I am an unwed single mother. Does my childs fathers family have visitation rights in the state of NJ if I live in Maine and the father is in prison?

Is the father trying to assert visitation rights?  Has he filed something in Maine or in New Jersey?  Technically, even though he is in prison he still has visitation rights, but most often the courts find that it is not in the child's best interest to have the parent exercise those rights, especially if the child is very young, or the sentence is very long.  If you have always lived in Maine, the biological father cannot get jurisdiction to have you bring the child to new Jersey just because that is where he is incarcerated.