Can my former custodial family take my unborn child from me?

The answer is no, they have not rights your child. But they could report you to child protective services that you are an unfit mother and the father is also unfit in some way. They know your history, and maybe there is something that will make CPS look into your situation. Babies have been taken by CPS from the birth hospitals on just such reports. Then your child is in dependency and you have to prove the reports are wrong or get help (services) so you can be the parent the state DSHS thinks you should be. In dependency extended family or former custodial parents can offer to take the child for care while you are in this proving stage. If you fail, then they can seek to adopt the child when your rights as parents are terminated. I would recommend if you can move to where these people do not know where you live, preferably another county. Look carefully at your situation, is there anything you need to do to change what you have done in the past (drug use, bad acts like prostitution or stupid childish stuff) and make sure none of that is happening now and will never happen again. Choose medical professionals who are not judgmental. Do Not tell you whole history to the new ones. Learn from the mistakes of others, ask around.

As the child's parent, you have primary rights to you child. Grandparents don't even have visitation rights in this State. However, I have seen grandparents obtain custody if they can prove that the parents are unfit, eg. that harm will co,e to the child.

I don't know what you mean by "former custodial family." No one can take your child from you without the permission of the court. If you and your husband can be shown to be unfit because of substance abuse, mental health or other serious problems that would not allow you to act as fit parents, a third party, including the child's grandparents can file a third party custody action. If these people were your foster parents, they would have less standing than a grandparent. Your age is irrelevant here because you're over 18. Where you live is irrelevant unless your husband's parents can be shown to be a danger to the child (e.g. a sex offender). Do not allow your child into the care of these people after he/she is born if you have any concerns that they may try to keep the child. You may want to call a lawyer to discuss this since the facts you provide are so minimal. More complete facts would allow for a more complete and clearer answer. That's not possible by simply writing this kind of question.