Generally speaking a biological parent has priority for custody of a child. From the question it appears that there were some kind of court orders in place regarding parenting time. If there are no such orders then the surviving parent should consider an action with the family law court to obtain the custody of their biological child. Consult with an experienced family law attorney for help in this matter.
Answered on Feb 07th, 2014 at 6:01 AM