do I have any legal grounds for custody

Lawyers.com offers general information and not legal advice.  In Virginia, issues of custody are based upon the child's best interest.  What constitutes the child's best interest is a factual question.  The court will look at the individual facts in each case and apply the law, as set forth in Virginia Code Section 20-124.3.  As grandparents, you have a legitimate interest to pursue custody.  However, the court will make a decision based on the overall best interest of the children.  Your questions cannot be answered in this forum.  It is recommended that you confer and consult with a family lawyer in your area to discuss this matter in greater details.  Most lawyers charge by the hour for office consultations, unless you qualify for some discount.  Best of luck~