Can a new wife adopt my son without my permission

I have joint custody of my son with an ex-boyfriend. he threw me out in 2008 and kept our son. I went to live with a relative who then in turn told me to leave. I moved out of state to live with my mother because I had no where else to go. Every time I tried to schedual visitation I was told no. He also refused to let me speak to our son on the phone. Now his wife wants to adopt our son and he says because I moved out of state I abandoned him and she can adopt him without my consent. He keeps moving and doesn't let me know and him and the wife have both been arrested for domestic violence and my son was temporarily in foster care and I was never notified. I recently found this out from my ex's sister. Can you please tell me of my rights? I currently live in Connecticut and he is in New York.