What do I do if the father is withholding my son from me?

I had my child and continually asked his father to come meet him to which he had no interest. I became very depressed after giving birth and fell into doing drugs. I was soon convicted of drug use and completed a year program. While I was incarcerated, my mother was granted custody. Then my son’s father finally took interest and was given joint custody with my mother. When I was released, I worked hard to do everything I needed to get custody back. I got full time employment, followed up with self-help groups and went to court. The judge ordered that I have supervised visits and attend drug/parenting classes and weekly testing. My work schedule started to interfere and so a verbal agreement was made with my mom and son’s father that if I stayed clean and productive, I would have joint custody with my son’s father. My mother still had the majority of the legal custody when she passed away, but I never went back to court because my son’s father and I had a great co-parenting relationship. We found residences near each other and parented as a unit. All this time I had paid for 90% of my son’s expenses including insurance, outside activities and school tuitions without complaint. Last year, my husband and I separated and there were a lot of issues. We ended up getting back together and my son’s father did not agree with my decision and withheld my son from me since he has legal custody. This was enforced more so because my son’s private school called cops with allegations against my husband while my son wasn't even residing in my home and hadn't seen my husband in months. On top of that, his family kept coming to my house and was directly trying to interfere with my relationship with my husband. Now I have lost my job and haven't been able to pay private school tuition so once again the school calls CPS and reports negligence and somehow this is all falling on me.