Can my ex husband take custody or place a restraining order on my new partner who has a felony 1?

I’m at the end of my divorce, it’s been a year and a half now, and I am dating someone that I used to date 18 years ago. Since then he had been convicted of a felony 1 man slaughter after a fight. He was imprisoned for 11 years. He’s been out for 2 years, and we’ve been seeing eachother for 11 months. He’s amazing with my 3 boys and myself. He has done things for us that their own father wouldn’t do, such as fix their dune buggy for free, fix our outdoor woodburner so we would have heat during the winter and he even co signed for my car when my ex wouldn’t help me at all, even though he is solely responsible for ruining my credit. I have witnesses and evidence of multiple reasons why my ex is not suitable to have custody but since he found out that I have been dating this man he and his attorney threatened to have a restraining order put on him and it may jeopardize our parental agreement if I allow our boys around my new partner or so much as let my partner come to my home