What do I have to do to terminate his rights and have no ties at all?

I have a 20 month old son whose father is on the sexual predator list for having involuntary sex with an 11 year old girl. I did not know this at the time but when I found out, I still had the baby. He did sign the birth certificate and now he scares me. He does not live in the same household. He goes back and forth if he wants to be the father or not. He has an angry criminal background that I am still finding out more about. Little by little. I do not and never will leave my son with him or let him go anywhere with him. I also want to protect myself and my son from him in case he gets angry and tries to retaliate. Do I have to give him some kind of rights?