Can he modify the child custody and what do you think the likelihood of this is to happen?

I left my husband (at the time) in Texas in September 2011. I came to Missouri and filed for divorced. Divorce was final on January 2013. Our agreement was, he does not pay child support and I get our daughter. Visitations are at my discretion. I was advised not to allow my daughter to see him. I have read some of the other questions posted on here. You have said that rights and duties come together, drop one and lose both. Is that the case here? Did he sign away his duties? He was abusive. I do have police reports and our daughter is currently in counselling. She is only 5 but is scared of daddy coming to get her. Ex wants to see her this summer and is threatening me that he can get around the system. I took him to court and got a restraining order got that but was denied child protective order. Please advise. I don't ask for anything. He also claimed her on his taxes when it was clearly stated that I was to claim her. So he has messed up a lot of things for us. Thanks for your time.