My daughters father is threatening to take me back to court over our daughter. What grounds does he have?

To begin. I have full custody of my daughter. He is non-custodial with a visitation agreement. My daughters father is mad at me because I brought to his attention that she is coming home telling me about several women staying over while she's there. She's told me other alarming things as well. Things she shouldn't know about. She's only there every other weekend. I reminded him that there is a cohabitation order in place and that I would recommend him not doing this because our daughter is not understanding. I don't care what he does outside of when my daughter is there. He doesn't work. I recently found out about 10 years ago he was charged and tried over beating and bruising a small child of a woman he use to see. What should I do about learning of this charge and to ensure he doesn't do mess like this to my daughter? And does he have any bit of ground to try and pursue changing his visitation agreement? I just get tired of him acting like he has some authority.