Is my daughter's father responsible for picking her up and dropping her off for visitation?

My daughter lives with me but is supposed to visit with her father on weekends. Is it his responsibility to pick her up and drop her off for visitation? I don't see how it is my responsibility. If he wishes to see her then he should pick her up and drop her off. He does not pay child support or help me with her at all financially. He is an alcoholic bum who cannot pass a drug test and refuses to work. He is harassing me claiming it is my responsibility to drop her off to his house and pick her up. Frankly, I don't even want her over there. He is an alcoholic and cannot pass a drug test, his father is very nasty to her and he constantly has my child around a sex offender. Should I file for full custody when he goes back to jail? Am I really responsible for pick up and drop off for his visitation?