Does Mom get kid when Dad is at work even if he has a babysitter?

My boyfriend and his ex have joint custody for their three year old and she is NOT fun to deal with. My boyfriend gets his son from Wednesday at 7 until Saturday at 7. Today is thursday, my boyfriend works. Does he ABSOLUTELY NEED to give him to the mother or does he get to decide who watches him because its his day? He lives with his mom and so his mom is right there to watch him...not to mention its not in the sons best interest to keep going back and forth multiple times in the day. Also, if he gets out of work at 8pm on HIS day and she refuses to give him up cause she claims hes "in bed" (he goes to bed at 930) is that breaking court order?