How do I keep the custody agrrement when it's not being followed by the other party?

If you have an agreement on custody that is not a court order, the usual solution is to file for a custody order that would have the court decide who has custody when.  Often parents have trouble with agreements because they have different expectations when they sign the agreement.  If possible, you might want to try to work out a specific schedule with the other parent -- joint custody does not really define what the parties are intending and often there are tensions because of the misunderstandings.  Another option, if it is difficult for you to discuss this with the other parent, is engaging a mediator to work with you both to develop an agreement that works for both of you.