Can I collect my son from another state by his request and bring him home?

My son moved out of state with his father less than 3 months ago, to "try it out" but is not happy and wants to return. We did not change the court order. We still have the same order the had both myself and his father with joint legal and physical custody, split week to week. We did make an agreement for my son to go with him and a visitation schedule for him to return here, which we printed up and notarized. But nothing was ever filed with the court, I had a feeling my son would want to return soon, so this seemed fine for the time being. He was supposed to be on a plane today returning home, but yesterday I was informed by his father he was not coming. Communication was severed between my son and I a couple days ago. I sent the sheriff for a wellness check, my son also told him he wanted to come home. He says his father won't take him to the airport (after agreeing with me via text that he would) Can I go pick him up? Or did signing that agreement make it so I can't now?