Can he take it back and say he's taking her on vacation?

If you have met all the requirements of your divorce judgment in terms of giving the father advance notice (including the location and phone number where the child will be), then you should not need his specific permission. However, he could come and try to take her on vacation with him. You might do best to have your lawyer file an expedited motion to clarify that you have the right to take the child on vacation during the time period you told him about months ago, and you can ask the court to order him not to try to take her from you at your scheduled departure time. Best of all, of course, is reaching some agreement with him if it is at all possible. Good Luck.