Can ex make children choose between planned vacations with their mother or father?

I have been divorced five years and have 2 teenage daughters. I received our timeshare in the divorce which I have gone to for 16 years now with the kids. It's the exact same calendar week every year and some years it falls over the 4th of July, like this year. The kids love it and look forward to it. Some years my ex and I have split the week (as he was vacationing nearby); other years he has planned his vacation the week after. This year, he just informed me that he wants his vacation for the exact same week and wants the kids. He is wanting the kids to choose between who they go with- if he gets them they don't go on the trip with me. I do not think this is fair to the kids and they are disappointed they can't do both like last year. He knew when my week was and he has his own business with more flexibility to take off time. I have the one week off of work and it’s always requested at the beg of the year. He now says it’s easier for him to take off the week with the holiday in it, but this wasn't an issue last year. He has a pattern of last minute trying to change things to benefit him. I have sole physical custody and joint legal. We have never gone by Friend of the Court holiday schedule instead we've been flexible. He now has a fiancé with kids of her own so he keeps trying to change things that we have always done in the past. I do not want my kids to have to choose and want them to have time with us both. I feel that he is causing unneeded stress and anxiety here. What grounds do I have to stand firm on taking them on their annual trip?