Can my daughter's mother take her to a different state without my consent?

My child's mother and I do not have any kind of court ordered custody. We share our daughter week on week off 50/50. We both currently live in the same place where the child has been raised till now (6 years). She wants to take our daughter to visit her grandmother and uncle for the summer while she is out of school. I have verbally agreed to let her go for up to 3 weeks which seems to long for her to be away. she is very attached to me. It is summer and she needs to know her grandmother too, so I can give a little. Now that her mother has revealed to me that she is going to find a job when she gets there and if it is going well she will move there. Is there anything I can do to make sure she comes back in the time that I specified. Or not let her take our daughter unless she signs a written agreement to the 3 week terms. Side note: Her mother lives in an RV and/or Stays illegally in vacation rentals that she used to clean and still has the door code for, all over town. she does not have a job. she has a 1 wet and reckless and a DUI she was arrested for domestic violence about 2 years ago. I did not press charges cause we were together at the time. she has recently given her boy friend that live with her in the RV two black eyes and our daughter says they drink to much and fight to much. most of the time when I pick her up the RV or whatever house they are staying in smells like pot smoke. Our child calls me crying all of the time wanting to "come home". My girl friend and I have a nice condo where she has her own room. We have lived there for over a year and plan on staying. I do not drink anymore or smoke pot I do have a 1 DUI. I am a wild land fire fighter so I do make alright money but I don't think I can afford an attorney. if there is anything I can do please let me know. Thank you.