is my ex allowed to take my 4 week old son for a long period of time even though I don't allow it?

Until there is a court order, there are few "rules."  But you don't supply enough information to know what the default is -- things are different depending on whether you were ever married to the father, whether his name is on the birth certificate, and what the background facts are, as to your relationship, etc. You should take a look at a couple of resources for background: http://willicklawgroup.com/child-custody-and-visitation/ http://willicklawgroup.com/paternity/ For specific advice you should confer with a family law specialist.