Do i have to repay a sign on bonus with no signed contract?

I recently accepted a sign on bonus for joining a new company however after being there for two days I left as the job was not what I was told it would be. I had already cashed the sign on bonus check. We did not have sign contract I only have a text message stating that the sign on bonus would be paid the first day of employment nothing says it needed to be repaid if I left the company. The owner of the company is now harassing me and threatening me with showing up to my new job if I do not pay him back the money. Do I have to repay it if there is no signed contract??