Do I have grounds to sue for defamation of character?

I was dating my former boss. I broke up with him. I have record that he initially stated to help me find employment and give me a positive reference in my job search. He stated that he forwarded my resume to local firms and for a position at his company. After doing this he requested that we still date, I reaffirmed the fact that I would not date my boss and work for his company at the same time. He later stated that I can no longer use the company or him as a reference in an attempt to destroy me. Solely because I refuse to date him or fraternize. I have evidence of this conversation in malicious text messages that he sent. One stating “It’s only just and fair that the same happens to you.” Meaning, if he’s hurt then I need to hurt too. He has also threatened to sue me for the gifts that he has given me, which I know he has no grounds to sue. Will proof harassment help my case? I’m considering filing charge with the EEOC. I’d ultimately like to sue for defamation.