I was threatened by a business partner of my company

I do credit and collections for my company one of the accounts is a partner to the company that shows past due invoices. There were no notes on the account prior to me putting them there to NOT to call them. I was instructed by the partners secretary to contact this person who is the partner of the company I work at. At first when I called the person was extremely rude and told me not to call him using my title (im the collections manager) or the word Collectors. and told me to call back. I was very upset the way he degraded me and vented to a co-worker over the phone. In which my supervisor over heard me and asked me what account it was. She then proceeded to the Owner of the company who demanded that I call back and apologize to him for calling. (REALLY?!?!) I then called that person back and re introduced myself and he then stated "thats much better" I was going to come down there and punch you in the face and break your nose while he was laughing. Do I have a case?