Is there a tax liability and early withdrawal penalty?

When I worked for Employer A I took out a 401k loan to purchase a primary residence. Now I work for Employer B and wanted to rollover my 401k. In order to do that I had to pay the balance of the 401k loan. I did so using the funds from an annuity I have that was created from a pension payout from a previous employer. To avoid early withdrawal penalties my goals was to take out a 401k loan with Employer B and repay the annuity before 60 days. Due to a series of missteps with Employer A's administrator the 60 days have nearly lapsed. They've offered to write a letter detailing their missteps but I'm not sure if that helps with the IRS, and then the plan with the annuity said there would not be a tax liability or early withdrawal penalties, but my financial advisor said there would be. Please advise. I tried to repay the loan and stayed on top of the administrator but their mistakes cost me precious time. Thank you.