Can an attorney cease representation over comments questioning his/her fees?

About 18 months ago, I learned about a fraud investigation against me. I hired a lawyer for representation. To this date there has been no charges filed. The attorney's fee was $10,000 to cover the fraud case which was paid in full, plus $400 per hour for any excess time. I agreed to let him hold an additional $45,000 in case I was arrested and my assets were frozen. I wanted to ensure I had him paid in full. Eventually, when I felt more comfortable, I asked for the escrow back. He only refunded $29,000. I have been in a dispute with him since. He has kept $26,000 and claims to have worked 65 hours which I don't see possible. There was no case; the only work that's been done was e-mail correspondence, some phone calls, and two brief in-person meetings. I have asked (on numerous occasions) for an accounting that details his time and services. He has always refused, but just today he agreed to provide it while still insisting he's worked 65 hours. I replied by saying that if a person spent 65 hours on this, he/she must a very incompetent person. The lawyer replied by withdrawing his services due to dispersions on his professionalism. Where do I go from here?