Do we have to pay to get our documentation and money back from our Lawyer?

My husband and I fired our lawyer about a month ago. We fired our lawyer because he was not keeping us informed of the status of our case and was argumentative when we tried to obtain information of our case. Since then, it has been a horrible communication battle to get our money, (that we were told was fully refundable), back and our documentation to our case so that we may be free to hire another lawyer. He also has refused to remove himself, in this process, from our court case as we have requested by certified mail. Now, one month later, after we contacted the partners of his law firm with certified mail concerning this issue, our former lawyer is now saying that he will have to charge us, from the refunded amount to send our documentation back and the remainder of the money or we can come to his office and pick it up.