Can an attorney hold your consulting funds if there were two consulting fees because he forgot what was in the first consult?

My son was seeking emergency temporary custody of his son. We were directed to this lawyer. His consulting fees were $250. We paid it. He took notes in the first meeting and we set an appointment for another time to get the papers going on this. We went back to our set appointment. He couldn't see us. He had court so of course, it was a wasted trip. We waited for another time to see him. There was another $200 consultation fee due to the fact that he said he didn't remember what was said in the first meeting. I remember he took notes and we were about to head to his office for his next appointment to get things rolling on my son’s child custody. Then he had his secretary call and she said he couldn't take the case because there wasn't enough time to get it done. He was running for judge at the time and he wouldn't have time to take my son's case. Should we be entitled to a refund for services untendered? He paid for two consults that was last year. We have contacted his office many times and requested at least part of the money back to no avail. His secretaries just said that they will give him the message and he doesn't try to contact us that is just wrong in my book. I think he is entitled at least the second consult fee because he was put off too many times.