Who is responsible for a mortgage loan that has one person on the closing douments and a different person on the deed?

I requested a mortgage refinance through countrywide in 2007, I was told my credit score was not good enough to do on my own and I needed a co-signer. My father accepted this responsibility. We went to the closing together and our guy through countrywide said that I could not be on the loan at all, but my father could finance by himself and in 6 mos, we could do a quit claim and get his name off and my name on. We agreed to this after advise from our countrywide mortgage broker. We did just that and everything has been fine. So, now 5 years later my monthy escrow which pays my taxes has raised my monthly mortgage payment $1000. I cannot afford this and requested a modification through my current mortgage company Bank of America. They are telling me I might not have any options because my father is the loan holder, but his name is not anywhere on the mortgage, as he was quit claimed off. They are trying to figure out who is responsible for the loan? What do I do?