Can i be sued for not paying the balance to a company for shoddy workmanship?

i hired a company to erect a privacy fence. i told him what i wanted. He quoted me a price. then, when it was time to do the work, the price had increased $600. i agreed to that. i paid over half the cost before the start of work. They built the fence and it is not what i asked for, and the work is horrible. He has refused to fix it to my satisfaction. He now says he will sue me for the remaining balance. What should i do?