Can I cancel my credit card charge for subpar work done by a contractor who has not finished the work?

We hired a contractor to do concrete pointing and concrete work. He said it was $1500. on the contract. I charged on my credit card $500. as a down payment. Two of his family members did most of the work; a week later he said it was a remainder of $2880. and included a surcharge of $23. for charging my $500. He said labor was also $50. pr hr for each worker. It felt like a bait and switch; I explained what was originally agreed and signed. I got confused and charged the rest on the spot. I also just had surgery and was on heavy pain medication when I did this. He was to come back that Mon. Since then, it's rained; concrete has been falling out from the pointing; other stuff & didn't finish the job. We called a few times; he said he was coming out and that I owed him a $100. for a surcharge for charging the $2880. He did not show. I did take pictures. I had another contractor come out and it's going to cost more than what I paid to correct the subpar work. I did a search on the PA