Contractor wanting payment for work not done

Had homeowner damage & subcontractor did not have general contractor license, so he brought in someone who did. The general contractor paid the subcontractor up front for the work without advising me. Subcontractor did not complete one small part of the job, so I have refused to pay for that work - it is $200 of a $6100 total. The general contractor is now threatening to sue me, saying he can go after me for the deductible that was involved as well as other fees. He has offered to send someone to do the small amount of work left, but I don't trust this guy and I do not want him on my property. He signed a waiver of lien, so he can't put a lien on the property. Am I wrong to just want to pay him for all but the incomplete work and be done with it? This small project has drug out almost a year now. My son has agreed to do the small job with the $200 I still haven't paid, and I would much rather let him do it and be done with this.