Does he have to file the joint sponsor form I-864A too or only I have to file form I-864?

Generally, to sponsor your parents you will need to file an Affidavit of Support, and if you have insufficient documentable income for sponsoring your parents, it will be necessary for another U.S. citizen or Permanent Resident (whether your husband or someone else) to serve as a joint sponsor, and that person will need to submit an Affidavit of Support too. Of course, there are many other details that determine eligibility and Affidavit of Support requirements. It would be wise for you and your family to consult with an immigration attorney who, after learning all of the relevant information, could advise about eligibilities, options and strategies to attain your goals, and who then could offer legal representation in the often complex application process.