is there a way to appeal the penalty for not having workers comp insurance?

There are a variety of ways that the requirement to maintain workers compensation insurance could have been avoided in the first place. However, from the limited facts you have posted, it is difficult to ascertain the procedural posture of this matter and whether all opportunities to raise those defenses have been exhausted. You need to review the situation and all relevant facts with an attorney. This is really not the right place to do that. You will need to engage the services of an attorney on a fee basis.