Will filing for unemployment potentially open me up to back taxes since approximately half my wages were unreported? Do I have other options?

For the last several years my employer has paid about half my wages off the books. I did not report the additional income. He repeatedly promised to handle payroll legitimately but never did, so I quit the job recently. If I file for unemployment, do I open myself up to back taxes and penalties even if the additional income cannot be verified? I was never comfortable with this arrangement, but felt I had no choice. Payments were made partially in cash and partially by checks on an irregular schedule, and I often had to ask repeatedly to be paid.