Common Law partner just up and leaves and leaves me with all the bills, debt and mortgage payments. What are my rights?

You didn't mention what state you are in, but there are very few that still allow for common law marriages.  So, you may have few rights to division of what would otherwise be called marital property.  But, if you are both on the deed and mortgage to the house, you can ask the court to grant you partition of the property, so that you can get your share out of it.  Depending on how much you put into the house, you may be able to refinance it in your own name or get your mortgage company to take him off the mortgage, but you would need an agreement or court order to do it.