He's not on the mortgage, we aren't married can I ask him to leave the house?

Anyone can ASK anyone else to do anything.  If the other party refuses, regardless of whether that refusal is justified, there will be a legal battle.  As for who would win that legal battle, it is impossible to predict without knowing a lot more facts, especially how title is held, and if you ever had any agreement about ownership.  If you are the only title holder and had no agreement that he would co-own the house, I think it most likely (again, there may be many relevant circumstances of which I am unaware) that you would be granted sole possession of the house, but would have to repay him for his contributions to its value.