Can my landlord kick me out of my apartment if I did not sign my renewal?

I did not sign my renewal lease, but paid the rent increase and have been paying each month. I told my landlord that I wasn't sure if I was staying, and wanted to keep my options open and would let him know by June if I was staying or not. He said he could not do a month to month lease, but was OK with me letting him know by June. I let him know that I was staying and would sign my lease, but now he's telling me to hold on to my rent, and that he wants me out. What are my rights as a tenant even if I don't have a current lease? What can I do to ensure I don't get evicted?