Lease agreement issues with my fitness studio's landlord?

I knew what my fitness business would offer before I found the location. I found a space that offered "advanced heating system-- perfect for hot yoga" and thought that was perfect because I wanted to offer hot yoga. It was directly next door to a yoga center. I asked the real estate agent REA) if that would be a problem and she said no because they're well established. We met with the landlord and I told him about my space and said he could look at my website to which he declined. We signed the lease. Yesterday, he contacted me saying that I need to remove my yoga classes because it violates his agreement to the yoga space not to rent out to competition and the REA made a mistake. I have already paid for advertising for my yoga, hired somebody, and signed people up. It says nothing in my lease about what I can and can't offer nor does it mention him being able to impart rules at random. Legally, can he say I need to get rid of all of my yoga classes (some the other place doesn't have).