WE bought our house 5 yrs ago The house had an existing privacy fence when we bought it Is there a limitations when it was put on neighbors property?

Yes and no.  There is a legal concept called adverse possession, which means that if you openly exercise control over someone else's real property without their permission for a period of time (the period varies between jurisdictions but is generally approximately 10 years; I don't know what it is in Indiana), you gain the right to continue to use the property as you had been, up to and including full ownership.  However, here you had the owner's permission to use the property, whcih they could revoke.  Any claim that you might make that they had agreed to allow the prior owner of your property to use the property in perpetuity is likely to fail, because such a contract is unenforceable unless it is in writing.