Will a verbal agreement hold up in the case of easement to our property?

When my husband and brother-in-law were inheriting land from their dad, their 40 acre parcels ran in a way that our 40 had all county hwy access and his brother was unhappy about that. So we agreed to change the dimensions of the 40's in such a way that would split up the land into 2 shorter but longer parcels (more rectangle) versus the traditional (more square) so that both properties would have hwy access SO long as we could use the old existing driveway to access our property and land because our land has a river through it and all hwy access is considered wetland. And his brother and wife agreed. We have been using the driveway for over 15 yrs. now since the agreement and 2 yrs. prior. Nothing was in writing and this is the only way into our property and home. Now they are saying they want clean property lines because they intend to sell their property and say they are going to put up a fence on August 1st. Can they do this? We have witnesses that will testify to our verbal easement agreement. We said we will try to work with them and have contacted the Dept. of Natural Resources regarding acquiring a permit to go through a wetland but prior to doing that, we need to get the property line surveyed and marked. We were both supposed to pay half the survey fee this spring but they didn’t have the $ and they said they wanted to pay for it themselves because they owe us $ for machinery, building materials and other misc. items, so we said fine then we would be even. However they never got the survey done to establish property lines which needs to be done prior to construction of a new driveway through the wetland. So, the most important question now is can they put up a fence keeping us from our home? We are living on 1 income and was slapped with this w/o any warning. Estimates for our driveway are as high as $19,000.00! We need help and advice and options. His brother keeps going back on his word and I need to know if we have a leg to stand on. Thank you.