Is it legal for my brother to make a document that gives him certain rights to the property?

It depends how involved your brother was in having this document executed. Most estate litigation centers around competency and undue influence. If your father was in his right mind, it's probably OK. But if there were circumstances suggesting that your brother fooled him into signing it or used his position as a confidant to get your father to do something that he didn't really want to do, then the will/deed/power of attorney (or whatever) can be successfully challenged.

If the will has a statement in the will that there will be a writing attached to the will making specific bequests, then the additional writing becomes part of the will. There may be issues of the validity of the writing or will not addressed here. It would be wise to consult an attorney on this matter to determine if the will and or the memorandum are valid.

If the document your brother has was executed as a last will and testament with all the requirements and formalities of a last will and testament and was executed last, that's the document that will prevail. However, just a simple document, although notarized but without the formalities of a will, will not supersede an existing Will.