Can a US citizen petition a brother that has overstayed his visa?

The US citizen can petition for his brother, but his brother would not be able to adjust status to permanent resident (get his green card) here in the US. He would have to return to his home country. But, depending on how long he has overstayed his visa, once he leaves the US he would be faced with a 10 year bar to reentering. Also, there is a long wait for siblings of US citizens to get a visa. Probably best to wait and see what happens with the immigration reform.

If you petition for your brother now it will take 10 years for him to get any legal documentation. There is also the issue of his overstay that will affect him at the time of legalization. He should wait for the new reform and that will help him.

Yes, you can and probably should file a petition for your brother. However, since he is not immediate relative, and is in the fourth preference category (F4), he will have to wait for several years (depending on where is from, the wait if different but it's still about 10 years or longer for most countries) to file his adjustment of status (greencard) application. Unless he has a prior petition that was filed for him or his behalf, which was filed on or before April 30, 2001, he would not be eligible for adjustment of status, under the current law. That would mean that he would have to rely on the passage of immigration reform.

A form I130 petition filed by a US citizen for his siblings is one of the options that may be availing to your brother and I strongly suggest that you consider doing so. The comprehensive immigration reform is still being proposed and it may or may not become law. Next, while a petition by a US citizen for his sibling under F4 category will take about 12 years, it is worth filing for two reasons. First, in the proposed immigration reform, that petition may not survive the final bill that passes Congress and it will be unwise to pass up on that opportunity. Second, it is highly likely that if you file the petition before Congress scraps that option, it is likely that processing times of petitions filed in that category will become a lot shorter and in any event, it may be the only option available to your brother to become a green card holder under current law.

A US citizen can petition for his brother or sister on form I-130 even if the sibling overstayed his authorized stay. However, unless the sibling has been grandfathered in under INA 245(i), he will not be eligible to apply for adjustment of status when his priority date becomes current.