As the Obama Administration attempts to rein in Wall Street pay, it’s no secret the restrictions imposed by “pay czar” Kenneth Feinberg have sent some executives scurrying for greener pastures, or at least to pastures where the administration hasn’t yet set limits on the amount of “green” available.

The "brain drain," as it is known, has hit all of the seven largest bailout recipients that fall under Feinberg’s jurisdiction, with more than 40 executives at those firms jumping ship.

But the drain hasn’t been limited to just executives. In the past year, dozens, if not hundreds, of lower level employees have walked away from those same companies, many of them blaming what they say is the “mob mentality” of lawmakers seeking to hold someone accountable for Wall Street’s near demise.

At Bank of America Merrill Lynch (BAC), which accepted some $45 billion in bailout funds and has yet to repay the money, several high profile stock analysts have left the firm in the past six months. Aamong them were most of the company’s energy analysts, its economic guru and several regulars on the Institutional Investor All-America list, which ranks the best analysts in the business.

In interviews, some cited the risk of compensation cuts as their reason for leaving. Others cited employee morale, damaged daily by continuing attacks from all levels of government and by the defection of high ranking employees. But whatever the reason, they said the decision to seek employment outside the glare of the government spotlight was not a hard one.

“It’s no coincidence that so many of us decided to leave at the same time,” said one stock analyst, a fixture on the various “best of” lists who left Bank of America this summer for a smaller Wall Street player. The analyst asked not to be identified, citing media policy at his new firm.

“If you go back to late last year and early this year, they were talking about claw backs of bonuses and maximum pay levels at all of these companies. That’s a tough atmosphere to work in, especially when there are opportunities out there at firms that didn’t take any TARP money.”

“Basically my bonus is my salary and I have to work a whole year to collect it,” said another who also asked not to be identified. “To face the potential bullet (of bonus restrictions or claw backs) at the end of the year isn’t worth it.”

And while the average taxpayer may feel little sympathy for high salaried Wall Streeters, they should, says one former Bank of America employee.

“The taxpayers are the major investors in these companies now by virtue of the bailout,” the analyst said. “If I’m an investor the last thing I want to do is undercut the top talent at my company.”

The brain drain also hasn’t been lost on some lawmakers who say the government’s compensation restrictions are counterproductive to its efforts at recouping taxpayers’ multi-billion investment in the foundering companies.

“If they jump ship and you don’t have top talent running these companies, the American taxpayer, who is the majority stockholder, has inferior people running the company. Doesn’t that concern you?” U.S. Rep. Dan Burton (R-Ind) asked Feinberg during a hearing last week.

Feinberg responded that his restrictions had walked a fine line between employee morale and reining in pay. Asked to comment on the defection of important employees, a Treasury spokeswoman echoed Feinberg’s response.

“As his rulings show, Mr. Feinberg has struck a careful balance between the need to retain talent so that taxpayers can recoup their investment and the need to structure compensation in a manner aligned with long-term value creation and financial stability,” said spokeswoman Meg Reilly. “In several cases, Mr. Feinberg has expressly identified employees critical to the future of the enterprise, and has taken those considerations into account when structuring the employees' compensation.”

And just how much effect the loss of any employee has on a company’s ability to perform is debatable, says one analyst who follows the banks closely.

“People in my business tend to be ego-maniacs,” said Dick Bove of Rochdale Securities. "They think that they are indispensable and that if a firm loses their services the firm is harmed. But if you are dealing with a company that has a trillion in assets it’s pretty hard to make that case.”

Bank of America also recognizes the loss, but says top-tier talent will always gravitate toward the biggest names in the business.

"If you;re looking for a global career in financial services, there are very few (companies) that can offer the opportunity that we can," said Bank of America spokesman Jerry Dubrowski.

Feinberg has begun work on 2010 compensation for the major TARP recipients, vowing to continue to structure pay that puts long term growth ahead of short term gains.