In an interview with FOX Business Network’s Liz Claman on Wednesday afternoon, Berkshire Hathaway (BRK.A) Chairman Warren Buffett said Goldman Sachs (GS), in which he just bought a $5 billion stake, is a strong company. He also expressed support for the financial-rescue package being debated by Congress now.

"If Congress doesn’t help us on this, then heaven help us," Buffett said. He added that Americans were "looking over a precipice" in terms of the economic condition of the country over the next few years.

Buffett said that he may regret having bought the stake in Goldman if the rescue package doesn’t pass.

“If congress doesn’t do anything… I may wish I hadn’t gone in….I think anybody buying anything this week will wish they hadn’t if Congress doesn’t act,” Buffett said.

However, Buffett said he doesn’t like to invest in companies that would fail without his money, countering some previous media speculation that Goldman would have gone under without the Buffett investment. Buffett said he has confidence in Goldman, adding, “I like their people, I like the economics of their business, and I like the deal that was offered to us,” Buffett said.

While Buffett stressed that Goldman Sachs would not have failed without his investment, he said it was certainly “helpful” to the company to have the money. Byron Trott, a Goldman Sachs managing director, approached Buffett yesterday to begin initial discussions about the $5 billion investment, according to Buffett.

Buffett also said he had been approached by a “great number” of financial institutions for possible investments, and had passed on taking stakes in Bear Stearns and Lehman Brothers. Bear Stearns was bought by JPMorgan Chase (JPM) in March, and Lehman filed for Chapter 11 bankruptcy protection earlier this month.

In the wake of Buffett’s investment, shares of Goldman Sachs jumped by 5% to $131.55. Buffett said his $5 billion investment in Goldman Sachs had already earned over $700 million by Wednesday afternoon. However, he stressed the true test of the investment would be in another five years, saying, “But I think it’ll work out fine.”

Buffett cautioned that the American public may be at the point where they are ready to stick their money under mattresses rather than invest it in the market, and this mindset has the potential to bring the U.S. economy to a screeching halt.

However, Buffett also said the U.S. Treasury may well make significant money because it has the staying power and the low borrowing costs that make for a successful vehicle.

"I would love to have a piece of that action," Buffett said.

Buffett has supported presidential candidate Barack Obama for candidacy this year, and Buffett confirmed that he has spoken with Obama about the credit crisis, though would not disclose Obama’s position on the issue.

What has happened to the financial markets so far pales in comparison to what will happen if Congress does not offer a bailout, Buffett said. He added that Congress would likely do the right thing, because legislators do not want anything bad to happen to the U.S. economy.

President Bush is expected to address the nation this evening around 9 PM Eastern time to discuss the current financial crisis.