A healthy -- but admittedly lonely -- Bernie Madoff told me the one thing he did right was not make his family a part of his infamous and tragic Ponzi scheme.

He looks healthy and strong for a man who is now 75 years old and will only leave prison in a coffin, since he is serving a 100-plus-year sentence. Madoff spent two hours talking with me about some of his former clients who he now says knew they were breaking the law, and he touched on his family and the role of banks like JP Morgan Chase which Madoff says were complicit in his crime. He claims to have no contact with his family and says he will, "forever live with the guilt of causing my son Mark's death." 

Four years after arriving at the Federal Correctional Complex in Butner, N.C., Madoff says he wants to help investigators recover the billions of dollars his scheme robbed from innocent investors. 

"All I care about is to get the money back," he said. 

But it is hard to believe what he says when you consider the enormity of Madoff's crime. The Trustee says investors lost $17 billion in capital they invested with Madoff and even Bernie says the scheme was "...a curse, it was a horror" that came to a calamitous end in December 2008, when Madoff was arrested. Now he's changing his story.

Madoff has always said he first told his brother Peter about the Ponzi scheme and then his sons. But for the first time publicly, Madoff claims a few days before the family confessions, he called three of his four biggest investors -- Carl Shapiro, Stanley Chais and Jeffry Picower -- to tell them the charade was over. Madoff told me, "they knew they were doing illegal stuff for years...back dating stuff." 

According to Madoff those three investors, along with New York real estate mogul Norman Levy, were dictating to Madoff fictitious profits they wanted to make and ordering him to back date stock transactions to deliver their desired returns starting in the early 1990s. When I asked Madoff if the four investors were aware of the Ponzi scheme he said, "they didn't care if it was a Ponzi scheme." 

Only Carl Shapiro is alive today and his attorney Stephen Fishbein issued a statement responding to Madoff's new claim, saying, " Carl Shapiro was as shocked as everyone else when Madoff confessed to a Ponzi Scheme. This after-the-fact statement by an admitted liar and convicted thief deserves no credibility whatsoever." 

FOX Business reached out to attorneys representing the estates of Levy, Picower and Chais for a response. The Levy family returned $220 million to the trustee, the Picower estate returned $7.2 billion and Carl Shapiro returned $625 million. The trustee is suing the estate of Stanley Chais for more than $1 billion.

The three former clients were the target of a criminal investigation but charges were never filed against them. Madoff claims he called the surviving clients in an attempt to recover money for his victims but his new statements contradict his admitted actions in the final days of his Ponzi scheme in which he was preparing to disburse the remaining millions of dollars in his bank account to family members and employees of his firm, not his victims.

That money was held in an account at JP Morgan Chase, the bank that handled Madoff's money for more than two decades. We discussed the bogus SEC reports Madoff filed monthly with the bank in which key information about client funds was missing. We also discussed loans JP Morgan made to Madoff as well as his client Norman Levy, who had an account at the bank too. 

Millions of dollars flowed between Levy's account and Madoff's account, and Madoff says JP Morgan employees asked him questions about discrepancies in reports regarding those transactions, which regulators say should have raised questions about money laundering. Madoff told me, "There's no question that JP Morgan is guilty; they would have to be idiots to not realize what was going on." 

A source close to J.P. Morgan says, "Madoff has made these false accusations many times in past interviews." Madoff says Eric Thorson, Inspector General of the United States Treasury, is investigating those transactions and other issues at JP Morgan. He says that the IG's office has contacted him about it.

The IG's office has not responded to a request for comment on the matter and JP Morgan Chase declined to comment. But many of these issues are raised in the Trustees lawsuit to recover billions of dollars from JP Morgan. That lawsuit was partially dismissed on jurisdictional issues. The court ruled that the Trustee lacked legal grounds to bring the suit. The Trustee is appealing.

As for Madoff, his days at Butner he says are basically stress free compared to his life as a multi-billion-dollar crook. He told me, "I was under such stress while all of this was going on, the stress was unbelievable." 

Now he meets regularly with a psychologist trying to come to terms with the lives he destroyed. 

"How was I able to block out what I was doing and the consequences? I can't figure that out. How could I have caused so much havoc and pain?" 

Bernie Madoff isn't the only one asking those questions four-and-a-half years after the biggest Ponzi Scheme in history collapsed.

Adam Shapiro joined FOX Business Network (FBN) in September 2007 as a New York based reporter.