For many people, their 401(k) account holds most of their wealth—but that doesn’t mean it should be considered a revenue source.

Borrowing from your 401(k) to pay off high interest-rate, credit-card debt may seem like a good idea. But it turns out, many workers who went this route now have borrowers’ regret.

According to TIAA-CREF’s  (https://www.tiaa-cref.org/public/index.html recent Borrowing Against Retirement survey, 46% of respondents report having tapped their 401(k) to pay off debt, but only 26% of respondents said paying off debt was a good reason for taking a loan against their retirement.

“Paying off debt with a loan from your retirement account is like paying off a credit card with another credit card; it could spiral into a cycle,” says Daniel Keady, certified financial planner and director of financial planning at TIAA-CREF.

The survey canvassed 1,000 adults who are currently contributing to their 401(k).

Borrowing from a 401(k), which became more commonplace during the 2008 financial crisis, can be a benefit, experts say, but it can also put your retirement savings at risk.

Forty-seven percent of borrowers took out more than 20% and 9% borrowed more than 50% of their accounts. Forty-four percent, took out more than two loans.

“This tells me people are thinking of their retirement savings plan as a secondary piggy bank and [think] ‘Why don’t I use it?’”

That being said, Keady acknowledges if used properly the loan option can be beneficial, but prefers workers have a three to six month emergency fund to fall back on. He says borrowing from your retirement plan can be helpful if you’re faced with an unexpected emergency like a job loss or catastrophic medical event, the number two reason 35% of survey respondents dipped into their accounts.

Trouble Ahead

Keady emphasizes that if you take out a loan from your 401(k) and can’t pay it back, you can run into trouble. You’ll be on the hook with the IRS which will consider the money a withdrawal—a taxable event. Plus, you’ll be socked with an extra 10% penalty if you’re younger than 59 ½, the age at which you’re allowed to make an early withdrawal from a 401(k) plan.

“Borrowing also takes money off the table,” Keady warns. “You may end up missing out on strong market performance that will grow your 401(k).”

What’s more, paying back a loan can put a crimp in your savings cycle. In fact, 57% say during the payback period they decreased their contribution rate, with 18 to 34 year olds cutting contributions by as much as 81%.

It’s a real catch 22, says Brad Klontz, managing partner at Occidental Asset Management.  “You’re motivated to contribute when you’re told you can borrow any time and don’t have to hold off till your sixties.”

“But, that’s looking at your 401(k) in totally the wrong way,” he cautions.  “It’s not a savings account, but a financial freedom account for your later years.”

Experts recommend these tips to avoid the lure of borrowing against your retirement plan:

Go back to basics. This will help break the cycle of over-spending, says Keady. Track and evaluate your daily spending, then develop a budget, he says. Regardless of how you got off kilter, have a strategy to right the ship.

Think income. Don’t consider your 401(k) a nest egg, cautions Keady. Understand you have to convert the money to income at retirement.  “That $100,000 doesn’t seem like much when you consider it has to keep your income going during your golden years.

Evaluate funding sources. Depending on your need, you may have choices for gaining access to a loan through the type of loan you select (e.g., a student loan or home equity line of credit). Some whole life insurance policies come with loan options, explains Keady. Another option, he says, particularly suited to younger people: turning to your Roth [IRA]. You can pull out Roth contributions tax-free.

Take small steps. This helps build your emergency fund, moving you from concept to execution, says Keady. Saving for six months’ expenses may seem insurmountable, so start small, he recommends, perhaps $500, and increase your goal incrementally once you reach each mini-benchmark.

Divide and conquer. Dividing your money into buckets is a natural way to deal with money, says Klontz. “We treat our diverse money buckets emotionally differently,” he says. Klontz recommends establishing three buckets: debt repayment, retirement savings and your emergency fund.

One caveat, he adds: the rule of three is easy if you have $1,000 in debt. “But if you’re in for $15,000 at 17%, you might consider throwing most of your earnings to pay off that credit account.” Then regroup with a clean slate.

Seek help. Workplace financial literacy education helps, as does working with a financial professional, suggests Keady.

Klontz says financial therapists are also helpful when uncontrollable spending becomes an unstoppable problem that can delay, or even worse, prevent you from retiring.