Not long ago, diet soft drinks were looked at as the beverage industry’s saving grace, a way to blunt a shift by health-conscious consumers away from sugary drinks.

But those hopes haven’t panned out. Diet sodas have suffered even bigger declines in recent years than regular carbonated soft drinks as Americans continue to guzzle more bottled water, energy drinks and teas instead.

Concerns about the growth prospects of carbonated soft drinks have serious financial implications for beverage behemoths Coca-Cola (KO) and PepsiCo (PEP), which are increasingly diversifying away from their namesake products, venturing into new markets and searching for all-natural solutions.

“People want healthier products. Carbonated soft drinks have never been positioned that way. The consumer is moving gradually to a healthier place,” said Gary Hemphill, senior vice president of information services at the Beverage Marketing Corporation.

While Americans consumed an average of 54 gallons of a year of carbonated soft drinks in 1998, they now drink 43.8 gallons, according to Beverage Digest.

At the same time, diet soft drink consumption has declined in each of the past six years, including a 3.4% tumble in 2012 and a 2.5% drop in 2011, according to BMC.

Health Concerns Take a Toll

Worries about the image of diet soft drinks were crystallized by a recent Purdue University survey study, which expressed serious doubt about the perceived benefits of drinking diet soda.

“Consumption of artificial sweeteners may exacerbate the negative effects of sugars by reducing the ability to predict the consequences of consuming sugars reliably and/or by altering cognitive processes that lead to overconsumption,” Purdue professor Susan Swithers wrote in the study, which analyzed a series of recent research papers.

The study comes on the heels of the contested move by New York City to limit non-diet soft drinks to 16 ounces.

“The problem is that the cumulative impact of all of these stories builds and it certainly has an impact on consumption,” said Timothy Calkins, a marketing professor at Northwestern University.

The beverage industry appears to be listening.

“With increasing category headwinds, it's imperative that we reinvigorate the [carbonated soft-drink] category and bring lapsed and new consumers back to the products they know and love,” Dr. Pepper Snapple Group (DPS) CEO Larry Young told analysts last month.

Young sounded agonized by the continued drop in diet soda consumption.

“People want [what’s] better for you but the diets are going down,” he said. “I wish I could wave a magic wand…It takes time to change people’s consumption habits.”

Trickling Soda Consumption

According to Harry Balzer, chief industry analyst at the NPD Group, 71% of all Americans have had a soft drink at least once in a two-week span, down from 81% in 2000. He said that makes carbonated soft drinks the fastest declining item in the American diet.

“I always get a kick out of the attempt to ban soft drinks -- as if consumers aren’t doing this on their own,” said Balzer.

Neither Coca-Cola nor Dr. Pepper Snapple responded to a request for comment on the Purdue review study, while PepsiCo referred questions to the American Beverage Association.

“This country is waiting for the Howard Schultz of carbonated soft drinks."

- Harry Balzer of NPD Group

“This is an opinion piece not a scientific study,” the industry group said in a statement last month. “Low-calorie sweeteners are some of the most studied and reviewed ingredients in the food supply today. They are safe and an effective tool in weight loss and weight management, according to decades of scientific research and regulatory agencies around the globe.”

To be sure, the carbonated soft drink category may be a mature market, but it remains a cash cow. Coca-Cola and PepsiCo have a combined market capitalization of more than $300 billion.

“The trend may be down, but this is still huge and an important part of the American diet. There’s still lots of money to be made in soft drinks. This country loves soft drinks,” said Balzer.

But it’s clear consumers continue to migrate to non-soda categories. While carbonated soft drink volume shrank by 1.8% in 2012, energy drink volume soared 14.3%, ready-to-drink coffee jumped 9.5%, bottled water increased 5.8% and ready-to-drink tea gained 4.9%, according to BMC.

Dietary Diversity is Key

With those shifts in mind, the industry continues to try to insulate itself from the soda slowdown.

“I think the big thing for investors is to realize that in the United States the big guys, Coke and Pepsi, have good portfolios of still beverages. They also have vast international diversification,” said Thomas Mullarkey, an analyst at Morningstar.

Despite its name, PepsiCo owns many non-soda brands, including Frito Lay products like Doritos, Tropicana juices, Quaker items such as Rice-a-Roni and sports-drink behemoth Gatorade.

In fact, just 31.3% of its second-quarter sales haul of $16.8 billion derived from PepsiCo Americas beverages, which posted a 2% drop in revenue.

The beverage category in the U.S. “has its challenges, especially carbonated soft drinks,” PepsiCo CEO Indra Nooyi acknowledged during a call last month. She pointed to beverage volume growth in emerging markets, including 24% in the Philippines, 13% in Pakistan and 11% in India.

“We are able to continue to deliver in these trying conditions largely due to the strength of our portfolio, both the geographic diversity and product diversity,” said Nooyi.

Coca-Cola generated 55% of its $12.75 billion worth of second-quarter revenue outside of its North American business. The company said unit case volume fell 1% in North America and 4% in Europe last month, but jumped 9% in Eurasia and Africa.  

Silver Bullet?

Dr. Pepper Snapple appears more exposed to the soda slowdown as it doesn’t own the food brands of PepsiCo and isn’t as geographically diversified as Coca-Cola.

The maker of 7 Up, Canada Dry and Hawaiian Punch relies on carbonated soft drinks for a whopping 80% of its volume. Dr. Pepper Snapple generated just $171 million in revenue last quarter from Latin America, representing less than 8% of its total sales of $1.61 billion.

All of this helps explain why the three beverage makers continue to invest in research in a quest to find an all-natural sweetener.

“This country is waiting for the Howard Schultz of carbonated soft drinks,” said Balzer, referring to the Starbucks (SBUX) founder. “Who is going to give me a reason to drink this stuff?”

Dr. Pepper Snapple touted the launch of TEN, which has just four grams of sugar per 20 ounces, compared with 64 grams in traditional Dr. Pepper.

The company spent $23 million on the launch and said 51% of customers who bought the new drink had left carbonated soft drinks or “not been a user before.”

“It takes time to change people’s consumption habits. But I feel very bullish that we've got the programs right now that can; and our R&D teams, as everyone else in the industry, is working aggressively to come up with a natural sweetener,” said Young.

Investors and health advocates alike are hoping those efforts come to fruition.

Follow Matt Egan on Twitter @MattMEgan5