The software glitch that cost Knight Capital Group $440 million in just 45 minutes reveals the deep fault lines in stock markets that are increasingly dominated by sophisticated high-speed trading systems. But Wall Street firms and regulators have few easy solutions for such problems.
Automated trading can handle massive volumes of transactions in milliseconds, something human traders could never do. But the benefits come at a cost: stock markets have become a jumble of exchanges, market makers, high-frequency traders, and investors using different systems that can interact in unexpected ways.
The May 2010 'Flash Crash', in which U.S. stocks inexplicably sank in a matter of minutes, illustrated how technological problems can cascade. These sorts of problems may be more likely given that many market participants are under pressure to cut costs - including technology spending - as trading margins narrow and regulation costs increase.
Since April, a series of embarrassing and costly technology issues have rocked markets and shaken the confidence of investors.
BATS Global Markets, an exchange, was unable to complete its own initial public offering because of a technical problem. Nasdaq botched the market debut of Facebook due to technical glitches, costing it tens of millions of dollars, while UBS AG lost more than $350 million in trading Facebook shares and is blaming Nasdaq.
"The structure just may be too complicated to work," said Larry Tabb, founder of Tabb Group, a consulting firm that focuses on capital markets.
"We may need to think about changing the structure of the market to simplify this," Tabb added.
Others argue that regulators can continue to take relatively simple steps, such as creating more finely tuned mechanisms to halt stock trading when trading volumes or prices move too much.
Bernard Donefer, who teaches at the NYU Stern Graduate Business School and CUNY's Baruch College, said that new, broader limits on stock swings that are slated to be tested by regulators in February would have helped.
If the latest cars can sense approaching objects and stop automatically, "we need those kinds of controls" in the markets as well, he added.
In the old days, it was simpler: human traders known as "specialists" worked on the floors of stock exchanges, such as the New York Stock Exchange, to match buyers with sellers and complete trades themselves if matches couldn't be made.
But over the past decade, those specialists have been replaced by automated trading systems, and much trading volume has moved away from exchanges and into other venues, such as "dark pools" - trading systems that let investors anonymously buy or sell larger blocks of stock without tipping their hand to a wider market.
Knight Capital, which makes most of its money by executing trades for other brokerage firms, said that a new trading system it installed sent a flood of bogus trades to the market. The loss means that the firm is now fighting for its survival.
Outsiders questioned why Knight did not pre-test the new software more assiduously, and why the bad trades continued to be generated for more than half an hour, instead of being shut down by internal systems almost immediately.
New York quantitative hedge fund owner Roy Niederhoffer said that there was "no excuse" for Knight failing to act sooner.
"This is like a nuclear reactor or aircraft," said Niederhoffer, whose R.G. Niederhoffer Capital Management uses Knight. "There has to be some way of seeing the state of the whole system."
Trading system glitches often arise with the installation of new systems or computer code. But experts said that market participants constantly need new systems to accommodate new rules and changing customer demands - particularly from high-frequency traders.
For example, many trading systems were modified this week to conform with a new retail order-taking system introduced at the New York Stock Exchange and to accommodate a French securities transaction tax that took effect on Wednesday.
The U.S. Securities and Exchange Commission has been grappling for years with ways to create a national market system that uses technology to ensure that orders to buy and sell shares are sent to the best possible exchange, dark pool, or other venue. After the shock of the 2010 Flash Crash, it also began exploring fail-safe mechanisms to prevent technology-induced disasters.
Some of those measures, including halting trading in a stock that rises or falls beyond predetermined limits, kicked in on Wednesday, protecting investors but not saving Knight from losing big.
To be sure, many experts pooh-poohed the wider significance of Knight's problem.
"Coding problems happen, but it's not an industry issue," said Matt Andresen, founder of proprietary quantitative trading firm Headlands Technology and a former trading head at a Knight competitor. "I don't know what happened at Knight, but they had a self-inflicted problem that only hurt them. That's the way the market is supposed to work."
Maureen O'Hara, a finance professor at Cornell University who sat on a special advisory panel that explored Flash Crash reforms, agreed. "I'm very worried people will take a look and say there is something fundamentally wrong with the market, and there isn't," she said.
Some traders and critics said that more details need to come out before they could draw too many conclusions other than the importance of software testing.
One thing that's clear: the financial system is much more elaborate now than it was a decade ago, and finding solutions to market problems is not easy.
"We have a very complex financial system, bordering on the baroque," said NYU's Donefer.
(Reporting By Jed Horowitz, Daniel Wilchins and Lauren Tara LaCapra in New York, Joseph Menn in San Francisco, Sarah Lynch in Washington and Jeff Lukes in London. Writing by Dan Wilchins and Jed Horowitz; Editing by Martin Howell and Muralikumar Anantharaman)