Just nine months ago BATS Global Markets was barreling towards an initial public offering that would have allowed the upstart exchange operator to invest new capital to better compete with Nasdaq and the New York Stock Exchange.

After an embarrassing glitch scuttled its own IPO in March, this week BATS is cleaning up another PR black eye, this one caused by a “system issue” that fouled up nearly half a million orders over a four-year span.

While the actual cost to compensate investors should be relatively small, the latest stumble threatens to further tarnish the exchange's reputation and simultaneously add to growing investor angst over the increasingly complex and intertwined equity market.

“Technology is supposed to make our lives easier, but the lack of any ability to assure investors and traders that technology is working -- either as required or as advertised -- is damaging our markets and those responsible for overseeing them,” Harvey Pitt, former chairman of the Securities and Exchange Commission, said in an email.

BATS Back Under Scrutiny

The latest BATS trouble emerged Wednesday night when the company fired off a notice to customers saying that a recently-discovered “system issue” caused over 400,000 transactions since 2008 to be processed at prices that weren’t the best available. In the industry, this type of problem is called a “trade through.”

Regulation National Market System, or Reg NMS, rules require that trade orders are executed at the best available price, no matter the location. For example, if a sell order comes in at $99 at BATS and $100 at NYSE, it must trade through the Big Board’s exchange.

Therefore, BATS appears to be admitting to breaking RegNMS rules on a large scale and over a long period of time, at the detriment to clients.

“Where the heck were the regulators?” said Joe Saluzzi, co-head of trading at Themis Trading and co-author of Broken Markets. “Fines need to be imposed. If I violated Reg NMS, the SEC is going to come down on me, I can tell you that. Do your job, Mr. Regulator.”

Pitt questioned how it was possible for these problems to have “gone undetected for four minutes, much less four years.” He added, “the SEC cannot rely on those who create, run and participate in modern markets to catch technological glitches.”

BATS said it is working with the SEC to see if it can compensate investors retroactively. The exchange sees the total impact to customers at just $420,360, which pales in comparison with the cost of other recent exchange problems.

While the cost seems relatively small, the problem comes at a difficult time for BATS, which is still recovering from a disastrous IPO that had to be scrapped due to technical glitches on its own exchange.

Those IPO troubles at least briefly undermined faith in the technology at the heart of the company’s business.

“Cut them a break,” Dennis Dick, a market structure consultant quipped on Twitter. “They are starting to become the Detroit Lions of the stock exchange world, just finding a way to lose.”

Lenexa, Kan.-based BATS didn’t respond to a request for comment.

Pitt said the new disclosure, on top of the IPO difficulties, “has to augur poorly for BATS' reputation unless it moves quickly to redress these problems.”

It’s too early to tell if the exchange’s business will also suffer damage from this week’s news, but BATS was able to quickly recoup orders after its IPO debacle.  

“I don’t think it’s going to harm them too much,” said Joel Hasbrouck, a professor at NYU who in the past has served as a consultant to exchanges and regulators. “BATS is a fairly major player and in order for an institution to trade at the minimum cost they really do need to be accessing all sources of liquidity.”

Gaston Ceron, who covers Big Board parent NYSE Euronext (NYX) at Morningstar, also believes it’s possible for BATS to bounce back.

“I would not discount the resiliency of their management team and their company,” said Ceron. “Yes, it will give them another black eye and there might well be some fallout. But this is group of folks who have built another strong competitor.”

Another Day, Another Glitch

Even if BATS can rebound, the latest disclosure highlights a growing uneasiness among some investors about the safety and soundness of the incredibly fragmented equity markets.

“It would just be nice if at some point we got to a place where this stream of various snafus at exchanges would take a pause for a bit,” said Ceron.

While most trades used to take place in one place, they are now routed through an opaque maze of pipes that don’t always work as planned. There are now concerns that the markets have become too complex or that the technology underpinning them is not as sound as it should be.

Jitters about market structure hit a crescendo after the May 2010 flash crash, but were also bolstered by Facebook’s (FB) flubbed IPO at Nasdaq OMX Group (NDAQ) last year and the near-implosion of Knight Capital Group (KCG) last summer after a $400 million trading glitch.

“These problems underscore how slender a grasp both market regulators and market participants have on the dominant reliance today's markets have invested in technology,” said Pitt.

Each of these events threatens to further undermine confidence in the system itself.

“People are throwing up their hands and saying, ‘How do I trust the stock exchanges?’ Not even just the large banks, but I can’t even trust the stock exchanges now,” said Saluzzi. “These events will happen more frequently until you create another flash crash environment and everyone in D.C. wakes up.”

Follow Matt Egan on Twitter @MattMEgan5