It was a perfect day in Atlanta, and I had just returned to my hotel room to write my column after a breakfast meeting with Oliver Rist and some PR representatives when our colleague Alyson Behr called with words that will forever bring chills. "Wayne," she said, "turn on your TV." I watched, stunned, as the crew on NBC's Today Show narrated the fall of first one and then the other tower of the New York's World Trade Center. The trade show we were attending suddenly seemed very far away.
Of the 2,606 people who died in the two towers on September 11, 2001, 658 were employees of financial services firm Cantor Fitzgerald. That number represented two-thirds of the company's workforce. Because the offices in the World Trade Center were the company's corporate headquarters, this meant the company's records as well as most of its corporate knowledge were lost. Yet, a week later, while smoke was still rising from the ruins, the company was back up and running and conducting trades.
The reason Cantor Fitzgerald was able to resume operations so quickly can be ascribed to two things. First, company CEO Howard Lutnick had been off-site in a meeting, and second, the company had created a backup site several miles away and linked the headquarters and the backup site with a fiber network. Most of the activities on the company servers were mirrored on the backup servers in near-real-time.
This was a case in which thoughtful planning on the part of the company executives and the IT department allowed Cantor Fitzgerald to stay in business while many other firms simply disappeared. That planning included disaster preparation, a means of disaster recovery (DR), and the will to make sure it happened. But it never would have been possible had the off-site data backup not been there.
Hurricane Florence and Other Disasters
While it's easy to say that such a disaster might never happen again, the fact is, different kinds of disasters happen every year. For instance, as I'm writing this, Category 4 Hurricane Florence is bearing down on the East Coast of the United States, and it's certain to destroy the facilities of thousands of small to midsize businesses (SMBs), not to mention disrupt the lives of millions of people. It will certainly be days, and possibly weeks, before some SMBs can resume operations and some of those millions of people can return to work. In the meantime, what happens to those SMBs?
If their data isn't preserved and a recovery plan isn't in place, those SMBs may never recover. A critical factor will be whether those SMBs have found a way to protect their data by getting it far away from the storm's impact area. While it's possible to accomplish data protection by loading backup tapes into a fleet of trucks and driving them away from a disaster, that only works if you have a few days' warning. If a disaster arrives with little warning, then you could be, well, screwed.
Real-Time Backup for Data Recovery
What you need instead is real-time backup, along with some means of recovering your data and for getting your SMB back up and running. There are two ways to accomplish this. The first and most comprehensive way is to involve a full-service DR service such as Sungard Availability Services, which will provide cloud-based backup and support for your business operations until you can return to your offices or build new ones. They will even provide office space, and reroute phone and data lines so you can continue operations.
The second method is to engage a DR-as-a-Service (DRaaS) company that will provide cloud-based storage and help with restoration. In our tests of such services, we found that many of the same companies that we reviewed for Infrastructure-as-a-Service (IaaS) also provide DR services; others are primarily cloud backup services, though a very few are specialized only in DR.
Off-Site Backup Step by Step
It's important that you keep several things in mind while you're getting set up for off-site backup, including the fact that you may need more than one kind of backup. Here are some things to consider:
- Not all cloud-based DRaaS providers are equal. This isn't necessarily a measure of quality. More often it has to do with how the vendor architects their solution and whether that architecture is what will work best for your business. You'll need to evaluate them, considering factors such as backup speed, the amount of data you need to back up, what kind of data it is and how fast your business is generating it, and, finally, how easy and quickly it can be recovered once you need it.
- Geographical diversity is crucial. You don't want the same disaster that just took out your primary data site to also take out your backup and recovery site. Because most natural disasters can cover a wide area, this means finding a provider that has its sites in some other part of the US or even outside the US. Often, having backups in three geographic locations makes the most sense for businesses that absolutely need to ensure their data will be available no matter what. As I mentioned earlier, as I write this, Hurricane Florence is barreling towards the Carolinas and she's a perfect example of a wide-site disaster in the offing. Hurricane Florence will cover a large enough area that a company in Savannah, Georgia won't want their backup to be in Baltimore, Maryland because both cities may be badly hit. And if she turns north, even if your data is stored in New York, it might not be safe. Having sites in New York and Chicago, for instance, would be a more effective DR architecture.
- Duplication of your data to multiple data centers is important for the same reason that geographical diversity is critical. You need to reduce the chance of your data being lost.
- Make sure you're implementing a rational recovery process. You need to be able to get your data back quickly enough to be useful. If your recovery choice is a download over the internet, then you probably won't be able to restore your business if it needs to be up in hours. Make sure your architecture will recover your infrastructure in a timeframe useful to your business and then test the solution on a regular basis to make sure it remains effective.
- Continuing on the point just discussed, make sure there's a way to practice this data recovery. You need to know exactly how to recover your stored data and you need to be able to document the process step by step. The only way to do this is through practice. There are few people more frustrated than a CEO whose IT staff spent money on a DR process, only to be unable to operate it once the fateful day actually arrives.
It's also important to remember that you may need two types of backup. This is typically dependent on the kind of data your company generates and the way in which it uses that data. For companies that manipulate data via stream or micro-second transactions, they'll likely need one link in the form of a high-speed fiber connection for real-time data. That connection will need to be relatively close due to latency considerations. Then a second link would be a remote backup that could operate at slower speeds, but which would avoid being hit by the same disaster.
By the time you read this, it will probably be too late to get ready for Hurricane Florence, especially if you're in the Carolinas. But when that storm's over, it will be a good idea to get ready for the next disaster—which will be coming far sooner than any of us want it to, never you doubt it.