Tips on How to Quickly Recover from Data Corruption

Bounce back from loss in minutes, not hours or days

Many businesses that rely heavily on network data to conduct business operations, interact with customers, process payments or for remote access for off-site employees believe they are protected from data corruption or network failure because they have a “backup and recovery” system in place.

Unfortunately, this false sense of security can leave a company vulnerable to a lengthy and potentially crippling business interruption, so much so, that many businesses each year experience significant loss of revenue, diminished customer confidence, and, yes, even go out of business completely – despite having some type of backup system in place.

Vulnerabilities of Network Backup and Recovery

Recovering data from backups can take many hours, and even days if associated with hardware failures. In addition to the critical nature of the data, a failure of the application, database or email server can leave the business unable to access that data, place orders, bill customers, etc.

For this reason, for companies that rely heavily on electronic medical records or transactions, such as healthcare providers and medical practices, any solution that does not restore the network to full functionality within minutes – no matter the cause or point of failure – is simply a non-starter.

“In healthcare, the ability to access data is extremely important,” said Justin Huckaby, IT manager at CMA Healthcare, an independent multi-specialty medical practice in South Carolina. “Physicians now rely on electronic medical records to make good healthcare decisions for their patients.”

Business Continuity in Layman’s Terms

The odds of a significant network event are much higher than most business owners realize. Data can be corrupted or lost for a variety of reasons including hardware failures of servers, hard drives, user error, malware, file sharing and poor connection issues during remote access. As a result, data backup and recovery has evolved into what the IT industry now calls “business continuity.”

At its core, there are two fundamental aspects to business continuity:

  1. Back-ups involve copying network data to tapes, network attached storage (NAS), local hardware appliances or the “cloud.”
  2. Recovering that information quickly in the event of data loss, corruption, or server failure is the blind spot in the system that leaves them vulnerable to periods of extended downtime.

Retrieving terabytes of data from tape, or the cloud, for example, can take many hours, or even days, depending on download speeds. If the cause of the data loss is hardware-related, the restoration cannot even begin until the system is repaired or replaced.

According to IT experts, the other issue is that just because a backup exists does not mean the data is pristine. In fact, backup data can be corrupted as easily as network data and is far from a rare occurrence.

Comprehensive Business Continuity Solutions

So what does a complete business continuity solution look like for those with only a working knowledge of IT? Here are five factors that, combined, will be found in a comprehensive business continuity solution.

1. Image-based backup of key systems

Some basic backup systems copy select files to tape or other network storage devices. This allows for limited retrieval of data, with some of it potentially excluded and the process of both backing up and recovering the data (when needed) is typically very slow.

When CMA Healthcare’s Huckaby was first hired, the business relied on tape backups that were time-consuming, difficult to manage and easily corrupted. Although he lobbied for a better solution, it was only after a major hardware failure to the server that stored electronic medical records in 2010 that he was able to convince ownership to make a change. The event took the system offline for 3-4 days.

Since that time, Huckaby says, CMA Healthcare has entrusted its data backup and recovery a business continuity solution that is designed to restore key systems and data within minutes, no matter what the cause, or point of failure. The solution offered more advanced, image-based backups that created a copy of the operating system and all the data associated with it, including the system state and application configurations. The backup is saved as a single file called an image. The advantages of this approach are that select files can be restored within minutes, or if necessary, the entire server can be restored even if brand new.

2. Backups occur frequently to a local hardware appliance

Ideally, backups would be completed hourly, and possibly even more frequently for critical real-time data. Tape backups are often set to perform this work during the night and, depending on the amount of data, may not even be finished by the morning.

Image-based backup, on the other hand, can be completed much faster. “The system that these new solutions offer can be configured to take ‘snapshots’ of our servers as frequently as every 5 minutes,” explained Huckaby. “So, we identified the servers that handle all of our healthcare information and we back them up multiple times per hour. Less critical servers are backed up once an hour, so we know that if there is ever an issue we have a backup that is no more than an hour old that we can restore quickly.”

3. For redundancy, locally stored data would also be backed up in the Cloud

When anticipating every possible scenario where the system could break down, it makes sense to backup any local hardware appliances to the Cloud. In this way, if the local hardware appliance fails or is destroyed in a fire, flood, earthquake or other natural disaster then the entire network can be accessed directly from the cloud. While rare, these events happen with more regularity than many realize.

4. Back-ups are tested daily to assure the data is not corrupted

Although there are some rudimentary tools for checking that a backup was completed successfully, and in some cases that the data is not corrupted, these are limited and often infrequent. For smaller companies, this monitoring is often assigned not to IT, but to business owners, office managers or other staff.

To protect against potential data corruption of the backups, these new IT service provides are going a step further and conducting daily testing and verification of image-based backups. The daily reports are then sent to the client once a week to show their clients that the backup was tested and in good working order.

5. Back-ups, whether local or in the cloud, can act as virtual servers in a pinch

In addition to the data itself, the backups are configured so that in the event of server or other hardware failure they can be booted up and act as a “virtual” server. For the users – whether employees or customers – the “virtual” network functions and acts exactly like original server did. So much so, they will not recognize the difference.

Once the hardware is repaired or replaced, tested and back online, all the data (including everything changed or added during the downtime) is copied to it and the switch made back to the actual server.

Cost of Network Recovery in Minutes

According to CMA Healthcare’s Huckaby, business owners should view a comprehensive business continuity solution like insurance. “Nobody likes paying for insurance but they are sure glad they pay for it when they need to use it,” said Huckaby.

About The Author