When Data Recovery became a reality in a room full of people

by Abhik Mitra on June 17, 2013 in Data Recovery with No comments

I’m feeling a little nostalgic today, so thought I’d share a little incident from the past. As I’ve expressed in prior blog postings, one of the biggest challenges Kroll Ontrack has is successfully communicating the risks of data loss to customers whom have never experienced it. There are multiple reasons why data loss can be a confusing subject, but I believe confidence and confusion are the main ones. Confidence is the idea that data loss can “never happen to me” because we invest in technology and planning to prevent data loss in the first place. Confusion stems from the term “Data Recovery” itself, as many folks like to think of it as yet another term for backup. I’ve been to quite a few conferences during my tenure at Kroll Ontrack, so I decided to do something a little more drastic during last year’s Flash Memory Summit 2012. I decided to crash my own computer!

Nothing quite beats the dramatic effect of a data recovery company experiencing a computer crash in the middle of one of the biggest tech conferences. Putting it together didn’t require much skill and all I really did was  introduce a few blank screens, the dreaded blue screen of death, and some acting skills. In the end my intent wasn’t to fool anyone, but to make a strong statement about what data loss is. It’s not something you plan or budget for, nor can you predict when data loss will strike. And that’s the point… Check out the video here (specifically at 2 minute 45 seconds) to see me pull this off.

Share it!

The Low Down on Recovering Deleted Files

by David Logue on May 28, 2013 in Data Recovery with No comments

Recently, I’ve received several questions related to the recovery of deleted files.  What happens when a file is deleted on a Windows-based system, and what causes those files to be lost and therefore unrecoverable?  Further, what could I have done to prevent their loss?  To answer those questions, we first need to answer another very important question.

How does Windows save file data on a NTFS volume?

When you create a new file, like a picture from your vacation (vacation.jpg), and save it to your hard drive (formatted with the NTFS file system), Windows does a couple things.  It finds an open file record in the metadata area of the disk (called the Master File Table or MFT) and writes some information about the file, such as the file name and date.  If there are no open file records, Windows will expand the MFT and create a new file record.

Windows then finds some free data blocks on the volume to write the actual file data to.  Once the data blocks are identified, Windows links the new file record to the data blocks and writes the actual data to the disk.  The picture below illustrates the vacation.jpg file as written to the disk.

degraded RAID 5 data loss

So what happens when a file is deleted (assuming it is not going into the Recycle Bin)? Two very important things happen (from a data recovery perspective):
1. The file record is marked as deleted and available for reuse.
2. The data area is marked as free space and available for reuse.

degraded RAID 5 data loss

The image above shows the areas of the disk that hold the data for the vacation.jpg file have now been marked as free space and are available for use for new files or to expand existing files. The file record has also been marked as deleted and is available for reuse by the file system.
To recover deleted data, your data recovery company or software needs to be able to find deleted file records that have not been overwritten and the data blocks that relate to those files. The DR company or software should also scan the unallocated space on the disk for data blocks that were in use, but whose file records have been overwritten.
An example of such a process is as follows:

  1.  Limit access to the disk (write blocker)
  2.  Scan volume metadata for file records marked as deleted
  3. Recover deleted file records and their related data blocks into new files
  4. Scan volume for raw data that is currently in unallocated or free areas of the drive
  5. Recover raw data blocks into new files

What are some of the reasons deleted data cannot be recovered?

  1. File record is overwritten and:
    1. No signature for the file data
    2. Data is fragmented
  2. Data is completely overwritten
  3. Data is partially overwritten

The figure below illustrates a file that has been deleted, its file record overwritten by a new file, and the data is fragmented on the drive.

degraded RAID 5 data loss

Our example file (vacation.jpg) has been deleted and the file record overwritten with a new file (birthday.jpg). The only recovery possible for the vacation.jpg file is to find and assemble the raw data blocks (assuming there isn’t another copy of the FR somewhere else on the volume). The success rate for this type of recovery is very high as the data blocks (Blocks 1-4) in our example have not been overwritten by new data.

If the new file (birthday.jpg) had overwritten some of the data blocks like in the example below, then the file would only be partially recoverable (blocks 2 and 3 overwritten).

degraded RAID 5 data loss

If all of the data blocks had been overwritten like the example below then the file would not be recoverable (blocks 1-4 overwritten).

degraded RAID 5 data loss

So what can you do to make sure this doesn’t happen to you?

  1. Backup/replicate your data. I know it sounds cliché, but a simple backup/replication can save a ton of heartache.
  2. If you accidentally delete a file, and don’t have a backup, stop using the system as soon as possible. Browsing the Internet or continuing to work writes additional data to the disk and can cause the data blocks to be overwritten.
  3. Restore a backup to a different drive to make sure that the backup contains the data you need and that the files are in working order.
  4. If you want to attempt the recovery yourself, make a copy of the drive if possible and work on the copy. If it is not possible to make a copy, make sure the drive is slaved as a secondary disk to the system. Do not install recovery software on the drive you want to recover from.
  5. Seek professional assistance. Good data recovery companies offer a free consultation, so you can discuss your specific needs with a data loss expert.

Share it!

Gun laws, smashed hard drives and data recovery

by Abhik Mitra on May 20, 2013 in Data Recovery with 1 Comment
Gun laws, smashed hard drives and data recovery

Yes, it’s all going down in Capital Hill right now with families of the Sandy Hook victims pressing senators for tougher gun measures. Regardless of this outcome, the tragedy of the Sandy Hook Elementary School points to a very disturbing trend in that today’s criminals may be becoming all too aware of the digital footprints they leave behind. Case in point, the perpetrator (no, I won’t add to his celebrity by mentioning his name) who took meticulous steps to erase his digital history. He removed the hard drive from his computer and smashed it in with either a hammer or a screwdriver. Sound a little primitive?

And while this primitive yet highly effective art of taking a hammer to a hard drive could work, I believe it points to an increasingly informed population of ways to permanently destroy data. How did the perpetrator know of this method? Was it the abundance of blogs and other information readily available on the Internet? One thing is clear, the perpetrator was intent on covering his tracks and destroying any evidence that may lead investigators to a motive. All is not lost though and given the multiple devices that we generate data from today, a hard drive isn’t the only key to the puzzle. Many of us tend to forget that the vast quantity of data tracks we create are via some cloud platform or multiple devices connected via the Internet. Whether it’s text messages, e-mails, chats or search history, everything has some sort of repository in an invisible cloud we tend to overlook. The FBI is yet to provide additional details on this front but I’m all ears when they do!

Of course, we here at Kroll Ontrack live by the “nothing is unrecoverable” mantra. Yes, we’ve seen some pretty bizarre and extreme instances of hard drives damaged by floods, fires, and even a melted disk drive from the space shuttle Columbia, which was destroyed in 2003. Whether or not a hard drive can be salvaged depends on the extent of damage to what are called “platters,” the small discs where the data is stored. If the perpetrator was meticulous enough to ensure that all the platters were thoroughly destroyed, data recovery just got a little tougher. But then you also have tablets or smartphones the perpetrator likely owned that operate at a different level. You then have an option to gain access to contacts, calendar information, text messages, pictures, location history, and browsing history.

And this begs the question of whether today’s users truly understand all the data they are truly creating? We don’t live in a “single point of failure” world anymore and cheaper/robust storage platforms have made it all so easy to create dual points for data. Of course, we’re not all aware of these points because our sole purpose is to keep creating data! We take Facebook, Gmail, LinkedIn all for granted because it’s not our headache as to where the information we feed to it is stored. Perhaps the notion of this invisible cloud did not occur to the perpetrator while the hard drive was being thoroughly smashed. Is it possible the perpetrator created a digital footprint by default and didn’t realize it?

Share it!

EMC World 2013: A Reflection

by Troy Ronning on May 14, 2013 in Data Recovery with No comments
EMC World 2013

Baseball season is in full effect, the NHL playoffs are entering round two, and EMC has just concluded its yearly EMC World show in Las Vegas.  With thousands of people attending from around the globe, what a great place to network, learn, and grow your IT knowledge.  To cap off the event, a Bruno Mars concert.  Like him or not, it’s definitely high energy entertainment.

So what did Kroll Ontrack see from our front row seats you ask? For starters, EMC dominated the exhibitor hall with massive booths as expected. The big thing this year was EMC VSPEX Proven Infrastructure.  EMC has gone to great lengths to simplify the solutions for you. If a customer or partner wants to deploy SharePoint, for example, they can choose from a list of proven options to complete their solution. It takes the guessing game right out of it.  I am excited to say that Kroll Ontrack was recently VSPEX Validated with Ontrack PowerControls for SharePoint.

In addition to the big presence of EMC and other various  phenomenal technology companies, the attendees were top notch.  This year I lost count of how many customers stopped by to say “thanks” for making a simple product to make life easier.  There was also an increasing amount of people looking to deploy our solution in conjunction with EMC BRS products (Avamar / Networker).  Data protection savvy customers are the best.

If you missed EMC World 2013, there will be one next year, so please stop by our booth. If you attended we would love to hear about your experiences at the show. What did you find most interesting?

Share it!

Preventing Data Loss During RAID and Drive Rebuilds – Part 2

by David Logue on April 22, 2013 in Data Recovery with 1 Comment
Minimizing Data Loss

This is a follow up to the original article on RAID vs. Drive Rebuilds. Several questions came up about RAID and drive rebuilds and preventing data loss since the original posting, and Part 2 will attempt to address those questions.

One of our readers noted:  “In your first example, parity is missing from stripe 4. You didn’t mention how that stripe can get rebuilt if there’s no parity.”

degraded RAID 5 data loss

The parity is missing from stripe 4 because it should be on the missing or damaged drive.  In other words, in a healthy array it would be at the top of stripe 4.  As to how it is rebuilt, in this example all of the data is intact and the parity sector on HDD 1 in Stripe 4 would be rebuilt by XOR’ing the data from drives 2-4 (P4 = XOR (D9, D8, D7).  See below for a picture of the rebuilt drive.

RAID 5 rebuilt drive

Another question that came up multiple times since the original post relates to other ways drive or RAID data loss or data damage can occur.

One of our readers asked:  “Your second example shows how the data can be lost if the wrong type of rebuild is done, “such as” rebuild parity. Is that the only case? “Such as” kind of implies you could do other rebuilds that would get you in trouble.”

Rebuilds that can cause data loss

There are several types of rebuilds that can happen where data can be lost. Below is a list of some of the types of rebuilds that can cause data loss.

1.  Rebuild parity with zeroed drive (parity overwritten)

2.  Rebuild parity with degraded drive (forced online and parity overwritten)

3.  Rebuild parity with drives out of order (parity and data overwritten)

4.  Rebuild RAID with missing drive (parity and data overwritten)

5.  Rebuild RAID with different stripe size (parity and data overwritten)

6.  Rebuild RAID with different configuration (parity and data overwritten)

As an example, one of the most common RAID data loss cases we see is when parity is updated with a zeroed disk in the RAID configuration (RAID rebuild instead of HDD rebuild). This type of rebuild effectively destroys the original parity and prevents a drive rebuild.  Once the parity is overwritten, the missing user data from the damaged or missing HDD cannot be recreated.

Another scenario where data could be lost is a disordered RAID array, especially during a RAID rebuild. Parity rebuilds on drives that are out of order can end up overwriting good user data. 

RAID 5 disordered array

In the example above, the data that was originally on HDD 3 on stripe 1 is now overwritten with new parity. The parity that is on HDD 4 in stripe 1 is now treated like user data instead of parity causing logical corruption. Furthermore, the data that is on HDD2 in stripe 1 is skewed, also contributing to the logical volume corruption. All of the areas marked in red would be damaged. 

Even if a parity rebuild is not done, there would still be logical volume corruption. This logical corruption often triggers volume repair tools to run (CHKDSK, FSCK, etc.). These repair utilities will try to “fix” the logical corruption when the damage is really at the RAID level, causing even more damage such as deleting metadata and making the system unrecoverable.

Another scenario is where a RAID is rebuilt after a two-drive failure using a degraded drive that has been forced online and a new drive. This rebuild with this combination will overwrite the “good” parity with new “bad” parity, often making the system unrecoverable or the data unusable.

The final example to illustrate is where the RAID configuration changes and parity and data areas are overwritten with the new configuration. 

Let’s assume for this example that we have a RAID 5 array with a stripe size of 64K. The OS will read the data from the stripes starting with HDD1 and the data represented by M1. Then, it will proceed to M2 and then to D1 and so on.

RAID 5 NTFS volume

If the array controller loses the configuration and the user forces the wrong configuration, data damage will occur. In our example, the user has forced a new configuration with a 32K stripe size, effectively splitting the data in half.

RAID 5 new configuration

The OS will read the first half of the first section of metadata represented as M1.1. Then, the OS will jump to the next disk in the stripe and read the first half of the next section of metadata represented as M2.1. This will cause logical corruption, making the data unusable. Often this will trigger volume repair tools to run and “repair” the logical damage, which in turn can cause additional damage and even make the volume unrecoverable.

tips to safely recover from this type of data loss

So how do you protect yourself in the event you run into a situation like this?  Here some tips on preventing this type of drive or RAID data loss :

  1. Image the drives before attempting a rebuild. That way if the rebuild is unsuccessful, your data is protected.  Make sure the imaging program you choose allows for a forensic or sector/block- level image of the disk.
  2. Restore backups to a different volume. This ensures that all important files on the backup are good before possibly overwriting data on the active volume. 
  3. If there is a RAID problem, test the backup by restoring it to a different location or image each drive from the RAID before attempting a rebuild. Sometimes a RAID rebuild does not work correctly and can make the problem worse.
  4. Do not create any new files on the disk requiring recovery or continue to run applications until the important data is recovered. New files can overwrite the files that need recovery.
  5. Do not run FSCK or CHKDSK file system repair tools on a virtual disk unless a good backup has been validated by restoring it to a different volume. These repair tools assume that there is a good backup of the data and can overwrite file pointers to make a file system consistent. If desired, these tools can be run in read-only mode to find any major corruption before repairs are made.
  6. Do not delete any additional files prior to a data recovery of deleted data.   Deleting files includes moving files from the source to another volume.  A move is simply a copy then delete.    If you need a copy of the data from the source, make sure to copy it and not move it.  Additional deleted files can complicate the data recovery.
  7. Do not try data recovery software unless you are sure it will not write anything to the disk that needs recovery. Some recovery software will attempt to write to the source disk and could damage later recovery attempts.
  8. Contact a data recovery professional before attempting the recovery on your own.  A professional can outline the possible impacts your plan will have on the recoverability of the data and offer suggestions for self-recovery.

Share it!

A Pro’s Perspective and other Random Thoughts…

by Troy Ronning on April 15, 2013 in Data Recovery with No comments

A happy day wish to all of the readers as spring is upon us – or it was until the Midwest was hit with a snow storm.  I guess spring is in a bit of a holding pattern.  So much for global warming. 

Despite the dreary weather, I recently had the privilege to be on a webinar with J. Peter Bruzzese, Microsoft Certified Trainer and CIO and co-founder of Clip Training.  For those of you that didn’t have a chance to attend, it’s hosted on MSExchange.org and can be replayed here.  To hear someone so tuned into Exchange talk candidly about the process gaps and challenges within the native toolset helps justify ISPs and our mission.  At Kroll Ontrack, we are about making your life easier by simplifying tasks and helping when disaster occurs.

It also happens to be the season for trade shows and technology events.  Kroll Ontrack will be at SMB Nation, EMC World, and TechEd.  I personally will be at EMC World and TechEd.  If you are attending any of these shows, please stop by to say “Hi” to our team, share some of your data recovery and exchange recovery stories, and register for one of our drawings!

Share it!

Survey reveals that 60% of respondents utilized backup solution but still experienced data loss

by Abhik Mitra on March 27, 2013 in Data Recovery with 4 Comments

According to our recent study of data recovery customers, we found that while 60 percent of respondents had a backup data recovery solution in place at the time of data loss, it was not current or operating properly. As we near World Backup Day on March 31, these findings provide key insight into the importance of diligently monitoring and verifying that a backup is successfully operating and capturing a current, accurate snapshot.

Leveraging a data backup solution is critical for any business or individual protecting against data loss. However, as our recent global survey results demonstrate, even a reputable cloud or external media solution does not always provide predictable results. An effective solution hinges on the user or IT administrator attentively validating that the solution is functioning as expected and verifying that the backup is complete.

Data Loss and Backup Survey Results

We surveyed 600 recent customers from North America (27 percent), Europe (58 percent) and Asia Pacific (15 percent), with one third of respondents experiencing personal data loss, and two thirds having lost business data. After experiencing data loss, 87 percent of respondents indicated they are extremely likely or somewhat likely to seek a backup solution. Of those, nearly 60 percent are seeking an external hard drive solution and roughly one fourth are looking to the cloud to protect their data. The remaining 13 percent of respondents who do not plan to seek a solution cited the time and expense associated with research and administration as the overwhelming barrier to consideration.

Additionally, survey results indicated that backing up to external hard drives was still the most used and sought after approach for both business and personal data. 60 percent of respondents utilized an external drive solution, while 15 percent leveraged the cloud and 15 percent backed up to tape. Regardless of solution, several common scenarios can lead to surprise data loss events:

  • External drive only connected on an occasional basis; backup not automated and instead performed on demand
  • Computer not on during scheduled backup and not configured to perform at a different time
  • Backup software failed
  • Backup ran out of destination space
  • Backup profile did not cover all of the device requiring backup
  • File lost before scheduled backup

Tips for Backup Success

  • Take the time to invest in a backup solution and set up a schedule
  • Ensure backups are running regularly in accordance with the determined schedule
  • Check backup reports for error indications or failure
  • Test backups on a regular basis to ensure data has been accurately captured and files are intact

If disaster does strike and you or your organization experience data loss, contact a data recovery expert at Kroll Ontrack – 800-872-2599.

Share it!

What exactly is the struggle with backups?

by Abhik Mitra on March 25, 2013 in Data Recovery with 1 Comment
Backup recovery solutions

Before I get started, let me be the first to acknowledge that I am guilty as charged. Like many of you, I’m creating an astronomical amount of critical data almost every day, with little to no knowledge that it is being securely and accurately backed up so that I may recover it someday. Working for a data recovery company for the better part of two years, I know a thing or two about the consequences of folks unable to restore backup data. In fact, prior to working at Kroll Ontrack, the idea of backing up data seemed mundane and tedious (unless it was happening behind-the-scenes somewhere). Needless to say, when you hear the data recovery horror stories that some of our customers share with us, you start to think that not backing up your data is a little short of a sin. But therein lays the question. Is the reason people experience data loss quite simply because they’re not backing up their data? The answer actually points to something far more compelling. 

60 Percent of Backup Solutions Not Operating Correctly

In a recent survey of our data recovery customers, 80 percent of respondents had a backup solution in place; however, in 60 percent of the cases the data backup was not current or not operating properly at the time of loss. What didn’t surprise me about the survey results was the 80 percent of responses that indicated a current data backup solution. What continues to surprise me is the 60 percent that have no utility from their existing backup solutions! Of those respondents whose backup was not current or not operating correctly, 60 percent utilized an external drive solution while 15 percent leveraged either cloud or tape backup. We might be on to something there… External hard drive backup is still the most used and sought after approach to backing up both business and personal data.

I would argue that the technique of backing up data to an external hard drive is far from automated. And it is that lack of automation that can make backup technology become one of the most overlooked factors in an overall IT or Disaster Recovery strategy. I think part of this is the cultural mindset of backups too. In fact, how many of you find yourselves skipping the page about backing up to iCloud when setting up your new iPad’s? That’s an example of an automated backup utility that we often choose to ignore. The fact is that unless there is a certain element of hand holding in the backup world, the technology can be pretty useless.

After all, what good is a backup without:

  • A user setting up a regular backup schedule
  • Ensuring backups are running regularly in accordance with the determined schedule
  • Checking backup reports for error indications or failures
  • Testing backups on a regular basis to ensure data has been accurately captured and the files are intact

Let’s face it, the security of having a backup solution, while comforting, doesn’t answer the question of how you’re going to use it when you lose your data. In fact, Kroll Ontrack gets called in on numerous occasions to help customers that are often clueless about their backup technology and in some instances have no idea how to recover backup data. All of this is, of course, based on my humble opinion of what we might be seeing with backup solutions.

Why do you think there is such a gap between the number of individuals that are backing up their data versus those that can recover backup data data during a data loss event?

Share it!

Mobile device data recoveries up 161%

by David Logue on March 21, 2013 in Data Recovery with No comments
Mobile business concept

If you can believe it, the International Telecommunication Union predicts the number of mobile phones worldwide is expected to exceed the world’s population by 2014. With the increase of mobile devices comes an increase in need for mobile device data recovery. Kroll Ontrack has seen major increases in mobile device data recovery over the past few years:

  • 2010-2011: 55% increase
  • 2011-2012: 161% increase

Physical damage is the most common cause of data loss seen by the company. This includes drops, human error and water damage. The rest of the data losses occur from logical failures, such as accidentally deleted files, corrupt software, password lockout and OS upgrade issues.

Ontrack Data Recovery engineers report that in 2012, for recovery resulting from physical failure:

  • 31% of cases were electronics-related physical damage
  • 23% were the result of water or moisture damage
  • 7% were related to damage to the exterior of the device

For recovery resulting from logical failure:

  • 26% were the result of deleted files
  • 7% were software corruption and six percent were cases of password lockout

Mobile device data recovery process

In instances of physical damage, Ontrack Data Recovery engineers open the device within a cleanroom environment and assess the physical condition of the circuit boards and parts through a comprehensive diagnostic process. The mobile device’s printed circuit board (PCB) parts are examined and repaired as needed to get the device to a state where the data can be read. When there is logical failure, engineers use specialized software to bypass the identified issue and then access and extract the data.

Tips for handling data loss

The most requested data to be recovered from mobile devices are photos/videos and contacts, followed by notes and text messages. To promote the best chance of success in recovering this valuable data, use these tips:

Time is of the essence

Power off the mobile device immediately and get it to a reputable data recovery provider. The longer you wait, the more likely critical data will be overwritten (deleted files) or the drive will corrode (physical damage such as water).

Backup, backup, backup

Before disaster strikes, back-up your data to another device, such as a laptop, the cloud or an external drive. If you get an operating system error, this backup is often the saving grace in the recovery process.

Know what you want.

The key to recovering data quickly is to know what data to target. Communicate to your data recovery provider what data is most critical to better ensure a timely and accurate recovery.

But these aren’t the only tips Kroll Ontrack has to offer. Visit the website for more tips on protecting your data.

Share it!

A view from the bullpen, are you ready when the phone rings?

by Troy Ronning on March 18, 2013 in Data Recovery with No comments
Baseball

If you stop and consider the complexities of an IT Department, you can draw many similarities between a baseball pitcher in the bullpen and a systems administrator.  Imagine it’s the bottom of the seventh inning when the phone rings, telling you to start warming up.  You know what your job requires and you have done it before, but were you really ready for the call? What things have you considered, or alternatively, not considered?

An IT systems administrator is in a similar position.  It could be late in the day, on a Friday none the less, when the phone rings and the Director of Human Resources has an urgent request for an employee’s mailbox from the past year.  To complicate matters, the Director needed this information yesterday.

To give the average corporate employee credit, the majority consider this to be a simple task; however most individuals are unfamiliar with the challenges of Exchange server recovery.

As a systems administrator, there are a variety of challenges to consider.  Do you have space for restores or even Exchange recovery servers ready for deployment?  Are you a PowerShell expert and do you have scripts ready for extraction?

If the answer is yes, you are probably in the minority. But what if you’re a systems administrator with a tool that can accomplish this type of task in minutes? Ontrack PowerControls is one way you can answer the call and quickly get the data you need while saving yourself the traditional headaches associated with these tasks.

So the question is, when the game is on the line and that phone rings are you ready for the call?

Share it!

← Older Entries