APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

Value of bit-level verify on backup?

© August 2013 Tony Lawrence

A bit level verify is where you read back a tape (or whatever you backed up to and compare the files one by one (usually by a "sum" or similar idea, but sometimes byte by byte).

You'd probably be surprised how often a bit level verify proves useful.

As many consultants do, I receive backup status email for several of my clients, so I can tell you that bit-level verification "failures" ( failure of one or more files to match the hard drive- not a complete backup failure) are not all that uncommon. Once in a while it's due to incipient tape failure, and it's very helpful to see a problem developing before it gets serious- if we're lucky the really important files aren't the ones the tape had trouble with.

Quite often the failure is due to unexpected activity on the system during the backup. Usually it's explainable, but a couple of times in my long history it has been an employee up to no good after hours and the bit-level verify helped show what they were doing.

Once or twice the bit-level verify has alerted us to incipient hard drive failures. Those usually would have been spotted by other means too, but it never hurts to have multiple channels of alerts. As someone said, if it's REALLY important that you get up at 4:00 AM tomorrow, you set two alarm clocks AND have the hotel desk call you.

Now and then we get "cosmic ray" failures. It's probably really just a wayward piece of dust finding its way to the tape, but the point is that one file fails to verify and we can't see any reason why. We always get suspicious of the tape of course, but sometimes it just goes away and the tape continues on for months. Just because of the odds, such failures are usually an unimportant or easily recreated file, but once in a great while it hits something very important, and that's when we're very happy to have the knowledge that, if we should need it, Tuesday's tape is NOT suitable for restoration because that important file is suspect. The point here of course is the KNOWLEDGE: we KNOW there is a problem on this tape. We probably don't need the tape (not planning on trashing the system today) but if we DID need it, we know that it has a problem.

And yes, we often do other things too. For convenience and redundancy I often have rsync or similar things tucking important bits of data here and there around the network. On really important systems we have more than one system that does tape or dvdram backup and lots of cross pollenization between the systems- it doesn't cost a lot to do this kind of thing and the peace of mind is much improved.

Another way to look at it: what value is there in a backup you aren't certain is accurate? Sure, sometimes it doesn't matter, but sometimes it matters a lot. Trust, but verify. Bit level verify, of course.

Got something to add? Send me email.

(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> Value of bit level bit-level verify backup

Inexpensive and informative Apple related e-books:

Take Control of Apple Mail, Third Edition

Are Your Bits Flipped?

Take Control of iCloud, Fifth Edition

Take Control of Pages

Sierra: A Take Control Crash Course

More Articles by © Tony Lawrence

Printer Friendly Version

Have you tried Searching this site?

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us

Printer Friendly Version

The three chief virtues of a programmer are: Laziness, Impatience and Hubris. (Larry Wall)

Linux posts

Troubleshooting posts

This post tagged:


Tape Drives

Unix/Linux Consultants

Skills Tests

Unix/Linux Book Reviews

My Unix/Linux Troubleshooting Book

This site runs on Linode