Best of the Newsgroups: cpio backup fails

If this isn't exactly what you wanted, please try our Search (there's a LOT of techy and non-techy stuff here about Linux, Unix, Mac OS X and just computers in general!):

From: Bela Lubkin <belal@sco.com>
Subject: Re: Help with solving an error message on a cpio backup
Date: Thu, 13 May 2004 07:30:40 GMT
Message-ID: <20040513073040.GK10272@sco.com> 
References: <40a2c606@news.iconz.co.nz> 


David Font wrote:

> I am trying to solve an error message with OSR5 site performing a 'relative'
> cpio backup of the Operating System "/" filesystem.
> 
> This is a OSR505 system, that was upgraded nearly 2 years ago to OSR506
> - It is patched with rs506a, oss635a, oss636a, oss639a.
> - There are 2 hard drives which are IDE IBM 30GB.
> - The tape drive is a Tandberg 4GB/8GB attached to SMDS-type SCSI adaptor
> installed with the 'bhba' driver.
> 
> When performing the / filesystem backup, the commands are as follows:
> 
> cd /
> find . -mount | cpio -ocv | compress -H | dd of=/dev/rct0 conv=sync
> 
> This has worked before at this site and at many other sites over the years,
> on OSR502 - OSR507.
> 
> This site however reports a failure about 5 minutes into the backup, the
> error, noted below, on descriptor 2 is intercepted by the script running the
> cpio:
> 
> dd: write error: No such device or address (error 6)

A cursory look at the "Stp" source makes me think that this must be due
to a tape "end of medium" condition.  That would be where the drive
reports to the driver that it's seen a media-specific "the end of this
tape is coming up soon" marker on the tape.

You shouldn't see that 5 minutes into the backup; at least, it seems
unlikely.

I don't have any definitive answers on this.  But let's look at your
backup command for a moment:

> find . -mount | cpio -ocv | compress -H | dd of=/dev/rct0 conv=sync

The actual writing to the tape is being done by `dd`.  You haven't
specified a block size, so dd will use its default, 512 bytes.  That is
almost certainly a horribly inefficient block size for that drive (or
almost any tape drive ever made).  As a result, the drive probably
spends a lot of time "shoeshining" -- seeking back and forth on the tape
because it's lost track of where it was.

Shoeshining shortens the life of both the drive and the media.  It also
reduces the effective size of the media, because after each write-seek
cycle, the write head is positioned somewhere after the previous write
-- you'd like it to be _immediately_ after, but in practice, most drives
waste a _lot_ of space between writes.

You're also using "conv=sync".  `dd` loops, reading a chunk of its block
size (here, 512 bytes), then writing that chunk out.  Without
"conv=sync", if one of those reads comes up short (say it only got 120
bytes), it writes a short block to match.  With "conv=sync", it writes
the full block size; in my example, the 120 bytes it successfully read
plus 392 bytes of 0.  You're compressing your data stream.  If dd ever
actually did this, the inserted 0 bytes would corrupt the compressed
data stream; you would be unable to uncompress from the tape.

What you really want to be doing is reblocking the `compress` output
into chunks large enough to make the tape drive happy, and absolutely
_not_ using "conv=sync".  Something more like:

find . -mount | cpio -ocv | compress -H | dd of=/dev/rct0 obs=16k

This uses the implied input block size (ibs) of 512 bytes.  When ibs !=
obs, dd reblocks.  You wouldn't want to use "bs=16k", which sets both
ibs and obs; when they're the same, dd doesn't reblock, it just copies
the result of each read directly out as a same-size write.  Since you're
reading from a pipe, you'll never get more than 5K in a single chunk;
not enough to make the drive happy, and an odd size that might make it
especially unhappy.

When dd is reblocking, nothing prevents the last block from being short.
If the drive is especially picky, that might be a problem.  You may
originally have added "conv=sync" because the very last block was being
written short, aggravating some particular drive.  Padding the end of
the file wouldn't hurt your compressed stream, and would mollify the
drive.  To handle that, you could use dd _twice_ in the pipeline, once
to create large blocks and a second time to pad the final short block;
something like:

... | dd obs=4k | dd ibs=4k obs=16k conv=sync

"conv=sync" pads _input_ blocks, so the final write is guaranteed to be
an exact multiple of 4K.

Next, you could set the block size being used by the tape _driver_ by
using `tape -a 16384 setblk`.  Sometimes this matters, sometimes it
doesn't.

I suspect that the overall solution to your problem lies somewhere in
the general area of block sizes.  As you can see, this may be a bit
tricky to figure out.

Automatically figuring out this sort of stuff is one of the many
benefits of the various "super-tar" commercial backup programs.  They
also have extended diagnostic capabilities and very capable tech support
staff.  Aside from whatever technical issue, you are creating this
problem for yourself by trying to build a "sophisticated" backup
procedure out of the crude tools at hand.  There are other problems with
the way you're compressing (for instance, a single bad bit on the
tape guarantees that every file past that point will be completely
unreadable).

> This error is occuring in directory:
> /usr/lib/custom/custom/DBCache/SCO:OpenServerCD
> 
> As the user had just decided to purchase their own media, I initially
> considered there was a media error or a media type incompatibility. The
> reason I originally thought this was because I sent one of my tapes down and
> the cpio to that tape had worked.
> 
> However there is only a finite number of times the user can go back to their
> retailer and request a replacement media, and the matter is dragging on and
> causing the user exasperation and for me concern in case there is a future
> catastrophic failure.
> 
> This site successfully performs  'cpio' backups of other directories and
> filesystems using the same commands, just the "/" filesystem backup fails.
> 
> Does anybody know what (error 6) actually means?
> Am I looking in the wrong direction by blaming the quality of the media.?
>  Is it possibly a disk error?

It sounds like it's in the tape subsystem (drive, driver, or bad
orders).

Who's paying for all this time?  Grab a free test install from
www.backupedge.com, www.lonetar.com, www.ctar.com, www.bru.com.
Whichever one you try, it will probably do the job much faster and take
up less tape; and if it fails, the failure will give you a lot more
information about what went wrong.

>Bela<

Take Control of Screen Sharing in Leopard

Take Control of Permissions in Leopard

Jump to Comments

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them. I resell or can earn commissions from the sale of some of these items. Links within these pages may be affiliate links that pay me for referring you to them. That's mostly insignificant amounts of money; whenever it is not I have made my relationship plain. I also may own stock in companies mentioned here. If you have any question, please do feel free to contact me.

Specific links that take you to pages that allow you to purchase the item I reviewed are very likely to pay me a commission. Many of the books I review were given to me by the publishers specifically for the purpose of writing a review. These gifts and referral fees do not affect my opinions; I often give bad reviews anyway.

We use Google third-party advertising companies to serve ads when you visit our website. These companies may use information (not including your name, address, email address, or telephone number) about your visits to this and other websites in order to provide advertisements about goods and services of interest to you. If you would like more information about this practice and to know your choices about not having this information used by these companies, click here.