APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed


Time Machine Details

On the face of it, you might say that Leopard's new "Time Machine" is just a fancy front end around rsync and something like Rsnapshot. In fact, if you look at the directory structure of "Backups.backupdb" on a Time Machine drive, you'll find something that looks like this:

2007-11-05-201839 2007-11-06-112856 2007-11-06-162908
2007-11-06-073038 2007-11-06-122633 2007-11-07-052821
2007-11-06-082639 2007-11-06-132636 2007-11-07-055440
2007-11-06-092632 2007-11-06-142912 2007-11-07-065450
2007-11-06-102629 2007-11-06-152913 Latest

Very familar. And if we look inside "Latest" at a file I know has not changed since the first backup, we see that, yes, it has multiple hard links:

540960 -rwxr-xr-x@ 14 apl apl 153 Jun 24 05:48 list.pl

A file that has changed shows fewer links:

618108 -rw-r--r--@ 9 apl apl 5 Nov 6 10:57 t

So, like we said, just a pretty front end around something like Rsnapshot, right? Umm, yes, but there's more than that..

Back in "Latest", let's see what's going on here. You may have noticed the "@" that "ls -li" showed after the file permissions? In Mac OS X Leopard, that means there is metadata stored with the file. What's in the metadata? Let's see:

First, we'll look at a file that has not changed since the first Time Machine backup. Here it is in the "Latest" directory:



$ ls -li list.pl; xattr -l list.pl
540960 -rwxr-xr-x@ 14 apl  apl  153 Jun 24 05:48 list.pl
com.apple.metadata:_kTimeMachineNewestSnapshot:
0000   62 70 6C 69 73 74 30 30 33 42 2D 63 C3 7F 00 00    bplist003B-c....
0010   00 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00    ................
0020   00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0030   00 11                                              ..

com.apple.metadata:_kTimeMachineOldestSnapshot:
0000   62 70 6C 69 73 74 30 30 33 41 A9 BF B4 32 00 00    bplist003A...2..
0010   00 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00    ................
0020   00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0030   00 11                                              ..

And here it is in the oldest:


540960 -rwxr-xr-x@ 14 apl  apl  153 Jun 24 05:48 list.pl
com.apple.metadata:_kTimeMachineNewestSnapshot:
0000   62 70 6C 69 73 74 30 30 33 42 2D 63 C3 7F 00 00    bplist003B-c....
0010   00 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00    ................
0020   00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0030   00 11                                              ..

com.apple.metadata:_kTimeMachineOldestSnapshot:
0000   62 70 6C 69 73 74 30 30 33 41 A9 BF B4 32 00 00    bplist003A...2..
0010   00 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00    ................
0020   00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0030   00 11                                              ..

As expected, the inode is the same (hard link) and so is the metadata. Notice the names of the metadata? OK, so now let's look at a file that has changed. First in "Latest":


$ ls -li t.pl; xattr -l t.pl
610745 -rwxr-xr-x@ 13 apl  apl  93 Nov  6 06:56 t.pl
com.apple.metadata:_kTimeMachineNewestSnapshot:
0000   62 70 6C 69 73 74 30 30 33 42 2D 63 C3 7F 00 00    bplist003B-c....
0010   00 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00    ................
0020   00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0030   00 11                                              ..

com.apple.metadata:_kTimeMachineOldestSnapshot:
0000   62 70 6C 69 73 74 30 30 33 41 A9 C1 29 E0 00 00    bplist003A..)...
0010   00 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00    ................
0020   00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0030   00 11                                              ..

And then in the oldest:


$ ls -li t.pl; xattr -l t.pl
551149 -rwxr-xr-x@ 1 apl  apl  84 Nov  5 17:14 t.pl
com.apple.metadata:_kTimeMachineNewestSnapshot:
0000   62 70 6C 69 73 74 30 30 33 41 A9 BF F0 DE 00 00    bplist003A......
0010   00 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00    ................
0020   00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0030   00 11                                              ..

com.apple.metadata:_kTimeMachineOldestSnapshot:
0000   62 70 6C 69 73 74 30 30 33 41 A9 BF B4 32 00 00    bplist003A...2..
0010   00 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00    ................
0020   00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0030   00 11                                              ..

As we'd expect, the inode is different, but so is the metadata. I'm not sure what the purpose of this extra information is: it doesn't seem necessary for any restore functions, but it might come into play for the "self-grooming" features. I've yet to see a full explanation of what "self-grooming" really means; the System Preferences say that TM will keep backing up "until your backup disk is full". That doesn't sound like "grooming", does it?

Part of "self grooming" is the folding of backups - that is, hourly backups are gathered up into one daily backup, and after a month, daily backups are folded into weeklies.

So after a day, you've lost records of hourly changes, after a month you've lost daily changes. From then on, no folding.

One other thing: The Arstechnica review says that Time Machine uses Leopard's new ability to hard link directories. If so, the links don't show up in "ls -li", so I don't know how you'd tell.. oops, yes, of course they do - I have no idea what I was not seeing: for unchanged directories, the inums are identical: they are hard links.. cool.

I think I'd like to partition off a small place and find out just what does happen when TM runs out of space.. maybe next week.

See Flyback needs more than ease of use also.

If you want a great guide to learning about backing up your Mac, consider Joe Kissel's Take Control of Mac OS X Backups , a $15.00 PDF E-book that will teach you everything you need to know.

"dd" backups

Once in a while you'll come across somebody like this recommending dd for backups.

Personally, this makes me a bit nervous. Unflushed disk buffers could mess this up, though apparently he's been ok with it

The system is running while this is dd'ing in either direction. Maybe *you* aren't doing anything, but background tasks are running, and I would think it would be easy to get the disk into an inconsistent state. Apparently that hasn't happened, but I would think that sooner or later dd'ing mounted drives is going to bite him. Were I going to use this, it would be in single-user mode only.

He's probably been lucky because buffers get flushed fairly quickly on a one user system. If he tried this on a multiuser system, it would almost certainly screw up his data somewhere. It may be doing that now; he might just not notice. The fact that the system complained about damage certainly shows that it notices a problem.

On the down side, OS X seems to get "confused" by my restore
instruction, so temporarily (for the last 105 backups) I am ignoring
the onscreen warning that appears immediately after the five minute
restore operation:

"Your disk was not put away properly, there may be damage to your files"

I have tried everything to try to get rid of that warning, but nothing
works, so I will pass it off as a false warning.
 

I would NOT do this other then in single user mode! Time Machine is far more useful anyway.





(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> -> Mac OS X Leopard Time Machine





More Articles by

Find me on Google+

© Anthony Lawrence



Kerio Samepage


Have you tried Searching this site?

Unix/Linux/Mac OS X support by phone, email or on-site: Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us
















This post tagged: