# # slack
APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

2004/02/22 slack

I've removed advertising from most of this site and will eventually clean up the few pages where it remains.

While not terribly expensive to maintain, this does cost me something. If I don't get enough donations to cover that expense, I will be shutting the site down in early 2020.

If you found something useful today, please consider a small donation.

Some material is very old and may be incorrect today

© February 2004 Tony Lawrence

Random data in drive sectors. This is data beyond the actual data of a file. "Ram slack" is used to fill the unused portion of the last sector of a file, and "drive slack" is whatever is in any unused sectors of the last cluster. See http://www.forensics-intl.com/def6.html

If you found something useful today, please consider a small donation.

Got something to add? Send me email.

(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> slack

1 comment

Inexpensive and informative Apple related e-books:

Take Control of Parallels Desktop 12

Take Control of Upgrading to El Capitan

Take Control of iCloud

Take Control of iCloud, Fifth Edition

iOS 10: A Take Control Crash Course

More Articles by © Tony Lawrence

It should be pointed out that the referenced article creates some confusion between sectors and clusters -- plus it focuses on the two principle Microsoft filesystems: FAT and NTFS and thus does not point out differences between Microsoft's notion of how to store data and UNIX's or Linux's.

Key to understanding "disk slack" is recognizing the difference between a sector and a cluster, and knowing the difference between a "low level" or physical format and a "high level" or logical format.

Physical formatting creates fixed sized storage units known as sectors, usually 512 bytes in size. As a general rule, a floppy disk sector will be 512 bytes, as will be the case with most IDE hard drives. SCSI drives are considerably more intelligent than their IDE cousins and can be instructed to format the medium to a different sector size (2048 bytes, for example). In any case, the sectors generated by a physical format represent raw, unorganized storage. Sectors are sometimes referred to as blocks.

Logical formatting organizes the raw storage into a filesystem. How this is done varies from one operating system to another. For example, Microsoft uses the (wasteful) concept of "clusters," in which an arbitrary number of sectors are treated by the operating system as a single storage entity. Exactly how many sectors make up a cluster depends on which filesystem is involved.

Microsoft's FAT12 floppy filesystem used with the modern, high density 3-1/2 inch drive, equates a cluster with exactly one sector and therefore, the cluster size is 512 bytes. It is the most efficient of the current Microsoft filesystems, but since is uses 12 bit numbers to manage storage, it is limited to a maximum of 2 megabytes.

FAT16, which is the native hard drive filesystem of MS-DOS and the original version of Windows 95, aggregates anywhere from 8 to 64 sectors in a cluster, which means a cluster can be anywhere from 4 KB to 32 KB, depending on the size of the drive. Because FAT16 uses 16 bit arithmetic to manage drive storage and because no more than 64 sectors can be aggregated into a cluster, FAT16 is limited to a maximum partition size of 2,048 megabytes. It works out as follows:

64 * 512 = 32,768 (32 KB, the max bytes per cluster)

16 bits = 65,536

65,536 * 32,768 = 2,147,483,648 bytes or 2 GB

Obviously, the 2 GB limit is grossly insufficient given the size of today's hard drives. Microsoft recognized that they had a problem on their hands and thus developed the FAT32 filesystem, which was first made available with Windows 95 OSR2. FAT32 follows the same general pattern as FAT16, but is able to manage a far greater amount of data because the operating system is able to use 32 bit arithmetic to keep track of clusters. The math works as follows:

64 * 512 = 32,768 (32 KB, the max bytes per cluster)

32 bits = 4,294,967,296

4,294,967,296 * 32,768 = Oops! Overflow!

Well, the number is huge, about 1.4 * 10^14 bytes, well beyond any hard drive available today. Incidentally, the NTFS filesystem used by Windows NT/2000/2003/XP has the same sort of capacity as FAT32, but a different internal structure.

It doesn't take a computer wiz to see that Microsoft's cluster storage methodology is wasteful. For example, suppose you have created a FAT32 filesystem on an 80 GB hard drive. The resulting layout would use 32 KB clusters. Now, suppose you create a file with exactly one byte. Since the operating system can only allocate storage in 32 KB chunks, your one byte file effectively becomes a 32 KB file! Store enough one byte files and you could end up with a seemingly "full" drive, because each one byte file gobbled up a cluster. Don't laugh! This happened with some degree of regularity back in the days of FAT16 and 20-40 megabyte hard drives.

A much more efficient storage method, which is found in modern UNIX-like operating systems, is to use a bitmap to track sector usage. In very simplified terms, it works like this:

11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111

The above represents the first 8 bytes of the disk allocation bitmap (in little endian order, for you purists). Each set bit corresponds to one unused sector. Sectors are logically numbered starting at zero, which means that each byte manages 8 sectors and therefore, the above map controls 64 sectors, all of which are unallocated (free).

Now, suppose you write your one byte file to this filesystem. Since one sector is the minimum storage that the filesystem can allocate, one sector will be claimed for the file (actually, we also would have to make an entry into a directory, which itself would consume some sectors, but I did say it was simplified). Now the bitmap might look like this:

11111110 11111111 11111111 11111111 11111111 11111111 11111111 11111111

Note how bit zero in the first byte is cleared, indicating that the corresponding sector has been allocated. Were you to continue writing bytes into your file, eventually you'd consume all 1,024 bytes in that first sector and the operating system would have to allocate another. Then the bitmap would appear like so:

11111100 11111111 11111111 11111111 11111111 11111111 11111111 11111111

The process would be reversed if the file were truncated or deleted.

To find an unused sector, the operating system would scan the bitmap looking for a non-zero byte. Some arithmetic shift operations on the byte would indicate which bit(s) is(are) set and thus the logical sector number to be used. There's more to it, of course, as a primary concern of any filesystem design is resistance to damage caused by errors, and performance -- the method of scanning the bitmap a byte at a time would cause disk writes to become slower and slower as more storage was consumed.

Incidentally, for historical reasons, a sector in UNIX is 1,024 bytes, so storage is allocated in 1 KB chunks, not 32 KB as is done in Windows. Obviously, the modern UNIX filesystem is far more efficient than what the Microsoft crowd provides.



Printer Friendly Version

Have you tried Searching this site?

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us

Printer Friendly Version

It has become appallingly obvious that our technology has exceeded our humanity. (Albert Einstein)

Linux posts

Troubleshooting posts

This post tagged:



Unix/Linux Consultants

Skills Tests

Unix/Linux Book Reviews

My Unix/Linux Troubleshooting Book

This site runs on Linode