Linux Logical Volume Manager (LVM) on Software RAID

More Articles

Logical Volume Manager is now included with most Linux distributions. The RedHat 8.0 installer even allows you to create LVM volumes during initial install. LVM offers capabilities previously only found in expensive products like Veritas.

If you plan on using LVM, I really recommend doing so on a RAID system, either hardware or software. Disks are dirt cheap nowadays, so there's really no excuse not to have mirroring (or RAID 5 if that's better for your usage). The RedHat installer will let you create LVM volumes on top of RAID volumes (it's a bit confusing, but it works), or you can do it later. Also remember that raid is not a substitute for regular and reliable backup!

Before we get to how you'd do this, let's go over what LVM gives you. First, LVM combines physical devices (partitions on disks in this case) into what it calls Volume Groups. Filesystems are then built on the Volume Group or groups. You can have a Volume Group that has only one disk partition in it, or several partitions on one or more disks. Each Volume Group can contain multiple filesystems. With this, you gain a lot of new capabilities.

• Increase or decrease filesystem size

  Decreasing the size of a filesystem returns space to the Volume Group. Increasing draws space from it. If you have too much space allocated for a file system, you can decrease it, and use that space somewhere else that needs it.

  Yes, you can do that with "parted", but it's not at all the same. You aren't dealing with contiguous disk blocks with LVM - all you need is free space in the Volume Group.

• Add more physical storage

  You can add more physical drives to an existing Volume Group, which of course immediately gives you more room to extend file systems.

• Create "snapshots" of filesystems.

  This is a great feature, but it requires a bit of explanation, so we'll leave the details for later. The use of snapshots is to freeze a filesystem in time, and let you go on using it while you leisurely back up the frozen data. What's wonderful about this is that it is NOT necessarily a full copy of your original file system - more on that later.

• Move Volume Groups to new physical storage

  I'm not going to cover this here, but it's a great feature, and you can even do it while the filesystems are in use!

  
  
  
  
  
  
  
  
• Striping

  I'm not going to cover that here (there are some references at the bottom of this article that do cover it), but you can get the performance benefit of striping as part of LVM. If you do this, it is even more critical that you mirror your drives. If you used RAID 5, there wouldn't be any point in striping in LVM as you already would have that benefit (see Raid if you don't understand that).

Configuring LVM on mirrored drives during RedHat Install

You might want to review Software Mirroring if you have never done that. Software mirroring is where we start by creating raid devices upon which we will create LVM volumes. For this test machine, I created a layout like this:

Note that there are two swap partitions, and that they will not be mirrored.

I then made two raid devices, choosing "Physical Volume (LVM)" as the filesystem type for the second.

Let's stop for a second and go over this carefully because it can be confusing. We've created mirrors of our boot partition and what would ordinarily become root. But there is no root partion yet. The place where it will go is mirrored, but we aren't creating a file system just yet. The next step is to click LVM to create a Volume Group. A volume group needs one or more Physical Volumes, and we have one. Note that is very different from Raid where you need at least two Software Raid partitions to do anything. For a LVM volume group, one physical volume is enough. It is here, in the Volume Group, that we create filesystems. Note that is plural. With raid, your partition is one filesystem, but with LVM you can create multiple filesystems within one Volume Group. To do that, just click the LVM button and it will bring up a new Volume Group. As we only have one Physical Volume here, that's what it will use, and it's ready to let you add file systems.

Note: although I put / into the LVM for this test setup, it may not be a good idea to do so because emergency boot media probably won't understand LVM filesystems. Supposedly RedHat 8.0 does support / on LVM (but not /boot).

So I added two filesystems, each 4GB in size, and had one mount at / and the other at /var. I had much more space available (this was a 30GB drive), but I didn't bother to use it at this time. I then proceeded with the install as usual. Well, almost as usual. Several times in testing I had the problem of not getting focus when it was time to put in the root password. The solution is to right click over the field, choose "Select All" and then click in the field again.

Extending File Systems

After booting up, I had the expected 4GB / and 4GB /var. As you will remember, the actual disk is much larger, so let's add more filesystems to it:

lvcreate -L 4G -n mylv Volume00
mkfs -t ext3 /dev/Volume00/mylv
mkdir /mylv
mount /dev/Volume00/mylv /mylv

That added a 4GB filesystem. Unfortunately, I made a mistake - I needed 6 GB. Well, at this point I could just blow it away (lvremove /dev/Volume00/mylv), but suppose I'd already added a lot of data? No problem, just unmount it and extend it:

umount /mylv
e2fsadm -L+2G /dev/Volume00/mylv
mount /dev/Volume00/mylv /mylv

The "ef2fsadm" actually runs lower level commands for you: it fscks the file system, runs lvextend to extend the logical volume, and then resizes the filesystem to fit. Much easier to let e2fsadm handle the whole job.

Adding more disks

If I really needed a lot more space, I could add a hard drive and use "pvcreate" to add it as a physical volume, and then "vgextend" picks up that space:

pvcreate /dev/sda1
vgextend Volume00 /dev/sda1

It just can't get much easier! Remember though: those disks should be mirrored too.

Snapshots

This is a very useful feature. Many of us have the situation where important data needs to be backed up, but it cannot be used while the backup is running because then the backed up files would be out of sync with each other. For example, you have an accounting system that is recording orders. The accounts receivable file gets backed up now, and you take an order. Both a/r and the customer file get updated to reflect the new order, but a/r has already been backed up. When the customer file finally makes it to tape, it's not consistent with a/r, and of course it needs to be. Without snapshots, your only recourse is to stop taking orders while the backup runs. If you have lots of disk space, you could copy the whole accounting system and backup the copy, but that can take a lot of time too, and you may not have the space. Snapshots are the solution. Before you do the next step, make sure you've put a few files in /little, and make at least one of them unimportant. Then create the snapshot.

lvcreate --size 200M --snapshot -n mysnap /dev/Volume00/mylv
mkdir /mylvsnap
mount /dev/Volume00/mysnap /mylvsnap
df

Right off the bat you should have noticed something strange. We created mysnap very specifically with a size of 200MB, and trust me, that's all it took away from us, but df shows it being the same size (6GB) as mylv. We'll get back to why this is in a minute, but first take a look at the files in /snap. They are identical to the files in /mylv, right? OK, now go edit a file in /mylv. Does it change in /snap? No, it does not. Remove a file in /mylv - it's still there in /snap. Add a new file to /mylv, and that does NOT appear in /snap. How is this done, and most especially how is it done in 200MB?

It's not magic

OK, it is magic. What is going on is that /snap contains absolutely nothing UNLESS something changes back at /mylv. If you ask for a file from /snap that has not changed, the data is read right from /mylv. But if a file IS changed, before the change is written, the data blocks that don't yet have the changes are written to /snap. Note that entire files are NOT written, just data blocks that are about to change. So, as long as we don't change more than 200MB worth of data in /mylv, we can have our cake and eat it too. Our procedure will be:

• Stop using the filesystem, shut down any databases that need to be shutdown etc.
• Create the snapshot
• Start up our databases, go back to work.
• Start backing up /snap

Our time without access is minutes or seconds - just however long it takes to stop the processes and restart them, basically. The backup can take its sweet time. Well it can if it doesn't take so long that we need more than 200 MB to store our data that is changing. That does mean that the size of mysnap does have to be a bit of an educated guess. It also means that as soon as you are done with the backup, mysnap should be removed:

umount /snap
lvremove /dev/Volume00/msnap

If you don't remove it, it will go on copying data as it is changed and eventually it will run out of room. You can't just leave it there for next time!

Some other helpful links:

http://tldp.org/HOWTO/LVM-HOWTO/index.html
http://ds9a.nl/lvm-howto/HOWTO/cvs/lvm-howto/output/lvm-howto.html

Publish your articles, comments, book reviews or opinions here!

Enter your email address for automatic notification of new posts here
(be sure to whitelist 'feedburner.com' if you use spam filtering)

63112-080107-548256-33

Have you tried Searching this site?

Unix/Linux/Mac OS X support by phone, email or on-site: Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more. We appreciate comments and article submissions.

Publishing your articles here