Managing Disk Space with LVM
Pages: 1, 2, 3, 4
Growth and Reallocation
Suppose that over the next year, the storage system fills up and needs to be expanded. Initially, you can begin allocating the unallocated space. For instance, to increase the amount of space available for shared files from 10GB to 15GB, run a command such as:
# lvextend -L15G /dev/datavg/sharelv
# resize_reiserfs /dev/datavg/sharelvBut over time, all the unallocated disk space will be used. One solution is to replace the four 250G drives with larger 800G ones.
In the case where you use RAID 1, migration is straightforward. Use mdadm to mark one drive of each of the RAID 1 mirrors as failed, and then remove them:
# mdadm --manage /dev/md0 --fail /dev/sda1
# mdadm --manage /dev/md0 --remove /dev/sda1
# mdadm --manage /dev/md0 --fail /dev/sdc1
# mdadm --manage /dev/md0 --remove /dev/sdc1Pull out the sda and sdc hard drives and replace them with two of the new 800G drives. Split each 800G drive into a 250G partition and a 550G partition using fdisk, and add the partitions back to md0 and md1:
# fdisk /dev/sda
# fdisk /dev/sdc
# mdadm --manage /dev/md0 --add /dev/sda1
# mdadm --manage /dev/md1 --add /dev/sdc1Repeat the above process with sdd and sdb to move them to the other two new drives, then create a third and fourth RAID device, md2 and md3, using the new space:
# mdadm --create /dev/md2 -a -l 1 -n 2 /dev/sda2 /dev/sdd2
# mdadm --create /dev/md3 -a -l 1 -n 2 /dev/sdb2 /dev/sdc2Finally, add these to LVM:
# pvcreate /dev/md2 /dev/md3
# vgextend datavg /dev/md2 /dev/md3The file server now has 1.6TB of fully redundant storage.
LVM and Desktops
So far, we've talked only about LVM and RAID for secondary disk space via a standalone file server, but what if you want to use LVM to manage the space on a regular desktop system? It can work, but there are some considerations to take into account.
First, the installation and upgrade procedures for some Linux distributions don't handle RAID or LVM, which may present complications. Many of today's distros do support it, and even provide tools to assist in creating and managing them, so check this first.
Second, having the root filesystem on LVM can complicate recovery of damaged file systems. Because boot loaders don't support LVM yet, you must also have a non-LVM /boot partition (though it can be on a RAID 1 device).
Third, you need some spare unallocated disk space for the new LVM partition. If you don't have this, use parted to shrink your existing root partition, as described in the LVM HOWTO.
For this example, assume you have your swap space and /boot partitions already set up outside of LVM on their own partitions. You can focus on moving your root filesystem onto a new LVM partition in the partition /dev/hda4. Check that the filesystem type on hda4 is LVM (type 8e).
Initialize LVM and create a new physical volume:
# vgscan
# pvcreate /dev/hda4
# vgcreate rootvg /dev/hda4Now create a 5G logical volume, formatted into an xfs file system:
# lvcreate rootvg ---name rootlv -size 5G
# mkfs.xfs /dev/rootvg/rootlvCopy the files from the existing root file system to the new LVM one:
# mkdir /mnt/new_root
# mount /dev/rootvg/rootlv /mnt/new_root
# cp -ax /. /mnt/new_root/Next, modify /etc/fstab to mount / on /dev/rootvg/root instead of /dev/hda3.
The trickiest part is to rebuild your initrd to include LVM support. This tends to be distro-specific, but look for mkinitrd or yaird. Your initrd image must have the LVM modules loaded or the root filesystem will not be available. To be safe, leave your original initrd image alone and make a new one named, for example, /boot/initrd-lvm.img.
Finally, update your bootloader. Add a new section for your new root filesystem, duplicating your original boot stanza. In the new copy, change the root from /dev/hda3 to /dev/rootvg/rootlv, and change your initrd to the newly built one. If you use lilo, be sure to run lilo once you've made the changes. For example, with grub, if you have:
title=Linux
root (hd0,0)
kernel /vmlinuz root=/dev/hda3 ro single
initrd /initrd.imgadd a new section such as:
title=LinuxLVM
root (hd0,0)
kernel /vmlinuz root=/dev/rootvg/root ro single
initrd /initrd-lvm.imgConclusion
LVM is only one of many enterprise technologies in the Linux kernel that has become available for regular users. LVM provides a great deal of flexibility with disk space, and combined with RAID 1, NFS, and a good backup strategy, you can build a bulletproof, easily managed way to store, share, and preserve any quantity of files.
Bryce Harrington is a Senior Performance Engineer at the Open Source Development Labs in Beaverton, Oregon.
Kees Cook is the senior network administrator at OSDL.
Return to the Linux DevCenter.
You must be logged in to the O'Reilly Network to post a talkback.
Showing messages 1 through 11 of 11.
-
lvcreate --name medialv --size 400G
2007-11-01 15:09:42 Steve Mallett |
[Reply | View]
I had to add the name of my volume group to the end of this:
lvcreate --name medialv --size 400G "name of vg"
for it to work.
-
Umounting /usr
2006-05-02 17:10:16 pretty-boy [Reply | View]
I have been using LVM on the desktop for quite a while for many of the reasons given in the article.
One problem that has arisen lately is trying to change (increase) the size of /usr using LVM and resizing the fs.
Recent version of bash now link into the glibc locale funstions. These in turn have links back into /usr/lib/locale. This means that even when pulling the system down to single user, as root you cannot umount /usr. And therefore you cannot resize it without rebooting into a rescue system.
Unix type systems have had the advantage of not requiring numerous reboots for system admin - but this seems to be changing.
-
Alternate RAID-1 migration
2006-04-30 19:04:48 jrzagar [Reply | View]
Instead of ending up with four raid-1 devices, wouldn't it be easier to do the following?
Use mdadm to mark one drive of each of the RAID 1 mirrors as failed, and then remove them:
# mdadm --manage /dev/md0 --fail /dev/sda1
# mdadm --manage /dev/md0 --remove /dev/sda1
# mdadm --manage /dev/md0 --fail /dev/sdc1
# mdadm --manage /dev/md0 --remove /dev/sdc1
Pull out the sda and sdc hard drives and replace them with two of the new 800G drives. Create 800G partitions on each drive and create md2:
# fdisk /dev/sda
# fdisk /dev/sdc
# mdadm --create /dev/md2 --level=1 \
--raid-devices=2 /dev/sd[ac]1--add /dev/sda1
Once you've activated the array, you can extend the volume group, move the logical volumes to the new drives, and remove the old raid arrays from the volume group:
# vgextend datavg /dev/md2
# pvmove /dev/md0 /dev/md2
# vgreduce datavg /dev/md0
# pvmove /dev/md1 /dev/md2
# vgreduce datavg /dev/md1
Now you can replace the hard drives for md[01], delete md0, recreate md1 with the second pair of 800G drives, and add the new md1 to datavg. -
Alternate RAID-1 migration
2006-04-30 23:37:05 bryceharrington [Reply | View]
Yes, there's definitely a few approaches to doing this, and like kees said, it's going to be driven by personal preference.
Note that part of the reason we picked this approach was just that it could be explained more concisely than other options we were considering. -
Alternate RAID-1 migration
2006-05-11 14:40:36 gr34t [Reply | View]
"Note that part of the reason we picked this approach was just that it could be explained more concisely than other options we were considering."
i agree with you my friend!!! take a look here:
http://www.filesharing-p2p.to/archive/english-subforum/p2p-news/dmca-20-t-41626.html -
Alternate RAID-1 migration
2006-04-30 19:45:07 keescook [Reply | View]
Excellent observation! :)
Doing it this way or the other is a matter of preference. Some people like maintaining the size of the original component devices, and some like maintaining the number of the original component devices.
For the purposes of the article, showing how to end up with more devices seemed to serve as a better teaching example, as it tangentially demonstrates how to just plain add more disks. :)
-
Root filesystem on LVM should be considered dangerous
2006-04-30 18:28:19 jrzagar [Reply | View]
The main advantage of having the root filesystem under LVM is that you can easily migrate your root filesystem to a new hard drive once you've added it to the volume group.
However, having the root filesystem under LVM creates new modes of failure that are extremely difficult to recover from...
The scenario quoted by the author, a corrupted filesystem, can be handled through an Ubuntu Live CD as it supports LVM and you will be able to fix broken filesystems.
A more catastrophic failure would be if the volume group itself became corrupted. How would you go about fixing a filesystem in a VG when the VG itself can't be activated? The answer is: with extreme difficulty, if it's possible at all.
My recommendations would be :
1) avoid using LVM for the root filesystem (recovery==easy),
2) if root is under LVM, don't let that VG span multiple disk devices (recovery==hard), and
3) if the VG *does* have multiple devices, don't let the logical volume with the root filesystem span more than one device (recovery==extremely difficult, but still possible).
Violate all three of these recommendations, and you're stuck doing a sector-by-sector forensic reconstruction to get your data back. -
Root filesystem on LVM should be considered dangerous
2006-04-30 19:38:41 keescook [Reply | View]
Yup, adding extra layers will always make dealing with failures more complex. I find the flexibility outweighs the complexity, though. That said, I personally don't recommend ever running without mirroring, especially on a root filesystem. And, I recommend doing regular backups with something nice like dirvish (http://dirvish.org/).
-
small correction: pvcreate
2006-04-29 08:50:22 keescook [Reply | View]
The example given for the very first "pvcreate" command has a typo. It should read:
pvcreate /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
Note the partition number after the drive name, without this, you won't benefit from having the partition type detected at boot-time.
I ran across a simple use case that wasn't explicitly covered in the article, but I was able to piece together the required commands from various examples in the article. I thought I'd share concisely.
Given:
Wanted:
Procedure:
# fdisk /dev/sdb
# pvcreate /dev/sdb1
# vgextend VolGroup00 /dev/sdb1
# vgdisplay | grep "VG Size"
# lvextend -L 28G /dev/VolGroup00/LogVol00
# mount
# ext2online -v /dev/mapper/VolGroup00-LogVol00
# df -h