Advertisement
MultiTail: tail for professionals!
MultiTail allows you to monitor logfiles and command output in multiple windows in a terminal, colorize, filter & merge.
| |||||||||||||
Fedora and LVMFedora and LVMPosted Oct 31, 2012 16:42 UTC (Wed) by dwmw2 (subscriber, #2063)Parent article: Fedora and LVM My biggest problem with LVM is that it's massively more fragile than the basic partitioning setup. And, as noted, with no real benefit in the basic laptop/desktop case. I run a number of machines, and mostly I remember to disable LVM when installing them. I have lost count of the number of times that something's gone wrong with the ever-increasingly-baroque initrd, leaving mount-by-UUID and LVM non-functional, while a simple boot with 'root=/dev/sda2' has been sufficient to get it working again. And of the number of times that I've attempted to perform a similar task on an LVM-inflicted machine, and been screwed. I have, on the other hand, resized partitions and switched from dual-500GB disks in RAID-1, to dual-1½TB disks in RAID-1, without having to use LVM. So even on the rare occasion that I've actually done disk-juggling stuff, LVM didn't help me.
(Log in to post comments)
Fedora and LVM Posted Oct 31, 2012 17:03 UTC (Wed) by tialaramex (subscriber, #21167) [Link]
I'm sure it's all possible, it's just not convenient. LVM makes it easier and safer to do this sort of thing, without fumbling around with disk layouts and other stuff you've no reason to care about.
The last time I looked the Fedora UI for LVM sucked pretty badly. But that's not a failing of LVM, it's an opportunity for someone with UI design skills to do a better job. This stuff can be made non-scary, even the Windows Server UI is less scary than what Fedora has now.
It also lets me put off decisions. I have 500GB free, is that space for more video recordings or for experimental software? Never mind, I'll leave it free and assign it when it's needed. Tricky with raw partitions, easy by leaving the blocks assigned to a volume group but not a specific logical volume.
Fedora and LVM Posted Oct 31, 2012 21:51 UTC (Wed) by epa (subscriber, #39769) [Link]
I can't really agree that 'LVM makes it easier and safer' to do disk-juggling operations. If your system is up and running happily from its system disk, and you have a couple of RAID arrays to manage, then yes. But for single-disk systems it seems to get in the way. For example, suppose you want to upgrade your hard disk. With Slackware it is a simple task to plug in the second disk, boot the machine, partition the second disk and copy everything across, and install the boot loader on the second disk. You're now ready to go. With Fedora and LVM, on the other hand, making it work requires a complex series of LVM commands; fdisk looks positively intuitive by comparison. In the end I gave up trying to move my system to a different hard disk and just reinstalled from scratch. Yay LVM!
To me, LVM is like git. It's hugely powerful; but as a beginner, or someone who doesn't use it often, you really long for a simple interface that will print out "WTF is going on" and guide you through the possible steps.
Fedora and LVM Posted Oct 31, 2012 22:59 UTC (Wed) by marcH (subscriber, #57642) [Link]
> To me, LVM is like git.
While LVM obviously add some complexity, the comparison with git is really not fair. If you understand git then you can certainly understand LVM with an extremely small effort.
There are only two/three LVM concepts you need to know:
1. Physical partitions: your real, good old disk partitions
Apologies for just making you fully understand LVM against your will.
Fedora and LVM Posted Nov 1, 2012 4:53 UTC (Thu) by bronson (subscriber, #4806) [Link]
Talk about oversimplification! He's going to have to read a lot more documentation to get even the simplest operation done in LVM.
Really, if if the desire is just to to move or resize partitions, it's probably best to use gparted and skip LVM entirely. Not many people actually need the additional features that LVM offers.
Fedora and LVM Posted Nov 1, 2012 7:53 UTC (Thu) by marcH (subscriber, #57642) [Link]
> He's going to have to read a lot more documentation to get even the simplest operation done in LVM.
I must be a genius then.
The only thing I do besides remembering the above is type the following:
lv[tab] --help
That's 15 minutes extra but it's only 15 minutes and it does not happen often.
> Really, if if the desire is just to to move or resize partitions, it's probably best to use gparted and skip LVM entirely.
The key LVM feature for me is: no reboot interruption(s) distracting me at the worst possible moment. If that's not considered important then yes LVM should be skipped.
Fedora and LVM Posted Nov 1, 2012 8:53 UTC (Thu) by epa (subscriber, #39769) [Link]
Yes, if your machine is up and running then you have recourse to "ten minutes of googling". If it won't boot because the initrd doesn't recognize the LVM setup and just falls over in a heap (rather than doing something helpful like, I don't know, displaying a list of volume groups and volumes and letting you choose which to mount as /) then everything becomes a lot more uncomfortable.
Fedora and LVM Posted Nov 1, 2012 9:29 UTC (Thu) by paulj (subscriber, #341) [Link]
That's not really LVMs fault, but the fault of the init root your distro supplied. That early-boot stuff gets rewritten en-masse regularly and then shipped to you when it still has lots of undiscovered bugs probably means you're the running the regression-guinea-pig version of your favoured vendors' distro.
Fedora and LVM Posted Nov 1, 2012 15:54 UTC (Thu) by bronson (subscriber, #4806) [Link]
Are you really suggesting that with only your previous post, 10 minutes of googling, and the output lvm --help, that epa (or typical Linux users) will be able to easily reconfigure their lvm setups?
Fedora and LVM Posted Nov 9, 2012 18:13 UTC (Fri) by nahoo (guest, #55570) [Link]
I think he is right. I did this last week. I did a backup, and half an hour later I had all my partitions + filesistems resized, and it was my first try at lvm.
Of course it depends on how you describer a typical user, but I think the statement is true for a user that knows what a filesystem/partition is and need to resize them.
Fedora and LVM Posted Nov 1, 2012 9:50 UTC (Thu) by mordae (subscriber, #54701) [Link]
> For example, suppose you want to upgrade your hard disk. With Slackware it is a simple task to plug in the second disk, boot the machine, partition the second disk and copy everything across, and install the boot loader on the second disk.
Yeah, and with LVM you can do that on-the-fly and safely move LVs while using them. You just move the LVs between PVs (the disks) within the same VG and finally you remove the old drive. You can even do it all incrementally.
Fedora and LVM Posted Nov 1, 2012 11:04 UTC (Thu) by epa (subscriber, #39769) [Link]
Do you have a link to some instructions about how to do this?
I had hoped to set up the new disk without touching the old disk - so there is always a fallback if something goes wrong. If you move the LV to the new disk, that means your old disk will no longer work. I think that for upgrading your hard disk, you usually just want to clone everything to the new disk - only with a larger partition size than before.
Fedora and LVM Posted Nov 1, 2012 12:57 UTC (Thu) by pbonzini (subscriber, #60935) [Link]
pvcreate "formats" a partition for use with LVM.
vgextend (mentioned in the man page for pvcreate) adds the partition to the existing volume group.
pvmove moves the logical volume from the old disk to the new disk.
vgremove removes the old partition from the volume group.
Remember to move /boot manually (it should be easy to do it with dd since it can be left read-only).
Fedora and LVM Posted Nov 2, 2012 0:34 UTC (Fri) by lacos (subscriber, #70616) [Link]
s/vgremove/vgreduce/ ; see above (commenting here so you get an email possibly and don't miss the correction)
Fedora and LVM Posted Nov 1, 2012 21:13 UTC (Thu) by agk (subscriber, #23332) [Link]
> pvcreate "formats" a partition for use with LVM.
BTW This step is optional now. Basic pvcreate functionality is included in vgextend (and vgcreate).
Fedora and LVM Posted Nov 2, 2012 0:33 UTC (Fri) by lacos (subscriber, #70616) [Link]
> vgremove removes the old partition from the volume group.
I think this is not correct (it caught my eye because I also have the most trouble remembering this step).
"vgremove" removes an entire volume group.
Instead "vgreduce" is needed here (evict the PV (ie. the partition) from the VG).
Afterwards "pvremove" may be invoked on the partition to "wipe[] the label on [the] device so that LVM will no longer recognise it as a physical volume".
Fedora and LVM Posted Nov 1, 2012 11:39 UTC (Thu) by Cato (subscriber, #7643) [Link]
Indeed - LVM also makes it harder to recover data (few tools directly support LVM), easier to lose data (on setups where the write caching is broken, but mostly older kernels) and harder to resize partitions (multiple LVM and FS resize commands, and if shrinking is required then it's easy to wreck the FS). And LVM snapshots are pretty slow.
See http://serverfault.com/questions/279571/lvm-dangers-and-c... for more.
Generally I think LVM should only be a default on server distros aimed at professionals. Once btrfs is more mature and has a complete fsck, it removes some of the need for LVM in any case, and has other advantages such as faster snapshots.
Fedora and LVM Posted Nov 1, 2012 20:49 UTC (Thu) by agk (subscriber, #23332) [Link]
> (multiple LVM and FS resize commands, and if shrinking is required then it's easy to wreck the FS).
To resize a logical volume containing a recognised filesystem with a single command, use:
This passes the size to the filesystem resizing command (via 'fsadm') and runs the steps in the right sequence.
Fedora and LVM Posted Nov 2, 2012 7:41 UTC (Fri) by Cato (subscriber, #7643) [Link]
I'm aware of the --resizefs feature, but it has been buggy over the years and was apparently only fixed some time in 2011 - see comments 5 and 6 on https://bugs.launchpad.net/ubuntu/+source/lvm2/+bug/174032
Most resizing HOWTOs for LVM don't use this command, probably because it was added in the last few years, but that may be no bad thing.
Shrinking an LVM logical volume and FS can be dangerous if you don't get the sizes exactly right, and I'm not 100% confident that the --resizefs feature is bug free. (See http://www.debuntu.org/how-to-install-ubuntu-over-lvm-fil... for a long discussion of issues, and the resizing part of http://serverfault.com/questions/279571/lvm-dangers-and-c...
Generally I have more confidence that Gparted (or parted) will not mess things up.
Fedora and LVM Posted Nov 2, 2012 14:27 UTC (Fri) by agk (subscriber, #23332) [Link]
> Shrinking an LVM logical volume and FS can be dangerous if you don't get the sizes exactly right
Which is why --resizefs exists: so that appropriate sizes are used and the steps are performed in the right order. The feature is fully supported.
> and I'm not 100% confident that the --resizefs feature is bug free.
"Bug free" will never be promised, of course:) The bugs you referenced look to be from before the code was included in lvm2 and were of the "feature gives an error and stops instead of resizing" variety.
Fedora and LVM Posted Nov 2, 2012 17:09 UTC (Fri) by paulj (subscriber, #341) [Link]
And if you insist on using the LVM and FS resizing commands separetely, well at least resize2fs doesn't /need/ any size argument. So just extend the LV, then run resize2fs on its own (no size arg) - it'll figure it out, completely safe.
Fedora and LVM Posted Nov 2, 2012 17:20 UTC (Fri) by nybble41 (subscriber, #55106) [Link]
That's true only when _growing_ the logical volume; the danger is in _shrinking_ it. To reduce the size of a LV you must first shrink the filesystem, and for that you need the final size of the LV. If you fail to reduce the filesystem to a size less than or equal to the final size of the LV before resizing the LV, the result will be data corruption. Thus the --resize option, which takes care of these details for you.
Fedora and LVM Posted Nov 5, 2012 12:08 UTC (Mon) by paulj (subscriber, #341) [Link]
For that case I first shrink the fs as much as possible, to size X say. Then I shrink the LV to desired size Y, where Y is unambiguously > X. Then I resizefs without args to increase the fs back to the full size of the LV.
That I do it that way is probably cause I pretty much never shrink fses, and the last time I had to do this was way back, when LVM was still newish and e2fsadm and/or the --resize option didn't exist, or didn't handle this.
Generally LVM lets me manage fses in such a way that I only allocate to them what is reasonable for the foreseeable future. I keep the extra space free in the VG. As/when fses need more space, I just extend them. That this is so painless to do with LVM and resize2fs - online and without interruption - makes it the best way to manage storage, I find.
So so so so much better than the olden days, when you had to rejiggle partitions, reboot several times, and potentially use your swap device as a temporary root, in order to shift space to where you needed it. That was hair-raising, and I never want to go back to that! There isn't much to LVM, it didn't take much to learn, and it's removed a lot of stress.
One lesson though: snapshots need attentive monitoring. Never let a snapshot get anywhere near full, otherwise you risk it getting full and your box will wedge! Also, the early implementations of pvmove were very risky. Apparently those problems have been ironed out and it's now a lot more reliable, but I'm conservative and tend to avoid it. Luckily, that's easy - just move PVs the old fashioned way, copy them like you had to do with FSes before LVM. ;)
Overall though, LVM is a massive win over what went before.
Fedora and LVM Posted Nov 5, 2012 12:23 UTC (Mon) by Cato (subscriber, #7643) [Link]
Your 'over-shrink then grow' method is a good one for ensuring the FS doesn't get corrupted when shrinking an LV. However the --resizefs option should be safe and quicker according to agk's comments elsewhere.
Re the "olden days" scenario: now that 2 to 3 TB external drives are very cheap, and USB3 or eSATA are very fast, you can simply backup all the partitions involved (a good idea anyway) to the drive, then repartition destructively and restore. (If you are using rsnapshot you only do an incremental backup since the overnight one, which will not take so long.)
This backup/restore model may be faster than doing the Gparted model of moving FSs to new location, as it uses 2 spindles not one. It also has the benefit of defragging your FSs - I know Linux FSs don't need defragging in theory, but in practice it will help somewhat particularly if FS has been in use for some years. (LVM's model will tend to increase fragmentation somewhat as you never do this sort of FS re-creation.)
I don't see why more than one reboot is needed with Gparted.
Fedora and LVM Posted Nov 5, 2012 12:32 UTC (Mon) by paulj (subscriber, #341) [Link]
Yes, I might try the --resizefs option next time. I just didn't know about it. It really has been a long time since I last had to shrink an fs.
That single reboot with gparted is still 1 more reboot than I need with LVM. Further, on an ongoing basis, it means you will need a lot more reboots than I will.
Fedora and LVM Posted Nov 2, 2012 18:02 UTC (Fri) by Cato (subscriber, #7643) [Link]
Since it seems that "lvextend --resizefs" generally works for common FSs, it's certainly worth a try, but the lack of documentation and HOWTOs mentioning this option doesn't fill me with confidence.
While you are here: I think that the LV size should be larger than the FS size by 2 x the LVM physical extent (PE) size. Is that correct?
Fedora and LVM Posted Nov 3, 2012 1:28 UTC (Sat) by agk (subscriber, #23332) [Link]
> the lack of documentation and HOWTOs mentioning this option doesn't fill me with confidence.
The LVM HOWTO at tldp hasn't been updated for several years.
> I think that the LV size should be larger than the FS size by 2 x the LVM physical extent (PE) size. Is that correct?
No. Where did that idea come from? LVM doesn't reserve any space *inside* the LVs it creates! The filesystem can use the entire LV if it wants to.
Fedora and LVM Posted Nov 1, 2012 22:05 UTC (Thu) by agk (subscriber, #23332) [Link]
> And LVM snapshots are pretty slow.
That can be qualified: They are optimised to act as short-term backups that will be accessed perhaps just once (if at all) and then deleted.
But because of the lack of alternatives they often get used in other situations where they are indeed 'slow' by design.
An alternative LVM "thin provisioning" snapshot implementation is now available that is optimised for having many long-term snapshots of the same volumes, sharing data between them where they can. Example:
# Set aside 10GB for a pool of thin logical volumes (thin_pool1) in vg1
Commands that allow the creation of thin snapshots of existing (non-thin) logical volumes are under development.
Fedora and LVM Posted Nov 1, 2012 15:16 UTC (Thu) by jwakely (subscriber, #60262) [Link]
> To me, LVM is like git.
Except Git is something I find immensely useful and am grateful for daily and therefore is worth bothering to learn.
As I said here last year (http://lwn.net/Articles/459233/) I still don't feel as though I'm missing out on anything by shunning LVM.
Fedora and LVM Posted Nov 2, 2012 7:26 UTC (Fri) by Cato (subscriber, #7643) [Link]
Your example of leaving 500GB free space for use by either video recordings or experimental software is easily handled with raw partitions and GParted:
1. Create the video and software FSs, say 200GB each, on partitions
I haven't included unmount/mount because in some cases using a live CD for Gparted is easier.
My point is that many common FS resizing operations are easily done by Gparted - it will take longer in the 'commit' stage when it does all the work than with LVM (due to moving the data blocks around), but you can leave your system running.
For the typical laptop/desktop case, Gparted + raw partitions are much easier than LVM commands + LVM. For servers where uptime is important, LVM is much better, but it requires some care to get all the commands right.
Fedora and LVM Posted Nov 2, 2012 16:13 UTC (Fri) by Tobu (subscriber, #24111) [Link] A LiveCD isn't just easier, it's necessary to be able to unmount partitions that are normally in use. More importantly, step 4 will take a long time and leave you with a thrashed filesystem if an interruption occurs. This is not user-friendly.
Fedora and LVM Posted Nov 2, 2012 16:27 UTC (Fri) by Cato (subscriber, #7643) [Link]
I'm aware of why using a live CD is sometimes necessary, didn't think that was essential to mention on LWN.
LVM on older kernels (< 2.6.33) that don't fully support write barriers is also rather dangerous for everyday operations, not just filesystem moves/resizes, unless you turn off write caching in the hard drive (long story, search for "lvm risks" in google).
So whether you use LVM or Gparted, you need to ensure your machine doesn't crash, that the moving-stuff-around app doesn't crash (pvmove has been known to cause data loss due to lack of RAM, see my comments elsewhere).
LVM is somewhat safer if you have a good UPS and have correctly configured write barriers and write caches, but since I use UPSes for my PCs and have yet to had a PSU failure or machine crash during a Gparted operation, I'm happy that the risk is quite low.
In fact the one time I lost data with GParted was when not using a live CD - the rather old kernel + Gparted versions meant that the partition table wasn't properly updated and I omitted to reboot. So a live CD is definitely the way to go.
Whatever method you use, you really need to do a full backup anyway, which guards against the sorts of data-loss mentioned here. So it comes down to speed, ease of use, reliability, etc.
Fedora and LVM Posted Nov 2, 2012 21:50 UTC (Fri) by Tobu (subscriber, #24111) [Link] I'm aware of why using a live CD is sometimes necessary, didn't think that was essential to mention on LWN.I mean, with LVM I don't need to bother with bootable media. I wanted to highlight this difference (this FESCO spat brought up the petty in me, sigh).
Fedora and LVM Posted Nov 2, 2012 17:38 UTC (Fri) by nix (subscriber, #2304) [Link]
Last I saw of it (which was years ago, I don't use partitions anymore), if you changed a partition table on a block device which was in use, you had to reboot before the kernel would pick up the new partition table.
That adds a new step 6, 'reboot'. That's a totally unnecessary reboot with LVM.
Fedora and LVM Posted Nov 12, 2012 3:34 UTC (Mon) by steffen780 (guest, #68142) [Link]
Just run partprobe, always worked for me. I assume there is a reason why this isn't called automatically, but I don't have the slightest clue of what that may be.
Fedora and LVM Posted Oct 31, 2012 17:27 UTC (Wed) by nix (subscriber, #2304) [Link]
That's not LVM's problem, though: it's the fault of the delicate initrd not handling the userspace initialization LVM needs properly, which could presumably bite booting off md/RAID as well. I've been using a simple initramfs linked into the kernel image for eight years now, giving me md, LVM-atop-md, md-atop-LVM and both-atop-NBD if needed. Total number of boot failures attributable to LVM in all that time: zero.
LVM has its nasty delicate corners (be scared of snapshots, for instance), but as a 'make it bigger when needed' tool, it still does the job, and is not notably unreliable in my experience.
Fedora and LVM Posted Oct 31, 2012 17:51 UTC (Wed) by mjg59 (subscriber, #23239) [Link]
I don't think anyone's worried about the quality of LVM in terms of its implementation and the core userland tools, but it's really not as well integrated into the distributions as you'd like.
Fedora and LVM Posted Oct 31, 2012 18:31 UTC (Wed) by dwmw2 (subscriber, #2063) [Link]
It's more than that. For most simple desktop/laptop installs, using LVM is a flagrant and gratuitous violation of the KISS principle. With the expected results.
Fedora and LVM Posted Oct 31, 2012 20:55 UTC (Wed) by rleigh (subscriber, #14622) [Link]
There are some serious quality issues with LVM. I can (and have) locked up the kernel hard many times (no panic, just completely dead) using nothing more than "lvcreate -s" and "lvremove".
The schroot tool uses LVM snapshots to create and destroy transient scratch build environments. We use them on the Debian autobuilders. If you kick off 24 parallel builds on a 24 core system, the system will be dead as a doornail in a few tens of seconds. Entirely due to lvcreate/lvremote triggering what looks like some kernel locking bug. Even on systems only running a single build, we still regularly have lvcreate and lvremove failures. There's some fundamental bugs in LVM which really need fixing, and which I'm surprised haven't been addressed given that they are easily reproducible. schroot is admittedly a special case--most people don't churn through as many LVs as we do--rebuilding the whole archive is ~14 hours with 24 parallel builds, and ~18000 transient LVs (though it always died in under 5 mins, less than 100 LVs in). But it should certainly work without killing your system.
We now also support btrfs snapshots, and while the filesystem itself is still not perfect, I've not yet run into a single issue doing heavy parallelised snapshotting.
Fedora and LVM Posted Oct 31, 2012 21:58 UTC (Wed) by BlueLightning (subscriber, #38978) [Link]
Sounds like a serious bug. Did you report it?
Fedora and LVM Posted Oct 31, 2012 22:18 UTC (Wed) by rleigh (subscriber, #14622) [Link]
IIRC it was sent to the LVM and/or kernel lists.
Fedora and LVM Posted Nov 1, 2012 21:39 UTC (Thu) by agk (subscriber, #23332) [Link]
> IIRC it was sent to the LVM and/or kernel lists.
Please would you dig up and post a link to this here?
Fedora and LVM Posted Nov 1, 2012 23:23 UTC (Thu) by rleigh (subscriber, #14622) [Link]
I've had a good search, but I'm afraid I can't find it on either list, sorry. I may be misremembering, or just looking in the wrong place.
Fedora and LVM Posted Nov 2, 2012 0:11 UTC (Fri) by agk (subscriber, #23332) [Link]
Well if you're still having problems, please let us know the details and we'll see what we can suggest. If it was 24 writeable snapshots of the same device, then it might be worth trying the new thin snapshots. (Set up the origin as a thin device, then drop the size parameter from CHROOT_LVM_SNAPSHOT_OPTIONS.)
Fedora and LVM Posted Nov 2, 2012 0:28 UTC (Fri) by rleigh (subscriber, #14622) [Link]
Thanks Alasdair, it was indeed all writable snapshots of the same device in this instance. I'll give thin snapshots a go with a current kernel. I no longer have access to the 24 core system (which was also remote, making debugging lockups difficult), but I'll see what I can do on a local quad core system.
Fedora and LVM Posted Nov 1, 2012 12:06 UTC (Thu) by Cato (subscriber, #7643) [Link]
I think most LVM users don't do much with snapshots, if they even use them, and the snapshot code is much more buggy as a result.
The non-snapshot LVM/DM code is not too bad, though there are some tools issues - e.g. pvmove can corrupt data if it runs out of memory - see http://serverfault.com/a/339899/79266 and http://deranfangvomende.wordpress.com/2009/12/28/a-primer...
Fedora and LVM Posted Nov 1, 2012 17:44 UTC (Thu) by nix (subscriber, #2304) [Link]
pvmove uses snapshots internally, so this is just another indication of the general principle that LVM snapshots are dangerous.
Fedora and LVM Posted Nov 1, 2012 21:32 UTC (Thu) by agk (subscriber, #23332) [Link]
> pvmove uses snapshots internally
pvmove uses mirrors, not snapshots.
It works by setting up temporary mirrors for the kernel to sync the data between the old and new locations.
Fedora and LVM Posted Nov 1, 2012 22:29 UTC (Thu) by nix (subscriber, #2304) [Link]
Oh. You're right, of course. (I've only ever encountered one pvmove failure, anyway -- a deadlock -- and it restarted fine on reboot.)
Fedora and LVM Posted Oct 31, 2012 23:32 UTC (Wed) by mezcalero (subscriber, #45103) [Link]
Actually I am concerned about the implementation. The fact that the LVM/dm userspace bits heavily rely on stuff such as sysv semaphores and things is just awful. If code uses SysV semaphores then this usually is a pretty strong sign that something is not right about the code, i.e. either that it hasn't been touched in decades, or that it simply is questionnable code.
But my main beef with LVM is and has been since years that it's assembly is just broken and wrong. They assume that during boot there was a time where all devices have shown up, and which point they can invoke vgchange -ay, and that's the only time where they need to enumerate devices. But that's really not how things work these days, and haven't been working in the last decade or so. Hardware devices show up all the time, and we do not know when all connected devices have been detected, so in the boot process we don't actually know when we could invoke "vgchange -ay". Also, harddisks in times of USB and iSCSI show up at any time, depending on network status and their own initialization time and there are no general rules about when initialization has to be complete. The way distributions hack around the fact that they don't know when to invoke vgchange -ay is by pulling in udev-settle and the atrocity that is scsi-wait-scan. These hacks make things work for the "majority" of runs, and as long as you only have SATA disks and other pretty standard stuff. But the question is whether the "majority" is good enough where reliability is required, and whether limiting stuff to SATA and friends is such a good choice. But on top of that, it's just slow, since it basically is little more than just delaying the boot arbitrarily in the hope that all possible devices might have shown up when some time passes. Of course, if you are unlucky the delay is not sufficient, but still everybody has to endure the delay.
This issue has been known by the LVM folks since many years, and pointed out to them again and again. But nothing ever happened. Now, they say they'll soon have the "option" to make LVM work like any other hw daemon and actually wait on its own for precisely the devices it needs and not longer, thus not delaying the boot any bit longer than necessary. And they'll investigate if they can make that "option" default one day... At that speed I am sure that LVM well get discovery/assembly right only after the time it already has been replaced by btrfs... (btrfs in turn does get all this right: assembly is based of device plug events and a btrfs raid will delay the boot only exactly until the point where all devices it actually needs have shown up. btrfs raid is hence hotplug-safe, snappy at boot and absolutely reliable).
Anyway, the take-away is: LVM is the one major slowness in Fedora's boot. It also adds fragility where it shouldn't. And this hasn't changed in years. LVM is written they way hardware was in the 80's and 90's, but it is not how hardware has beeen working in the past 15 years or so.
Fedora and LVM Posted Nov 1, 2012 0:31 UTC (Thu) by marcH (subscriber, #57642) [Link]
While I use and like LVM on "normal" disks, I really never expected anyone would try to use it on removables or on top of iSCSI... What are the use cases here?
Fedora and LVM Posted Nov 1, 2012 1:25 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
Using an external USB drive to temporarily expand a small internal SSD?
Fedora and LVM Posted Nov 1, 2012 14:02 UTC (Thu) by TRS-80 (subscriber, #1804) [Link] Proxmox VE uses it to store virtual machine images, which it can then LVM snapshot for crash-consistent backups while the VM is running. Being a VM hypervisor, iSCSI is a natural choice to allow shared storage and migration between hosts. I have it running at work and it's pretty awesome.
Fedora and LVM Posted Nov 2, 2012 19:06 UTC (Fri) by phiggins (subscriber, #5605) [Link]
I used to run Fedora as my primary OS on my work laptop. I resized the Windows partition down but didn't remove it, and that left me with little space to actually work in. I put /home on a USB drive and used LVM. It was a huge mistake for exactly the reasons mentioned. It rarely was mounted automatically and caused big problems (I don't remember exactly what they were now) whenever the USB cable accidentally became disconnected.
I honestly can't remember why I used LVM in the first place, but Fedora using it by default probably influenced that decision.
I used to be in the "LVM is worthless confusing overhead for a desktop and even small servers" camp. Now that I have more experience with it after using it for years *because* Fedora made it the default, I have found its features useful a few times per year. However, I am still nervous that one day I will be unable to recover my data because LVM is too complicated and I won't know how to restore a non-booting system.
Fedora and LVM Posted Nov 2, 2012 20:39 UTC (Fri) by nix (subscriber, #2304) [Link]
One saving grace of LVM is that the metadata backups are not just kept in /etc/lvm. They are *also* kept in a defined place on every PV (IIRC at the start), in ASCII. So if you are so badly shagged that you've lost / and vgscan can't reconstruct your VG, you can get started by sucking the metadata off the PV with dd. (But normally as long as the PVs are there the VG can be reconstructed. A VG doesn't have any existence beyond its PVs, after all: there is no 'VG area' that can get damaged to destroy your VG.)
Fedora and LVM Posted Nov 1, 2012 21:06 UTC (Thu) by agk (subscriber, #23332) [Link]
> The fact that the LVM/dm userspace bits heavily rely on stuff such as sysv semaphores
Semaphores are used only by the LVM/dm code for synchronisation with the udev rules that have to handle asynchronous uevents.
On a system using udev, after LVM/dm creates a device (or group of related devices) it needs to wait until udev has finished setting up those particular devices before it continues. This is done by having the last udev rule decrement a (per-transaction) semaphore to zero, which the blocked LVM/dm code waits for.
Fedora and LVM Posted Nov 2, 2012 2:17 UTC (Fri) by mezcalero (subscriber, #45103) [Link]
There is no excuse for every using SysV semaphores, unless you are living in the 80s. There are a multitude of alternatives around. But SysV semphores, na, thank you.
Also, LVM could just subscribe to udev devices coming and going with libudev like everyvody else, and things would be good...
Fedora and LVM Posted Nov 2, 2012 3:58 UTC (Fri) by agk (subscriber, #23332) [Link]
> LVM could just subscribe to udev devices coming and going with libudev
If only! We had lengthy discussions with the udev developers which led to the existing synchronisation mechanism.
Fedora and LVM Posted Nov 1, 2012 22:36 UTC (Thu) by agk (subscriber, #23332) [Link]
> This issue has been known by the LVM folks since many years, and pointed out to them again and again. But nothing ever happened.
If it had been easy, people would have submitted patches long ago:)
But the final piece of the assembly jigsaw is now passing our tests and should hit rawhide builds this week.
Fedora and LVM Posted Nov 3, 2012 17:54 UTC (Sat) by Cato (subscriber, #7643) [Link]
I've just realised that I seem to have had this problem on a server using LVM a while back - I didn't have time to investigate so I just put a "vgchange -ay" into the /etc/init.d/checkfs script. This is on Ubuntu 8.04 LTS server, so it's a cross-distro issue, and the hard disks are SATA.
Presumably an earlier invocation had failed, with the result that the fsck step failed, causing boot to fail. A shame that LVM, which generally helps uptime in many cases, caused downtime here - presumably btrfs wouldn't have this issue as it doesn't require this extra 'vgchange' type step.
I really look forward to btrfs (or perhaps ZFS in-kernel) being mature enough to use, largely for the parent-block checksumming to catch various errors.
This is on a home server in an inconvenient location, where there is only voltage regulation not UPS, and the utility power is flaky - so reliable unattended reboots are really helpful.
Fedora and LVM Posted Nov 5, 2012 10:51 UTC (Mon) by nmav (subscriber, #34036) [Link]
> Actually I am concerned about the implementation. The fact that the LVM/dm
> userspace bits heavily rely on stuff such as sysv semaphores and things is > just awful. If code uses SysV semaphores then this usually is a pretty > strong sign that something is not right about the code, i.e. either that > it hasn't been touched in decades, or that it simply is questionnable code.
I like those generalizations. If a thing does A it is crap. While semaphores may be used in bad code, there is nothing to indicate bad code because it uses semaphores. Semaphores are a tool (arguably an old one), but as every tool it can be used in a bad way or _not_.
Fedora and LVM Posted Nov 5, 2012 11:04 UTC (Mon) by dwmw2 (subscriber, #2063) [Link] It's a generalisation. Of course it isn't 100% accurate but it's a good predictor. It's much the same as "code stored in BZR or SVN probably isn't worth the pain of trying to get it out of its antiquated version control system to look at it" — there are exceptions, but they are few.
Fedora and LVM Posted Nov 6, 2012 13:33 UTC (Tue) by nix (subscriber, #2304) [Link]
The thing is, that's pretty much not true. Sure, dealing with such version control systems is painful, but if I followed those rules, I'd lose the source trees for Emacs, KDE (before the recent move to git), Enlightenment, Calibre... there's no real correlation between quality and choice of VCS, though if someone is still using CVS it is probably a sign that the project is not very actively maintained. Equally, there's no correlation between 'project is forced to use SysVIPC to interoperate with other software' and general quality -- only if the project used SysVIPC when it had another choice can such a conclusion be drawn.
Fedora and LVM Posted Nov 6, 2012 16:19 UTC (Tue) by admax88 (subscriber, #75035) [Link]
OpenBSD uses CVS, and its pretty actively maintained and very high quality.
Fedora and LVM Posted Nov 6, 2012 16:45 UTC (Tue) by nix (subscriber, #2304) [Link]
True. There are rare exceptions even here. (I can't think of many, though.)
Fedora and LVM Posted Nov 6, 2012 16:52 UTC (Tue) by admax88 (subscriber, #75035) [Link]
Most of the original GNU software is still in CVS.
Just because a project doesn't adapt to the new hotness in version control doesn't say anything about the quality of the code.
The choice of VCS is more likely an indicator of the age of the project and/or the age of the developers.
Fedora and LVM Posted Oct 31, 2012 18:50 UTC (Wed) by paulj (subscriber, #341) [Link]
My killer-app use-case for LVM is doing upgrades to a new root, while preserving ability to boot to the old. I snapshot my existing root, then clone it onto to a new root LV. This allows me to preserve my old root, and be able to fallback to it should something go wrong with the upgrade (which it does - Fedora Linux, perhaps Linux generally, seems to be getting ever more finicky about installed versions of initrd, kernel and inits).
Without LVM, I'd have to do the upgrades in place. With the risk of being fscked if it goes wrong. (Yes, I should burn recovery CDs in advance - Yum upgrade fails catastrophically infrequently enough that I often don't, but frequently enough though that I have lost day+ to reinstall and reconfigure more than once).
Fedora and LVM Posted Oct 31, 2012 18:54 UTC (Wed) by paulj (subscriber, #341) [Link]
Oh, this is so useful, that I'm surprised that Fedora doesn't make use of LVM snapshots to do revertable upgrades.
If LVM snapshots could be promoted to writeable LVs, this process would be even easier. There doesn't seem be a way though. Since Solaris switched to ZFS and IPS, it does upgrades this way I think - through writeable snapshots.
Fedora and LVM Posted Oct 31, 2012 20:11 UTC (Wed) by dtlin (✭ supporter ✭, #36537) [Link] Not sure what you mean by "writable LV" – LVM snapshots are already writable. If you want an wholly independent LV – perhaps you could just copy the snapshot to a new LV. lvconvert --merge merges a snapshot back into the origin, for when you only want the snapshot's contents and not the origin's anymore. (Hypothetically, at least. I've not yet tried it myself.)
Fedora and LVM Posted Oct 31, 2012 22:28 UTC (Wed) by paulj (subscriber, #341) [Link]
They're writable now? That's changed then, and good news for me. Thanks! I'd heard DM might have merge back support somewhere, but hadn't noticed the LVM UI for it was in lvconvert - I've been looking at lvchange the whole time.
Thanks :)
Fedora and LVM Posted Nov 1, 2012 12:10 UTC (Thu) by Cato (subscriber, #7643) [Link]
LVM snapshots are as buggy as a very buggy thing ... search "lvm bugs snapshots" for more and see the comment from the schroot developer above. They are also very slow because they multiply number of I/Os required for a changed block.
Seriously, do a lot of testing before you rely on LVM snapshots. Or just use btrfs.
Fedora and LVM Posted Nov 1, 2012 23:02 UTC (Thu) by agk (subscriber, #23332) [Link]
> LVM snapshots are as buggy as a very buggy thing
They are pretty robust nowadays: the last kernel race I can recall that could lead to incorrect data was found a few years ago now.
But they are not designed to be efficient if they grow large or if you have many of them; and make sure they never run out of space (use the snapshot_autoextend* lvm.conf settings / --monitor y / dmeventd).
Fedora and LVM Posted Oct 31, 2012 21:02 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
on my servers, I just do two root partitions and install the new OS into the currently unused parition.
works very well, no LVM needed.
I've also been bit my LVM and mount by UUID several times (did you know that mounting by UUID doesn't work if you have too many drives in your system? at least on some distros)
Fedora and LVM Posted Nov 1, 2012 3:41 UTC (Thu) by faramir (subscriber, #2327) [Link]
>On my servers, I just do two root partitions and install the new OS into the currently unused parition.
That's fine for a fresh install. I believe the use case was for a easily revertable upgrade. Not the same thing.
Fedora and LVM Posted Nov 7, 2012 13:00 UTC (Wed) by emmi3 (subscriber, #62443) [Link]
But it's not so much different either. I've ben using this "two primary root partition"-setup for quite some time on all my systems at work. Although I usually prefer to reinstall my desktop systems, this is how I would handle the "revertable upgrade" case:
- create an new filesystem with a different label on the unused partition
If anything goes wrong just reboot into the old root. Using "MBR" this is a simple matter of pressing the shift-key and choosing the right partition at boot time.
Fedora and LVM Posted Nov 7, 2012 18:26 UTC (Wed) by faramir (subscriber, #2327) [Link]
That's quite a lot of manual labor for every time you want to do a revertable upgrade. Which if it was easy enough, I think many would want to do for every upgrade.
Setting up LVM (with snapshots for /) might be a bit more difficult during the initial install, but it seems to me it is going to be a lot more convenient in the end.
In either case, you have to have done something up front during the initial install. i.e. Either used LVM or set aside a partition for your second root.
Fedora and LVM Posted Nov 2, 2012 7:45 UTC (Fri) by Cato (subscriber, #7643) [Link]
I would far prefer to avoid buggy LVM snapshots, and the fragility of root on LVM that others have mentioned.
There's a simple non-LVM approach I've used a lot, which is to create two root FS partitions - install your current distro on one, then do 'rsync -avH' from one to the other, boot into that one, and do the upgrade there.
Or simply use a good backup tool to make a backup first - a 2TB drive is not much over $100 these days, just use rsnapshot or your preferred backup tool. If you run rsnapshot daily, it will complete very quickly on a root FS that doesn't change very much, and you can then easily do file-restores and inter-day comparisons of config files (not quite as nice as etckeeper but pretty good).
Once btrfs is more mature, its snapshots look really nice, and much faster than LVM's. Of course, backups are a really good idea either way.
Fedora and LVM Posted Nov 1, 2012 23:55 UTC (Thu) by Tobu (subscriber, #24111) [Link] Flexibility. It removes two restrictions that feel idiotic when one encounters them, namely: “oh you still have some space left but you can't give it to that filesystem, it's in the wrong place silly”; “oh you added a disk but you still can't give extra space to that filesystem”. These are punishment for not having supernatural foresight when buying drives and they are enraging. Now Btrfs would also fix these, but stability and performance aren't up to scratch yet. The other benefit is not nearly as important, but still a good thing. LVM gives the ability to spin up VMs easily without relying on the host filesystem.
Fedora and LVM Posted Nov 1, 2012 23:58 UTC (Thu) by Tobu (subscriber, #24111) [Link] That was meant as a reply to dashesy, sorry.
Fedora and LVM Posted Nov 2, 2012 0:19 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
the question is which issue do you run into more frequently
the need to resize anything other than the last partition on a disk (or have a filesystem span multiple drives)
or
the problames caused by LVM
for many of us, the cases where LVM would be useful are much more rare than the cases where LVM is a problem.
I almost never resize partitions after the system is created (especially on production systems)
I would not WANT to have a single filesystem that spanned multiple drives, that's what's known as RAID 0, and it means that if _either_ drive has problems, I can loose the entire filesystem.
meanwhile, the downsides have caused me problems more than once (including cases with colo systems where a drive has partially failed, the facility has replaced the drive and then 'helpfully' plugged in the old drive, now both have default LVM configs and the system is unusable)
Fedora and LVM Posted Nov 2, 2012 0:46 UTC (Fri) by Tobu (subscriber, #24111) [Link] That's “often” in the first column, and “huh?” in the second. The functionality is pretty basic and I've found it reliable. Most of my experience is from a desktop (long-lived in the same way as Theseus' ship). That said, I don't mind learning from your colo problem. What made it hard to fix? Was the collision on UUIDs or logical names?
Fedora and LVM Posted Nov 2, 2012 1:13 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
I think it was in logical names (others in this thread have mentioned being hit by that problem as well)
Fedora and LVM Posted Nov 2, 2012 7:49 UTC (Fri) by Cato (subscriber, #7643) [Link]
I've seen similar issues on mailing lists when a hard disk from one Fedora machine is plugged into another Fedora machine - since they both use the default VG and LV names, much confusion results and it's not easy to recover from.
One tip for using LVM is to never use the default VG and LV names, to avoid this sort of thing.
I've lost far more time (and data) due to LVM complexities/issues than I have saved through easy expansion/moving of FSs. It makes upgrades and transferring to new PCs much more complex.
Fedora and LVM Posted Nov 2, 2012 13:07 UTC (Fri) by Tobu (subscriber, #24111) [Link] I've just tried it (using a VM to create a colliding VG name, then kpartx -a to make the nested partitions show up). $ sudo pvscan WARNING: Duplicate VG name V2: Existing 5daecde1-e30c-4ef8-bcf6-e501382f67b7 (created here) takes precedence over 8b814833-1a37-47d4-896d-5a8aa74f458f PV /dev/mapper/sda2p1 VG V2 lvm2 [1016,00 MiB / 0 free] PV /dev/sda5 VG V2 lvm2 [1,82 TiB / 13,00 GiB free] $ sudo vgrename 8b814833-1a37-47d4-896d-5a8aa74f458f V3 Volume group "V2" successfully renamed to "V3" Accessing the partition was very easy, but then I had to edit the root= boot parameter to boot the VM (vgcfgrestore to undo the rename would also work). This could be improved by having root= be uuid-based (which Debian/Ubuntu don't do for LVM, don't know about Fedora), as well as by having the Fedora installer pick a more unique VG name. Debian and Ubuntu pick the hostname as the VG name, this seems like a good way to prevent these collisions entirely.
|
|||||||||||||