Fedora and LVM [LWN.net]

Fedora and LVM

Posted Oct 31, 2012 20:55 UTC (Wed) by rleigh (subscriber, #14622)
In reply to: Fedora and LVM by mjg59
Parent article: Fedora and LVM

There are some serious quality issues with LVM. I can (and have) locked up the kernel hard many times (no panic, just completely dead) using nothing more than "lvcreate -s" and "lvremove".

The schroot tool uses LVM snapshots to create and destroy transient scratch build environments. We use them on the Debian autobuilders. If you kick off 24 parallel builds on a 24 core system, the system will be dead as a doornail in a few tens of seconds. Entirely due to lvcreate/lvremote triggering what looks like some kernel locking bug. Even on systems only running a single build, we still regularly have lvcreate and lvremove failures. There's some fundamental bugs in LVM which really need fixing, and which I'm surprised haven't been addressed given that they are easily reproducible. schroot is admittedly a special case--most people don't churn through as many LVs as we do--rebuilding the whole archive is ~14 hours with 24 parallel builds, and ~18000 transient LVs (though it always died in under 5 mins, less than 100 LVs in). But it should certainly work without killing your system.

We now also support btrfs snapshots, and while the filesystem itself is still not perfect, I've not yet run into a single issue doing heavy parallelised snapshotting.

(Log in to post comments)

Fedora and LVM

Posted Oct 31, 2012 22:18 UTC (Wed) by rleigh (subscriber, #14622) [Link]

IIRC it was sent to the LVM and/or kernel lists.

Fedora and LVM

Posted Nov 1, 2012 21:39 UTC (Thu) by agk (guest, #23332) [Link]

> IIRC it was sent to the LVM and/or kernel lists.

Please would you dig up and post a link to this here?

Fedora and LVM

Posted Nov 1, 2012 23:23 UTC (Thu) by rleigh (subscriber, #14622) [Link]

I've had a good search, but I'm afraid I can't find it on either list, sorry. I may be misremembering, or just looking in the wrong place.

Fedora and LVM

Posted Nov 2, 2012 0:11 UTC (Fri) by agk (guest, #23332) [Link]

Well if you're still having problems, please let us know the details and we'll see what we can suggest. If it was 24 writeable snapshots of the same device, then it might be worth trying the new thin snapshots. (Set up the origin as a thin device, then drop the size parameter from CHROOT_LVM_SNAPSHOT_OPTIONS.)

Fedora and LVM

Posted Nov 2, 2012 0:28 UTC (Fri) by rleigh (subscriber, #14622) [Link]

Thanks Alasdair, it was indeed all writable snapshots of the same device in this instance. I'll give thin snapshots a go with a current kernel. I no longer have access to the 24 core system (which was also remote, making debugging lockups difficult), but I'll see what I can do on a local quad core system.

Fedora and LVM

Posted Nov 1, 2012 12:06 UTC (Thu) by Cato (subscriber, #7643) [Link]

I think most LVM users don't do much with snapshots, if they even use them, and the snapshot code is much more buggy as a result.

The non-snapshot LVM/DM code is not too bad, though there are some tools issues - e.g. pvmove can corrupt data if it runs out of memory - see http://serverfault.com/a/339899/79266 and http://deranfangvomende.wordpress.com/2009/12/28/a-primer...

Fedora and LVM

Posted Nov 1, 2012 17:44 UTC (Thu) by nix (subscriber, #2304) [Link]

pvmove uses snapshots internally, so this is just another indication of the general principle that LVM snapshots are dangerous.

Fedora and LVM

Posted Nov 1, 2012 21:32 UTC (Thu) by agk (guest, #23332) [Link]

> pvmove uses snapshots internally

pvmove uses mirrors, not snapshots.

It works by setting up temporary mirrors for the kernel to sync the data between the old and new locations.

Fedora and LVM

Posted Nov 1, 2012 22:29 UTC (Thu) by nix (subscriber, #2304) [Link]

Oh. You're right, of course. (I've only ever encountered one pvmove failure, anyway -- a deadlock -- and it restarted fine on reboot.)