|
|
Subscribe / Log in / New account

New tricks for XFS

New tricks for XFS

Posted Feb 22, 2018 21:42 UTC (Thu) by nix (subscriber, #2304)
In reply to: New tricks for XFS by dgc
Parent article: New tricks for XFS

That's *clever*. (And I clearly need to watch your talk, which has been burning a hole in my disk space for a few weeks now.)

I saw the thin provisioning-aware stuff go by, but didn't realise it would have *this* consequence. I mean, thin provisioning is all very well but unless you're running a data centre or testing filesystems isn't really important in your day-to-day life. But everyone likes snapshots :)

Of course this also obsoletes LVM: instead of LVM, you have a single giant XFS filesystem with each "partition" consisting of a sparse disk image containing whatever filesystem you like (as long as it supports trimming of unused regions): if it's XFS, it asks the containing fs to CoW the appropriate metadata regions whenever a subvolume is created, and bingo.

There isn't even more metadata than there would otherwise be, despite the existence of two filesystems where before you had one, since the outermost one has minimal metadata tracking the space allocation of the contained fs and that's just about all. As lots of subvolumes are created, the extents in that fs tracking the increasingly fragmented thin file containing the subvolumes will grow ever smaller (the same problem CoW filesystems have traditionally had with crazy fragmentation), but defragmenting single files with XFS is something it's been very good at for a long time, and it's a heck of a lot easier than defragmenting a filesystem with CoW metadata would be: the metadata in the contained fs, of course, is completely unchanged by any of this, rather than having to go through rewrite-and-change hell.

I'll admit it. I want all this cool stuff *now* and I'm kicking myself that I re-mkfsed my big fileserver only eight months ago and can't convince myself to redo it like this instantly. :)


to post comments

New tricks for XFS

Posted Feb 22, 2018 22:12 UTC (Thu) by rodgerd (guest, #58896) [Link] (3 responses)

> I'll admit it. I want all this cool stuff *now* and I'm kicking myself that I re-mkfsed my big fileserver only eight months ago and can't convince myself to redo it like this instantly. :)

1. Convert big fileserver into two Gluster servers.
2. Reformat them with Cool Stuff one by one.
3. Beg the Gluster maintainers to integrate their tooling with the new XFS features.

New tricks for XFS

Posted Feb 22, 2018 22:44 UTC (Thu) by nix (subscriber, #2304) [Link] (2 responses)

I'm not sure how you convert one big machine into two servers. With an axe? (It has an odd number of disks in a RAID array -- not something I can evenly split :) ).

I am vaguely tempted to do something kinda similar with my new desktop, which will likely come with an MMC far too small to hold the fs I want to put on it, but *with* nice fast 10GbE (imagine my surprise to find that managed switches with a couple of 10GbE ports on have plunged in price to under £200 nowadays). I was thinking of putting Lustre on it as a way to split the metadata onto the MMC and the data onto the big machine over the 10GbE :) however, it's quite possible that NFSv4 onto a big RAID array over the 10GbE would be *faster*, particularly for writes. In practice I suspect I'll be happy with just nfsroot and ignoring the MMC entirely. (Which means I'll have to thunk to the fileserver to do snapshots etc in any case... though if I go to NFSv4.2 I will at least be able to do reflinks on the client.)

New tricks for XFS

Posted Feb 23, 2018 4:52 UTC (Fri) by rodgerd (guest, #58896) [Link] (1 responses)

> I'm not sure how you convert one big machine into two servers. With an axe?

I find "being a pack rat with more cases and motherboards lying about than is reasonable" seems to work acceptably well ;). The odd number of disks is a bit trickier.

> I was thinking of putting Lustre on it as a way to split the metadata onto the MMC and the data onto the big machine over the 10GbE :)

Good heavens. That's a dedication to complexity! I'm moving my performance-sensitive Gluster FSes to use tiering, so I'm in no position to criticise, though.

> imagine my surprise to find that managed switches with a couple of 10GbE ports on have plunged in price to under £200 nowadays

The plunging price of network equipment continues to astonish me. They days where just management was a prohibitively expensive option are long gone, and I'm loving it.

New tricks for XFS

Posted Feb 24, 2018 20:06 UTC (Sat) by naptastic (guest, #60139) [Link]

> I find "being a pack rat with more cases and motherboards lying about than is reasonable" seems to work acceptably well ;). The odd number of disks is a bit trickier.

I've learned that the only way to know for sure that you have enough is to have too much.

New tricks for XFS

Posted Feb 23, 2018 22:08 UTC (Fri) by dgc (subscriber, #6611) [Link] (1 responses)

> Of course this also obsoletes LVM: instead of LVM, you have a single giant XFS filesystem
> with each "partition" consisting of a sparse disk image containing whatever filesystem you
> like (as long as it supports trimming of unused regions): if it's XFS, it asks the containing
> fs to CoW the appropriate metadata regions whenever a subvolume is created, and bingo.

No, it doesn't obsolete LVM. You still need something to manage your physical storage that the base XFS filesystem is placed on. What it allows is much more flexible use and management of the space within that XFS filesystem, but it doesn't change how you manage the underlying devices. Especially if you want to grow the base filesystem in future....

New tricks for XFS

Posted Mar 3, 2018 14:00 UTC (Sat) by nix (subscriber, #2304) [Link]

You still need something to manage your physical storage that the base XFS filesystem is placed on.
I'm not sure why one giant filesystem completely filling the physical storage needs management -- but I suppose if it's sitting on a RAID array or something, that array is more growable than if it were a simple disk of that size, so you might still need some layer between the physical storage and the top-level XFS.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds