Changing filesystem resize patterns [LWN.net]

Changing filesystem resize patterns

Posted May 11, 2022 20:50 UTC (Wed) by NHO (subscriber, #104320)
Parent article: Changing filesystem resize patterns

Why not have a --preset parameter for mkfs, with some build-in sane parameters for other conflicting situations? And maybe hint in stderr if preset is most likely wrong. And add command line override of some preset parameters.
Trying to guess future isn't best idea, as we are all know, so user with plans will be able to easily select file system configuration that works, user who's making a thumbstick will get good thumbstick, and user who naively creates fs with wrong settings for backing device will get notification that fs may better be recreated differently.

to post comments

Changing filesystem resize patterns

Posted May 11, 2022 23:52 UTC (Wed) by gerdesj (subscriber, #5446) [Link] (1 responses)

The wizard approach is a great idea, I think. However, filesystems are sodding complicated. Trying to work out how to get the best out of a given setup is hard.

I recently specified a box for around 100TB of backup (Veeam) storage on a budget. I went for a reconditioned Dell T430, rack mounted (gets you eight 3.5" bays), a PERC RAID controller with 8GB RAM and eight 16TB SATA. RAID 6 takes about a day to build with nothing else going on so probably two to three days in use. Hence RAID 6 and not 5. I went for a larger stripe size than default because this thing holds a few large files. Now for the file system.

It's XFS for the reflinks or ReFS and Windows (no thanks)! Much searching later I realized that this hardware RAID doesn't need to worry about the same things as when its on md and the like, ie worrying about the number of disc in the RAID and number of parity discs and stripe size. It seems that above a certain stripe size (1024KB I think), the PERC reports the maximum as that anyway via /sys/. It's some sort of secret sauce in the PERC according to some responses by Dell employees in some forum postings I found.

I ran mkfs.xfs with defaults!

Did I get the best out of the box? Well it twiddles its thumbs waiting for data to turn up and has about 88TB (real TB) of file system. I'm quite risk averse, hence RAID 6 and not 5, even though I have a spare disc on the shelf. I went for dual PSU, 64GB RAM and dual Gold Xeon CPUs, full iDRAC etc. It costed about £5000 + VAT (sales tax). The box has three years, next business day hardware support priced in too.

I priced up quite a few options and that works out at less than £60/TB useable space.

So in that case I think XFS just works out of the box - I think it reads the /sys/ files to work out what to do already - with a RAID.

However, the cases described in the article are simply mad in comparison with my rather normal, pedestrian use case. In general I want to create a 100GB to 10TB fs and bolt on more in increments of 10-50% every now and then. Mr Cloudy and his 10GB basic offering, flexing to 1PB in increments of 1GB can design their own fs to work with that use case.

I think that the state of the art fs development has already got it pretty much spot on and Mr Cloudy and co can damn well do the research and spend the money and work out what they need well away from the real world. Ideally they would collaborate on this in some sort of open way and we'd all benefit. There are billions of individuals who benefit from ext4, xfs, btrfs and all the rest being at the (b)leeding edge and easily available. There are billions of customers of cloud systems who would benefit from cloudy optimised file systems. There is a lot of crossover.

Perhaps we need to see xfs.orc, ext4.ggl, btrfs.faceb00c and the like being developed. I've no idea how that would work or if it is even possible.

Changing filesystem resize patterns

Posted May 12, 2022 5:45 UTC (Thu) by Wol (subscriber, #4433) [Link]

> So in that case I think XFS just works out of the box - I think it reads the /sys/ files to work out what to do already - with a RAID.

So I was told by the XFS guys ...

But they did say if you want to grow the volume, don't. Just create a new volume and move the data across (or grow it in increments of 100%).

Cheers,
Wol

Changing filesystem resize patterns

Posted May 12, 2022 3:38 UTC (Thu) by zev (subscriber, #88455) [Link]

e2fsprogs already has the logic for providing bundles of preset parameters for different usage patterns -- see the '-T' option to mke2fs, and the '[fs_types]' section of mke2fs.conf. The tricky part is in trying to automatically figure out which one is the "right" one to use, which might depend on the nature of the backing device (maybe known at mkfs-time, maybe not if you're just prepping an image to dump to the real device) and on future usage/resizing (much harder to predict, though the backing device might provide some clues to base guesses on).