User: Password:
|
|
Subscribe / Log in / New account

XFS: the filesystem of the future?

XFS: the filesystem of the future?

Posted Jan 23, 2012 0:26 UTC (Mon) by dgc (subscriber, #6611)
In reply to: XFS: the filesystem of the future? by hmh
Parent article: XFS: the filesystem of the future?

Hi hmh,

> The thing is, on any server worth of notice you will have 8 to 32 cores,
> and mkxfs will usually give you at least 4 aggregation groups per
> filesystem.

The default for non-raid devices (single drives or hardware RAID that does not advertise it's configuration) is 4AGs below 4TB, and then scales at 1AG per TB. I was testing on a 17TB volume, so the default mkfs config gave 17 AGs because the virtio device doesn't pass on RAID alignment from the host.

I even pointed out in the talk some performance artifacts in the distribution plots that were a result of separate threads lock-stepping at times on AG resources, and that increasing the number of AGs solves the problem (and makes XFS even faster!) e.g. at 8 threads, XFS unlink is about 20% faster when I increase the number of AGs from 17 to 32 on teh same test rig.

If you have a workload that has a heavy concurrent metadata modification workload, then increasing the number of AGs might be a good thing. I tend to use 2x the number of CPU cores as a general rule of thumb for such workloads but the best tunings are highly depended on the workload so you should start just by using the defaults. :)

> Desktops and laptops might well be best served by a different fs, which is fine.

Which for me all run XFS.

Indeed, laptops and desktops are signficantly more powerful than you give them credit for. My desktop has 8 CPU threads, 500MB/s of IO bandwidth and can do 70,000 random 4k write IOPS and that cost less than $AU1500 when I bought it a couple of years ago. That's a serious amount of capability at very low cost, yet ext4 struggles to make use of all that capability. XFS, OTOH, is a perfect fit for such configurations, especially if you are running highly parallel applications (like kernel builds) all the time on your desktop....

This is one of the things I'm trying to make people aware of - that high performance (SSD) and large scale storage (4TB drives) are here right now and are affordable on your desktop and laptop. Filesystem concurrency and high throughput is not a supercomputer or high end server problem anymore - what you need from your desktop filesystem to use the maximum potential of your storage hardware is very different from what was needed 5 years ago....

> Use the best tool for the job, and the tools need not all be the same
> tool.

I couldn't have said it any better myself. :)

Dave.


(Log in to post comments)

XFS: the filesystem of the future?

Posted Feb 7, 2012 2:14 UTC (Tue) by jmalcolm (guest, #8876) [Link]

This is really one of my favourite things about Open Source. Much of the amazing technology that allows me to get the most out of my modest hardware was originally designed for very high-end workstation and data-center type use. As commodity hardware approaches the capabilities of yesterday's high end the benefits of the software trickle-down to the rest of us.

XFS was not designed for home media server or my development laptop but I sure enjoy it anyway.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds