User: Password:
|
|
Subscribe / Log in / New account

XFS: the filesystem of the future?

XFS: the filesystem of the future?

Posted Jan 25, 2012 18:00 UTC (Wed) by sbergman27 (guest, #10767)
Parent article: XFS: the filesystem of the future?

The video is available on Youtube. I would suggest watching it. Then watching it with the sound turned down, looking at the graphs and asking "Which filesystem looks best for *my* workloads?", at each step of the way. Even with all the new improvements to XFS, and despite the common disparagement of EXT4 as being old fashioned, EXT4 still beats XFS handily in Chinners own graphs for 1, 2, and 4 threads. XFS's domain begins at 8 threads. Why do we need EXT4? The question is almost absurd. We need it because it's still the best fs out there for most machines today and for some finite time into the future.

Sometimes I wish that certain people could get their heads out of their Fortune 100 Datacenters for a bit and take a look at how the rest of us live.

For balance, I'd recommend Avi Miller's "Best of #1" talk "I can't believe this is Butter", about btrfs. One of his messages is "use the right fs for the job". Far less defensive, dismissive, and confrontational than Dave Chinner's talk. And perhaps a bit more pertinent to those of use out here managing a few Dell T310's without fancy petabyte RAID arrays.


(Log in to post comments)

XFS: the filesystem of the future?

Posted Jan 26, 2012 4:12 UTC (Thu) by dgc (subscriber, #6611) [Link]

> Then watching it with the sound turned down

That's just absurd. When you strip something of all context, you can make it mean anything you want regardless of what the original message was. I was expecting to be quoted badly out of context, but that just takes the cake....

> And perhaps a bit more pertinent to those of use out here managing a
> few Dell T310's without fancy petabyte RAID arrays.

That's *exactly* the point of my talk - to smash this silly stereotype that XFS is only for massive, expensive servers and storage arrays. It is simply not true - there are more consumer NAS devices running XFS in the world than there are servers running XFS. Not to mention DVRs, or the fact that even TVs these days run XFS.

But to address you real world concern, all those benchmarks were run on a low-spec Dell R510 that cost a bit over $AU10,000 _two years ago_. You find them in datacenters everywhere. Hence those results are completely relevant to the low end server users that only have a handful of disks. The graphs clearly show that ext4 cannot scale to the capabilities of even low end servers and storage, let alone the massive servers where XFS dominates.

What you get in a $10000 server these days is the equivalent of a half million dollar supercomputer from five years ago. XFS worked (and still works) better than anything else on those 5 year old supercomputers. The talk showed that XFS also works better than anything else on new, low end hardware that, just co-incidentally, has the with equivalent capability of that five year old supercomputer....

The point is I then extrapolated from there - if ext4 can't handle current low end hardware, then what about the sort of hardware that will be available in 5 years time? There is no coherent plan to acheive this for ext4, while for XFS we are already running on supercomputers that will be the low end of the server market in 5 years time.

The difference is that the XFS developers tend to look years ahead to what we'll need in 5 years time to continue to be relevant to our users. Much of what I talked about I documented in mid-2008 on the XFS wiki. At the time I knew there was 5-10 years worth of work in everything I documented, but it would be necessary to implement it in that same time frame.

In contrast, the ext4 guys are still trying play catchup with what we could do more than 10 years ago and are now trying to work around architectural deficiencies to keep up. Do we really need to keep throwing good money after bad trying to re-invent the wheel with ext4, or should we just cut our losses and move on?

That's the question I'm asking, and to just look at a few of the graphs and ignore the rest of the context of the talk is doing everyone a disservice. We need to talk about issues, not perpetuate myths and stereotypes that are long out of date....

Dave.

XFS: the filesystem of the future?

Posted Jan 26, 2012 15:51 UTC (Thu) by SLi (subscriber, #53131) [Link]

I doubt that 5 years from now the norm for *desktops* will be 12 disks on a RAID and 8 threads writing on it.

I think <8 threads writing concurrently is going to be the normal case for desktop systems for quite a while, and I'm not sure any amount of hand-waving is going to dismiss XFS being 50% slower compared to ext4 in your benchmarks in the case that is important to desktop users.

No, 3 seconds compared to 2 seconds doesn't sound so bad, but I'd expect that to mean also that the operations that now take 2 minutes on my ext4 to take 3 minutes on XFS. A 50% difference certainly means it is appreciably slower.

Simply put, I don't think scalability in the way it is exercised in your tests is relevant to desktop users currently or in the near future. More than 8 threads may be imaginable in the future, but 12 disks is not. At least currently I care way more about a single-threaded workload being much slower on XFS (per your benchmarks). It *is* the absolute numbers that matter, after all.

XFS: the filesystem of the future?

Posted Jan 26, 2012 20:33 UTC (Thu) by dlang (subscriber, #313) [Link]

today most SSD drives internally act as if they are a raid set, so saying that in 5 years that RAID will be a common desktop/laptop feature is actually a fair statement.

as for multiple thread writing to it, that's more common than you think as well (media player streaming, desktop environment saving state, word processor saving state, spreadsheet saving state, .... )

XFS: the filesystem of the future?

Posted Jan 28, 2012 2:48 UTC (Sat) by sbergman27 (guest, #10767) [Link]

"today most SSD drives internally act as if they are a raid set"

Maybe. But SSD drives act, externally, as 1 drive. A single drive with excellent seek time and a sometimes very good data rate. If the drive can do parallelization of a single stream of data that means the FS and block i/o layers don't have to be so fancy. +1 for EXT4.

Regarding multiple threads writing... do you really think media player streaming and word processor saving state is going to require a half gig a second via 5+ threads for extended periods in the near future? If so, I think I'll go dust off my TV and typewriter. +1 for Magnavox and Royal.

XFS: the filesystem of the future?

Posted Jan 27, 2012 0:50 UTC (Fri) by dgc (subscriber, #6611) [Link]

> I doubt that 5 years from now the norm for *desktops* will be 12 disks
> on a RAID and 8 threads writing on it.

That device used RAID to get the capacity over 16TB, and 512MB of BBWC to keep the IOPS and bandwidth high to/from spinning disk storage. It sustained up to 10,000 IOPS and 600MB/s on some of those workloads. That's the only reason RAID was used.

To put that IO capability in context, my desktop already has 8 processor threads and a pair of SSDs that sustain 70,000 random write IOPS and 500MB/s. That cost me about $1500 a couple of years ago. It has more capability from the IO perspective than the hardware that I ran the tests on (except for capacity). And I do 8-way kernel builds on it, so I'm already writing with 8 threads to the filesystem on my desktop machine.

IOWs, the test system and tests I described are relevant not only to people with current low end servers but also relevant to those with current medium-to-high-end desktops and laptops. As I've said previously, that was the whole point of the exercise - to show how capable cheap, low end storage is these days and how only XFS can make full use of it....

> Now, 3 seconds compared to 2 seconds doesn't sound so bad, but I'd expect
> that to mean also that the operations that now take 2 minutes on my
> ext4 to take 3 minutes on XFS. A 50% difference certainly means it is
> appreciably slower.

You simply can't make that sort of generic extrapolation from one workload to all workloads. Because the main tests I ran reflected one -specific- aspect of metadata workloads (creates in isolation, read in isolation, unlink in isolation) you can use those numbers to get an idea of how a workload that is heavy in those behaviours might perform and scale. But you cannot say that all workloads will end up with a specific performance differential just because one workload had that differential...

Dave.

XFS: the filesystem of the future?

Posted Jan 28, 2012 2:32 UTC (Sat) by sbergman27 (guest, #10767) [Link]

"That's just absurd. When you strip something of all context, you can make it mean anything you want regardless of what the original message was."

I suggested watching the video with the sound up first. And then turned down to remove the salesman's pitch and allow one to concentrate on the actual data being presented. Nothing absurd about that.

I administer point of sale and accounting servers that typically handle ~100 users or registers. And also servers which handle ~80 users NX desktops. Not one of them could benefit from > 4 thread write performance. More than anything, I might could use improved fs performance for 1 or 2 threads.

Mine are not the sorts of customers who pay Red Hat the big bucks. But customers *like mine* outnumber Red Hat's customers.

I don't really care how XFS might perform on hardware that will be available in 5 years time. I care what makes sense for our current hardware. In 5 years, we'll decide what FS is suitable for our new hardware at that time. And as always, I will consider XFS along with the other options. Just because it has never been quite suitable before does not mean that it can't be next time.

-Steve

XFS: the filesystem of the future?

Posted Jan 28, 2012 17:01 UTC (Sat) by sbergman27 (guest, #10767) [Link]

"there are more consumer NAS devices running XFS in the world than there are servers running XFS"

BTW, can you point me to the reasons for chosing XFS in such an environment? The kernel module for XFS is huge. 1.7MB on the SL 6.1 machine in front of me. By "consumer" are you referring to the sub-$200 stuff that can be bought at, say, BEST BUY? I would be very interested in the rationale. I have a few Buffalo router/NAS devices. 32MB flash/64MB ram. USB 2.0. What would you say the advantages of running XFS rather than EXT4 might be on this device?

-Steve

XFS: the filesystem of the future?

Posted Feb 6, 2012 1:05 UTC (Mon) by dgc (subscriber, #6611) [Link]

> BTW, can you point me to the reasons for chosing XFS in such an
> environment?

XFS is very reliable and performs extremely consistently over the expected life of such devices? And XFS tends to perform better on NAS workloads than ext3/4 on the same hardware? And that mkfs doesn't make the user wait for a *long time* before they can use the device? And that there are tools in xfsprogs designed to optmise the manufacturing process (e.g. xfs_copy), and so on?

> By "consumer" are you referring to the sub-$200 stuff that can be
> bought at, say, BEST BUY?

Yes, exactly. Brands like thecus, netgear, dlink, etc either use XFS by default or recommend it over ext3/4 in most situations.

> I have a few Buffalo router/NAS devices.

I'm pretty sure they use XFS, too. Certainly google indicates that even the low end buffalo NAS boxes go faster with XFS on them....

Dave.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds