The POSIX talk is pointless [LWN.net]

The POSIX talk is pointless

Posted Mar 16, 2009 23:15 UTC (Mon) by sbergman27 (guest, #10767)
Parent article: Garrett: ext4, application expectations and power management

The situation here, from a bird's eye view, is quite simple. If ext4, with real applications running in the real world, exhibits this behavior then ext4 and Linux, not necessarily in that order, are going to come to be considered unstable crap. This includes old abandoned FOSS code as well as proprietary code used by businesses that is not likely to get rewritten any time soon. The Linux and Ext4, and not the application developers, will take the fall. If the solution involves hurting Ext4 in the benchmarks, then so be it. If it means a little more fragmentation, so be it. I cannot believe that there is any question about this. If nodelalloc needs to be made the default, with a big warning slapped onto the option that enables delayed allocation, then so be it. It seems absolutely surreal to see extX devs debating whether an easily demonstrable and severe lack of data integrity might be worth it to get better benchmark numbers. (And it makes me start wondering how much I should trust ext4, in general.) I guess maybe they've just been obsessing so much on bechmark figures that they've totally lost perspective. Hopefully, it is just a temporary condition.

to post comments

The POSIX talk is pointless

Posted Mar 16, 2009 23:45 UTC (Mon) by foom (subscriber, #14868) [Link] (5 responses)

> If ext4, with real applications running in the real world, exhibits this behavior then ext4 and
Linux, not necessarily in that order, are going to come to be considered unstable crap.

Please note that they already made and applied patches to restore the behavior people are arguing
for. You don't have to convince anybody that the most practical thing to do right now is fix this
behavior in the filesystem. So ext4 does *not* exhibit this behavior, anymore.

The only question remaining is if the filesystems *should* have had to make this change, or if what
they were doing before is really perfectly okay, except for all those darn applications people wrote
all of which are broken.

The POSIX talk is pointless

Posted Mar 17, 2009 0:40 UTC (Tue) by sbergman27 (guest, #10767) [Link] (4 responses)

I started to post again to clarify that point. But decided not to.

My understanding is that those patches keep existing files from being zapped, but still provides significantly less in the way of guarantees for the data integrity of new files. I simply don't see any room for reliability regressions at default settings, relative to ext3, in the name of performance, period. Old files, new files, whatever. If delayed allocation makes your data less safe and that can't be fixed, then it needs to be turned off by default. People can turn it on to get their benchmark numbers.

One of the major benefits of delayed allocation is supposed to be reduced fragmentation. Well, we're supposed to be getting an online defragger, aren't we? And I thought ext filesystems were already supposed to be doing well enough on fragmentation avoidance that we didn't really have to worry about it anyway.

The POSIX talk is pointless

Posted Mar 17, 2009 7:09 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (3 responses)

The patches fix the behaviour to match application usage patterns and reliability doesn't seem to be affected anymore than Ext3 for the typical use cases plus you get a nice performance benefit.

Ext3 avoids fragmentation to a good extend but it is still possible to run into situations where the filesystem is badly fragmented especially as you get closer to reaching your storage limits. Ext4 improves the fairly simple and stupid allocation method in Ext3 with a more intelligent one. A online defragmenter is a additional feature and not a replacement of the allocator.

The POSIX talk is pointless

Posted Mar 17, 2009 13:46 UTC (Tue) by sbergman27 (guest, #10767) [Link] (2 responses)

"""
...reliability doesn't seem to be affected anymore than Ext3 for the typical use cases...
"""

That sounds like a "weasel phrase". "Typical Use Cases"? The lofty goal of ext4 is now to preserve user data in "typical use cases"?

If I mount two filesystems, one ext3 and one ext4, both using the defaults. And then open a new file on both, write some data to both, close both, wait 30 seconds, and pull the power plug, what happens?

If the answer is "the same thing for both" or "ext4 is more reliable in that scenario" then great. If the answer is "ext4 is less reliable" then ext4 will never see my production systems, with or without nodelalloc.

The POSIX talk is pointless

Posted Mar 17, 2009 17:04 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (1 responses)

I can't speak on behalf of Ext4 developers but no filesystem will preserve your data in all cases. The principle is to optimize for the common use cases and then add things to cover the corner cases as well to the extend possible. That is what has happened in Ext4 as well.

With the current patches, you should see the same behaviour in both filesystems. Feel free to test and report if you see any changes.

The POSIX talk is pointless

Posted Mar 17, 2009 17:31 UTC (Tue) by sbergman27 (guest, #10767) [Link]

"""
...no filesystem will preserve your data in all cases...
"""

Ext3 with data=journal will come quite close to it, though. In fact, ext3 with data=ordered comes impressively close to it.

"""
With the current patches, you should see the same behaviour in both filesystems.
"""

I'll be running exactly this scenario when 2.6.30 is released to see what happens.