User: Password:
Subscribe / Log in / New account

Shared pain

Shared pain

Posted Feb 8, 2012 4:08 UTC (Wed) by neilbrown (subscriber, #359)
In reply to: Shared pain by mjg59
Parent article: XFS: the filesystem of the future?

> we've just spent the best part of a decade using a filesystem that crippled any application that did.

That's the heart of the matter to me.... but now XFS - a filesystem that didn't cripple correct applicatons - is getting a hard time because it doesn't follow the lead of a filesystem that did.

And yes, I know, technical excellence doesn't determine market success, and even the best contender must adapt or die when facing with an ill-informed market. So maybe XFS should adopt the extX model for rename even though it hurts performance in some cases - because if it doesn't people might choose not to use it - and who wants to be the best filesystem that nobody uses (though XFS is a long way from that fate).

So I'm just being a lone voice trying to teach the history and show people that the feature they like so much was originally a mistake and the programs that use it are actually incorrect (or at least not-portable) and maybe there are hidden costs in the thing they keep asking for..

I don't expect to be particularly successful, but that is no justification for being silent.

(Log in to post comments)

Shared pain

Posted Feb 8, 2012 12:38 UTC (Wed) by mjg59 (subscriber, #23239) [Link]

Arguing "The specification allows us to do this" isn't something that convinces the people who consume your code. Arguing "Our design makes it difficult" is more convincing, but implies that your design stage ignored your users. "We made this tradeoff for these reasons" is something that people understand, but isn't something I've seen clearly articulated in most of these discussions. It just usually ends up with strawman arguments about well how did you expect this stuff to end up on disk when you didn't fsync, which just makes people feel like you don't even care about pretending to understand what they're actually asking for.

(Abstract you throughout)

Shared pain

Posted Feb 8, 2012 13:24 UTC (Wed) by nye (guest, #51576) [Link]

>That's the heart of the matter to me.

Then you have misunderstood the nature of the problem.

The problem is that there are cases when atomicity is required but durability is not so important. With ext3 (et al.) it is possible to get one without the other, but with XFS (et al.) atomicity can only be gained as a side-effect of durability, which is more expensive.

Thus, ext3 provides a feature which XFS does not - one which filesystem developers, as a rule, don't seem to care about, but application developers, as a rule, do. The characterisation of anyone who actually cares for that feature as 'ill-informed' is grating, even offensive to many.

General addendum, not targeted at you specifically: falling back to the observation that XFS's behaviour is POSIX-compliant is pointless because - though true - it is vacuous. In fact POSIX doesn't specify anything in the case of power loss or system crashes, hence it would be perfectly legal for a POSIX-compliant filesystem to fill your hard drive with pictures of LOLcats.

Shared pain

Posted Feb 8, 2012 22:29 UTC (Wed) by dlang (subscriber, #313) [Link]

and with ext3 it's not possible to get durability without a huge performance impact

with any filesystem you have atomic renames IF THE SYSTEM DOESN'T CRASH before the data is written out, that's what the POSIX standard provides.

ext3 gains it's 'atomic renames' as a side effect of a bug, it can't figure out what data belongs to what, so if it's trying to make sure something gets written out it must write out ALL pending data, no matter what the data is part of. That made it so that if you are journaling the rename, all the writes prior to that had to get written out first (making the rename 'safe'), but the side effect is that all other pending writes, anywhere in the filesystem also had to be written out, and that could cause 10's of seconds of delay.

for the casual user, you argue that this is "good enough", but for anyone who actually wants durability, not merely atomicity in the face of a crash has serious problems.

ext4 has a different enough design that they can order the rename after the write of the contents of THAT ONE file, so they can provide some added safety at relatively little cost

you also need to be aware that without the durability, you can still have corrupted files in ext3 after a crash, all it takes is any application that modifies a file in place, including just appending to the end of the file

Shared pain

Posted Feb 8, 2012 19:48 UTC (Wed) by Wol (guest, #4433) [Link]

Lets just say that governments (and businesses) have wasted billions throwing away applications where the application met the spec but in practice was unfit for purpose.

And a filesystem that throws away user data IS unfit for purpose. After all, what was the point of journalling? To improve boot times after a crash and get the system back into production quicker. If you need to do data integrity check on top of your filesystem check you've just made your reboot times far WORSE - a day or two would not be atypical after a crash!


Shared pain

Posted Feb 8, 2012 20:51 UTC (Wed) by raven667 (subscriber, #5198) [Link]

The hyperbole is getting a little out of control. Journaled filesystems have traditionally only journaled the metadata so any file data in-flight at the time of a crash would be lost and corruption would be the result. Pre-journaling any filesystem with a write cache would be susceptible to losing in-flight data and corrupting metadata leading to long fsck times after crash to repair the damage. All filesystems lose data in those circumstances, that doesn't mean that all filesystems are unfit for any purpose or that computers are fundamentally unfit for any purpose. The current state of the art is to be safer with regular data writes, even to the point of checksumming everything, that's nice but the world didn't end when this wasn't the case.

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds