Two more

Posted Mar 31, 2009 15:40 UTC (Tue) by sbergman27 (guest, #10767)
In reply to: Two more by clugstj
Parent article: From ext3 to ext4: An Interview with Theodore Ts'o (Linux Magazine)

"""
It is his competence that matters.
"""

There is competence, and there is judgment. And the two are distinct. I think that it is his judgment on this matter that is in question. I've been waiting for Linus to speak on the matter. I would be very interested in his view of this matter. Of course, the distros have the final say as to what are the effective defaults, even down to the patches they choose to apply. And *savvy* users have the ultimate decision as to the configuration of their systems. Unsavvy users, of course, are stuck with what they get.

Two more

Posted Mar 31, 2009 19:18 UTC (Tue) by sbergman27 (guest, #10767) [Link]

Apparently, Linus has spoken, and I missed it. And he does choose the word "incompetent":

========================
On Tue, 24 Mar 2009, Theodore Tso wrote:
>
> Try ext4, I think you'll like it. :-)
>
> Failing that, data=writeback for single-user machines is probably your
> best bet.

Isn't that the same fix? ext4 just defaults to the crappy "writeback"
behavior, which is insane.
Sure, it makes things _much_ smoother, since now the actual data is no
longer in the critical path for any journal writes, but anybody who thinks
that's a solution is just incompetent.

We might as well go back to ext2 then. If your data gets written out long
after the metadata hit the disk, you are going to hit all kinds of bad
issues if the machine ever goes down.

Linus

=======================

Where competence meets judgment

Posted Mar 31, 2009 19:18 UTC (Tue) by man_ls (guest, #15091) [Link] (5 responses)

I had the impression that Linus had already spoken against data loss, and he has indeed:

Sure, it makes things _much_ smoother, since now the actual data is no longer in the critical path for any journal writes, but anybody who thinks that's a solution is just incompetent.
We might as well go back to ext2 then. If your data gets written out long after the metadata hit the disk, you are going to hit all kinds of bad issues if the machine ever goes down.

Gods how I enjoyed that quote. And:

But I also think that the "we write meta-data synchronously, but then the actual data shows up at some random later time" is just crazy talk. That's simply insane. It _guarantees_ that there will be huge windows of times where data simply will be lost if something bad happens.
And expecting every app to do fsync() is also crazy talk, especially with the major filesystems _sucking_ so bad at it (it's actually a lot more realistic with ext2 than it is with ext3).
So look for a middle ground. Not this crazy militant "user apps must do fsync()" crap. Because that is simply not a realistic scenario.

And:

Doesn't at least ext4 default to the _insane_ model of "data is less important than meta-data, and it doesn't get journalled"?
And ext3 with "data=writeback" does the same, no?
Both of which are - as far as I can tell - total braindamage. At least with ext3 it's not the _default_ mode.

Linus is tha man.

Speed doesn't matter if you cannot trust it

Posted Mar 31, 2009 19:39 UTC (Tue) by oak (guest, #2786) [Link]

I have an insanely fast file system with a really simple design:
cat >/dev/null

If /dev/null writes aren't zero-copy, it's journaled too!

The window for data retrieval is (infinitely) small though.

Where competence meets judgment

Posted Mar 31, 2009 22:37 UTC (Tue) by bojan (subscriber, #14302) [Link] (3 responses)

> And expecting every app to do fsync() is also crazy talk, especially with the major filesystems _sucking_ so bad at it (it's actually a lot more realistic with ext2 than it is with ext3).

Major filesystems being "ext3 in ordered mode only", of course. The rest could be just fine with fsync(), as we can see above from his ext2 comment. And as Ted pointed out, ext4 doesn't have a big penalty on fsync(), because it doesn't have to flush out MBs of stuff that are unrelated to this particular fsync(), every time this system call is used.

Just as Linus says that ext4 is brain damaged for doing delayed allocation by default, so can it be claimed that is ext3 brain damaged for locking up people's machines for a few seconds on a perfectly reasonable system call: fsync(). We have seen this from the FF fiasco. In fact, when Linux says that having an interactive application do fsync() is impossible, he must mean on ext3 in ordered mode, because that's what FF complaints were about. As Alan Cox and Ted pointed out, one can already do fsync() in another thread and be fully interactive.

As for configuration files of KDE (which is where the problem started), the library can trivially do backup of these files on startup and _never_ use fsync() after that. Other problems should probably be solved by a proper system call that does guarantee ordering (I think Ted provisionally called it fbarrier() or something). Then we'd have a real guarantee of the behaviour, instead of relying on whims of implementations.

Claiming the rename() always did "data before metadata" commits is ahistorical. So, the crazy talk ain't that crazy after all. We just got caught we our pants down.

Surely, Linus is "tha man" when it comes to Linux and what he says will eventually go. But, removing any criticism from what he says is just arse licking, IMNSHO.

Where competence meets judgment

Posted Mar 31, 2009 22:43 UTC (Tue) by bojan (subscriber, #14302) [Link] (1 responses)

> when Linux says

Gee, he should have called it something else. It is impossible to get the man's name right after having "Linux" :-)

Where competence meets judgment

Posted Apr 12, 2009 7:59 UTC (Sun) by Duncan (guest, #6647) [Link]

You likely know this as I'm sure most Linux veterans do by now, and the
above "he should have called it something else" was simply a figure of
speech, but maybe the below will be new to the newbies at least.

Actually, "he" (Linus) did call it something else, "Freeix". It was
Linus' colleague that put it up on the ftp-site that put it in a
directory he named "linux", and so history was made.

(Just google freeix linux for more. "I'm feeling lucky" does it for me.)

Duncan

Judgments must take into accounts users

Posted Apr 2, 2009 12:01 UTC (Thu) by renox (guest, #23785) [Link]

Even the best fsync on earth can take a long time if there's a lot of data to be written on the disk, so fsync is always a 'potentially time consuming' operation.
Which means that whatever the FS if you must use fsync to have the correct behaviour then to avoid showing freeze to the user you must go to the dreaded multi-threaded world.
Sure the FS can provide a (Linux specific) write barrier, but it's very likely that nobody will use this.

OR the other possibility is to use a FS which does the operations in-order which simplify a lot the application programming.
There may be a small performance cost, somehow I doubt that users will care.