Ext4 to be standard for Fedora 11, Btrfs also included (heise online) [LWN.net]

Ext4 to be standard for Fedora 11, Btrfs also included (heise online)

Posted Jan 23, 2009 23:10 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]

which filesystem are you considering unsafe on commodity hardware?

if you are thinking that XFS requires systems with special power supplies, that is a myth, and that bog that was fixed in may 2007 (linked to elsewhere in this topic) was the fix for it.

if anything, the fact that ext3 ignores write barriers by default while XFS honors them would make me say that XFS is safer on commodity hardware :-)

Write barriers

Posted Jan 24, 2009 1:49 UTC (Sat) by i3839 (guest, #31386) [Link]

> if anything, the fact that ext3 ignores write barriers by
> default while XFS honors them would make me say that XFS
> is safer on commodity hardware :-)

Depends. If it ignores write barriers because it is reliable without them in some other way, then it's not less safe. Similarly, if some drives ignore write barriers, then not counting on them in the fs is more reliable as well.

Write barriers

Posted Jan 24, 2009 2:21 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]

in this case it is known that ext3 ignoring barriers puts data at risk, but the performance overhead of turning them on by default (something like 30%) was deemed 'too expensive' by one person, so they remain off

I think you will find arguments where Ted Tso is arguing that they need to be on for data safety and Andrew says that it's unacceptable to introduce the performance regression that would result from doing so.

Write barriers

Posted Jan 26, 2009 20:29 UTC (Mon) by zlynx (subscriber, #2285) [Link]

Ext3 generally works without barriers because hard drive write caches usually write the journal first for whatever reasons. From what I recall reading, because the journal is all in one chunk at the beginning of the drive far away from the data, and that write caches usually flush in-order from lowest block to highest.

Note all the generally and usually qualifiers in there.

I recall reading about Ext3 journal corruption issues when the power fails as the journal is wrapping around, because the drive writes the end of the journal, some data, then back to the beginning of the journal instead of writing end of the journal, start of the journal, some data.

Write barriers

Posted Jan 29, 2009 16:47 UTC (Thu) by anton (guest, #25547) [Link]

Who knows what drives do or don't do when in write-caching mode? The last time I tested this 10 years ago the drives I used apparently wrote the blocks in-order, except when a block was overwritten; then the data might never reach the platter (if it's overwritten again and again). I should do some experiments with my new drives.

Also, why should writing to the journal first be safer than writing the other stuff first? With meta-data journaling as typically used in ext3 this would lead to typical meta-data update bugs.

Actually the data=ordered mode writes the data before the corresponding journal entries to prevent that from happening. So if the disks then write the journal entries before the data thanks to write-caching disks without barriers, that subverts the safety data=ordered provides. If the disks write stuff out-of-order in other ways, other failure modes are possible.

I guess that the reason we don't see lots of reports of data destroyed by ext3 without barriers is that most people don't have lots of write traffic going on (but then they won't notice any performance penalty from barriers either), and you expect to lose the file you write when the system dies anyway.

Ext4 to be standard for Fedora 11, Btrfs also included (heise online)

Posted Jan 25, 2009 19:29 UTC (Sun) by Nelson (subscriber, #21712) [Link]

I think the data safety claims are spurious at best, XFS is a serious filesystem with serious developers, made by a serious company. There simply aren't semantics for UNIX defined for what is supposed to happen when you lose power mid-write. My experience, they are both par. All the stories seem to be anecdotal, and the fact of the matter is whatever filesystem you're running when you lose data is going to become a defacto POS in your mind after that.

The big difference between the ext(x) family and XFS and JFS is ext(x) seem to be supported, really well. The others are still kind of bolt-ons to various distributions. I ran an all XFS Mandrake/Mandriva and Fedora systems for years, you'd be shocked how many times you'd' install a kernel upgrade and have a non-bootable system. I got to the point I'd never even consider doing a full distribution upgrade. The situation might be better now but for a very long time it looked like just a handful of kernel devs so much as cared about XFS, let alone enhanced it and kept it fresh with the rest of the kernel.

Ext4 to be standard for Fedora 11, Btrfs also included (heise online)

Posted Jan 25, 2009 20:35 UTC (Sun) by cantsin (guest, #4420) [Link]

What doesn't seem to be anecdotal is that XFS caches data more aggressively and over longer time in RAM than other filesystems including ext3. This results in performance advantages, but reliability issues on any other but high quality server hardware with ECC RAM. The anecdotal evidence mostly comes from users running PCs with standard (cheap) RAM and in non-data center-/USV-usage scenarios where loss of power is more likely or even common (such as on laptops).

Ext4 to be standard for Fedora 11, Btrfs also included (heise online)

Posted Jan 25, 2009 20:39 UTC (Sun) by jengelh (subscriber, #33263) [Link]

I have to object - laptops have statistically more batteries than desktop computers :-)

Ext4 to be standard for Fedora 11, Btrfs also included (heise online)

Posted Jan 26, 2009 2:25 UTC (Mon) by salimma (subscriber, #34460) [Link]

But, in countries with relatively stable electricity supplies, laptops are more likely than desktops to be uncleanly shut down because the battery ran out.

Ext4 to be standard for Fedora 11, Btrfs also included (heise online)

Posted Jan 25, 2009 23:56 UTC (Sun) by nix (subscriber, #2304) [Link]

UNIX doesn't define semantics for what happens when you lose power
mid-write. That means that what a filesystem does when that happens is a
quality-of-implementation issue: but given that power supplies are not
perfectly reliable it's a fairly important one, especially in countries
like the USA with a third-world electricity distribution network (as a UK
resident I'm privileged to have a fairly reliable power supply and the
most badass electrical regulations on earth: it's one of the few things
the UK's done right infrastructure-wise).

However, I'm afraid I consider 'files you were writing right now might be
chewed up' to be a *lot* better than 'files you were writing right now
might be chewed up, oh, and so might anything else on your disk, even if
you haven't touched it for months'.

I too saw this behaviour when I used XFS (years ago now): I've never seen
it using ext2 or ext3, even when I had multiple consecutive power cuts
during a lightning storm, even when I had bad RAM on the machine and was
doing massive cross-filesystem renames. Of course this too is 'anecdotal':
all reports from third parties are necessarily anecdotal. I don't see how
this makes them invalid (if they come from multiple sources).

Ext4 to be standard for Fedora 11, Btrfs also included (heise online)

Posted Jan 26, 2009 23:44 UTC (Mon) by lmb (subscriber, #39048) [Link]

Unix/POSIX doesn't define what happens to data in when you crash "mid-write", but it has one hell of an opinion on writes that have been committed and where the write()/fsync()/... calls have returned. So the durable state of the data is quite defined, unless the application is too dumb to make proper use of those features. (And, of course, unless the fs bites the dust.)