Wishful thinking [LWN.net]

Wishful thinking

Posted Mar 16, 2009 22:26 UTC (Mon) by dlang (guest, #313)
In reply to: Wishful thinking by quotemstr
Parent article: Garrett: ext4, application expectations and power management

the problem is that making the write barrier be part of the rename requires changes to all filesystems on all operating systems, and applications are only safe if they are running on a new enough version of an OS.

doing the fsync before rename works on all filesystems on all operating systems, but requires changing the applications.

if the applications push this into the OS/filesystem they will need to document that they are unsafe on any but (...) which is a list that will change over time without any control (and probably without the knowledge) of the application developers. but if they put in the fsync they work with everything that's on the market today.

they don't have to do the fsync of the directory if they don't care which version of the file exists after the crash, just doing it for the data is enough.

to post comments

Wishful thinking

Posted Mar 16, 2009 22:35 UTC (Mon) by quotemstr (subscriber, #45331) [Link] (2 responses)

the problem is that making the write barrier be part of the rename requires changes to all filesystems on all operating systems, and applications are only safe if they are running on a new enough version of an OS.

True enough, but applications work without fsync already, so upgrades will be required in either case. But there are far fewer filesystems than applications, so from a purely logistic point of view, it makes more sense to change the filesystem.

if the applications push this into the OS/filesystem they will need to document that they are unsafe on any but (...) which is a list that will change over time without any control (and probably without the knowledge) of the application developers. but if they put in the fsync they work with everything that's on the market today.

There are plenty of applications that don't include fsync. Making this change turns existing incorrect applications into correct ones, and doesn't harm any application that already uses fsync. Besides -- how many of these applications are portable to other systems anyway? A certain degree of unportability is required to drive change. Sometimes you don't need to worry about supporting every POSIX-compliant system and can take advantage of better functionality.

they don't have to do the fsync of the directory if they don't care which version of the file exists after the crash, just doing it for the data is enough.

...which causes performance problems. In fact, Theodore Tso recommends running fsync in a separate thread, which drastically increase the complexity of simple applications. Applications are left to choose one of incorrect, slow, or complicated. They shouldn't be required to make that choice.

Not to mention the problems rampant fsync would cause for laptop_mode.

Wishful thinking

Posted Mar 16, 2009 22:39 UTC (Mon) by dlang (guest, #313) [Link] (1 responses)

no, applications _sometimes_ or _mostly_ work without fsync on some OS/filesystem/mount option combinations

that's far from what you are asserting "applications work without fsync already"

Wishful thinking

Posted Mar 16, 2009 22:44 UTC (Mon) by quotemstr (subscriber, #45331) [Link]

You can add those qualifiers to almost any statement. The fact is that these applications are written to assume rename inserts a write barrier, and the vast majority of the time, these applications get what they want. With pre-patch ext4 and XFS, this assumption breaks. The breakage was noticed very quickly after ext4 entered wide use. If these applications' assumptions were so unreliable, we would have been an uproar well before ext4 was released.