Better than POSIX? [LWN.net]

Better than POSIX?

Posted Mar 18, 2009 2:48 UTC (Wed) by zlynx (subscriber, #2285)
In reply to: Better than POSIX? by christian.convey
Parent article: Better than POSIX?

That's a very good point.

So no matter what ext4 does in this case, the programs with the problems are still not POSIX-
compliant.

They should either fsync, or adopt the other common pattern of renaming the target file to a
backup with the tilde before renaming the new file into place. Then have recovery code to pick up
the tilde file if there's a problem with the regular one.

(Log in to post comments)

Better than POSIX?

Posted Mar 18, 2009 4:11 UTC (Wed) by quotemstr (subscriber, #45331) [Link]

Huh? If a program is truly POSIX-compliant, it can't make any assumptions about what happens after a crash. That's undefined by POSIX. There's no sense in playing games with backup files because the backups are undefined by POSIX too!

Really, all a program that depends on nothing more than POSIX can do it take advantage of the atomic rename support on a running system and hope for the best on a crash. POSIX guarantees nothing else.

Clearly, this is not an acceptable state of affairs, so we make further assumptions about the exact behavior fsync and friends. But these assumptions go beyond POSIX.

Now, an ordered rename makes a whole lot more sense than a rename that only preserves contents after an fsync. But don't pretend that either alternative is mandated by POSIX. This whole damn problem has nothing to do with POSIX, so stop bringing it up. (And that means you too, bojan.)

Better than POSIX?

Posted Mar 18, 2009 5:26 UTC (Wed) by bojan (subscriber, #14302) [Link]

> And that means you too, bojan.

Sorry. I'll bring up whatever I see fit, whether you like that or not. But, by all means, don't listen to me and don't reply to my comments. Ignore me - that's OK. But at least do try to understand what Ted's saying.

> If a program is truly POSIX-compliant, it can't make any assumptions about what happens after a crash.

Not, it cannot make assumptions. It can make preparations as best it can (which are defined in the standard) to have data on disk. These preparations are called fsync(). Or, it can be smart and create little tiny backup files with fsync() beforehand and then be fast and keep renaming in the hope that the system doesn't crash all the time.

Either intentionally or otherwise, you keep misinterpreting what POSIX does or does not define. POSIX defines that fsync() is your best bet on having the data on disk. It doesn't define anything about rename() having ordered characteristics. It also doesn't define anything about the situation after the crash.

Out of this, you are giving people advice that the best thing to do is to go with undefined behaviour of rename() if you want your data on disk after the crash. The mind boggles...

Better than POSIX?

Posted Mar 18, 2009 14:22 UTC (Wed) by nlucas (subscriber, #33793) [Link]

Even if you go the POSIX way, many cheap disk drives lie to the system saying the data is on the disk plater when it's still on it's internal buffers. So even that can not guarantee you the operation is atomic.

An anecdotal case some years ago was Windows 98 corrupting the disk on "modern" PCs, because the hardware was so much faster than the disk flushing. When windows was shutdown it would kill the power before the disk finished buffering it's writes, corrupting the file system.

The only solution was to add some wait time before killing the power.