User: Password:
Subscribe / Log in / New account

POSIX v. reality: A position on O_PONIES

POSIX v. reality: A position on O_PONIES

Posted Sep 12, 2009 0:46 UTC (Sat) by spitzak (guest, #4593)
Parent article: POSIX v. reality: A position on O_PONIES

Yay! Finally some sense in talking about this. Please everybody understand what this article is saying, and if you disagree realize you are wrong.

I would like to go further and really extend POSIX to get the atomic operations everybody really wants. I would actually redefine the flags sent by creat() (O_CREAT|O_WRONLY|O_TRUNC) to be this "ponies" flag with the following rules:

1. If the file already exists, then other programs either see the old file or the contents of the new file when close() was called on it.

2. If the file does not already exist then other programs either see no such file or they see the file with the contents when close() was called on it.

3. If the program crashes or exits without calling close() then it is exactly like nothing ever happended.

3a. It may be useful to add some new call that "forgets" the file, ie it is "closed" in the same way as when the program exits.

This will avoid the need to use a temporary file and then rename it to get real atomic writes. And it would not use a new flag, because all programs using the creat() flags are already acting exactly like it works this way.

(Log in to post comments)

POSIX v. reality: A position on O_PONIES

Posted Sep 12, 2009 3:50 UTC (Sat) by cras (guest, #7000) [Link]

I don't think close() is the proper checkpoint here. I think it'll already break POSIX guarantees, and
it's quite likely that it'll break assumptions by some programs. Not all programs close() the file once
they're done with it. After all, there are less syscalls to do if you want to append a few lines to a file
you already once wrote.

POSIX v. reality: A position on O_PONIES

Posted Sep 14, 2009 18:20 UTC (Mon) by spitzak (guest, #4593) [Link]

I don't think POSIX guarantees *anything* until close() is called, so anybody relying on seeing what is there is relying on non-POSIX behavior.

That said, I'm fairly certain that any lseek() on the file can be a trigger that this behavior is not wanted and that the partial file should become visible at that point. Of course no guarantee until close() is called...

This has no effect on pipes. So at absolute worst, you can write a logfile writer, so instead of "foo > logfile" you write "foo | write-old-style logfile".

I very much believe the improvements to atomicity from this so vastly outweigh any incompatibility problems that they are irrelevant.

POSIX v. reality: A position on O_PONIES

Posted Sep 13, 2009 19:40 UTC (Sun) by njs (guest, #40338) [Link]

IIUC, with your suggestion, running 'long_running_process > logfile' would make it impossible to track progress in the logfile, because the logfile would not become visible until after long_running_process exited. And if your box crashed, then the logfile would be impossible to read, period. (Think of checking /var/log/Xorg.0.log to debug X crashes...)

POSIX v. reality: A position on O_PONIES

Posted Sep 14, 2009 18:15 UTC (Mon) by spitzak (guest, #4593) [Link]

Most such software does ">>" or O_APPEND or otherwise opens the file with different flags, so it would act normally. Otherwise you are correct that something would have to be done, I believe any call to lseek() could indicate that you want the previous behavior, and it would be easy to alter software to do this, or add some syntax to the shell to do this.

Also it is possible that the file will revert to previous behavior after a certain size is achieved.

I still feel the benifits outweigh any compatibility problems. By far the majority of programs writing files with creat() act as though it works exactly like I describe.

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds