Luu: Files are hard

I think the solution is pretty simple (at least in concept) and steals the database transaction pattern:

begin_transaction(fd, flag)

(critical IO operations)

commit_transaction(fd)

while the names are the same as DB transactions, the feature wouldn't necessarily have any sort of rollback functionality, although that would be cool. Additionally, "commit" means more of a "I'm done with the transaction, go back to normal" rather than an "only now put all the data on disk." Begin_transaction() does two things. 1) IO reordering is disabled through all storage layers for blocks associated with that FD.* 2) Every IO syscall in the transaction for that FD automatically has fsync() called. Commit_transaction just turns it off -- the data have been hitting the disk as normal, though without reordering, the whole time.

Through a feature flag argument on begin, and you could allow tuning how each subfeature works.

The single biggest issue, in my opinion, from the article and comments is the unpredictable ordering of IO operations, and the only solution is to utilize synchronization methods (O_SYNC, fsync, custom in user space with O_DIRECT/AIO, etc.) that have serious downsides. You know, sometimes I want the Atomicity and Isolation in ACID but don't care so much about Durability. The current kernel features force you to do both -- hurting performance -- and even then, you'll probably still have subtle bugs.

* Much easier said than done.

> With that in place, one could argue about whether it's best done in the kernel or in user space.

At least in my idea, the principal concern is allowing the user to definitely prevent IO reordering on specified FDs. That's impossible to do in user space -- at least without going down the undesired existing-synchronization paths.

Or, hell, perhaps just go full-bore:

begin_acid(fd, flag)

(Critical region)

end_acid(fd)

where flag is simply a 5-bit flag corresponding to enabling each property in the acronym. (Again, no rollback necessarily, so atomicity is only at the individual syscall level within that block, and there is no larger idea of an atomic transaction. Of course, modern, COW file systems can simply reflink at begin_acid and do some FD magic if there is an error if they want to support developer-friendly features.)