|
|
Subscribe / Log in / New account

Asynchronous commit

Asynchronous commit

Posted Mar 26, 2014 21:11 UTC (Wed) by zblaxell (subscriber, #26385)
Parent article: PostgreSQL pain points

I was kinda hoping to read something like: "There was a meeting of PostgreSQL and kernel developers. Linux is going to get asynchronous commit."

Suppose I am using the Unix write-close-rename atomic file update idiom, and I write five versions of a single file (A, B, C, D, and E) in rapid succession, and then there's a crash. I usually don't care which version of A-E I end up with, as long as I get some version intact (or, if version A created the file in the first place, no version at all). In PostgreSQL, I can ask for asynchronous commit, which does more or less exactly that.

Unfortunately there's no way (that I know of) to ask the kernel for this and still be fast. I seem to have three bad choices: call fsync() where I want a write barrier, which slows down both the process that is writing the file and every other process sharing a disk with the writing process; or don't call fsync(), which may under some circumstances give me a file that is empty or broken; or put clunky workarounds in application code (e.g. use a tmpfs for the file and copy the file to a persistent disk with rsync in a loop running at whatever speed the disk can keep up with, or put the filesystem in a database server that implements asynchronous commit).

Ideally the kernel would provide some mechanism that preserves write order around user-supplied barriers, but does not otherwise affect the scheduling of writes or the process doing the writing, and still allows the kernel to freely reorder or coalesce writes between the barriers.


to post comments

Asynchronous commit

Posted Mar 27, 2014 8:01 UTC (Thu) by marcH (subscriber, #57642) [Link] (1 responses)

You mean, like... a database?

Sorry could not resist :-)

Asynchronous commit

Posted Mar 27, 2014 16:36 UTC (Thu) by zblaxell (subscriber, #26385) [Link]

I thought I covered that as undesirable option #3 ("put the filesystem in a database server that implements asynchronous commit"). ;)

The RDBMS state of the art has progressed over the last 20 years, providing a wide variety of reasonable data integrity and performance tradeoffs. It would be nice if filesystems could catch up. fsync() is a sledgehammer.

Asynchronous commit

Posted Apr 4, 2014 10:49 UTC (Fri) by XTF (guest, #83255) [Link]

Kernel (FS) devs appear to be allergic to such a sound solution. I've requested it before (O_ATOMIC), they said it was too hard to do.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds