PostgreSQL might benefit from a fd-list fsync() API

Posted Jan 19, 2021 14:51 UTC (Tue) by Wol (subscriber, #4433)
In reply to: PostgreSQL might benefit from a fd-list fsync() API by ringerc
Parent article: Fast commits for ext4

> Finally, for checkpoints PostgreSQL must fsync() all the dirty files it has open. This will force all pending writes on those files to disk, not just the writes made before the current checkpoint target position. On most Linux FSes it also flushes a bunch of other irrelevant and uninteresting files that don't need to be durable at all.

Does PG tend to have a lot of these (dirty files)? Because Pick, the database I'm thinking of, typically stores a single "table" in one OR MORE os files, so I could have a *lot* of dirty files open ...

Although of course, like PG, the main thing I'm concerned about is knowing that the journal is flushed ...

Cheers,
Wol

PostgreSQL might benefit from a fd-list fsync() API

Posted Jan 22, 2021 1:15 UTC (Fri) by ringerc (subscriber, #3071) [Link]

Yes. PostgreSQL may have many dirty FDs. Each tables or index is stored as separate file, and split into 1GB extents. I imagine that the extent splitting could be changed if there's a benefit to doing so, but the separate files per relation not so much.