User: Password:
Subscribe / Log in / New account

Dirty pages, faster writing and fsync

Dirty pages, faster writing and fsync

Posted Mar 26, 2014 20:08 UTC (Wed) by jhhaller (subscriber, #56103)
In reply to: Dirty pages, faster writing and fsync by roblucid
Parent article: PostgreSQL pain points

I agree that coalescing writes have advantages, but when a single fd has written 2-3GB to the buffer cache and it's only memory pressure forcing them to be written, that we are long past where the write coalescing benefit is useful. In the case of PostgreSQL, the pages should be written relatively quickly, as fsync is coming, and there is no point waiting for it. Linux doesn't behave well during either an explicit fsync or an implicit one caused by memory pressure. Another example, mythtv, calls fsync once per second while recording TV just to avoid a huge file operations delayed when memory pressure causes an implicit sync operation.

Having too many contiguous blocks written at once is another problem, as the drive will be busy writing all those blocks for as long as it takes. Once the write exceeds the amount in one cylinder (not that we can really tell the geometry), it has to seek anyway, and might as well let something else use the disk. Otherwise we have the storage equivalent of bufferbloat, where high priority writes get backed up behind a huge transfer.

(Log in to post comments)

Dirty pages, faster writing and fsync

Posted Mar 27, 2014 10:45 UTC (Thu) by dgm (subscriber, #49227) [Link]

As with bufferbloat, one approach could be to measure the buffer cache in terms of time it takes to write it back, instead of bytes and megabytes.

As others have suggested, having multiple queues mapped somehow to ionice levels, could be of help too.

Copyright © 2018, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds