User: Password:
|
|
Subscribe / Log in / New account

Dirty page caching not mostly harmless

Dirty page caching not mostly harmless

Posted Mar 27, 2014 8:07 UTC (Thu) by marcH (subscriber, #57642)
In reply to: Dirty page caching not mostly harmless by dlang
Parent article: PostgreSQL pain points

> > I think the default is 20% or so

> I beleive it used to be 20% but is now 5% or so (or based on RAM size)

Is this [disk] bufferbloat here again? If yes there was a discussion about it on this site recently.

Without getting into detail and duplicating existing discussions (thanks for pointers) should the ideal solution be based on time just like for network bufferbloat, as opposed to RAM size now?


(Log in to post comments)

Dirty page caching not mostly harmless

Posted Mar 27, 2014 8:20 UTC (Thu) by dlang (subscriber, #313) [Link]

probably, there has been a lot of talk over the last couple of years that the disk I/O needs to resemble network I/O a lot more. It used to be that the spinning rust was so slow compared to the rest of the system that is was worth just about any computational effort to optimize it. But with large arrays and SSDs becoming much more common, 'disk' I/O is having trouble keeping up.

So a full time and queue based approach similar to what's happening in the network space may end up being a good fit. The problem is how to get there from here (especially safely)

Dirty page caching not mostly harmless

Posted Mar 27, 2014 16:25 UTC (Thu) by zblaxell (subscriber, #26385) [Link]

> It used to be that the spinning rust was so slow compared to the rest of the system that it was worth just about any computational effort to optimize it

Computational effort is one thing. RAM cost is another, and it frequently dominates.

Linux has worse problems with slow devices than with fast ones. SSDs are raising the high end, but at the same time enormous USB sticks and slow "green" rust drives are lowering the low end. It's annoying to have your fast SSD be idle 10% of the time because the filesystem can't keep up with it, but it's much more annoying to have most of your RAM be unusable for hours because someone plugged the slowest USB stick money can buy into your server, then tried to copy a few dozen GB of data onto it.

The solution is the same at both extremes--limit the number of dirty pages and stop delaying writes so much. It's the spinning rust in the middle of the range that still needs big dirty page caches and big write delays to form them--and even then, there are pretty low limits to the useful data size for optimization. Fast SSDs don't need optimization and slow "green" drives are so slow that the first page write alone will provide plenty of time for an optimizable set of dirty pages to accumulate.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds