| |||||||||||||
The Linux Storage and Filesystem Summit, day 1The Linux Storage and Filesystem Summit, day 1Posted Aug 9, 2010 9:31 UTC (Mon) by neilbrown (subscriber, #359)In reply to: The Linux Storage and Filesystem Summit, day 1 by james_ Parent article: The 2010 Linux Storage and Filesystem Summit, day 1
I completely agree - the case where I have seen the need for a low ratio was also when the writes were going out via NFS. The machine had, I think, 32Gig of RAM, so 40% was a lot. Even 1%, the smallest non-zero setting, took longer to flush than we really wanted.
Problems with out-of-order writes is an interesting twist on that!
(Log in to post comments)
The Linux Storage and Filesystem Summit, day 1 Posted Aug 15, 2010 17:52 UTC (Sun) by kleptog (subscriber, #1183) [Link]
I have a another case where the default settings don't work well. A process that is somewhat realtime produces 30MB/s of data writing to disk. Under the default settings the kernel will wait 30 seconds before writing anything (900MB) and if it doesn't get it out in a reasonable way that the process gets stuck because it used up all of the 20% of memory for its data.
The solution is to have the kernel check much more often the amount of data waiting (every second rather than every 5 seconds) and drastically reduce the amount of dirty memory there's allowed to be before write back happens.
Without this the kernel suddenly realises it has more than a gigabyte of data to writeback (20% of 8GB = 1.6GB) and manages to starve other processes trying to get it out. Whereas if it just writebacks small amounts in the background continuously everything goes smoothly. 1% works well, since that's what the storage subsystem can handle quickly.
Pity it's a global setting though, other processes would probably work better with a higher writeback threshold, but you can't pick and choose.
The Linux Storage and Filesystem Summit, day 1 Posted Aug 19, 2010 13:04 UTC (Thu) by cypherpunks (guest, #1288) [Link]
I can't help thinking that the solution looks a lot like a PID controller. That is, page write speed is defined by the sum of three terms: one proportional to the amount of excess dirty pages, one proportional to the integral, and one proportional to the derivative.
The latter is the "feed-forward" term, and helps respond quickly to sudden changes. If the rate of page dirtying increases sharply, the rate of writeback should likewise take a sudden jump.
|
|||||||||||||