Andrea Arcangeli's per-task-predictive-write-throttling [LWN.net]

Andrea Arcangeli's per-task-predictive-write-throttling

Posted Sep 29, 2005 10:34 UTC (Thu) by farnz (subscriber, #17727)
In reply to: replaced by new patch from .. by ncm
Parent article: Swap prefetching

I believe he's referring to Andrea Arcangeli's per-task-predictive-write-throttling.

If I've understood it properly, the patch measures the rate at which each task is writing to disk; if maintaining that rate would cause the kernel to start flushing buffers via pdflush in the near future, the task's timeslice is used to flush instead.

The result is that tasks doing the odd write here and there aren't affected, since they don't cause enough dirty pages. Tasks like cp, which dirty lots of pages, get paused to write out these pages (cleaning them), and making these pages eligible for eviction. This reduces the memory pressure cp-type tasks can induce without killing their performance; cp would eventually pause for the writeout, once it had dirtied as much RAM as it could. The patch just brings this pause forward, so that it doesn't dirty too much RAM before it writes out.

(Log in to post comments)

Andrea Arcangeli's per-task-predictive-write-throttling

Posted Sep 29, 2005 10:41 UTC (Thu) by nix (subscriber, #2304) [Link]

It's a good idea, that patch, definitely. For a pathological example, my backup process involves multi-hundred-Mb copies to a packet-written CD-RW. This generally floods almost all of memory with dirtied pages and then flushes them to this rather slow device, making the machine slow and swappy for a quarter-hour or so until the flush is done: with predictive write throttling, I'd expect to see a steady trickle instead, at something close to it.

(Even the current behaviour is much better than Linux-2.4, where the fact that block devices didn't have their own queues led to the entirety of X and everything I was doing freezing solid within a second or two of the flush beginning; the system was more swappy than normal because memory was filled with all those dirty pages, and a tiny and otherwise-unnoticeable bit of swapping or paging was stuck behind the vast heap of stuff destined for the CD-RW.)