Flushing out pdflush

Posted Apr 4, 2009 20:59 UTC (Sat) by nix (subscriber, #2304)
In reply to: Flushing out pdflush by mjcoder
Parent article: Flushing out pdflush

Yeah, but how many new kernel threads do we end up with? We already have a
crazy number.

Flushing out pdflush

Posted Apr 5, 2009 0:09 UTC (Sun) by knobunc (guest, #4678) [Link] (8 responses)

I thought that too... but what's the downside to more threads?

Flushing out pdflush

Posted Apr 5, 2009 11:56 UTC (Sun) by nix (subscriber, #2304) [Link] (7 responses)

Inelegance? 12K unswappable kernel memory per thread?

A thread pool would surely be better (for something like this, anyway, in
which the full complement will only be needed in extreme situations).
There are several thread pools in the kernel (pdflush) or out-of-tree
patches (fs-cache's slow-work threads) already. It's quite doable.

Flushing out pdflush

Posted Apr 5, 2009 14:27 UTC (Sun) by i3839 (guest, #31386) [Link] (6 responses)

It seems that by default you will have one thread per block device, which seems totally reasonable, no matter how many block devices you have. Reducing this number to less than that will mean you will be able to write data out slower, because you can't keep all devices busy. Except if you switch to a non-blocking multiplexing thread doing all write-outs, but there's probably a good reason why that's not done. For fast devices it may be better to increase the number of threads, but again, not doing that will result in slower write-out throughput.

A thread pool would only be better if you would otherwise allocate too many threads per device. But if you allocated too many, there's nothing that prevents having a too high number of threads in the pool either, so it's just shuffling the problem around.

Flushing out pdflush

Posted Apr 5, 2009 18:27 UTC (Sun) by nix (subscriber, #2304) [Link] (5 responses)

Most block devices are never or very very rarely written to (e.g. one
containing only /usr is only going to be written to during a package
upgrade). Why devote a thread to it which will be idle nearly all the
time?

Flushing out pdflush

Posted Apr 6, 2009 11:44 UTC (Mon) by i3839 (guest, #31386) [Link] (4 responses)

The thread is only created when needed and exists after a time-out, IIRC.

Flushing out pdflush

Posted Apr 6, 2009 19:31 UTC (Mon) by nix (subscriber, #2304) [Link]

So it is, in effect, a thread pool. Excellent.

Flushing out pdflush

Posted Apr 17, 2009 12:41 UTC (Fri) by axboe (subscriber, #904) [Link] (2 responses)

Yes that is correct, threads are created on-demand and exit if they have been idle for some period of time.

It's mostly to satisfy the people getting annoyed when looking at the ps output. An idle kernel thread is essentially free. Of course if you have 10000 logical disks in your system, you'll probably appreciate not spending memory on the threads at least.

Flushing out pdflush

Posted Apr 17, 2009 22:29 UTC (Fri) by nix (subscriber, #2304) [Link] (1 responses)

Last I heard the set of PIDs was limited to 32767 by default, and having
huge numbers of even idle processes tended to slow down the PID allocator
horribly.

(Also, kernel threads still need a kernel stack: that's 8Kb of memory you
won't see again. Not much by modern standard perhaps...)

Flushing out pdflush

Posted Apr 17, 2009 23:43 UTC (Fri) by njs (subscriber, #40338) [Link]

The PID allocator supposedly got fixed a few years ago, around when NPTL landed. (Here's an interview with Ingo that confirms this: http://kerneltrap.org/node/517) And if the kernel needed thousands of threads for some reason, presumably it could tweak the kernel.pid_max sysctl itself...

But anyway, yeah, for ordinary systems the memory usage matters a little but not much.