User: Password:
|
|
Subscribe / Log in / New account

Flushing out pdflush

Flushing out pdflush

Posted Apr 4, 2009 20:17 UTC (Sat) by mjcoder (guest, #54432)
Parent article: Flushing out pdflush

> With a ten-disk btrfs filesystem, per-BDI flushing performed 25% faster

Wow, this is a massive improvement!


(Log in to post comments)

Flushing out pdflush

Posted Apr 4, 2009 20:59 UTC (Sat) by nix (subscriber, #2304) [Link]

Yeah, but how many new kernel threads do we end up with? We already have a
crazy number.

Flushing out pdflush

Posted Apr 5, 2009 0:09 UTC (Sun) by knobunc (subscriber, #4678) [Link]

I thought that too... but what's the downside to more threads?

Flushing out pdflush

Posted Apr 5, 2009 11:56 UTC (Sun) by nix (subscriber, #2304) [Link]

Inelegance? 12K unswappable kernel memory per thread?

A thread pool would surely be better (for something like this, anyway, in
which the full complement will only be needed in extreme situations).
There are several thread pools in the kernel (pdflush) or out-of-tree
patches (fs-cache's slow-work threads) already. It's quite doable.

Flushing out pdflush

Posted Apr 5, 2009 14:27 UTC (Sun) by i3839 (guest, #31386) [Link]

It seems that by default you will have one thread per block device, which seems totally reasonable, no matter how many block devices you have. Reducing this number to less than that will mean you will be able to write data out slower, because you can't keep all devices busy. Except if you switch to a non-blocking multiplexing thread doing all write-outs, but there's probably a good reason why that's not done. For fast devices it may be better to increase the number of threads, but again, not doing that will result in slower write-out throughput.

A thread pool would only be better if you would otherwise allocate too many threads per device. But if you allocated too many, there's nothing that prevents having a too high number of threads in the pool either, so it's just shuffling the problem around.

Flushing out pdflush

Posted Apr 5, 2009 18:27 UTC (Sun) by nix (subscriber, #2304) [Link]

Most block devices are never or very very rarely written to (e.g. one
containing only /usr is only going to be written to during a package
upgrade). Why devote a thread to it which will be idle nearly all the
time?

Flushing out pdflush

Posted Apr 6, 2009 11:44 UTC (Mon) by i3839 (guest, #31386) [Link]

The thread is only created when needed and exists after a time-out, IIRC.

Flushing out pdflush

Posted Apr 6, 2009 19:31 UTC (Mon) by nix (subscriber, #2304) [Link]

So it is, in effect, a thread pool. Excellent.

Flushing out pdflush

Posted Apr 17, 2009 12:41 UTC (Fri) by axboe (subscriber, #904) [Link]

Yes that is correct, threads are created on-demand and exit if they have been idle for some period of time.

It's mostly to satisfy the people getting annoyed when looking at the ps output. An idle kernel thread is essentially free. Of course if you have 10000 logical disks in your system, you'll probably appreciate not spending memory on the threads at least.

Flushing out pdflush

Posted Apr 17, 2009 22:29 UTC (Fri) by nix (subscriber, #2304) [Link]

Last I heard the set of PIDs was limited to 32767 by default, and having
huge numbers of even idle processes tended to slow down the PID allocator
horribly.

(Also, kernel threads still need a kernel stack: that's 8Kb of memory you
won't see again. Not much by modern standard perhaps...)

Flushing out pdflush

Posted Apr 17, 2009 23:43 UTC (Fri) by njs (guest, #40338) [Link]

The PID allocator supposedly got fixed a few years ago, around when NPTL landed. (Here's an interview with Ingo that confirms this: http://kerneltrap.org/node/517) And if the kernel needed thousands of threads for some reason, presumably it could tweak the kernel.pid_max sysctl itself...

But anyway, yeah, for ordinary systems the memory usage matters a little but not much.

Flushing out pdflush

Posted Apr 7, 2009 20:08 UTC (Tue) by jzbiciak (subscriber, #5246) [Link]

Hmmm...
At a given point of time, there are between two and eight pdflush threads running in the system.
vs.
With a ten-disk btrfs filesystem, per-BDI flushing performed 25% faster
Could a noticeable part of the 25% boost be attributed to a 25% boost in number of flushing threads? A 10-disk btrfs filesystem ought to be generating traffic on all 10 spindles, right?

Flushing out pdflush

Posted Apr 17, 2009 12:39 UTC (Fri) by axboe (subscriber, #904) [Link]

Actually, no. With btrfs, currently it assigns a per-fs backing device to each inode. So for that particular case, you have just a single bdi flusher thread running even for the 10 disks.

Flushing out pdflush

Posted Feb 10, 2011 8:25 UTC (Thu) by bergwolf (subscriber, #55931) [Link]

Then why does per-bdi flusher improves that much performance for btrfs? If the key idea is *per-bdi*, there is little difference for btrfs. But where does the performance come from?


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds