| |||||||||||||
Some approaches to parallelism avoidanceSome approaches to parallelism avoidancePosted Nov 19, 2009 17:23 UTC (Thu) by martinfick (subscriber, #4455)Parent article: Some approaches to parallelism avoidance
I would think that a slight tweak to this idea might also be desired: a "run only one of these processes per CPU at a time" flag. This way, I could start my make with a -j <infinity> and avoid useless cache flushing behavior. New processes would only run when another process is actually waiting for IO.
(Log in to post comments)
Some approaches to parallelism avoidance Posted Nov 23, 2009 10:42 UTC (Mon) by nye (guest, #51576) [Link]
I think the idea (or one of the variants proposed) was to have a 'maximum parallelism' knob, that you could set to the number of cores/CPUs in order to get this behaviour.
This sounds very much like how goroutines were described, which is interesting.
Some approaches to parallelism avoidance Posted Dec 2, 2009 18:58 UTC (Wed) by HIGHGuY (guest, #62277) [Link]
The original idea is geared towards threadpools. If a user enqueues work which blocks or might block (e.g. lock a mutex, send data on a socket, ...)
then the threadpool's efficiency is temporarily decreased.
One way to solve would be to overallocate threads, but that only brings unnecessary context switching. Another would be to require the user to use async I/O but this tends to be rather complex and impacts the threadpool user's code instead of the library itself.
The prototype implementation, as I saw later on, actually resembles what Tejun Heo is doing with his workqueue implementation. When one thread of the workqueue blocks, another one is woken to resume work. The big difference, of course, being that workqueues are kernel-only, while userspace might definitely benefit from a similar approach.
At this moment I'm geared towards implementing Ingo's solution as that would not only benefit my use-case, but could also be used as a true perf-counter to measure how well a given workload is using the available CPU-power. It's also quite flexible.
|
|||||||||||||