SMP? [LWN.net]

SMP?

Posted Mar 23, 2012 13:02 UTC (Fri) by ScottMinster (subscriber, #67541)
Parent article: Toward better NUMA scheduling

I've noticed that when a single thread process runs on a Linux SMP system, it tends to be distributed to all the cores almost equally (based on looking at top output, etc). Perhaps naively, it seems to me that it would be better to run that thread on a single core, to take advantage of caches in that core rather than switching between cores.

So, my question is, would some of these changes, like the concept of a "home node" make sense in the normal SMP world?

to post comments

SMP?

Posted Mar 23, 2012 21:34 UTC (Fri) by zlynx (guest, #2285) [Link] (3 responses)

Hmm. That is counter to what I observe, which is that a high-cpu using process tends to stick to one core.

Now, a process that does a lot of sleeping on IO does shift around a lot. It doesn't much matter where it wakes up, after all.

SMP?

Posted Mar 24, 2012 4:29 UTC (Sat) by jzbiciak (guest, #5246) [Link]

Now, a process that does a lot of sleeping on IO does shift around a lot. It doesn't much matter where it wakes up, after all.

What about whatever working set it had in the caches? It may be sleeping on I/O, but if your number of running tasks stays below your number of CPUs, there's a good chance that a good fraction of that working set is still in the L2 of the last processor it ran on. So, it seems like you'd want to wake it on that same CPU.

Then again, with modern multiprocessors, that state might be in an L3 shared by multiple CPUs, even if it isn't in L2. (And, the trend seems to be to increase the size of L3 at the expense of the size of L2, or at least in lieu of increasing L2.) So then you want to wake up on a processor connected to the same L3...

SMP?

Posted Mar 24, 2012 12:21 UTC (Sat) by ScottMinster (subscriber, #67541) [Link] (1 responses)

You're right. I just verified with a simple CPU heavy application that it does tend to stay on a single core. I think I've noticed migration in the past with compilers, which tend to use more I/O.

SMP?

Posted Apr 1, 2012 21:38 UTC (Sun) by Wol (subscriber, #4433) [Link]

This was a well-known failure mode with - I think - NT4.

If you were running a single heavy-CPU process the scheduler would make it jump from CPU to CPU as it tried to balance the load ... :-)

Cheers,
Wol