A potential NUMA scheduling solution
Posted Nov 3, 2012 2:41 UTC (Sat) by Trelane (subscriber, #56877)
Posted Nov 3, 2012 2:43 UTC (Sat) by Trelane (subscriber, #56877)
Posted Nov 4, 2012 23:56 UTC (Sun) by marcH (subscriber, #57642)
Thanks for correcting me!
Posted Nov 15, 2012 18:24 UTC (Thu) by slamb (guest, #1070)
I don't know about need, but anyone who needs a lot of computing power might prefer N M-package machines to N*M 1-package machines for efficiency's sake.
At Google, we certainly do. I won't reveal anything new, so take a look at this public talk (by Jeff Dean and Sanjay Ghemawat, always interesting people to follow). Multiple things per machine; their thing alone uses "144 machines, ~2300 cores", or 16 cores/machine.
This sort of configuration makes sense if you have to bin-pack applications with different needs, like a memory-heavy memcached, a CPU-heavy application server, and a disk/SSD-heavy filesystem daemon. Or if you have processes with a lot of fixed overhead to run - better to get as much out of each one as possible. Or if the N M-package machines use less power than the N*M 1-package machines.
Posted Nov 15, 2012 20:45 UTC (Thu) by dlang (subscriber, #313)
Using a bunch of 1-package machines imposes a much higher communication cost than multi-package machines.
There's also a question of the features provided on the motherboards. There are very few 1-package motherboards out there that have 'server' features like ECC memory, serial or network consoles, etc. If any of these features are desirable to you, you will probably be using 2-package motherboards at the smallest.
In terms of cost for computing power, 2-package machines seem to still be the sweet spot, you avoid having to pay for multiple power supplies, drives, and other per-system infrastructure, but you don't have the price premium that larger multi-package systems tend to have.
But even a 2-package machine can see advantages with proper NUMA handling (not as drastic as larger systems, but there is still an advantage)
In addition, a large amount of workload on large systems nowdays is virtual machine based with the hardware systems being oversubscribed. While a VM can be moved from one machine to another, it can be a fairly expensive thing to do, so larger systems work better in practice as they allow for the peaks and valleys of demand to average out better, so you can oversubscribe your hardware more.
Posted Nov 15, 2012 23:12 UTC (Thu) by intgr (subscriber, #39733)
Even most desktop motherboards support ECC these days and have serial port pin headers at the very least. All Intel server motherboards (and probably HP and IBM too) come with BMC and an optional IP KVM addon.
Posted Nov 15, 2012 23:39 UTC (Thu) by dlang (subscriber, #313)
how many of these Intel server motherboards are single socket?
I've purchased single-package systems from HP, IBM, Sun and other tier-1 vendors. In every case the system that I received had a multi-package motherboard with only one package installed in it.
In that situation, the most cost-effective way to get compute capability is to go ahead and fully populate the motherboard. Even if that (usually second) package isn't as efficient as it would have been in a separate system (with no contention for RAM bandwidth, etc) it's still the best thing to do because the marginal cost of adding it to an existing system is so small.
Posted Nov 16, 2012 1:26 UTC (Fri) by intgr (subscriber, #39733)
Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds