How will Linux be leveraged in next-gen supercomputers? (NewsForge) [LWN.net]

How will Linux be leveraged in next-gen supercomputers? (NewsForge)

Posted Sep 27, 2005 4:37 UTC (Tue) by drag (subscriber, #31333)
In reply to: How will Linux be leveraged in next-gen supercomputers? (NewsForge) by dlang
Parent article: How will Linux be leveraged in next-gen supercomputers? (NewsForge)

I think the kernel itself (at least 2.6 series) scales a bit better then was given credit for when one of you said it could use up to 32 cpus.

HP with it's 'BigTux' project showed that a out-of-the-box kernel is able to scale perfectly well (with benchmarks and such) in a 64-way NUMA architecture. I beleive they used a Suse smp kernel for testing. And a unique thing it can do is do it dynamicly. That is when they plug in more cpus to a system the kernel is able to automaticly detect them and set up memory management so that it's all optimized for regular numa-style operation.

This is something special, I think. Propriatory unix kernels would have to be modified code-wise or have to be reconfigured at least to be able to run efficiently in a single cpu or dual cpu enviroment vs a 64 cpu enviroment. At least that's the impression that I get.

Also don't forget that SGI has it's Linux supercomputers that are able to use up to 2048 cpus in a single machine. Of course this isn't the vanilla kernel.

(Log in to post comments)

How will Linux be leveraged in next-gen supercomputers? (NewsForge)

Posted Sep 27, 2005 5:11 UTC (Tue) by dlang (subscriber, #313) [Link]

I did say at least 32 CPU's :-)

the post a week or so ago about the 512 CPU machine was useing something pretty close to a stock kernel and they were still at ~50% scaleing per cpu.

I've also seen posts with IBM 64 CPU machines running very nicely with stock kernels.

in any case, even with multi-core CPU's the limit is still comfortably ahead of what's easy to get hold of.

and with the steady trickle of fixes for the bottlenecks that they do run into on the 512 CPU machines it's just getting better.

when you go from a 1 CPU machine to a SMP machine you do need to recompile the kernel, and when you move from a SMP setup to a NUMA setup (useually around 16-32 CPU's) there are a bunch of new things that have to be set right for the exact hardware (to have the memory management work optimily, although there is a lot of work being done to autodetect the nessasary settings), but there's a very broad range in there from 1.5 (HT) to ~64 cpu's where things just work

How will Linux be leveraged in next-gen supercomputers? (NewsForge)

Posted Sep 27, 2005 8:21 UTC (Tue) by drag (subscriber, #31333) [Link]

Cool.

Linux rocks. :P

Pretty soon we will be all running Numa kernels anyways.

Take a dual socket AMD64 board, which is nice enough to have a seperate memory bank for each memory controller, and then stick 2 dual core cpus on it. Instant complex numa machine. 4 cores.. 2 cores share one level of cache and main memory on one half, the other 2 on the other with the ability to access each other's stuff in a pinch.

Fun stuff.

How will Linux be leveraged in next-gen supercomputers? (NewsForge)

Posted Sep 27, 2005 9:02 UTC (Tue) by joib (guest, #8541) [Link]

Actually, the dual core AMD64:s don't share cache. Each core has a separate 1 MB (or 512 on some of the lower end desktop models) L2 cache. Although AFAIK they are able to copy stuff directly between caches without having to go via main memory.