Building a High-Performance Cluster with Gentoo

Posted Apr 11, 2007 7:14 UTC (Wed) by amacater (subscriber, #790) [Link]

The classic answer on the Beowulf list: It depends. It depends on whether
you admin. your own server or have to rely on central admin. It depends on
the size of your cluster and, more importantly, who your hardware vendors
are. If you buy a 2048 node cluster from IBM, to some exent it's easier to
take the hardware vendor's choice of distro and cluster admin tools. HP's
choice may be different from Penguin's. Two further considerations: fast
interconnect hardware (Quadrics/Mellanox ...) which is an essential for
some classes of problem needs drivers. The companies are relatively small
in terms of staff size and are operating on tight margins in a small
market. It may be that they haven't time to sort out a Debian/Gentoo/Yellow
Dog ... hardware card driver. Lastly, there's the high performance compiler
writers and high-end proprietary software types: they want to debug a known
kernel/memory combination when they get oops reports. You can run highly
successful infrastructures on whichever distribution you like - as ever,
your problem set, resources, time and effort will differ from everyone
else's and, sometimes, it's easier to buy a system off the shelf so that
your users can concentrate on coding and running jobs. Read the Beowulf
list archives for this discussion and minor variants - many times :)

Posted Apr 11, 2007 16:21 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]

useing the approach described above you don't have different systems running different versions of things (unless you want them to). with the binary package server you have one box compile the code with the optimizations that you want, and then it makes the results available to all the other systems (assuming that they are identical)

I haven't done head-to-head performance comparisons with gentoo, but I have seen cases where optimizing the kernel could result in 20-30% performance improvements in the past (back in the 1GHz athlon days). on modern 64 bit hardware it's less of an issue becouse there's less variability between hardware, and therefor less difference betwen optimized versions and the generic versions.

where I actually see the benifit of gentoo where I use it (my home server) is in the ability to configure the packages with the options and dependancies that I want them to have (this means turning on some that other distros would leave off, but mostly turning off options that other distros turn on, but I don't care about)

Posted Apr 19, 2007 12:35 UTC (Thu) by piggy (subscriber, #18693) [Link]

I would question the claim that a source-based distro necessarily sees a higher risk of encountering obscure and subtle bugs than a binary-based distro. Your reasoning is sound, but my empirical experience suggests that the reverse may be true.

My experience as a developer for a vendor of a binary-only commercial Unix clone demonstrates that the range of strange PC hardware out there is more than sufficient to exercise plenty of unique corner cases.

My other stint of experience comes from working for an embedded Linux vendor. We saw a LOT more trouble from people trying to piece together tiny distributions from prebuilt binaries (even all from the same source) than from people willing to build everything from source. A very common problem we saw with people who tried to do all of their system work with binaries only was subtle version dependencies among libraries as people upgraded individual packages over time. These problems simply do not occur if every library is built successively against the existing set of binaries on the system.