LWN: Comments on "Gentoo Linux 2006.1 released"

Wow, some misinformation never dies.

nix — Sun, 03 Sep 2006 14:40:22 +0000

The `more conservative' recommendation is both faster and smaller on
register-poor platforms like x86, and to some extent on x86_64 as well.

Wow, some misinformation never dies.

sholdowa — Thu, 31 Aug 2006 21:13:06 +0000

Err, no you're pretty much completely wrong there. I see what you're saying, and build my internet facing servers from source for exactly that reason.

My point was that the OP made a sweeping statement that IMHO was almost totally untrue. A package management system is a package management system, is a package management system and the only thing that really makes gentoo's different is the ability/expectation to build from scratch. They're all as good as the data contained within them, and because that's put there by people, they all make mistakes at some time or other.

To address your point, debian allows me to optionally install the following modules for apache 2.0. I could have just as easily used php for an example.

libapache2-mod-annodex - Provides server-side support for Annodex media
libapache2-mod-apreq2 - generic Apache request library - Apache module
libapache2-mod-auth-kerb - apache2 module for Kerberos authentication
libapache2-mod-auth-mysql - Apache 2 module for MySQL authentication
libapache2-mod-auth-pam - module for Apache2 which authenticate using PAM
libapache2-mod-auth-pgsql - Module for Apache2 which provides pgsql authentication
libapache2-mod-auth-plain - Module for Apache2 which provides plaintext authentication
libapache2-mod-auth-sys-group - Module for Apache2 which checks user against system group
libapache2-mod-bt - BitTorrent tracker for the Apache2 web server
libapache2-mod-bt-dev - Header files for mod_bt
libapache2-mod-cband - An Apache 2 module for bandwidth limiting the webserver
libapache2-mod-chroot - run Apache in a secure chroot environment
libapache2-mod-dnssd - Apache 2 module which adds Zeroconf support via avahi
libapache2-mod-encoding - Apache2 module for non-ascii filename interoperability
libapache2-mod-fcgid - an alternative module compat with mod_fastcgi
libapache2-mod-geoip - GeoIP support for apache2
libapache2-mod-ifier - Filter and reject incoming client requests
libapache2-mod-jk - Apache 2 connector for the Tomcat Java servlet engine
libapache2-mod-layout - Apache2 web page content wrapper
libapache2-mod-ldap-userdir - Apache2 module that provides UserDir lookups via LDAP
libapache2-mod-macro - Create macros inside apache2 config files
libapache2-mod-mime-xattr - Apache2 module to get MIME info from filesystem extended attributes
libapache2-mod-musicindex - Browse, stream, download and search through MP3/Ogg/FLAC files
libapache2-mod-ngobjweb - Apache2 module for the SOPE application server
libapache2-mod-perl2 - Integration of perl with the Apache2 web server
libapache2-mod-perl2-dev - Integration of perl with the Apache2 web server - development files
libapache2-mod-perl2-doc - Integration of perl with the Apache2 web server - documentation
libapache2-mod-php4 - server-side, HTML-embedded scripting language (apache 2.0 module)
libapache2-mod-php5 - server-side, HTML-embedded scripting language (apache 2.0 module)
libapache2-mod-proxy-html - Apache2 filter module for HTML links rewriting
libapache2-mod-python - An Apache module that embeds Python within the server
libapache2-mod-python-doc - An Apache module that embeds Python within the server
libapache2-mod-python2.2 - An Apache 2 module that embeds Python 2.2 within the server
libapache2-mod-python2.3 - An Apache 2 module that embeds Python 2.3 within the server
libapache2-mod-removeip - Module to remove IP from apache2's logs
libapache2-mod-rpaf - module for Apache2 which takes the last IP from the 'X-Forwarded-For' header
libapache2-mod-ruby - Embedding Ruby in the Apache2 web server
libapache2-mod-scgi - Apache module implementing the SCGI protocol.
libapache2-mod-speedycgi - apache2 module to speed up perl scripts by making them persistent
libapache2-mod-suphp - Apache2 module to run php scripts with the owner permissions
libapache2-mod-vhost-hash-alias - Fast and efficient way to manage virtual hosting
libapache2-mod-vhost-ldap - Apache 2 module for Virtual Hosting from LDAP
libapache2-mod-xmlrpc2 - XMLRPC Server module for Apache2 web server
libapache2-modbt-perl - Perl bindings for mod_bt
libapache2-modxslt - XSLT processing module for Apache 2.0.x based on libxml2

Wow, some misinformation never dies.

g2boojum — Thu, 31 Aug 2006 17:12:22 +0000

>> That whole "Gentoo = ricer" thing is a load,

> No, it's historically what Gentoo was. It used to have options like
> -funroll-all-loops which nobody would recommend, and -O4 which doesn't even
> exist with gcc.

I could be wrong, but that doesn't agree with my recollection, and I've been using Gentoo since the 1.0_pre?? days. I remember using -O3 -pipe as the recommended default back then, and now we recommend -O2 -pipe. Yes, the current recommendation is more conservative, but not really by all that much. (It is also worth remembering that back then many distros compiled their software for i386 by default, so optimizing w/ march=i686 actually did make a noticeable difference.)

Minimalist builds untested

cventers — Thu, 31 Aug 2006 15:01:07 +0000

Perhaps, but one thing I find interesting about building your system from
source (and then prelinking your magic blend) is that it would presumably
make things much more difficult for someone that wanted to inject
shellcode through an overflow vulnerability. If you want to target a
bunch of Linux systems, you can go after a distribution like Red Hat
where everyone is running exactly the same binaries.

In a world like Gentoo, though, they're running the same software, but
all with different combinations of features and load addresses. How do
write shellcode to jump to a different function in the app if everyone's
compiler has put the function at a different address or GOT offset, for
reasons of different compiler versions, USE flags, etc?

That doesn't alleviate the problem the way some other solutions (like
PaX) do, but I don't think that property could hurt at all -- certainly
not if our systems were facing a threat from a worm, for example.

Some information

cventers — Thu, 31 Aug 2006 14:53:35 +0000

I'm still a Gentoo user.

I used to buy into this 'Gentoo Ricer' bs as well, reading sites like
'funroll-loops' that made fun of the segment of naive Gentoo users that
like to add to their CFLAGS every parameter they can find in the gcc
manpage.

I eventually decided to try Gentoo on the desktop when I started a new
job. I was _so_ impressed with Portage that I immediately got rid of
Slackware at home. I've been a happy Gentoo user ever since.

Wow, some misinformation never dies.

rsidd — Thu, 31 Aug 2006 12:47:45 +0000

That whole "Gentoo = ricer" thing is a load,

No, it's historically what Gentoo was. It used to have options like -funroll-all-loops which nobody would recommend, and -O4 which doesn't even exist with gcc.

The flexibility with USE flags is newer, and yes, it's neat. The thing is, if I'm on a reasonably modern machine, omitting build dependencies Gentoo-style doesn't really help that much (eg, I don't use gnome but I don't really care if firefox is built with gnome support). I may save 2% of the used disk space. While if I'm on an older, slower machine, compiling everything on it gets pretty tiresome. That, and when I used it last (some years ago) I managed to hose it twice while updating (it updated libc to an incompatible version breaking most of the system in the process). So I don't have much use for Gentoo, but obviously some people do.

Hurrah Gentoo Linux 2006.1 released

job — Thu, 31 Aug 2006 12:36:32 +0000

I just wanted to add to the comments you've already got on the subject:
In short, adding support for, say Kerberos, is done by setting a flag and
rebuilding all packages that support it where a project such as Debian
has to offer the same package in a kerberized variant. Supporting many
variants on a packages quickly becomes coplex for binary based distro,
while with a source based one you can choose an arbitrary combination.

Hurrah Gentoo Linux 2006.1 released

edomaur — Thu, 31 Aug 2006 11:56:22 +0000

I have to answer this...

Okay : one year ago, my then employer bought a new dev server, based upon some too new hardware for the RedHat and friends distros. After spending a day trying to get the drivers and the system to work, I switched to Gentoo. Something like 15 minutes to set up, then letting the system build itself during the night. After that, it just worked, without any problems because of the too new hardware.

Minimalist builds untested

jimmybgood — Thu, 31 Aug 2006 10:46:05 +0000

This is certainly possible, but seems unlikely unless one chooses to build without security features like selinux or privilege seperation. Do you have any examples? Has there been an abiword built the Debian way with four spell checkers and enchant that doesn't have a vulnerability that abiword built with just one spell checker *does* have? Has there been a cairo built with directfb, svga, ggi and aalib support that is more secure than a cairo built only with X support?

By the same token, upgrading in response to a security fix has been known to leave you more vulnerable than before the security fix. But common sense and experience dictate that you will more likely be safe than sorry for upgrading in such a situation.

While I can't be sure that subtle bugs won't be introduced by building with a reduced feature set, I think, with Linux anyway, the less software I have, the safer I am. With OpenBSD, though, I might agree with you. The consideration being the experience and dedication to security of the coder and the rigor with which the code is tested and audited before being released.

Wow, some misinformation never dies.

flewellyn — Thu, 31 Aug 2006 07:01:08 +0000

Yes, but it's not a very good one.

See, you're missing that many of these packages (especially complex ones like PHP or Apache) have compile-time options that determine what features are built in.

If you only use precompiled packages, you can't choose those. This should be readily apparent.

Some information

man_ls — Thu, 31 Aug 2006 06:52:34 +0000

In short: with Gentoo you get to choose the dependencies you want. You don't want a dependency with GNOME, fine: you can remove it. You cannot do that with apt. emerge then figures out the remaining dependencies for you.

Gentoo makes for a very nice desktop; try it one time instead of blindly bashing the fine work of many people. It was not lost time: I learned a lot. In the end I got tired of synchronizing with the repos and recompiling: yes, it's tiring, I felt like a distro maintainer (which in a sense I was). But then I had been spoilt by apt and rpm...

Wow, some misinformation never dies.

sholdowa — Thu, 31 Aug 2006 04:01:27 +0000

No. Portage is *currently* at least as good as / better than rpm or dpkg in this respect at the moment. They all stuff up on a regular basis ( yes, even SuSE (: ), it's just not currently gentoo's turn.

( Although I've been hearing comments to the contrary, and complaints about the amount of diskspace required lately - but that's probably just a knock-on effect of 2006.0 getting a bit long in the tooth )

Wow, some misinformation never dies.

sholdowa — Thu, 31 Aug 2006 03:57:05 +0000

To quote "It's not about compiler optimization, it's about being able to choose the optimal feature set for your needs."

Now, reread *your* post in the light of that statement, which was the one that *I* was responding to. In which way does gentoo's emerge system differ to yum or apt when choosing feature sets?

See my point now?

Minimalist builds untested

xoddam — Thu, 31 Aug 2006 03:47:26 +0000

> This translates into better security.

There are two potential schools of thought on this. One is, "deleted code is debugged code". For sure, vulnerabilities which are specific to the subsystems you're not including will not exist in your minimalist build.

But another point is that what you're running is rather different from anything that has had serious production testing by the upstream project or distributors' QA people. It's a product release uniquely made for and by you, and although you might *generally* be able to trust that upstream will ensure that builds with options switched off do basically work, no-one but you can promise that your exact combination is solid and security-hole free. Can you ever be sure that omitting a major component which everyone else uses won't introduce subtle bugs?

Wow, some misinformation never dies.

corey_s — Thu, 31 Aug 2006 01:50:55 +0000

> I see, so Gentoo is just the same as all other distros that use yum/rpm,
or apt/dpkg, etc then?

Apparently you _don't_ see. When learning about a subject which you have
no actual/practical experience with ( gentoo ), it helps to ask real,
honest questions rather than make assertive assumptions under a false
pretense of actual curiosity.

With a binary package management system, such as "yum/rpm or apt/dpkg" -
you get whatever fat-ass binary was built for you by the distribution's
maintainer(s), i.e. generally everything and the kitchen sink. This adds
layers and layers of extra software you don't actually want or need due to
the depencies that were built into the package in question, such as
including perl, python, _and_ ruby support into vim, when you only ever
use ruby - or such as building X with every single extra/unnecessary
driver, utility, service, support, etc, etc, that you don't need... etc.,
etc. ad infinitum.

With a source-based package management system such as gentoo's portage,
you can explicitly configure what you _want_ and what you dont want
built/included/extended into your packages and into your system.

Never use gtk or gnome, prefer qt and kde, and only ever script in perl?
Then add:

USE="-gtk -gnome qt kde perl"

into your portage 'make.conf' config, and _every_ package you ever install
will abide by those rules.... meaning that when installing, say, firefox -
using the above portage configuration ( "USE flags" ), then I'll get a
firefox installed on my system that doesn't also require gnome and gtk
libraries/software:

scanner ~ # emerge -s firefox
Searching...
[ Results for search key : firefox ]
[ Applications found : 2 ]

* www-client/mozilla-firefox
Latest version available: 1.5.0.5
Latest version installed: [ Not Installed ]
Size of files: 39,211 kB
Homepage: http://www.mozilla.org/projects/firefox/
Description: Firefox Web Browser
License: MPL-1.1 NPL-1.1

* www-client/mozilla-firefox-bin
Latest version available: 1.5.0.5
Latest version installed: 1.5.0.5
Size of files: 12,759 kB
Homepage: http://www.mozilla.org/projects/firefox
Description: Firefox Web Browser
License: MPL-1.1 NPL-1.1

scanner ~ # emerge -pv mozilla-firefox

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild N ] dev-libs/libIDL-0.8.6 USE="-debug -static" 342 kB
[ebuild N ] dev-libs/nspr-4.6.1-r2 USE="ipv6" 1,301 kB
[ebuild N ] app-arch/zip-2.31 USE="crypt" 783 kB
[ebuild N ] dev-libs/nss-3.11-r1 4,885 kB
[ebuild N ] www-client/mozilla-firefox-1.5.0.5
USE="ipv6 -debug -gnome -java -mozdevelop -xinerama -xprint"

... notice the '-gnome' and '-java', etc for
the 'www-client/mozilla-firefox' entry?

That means the firefox on this particular system wont build with those
extra dependencies.

But what happens if I 'emerge' using different USE flags? Let's see:

scanner ~ # USE="gnome java -ipv6" emerge -pv mozilla-firefox

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild N ] dev-libs/libIDL-0.8.6 USE="-debug -static" 342 kB
[ebuild N ] dev-libs/nspr-4.6.1-r2 USE="-ipv6" 1,301 kB
[ebuild N ] app-arch/zip-2.31 USE="crypt" 783 kB
[ebuild N ] dev-libs/nss-3.11-r1 4,885 kB
[ebuild N ] gnome-base/orbit-2.14.0 USE="ssl -debug -doc -static" 687
kB
[ebuild N ] gnome-base/gconf-2.14.0 USE="-debug -doc" 1,851 kB
[ebuild N ] gnome-base/gnome-mime-data-2.4.2 USE="-debug" 829 kB
[ebuild N ] gnome-base/libbonobo-2.14.0 USE="-debug -doc" 1,354 kB
[ebuild N ] gnome-base/gnome-vfs-2.14.2 USE="hal
ssl -avahi -debug -doc -gnutls -ipv6 -samba" 1,773 kB
[ebuild N ] gnome-base/libgnome-2.14.1 USE="esd -debug -doc -static"
971 kB
[ebuild N ] gnome-base/libglade-2.5.1 USE="-debug -doc" 310 kB
[ebuild N ] gnome-base/libgnomecanvas-2.14.0
USE="-X -debug -doc -static" 597 kB
[ebuild N ] gnome-base/gnome-keyring-0.4.9 USE="-debug" 386 kB
[ebuild N ] gnome-base/libbonoboui-2.14.0 USE="-X -debug -doc" 872 kB
[ebuild N ] gnome-base/libgnomeui-2.14.1 USE="jpeg -debug -doc" 1,847
kB
[ebuild N ] www-client/mozilla-firefox-1.5.0.5 USE="gnome
java -debug -ipv6 -mozdevelop -xinerama -xprint"
[ebuild N ] x11-themes/hicolor-icon-theme-0.8 30 kB
[ebuild N ] dev-perl/XML-NamespaceSupport-1.09 7 kB
[ebuild N ] dev-perl/XML-SAX-0.14-r1 57 kB
[ebuild N ] virtual/perl-Storable-2.15 0 kB
[ebuild N ] dev-perl/XML-Simple-2.14 64 kB
[ebuild N ] x11-misc/icon-naming-utils-0.7.0 59 kB
[ebuild N ] x11-themes/gnome-icon-theme-2.14.2 USE="-debug" 2,878 kB

Whoah... completely different result. LOTS AND LOTS of CRAP that my
WORKSTATION or SERVER DOESN'T NEED. That's exactly the sort of massive
extra overhead and cruft you'll get in a binary-based package management
system... among other undesirable artifacts.

This translates into better security.

This translates into more simple maintenance.

This translates into better performance.

If security, reliability and performance aren't something you much care
for or require in your own use of a linux-based operating system, then
that's cool, bro. To each his own, and all that.

Wow, some misinformation never dies.

flewellyn — Thu, 31 Aug 2006 01:18:15 +0000

If you use a binary package from Debian, you get the compile-time options that the maker of the package selected.

If you use Gentoo, you get the compile-time options that YOU selected.

The benefits here should be readily apparent.

Another thing to note is that Portage is a lot better at figuring out dependency information than APT. I have used both extensively, and Portage wins hands down at handling thorny dependency issues, where APT would simply scream and die. And don't even get me started on RPM/yum, that's just a horrid mess.

Also of note: if you have the ability to rebuild any necessary package (or its dependencies) at need, API or ABI changes are much less problematic. My Gentoo box handled the change from gcc-3.3 to 4.0 without problems, all C++ applications were rebuilt and just worked.

Wow, some misinformation never dies.

sholdowa — Thu, 31 Aug 2006 01:08:31 +0000

I see, so Gentoo is just the same as all other distros that use yum/rpm, or apt/dpkg, etc then? You can choose the features you want, and they handle all the dependencies too, don't they?

So what was the advantage again, if it's *not* optimizing your kernel for your hardware?

Gentoo Linux 2006.1 released

flewellyn — Wed, 30 Aug 2006 22:48:39 +0000

Yes, you can use options to Portage that request binary packages, choose from a repository other than the defaults, and so on. It's very configurable.

Wow, some misinformation never dies.

flewellyn — Wed, 30 Aug 2006 22:47:20 +0000

That's not what Gentoo people mean when we're talking about "optimizing" the system. It's not about compiler optimization, it's about being able to choose the optimal feature set for your needs. The packages you install have just the options you want built in, with dependencies handled automatically.

That's the advantage, not compiler optimization flags. That whole "Gentoo = ricer" thing is a load, and people should really do some research before making claims like that.

Hurrah Gentoo Linux 2006.1 released

stuart — Wed, 30 Aug 2006 22:41:18 +0000

and best of all the electricity used in compiling all the packages is a small sacrifice for the 1% speed up or more if one considers robust gcc options like -fomit-instructions.

Gentoo, an American word meaning "for people who have too much time."

Gentoo Linux 2006.1 released

rvfh — Wed, 30 Aug 2006 22:38:11 +0000

And have you got a way to tell the system which packages you prefer pre-built, and if not found revert to local build? Does all that work for upgrades too, or just first install? Still curious..

Gentoo Linux 2006.1 released

flewellyn — Wed, 30 Aug 2006 21:43:31 +0000

Yes, Portage can use pre-built packages. They just don't provide them for most things (a few, like X.org, KDE, and OpenOffice).

I know of one admin for a cluster of Gentoo workstations at a university in London who has a central "master builder" system, configured identically to the other machines in the cluster, that every week pulls down and compiles the latest package upgrades from the central Portage tree. The machine uses distcc to distribute the compilation across the other machines in the cluster. Once everything is built, it stores the compiled packages in its repository, and has the other machines in the cluster sync with it and download the binaries to install.

This way, all of the machines get upgraded at once, the packages only need to be compiled once, and the compilation goes faster because it's distributed across the cluster.

Gentoo Linux 2006.1 released

johnkarp — Wed, 30 Aug 2006 21:30:30 +0000

To avoid the chicken/egg problem you have to bootstrap your system from
*something*, so a minimal prebuilt environment is included.

Yes, there is support for binary packages, though its generally used for
the scenario where you build your own packages centrally, and distribute
them to your other computers.

Gentoo Linux 2006.1 released

tony.taylor — Wed, 30 Aug 2006 21:23:15 +0000

Yes, you can install straight from pre-built binaries. However, each architecture also has architecture-specific patches, so even if you build from scratch, you want your particular brand.

Gentoo Linux 2006.1 released

rvfh — Wed, 30 Aug 2006 21:17:36 +0000

The AMD64, HPPA, x86, 32- and 64-bit PowerPC releases are built with and include GCC 4.1 <snip>

I though Gentoo was about building everything from source? Is it possible to use pre-built packages in Gentoo like in other distributions then? Just curious...