Development Gentoo for developers

The Grumpy Editor didn't have enough experience with Gentoo Linux to cover it in his review of development distributions, so the time is nigh. A few commenters on the review also said they'd hoped it was about the best distributions to develop with, so I'll show you here what makes developing with Gentoo Linux a pleasure.

Many of you know Gentoo is a moving target. You may not know, however, that despite that, it's divided into two main chunks: stable and testing. All packages go through a phase in testing (a.k.a. ~arch, with the tilde designating testing status) before they go stable. Stable resembles the testing level of Debian, and Gentoo's ~arch resembles Debian Sid. Developers often want to use the latest and greatest development tools and libraries to develop against, and running Gentoo ~arch (like any good development distribution) allows you to keep all these tools and libraries under package management. First, we'll cover the same points as our Grumpy Editor's review, then we'll move into using Gentoo for development.

Stability

Although complaints about Gentoo's stability and breakages are fairly common, many of those complaints involve compile-time rather than run-time failures. Perhaps the largest complaint about run-time issues with Gentoo involves shared-library version bumps. Since you, rather than an upstream provider, are doing all the building, you need to run a program called revdep-rebuild to track down and rebuild packages that broke from a shared-library version change. While you're waiting for the rebuilds, packages using the bumped library may or may not break, depending on its maintainer's choice. Fortunately this will change in Portage 2.2, planned for release by the end of the year, with the addition of a global setting to always save the old libraries.

Tracking ~arch is possible with the --changelog and --pretend options to emerge. The --changelog option, when used in combination with a --pretend merge (which simply shows which packages would be installed or updated), displays the ChangeLog entries made since adding the new package . Other than that, the new Gentoo packages site has an Atom feed with all version bumps. There is no mailing list, however, dedicated to ~arch users. Instead, one would hope people encountering bugs would report them, and others could search Gentoo's Bugzilla for bugs filed in the past few days. Advance warnings are fairly scarce, but that may change as Portage 2.2 releases late this year with support for distributing news announcements. Last-minute warnings now show up as messages printed just before or after installing the update (and saved for later reading), but sometimes this is too late. Here's a look at --changelog output:

Package management and updates

LWN covered in July the basics of Gentoo's package management. The basic thrust is that Gentoo has three competing low-level package managers: portage (the official one), paludis and pkgcore. Since much of package management in Gentoo is done on the command line, the CLIs are very colorful and featureful: emerge --help --verbose produces 334 lines of options and descriptions. Here's an example of colorful, verbose emerge output:

Development of GUI package-management tools has increased lately, with portato and himerge giving strong showings as fairly full-featured GUIs. Other GUIs exist for more specific purposes, for example configuration-file updating and USE flag (compilation feature) editing. One of the biggest problems with Gentoo is choosing which of the many tools you like best. For example, there are at least 5 different packages that allow you to do a package search.

Configuration-file updating is superb in Gentoo compared to many other distributions. This may be in part because Gentoo assumes some level of familiarity with the command line and with config-file editing. Again, there are a few tools: etc-update and dispatch-conf on the CLI side and etcproposals on the GUI side, among others. After each installation, Gentoo tells you how many configuration files you have to update, and the various tools help you merge the old and new configurations. Some of them even have true version control.

Package selection

Gentoo has 12,037 packages in its main repository as of this writing, which puts it among the largest package collections. A quick check of my Fedora CVS checkout revealed roughly 5,200 packages, to put this in perspective. Since the state of ~arch has no relationship to Gentoo's releases, there are no freezes and packages are never artificially forced to become out of date. There are also many more packages available via overlays, additional package repositories that are easy to find using layman, which provides a centralized list of pointers to repositories and manages them for you, and easy to search (even without downloading them) with eix, which can download a pregenerated cache of information from all repositories. Here's an example of the additional repositories on my system:

Since Gentoo is a pragmatic distribution, it doesn't force any particular ideology upon users so it's easy to use patent-encumbered formats. Whether this is a good or bad thing philosophically remains open to question, but Gentoo is a tool; it tries to get out of your way to let you do what you want. For those more interested in freedom, Portage 2.2 will add license restrictions—you can refuse to install any packages with nonfree licenses. Other Gentoo package managers already allow this.

Why develop with Gentoo?

Gentoo provides a powerful set of tools for a development machine. Since it's a source-based distribution, it comes by default with the full toolchain and related tools (autotools, flex, bison, etc). Furthermore, the packages are not split up, so if you have a package installed, you have all of that package: no searching around for -devel, -headers, -debuginfo, or whatever else.

Two extremely powerful tools to speed up development are ccache and distcc, both of which Gentoo has quite well-integrated and documented. CCache is the compiler cache, which can significantly speed up compile times by caching the object files. DistCC is the distributed C/C++ compiler, which distributes parallel compilations among multiple hosts and is really handy for anything that can build in parallel and takes a while, such as a kernel.

To set up a decent development system, you often want to install debugging versions of a lot of libraries and maybe even install the source code too. Gentoo provides for that with a couple of FEATURES settings: splitdebug and installsources. The splitdebug feature installs separate debuginfo files that are automatically found by gdb, and installsources will do what it sounds like. Combined with an addition of -g or similar to to your CFLAGS setting in /etc/make.conf, that will get you off and running.

On a lower level, you might want to make sure your code compiles on a wide variety of compilers before shipping or releasing it. Gentoo's gcc-config lets you dynamically switch between multiple installed compilers, and you can even use non-GCC compilers for Gentoo packages by simply setting CC in /etc/make.conf or the environment. Here's all of the GCCs installed on my system, including cross-compiling toolchains:

Gentoo supports cross-compiling quite nicely with a tool called crossdev that integrates with Portage to make building cross-compiling toolchains completely trivial. All you have to do is pass crossdev the target and it takes care of the rest. You may want to cross-compile with distcc, too, and Gentoo also supports that. There's a draft version of the Gentoo Embedded Handbook that goes into detail on creating and using cross-compilers, but remember that it is just a draft.

Another feature that makes Gentoo excellent for development is its ability to install packages into a chroot with the variable ROOT. This makes it easy to test things independent of your development system without getting into virtualization. It also helps a lot for embedded development (when coupled with crossdev) and netbooting (when exporting the client / over NFS).

The last, most advanced and possibly most useful aspect of Gentoo for development is the ease of packaging your code. Writing ebuilds is just writing bash shell scripts. If you can build it by hand, you can make an ebuild for it. Here's an example of a simple ebuild, to wrap things up: