A look at rpath Linux [LWN.net]

June 8, 2005

This article was contributed by Joe 'Zonker' Brockmeier.

Michael K. Johnson announced the 0.24 release of rpath Linux, formerly known as "Specifix Linux," last Thursday. This release includes 0.60.4 of the Conary Software Provisioning System. Conary is meant to replace package managers like RPM and dpkg.

We downloaded the rpath ISOs and took the distribution for a little test drive, and we e-mailed Johnson and chatted with him in the #conary channel on freenode.net about the distribution and Conary to find out what it has to offer over other packaging systems.

In terms of the rpath distribution itself, Johnson said that it wasn't particularly unique, apart from the Conary packaging system. "Because the whole point is to be quite 'vanilla' outside of Conary, rpath Linux's main unique feature is that it is built with Conary. (Even that is not quite unique, actually, since Foresight Linux exists and is built with Conary as a derivative of rpath Linux.)" The rpath distribution uses an Anaconda installer, and basically is a "vanilla" distribution with a GNOME 2.10 desktop and a lot of the applications you'd expect to see in a basic desktop distribution.

As the introduction to the Conary system explains, Conary is a packaging system that works like a Source Control Management (SCM) system. Everything is stored in a distributed repository, rather than package files. Components and packages in Conary are called "Troves." A source component may be built with different configurations and/or for different architectures. This is called a "flavor." A good example of this would be kernels built for SMP systems, or with different instruction sets. The SMP and UMP kernels would be different "flavors" of a component.

Versioning in Conary works a bit differently than with package systems like RPM and dpkg. For example, the RPM naming convention provides the name of the package, the version, the package release number, and the architecture. So the Abiword package for Fedora Core 3 is abiword-2.0.12-3.i386.rpm. Conary, on the other hand, names files according to the repository, the version number of the software, source revision number and binary revision number.

In practice, Conary's design allows one to install a package like Abiword and its dependencies without necessarily installing additional packages. For example, installing the Abiword "Trove" added the Enchant library component, but not the Enchant runtime or document components. Johnson also said that Conary makes it easy to install multiple versions of libraries. For example, users who run x86_64 should be able to easily install x86 and x86_64 versions of libraries.

Conary divides files up between components automatically (with manual overrides, of course) and the defaults make it easy to have multiple non-conflicting libraries installed on a system that supports them both. We simply build all packages with these default settings. Furthermore, Conary automatically checks for several problems that would break a multi-lib setup, and halts the build with errors if it sees them. This means that we don't have to have a special rebuild to make some core set of x86 libraries available on x86_64; instead, any of the x86 :lib components can install and function on x86_64. This is going to become more and more important now that both AMD and Intel's mainstream is entirely 64-bit x86_64.

Conary also works like an SCM in that one can rollback transactions. By running "conary rblist" one can see the recent commits to the system and one can also move backwards by running "conary rollback r.nnn" where "nnn" is the number of the revision. The list of commits to the system appears to start from the very beginning of the installation, so one could conceivably roll back quite a few changes rather easily. Note that rollbacks cannot be applied out of order, so one must progress backwards one rollback at a time.

The system can also be used to generate local changesets that can be committed to a local repository, and updated on other machines from that repository. This makes Conary interesting for system admins who need to customize software across a group of machines.

Conary also supports "branches" for development of Troves, so one may install a branch of an application and continue to follow that development tree rather than worrying about a conflict between versions of the application. If the main rpath distribution includes Firefox, for example, and there's an experimental version of Firefox in the "contrib" repository the user can install the experimental version from the contrib repository and then follow that branch of development without worrying about conflicts with the "official" version in the main repository. This also works with Conary flavors, so once one installs a specific flavor, that flavor will be installed when the user updates the package.

The rpath distribution also includes a Conary GUI application that serves as a browser for repositories, and which makes it easy to see what Troves are available for installation and so forth. It was easy to install Abiword and other applications from the Conary GUI, though the GUI works on the metaphor of applying updates rather than "installing" a package -- which might throw some users off. The Conary command-line tools took a bit of getting used to, but this is probably more a symptom of many years experience with RPM and dpkg, rather than a sign that Conary is overly complex. It's not quite as slick as APT or Yum just yet, but Johnson did say that work is still being done on Conary.

We also asked Johnson what the goals for rpath Linux were, and where rpath could "fit" in the already-crowded distribution market. According to Johnson, the problem is not that the market is too crowded, but that it's "crowded in the wrong way."

It is crowded with lots of little effectively unrelated operating system images, all different, and different in ways that aren't immediately obvious, easily discoverable, or even intentional. There's no real reason, for example, to think of every Knoppix derivative as a separate "distribution", except that the technology doesn't explicitly working with them as a set of related and interoperable sources of operating system data. With Conary and rpath Linux, we are separating the concepts of "distribution" and "installation image". The repository is the canonical source of the bits, not a set of ISO images. Why should it be hard to create a custom installation image that represents exactly what you want to install on your system? That's a trick question; the answer is that it shouldn't be! Doing that should not be counted as creating "another distribution".

Think more about a set of custom, interoperable operating system images instead of "distributions". Then you can pick the best operating system image without worrying about choosing a distribution putting you in a corner. Conary is the core technology which enables this view, and rpath Linux is a foundation or cornerstone.

He also said that the goal for rpath was "to make it a good distribution on which to base a derivative."

Ken VanDine joined the Conary community immediately after we announced Conary, and he quickly saw the potential Conary's new model provides. He then set to work on a derivative distribution called Foresight Linux which has about 20% changed or new content relative to rpath Linux...

Being a good source for derivative operating system images has some definite implications. rpath Linux must not be too heavily patched, because the more patches we apply to an upstream project, the less likely it is that some other patch (which someone building a derivative wants to apply in their derivative) will apply. The distribution needs to be functional and coherent, because otherwise who will want to use it as a source for their derivative work? It needs to be relatively current, because new patches aren't likely to apply easily to old source code.

Some people ask whether this approach will make "distribution hell" that much worse. Fortunately, the answer is, "no". When Conary is widely adopted (the only case that actually matters from this perspective), we'll have lots of interoperable slices, with rich dependencies that make it clear what actually interoperates. Already, rpath Linux users sometimes cherry-pick the bits that they want from the Foresight Linux repositories. Rich dependencies and explicit distribution and package inheritance will make this continue to work. Conary will mean that there are more customized installation images available, but will alleviate unnecessary incompatibilities by allowing derivatives to differ in distinctives only, and not drift apart into mutually-incompatible projects.

Obviously, Conary will achieve these goals by being adopted. Since Conary is currently in beta, and rpath Linux is in the last few stages of being alpha, I'm looking a little bit into the future here!

Conary is not limited to Linux systems. Johnson said that Conary should work "just fine on BSDs, and that they've had a report of successful Conary installation on Cygwin. The rpath distribution is probably not ready for production use, we ran into some spectacular Python errors using Conary after just a few updates and rollbacks, but the Conary package system is definitely worth a look. It should be interesting to see whether or not the Conary package system catches on. It has some worthwhile features, but it won't be easy to convert users who are already familiar (and have strong biases towards) existing packaging systems.

(Log in to post comments)

Buzzwords, marketingspeak, and/or other jargon

Posted Jun 11, 2005 3:04 UTC (Sat) by msw (subscriber, #3733) [Link]

> But based on that document, conary seems to mostly solve problems found in
> the RPM world. As a typical desktop user, it's still not clear to me how
> conary would be better than emerge or apt.

Enabling people to modify and customize the software components in a Linux based distribution was is of the major design goals for Conary. dpkg, rpm, portage, etc. don't give you the facility to do distributed development of an entire Linux-base distribution.

I was talking to a help desk staffer after my Conary presentation at TruLUG last night. He was asking why we didn't have Conary three years ago when the organization was deploying Red Hat Linux 7.3 to non-technical desktop users. They added StarOffice as the office suite, but integrating it into the distribution was extremely difficult. They wanted to modify the mailcap package so that StarOffice would automatically be used to open .doc files downloaded in Netscape. They ended up creating their own "ourcompany-mailcap" package because there wasn't a way to maintain that local modification to the mailcap package.

With Conary, the company would be able to create a special type of branch of the mailcap package. We call these branches "shadows". They would be able to make their local modifications on that shadow, which could be stored on a local Conary repository. Shadows keep track of the ancestry of the package, which allows you to easily merge in changes that happen "upstream" of the shadow.

Conary brings a paradigm shift in how software is managed on Linux systems, and how Linux distributions are built and customized. It takes a bit of time to see the whole system when you're looking at it from a "how is it better than APT, YUM or Portage" perspective.

Buzzwords, marketingspeak, and/or other jargon

Posted Jun 13, 2005 1:41 UTC (Mon) by hazelsct (guest, #3659) [Link]

I'm sorry, but I too fail to see the advantage, at least over .debs (don't know about RPMs). The mailcap examples can be treated as a simple case of configuration file customization, which dpkg tracks straightforwardly -- and debconf does even better, as it substitutes configuration options correctly even as file formats change, where "shadows" might manage for small changes but will inevitably fail in the general case. (In fact, the http backend of debconf allows one to set site-wide configuration options which are seamlessly integrated into every machine's package-by-package conf files, automagically upgraded, etc.)

Perhaps this is why Debian has been so successful as a base for new distributions...

But again, how is it again that conary does this better than dpkg?