Fedora ponders Software Collections

A feature in Red Hat Enterprise Linux (RHEL) that supports multiple, parallel installations of programming languages and other normally system-wide tools was recently discussed on the fedora-devel mailing list. Matthew Miller, who recently started as the Fedora cloud architect, raised the idea of bringing Software Collections from RHEL to Fedora. The idea behind Software Collections is interesting, but no clear consensus on how appropriate they might be for Fedora emerged. As with many Fedora discussions of late, this one at least partly comes back to the question of the role that the distribution is meant to fill.

The problem that Miller initially presented is particularly acute in the Ruby and Java worlds, though Python and other tools (e.g. databases) sometimes suffer from it as well. Various packages may depend on different versions of the underlying tools, which makes it difficult to have them coexist on the same system. As an example he noted that the Fedora packages for the Puppet configuration management tool are broken because the Fedora Ruby version is too new. One way to solve that problem is to have multiple Ruby versions available that can be installed in parallel and chosen at runtime. That's exactly the problem that Software Collections sets out to solve.

A Software Collection (SC) uses the same packaging tools (RPM, Yum) already used by RHEL and Fedora, but installs the packages and their dependencies in the /opt/provider hierarchy. The provider piece is a specific string assigned to a vendor, which will allow multiple software providers to share the hierarchy without name collisions. There is also an scl tool that allows choosing one or more SCs to be active when running a specific command. Whatever is needed in the environment for the particular SC will be set up by "scriptlets" that get installed with the collection and are run when the SC is selected.

As Miller notes, there is nothing inherent in Java or Ruby that leads to version-mismatch problems, instead they are caused by the expectations of developers building software using those languages. There is a strong preference for bundling various toolkits and libraries with such packages, which runs counter to the way Fedora and many other distributions do things. Red Hat Eclipse team member Alexander Kurtakov put it more bluntly:

But, the fast-paced nature of Fedora (normally a release every six months or so) may not make for a good match with SCs. Former Fedora project leader Jared Smith was a bit skeptical about the fit:

Miller had a ready answer. He outlined three separate uses he saw for SCs in Fedora, starting with handling problems like the Puppet issue. Allowing multiple languages would give more choices for Fedora as a development platform. He also noted that RHEL and Fedora make up an ecosystem where developers targeting the former may well be developing on the latter. Access to SCs on Fedora might be quite useful since they are available for RHEL. Those two potential use cases did not require too much discussion, but the other one did:

Bill Nottingham responded with a self-proclaimed "heretical" suggestion that Fedora be turned into a much smaller platform, with packages from the "grand Fedora universe" that target one or more of those platform releases. In that model, the enormous pile of software that Fedora deals with for each release would be greatly reduced. Miller and others—notably enterprise-leaning participants—looked favorably on the heresy, but it was recognized that it would be a difficult direction for Fedora to take.

There are, of course, downsides to managing software via SCs. The "library bundling problem" comes to mind, for example. If multiple SCs all include a library (or other component) that needs to be upgraded for security reasons, it may require a great deal of work. One could imagine several different vulnerable versions of a library lurking in SCs that are still being used. Each of those needs to be fixed and all of the SCs need to be updated. For RHEL, that's par for the course, but Fedora has generally moves on before those kinds of problems become acute.

The conversation soon pivoted from Nottingham's suggestion to whether SCs might help external projects or companies in making their software available for Fedora. By providing a stable platform and a way for those entities to bundle up and install all of the needed pieces, more software might be made available for Fedora. Much of that software is, of course, proprietary, but there are advocates for making Fedora an easier target for those kinds of applications.

One of the problems that Fedora has faced over the years is the explosion of packages that it tries to maintain, test, and release in a short six-month time frame. Several commenters pointed to SCs (or something like them) as a way to decouple Fedora from that enormous list of packages. Projects could target Fedora, without actually becoming part of Fedora, as Fenando Nasser suggested.

Those kinds of ideas hearken back to an earlier arrangement for the Fedora distribution. While Adam Williamson's humorous idea of a return to Fedora Core and Fedora Extras was not taken seriously, some kind of similar split seemed to gain quite a bit of traction in the discussion. Whether it goes any further than that down the road remains to be seen, but SCs could potentially help the process if it does.

There are logistical and licensing questions—along with plenty of others—that would have to be resolved, of course. If there were a split, how would non-core (for lack of a better term) components manage their SCs and repositories? If they were under the Fedora umbrella, the distribution would have some responsibility for the contents of the packages. If they were not under the umbrella, users would have to somehow enter those repositories into their Yum configuration. Richard Jones outlined a number of issues that would need to be resolved under such a system.

While the conversation veered in a direction that Miller may not have expected, it does give an interesting view into the thinking of some (many?) in the Fedora development community. Some of the interest in a fairly radical change may come from frustration with the delays in the Fedora 18 cycle, but there seems to be more to it than that. For some time now, Fedora has been trying to find (or define) its niche. This conversation is another step along that path.