Distributions

Fedora opens up to bundling

By Jonathan Corbet
October 13, 2015

The term "bundling" refers to the practice of distributing a copy of one software project (usually some sort of library) within another one. Software developers may have a number of reasons for bundling, but Linux distributors tend to dislike it for reasons of their own. The Fedora project, in particular, has long forbidden bundling except in a few cases where it could not be avoided. It now seems, though, that Fedora has decided to back off a bit on its anti-bundling policy — a decision that is not uniformly popular in its development community, but which may well be necessary to help ensure the distribution's ongoing relevance.

The ups and downs of bundling

There can be any of a number of reasons for the bundling of software. Typically, developers of a large package want to have stable versions of important libraries to develop against; if they rely on libraries installed by others, they lose control over which versions are installed. Bundling can make the build process easier for others by freeing them from the need to chase down dependencies. Sometimes these projects have their own special patches that they apply to make a bundled library work the way they want. And sometimes bundling just seems like the easiest way to go, even if there is no particular need to do things that way.

Distributors, of course, would rather not distribute multiple versions of libraries, so they tend to push back against bundling. Multiple copies of libraries bloat the size of the distribution, of course, but they also raise security concerns: if a given library proves to have a vulnerability, the distribution must somehow locate and patch every version of that library it ships. In practice, that means that vulnerabilities can lurk for a long time in hidden copies of libraries; the "zlib" compression library has been particularly famous for propagating hidden vulnerabilities in distributions.

To avoid this kind of problem, Fedora has long had a strong policy against the bundling of libraries; if a library is used by more than one package, policy requires that library to be shipped as an independent package. Any exceptions to this rule had to go before the Fedora packaging committee (FPC) for approval, and that approval was often not forthcoming.

This intransigence with regard to bundled libraries has certainly had its intended effect: Fedora ships relatively few of them, and a number of upstream projects that bundled libraries have been fixed to not do so. But the policy also has its costs. Unbundling libraries is not always a trivial task, especially if patches have been applied by the project doing the bundling. Upstream projects may not be willing to accept the patches needed to successfully unbundle a library, with the result that Fedora ends up carrying its own private fork — a task for which few resources are available. The work required to undo bundling is such that, it seems, developers simply give up on the idea of packaging some programs entirely. Bundling issues are at the core of why Google's Chromium browser does not ship with Fedora, for example.

The bundle of straw that broke the camel's back this time around may well have been the darktable raw image editor. Darktable bundles a number of libraries, a fact that was first noted in a bug report in 2013. Over time, some of that bundling has been undone, but it still carries its own copy of rawspeed, a library for raw image decoding. The issue was brought before the FPC last July; the FPC rejected the request for a bundling exception in its July 9 meeting.

The interesting thing here is that the rawspeed library is evidently intended by its developer to be shipped as a bundled library; no effort has gone into distributing it as a system shared library. Some saw this decision as taking the anti-bundling policy too far; in response, Adam Williamson complained that the FPC went beyond its mandate:

My best inferral is that FPC wants a timeline from upstream for turning Rawspeed into a 'real' shared library, but I find it frankly difficult to see where FPC thinks it has a mandate to demand upstream projects that have been intended from day 1 to be used as copylibs to convert themselves into system shared libraries? FPC seemed to start out asking for upstream to clarify the status of rawspeed, then pivot to demanding it be rewritten without ever justifying the switch.

As it happens, the FPC did eventually backtrack a bit and grant a temporary exception, good only for the Fedora 23 and 24 releases, to darktable; this was done because the "Design Suite" spin needs it. But a number of developers seemingly saw this incident as an example of the sort of collateral damage that the no-bundling policy can cause.

A proposal

Stephen Gallagher decided to do something about it; the result was, initially, this proposal to weaken Fedora's anti-bundling policy. In short, he proposed a change of rules such that, in cases where a package is unable to link against a shared system library, it would be allowed to bundle its own version instead with no special exception required. The proposal did require the packagers to occasionally nag the upstream project about fixing the problem. Unbundled libraries are still better, he said, but:

The reason for this proposal is relatively simple: we know the advantages to unbundling, particularly with security and resource- usage. However, the world's developer community largely *does not care*. We fought the good fight, we tried to bring people around to seeing our reasoning and we failed.

Stephen expected a lively discussion to follow from this proposal, and he wasn't disappointed. Some participants decried it as a surrender to poor practices that would end up burying Fedora in inferior bundled libraries. Others were cautiously positive, noting that, if users cannot get the software they want from Fedora, they will simply get it from elsewhere, possibly ditching Fedora entirely in the process. Still others got into the details of the proposed policy, suggesting various tweaks.

The result of that latter discussion was a new proposal, posted in early October. It made the policy somewhat more complicated, and, in particular, banned outright the bundling of software in critical-path packages. These are packages that are deemed to provide essential, core functionality; in practice, there are a lot of them in Fedora. Desktop environments, for example, are so designated. In cases where a non-critical-path package bundles a library, packages would be required to publicly contact the upstream community about fixing the bundling issue; if upstream declines to cooperate, then the bundled libraries could be shipped without a special exception.

This is the version of the proposal that was taken to the Fedora engineering steering committee (FESCo) for approval. When the minutes from the October 7 meeting were published, though, it became clear that what had emerged was a bit different. The policy approved by FESCo is relatively simple, reading, in full:

All packages whose upstreams allow them to be built against system libraries must be built against system libraries.
All packages whose upstreams have no mechanism to build against system libraries must be contacted publicly about a path to supporting system libraries. If upstream refuses, this must be recorded in the spec file using a persistent mechanism to be clarified in the packaging guidelines.
All packages whose upstreams have no mechanism to build against system libraries may opt to carry bundled libraries, but if they do, they must include Provides: bundled(<libname>) = <version> in their RPM spec file.

This text, which, among other things, entirely drops the "critical path" language, was arrived at in the meeting itself; it was not posted for discussion prior to its approval. That has led to charges that FESCo is acting in haste to try to resolve an issue that has been discussed for many years. It looks, to some, like a move to take some power away from FPC in response to a decision that some on FESCo didn't like, and a vote for packaging more software regardless of quality concerns. It would seem, though, that much of the community has accepted the view posted by FESCo member Adam Jackson after the vote:

Policies that are out of line with reality are bad policy: the war on drugs does not fix drug abuse, vagrancy laws do not fix poverty, and the war on bundling merely ensures that bundled software goes unreported. We should acknowledge that bundling is a real thing that solves real problems for both app developers and end users, we should codify it in our policies, and we should build the tools that enable us to track and manage it rather than pretend it doesn't happen just because a package passed review once.

So now the policy has been adjusted; the part about building the tools to track and manage bundled software, needless to say, will take somewhat longer.

Changing roles

In the end, this policy change may well reflect a realization that the role of distributors is changing and that distributions are becoming less central. The prominence of non-distributor packaging mechanisms is increasing; most language communities have such a mechanism, for example. Upstream projects are increasingly interested in shipping their software directly to users without having middlemen in the way. And the real elephant in the room, arguably, is containers, which, by their very nature, are large blobs of bundled software.

Distributors have been an important actor in the free-software ecosystem; they see to the needs of users and provide an integrated, supported system for easy use. The lessening of their role may thus not be without its costs. But, if distributors are able to adapt to the changing software-distribution environment, they should be able to provide a lot of value for a long time to come. The policy change in the Fedora community is an attempt to adapt in this way. Whether the new policy will be successful remains to be seen, but one can easily argue that what came before was not.

Comments (162 posted)

Brief items

Distribution quotes of the week

We just plain don't have the clout to use inclusion in Fedora as a lever to fix this. And I don't see the benefit to Fedora in excluding software which uses them from the Fedora universe. I mean, if we insisted on perfection, the distribution would consist solely of software written personally by Donald Knuth.

-- Matthew Miller

Well, following the beginner's guide, I finally got a system up and running in a VM! I'm not sure what clicked with me this time (I honestly attribute it to using nano instead of vi this time), but I am officially ready to try installing Arch on my personal laptop! I'm very excited to start breaking my system, and to spending many late, late nights getting it working again! I'm a tinkerer at heart, and Arch just screams my name.

-- r3djak

Comments (4 posted)

Announcing NetBSD 7.0

The NetBSD Project has announced the release of NetBSD 7.0. Some highlights include DRM/KMS support to bring accelerated graphics to x86 systems, multiprocessor ARM support, support for many new ARM boards, network packet filter (NPF) improvements, and much more.

Full Story (comments: none)

Distribution News

CentOS

CentOS Linux 7 for x86 (i386) architecture

The CentOS AltArch Special Interest Group has released CentOS 7 for the 32-bit x86 (i386) architecture. "This is the first major release of the 32 bit x86 by the AltArch Special Interest Group. This release is based on the Source Code from the CentOS 7 (1503) x86_64 architecture and includes all current updates from the main CentOS 7 tree."