Modularizing Fedora

In that model, distributors are the gatekeepers; they gather together a bunch of software, make it all work well together, and ship it on a regular schedule. After that, they support what they have shipped for some period of time while working on the next big release. This mode of operation seemed logical in the early days of Linux, but it is an awkward fit in a world of ever more complex software stacks, faster release schedules, and a desire within projects to deal more directly with their users.

One particular pain point is the disconnect between any given distribution's release cycle and the release cycles of any projects of interest. Waiting for the next major release from a distribution, which could happen either months or years in the future, can be a problem for impatient users. This is especially true for developers building stacks of software; waiting for a distributor to update the lower-level dependencies in such a stack is often just not an option. So, for a while now, distributors have been looking for ways to stay more relevant for their users, whether it be moving to a rolling model, adding "app stores," jumping on the container bandwagon, or something else.

Back in 2013, Fedora launched an initiative called "rings," which was an attempt to partition the distribution into layers, some of which could move more quickly than others. Splitting the distribution into three editions was one outcome of this effort, one which seems to have worked out reasonably well. But much of the hoped-for benefit of rings has not really materialized. As the Fedora modularization wiki page points out, "it seems that the simple metaphor of concentric rings doesn't actually work very well for our increasingly messy open source software world." Modularization is an attempt to address that messiness more directly.

A "module," in short, is a collection of software that serves a specific purpose, carries its own dependencies, is developed and tested as a unit, and is shipped together. Modules can have their own lifecycles that do not match up with that of the distribution as a whole. Beyond that, the definition gets a bit fuzzy; the modularization page adds text like:

The hope is that, with software chunked into modules in this way, it can be more easily supported and updated independently of the rest of the distribution. There is some clear enthusiasm behind the idea, as is evidenced by the Fedora Council's March 14 decision to approve the modularization effort as an official Fedora project.

The current plan, as described in this page, is to have a prototype implementation available at the time of the Fedora 25 release, so that it can be demonstrated in early 2017. That means that much of the initial work must be done by Fedora 24, due in June of this year. Needless to say, that is an ambitious schedule.

So what will be done for this prototype? The developers provide the diagram to the right as a way of describing what they are aiming for, but readers may not find it all that enlightening. Once those developers manage to put away their diagram editors, they plan to make a working prototype, "functional enough that people can use it, understand it, hammer on it, and improve it", in the next year. The number of actual modules is likely to be small, consisting of a "core" module and a handful of applications. There should also be the various mechanisms in place to make modularization work: a way to automatically test modules, a delivery setup, and more.

One interesting issue has to do with the plan to have each module contain its own dependencies. Having modules move at different speeds means that they will certainly, at times, need different versions of shared dependencies, so each module is likely to have its own. This starts to sound a lot like the bundling of dependencies, a practice that Fedora (like most Linux distributors) has fiercely resisted. This blog post by Langdon White (who is taking the lead on much of this work) states that "we need to be able to define an 'application' (or module) without resorting to bundling", but the RPM package-management system does not currently respond well to attempts to install multiple versions of the same package. So it will be hard to install dependencies in a way that they can be shared between any modules that can use the same versions.

That is just one of the many tooling issues that the modularization developers will have to solve, but it looks to be one of the more difficult ones. Other problems include things like security updates; patching a dependency will require creating new modules that use that dependency and testing them all. Some modules may not work properly with the patch applied, leading to further version skew.

All told, it looks like a significant technical challenge to make all of this work. But the benefits, in the form of a more amorphous, flexible, and useful Fedora, may well be worth it. Langdon has indicated that Red Hat is putting a number of developers on the job, so the necessary resources might just be available. One way or another, the community as a whole is likely to learn more about the problem of running a distribution in the year of the 25th anniversary of Linux.