CentOS and Red Hat

CentOS board member Karsten Wade, who is also Red Hat's engineering manager for CentOS, came to the 2014 Linux Foundation Collaboration Summit to explain how CentOS and Red Hat are joining forces and what it means for the future of both. Back in early January, the two announced that CentOS was joining the Red Hat family. According to Wade, that was the completion of one-and-a-half years of effort to put the two together, but it was just the beginning of actually figuring out what the partnership means.

Wade is a longtime Red Hat employee (12–13 years) who was part of the early days of Fedora. More recently he has worked on more "meta" open source things as part of the Open Source and Standards team. That includes projects like "The Open Source Way" and other, similar initiatives.

A bit of history

When Red Hat made the decision to split Red Hat Linux into Red Hat Enterprise Linux (RHEL) and Fedora, it knew it was leaving some users behind. The company wanted a fast-moving distribution, as well as one that was commercially supportable. That split led to RHEL rebuilds, and CentOS was the most prominent of them. In fact, he said, CentOS might have a bigger user base than RHEL and Fedora combined; no one really knows for sure.

There are some overlaps, but for the most part there are fairly distinct use cases for Fedora, CentOS, and RHEL. Fedora always has the latest and greatest packages; it tends to be ahead of the curve in comparison to other distributions. The differences between CentOS and RHEL are not as sharp. CentOS has probably been used to "beat down" the Red Hat salespeople on the price for RHEL support, Wade said, but the key is that CentOS provides a base that doesn't move quickly and is not going to break (itself or applications running on top of it) on the next update. For open source development, it is sometimes important to have that stable base, he said.

CentOS was interested in working with Red Hat so that it would have more resources both to create that stable base and to add new things on top of the base. Wade said he understands Red Hat's motivations better because he has worked there so long; he is a longtime CentOS user, but a new member of its community. Red Hat knows that the open source development model works, and not just for R&D.

Ultimately, Red Hat is trying to grow its user base. For many years, Linux itself was an interesting platform, but we are moving away from that, at least in some places. For example, OpenStack is the platform of interest to many today—what's underneath is much less interesting. The RDO project is a Red Hat effort to make it easy to get the latest OpenStack code up and running on Fedora, CentOS, and RHEL. Users of RDO just care about OpenStack for the most part.

CentOS had some of the same problems, but were viewing them "from the other side", Wade said. He met CentOS project leader Karanbir Singh at a Puppet conference in 2012. At that point, Wade knew that Red Hat saw the need to have a community distribution with a slow moving base in addition to the fast-moving Fedora. He knew that the company would either create its own rebuild or join forces with the leader, but he couldn't say anything to Singh at that point.

Wade talked with Singh and Fedora project leader Robyn Bergeron for several hours at that conference. One thing he learned was that CentOS had run into a problem when RHEL 6 was released: RHEL no longer shipped the Xen virtualization system. But CentOS had lots of users who wanted to continue using Xen, so CentOS teamed up with Xen to put the virtualization solution on CentOS 6. In that case, the vast majority of core CentOS is still used, but there are some minor tweaks (including a different kernel) atop the stable base.

So the goal for CentOS is to create a next-generation platform that is supported for a longer period of time than Fedora is. Ten years would be good, but most people just want something longer than 13 months, he said, and 2–3 years seemed to be a sweet spot. There will be CentOS variants that mostly ship code from the CentOS repositories, with some differences, and, importantly, with a special interest group (SIG) to feed and care for it. Variants can differentiate at different levels and could include things like variants for storage, cloud, or the Mate desktop.

Bringing CentOS into Red Hat is also meant to help defragment the Linux ecosystem, Wade said. RHEL Rebuilds were proliferating, so he is hoping that some of that energy can be pulled into CentOS by creating some "gravity" around the project. In the end, CentOS is something like the third leg of the stool for Red Hat, and it was needed for Red Hat to grow its communities.

Post-announcement

Many of the organizational and governance decisions about CentOS moving forward were deferred until after the announcement so that they could be made in public. Every other board meeting is now held in Google Hangouts so that people can see what's going on (and put faces to the participants). One of the more prominent community rebuilds, Scientific Linux, has been participating as well. It has been invited to join the community, and a board seat is being saved for it, but they are currently "talking among themselves" about how to work with CentOS. It is possible that it could become a SIG or that its community may want to keep its autonomy; time will tell, Wade said.

SIGs started cropping up right away. Around a dozen have been discussed, but the board wants to see broad SIGs, so it would rather there be a storage SIG instead of a Ceph SIG, for example. Four SIGs have been accepted so far, including core (the base distribution), virtualization (both in CentOS and of CentOS), and cloud instances (creating CentOS images for various cloud providers). More will be coming soon, he said.

Historically, there have been perceptual and actual barriers to participation in the CentOS project. Wade has something of a personal crusade to lower those kinds of barriers, and CentOS won't be any different in that regard. Singh has said that CentOS never really was an open-source project because it didn't really produce any software, but that is changing with SIGs and variants. There is a lot of "pent-up energy" for CentOS to be an open-source project, Wade said, and that energy will start to move into the project now.

Instead of roughly one-and-a-half full-time people on the project, there will now be five. But there will also be support from the Open Source and Standards group at Red Hat, so that the team can focus on technical problems rather than having to order T-shirts or organize conferences. The team is firewalled from the RHEL team, however.

One of the outcomes of that firewalling is that CentOS will get its own build system, which will provide visibility and access to users. It will likely be based on Fedora's koji build system. There will be two parallel build systems, actually, one for the core SIG and one for all of the other SIGs. To start out, the release process for SIGs will be somewhat akin to the Apache model: SIGs will demonstrate that the release is sound and it will be signed and released. After some time, the SIG will be able to sign its own releases. One other thing to note: it will be possible for people outside of Red Hat to join the core SIG, though it will likely be a rather arduous process.

All in all, Wade painted a highly optimistic picture for the future of CentOS and Red Hat. It could be argued that CentOS already had a bright future, but it should be able to do a lot more with Red Hat's help. He also did not leave Fedora out of the picture, as it is an integral part of the family, complementary in many ways to both RHEL and CentOS. Fedora effectively becomes the upstream for many of the add-ons that will be shipped in the CentOS variants, for example. While it may have seemed a bit puzzling at first, CentOS joining Red Hat seems to make more and more sense over time.

[ Thanks to the Linux Foundation for supporting my travel to the Collaboration Summit. ]