Big decisions loom for Fedora [LWN.net]

The Fedora Project is in one of those relatively rare periods where the deadlines have passed, the distribution has been shipped, and no new deadlines have yet been set. Now is the time when participants in the project can engage in a bit of introspection, and that's exactly what is going on. Over the next week or so, decisions will be made which could significantly change the way this project works.

For some background, readers may want to look at this posting from Thorsten Leemhuis and Max Spevack's state of Fedora note. The developers involved with Fedora seem to think that the Fedora Core 6 process went well, and that, as a result, FC6 is a solid distribution. They are justifiably proud of their work. That said, there are a number of issues on the Fedora developers' minds, and a number of changes which, seemingly, need to be made.

To that end, the Fedora Project Board will be meeting on November 7. The real discussion, however, will happen at a special "Fedora Summit" happening from November 11 through the 15th. It is a closed affair, featuring Max Spevack, Greg DeKoenigsberg, Bill Nottingham, Chris Blizzard, Warren Togami, Dave Jones, Jeremy Katz, Jesse Keating, and perhaps various others at times. This group of people will try to make a plan for the development of Fedora Core 7 and the future organization of the project.

Since its inception, Fedora has been criticized for not being as open to the community as its early PR had led people to hope. Much progress has been made in that direction over the last year or so, but much remains to be done. Greg DeKoenigsberg is quite clear that making the project more open is a priority, and that the time has come:

We've got a lot of work to do inside the fenceline, though. Honestly, a lot of that work requires the disentanglement of Fedora and RHEL -- we need the ability to innovate freely in Fedora without adversely impacting RHEL. We didn't really have that opportunity in the FC6 timeframe.

But now we do.

From the resulting discussion, it would appear that one significant decision has already been made, at least in principle: the Fedora Core distribution, as such, will be abolished. Fedora Extras has been sufficiently successful that it increasingly looks like the model for Fedora as a whole in the future. There does not appear to be any dissent to this idea; the hot topic, instead, seems to be how the new distribution will be named. "Fedora Linux" appears to be the leading choice at the moment.

But, then, nobody has really gotten down to discussing - in public, at least - how the new, more open Fedora will work. There will still have to be a decision-making mechanism, a way for setting the goals and priorities for the project. Red Hat is still picking up most of the tab for work on Fedora, so there are still likely to be limits to how much latitude the company is willing to give the project to set its own priorities. A good place to start might be to establish the Fedora Steering Committee - first promised in 2003 - with a significant number of outside contributors and let it provide some direction (in the open) for the project as a whole.

Another topic for the discussion is the future of the Fedora Legacy project, which was discussed here last month. It appears that the project has finally come to see Fedora Legacy - or its absence - as a problem. How that problem will be solved is far from clear at this point, however. Another nagging problem is the ongoing maintenance of rpm; that, too, looks like it may be addressed by the board meeting and the summit.

Then there are issues like the ongoing lack of a Fedora live CD. Desktop support is getting more attention, though it is hard to see how Fedora can address many of the complaints in this area (lack of official Java, flash support, etc.) while remaining true to its "free software only" rules. Making a source code management system available to the wider community remains on the "to do" list. And so on.

In other words, Fedora has a lot of work to do, still, before it becomes a truly open, community project. Nothing illustrates that better than the fact that the directions and priorities for the next Fedora release will be set in closed board and summit meetings. What seems different now is that the project insiders appear more determined than ever to get this work done. For all that Fedora is a great distribution, it needs its community to continue to grow and reach its potential. Given all that needs to be done to become more open to its community, Fedora is likely to still be very much a work in progress by the time the Fedora Linux 7 (or whatever it is called) is released. But, then, that is true of a great many free software projects.

(Log in to post comments)

What's the big deal?

Posted Nov 7, 2006 0:48 UTC (Tue) by drag (subscriber, #31333) [Link]

What if somebody started a Linux distribution and nobody downloaded it?

I don't think that there is a problem with being a beta release of Redhat, but I don't think that is what actually Fedora realy wants.

Sure a few years ago Fedora would be considured kick-ass in terms of it's structure and such, but I think that with Ubuntu people's expectations have risen. So now you have situation were people are migrating away from Fedora instead of TO Fedora. These users, the community, is a absolute nessicity for creating a top-notch Linux distro.

Red Hat should know

Posted Nov 7, 2006 1:06 UTC (Tue) by man_ls (subscriber, #15091) [Link]

That is not how Red Hat is selling Fedora:

The Fedora Project is a Red Hat sponsored and community supported open source project. Its goal? The rapid progress of Free and Open Source software and content. Public forums. Open processes. Rapid innovation. Meritocracy and transparency. All in pursuit of the best operating system and platform that Free software can provide.

(Emphasis mine.) So Red Hat is definitely not comfortable with Fedora being just a "beta version". Maybe because people do not usually like to work for Red Hat for free, but they cooperate freely with community distros.

Also, it has been speculated that the failure of Fedora Legacy is related to Red Hat's failure to position Fedora as a community distro. As a public company Red Hat would have a hard time justifying support for a free distro; but many community distros do that aspect just fine.

Red Hat should know

Posted Nov 7, 2006 3:28 UTC (Tue) by davej (guest, #354) [Link]

it has been speculated that the failure of Fedora Legacy is related to Red Hat's failure to position Fedora as a community distro.

And that speculation would be way off base.
The facts are that backporting fixes to older releases is boring as hell work, and very few people want to do it.

it's not like it's difficult to get involved with Fedora legacy, it's just that there are way more consumers than developers. Which is to be expected, but when you only have a handful of people actually turning out updates, it doesn't take long until they become overloaded.

Before a transition to Fedora legacy, there's almost a 1:1 mapping between package:maintainer. After the transition to legacy, there's a handful of developers maintaining the whole distro. This clearly, doesn't scale.

Legacy inadequacy

Posted Nov 7, 2006 8:04 UTC (Tue) by man_ls (subscriber, #15091) [Link]

I don't even use Fedora; I just follow the news on LWN. Let me point out, from my utmost ignorance about Fedora (and most other things), the obvious flaws in your arguments.

The facts are that backporting fixes to older releases is boring as hell work, and very few people want to do it.

Indeed, but other community distros do just fine with volunteers.

Before a transition to Fedora legacy, there's almost a 1:1 mapping between package:maintainer. After the transition to legacy, there's a handful of developers maintaining the whole distro.

It is not the same task at all; Legacy's stated mission is to "provide security and critical bug fix errata packages", not to correct every bug. It should be orders of magnitude less work.

[...] when you only have a handful of people actually turning out updates, it doesn't take long until they become overloaded.

No fixes since July for FC3 and no fixes at all for FC4. This does not look like overloaded developers, but complete paralysis.

In support of my argument: if people saw working for Fedora Legacy as doing the grunt work for Red Hat for free, then they would not be willing to do it. Witness this quote from Jake Edge:

Fedora Legacy is a great idea, but appears to suffer from a lack of participation from the community.

Or (at the risk of being pedantic, but consider the imbalance in this conversation) this one from Jon:

Perhaps the time has come to ask the question: is there any point in continuing to pretend that Fedora Legacy is a viable, successful project?

Legacy inadequacy

Posted Nov 7, 2006 8:14 UTC (Tue) by bojan (subscriber, #14302) [Link]

It's probably very simple: people that want a free of charge, stable Red Hat flavoured distro go with CentOS, the ones that want a bleeding edge one pick FC. So, the FL is mostly uninteresting, therefore nobody maintains the ever growing number of versions there.

Legacy inadequacy

Posted Nov 7, 2006 14:22 UTC (Tue) by mrshiny (subscriber, #4266) [Link]

I agree; long-term stability isn't what Fedora is looking for. But personally, I wish they supported FCX until FCX+2 was released, not FCX+2test2. This small gap leaves users vulnerable who want to upgrade slightly less frequently.

Legacy inadequacy

Posted Nov 7, 2006 17:37 UTC (Tue) by man_ls (subscriber, #15091) [Link]

I don't agree. I have seen lots of complaints about volatility of Fedora. CentOS on the desktop is probably not a good choice; OTOH Fedora's forced upgrades are not nice.

I use Ubuntu 6.04; 6.10 is already out there, but my current desktop is working fine, so why change it? When I feel like it (or need it) I will take the time to upgrade; meanwhile I appreciate the support provided by Canonical, in this case (LTS version) exceptionally long. I would expect Fedora users to feel more or less the same; otherwise why would anyone have started the Legacy project?

Legacy inadequacy

Posted Nov 7, 2006 22:20 UTC (Tue) by bojan (subscriber, #14302) [Link]

Obviously people that think that Fedora's forced upgrades are not nice, like yourself (and there is nothing wrong with that - I'm just making an observation), already picked another distro, which is about 3 orders of magnitude easier than maintaining Fedora Legacy.

Initially, people thought that FL will be the answer to what RHL used to be. In order words a free, "forever" maintained, recent, Red Hat flavoured distro. It turned out that the intersection of:

- has to be Red Hat flavoured
- has to be free
- has to have updates coming
- has to be more recent than RHELx

Didn't have that many interested users/developers. Therefore, people picked different distros, including CentOS, FC, Ubuntu, Debian etc. And FL ended up being unmaintained after this initial reaction to FC/RHEL split cleared up.

Granted, there is still occasional "don't keep changing so fast" on various Fedora lists, but most people understand by now that the Fedora is life in a fast lane.

And once RHEL5 gets released and CentOS 5 gets built, the current level of interest in FL will drop by a few notches again. It's just a natural reaction to the fact that there is no point in doing the hard work that someone already did.

Legacy inadequacy

Posted Nov 7, 2006 22:57 UTC (Tue) by sbishop (guest, #33061) [Link]

Exactly. Thank you, bojan. That's exactly what I was getting at in my initial post.

Legacy inadequacy

Posted Nov 19, 2006 19:55 UTC (Sun) by dag- (subscriber, #30207) [Link]

Actually, CentOS is fine for the desktop. Since it's released every 18 months you may want to check for hardware compatibility or delay until the next release to support recent hardware.

But CentOS 4 as a desktop is perfect for anyone except maybe technical people that desire to have the latest and greatest technology.

For everyone else, stability, reliability, low maintenance, low-risk non-disruptive updates and long term support (7 years) is exactly what one needs.

What's the big deal?

Posted Nov 7, 2006 7:26 UTC (Tue) by csamuel (✭ supporter ✭, #2624) [Link]

For my line of work (HPC clusters for scientific computing) Fedora is
actually better to use than RHEL, which is too straight-jacketed (cut
down & old kernels) and their (admirable) aim of being a long-term stable
release for servers means it is often too old to be of much interest.

The single exception to that is commercial software where the ISVs often
stipulate RHEL or SLES (even though their code usually works quite
happily with Fedora, just don't try and tell them that).

Structural impediments

Posted Nov 7, 2006 14:02 UTC (Tue) by brugolsky (✭ supporter ✭, #28) [Link]

Fedora has accumulated a number of structural impediments that make certain improvements difficult, and keeps the average sysadmin mired in operational issues, rather than incremental improvements to Fedora that can be pushed back upstream.

Top of the list, methinks, is RPM/yum. RPM provides five things: a pristine source + patches build technology, a container for bits, versioning, dependency management, and scriptlets that are triggered on install/update/removal.

When managing more than one host, or virtual machines on a host, RPM shows its weaknesses compared to Conary, or other potential solutions. As a Fedora user, I keep downloading the same bits, because the contents of the (N+1)th release of a package is often nearly identical to the Nth release -- especially if that package has "font," "langpack," or "i18n" in its name! RPM makes it unnecessarily difficult to maintain a set of hardlinked chrooted environments, or virtual disks, snapshots, etc., because it churns the filesystem. Because sharing an image is inordinately difficult, I have prelink turned off, so that identical packages installed on different hosts have identical files.

On the package building side, the spec file format suffers from all the traditional problems of poorly implemented macro languages and policy, and has discouraged abstraction and higher-level tools (such as the use of the various fedora-rpm* tools). Code is replicated hundreds of times, with slight variations (e.g. $RPM_BUILD_ROOT v. %{buildroot}, "make install" v. %makeinstall) all throughout the distro, rather than being centralized into recipes for GNOME apps, KDE apps, kernel modules, ... Packages are littered with scriptlets that only work in the forward direction, and there is no easy way to hook it to do otherwise, because all of the scriplets are open-coded.

Maintaining local versions of packages is a nightmare. Rather than being able to specify that I want my build to be the same as the upstream build, except that I want my locally-maintained path series applied, and different options to configure, instead I have to hand-edit the spec file every time I roll a package forward. On FC4, I have 95 locally built packages, most of which involved some hand-editing, some to apply patches (which have been rejected or dropped upstream, for whatever reason), others to fix up things like SELinux and XFree86/Xorg dependencies/build-options from Fedora development packages.

Another problem is network config. The initscripts that Fedora inherited from Red Hat Linux are more suitable to the BSD-like Linux-2.0 view of the world. Alexey Kuznetsov and others added a mind-boggling array of networking functionality starting with Linux-2.2 (circa Red Hat Linux 6.x timeframe), and it is largely useless in Fedora without lots of custom scripting. For the most part, the initscripts package does nothing useful with: secondary IP addresses, policy routing, packet marking and filtering, and traffic control and QoS. NetworkManager addresses some of these issues for wireless users, but various important pieces are missing. The /etc/net framework provides an improved static configuration, but there is as yet no dynamic network management framework that would provide the simple interface of a Cisco router, despite the Linux kernel being more featureful than Cisco IOS in many regards. [Quagga offers a limited subset of Cisco CLI functionality, but because it is cross-platform, does not solve the whole problem of interface and link management, etc. The commercial version of Zebra in ZebOS includes some of the desired functionality.]

Also on my hit list is the sorry state of LVM snapshotting. Things have only stabilized in the last six months, and the current snapshot implementation is very slow and only suitable for taking backups. Running with multiple snapshots brings a fast block device to a crawl. The wandering log technology in Reiser4 is interesting from an in-filesystem snapshot perspective, but that appears to be going nowhere fast. One can only hope that something suitable will find its way into Ext4.

I am hopeful that the intense interest in virtualization, as well as the needs of OLPC will spur Red Hat and the Fedora Project to address the need to make Fedora easily configured and managed, and readily branched for development, customization, or isolation.

Interesting post

Posted Nov 7, 2006 16:03 UTC (Tue) by scottt (subscriber, #5028) [Link]

I summarized the problems you raised as follows:

update download size is large since Fedora does not use delta RPMs esp. font langpack and i18n packages
RPM and prelink make it hard to maintain identical program binaries for multiple hard linked chroot environments or disk images
The spec file format that control RPM building is a terrible macro language. The same scriptlets gets copy-and-pasted into many packages and there's no way for the sysadmin to replace the scripts system wide.
maintaining local packages that apply custom patches and pass different compile time options to configure should be easier.
network configuration: secondary IP addresses, policy routing, packet marking and filtering, and traffic control and QoS.
LVM snapshots are slow.

I agree with most of your comments.

For problem 1, my understanding is that the Fedora project would not use an update system with that only downloads binary deltas like firefox's because their mirrors would not like to run any custom server programs.

I agree that the spec language sucks whole heartedly. Here's an example, bug 159792 . Dave Jones wanted %patch to use a "fuzzy level" switch of "-F1". Costomizing the behavior of "patch" and "configure" should be easy. Then there's the one that rpmbuild expands macros even in the changelog section of the spec file.

I believe conary has a way to deal with the cut-and-paste scriptlet problem by tagging a GNU info file or gconf spec file as such and providing a way for the sysadmin to specify the scripts to install them.

Prelink provides valuable address space randomization protection for most systems and I think Fedora is right to have it on by default.

For maintaining custom packages, there's the additional problem of making sure your package doesn't get overrided by anything that comes from the update repository.

Structural impediments

Posted Nov 7, 2006 17:50 UTC (Tue) by PaulDickson (subscriber, #478) [Link]

Actually, this has been solved in Sylpheed's spec file for several years. All I have to do is:

SYLPHEED_CONFIGURE_FLAGS="--disable-ipv6" SYLPHEED_REL_DIST="1mypkg.fc6" rpmbuild -tb --clean sylpheed-2.2.7.tar.bz

Structural impediments

Posted Nov 9, 2006 3:51 UTC (Thu) by proski (subscriber, #104) [Link]

The solution should be across the board, not just for one package. Besides, I don't think you cannot specify additional patches on the command line, even for Sylpheed.

Structural impediments

Posted Nov 9, 2006 10:26 UTC (Thu) by nix (subscriber, #2304) [Link]

LVM snapshots aren't terribly useful for backups either because snapshotting / may deadlock. (It's rare, but it can happen, especially when under significant memory pressure.)

The last time I looked, the LVM docs didn't actually *say* that snapshotting the filesystem(s) on which /sbin/lvm and /etc/lvm.d reside is unsupported :/