The hard side of the Bazaar [LWN.net]

The "Bazaar" style of project management, as described by Eric Raymond and typified by the Linux kernel development model, is undoubtedly effective at producing quality software, at least in some situations. It can also, however, be a harsh environment in which to operate, as demonstrated by events in the kernel community over the 2.5 series, and especially over the last week.

Readers of the LWN.net weekly Kernel Page will have been following the development of the IDE/ATA layer in the 2.5 series for some time. For the rest, here is some quick background to provide context for the rest.

The IDE layer, of course, is the low-level code that handles the disk (and CD) drives found on most Linux systems. This code operates under a number of serious constraints. It must be fast - able to drive the hardware at its maximum speed; the performance of a Linux system as a whole is highly dependent on how fast its disks can go. It also must be absolutely correct; users get grumpy when their data is lost or corrupted. And it must deal with a wide variety of, um, "inexpensive" hardware that does not always behave as the documentation and standards say it should. Hacking on the IDE subsystem is not for the faint of heart.

In recent times the IDE maintainer has been Andre Hedrick. Andre has had numerous communication problems with Linus (and others) which have made it difficult for him to get patches into the kernel. It is also fashionable in certain quarters to criticize the quality of Andre's code. But, it should be said: Andre's IDE layer has proved, over time, to be rigidly standards compliant and highly reliable.

Andre's inability to get patches into the kernel left a void in the 2.5 series, however. That void was filled by Marcin Dalecki, who started posting his "IDE cleanup" patches back in February. The "cleanups" began to look increasingly like a complete rework (and hostile takeover) of the IDE code, and, with IDE 18, Marcin put his name into the MAINTAINERS file.

Marcin's work has been controversial all along - especially after he started removing features that people were using, and when the IDE layer started breaking for some users. His approach was not subtle, and he seemed untroubled by the concerns of the other Linux kernel hackers. After all, said Marcin, "Breakage is the price you have to pay for advancements."

Linus, for the most part, seemed to agree; he merged almost every patch from Marcin through IDE 115, posted on August 9.

All this changed on August 16, when Linus, without fanfare, deleted the entire 2.5 IDE subsystem and replaced it with the "foreport" of the 2.4 IDE layer, done by Jens Axboe and others. The word from Linus is that Marcin got tired of all the criticism and quit; Marcin, himself, has been silent since then. It is telling, though, that Linus responded by simply deleting and replacing the entire body of 2.5 IDE work, rather than trying to find somebody who would continue that task. Either Linus came to agree with other kernel hackers about the quality of the reworked IDE code, or he concluded that nobody else would be willing to work with that code.

The end result is that six months worth of Marcin's work, in the form of 115 IDE patches, has just been dumped into the bit bucket.

And that is an example of the harsh side of participating in the kernel bazaar. One can work for months, see that work apparently accepted, then have it vanish in a moment. Linus has said numerous times that the doesn't much care about the feelings of kernel hackers; he is far more concerned about the quality of the code. This approach may well be part of why Linus is a good manager for Linux development - in the end, the code quality must remain high or the whole thing will collapse under its own weight. But it also explains why kernel hackers occasionally get frustrated and leave the kernel development community. The bazaar can be fun and effective, but it's not always nice.

(Log in to post comments)

Is it bazaar specific?

Posted Aug 22, 2002 11:24 UTC (Thu) by nenieorg (guest, #3378) [Link]

I guess a large proportion of code produced within commercial
and/or "cathedral" projects never sees production, either for
small portions of code, or on a larger scale when the whole
project fails or gets cancelled for internal or external
reasons, and that's far from an unusual occurrence.

Of course, on commercial projects programmers still get paid
before the project is cancelled, but the open source
participant can still get a lot of fame (or notoriety)
especially when having worked on such a core subsystem
of the Linux kernel. And the code is still there, so
some of it will get reused one way or another if it's
any good.

The public humiliation is

Posted Aug 22, 2002 15:52 UTC (Thu) by cwong15 (guest, #3013) [Link]

In a closed source environment, you can see code cancelled, and you can receive criticism. But on open source projects, you can get flamed publicly on mailing lists. Since a lot of the motivation for contributing to open source is public recognition, this sort of public humiliation tends to negate any payoffs. Not only do open source developers not get a paycheck to compensate for the grief or lost work, but a public wrecking of their reputation can be detrimental to their rent-paying careers. Email shows off one's worst side. An interviewer might try doing a Google search of a job seeker before an interview. Think about it: how would you like to walk into an interview and be asked about a supposed reputation of being anti-social, arrogant, careless, incompetent or not being "a team player"? That is something Marcin might have to deal with.

Is it bazaar specific?

Posted Aug 23, 2002 15:03 UTC (Fri) by alexr (guest, #2752) [Link]

Several years ago, I spent months working on a localization project for
an accounting system. When a lot of stuff was fully translated, the customer
decided to cancel the project.

This happens with cathedral/commercial projects. The good side is that you
(usually) still get paid. The bad side is that you spent so much time
working on stuff that nobody will ever use. With free software/free docs,
your work is still available for reuse; you may hope it's not fully wasted.