Looking a Novell gift horse in the mouth [LWN.net]

Novell's February 7 press release proclaiming its contributions to the X.org and GNOME projects was generally well received. It is hard to disagree with better graphics and more fun eye candy, after all. Novell's work shows that the free software community has the potential to take the leadership on desktop issues, and that is a good thing. Free software desktops will only take over the world if the community can produce a desktop experience that is truly better than the alternatives.

The issue that has come up in some quarters, however, is that of "community." Developers in the wider GNOME community, in particular, are feeling somewhat excluded from the process. Novell's work, to them, is not a community development - it's a product which has emerged in complete form from a corporate cathedral. While it is great that Novell is doing this work, they say, wouldn't it have been better to involve the community from the outset? Now community members are in a position of reviewing a large drop of code that they had no part in designing, and not all of them are happy about it.

If the words of Novell's Dan Winship are representative of the company's position (he claims to be speaking only for himself), Novell believes it has taken the right approach:

If we had proposed the changes on the mailing lists, it would have started a huge discussion about what people hated about the design ("you can't make the panel menu depend on beagle!!!") and how it should be different. And then we could have either (a) completely ignored everyone and done it ourselves anyway, or (b) had a long conversation about the merits of the design and then not actually finished the code in time for NLD10.

So we did it ourselves, and now either GNOME will like what we did, in which case, yay, free code for GNOME, or GNOME won't like what we did, in which case, no harm no foul for GNOME, and yay, brand differentiation for Novell.

Dan goes on to say that it simply is not possible to perform software design in a community setting. Everything good which has ever been done in the GNOME project has been the result of a small group's work. All big community debates tend to do is to slow down or stop the process. "Design by committee," says Dan, does not work.

GNOME hacker Jeff Waugh ~~disagrees~~ does not want to give up on community involvement in design:

This is a very sorry state of affairs for GNOME. But it is not only Novell and its employees who have adopted this commons-sapping, community-tearing, morally and intellectually lazy approach to open design and development in GNOME.... Ultimately, this is *killing our community*. And it must be fought.

One useful perspective in the discussion came from Alan Cox, who made a distinction between "design by community" and "design in the community." The latter approach leaves the bulk of the design work in the small group which is most interested in it, but which recognizes that the community may have something to add. When design work is taken out of the community altogether, something is lost:

If you design stuff in secret then publish it, it will have no review of quality, no style checking, no security audit, no extra pairs of eyes and extra brains on it. Mouths are in oversupply but brains/eyes are not.

Jeff agreed, and went on to compare GNOME development with how the Linux kernel is managed:

While the Linux process has its warts, there are two things it is great at that we should mention here: First, a fairly easy to understand technical and social leadership - decisions get made. Second, a pretty uncompromising approach to design in the community - it's really hard to drop a pre-cooked hairball (cat hair *or* angel hair) into the kernel process without getting roasted, spanked and harshly reviewed.

If, from this particular perspective, the kernel process is seen as being more successful than GNOME, it might be worth asking why. It is indeed true that the kernel community responds poorly to large piles of code which appear out of the blue. Often, such code must go through substantial revisions before it will be considered for merging - the community gets its say in the end after all. The reiser4 experience is a good example; the new version of the Reiser filesystem showed up in complete form, with a request for expedited merging. Numerous problems were found with the code, however, and reiser4 remains out of the kernel years later, even with numerous fixes made and features removed. In the kernel space, most developers learn, sooner or later, to involve the community early in a project's life.

The leadership issue is worth a look as well. As Jeff pointed out, the kernel has a relatively clear decision-making process, though it can be frustrating for contributors to work with. Discussions tend not to go on forever because, in one way or another, decisions get made. Instead, says Jeff, GNOME is "without coherent leadership." He would like to see the GNOME project structure reworked so that decisions are easier to make - though what form those changes would take is yet to be worked out.

Another issue, raised by Havoc Pennington, is the vision the project has for itself. The GNOME project, says Havoc, needs to come up with a better idea of who its user community is and to not be afraid to lose users who are outside of that community. When the project has a clearer sense of what it is trying to do, decisions will be easier to make. The kernel project knows what it is trying to do:

They are writing a component for use by developers, not an end-user product. And they aren't ashamed of it and they optimize for it and they do it well.

Havoc suggests that GNOME might want to take a similar approach: create a series of components which can be rearranged and customized by distributors and gadget-makers to fit their specific needs. Such an orientation would let GNOME focus on making the best tool possible while allowing others, who are arguably closer to the ultimate users, to make the desktop fit those users' needs.

That leads to one of the driving forces behind this entire debate. To a great extent, companies distributing Linux (or products incorporating Linux) tend not to differentiate their offerings with kernel features. Distributors do add kernel patches, but the size of those patches has gone down considerably with the advent of the 2.6 development process. This is an important point: the development process change has had the effect of significantly reducing the differences between distributors' kernels. But user interface changes are visible to all who work with a system in a way which most kernel changes are not. Distributors will thus always have a strong incentive to put their particular mark on the desktop and to try to have the coolest features first. So, at best, we are likely to see more desktop work done in relative secret until it is deemed ready to be shipped. At worst, we could see a repeat of the highly tweaked desktops shipped during the worst of the proprietary Unix days.

Distributors have strong reasons to differentiate their offerings, but they also depend heavily on projects like GNOME to provide the foundation on which they can create those offerings. Taking much of their development semi-proprietary may help sales in the next year or so, but that could happen at the cost of eventually tearing apart the community upon which they depend, even if they do not necessarily respect its design guidance. If GNOME is to remain healthy well into the future, these two forces will have to be reconciled. The solution will likely involve a combination of project governance changes and a more community-oriented approach by all participating companies. This should be something the community can achieve.

(Log in to post comments)

Looking a Novell gift horse in the mouth

Posted Feb 8, 2006 21:15 UTC (Wed) by cventers (subscriber, #31465) [Link]

Yeah, but I think that it's the GNOME guys' attitudes (not their
underlying concern) that is at fault.

Huge code drops are bad.

Waugh's comment about "killing the community" makes it sound a lot like
he feels an inherent obligation to swallow anything anyone tries to feed
him. If it were the kernel, they wouldn't be saying "you're killing the
community," rather, the author(s) would be potentially chastised after
which their product would be subjected to a slow and careful review
(speed actually dependant somewhat on how much interest there is in
feature Y).

So I think that there is a problem when development of a critical
component of open source like Xgl goes behind closed doors for a while.
But I don't think Waugh's comment was at all justified.

Looking a Novell gift horse in the mouth

Posted Feb 8, 2006 21:32 UTC (Wed) by jdub (subscriber, #27) [Link]

It's worth considering that my comments were not specifically about Novell. They were about a compounded set of problems that have become assumptions in our community over a long period of time - unfortunately, these things come to the fore when examples present themselves.

Novell are doing brilliant work, and making awesome contributions to the Free Software desktop and GNOME in particular - they should receive full credit for funding and driving this work.

The problem for GNOME is broader than the immediate examples discussed, and without any value judgement implied about the contributions being made.

Looking a Novell gift horse in the mouth

Posted Feb 9, 2006 0:09 UTC (Thu) by cventers (subscriber, #31465) [Link]

Well, whether or not it has anything to do with Novell and XGL, the point
I was trying to make is simply this. If the GNOME community feels
threatened by large code drops and private development efforts, the GNOME
community's response should simply be to not tolerate it. If you choose
not to tolerate it, then it is immediately of no threat to your community
(unless, of course, you used a license like the BSD license that would
allow someone to take your code and run).

Don't 'whine' about it to Novell or anyone else. Tell them that you
appreciate their great contribution, but that huge code drops aren't easy
to merge. It may be that the only way to make your point is to let time
pass and let the problem illustrate itself.

This, by the way, is why the occasional flames from people like Linus are
exceptionally valuable - because it puts forth a strong, commanding
position. And it's clear that the effect this has had on Linux is that
companies interested in Linux development *adapt* to support its
procedures.

I remember a comment Andrew Morton made in an interview. I'm
paraphrasing, but basically he gave an example of how IBM management
adapted. The IBM bosses couldn't simply tell their engineers "We need
feature X in the kernel by Friday". The picture Andrew painted was such
that when IBM management asked for something objectionable, the answer
they got back wasn't "the kernel guys won't accept this," rather, "we
won't accept this".

Of course, they wouldn't be empowered to say such things if that weren't
made possible by good open source management. Are you taking notes?

Hitting the nail on the head: LEADERSHIP

Posted Feb 9, 2006 10:33 UTC (Thu) by hummassa (subscriber, #307) [Link]

If this were to be happening in the kernel code, Linus would use his
CommandVoice(TM): "go away, and don't come back unless it's with a lot of
small, each individually well-explained patches. Then, and only then,
we'll think about it". Instead, the GNOME guys are (IMO) just whining:
"oooh they are destroying our community". Oh, come on.

Hitting the nail on the head: LEADERSHIP

Posted Feb 9, 2006 11:34 UTC (Thu) by smitty_one_each (subscriber, #28989) [Link]

>guys are (IMO) just whining
Look at the resulting advertising! Brilliant!

Looking a Novell gift horse in the mouth

Posted Feb 9, 2006 16:18 UTC (Thu) by GreyWizard (subscriber, #1026) [Link]

Please don't suppose everyone on LWN agrees with the semi-coherent rant to which you have responded. I suspect the silent majority have read what you actually wrote and are considering the issue rather than inventing straw men in your image. At least some of the people whining about your message (on a charge of "whining" -- oh the irony) can be counted on to post angry nonsense on just about any article that mentions GNOME or GTK+. Why they feel qualified to educate anyone about community project management is something of a mystery.

Looking a Novell gift horse in the mouth

Posted Feb 9, 2006 18:04 UTC (Thu) by pimlott (guest, #1535) [Link]

since [Novell] gave the code away, Gnome can take its time to review and integrate at their pace.

This is harder than you make it sound. Incorporating a piece of independently developed code, so that you can understand and maintain it, while retaining a coherent design, is a significant project. Especially as the original developers may not have much incentive to help--they've already shipped, and probably gone on to the next project. So a code drop puts the GNOME developers into a difficult position: Either accept the code as is, creating a less maintainable code-base; go through the painful process of incorporating it gradually; or reject valuable funtionality. If the code were developed in the open, they wouldn't be pushed into this corner.

I don't think that Jeff's comments were hyperbole. This is how fragmentation happens.

Do or Do Not Do: Either is Fine

Posted Feb 11, 2006 6:00 UTC (Sat) by AnswerGuy (guest, #1256) [Link]

They can look at the code; and decide if there are bits of if that they
want. They can decide that there are big enough chunks with clear enough
interfaces that they can be integrated into their work.

Or they can decide that it's too hard and that it'll be easier to just re-write.

Or they can decide that the features aren't sufficiently compelling.

Probably what's going to happen is that very "theys" will differ in their conclusions and some of them will go off in various directions. Some of the code will be merged; other bits won't --- some will be buggy and some bugs will be discovered and fixed during the merging process.

There will also be those who spend more time whining and pontificating than actually code or even reading code.

JimD

Perhaps the Apache model can be useful to Gnome?

Posted Feb 8, 2006 22:00 UTC (Wed) by fredrik (subscriber, #232) [Link]

First I must confess that I know zilch of the decision making process in the various Gnome projects today, so I may be barking up the wrong tree entirely. Having said that...

I make my living developing J2EE solutions, so I tend to spend a lot of time on the Apache developer mailinglists. And the Apache projects seems to have a neat solution to neverending mailing list design debates.

The Apache decision process AFAIU works like this: Proposals on modification of an arhcitecture, changes in roadmaps, and adding of new contributers, are brought up in the mailing list. Then everyone who is involved or affected can join in and discuss the merits and drawbacks of the proposal. Finally a formal open vote is performed on the list, people simply respond to a vote with +1 or -1 (with some variations). Most decisions are more or less taken in consensus, since most objections often have been delt with already in the proposal stage. My impression is that most proposals don't linger for ever either - quite the opposite - so the Novell guys fears of not meeting their deadlines seems moot too.

Now, of course there are exceptions, one of my favourite projects is Cocoon. And right now they seem to be lost in some serious decision making anguish, and noone seems to be able to put their foot down either. So, the Apache model may not solve every management issue, but still I think it can serve as a role modell for how to breed open source community thinking into committers that mostly comes from a corporate closed source world.

Community vs developers

Posted Feb 8, 2006 23:22 UTC (Wed) by elanthis (subscriber, #6227) [Link]

I'd also like to quickly add that I myself am one of the over-opinionated whiny users. Possibly one of the worst. ;-)

That's why I unsubscribed from all of the GNOME and FreeDesktop mailing lists a year or two back. I was doing a lot more mouthing off than I was doing real contributing, and I was part of the problem.

The real developers would get a lot more (good) work done if more of the non-developers would follow my lead and censor themselves.

Community vs developers

Posted Feb 9, 2006 0:19 UTC (Thu) by cventers (subscriber, #31465) [Link]

I agree totally with your sentiment, but disagree with the mechanism.
There's no reason to have "read-only" mailing lists. What you need is a
strong community like LKML where badness is immediately and brutally
gunned down.

Be one hundred thousand percent blunt about every real point you make.
Don't beat around the bush. Don't whine, and then the people who choose
to whine will stand out from the crowd.

Often, LKML 'protocol violations' if you will only have to be dismissed
by one or two people. The rest of the developers just ignore it. This
comes from having a strong sense of "this is the way we do development".
If that's your attitude, who can argue?

What I was getting at elsewhere in the thread about Novell and XGL is
simply this. It's fine to ask questions and get upset when open source
development goes closed.

But you do that personally. When you're wearing your Linux dev hat, or
your GNOME dev hat, your approach to the problem is simply "I am
uninterested in development that doesn't take place at our pace. I am
grateful if they decide to come submit patches / join up, but if they do
that, they'll need to understand that there will need to be allowances
made for the way we do things around here."

So, in closing, have STRONG personalities like Torvalds. If a whiner
enters your list with some idea that is outright wrong, don't entertain
it for very long. The number one attitude to remember is "SHOW ME THE
CODE." Then your community won't be hostile to fast-paced development.

Community vs developers

Posted Feb 9, 2006 1:17 UTC (Thu) by elanthis (subscriber, #6227) [Link]

"So, in closing, have STRONG personalities like Torvalds."

Unfortunately, you can't really manufacturer people like that. ;-)

Looking a Novell gift horse in the mouth

Posted Feb 9, 2006 10:15 UTC (Thu) by kleptog (subscriber, #1183) [Link]

Interesting article. A bit odd though, especially the comment about how you don't want to post design because people complain with comments like:

"you can't make the panel menu depend on beagle!!!"

Huh? That's not a design issue, that's an implementation detail. A design is more like saying we take layer K out of the X server and put it into a library that depends on OpenGL. Rationale is Y.

The design should be aimed at people who are knowledgable about the program (in this case the X server). What you put forth is an overview about what you want to acheive and which parts are likely to be affected and let people comment on it. If you don't like the responses, you can ignore them. You don't need to have a vote, if you get good ideas, incorporate them. If you don't, don't.

In this sense it is good to have seperate mailing lists for users, advanced users and hackers. Users are end users, advanced users are getting familiar with the code and hackers is where the real hacking happens. The last group will be quite small as most people have no idea how the server works. This is the group your design should be aimed at. The other groups are not relevent to the discussion because they don't have the background.

(Just in case people think I'm discriminating against users, most things that get discussed are design changes that have almost zero impact on the end user. Things might get faster or more flexible, but the end result isn't going to look fundamentally different. The end users in this case are designers of applications for X, not the end users of gnome. As long as source code compatability is preserved, they don't need to care.)

Anyway, just my 2c.