One of the mini-confs which happened ahead of linux.conf.au proper was the
"distribution summit," meant to be a place where representatives and users
of all distributions could talk about issues of interest to all. The
highlight of this event, perhaps, was Jeff Waugh's talk on
disintermediating distributions - or, as he rephrased it, "distributed
distributions." If his ideas take hold, they could be the beginning of a
new relationship between free software projects and their users.
It all started, says Jeff, some years ago, when he ran into Mark
Shuttleworth fresh from a visit to Antarctica. Mark's pitch, says Jeff,
"sounded like crack" at the time. By 2003 or so, it just didn't seem like
there was a whole lot of room for a new distribution. But Mark had some
interesting ideas, and Jeff signed on; the result, of course, was Ubuntu.
Ubuntu has clearly had some success, but, in some important ways, it has
failed to work out - at least for Jeff. He found himself distracted by Ubuntu's lack of
participation in Debian, from which it derived its product. There was
a real tension between tracking Debian and tracking upstream projects
more directly. Despite Jeff's insistence that Ubuntu should be tracking
(and pushing updates into) Debian's unstable distribution, Ubuntu often
chose to go with upstream, resulting in what is, in effect, a fork of the
Debian distribution - in terms of both the technology and the community.
What Ubuntu was doing was taking upstream packages, modifying them,
bringing in shiny new features, and generally looking for ways to
differentiate itself from the other distributors. So, for example, the
first Ubuntu release contained a great deal of Project Utopia work (aimed
at making hardware "just work" with Linux) which had been done by
developers from other distributions; Ubuntu shipped it first, though, and
got a lot of credit for it. Novell's behind-closed-doors development of
Xgl was motivated primarily by the wish to keep Ubuntu from shipping it
first. Meanwhile, Red Hat had slowly learned that trying to differentiate
itself by diverging from upstream was a path to pain. So Red Hat's
developers created AIGLX,
in an open, community oriented manner; the result is that AIGLX has proved
to be the winning technology.
Events like these led Jeff to wonder about just where the integration
of packages should be done - upstream or downstream? From Jeff's
(GNOME-based) upstream point of view, he wonders why he doesn't have a
direct relationship with his users. While most projects deliver their code
through middlemen (distributors), there is an example of a project which
has managed to maintain a much more direct relationship: Firefox. Most
Firefox users are direct clients of the project - though most of them are
Windows users. The Firefox trademark has been used to ensure that, even
when distributors are involved, the upstream developers get a say in what
is delivered to users.
So, what happens if you take out the middleman? It's instructive to look
back at what life was like before there were distributors. It was, Jeff
says, much like pigs playing in mud; perhaps they enjoyed it, but it was
messy. There are, in fact, a lot of good things that distributors have
done for us. You can get a fully integrated stack of software from one
source, and the distributor acts, in a way, as the user's advocate toward
the upstream project. We don't want to lose out on all that.
But, if one were to look at facilitating a more direct relationship between
development project and their users, one would want to take advantage of a
number of maturing technologies. These include:
- OpenID. Any process of distributing distributions must look at
distributed identity, and OpenID is the way to do it.
- DOAP. "Sounds terrible" but it's a useful way of describing a project
with XML. With a DOAP description, a user can find a project's
mailing lists, bug tracker, source repository, etc.
- Atom. This is how projects can distribute information about what they
- XMPP. This is a Jabber-based message queueing and presence protocol.
It can be used to more active publishing of information than Atom can
- Distributed revision control. Lots of functionality for integration
between projects, and between upstream and downstream. Jeff sees git
as a step backward, though; some
of the other offerings, he thinks, have much better user interfaces.
Also important are the packaging efforts which are underway in a
number of places. These include Fedora, which is "becoming competitive
with Debian" as a community project. OpenSUSE has put together a build
system which can create packages for a number of distributions. Debian has
had a community build system for years; there is interest in Debian in
going the next step, though - ideas like building packages directly from a
distributed version control system. Ubuntu's Launchpad was "a spectacular
vision," though the reality is "a bit of a snore"; it didn't achieve its
goal of helping upstream and downstream work together.
Then there's Bugzilla, which is the "bug filing gauntlet" between projects
and their users. The Debian bug tracking system has done a better job of
facilitating bug reports by
allowing them to be submitted by email. But most big projects are
using Bugzilla. It would be much improved by using OpenID (so that users
would not have to register to file bugs) and some sort of Atom-based feed
which would make querying bugs easy.
If you take out the distribution, what do you replace it with? How do we
achieve consistency? We need to create standards for how we interact with
each other. And we can, in fact, be very good at consistency and standards
when the need
is clear. Good release management is a step toward that goal. GNOME once
had very bad release management, but has pulled it together. Doing
time-based releases was a hard sell, but few developers would want anything
else now. Now GNOME release management just works.
Consistency in source management is needed. Once upon a time that was done
through CVS, but CVS is no longer up to the job, and now every project is using
a different distributed version control system. But, sooner or later, one
of the competing projects will win out and "hopefully we'll have clarity
again." Autotools and pkgconfig can also go a long way toward creating
consistency between projects.
So, if we can push the available tools up into the upstream projects, those
projects can get better at producing packages for distributions themselves.
Once the tools (like bug trackers) can talk to each other, people will
start making more use of them and network effects will take over. But, at
the moment, the knowledge about integration remains at the distribution
Debian, Jeff thinks, is well placed to take on a project like this
and push its integration knowledge upstream. While Debian has typically
been ten years ahead of everybody else in its packaging and integration
abilities, it currently has a "relevancy problem." Finding ways to help
upstream projects support their users more directly while maintaining
overall integration and consistency would be a perfect way for Debian to
maintain its leadership in this area. That could change the game
for everybody, bringing projects closer to their users and making us all
"happy as pigs in mud."
to post comments)