Shuttleworth: On cadence and collaboration

Hi folks

I've stayed quiet in this discussion, though several folks have invoked my
name and ascribed motivations to me that were a little upsetting. I'm not
responding to that here, instead I'd like to focus on what we can achieve
together, and how we can lead a very significant improvement in the health
of the whole free software ecosystem.

Apologies in advance if this mail is lengthy and not particularly witty!

Imagine you are the leader of a key upstream component. You care about
your users, you want them to appreciate and love the software you write.
But you also know that most users won't get the code from you - your code
will land in most users hands through one or other distribution - maybe
RHEL, maybe Fedora, maybe Debian or Ubuntu or Gentoo. And you can maintain
a few personal relationships with distribution-space that help to
straighten things out, but more often than not, users will get your code
from a distribution with whom you have little contact. To make things
worse, at any given time, different distributions may be shipping wildly
different versions of your code. That makes all the bug reports harder to
evaluate, and all the patches harder to apply. It also makes it harder to
know where to commit precious resources to stable version maintenance.

I hear this story all the time from upstreams. "We'd like to help
distributions, but WHICH distribution should we pick?" That's a very
difficult proposition for upstreams. They want to help, but they can't.
And they shouldn't have to pick favorites.

Adopting a broad pattern of cadence and collaboration between many
distributions won't be a silver bullet for ALL of those problems, but it
will go a very long way to simplifying the life of both upstreams and
distribution maintainers. If upstream knows, for example, that MANY
distributions will be shipping a particular version of their code and
supporting it for several years (in fact, if they can sit down with those
distributions and make suggestions as to which version would be best!)
then they are more likely to be able to justify doing point releases with
security fixes for that version... which in turn makes it easier for the
security teams and maintainers in the distribution.

We're already seeing a growing trend towards cadence in free software,
which I think is a wonderful move. Here, we are talking about elevating
that to something that the world has never seen in proprietary software
(and never will) - an entire industry collaborating. Collaboration is the
primary tool we have in our battle with proprietary software, we should
take the opportunities that present themselves to make that collaboration
easier and more effective.

OK, so that's the theory. How do we get there? How do we get many
distributions to sit down and explore the opportunities to agree on common
base versions for major releases?

Well, the first thing is to agree on the idea of a predictable cadence.
Although the big threads on this list are a little heartbreaking for me to
watch, I'm glad that there hasn't been a lot of upset at the idea of a
cadence in Debian so much as *which* cadence. We can solve the latter, we
couldn't solve the former. So I'm happy at least at that :-)

The second thing is to find the opportunities that are most likely to be
successful. That depends as much on psychology and practical interaction
as anything else.

As pointed out on this list, Debian and Ubuntu share a great deal. We have
largely common package names (imagine what a difference that will make to
practical discussions over IRC ;-)) and we have established relationships
between folks who care about most of the major components already. We have
lots of people with shared experience in both projects (most of the
strongest Ubuntu contributors are or have been very strong Debian
contributors too, and many new Debian maintainers have come to the project
through Ubuntu). When I look over the commentary on debian-devel and in
debbugs and on #debian-devel, I see a lot of familiar names from Ubuntu,
especially on the deep, hard problems that need solving at the core. I'm
proud of that.

So, practically, we would be in a good position to collaborate.

Psychologically, I don't know so much. Have you ever noticed how family
disputes can be the most bitter? Or how neighboring countries that share
the same food, the same dress, the same values, can often be the bitterest
feuds? Some days I think that applies between us. I see mails on this list
saying it would be easier and better for Debian to coordinate with
distributions that I think would be almost *impossible* to work with
practically, but somehow they are more attractive because they are not
family. Perhaps we know each other too well. It's hard to be a prophet in
your home town.

How do I think it could work in practice? Well, if Debian and Ubuntu went
ahead with the summit in December, where we reviewed plans for 2010 and
identified opportunities to collaborate, I think we would get (a) several
other smaller distributions to participate, and (b) several upstreams to
participate. That would be a big win. It would set us off on a good
course. If we delivered, then, we would virtually guarantee that almost
all the distributions and key upstreams would participate the next time
around. And if *that* worked, we'd win RHEL over too.

A December summit is not about tying anybody's hands. It's about looking
for opportunities, where they exist naturally, and communicating those
more widely. At the moment, if we happen to ship the same version, it's
partly an accident, and upstream doesn't know about it till afterwards.
With an effort made on reviewing and thinking about it, we should get much
better information and communicate much better. Which is a win, right?

So, I'd like to address some of the comments and ideas expressed on this
list recently.

First, there has been no secret cabal or skunkworks effort to influence
Debian. As best I can tell, folks from both Debian and Ubuntu who have
deep insight into release management established a shared interest in
working together better, at many levels, and this was one idea that came
forward. The fact that those discussions were open and ongoing was no
secret - I wouldn't have talked about it in the media if it were!
(Ironically, someone suggested that the fact that I was talking publicly
about something in Debian implied there was a secret cabal. Aiieee.)

I have always tried to make sure that I speak regularly with the DPL -
some DPL's have not responded to that at all, others have been happy to
speak. Steve and I have spoken about every quarter, which is great, and we
focus those conversations on ways we can make collaboration better.
Finding teams we can introduce to one another. Finding ways to communicate
better. This was again, one of the things that came up, as was the idea of
a joint sprint on boot process, which was very successful.

In both cases, the individuals and teams concerned have a mandate from
their organisations to think problems through and speak for the project.
Large organisations can't work any other way. I was stunned when I saw the
announcement of a "decision" because I know that Debian works by building
steady consensus (and by small groups who Just Do It now and then, but
that won't work on something like this). I had expected there would be
more of a proposal for discussion. As far as I can tell, that's what
happened at DebConf, but the announcement afterwards was abrupt. A pity,
because the discussions have been colored by the perception of an imposed
decision, when they needn't have been.

Second, this is not about Debian changing to meet the needs of Ubuntu.

As I've said elsewhere, Ubuntu would be happy to reach a compromise if
needed to work with Debian and others. I think there's agreement on a two
year cadence, and if needed we can change one of our cycles to help bring
multiple distributions into line. Alternatively, with Debian specifically,
we can contribute resources to help Debian meet a stretch (or squeeze ;-))
goal. From my perspective, committing Canonical employees to help Debian
freeze in December, or stretching our one cycle to get us both onto a two
year cadence, are roughly the same. It would be unreasonable to expect us
to do BOTH of those, but I'm happy to work one either basis. Compromise
requires some give from both parties, though.

But most importantly, this whole thing will have it's best and biggest
impact if it goes beyond Ubuntu and Debian. The debate on this list has
mostly been about "Ubuntu vs Debian", which misses the real goal: let's
send a signal to upstreams that they can participate and help shape the
way end users will experience their software. To do that, we need to get
multiple distributions. And looking at it that way, a December summit
gives us a much stronger ability to influence multiple distributions that
are planning releases in 2010. Based on the feedback from the Debian
release team that they liked the idea, I've been reaching out to other
distributions to try to get more of them together. This gets much more
powerful the more of them we bring to the same forum. I'm saddened that
the aggressive tone of this debate has thrown the exercise into question -
I think largely because of unfortunate communications after the
discussions at DebConf. C'est la vie.

Third, I think we need to call on the people who are not fundamentally
prejudiced to speak out.

I see many mails on this list from people who are clearly absolutely
certain in their minds that "Ubuntu is an evil thief of Debian's work". I
don't see any way to change their minds. No matter how many positive
examples of effort made by Ubuntu folks to collaborate we find, they will
always find examples that reinforce their view. If that view dominates the
discussion, we can never improve the situation, because that view says
"don't bother doing anything different, don't look for opportunities to
collaborate, don't make any offer to compromise". How can we achieve
anything from that basis? Debian is made up of hundreds of contributors,
many stay silent. I'm saddened that the loudest voices seem to be those
who are vociferous in their opposition to Ubuntu, rather than those who
are finding ways to make things better. I'm saddened that a good idea - a
sounder basis for collaboration, backed by real investment and effort -
gets crushed on the rocks of hate from folks who do not make the bulk of
the contribution.

It's not hard to tell if someone is expressing an opinion based on
prejudice or one based on openness. Anyone who says, definitively, that a
whole organisation or hundreds of people is "bad", is making a
generalisation that can only be harmful to relationships. I enjoyed Linus
Torvalds' recent interview where he talked about prejudice against
Microsoft in the Linux community, and how poisonous it is. The same is
true of prejudice against Ubuntu here in Debian.

There are very good people, with long histories in Debian, who have
pointed out the positive things that have come from Ubuntu. Listen to
them. Ubuntu is in a great position to help with big and deep changes that
need to be made. Look at the folks who have been instrumental in
discussions around multi-arch, or the move to event-based booting, for
recent examples, and you'll see people who work effectively in both
projects. Neither project can claim credit for all the good work that goes
on, but it would be very wrong to make a sweeping statement that Ubuntu
makes no contribution.

As for my motivations - I love free software and want it to win. If it
wins properly, it will not come in a single package branded "Debian" or
"Ubuntu" or "Red Hat", it will come in a coordinated diversity. I have no
interest in seeing anything bad happen to Debian. Quite the reverse, I've
acted in the ways I thought would carry the greatness of Debian into new
places in the most effective way possible. I'm sorry that some folks have
responded to that as if it were a threat, and sought to create divide and
disharmony. I stayed away from DebConf this year - the first time in six
years - because I didn't want to be a flashpoint for division, when there
are so many positive threads of collaboration under way. I hope this mail
doesn't turn into a magnet for flies and pus. If it suits your brain to
think that I'm an evil capitalist thieving pig, so be it, I doubt there's
anything I could do to change your mind. Paranoia will only get you so
far. But if you're open minded, take the time to look again at what I've
done and said, and ask if it's made a positive difference to Debian and
free software in the last five years, and make your own mind up.

Rather than turning this into a debate on the integrity of individuals or
organisations or projects, let's look at how to make a big improvement in
the free software ecosystem.

In summary (and thank you to anyone who made it this far :-)):

To achieve anything together, we'll both need to work together, we'll need
to make compromises or we'll need to contribute effort to the other side.
If the Debian community is willing to consider a December freeze, then
Ubuntu (and Canonical) will commit resources to help Debian meet that
goal. It means we'll get less done in Ubuntu, but the benefits of having a
schedule which could attract many other distributions would outweigh that.
I think multiple other distributions, who tend to think in financial years
(2010) and plan accordingly, will join a December freeze summit, and there
are significant benefits to Debian to being part of that rather than on a
different schedule.

This is a good faith offer of help and support in order to reach a tough
but noble and achievable goal. It won't be easy, the first time or the
next, but it will kickstart a process that will bring dividends to Debian,
and to the whole broader ecosystem. Ask upstreams what they think, and
whether they would want to participate, and you'll hear a very positive
response.

Mark