Ten-year timeline part 6: almost to the present

Part 5 of this increasingly long series stopped in March, 2004, when BitMover loudly proclaimed that the use of BitKeeper had doubled the pace of kernel development. This installment picks up from there, looking at a year when BitKeeper remained in the news, the SCO case was in progress, software patents became more threatening, and more.

• April 8, 2004: The first X.org release. SELinux shows up in a Fedora Core 2 test release. Red Hat v. SCO is put on indefinite hold (where it remains to this day). Anti-software-patent demonstrations are held in Europe.

This week featured some important news. The launch of X.org signaled the resurrection of Linux desktop work and the beginning of a much more interesting and promising era. Meanwhile, Fedora took the lead in pushing SELinux-based mandatory access control technology into a general-purpose system. That work is still very much in progress nearly four years later, but, like it or not, SELinux has become an important part of our defensive arsenal.

• April 15, 2004: The 2.6.6 kernel gains POSIX message queues, filesystem speedups, internal API changes, laptop mode, 4K stacks, auditing, the CFQ I/O scheduler, and more. Sun and Microsoft make a $2 billion deal. Lindows becomes Linspire.

• April 22, 2004: Linspire files to go public. BayStar tells SCO it wants its money back.

• April 29, 2004: Gentoo founder Daniel Robbins leaves the project.

Something else which was going on during this time was a rising level of discontent over the management of the Fedora project, which was not turning out to be the open community that many had hoped for. Pause for a moment and revisit this classic dialog posted by Konstantin Ryabitsev, which so clearly documented how the situation was seen by the community at that time. Fedora has come a long way since then.

• May 20, 2004: The European Council approves the software patent directive, sending it back to the Parliament for final passage.

Remember: the directive approved by the Council was the original version which legitimized software patents, not the version amended by the Parliament which did not. Thus started the final (so far) round in the fight against European software patents - a round which we eventually won.

• May 27, 2004: The kernel adopts the Signed-off-by: convention. The 2.6.7 kernel gains scheduling domains, the object-based reverse mapping VM, filtered wakeups, and more.

The thing to remember here is that 2.6 was alleged to be a stable kernel series, and everybody was still waiting for 2.7 to start. Linus defended the massive VM changes with the claim that they were, in fact, an "implementation detail." The realization that the kernel development process had, in fact, already changed did not come through until...

• July 22, 2004: The "new" kernel development process is adopted.

This kernel summit decision - which, among other things, said that there would be no 2.7 kernel - surprised almost everybody. Certainly there have been some issues since then, but nobody really wants to go back to the old, pre-2.6 days.

• August 5, 2004: Open Source Risk Management funds a study showing that the kernel infringes on 283 patents, offers patent suit insurance. SCO Forum is held, featuring a keynote by Rob Enderle; the rest of the world looks on incredulously. The Munich Linux deployment is put on hold as a result of software patent fears.

• August 19, 2004: Lindows gives up on its IPO. The 2.6.8.1 kernel is released.

There were interesting cross-currents happening at this time. On the one hand, companies like Open Source Risk Management were trying to use SCO as a way to scare companies (and individual developers) into buying its insurance offerings. On the other, there was a hallucinogenic aspect to the SCO Forum discussions that escaped nobody; SCO's time of being taken seriously by the wider world was already done.

It's worth noting that OSRM still exists, but its insurance offering now is for companies worried about GPL-infringement suits.

Meanwhile, 2.6.8.1 was the first three-dot kernel release ever; it was rushed out in response to an unpleasant, last-minute bug in 2.6.8.

• August 26, 2004: IBM brings GPL-infringement charges against SCO. LWN fails to reproduce the posted reiser4 filesystem benchmarks, gets in trouble with Namesys.

• September 16, 2004: Sun announces plans to open-source Solaris. OSDL and the Free Standards Group announce a plan for cooperation on the Linux Standard Base.

OSDL and the FSG were, at this point, separate groups which, at times, almost seemed to be in competition with each other. Those days, of course, are no more: the two have since merged and become the Linux Foundation.

• September 23, 2004: the Ubuntu distribution announces its existence.

Who would have thought that one could create a major new distribution in 2004? One might well wonder whether the situation is any less open now.

• October 7, 2004: the bnetd developers lose their DMCA case. Concerns about kernel quality are expressed. Microsoft's FAT patent is overturned.

• October 14, 2004: Novell says it will use its patents "as appropriate" to defend free software projects against patent attacks. Jeff Merkey offers $50,000 for the right to take the kernel proprietary. The realtime preemption patch set gets started.

• October 21, 2004: the first Ubuntu release (4.10) comes out. Busybox 1.0 is released at last. Mozilla begins fund raising to advertise Firefox in the New York Times.

• November 11, 2004: Firefox 1.0 is released. Novell gets $500 million in anti-trust cash from Microsoft.

The Firefox 1.0 release was, in a very real sense, the much-delayed culmination of the process which began back in 1998, when Netscape announced that it would be releasing its code. Firefox was almost seven years in the making, but, sometimes, late really is better than never. Even those of us who use a different browser should be thankful for the effect Firefox has had toward the creation of a standard-compliant web and a competitive environment for web browsers.

• November 18, 2004: the Linux Core Consortium is formed by Conectiva, MandrakeSoft, Progeny, and Turbolinux.

• December 2, 2004: MandrakeSoft turns a profit.

Whether it's called United Linux, the Linux Core Consortium, or Manbo-Labs, this is an idea which returns on occasion: pool effort on the creation of a base distribution so that each player can concentrate their differentiation efforts on the higher levels. It often seems not to work, though. It is hard to compete with more community-based distributions through the establishment of a base platform by corporate fiat. It seems that the true "base" distributions have names like Debian or Fedora.

• January 13, 2005: Debian runs afoul of the Mozilla trademark policy. The European Parliament attempts to restart the software patent discussion from the beginning.

• January 27, 2005: Sun starts releasing Solaris code under the CDDL.

• February 3, 2005: The Software Freedom Law Center is founded. Eben Moglen starts talking about GPLv3. Russ Nelson becomes the president of the Open Source Initiative - briefly.

• February 10, 2005: IBM's requests for summary judgment in the SCO case are dismissed - temporarily - by Judge Kimball. BitKeeper flame wars return, this time about the locking-up of history metadata and license-based prohibitions on its extraction.

The locking-up of metadata within BitKeeper was a sore point even for developers who had accepted BitKeeper in general. Larry McVoy was unsympathetic, though, stating that he was operating within his rights. This episode was the beginning of the end for BitKeeper and the kernel.

• March 3, 2005: MandrakeSoft acquires Conectiva. The European Commission ignores the European Parliament's request to restart the software patent directive process.

• March 10, 2005: Kernel quality concerns lead to the creation of the -stable tree.

Those quality concerns are not gone now, though they have diminished somewhat. The -stable tree seemed like an experiment at the time, but it has proved successful and is still being produced almost three years later.

• April 7, 2005: The BitKeeper era comes to an abrupt end when the free-beer license for the software is terminated by BitMover. (Unfounded) rumors about a merger between UserLinux and Ubuntu circulate.

• April 14, 2005: Linus posts the first version of git. MandrakeSoft becomes Mandriva.

The termination of free-beer BitKeeper was probably inevitable from the very beginning of its existence; trying to maintain a closed system with proprietary data formats in the middle of a highly open process was always a losing proposition. For some time, many of us had feared that it could end in a much uglier way than it actually played out. We, the community, had danced on some thin ice for a while, but, when it broke, the water was only ankle-deep. We got lucky.

As your editor has said before, BitKeeper did us a lot of good by bringing order to the kernel development process when things had been working very poorly, and by showing the world what distributed revision control could do. It set the stage for what came after. Git was not the first free distributed revision control system, but it was the first to be employed on such a massive scale. In a real sense, git launched a new era of free software development.

On that note, this article will end - and, probably, the retrospective series ends as well. As events become more recent, the difficulty of putting them into historical perspective gets greater. A retrospective covering the remaining 2+ years risks becoming a repeat of the annual timelines and adding little of value. That period is best left for the 20-year retrospective.

So, the entire LWN staff would like to say "thanks!" one last time to our readers, who have treated us so well for the last ten years. It has been an incredible ride.

---

to post comments

git was, for me, final proof of the `release early, release often' idea. 
The initial git was very nearly unusable by mortals and was certainly far 
too disk-space-inefficient to actually be used for a project with a churn 
rate as high as the kernel for long.

But because the *representation* was right, the other problems could be 
solved later (and were, yet fast enough that nobody's disks filled up): 
and a whole bunch of really nasty ones that had bedeviled other VC systems 
for ages just ceased to exist, like rename tracking (what? you need to 
track renames? why not just search for similar content when you pack? that 
way you can merge stuff that's similar whether or not it originated in a 
rename.)

And it keeps improving. My kernel git repo actually uses *less* space now 
than it did in the 2.6.18 era because of repacker improvements...

It continues to bother me that Linus didn't give credit to Monotone in his git release
announcement.  Git at its inception was, as I understand it, a clone of a subset of monotone,
perhaps with the addition of some tweaks that I haven't understood.  That's fine -- it's a
wonderful thing to draw ideas from other people and fit them into your needs, and to extend
them to work better.  It's more wonderful when you give them credit -- that's a useful part of
the scientific process, and people should treat it as a moral obligation to do so as well as
they can.

Instead Linus still trash-talks Monotone, even though its implementation is completely
different than when he raided it, and he (evidently) hasn't looked at it since.

Reference, please?

Oh by, Graydon Hoare -- author of monotone -- posted, at the time, his summary of his
discussion with Linus and reasons why he thinks Linus rejected monotone:

http://www.mail-archive.com/monotone-devel@nongnu.org/msg...

I greatly appreciate the way Graydon is precise and to the point while also being soft-spoken
and charitable towards others.  Not to put too fine a point on it, but I find myself
noticeably happier at the prospect of reading something Graydon has written than something
Linus has written.

A related story to "Linus rejects monotone" is "Linus rejects Mercurial".  There is an
interesting thread on lkml about that.  Here is a climactic point where Linus seems to be
wavering about adopting Mercurial instead of git:

http://lkml.org/lkml/2005/5/3/2

I couldn't find the next inflection point -- where Linus decided to keep git.

I think maybe the last word on the subject belongs to the long-lost ntk.net:

http://www.ntk.net/2005/04/29/

"Given the surfeit of next generation systems - including darcs, codeville, arch, monotone,
bazaar, bazaar-ng, vesta, svk, ArX, aegis, we suspect that the winner will be git, just out of
the Mighty Power Of Fanboyism."

:-)

Well speed was a very very high priority for Linus with Git. His opinion is that it puts a
different dynamic on a feature if it's slow vs fast. That is people will take much more
advantage of features and use them in creative ways if they are fast, and if they are very
slow it makes them very much less useful in real world situations.

Seems like he felt that people were caring about features and code structure (ie being a bit
to academic) rather then concentrating on making quick and essential functionality.

I donno. Stuff your guys are talking about are probably much improved but git exists now and
it's gotten popular so I doubt there is a reason to switch now.

I don't know how this compares to other things like Monotone, BK, or any other 'third
generation' version control system else like that, but it is also important for him that each
developer can have his own private tree local for his own work. That there is no enforced
hierarchy, no secondary players in git-land. No centralized anything. Everything is
distrubted. Each personal repository is on equal footing with everybody else's and all you
essentially have is a multitude of separate git trees that can share code equally. 

There is a lovely talk here:
http://video.google.com/videoplay?docid=-2199332044603874737


Also keep in mind that jives and insults matter different in different contexts. Friends often
are very insulting to each other were I am from, it's actually quite friendly because they
know that they can behave like that and trust each other not to take it personally. That is as
long as it stays 'good nature'. Reassuringly friendly sometimes. (this does not mix well with
alcohol, though. Not at all) 

I notice that often people don't understand that and they come from a place were politeness
and careful attention to social sensitivities is very important.  This is fine, it's just
different. Depending on context this can be also mis-interpreted and lack of trust/empathy and
taken sort of indicator of a person with a  self-superior attitude.

It's not really that important, but it's something to keep in mind I guess. Of course Linus
can be abrasive and he is proud of it, so lots of his BS just should be ignored completely.

You're wrong, but it doesn't matter. The only thing that really matters -- to anyone outside
those few of us mildly offended by being misrepresented -- is that the idea, and some
implementation of it, is now spreading like wildfire.

The fact that git didn't invent the idea is one of those easily-overlooked juicy details lost
in history. It has technical and political momentum to dominate, so ... run with it.

Hi Graydon,

Nice to see you here, and nice to see you getting credit for the considerable advances you
brought about in the state of this art.

I wonder if anybody will ever chronicle my part in the story?

Regards,

Daniel

Do tell.  I don't know your part of the story.  Nor your last name.

Nice to see you again too! But I must admit to not knowing all the twists and turns of your
part of the story to chronicle them correctly. I know some bits but I'd probably blurt them
out wrong. 

I didn't mean to imply that I invented the interesting ideas in monotone. Merely that it, as a
social and researchy development project, both discovered a few fresh ideas and consolidated /
refined many others, and has subsequently been a ripe source of ideas for its successors. I
did some of the work, but also made a ton of mistakes; the key theoretical work we stumbled
through during the course of monotone development was mostly the doing of others. Jerome
Fisher, Nathaniel Smith, Derek Scherger, Bram and Ross Cohen, Timothy Brownawell, Christof
Petig, Richard Levitte, Zack Weinberg, Peter Simons, Daniel Phillips, Emile Snyder, Markus
Schiltknecht, Paul Crowley ... and a long list of others who I am probably implicitly
insulting by not mentioning here (sorry, limited comment space).

We really lucked out, for whatever reason, in drawing together a group of exceptional people
to mull over the problem and push around potential solutions in code, without anyone getting
too pissy about "being right". It's been a really enjoyable and open community.

I was referring to 

  http://lwn.net/Articles/249457/

I think part of what bothers me about this story is our Loyal Editor's assertion: "Git was not
the first free distributed revision control system, but it was the first to be employed on
such a massive scale. In a real sense, git launched a new era of free software development."

The first sentence is true, as far as I goes, but it was Linux switching to a Free Software,
decentralized revision control tool that was so important, not the invention of yet another
Free Software, decentralized revision control tool.  The combination of Linus not giving
credit where credit was due and of Linus-fans subsequently misattributing monotone's
innovations to git bothers me.

The use of a decentralized, Free, revision control tool for the kernel was a major step
forward.  The invention of git was a minor tweak to the state of the art -- an exploration of
other parts of the design space.

Don't get me wrong -- I like git, and I'm glad it exists.  I like diversity and redundancy.  I
like exploring the design space widely instead of everyone congregating on the first part of
the design space that is Good Enough.

And I'm sure that git serves the needs of linux kernel developers -- and of many other people
-- as well or better than various alternatives would.  But I don't like for the history of
scientific invention to be obscured by enthusiasm for Linus's personality.

What did our Loyal Editor mean by writing that git launched a new era of free software
development?

You're apparently reading things into what I wrote that I didn't say.

I didn't say `Linus invented all this stuff'. I said that he got the 
representation right, not that nobody else had ever done the same. I'm not 
such a fool as to imagine that nobody else ever tried content-addressable 
storage in version control systems before. (However, I'm fairly sure 
nobody ever released a VCS in such an embryonic state before: generally 
release schedules for VCSes are quite conservative because people hate 
losing their work. It's impressive that git has managed to go so long with 
an aggressive release policy with so few incidents of significant data 
loss.)

Sorry -- I didn't really mean *you*.  Your point about releasing a version control system was
an interesting and valid point, I thought.  I didn't really mean any specific person on this
thread -- more the general folklore that I imagine exists in which people think that Linus
took a break from his kernel hacking in order to singlehandedly move forward the state of the
art of revision control.

Ah, OK. Damn English: why can't we have visibly distinct singular and 
plural second person pronouns anyway?

(Totally off-topic)

There are regional dialects which make the distinction at least in part.  For example, the
American South offers us the term "y'all", which is universally used (among those who use it)
as a second-person plural.  Although I'm not from the South, I find the term useful enough
that I occasionally drop it into informal speech or writing.  Unfortunately, I don't know of
any equivalent that is unambiguously singular.

Scots offers 'youse' as another effective second-person plural (this has also become common in
Australian vernacular in recent decades).

There is no modern-sounding English pronoun that is unambiguously singular, but the archaic
(some Northern English dialects preserved this usage up to the 1950s) 'thee' and accusative
'thou' will do, if you don't mind sounding vaguely biblical.

If you do use these, please please conjugate your funny old verb forms (thou dost, she doth)
correctly!

>I didn't say `Linus invented all this stuff'. I said that he got the representation right,
not that nobody else had ever done the same. I'm not such a fool as to imagine that nobody
else ever tried content-addressable storage in version control systems before. (However, I'm
fairly sure nobody ever released a VCS in such an embryonic state before: generally release
schedules for VCSes are quite conservative because people hate losing their work. It's
impressive that git has managed to go so long with an aggressive release policy with so few
incidents of significant data loss.)

Point of history: Git got the representation (by which I'm assuming you mean the core
file/tree/commit blob design) right because it copied the hard parts from Monotone.  Monotone
got the representation right because it started with a good idea (i.e., "hey, let's use
content-addressing to decouple storage and history representations"), and then evolved it over
several years (including two major representation rewrites, one of which added the crucial
"commit" object), using monthly time-based releases (i.e., "oops, it's Monday, time to ship
whatever's in trunk"), and was self-hosting from ~the very beginning -- I think before Graydon
had even received a single outside patch.  It did all this with continuous field upgrades for
all storage/representation changes, minimal segfault bugs -- I'm remembering ~2-4? (mtn is
written in C++) -- and no reported data loss by any users.

None of which is to say that git is unimpressive -- I don't subscribe to the peculiar notion
whereby any achievement seen once becomes unimpressive forever after -- and git is a
well-done, well-maintained project containing other innovations and that is making a lot of
users happy.  That's always impressive :-).  ATM, in fact, it's doing a better job of it than
monotone is -- probably as a result of Linus's emphasis on building a tool that would be
immediately useful under extreme conditions, while monotone was noodling around trying to
invent about three different novel technologies.  Turns out that one suffices.  Oops.  OTOH,
it's not like this is the first time an idea had to jump projects to move from research to
mainstream; they reward different approaches.  It's entirely possible that if monotone had
started out with the attitude that made git so successful, neither would exist at this point.
So... *shrug*.

(You just *wait* 'til we get those other two nailed down, though!  Muahaha!)

> and a whole bunch of really nasty ones that had bedeviled other VC
> systems for ages just ceased to exist, like rename tracking (what?
> you need to track renames? why not just search for similar content
> when you pack? that way you can merge stuff that's similar whether
> or not it originated in a rename.)

The primary reason for wanting to track renames in a version control system is not storage
efficiency: it is to correctly merge a branch even if files have been moved around.

Git's answer to the merge problem is to infer the renames based on the file content, which
works quite well except for a few cases where it doesn't (some cases with renamed directories
and added files come to mind).

In contrast, VCS's that do track renames generally only use the information they captured when
performing the merge.  This can break down in cases where a file has been split in two, since
they'll often try to apply the changes to just one of the halves.

One thing to note is that the data model of a VCS that tracks renames does not preclude
performing git-style content based merging, while the reverse is not true.  Perhaps the ideal
merge algorithm will turn out to be a combination of tracking renames and content based
heuristics.

As both approaches have their faults, I'd prefer to keep my data in a form that tracks the
renames.  5 to 10 years down the track, if it turns out that content based merges are state of
the art I can always throw that information away.

It's true about the merging stuff, I quite forgot about that thanks to using VCs that had
largely solved this problem for so long now :)


Tracking renames explicitly works fine until your users forget to tell the VCS as well as the
filesystem about the rename. Then it breaks. I find the users forget to do this all the damn
time.

I suspect manual rename tracking (as opposed to content-detection inference at some stage)
will work properly only when the FS and VCS are merged, and look at ClearCase for an example
of how ugly *that* can get. (Also, manual rename tracking intrinsically can't handle the case
you mentioned of 'oh that file got cut in half', let alone more complex cases.)

It's easy for the VCS to detect when the user has forgotten to record a rename -- there's an
inconsistency between the VCS's idea of what the tree should look like and what the tree
actually looks like on disk.  (This is true both when the VCS thinks a rename should have
occurred but none did, or vice-versa.)  Most VCSes already refuse to allow a commit when
there's a "missing file" situation like this.  So just add some code to that routine that says
"such and such files are missing -- hmm, but it looks like very similar files are in an
'unknown' state over there.  (Do you want me to record the following renames to fix things
up?)/(Do you want me to rename the following files on disk to fix things up?): a -> b, c ->
d".

This combines the ease-of-use of the automatic content tracking systems with the
predictability of explicit rename tracking.  It still doesn't help with files getting cut in
half, though, sure.  (Handling that case in an explicit and predictable manner looks hopeless,
though, which is why I personally would prefer that such heuristics only be used as part of a
manual merge with a human checking the results.  YMMV.)

Hooking into the FS doesn't really help, and probably hurts, because a VCS "rename" is at a
higher semantic level than a FS "rename".  It's perfectly common for people to, say, copy a
file, edit the copy, and then replace/delete the original.  rename(2) is a way to move bits
around, "myvcs rename" is a way to record intentions.

> So, the entire LWN staff would like to say "thanks!" one last time to our readers, who have
treated us so well for the last ten years. It has been an incredible ride.

It has been indeed. Thanks!

A better link for 'The "new" kernel development process is adopted' is:
http://lwn.net/Articles/94605/

I had forgotten when Ubuntu came on the scene.  Not even three and a half years ago.  They
have accomplished a lot in such a short time, especially with name recognition.

"The Firefox 1.0 release was, in a very real sense, the much-delayed culmination of the
process which began back in 1998, when Netscape announced that it would be releasing its code.
Firefox was almost seven years in the making, but, sometimes, late really is better than
never. Even those of us who use a different browser should be thankful for the effect Firefox
has had toward the creation of a standard-compliant web and a competitive environment for web
browsers."

I'd like to agree, but there seems to be a very recent backlash of sorts occurring on the part
of some Web developers.  It seems as though there is suddenly a willingness to code for IE
only citing "Firefox security issues".  Digging a bit deeper one comes to the realization that
it is the open development of Firefox  and "a lack of development security" that the web
developers are referring to.

Now, who would be promoting that idea?  Who would benefit?  When pressed, those developers are
adamant that only IE will be "supported".  Something is going on in the back rooms, especially
just at the point it seemed that cross-platform compatibility was a done deal.  It looks like
as though while MS has been making a big deal about playing nice with the F/OSS community,
they have been up to other actions behind the scenes.

Here is a thread that alerted me to this kind of thinking:

http://kzrider.com/component/option,com_joomlaboard/Itemi...

I've since learned from communications with others of similar responses from certain web
developers.

The link you posted is 404. Mistake in copy/paste, or did it get changed on the server?

Corbet -- Thank you for this series, I enjoyed it tremendously.

However, I am stunned, looking back, on the consistent slant in your coverage of the Bitkeeper
affair; your unwavering support of Larry McVoy as he grew increasingly unreasonable, up until
he pulled the rug out from under Linux, prompting you to essentially call Andrew Tridgell a
'hypocrite' (after mischaracterizing what happened with the conditional "if Tridge really did
eat babies then he's a bad bad man").

I get the sense from the articles that McVoy is a friend of yours and I can understand that it
must be difficult to keep journalistic detachment when reporting on people you know and like.
But I can't help but wonder if you, with the passage of time, now see the Bitkeeper affair as
a black mark on your otherwise superb publication?

Am I right if I think the 'hypocrite' thing you mention is this?

>If BitKeeper users were violating the license under which they received the software, they
have indeed done something wrong. Every time we release code under a free license, we do so
with the expectation that the terms of that license will be respected. To treat somebody
else's license with less respect is hypocritical; if the license terms are not acceptable, do
not use the software. 

I think that LWN is balanced and reasonable here. I also think that using bitkeeper in the
first place was a brave thing for Linus to do. 

The main reason I am still reading LWN after all these years is that the editors, in my view,
are able to write balanced articles where they are able to see things from several different
perspectives. I also think that the main reason that Linux is so popular is that so many of
the Linux supporters are seen as reasonable and friendly people. (And there are lots of failed
OS'es where that is not the case.)