LWN.net Weekly Edition for December 24, 2009

A 2009 retrospective

It's that time of year again: 2009 is coming to a close, so it is time for your editor to return to his ill-advised predictions made in January and see how things went. As usual, it's a mixed bag; some predictions went better than others. As they say, even a broken clock is right twice a day - at least for those of us old enough to remember analog clocks - so your editor could hardly be expected to get everything wrong. But he tried.

The first prediction was that free software would emerge from the economic mess stronger than ever. Whether anybody has truly emerged from this crisis is a bit of a controversial subject; it may be a while before we really know. But it is already clear that this prediction hit the mark. Companies dealing in free software are generally doing well, and developers are having a much easier time finding jobs than many others. The dot-com crash was hard on our community; this time around has not been a whole lot of fun, but we're coming through it in good form.

A related prediction was that open embedded systems would grow in appeal, and that Android, in particular, would do well. Android's success seems no longer to be in doubt; it is showing up on a wide variety of devices. Truly open systems are still rather more scarce than one might like, though handsets like Nokia's N900 are a nice step in the right direction. What we are seeing, in any case, is that even closed devices are quickly opened up by their customers; sometimes it seems like the industry isn't even trying all that hard to stop device liberation anymore.

Your editor thought that there would be fewer GPL enforcement actions this year. Without taking the time to do a proper count, your editor thinks that happened, though the new round of BusyBox suits announced in mid-September made that conclusion less clear than it would have otherwise been. But, as seen by Bradley Kuhn's successful project to find a new GPL violation every day, respect for our licensing remains far from universal.

The fear that a formerly friendly company might go to the dark side and follow the SCO path hasn't been realized - so far. On the other hand, your editor didn't even bother to predict that SCO itself would be gone by this time; one wonders if that story will ever really come to an end.

Rather cynically, your editor said that we still would not know about the 2008 Fedora break-in. Apologies are due for that one: the Fedora project posted its report at the end of March. There are still unanswered questions, naturally, but this report is as complete as could have been expected.

On the prediction that the 2.6.33 kernel would be released: does 2.6.33-rc1 count? Probably not. At this point, the kernel seems to have settled pretty firmly into a three-month development cycle; that's unlikely to change in the near future. On the other hand, the prediction that the numbering scheme would not be changed proved to be correct.

"The realtime patch set will be mostly merged by the end of the year." Oh well.

"3D graphics will be a solved problem." That was a bit on the optimistic side, but we are getting much closer. The big problem in the future is not going to be 3D in general, but graphics chipsets used in mobile platforms in particular.

On the other development-oriented subjects: the "make or break year" for Perl looks mostly like another year of stasis for Perl; the Parrot 1.0.0 release does not appear to have brought a lot of new energy to this project, but neither is Perl fading away. KDE 4 has indeed stabilized, and GNOME 3 is indeed coming into focus. People are still debating distributed version control systems, and more projects are making the switch. On the other hand, Go-oo notwithstanding, it's hard to say that OpenOffice.org has truly been forked.

There is one other important thing to do when reviewing predictions: evaluate what was missed altogether. Predictions which are always right, but which fail to anticipate the truly important events of the year are of limited utility. So, with that in mind, one might well ask: what did your clueless editor miss entirely back in January?

Arguably the item at the top of the list would have to be the acquisition of Sun Microsystems, which had been in trouble for some time already. Since Sun claims to be the world's largest contributor to free software projects, any change of control must be an important event. In this case, the proposed acquisition of Sun by Oracle has put an important free software project (MySQL) into play as various parties try to either use it to affect regulatory acceptance of the acquisition or, instead, use the regulatory process to gain some degree of control over a post-acquisition MySQL. It is not a pretty picture, but it does demonstrate the sort of importance that free software projects can attain in the wider world.

While your editor predicted success for Android, the announcement of ChromeOS came as a bit of a surprise, despite long-lived rumors that Google was going to get into the Linux distribution business.

Your editor certainly wishes he had gone on record with a prediction that Microsoft would become a contributor to the Linux kernel. Such a prediction would have certainly drawn a number of skeptical comments; ah, what joy it would have been to post "I told you so" responses to those. Alas, your editor was not that on top of things. But, then, it appears (again) that Microsoft's time as a kernel contributor might be short.

Finally, something that really should have been predicted was the increasing focus on identifying and discouraging behavior which discourages people from joining (or remaining part of) our community. We have seen a number of discussions resulting from ill-considered comments by high-profile people, the imposition of codes of conduct, and more. There can be no doubt that the atmosphere in many of our public spaces is seen as hostile by many talented people we would like to have as contributors. There also can be no doubt that we will drive away contributors with excessive criticism of community members whose comments are seen as unwelcome or heavy-handed conduct enforcement schemes. Finding a balance which works for the community as a whole is going to be a long-term project.

With that, your editor wishes the best of holidays and year-end festivities for all LWN readers. LWN traditionally does not publish an edition in the last week of the year - there is usually not much happening anyway - so we will not be back until January 7, when we will resume our normal publication schedule and your editor will return with another set of doomed predictions. Many thanks to all of you for supporting LWN through another great year.

Comments (3 posted)

EtherPad source code is free, now what?

Google's newly-acquired startup AppJet released the source code to its popular EtherPad web editor recently, making good on a promise to EtherPad's users who were previously faced with a service shutdown following the acquisition. The source is under the Apache 2.0 license, which is GPL-compatible, making the code potentially useful to a wide array of free software projects. The release has the community debating the impact on similar and related software, and revisiting the contentious question of how free software in general can and should transition to the web-hosted environment.

Pad timing

EtherPad is a collaborative in-browser text editor. AppJet launched the product in the fall of 2008 with both commercial and free (limited to eight concurrent editors) versions, and it quickly gained popularity in the first half of 2009. When Google unveiled its own real-time collaboration system Wave in June, comparisons were inevitable. Many users found EtherPad's interface simpler to use and easier to understand, however, so it was no great surprise when Google announced that it had purchased AppJet and EtherPad on December 4. The AppJet engineers would work on Wave, ostensibly making it as easy to use as EtherPad itself.

What did come as a surprise to most EtherPad users was AppJet's announcement that due to the acquisition, it would be unceremoniously switching off the service for all users on April 1, 2010 — and to reinforce that the move was no April Fools' joke, no new documents could be created, effective immediately. There would also be no refunds to customers who had already paid for the "professional" service. The subsequent backlash from users and fans was forceful enough that, less than 24 hours later, AppJet CEO Aaron Iba posted a personal apology and announced a new "transition plan" — document creation would be re-enabled, EtherPad itself and the underlying AppJet Web Framework would both become open source projects, and AppJet would try to get Google Wave invites for EtherPad users.

Source at last

The source code release came on December 17, accompanied by the proclamation that AppJet's goal "is to let the world run their own etherpad servers so that the functionality can live on even after we shut down etherpad.com". The shutdown is still scheduled to take place on March 31, 2010, and new document creation may be again switched off sooner than that, if traffic is seen to "taper off". The source is hosted at Google Code, and includes instructions for compilation on Mac OS X and Linux. The actual code implements an EtherPad server running as a stand-alone HTTP server on port 9000. The server is written in Java and Scala, and requires MySQL. The client-side editor is implemented in JavaScript.

Some pieces of the service as it was provided at etherpad.com are not present in the open source release, however, notably file upload, document import/export, the email invitation system, and the framework for managing "professional" accounts. The file upload capability was provided by a proprietary servlet that AppJet could not include with the release; the other capabilities appear to have been left out for the sake of convenience. Perhaps those missing pieces, when taken with the news that the AppJet team still intended to shut down the service and not pursue further work on the code, contributed to those in the open source sphere describing the move as "dumping code over the wall" — a pejorative typically indicating the community's belief that the company has no interest in what happens next.

Source is still source, though

Nevertheless, the Etherpad release attracted many eyes and many comments from open source circles. Two topics dominated the conversation: what impact the EtherPad code would have on other projects, and how free software could protect users from suffering the inconveniences of a similar web service shutdown.

As it currently stands, the open source EtherPad code seems unlikely to develop as a viable project on its own. The Google Code site refers to the project as an "exhibition" and says that "we will try to support you in our spare time until we begin working full-time on Google Wave." There is an open mailing list, however, and several developers with non-Google IDs have been granted the Owner role. At least one independent public server has already been launched, PiratePad.net.

The other projects most likely to be affected by the availability of EtherPad source code are Google Wave (naturally) and other real-time collaborative editing tools like Gobby, AbiCollab (which we recently covered), and Bespin (also recently covered). Although Wave's document-sharing and editing capabilities are less mature than EtherPad's, it does have one notable advantage: federation is built in to the protocol, allowing editing sessions to be shared between multiple Wave servers, a feature EtherPad never had.

As for EtherPad's "threat" to other editors, the prevailing attitude is that in-browser editing trumps any desktop client editor because of the sheer ease-of-deployment, a feature that is critical to collaboration. On the other hand, Gobby's conflict-resolution algorithms are highly-regarded and well-documented (unlike EtherPad's), and the editor features niceties like syntax highlighting not found in the web editor. Gobby maintainer Armin Burgmeier commented on one blog discussion that the best way forward might be adding Gobby's concurrency control (via Gobby's libinfinity library) to an Etherpad-like web editor.

Branching out from pure editing alone, Red Hat's Máirín Duffy suggested that EtherPad's slick editing capabilities would be a good addition to some other web-based tools, MediaWiki in particular. MediaWiki is designed to encourage collaborative writing, after all, but it currently relies on HTML's "textarea" element and its own peculiar markup as an editing interface.

Web versus Desktop; collaboration versus solo work

However the EtherPad application evolves, the fiasco surrounding the shutdown announcement and subsequent code dump again raises the weighty and still unsolved problem of how free software ideals and practices should migrate from the desktop paradigm to the web service paradigm.

In her blog, the GNOME Foundation's Stormy Peters wrote that hosting free and open source web applications is fundamentally hard — open source web applications that thrive have always offered end users a hosted service (such as Wordpress.com or SugarCRM); those that have not tend to fail. There are varying business models, including advertising-supported free services, paid professional alternatives, and more, but unlike hosting a download site for desktop applications, there are ongoing support and labor costs that must be borne somehow. As long as the shepherding organization is a company that remains in business and actively involved, a hosted service is reasonably safe for users to rely on. The trouble arises when an acquisition, a change of business plan, economic woes, or other real-life events threaten the business itself. Consequently, Peters asked: Should software projects start non-profit foundations to provide web services?

Ubuntu's Jorge Castro opined in his own blog piece that existing free software groups such as GNOME and KDE ought to offer web services like EtherPad, just as they currently host mailing lists, revision control systems, IRC channels, and other collaboration tools. According to the post, Castro recently undertook a self-imposed experiment to use only web-based applications for a set period of time, just to see how the experience compared to desktop applications. He liked it so much, he has no plans to go back.

It is interesting to note, however, that the services Castro cites as examples are all communication tools: email, instant messaging, microblogging, and real-time note-taking at conferences. There are other web application use cases that do not inherently involve sharing data with other remote users, and as a result, might not inherently benefit from running solely on the web. Financial records, for example, might be convenient to access from multiple locations, but a hypothetical "Gnucash Online" service would not need to share information between users concurrently. Media players, to take an unrelated issue, are hamstrung by copyright holders' rights when online storage comes into play. Image, sound, and video editing, on the other hand, have low network latency requirements that make for a poor user experience under anything but the best network conditions — even if sharing the final product on the web is something the user intends to do.

Castro suggests that the myriad of free software groups provide hosting of web services for participating developers, not for the public at large, so it might not offer the protection-from-corporate-disappearance that Peters asked about. But for a collaborative editor like EtherPad, it might be just the thing.

Code drops are a gift to the open source software world and, as such, they are always welcome events, but rarely are they game-changers. EtherPad was a wildly popular product in its lifetime, but judging by the reaction to recent events, its popularity may have been more due to its implementation as a free web-based service than to ingenuity of the code itself. Thus, the bigger question going forward is one that free software has been struggling to answer for the past several years and will likely continue to struggle with for years to come: how can open source not just compete with closed-but-freely-accessible web services, but beat them on the critical question of protecting users from the catastrophe of being deserted by a service that disappears.

Comments (14 posted)

The 2009 Linux and free software timeline - Q3

Here is LWN's twelfth annual timeline of significant events in the Linux and free software world for the year.

2009 offered few surprises to those that have been following Linux and free software for as long as we have. As expected, there were new releases of many of the tools and underlying infrastructure that we use on a daily basis. There were also lawsuits over software patents, arguments over licensing, and various security flaws found and fixed. Distributions were packaged up and released, more phones and other devices with Linux and free software were sold, and so forth. All part of the march to "world domination". We look forward to 2010—and beyond.

This year we will be breaking things up into quarters, and this is our report on July-September 2009. We got a bit behind, so the timeline for the last quarter directly follows this one.

• First quarter timeline (Jan-Mar 2009)
• Second quarter timeline (Apr-Jun 2009)

---

This is version 0.8 of the 2009 timeline. There are almost certainly some errors or omissions; if you find any, please send them to timeline@lwn.net.

LWN subscribers have paid for the development of this timeline, along with previous timelines and the weekly editions. If you like what you see here, or elsewhere on the site, please consider subscribing to LWN.

For those with a nostalgic bent, our timeline index page has links to the previous eleven timelines and some other retrospective articles going all the way back to 1998.

July

PostgreSQL 8.4 is released. (announcement)

Google announces Chrome OS, a Linux-based, web-centric OS for ARM and x86. (announcement, LWN coverage)

VLC media player 1.0 is released. (announcement, LWN review)

Mercurial releases version 1.3 of the Python-based distributed version control system. (announcement)

The Gran Canaria Desktop Summit is held in the Canary Islands—it is the first time that GNOME and KDE co-located their annual conferences. (KDE.News coverage)

Maemo announces a switch from GTK/Hildon to Qt, something that doesn't come as a complete surprise after Nokia acquired Qt provider Trolltech. (LWN coverage)

The International Free and Open Source Software Law Review is launched. (announcement)

A local user privilege escalation vulnerability in the kernel, which (ab)uses NULL pointer dereferences is announced with a proof-of-concept exploit. (LWN coverage part 1 and part 2)

The Nmap security scanner releases version 5.0. (announcement)

Canonical, the company behind Ubuntu, released its Launchpad source code under a free software license. (announcement)

Django 1.1 is released; Django is a Python-based web framework. (announcement)

Amazon fails in its irony detection and deletes George Orwell's 1984 (and Animal Farm) from users' Kindle e-book readers. (New York Times coverage)

Emacs 23.1 is released. (announcement)

Botnet simulation boots one million virtualized Linux kernels at Sandia National Laboratories. (LinuxInsider article)

August

KDE 4.3 is released. (announcement)

Novell devotes ten engineers to the openSUSE project, rather than have them work as time is available. (announcement)

openSUSE reduces maintenance period for new distribution releases to 18 months, down from 24 months. (announcement)

An injunction against the OpenBTS cellular base station project is lifted, allowing discussion of the project by certain members once again. (announcement, LWN injunction article)

Ubuntu removes the controversial "multisearch" feature from Karmic Koala (9.10), because of privacy and usability concerns. (LWN coverage)

Arch Linux 2009.08 is released. (announcement)

KMyMoney 1.0 is released, after two years of development on the personal finance management application. (announcement, LWN review)

Yet another kernel NULL pointer vulnerability is reported, in what is becoming a steady stream of such reports. (linux-kernel posting, more LWN coverage)

Desktop publisher Scribus releases version 1.3.5 (release notes, LWN review)

O'Reilly publishes The Art of Community by Ubuntu community manager Jono Bacon. (announcement)

The Linux Foundation updates its kernel development statistics white paper, authored by Jonathan Corbet, Greg Kroah-Hartman, and Amanda McPherson. (announcement, white paper [PDF])

An appeals court rules that SCO's claims about Unix copyrights should go to trial, overturning the summary judgment that Novell "won" in 2007 and breathing new life into the SCO litigation circus. (LWN coverage)

openSUSE defaults desktop choice to KDE, though GNOME and others still remain as supported choices. (announcement, LWN coverage)

Unix celebrates its 40th birthday. (BBC article)

Slackware 13.0 is released, with support for 64-bit processors. (announcement, LWN review)

September

Red Hat Enterprise Linux 5.4 is released, with x86_64 KVM support, FUSE, the XFS filesystem, and more. (release notes)

Linux 2.6.31 is released with performance counter support, kernel mode setting for ATI Radeon chipsets, kmemleak, USB 3.0 support, and more. (announcement, KernelNewbies coverage)

Debian announces a switch to Upstart for boot-time initialization. (announcement)

Microsoft forms the CodePlex foundation to foster cooperation between software companies and open source communities. (announcement, LWN coverage)

Alan Turing gets a long-belated apology from the UK government for his treatment for being gay. (Prime Minister Gordon Brown's apology)

The first-ever LinuxCon is held in Portland, Oregon co-located with the second-ever Linux Plumbers Conference. (LinuxCon event site)

Puppy Linux 4.3 is released. (announcement, LWN review)

LWN finally makes T-shirts and other branded items available for sale. (LWN.net CafePress store)

GNOME 2.28 is released. (announcement)

libtheora 1.1 "Thusnelda" is released bringing faster decoding and better quality to the Theora video codec. (announcement)

Comments (none posted)

The 2009 Linux and free software timeline - Q4

October

X.org releases xorg-server 1.7 (announcement, LWN coverage)

Gentoo celebrates its tenth birthday by releasing a Gentoo Linux 10.0 LiveDVD. (announcement)

OpenSSH also celebrates its tenth anniversary with the release of OpenSSH 5.3. (announcement)

TurboGears releases version 1.1 of the Python-based web framework. (announcement)

The Real Time Linux Workshop is held in Dresden, Germany. (LWN coverage)

Amarok 2.2 "Sunjammer" is released. (KDE.News report)

Nokia releases the N900 based on Maemo 5 and quite hackable. (LWN report from the Maemo Summit)

GDB 7.0 is released with reverse debugging, Python scripting, and more. (announcement)

CentOS 5.4 is released. (announcement)

OpenBSD 4.6 is released. (announcement)

Darl McBride is terminated as SCO CEO and as the longtime "face" of SCO's litigation strategy. (Groklaw coverage)

The Linux Kernel Summit is held in Asia, specifically Tokyo, for the first time. It is co-located with the Japan Linux Symposium. (LWN Kernel Summit coverage)

X11R7.5 is released with multi-pointer X, RANDR enhancements, and more. (announcement, Peter Hutterer's disambiguation)

SeaMonkey 2.0 is released—the heir to Netscape Communicator as an all-in-one internet suite. (announcement)

Version 2.6 of the LLVM compiler is released with the first release of the Clang C/Objective-C compiler, better code generation, and more. (announcement)

Word processor AbiWord releases version 2.8 with collaboration support, "true" SVG support, and more. (announcement, LWN review)

Ubuntu "Karmic Koala" (9.10) is released. (announcement, LWN review)

November

Mandriva 2010.0 is released. (release notes, LWN review)

One Laptop Per Child cancels the XO-2, opting instead for an ARM-based XO-1.75 in the near term and an XO-3 in 2012. (OLPC News report)

Python declares a moratorium on syntax and grammar changes through the 2.7 and 3.2 releases and possibly longer. (LWN coverage)

GNOME plans for a 3.0 release in September 2010 and 2.30 in March. (announcement)

Google announces a new systems programming language: Go—released under a BSD license. (web site, language tutorial)

Cavium Networks acquires MontaVista Software one of the first commercial embedded Linux vendors. (press release)

A fundamental flaw is found in the Transport Layer Security (TLS) protocol, which allows man-in-the-middle plaintext injection attacks. (LWN coverage)

openSUSE 11.2 is released with KDE 4.3, GNOME 2.28, OpenOffice.org 3.1, and more. (announcement, LWN review)

Fedora 12 is released with rpmdelta support, virtualization improvements, and more. (announcement, LWN conversation with Paul Frields)

Knoppix 6.2 is released with kernel 2.6.31.6, X.org 7.4, and more. (The H article)

Google releases the Chromium OS source under a BSD license. (announcement)

Fedora 12 initially ships with a security hole by default allowing unprivileged users to install signed packages from signed repositories without requiring a password. (LWN coverage)

KDE repositions its "brand" by separating the KDE software into different groups: KDE Plasma Desktop, KDE Platform, KDE Applications, and KDE Software Compilation. (KDE.News report)

Vector drawing program Inkscape releases version 0.47, which has been massively overhauled from previous versions. (release notes)

FreeBSD 8.0 is released. (announcement, LWN review)

Linux Mint 8 "Helena" is released. (announcement)

December

Qt 4.6 is released with multi-touch and gesture support, new graphical capabilities, more platforms supported, and more. (announcement, LWN coverage)

Linux 2.6.32 is released with devtmpfs, HWPOISON, more perf events features, kernel shared memory, and more. (announcement, KernelNewbies coverage)

Twisted 9.0.0 is released; Twisted is a Python-based event-driven networking engine. (announcement, LWN review)

OpenInkpot releases version 0.2 of the free firmware for e-book readers. (announcement, LWN coverage)

Email client Thunderbird 3.0 is released (release notes)

Sugar on a Stick v2 "Blueberry" is released. (announcement)

Various efforts are made to get MySQL out from under the control of Oracle, either by license or ownership change. (LWN coverage)

The Software Freedom Law Center sues Best Buy, Samsung, Westinghouse, and others for GPL violations on behalf of the BusyBox project (announcement)

Malware disguised as a screensaver is made available at GNOME-Look.org, which serves as a reminder to be careful where you get your bits. (LWN coverage)

Fedora 10 reaches end of life. (announcement)

digiKam 1.0 is released. (announcement, LWN review)

Moonlight 2 is released. (announcement)

Mark Shuttleworth announces that he is stepping down as Canonical CEO effective March 2010, in favor of Jane Silber; Shuttleworth will focus on design and quality for Canonical. (announcement)

To be continued ...

Comments (none posted)