LWN.net Weekly Edition for December 23, 2010

A 2010 retrospective

Your editor is not really prepared for the end of 2010; truly, he has not yet come to terms with the end of the 20th century, but so be it. Ready or not, it's time to look back at the just-finished year, with an eye toward making fun of the predictions which were made back at the beginning. Your editor is shocked to discover that he didn't get everything right.

There were two hardware-related predictions: that the awareness of the value of open hardware would grow, and that there would be a number of Linux-based tablets. Neither was realized in any complete sort of way. The success of Android shows some level of appreciation for openness; it can be instructive to look at second-hand sales of Android handsets and note how many of them are described as "rooted." The Free Software Foundation also tried to raise awareness with its endorsement program but, as your editor said at the time, that program appears unlikely to have much real-world effect.

As far as Linux-based tablets go: we have seen a few Android devices, with Samsung's Galaxy Tab being the most prominent. But Android on tablets has been surprisingly slow to arrive, and MeeGo, needless to say, is slower yet. Perhaps 2011 will be the year of the Linux tablet.

The prediction that software patents would be a problem was not particularly hard to make. Sure enough, a number of suits have been launched, mostly in the mobile computing area.

Copyright assignment policies: the prediction that there would be debate around such policies was accurate. The LibreOffice project, in particular, had a surprisingly high-volume (in both the amplitude and quantity sense) debate on copyright assignment, but the developers behind LibreOffice seem determined that they will be more successful without any such policy. The GNOME project and the newly-formed MeeGo project also came out strongly against copyright assignment. These policies remain firmly entrenched in many projects, but the trend appears to be against them.

That Oracle's acquisition of Sun would proceed was also a relatively easy prediction. Your editor said that MySQL would be treated with a relatively light hand, which has proved mostly to be the case. What your editor missed was how badly most of the rest of Sun's free software projects would fare. Significant forks of OpenSolaris and OpenOffice now exist, and there is discontent in other projects as well. Oracle's relationship with the kernel community remains good, but the company seems to care little about projects higher in the stack.

The browser wars: perhaps they have heated up again, as predicted; certainly Google Chrome seems to be gaining strength. Mozilla is competing with a number of interesting initiatives, including a mobile version of Firefox. Good stuff is happening - but Internet Explorer still hangs on to over half of all traffic.

The prediction that solid-state storage devices would go into wider use was boring and obvious. Perhaps more interesting was the claim that some distributors would be offering Btrfs. That has certainly happened; your editor did not foresee, though, that the MeeGo project would adopt Btrfs as its default filesystem.

The rumors of the death of the big kernel lock were only exaggerated by a little; the 2.6.37 kernel (which should come out just after the new year) can be built in a useful mode without the BKL entirely.

The growth of LLVM was another fairly obvious prediction; a number of interesting things have happened with that project in the last year. Identifying Unladen Swallow as one of those things turned out to be a bad choice, though; whether Unladen Swallow is dead or just resting remains to be seen, but it is not a hotbed of activity at the moment.

Your editor predicted a "scary security incident" involving mobile devices. There have been some examples of malicious Android applications, but nothing that qualifies as a truly scary incident - that we know about, anyway. The scary stuff, instead, happened at other levels, with the Google attacks and the Stuxnet worm being the most prominent examples. The "year of the sandbox" was also predicted, but nothing of any real interest seems to have happened in that area.

It's not surprising that there was a lot of discussion of cloud computing, as predicted. The sun also rose every morning. On the other hand, the predicted release of GNOME 3 did not happen. The predicted increase in Python 3 adoption is also hard to find; there does seem to be a little more interest, but most developers seem to be in no hurry to leave 2.x behind.

The last prediction - on the importance of community distributions - is hard to measure, but it's not clear that the situation has changed markedly. What we are seeing is a bit more attention to staying close to upstream projects and working more closely with them. In its own way, Oracle's decision to slip a 2.6.32-based kernel into its RHEL5 clone is an example of this. MeeGo's desire to push patches upstream rather than carrying them is another.

So what did your editor miss entirely? The seeming increase in high-profile forks (LibreOffice, Mageia, IllumOS, ...) is one of them. The creation of MeeGo through the merger of Moblin and Maemo was another. In retrospect, it's not surprising that the sharks would start to circle around Novell, but your editor certainly did not think that the company might be in different hands by the end of the year. The failure of PHP6 was also obvious in retrospect.

One other interesting omission might, at the beginning of the year, have been phrased something like "the embedded Linux world will begin to get its act together." In this year, we've seen the creation of the Linaro project to try to improve tools and support for the important ARM architecture. The Yocto project - meant to ease the process of creating embedded distributions - launched. A number of embedded vendors came together and decided to standardize on the 2.6.35 kernel, which will receive improved long-term support as a result. The number of embedded vendors contributing to free software projects is growing. There is plenty of room for improvement yet, but things seem to be headed in the right direction.

The most obvious prediction of all was that free software would be stronger than ever. Despite our ups and downs, our flame wars and lawsuits, our bugs and our forks, we're doing great. It's been another good year for Linux and free software, and it has been a pleasure covering it for this audience. Thanks to all of you for making this community happen.

Comments (9 posted)

In Memoriam: the free software projects we lost in 2010

In November I wrote about the apparent demise of the well-known mail delivery agent procmail, which has not been updated since 2001, but is still routinely packaged by Linux distributions. Whatever your feelings about procmail itself, the story prompted a discussion in the comment threads that we revisit periodically as a community: what exactly does it mean for a free or open source project to die anyway?

There is no one "right answer;" the context of the project, its governance, user community, and the opinion of the debaters make for a wide spectrum of definitions. The process of single-vendor project shutting down can instantaneously switch off source code access, online documentation, and mailing lists. More often, the tools and trappings of a project atrophy one at a time; the documentation wiki slips behind the current release, the milestones never make it past "alpha" announcements on the mailing list and onto the Downloads page, and the user forum slowly transforms into a refugee camp where abandoned users help each other patch and shoehorn the aging code into compatibility with newer system libraries.

In most cases, however, the code itself is still available — somewhere. But if we look back at the projects that did close up shop during 2010, it is clear that the source availability factor alone is not always sufficient to regard a project still among the living. In the end, what makes a free software project dead comes down to practical questions. When was the last release? Are there any support plans for businesses? Is there reliable user-to-user support? Is there support for new developers wishing to leverage the code?

For all practical purposes, there are degrees of mortality to be considered. The owner of the code can walk away, shut down the resources, and fire the developers. Projects that meet that fate have little hope unless an entirely new team revives the work from scratch. But almost as serious is when the project owner or leadership pulls all of the developers and puts them on some successor project — however legitimate the ordination is, there is always a risk that the successor project will never see the light of day, and users cannot make the jump until it does.

In any event, it is the end of the year, which puts many of us in a reflective frame of mind, so taking stock of the wreckage from the past twelve months can be illuminating — both with regard to how open source projects die, and the different directions events can take afterward.

The dead and buried

The easiest casualties to identify are those marked by a corporate owner's official press release or a project's clear statement of discontinuation. A few common factors precede many of these cases — generally predictable ones like lack of consumer adoption or legal woes; risks every project endures on a daily basis.

Easily the most high-profile FOSS project to get the axe this year was Google's real-time collaborative editor Wave. Despite a highly-publicized fall 2009 launch, the subsequent releases of much of the code, and worldwide Wave developer events, the search behemoth pulled the plug in August 2010. For reasons that still baffle me (although the "highly-publicized" part is no doubt a key ingredient), this decision was met with sheer joy from many in the technology press, and outright celebration ensued in some darker quarters. The fact that many of those rejoicing continued to describe Wave as an "instant messaging" tool — which it was not — points to poor product management and muddled marketing as critical mistakes on Google's part.

The same blunders may have stricken the Open Source Applications Foundation (OSAF)'s Chandler, a cross-platform email-and-calendaring application, although its end was not nearly as widely observed. The last announced release was made in July of 2009, although commits trickled in until the end of the year, and while the user and developer mailing lists survive, they now consist solely of requests for install-time help. Despite a well-funded benefactor underwriting its development, the project never achieved a fraction of the mindshare enjoyed by Mozilla Thunderbird (and its Lightning calendar add-on), which itself remains a minority player. I'd be willing to wager less than half of the people who read this paragraph knew what Chandler was, much less had tried it.

In contrast, Linux on the PlayStation 3 met a quick demise in 2010, and it was big news. While the firmware that allowed installing Linux wasn't free software itself, of course, it did allow booting the device with a user-installed OS, and was supported by several Linux distributions. It was such big news because PS3's corporate parent, Sony, knifed the project intentionally, publicly, and without remorse. Citing "security concerns," Sony pushed an April firmware update out to PS3s that disabled the "Other OS" feature the devices had supported since their launch in 2006. Rumors were that fear of Blu-Ray piracy enabled through PS3 hacking were the "concern" in question, although no such exploits were ever published.

A peculiar footnote to the PS3 Linux obituary was Sony's sudden announcement of an OpenStep-based application development framework it named SNAP, which was quickly followed by Sony's sudden announcement that SNAP was canceled. The stated idea was to create an open development framework for Apple's iOS, thus prying the tightly-closed lid off of the iPad/iPhone platform to allow in fresh rays of freedom. Considering that the dream of an open homebrew-development community was the initial justification for allowing Other OS on the PS3, SNAP's brief moment in the sun is probably no big surprise.

The LimeWire peer-to-peer (P2P) file sharing tool was scuttled in October, a move dictated by court order. Sources "close to the company" told PCMag that the application will be reborn as a "copyright-friendly service." Because the court order prevents LimeWire from distributing a client capable of uploading or downloading from the Gnutella P2P network, though, there is little chance that the open source "LimeWire Basic" version of the client will return at all.

Not all project terminations were the result of corporate mismanagement or copyright paranoia, however. Linux's HAL hardware abstraction layer, for example, is officially deprecated in favor of udev. Although this transition has been planned since 2008, both that year and 2009 continued to see additional HAL point releases. As of 2010, the major desktop distributions have migrated away from HAL, although several individual applications still pull it in as a dependency. HAL may continue to receive security patches, but its active life is essentially over.

The lost

Speaking of patches and active lives, several large projects fell into the awkward "dead but still claiming lots of users" category, which poses its own unique set of challenges.

Consider the Moblin and Maemo siblings, for example. Intel's netbook distribution and Nokia's handset distribution were welded together into the brand-new MeeGo initiative in February of 2010, which bodes well for the future of the code itself. But both of the parent projects targeted embedded (or at least, non-standard-hardware) devices. Consumers who purchased an N900 phone from Nokia might be miffed to learn that there will be no MeeGo release for the device.The daring can boot MeeGo builds on the N900 from an SD card, but they do so at their own risk.

OpenSolaris was just one of many Sun projects acquired by the proprietary database vendor Oracle, and although several of the others (Java, OpenOffice, and MySQL) have had their fair share of headaches and battles since the acquisition, OpenSolaris is the only one to be scrapped outright. A leaked Oracle memo announced the move in September, under which upcoming "Solaris 11" releases might be available through a "technology partner program," but the open source version marches straight for the grave.

In November, the Symbian Foundation met an unceremonious end when majority stakeholder Nokia announced that it would re-absorb the Symbian unit and shut down all of the Foundation's web assets. Those assets disappeared on December 17th, though Nokia reportedly still employs the Symbian development team. Officially, Symbian will remain open source software, and what was the Symbian Foundation will morph into a "licensing body" — but the actual source code will disappear entirely sometime in March. One would be excused for thinking that that doesn't sound particularly open source; anyone who needs the code is encouraged to drop an email to contact@symbian.org — a friendly offer, but not one that alleviates fears of abandonment.

The gone but not forgotten

Sometimes, of course, a corporate parent can cut a project loose, and the project can continue to survive or even grow. Such was the case for Etherpad, the web-based collaborative editor acquired by Google in 2009. Google opened up the code right after the acquisition, but snuffed out the service in May of 2010. Prior to the switch-off date, several replacement services sprouted up based on the Etherpad source code — Pirate Pad, PrimaryPad, OpenEtherpad, and more.

In addition to straight derivatives, the existence of forks sometimes makes it hard to determine when to declare a project dead, but at least one project is a plausible candidate in 2010.

The PHP-based content management system (CMS) Mambo suffered an acrimonious leadership battle in 2005 that led to the departure of the bulk of the developers, who started the Joomla CMS. As is often the case is such a fork, the remaining owners of the Mambo trademark and source code copyrights asserted that nothing was wrong and that development would continue unabated. Although that may have been true for a while, here at the end of 2010 it has been a full calendar year since there were any signs of life from Mambo (longer still since there was a release), apart from the occasional Twitter alert that the project's servers had been attacked. Joomla, on the other hand, seems fine.

The great unknowns

The final category is made up of those projects that have disappeared or show no signs of life, but which, for one odd reason or another, are impossible to outright pronounce them as dead.

Take Xandros, for example. The commercial Linux distribution has not made a release since 2007, although it has acquired a handful of other companies since then, which indicates that capital is not the problem. One of those acquisitions was even fellow distribution Linspire, which has also failed to make a release since 2007. It's not clear whether or not the distribution is dead, though the company itself still exists and sells support contracts for existing Xandros Linux users. The company does have other products, but also it went all of 2010 without making a press release. Any new products the company may be developing is being done behind closed doors

Snort also has a corporate parent that continues to do business, but the tool itself still faces uncertainty. Some in security circles worry that the popular intrusion detection system is on life support if not actually terminal. The reason is that the long-discussed 3.0 rewrite, in planning since 2007, still has yet to appear. The project continued to make incremental updates to the existing version of Snort in 2010, but that apparently was not good enough to satisfy the US government, which paid to have a Snort replacement written.

Raindrop was a combined-messaging inbox system developed by Mozilla Labs, and offered an unusual combination of features: merging email, instant messages, and microblogging discussions into a single stream, and intelligently filtering one-to-one, group, and automated messages. Despite optimistic beginnings, the project quietly stopped receiving updates in late spring, and the mailing lists fizzled. A Mozilla Labs developer told me in October that a Raindrop replacement would arrive "soon" ... but it never has.

The XUL-based cross platform media player Songbird did not shut down entirely in 2010, but it did drop all Linux support in April. Shortly thereafter, it looked as though things on the Linux front were going to be A-OK, when a group of contributors announced the Nightingale project that would pick up where Songbird left off. Eight months later, however, and there has still been no code released.

The caveats on these seemingly expired projects vary. One has to give a some leeway to Nightingale; starting a project from the ground floor but with a large, pre-existing codebase is never easy. With regard to Raindrop, Mozilla Labs is explicitly marketed as the experimental wing of the browser maker, where R&D happens, and actual projects come and go. Either project could still awaken from its slumber and lead a long, happy life. Snort 3.0 could drop tomorrow, of course; perhaps Uncle Sam is just impatient, and the 3.0 rewrite is close to perfection. Who knows what Xandros HQ could have up its sleeve; the ISO downloads are cryptically marked as "out of stock" so maybe it's as simple as a missing hard disk.

2010, we hardly knew ye

Looking back at the list of 2010's dearly departed, you see a snapshot of the open source ecosystem as a whole. Some projects, like Google Wave, Symbian, and Chandler, never found the user-base their creators were hoping for. Others, like LimeWire and PS3 Linux, were forced to walk the plank thanks to legal threats from the code-meets-commercial-media arena. Songbird and Xandros were both popular when they were available, but appear to have simply lost support among the people who write the paychecks at their respective companies (and who knows what happened to SNAP, but at the very least we can agree that "too much support among management" was not among its problems). Mambo got taken out by infighting between its leadership and core developers. If you looked at the active open source projects making the news today, you'd likely find the same kinds of problems.

What is interesting to note about 2010's obituary is that there was only one Oracle acquisition among the fallen. Despite the database company's dominance of the news cycle for lawsuits and anti-community practices, it did not actually succeed in killing that many open source projects. Whether that tells you something about the hype factor of the acquisition or the resilience of the free software community is anybody's guess.

The developers behind OpenIndiana, the community-driven replacement for OpenSolaris, would probably say the latter. That brings us to the other potential lesson from 2010: the number of open source projects that survived, in one form or another. Etherpad is positively flourishing, Joomla is more popular than ever now, MeeGo is growing and even expanding into new areas, and even the much-maligned Wave has been resurrected as an Apache project (presumably to the consternation of some members of the FOSS press).

Three years ago I looked at the projects that perished in 2007 for NewsForge. There were nine projects on that year's Big Sleep list, and although this is not an exact parallel (the 2007 article only covered projects I personally had written about during the preceding year), I can't help but notice that only one of them has survived in any form that I can identify today. There is reason to be hopeful about at least three or four of this year's victims.

The difference could be due to random variation, but it is also possible that the community has learned from experience. For example, there have been large-scale dump-the-code-over-the-wall releases in the past that did not work out as well as Etherpad; perhaps Etherpad's continued existence ought to make it a case study for other such "if the community wants it, the community can have it" divestments. It might not even be too late for some of this year's casualties, say, Raindrop and Symbian. Even though both of them have some prospect for survival, good intentions offer no guarantee either will still be here in 2011.

Comments (34 posted)

The 2010 Linux and free software timeline - Q4

Here is LWN's thirteenth annual timeline of significant events in the Linux and free software world for the year.

In what is becoming a fairly standard pattern, 2010 brought various patent lawsuits, company acquisitions, new initiatives, and new projects. It also brought new releases of the software that we use on a daily basis. There were licensing squabbles and development direction disagreements—all things that we have come to expect from the Linux and free software world over a year's time. Also as expected, though, were the improvements in the kernel, applications, distributions, and so on that make up that world. Linux and free software just keep chugging along, and we are very happy to be able to keep on reporting about it.

Like last year, we broke things up into quarters, and this is our report on the final quarter, October-December 2010, though there may be an addition or two for December. The previous quarters can be found as follows:

• January-March 2010
• April-June 2010
• July-September 2010

---

This is version 0.8 of the 2010 timeline. There are almost certainly some errors or omissions; if you find any, please send them to timeline@lwn.net.

LWN subscribers have paid for the development of this timeline, along with previous timelines and the weekly editions. If you like what you see here, or elsewhere on the site, please consider subscribing to LWN.

For those with a nostalgic bent, our timeline index page has links to the previous twelve timelines and some other retrospective articles going all the way back to 1998.

October

Smeegol, an openSUSE-based version of the MeeGo UI, is released. The project soon runs afoul of MeeGo trademark issues (announcement, LWN trademark issue coverage and Smeegol review).

The LLVM compiler project releases version 2.8, including major improvements to the Clang C++ support and two new projects: libc++ and LLDB (announcement).

The Software Freedom Conservancy appoints Bradley M. Kuhn as its full-time executive director (LWN blurb and interview).

Red Hat settles a patent case with the patent troll Acacia, but shares no details of the settlement terms (InternetNews blog posting).

The Utah Open Source Conference is held in Sandy, UT (LWN coverage: Learning from failure, Inexpensive audio/video recording, and Applying open source ideals).

Microsoft VP Scott Charney suggests barring computers without a "health certificate" from the internet as a way to fight botnets and other internet security threats. Of course, those certificates would have to be issued by Microsoft. (blog posting).

Ubuntu 10.10 ("Maverick Meerkat") is released (announcement).

Debian welcomes non-packaging contributors as project members in a landslide vote: 285-14 (vote results).

The Open Document Format Plugfest is held in Brussels, Belgium to discuss interoperability between ODF-supporting applications (LWN coverage).

The AsbestOS bootloader, which allows Playstation 3s to run Linux once again, is released (announcement).

The Free Software Foundation announces a hardware endorsement program to distinguish hardware that "respects your freedoms" (announcement, LWN coverage).

Linux 2.6.36 is released (announcement, KernelNewbies summary).

The first ever GStreamer conference is held in Cambridge, UK (LWN coverage).

The 2010 openSUSE conference is held in Nürnberg, Germany (LWN coverage: The state of openSUSE, The future of LibreOffice, and Making testing easier).

Mark Shuttleworth announces that Unity will be the default desktop for 11.04 ("Natty Narwhal") in preference to the GNOME 3 Shell (ars technica report).

The Consumer Electronics Linux Forum (CELF) announced a merger with the Linux Foundation at the Embedded Linux Conference Europe (ELCE), which was held in Cambridge, UK. (CELF/LF merge blurb and ELCE coverage: The state of embedded Linux and Device trees).

The Yocto project for easing embedded Linux development is announced at ELCE (project home page).

A plugin for Firefox that sniffs web application credentials from wireless networks, called Firesheep, is released (LWN article).

MeeGo 1.1 is released (announcement).

November

The 2010 Kernel summit is held in Cambridge, MA (extensive LWN coverage).

Fedora 14 is released (announcement).

Stormy Peters announces that she is leaving her position as GNOME foundation executive director to work at Mozilla on the open web (blog post)

The Linux Plumbers Conference is held in Cambridge, MA (LWN coverage: LibreOffice and code ownership and Life after X).

Red Hat Enterprise Linux 6 is released (press release).

The Apache Software Foundation issues a warning that it will stop participating in the Java Community Process if the TCK tests are not made available to it; access to the TCK has been promised for some time (Apache statement).

The first MeeGo conference is held in Dublin, Ireland (LWN coverage: Visions of MeeGo, Beyond mobile devices, MeeGo security high-level view, MeeGo security framework).

AMD joins the MeeGo project (press release).

Novell agrees to be acquired by Attachmate, while selling off 882 patents to a consortium owned by Microsoft, Apple, Oracle, and EMC (LWN blurb and article).

GNU's Savannah project hosting site suffers a SQL injection attack that reveals users' encrypted passwords (LWN blurb).

CentOS struggles with its efforts to release its rebranding of RHEL 6 (LWN coverage).

Novell puts out a message to assure those worried that Attachmate will retain the Unix copyrights even after the acquisition closes (brief message).

December

A generic anti-harassment policy for open source conferences is developed in the wake of numerous sexual (and other) harassment incidents (LWN article).

The Linux Foundation publishes its annual kernel development report (announcement).

The openSUSE "Tumbleweed" project to create a rolling release is announced (announcement, LWN coverage).

A Linux client for the Ryzom MMORPG is released (LWN article).

The GRUB bootloader accepts code to support booting from ZFS and releases the code under the GPLv3, without a copyright assignment (LWN article).

KOffice forks (or splits) and becomes the Calligra Suite (LWN article).

The Hudson continuous integration server runs into Oracle interference when trying to change its development infrastructure in yet another example of the software giant not quite understanding free software communities (LWN blurb).

Google announces the availability of Android 2.3 ("Gingerbread"), along with a software development kit and a new flagship phone: the Nexus S (2.3 announcement, Nexus S announcement, code release).

Matt Asay announces his resignation as Canonical's COO in order to join a mobile web application startup (blog post).

The Yocto project has a two-day summit in San Francisco involving 40 members of the embedded Linux community (LWN coverage).

An allegation is made that the US FBI paid to have a backdoor put into OpenBSD's IPSEC implementation, though it is still unclear whether there is any truth to it (LWN blurb, update from Theo de Raadt).

The Apache Software Foundation resigns from the Java Community Process executive board as it previously warned that it would over the availability of the TCK tests (LWN blurb).

Richard Purdie is named as a Linux Foundation fellow to work on the Yocto project and other related tools (announcement).

Several projects announce that they have become licensees of the Open Invention Network, which collects patents for the defense of free software projects (LWN blurbs: Gentoo, The Document Foundation (LibreOffice), and KDE).

FOSS.IN announces that 2010 will be the last year it is held; it has been the premier free and open source conference in India over the last decade or so (LWN posting).

X11R7.6 is released (announcement).

Openwall GNU/*/Linux 3.0 is released, which marks the ten year anniversary of the security-enhanced Linux distribution (announcement).

Comments (none posted)

A happy new year to all

This is the final LWN.net Weekly Edition for 2010; as is traditional, we will be taking the final week of the year off to rest, recover, and get ready for 2011. Many thanks to LWN readers worldwide for supporting us for all these years and for being so fun to write for; it's hard to imagine how we could find a better audience. Happy holidays to all, and we'll see you in 2011.

Comments (9 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

• Security: Linux capabilities for user namespaces; New vulnerabilities in chromium, eucalyptus, kvm, tor,...
• Kernel: SCSI targets; Radar detection with Linux; Realtime group scheduling; NoHZ tasks.
• Distributions: Gargoyle: completely open source and easy to use; DACA project, Openwall, ...
• Development: A first look at Xfce 4.8; Android 2.3, PostgreSQL, Scilab, X11R7.6, ...
• Announcements: MariaDB trademark policy; new OIN members; Friendly conference update; CENATIC Foundation's OSS report, ...

Next page: Security>>