LWN.net Weekly Edition for July 9, 2009

Google Chrome OS and the community

On July 7, Google let the world know about a project called "Google Chrome OS." It is a new operating system, meant to run (initially) on netbooks. As would be expected from Google, there will be a strong emphasis on web applications; much work is also apparently going into fast booting, security, and a simplified user interface. Google promises to open-source the code toward the end of the year; commercial shipments are expected in the latter half of 2010.

Much of the mainstream press sees this move as a frontal assault on Microsoft, and that it may well be. Microsoft appears to have regained the upper hand on the netbook platform for now, but Windows does not come across as a perfect fit for that sort of platform. But it might not just be Microsoft which feels discomfort from this new operating system; it's not clear that this effort will be good for Linux either. Much depends on how Google works with the free software community; past experience suggests that there could be cause for worry.

Those who would criticize Linux like to point at the vast number of distributions available. They charge (rightly) that fragmentation did a lot of damage to proprietary Unix; Linux, they say, is far more fragmented than Unix ever was. In truth, fragmentation has been a relatively small problem for Linux. It is worth spending a moment to look at why.

One of the reasons, clearly, is that all Linux distributions are based on the same kernel. Some distributors apply more patches than others, but it is, for all practical purposes, the same platform underneath. The accelerated development process adopted for 2.6 has helped in this regard; useful code gets into the mainline quickly enough that there is little reason for distributors to patch significant functionality into their own kernels. On top of that, the "upstream first" ethic ensures that enhancements to the kernel are available to all distributors and, thus, to all users.

On top of that, much of the "plumbing layer" on top of the kernel is also common to all distributors. The availability and management of libraries works well enough that it's often possible to move complicated binaries between distributions and expect them to run. That is a high degree of compatibility for a "fragmented" platform. The end result is almost zero lock-in for most Linux users. The ability to move to a different distribution while still running Linux is one of the greatest strengths of the platform; it is a direct manifestation of the value of free software for users. As long as the ability to switch remains such a fundamental feature of Linux, we need not fear fragmentation.

So the real question is: will Google's new operating system play by the rules which have provided such consistency across Linux distributions? The real answer won't be known for some time. But Google Chrome OS will not be Google's first Linux-based operating system; that distinction belongs to Android. So perhaps we can get a foreshadowing of how things will work by looking at what was done with Android:

• The kernel was indeed Linux, but what Android ships is far removed from a mainline release. A great deal of code was added behind closed doors and committed to the platform before any sort of public release or review. Much of it has no real hope of getting into the mainline kernel ever. Even now, Android kernel code, while being available in a public git tree, is developed separately from the mainline. With some small exceptions, nobody from Google is making any real effort to get Google's code reviewed in the wider community or merged into the official kernel.

• The plumbing layer is totally different; Google rolled its own C library for Android. The motivations for this work are not entirely clear, but it does seem that Google has gone out of its way to avoid GPL-licensed code, and code owned by the Free Software Foundation in particular.

• Several of the applications are proprietary.

The end result is that, while Android is based on the Linux kernel, it does not, in its default form, feel much like a Linux system. Ordinary Linux applications do not just run on Android. With effort, one can supplement Android with the features needed to run "normal" Linux; one can even put a full Debian environment onto it. But it's an add-on, not part of the platform itself.

One could argue that Android sits in a special niche: it runs on mobile phones and must, among other things, operate in a way acceptable to handset manufacturers and cellular providers - not always the most accommodating sorts of companies. Google Chrome OS is, instead, aimed at desktop-like applications. It will operate in a niche where ordinary Linux can be found; perhaps, as a result, it will be more like ordinary Linux. Time for a closer look at the announcement:

• Code is to be released "later this year." But this is a project which has been underway for a while, and which will, undoubtedly, proceed quickly during this time. So we are not starting with community-based development; we'll get another code dump some months from now.

• Google is "completely redesigning the underlying security architecture of the OS." How that security model will be enforced is unclear - it could involve kernel changes, or it could be embedded within a virtual machine. Either way, it does not sound like a feature which will enhance compatibility with other Linux distributions. Security is important, and it does not come out well when designed behind closed doors. If Google has a better way to do security on Linux, it should be sharing its ideas and getting community input now; presenting a new security model as a fait accompli months from now will not be helpful.

• There will be a new windowing system; no more details than that are available. How new and different will it be? Will Google Chrome OS be able to run X applications?

The picture which emerges looks a lot like Android: a platform which takes a number of pieces from Linux, but which is not like Linux, and which does not really give back to Linux.

Perhaps that picture is wrong. Perhaps Google is secretly working with one or more Linux distributors, or with projects like Moblin or Maemo, which are doing a great job of achieving many of the objectives Google has set for its new operating system. Just maybe, Google is working to strengthen the projects its work is based upon, rather than trying to supplant them. Possibly, when Google says:

it really means to work with the community and not just absorb work from the community. Your editor very much hopes so, but your editor also recognizes that this would require a different approach to the community than Google has shown in the past.

Android is a good thing: it has brought Linux to a new class of platforms and created a new development community based on free software. The Android developers have taken the time to rethink how the system works and to attempt some innovative new approaches; we can never have too much of that. There can be no doubt that the same will be true of Google Chrome OS; it will be interesting to see what they come up with. But also can be no doubt that Google Chrome OS could be a lot better if it were developed within the community instead of on top of it. Your editor wishes Google the best of luck with this ambitious project and hopes that the larger community will truly be able to be a part of it.

Comments (45 posted)

Identi.ca pushes forward

Identi.ca, the open microblogging site popular with free software advocates, just passed its one-year anniversary — a milestone that happened to roughly coincide with an major upgrade to Laconica, the software package that runs the site. The upgrade brings new functionality to Identi.ca, and lays the groundwork for a new commercial microblogging offering from the free service's creator. The dual commercial and free product lines are an important move for the company, as the free software community seeks business models fit for distributed network services.

Identi.ca is run by Montreal-based Control Yourself. CEO Evan Prodromou chuckles at the term "Twitter clone," but that is how many outsiders refer to the service. The Laconica software implements a Twitter-like web application, and can use the Twitter API to connect to many of the same software applications and third-party services. It has its own, native API, however, based on the Open MicroBlogging (OMB) specification. OMB supports features not found on Twitter (such as groups), and allows federation, meaning a user on any OMB site can subscribe to notices from users on every other OMB site, without the hassle of creating additional accounts.

The 0.8 update to Laconica, codenamed "Shiny Happy People," brought several new features to Identi.ca, including file attachments, page theming, conversation threading, and Facebook support. Behind the scenes, 0.8 adds offline processing of queued messages via Streaming Text Orientated Messaging Protocol (STOMP) queue servers like ActiveMQ or RabbitMQ, reducing strain on the database server and making it easier to scale the service up. A statistics package will optionally report non-privacy-invading data (such as the number of users and messages, version number of dependencies, etc.) back to the Laconica project.

Paid service on free software

Although the public got its first taste of Laconica 0.8 when Identi.ca upgraded, others have been testing the code out in private. Control Yourself's paid service Status.net is not yet open for business, but a private group of invited beta testers have been coming online . Status.net offers customers a fully hosted Laconica microblogging service courtesy of their choice of subdomains (e.g., yournickname.status.net). In Prodromou's preview announcement, he listed several facets of Status.net accounts that distinguish it from a personal account at Identi.ca or another free service, including the ability to incorporate advertising, to make the site private for internal or team use only, to integrate the site with other existing sites or user databases, and to change the license terms attached to status updates (by default, Identi.ca requires a liberal Creative Commons Attribution license applied to all content).

Prodromou anticipates the paid service attracting customers from blogging, media, and corporate circles who want to make use of microblogging but are not interested in the overhead required to run an internal OMB server. He said about 20 customers are already up and running in the private beta, with another 30 on track before the public launch at the end of the summer. Although he could not disclose any names, he described them as "marquee clients" that will help show off the platform.

Status.net will have tiered pricing based on the type of account, and customers' subdomains will be able to be mapped to external domains to better integrate with existing web sites. Prodromou said the service will cater to three distinct classes of client: Enterprise, Publisher, and Community. Enterprise customers will get a private in-house microblogging environment similar to offerings from yammer.com, paying by-the-user. Publisher clients are more interested in the broadcast model, sending out status messages linked back to their own site's material, and delivered to multiple channels including Twitter, the Web, and Facebook. Community service is intended to serve groups and organizations who want to create a focused microblogging community around a specific topic or theme; Prodromou described this as similar to what is offered by ning.com.

He added that Status.net will offer pricing competitive with other players in the microblogging market, but that the new service will beat them "hands down" on features, functionality, and client support. "We also think the flexibility of being able to easily move off our platform makes us a great choice."

Openness changing the landscape

That last sentiment is where open source software breaks from traditional businesses' conventional wisdom. Vendor lock-in is a tried and proven strategy; open source has used freeing customers from it as a selling point in the desktop and server market for years. Prodromou does not see any difference in the network service market. "One of the nice things about Open Source in the cloud is that it gives you a lot of choice," he said — including the ability to change directions if your first vendor fails or changes the terms. "People have Web site fatigue — they're tired of investing time, energy, and social capital into new sites where they're not sure what the endgame of the company is."

Clients can easily migrate from a hosted Laconica service to a rival, or to running their own instance. Individual users and entire sites can export the accumulated "friend" and "follower" relationships between accounts in the public Friend Of A Friend (FOAF) format. Laconica was initially the only implementation of the OMB protocol, but others (such as Google's Apache-licensed JaikuEngine) now support it as well. Prodromou is pleased that other service providers are involved. "I don't think a single-implementation protocol can be robust enough. You need to have lots of implementers stretching the boundaries."

Microblogging is the latest communication tool, but the community can learn from the past. "We haven't had an important communications medium on the Internet succeed without a leading Open Source implementation," he noted, citing SMTP and HTTP as examples. The prime counter-example is instant messaging, which was long dominated by AOL Instant Messenger (AIM), and remains a fractured field to this day. "We have two good, competing protocols (SIMPLE and Jabber) and one good Open Source implementation on the server side. The main IM vendors never got synched on the protocol, and what resulted was a solution based on multi-protocol clients. I think that's been a downside of IM."

Consequently, he is amused by Twitter creator Jack Dorsey's dismissal of Identi.ca as one of "a lot of Twitter clones." Laconica and OMB are considerably more feature-rich that Twitter, he observed, and although the Identi.ca web site preserves the minimalist outlook of Twitter's site, "I don't think we're a copy, though; more of a next iteration on the concept." When he heard Dorsey's remark, "All I could think was, 'You're gonna see a lot more, if we get our way!'"

How now, network service

OMB undoubtedly offers users more than Twitter does; Twitter has even removed popular features like tracking. Prodromou is breaking relatively new ground by trying to support Laconica development with the commercial Status.net service, though.

Control Yourself has been privately funded, underwriting the development time and bandwidth costs of Identi.ca's 70,000 user accounts, but that cannot continue indefinitely. Open source desktop and server software companies have faced the same question for years, and several business models have proven themselves popular and sustainable: consulting on private installations, selling proprietary add-ons, and enterprise support contracts, for example.

The Status.net service is a mix of the private consultation and enterprise support models. Prodromou sees it as similar to the approach taken by SugarCRM and Wordpress, both of which offer their core software as open source, but sustain development with commercial services based on the same code base. "Clearly the choice that they offer users makes people comfortable with putting their time and energy into those platforms. Comparing Sugar to SalesForce.com, or WordPress to Blogger, you can see that open source is their secret weapon — what differentiates them from the market leader."

Bradley Kuhn of the Software Freedom Law Center said that he thinks network services like microblogging are a natural fit for the enterprise support free software business model:

A more direct comparison than SugarCRM or Wordpress might be to Jabber, original creator of the Extensible Messaging and Presence Protocol (XMPP) protocol and its first server. Jabber also ran a free public service at jabber.org and sold enterprise consulting services to support the software and protocol development. By all accounts, the business was successful; Cisco acquired Jabber in September of 2008 for an undisclosed amount. XMPP has gone on to become an IETF standard under the guidance of the XMPP Standards Foundation.

On the other hand, an excellent, open technical standard is no guarantee of success — one needs only to look at the voice over IP (VoIP) marketplace for that. The closed, proprietary, and non-interoperable Skype still dominates the consumer VoIP market, in spite of Session Initiation Protocol's (SIP) technical superiority and IETF endorsement. In 2004, French telecom provider Wengo attempted to compete head-to-head against Skype with a SIP-based, cross-platform VoIP application called WengoPhone, but by 2007 it abandoned the project and left the VoIP business.

There are certainly successful commercial companies in the Internet telephony business, most notably Digium, the sponsor of telephony server Asterisk. Prodromou is not oblivious to the challenges of commercial competition; he co-founded Wikitravel, which competes successfully against much larger and industry-backed travel and tourism web sites. For microblogging, he thinks that Status.net offers customers not merely a private status update system, but a method to build focused communities — a service not possible on the broad social networking sites. "We've heard a lot of people talking about building communities on Twitter or Facebook, and how inter-community communication gets lost in the noise of those general purpose sites." Status.net will allow the world to see what that looks like, and hopefully push microblogging in new directions at the same time.

[ Editors Note: You can follow LWN on Identi.ca as lwnnet, as well as on Twitter as—you guessed it—lwnnet. ]

Comments (10 posted)

Why people don't test development distributions

Development distributions play a crucial role in the free software ecosystem. They are the proving ground where much new software is first exposed to a wider user community; they are also the place where this software demonstrates how well it plays with other packages. Distributors would like to see wider testing of their development releases, but, as your editor's recent experience shows, there are limits to how wide this testing community can be expected to be.

Your editor has a habit of running development distributions on real-work machines. There is no better way to stay on top of what the development communities (at both the distributor and upstream levels) are up to; it's also a way to help the community by finding and reporting bugs. Much of June was spent traveling, though, with the result that these machines were generally on the wrong side of an ocean and, thus, fell behind the leading edge. On return, after shoveling out a horrifying inbox, your editor decided to bring his desktop system up to current Rawhide. After all, what could possibly go wrong?

Anybody who has worked with development distributions for any period of time knows that the early part of the distribution development cycle is when things are most likely to go wrong. That's when the distribution-wide, disruptive changes go in. Traffic on the mailing lists suggested that, after the Fedora 11 release, Rawhide did not disappoint anybody looking to add a little adrenaline to their working day. Still, it seemed that things had settled out a bit; one tester responded to a query from your editor by saying:

So your editor upgraded. Sound stopped working. The screen saver started leaving the display in a weird, low-color-resolution state. And, most annoyingly, the keyboard layout went fully into psychedelic country. Selecting the indispensable GNOME "caps lock is another Control" option yielded a keyboard with no Control key at all; turning that option off restored control - to the Alt-left key. The Alt modifier appeared to be entirely unobtainable - a situation which can only serve to cause extreme misery to any serious Emacs user.

All inconvenient, but, then, development distributions can be like that; one should not venture into that world if one is not prepared to encounter occasional bizarre behavior. Often, in cases like this, the best thing to do is to report the problems and follow the leading edge closely in the hope that fixes will be uploaded soon. So that's what your editor did.

[PULL QUOTE: Your editor, drawing on many years of system administration experience, had come to the reasoned conclusion that it was a good time to run away screaming. END QUOTE] Big mistake. Just before the holiday weekend in the US, somebody uploaded a broken prelink which hosed most important executables on the system. The result was a box which wouldn't boot and which couldn't really even be fixed from a rescue disk. It now seems that running prelink -au * from a rescue disk might be a way for other afflicted users to get their boxes back. By the time that was posted, though, your editor, drawing on many years of system administration experience, had come to the reasoned conclusion that it was a good time to run away screaming.

A helpful hint for development distribution users: have at least one other root-suitable partition set aside on the system. All useful files not directly tied to the distribution should be stored elsewhere. If things get really ugly, one can always boot an emergency backup partition and end up with a usable system. This article is currently being typed using a system kept on such a partition.

Others recommend running development distributions within virtualized guests or on sacrificial boxes. Both of those techniques are useful, but they miss an important point: the best way to find problems in new software is to use it for real work. If people are not trying to actually get things done with a development distribution, they are going to miss a lot of the bugs. Those bugs will then turn up after the (allegedly) stable release, biting users who didn't think they were signing up for alpha-level software. We need people doing more than just convincing themselves that the testing box boots properly.

For this reason, Fedora, like other distributors, would like to see more people testing its development distribution. Your editor would like to see that too; testing of early releases is one of the "prices" that many of us need to pay to help ensure that our free software is as good as we expect it to be. Besides, tracking an evolving system is often fun; it can help to bring users further into our community. But it is hard to tell most users that they should be running a development distribution if it's liable to leave them with a smoking wreckage of a system when they really need to get some work done.

And, it should be noted, problems like this are certainly not limited to Rawhide; Ubuntu testers who updated gdm at the wrong time will certainly be questioning their karma as this is being written.

So, what can be done to make development distributions safer for a wider community of testers? Absolute safety seems unattainable, but there are some things which could be done:

• Create a version of the distribution containing packages which have shown a relatively low level of combustibility. The alpha releases done by some distributors are a step in this direction; there is usually an attempt made to stabilize things a little bit prior to the release. But these releases tend to leave testers somewhat behind the current state of the art. Debian's "testing" distribution is probably the best example of how this can be done on an ongoing basis.

• Provide an indication of the state of the distribution. Many beaches are equipped with red flags which are posted when dangerous currents are present. Wouldn't it be nice if an apt-get upgrade could respond with a message like "the current threat condition is orange, you may want to reconsider"?

• A built-in rollback system which can undo the effects of an ill-advised upgrade, even if the system as a whole has been reduced to rubble. The Btrfs snapshot mechanism should be well suited to this sort of feature - once Btrfs is stable enough to be used on a root partition.

This is an issue which merits some thought. If we can make testing easier and safer, we should end up with more testers. That, in turn, should lead to more stable releases and, just importantly, users who have more invested in the software and the process which creates it. It is hard to see how those could fail to be good things.

Comments (76 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

• Security: DNSCurve: an alternative to DNSSEC; New vulnerabilities in drupal, kernel, mod_security, nagios,...
• Kernel: Some ado about zero; A lockless ring-buffer; Transcendent memory
• Distributions: Tiny Core Linux 2.1: Less is more; Slackware 13.0 RC1; PC-BSD 7.1.1; FreeBSD 8.0-BETA1; The Google Chrome OS.
• Development: Sugar on a Stick Brings Sugar to a Wider Audience, C# and Community Promise, HTML5 dropps ogg, new versions of PyODB, SQLite, Samba, CUPS, Audacious, CELT codec, python-graph, SUMMON, GNOME, GNU Radio, Python gnupg, Wine, Claws Mail, Moovida, VLC, twander, Milepost GCC, Mercurial.
• Press: Tiemann at FISL 10, Gran Canaria Desktop Summit day 1 and 2, Linux desktop adoption, GAMC uses Linux, Smári McCarthy interview, Justin Steinman interview, Pianoteq3 review, unlocking the iPhone.
• Announcements: EFF on ringtone copyrights, KDE e.V board election, new LF director, Akademy Awards, ODBMS.ORG award, OOoCon cfp, PyCon IN cfp, Security Byte cfp, DjangoCon, InsideMobile conf, Perl-IT, IT bottleneck webinar, OpenNICTA launched, DBMS webcast.

Next page: Security>>