LWN.net Weekly Edition for December 16, 2010

Storm clouds

Richard Stallman recently went on record against cloud-based computing, and against Google's ChromeOS in particular. Putting one's personal data on remote servers, he says, necessarily entails loss of control over that data. It is far better to keep one's data on a system which is under one's physical control. As with most things, Richard has been most consistent with this message; he has been saying similar things for a long time. But the increase in cloud-based services - and systems designed to direct users toward them - is adding urgency to this message.

Your editor does not always agree with Richard, but Richard has a point here. We have worked for many years to build systems which we have some degree of control over, with quite a bit of success. Even systems which have traditionally been severely closed - phone handsets, for example - are becoming more hackable over time. A suitably motivated and skilled user can avoid proprietary software, and the long list of antifeatures such software tends to include, much of the time. The situation is not perfect, but things could certainly have been a lot worse.

When our systems become little more than a window into somebody else's server, though, that control disappears. The results are predictable:

• People can come to depend on cloud-based services, but the providers of those services assert their right to pull the plug at any time. The eviction of Wikileaks from Amazon's cloud is a recent, high-profile example, but almost every well-known network-based service is followed by stories of users who have been locked out for seemingly trivial (or nonsensical) reasons.

• Stories of data misuse abound. Facebook puts profile pictures into advertisements served to others. Gmail reads messages and tailors advertisements to match. Email addresses find their way onto spam lists. Many sites track their users' activity across the web as a whole and do their best to monetize that information. And so on.

• Resources in the cloud are cloudy at best; reports that Amazon has resumed deleting books that Kindle owners believed they owned are just the latest example of when can happen when "our" stuff lives at somebody else's will.

• Security breaches and data loss are a common occurrences.

• Many cloud-based services seem to maintain an open-door policy for governmental agencies looking for information. There is no way to know what information has been disclosed to whom.

With regard to the last item above, it is encouraging that a US appeals court has just ruled that email cannot be seized from a third-party provider without a search warrant. But it is highly discouraging that such a ruling was necessary in the first place. Seemingly obvious concepts - like the privacy of email - seem to fall by the wayside when network-based providers are involved.

Given all this, one might well wonder why such services are seeing any use at all. The simple fact of the matter is that they are awfully convenient. A web-based email account is far easier to set up and maintain than an independent mail server - even for those who have the skills to maintain such a system. Anybody who has been through the tiresome experience of moving into a new phone can only be thrilled when that new Android handset automatically downloads the contact list - and all previously-installed applications. Establishing contacts and sharing information is easy on social networking sites - and essentially impossible otherwise. These services have brought a wide range of capabilities and features to a wide community of users; there is clear value in what these companies are providing.

It is well to warn users of what they are giving up when they place their personal information on such a site. Making sure people know when a cloud provider misbehaves is clearly the right thing to do. Many LWN readers heed those warnings and take a great deal of care to limit the information given to cloud providers and to maintain their own infrastructure. But it is futile to tell the rest of the world to avoid cloud-based services when we cannot point them to any alternatives that are useful to them. Such advice will be ignored, and the message as a whole may be lost.

The right response to the cloud problem is to create alternatives which give a higher degree of control - and which are usable by people who have no interest in putting their time into making things actually work. That means solving problems at a number of levels. We need applications which provide a rich experience to users which are not tied to any specific machine; the web is the obvious way to provide that experience, but it might not be the only way. Needless to say, these applications must be free software if we are to trust them at all. We need freedom-friendly policies that raise the bar for what users expect and demand. We need a mechanism for deploying these applications on the net which allows users to easily create and maintain their own instances while interoperating with others. It would be good to contemplate what could be done when terabyte storage on mobile platforms is commonplace - we can always have all of our data in our pockets. With pieces like these in place, we might begin to have a story which can compete with the existing providers.

Something else is needed, though: a means for financing these services must be developed. "Free" is awfully nice, but, as people far wiser than your editor have observed, if you are not paying for a service, you are not the provider's customer - you are their product. That is a relationship which will inevitably lead to conflicts of interest. Establishing a more straightforward relationship between users and providers of online services seems like an important step toward improving both control and privacy. That does not mean getting companies out of the services business - indeed, it could mean the opposite - but it does mean renegotiating the relationship. (And, naturally, companies have all the same freedom and privacy interests that individuals do when it comes to obtaining services on the net).

Recent events have convinced many people that, as we have become increasingly dependent on the net, we have also lost control over it. We may see a more focused effort in the coming years to take back control and freedom at the network level. As with all of these battles, it will be difficult; there is no shortage of powerful interests pushing toward central control. But it's one that we should be able to win.

Comments (45 posted)

Embedded Linux developers meet for a Yocto project summit

On December 1st and 2nd, a group of about 40 leaders and engineers in the embedded Linux community met in San Francisco for a for two-day summit. On the table was the Linux Foundation's Yocto project, which was announced in October at the Embedded Linux Conference Europe (ELCE), and aims to make the development of custom embedded Linux systems simpler and easier. The Linux Foundation wants to see broad industry adoption of and involvement in Yocto, and this meeting was called to provide an overview of the project and to set up a community governance process. As such, the engineers and management from a wide cross-section of the industry were invited including representatives from Intel, Wind River, MontaVista, Mentor Graphics, Linaro, Dell, Texas Instruments, Qualcomm, Red Hat, and HP.

Jim Zemlin, executive director of the Linux Foundation opened the summit and welcomed all the attendees with the results of a survey that attendees were asked to fill out before the meeting. In typical Zemlin style, he pronounced that in the embedded space, "Linux is kicking ass". The bad news is that there is a shortage of well-trained embedded Linux developer and, in particular, system engineers. There is a desperate need to get more people "in the tent". The merger of the Consumer Electronics Linux Forum (CELF) with the Linux Foundation, announced in October, along with the Yocto project creation announced at the same time, means that the Linux Foundation is now directing significant resources toward improving the state of embedded Linux development.

As a first step, the Zemlin announced that Richard Purdie, the leader of the Poky project, has become a Linux Foundation fellow in the role of Yocto project architect.

Before the summit, attendees were surveyed about their view of the Yocto project and what they would need to make it a useful technology. Rather than attempting to analyze the results, Zemlin provided the raw answers to the room (with names removed) so that attendees could make of it what they would. Not surprisingly, it appears that few people are familiar with the Yocto project, and many were hoping to come out of the meeting with a better understanding of what it was and whether or not it would be useful for their work.

An overview of the project

The rest of the morning was devoted to an overview of the Yocto project by Purdie and Dirk Hohndel, Intel's Chief Linux and Open Source Technologist. According to Hohndel and Purdie, Yocto is an umbrella project covering several initiatives with the goal of reducing the duplication of effort required to build custom embedded Linux systems. While there are numerous tools for building embedded systems, such as OpenEmbedded and BuildRoot, all of them still require a lot of pain and effort to stabilize into something usable. Hohndel wants to "raise the bar and produce a great shared base technology which is easy to build products on top of". Yocto aims to achieve this goal by directing effort to improving the build tools, providing tested releases, and improving documentation.

The most prominent component under the umbrella is the Poky build system which is a derivative of OpenEmbedded. Additionally, Yocto sponsors related projects including Pseudo: a fakeroot replacement, Swabber: a tool that detects cross-build contamination from the host filesystems, Eclipse and Anjuta plugins, and an SDK generator. Documentation, testing, and a build/test infrastructure is also covered. An complete list of projects can be found on the Yocto website.

Many questions were asked about the relationship between Poky, Yocto, and OpenEmbedded. Rather than reiterate the discussion here, an openembedded-devel mailing list posting from Purdie provides an excellent overview, and covers all of the points touched on in his presentation.

Poky is closely related to OpenEmbedded, and both projects share the BitBake build tool and much of the build package metadata. Poky is sometimes characterized as a fork of OpenEmbedded, yet it does not appear to be a competing fork since a great deal of care has been taken to maintain compatibility between the two projects, and there is a lot of cross-pollination of metadata and features. Poky differs from OpenEmbedded in its focus and policies. According to Purdie, while OpenEmbedded is a wonderful tool, it is actually quite difficult to base a product on top of it. Most companies using it need to make a large number of changes. Poky, on the other hand, aims to be a well-tested, stabilized set of build metadata in a form that requires far fewer changes for use in a commercial product.

Questions were also asked about the relationship between Yocto and MeeGo, and if the two projects were duplicating effort. The answer is simply that the two projects address completely different segments of the embedded Linux ecosystem. Where MeeGo is about a common application framework and environment with a stable ABI and a consistent interface, Yocto is instead focused on highly customized device-specific embedded systems where all the choices about what is included is up to the system builder.

Much concern was expressed about whether or not Yocto will be effectively perceived as a single vendor project since it is being championed by Intel and its subsidiary, Wind River. Ironically, Intel and Wind River employees were within the group expressing this concern. It would appear that the developers of the project deeply believe in the Yocto approach, and don't want to see it written off as an Intel-only tool. Both Hohndel and Zemlin expressed several times that they really want to see Yocto become a vendor-neutral project that improves the state of embedded Linux for everyone. For this reason, ownership of the Yocto and Poky trademarks have been transferred to the Linux Foundation, and the steering group is open to all eligible member companies.

Project governance

After lunch the meeting split into a technical track and a governance track. The governance track was focused on choosing the structure for the Yocto steering group. Prior to the summit, a governance structure was drafted to be discussed at the meeting. As much as possible, the governance structure is to take a hands-off approach to the day-to-day and technical decisions of the project, while still providing the advantages of a legal entity and to have a mechanism for managing engineering and financial resources.

A steering group will be formed with one seat held by each of the initial participating companies. In the spirit of hands-off governance, the membership agreement is low-key with primary requirements being to maintain at least one full time Yocto engineer and, in Zemlin's words, "don't be a jerk". He wants to protect against the possibility of a company joining the steering group in order to sabotage the project, and so the membership agreement provides a mechanism for removing unfriendly members. It would appear that the history lessons of SCO and UnitedLinux have been learned, and Zemlin wants to protect against the possibility of the entire project getting sabotaged by a single hostile member.

From the companies represented in the governance track, the reception was generally positive. A few people voiced concern that Yocto would be yet another embedded Linux solution to support over and above Android and OpenEmbedded. However, the close relationship between Poky and OpenEmbedded somewhat mitigates those fears and some hope was expressed that the two projects would continue to work together, and possibly even merge.

Concern was also expressed that the Linux Foundation membership requirement effectively excludes independent and community developers from holding a seat on the steering group. According to Zemlin, the intent of the membership requirements are to raise the bar so that only committed companies become involved. The steering group is completely free to waive requirements as needed to make sure that key people are not excluded from participating.

Zemlin proposed polishing up the membership agreement and sending it out to prospective members in the next week or so with the goal of forming the initial steering group by mid-December. Once the initial group is formed, the group itself can vote to change and adapt the steering requirements as needed by the project. In fact, one of the first tasks that will be put to the steering group after it is formed is to review the membership policies and make decisions about how the group should be composed. Questions like how large it should be, how many seats will be held by community developers, and how members are selected, will be addressed.

Preparing for 1.0

Meanwhile the technical track set out to spend the afternoon discussing the current state of the project and what needs to be fixed or completed before cutting a 1.0 release. A lot of time was devoted to discussing the differences between OpenEmbedded and Poky, and by the end of the day the technical track probably spent about as much time on governance issues as the governance track did.

The development model is one of the major differences between OpenEmbedded and Poky, and has been characterized as a push vs. pull model. "Push" refers to the fact that many OpenEmbedded developers have direct commit access to the source repositories. Poky has instead adopted a "pull" model where one developer maintains the canonical source tree and other developers send pull request to the maintainer of said tree (similar to how Linus Torvalds maintains his Linux tree).

OpenEmbedded and Poky also differ in policy about what recipes are accepted into the core repository. OpenEmbedded has historically taken a laissez-faire approach where anything is acceptable, as long as it doesn't break other things, and so it includes a huge number of recipes. Unfortunately it's openness is double-edged sword since many of those recipes are broken, unmaintained, or for ancient versions of software. It is easy to get new recipes into OpenEmbedded, but it is also very difficult to know which recipes actually work.

Poky on the other hand has strict policies about what is acceptable in the core and only maintains recipes that are actually used and tested. Typically Poky will not contain more than two recipes for a given package; one known working version, and an unstable testing version. It also keeps the scope limited to the core recipes required to get a working build. It is not surprising that the 700 recipes currently in Poky are an order of magnitude fewer than the 8000 in OpenEmbedded.

Instead of adding recipes to the core, Poky is extended by using "layers". Layers are a mechanism for adding additional recipes for things like board support, special toolchains, new architectures, and applications on top of the Poky core. Since anything in Poky can be overridden or extended with a layer, Poky users have lots of flexibility to adapt it to their needs. Mark Hatle commented that a typical Wind River build will consist of about 7 layers; the core, a toolchain layer, the kernel layer, a board support package layer, and one or more user-space layers.

Poky and OpenEmbedded

The push vs. pull debate came to the forefront when the idea of merging Poky with OpenEmbedded was raised. Some developers in the Yocto project would like to see Poky work more closely with OpenEmbedded, and effort has already been expended in that direction. Koen Kooi, the founder of the OpenEmbedded-based Ångström project and a Texas Instruments engineer, stated that he's prototyped using the OpenEmbedded recipes required by Ångström as a layer on top of Poky. It isn't a full OpenEmbedded port, but it does demonstrate feasibility.

Specifically, the proposed idea is to use Poky as a "common core" layer for both the Yocto and OpenEmbedded projects, and OpenEmbedded would be maintained as layers on top of Poky. The advantage would be a larger block of common functionality being used and tested by both projects, which presumably would result in a better quality core. Several OpenEmbedded developers were in the room, and while all seemed favorable to the idea, concern was expressed that the OpenEmbedded community at large would not take kindly to the change for a few reasons.

Requiring the developers to switch to a pull model for the core code is the first concern, but it ended up being easy to address. Since each layer gets its own source repository, layer maintainers can make their own decisions about development process and commit access. Plus, since anything can be overridden by a layer, an OpenEmbedded layer would be able to change anything in the core code that doesn't fit OpenEmbedded's needs.

Second was the concern that it would be viewed as a "hostile takeover" of OpenEmbedded by Poky. Much hand wringing and worry accompanied this possibility and a fair bit of time was consumed by this discussion. Politically, any such move would require the agreement of the OpenEmbedded board of directors, and have general assent from the development community at large.

There was also some discussion about the name, and whether or not using the "Poky" name would be palatable to the OpenEmbedded community, or if it would be viewed as diluting the "OpenEmbedded Brand". However, this seems to be the least of these concerns as both Poky and OpenEmbedded developers expressed flexibility on naming issues if it actually becomes an issue.

While many of the concerns raised during this discussion were important and needed to be considered, there isn't much evidence that there is actually a problem yet. The OpenEmbedded developers took an action item away from the meeting to continue discussing using Poky as the base layer for OpenEmbedded with the rest of the developer community, with the intent of having some resolution by the end of December. Since a proposal to include OpenEmbedded representation in the Yocto steering group is also on the table, the initial formation of the steering group will probably be delayed until early January to give the OpenEmbedded community time to resolve their concerns and to nominate a representative.

After that, the summit seemed to wind down and focus on technical details or the finer points of what Yocto would provide. Tim Bird's statement that he's always like the design of OpenEmbedded, but has never been able to get it to work became somewhat a mantra for the remainder of the event, and "solve Tim's problem" was often heard in relation to making Yocto a reliable and easy to use set of tools. The Yocto 1.0 roadmap was touched upon, as well as work needed on the BitBake user interface and the IDE integration features. More questions were asked to clarify details about policy and governance issues, but for the most part no more big issues came up.

A number of action items were identified as the summit wound up. The business folks from represented companies are going to get together and come back with some form of joint declaration of support for the project in the next month. The OpenEmbedded representatives will discuss with the rest of their community about closer cooperation with Poky and Yocto. Jon Masters asked if there was a regular technical conference call for the project. Hohndel replied that there is currently an Intel/Wind River call, but that the project needs to decide if continuing with a conference call is the best way to communicate.

Clearly the Yocto project is still in its infancy and there is a lot of work to be done before it can truly be considered a vendor-neutral project. However, judging from this meeting and from an informal poll of attendees afterward, the project seems to be off to a good start and it is likely to be adopted by the major embedded Linux and silicon vendors. Overall the summit ended with a positive and optimistic tone any by all indications Yocto is a project that embedded Linux engineers should be keeping their eyes on.

Comments (8 posted)

The 2010 Linux and free software timeline - Q3

Here is LWN's thirteenth annual timeline of significant events in the Linux and free software world for the year.

In what is becoming a fairly standard pattern, 2010 brought various patent lawsuits, company acquisitions, new initiatives, and new projects. It also brought new releases of the software that we use on a daily basis. There were licensing squabbles and development direction disagreements—all things that we have come to expect from the Linux and free software world over a year's time. Also as expected, though, were the improvements in the kernel, applications, distributions, and so on that make up that world. Linux and free software just keep chugging along, and we are very happy to be able to keep on reporting about it.

Like last year, we will be breaking this up into quarters, and this is our report on July-September 2010. Sometime in the next week or two, we'll put out the timeline for the last quarter of 2010.

• January-March 2010
• April-June 2010

---

This is version 0.8 of the 2010 timeline. There are almost certainly some errors or omissions; if you find any, please send them to timeline@lwn.net.

LWN subscribers have paid for the development of this timeline, along with previous timelines and the weekly editions. If you like what you see here, or elsewhere on the site, please consider subscribing to LWN.

For those with a nostalgic bent, our timeline index page has links to the previous twelve timelines and some other retrospective articles going all the way back to 1998.

July

Python 2.7 is released as the last major version in the 2.x series (announcement).

3D graphics drivers that require proprietary user-space drivers (or "blobs") are blocked from inclusion in the kernel by graphics maintainer Dave Airlie (LWN coverage).

OpenSolaris governing board issues an ultimatum to Oracle threatening dissolution if no contact person is put forward by Oracle (news article).

The first draft of an Open Source Hardware Definition is released (draft, version 1.1).

Akademy, KDE's yearly conference, is held in Tampere, Finland (wrap-up press release).

ISO changes its standardization processes to avoid some of the abuses seen in Microsoft's OOXML push (LWN coverage).

The Battle for Wesnoth struggles with possible GPL violations because of its appearance in the Apple App Store, which to some extent parallels the Gnu Go problems in May (LWN article).

Ubuntu experiments with open font development, though the closed beta of the Ubuntu font does raise some eyebrows (LWN coverage).

The Women's Caucus publishes recommendations for increasing the participation of women in open source (recommendations).

The GNOME Users and Developers European Conference (GUADEC) is held in The Hague, Netherlands (LWN coverage: Luis Villa keynote, GNOME 3 release plans, Privacy, encryption, and the desktop, GNOME Shell, Banshee, and GUADEC notes).

OSCON is held in Portland, Oregon (LWN coverage: "Open phones" and Building communities).

Jos Poortvliet becomes the new openSUSE community manager, succeeding Joe "Zonker" Brockmeier (announcement).

The EFF wins three DMCA exemptions for cellphone unlocking, cellphone "jailbreaking", and fair use of DVD content (press release).

GNOME and KDE announce a second Desktop Summit to be held in Berlin, Germany in August 2011—it will combine GUADEC and Akademy much as was done at the Gran Canaria Desktop Summit in 2009 (announcement).

August

Linux 2.6.35 is released (announcement, KernelNewbies summary).

AppArmor, the long out-of-tree kernel security module, is merged for the 2.6.36 kernel largely due to the efforts of Canonical (LWN blurb).

BusyBox once again prevails in a GPL enforcement suit (Groklaw coverage).

The fourth Linux Storage and Filesystems Summit is held just prior to LinuxCon (LWN reports: day 1 and day 2).

GNOME announces a new copyright assignment policy such that new modules that require copyright assignment need to get explicit approval (announcement, guidelines).

LinuxCon is held in Boston (LWN coverage: MeeGo, Media panel, One billion files, Two bootcharts, and LinuxCon moments).

Illumos, an fork of OpenSolaris, is announced in the wake of much uncertainty in the OpenSolaris community (LWN coverage).

The Linux Foundation announces the Open Compliance Program to help companies ensure they are complying with the free software licenses of the code they use (Linux.com article).

Oracle sues Google over the Dalvik Java reimplementation that's used in Android; the suit is for both patent and copyright violations (LWN's thoughts).

Allison Randal is named as Ubuntu Technical Architect in addition to her role as Chief Architect for the Parrot virtual machine (announcement).

Vim 7.3 is released after two years of development on "Vi IMproved" (announcement, LWN review).

The OpenSolaris governing board resigns en masse as expected due to Oracle's unwillingness to appoint a liaison to the board (Simon Phipps's blog posting).

LinuxCon Brazil is held in São Paulo (LWN coverage: Linus & Andrew and Consumers, experts, or admins?).

CyanogenMod 6.0—replacement firmware for Android phones—is released (LWN blurb and review).

September

Linux 2.4.37.10 is released for those still hanging onto the 2.4 series (announcement and 2.4 EOL plans).

The Mozilla Labs Gaming project is announced to foster web gaming (LWN blurb).

Microsoft's CodePlex.com code hosting site donates $25,000 for Mercurial development, assisting with project lead Matt Mackall's efforts to fund his work on the distributed version control system (DVCS) (announcement).

Linus Torvalds becomes a US citizen, which delays some patch testing while he registers to vote (lkml mention).

Broadcom releases an open source driver for its current wireless chipsets, but notably does not free the firmware for earlier chipsets (announcement, LWN coverage).

Fedora decides not to ship systemd in Fedora 14 in a rather late-breaking change; it should reappear in Fedora 15 (FESCO meeting minutes, related LWN coverage).

The Mageia community fork of Mandriva is announced (announcement, LWN article).

PostgreSQL 9.0 is released (announcement, LWN article).

Diaspora makes its first code release of the alternative free social networking platform, though there are some major security concerns with the early release (announcement, security issues).

Qt 4.7 is released (announcement, new features).

Oracle updates the kernel used in its RHEL-based "Unbreakable Linux", moving from 2.6.18 to 2.6.32 which may show other enterprise Linux vendors that they don't have to drag old kernels forward forever (LWN blurb).

The LibreOffice fork of OpenOffice.org is announced by a large portion of the OOo development community (announcement, LWN interview with Michael Meeks).

Linux-Kongress is held in Nürnberg, Germany (LWN coverage: GSM security).

LinuxCon Japan is held in Tokyo (LWN coverage: Stable kernels, Kernel messages, and SSDs and the block layer).

GNOME 2.32 is released as the last in the 2.x series (announcement, new features).

Comments (none posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

• Security: OATH; New vulnerabilities in bind, exim, Firefox, kernel,...
• Kernel: Dcache scalability and RCU-walk; Unmapped page cache control; Linux Kernel Development, third edition.
• Distributions: Lightweight Portable Security; DebXO, OWL, Debian. openSUSE, Mint, Ubuntu, ...
• Development: Behind the KOffice split; Denemo, Modula-2, MySQL, PacketFence.
• Announcements: ASF resigns from JCP executive committee; Richard Purdie appointed as LF fellow; CELF proposals; apache-extras.org; articles from Guardian, opensource.com, ...

Next page: Security>>