The N9 and the future of MeeGo at Nokia

The story has been told many times at this point: just as work on MeeGo was reaching some sort of stable point, Nokia, one of the principal partners in the MeeGo project, went through a change in management, dropped MeeGo, and committed the company to a close partnership with Microsoft. MeeGo remains under development, but, with the primary handset manufacturer gone, the mood is somewhat subdued. Nokia's plan was always to produce one MeeGo handset, though; that handset has now been announced in the form of the Nokia N9. Is the N9 a one-off device, or might there be more where it came from?

If your editor had an N9 in hand (hint), he could write a proper review of it; in the absence of such luck he'll have to take the word of others. By all accounts, the N9 is a nice device indeed. The hardware is nicely designed with entirely respectable specifications. The software is said to be the realization of much of the potential that many of us have seen in MeeGo. All told, it seems to be a clear statement that, despite indications to the contrary, Nokia can indeed make an interesting, competitive handset in 2011. It's the touchscreen-based smartphone that Nokia desperately needed to make.

There have been some questions as to whether the system running on this phone is actually MeeGo or not. By the terms of the MeeGo specification, it is not; it retains too much of its original Maemo heritage. That said, Nokia did obtain permission to use the MeeGo trademark with this system, which has been dubbed "MeeGo 1.2 Harmattan," so it is, in some sense, MeeGo. One thing this version of MeeGo does have is Qt as the graphics toolkit, so it does show Nokia's intended direction for the MeeGo user interface. Of course, the use of proprietary applications at the "user experience" level has also been part of the MeeGo plan from the beginning.

Given how well this phone has been received, one might well hope that Nokia might reconsider its plan to move away from the MeeGo platform. After all, it has demonstrated that it can produce an interesting handset based on some version of MeeGo; why not continue, especially if the N9 sells well? Unfortunately, it does seem like Nokia is not hugely interested in making this phone a success. Developers will be understandably reluctant to put time into creating applications for a one-off device. The "check availability" page lets potential customers request a notification when the phone becomes available - but not if they are in the United States or much of Europe. And then there is the matter of that deal with Microsoft; the folks in Redmond may think that the billions of dollars apparently sent to Nokia should be enough to keep competing operating systems off Nokia's smartphones.

So the situation looks grim. Still, the world can be a surprising place, especially where corporations are involved. An awful lot could happen over the course of the next year. The N770 and N800 tablets did not have clear successors either - they weren't even phones. The N900 was expected to be a one-off as well. So history suggests that Nokia might yet see an interest in creating more MeeGo-based phones in the near future, regardless of what has been said in the last few months.

For those who like to read tea leaves, there is this posting from Nokia's Quim Gil. Quim says that there are four important software components to the N9: Qt, the Linux kernel, WebKit, and the "swipe" user interface. Nokia, he notes, continues to invest in the development of all of these components. He says:

One could speculate endlessly on what "a form or another" means; it might not be handsets. Regardless, it seems possible that MeeGo is not entirely dead at Nokia. A breeze from the right direction might just get things moving in a more interesting direction again.

That breeze could come from the corporate direction; Stephen Elop, having presided over a nearly 50% drop in Nokia's stock price in just a few months, might struggle to retain his position. For a number of reasons, Windows has struggled to gain any serious success in the handset market; that track record may not change in the next year or so. For some wilder (and voluminous) speculations, see this article suggesting that Microsoft's acquisition of Skype has turned the carriers against Windows-based handsets and that Nokia is in the middle of a big about-face.

What will come of all this is unknowable; expecting rational, consistent, or predictable behavior from corporations is a good way to be surprised. But Nokia may just change its mind about MeeGo phones again; we may also see other manufacturers develop an interest in Linux-based alternatives to Android. All we really know is that the N9 has seemingly shown the world how good a MeeGo-based handset (for some value of "MeeGo") can be. Hopefully good things will come from that.

Comments (30 posted)

Echoprint: Open acoustic fingerprinting

Acoustic fingerprinting has been given a tremendous boost by the mobile smartphone business. You have probably seen the basic scenario in television commercials, if not in person: a user holds up a phone to capture a few seconds of audio playing nearby, and the application computes a "fingerprint" of the track, which is then used to query a remote database for the mystery artist and track name. The space has been dominated by proprietary software, but a new — and open source — project was unveiled last week, named Echoprint.

Fingerprints on the databases

Despite the name, acoustic fingerprinting has little in common with hash-based digital fingerprinting techniques used to detect alteration of a file. While a hash function is sensitive to changes in individual bits, an acoustic fingerprint function must robustly analyze the way the audio sounds, in a manner independent of the codec used, bitrate, or even static and ambient noise. Acoustic fingerprints focus on extracting perceptual data from the audio track, such as its tempo, average spectrum, and pattern of recurring tones. The canonical use is to discover the track information of an unknown audio clip, but other uses are possible as well, such as finding similar-sounding music (on any number of factors supported by the algorithm).

The proprietary software market is home to several acoustic fingerprinting services, most famously Shazam, SoundHound, and Gracenote. Gracenote is known to many in the free software community due to the controversy that erupted a decade ago when its corporate parent suddenly restricted the usage of CDDB, its user-built compact disc identifying database. Many felt betrayed by the policy change, because the CDDB data was submitted voluntarily by users when playing or ripping CDs, not entered or harvested by CDDB's owners themselves. The database was an early example of Internet crowd-sourcing, and many saw the sudden cut-off of access as outright exploitation of their effort.

Fast-forward to 2011, and most open source applications use competing, "open content" services instead, such as MusicBrainz, which is managed by the 501(c)(3) nonprofit MetaBrainz Foundation. For the past several years, MusicBrainz has supported acoustic fingerprinting through the closed-source MusicDNS service provided by MusicIP (later renamed AmpliFIND).

Although MusicBrainz had a perpetual contract with AmpliFIND for the service, it was never considered a good fit, since MusicBrainz's social contract requires it to remain "100% free". Recently, a handful of open source acoustic fingerprinting projects started picking up steam — such as Lukáš Lalinský's Acoustid — and MusicBrainz decided to start looking for open source and open content replacements for MusicDNS. Around the same time, the team at acoustic fingerprinting startup Echo Nest decided that its best strategy was to take its entire product open source and attempt to commoditize acoustic fingerprint services, rather than attempt to take on the entrenched players head-to-head.

Echo Nest and MusicBrainz had collaborated in the past on projects such as Rosetta Stone — a utility to match artist and track IDs between various music services' ID databases — so the mutual decision to launch Echoprint as an open project and begin integrating it with MusicBrainz was a good fit from both perspectives. But it did not hurt that AmpliFIND also sold off its intellectual property holdings — including MusicDNS and the portable unique identifier (PUID) database — to none other than Gracenote.

The Echoprint release

The Echoprint system consists of three components. The Codegen fingerprint generator takes an audio file (or audio sample) as input, and generates a fingerprint based on the Echo Nest Musical Fingerprint (ENMFP) algorithm. The Echoprint server maintains a database of fingerprints indexed to track information, and supports remote queries as well as inserting new fingerprints and tracks. The Echoprint database itself contains publicly-accessible track and fingerprint data. The database contains fingerprint codes for the entire duration of each track, but as in most acoustic fingerprinting techniques, only a shorter segment is usually sent for comparison. Echo Nest claims that Echoprint provides accurate matches for fingerprint blocks computed from samples of at least 20 seconds in length.

In practical usage, an application would sample audio (either captured or from a file), use the Codegen library to compute a fingerprint, and query a compatible Echoprint server. The server would return any matching track records in JSON format. Alternatively, if there are no decent matches, the application could submit its fingerprint information to the server's database along with track metadata acquired through some other means.

The code for Codegen, the server, and various utilities (including an example iPhone app) are hosted at GitHub. The Codegen application and shared library are available under the MIT license, while the server (which is based on Apache Solr and Tokyo Tyrant) are under the Apache License 2.0.

The public Echoprint database is provided under its own terms, dubbed the "Echoprint Database License". It allows for commercial and noncommercial usage, and requires that anyone who downloads the data and adds to it contributes the additional data back to Echo Nest. That clause is something less than a Creative Commons-style "Share Alike" requirement, because it requires sending the data to Echo Nest alone. The preamble to the license seems to indicate that all such contributions will be shared with the public, but Echo Nest assumes no obligation to share the data. The initial release is "seeded" with approximately 13 million fingerprints generated from online music vendor 7Digital's digital holdings, with metadata provided by MusicBrainz.

There are some other potentially worrisome terms in the agreement, including a requirement to use Echoprint "powered by" logos in any application that accesses the data. In addition, the agreement is not clear about how Echo Nest can modify or terminate the agreement down the road. For those who were burned by the CDDB debacle, this agreement should give them pause as it is not at all clear that the same couldn't happen with the Echoprint database.

At the moment, Echo Nest has not published the details of its algorithm in a form suitable for casual reading. The source code for Codegen is provided, of course, but a white paper is supposed to be released shortly that will explain the process at length. Unfortunately, the current legal documents do not explicitly address patent grants in relation to the software (the MIT license is very brief), which might concern some developers. Acoustic fingerprinting is a patent-laden field, and indeed a little searching reveals several relevant filings in the name of Echo Nest and its founders Brian Whitman and Tristan Jehan. On the plus side, all of the proprietary acoustic fingerprinting services are in roughly the same position.

Currently Echo Nest's own "song/identity" server is the only up-and-running Echoprint database, although obviously any application authors could set up their own servers for testing purposes. The Codegen command-line application will build on any reasonably modern Linux system; the only significant dependencies are TagLib, Boost, and FFmpeg. The application generates a fingerprint from a file argument (optionally followed by a start time and duration, both in seconds). The output is a JSON object including ID3 tag information from the file and a base64-encoded representation of the fingerprint. This output can be posted directly to the Echo nest server with cURL or a similar tool, as documented in the Codegen README file.

Play or Pause

MusicBrainz's Robert Kaye said that the project plans to retain support for PUIDs and MusicDNS in the MusicBrainz database for the foreseeable future (or until "people pester me to get rid of it."). The project is running a test server that uses Echoprint in lieu of MusicDNS, but there is no time frame to add tables to the main database to support Echoprint.

Kaye said that he expects more tuning to be done to the Echoprint product before it is ready for widespread adoption, but he observed that "critical mass" is the most important factor — meaning support in client applications and a sizable database of reliable fingerprints. The 13 million tracks pre-loaded with 7Digital's help may sound like a lot, but for comparison, Shazam claims more than one billion songs in its database to have identified more than one billion songs.

Given the number of open source audio projects that use MusicBrainz, it is safe to say that Echoprint has its foot in the door. It is the first entirely open source acoustic fingerprinting system to hit the market in "ready to use" form, so it may spawn considerable development of song recognition in open source mobile applications. Without the burden of licensing fees, the technology could spread beyond stand-alone song-recognition-apps, open or closed.

Nevertheless, Kaye emphasized that MusicBrainz post-MusicDNS move is meant to make the project agnostic to acoustic fingerprinting algorithms. Acoustid is still in active development, too, has documented the details of its algorithm, and does not require changing the MusicBrainz database format for support.

Whether the two fingerprinting techniques overlap, complement, or compete may ultimately be up to the users to determine. Echoprint is so new that it is difficult to predict where it will go from here. The MusicBrainz support is naturally a big boost, but better technical documentation and clarification of the fuzzy legal questions may be required before application authors can be expected to pick up the technology in large numbers. But without doubt, it is poised to fill a visible hole in open source mobile software. An open solution that works well with the crowd-sourcing techniques needed to build the fingerprint database will likely have staying power in a niche with so many similar proprietary offerings.

Comments (11 posted)

Who is that code for?

Free software development projects put a lot of thought and energy into what they are trying to create. Some projects also put effort into communicating their goals clearly to the world. Arguably fewer projects ponder the question of who their users are. But a project's vision of who it is developing for deeply influences the code it produces and the way it manages its releases. Some high-profile projects have been the target of extensive criticism storms recently; arguably the real problem in these incidents has been confusion over who the target users are. In particular, when groups find that they are not seen as users by a project that they had thought was keeping their interests in mind, they tend to get upset.

The recent examples are fairly obvious. Consider the firestorm which has resulted from the Firefox 5 release, the quick abandonment of Firefox 4, and the prospect of the same thing happening again in the near future with Firefox 6. For individual desktop users, this release scheme may work just fine, depending on how many extensions break (your editor reports with dread that this update broke Firemacs - and, thus, Emacs keybindings - in the browser). If, instead, you run a corporate network where software must go through an extended approval cycle prior to deployment, losing support for a major release constitutes a big setback. For "enterprises" with this type of policy, Firefox is increasingly unsuitable; it is thus not surprising that a lot of complaints have come from that direction.

The response from Firefox developers has been clear: the Firefox project is not targeting enterprise users. Firefox is successful because it was adopted by individuals, not their employers, and the plan is to continue to target those individuals. This view can be seen in the draft vision statement under consideration by the project now:

Making life easier for corporate information technology managers does not really figure into this plan. One can argue - as many have - that this approach can only serve to cement the role of Internet Explorer in corporate settings, but that is the choice the Firefox developers have made.

The other recent, high-profile misunderstanding with regard to target users is the GNOME project, and the GNOME 3 release in particular. There can be no doubt that some users are upset by the changes brought by GNOME 3 and GNOME Shell, even if there is disagreement over how large that group is. GNOME developer Bastien Nocera recently made it clear that at least some of those people are not in the GNOME project's target audience:

The project has made numerous other decisions which implement that same view of its user community. In this case, many of the affected users felt that, once upon a time, they were somebody that the GNOME project cared about. If GNOME has ever truly targeted users who care about their terminal emulator, it has since seen greener grass elsewhere and left those users behind. Those users have made their feelings known; at this point, the best thing to do is almost certainly to wish GNOME success with the users it is targeting and, if GNOME is no longer a suitable working environment, move on to something else that is.

The community is full of examples of how a view of the users affects the way the code is managed. Consider PostgreSQL; this is a project which does see "enterprise" users as interesting. PostgreSQL developers also assume that their users care a lot about their data. The results include a highly conservative, review-intensive patch merging policy and a five-year support policy. PostgreSQL 8.2, released in late 2006, will continue to be supported through the end of this year. The wait for new features may be longer than one might like, but rapid feature development is usually not at the top of the "must have" list for users of database management systems.

For a completely different view, see OpenBSD, which only grudgingly admits the existence of a user base outside of its development community at all. OpenBSD leader Theo de Raadt put it clearly a while back:

This position evidently works for this particular project, though it does lead to occasional misunderstandings when users forget their place.

Other examples abound. Git and Mercurial are similar tools with different views of their users - a difference especially felt, it seems, by Windows users. Fedora and Red Hat Enterprise Linux share a common ancestor and a lot of code, but they are developed for different people. Prior to the early 2.6.x days, the kernel community (mostly) saw itself developing directly for end users; the even/odd development process was a direct result of that perception. Current kernels are developed (mainly) for distributors, with an associated focus on frequent stable releases and rapid merging of features. Pulseaudio targets different users than JACK. And so on.

It is a rare project that can be all things to all people; projects which try often do not succeed. A clear idea of who the users are can do a lot to focus a project's activity, attract like-minded developers, and increase adoption. So projects need a good idea of who they are developing for; they also need to communicate that vision clearly. If a project does not prioritize the needs of specific users, it is best if those users understand that before they invest a lot of time into the software. Users, in turn, should pay attention: just as we would not expect a video editor to do source code highlighting, we should not fault a project which is clearly targeting a specific group of users for not meeting the needs of others. We are a large and rich community; if one project does not meet a specific set of needs, there are probably several others with a different and more suitable focus.

Comments (54 posted)

The next weekly edition will be published July 8

The US Independence Day holiday falls on Monday, July 4, so next week's weekly edition will be delayed until Friday, July 8 (in the early hours UTC for those who can't wait). Too much sun and food, along with some fireworks, are in our plans for Monday, but whether you celebrate the holiday or not, we wish all of our readers a happy 4th of July.

Comments (none posted)

Fedora reexamines "trusted boot"

When we looked in on Fedora's decision on enabling "trusted boot" a little over a year ago, the consensus seemed to be that adding a feature that depended on a closed-source binary firmware blob ran counter to the distribution's goals. Since then, that blob is in the process of being added into the BIOS of various systems—though it remains closed—and the "tboot" (trusted boot hypervisor) package has been added to Fedora. Those changes might make it more attractive for Fedora to provide a way for users to enable it at install time. But there is still resistance to adding the feature, and it didn't make the cut for Fedora 16. It seems likely to come up again for Fedora 17 or beyond, so it may well make its way into the distribution sooner or later.

Trusted boot is a way to use the Trusted Platform Module (TPM) hardware that comes in many systems to only boot signed binaries. The fear is that it will enable hardware vendors to lock down their systems—by requiring binaries signed by their keys in order to boot—but it could also be used to protect against various kinds of malware. As with any hardware-based integrity scheme, it all depends on who holds the keys.

Matthew Garrett opened the discussion on fedora-devel, pointing to a proposed feature on the Fedora wiki. That wiki page is fairly thin on details, which is one of the things that sunk the proposal this time, but it essentially proposes changing the anaconda installer to allow users to choose trusted boot and to provide a way to change the GRUB configuration to support it.

One of the first questions was about the benefit of the feature to Fedora. Camilo Mesias asked about the use cases for trusted boot. Daniel Walsh listed a number of possibilities:

Several of those use cases rely on the "remote attestation" feature of trusted boot, which means that a suitably configured system that has the "proper" keys stored in the TPM can provide a signed cryptographic hash corresponding to the kernel in use. There was some confusion in the thread about what remote attestation means, but, beyond that, there are concerns that it could be used to thwart users installing their own kernel, even on free software OSes. As Gregory Maxwell put it:

A bigger problem might be that services start requiring some proprietary OS, rather than a particular Fedora kernel. That would, of course, suit the proprietary OS vendors just fine. But that could happen regardless of what Fedora does. If requiring a Fedora kernel is important, services can require that only approved kernels be used and require that users enable and configure trusted boot in order to access them. The more likely outcome under that scenario would leave Fedora (and other Linux distributions) out in the cold, of course.

While the closed-source nature of the SINIT blob (aka SINIT AC), which is the code that does the low-level integrity checking, worries some, Intel is evidently adamant that the code remains closed. From a practical standpoint, having the code wouldn't do much for users, as Miloslav Trmač points out:

As was noted in the thread, we are already largely at the mercy of the hardware and BIOS vendors with respect to the code that runs on our systems. The relatively recent System Management Mode (SMM) additions to the BIOS are just one example. As Adam Williamson put it:

But, the presence of Intel signing keys stored in the TPM, which is required to verify the SINIT blob, may lead to some worrisome restrictions. Björn Persson points out that there are alternative, free software BIOS implementations (e.g. Coreboot), but that it may be impossible to replace SINIT:

Aside from the technical issues, and concerns, the actual feature request on the wiki didn't provide a whole lot of information about how the feature would be used, and how it would be integrated into the Fedora install and boot processes. What facilities would be present for remote attestation, how they could be disabled by users, and what happens when a user tries to boot an "untrusted" kernel are all left up in the air in the feature proposal. It also didn't provide much in the way of justification for why trusted boot would be useful to Fedora users. It is clear to some that it could be useful, if users could create and install their own keys, but the wiki page didn't really make that clear. That led to several requests for more information to be added to the feature request.

The Fedora Engineering Steering Committee (FESCo) took up the request at its June 27 meeting. The discussion in the IRC log of the meeting paralleled that of the mailing list to some extent. The FESCo members seemed a bit puzzled by what exactly was being proposed—and why it would be useful. As Kevin Fenzi (aka nirik) said: "I think there's cool things we could use this for, but 'enable it now so we can do ~wave hands~ later' seems non ideal. Why not set it up to do something our users care about and _then_ enable it."

There was also some discussion of which pieces of Fedora needed changing (anaconda and grubby being two obvious places that might require modification). But the overall sense was that there just wasn't enough meat in the proposal to add it as a Fedora feature. Fenzi is optimistic that it could be done right for Fedora users, but it needs to be fleshed out more:

So FESCo decided to decline the feature for Fedora 16, but agreed that interested parties should continue working on it for possible inclusion down the road. Any work that is done in anaconda, grubby, or elsewhere will have to be negotiated between the trusted boot proponents and the other projects, as there is no FESCo mandate for the feature. It would not be surprising to see this feature come up again in six months or a year.

The TPM and "trusted computing" (aka "treacherous computing") evoke strong reactions. While it is clear that they can be used in ways that essentially lock users out of their own hardware, they could also be used in ways that allow users to ensure the integrity of the code running on those same systems. Like many technologies, trusted boot can be used for good or ill.

If Fedora can provide ways for its users to take advantage of this technology, without being taken advantage of, it will definitely be something that some security-conscious users will benefit from. Intel's secrecy with regard to the SINIT blob is somewhat troubling, but doesn't change the closed-source BIOS problem all that much really. A bigger problem for Fedora down the road may be trying to support users who enable the feature but run into problems getting their Fedora (or custom-built) kernels to boot. But, that's a problem for another day.

Comments (33 posted)

Security quotes of the week

-- Will Stephenson

-- Tom Henderson

-- Loz Kaye

Comments (5 posted)

asterisk: denial of service

Package(s):	asterisk	CVE #(s):	CVE-2011-2216
Created:	June 27, 2011	Updated:	July 13, 2011
Description:	From the Red Hat bugzilla: A denial of service flaw was found in the way Asterisk processed malformed Contact headers in SIP calls. A remote attacker could use this flaw to cause asterisk server crash via specially-crafted Contact header sent in a reply upon initialization of a SIP call.
Alerts:	Fedora FEDORA-2011-8983 asterisk 2011-07-02 Fedora FEDORA-2011-8319 asterisk 2011-06-14

Comments (none posted)

curl: exposed client credentials

Package(s):	curl	CVE #(s):	CVE-2011-2192
Created:	June 24, 2011	Updated:	March 6, 2012
Description:	From the Ubuntu advisory: Richard Silverman discovered that when doing GSSAPI authentication, libcurl unconditionally performs credential delegation, handing the server a copy of the client's security credential.
Alerts:	Gentoo 201203-02 curl 2012-03-05 openSUSE openSUSE-SU-2012:0199-1 curl 2012-02-09 CentOS CESA-2011:0918 curl 2011-08-14 Mandriva MDVSA-2011:116 curl 2011-07-22 CentOS CESA-2011:0918 curl 2011-07-06 Scientific Linux SL-curl-20110705 curl 2011-07-05 Red Hat RHSA-2011:0918-01 curl 2011-07-05 Fedora FEDORA-2011-8640 curl 2011-06-24 Debian DSA-2271-1 curl 2011-07-02 Fedora FEDORA-2011-8586 curl 2011-06-24 Ubuntu USN-1158-1 curl 2011-06-24

Comments (none posted)

gdk-pixbuf2: excessive memory use

Package(s):	gdk-pixbuf2	CVE #(s):	CVE-2011-2485
Created:	June 27, 2011	Updated:	March 15, 2013
Description:	From the Fedora advisory: It was found that gdk-pixbuf GIF image loader gdk_pixbuf__gif_image_load() routine did not properly handle certain return values from their subroutines. A remote attacker could provide a specially-crafted GIF image, which once opened in an application, linked against gdk-pixbuf would lead to gdk-pixbuf to return partially initialized pixbuf structure, possibly having huge width and height, leading to that particular application termination due excessive memory use.
Alerts:	Oracle ELSA-2013-0646 pidgin 2013-03-14 Gentoo 201206-20 gdk-pixbuf 2012-06-23 Gentoo 201206-11 pidgin 2012-06-21 Mandriva MDVSA-2011:132 pidgin 2011-09-06 Fedora FEDORA-2011-8667 gdk-pixbuf2 2011-06-24 Fedora FEDORA-2011-8917 pidgin 2011-07-01 Fedora FEDORA-2011-8966 pidgin 2011-07-01 Slackware SSA:2011-178-01 pidgin 2011-06-28 Fedora FEDORA-2011-8672 gdk-pixbuf2 2011-06-24

Comments (none posted)

git-web: cross-site scripting

Package(s):	git-web	CVE #(s):	CVE-2011-2186
Created:	June 28, 2011	Updated:	June 29, 2011
Description:	From the openSUSE advisory: Users with commit access to repos served by git-web could cause cross site scripting (XSS) issues with XML files
Alerts:	openSUSE openSUSE-SU-2011:0705-1 git-web 2011-06-28

Comments (none posted)

kernel: privilege escalation

Package(s):	kernel	CVE #(s):	CVE-2011-1169
Created:	June 28, 2011	Updated:	August 9, 2011
Description:	From the Ubuntu advisory: Dan Rosenberg discovered that some ALSA drivers did not correctly check the adapter index during ioctl calls. If this driver was loaded, a local attacker could make a specially crafted ioctl call to gain root privileges.
Alerts:	Ubuntu USN-1202-1 linux-ti-omap4 2011-09-13 Ubuntu USN-1187-1 kernel 2011-08-09 Ubuntu USN-1167-1 linux 2011-07-13 Ubuntu USN-1160-1 kernel 2011-06-28

Comments (none posted)

libgnomesu: privilege escalation

Package(s):	libgnomesu	CVE #(s):	CVE-2011-1946
Created:	June 27, 2011	Updated:	June 29, 2011
Description:	From the openSUSE advisory: The libgnomesu pam backend did not check the return value of the setuid() functions. Local users could exploit that to gain root privileges
Alerts:	openSUSE openSUSE-SU-2011:0694-1 libgnomesu 2011-06-27

Comments (none posted)

libgssglue: privilege escalation

Package(s):	libgssglue	CVE #(s):	CVE-2011-2709
Created:	June 27, 2011	Updated:	April 8, 2013
Description:	From the SUSE advisory: This update fixes insecure getenv() usage in libgssglue, which could be used under some circumstances by local attackers to gain root privileges.
Alerts:	Mandriva MDVSA-2013:043 libgssglue 2013-04-05 Ubuntu USN-1612-1 libgssglue 2012-10-15 Gentoo 201209-22 libgssglue 2012-09-27 Mageia MGASA-2012-0159 libgssglue 2012-07-10 Fedora FEDORA-2012-8067 libgssglue 2012-06-10 Fedora FEDORA-2012-7971 libgssglue 2012-06-15 SUSE SUSE-SU-2011:0696-1 libgssglue 2011-06-27

Comments (none posted)

Kernel release status

The current development kernel is 3.0-rc5, released on June 27. "Nothing terribly exciting here. The most noteworthy thing may be that only about a quarter of the changes are in drivers, filesystem changes actually account for more (40%): btrfs, cifs, ext4, jbd2, nfs are all present and accounted for." Details can be found in the full changelog.

Stable updates: the 2.6.32.42, 2.6.33.15, and 2.6.39.2 updates were released on June 23 with a long list of fixes. The 2.6.34.10 update - the first since April - came out on June 26 with a very long list of fixes.

Comments (none posted)

Quotes of the week

-- Frederic Weisbecker

-- Mauro Carvalho Chehab

-- Andrew Morton

Comments (none posted)

2011 Kernel Summit Planning starts

The kickoff message for the 2011 Kernel Summit planning process has been sent out. The event will be held October 24-26 in Prague, with some tweaks to the longstanding formula. "This year, the biggest change is that the conference will be running three days, where the first day will be dedicated to some kernel subsystem workshops. The second day will be focused on development process issues and be more discussion oriented; for this reason, it will be limited to core kernel developers picked through a nomination and selection process which as in previous years. The third day will be more presentation oriented (although hopefully we will have some discussion); and all kernel summit workshop attendees will be welcome attend the 3rd day." Now is the time for interested folks to help shape the agenda.

Full Story (comments: none)

-EWHICHERROR?

Users of the Video4Linux2 API know that it is a rather complicated one, involving some 91 different ioctl() commands. The error-reporting side of the API is much simpler, though; if something goes wrong, the application is almost certain to get EINVAL back. That error can be trying to tell user space that the device is in the wrong state, that some parameter was out of range, or, simply, that the requested command has not been implemented. Needless to say, it can be hard for developers to figure out what is really going on.

V4L2 maintainer Mauro Carvalho Chehab recently posted a patch to change the return code to ENOIOCTLCMD in cases where the underlying driver has not actually implemented the requested command. That change would at least distinguish one set of problems - except that the VFS code silently translates ENOIOCTLCMD to EINVAL before returning to user space. So, from the point of view of the application, nothing changes.

Interestingly, the rules for what is supposed to happen in this situation are relatively clear: if an ioctl() command has not been implemented, the kernel should return ENOTTY. Some parts of the kernel follow that convention, while others don't. This is not a new or Linux-specific problem; as Linus put it: "The EINVAL thing goes way back, and is a disaster. It predates Linux itself, as far as I can tell." He has suggested simply changing ENOIOCTLCMD to ENOTTY across the kernel and seeing what happens.

What happens, of course, is that the user-space ABI changes. It is entirely possible that, somewhere out there, some program depends on getting EINVAL for a missing ioctl() function and will break if the return code changes. There is only one way to find out for sure: make the change and see what happens. Mauro reports that making that change within V4L2 does not seem to break things, so chances are good that change will find its way into 3.1. A tree-wide change could have much wider implications; whether somebody will find the courage to try that remains to be seen.

Comments (7 posted)

PCIe, power management, and problematic BIOSes

Back in April, Phoronix announced with some fanfare that the 2.6.38 kernel - and those following - had a "major" power management regression which significantly reduced battery life on some systems. This problem has generated a fair amount of discussion, including this Launchpad thread, but little in the way of solutions. Phoronix now claims to have located the change that caused the problem and has provided a workaround which will make things better for some users. But a true fix may be a while in coming.

As a result of the high clock rates used, PCI-Express devices can take a lot of power even when they are idle. "Active state power management" (ASPM) was developed as a means for putting those peripherals into a lower power state when it seems that there may be little need for them. ASPM can save power, but the usual tradeoff applies: a device which is in a reduced power state will not be immediately available for use. So, on systems where ASPM is in use, access to devices can sometimes take noticeably longer if those devices have been powered down. In some situations (usually those involving batteries) this tradeoff may be acceptable; in others it is not. So, like most power management mechanisms, ASPM can be turned on or off.

It is a bit more complicated than that, though; on x86 systems, the BIOS also gets into the act. The BIOS is charged with the initial configuration of the system and with telling the kernel about the functionality that is present. One bit of information passed to the kernel by the BIOS is whether ASPM is supported on the system. The kernel developers, reasonably, concluded that, if the BIOS says that ASPM is not supported, they should not mess with the associated registers. It turns out, though, that this approach didn't quite work; thus, in December, Matthew Garrett committed a patch described as:

In other words, sometimes the BIOS will tell the system that ASPM is not supported even though ASPM support is present; for added fun, the BIOS may enable ASPM on some devices (even though it says ASPM is not supported) before passing control to the kernel. There are reasons why operating system developers tend to hold BIOS developers in low esteem.

Had Andrew Morton read the above changelog, he certainly would have complained that "can cause problems" is a rather vague justification for a change to the kernel. Your editor asked Matthew about the problem and got an informative response that is worth reading in its entirety:

It's not hard to imagine that putting devices into a state that the kernel was told should not exist might create confusion somewhere. Some research turns up, for example, this bug report about system hangs which are fixed by Matthew's patch. If the BIOS says that ASPM is not supported, it would seem that ensuring that no devices think otherwise would make sense.

That said, this patch is the one that the bisection effort at Phoronix has fingered as the cause of the power regression. Apparently, the notion that disabling low-power states in hardware may lead to increased power consumption also makes sense. The workaround suggested in the article is to boot with the pcie_aspm=force option; that forces the system to turn on ASPM regardless of whether the BIOS claims to support it. This workaround will undoubtedly yield better battery life on some affected systems; others may well not work at all. In the latter case, the system may simply lock up - a state with even worse latency characteristics combined with surprisingly bad power use. So this workaround may be welcomed by users who have seen their battery life decline significantly, but it is not a proper solution to the problem.

Finding that proper solution - preferably one which Just Works without any need for special boot parameters - could be tricky. Quoting Matthew again:

Given the uncertainty in the situation, the kernel developers have reached the conclusion that "waste a bit of power" is a lesser evil than "lock up on some systems." In the absence of a better understanding of the problem, any other approach would be hard to justify. So some users may have to use the pcie_aspm=force workaround for a while yet.

Meanwhile, the power usage problem has, as far as your editor can tell, never been raised on any kernel development mailing list. It never appeared in the 2.6.38 regression list. So this issue was invisible to much of the development community; it's not entirely surprising that it has not received much in the way of attention from developers. For better or for worse, the development community has its way of dealing with issues. Reporting a bug to linux-kernel certainly does not guarantee that it will get fixed, but it does improve the odds considerably. Had this issue been brought directly to the developers involved, we might have learned about the root cause some time ago.

Comments (45 posted)

Dealing with complexity: power domains and asymmetric multiprocessing

When one thinks of embedded systems, it may be natural to think of extremely simple processors which are just barely able to perform the tasks which are asked of them. That view is somewhat dated, though. Contemporary processors are often put into settings where they are expected to manage a variety of peripherals - cameras, signal processors, radios, etc. - while using a minimum of power. Indeed, a reasonably capable system-on-chip (SoC) processor likely has controllers for these peripherals built in. The result is a processor which presents a high level of complexity to the operating system. This article will look at a couple of patch sets which show how the kernel is changing to deal with these processors.

Power domains

On a desktop (or laptop) system, power management is usually a matter of putting the entire CPU into a low-power state when the load on the system allows. Embedded processors are a little different: as noted above, they tend to contain a wide variety of functional units. Each of these units can be powered down (and have its clocks turned off) when it is not needed, while the rest of the processor continues to function. The kernel can handle the powering down of individual subsystems now; what makes things harder is the power dependencies between devices.

Power management was one of the motivations behind the addition of the kernel's device model in the 2.5 development series. It does not make sense, for example, to power down a USB controller if devices attached to that controller remain in operation. The device model captures the connection topology of the system; this information can be used to power devices up and down in a reasonable order. The result was much improved power management in the 2.6 kernel.

On newer systems, though, there are likely to be dependencies between subsystems that are not visible in the bus topology. A set of otherwise unrelated devices may share the same clock or power lines, meaning that they can only be powered up or down as a group. Different SoC designs may feature combinations of the same controllers with different power connections. As a result, drivers for specific controllers often cannot know whether it is safe to power down their devices - or even how to do it. This information must be maintained at a level separate from the device hierarchy if the system is to be able to make reasonable power management decisions.

The answer to this problem would appear to be Rafael Wysocki's generic I/O power domains patch set. A power domain looks like this:

    struct generic_pm_domain {
	struct dev_pm_domain domain;	
	struct list_head sd_node;	
	struct generic_pm_domain *parent;
	struct list_head sd_list;
	struct list_head dev_list;
	bool power_is_off;
	int (*power_off)(struct generic_pm_domain *domain);
	int (*power_on)(struct generic_pm_domain *domain);
	int (*start_device)(struct device *dev);
	int (*stop_device)(struct device *dev);
        /* Others omitted */
    };

Power domains are hierarchical, though the hierarchy may differ from the bus hierarchy. So each power domain has a parent domain (parent), a list of sibling domains (sd_node), and a list of child domains (sd_list); there is also, naturally, a list of devices contained within the domain (dev_list). When the kernel is changing a domain's power state, it can use start_device() and stop_device() to operate on specific devices, or power_on() and power_off() to power the entire domain up and down.

That is the core of the patch though, naturally, there is a lot of supporting infrastructure to manage domains, let them participate in suspend and resume, etc. The one other piece is the construction of the domain hierarchy itself. The patch set includes one example implementation which is added to the ARM "shmobile" subarchitecture board file. In the longer term, there will need to be a way to represent power domains within device trees since board files are intended to go away.

This patch set has been through several revisions and seems likely to be merged during the 3.1 development cycle.

Asymmetric multiprocessing

When one speaks of multiprocessor systems, the context is almost always symmetric multiprocessing - SMP - where all of the processors are equal. An embedded SoC may not be organized that way, though. Consider, for example, this description from the introduction to a patch set from Ohad Ben-Cohen:

Asymmetric multiprocessing (AMP) is what you get when a system consists of unequal processors running different operating systems. It could be thought of as a form of (very) local-area networking, but all of those cores sit on the same die and share access to memory, I/O controllers, and more. This type of processor isn't simply "running Linux"; instead, it has Linux running on some processors trying to shepherd a mixed collection of operating systems on a variety of CPUs.

Ohad's patch is an attempt to create a structure within which Linux can direct a processor of this type. It starts with a framework called "remoteproc" that allows the registration of "remote" processors. Through this framework, the kernel can power those processors up and down and manage the loading of firmware for them to run. Much of this code is necessarily processor-specific, but the framework abstracts away the details and allows the kernel to deal with remote processors in a more generic fashion.

Once the remote processor is running, the kernel needs to be able to communicate with it. To that end, the patch set creates the concept of "channels" which can be used to pass messages between processors. These messages go through a ring buffer stored in memory visible to both processors; virtio is used to implement the rings. A small piece of processor-specific code is needed to implement a doorbell to inform processors of when a message arrives; the rest should be mostly independent of the actual system that it is running on.

This patch set has been reasonably well received as a good start toward the goal of regularizing the management of AMP systems. A complete solution is likely to require quite a bit more work, including implementations for a wider variety of architectures. But, then, one could say that, after twenty years, Linux as a whole is still working toward a complete solution. The hardware continues to evolve toward more complexity; the operating system will have to keep evolving in response. These two patch sets give some hints of the direction that evolution is likely to take in the near future.

Comments (1 posted)

Sanitizing log file output

Handling user-controlled data properly is one of the basic principles of computer security. Various kernel log messages allow user-controlled strings to be placed into the messages via the "%s" format specifier, which could be used by an attacker to potentially confuse administrators by inserting control characters into the strings. So Vasiliy Kulikov has proposed a patch that would escape certain characters that appear in those strings. There is some question as to which characters should be escaped, but the bigger question is an age-old one in security circles: whitelisting vs. blacklisting.

The problem stems from the idea that administrators will often use tools like tail and more to view log files on a TTY. If a user can insert control characters (and, in particular, escape sequences) into the log file, they could potentially cause important information to be overlooked—or cause other kinds of confusion. In the worst case, escape sequences could potentially exploit some hole in the terminal emulator program to execute code or cause other misbehavior. In the patch, Kulikov gives the following example: "Control characters might fool root viewing the logs via tty, e.g. using ^[1A to suppress the previous log line." For characters that are filtered, the patch simply replaces them with "#xx", where xx is the hex value of the character.

It's a fairly minor issue, at some level, but it's not at all clear that there is any legitimate use of control characters in those user-supplied strings. The strings could come from various places; two that were mentioned in the discussion were filenames or USB product ID strings. The first version of the patch clearly went too far by escaping characters above 0x7e (in addition to control characters), which would exclude Unicode and other non-ASCII characters. But after complaints about that, Kulikov's second version just excludes control characters (i.e. < 0x20) with the exception of newline and tab.

That didn't sit well with Ingo Molnar, however, who thought that rather than whitelisting the known-good characters, blacklisting those known to be potentially harmful should be done instead:

But, in order to create a blacklist, one must carefully determine the effects of the various control characters on all the different terminal emulators, whereas the whitelist approach has the advantage of being simpler by casting a much wider net. As Kulikov notes, figuring out which characters are problematic is not necessarily simple:

The disagreement between Molnar and Kulikov is one that has gone on in the security world for many years. There is no right answer as to which is better. As with most things in security (and software development for that matter), there are tradeoffs between whitelists and blacklists. In general, for user-supplied data (in web applications for example), the consensus has been to whitelist known-good input, rather than attempting to determine all of the "bad" input to exclude. At least in this case, though, Molnar does not see whitelists as the right approach:

It won't come as a surprise that Kulikov disagreed with that analysis: "What do you do with dangerous characters that are *not yet known* to be dangerous?" While there is little question that whitelisting the known-good characters is more secure, it is less flexible if there is a legitimate use for other control characters in the user-supplied strings. In addition, Molnar is skeptical that there are hidden dangers lurking in the ASCII control characters: "This claim is silly - do you claim some 'unknown bug' in the ASCII printout space?"

In this particular case, either solution should be just fine, as there aren't any good reasons to include those characters, but Molnar is probably right that there aren't hidden dangers in ASCII. There is a question as to whether this change is needed at all, however. The concern that spawned the patch is that administrators might miss important messages or get fooled by carefully crafted input (Willy Tarreau provides an interesting example of the latter). Linus Torvalds is not convinced that it is really a problem that needs addressing:

Given Torvalds's skepticism, it doesn't seem all that likely this patch will go anywhere even if it were changed to a blacklisting approach as advocated by Molnar. It is, or should be, a fairly minor concern, but the question about blacklisting vs. whitelisting is one we will likely hear again. There are plenty of examples of both techniques being used in security (and other) contexts. It often comes down to a choice between more security (whitelisting typically) or more usability (blacklisting). This case is no different, really, and others are sure to crop up.

Comments (19 posted)

Porteus 1.0: A portable distribution for old-school KDE users

Users who've missed KDE 3.5.x are in for a treat with Porteus, a portable Linux distribution that offers a 32-bit release with the Trinity fork of KDE 3.5.x, and a 64-bit release that offers KDE 4.6.4. While not a distribution that will appeal to everyone, it might be of interest to enthusiasts of live CD distributions and old-school KDE fans.

Porteus is a distribution that was originally based on Slax, and is now based on Slackware Linux since Slax has gone without a release since 2009. Porteus developer Jay Flood says that Porteus started as "Slax Remix" with a 32-bit version only, then moved to an independent project with the name change to Porteus at the beginning of 2011. What's the philosophy for Porteus? In a word, speed. Flood says that the project's goals are "being lightweight (under 300Mb), super fast, portable and complete" and the project FAQ says that Porteus is for "anyone who likes an extremely fast and light operating system that boots in seconds, and stays up to date with the latest software and kernel versions."

Unlike many distributions, Porteus does not have a unified set of packages between its 32-bit and 64-bit releases. Instead, the 32-bit release offers KDE 3.5.12 or LXDE as its desktop, while KDE 4.x users will need to choose the 64-bit release. Why the difference? Flood says that the choice was made to optimize Porteus 32-bit for older hardware, and there's user demand for the prior KDE release. He also says that KDE 4.x is available for the 32-bit version, but not by default. LXDE is available for both releases.

Flood says that the project is also testing Trinity for the 64-bit release, but they haven't yet provided it publicly for users to try out. He did note that users can only run one version of KDE, as Porteus has compiled Trinity to run from the /usr directory unlike the upstream project, which is designed to sit alongside KDE 4.x.

Porteus is not currently doing any development on Trinity, though it is reporting bugs upstream. The Trinity project itself is currently frozen due to work on converting to using cmake. Should Trinity fall by the wayside, Flood says that Porteus would likely look to moving to Xfce or KDE4 for the 32-bit edition.

The distribution is meant to be run off of a CD, or off a USB flash drive. It can be copied to a hard drive, but the project recommends Slackware instead if the user wants to do a traditional installation that runs off a hard disk. The live CD also offers an option to set up a PXE boot server, so users could boot Porteus over the network as well.

To try out Porteus, I decided to go with the 32-bit release. It's been a while since I've sat down to a KDE 3.5 desktop, and I wanted to see how it compared with today's KDE. Porteus is using Trinity KDE 3.5.12, which was released in October of 2010. The last official release of KDE was 3.5.10, so Trinity has put out two releases since picking up the project. The 3.5.11 Trinity release added a number of new features, like SmartCard support for KDM, remote folder synchronization in Konqueror, and ICC color profile support. The 3.5.12 release had little in the way of major features but had a number of fixes for Kontact, changed the default to double-clicking for launching applications, and added features to make GTK applications fit in better on the KDE desktop.

For longtime KDE users, then, the Trinity desktop will look very familiar but contains no major improvements since the KDE project focused on the 4.x series. Which is not to say that it's aged poorly. Trinity KDE is perfectly usable, and very fast even with a limited amount of memory. I started testing Porteus in a VM with only 512MB of RAM (to better simulate older hardware), and found KDE very responsive and usable. With the RAM bumped up to 1GB and copying the entire live CD into RAM, I found Porteus extremely responsive for normal desktop use.

The application selection may be a bit limiting for some users. There's no LibreOffice/OpenOffice.org, GIMP, Inkscape, etc. Porteus does include KOffice (1.6) and the KDE PIM applications from the 3.5.x series. Firefox is already configured with Adobe Flash and the Flashblock Add-On.

If you want applications that aren't installed by default, Porteus uses a "module" system derived from Slax to install packages. These are much like Slackware packages, but they're an LZMA2-compressed squashfs filesystem, which provides a smaller download and allows modules to be unmounted from a live system. The project has some packages available for installation via a Porteus Package Manager, which is a serviceable if clunky utility for finding, downloading, and installing modules. Porteus also includes a utility to remove modules from a running system, so it's possible to conserve memory by (for example) removing the KOffice module.

Since Porteus is based on Slackware, you might expect to be able to install Slackware packages as well. This is possible, but with a few hurdles. Specifically, Porteus expects you to convert the packages into Porteus modules instead. The distribution does include tools to do this, and the steps are not particularly difficult — but tend to rule Porteus out for users who want to install a variety of software without too much hassle.

Naturally, Porteus has tools for saving configuration between reboots, and even tools for copying directories to USB or hard disk in case you want to preserve the files between reboots. One downside, though, is that the project doesn't really have an update system — beyond grabbing a new image at this time.

Most of the work that's gone into Porteus has been done, or led by, Flood and founder Tomasz Jokiel. The distribution also has a documentation team that has developed quite a few guides and tutorials for working with Porteus. Flood says that the distribution has a "friendly and active development community" and invites users to register on the Porteus forum and get involved.

In general, Porteus looks like a nice portable Linux distribution, aimed at expert or at least experienced Linux users. It's not something that will appeal to the majority of Linux users, particularly users who prefer a slightly larger depth of available packages. But, for users who are nursing older hardware or prefer a portable distribution, Porteus is an interesting project.

Comments (2 posted)

Distribution quotes of the week

-- Philip Van Hoof

-- Juan Rodriguez announces Fedora remix BlueBubble

-- Juan Rodriguez

Comments (none posted)

Red Hat Enterprise MRG 2.0 released

Red Hat has announced the availability of version 2.0 of its Enterprise MRG ("messaging, realtime, and grid") distribution. It features an update to RHEL 6.1, a number of claimed performance improvements, and a bunch of buzzword-compliant cloud features. "Enterprise MRG Messaging 2.0 services as a foundational layer of infrastructure in Red Hat's OpenShift Platform-as-a-Service (PaaS) solution, introduced in May 2011. With Enterprise MRG 2.0, OpenShift gains a reliable, scalable mechanism for transferring data in the cloud, as well as the interoperability of the AMQP open standard."

Comments (none posted)

Mageia: preview of ARM port is now available

Arnaud Patard has been working on an ARM port of Mageia, and the first technical preview is now available. "This first release has been built using Mageia tools but not integrated into the Mageia build system. This is one of the main items on the current todo list. A PandaBoard is waiting now to be installed in the Mageia build system so that a parallel build can be done in ARM when a package is submitted to the build system. The hard part will be managing different ARM machines using different socs meaning different kernels."

Comments (12 posted)

Updated Debian 6.0.2 released

The Debian project has announced the first update of its stable distribution Debian 6.0 "Squeeze". "This update mainly adds corrections for security problems to the stable release, along with a few adjustments to serious problems."

Comments (1 posted)

How YOU can help Debian!

Martin Zobel-Helas looks at how people might get involved in Debian's development process. "If you are using Debian, speak about it! If you have problems with Debian, speak about it! If you like Debian, speak about it. Read the debian-user mailing list (or a localized one) and jump in if users have the same problem you had, and help them. All sort of publicity will help Debian." (Thanks to Paul Wise)

Comments (none posted)

Bits from the Release Team

Neil McGovern has a report from the Debian Release Team, covering Release Team sprint minutes, Squeeze wrap up/retrospective, Release team membership/workload, Time based freezes, Migrations from unstable to testing, and what's in the next update.

Full Story (comments: none)

Multiarch in Debian unstable

Steve Langasek covers the current status of multiarch support in Debian. "People familiar with the FHS and with other Linux distributions probably know that they require amd64 libraries to be shipped in /lib64 instead of in /lib - a so-called "biarch" solution because it only scales to two architectures. Debian has never implemented this provision of the FHS for two reasons: first, because the package manager up to now has not had support for installing copies of a package for more than one architecture side by side, and second because accomodating /lib64 in Debian packaging is sufficiently intrusive that we've held out for a solution that would address more than just the 32+64-bit library case. Multiarch is the solution to the second problem, providing a policy for combining library packages from an arbitrary number of architectures on a single filesystem."

Full Story (comments: 2)

Fedora 13 End of Life

This is a reminder that Fedora 13 has reached its end of life. There will be no further updates, including security updates.

Full Story (comments: 2)

Appointment to the Fedora Board

Ruediger "Rudi" Landmann has been appointed to the Fedora Board for a full one-year cycle. "Rudi has been active on the Fedora Documentation team, and maintains several documentation-related packages in Fedora."

Full Story (comments: none)

Mageia: Security, Updates and More - Oh My!

The fledgling Mandriva fork, Mageia, has gotten their first release out the door. This report takes a look at the update process and progress. "If you've been using Mageia 1, you may have been wondering where all the updates are. It's customary to get quite a few updated packages in the first month or so of a new distribution to correct bugs and address security issues. Don't worry, we've been working on that too. As a new organization, and a community driven one, we first had to work out how to do the updates. While some of us have experience from previous lives, we weren't entirely satisfied with the old process and wanted to make sure our new community of users and packagers had an input into how we'll do things."

Comments (none posted)

Distribution newsletters

• Debian Project News (June 24)
• DistroWatch Weekly, Issue 411 (June 27)
• Fedora Weekly News Issue 279 (June 22)
• Maemo Weekly News (June 27)
• openSUSE Weekly News, Issue 181 (June 25)
• Ubuntu Weekly Newsletter, Issue 222 (June 25)

Comments (none posted)

The 2011 Linux Distro Scorecard (Linux.com)

Joe 'Zonker' Brockmeier has a brief review of his top 6 distribution picks; Debian, Fedora, Linux Mint, openSUSE, Slackware, and Ubuntu. "Which distribution is the best? None of them, or all of them. It's really about what meets your needs. Some people want a distribution that's really easy to use, and don't care much about licensing. Some people choose a distribution because of the licensing, and ease of use isn't really that important. You might only want to look at distributions that have KDE or GNOME as a desktop. It's sort of like picking a restaurant, what makes one person happy is going to be a really bad experience for another person. I like spicy food, other folks can't handle it or just don't like it."

Comments (none posted)

Looking in on Apache OpenOffice.org

In the few weeks since Apache accepted OpenOffice.org (OOo) as an incubator project, the project has gotten off to a fast start, at least from a planning perspective. It is an interesting case study in adapting an existing project into the infrastructure and governance of an entirely different organization. That leads to some technical and organizational challenges that are being addressed, but it remains to be seen how well OOo will fit under the Apache umbrella. Apache is an excellent organization, and likely a good home for OpenOffice.org developers who are not concerned about copyleft, but it hasn't ever really absorbed a codebase and community of this size before.

The biggest hurdle may be in the area of tools, in the form of adopting (and adapting to) the Subversion (svn) version control system. There is nothing wrong with svn per se, but it may not be the right tool for this particular job. The existing OOo repositories are stored in Mercurial (hg), which is a distributed version control system (DVCS), so the existing developers clearly saw a benefit to DVCS-style development. LibreOffice (LO), on the other hand, has adopted Git, which also has the distributed features many projects are finding useful. But Apache projects must use Apache infrastructure, which requires svn, at least for now.

The problems of converting the existing repositories from hg to svn are certainly solvable, but the bigger problems may arise in trying to coordinate development with a widely distributed contributor base. There are, undoubtedly, ways to make that work (and various Apache projects already do), but it seems a bit odd to choose a VCS based on the required infrastructure, rather than the other way around. In the end, a decision made based on the licensing that Oracle and IBM were comfortable with trickles down into the development style of the project.

None of that is to say that svn will necessarily be a barrier, or that the Apache infrastructure is inadequate to the task. One would guess that any problems there will be worked around one way or another. But community-led projects typically make their decisions rather differently.

There are lots of discussions going on in the new incubator ooo-dev mailing list that clearly indicate a community that is bootstrapping itself. That list has well over 1000 posts in just the two weeks or so that it has existed. It's an interesting transition to watch, at least partly because we haven't seen anything like it in the free software world. There is a huge existing infrastructure (web sites, build farms, code repositories, etc.) to support OOo that currently exists on Oracle's servers, so moving that to its new home is a major part of the transition.

IBM's Rob Weir has been coordinating much of the work that is being done on the Apache project. In addition, he has put out a pre-proposal for an ODF Toolkit project, which would include several, mostly Java-based, tools for manipulating Open Document Format (ODF) files. Whether that ends up as a top-level Apache project or gets added into an existing Apache project (perhaps POI or OOo itself) is an open question, but it's clear that there is more than just OOo that Oracle and IBM would like to put under the Apache banner.

Governance issues are also being discussed as a corporate-controlled (and dominated) project transitions to the community-oriented, meritocratic style that is expected of Apache projects. There is a project management committee (PMC) to set up; in this case it's a podling PMC (PPMC) as OOo is still in the incubator (and thus a "podling"). The list of project committers also need to be formalized out of the list of initial committers that was gathered during the incubation proposal. These are the people who will have commit access to the repositories, and additional committers will be added to the project as their work merits it.

There are also format questions, in terms of both documentation for OOo itself and for project planning purposes. For a document suite, creating documentation using its formats makes a great deal of sense, but it isn't necessarily the format that developers are used to. There is a bit of a clash between the text-only and "rich text" (i.e. ODF) worlds. Like many projects, Apache typically uses diffs to review patches, but that is somewhere between difficult and impossible to do with ODF. It would seem that a consensus is arising that planning documents will be text-oriented and put into the wiki, while product documentation will continue to use ODF and the ODFAuthors infrastructure.

There is also the minor task of starting to do builds of the code, and trying to ferret out any missing pieces, along with replacing dependencies that are not available under the Apache license. There are still some lingering questions about whether the grant from Oracle actually contains all of the necessary files—though it is believed that is only a procedural hurdle. OOo 3.4 had a beta release recently, and there are thoughts that the release should be made using the Oracle servers, before making the big switch over to Apache. And so on.

Many more of these kinds of discussions are taking place, most of them very amicable, and problems are being worked out as they come up. Undoubtedly LibreOffice struggled with many of the same kinds of things as it came up to speed over the last eight months or so. In some sense, LO had two parent projects to digest, the Go-OO project (which was mostly a set of patches against OOo) as well as the OOo upstream code from Sun/Oracle. But LO did not have any existing structure like Apache that it needed to mesh with. Watching both projects develop over the next few years should prove interesting.

Comments (1 posted)

Quotes of the week

-- Brien Behlendorf

-- Asa Dotzler

-- Peter Hutterer

-- Richard Stallman

Comments (12 posted)

Announcing Echoprint

The Echoprint project has announced its existence and initial release of code and data. "The Echo Nest has been focusing on a crucial component of the oncoming music cloud for some time: we spend a lot of time and engineering resources on music resolving. This extends from mapping a query for a band name to its ID, to uncovering mentions of songs on blogs, to identifying the song in an audio stream without any metadata - otherwise known as fingerprinting. The Echo Nest's existing fingerprint technology, 'The Echo Nest Musical Fingerprint' aka ENMFP, has been in wide use privately and via our API for 18 months. Today we are unveiling a new fingerprint technology called 'Echoprint,' whose main feature is its complete openness - everything from the program to analyze the audio to the server and data to make the match are available for anyone to use, under a permissive open source license, for free."

Comments (9 posted)

EuroPython Language Summit report

Mark Dickinson has put together a brief report from the EuroPython Language summit, held in Florence on June 19. Topics covered include Python 3 adoption, various open PEPs, the Linux kernel version number change, and more.

Full Story (comments: none)

Removing the PyPy global lock

The PyPy Status Blog has an article describing a plan to remove the global interpreter lock and switch to an transactional memory scheme. "During a transaction, we don't actually change the global memory at all. Instead, we use the thread-local transaction object. We store in it which objects we read from, which objects we write to, and what values we write. It is only when the transaction reaches its end that we attempt to 'commit' it. Committing might fail if other commits have occurred in between, creating inconsistencies; in that case, the transaction aborts and must restart from the beginning."

Comments (16 posted)

Rockbox 3.9

Version 3.9 of the Rockbox audio player firmware system has been released. Changes include a playback engine rework, a number of hardware-related improvements, support for antialiased fonts, and more; see the release notes for details.

Comments (none posted)

Development newsletters from the last week

• Caml Weekly News (June 28)
• LibreOffice development summary (June 28)
• Mozilla Engineering Newsletter (June 22)
• Python-URL! (June 24)
• Tahoe-LAFS Weekly News (June 26)f
• Tcl-URL! (June 29)

Comments (1 posted)

Applying Graph Analysis and Manipulation to Data Stores

Roberto V. Zicari talks with Marko Rodriguez and Peter Neubauer about the Tinkerpop project. "TinkerPop is an open-source graph software group. Currently, we provide a stack of technologies (called the TinkerPop stack) and members contribute to those aspects of the stack that align with their expertise. The stack starts just above the database layer (just above the graph persistence layer) and connects to various graph database vendors - e.g. Neo4j, OrientDB, DEX, RDF Sail triple/quad stores, etc."

Comments (none posted)

Zeuthen: Writing a C library, part 1

David Zeuthen has posted a set of guidelines for low-level library implementers. "Unless it's self-evident, all functions should have documentation explaining how parameters are managed. It is often a good idea to try to force some kind of consistency on the API. For example, in the GLib stack the general rule is that the caller owns parameters passed to a function (so the function need to take a reference or make a copy if the parameter is used after the function returns) and that the callee owns the returned parameters (so the caller needs to make a copy or increase the reference count) unless the function can be called from multiple threads (in which case the caller needs to free the returned object)."

Comments (108 posted)

Zeuthen: Writing a C library, part 2

David Zeuthen has posted the second installment in his series on best practices for low-level library development. "It is important for users of a library to know if calling a function involves doing synchronous I/O (also called blocking I/O). For example, an application with an user interface need to be responsive to user input and may even need to update the user interface every frame for smooth animations (e.g. 60 times a second). To avoid unresponsive applications and jerky animations, its UI thread must never call any functions that does any synchronous I/O."

Comments (5 posted)

Ada Initiative fund-raising campaign a success

The Ada Initiative, a new organization which "focuses on scalable, reusable, and effective programs aimed at both recruitment and retention of women in open technology and culture, has announced that it has completed its initial fund-raising drive one week ahead of schedule. "The Seed 100 campaign raised over $80,000 from 102 donors of $512 or more in just 24 days."

Full Story (comments: 3)

The Power of Open from Creative Commons

The Creative Commons project has announced the release of a 47-page book highlighting the stories of a number of people using CC licenses for their work. "The Power of Open collects the stories of those creators. Some are like ProPublica, a Pulitzer Prize-winning investigative news organization that uses CC while partnering with the world's largest media companies. Others like nomadic filmmaker Vincent Moon use CC licensing as an essential element of a lifestyle of openness in pursuit of creativity. The breadth of uses is as great as the creativity of the individuals and organizations choosing to open their content, art and ideas to the rest of the world." Unsurprisingly, it's downloadable under a CC license.

Comments (none posted)

LinuxQuestions.org Turns Eleven

LinuxQuestions.org celebrates 11 years on the internet. "4,382,316 posts and 457,176 registered members does not even begin to tell the story. The community and mod team that has grown at LQ is truly amazing and something that I'm very proud to be a part of."

Full Story (comments: none)

Desktop Summit Keynote Interview with Dirk Hohndel

Dirk Hohndel is the Chief Linux and Open Source Technologist at Intel and the opening keynote speaker at the Desktop Summit. William Carlson traded emails with Dirk as part of a series of interviews leading up to the Summit in August. "[Intel's] Open Source Technology Center (OTC) has a rather wide charter. Basically of course, we are doing much of the enabling of future Intel hardware (and software) innovations in Linux and other open source software based environments. But beyond that we work on a lot of interesting new technologies in open source. Examples include MeeGo or the Yocto Project. And there are many more projects that have been initiated by the OTC: PowerTop, LatencyTop, many other projects ranging from bootloader to client applications. Fundamentally we view ourselves as part of the larger open source community."

Comments (1 posted)

FSFE: Fellowship interview with Guido Arnold

Chris Woolfrey talks with Guido Arnold, "Deputy Coordinator of [Free Software Foundation Europe] FSFE's Education Team, as well as a member of the German team, and a translator of fsfe.org and gnu.org." Guido: "The thing with Free Software in education is that there are already many, many groups throughout Europe working in this field. Many of them are inactive however, because there are only a few people who are active and the rest stay silent. It would be great to get all those people who are active to work together, and that's part of our aim. I spent some time introducing new members to the Education Team. And we've had to deal with issues internally: we were asked if we knew of any "free" material for teaching kids the concepts of Free Software; at first I thought this would be easy, but I was mistaken. So, I spent some time researching and asking around."

Comments (none posted)

Patent Reform - If Your Head Hurt Before . . . (Groklaw)

For those who are curious about the "patent reform" bill working its way through the U.S. Congress, Groklaw has a critical summary of the changes. "And where are most of those 'fixes' aimed? At addressing the reexamination of patents that contain claims that likely should have never been allowed. Doesn't it make more sense to focus and invest on achieving thorough examinations in the first place? Well, yes, it does, but there are serious interests out there that really don't want that to happen. Why? Because regardless of whether a claim is ultimately found valid, a patent has value by its mere existence because of the high cost of patent litigation. This legislation is not going to fix that problem."

Comments (12 posted)

Linux Foundation Monthly Newsletter: June 2011

This edition of the Linux Foundation Monthly Newsletter covers the LinuxCon Program, a new Scholarship Program, LexisNexis Joins LF, and several other topics.

Full Story (comments: none)

Mozilla to Businesses: We're Not Interested (PC Mag)

Here's a PC Magazine article taking a sharp look at the apparent disconnect between the new Firefox development process and the needs of businesses. "[Asa] Dotzler's comment is both sneering and contemptuous of the businesses that have deployed Firefox in their organizations. And, sometimes, at their own peril, may I add. While Firefox is a wonderful browser with its own unique set of features, the frequent updating, occasional lack of good documentation, extension breaking whenever a new update comes out - it all makes it a dicey choice of browser for businesses."

Comments (115 posted)

Call for Presentations open for Postgres Open

Postgres Open takes place September 14-16, 2011 in Chicago, Illinois. The call for presentations is open until July 8. "Talks should be oriented towards the business or development user of PostgreSQL, and should have substantial technical content."

Full Story (comments: none)

LinuxCon/ELC Europe CFP deadline is July 8

The Linux Foundation has sent out a reminder that the proposal deadline for LinuxCon Europe and the Embedded Linux Conference Europe (Prague, late October) is July 8. The 2011 Kernel Summit and Realtime Linux Workshop will also be happening during this time; it's going to be a busy couple of weeks.

Full Story (comments: 1)

KVM Forum 2011 Schedule Published

The KVM Forum 2011 takes place in Vancouver Canada, August 15-16, 2011. The final schedule has been announced.

Comments (none posted)

PyCon Australia to be held in August 2011

The second Australian Python Conference will be held August 20-21 in Sydney. "PyCon Australia 2011 will host dozens of presentations on web programming, business applications, game development, science and mathematics, social issues, education, design, testing, documentation and more. Scheduled presentations include "How Python Evolves", "Behaviour Driven Development", "Benchmarking stuff made ridiculously easy", and a discussion panel on Python 3."

Full Story (comments: none)

PyCon AU gender diversity grants for women in Python

PyCon AU has announced that it will be offering two gender diversity delegate grants to women who wish to attend PyCon AU in 2011. "These grants will *both* cover full registration costs; in addition, one of the grants will cover up to $AUD500 of travel and accommodation costs for a woman living outside of the Sydney region to attend." Applications are due by July 8. PyCon AU will take place August 20-21, 2011 in Sydney, Australia.

Full Story (comments: none)

Tutorials at PyCon DE 2011, Leipzig, Germany

PyCon DE 2011 takes place October 4-9, with tutorials taking place on October 4. The tutorial program has been announced. "There are 12 three-hour tutorials covering a wide range of Python topics such as Python for newbies, web development, algorithms, tests, data analysis, databases or Cython."

Full Story (comments: none)

Events: July 7, 2011 to September 5, 2011

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
July 9 July 14	Libre Software Meeting / Rencontres mondiales du logiciel libre	Strasbourg, France
July 11 July 16	SciPy 2011	Austin, TX, USA
July 11 July 12	PostgreSQL Clustering, High Availability and Replication	Cambridge, UK
July 11 July 15	Ubuntu Developer Week	online event
July 15 July 17	State of the Map Europe 2011	Wien, Austria
July 17 July 23	DebCamp	Banja Luka, Bosnia
July 19	Getting Started with C++ Unit Testing in Linux
July 24 July 30	DebConf11	Banja Luka, Bosnia
July 25 July 29	OSCON 2011	Portland, OR, USA
July 30 July 31	PyOhio 2011	Columbus, OH, USA
July 30 August 6	Linux Beer Hike (LinuxBierWanderung)	Lanersbach, Tux, Austria
August 4 August 7	Wikimania 2011	Haifa, Israel
August 6 August 12	Desktop Summit	Berlin, Germany
August 10 August 12	USENIX Security ’11: 20th USENIX Security Symposium	San Francisco, CA, USA
August 10 August 14	Chaos Communication Camp 2011	Finowfurt, Germany
August 13 August 14	OggCamp 11	Farnham, UK
August 15 August 16	KVM Forum 2011	Vancouver, BC, Canada
August 15 August 17	YAPC::Europe 2011 “Modern Perl”	Riga, Latvia
August 17 August 19	LinuxCon North America 2011	Vancouver, Canada
August 20 August 21	PyCon Australia	Sydney, Australia
August 20 August 21	Conference for Open Source Coders, Users and Promoters	Tapei, Taiwan
August 22 August 26	8th Netfilter Workshop	Freiburg, Germany
August 23	Government Open Source Conference	Washington, DC, USA
August 25 August 28	EuroSciPy	Paris, France
August 25 August 28	GNU Hackers Meeting	Paris, France
August 26	Dynamic Language Conference 2011	Edinburgh, United-Kingdom
August 27 August 28	Kiwi PyCon 2011	Wellington, New Zealand
August 27	PyCon Japan 2011	Tokyo, Japan
August 27	SC2011 - Software Developers Haven	Ottawa, ON, Canada
August 30 September 1	Military Open Source Software (MIL-OSS) WG3 Conference	Atlanta, GA, USA

If your event does not appear here, please tell us about it.