LWN.net Weekly Edition for February 16, 2012

Book review: Open Advice

The recently released Open Advice has much to offer those who are new to free software and its communities, but there is plenty of interest to veterans as well. It is a collection of essays from an auspicious number of contributors (42) to free and open source software (FOSS) that centers around the idea of "what we wish we had known when we started". As might be guessed, the book encompasses more than that—it ranges all over the FOSS map—including recollections, war stories, philosophical musings, academic research, and good advice.

The book was the brainchild of KDE contributor Lydia Pintscher, who served as the editor as well as contributing one of the essays, "Being Allowed to Do Awesome". The book was released at the recent FOSDEM conference in Brussels and is available in a variety of formats (PDF, EPUB, Mobi). But the book is licensed under the CC-BY-SA license, which means that the LaTeX source is also available. In addition, printed versions of the book can be ordered from Lulu and, soon, Amazon.

The material is spread out over 16 chapters after a foreword by Free Software Foundation Europe founder Georg Greve. He sets the tone for the book by introducing the ideas of free software and communities, specifically connecting the rise of the internet and free software as "co-dependent" developments. Not only does free software run much of the internet, but many of the internet giants that are popular today (Google, Facebook, and Twitter are specifically mentioned) could not have gotten so far so fast without depending on free software.

What follows is a bit of a wander through the different facets of our communities, with an eye toward passing on some of the hard-won knowledge that these contributors have gained. Armijn Hemel notes that projects need to coalesce around code, so that there is something to work with and improve. All the design documents and ideas in the world will not actually build into a community. Evan Prodromou furthers that idea by noting that once there is code, the founder needs to step back and let others contribute. It is, he says, sometimes hard to do, but is essential:

Markus Krötzsch and Felipe Ortega talk about the connection between academia and FOSS, but come at it from different angles. Krötzsch looks at the challenges that researchers face when opening their code, many of which are applicable to anyone trying to form a community of users and contributors. He likens the effort to gardening. Ortega looks at how different FOSS communities evolve as project members come and go. He notes several studies of different kinds of projects and how they have overcome the "generational relay" problem—handing off the project to new leadership over time.

But in order to increase a project's contributors, some recruitment and mentoring are needed. That's the subject of one of the chapters. In it, Leslie Hawthorn points out that contrary to their belief, people new to the project have something unique to offer:

Hawthorn's point crosses into the realm of documentation, which is the subject of a later chapter, but that just highlights one of the strengths of the book. Few of the essays neatly fit into the categories of the chapter they appear in. They largely represent a personal view of the experiences of the author, which often range across various parts of the FOSS world. They are also uniformly encouraging to those who may know little or nothing about how to participate and why they might want to.

Pintscher's entry notes that the biggest enemy of free software is "not who most people on the Internet think it is", but is, instead, a lack of participation. She notes that it takes active effort from existing project members to get new contributors involved, but that it's worth the effort. It's also important to recruit from outside the existing contributor base:

Henri Bergius writes about cross-project collaboration and notes that creating libraries, rather than frameworks, better fosters that collaboration. In addition, he says, meeting in person can go a long way toward helping projects work more closely together:

That sentiment is echoed by others, including Nóirín Plunkett in the chapter on "Conferences and Sprints":

But, all is not "sweetness and light" in the free software world as Máirín Duffy Strode describes in her look at the interaction between designers and developers. Her essay, "Don't Be Shy", suggests that designers (and, by extension, all new contributors) make their needs known so that they can "help the project help you help them", which can't happen without making it clear what's needed to get the job done. She also notes a bit of cautionary tale about being chased away from a project and suggests that new contributors be persistent:

There are several essays on various aspects of documentation. From Atul Jha's reflection on how Eric S. Raymond's writings (in particular the jargon file and "How To Ask Questions The Smart Way") inspired him, to Shaun McCance's look at how to use the "crowd" to generate project documentation, there is a wealth of interesting information. Rich Bowen plays off Larry Wall's famous "virtues of a good programmer" (laziness, impatience, and hubris) and notes that those virtues unfortunately give some programmers a "license to be jerks". He goes on to describe virtues for another group:

Anne Gentle introduces an interesting idea about how a new contributor can "take the pulse" of a project to get a feeling for how hard or easy it will be to get involved:

In the realm of usability, Guillaume Paumier offers up some things he's learned working with the Wikimedia Foundation. He suggests that developers sit down and passively watch users interact with their application. It is "truly an eye-opening experience". Furthermore, he offers up an important point that sometimes gets lost in the development process: "Users are an unpredictable species. But they are on your side. Learn from them."

Federico Mena Quintero has a lengthy essay on "Software that Has the Quality Without A Name". In it he takes concepts from the architectural (i.e. buildings, not software) studies done by Christopher Alexander and others and applies them to software design. The "quality without a name" is embodied in buildings and spaces that are eminently livable. Alexander has come up with a number of properties that govern such spaces and Quintero (by way of Richard Gabriel's Patterns of Software) applies those ideas to software. It is one of the more philosophical essays in the book, and well worth reading in its entirety.

There is also some real "nuts and bolts" advice on things like conference planning (from Dave Neary) and a nice explanation of "How to Ask for Money" (for conferences) by Selena Deckelmann. Beyond that, there is advice on community management and the role of a community manager from Jono Bacon, thoughts on the intersection of law and FOSS from Till Jaeger, ideas about testing from Ara Pulido (and others), and on and on. Not mentioning one of the essays in this review is in no way an indictment of the essay or author, there is more there than one could hope to cover. In addition, each reader will undoubtedly have their own slant on the most interesting and useful essays in the book.

Open Advice is a book that will be helpful to those who are new to FOSS, but, because of the individual voices, styles, and tones, it doesn't read like a "how to". It could even be recommended to those who aren't necessarily interested in contributing, but are curious about what this "free software thing" is all about. It is, in short, a great book for a variety of audiences and the (mostly) two or three page essays make it easy to read, while the anecdotes and recollections personalize it. The authors, editor, and everyone else who helped should be very pleased with the result. Readers will be too.

Comments (16 posted)

FOSDEM: The Wayland display server

In his talk at FOSDEM (Free and Open Source Software Developers' European Meeting) 2012 in Brussels, Kristian Høgsberg gave an overview of the Wayland display system, where it comes from and how the Linux graphics stack has evolved to make Wayland feasible. He also shared some information about the schedule of the 0.85 and 1.0 releases. If all goes according to plan, we'll see the first stable version of Wayland by the end of the year.

Replacing a tried-and-true technology such as X by a new one such as Wayland is no small task, and the idea alone has already stirred up the Linux community in the past. However, not every critic understands exactly what Wayland is. Apparently some even have various strange opinions about Kristian, for instance that he holds a grudge against X or that he even doesn't know anything about how X works. To dot the i's, Kristian started his talk by clarifying that he has been a core X developer since 2004. "I admire the design and implementation of X, it just happens that it's the wrong solution for what we do now." Kristian started developing Wayland in 2008 in his spare time while working for Red Hat, and now he is working on it at Intel in the graphics team of the Open Source Technology Center.

Where do we come from?

A while ago, X did lots of things related to its primary task as a display server. For instance, it did font management and font rendering, graphical mode setting and acceleration code was tied up in X specific drivers, the X server had input drivers to parse various input device serial formats, and so on. However, bit by bit many of these tasks were moved into separate components, many of them in the kernel. Wearing his Red Hat hat, Kristian worked on AIGLX (which allowed compositors to run with hardware acceleration on X), DRI2 (which provides OpenGL hardware acceleration), KMS (Kernel Mode Setting), and GEM (Graphics Execution Manager, doing memory management for graphics drivers). In principle, all hardware drivers needed by X are now in the Linux kernel (KMS) or in Mesa (an OpenGL implementation).

Much of this work was done as part of Kristian's job as a member of Red Hat's X team to make a composited desktop possible. Without a compositor, each application renders its output directly into the buffer of the X server. With a compositor (such as Compiz, KDE's KWin, GNOME 3's Mutter, and Xfce's Xfwm), applications render their output in their own private off-screen buffers. The compositor renders the final desktop output by painting those buffers onto the screen's frame buffer. On a modern Linux desktop, compositing has become a basic expectation and requirement, Kristian says: "Compositing is not only about those 2D and 3D visual effects, but it has other fundamental advantages, such as flicker-less movements of windows."

But now that we have reached a point where most of the hardware drivers implement KMS or Mesa and compositing is used for drawing application windows on the screen, most of the work in the window system is done by the compositor and the applications themselves; the X server is just used as a middle-man for input. Kristian wanted to create a display server that directly supports this new window system architecture from the ground up. This became Wayland, which integrates the display server, window manager, and compositor into one component, and is, according to Kristian, "essentially just consolidating existing practices and simplifying the X architecture to what we are using it for nowadays." The result is a window system architecture that is more responsive and has less graphical artifacts such as tearing or flickering. More details about Wayland's architecture and the difference with X can be found on the project's wiki.

Hardware support

So, with this new window system architecture, Wayland merges the compositor into the display server, and applications talk directly to the compositor. The compositor reads input from devices such as the keyboard and mouse using the kernel's evdev subsystem, and it distributes this input to the applications. The compositor uses KMS to bring up the display, and applications push their buffers to the compositor, which combines the buffers from the various applications and renders the result using direct rendering, through OpenGL ES, to the KMS buffers. "There is no hardware-specific code whatsoever in the compositor code," Kristian emphasizes, "and there is also no rendering API for drawing lines, text, and so on: it's all just direct rendering."

Kristian's short answer to the question which graphics hardware support Wayland is "If the Mesa driver for your graphics hardware supports DRI2 under X and has KMS support, you can run Wayland." Currently, this means that you'll have most luck with the open source drivers for Intel, AMD and NVIDIA graphics hardware, but when driver writers add new chipset support in Mesa, it will automatically support Wayland. It's still unclear what AMD and NVIDIA will do with their proprietary drivers.

Wayland support in graphical toolkits

But applications have to be modified to be able to use Wayland instead of X, of course. Fortunately, most applications don't talk to the X window system directly but are using graphical toolkits, which provide the common user interface elements such as text fields, scrollbars, buttons, and so on. So most of the work in porting an application from X to Wayland is in porting the toolkit it uses. According to Kristian, all major toolkits are being ported to Wayland as we speak, including GTK+ 3, Qt 5, EFL (Enlightenment Foundation Libraries), Clutter, and SDL (Simple DirectMedia Layer). The Qt team at Nokia is maintaining and driving the Qt port themselves, he said. Wayland's web site has instructions on how to use these toolkits with Wayland. Applications that are using their own toolkit, such as Blender and LibreOffice, will have to do the porting effort themselves.

Porting a graphical toolkit to Wayland poses some interesting challenges, for instance because Wayland doesn't support some things that are quite natural in X, such as pointer grabbing: in Wayland an application currently cannot claim the keyboard or mouse input. However, games regularly grab the mouse cursor to prevent the player from accidentally losing mouse control over their game, and pop-up windows rely on grabbing the keyboard and pointer too. Another challenge is the support for client-side window decorations, which allows individual applications to control their appearance. Both challenges are being worked on.

Applications that aren't using a Wayland-compatible graphical toolkit or that have some problems running on Wayland can still run on an X server that runs as a Wayland client. Of course this adds some overhead, but it should be minimal. Running a rootless X Server as a Wayland client could also be a temporary solution to keep using network transparency, a feature that Wayland lacks. In his talk, Kristian reassured the audience that the current lack of network transparency is not definitive: "While supporting remote displays is not a priority now for Wayland, nothing in Wayland's architecture makes this impossible."

Toward a stable API

The first real release of Wayland happened a few days after Kristian's talk at FOSDEM: Wayland 0.85. The code is divided in two parts: Wayland is the protocol and IPC mechanism, while Weston is the reference implementation of the compositor (and thus also the display server). "Weston has around 10,000 lines of code and it can be used as the base for a compositor for mobile or embedded systems," according to Kristian, who ran Weston on his laptop during his FOSDEM presentation. However, existing X compositors such as KWin and Mutter can also be modified to become a Wayland compositor.

Kristian promises that the 0.85 release is going to be protocol and API stable. The point of the release is that developers can begin experimenting with Wayland and porting their toolkits and applications. While it used to be that you had to compile a special KMS "pageflip" kernel, a patched Mesa branch, and Kristian's own EGL library to be able to run Wayland on your system, now everything should be upstream. Kristian hopes to put out a Wayland 1.0 release in late 2012 or early 2013. So while Wayland will not replace X overnight, we will likely see the first mainstream use of the new window system architecture in 2013.

Comments (38 posted)

Gray areas in software licensing

This year saw a new addition to FOSDEM in Brussels - "legal issues" gained its own DevRoom and joined the long list of topics presented and discussed during the conference. I was privileged to be asked to speak at the track, and when Bradley Kuhn asked me what I'd like to speak about, my initial reaction was the theme: "how much I hate having to care about the law".

Developers hate the law. Or, to be more precise, I hate the law. At least, whenever I'm working on a software project. In my opinion, every time a software developer has to stop and think about the law, that's a bug. Either he wants to do something that he shouldn't be doing, or the law is getting in the way of him doing something which it would be reasonable for him to do.

In case it's not obvious yet, I am not a lawyer, so nothing in here should be taken as legal advice. But, over the years, while working as a GIMP developer, as community and product manager of the OpenWengo project, or as an independent consultant, I have had to stop and consider the potential impact of the law on what I was doing. And every time that I have sought legal advice on the issues, the result has been unsatisfactory. Here are a few examples from my personal experience.

GIMP plug-ins

In the late 1990s and early 2000s, the licensing of plug-ins was a hot topic in the GIMP community that came up repeatedly. Does the GPL allow people to develop proprietary GIMP plug-ins? It was certainly the intention of the developers active at the time to allow proprietary plug-ins. To explain why this question is even interesting, some technical details on GIMP plug-ins are useful.

GIMP plug-ins run in a separate process from the GIMP core. They link to libgimp, a library that was licensed under the LGPL in 1997, which is a wrapper around the PDB, the Procedural Database. The Procedural Database is how plug-ins communicate with the core GPL application: data is sent via shared memory, with the PDB defining arguments and return values from plug-in functions. Plug-ins can call core functions, again via the PDB, to get image data from a drawable (a GIMP object representing anything you can draw on), or for any of a range of core functions.

But some argued that since GIMP plug-ins could only work with the GIMP core, that they were really derivative works, and were thus covered by the GPL. When we asked for definitive legal advice on the issue, the answer we got back was not entirely satisfying.

Maybe.

Ugh. So to clarify the situation, the GIMP developers added a license clarification to the source code, alongside the GNU GPL COPYING file, stating that we considered plug-ins to be "mere aggregation", and not a "derivative work". This may not be legally rigorous, but it was enough to allow us to forget about the issue and get back to coding.

Distributing GIMP plug-ins

That is, until it was pointed out by Debian that GIMP was shipping some GPL-incompatible code in plug-ins that we were distributing in the gimp .tar.gz. This issue ended up blocking the release of GIMP 1.2.4 for over a year. So the question arose: can we include non-GPL plug-ins in a GIMP package? The answer was, you guessed it, "maybe". At least, that's as far as we could tell by discussing it in a Debian bug report and on gimp-developer.

While GIMP plug-ins might be a "mere aggregation" with the GIMP core, apparently, there was an expectation that all the plug-ins which were shipped with GIMP would be GPL compatible. Over the years, we had included a number of code snippets from other open source projects which were licensed under other licenses. One of the projects was SIOD, Scheme in one Defun, a Scheme implementation by George Carrette. SIOD was the back-end of Script-fu, GIMP's scripting plug-in, at the time. SIOD had a very liberal license which included an "advertising clause", requiring that "the permission notice appear in supporting documentation", a clause which was widely argued to be incompatible with the GPL at that time. It turned out that there were about a dozen occurrences of this advertising clause in the code for plug-ins at the time.

Since Debian wouldn't ship GIMP with these plug-ins unless we made the licensing consistent, and since some GIMP developers also expected us to ship only GPL-compatible plug-ins, this was a major problem. So I and others spent a month in 2003 chasing down authors at five-year-old email addresses, removing or rewriting code snippets when we could do so, and on occasion removing offending copyright headers when we decided that the amount of code covered by the problematic licenses was so small that it was not a problem. I am sure I would have preferred to do other things with my time - and I was certainly not as methodical as Gervase Markham when he was relicensing Mozilla code - a process that took over 4 years. Were we legally rigorous? Maybe.

OpenWengo codecs

My next example is audio codecs from the OpenWengo project (site now dead). Some codecs like G729 were patent encumbered. In addition, widely used reference implementations of codecs like iLBC were included as sample code in standards documents and were only available under GPL-incompatible and non-free software licenses. The way we tried to get around this problem was to encapsulate these codecs in a standard API, and install them in a common directory. At start-up, we scanned the directory, and used dlopen() to open the libraries present there. We could then call the standard function to register the codecs and pass data streams for decoding and encoding.

A question arose for some of the non-GPL codecs: could we do something in the start-up process to make it easier for the user to obtain these codecs? One possibility would be to present the user with a dialog, as part of the first application start-up, which fetched the proprietary codecs and installed them in the common directory. But, would that be against the spirit of the GPL, and have us running the risk of being called a GPL violator by one of our users?

The answer, apparently, is "maybe".

By divorcing the loading of the plug-in at run-time from the distribution of the plug-in, we were probably absolving ourselves of any GPL obligations. At least, that is the legal advice Wengo obtained at the time. However, whether making it easier to get the codecs at start-up was the legal equivalent to including the codecs in the package was, at best, ambiguous.

Hunspell dictionaries

Last year, I was asked an interesting question by a client: Can I include Hunspell and the Czech dictionary in a proprietary application?

The answer, of course, is "maybe".

Hunspell changed its license in 2006, to the MPL/GPL/LGPL tri-license to enable inclusion in Mozilla. It is used as the spell-checker for OpenOffice.org/LibreOffice as well as Mozilla Thunderbird and Firefox. Of course, the tri-license enables the embedding of Hunspell in proprietary applications. So where's the problem? It turns out that some Hunspell dictionaries, including Catalan, Danish and Czech, are licensed under GPL only. It's a bit curious why anyone would use the GPL, a license intended for use with source code, for a dictionary. Regardless, the question is: does including Hunspell in another application, with a GPL dictionary, constitute a derivative work, or an aggregation?

At one level, the question is whether it even makes sense for a dictionary to have a license. Thanks to Feist v Rural, copyright only applies to a "work" if there is significant creativity involved in it. If a Hunspell dictionary were just a list of words in alphabetical order, then at least in the US, copyright would not apply. However, in other jurisdictions, there are laws covering the constitution of databases, and some form of protection is afforded to the author. In addition, a Hunspell dictionary isn't really just a list of words - it also contains grammatical and hyphenation information, which involves some creativity. So there's a solid argument that the dictionary is copyrightable.

The second question is whether including the dictionary with Hunspell and other software makes a derivative work, or an aggregate work. One part of that is whether the component is useful without the Hunspell spellchecker. Since it's just a dictionary, it can be used with other spell-checkers, including aspell. Hunspell could easily use another Czech dictionary released under another license. So, since the two bits are easily dissociable, Hunspell with the dictionary probably makes an aggregate work, and the license of the dictionary does not impose any requirements on other software shipped with it. This is the advice that the OpenOffice.org project received from the FSF licensing list, and so the GPL dictionaries are included in the LibreOffice and OpenOffice.org distributions.

Mozilla, on the other hand, do not ship any GPL dictionaries. When I asked Gervase why, he answered:

In other words, the intent of the author trumps all in this case, for Mozilla.

So - maybe.

Conversations with lawyers

The problem for developers is that conversations with lawyers typically end up with "maybe". We're looking for what we may and may not do, and often, in the absence of jurisprudence, lawyers deal in shades of gray. All too often, the end result for developers is to avoid any risk.

A friend once told me a story of a meeting with her boss and "legal", where she came out of the meeting disappointed. When her boss asked her what was wrong, she said "the lawyers told us we can't do what we'd like to." To which he replied, "No, they just told us the risks involved. Now we have to decide whether we're prepared to accept those risks or not." Instead of being the end of the conversation, "maybe" could be the start of it. Lawyers (and , in the free software world, some non-lawyers) will help you identify risks. But then it is up to the project to decide whether to take the risks or not.

Comments (17 posted)

Page editor: Jonathan Corbet

Security

Tor offers SSL obfuscation for users behind censorship walls

On February 10, the Tor project posted reports from users inside Iran that the country's government had begun blocking all SSL/TLS traffic, a major escalation of firewall policies that had already cut off users from specific services. Tor responded by pointing readers to obfsproxy, its comparatively-little-known project that can disguise SSL traffic to evade detection by the deep packet inspection (DPI) filters used to flag and shut down encrypted connections.

Word from users in Iran was that the government SSL-blocking effort appeared to be DPI-based, because of the fact that connections were being terminated only after the first few steps of the SSL handshake. Other methods to bypass the firewall, including VPNs, were still functional, although they are an impractical (and sometimes expensive) solution for the majority of Iranian Internet users. However, the block prevents standard Tor usage in particular, because the system relies on an encrypted connection between the user's machine and the first relay in the Tor network.

Earlier filtering techniques, such as blocking access to specific IP addresses, had been bypassed by using bridges — Tor relays that ran on unpublished IPs. Reports indicated that the SSL-block varies by ISP, and had not affected the entire country, but the project's metrics showed a sharp decline in the number of Tor users originating from Iran, starting around February 9. Alongside the announcement of obfsproxy, the Tor project asked users to help restore connectivity for people in the region by setting up obfuscated bridges — but cautioned that drawing too much public attention to the project could prompt authorities to implement countermeasures.

It rolls right off the tongue: "obfsproxy"

Obfsproxy — which, although just announced to the public, has been in development since early 2011 — offers relief from DPI filtering. It is a transport proxy that encapsulates protocol traffic between endpoints within an "innocent-looking" wrapper. The system is modular enough that the project says it can be used with a variety of protocols, but the default usage is designed to wrap SSL traffic between a Tor client and Tor bridge inside another application-layer connection. Furthermore, within those faux application-layer packets, the genuine SSL packets are encrypted by a stream cipher, making their contents immune to detection by DPI filters (which catch protocols by matching characteristic strings or regular expressions in the TCP stream).

Obfsproxy's default module is called obfs2, and merely disguises SSL traffic as unencrypted SOCKS traffic. It does not itself provide authentication, confidentiality, or guarantee data integrity. Those features must be provided by the traffic being obfuscated (e.g., SSL and Tor). Nor does it protect against protocol fingerprinting using other means (such as timing or packet size), nor against attackers looking specifically for obfsproxy itself. The threat model document in the project's Git repository outlines the assumptions and characteristics in specific detail, and argues that although the scope is limited: it "protects against many real-life Tor traffic detection methods currently deployed, since most of them currently use static SSL handshake strings as signatures."

Tor executive director Andrew Lewman told Forbes magazine that other protocol wrappers are a possibility for future releases, including XMPP and vanilla HTTP, and described a simpler client-side interface. At the moment, however, he described the project as "very much a work in progress, and the various pluggable transports are still in design and development."

The anatomy of obfs2

Obfsproxy is a recent addition to Tor, and although the project has released updated Tor Browser Bundle binaries pre-configured to use it, for most existing Tor users it requires compiling and running the client code — as well as knowledge of a Tor bridge also configured to run obfsproxy. Still, Tor reports that users in Iran are taking advantage of the code to successfully restore their lost connectivity.

Masking a Tor connection between client and bridge requires both participants to be running obfsproxy, but the client-bridge connection is the only one involved in the obfuscation — no general-purpose Tor nodes (including exit nodes) are required to install obfsproxy or need to alter their configuration.

Interested client and bridge users should fetch obfsproxy from the Tor project's Git repository and compile it with Autogen and GNU make. All versions of Tor newer than 0.2.3.11 can be configured to use obfsproxy simply by editing the Tor configuration file. Clients must add the IP address and port number of an obfsproxy-aware bridge and path to the obfsproxy executable. Bridge operators must start Tor and watch their logs, because Tor randomly selects an open, higher-numbered TCP port for obfsproxy to listen on the first time it is run. Older versions of Tor can use obfsproxy, too, using additional steps to configure a localhost-only relay between Tor and the obfsproxy program.

Whichever setup is involved, the obfs2 protocol operates in the same manner. It is based on Bruce Leidl's older work to obfuscate SSH handshakes. The client and the bridge first exchange session keys with each other, after which they "superencipher" their SSL session by encrypting it with 128-bit AES.

By default the protocol uses a relatively weak key-exchange method that could be compromised by an eavesdropper listening to both sides of the conversation — although the use of a pre-shared secret to strengthen this step is supported as well. It may sound as if the weak key exchange method undermines the whole process, but the important thing to remember is that the obfuscation protocol's only goal is to defeat automatic detection by pattern-matching DPI filters. Furthermore, the seed values that the client and bridge exchange are concatenated with constants and then hashed with SHA256, a step that does not make them unrecoverable, but that is computationally expensive to perform on the class of high-throughput networking hardware generally used to do DPI traffic analysis.

Obfuscation for all

To the paranoid, obfs2 may sound like an imperfect solution to the filtering crisis. After all, it could be defeated by closer inspection of packets or filtering out SOCKS traffic. In that sense, obfsproxy might be likened to steganography — its goal is to hide the traffic of estimated tens-of-thousands of Tor users in a censored region like Iran among the connections of millions.

The Tor project reports that it has performed experiments of its own, and found obfsproxy to be effective "in all censored countries" when used as-is. However, as Jacob Appelbaum mentioned in the call for obfsproxy bridges, "it might even only last for a few days at the rate the arms race is progressing, if you could call it progress." Then again, thanks to the modular design of obfsproxy, the obfs2 module itself can be replaced or upgraded in future releases, both to disguise traffic better, or to implement completely different security features. The sudden crackdown on all SSL traffic in Iran might have hit before a perfect system was in place, but obfsproxy is still a welcome relief for those who are affected, and have no other practical options.

Comments (8 posted)

Brief items

Security quotes of the week

Comments (8 posted)

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

Comments (40 posted)

Horde Groupware contains backdoor (The H)

Comments (none posted)

Garrett: Some things you may have heard about Secure Boot which aren't entirely true

Matthew Garrett clears up some Secure Boot myths on his blog:

Comments (2 posted)

New vulnerabilities

apr: denial of service

Comments (none posted)

bugzilla: multiple vulnerabilities

Comments (none posted)

cvs: remote code execution

Comments (none posted)

devscripts: multiple vulnerabilities

Comments (none posted)

ettercap: insecure settings file

Comments (none posted)

firefox: multiple vulnerabilities

Comments (none posted)

glpi: file inclusion vulnerability

Comments (none posted)

gnutls: denial of service

Comments (none posted)

java: multiple vulnerabilities

Comments (none posted)

kernel: denial of service

Comments (none posted)

kernel: memory corruption

Comments (none posted)

kernel: multiple vulnerabilities

Comments (none posted)

kernel: unauthorized file access

Comments (none posted)

mozilla: code execution

Comments (none posted)

mysql: multiple unspecified vulnerabilities

Comments (1 posted)

mysql: multiple unspecified vulnerabilities

Comments (none posted)

php: multiple vulnerabilities

Comments (none posted)

phpldapadmin: cross-site scripting

Comments (none posted)

puppet: unintended access to resources

Comments (none posted)

samba: denial of service

Comments (none posted)

selinux-policy: policy enhancements

Comments (none posted)

sysconfig: code execution

Comments (none posted)

wireshark: multiple vulnerabilities

Comments (none posted)

xchat-ruby: null pointer dereference, remote DoS

 static_ruby_custom_command_hook(char *word[], char *word_eol[], void *userdata)
 static_ruby_custom_server_hook(char *word[], char *word_eol[], void *userdata)

parameter 'word' used in a for cycle without break [1]
 for( i = 1; word[i][0] != '\0'; i++ )

Comments (none posted)

znc: denial of service

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

Stable updates: 3.0.21, 3.2.6 and 2.6.32.57 were released on February 13 with a long list of important fixes.

For those still using the 2.6.27 kernel, 2.6.27.60 was released on February 12, quickly followed by 2.6.27.61 to fix the inevitable build error. There's lots of fixes that have gone in since 2.6.27.59 was released last April.

Comments (none posted)

Quotes of the week

-static inline void *kcalloc(size_t n, size_t size, gfp_t flags)
+static inline void *wtf_do_i_call_this(size_t n, size_t size, gfp_t flags)
 {
 	if (size != 0 && n > ULONG_MAX / size)
 		return NULL;
-	return __kmalloc(n * size, flags | __GFP_ZERO);
+	return __kmalloc(n * size, flags);
+}
+
+static inline void *kcalloc(size_t n, size_t size, gfp_t flags)
+{
+	return wtf_do_i_call_this(n, size, flags | __GFP_ZERO);
 }

Comments (2 posted)

Btrfs: The Swiss army knife of storage (;login:)

Comments (14 posted)

Lima driver code for the Mali GPU released

Comments (9 posted)

Kernel development news

ABS: Android and the kernel mainline

On the first day of this year's Android Builders Summit, a panel was held to discuss the Android patches to the Linux kernel, including the progress on getting them upstream. The panel consisted of Zach Pfeffer from Linaro, Tim Bird from Sony, Arnd Bergmann of IBM/Linaro, and Greg Kroah-Hartman of the Linux Foundation (LF), with LWN executive editor Jonathan Corbet moderating. The overall feeling from the panel was that things with the Android kernel patches were proceeding apace—recent kernels can boot into an Android user space—but there is still work to be done. While I could not attend ABS this year, this report comes via the magic of streaming video.

Each of the panelists introduced themselves and connected themselves to Android in various ways. Pfeffer is the Android platform lead for Linaro and is responsible for creating Android builds for each of the member companies, Bird represents Sony as the architecture chair of the LF Consumer Electronics working group, Bergmann has recently been working on the cleanup and consolidation of the ARM subtree, and Kroah-Hartman maintains the staging tree where most of the Android code currently lives.

Corbet kicked off the discussion by noting that he had recently looked at the current Red Hat Enterprise Linux (RHEL) kernel, which includes more than 7600 patches on top of the mainline kernel. He also pointed out that the Fedora kernel carries a number of different patches that aren't likely to go upstream anytime soon (utrace, in particular). Given that, "why are we here?", why is Android (and its patches) treated differently than other distributions' kernels, he asked.

Kroah-Hartman said that the Android patches are only about 7000 lines of code and that some UART drivers are larger. But people care more about the Android patches because device makers and others have to pull down all those patches and apply them to get a mainline kernel working with Android. It is different from enterprise kernels, he said, because there are no real downstream users of the source code of those. Bergmann also pointed out that much of the change in the RHEL or Fedora kernels was for things like drivers which don't change the operating system core as some of the Android patches do.

Pfeffer noted that in the past the kernel developers were seen as "the rebel alliance", but that now the Android developers have assumed that role to some extent. Bird pointed out that part of the problem is that due to the success of Android, the kernels for board support packages (BSPs) are being built with Android kernels, rather than kernel.org kernels. That essentially causes a split in the community.

The Android patches are largely in the mainline at this point (in staging), Kroah-Hartman said, except for the wakelocks code. The 3.3-rc3 kernel can boot an Android user space, but lacks the power management features that wakelocks provide, so battery life is poor. Bergmann said that he had seen a demo of Android running on a mainline kernel, and there is "still a long way to go".

One area that needs attention, Bergmann said, is the user-space interfaces of some of the Android features. Those may not be what the kernel developers want to support long-term, so they need to be addressed before the Android patches make their way out of staging and those interfaces become part of the kernel ABI.

The PMEM patches, which provide a means to reserve contiguous memory and to share buffers between the kernel and user space, was the next topic. Corbet noted that PMEM had been in and out of staging twice, but had never been merged. Since then, Android has moved on and is not using PMEM any longer. So, was it the right move not to merge PMEM, he asked.

The panel seemed in agreement that it was right not to merge those patches, with Pfeffer noting that they were simply an expedient to get products out the door. As ARM matures, he said, common usage models will come about, rather than various quick fixes. Kroah-Hartman pointed out that the memory management kernel developers told the PMEM developers "how to do it right", but that no one ever did that work, which is "a problem that we've had forever". Bird agreed with Pfeffer, saying that he is pushing to get things into the mainline, but that if there is pushback, "that's fine", as there are sometimes "quick and dirty" things done to ship products.

Corbet pursued that topic with a mention of the contiguous memory allocator (CMA) that is being pushed by Samsung and is aimed at Android. But he noted that Android itself has moved from PMEM to the ION memory allocator, which duplicates some of the CMA functionality while adding another user-space interface. What should be done about ION, he asked.

Android is not "pushing" ION, Pfeffer said, and if the kernel community doesn't want it, it shouldn't take it. There was a need for a solution that didn't exist in the mainline, so Android went ahead with ION. It may not necessarily be easy to work with all of the Android kernel patches, he said, because of the time pressures it is operating under. In the end, ION could be handled much like PMEM was, he said.

But Bergmann pointed out that ION could be integrated with CMA and the DMA buffer sharing work that are getting close to the mainline. ION and the soon-to-exist mainline solutions are not mutually exclusive, he said. Pfeffer said that the "entire room" doesn't really care which solution is chosen, they just want something that has been integrated and tested. That may be ION now, and something else down the road.

In a sense, this is a result of the longstanding conflict between the needs of the embedded world vs. those of the rest of kernel, Corbet posited. But Kroah-Hartman disagreed, saying that the enterprise distributions have the same problems, just on a different timescale. Those distributions need to ship, often well before the code is upstream. Embedded isn't really different, they just need to get their code upstream as well, he said.

One place where it is different, though, is that in the embedded world things may move from hardware to software at a relatively rapid pace, Pfeffer said. That means that a driver written for hardware JPEG decoding may really only be needed for one six-month product cycle, so that driver really doesn't need to go upstream. Because of that, Bergmann said, many of those drivers are designed to stay out of the mainline. Bird echoed that, saying that sometimes a "fatalistic approach" to kernel development is the pragmatic choice. Even for long-lived code, if there is no hope that it can go upstream, developers will just try to focus on making it maintainable out of tree.

For a long time, the trajectory of Android code was heading away from the mainline, Pfeffer said, but that has started to correct itself. Though the kernel needs more maintainers so that code can get upstream faster, he said. Bergmann agreed that more maintainers would help, but the work needs to be done in a more organized fashion with an eye toward getting the user-space interfaces right.

Bird said that he doesn't expect there to be a need for another panel of this sort because the problem is solving itself at this point. Kroah-Hartman more or less agreed, noting that the Android problems are "nothing new" and that the kernel community has been solving these kinds of problems for 20 years.

With luck, the Android developers won't be adding much more to the core, Corbet said, but what about drivers? CyanogenMod is trying to get Ice Cream Sandwich (Android 4.0) in the hands of its users, but is running into problems with getting drivers from some vendors. What can be done to solve that problem?

Kroah-Hartman noted that things are getting better in that department, but that some companies care about getting their drivers into the mainline, while others don't. The latter don't realize that it will save them money in the long run. He is often talking with various companies, so if there are specific instances of problem drivers, he wants to know about them.

But kernel drivers are not the whole problem, Pfeffer said. In the graphics and video realms, the line between the kernel and user space has been "blown away", he said. There is now firmware, kernel interfaces, binary blobs, and user-space interfaces to do graphics which are all released in lockstep, and often all as binary blobs, which is not a good thing. It isn't an engineering issue, but a legal one, he said. Google's engineers do not want binary blobs, he said, and have been pushing back on vendors for things like the Nexus line of phones.

Bergmann also pointed to the open source driver for the Mali GPU as an indicator of the direction where things are headed. If the other graphics vendors don't get their act together with respect to free drivers, they will not survive, he said.

With that, the 45-minute slot had expired. The upshot seems to be that mainline kernel support for Android is moving along reasonably well. It won't be too long before it will be possible to run Android on a mainline kernel while still maintaining some reasonable battery life. Beyond that, though, the process is working more or less as it should. The out-of-tree Android patches were just another in a long line of hurdles that the kernel community has overcome.

Comments (none posted)

Trusting the hardware too much

Consider this snippet of code from drivers/char/hpet.c:

	do {
		m = read_counter(&hpet->hpet_mc);
		write_counter(t + m + hpetp->hp_delta, &timer->hpet_compare);
	} while (i++, (m - start) < count);

Here, read_counter() is a thin macro wrapper around readl(). The driver is writing to the timer compare register in a loop, assuming that the "main counter" value read from the HPET will eventually exceed the threshold value. Almost always, that is exactly what happens. But if the HPET ever goes a little bit weird and stops returning something meaningful when the main counter is read, the above code could easily turn into an infinite loop. The kernel is trusting the hardware to be rational, but the hardware may not choose to live up to that expectation.

"Usbmuxd" is a daemon which facilitates communications with various Apple iDevices. Recently, this patch to usbmuxd was recognized to be a security fix for a bug eventually designated as CVE-2012-0065. In short, this daemon would read a serial number string from the device and copy it into an internal array without checking its length. Exploiting this vulnerability is not easy - it requires the ability to plug in a USB device that has been designed to overflow that particular buffer with something interesting. But it is a vulnerability, and it is worth noting that an increasing number of USB devices are really just Linux systems using the "USB gadget" code; creating that malicious device would not be hard to do. So this vulnerability could be interesting to the "leave a malicious USB stick in the parking lot" school of attacker.

This bug, too, is the result of trusting the hardware. As seen here, the hardware could be overtly evil. In other cases, it is just subject to electrical wear, power spikes, cosmic rays, and the varying skills of those who write the firmware - closed source which is never reviewed by anybody. Even in a world where price pressures didn't mandate that each component must cost as little as possible, hardware bugs would be a problem.

By now, the lesson should be clear: driver developers should always regard their hardware with extreme suspicion and take nothing for granted. The problem is that even highly diligent developers (and reviewers) can easily let this kind of bug slip by. In almost all cases, the driver appears to work just fine without the extra sanity checks; the hardware plays along most of the time, after all, until that especially inopportune moment arrives. Sometimes the developer sees the resulting failure, resulting in that "oh, yeah, I have to make sure that the hardware doesn't flake there" moment that is discouragingly common in driver development. Other times, some far away user sees strange problems and nobody really knows why.

What would be nice would a way for the computer to tell developers when they are being overly trusting of the hardware; then it might be possible to skip the "tracking down the weird problem" experience. As it happens, such a way exists in the form of a static analysis tool called Carburizer, developed by Asim Kadav, Matthew J. Renzelmann and Michael M. Swift. Those wanting a lot of information about this tool can find it in this one-page poster [PDF], this ACM Symposium on Operating Systems Principles (SOSP) paper [PDF], or in this rather over-the-top web site.

In short: Carburizer analyzes kernel code, looking for insufficiently robust dealings with the hardware. Its key strength at the moment appears to be the identification of possible infinite loops - loops whose exit condition depends solely on a value obtained from the hardware. There are, it seems, over 1000 such loops in the 3.2.1 kernel. The tool also looks for cases where unchecked values from hardware are used to index arrays or are used directly as pointers, though the false-positive rate seems to be higher for these checks. The result is an output file as linked above, from which developers can go and investigate.

Naturally enough, the tool shows some signs of its academic origins. It is written in Ocaml and requires some modifications to the kernel source tree before it can be run. Carburizer also requires that multi-file drivers be merged into one big file, with the result that the line numbers in the resulting diagnostics do not correspond to the source tree everybody else has. That may be part of why, despite a positive response to a posting of the tool on kernel-janitors in January, 2011, little in the way of actual fixes seems to have resulted. Or it may just be that, so far, these results have only been seen by a relatively small group of developers.

Interestingly, Carburizer can propose fixes of its own. These include putting time limits into potentially infinite loops and adding bounds checks to suspect array references. While it is at it, Carburizer fixes up seemingly unnecessary panic() calls and adds logging code to places where, it thinks, the driver neglects to report a hardware failure. With a separate runtime module, it can even deal with stuck interrupts (the driver is forced into a polling mode) and more. The resulting code has not been posted for consideration, which is not entirely surprising; the fixes are, necessarily, of a highly conservative "don't break the driver" nature. Such fixes are almost certain not to be what a human would write after looking at the code. But the tool is open source, so interested developers can run it themselves to see what it would do.

Meanwhile, even without automatic fixes, these results seem worthy of some attention. Computers can be far better than humans at finding many classes of bugs; when computers have been used in that role, some types of bugs have nearly disappeared from the kernel. Maybe someday we'll have a version of Carburizer that can be folded into a tool like checkpatch; for now, though, we'll have to look at its complaints about our code separately and decide what action is needed.

Comments (11 posted)

Linux support for ARM big.LITTLE

ARM Ltd recently announced the big.LITTLE architecture consisting of a twist on the SMP systems that we've all gotten accustomed to. Instead of having a bunch of identical CPU cores put together in a system, the big.LITTLE architecture is effectively pushing the concept further by pulling two different SMP systems together: one being a set of "big" and fast processors, the other one consisting of "little" and power-efficient processors.

In practice this means having a cluster of Cortex-A15 cores, a cluster of Cortex-A7 cores, and ensuring cache coherency between them. The advantage of such an arrangement is that it allows for significant power saving when processes that don't require the full performance of the Cortex-A15 are executed on the Cortex-A7 instead. This way, non-interactive background operation, or streaming multimedia decoding, can be run on the A7 cluster for power efficiency, while sudden screen refreshes and similar bursty operations can be run on the A15 cluster to improve responsiveness and interactivity.

Then, how to support this in Linux? This is not as trivial as it may seem initially. Let's suppose we have a system comprising a cluster of four A15 cores and a cluster of four A7 cores. The naive approach would suggest making the eight cores visible to the kernel and letting the scheduler do its job just like with any other SMP system. But here's the catch: SMP means Symmetric Multi-Processing, and in the big.LITTLE case the cores aren't symmetric between clusters.

The Linux scheduler expects all available CPUs to have the same performance characteristics. For example, there are provisions in the scheduler to deal with things like hyperthreading, but this is still an attribute which is normally available on all CPUs in a given system. Here we're purposely putting together a couple of CPUs with significant performance/power characteristic discrepancies in the same system, and we expect the kernel to make the optimal usage of them at all times, considering that we want to get the best user experience together with the lowest possible battery consumption.

So, what should be done? Many questions come to mind:

• Is it OK to reserve the A15 cluster just for interactive tasks and the A7 cluster for background tasks?
• What if the interactive tasks are sufficiently light to be processed by the small cores at all times?
• What about those background tasks that the user interface is actually waiting after?
• How to determine if a task using 100% CPU on a small core should be migrated to a fast core instead, or left on the small core because it is not critical enough to justify the increased power usage?
• Should the scheduler auto-tune its behavior, or should user-space policies influence it?
• If the latter, what would the interface look like to be useful and sufficiently future-proof?

Linaro started an initiative during the most recent Linaro Connect to investigate this problem. It will require a high degree of collaboration with the upstream scheduler maintainers and a good amount of discussion. And given past history, we know that scheduler changes cannot happen overnight... unless your name is Ingo that is. Therefore, it is safe to assume that this will take a significant amount of time.

Silicon vendors and portable device makers are not going to wait though. Chips implementing the big.LITTLE architecture will appear on the market in one form or another, way before a full heterogeneous multi-processor aware scheduler is available. An interim solution is therefore needed soon. So let's put aside the scheduler for the time being.

ARM Ltd has produced a prototype software solution consisting of a small hypervisor using the virtualization extensions of the Cortex-A15 and Cortex-A7 to make both clusters appear to the underlying operating system as if there was only one Cortex-A15 cluster. Because the cores within a given cluster are still symmetric, all the assumptions built into the current scheduler still hold. With a single call, the hypervisor can atomically suspend execution of the whole system, migrate the CPU states from one cluster to the other, and resume system execution on the other cluster without the underlying operating system being aware of the change; just as if nothing has happened.

Taking the example above, Linux would see only four Cortex-A15 CPUs at all times. When a switch is initiated, the registers for each of the 4 CPUs in cluster A are transferred to corresponding CPUs in cluster B, interrupts are rerouted to the CPUs in cluster B, then CPUs in cluster B are resumed exactly where cluster A was interrupted, and, finally, the CPUs in cluster A are powered off. And vice versa for switching back to the original cluster. Therefore, if there are eight CPU cores in the system, only four of them are visible to the operating system at all times. The only visible difference is the observable execution speed, and of course the corresponding change in power consumption when a cluster switch occurs. Some latency is implied by the actual switch of course, but that should be very small and imperceptible by the user.

This solution has advantages such as providing a mechanism which should work for any operating system targeting a Cortex-A15 without modifications to that operating system. It is therefore OS-independent and easy to integrate. However, it brings a certain level of complexity such as the need to virtualize all the differences between the A15 and the A7. While those CPU cores are functionally equivalent, they may differ in implementation details such as cache topology. That would force every cache maintenance operation to be trapped by the hypervisor and translated into equivalent operations on the actual CPU core when the running core is not the one that the operating system thinks is running.

Another disadvantage is the overhead of saving and restoring the full CPU state because, by virtue of being OS-independent, the hypervisor code may not know what part of the CPU is actually being actively used by the OS. The hypervisor could trap everything to be able to know what is being touched allowing partial context transfers, but that would be yet more complexity for a dubious gain. After all, the kernel already knows what is being used in the CPU, and it can deal with differing cache topologies natively, etc. So why not implement this switcher support directly in the kernel given that we can modify Linux and do better?

In fact that's exactly what we are doing i.e. take the ARM Ltd BSD licensed switcher code and use it as a reference to actually put the switcher functionality directly in the kernel. This way, we can get away with much less support from the hypervisor code and improve switching performances by not having to trap any cache maintenance instructions, by limiting the CPU context transfer only to the minimum set of active registers, and by sharing the same address space with the kernel.

We can implement this switcher by modeling its functionality as a CPU speed change, and therefore expose it via a cpufreq driver. This way, contrary to the reference code from ARM Ltd which is limited to a whole cluster switch, we can easily pair each of the A15 cores with one of the A7 cores, and have each of those CPU pairs appear as a single pseudo CPU with the ability to change its performance level via cpufreq. And because the cpufreq governors are already available and understood by existing distributions, including Android, we therefore have a straightforward solution with a fast time-to-market for the big.LITTLE architecture that shouldn't cause any controversy.

Obviously the "switcher" as we call it is not replacing the ultimate goal of exposing all the cores to the kernel and letting the scheduler make the right decisions. But it is nevertheless a nice self-contained interim solution that will allow pretty good usage of the big.LITTLE architecture while removing the pressure to come up with scheduler changes quickly.

Comments (60 posted)

Patches and updates

Kernel trees

• Linus Torvalds: Linux 3.3-rc3 . (February 9, 2012)

• Greg KH: Linux 3.2.6 . (February 13, 2012)

• Greg KH: Linux 3.0.21 . (February 13, 2012)

• Steven Rostedt: 3.0.20-rt36 . (February 10, 2012)

• Greg KH: Linux 2.6.32.57 . (February 13, 2012)

• Willy Tarreau: Linux 2.6.27.60 . (February 13, 2012)

• Willy Tarreau: Linux 2.6.27.61 . (February 13, 2012)

Core kernel code

• Gilad Ben-Yossef: Reduce cross CPU IPI interference . (February 9, 2012)

• Konstantin Khlebnikov: radix-tree: general iterator . (February 10, 2012)

• Rakib Mullick: [ANNOUNCEMENT] The Barbershop Load Distribution algorithm for Linux kernel scheduler. . (February 13, 2012)

• Paul E. McKenney: rcu: direct algorithmic SRCU implementation . (February 13, 2012)

• Cyrill Gorcunov: Resending, c/r series v2 . (February 14, 2012)

• MyungJoo Ham: [RFC PATCH] PM / QoS: Introduce new classes: DMA-Throughput and DVFS-Latency . (February 14, 2012)

• MyungJoo Ham: PM / QoS: add pm_qos_update_request_timeout API . (February 14, 2012)

• Stanislav Kinsbursky: IPC: message queue checkpoint support . (February 15, 2012)

Development tools

• Xiao Guangrong: KVM: perf: kvm events analysis tool . (February 9, 2012)

• Stephane Eranian: perf: add support for sampling taken branches . (February 10, 2012)

• Jiri Olsa: ftrace, perf: Adding support to use function trace . (February 15, 2012)

Device drivers

• Mauro Carvalho Chehab: Hardware Events Report Mecanism (HERM) . (February 10, 2012)

• Alan Cox: SEP resynchronize . (February 10, 2012)

• MyungJoo Ham: Introduce External Connector Class (extcon) . (February 10, 2012)

• Jon Brenner: TAOS tsl2x7x . (February 10, 2012)

• Oskar Schirmer: add support for mc13892 I2C based touch panel for mx35_3ds . (February 14, 2012)

• Chris Boot: firewire-sbp-target: FireWire SBP-2 SCSI target . (February 15, 2012)

Filesystems and block I/O

• Joel Reardon: Adding Secure Deletion to UBIFS . (February 9, 2012)

• Jan Kara: Generic O_SYNC AIO DIO handling . (February 10, 2012)

• Andrea Righi: [PATCH v5 0/3] fadvise: support POSIX_FADV_NOREUSE . (February 13, 2012)

• Vivek Goyal: block: online resize of disk partitions . (February 14, 2012)

• Evgeniy Polyakov: pohmelfs: call for inclusion . (February 15, 2012)

• Richard Weinberger: UBI checkpointing support . (February 15, 2012)

Memory management

• Xi Wang: slab: introduce knalloc/kxnalloc . (February 9, 2012)

• Cong Wang: highmem: remove the second argument of k[un]map_atomic() . (February 10, 2012)

• Marek Szyprowski: Contiguous Memory Allocator . (February 10, 2012)

• Wu Fengguang: readahead stats/tracing, backwards prefetching and more (v5) . (February 13, 2012)

• KAMEZAWA Hiroyuki: memcg: page cgroup diet . (February 14, 2012)

• Dan Magenheimer: staging: ramster: multi-machine memory capacity management . (February 15, 2012)

• Aneesh Kumar K.V: hugetlbfs: Add cgroup resource controller for hugetlbfs . (February 15, 2012)

Architecture-specific

• Peter De Schrijver: Add tegra30 support for secondary cores . (February 10, 2012)

• Marek Szyprowski: ARM: DMA-mapping framework redesign . (February 15, 2012)

Security-related

• Will Drewry: user-supplied seccomp filtering using BPF . (February 13, 2012)

Virtualization and containers

• Tang Liang: xen: patches for supporting efi . (February 9, 2012)

• Andrew Stiegmann (stieg): RFC: VMCI for Linux . (February 15, 2012)

Miscellaneous

• Etienne Lorrain: Gujin GPL bootloader version 2.8.5 . (February 9, 2012)

Page editor: Jonathan Corbet

Distributions

A chat with new Fedora project leader Robyn Bergeron

Robyn Bergeron, the new Fedora project leader (FPL), described herself on the OpenStack wiki as an "all-around untechnical person." Given her background, the description is too modest, but it does emphasize that she brings to her new position perspectives that are different from that of her predecessors — in particular, that of a industry analyst. Although Bergeron studied economics in college, her first job combined duties on a help desk with part-time system administration. She later became a business analyst at Intel, focusing on the embedded chip vertical markets.

For several years, she was a full-time mother, but "I started missing my technical roots," she said. "I started getting involved with various open-sourcey things: I did the editing for papers for the Linux Symposium, and then I sort of stumbled into Fedora Marketing, mid-to-late 2009. From there, I sort of steadily progressed." Hired in November 2010 by Red Hat as Fedora Program Manager to oversee the features of each release, she has also been the Fedora Marketing team lead, and a facilitator for the Fedora Cloud SIG.

With this background, her appointment by Red Hat was "not a shock," she said, to either her or the Fedora community. Referring to Jared Smith, the previous FPL, she added:

Setting technical priorities and directions

Staying true to her analyst perspective, Bergeron said, "One of the first things I'd like to do is get a handle on statistics. If you don't have some specific statistical data that you know is valid, it's very hard to choose a direction in which to go." Asked to elaborate on the statistics that interest her, Bergeron suggested:

In addition, she would like to better track the personal goals of members of the Fedora Board and other active project participants. That is, she would like project members to be able to answer questions like: "What are the milestones you need to get there? What are the data points that you want to hit so we're being successful? I think people feel better when they can concretely see that they're meeting milestones and progressing somewhere."

This kind of tracking, Bergeron suggested, is becoming increasingly important as Fedora moves away from being focused on the desktop and server markets and starts to enter the mobile markets. "A lot of [my role] is finding what the big interests are people have in Fedora," she said. "I mean, what do they want to do with it?"

Despite this caution, Bergeron has already identified some of the directions in which she would like to see Fedora develop. Although she describes Fedora's support for the ARM platform as "one of the areas where I'm still learning a lot," she already recognizes it as an area where the Fedora community would like to see more activity. "I know for sure, particularly based on the number of people who sat in on some of the sessions at FUDCon that we have a tremendous lot of people who are extremely, extremely interested in the ARM platform who are diligently getting it ready to work, and hoping that everything will be set for Fedora 17 to have an ARM release."

Another priority is support for cloud computing. In Fedora 17, Bergeron said:

In addition, Bergeron remains interested in making Btrfs the default filesystem for Fedora. However, contrary to previous announcements, that change will not happen in Fedora 17. According to the ticket in the Fedora bug-tracking system, neither Anaconda, the Fedora installer, or Kickstart, the automatic installer, supports manual repartitioning with Btrfs, and with Anaconda scheduled for a rewrite for Fedora 18, a change of plans seems only logical. "It was a question of whether we could support it in the way we would need to if it's going to be the default filesystem," Bergeron explained.

In general, Bergeron professed to be generally satisfied with Fedora's state. Although complaints about overall quality erupt periodically on the user mailing list, Bergeron pointed out Fedora's well-defined release criteria and process and active QA team as proof that Fedora's quality remains high.

Commenting on the perception that Fedora's quality is compromised by its tradition of introducing new features, Bergeron said:

Setting policy

Inevitably, some of the media coverage of Bergeron's appointment highlighted that she was the first woman to be FPL. Similarly, comments on the reporting often focused on whether this milestone mattered. However, Bergeron herself is non-committal on the topic:

Asked if she has any plans to increase women's participation in Fedora, her reply was, "I have plans to get more contributors involved" — her emphasis suggesting yet again that she is not much interested in the subject.

By contrast, Bergeron sounded far more enthusiastic about using Fedora Ambassadors to increase contributors in general. She suggested continuing the practice of targeting regions for attention by Ambassadors. She also suggested specifically targeting potential contributors with design and programming skills, and taking another look at how easy joining Fedora really is, and how newcomers can become active more quickly.

Still another policy Bergeron plans to put in place is to start the process of preparing potential candidates for her eventual replacement. "Part of being a leader is thinking of who might follow you from Day One," she said. "It's part of leadership, picking out people you think might be good and encouraging them to step up and take new tasks, and seeing how they might perform in those areas."

However, one area that Bergeron has no plans for changes is Red Hat's appointment of the FPL, which is done with little input from the Fedora board or community. Outsiders often criticize the lack of community control and input in the appointment of the FPL, some suggesting that the process proves that Fedora is not a true community distribution, but still controlled by Red Hat. But when the question came up, Bergeron said simply, "No one in the Fedora community has come out screaming that it's the wrong way to do it. I think it's always easy from the outside to say that's the wrong way, but so long as the community is comfortable with it, that's an okay thing to do."

Settling in

When LWN talked to Bergeron, she was obviously still settling into her new role. However, she characterized this period of transition as a time to evaluate:

As though to emphasize her whimsy, Bergeron concluded the conversation with, "I'm happy to be here. And I love hot dogs." She is referring, of course, to Fedora 17's code name, Beefy Miracle, but the comment hints that whimsy might, as she promised, be yet another aspect of her leadership. At the same time, the comment leaves no doubt of her familiarity with the distribution she is now leading.

Comments (3 posted)

Brief items

Distribution quotes of the week

Comments (2 posted)

End of Life for Debian GNU/Linux 5.0

Comments (none posted)

NetBSD 5.1.2 released

Comments (none posted)

Scientific Linux 4.x End Of Life Notice

Full Story (comments: none)

Newsletters and articles of interest

Distribution newsletters

• DistroWatch Weekly, Issue 443 (February 13)
• Maemo Weekly News (February 13)
• Ubuntu Weekly Newsletter, Issue 252 (February 12)

Comments (none posted)

CrunchBang 10 update split into two editions (The H)

Comments (none posted)

Hall: A new way to Jam

Comments (none posted)

Shuttleworth: Remixing Ubuntu for the Enterprise Desktop

Comments (none posted)

Celebrating SUSE’s 20 Years Of Linux Love (TechWeek Europe)

Comments (none posted)

Page editor: Rebecca Sobol

Development

Radio station management with Airtime

Audio broadcasters gained a valuable tool in late January with the 2.0 release of Airtime, a Linux-based radio station management suite. Although it started off targeting independent over-the-air news stations, Airtime has developed into a tool that can handle recording and programming streams delivered over the Internet, too. The new release adds more integration with online services of interest to broadcasters, as well as streamlining the configuration and workflow of running a broadcast service.

What is Airtime?

Airtime is developed by Sourcefabric, a Prague-based non-profit that produces free software tools for independent media. Its other projects include news-oriented content management and publishing systems, and a recently launched book publishing platform. The group started in 1998 as the electronic media wing of the Media Development Loan Fund, a charity chartered to support "independent news outlets in countries with a history of media oppression," before spinning off on its own in 2010.

Sourcefabric's origins have impacted its software projects in several key ways, however. For example, all of the products are built with internationalization and multilingual-output (often including simultaneous publication in several languages) from the start, which is not true of every CMS. But with Airtime, the original mandate — to develop station management software for radio broadcasters under oppression — resulted in a web-based design that allows users to configure and program broadcasting equipment from remote locations, even across national borders.

In practice, Airtime generates audio streams using the Liquidsoap stream-mixing framework, and can output the results simultaneously to Icecast or Shoutcast servers (as MP3 and/or Vorbis) or through a sound card connected to traditional over-the-air broadcasting equipment. It supports several audio source types, including a library of recorded files (which are indexed by PostgreSQL and may be stored in a remote location), live studio audio that is captured by Ecasound, and "playout" content mixed in a variety of ways by the Mixxx DJing application. The audio features include fade-in/fade-out, cue points, and automatic cutoff when a program reaches its scheduled end.

The station management features include a calendar-like system for programming shows, ads, and other audio, complete with user-configurable recurrence and re-broadcast settings. The programming interface allows drag-and-drop reordering of the schedule and of tracks within a given program, and full metadata searching through the station library. Individual user accounts can be configured to give producers and staff access to program only their own shows, and administrators can monitor station status status and resources remotely. The media management component automatically imports and indexes new content, so staff can pre-record shows and drop them into the central storage location, where Airtime will pick them up for later playback.

Although Airtime itself is a management interface, the project also includes jQuery widgets that stations can add to their public-facing web sites to display information about the current on-air content and schedule information for the future.

Airtime is Linux-only, and while the project only makes official, packaged releases for Debian and Ubuntu, users of other distributions can download the source code and should be able to install it using the bundled installation scripts. In addition to Liquidsoap, Ecasound, and PostgreSQL mentioned above, Airtime depends on a variety of sound packages, although nothing too out-of-the-ordinary (assuming you do not count Liquidsoap's OCaml underpinnings as extraordinary). The code itself is written mostly in PHP, and is designed to work with Apache.

New in 2

The last major Airtime release (1.9) was in August of 2011. The project has been adding new features at a measured pace, so Airtime 2.0 is not a radical departure. However, it does implement some changes that can affect users and station administrators.

The most prominent is the ability to deliver output to up to three separate Icecast/Shoutcast streams simultaneously, potentially on three separate streaming servers. This allows stations to offer multiple bitrates, or to provide separate MP3 and Vorbis streams for listeners who prefer one or the other (although the limitation to three streams seems to be an arbitrary decision). A related feature is the ability to automatically upload clips or full programs to a station's SoundCloud account for non-live access from directly within the Airtime interface. Bulk uploads are supported, as are re-uploads, and complete control over the metadata and sharing settings of files.

A few lower-level improvements are potentially important to server administrators. 2.0 now supports PulseAudio as a sound server (in addition to ALSA, OSS, libao, and PortAudio), thus removing the need for administrators to manually uninstall PulseAudio, which was an increasingly tricky proposition once PulseAudio became the default sound server in common distributions.

The "media monitor" has also been enhanced in the new release. It is the component responsible for watching designated storage directories for new content and indexing added files. Previous releases failed to handle some common scenarios, such as renaming directories, and could occasionally attempt to index a new file while it was still being copied into storage, thus corrupting the index data. There is new error-checking to prevent to users from inadvertently editing the same program simultaneously, which could also result in undefined behavior.

Many more of the improvements in the new release fall under the banner of simplifying station management through the web front-end. This is in keeping with the project's stated focus to "reduce the time and effort station managers and DJs spent on behind-the-scenes work," and most of the changes are said to have originated in feedback from existing Airtime users.

The changes include a simpler interface to the playlist builder, a more configurable calendar/schedule editor that remembers a user's view settings between visits, and the ability to "live preview" a program directly from the browser (in previous releases, previewing a program from the web required using an auxiliary application such as VLC). There are some security-hardening fixes as well (such as input validation and a reCAPTCHA option that is triggered when there is a series of failed login attempts), and more settings have been exposed in the web frontend that previously required editing .htaccess or php.ini. Finally, there are new widgets in the administration interface to monitor disk space usage and service status, and theme-able HTTP error pages.

Stay tuned

Sourcefabric lists an assortment of real radio stations using Airtime on its "Who's Using" page; they are scattered around Europe, Africa, and Asia. Some are commercial stations, others are community broadcasters or non-profits. Unfortunately, if you do not happen to live near one of the FM transmitters, your only option for listening to Airtime programming "live" is through one of the Icecast/Shoutcast feeds.

Anybody contemplating their own Airtime deployment has one other new option, though. Alongside the debut of the 2.0 code, Sourcefabric announced the availability of "Airtime Pro," a hosting service that offers Airtime in a variety of month-to-month plans. Naturally, this service is geared for Internet streaming and not AM or FM broadcasting, but for many users that may suffice.

Sourcefabric has offered customization services and support packages to commercial users of its products in years past; now that it is an independent organization it is presumably responsible for finding sustainable revenue streams. Given the project's commitment to "free as in freedom", hosted services are preferable to offering the software as open-core or another business model.

The project has an extensive roadmap for Airtime, hinting at more features and eventually a REST API. Airtime may only account for a fraction of a percent of the all the active radio stations on the globe, but it is a field currently dominated by expensive, proprietary software solutions — which, if nothing else, means the project has huge potential for growth.

Comments (3 posted)

Brief items

Quotes of the week

Comments (10 posted)

LibreOffice 3.5 released

Full Story (comments: 26)

PyPy 1.8 released

Full Story (comments: none)

Wayland and Weston 0.85.0 released

Full Story (comments: 26)

Newsletters and articles

Development newsletters from the last week

• Caml Weekly News (February 14)
• LibreOffice Development Summary (February 13)
• Perl Weekly (February 13)
• PostgreSQL Weekly News (February 12)

Comments (1 posted)

The Chromium Blog on the future of JavaScript

Comments (14 posted)

Day: A New Approach to GNOME Application Design

Comments (159 posted)

Wayland - Beyond X (The H)

Comments (434 posted)

Page editor: Jonathan Corbet

Announcements

Brief items

Booktype, an open source platform to write and publish print and digital books

Full Story (comments: 5)

Articles of interest

Patent troll claims ownership of interactive Web—and might win (ars technica)

Comments (5 posted)

Jury rules that Eolas's "interactive web" patent is invalid (ars technica)

Comments (57 posted)

FSFE: Nortel/Rockstar and Google/Motorola deals

Full Story (comments: 4)

Welte: Some comments on the heated debate on SFC / Busybox / Linux GPL enforcement

Comments (36 posted)

New Books

Practical Malware Analysis--New from No Starch Press

Full Story (comments: none)

Upcoming Events

Events: February 16, 2012 to April 16, 2012

If your event does not appear here, please tell us about it.

Page editor: Rebecca Sobol