LWN.net Weekly Edition for September 22, 2011

Non-modular X?

It has been nearly six years since the release of X11R7.0, the first major release done by the X.Org Foundation. Among the many changes in this release was the change to a new modular architecture that allowed individual graphics drivers to be built and distributed independently from the server itself. Separating out the drivers was seen as a way to make it easier to contribute to their development and to get support for new hardware out to users as quickly as possible. By all accounts, X.Org has been a success, and the modular server architecture would seem to have worked out well.

So it is interesting to see a surge of new discussion of the architecture of our graphics subsystems at all levels. At the kernel level, developers are rethinking the relationships between graphics subsystems; that discussion will be covered in a separate article. At the X.Org level, instead, developers are considering undoing one of the headline changes from X11R7.0 and pulling graphics drivers back into the server itself.

The topic was evidently discussed at the recently-concluded X.Org Developer Conference; Jesse Barnes then brought it to the mailing list with the goal of discussing the good and bad aspects of returning to a monolithic server. At the top of his list of "pros" was that it would make it easier to make API changes in the server and push them into drivers. The kernel benefits from the ability to make internal API changes quickly; X could also make use of this flexibility. Being able to effect API changes immediately would also enable the removal of a bunch of backward-compatibility code intended to make it easier to mix driver and server versions.

The down side, of course, is that life gets harder for developers maintaining out-of-tree drivers - though it must be said that not everybody saw that as a disadvantage. Perhaps a monolithic server will encourage more drivers to move into the X.Org repository. But there is an interesting twist: some drivers, like the Wacom input driver, are licensed under the GPL. X, of course, is under the MIT license; adding GPL-licensed code into the mix would raise some awkward questions. What that means, in reality, is that some drivers will remain outside of the X server repository regardless of what happens to the rest.

Alex Deucher made the claim that API changes in the X server have been decreasing in frequency for some time, so the ability to make them more easily may not be all that valuable. The movement of much of the driver code into the kernel may have caused a reduction in disruptive driver changes in the X.Org server. But it may also be that, as Keith Packard said: "We don't get ABI changes because they're nearly impossible to handle." If those changes could be made more easily, chances are that the server would see more of them. Dave Airlie noted that he has a scheme in the works to make major changes to the driver API, but that he does not see how it can be done with the current, modular model.

How merging the drivers would affect testing is not entirely clear. On the one hand, testers working with new drivers would be building a new server as well, improving testing of the whole system. On the other, testing just a new driver without moving to a new server as well would become harder with the effect that some users are likely to just not bother. So the amount of driver testing could decrease. Nobody really knows how that would settle out, though, without actually trying the experiment.

Distributors could end up working harder to backport drivers to older servers, but the level of concern expressed by distributors in this discussion has been relatively low. As Ubuntu maintainer Bryce Harrington put it: "I favor just doing whatever the upstream developers feel is best for their own needs."

A different concern expressed by driver developers has to do with the core X.Org development process. The rules say that all patches must be reviewed before they can be merged into the master branch. Like all other projects, X.Org is short of reviewer time, so getting the requisite Reviewed-by tags for obscure changes can be a long and frustrating process. Driver code is typically understood by - and reviewed by - fewer developers than core code; that suggests that making changes to drivers in a monolithic server could be hard. The project might have to make some process changes in response; perhaps, as is the case in the kernel, the bar for driver changes could be set a little lower than for changes to the core code.

From your editor's reading of the discussion, it seems that there is a stronger sentiment in favor of making this change than there is against it. Those in favor see a way of making the development process more efficient and more closely integrating all the pieces of the X server. Those who have expressed opposition are mostly concerned about peripheral issues: licensing, review process, etc. Even the developer who is arguably the strongest opponent of the change - Luc Verhaegen - has argued that the real concerns are elsewhere. A better "mindset" when it comes to the design of the core API, he said, would have more far-ranging effects than the organization of the source tree.

As of this writing, no decision has been made public. It is not even 100% clear how such a decision can be made in the X.Org environment, which lacks a dictator (benevolent or otherwise) with the power to impose or block a change. Any project wanting to be successful in the long term must occasionally examine and tweak its institutions and processes. X.Org has already achieved the "long term," but not without some significant changes on the way. Relative to those changes, a decision to pull the drivers back in - or not to do so - seems relatively insignificant. The project, its processes, and its code have all improved considerably over the course of the last six years or so; that trend seems likely to continue into the future either way.

Comments (15 posted)

PostgreSQL and the SQL standards process

Open source and free software projects often encounter culture clash whenever they have to work with standards bodies. The most obvious problem is the secrecy that many proprietary-vendor-driven standards processes demand of participants, but that is not the only challenge. The PostgreSQL database project has been grappling with these challenges in recent weeks in an effort to strike a balance between its needs as a project and the closed structures and process of the ISO, which is the publisher of the official standard for SQL.

Secrecy

The topic arose on the pgsql-hackers mailing list in mid-September, when Susanne Ebrecht lamented the apparent lack of interest in the SQL standards process among PostgreSQL developers, prompted by her experience having a conference talk proposal on the subject rejected. She noted that another ISO meeting was fast approaching, and although rules prevented her from disclosing new drafts of the standard to "the public," she was permitted to discuss them privately with the organization that supported her (PostgreSQL), and asked if there was sufficient interest to set up a private mailing list for such discussions.

It apparently came as a surprise to several on the list that Ebrecht was an official representative in the ISO process. However, as she elaborated to the list, her role is not a direct (or a particularly powerful) one. The ISO has managed the SQL standard since 1987, as ISO/IEC 9075. But the ISO itself is composed of representatives — one per country — from 162 separate national standards bodies. The German standards body Deutsches Institut für Normung (DIN) solicited Ebrecht's input for their own work on SQL.

The final voting on changes to the ISO standard for SQL is done by the assembled national representatives, however. Thus, even though Ebrecht can present PostgreSQL's concerns to the DIN SQL committee, they are still several steps removed from making it into the eventual standard — steps where the vested interests of corporations and other nations gain more and more influence on the outcome. The real practical question posed to PostgreSQL is how Ebrecht could communicate about the process to the developers without running afoul of the committees' secrecy rules.

It might be possible to avoid violating the non-disclosure rule by discussing broad changes to the drafts on a public mailing list without going into detail. But in SQL as in so much of life, the devil is in the details, so the consensus eventually was that a private list would be set up, to which Ebrecht could forward updates from the standards-writing process. To keep the list traffic confidential, it would be limited to known PostgreSQL contributors.

Standards: who needs 'em?

On the plus side, there does seem to be a healthy interest among project members in following the ISO standards process. As Heikki Linnakangas said, the process may not have sparked much discussion over the years, but "it's hard to get excited about something if you don't know what's happening." As core team member Josh Berkus said in an email, though, the non-disclosure rules are just one of several challenges.

On the third point, Berkus offered a specific example where influential vendors appear to have used the standards process as a weapon. Both PostgreSQL and MySQL supported a simple syntax for the retrieval of a subset of the rows returned by a query using the LIMIT and OFFSET operators, he said, syntax which was well-understood and well-liked by users. But the standards committee adopted a different syntax that was more verbose, but which added no additional features or flexibility. He said:

Open source projects are not the only players put at a disadvantage by this sort of tactic, either, he observed. The same hurdles affect startup companies, to the protection of entrenched players against competition.

Distrust of the ISO process was visible from others in the project as well. PostgreSQL's resident standards guru Peter Eisentraut commented in an off-list email that, for end users, SQL is "pretty useless as a 'standard'" when compared to more complete specifications like C and XML. SQL lacks specifications for important features like optimization and administration, he said, and worse still, the language itself is "baroque," with every new feature adopting a completely new syntax. As a result, there is no clear way to extend the language in a consistent fashion, which is problematic for PostgreSQL and other projects.

Open source, proprietary vendors, and incompatibility

Joe Abbate mused that perhaps it was time for the open source database players to establish their own standard not controlled by incumbent vendors out to protect their business. Abbate's initial message to that effect came across as a call to form an "open source fork" of SQL itself, which most of the PostgreSQL team seemed to think was a bad idea. In addition to the confusion it would create for users, attempting a fork would require tremendous time and energy — and as Greg Smith commented, "standardization tends to attract lots of paperwork. Last thing you want to be competing with a big company on is doing that sort of big company work."

On the other hand, some, like Christopher Browne, pointed out that open source projects should consider participating in new standards processes that are just beginning, such as the UnQL specification proposed for NoSQL database queries. Darren Duncan suggested much the same thing with respect to the Muldis D language. Abbate clarified his intention in a follow-up message, saying he did not mean to propose embarking on a standards-fork. "I only think it may be useful to discuss SQL features, informally or otherwise, with other open source 'competitors' such as SQLite, MySQL (brethren), Firebird, etc.."

With regard to Abbate's idea, Berkus affirmed the value of communication between the various open source database projects, noting that they already meet annually at OpenSQL Camp. But there are essentially only three open source relational databases that matter, he said: PostgreSQL, MySQL, and SQLite. Among those, MySQL is now split into several competing fragments, the largest of which is owned by Oracle. As a result, cross-project communication boils down to PostgreSQL concurring with SQLite, he said, "which we already mostly do."

Realistically, though, Berkus does not feel that SQL users are demanding more features and syntax:

In the broader open source community, then, relational databases may have it easy because SQL is old enough that it is both well known and established (not to mention the fact that most users are resigned to incompatibility between competing vendors). Other software projects are not so lucky, from patent-driven fights about video codecs in HTML5 to supporting new hardware specification in the Linux kernel. The roadblocks Berkus mentioned are problematic no matter what the standard. Large projects or well-funded organizations may be fortunate enough to get a representative into the process (as PostgreSQL has), but a closed process dominated by proprietary vendors cannot be reformed in a day.

Comments (55 posted)

Managing GNOME shell extensions

The GNOME project is currently readying its 3.2 release, the first update to the re-vamped infrastructure and environment introduced in April 2011. Although many minor enhancements and changes are slated to roll out with 3.2, the one with the greatest potential to affect end users is the extensions mechanism for the GNOME Shell desktop interface.

Background

When 3.0 was released, a large slice of the negative criticism it received centered around the difficulty of customizing GNOME Shell when compared to the GNOME 2.x panel and menu system. The placement and orientation of GNOME Shell's interface elements was fixed; fonts, key bindings, and icons could not be changed; popular informational applets and controls were missing (and there was no facility for bringing them back); there were no UI or window-manager themes, et cetera. A stopgap measure called Gnome Tweak Tool appeared later that restored user control over a number of basic settings, but only for a fixed set of options.

In the meantime, although the GNOME marketing crew maintained that the new distraction-free environment of GNOME Shell would eventually win over the hearts and minds of its critics, when pressed about specific issues GNOME developers often referred users with concerns to the forthcoming extensions mechanism that would make every aspect of GNOME Shell mutable and scriptable with JavaScript and CSS. To those (like myself) with specific UI nits to pick, it sounded like a dream come true, albeit one to arrive at an unspecified point in the future.

In the months since, that extension system has slowly begun to take shape. Initially, individual developers would announce extensions on their personal blogs, which were periodically rounded-up on third-party discussion-and-review blogs. That process made locating them difficult, and knowing which ones to trust dicey. An official collection is now hosted in the GNOME Git repository, currently consisting of nine extensions, but it became clear in recent months that a real extension infrastructure would be needed — to handle hosting public extensions, validating and reviewing code, and providing users with a simple way to install and uninstall their selections from within GNOME.

The foot with a sweet tooth

Sweet Tooth is the codename for the new GNOME Shell extension infrastructure project. The user-facing front end of the system is a planned extension-hosting web site à la Mozilla's addons.mozilla.org, at which visitors can search for and download extensions. The URL for the site is variously reported as extensions.gnome.org or extensions.gnome3.org, although neither is active yet. There will be (at least) two substantive differences between the GNOME extension site and Mozilla's Add-Ons repository, however.

First, GNOME Shell extensions will be bundles of JavaScript and CSS that, when executed, alter the GNOME Shell environment itself — thus, in order to avoid forcing the user to shut down his or her session entirely and restart GNOME, they take effect immediately once downloaded and enabled. These factors raise the undesirable possibility of malicious extensions being delivered from any site that could then seize control of the local machine. While extensions would not run with root privileges, they would still have access to potentially sensitive user data or contain other types of malware payloads.

While one possible solution would be to limit the installation of extensions to a specialized application, the current scheme is to access to the extension site with any modern browser, and use other measures to detect and block malware. A sticking point in this approach is how to permit the site's web application to safely learn which Shell extensions are currently installed (and at which version numbers) in order to present the correct options to the use in the UI (i.e., "Install" versus "Uninstall / Upgrade"); it is also necessary to create a mechanism to actually install extensions from the browser.

To provide those capabilities, some sort of go-between is required, perhaps a local process that can speak HTTP to the browser, although there are of course inherent security risks to running a local server that responds to queries about the local filesystem. On the extension site's side of the equation, the plan is to implement a code review policy with cryptographically-signed uploads of each extension. Reviewers will only be responsible for checking new extensions for malicious behavior, not grading them on importance, functionality, or style.

The GNOME Shell discussion list has been debating several approaches to the extension-signing and review-process pieces of the puzzle. Owen Taylor's preferred plan involves two signatures: one from each reviewer, and a separate one from the site — although he noted that the manual steps could constitute a weak spot.

Extension security is an issue, but as was noted elsewhere in the thread, it is not a greater security risk than that of running any other desktop application.

The other distinction between the GNOME Shell extension service and Mozilla's is that essentially any aspect of the GNOME environment is fair game for extensions, and is configurable through JavaScript thanks to GNOME 3's pervasive GObject introspection. Firefox and Thunderbird extensions could alter fundamental application behavior, but most do not — they make small changes to tie in a specific new feature or service, or alter a particular behavior.

As the example extensions linked to above show, however, a fair number of GNOME Shell extension authors appear interested in substantially changing the desktop experience. The GNOME Shell team seems to be taking a hands-off approach (noting on the Sweet Tooth wiki page that the project will not endorse or support third-party extensions).

Hopefully making that policy prominent on the extensions site will appease the camp that worries about customization diluting the "GNOME brand," but it does leave open the possibility of mutually-conflicting extensions. So far there appear to be no safeguards in place, although there was some discussion about an SELinux-like permission system. Keeping track of permission sets is primarily a security policy tool, but would also assist in managing collection of extensions.

Right now, when a user downloads an extension, it is unpacked into the $HOME/.local/share/gnome-shell/extensions/ directory, with one subdirectory per extension. The subdirectory name should conform to an email-address-like syntax of the form extensionname@yourdomain.com. The Looking Glass tool is GNOME Shell's JavaScript inspector and debugger, and it can be used to investigate installed extensions (and directly evaluate user-provided JavaScript code).

The developer angle

Looking Glass offers a way for aspiring extension writers to explore the GNOME Shell environment. One can select items in GNOME Shell with the mouse and copy the underlying GObject structure to the Looking Glass evaluator, and there is a special function to slow down animations for easier debugging. But it still is not complete enough to serve as a full development environment.

A bigger issue is that GNOME Shell still lacks comprehensive documentation of its structure, methods, and conventions — despite the fact that such documentation is part of the official roadmap. The extension system is potentially powerful, but the only way for outsiders to learn how to write extensions is to hunt for tutorials and examples on individual blogs. Some of them are quite good, such as Finnbarr Murphy's. But without a real effort to maintain official documentation, they rapidly fall out of date. Providing add-on developers with adequate resources is an area where Mozilla excels with its Mozilla Developer Network sites; GNOME will need to play catch-up in order to grow a healthy GNOME Shell extension community.

Soft landing

Now that the freeze for 3.2 is upon us, and the extensions site is still not up and running, it appears that the framework will be relegated to a "soft-launch" in the 3.2 cycle. Taylor described it as "a bit of a stealth-beta for this release ... something we're still working on, something we don't advertise as a release feature, but something that you can already use."

Perhaps that is for the best. Although the extensibility of GNOME Shell is promising (perusing the extensions already written is quite an experience), the human side of the framework still needs work. One only needs to take a look at the public response to GNOME 3.0 to see how poor messaging can undermine a technical success.

The 3.0 public relations and marketing blitz completely failed to communicate to users and developers that an extension mechanism was in the works, or even a possibility for the future — it got no mention in the release notes, the talking points, or even the design documents. GNOME developers have been talking about GNOME Shell extensions in the run-up to 3.2, but with API documentation and developer outreach still missing in action, the risk is that yet another major release will pass with the project missing an chance to attract coders and strengthen its software ecosystem.

Comments (30 posted)

Page editor: Jonathan Corbet

Security

LSS: The kernel hardening roundtable

Hardening the kernel to make attackers' jobs harder was the topic of a wide-ranging discussion at the Linux Security Summit (LSS) held on September 8, 2011. Reducing the attack surface of the kernel, protecting it from user-space attacks, and finding ways to mitigate entire classes of exploitable bugs were all on the table. As might be expected, the biggest barrier to getting these hardening patches accepted into the mainline is often performance concerns. While no firm conclusions were drawn, many ideas were discussed, some of which may eventually find their way into the mainline.

Attack surface

The discussion began with an effort to quantify the "exposed surface" of the kernel as roundtable leader Will Drewry of Google's Chrome OS team put it. He and the other roundtable leader, Kees Cook of the Ubuntu security team, put together their own list, but also asked those present to add to it. Obvious attack surfaces like the system call interface, /proc and sysfs files, the networking stack, and device drivers were mentioned, but also less obvious things like filesystem parsing, auto-loaded kernel modules, device scanning, CPU or other hardware bugs, side-channel timing attacks, and so on.

Enumerating the attack surface "helps define what to pay attention to", Cook said. The intent of many of the kernel hardening patches is to try "to kill off a whole class of problems, rather than shooting individual bugs", he said. The latter is where most of the current kernel security effort goes, he said. Drewry added that the intent is to figure out what can be done now to reduce those attack surfaces. Many of the attack surfaces are still present even in a system that runs a mandatory access control (MAC) system like SELinux, Cook said, because the system call interface is still available to be used (and abused). That is one of the problems with looking to the LSM interface to provide confinement, he added.

Casey Schaufler also pointed out that there is often special-purpose hardware in Linux systems—in years past it was graphics hardware, but these days tends to be video hardware—that is allowed to be directly accessed from user space. That opens up a number of potential security problems, he said, but that won't stop it from happening. The capabilities provided by allowing direct access to these devices are "so compelling that security concerns are secondary".

But there are kernel installations that are more security-sensitive, Cook said, that could benefit from restricting some features even at the cost of performance. If a particular hardening feature has no real cost, it could be put into the kernel without providing a configuration option to disable it. Others, that do have a cost, could be optional and distributions or users could enable them based on their needs.

API/ABI restrictions

The "biggest single exposure" in Linux systems is applications that run as root, Schaufler said, like the X server. Because the kernel is one "gigantic privileged application" it can't be protected against other privileged applications like X, Cook said. But, applications could have the ABI available to them reduced, Drewry said, which would reduce the damage they could do if they are compromised.

The only existing "API management" tool in the kernel (besides the LSM interface) is seccomp, but it is too restrictive to be useful for many applications, Drewry said. Since seccomp only allows four system calls (read(), write(), exit(), and sigreturn()), it is too limited for many possible reduced-ABI applications. The Chrome/Chromium browser team would like to be able to reduce the system calls that its rendering processes can make. Seccomp is too limited for Chromium's needs, so they have implemented a more complicated solution, with a "trusted" assembly language thread that mediates system calls. System call restrictions could also be enforced using ptrace(), Drewry said, but there is an "intense amount of overhead".

What Drewry is looking for is some kind of expanded seccomp where a subset of system calls would be allowed. So far, his patches to implement that have been shot down from various directions, but there is hope that there may be some kind of resolution at the upcoming Kernel Summit.

Some of the attendees were skeptical of an expanded seccomp approach. Schaufler pointed out that there is already a mechanism in the kernel (capabilities) for reducing the impact of vulnerabilities, but "no one uses it". Cook was not convinced that the granularity of capabilities was really all that useful because the number of capability bits that are equivalent to root is so large.

As Drewry cast about for a way to limit system calls, there was discussion of possibly augmenting the LSM interface. As Cook pointed out, the current interface does not mediate all system calls, so it can't be used for Drewry's use case as it stands. James Morris noted that LSM is intended to be an access control framework and not anything more than that. In the end, Drewry doesn't particularly care how to get there, he is just looking for a way for "reducing what I expose to untrusted applications", he said.

Schaufler also pointed out that reducing the ABI available to an application doesn't help "if the ABI is completely well-defined and if it is consistent with the security policy" of the system. "That's a lot of 'if's", Drewry responded, to general agreement, that neither of the two conditions are met on Linux systems. Because the system call interface is not well-defined, nor necessarily consistent with the system security policy, reducing the exposure of parts of that interface can help. Schaufler cautioned that the ad hoc documentation makes it hard to decide where the bugs actually are: "If the code is the documentation, it is impossible to have a bug".

There were questions about whether seccomp filtering (in whatever form) would actually be used by applications. Cook noted that, in addition to Chromium, several other projects popped up on linux-kernel to express interest in the feature, including QEMU, vsftpd, and others. One attendee also hypothesized a DNS server that was limited to recvmsg(), sendmsg(), and write() (to a log file) as another possible use-case.

There were also concerns that seccomp filters would spread security policy throughout the system, but others saw that as a feature. Unlike MAC policy, which tends to be imposed from the outside, seccomp filter policy would embody "the programmer's idea of what it should be able to do", as Cook put it. While the system call granularity may not be exactly right, it is the place where user space enters the kernel, so mediating at that point makes some sense.

Attendees theorized that if a flexible seccomp filter facility was available, multiple applications would take advantage of it. Smalley was a bit skeptical that it would be straightforward for most applications to use the facility because it might require a major rework of the program. He pointed to the privilege separation efforts that went on in OpenSSH as an example. That required "significant refactoring", he said.

Drewry said that the Chromium team's plan is to move the browser to whatever solution becomes available to better contain the renderers. Right now, that is the "trusted thread" sandbox, but if there are other facilities available, Chromium will use them. That could be some kind of SELinux containment, seccomp filtering, or something else entirely. In the future, the team would also like to confine renderers based on where the data comes from, he said, so that all renderers running for a given site were protected from each other as well.

PaX and grsecurity

The roundtable wrapped up with some discussion of bringing more of the grsecurity and PaX hardening patches into the mainline. Those patches tend to be fairly intrusive and have performance implications that make them undesirable to many kernel hackers, but they do provide protections that some would find valuable. According to Cook, there are many pieces of grsecurity and PaX that could make their way into the mainline.

Simple things, like constifying function pointers, are essentially free and should be mainlined immediately: "It's a shame that hasn't been done long ago", one attendee said. Others that have more impact are trickier. Making them optional is one possibility, but even that has a cost that maintainers are likely to push back against. Adding another path through core kernel code can be a maintenance headache, and those may be difficult to get into the mainline.

Andre Hedrick mentioned that he has been pulling apart the grsecurity/PaX patches to try to make them more palatable. For one thing, grsecurity depends on a role-based access control (RBAC) mechanism that isn't present in the mainline (and isn't implemented as an LSM, so it isn't likely to ever be, at least in that form). Hedrick is trying to remove that dependency from the grsecurity features of interest, like better address-space layout randomization (ASLR) and a fully relocatable kernel, both of which can thwart various kinds of attacks.

One goal would be to find the grsecurity/PaX changes that have minimal impact and to get those into the mainline as non-optional protections. Turning RBAC into an LSM might be another useful exercise. grsecurity developer Brad Spengler provided a "long list" of features that could make their way into the kernel at last year's LSS, Cook said. That list would make a good starting point.

Cook also noted several other efforts aimed at hardening the kernel. Those include the work that Openwall hacker Vasiliy Kulikov has been doing, much of which is being discussed on the kernel-hardening mailing list. Also, the Ubuntu security team has been working on a kernel hardening project of its own. There is no lack of ideas out there, and a clear need to make the kernel more resistant to attacks. Based on the discussion, and the various ongoing efforts, we are likely to see more and more hardening patches aimed at the mainline over the next few years.

[ I'd like to thank LWN subscribers for supporting my travel to LSS. ]

Comments (12 posted)

Brief items

Quotes of the week

-- Microsoft makes .txt files dangerous

-- FEDORA-2011-12302 (Thanks to Rahul Sundaram; see also FEDORA-2011-10266 and FEDORA-2011-2890)

Comments (13 posted)

An alleged SSL/TLS protocol vulnerability

Here are articles in the Register and Threat Post on a new attack that, it is said, can extract cookies from SSL streams. Details are scarce, but it seems to be a man-in-the-middle attack that injects a bit of JavaScript into the victim's browser. That JavaScript can then take advantage of the fact that SSL connections are reused across page fetches to carry out a known-plaintext attack against that connection. TLS versions 1.1 and 1.2 are apparently not vulnerable, but, alas, nobody uses those versions. Those wanting to do some digging can learn a bit more from conversations on the TLS list and Hacker News.

Comments (26 posted)

Garrett: UEFI secure booting

Matthew Garrett has posted an article about the UEFI "secure boot" feature and its potential impact on Linux.

As he notes, it is not time to panic yet, but it is worth being concerned about. Those who are interested in learning more about Microsoft's plans may want to watch this video which describes them in detail.

Comments (85 posted)

New vulnerabilities

django: multiple vulnerabilities

Package(s):	django	CVE #(s):
Created:	September 19, 2011	Updated:	September 21, 2011
Description:	Django 1.2.6 and Django 1.3.1 fix several security issues, including session manipulation, a denial of service attack via URLField, URLField redirection, and host header cache poisoning. See the Django advisory for details.
Alerts:	Fedora FEDORA-2011-12493 2011-09-10

Comments (none posted)

ffmpeg: denial of service/code execution

Package(s):	ffmpeg	CVE #(s):	CVE-2011-1196 CVE-2011-2161 CVE-2011-3362
Created:	September 20, 2011	Updated:	August 30, 2012
Description:	From the Ubuntu advisory: It was discovered that FFmpeg incorrectly handled certain malformed ogg files. If a user were tricked into opening a crafted ogg file, an attacker could cause a denial of service via application crash, or possibly execute arbitrary code with the privileges of the user invoking the program. This issue only affected Ubuntu 10.10. (CVE-2011-1196) It was discovered that FFmpeg incorrectly handled certain malformed APE files. If a user were tricked into opening a crafted APE file, an attacker could cause a denial of service via application crash. (CVE-2011-2161) Emmanouel Kellinis discovered that FFmpeg incorrectly handled certain malformed CAVS files. If a user were tricked into opening a crafted CAVS file, an attacker could cause a denial of service via application crash, or possibly execute arbitrary code with the privileges of the user invoking the program. (CVE-2011-3362)
Alerts:	Debian DSA-2336-1 2011-11-07 Ubuntu USN-1209-2 2011-09-19 Ubuntu USN-1209-1 2011-09-19 Mandriva MDVSA-2012:074 2012-05-14 Mandriva MDVSA-2012:075 2012-05-15 Mandriva MDVSA-2012:076 2012-05-15 Mandriva MDVSA-2012:074-1 2012-08-30 Mandriva MDVSA-2012:148 2012-08-30

Comments (none posted)

kernel: denial of service

Package(s):	linux-ti-omap4 linux kernel	CVE #(s):	CVE-2011-1771
Created:	September 21, 2011	Updated:	September 21, 2011
Description:	The CIFS filesystem implementation does not properly handle direct I/O, allowing a local attacker with access to a CIFS filesystem to cause a kernel oops.
Alerts:	Ubuntu USN-1256-1 2011-11-09 Ubuntu USN-1212-1 2011-09-21

Comments (none posted)

httpd: denial of service

Package(s):	httpd	CVE #(s):	CVE-2011-3348
Created:	September 16, 2011	Updated:	October 20, 2011
Description:	From the Fedora advisory: mod_proxy_ajp when combined with mod_proxy_balancer: Prevents unrecognized HTTP methods from marking ajp: balancer members in an error state, avoiding denial of service.
Alerts:	Ubuntu USN-1259-1 2011-11-11 openSUSE openSUSE-SU-2011:1217-1 2011-11-04 SUSE SUSE-SU-2011:1215-1 2011-11-04 Mandriva MDVSA-2011:168 2011-11-09 Scientific Linux SL-http-20111020 2011-10-20 Red Hat RHSA-2011:1391-01 2011-10-20 Slackware SSA:2011-284-01 2011-10-17 Fedora FEDORA-2011-12715 2011-09-14 Gentoo 201206-25 2012-06-24

Comments (none posted)

mantis: multiple vulnerabilities

Package(s):	mantis	CVE #(s):	CVE-2011-2938 CVE-2011-3356
Created:	September 19, 2011	Updated:	November 9, 2012
Description:	Mantis 1.2.7 fixes cross site scripting and remote SQL injection vulnerabilities. See the Mantis bug report for details.
Alerts:	Fedora FEDORA-2011-12369 2011-09-09 Gentoo 201211-01 2012-11-08

Comments (none posted)

openttd: multiple vulnerabilities

Package(s):	openttd	CVE #(s):	CVE-2011-3341 CVE-2011-3342 CVE-2011-3343
Created:	September 20, 2011	Updated:	January 12, 2012
Description:	From the CVE entries: Multiple off-by-one errors in order_cmd.cpp in OpenTTD before 1.1.3 allow remote attackers to cause a denial of service (daemon crash) or possibly execute arbitrary code via a crafted CMD_INSERT_ORDER command. (CVE-2011-3341) Multiple buffer overflows in OpenTTD before 1.1.3 allow remote attackers to cause a denial of service (daemon crash) or possibly execute arbitrary code via vectors related to (1) NAME, (2) PLYR, (3) CHTS, or (4) AIPL (aka AI config) chunk loading from a savegame. (CVE-2011-3342) Multiple buffer overflows in OpenTTD before 1.1.3 allow local users to cause a denial of service (daemon crash) or possibly gain privileges via (1) a crafted BMP file with RLE compression or (2) crafted dimensions in a BMP file. (CVE-2011-3343)
Alerts:	Gentoo 201111-03 2011-11-11 Fedora FEDORA-2011-12975 2011-09-19 Debian DSA-2386-1 2012-01-11

Comments (none posted)

php: denial of service

Package(s):	php	CVE #(s):	CVE-2011-3182
Created:	September 19, 2011	Updated:	April 13, 2012
Description:	From the CVE entry: PHP before 5.3.7 does not properly check the return values of the malloc, calloc, and realloc library functions, which allows context-dependent attackers to cause a denial of service (NULL pointer dereference and application crash) or trigger a buffer overflow by leveraging the ability to provide an arbitrary value for a function argument, related to (1) ext/curl/interface.c, (2) ext/date/lib/parse_date.c, (3) ext/date/lib/parse_iso_intervals.c, (4) ext/date/lib/parse_tz.c, (5) ext/date/lib/timelib.c, (6) ext/pdo_odbc/pdo_odbc.c, (7) ext/reflection/php_reflection.c, (8) ext/soap/php_sdl.c, (9) ext/xmlrpc/libxmlrpc/base64.c, (10) TSRM/tsrm_win32.c, and (11) the strtotime function.
Alerts:	Mandriva MDVSA-2011:165 2011-11-03 Ubuntu USN-1231-1 2011-10-18 Gentoo 201110-06 2011-10-10 Fedora FEDORA-2011-11537 2011-08-26 Fedora FEDORA-2011-11528 2011-08-26 Fedora FEDORA-2011-11537 2011-08-26 Fedora FEDORA-2011-11528 2011-08-26 Fedora FEDORA-2011-11537 2011-08-26 Fedora FEDORA-2011-11528 2011-08-26 Debian DSA-2408-1 2012-02-13 SUSE SUSE-SU-2012:0496-1 2012-04-12 Mandriva MDVSA-2012:071 2012-05-10

Comments (none posted)

roundcube: cross-site scripting

Package(s):	roundcubemail roundcube	CVE #(s):
Created:	September 15, 2011	Updated:	September 15, 2011
Description:	The Roundcube web mail system suffers from cross-site scripting vulnerabilities in its user interface messages.
Alerts:	Fedora FEDORA-2011-12131 2011-09-06

Comments (none posted)

wireshark: denial of service

Package(s):	wireshark	CVE #(s):	CVE-2011-3266
Created:	September 19, 2011	Updated:	September 21, 2011
Description:	From the CVE entry: The proto_tree_add_item function in Wireshark 1.6.1, when the IKEv1 protocol dissector is used, allows user-assisted remote attackers to cause a denial of service (infinite loop) via vectors involving a malformed IKE packet and many items in a tree.
Alerts:	openSUSE openSUSE-SU-2011:1263-1 2011-11-21 SUSE SUSE-SU-2011:1262-1 2011-11-21 openSUSE openSUSE-SU-2011:1142-1 2011-10-18 Gentoo 201110-02 2011-10-09 Fedora FEDORA-2011-12423 2011-09-09 Fedora FEDORA-2011-12403 2011-09-09

Comments (none posted)

vsftpd: denial of service

Package(s):	vsftpd	CVE #(s):	CVE-2011-2189
Created:	September 19, 2011	Updated:	December 7, 2011
Description:	From the Debian advisory: Maksymilian Arciemowicz discovered that vsftpd is incorrectly handling certain glob expressions in STAT commands. This allows a remote authenticated attacker to conduct denial of service attacks (excessive CPU and process slot exhaustion) via crafted STAT commands.
Alerts:	Ubuntu USN-1288-1 2011-12-07 Debian DSA-2305-1 2011-09-19

Comments (none posted)

zabbix: remote information disclosure

Package(s):	zabbix	CVE #(s):	CVE-2011-3265
Created:	September 19, 2011	Updated:	September 21, 2011
Description:	From the CVE entry: popup.php in Zabbix before 1.8.7 allows remote attackers to read the contents of arbitrary database tables via a modified srctbl parameter.
Alerts:	Fedora FEDORA-2011-12489 2011-09-10 Fedora FEDORA-2011-12485 2011-09-10

Comments (none posted)

Page editor: Jonathan Corbet

Kernel development

Brief items

Kernel release status

The current development kernel remains 3.1-rc6; there have been no releases - development or stable - in the last week.

Comments (2 posted)

Quotes of the week

-- Neil Brown

-- John Watlington

-- Thomas Gleixner

Comments (1 posted)

linux-next on github

The linux-next tree has been unavailable since kernel.org went down; maintainer Stephen Rothwell had said previously that he was unable to post it at an alternative location. That problem has evidently been overcome; those wanting the current linux-next tree can find it on github.

Full Story (comments: 28)

Where's that tree?

As of this writing, the kernel.org outage continues with no word as to when the site will be back up. Kernel development never stops, though, and one of the advantages of the git model is that copies of repositories exist all over the place. A number of developers have announced new locations for their trees; some of the relocated repositories are:

ACPI	https://github.com/lenb/linux.git
ALSA SOC	git://opensource.wolfsonmicro.com/linux-2.6-asoc.git
amd64 EDAC	git://amd64.org/linux/bp.git
APM	git://twin.jikos.cz/jikos/apm
arm-soc	git://git.linaro.org/people/arnd/arm-soc.git
HID	git://twin.jikos.cz/jikos/hid
infiniband	https://github.com/rolandd/infiniband
input	https://github.com/dtor/input
kbuild	http://repo.or.cz/w/linux-kbuild.git
libata	git://github.com/jgarzik/libata-dev.git
mmc	git://dev.laptop.org/users/cjb/mmc mmc-next
pm	git://github.com/rjwysocki/linux-pm.git
regmap	git://opensource.wolfsonmicro.com/regmap.git
SCSI	git://bedivere.hansenpartnership.com/git/scsi-rc-fixes-2.6.git git://bedivere.hansenpartnership.com/git/scsi-misc-2.6.git
slab	git://github.com/penberg/linux.git
tip	git://tesla.tglx.de/git/linux-2.6-tip
trivial	git://twin.jikos.cz/jikos/trivial
wireless	git://git.infradead.org/users/linville/wireless.git git://git.infradead.org/users/linville/wireless-next.git git://git.infradead.org/users/linville/wireless-testing.git
xen	git://oss.oracle.com/git/kwilk/xen.git

Most of these relocations have been advertised as being temporary. It will be interesting to see how many of them move back to kernel.org on its return, especially if that site comes back with stricter rules about access.

Comments (12 posted)

Kernel development news

LPC: Control groups

Control groups remain a controversial topic in kernel circles; some developers like them, others hate them. The latter group would like to see the feature removed altogether, but that seems unlikely to happen; there are too many users for control groups already, with more to come. The 2011 Linux Plumbers Conference featured a discussion among those users that gave some insights into why control groups are useful and what could be done to make them more so.

The session started with a brief talk by Kir Kolyshkin of Parallels; for him, control groups are all about implementing containers. Containers can be seen as a sort of poor user's virtualization; it enables the running of multiple, isolated user-space systems all on the same kernel. Containers tend to be more efficient than pure virtualization; they are also, he said, the only form of virtualization available for the ARM architecture at the moment. Control groups help in the implementation of containers by isolating groups of processes from each other and by allowing the imposition of resource limits on each group.

The bulk of the session, though, centered around a presentation by Tim Hockin on Google's isolation and resource limitation needs. Google's cluster runs all kinds of jobs which, internally, are divided into "tier 1" and "tier 2" tasks. The general problem Google has is that tasks normally do not use 100% of the resources they request; that means that systems in the cluster tend to be underutilized. Google would like to be able to pack more jobs onto each box, but they have to be very careful about overcommitting resources. If that is not done carefully, resource-intensive jobs can get in the way of urgent tasks like responding to search queries.

Google uses its own form of containers to be able to overcommit systems safely. Containers let Google place limits on the CPU usage, memory usage, I/O bandwidth consumption, etc. of each group of processes on the system. The goal, when all goes well, is to isolate each group from the others, provide predictable resources to each, and to lose very little time on the container implementation itself. Control groups are used when they are available and suitable to the task; in other places, a lot of user-space control code is used instead. The user-space code is complex and racy, Tim said; they would like to be rid of it.

There is a special daemon running on each system that wakes up about every 100ms to have a look at what is going on. Should it detect a load spike originating from the system's tier-1 work, it will stop or kill any tier-2 tasks needed to make room. This all works, but it could work better; more support from the kernel would be helpful.

For example, memory use needs to be tightly controlled on these systems. At the moment, Google is using the "fake NUMA" feature to partition system memory and parcel it out as needed (see this article for a bit more information on how that works). Fake NUMA is a hack, though, with resource costs of its own. They are moving to the kernel's memory controller, but it is not yet suitable for their needs because it cannot work with nested control groups. They had similar problems with the disk bandwidth controller, but that problem has been resolved recently. In general, Tim said, anybody who is designing a controller for Linux should think about how it will nest from the beginning.

One other problem with the memory controller is its handling of shared memory. Currently shared pages are billed to the control group that touches it first. That makes deterministic resource control hard, especially in situations where the limits are set tightly. Tim didn't like the idea of proportional billing (dividing the charge for each page across each group that has it mapped) any better. That, he said, takes memory billing out of the control of each group; if one control group exits, the others will suddenly find themselves over their limits as their portion of the shared pages grows. What he would like would be the ability to manually arrange for pages backed by certain files to be billed to specific groups. Then he could set up a system group to be billed for, say, the C library.

There are some other problems as well. The memory overhead of the memory controller is painfully high, for example. Google would really like a way to query the size of the working set for each control group, but that capability is not currently there. They also really want per-control-group reclaim to focus the memory management code on the control groups that are currently exceeding their limits. And, if a container goes so far over its limits that the out-of-memory killer gets involved, it would be really nice to have a way to kill a whole control group at once instead of having to do it one process at a time. (It's worth noting that patches for many of these features exist; many of them come from Google).

Beyond that, there is a lot of interest in the I/O bandwidth controller. A lot of Google jobs, he said, are "seek locked"; controlling how much I/O bandwidth they use is important. Controllers for other types of resources (number of threads, number of open file descriptors, network ports, etc.) would be useful. And so on.

The session spent some time on other topics - primarily user-space checkpoint/restart. It was agreed that everybody in the room was interested in better isolation, and that the memory controller was the area in need of the most work at the moment. The session was dominated by users of control groups, though; there were not a lot of implementers present. Even more notable in their absence were those developers who are opposed to control groups in their current form; it would have been interesting to hear their ideas about how the needs expressed there should really be met.

Comments (1 posted)

Toward a unified display driver framework

In a separate article, LWN looked at the discussion around how display drivers should be managed in the X server; one of the things that was noted there was that the movement of much of the driver logic into the kernel has reduced the rate of change on the user-space side. Seemingly simultaneously, the kernel community got into an extended discussion of how display drivers should be managed within the kernel. Here, the complexity of contemporary hardware is likely to drive both a consolidation of and some extensions to the kernel's interfaces.

It all started rather innocently with Tomi Valkeinen's description of the challenges posed by the display system found on OMAP processors. System-on-chip architectures like OMAP tend not to bother with the nice separation between devices found on desktop- and server-oriented architectures. So, instead of having a "video card," the OMAP has, on one side, an acceleration engine that can render pixels into main memory and, on the other, a "display subsystem" connecting that memory to the video display. That subsystem consists of a series of overlay processors, each of which can render a window from memory; the output of all the overlay processors is composited by the hardware and actually put on the display. Or, more specifically, the output from these processors is handed to the panel controller, which may be a complex bit of hardware in its own right.

So OMAP graphics depends on a set of interconnected components. Filling video memory can be done via the framebuffer interface, via the direct rendering (DRM) interface, or, for video captured from a camera, via the Video4Linux2 overlay interface. Video memory must be managed for those interfaces, then handed to the display processors which, in turn, must communicate with the panel controller. All of this works, but, as Tomi noted, there seems to be a lot of duplication of code between these various interfaces and no generic way for the kernel to manage things at any of these levels. Wouldn't it be nicer, he asked, to create a low-level display framework to handle these tasks?

He is not the first to ask such a question; the graphics developers have been working on this problem for some years, and much of the solution seems clear. The DRM code is where the bulk of the work has been done in the last few years; it is the only display subsystem that comes close to being able to describe and drive contemporary hardware. As the memory management issues associated with graphics become more complex, it becomes increasingly necessary to use a management framework like GEM, and that means using DRM. It also, as a result of its X-server heritage, contains a couple decades' worth of experience on dealing with the quirks of real-world video hardware. So most developers seem to believe that, over time, DRM should become the interface for mode setting and memory management, while the older framebuffer interface should become a compatibility layer over DRM until it fades away entirely.

That said, Florian Tobias Schandinat, who recently took over the maintainership of the framebuffer code, has a different opinion. To Florian, the framebuffer layer is alive and well, it has more users than DRM does, and it will not be going away anytime soon. His biggest complaint with DRM appears to be that (1) it is significantly more complex, making the drivers more complex, and (2) exposing the acceleration capabilities of the graphics processor makes it easy for applications to crash the system. The fact that the framebuffer API does not provide any mechanism for acceleration is, in his view, an advantage.

Florian would appear to be in the minority here, though; most developers seem to feel that it will be increasingly hard to manage contemporary hardware without the capabilities that the DRM layer provides. The presence of bugs in DRM drivers that can crash the system - especially when acceleration is used - is not really denied by anybody, but it was pointed out that use of DRM does not require the use of acceleration. The hardware is also apparently getting better in that it makes it easier for the operating system to regain control of the GPU when necessary. In any case, crashes and bugs are seen as something to fix and not as a reason to avoid DRM outright.

That leaves the question of how to handle the Video4Linux2 overlay feature. Overlay has been somewhat deprecated and unloved for some years, though it remains an official part of the interface; it was designed for an earlier, simpler era. When CPUs reached a point where they could easily manage a video stream from a camera device, the motivation for overlay faded - for a while. More recently, the resolution of video streams has increased notably and power consumption has become a much more important consideration. Even if the CPU can process a video stream in real time on a mobile device, the battery will last longer if the CPU sleeps and the stream goes straight to video memory. That means that the ability to overlay video streams onto the display in a zero-copy manner has become interesting again.

Given that the old overlay interface is seen as inadequate, there is a clear need for a new one. Jesse Barnes floated a proposal for a new overlay API back in April; the DMA buffer sharing proposal posted more recently is also aimed at this requirement. The word is that this topic was discussed at the X Developers Conference and that a new proposal is forthcoming soon.

As an indication of where things could be heading in the longer term, it is worth looking at this message from Laurent Pinchart, the author of the V4L2 media controller subsystem. The complexity of video acquisition devices has reached a point where treating them as a single device no longer works well; thus the media controller, which allows user space to query and change the connections between a pipeline of devices. The display problem, he said, is essentially the same; perhaps, he suggested, the media controller could be useful for controlling display pipelines as well. The idea did not immediately take the world by storm, but it may give an indication of where things will eventually need to go in the future.

The last few years have seen the consolidation of a lot of display-oriented code into the kernel; that code is increasingly using common infrastructure like the GEM memory manager. It is not hard to imagine that this consolidation will continue to the point where the DRM subsystem becomes the supported way for controlling displays, with the other interfaces implemented as some sort of compatibility layer. The complexity of the DRM code is, in the end, driven by the complexity of the hardware it must drive, and that hardware does not look like it will be getting any simpler anytime soon.

Comments (5 posted)

dm-verity

It is not often that Netflix employees show up on linux-kernel to advocate for the merging of specific patches. But that is exactly what has happened after a posting of a new device mapper module called dm-verity. As one might expect, dm-verity has little to do with, say, efficient sorting of DVD mailings. It is, instead, a classic piece of security technology with the potential to work in the user's interests - or against those interests.

The purpose of dm-verity is to implement a device mapper target capable of validating the data blocks contained in a filesystem against a list of cryptographic hash values. If the hash for a specific block does not come out as expected, the module assumes that the device has been tampered with and causes the access attempt to fail. It has been put forward by Mandeep Singh Baines of Google's Chromium OS team, but there appears to be interest in this capability beyond that small group.

At the core of this new facility is a module called dm-bht, which works with a list of block numbers and their associated hash values. This list is organized into a simple tree for quick access to the hashes for arbitrary blocks. In essence, the leaves of the tree are pages containing hash values; each higher level in the tree contains hashes of the blocks below it. Verifying a block requires not only checking the hash value for that specific block; it is also necessary to verify hashes up to the root of the tree. If the hash for the tree root (which is assumed to be trusted) checks out, all is well. The dm-bht code can use any hash algorithm supported by the kernel's crypto API; SHA1 is given as an example, but others can be used as well.

dm-verity implements a read-only target; it is assumed that there is no need to change the data protected by this scheme (being, most likely, the binaries run by the system itself) during operation. The tree of block hashes is stored with (or near) the data itself, but the root hash must be passed in externally. If that root hash comes from a trusted source, it should be possible to detect any modification of the disk, in either the data itself or in the stored hash tree. So, if all goes well, a system running with dm-verity can be assured that the underlying software has not been changed. It's worth noting that integrity checking for any specific block does not happen until the kernel tries to read that block into the page cache. There is, thus, no need for a lengthy verification process at boot or mount time.

All of this depends on getting the right hash value into the system at startup time. If some sort of hardware-verified trusted bootloader is in use, that can probably be done in some sort of secure manner. Device mapper setup is a complex task requiring some sort of running user space. This means that a system using dm-verity will need some other mechanism to load a trusted kernel and initramfs or the whole chain will break. A trusted bootloader can achieve that kind of setup; another example given by the developers is a system booting from a "known good" source like a USB stick that is never left unattended.

One might wonder how dm-verity differs from existing features like the extended verification module. EVM requires and uses a trusted platform module (TPM) on the system to be verified; as long as the initial boot step can be secured, dm-verity is able to work without a TPM. It also seems likely that dm-verity will be faster since it does on-demand verification of blocks; there is no need to verify entire files before the first block can be accessed.

Wesley Miaw of Netflix made it clear that this patch is seen with favor there:

The reasons for this interest should be fairly clear: dm-verity will make it easier to create locked-down Linux-based systems that will enforce whatever DRM requirements the movie studios may see fit to impose. Thanks to dm-verity, there will no longer be pirated films circulating on the Internet; or, perhaps, that's the sort of outcome that only happens in the movies. Whether or not the effort is futile, it shows that tools like dm-verity can be used to harden Linux-based systems in ways that are hostile to their users.

To an extent, Google's interests may align with those of Netflix: Chromebooks that can stream content from Netflix will be more attractive than those that cannot. But dm-verity also fits the ChromeOS concepts of minimal, trustworthy devices with data stored on Google's servers. For users who like this mode of operation, this kind of built-in integrity protection is a positive feature. Google can, one hopes, be trusted to hold the user's data; a suitably verified device can be trusted not to leak that data or the user's credentials to an attacker. Even if the running system is compromised through some sort of malware attack, a simple reboot should either put things right or make it clear that the machine can no longer be trusted.

As long as this functionality is under the user's control, it can be made to serve the user's interests. The "developer mode switch" designed into Chromebooks seems like a good compromise in this area. Some vendors will, beyond doubt, choose to incorporate tools like dm-verity without giving owners the ability to turn it off. That is not a good thing, but neither is it anything new.

Comments (12 posted)

Patches and updates

Kernel trees

• Thomas Gleixner: 3.0.4-rt14 . (September 15, 2011)

Core kernel code

• Marc Zyngier: genirq: handling GIC per-cpu interrupts . (September 19, 2011)

• Srikar Dronamraju: Uprobes patchset with perf probe support . (September 20, 2011)

Development tools

• Shubham Goyal: [ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2011. . (September 16, 2011)

Device drivers

• Jassi Brar: DMAEngine: Define generic transfer request api . (September 15, 2011)

• Linus Walleij: pin controller subsystem v7 . (September 16, 2011)

• Javier Martinez Canillas: Input: Cypress TTSP device driver . (September 19, 2011)

• Jason Wang: Multiqueue support in tun/tap . (September 19, 2011)

• Rob Clark: RFCv2: omapdrm DRM/KMS driver for TI OMAP platforms . (September 20, 2011)

• Deepthy Ravi: OMAP3EVM: Add support for MT9T111 sensor . (September 20, 2011)

• Martin Hostettler: v4l: Add driver for Micron MT9M032 camera sensor . (September 20, 2011)

• Sylwester Nawrocki: [PATCH v1 0/3] Add v4l2 subdev driver for S5K6AAFX sensor with embedded SoC ISP . (September 20, 2011)

Documentation

• Michael Kerrisk: Man page doc for SEEK_DATA/SEEK_HOLE . (September 19, 2011)

• Michael Kerrisk: rt_sigqueueinfo/rt_tgsigqueueinfo man page . (September 19, 2011)

• Michael Kerrisk: man-pages-3.33 is released . (September 20, 2011)

Filesystems and block I/O

• Andi Kleen: Improve lseek scalability v3 . (September 16, 2011)

• Evgeniy Polyakov: POHMELFS is back . (September 19, 2011)

Memory management

• Dan Magenheimer: [PATCH V10 0/6] mm: frontswap: overview (and proposal to merge at next window) . (September 19, 2011)

• Michel Lespinasse: idle page tracking / working set estimation . (September 19, 2011)

• Johannes Weiner: 50% faster writing to your USB drive!* . (September 20, 2011)

Networking

• Glauber Costa: per-cgroup tcp buffer pressure settings . (September 15, 2011)

Architecture-specific

• Jeremy Fitzhardinge: [PATCH RFC V2] Paravirtualized ticketlocks . (September 15, 2011)

• Pawel Moll: arm: Add platform bus driver for memory mapped virtio device . (September 19, 2011)

• Don Zickus: x86, nmi: new NMI handling routines . (September 20, 2011)

• Robert Richter: perf, x86: Implement AMD IBS . (September 21, 2011)

Security-related

• Mandeep Singh Baines: dm: verity target . (September 19, 2011)

Page editor: Jonathan Corbet

Distributions

SmartOS: virtualization with ZFS and KVM

On August 15, at the KVM forum 2011, Bryan Cantrill, VP Engineering at Joyent, gave a presentation entitled "Experiences Porting KVM to SmartOS." The SmartOS in the title is Joyent's illumos-based operating system that is the foundation of its public cloud and its SmartDataCenter product. With this talk, Cantrill essentially announced that Joyent has ported KVM to the illumos (Solaris) kernel.

Thanks to its illumos base, Joyent's SmartOS already had several key features for a cloud operating system, such as the ZFS file system, the dynamic tracing possibilities of DTrace, network virtualization with Crossbow, and operating system-level virtualization (Zones) to isolate virtual operating systems, all running on the same kernel. However, one essential piece was missing in this puzzle of enterprise technologies: hardware virtualization. Granted, a few years ago OpenSolaris had Xen Dom0 support (called xVM), even with hardware virtualization, but the project was abandoned even before Oracle walked away from OpenSolaris.

Joyent (which is a member of the Open Virtualization Alliance dedicated to the awareness and adoption of KVM) believes in the thesis that the best hypervisor is the host operating system itself, because anyone attempting to implement a thin hypervisor would end up retracing the history of operating systems. This is exactly the vision of KVM, so when Joyent decided in the fall of last year that it needed to port KVM to SmartOS, this was a natural (but not trivial) choice.

Because its resources were constrained, Joyent decided to focus exclusively on KVM support for Intel processors. More specifically, a machine running KVM on illumos needs an Intel processor with VT-x and EPT (Extended Page Tables), such as the Nehalem Core i3/i5/i7. However, the developers made sure that they didn't make decisions that would impede later AMD support. Also, only x86-64 hosts and x86 and x86-64 guests are supported. Apart from these constraints, one of the design goals was that the KVM port to illumos would maintain compatibility with the QEMU/KVM interface as much as possible.

Porting an unportable component

At first sight, it seems impossible to port a component that is essentially not designed to be portable: KVM is very specific to Linux. Some of the Linux-specific facilities that KVM uses could be emulated in the illumos kernel, but, because of the big differences between the Linux and the illumos kernel, this would not be a clean solution. Joyent engineer Max Bruning started working on the port in the fall of 2010 by copying the KVM bits from the stable Linux 2.6.34 source and getting it to compile on illumos; in April 2011 he was joined by Robert Mustacchi and Bryan Cantrill. As Cantrill explained in his presentation, DTrace (invented by Cantrill when he was working at Sun) was essential in the porting process: it let them understand how much the still unported code was used by virtual machines.

In his blog post KVM on illumos, Cantrill explains why the porting effort was so grueling:

According to Joyent's measurements, the illumos KVM port performs as well as Linux KVM, at bare-metal speeds for entirely CPU-bound workloads. For other workloads, such as a MySQL benchmark, the performance is obviously not at full bare-metal speed, but the Linux and illumos implementations of KVM don't diverge much from each other. Tested guest operating systems include SmartOS, 64-bit Linux, Windows Server 2008 R2, FreeBSD, OpenBSD, QNX, Plan 9 and Haiku. As for VM resources, the same limitations as for Linux KVM apply: up to 256 virtual CPUs per virtual machine and up to 2 TB virtual memory.

Limitations and enhancements

Illumos KVM diverges from Linux KVM in some areas. For starters, apart from the focus on Intel and x86/x86-64 there are some limitations in functionality. As a cloud provider, Joyent doesn't believe in overselling memory, so it locks down guest memory in KVM: if the host hasn't enough free memory to lock down all the needed memory for the virtual machine, the guest fails to start. Using the same argument, Joyent also didn't implement kernel same-page merging (KSM), the Linux functionality for memory deduplication. According to Cantrill's presentation, it's technically possible to implement this in illumos, but Joyent doesn't see an acute need for it. Another limitation is that illumos KVM doesn't support nested virtualization.

However, tying KVM to illumos instead of Linux also makes some interesting enhancements possible. For instance, you can create ZFS volumes for your virtual machine images. At first sight, this looks like just a convenient way to store your VM images, but it really improves the virtualization experience. Because ZFS can clone volumes in constant time, you can provision new KVM guests nearly instantly if you already have a reference image. Moreover, ZFS remote replication with the zfs send and zfs receive commands makes an efficient foundation for remote cloning and migration of virtual machines over the network. To streamline this, Cantrill intends to integrate QEMU migration with ZFS remote replication. Also, because ZFS has a unified adaptive replacement cache (ARC), guest I/O will be efficiently cached in the host, resulting in improved random I/O operations.

Another improvement that illumos makes possible is the use of Zones, the OS virtualization feature. While the illumos KVM implementation doesn't require it, SmartOS runs KVM guests in a local zone, with the QEMU process as the only process running in the zone. While zones were originally used for resource management, quality of service, I/O throttling, and so on, containing QEMU in its own zone also improves security by reducing the attack surface for QEMU exploits. If an exploit 'breaks out of the virtual machine', it's still contained in the local zone and has no access to the global zone of the virtualization host or the local zones of the other virtualization guests.

Another interesting feature of illumos is network virtualization, also known as Crossbow. With a few commands, you can create a virtual network interface card (VNIC) per KVM guest. The SmartOS developers have written some glue code to connect this feature to virtio and have been able to attain 1Gbit/second data rates to/from a KVM guest. VNICs also makes managing the virtual machine's network usage more easy. Thanks to flow management, guests can be capped at specified levels of bandwidth, and guests can be confined to specified IP addresses, hereby making IP spoofing impossible.

And last but not least, the dynamic tracing possibilities of the DTrace framework let the system administrator understand the workload characteristics of his virtual machines and helps with troubleshooting. For this purpose, Joyent has added some DTrace probes to QEMU and KVM to examine the behavior of KVM guests. They have even integrated several of these metrics into their Cloud Analytics tool to visualize the KVM guest behavior in graphs. In his presentation, Cantrill even suggests that, thanks to the better visibility in guest behavior, DTrace will help in finding performance improvements for KVM, which will likely carry from illumos KVM to the original Linux implementation.

The source

Joyent was already a big contributor to illumos, the successor to the OpenSolaris community. However, their KVM port is the first major addition of functionality to the illumos source since Oracle let the OpenSolaris community die. The source code is published in two parts: a GitHub repository for KVM itself (illumos-kvm) and one with some minor patches to QEMU 0.14.1, all of which they intend to upstream (illumos-kvm-command). Other KVM-specific tooling (such as the kvmstat command for monitoring of KVM statistics) has already been upstreamed to illumos itself. According to Joyent, this port is at or near production quality.

Joyent hasn't pushed any bug fixes back to KVM, but the reason for this is quite simple: they didn't find any bugs in KVM. Cantrill explains this in an email interview:

As for the enhancements Joyent's team made to illumos KVM: all of them are specific to illumos features like ZFS, DTrace, Zones, Crossbow, kstats, and the like. As these features do not exist on a Linux host, it doesn't make any sense to upstream them.

But of course there have been a lot of changes to Linux KVM since 2.6.34, the version on which illumos KVM is based. Cantrill is not very concerned about this, he explains:

The license

Of course some questions arose about the license: Joyent has copied the GPL-ed KVM code from Linux, while the illumos kernel uses the CDDL (Common Development and Distribution License). However, according to Cantrill this doesn't pose any problems. On his blog he answers a question from a reader about the issue:

And on Hacker News he clarifies that their KVM port doesn't use the hooks that Linux KVM has into the Linux kernel (which are marked as EXPORT_SYMBOL_GPL in the Linux kernel): "Actually, our port does not use these hooks – there were zero mods to the illumos kernel to support KVM per se." So, although there seem to be some questions about the legality of the KVM module in illumos, the developers are fairly confident that the problems don't apply because the illumos kernel (CDDL-licensed) is not a derived work of the illumos KVM module (GPL-licensed).

SmartOS and OpenIndiana

SmartOS, for which you can find the source on GitHub (smartos-live), can be downloaded as an ISO or USB image, and it's a minimal live distribution meant to run as a virtualization server. SmartOS just boots from a USB stick or CD-ROM and runs from RAM. It's not meant to be installable, and Joyent doesn't intend to develop an installer, but with some elaborate commands it's possible to install SmartOS on a hard disk.

When running SmartOS from RAM, changes made on the running system naturally don't persist across boots. This doesn't have to be a big issue, as long as you don't want to change your operating system's configuration. Just create one or more ZFS pools with zfs create to initialize your hard disks and to be able to store data on them. However, because of the transient nature of SmartOS, you have to manually import all pools with the zpool import command after each boot.

There's some scarce and fragmented documentation on the SmartOS wiki, with some help about creating zones, creating virtual machines, and other tasks. If you're not comfortable with the Solaris commands, you can also read the topic Finding Your Way Around a SmartMachine on the wiki of SmartMachines, Joyent's commercial cloud offering based on SmartOS.

As SmartOS is really stripped down to only have the minimal bits to work as a virtualization hosts, many tools for other purposes are lacking out-of-the-box. To install extra software, you can use pkgsrc, NetBSD's portable package manager, by downloading the pkgsrc bootstrap image and unpacking it.

If you really want to have illumos KVM installed on hard disk instead and don't want to get your hands dirty with the manual installation of SmartOS, there's another option. OpenIndiana, the spiritual successor of the OpenSolaris distribution, recently released their development build 151a exactly one year after the initial release of the distribution. It's the first build based on the illumos core and it also has integrated Joyent's KVM support. Installing OpenIndiana oi_151a gives you a Gnome desktop for a workstation or a text-based installation for headless servers, and the KVM bits can be installed with the pkg install command of the package manager IPS (Image Packaging System). The OpenIndiana wiki shows you the needed commands.

Conclusion

If anyone doubted that illumos would be able to build enough momentum, Joyent's KVM port to illumos and the subsequent illumos-based OpenIndiana development release have surely answered these doubts. Illumos appears to be here to stay, and it offers a lot of interesting technology, such as ZFS, DTrace, Crossbow, Zones, and now KVM. For Linux users who were interested in these Solaris technologies but wouldn't want to lose their favorite hypervisor KVM, SmartOS and OpenIndiana are now able offer the best of both worlds.

Comments (15 posted)

Brief items

Distribution quotes of the week

-- Rich Freeman

-- Barry Warsaw

Comments (none posted)

Tumbleweed backs off on systemd for now

The openSUSE "Tumbleweed" distribution has removed systemd (which had been available as an optional package) for now. "Due to a number of interdependancies on packages that are not ready for Tumbleweed, and other interactions with the system that are causing problems for some users, I'm going to remove systemd from Tumbleweed today to allow the developers to spend more time on getting it stable for Factory and 12.1 instead of having to chase down problems that are specific to Tumbleweed only." Interestingly, almost all of the followup discussion is about whether this change should be announced more widely or kept quiet.

Full Story (comments: 65)

Distribution News

Debian GNU/Linux

Upcoming Debian point releases and call for test

The Debian Project has announced that the upcoming point releases for Debian 5 "Lenny" and Debian 6 "Squeeze" are scheduled for October 1st and October 8th respectively. Interested users are encouraged to test these packages before their release, especially on systems that use the updated drivers.

Full Story (comments: none)

Newsletters and articles of interest

Distribution newsletters

• Debian Project News (September 19)
• DistroWatch Weekly, Issue 423 (September 19)
• Maemo Weekly News (September 19)
• openSUSE Weekly News 193 (September 17)
• Ubuntu Weekly Newsletter, Issue 233 (September 18)

Comments (none posted)

Arch Linux - "It is what you make it" (The H)

The H looks at the simplicity of Arch Linux and its usefulness as a base for custom distributions. "Every Arch installation is different to every other Arch installation and is defined by its technical elegance and its adherence to the demands of the individual user or set of users. The aim is not a Linux for every man, but a Linux that is moulded to fit the demands of the individual user. So, as the Arch wiki expresses it, simplicity in the context of Arch Linux means "without unnecessary additions, modifications, or complications. In short; an elegant, minimalist approach.""

Comments (38 posted)

Fresh wind for openSUSE (The H)

The H talks with Andreas Jaeger at the openSUSE Conference in Nuremberg. "However, 'zero versions' reportedly tend to be regarded as 'major updates' and generate a high level of expectation among users. Apparently, this has repeatedly given rise to the opinion that a version wasn't ready for release, and that it contained too few new features for a zero release. Internally, the openSUSE team has never treated the zero versions as major releases, said Jaeger. The developers therefore decided to skip the 'zero release' and not release a version 12.0."

Comments (12 posted)

Poortvliet: openSUSE Conference Fun!

Jos Poortvliet wraps up last week's openSUSE Conference. "Lots of talks and discussions ranging from development and low-level kernel tools to social and marketing sessions have taken place over the last four days, all focused on world domination of course. There was a large number of sessions around packaging, both focusing on teaching as well as improving current packaging quality and more steam lined maintenance of our repositories. Robert Schwelkert's talk on "Where do we improve?" proposed a lot of changes like improved translations, documentation, separating the bugzilla and getting rid of Novell's iChain. The openSUSE Project Meeting discussed a number of interesting ideas and developments including the current status of the openSUSE Foundation and upcoming elections. The board said it was working on the foundation but it is a slow process. We want to have a long-term solution with buy-in from all parties. As Attachmate has just joined this process it has taken time to get them up to speed but there is progress now. "

Comments (none posted)

Zacchiroli: why there are so many debian derivatives

Stefano Zacchiroli considers the advantages of using Debian as a base for derivative distributions. "But there are also a couple of "political" reasons for basing derivatives on Debian. One is quite subtle and applies mostly to commercial distributions. If you are designing one such commercial distro, you have to be based on an independent distro with no commercial interests, lest risking that petty (technical or otherwise) choices might be made just to undermine your business. Among "popular" GNU/Linux distros, Debian is essentially the only one which is both volunteer-based and not ascribable to any specific company." (Thanks to Paul Wise)

Comments (none posted)

Page editor: Rebecca Sobol

Development

Why CouchDB?

Why should developers use CouchDB? It all comes down to replication, says Benjamin Young. Data is "lonely," trapped in proprietary formats and cloud silos, or remote applications that require dependable Internet connections. When the power goes out, when you want to work on the plane, or when you just want control over your data — Young says you want CouchDB.

Young was speaking at the Strange Loop 2011 conference on Tuesday, September 20 in St. Louis. The mission? To convince some of the 900-plus developers at the conference to consider CouchDB as database to power their applications. It wouldn't be very convincing if Young wasn't a fan himself, of course. Young's history with CouchDB goes back to work with BlueInk, a content management system for building simple sites for small companies. BlueInk used CouchDB, which caught the attention of CouchOne (now Couchbase, following a merger of CouchOne and Membase). BlueInk was released as open source a year ago when Young joined CouchOne.

What is CouchDB?

After giving a bit of his history, Young moved on to talking about CouchDB and what it actually is. CouchDB is a document storage database written in Erlang. It is open source, and part of the Apache project — though several different implementations are offered by a handful of companies like Couchbase. It's one of the current crop of NoSQL, or coSQL if you prefer, databases.

Documents in CouchDB are stored in JavaScript Object Notation (JSON) format, plus attachments if you have binary data (photos, for example) to put into the database. Queries are written using JavaScript. It's a distributed database that replicates statelessly. That is, as Young describes, "it can be interrupted and doesn't care." CouchDB counts the number of changes that have been made since it last connected to another instance, and picks up where it left off.

Young described a scenario of deploying an application using CouchDB in rural Africa. The users might only have Internet a couple of hours per day in rural Africa. If they were using a cloud-based application that depends on a constant connection "they'd be completely screwed." Using CouchDB, the data can be easily synced during the window of connectivity without any hassle. The database doesn't have to keep a constant state, CouchDB operates on the principle that it will be "eventually consistent."

As the scenario Young described implies, CouchDB is not limited to running on a server. It will run on a desktop, server, or even on a mobile device. Any device running CouchDB can communicate and send data to another instance of CouchDB, so replication can be peer-based or follow a server-client model. Replication can also be filtered, so that an instance only receives some of the information relevant to its local application. For instance, Young mentioned you might use CouchDB for distributing shipping information — the primary instance of CouchDB getting all of the order information, and filtering the appropriate orders to the users who are filling some of the orders.

Another argument for CouchDB? Young says "you already know the API," because it's basically HTTP. CouchDB provides a RESTful API with GET, PUT, POST, DELETE, as well as a non-standard extension to HTTP, COPY. The COPY API does what one would expect: it duplicates the contents and attachments of a document to a new document using a new name, and without requiring it to be retrieved first.

Web 2.5: Work Locally, Sync Globally

The big reason for choosing CouchDB according to Young? Replication. Like many in the open source community, Young doesn't express a lot of love for cloud services — which he describes as "managed and terms-of-serviced and fascist." That's not to say Young doesn't want to collaborate with his friends and co-workers. Far from it, but Young argues for a return to a "decentralized and very organic" Web. He calls it "Web 2.5" (because "Web 3.0 is already taken").

Web 2.0 gave us "cloud powers" but took away ownership of data. "For cloud powers we traded ownership, privacy, security, safety, stability... we need to get all those things back, to get that we need replication. Not only to move data, but applications." Web 2.5, says Young, is a situation where you have the browser and server "and then the .5 lives in the cloud, and that's a service that watches for changes in your CouchDB." Data a user chooses to share can still be synced up to cloud services, but it lives on the users' machine as well. "I as a person shouldn't have to keep everything that is me in a little package in the cloud... I can work locally on my laptop and then if I need to share I can go somewhere people are."

Get Started

Young suggests that developers interested in CouchDB start at Iris Couch as the fastest way to get started. He also suggested checking out the Apache project, and its mailing lists. For more immediate questions and feedback, Young noted that there was a decent community of CouchDB developers on Freenode on the #couchdb channel as well as in the #couchbase channel.

Comments (2 posted)

Brief items

Quotes of the week

-- Greg Ward

Jim Sangwine

-- Greg Smith

Comments (none posted)

GNU Health 1.3.3 released

GNU Health is a hospital information system with modules for electronic medical record, hospital information systems, and health information systems. The 1.3.3 release is out; beyond a name change (from GNU Medical) and improved installation scripts, it includes: "major improvements in the Laboratory module. Now we have automatic visual alerts, possibility to exclude analytes from the analysis, qualitative and quantitative testing."

Full Story (comments: none)

Greedy register allocation in LLVM 3.0

Readers interested in the grungy details of compilers may enjoy this LLVM project blog entry on their new register allocation algorithm. "The new basic allocator does away with linear scan's dependence on visiting live ranges in linear order. Instead, it uses a priority queue to visit live range in order of decreasing spill weight. The active list used for interference checks is replaced with a set of live interval unions. Implemented as a B+ tree per physical register, they are an efficient way of checking for interference with already assigned live ranges. Unlike the active list, live interval unions work with any priority queue order."

Comments (7 posted)

Openshot 1.4

Version 1.4 of the OpenShot video editor has been announced. Changes include timeline improvements, a "more stable" effects engine, new color correction tools, more animations and transitions, and more. (LWN reviewed OpenShot in February).

Comments (none posted)

Newsletters and articles

Development newsletters from the last week

• Bzr team weekly notes (September 20)
• Caml Weekly News (September 20)
• LibreOffice development summary (September 21)
• PostgreSQL Weekly News (September 18)

Comments (1 posted)

Bray: Preparing for [Android] Handsets

Tim Bray Scott Main notes that the upcoming Ice Cream Sandwich (ICS) release will support more form factors compared to the current Honeycomb. "Early this year, Honeycomb (Android 3.0) launched for tablets. Although Honeycomb remains tablets-only, the upcoming Ice Cream Sandwich (ICS) release will support big screens, small screens, and everything in between. This is the way Android will stay from now on: the same version runs on all screen sizes. Some Honeycomb apps assume that they'll run only on a large screen, and have baked that into their designs. This assumption is currently true, but will become false with the arrival of ICS, because Android apps are forward-compatible - an app developed for Honeycomb is compatible with a device running ICS, which could be a tablet, a phone, or something else."

Comments (18 posted)

Page editor: Jonathan Corbet

Announcements

Brief items

The Maemo community loses Gary "lcuk" Birkett

The Maemo community is mourning the loss of Gary Birkett, also known as "lcuk," who passed away recently. Thoughts and memories are being posted on this maemo.org forum page. Condolences to all, especially the family that Gary left behind.

Comments (none posted)

Articles of interest

Is Android really free software? (Guardian)

Richard Stallman has written a lengthy article for The Guardian on the freedom - or lack thereof - of Android. "Putting these points together, we can tolerate non-free phone network firmware provided new versions of it won't be loaded, it can't take control of the main computer, and it can only communicate when and as the free operating system chooses to let it communicate. In other words, it has to be equivalent to circuitry, and that circuitry must not be malicious. There is no obstacle to building an Android phone which has these characteristics, but we don't know of any."

Comments (55 posted)

MyGreatFest - The iDroidProject Takes To The Stage (AppAdvice)

Developers from the iDroid Project announced at the MyGreatFest conference that they are working on porting Android to iOS devices. "Back in 2008, the project initially began under the name of the "iPhone Linux Project." This project aimed to boot a Linux kernel, and by the end of 2008, this aim had been achieved. The year after, the hacker behind the iPhone Linux Project surprised everybody by presenting a working first-generation iPhone running Android OS: The iDroid Project was born."

Comments (5 posted)

New Books

The Art of R Programming--New from No Starch Press

No Starch Press has released "The Art of R Programming" by Norman Matloff.

Full Story (comments: 1)

Upcoming Events

HelloGCC Workshop 2011

HelloGCC 2011 is the 4th Workshop on open source toolchains taking place in Beijing, China on September 24, 2011.

Full Story (comments: none)

PyCon Finland 2011

PyCon Finland takes place October 17-18 in Turku. Click below for some conference updates.

Full Story (comments: none)

Program of LibreOffice Conference

The first LibreOffice Conference will take place October 13-15, 2011 in Paris, France. "On October 12, La Cantine will be open in the afternoon for registration and for a meeting of TDF Steering Committee, followed by a public Q&A session open to members of The Document Foundation and conference attendees. In the evening at 7 pm, Cap Digital will organize a welcome cocktail at their headquarters near the Bastille."

Full Story (comments: none)

Registrations for linux.conf.au 2012 now open

Registrations are open for linux.conf.au 2012, which takes place January 16-20 in Ballarat, Australia. "Early bird registrations will remain open until the end of October 2011 or until sold out. If previous years are anything to go by, the early bird registrations will prove very popular and see significant demand, so attendees will need to get in early to avoid disappointment."

Full Story (comments: none)

Events: September 29, 2011 to November 28, 2011

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
September 27 September 29	Nagios World Conference North America 2011	Saint Paul, MN, USA
September 27 September 30	PostgreSQL Conference West	San Jose, CA, USA
September 29 October 1	Python Brasil [7]	São Paulo, Brazil
September 30 October 3	Fedora Users and Developers Conference: Milan 2011	Milan, Italy
October 1 October 2	WineConf 2011	Minneapolis, MN, USA
October 1 October 2	Big Android BBQ	Austin, TX, USA
October 3 October 5	OpenStack "Essex" Design Summit	Boston, MA, USA
October 4 October 9	PyCon DE	Leipzig, Germany
October 6 October 9	EuroBSDCon 2011	, Netherlands
October 7 October 9	Linux Autumn 2011	Kielce, Poland
October 7 October 10	Open Source Week 2011	Malang, Indonesia
October 8	PHP North West Conference	Manchester, UK
October 8	FLOSSUK / UKUUG's 2011 Unconference	Manchester, UK
October 8 October 9	PyCon Ireland 2011	Dublin, Ireland
October 8 October 9	Pittsburgh Perl Workshop 2011	Pittsburgh, PA, USA
October 8 October 10	GNOME "Boston" Fall Summit 2011	Montreal, QC, Canada
October 9 October 11	Android Open	San Francisco, CA, USA
October 11	PLUG Talk: Rusty Russell	Perth, Australia
October 12 October 15	LibreOffice Conference	Paris, France
October 14	Workshop Packaging BlankOn	Jakarta , Indonesia
October 14 October 16	MediaWiki Hackathon New Orleans	New Orleans, Louisiana, USA
October 15	Packaging Debian Class BlankOn	Surabaya, Indonesia
October 17 October 18	PyCon Finland 2011	Turku, Finland
October 18 October 21	PostgreSQL Conference Europe	Amsterdam, The Netherlands
October 19 October 21	13th German Perl Workshop	Frankfurt/Main, Germany
October 19 October 21	Latinoware 2011	Foz do Iguaçu, Brazil
October 20 October 22	13th Real-Time Linux Workshop	Prague, Czech Republic
October 21	PG-Day Denver 2011	Denver, CO, USA
October 21 October 23	PHPCon Poland 2011	Kielce, Poland
October 23 October 25	Kernel Summit	Prague, Czech Republic
October 24 October 25	GitTogether 2011	Mountain View, CA, USA
October 24 October 25	GStreamer Conference 2011	Prague, Czech Republic
October 24 October 28	18th Annual Tcl/Tk Conference (Tcl'2011)	Manassas, Virgina, USA
October 26 October 28	Embedded Linux Conference Europe	Prague, Czech Republic
October 26 October 28	LinuxCon Europe 2011	Prague, Czech Republic
October 28 October 30	MiniDebConf Mangalore India	Mangalore, India
October 29	buildroot + crosstool-NG Developers' Day	Prague, Czech Republic
October 31 November 4	Ubuntu Developer Summit	Orlando, FL, USA
October 31 November 4	Linux on ARM: Linaro Connect Q4.11	Orlando, FL, USA
November 1 November 3	oVirt Workshop and Initial Code Release	San Jose, CA, USA
November 1 November 8	2011 Plone Conference	San Francisco, CA, USA
November 4 November 6	Fedora Users and Developer's Conference : India 2011	Pune, India
November 4 November 6	Mozilla Festival -- Media, Freedom and the Web	London, United Kingdom
November 5 November 6	Technical Dutch Open Source Event	Eindhoven, The Netherlands
November 5 November 6	OpenFest 2011	Sofia, Bulgaria
November 7 November 11	ApacheCon NA 2011	Vancouver, Canada
November 8 November 12	Grace Hopper Celebration of Women in Computing	Portland, Oregon, USA
November 10 November 12	Clojure/conj 2011	Raleigh, NC, USA
November 11 November 12	Zentyal Summit	Zaragoza , Spain
November 11 November 13	Free Society Conference and Nordic Summit 2011	Gothenburg, Sweden
November 12	London Perl Workshop 2011	London, United-Kingdom
November 12	Emacsforum 2011	Copenhagen, Denmark
November 14 November 17	SC11	Seattle, WA, USA
November 14 November 18	Open Source Developers Conference 2011	Canberra, Australia
November 17 November 18	LinuxCon Brazil 2011	São Paulo, Brazil
November 18	LLVM Developers' Meeting	San Jose, CA, USA
November 18 November 20	Foswiki Camp and General Assembly	Geneva, Swizerland
November 19 November 20	MediaWiki India Hackathon 2011 - Mumbai	Mumbai, India
November 20 November 22	Open Source India Days 2011	Bangalore, India
November 24	verinice.XP	Berlin, Germany

If your event does not appear here, please tell us about it.

Page editor: Rebecca Sobol