Reporting bugs upstream

Bug reports from users are an important way for projects to find and eventually fix problems in their code. For most Linux users, though, the code from those projects makes its way to them via distributions, so that's where they report any bugs they find. What happens next is something of an open question, as a thread on the debian-devel mailing list highlights. Should it be up to the bug reporter to push the bug upstream if it is found to not be specific to the distribution? Or should it be up to the package maintainer?

Brian M. Carlson started the thread by noting that he has been frequently asked recently to forward bugs reported to the Debian bug tracking system (BTS) to upstream projects. For a number of reasons, he doesn't think that's the right way to deal with these bugs:

Essentially, Carlson is arguing that package maintainers already have the requisite relationships and accounts to properly move the bug upstream, whereas a regular user, even one who is fairly technically sophisticated, will not. But, of course, it is the user who has the intimate knowledge of the bug and can hopefully reproduce it or at least give the upstream developers a better idea of how to. Inserting the package maintainer into the middle of the bug triaging/fixing process may just slow things down. Unfortunately, most bug tracking systems don't provide a way for users without accounts to be copied on bug report traffic, which leads back to the problem that Carlson identified: users aren't interested in creating an account for each project's bug tracker, instead they rely on their distribution to upstream bugs.

In addition, it may well be that the user doesn't really have the knowledge necessary to interact with upstream, especially for distributions that target less-technical users. Many of the bug reports that come into any bug tracker (Debian's, other distributions', or the projects') are inadequate for various reasons, so forwarding them or asking the user to report them upstream is pointless—potentially counterproductive because it annoys the upstream developers.

It comes down to a question of what the responsibilities of a package maintainer are. Many seem to take a proactive approach by trying to reproduce the problem and, if they can, reporting it upstream. Others, though, are completely swamped by the sheer number of bugs that get reported, finding it difficult to do more than try to determine if the bug is Debian-specific and asking for it to be reported upstream if it isn't.

There is a large disparity in the size of the packages that are maintained by Debian (or any other distribution), of course. Some smaller or less-popular packages may allow their maintainers to keep up with the bugs and shepherd those that deserve it upstream. While other, larger packages, like the kernel, X, GNOME, KDE, LibreOffice, and so on, just don't have enough people or time to do that. Thus it is very unlikely that there can be a "one size fits all" solution; each package and maintainer may necessarily have their own bug management style.

John Goerzen explained how the process works for the Debian package of the Bacula backup program, noting that he sits between the bug reporter and the upstream project, but that it is often wasted effort:

Being a "copy and paste monkey between emails and web forms" is not what he wants to be doing. He points out that various unnamed major packages tend to just ignore many of their bug reports for years at a time. Overall, he doesn't think that being a maintainer should necessitate being a bug intermediary as well:

Ben Finney (and others) see it as, at least partially, a technical problem in the bug tracking software. While requiring bug reporters and those copied on the bug comments to be registered in the system may make sense—for reducing spam problems if nothing else—it definitely puts up a barrier for users:

Finney looks at the problem as an opportunity to try to find better technical solutions. There may be ways to enhance the Debian BTS to file bugs upstream and/or CC the Debian bug reporter on upstream questions, as Don Armstrong suggests. Ubuntu's Launchpad has made some efforts to link downstream and upstream bugs to address some parts of the problem, but it is more than just a technical problem as various posters pointed out.

Many upstream projects are largely uninterested in bugs reported against older versions of their code. Unless the bug can be reproduced in the latest release—or sometimes the latest commit to the project's source code repository—it may not be investigated at all, even if the bug is reported upstream. It is not just package maintainers that have far more work, and bugs, than they can possibly deal with.

But distributions have something of a contract with their users to support the package versions that make up the distribution release. It can be very difficult to have a user test with updated versions of the code in question to try to reproduce the problem. Bugs that can be easily reproduced are obviously much easier to handle, but they are also, seemingly, much less common.

Fedora has struggled with similar issues, including discussions last July on the fedora-devel and test mailing lists; undoubtedly other distributions have as well. Also, it isn't just the distribution and user side that suffers, as there have been cases where bugs and fixes known to downstream distributions haven't made their way upstream. It would seem that there is an opportunity for projects and distributions to work more closely together, but that won't be easy given the workloads on both ends of the stream.

Bug reports are a difficult problem in general, as good reports are sometimes hard to find. There may also be a few layers between the reporter and someone who might be able to actually fix the problem, which can cause all manner of frustrations for anyone involved. On the bright side, though, the situation is far better than the proprietary software world, where reporting a bug may require paying for a "trouble ticket" and waiting for a release that actually addresses it. At least free software allows an interested hacker to poke around in the code and fix it for themselves (or their customers).

Comments (31 posted)

The Cr-48 and Chrome OS: Google's vision of the net

The Cr-48 is, according to Google, the "first of its kind - a notebook built and optimized for the web." It is the next step in the promotion of Chrome OS, Google's other Linux-based distribution. As a way of showing off what it has accomplished and building interest in the system, Google has distributed Cr-48 machines widely. Your editor was a lucky, if late, recipient of one of these devices; what follows are his impressions after some time playing with it. The Cr-48 and Chrome OS are an interesting vision of where computing should go, even if that vision is not for everybody.

The hardware itself is quite nice at a first glance. This machine is not a netbook; it is a small notebook device which clearly has taken some inspiration from Apple's hardware. Except, of course, that Apple's machines are not jet black, with no logos or markings of any type. It exudes a sort of Clarke-ian "2001 monolith" feel. There's an Intel Atom dual-core processor, 2GB of memory, and a 16GB solid-state drive. The silence of the device is quite pleasing; also pleasing is the built-in 3G modem with 100MB/month of free traffic by way of Verizon (which, unsurprisingly, is more than prepared to sell you more bandwidth once that runs out). Other connectivity includes WiFi and Bluetooth (though there appears to be no way to use the latter); there is no wired Ethernet port. There's a single USB port, an audio port, a monitor port, and what appears to be an SD card reader. Battery life is said to be about eight hours. Despite the small disk, it's a slick piece of hardware.

Using Chrome OS

The operating system and the hardware work nicely together. A cold boot takes a little over ten seconds; suspend and resume are almost instantaneous. In normal use, one simply lifts the lid and the system is ready to go; by default, the system does not even request a password at resume time if somebody is logged in - a setting that security-conscious users may want to change. There is a large trackpad with some simple multitouch capability. Interestingly, there is no "caps lock" key; Google, in its wisdom, replaced it with a "search" key. Happily, Google was also wise enough to allow the key to be remapped by the user; it can be restored to caps lock or, instead, as $DEITY intended, set to be a control key. Where one would expect to find the function keys are more web-centric buttons: Google has dedicated keys to operations like "back," "forward," and "reload." Of course, they're really just function keys underneath as far as the X server is concerned.

The system software is Linux-based, of course, but there's no way for a casual user to notice that. The core idea behind Chrome OS is that anything of interest can be had by way of a web browser, so that's all you get. Like an Android phone, the system starts by asking for the user's Google account; everything after that is tied to that account. Email is to be done via GMail (there appears to be no way to read mail directly from an IMAP server), document editing with Google Docs, conferencing with Google Talk, and so on. Like an Android phone, a Chrome OS device is meant to be a portable front-end to Google-based services.

That is why the Cr-48 comes with such a small SSD; very little is stored there beyond the operating system image itself, and that image is small. Most of the space, in fact, is set aside for a local cache, but it's entirely disposable; everything of interest lives in the Google "cloud." So if, as the startup tutorial says, the device succumbs to an "unexpected steamroller attack", nothing is lost except the hardware. The user can sign onto a new device and everything will be there.

The appeal of this arrangement is clear: no backups, no lost data, no hassles upgrading to a new machine. Just browse the web and let Google worry about all the details. Of course, there are some costs; the Cr-48 can do almost nothing which cannot be done via the web. There is no way to get a shell (though see below) and no way to install Linux applications. Even updates are out of the user's hands: they happen when the Chrome OS Gods determine that the time is right.

There is a "web store" where browser-based applications can be had. At this time there is a surprising variety of them, almost all of which are free of charge. The application selection still falls far short of what is available with a standard Linux distribution or on Android, though. It's also not at all clear how many (if any) of these applications are actually free software. The "no local installations" philosophy means that Chrome browser plugins (which hook into the browser at a lower level than "applications" do) cannot be installed; that, in turn, means that any application which requires a plugin, while usable on regular Linux or Windows, is not installable on Chrome OS. It turns out that quite a few web store applications need plugins; annoyingly, the only way to find out if any given application can be installed is to try.

Your editor wanted to take a screenshot or two of the system in operation. The store offers a few screenshot applications, one provided by Google itself. The Google tool, though, needs a plugin and thus refused to install. An alternative application did install, but the "save" button, needing a plugin, was not able to save the result anywhere. The application could, though, "share" the screenshot through any of a number of web services - though the image itself (to your editor's surprise) is stored on the web site of the company providing the screenshot application. Something as simple as taking a screenshot should not be so hard - and it should not broadcast screenshots to the world by default.

Under the hood

The Cr-48 is a locked-down system. Its firmware will only load Google-signed images, so it's not possible for the user to make any changes. The root filesystem is mounted read-only. The whole verified boot mechanism is designed to ensure that the device's software has not been compromised and that the user can trust it. That said, the design goals are also expressed this way:

The way this works on the Cr-48 is through a "developer switch," which is cleverly hidden behind a piece of tape inside the battery compartment. The instructions describe a lengthy series of events that will happen when that switch is flipped, including a special warning screen and a five-minute delay while the system cleans up any personal data which may be cached locally. What actually happened was a warning that the system is corrupted; hitting control-D at that screen did manage to boot the system into the developer mode, though.

Developer mode looks much like the regular operating mode with one exception: the other virtual consoles are now enabled, allowing the user to get to a shell and explore the system a bit. The system, it turns out, is based on a 2.6.32.23 kernel; it's said to be based on Ubuntu Gentoo, but any such parentage is hard to find. It uses the trusted platform module for integrity measurement, but it does not appear to be using the IMA or EVM modules shipped with the mainline kernel. The devtmpfs filesystem is used to populate /dev.

The system uses the ext3 filesystem for local data storage. There are two sets of root filesystem partitions; one is in use while updates are loaded into the other. It also uses eCryptfs to store user-specific data; in theory that means that such data is safe from prying eyes when the user is not actually logged into the system.

Given access to developer mode, one can go as far as installing an entirely new operating system on the device. The instructions for doing so are intimidating at best, though; Google has not gone out of its way to make displacing Chrome OS easy. Your editor will probably give it a try at some point, but the job did not look like something which could be done within any sort of deadline. It sure would have been nice if the system could just boot from an external device.

What it's good for

The appeal of a system like this is easy enough to understand. Here is a computer which can access all kinds of web-based services, never needs to be backed up, is highly malware-resistant, and which can be easily replaced. It could be handed to one's children with minimal fear of the consequences, and it is easily operated by people who are intimidated by any sort of system management task. A Chrome OS device is the contemporary equivalent of an X terminal; it is little more than a window into services which are managed elsewhere.

Your editor, who is not afraid to break manage his systems, and who prefers more control over his data, does not find this approach to computing to be hugely attractive. It is not useful for software development at all, and the things it can do are contingent on having network access. Google Docs might be able to handle a presentation, but the idea of depending on a conference network to be able to give a talk is frightening. There are those of us who will always want our systems to be more self-contained and locally controlled.

That said, such machines are not without their applications. Thousands of people, it seems, have had their laptops searched at the US border; your editor, who crosses that border frequently, has not, yet, had that experience. Should it ever come to pass, it might be nice to have a laptop which contains no local data at all. A throwaway Google account could be used for plausible deniability, and, in the unlikely case of a border agent who knows about the developer switch, any user-specific data on the system (which is encrypted anyway) should be gone by the time it becomes accessible. "Data in the cloud" systems have security concerns of their own (it would be nice if a Chrome OS system could be backed up by providers other than Google, for example), but there are times when having all of one's data be elsewhere can be comforting.

The locked-down nature of Chrome OS is thus not without its value, but locked-down is only good as long as the owner wants things that way. The Chrome OS documentation suggests that Google wants all devices to include a developer switch. In the real world, it would be unsurprising if some vendors somehow never quite got around to adding that switch. Without full access, one of these laptops becomes something more like a television: useful for displaying content, but something short of a real computer.

Chrome OS is clearly not meant to be a "real computer" of the sort that LWN readers are likely to want. The target user base is different, to say the least. As such, it is an interesting exercise in what can be done to package Linux for other classes of users. At the beginning of the year, your editor predicted that Chrome OS would struggle; who wants such a limited system when a real computer can be so easily had? Based on this experience, your editor is not quite ready to change his mind, but he is willing to admit that Chrome OS may be the experience some people are looking for.

Comments (91 posted)

Review: The Linux Programming Interface

Michael Kerrisk's (relatively) new book, The Linux Programming Interface (TLPI), is targeted at Linux system programmers, but it is not just those folks who will find it useful. While it is a hefty tome ("thick enough to stun an ox" as Laurie Anderson might say), it is eminently readable, both by browsing through it or by biting the bullet and reading it straight through. The coverage of the Linux system call interface is encyclopedic, but the writing style is very approachable. It is, in short, an excellent reference that will likely find its way onto the bookshelves of user-space developers and kernel hackers—including some who aren't necessarily primarily focused on Linux.

Kerrisk has been the maintainer of the Linux man pages since 2004, which gives him a good perspective on the Linux API. As he says in the preface, it is quite likely that you have already read some of his work in sections 2, 3, 4, 5, and 7 of those pages. But the book is not a collection of man pages though it covers much of the same ground. The style and organization is much less dry, and more explanatory, than a typical man entry.

The book is some 1500 pages in length, which makes it a rather daunting prospect to review. Once I started reading it, though, it was quite approachable. Kerrisk's clear descriptions of various system calls and other parts of the Linux API made it easy to keep reading. I set out to pick and choose certain chapters to read, and just skim the others, but found myself reading quite a bit more than that—which might partially explain the lateness of this review.

The book is organized into 64 chapters of around 20 pages each, which makes for nice bite-sized chunks that allow for reading the book around other tasks. While the focus is on Linux, Kerrisk doesn't neglect other Unix varieties and notes where they differ from Linux. He also pays careful attention to the various standards that specify Unix behavior—like POSIX and the Single Unix Specification (SUS)—pointing out where Linux does and does not follow those standards.

TLPI was written for kernel version 2.6.35 and glibc 2.12. In the text, though, Kerrisk is careful to indicate which kernel version introduced a new feature, so that those working with older kernels will know which they can use. While it is primarily looking at the 2.6 series, 2.4 is not neglected, and the text notes features that were introduced at various points in the 2.4 kernel history.

The book starts with a bit of history, going all the way back to Ken Thompson and Dennis Ritchie and then moving forward to the present, looking at the various branches of the Unix tree. It then moves into a description of what an operating system is, the role that the kernel plays, and some of the overarching concepts that make up Unix (and Linux). While this information may be unnecessary for most Linux hackers, it will come in handy for those coming to Linux from other operating systems. The ideas that "everything is a file" and that files are just streams of bytes are described in ways that will quickly get a system programmer up to speed on the "Unix way".

After that introductory material, Kerrisk launches into the chapters that cover aspects of the system call interface. This makes up the vast majority of the book and each of these chapters is fairly self-contained. They build on the earlier chapters, but the text is replete with references to other sections. In the preface, Kerrisk says that he attempted to minimize forward references, but that clearly was a difficult task as there are often as many forward as backward references in a chapter.

Navigating within the book is easy to do because there are frequent numbered section and subsection headings, along with the chapter number on each page. Other technical books could benefit from that style. There is also an almost too detailed index that runs to more than 50 pages.

Each chapter comes with sample code that is easy to read and understand. Importantly, the examples also do a good job of demonstrating the topic at hand and some of them could be adapted into useful utilities. The code is available from the TLPI web site and is free software released under the Affero GPLv3. Each chapter also has a handful of exercises for the reader, some of which have answers in one of the appendices.

So, what does the book cover? It would be easy to say "all of it", but that would be something of a cop-out, and a bit inaccurate as well. There are multiple chapters on files, file I/O, filesystems, and file attributes, extended attributes, and access control lists (ACLs). There is a chapter covering directories and links, as well as one that looks at the inotify file event notification call.

There are multiple chapters on processes, threads, signals, as well as chapters covering process groups and sessions, and process priorities and scheduling. Of particular interest to me were a chapter on writing secure privileged programs and one on Linux capabilities. There are two chapters on shared libraries, the first of which is more about the ideas underlying libraries and shared libraries along with how to build them, rather than the dlopen() system call (and friends), which is covered in the second.

There are, perhaps, too many chapters covering interprocess communication (IPC), with separate chapters devoted to each System V IPC mechanism (shared memory, message queues, and semaphores). There is also a chapter for each of the POSIX variants of those three IPC types. Both POSIX and System V IPC get their own introductory chapter in addition to the chapters focusing on the details of each type. Sandwiched between the System V and POSIX IPC mechanisms are two chapters on memory mapping and virtual memory operations that might have been better placed elsewhere in the book. There is also a chapter devoted to an introduction to IPC and one that looks at the more traditional Unix pipes and FIFOs. In all, there are twelve chapters on IPC before we even get to the sockets API.

After IPC, comes a chapter on file locking followed by six chapters covering sockets. Those chapters look at Unix and internet domain sockets, along with server design and advanced sockets topics. The book wraps up with a chapter on each of terminals and pseudoterminals, with something of an oddly placed "Alternative I/O Models" chapter in between them. It's an interesting chapter, covering select(), poll(), epoll(), signal-driven I/O, and a few other topics, but it seems weird where it is.

There is more, of course, and looking at the detailed table of contents will fill out the list. One thing that stands out from the book is the vast size of the Linux/Unix API. It also points out some of the warts and historical cruft that is carried along in that API. Kerrisk is not shy about noting things like that where appropriate in the text: "In summary, System V message queues are often best avoided."

There were two specific topics that I looked forward to reading about but were only marginally covered by the book. The first is containers and namespaces, which are very briefly mentioned in a discussion of the flags to the clone() system call. A more puzzling omission is that there is almost no mention of the ptrace() system call. In the few places it does come up, readers are referred to the ptrace(2) man page.

There are certainly other parts of the Linux API that could have been covered, beyond the system call interface—sysfs, splice(), and perf come to mind—but Kerrisk undoubtedly needed to draw the line somewhere. Overall, he did an excellent job of that. Technical books, especially those covering Linux, have a tendency to get stale rather quickly, but TLPI shouldn't suffer from that as much as a kernel internals book would, for example. There should really only be additions down the road as the user-space API is maintained by the kernel developers "forever", but updates will presumably need to be made eventually.

There are a handful of additional complaints I could make about the book, but they are all quite minor, as were those mentioned above. The biggest nit is that the "asides" in the text, which are numerous, are really often much more than just asides. Each is set off from the rest of text, indented and rendered in a slightly smaller font (which is typographically a bit annoying to me), and are meant to contain additional information that is not necessarily critical to understanding the topic. In my experience, though, many of them might best have been worked into the main text. See what I mean about minor complaints?

This is a book that will be useful to application and system-level developers, primarily, but there is much of interest for others as well. Kernel hackers will find it useful to ensure their new feature (or fix) doesn't break the existing API. Programmers who are primarily targeting other Unix systems may also find it useful for making their code more portable. I found it to be extremely useful and expect to return to it frequently. Anyone who has an interest in programming for Linux will likely feel the same way.

Comments (31 posted)

Tarsnap advisory provides a few lessons

An interesting and brutally honest security advisory for the Tarsnap "secure online backup service" was released on January 18. It certainly shows a refreshing amount of candor that other projects and companies would do well to emulate. But there are some other lessons to be learned from the vulnerability including the value of source code availability and the perils of refactoring.

Tarsnap is a company founded by Colin Percival that provides encrypted online storage for backups. The client code is available, but it is not free software. The code can only be used, unmodified, to talk to the Tarsnap service. The server code is evidently completely unavailable, but Percival is interested in hearing from folks with ideas for improvement to the client—or those who have found a security hole.

Percival was contacted by Taylor R. Campbell on January 14 with just such a bug. It turns out that a refactoring of the code for the 1.0.22 release, which was made in June 2009, introduced a bug that potentially would allow anyone with access to the data to decrypt it. The data is stored in the Amazon S3 "cloud", which limits the access to a small group, but that doesn't really fit well with the security model espoused by Tarsnap. In the advisory, Percival makes that clear:

In doing the refactoring, Percival removed an auto-increment of a nonce value used in the Advanced Encryption Standard (AES) Counter (CTR) mode for encrypting blocks of data. The impact of that is that someone can decrypt the data without having the key.

There are two ways that the decryption could be done when the nonce value is reused, either by comparing two ciphertexts or by using known plaintext. The former attack is considered by Percival to be unusable on the Tarsnap data because of the compression done to the data blocks before they are encrypted. On the other hand, known plaintext attacks are quite plausible if there is some known data in the blocks. As Percival points out, full backups are likely to have any number of files with known contents, namely the files that are installed by the operating system—binaries, configuration files, and so on.

The bug was found by Campbell by "reading the Tarsnap source code purely out of curiosity", which certainly shows the advantage of making that source available. One wonders if the server code might also benefit from curious hackers. Percival is creating a bug bounty program (and seemingly retroactively paying one out to Campbell) to hopefully ferret out any other problems in the client sooner.

Refactoring is meant to be strictly a clean-up operation that does not change the semantics of the code in question. When doing refactoring, it is helpful if there are a set of regression tests that can detect when refactoring has gone awry. In the comments on the advisory, Percival said that Tarsnap does not have a test suite of that sort, and pointed out that it is difficult to create one for cryptographic software, but "I should probably find some way of automatically testing and/or assert()ing for nonce-reuse bugs though".

The lack of regression tests is unfortunate, but Tarsnap is hardly alone in that. There are countless projects that refactor their code without such a test suite. This particular incident should serve as something of a reminder to projects, especially those that are implementing security features, that refactoring can and does introduce bugs. A test suite is great, but even just some regression testing of the areas that have been refactored may find bugs like this one.

Percival is to be congratulated for quickly turning around a fix for the problem, as well as for being so forthright with the gory details of the bug and its impact. It is far too often that we see companies trying to sweep the details of their security holes under the rug—free software projects sometimes do as well. Bugs happen, security or otherwise, and there is value in seeing what they are and how they came about. We can learn from incidents like this.

Comments (12 posted)

A critical security bug in tarsnap

The author of tarsnap ("online backups for the truly paranoid") has sent out an advisory describing a "critical" security bug in versions 1.0.22 through 1.0.27. "It may be possible for me, Amazon, or US government agencies with access to Amazon's datacenters to decrypt data stored with those versions of Tarsnap. This is an absolutely unacceptable compromise of Tarsnap's security principles, and I sincerely apologize to everyone affected." The posting describes how to respond to the problem and is an interesting discussion of how easily things can go wrong in security-related code.

Comments (12 posted)

ccid: arbitrary code execution

Package(s):	ccid	CVE #(s):	CVE-2010-4530
Created:	January 14, 2011	Updated:	January 22, 2014
Description:	From the Red Hat bugzilla: An integer overflow, leading to array index error was found in the way USB CCID (Chip/Smart Card Interface Devices) driver processed certain values of card serial number. A local attacker could use this flaw to execute arbitrary code, with the privileges of the user running the pcscd daemon, via a malicious smart card with specially-crafted value of its serial number, inserted to the system USB port.
Alerts:	Gentoo 201401-16 ccid 2014-01-21 Scientific Linux SLSA-2013:1323-1 ccid 2013-10-10 Oracle ELSA-2013-1323 ccid 2013-10-02 Red Hat RHSA-2013:1323-01 ccid 2013-09-30 CentOS CESA-2013:0523 ccid 2013-03-09 Scientific Linux SL-ccid-20130304 ccid 2013-03-04 Oracle ELSA-2013-0523 ccid 2013-02-25 Red Hat RHSA-2013:0523-02 ccid 2013-02-21 SUSE SUSE-SR:2011:003 gnutls, tomcat6, perl-CGI-Simple, pcsc-lite, obs-server, dhcp, java-1_6_0-openjdk, opera 2011-02-08 Pardus 2011-22 ccid 2011-02-02 openSUSE openSUSE-SU-2011:0092-1 pcsc-ccid pcsc-lite 2011-02-02 Mandriva MDVSA-2011:014 ccid 2011-01-20 Fedora FEDORA-2011-0143 ccid 2011-01-05 Fedora FEDORA-2011-0162 ccid 2011-01-05

Comments (none posted)

chromium: mysterious vulnerabilities

Package(s):	chromium	CVE #(s):	CVE-2010-2898 CVE-2010-2899 CVE-2010-2900 CVE-2010-2901 CVE-2010-2902 CVE-2010-2903
Created:	January 19, 2011	Updated:	August 23, 2011
Description:	The chromium browser suffers from a number of "unspecified" vulnerabilities with "unknown" impact.
Alerts:	Gentoo 201412-09 racer-bin, fmod, PEAR-Mail, lvm2, gnucash, xine-lib, lastfmplayer, webkit-gtk, shadow, PEAR-PEAR, unixODBC, resource-agents, mrouted, rsync, xmlsec, xrdb, vino, oprofile, syslog-ng, sflowtool, gdm, libsoup, ca-certificates, gitolite, qt-creator 2014-12-11 Ubuntu USN-1195-1 webkit 2011-08-23 SUSE SUSE-SR:2011:009 mailman, openssl, tgt, rsync, vsftpd, libzip1/libzip-devel, otrs, libtiff, kdelibs4, libwebkit, libpython2_6-1_0, perl, pure-ftpd, collectd, vino, aaa_base, exim 2011-05-17 Debian DSA-2188-1 webkit 2011-03-10 openSUSE openSUSE-SU-2011:0482-1 webkit 2011-05-13 Fedora FEDORA-2011-1224 webkitgtk 2011-02-09 MeeGo MeeGo-SA-10:23 chromium 2010-09-03

Comments (2 posted)

gif2png: denial of service

Package(s):	gif2png	CVE #(s):	CVE-2010-4694
Created:	January 17, 2011	Updated:	March 16, 2012
Description:	From the Mandriva advisory: Buffer overflow in gif2png.c in gif2png 2.5.3 and earlier might allow context-dependent attackers to cause a denial of service (application crash) or have unspecified other impact via a GIF file that contains many images, leading to long extensions such as .p100 for PNG output files, as demonstrated by a CGI program that launches gif2png, a different vulnerability than CVE-2009-5018.
Alerts:	Gentoo 201203-15 gif2png 2012-03-16 Mandriva MDVSA-2011:009 gif2png 2011-01-14

Comments (none posted)

hplip: arbitrary code execution

Package(s):	hplip	CVE #(s):	CVE-2010-4267
Created:	January 18, 2011	Updated:	August 22, 2013
Description:	From the Red Hat advisory: A flaw was found in the way certain HPLIP tools discovered devices using the SNMP protocol. If a user ran certain HPLIP tools that search for supported devices using SNMP, and a malicious user is able to send specially-crafted SNMP responses, it could cause those HPLIP tools to crash or, possibly, execute arbitrary code with the privileges of the user running them.
Alerts:	Slackware SSA:2013-233-01 hplip 2013-08-21 Oracle ELSA-2013-0133 hplip3 2013-01-12 Gentoo 201203-17 hplip 2012-03-16 CentOS CESA-2011:0154 hplip 2011-04-14 CentOS CESA-2011:0154 hplip 2011-04-14 SUSE SUSE-SR:2011:005 hplip, perl, subversion, t1lib, bind, tomcat5, tomcat6, avahi, gimp, aaa_base, build, libtiff, krb5, nbd, clamav, aaa_base, flash-player, pango, openssl, subversion, postgresql, logwatch, libxml2, quagga, fuse, util-linux 2011-04-01 Pardus 2011-33 hplip 2011-02-12 Debian DSA-2152-1 hplip 2011-01-27 Fedora FEDORA-2011-0524 hplip 2011-01-18 Fedora FEDORA-2011-0525 hplip 2011-01-18 Ubuntu USN-1051-1 hplip 2011-01-25 openSUSE openSUSE-SU-2011:0068-1 hplip 2011-01-21 Mandriva MDVSA-2011:013 hplip 2011-01-19 Red Hat RHSA-2011:0154-01 hplip 2011-01-17 SUSE SUSE-SR:2011:002 ed, evince, hplip, libopensc2/opensc, libsmi, libwebkit, perl, python, sssd, sudo, wireshark 2011-01-25

Comments (none posted)

java-1_6_0-openjdk: security manager bypass

Package(s):	java-1_6_0-openjdk	CVE #(s):	CVE-2010-4351
Created:	January 19, 2011	Updated:	April 21, 2011
Description:	The IcedTea JNLP security manager implementation will, in some cases, fail to throw an expected exception when security policy is violated.
Alerts:	Gentoo 201406-32 icedtea-bin 2014-06-29 CentOS CESA-2011:0176 java-1.6.0-openjdk 2011-04-14 Mandriva MDVSA-2011:054 java-1.6.0-openjdk 2011-03-27 Debian DSA-2224-1 openjdk-6 2011-04-20 Ubuntu USN-1055-1 openjdk-6, openjdk-6b18 2011-02-01 Ubuntu USN-1052-1 openjdk-6, openjdk-6b18 2011-01-26 Red Hat RHSA-2011:0176-01 java-1.6.0-openjdk 2011-01-25 Fedora FEDORA-2011-0521 java-1.6.0-openjdk 2011-01-18 Fedora FEDORA-2011-0500 java-1.6.0-openjdk 2011-01-18 openSUSE openSUSE-SU-2011:0057-1 java-1_6_0-openjdk 2011-01-19

Comments (none posted)

kernel: denial of service

Package(s):	kernel	CVE #(s):	CVE-2010-4526
Created:	January 19, 2011	Updated:	September 13, 2011
Description:	Yet another bug in the SCTP network protocol code allows a remote attacker to oops the kernel.
Alerts:	Ubuntu USN-1204-1 linux-fsl-imx51 2011-09-13 Red Hat RHSA-2011:1253-01 kernel-rt 2011-09-12 Ubuntu USN-1170-1 linux 2011-07-15 CentOS CESA-2011:0163 kernel 2011-04-14 Red Hat RHSA-2011:0421-01 kernel 2011-04-07 Ubuntu USN-1093-1 linux-mvl-dove 2011-03-25 SUSE SUSE-SA:2011:015 kernel 2011-03-24 SUSE SUSE-SA:2011:012 kernel 2011-03-08 Ubuntu USN-1080-2 linux-ec2 2011-03-02 Ubuntu USN-1080-1 linux 2011-03-01 Debian DSA-2153-1 linux-2.6 kernel 2011-01-30 Red Hat RHSA-2011:0163-01 kernel 2011-01-18

Comments (none posted)

kernel: multiple vulnerabilities

Package(s):	kernel	CVE #(s):	CVE-2010-4238 CVE-2010-4243 CVE-2010-4255 CVE-2010-4343
Created:	January 13, 2011	Updated:	September 14, 2011
Description:	From the Red Hat advisory: * A missing sanity check was found in vbd_create() in the Xen hypervisor implementation. As CD-ROM drives are not supported by the blkback back-end driver, attempting to use a virtual CD-ROM drive with blkback could trigger a denial of service (crash) on the host system running the Xen hypervisor. (CVE-2010-4238, Moderate) * A flaw was found in the Linux kernel execve() system call implementation. A local, unprivileged user could cause large amounts of memory to be allocated but not visible to the OOM (Out of Memory) killer, triggering a denial of service. (CVE-2010-4243, Moderate) * A flaw was found in fixup_page_fault() in the Xen hypervisor implementation. If a 64-bit para-virtualized guest accessed a certain area of memory, it could cause a denial of service on the host system running the Xen hypervisor. (CVE-2010-4255, Moderate) * A missing initialization flaw was found in the bfa driver used by Brocade Fibre Channel Host Bus Adapters. A local, unprivileged user could use this flaw to cause a denial of service by reading a file in the "/sys/class/fc_host/host#/statistics/" directory. (CVE-2010-4343, Moderate)
Alerts:	Oracle ELSA-2013-1645 kernel 2013-11-26 openSUSE openSUSE-SU-2013:0927-1 kernel 2013-06-10 Ubuntu USN-1204-1 linux-fsl-imx51 2011-09-13 Ubuntu USN-1202-1 linux-ti-omap4 2011-09-13 Red Hat RHSA-2011:1253-01 kernel-rt 2011-09-12 Ubuntu USN-1186-1 kernel 2011-08-09 Ubuntu USN-1167-1 linux 2011-07-13 Ubuntu USN-1159-1 linux-mvl-dove 2011-07-13 Ubuntu USN-1162-1 linux-mvl-dove 2011-06-29 Ubuntu USN-1141-1 linux, linux-ec2 2011-05-31 SUSE SUSE-SA:2011:017 kernel 2011-04-18 openSUSE openSUSE-SU-2011:0346-1 kernel 2011-04-18 Ubuntu USN-1093-1 linux-mvl-dove 2011-03-25 SUSE SUSE-SA:2011:012 kernel 2011-03-08 Ubuntu USN-1080-2 linux-ec2 2011-03-02 Ubuntu USN-1080-1 linux 2011-03-01 openSUSE openSUSE-SU-2011:0399-1 kernel 2011-04-28 Red Hat RHSA-2011:0283-01 kernel 2011-02-22 Debian DSA-2153-1 linux-2.6 kernel 2011-01-30 Red Hat RHSA-2011:0017-01 kernel 2011-01-13

Comments (none posted)

libtiff: denial of service

Package(s):	libtiff	CVE #(s):	CVE-2010-2596 CVE-2010-2630 CVE-2010-2631 CVE-2010-2482
Created:	January 19, 2011	Updated:	March 15, 2011
Description:	The libtiff library contains a number of flaws which can be exploited to crash a running application.
Alerts:	Debian-LTS DLA-610-1 tiff3 2016-09-05 Arch Linux ASA-201611-26 libtiff 2016-11-25 Arch Linux ASA-201611-27 lib32-libtiff 2016-11-25 Scientific Linux SLSA-2014:0222-1 libtiff 2014-02-27 Red Hat RHSA-2014:0222-01 libtiff 2014-02-27 Oracle ELSA-2014-0222 libtiff 2014-02-27 CentOS CESA-2014:0222 libtiff 2014-02-28 Debian DSA-2552-1 tiff 2012-09-26 Gentoo 201209-02 tiff 2012-09-23 Ubuntu USN-1085-2 tiff 2011-03-15 Ubuntu USN-1085-1 tiff 2011-03-07 MeeGo MeeGo-SA-10:27 libtiff 2010-09-03

Comments (none posted)

mydms: directory traversal

Package(s):	mydms	CVE #(s):	CVE-2010-2006
Created:	January 17, 2011	Updated:	January 19, 2011
Description:	From the Debian advisory: D. Fabian and L. Weichselbaum discovered a directory traversal vulnerability in MyDMS, a open-source document management system based on PHP and MySQL.
Alerts:	Debian DSA-2146-1 mydms 2011-01-16

Comments (none posted)

pcsc-lite: arbitrary code execution

Package(s):	pcsc-lite	CVE #(s):	CVE-2010-4531
Created:	January 14, 2011	Updated:	January 22, 2014
Description:	From the Red Hat bugzilla: A stack-based buffer overflow flaw was found in the way PC/SC Lite smart card framework decoded certain attribute values of the Answer-to-Reset (ATR) message, received back from the card after connecting. A local attacker could use this flaw to execute arbitrary code with the privileges of the user running the pcscd daemon, via a malicious smart card inserted to the system USB port.
Alerts:	Gentoo 201401-17 pcsc-lite 2014-01-21 CentOS CESA-2013:0525 pcsc-lite 2013-03-09 Scientific Linux SL-pcsc-20130228 pcsc-lite 2013-02-28 Oracle ELSA-2013-0525 pcsc-lite 2013-02-25 Red Hat RHSA-2013:0525-02 pcsc-lite 2013-02-21 SUSE SUSE-SR:2011:003 gnutls, tomcat6, perl-CGI-Simple, pcsc-lite, obs-server, dhcp, java-1_6_0-openjdk, opera 2011-02-08 Ubuntu USN-1125-1 pcsc-lite 2011-04-27 Pardus 2011-24 pcsc-lite 2011-02-02 openSUSE openSUSE-SU-2011:0092-1 pcsc-ccid pcsc-lite 2011-02-02 Debian DSA-2156-1 pcscd 2011-01-31 Mandriva MDVSA-2011:015 pcsc-lite 2011-01-20 Fedora FEDORA-2011-0164 pcsc-lite 2011-01-05 Fedora FEDORA-2011-0123 pcsc-lite 2011-01-05

Comments (none posted)

perl-CGI: HTTP response splitting attacks

Package(s):	perl-CGI	CVE #(s):	CVE-2010-4411
Created:	January 17, 2011	Updated:	January 31, 2011
Description:	From the Mandriva advisory: Unspecified vulnerability in CGI.pm 3.50 and earlier allows remote attackers to inject arbitrary HTTP headers and conduct HTTP response splitting attacks via unknown vectors. NOTE: this issue exists because of an incomplete fix for CVE-2010-2761.
Alerts:	Gentoo 201110-03 bugzilla 2011-10-10 Fedora FEDORA-2011-0653 perl-CGI-Simple 2011-01-21 Fedora FEDORA-2011-0631 perl-CGI-Simple 2011-01-21 openSUSE openSUSE-SU-2011:0083-1 perl-CGI-Simple 2011-01-28 openSUSE openSUSE-SU-2011:0064-1 perl 2011-01-20 Mandriva MDVSA-2011:008 perl-CGI 2011-01-14 SUSE SUSE-SR:2011:002 ed, evince, hplip, libopensc2/opensc, libsmi, libwebkit, perl, python, sssd, sudo, wireshark 2011-01-25

Comments (none posted)

pimd: insecure temporary files

Package(s):	pimd	CVE #(s):	CVE-2011-0007
Created:	January 17, 2011	Updated:	January 19, 2011
Description:	From the Debian advisory: Vincent Bernat discovered that pimd, a multicast routing daemon, creates files with predictable names upon the receipt of particular signals.
Alerts:	Gentoo 201412-09 racer-bin, fmod, PEAR-Mail, lvm2, gnucash, xine-lib, lastfmplayer, webkit-gtk, shadow, PEAR-PEAR, unixODBC, resource-agents, mrouted, rsync, xmlsec, xrdb, vino, oprofile, syslog-ng, sflowtool, gdm, libsoup, ca-certificates, gitolite, qt-creator 2014-12-11 Debian DSA-2147-1 pimd 2011-01-16

Comments (none posted)

prewikka: password leak

Package(s):	prewikka	CVE #(s):	CVE-2010-2058
Created:	January 17, 2011	Updated:	January 19, 2011
Description:	From the CVE entry: setup.py in Prewikka 0.9.14 installs prewikka.conf with world-readable permissions, which allows local users to obtain the SQL database password.
Alerts:	Gentoo 201101-07 prewikka 2011-01-16

Comments (none posted)

sssd: denial of service

Package(s):	sssd	CVE #(s):	CVE-2010-4341
Created:	January 19, 2011	Updated:	September 23, 2011
Description:	Sssd suffers from a bug in pam_parse_in_data_v2() which allows a local attacker to prevent other users from logging into the system.
Alerts:	CentOS CESA-2011:0975 sssd 2011-09-22 Scientific Linux SL-sssd-20110721 sssd 2011-07-21 Red Hat RHSA-2011:0975-01 sssd 2011-07-21 Scientific Linux SL-sssd-20110519 sssd 2011-05-19 Red Hat RHSA-2011:0560-01 sssd 2011-05-19 Fedora FEDORA-2011-0364 sssd 2011-01-13 openSUSE openSUSE-SU-2011:0058-1 sssd 2011-01-19 SUSE SUSE-SR:2011:002 ed, evince, hplip, libopensc2/opensc, libsmi, libwebkit, perl, python, sssd, sudo, wireshark 2011-01-25 Fedora FEDORA-2011-0337 sssd 2011-01-13

Comments (none posted)

subversion: denial of service

Package(s):	subversion	CVE #(s):	CVE-2010-4539 CVE-2010-4644
Created:	January 14, 2011	Updated:	April 15, 2011
Description:	From the Mandriva advisory: The walk function in repos.c in the mod_dav_svn module for the Apache HTTP Server, as distributed in Apache Subversion before 1.6.15, allows remote authenticated users to cause a denial of service (NULL pointer dereference and daemon crash) via vectors that trigger the walking of SVNParentPath collections (CVE-2010-4539). Multiple memory leaks in rev_hunt.c in Apache Subversion before 1.6.15 allow remote authenticated users to cause a denial of service (memory consumption and daemon crash) via the -g option to the blame command (CVE-2010-4644).
Alerts:	openSUSE openSUSE-SU-2013:1869-1 subversion 2013-12-13 Gentoo 201309-11 subversion 2013-09-23 CentOS CESA-2011:0257 subversion 2011-04-14 SUSE SUSE-SR:2011:005 hplip, perl, subversion, t1lib, bind, tomcat5, tomcat6, avahi, gimp, aaa_base, build, libtiff, krb5, nbd, clamav, aaa_base, flash-player, pango, openssl, subversion, postgresql, logwatch, libxml2, quagga, fuse, util-linux 2011-04-01 openSUSE openSUSE-SU-2011:0136-1 subversion 2011-02-25 Red Hat RHSA-2011:0257-01 subversion 2011-02-15 Red Hat RHSA-2011:0258-01 subversion 2011-02-15 Pardus 2011-32 subversion mod_dav_svn 2011-02-12 Ubuntu USN-1053-1 subversion 2011-02-01 Fedora FEDORA-2011-0099 subversion 2011-01-04 Mandriva MDVSA-2011:006 subversion 2011-01-14

Comments (none posted)

sudo: group-related vulnerabilities

Package(s):	sudo	CVE #(s):	CVE-2011-0008 CVE-2011-0010
Created:	January 19, 2011	Updated:	March 22, 2012
Description:	It turns out that sudo does not ask for a password on group ID changes. CVE-2011-0008 is the return of CVE-2009-0034 (another group-oriented vulnerability) as the result of upstream changes.
Alerts:	Scientific Linux SL-sudo-20120321 sudo 2012-03-21 Oracle ELSA-2012-0309 sudo 2012-03-07 Gentoo 201203-06 sudo 2012-03-05 Red Hat RHSA-2012:0309-03 sudo 2012-02-21 Pardus 2011-31 sudo 2011-02-12 Slackware SSA:2011-041-05 sudo 2011-02-11 Red Hat RHSA-2011:0599-01 sudo 2011-05-19 Mandriva MDVSA-2011:018 sudo 2011-01-21 Ubuntu USN-1046-1 sudo 2011-01-20 openSUSE openSUSE-SU-2011:0050-1 sudo 2011-01-19 SUSE SUSE-SR:2011:002 ed, evince, hplip, libopensc2/opensc, libsmi, libwebkit, perl, python, sssd, sudo, wireshark 2011-01-25 Fedora FEDORA-2011-0455 sudo 2011-01-17 Fedora FEDORA-2011-0470 sudo 2011-01-17

Comments (none posted)

tor: multiple vulnerabilities

Package(s):	tor	CVE #(s):	CVE-2011-0427
Created:	January 17, 2011	Updated:	June 9, 2011
Description:	From the Debian advisory: The developers of Tor, an anonymizing overlay network for TCP, found three security issues during a security audit. A heap overflow allowed the execution of arbitrary code (CVE-2011-0427), a denial of service vulnerability was found in the zlib compression handling and some key memory was incorrectly zeroed out before being freed. The latter two issues do not yet have CVE identifiers assigned.
Alerts:	Gentoo 201110-13 tor 2011-10-18 Fedora FEDORA-2011-0650 tor 2011-01-21 Fedora FEDORA-2011-0642 tor 2011-01-21 Debian DSA-2148-1 tor 2011-01-17

Comments (none posted)

wireshark: arbitrary code execution

Package(s):	wireshark	CVE #(s):	CVE-2011-0444
Created:	January 14, 2011	Updated:	April 19, 2011
Description:	From the Mandriva advisory: Buffer overflow in the MAC-LTE dissector (epan/dissectors/packet-mac-lte.c) in Wireshark 1.2.0 through 1.2.13 and 1.4.0 through 1.4.2 allows remote attackers to cause a denial of service (crash) and possibly execute arbitrary code via a large number of RARs.
Alerts:	Gentoo 201110-02 wireshark 2011-10-09 SUSE SUSE-SR:2011:007 NetworkManager, OpenOffice_org, apache2-slms, dbus-1-glib, dhcp/dhcpcd/dhcp6, freetype2, kbd, krb5, libcgroup, libmodplug, libvirt, mailman, moonlight-plugin, nbd, openldap2, pure-ftpd, python-feedparser, rsyslog, telepathy-gabble, wireshark 2011-04-19 Red Hat RHSA-2011:0369-01 wireshark 2011-03-21 Fedora FEDORA-2011-0450 wireshark 2011-01-17 Fedora FEDORA-2011-0460 wireshark 2011-01-17 Pardus 2011-21 wireshark 2011-01-31 Mandriva MDVSA-2011:007 wireshark 2011-01-14

Comments (none posted)

xfig: multiple vulnerabilities

Package(s):	xfig	CVE #(s):	CVE-2009-4227 CVE-2009-4228
Created:	January 17, 2011	Updated:	December 15, 2014
Description:	From the Mandriva advisory: Stack-based buffer overflow in the read_1_3_textobject function in f_readold.c in Xfig 3.2.5b and earlier, and in the read_textobject function in read1_3.c in fig2dev in Transfig 3.2.5a and earlier, allows remote attackers to execute arbitrary code via a long string in a malformed .fig file that uses the 1.3 file format. NOTE: some of these details are obtained from third party information (CVE-2009-4227). Stack consumption vulnerability in u_bound.c in Xfig 3.2.5b and earlier allows remote attackers to cause a denial of service (application crash) via a long string in a malformed .fig file that uses the 1.3 file format, possibly related to the readfp_fig function in f_read.c (CVE-2009-4228).
Alerts:	Gentoo 201412-14 xfig 2014-12-13 Fedora FEDORA-2012-11737 transfig 2012-08-27 Fedora FEDORA-2012-11718 transfig 2012-08-27 Fedora FEDORA-2012-11801 xfig 2012-08-22 Fedora FEDORA-2012-11813 xfig 2012-08-22 Mandriva MDVSA-2011:010 xfig 2011-01-15

Comments (none posted)

Kernel release status

The current development kernel is 2.6.38-rc1, released on January 18. "It's been two weeks, and the merge window for 2.6.38 is thus closed. And an interesting merge window it has been." All told, a little over 7,600 changes were merged in this merge window. Some of them affect core code in fairly invasive ways, and there have been some significant regressions reported already. People testing 2.6.38-rc1 might want to be just a little more careful (and better backed up) than usual. See the full changelog for all the details.

Stable updates: there have been no stable updates released in the last week.

Comments (none posted)

Quotes of the week

-- Matthew Garrett

-- David Woodhouse

Comments (3 posted)

Bypassing linux-next

It has been almost three years since the creation of the linux-next tree; during that time, it has become an indispensable part of the kernel development process. By the time code is merged into the mainline during the merge window, it has already seen a fair amount of integration and compilation testing in linux-next - and even some actual run testing. That has helped to make the merge window run more smoothly. So it's not surprising that developers are getting increasingly grumpy when code is seen to be circumventing linux-next and creating problems in the mainline.

We've had a couple of examples of that grumpiness in the 2.6.38 cycle. When Al Viro posted his first VFS pull request, linux-next maintainer Stephen Rothwell complained that this was his first sighting of that code, despite the fact that it had apparently been around for a few months. Al is known for pulling together mainline submissions at the last minute, so this sort of thing is not entirely surprising; it remains to be seen whether he can be pushed into changing his ways.

The other complaint came after the merging of the transparent huge pages patch set, which went in by way of Andrew Morton's -mm tree. Tony Luck, having discovered that the ia64 architecture no longer built in the mainline, asked:

Andrew responded that "It's taking a while - Stephen and I are discussing a plan." Integrating -mm was always going to be a bit of a challenge; linux-next is supposed to contain code which is ready for merging into the mainline, while -mm can carry under-development code for years. Until that gets worked out, though, memory management developers are going to be in a bit of a difficult position; there is no maintainer tree they can get into which feeds into linux-next. Those developers will need to either get their own trees into linux-next (an easy thing to do) or take the complaints when code which lived in -mm is seen by testers for the first time when it hits the mainline.

Comments (2 posted)

2.6.38 merge window part 2

As of the 2.6.38-rc1 release, some 7616 non-merge changesets had been pulled into the mainline kernel. A number of significant changes have been merged since last week's summary; the most interesting changes visible to users are:

• The transparent huge pages feature has been merged. THP attempts to maximize the use of huge pages in the system (boosting performance) without requiring application changes or administrator overhead.

• A new tool called turbostat has been added; it can be used to obtain various types of performance statistics from Intel processors. Also added is x86_energy_perf_policy, which can be used to tweak the performance/power usage tradeoff on Intel CPUs.

• The taskstats API has been changed to use different alignments for returned values; this may break applications which were dependent on the old arrangement.

• The kernel can now synchronize its internal time to an external pulse-per-second (PPS) signal with a high degree of accuracy. The kernel has also gained the ability to generate (and accept) PPS signals on a parallel port, assuming one can still find a computer with such a port.

• The x86 architecture can now boot XZ-compressed kernels.

• Basic support for multitouch panels has been added to the human input devices (HID) layer.

• The kernel now has support for the RFC4106 AES-GCM cryptographic algorithm.

• The fallocate() system call can now be used to punch holes in the middle of files. Currently this feature is supported by XFS and OCFS2.

• The XFS filesystem supports the FITRIM ioctl(), allowing discard operations to be initiated from user space. "This is not intended to be run during normal workloads, as the freepsace btree walks can cause large performance degradation."

• The LIO SCSI target core has been merged.

• The block I/O bandwidth controller can now be used with hierarchical control groups.

• The block layer has a new "events handling" mechanism. What that means is that detection of device events (the insertion of an optical disc, for example) can be done in the drivers, eliminating the need to poll devices from user space.

• The device mapper dm-crypt target has a new "multikey" mode whereby different blocks can be encrypted with different keys. The crypt target is also now able to access encrypted partitions created with the out-of-tree loop-AES package.

• The device mapper has gained the ability to manage RAID 4/5/6 volumes using the MD RAID drivers.

• The clone() system call no longer honors the long-deprecated CLONE_STOPPED flag.

• The btrfs filesystem has gained support for read-only snapshots and LZO compression.

• New drivers:
  • Systems and processors: ALPHAPROJECT AP-SH4A-3A and AP-SH4AD-0A reference boards, Acme Systems srl FOX G20 boards, GeoSIG GS_IA18_S boards, and Atheros AR71XX/AR724X/AR931X SoCs.

  • Audio: HP t5325 audio devices, Realtek alc562x codecs, Wolfson Micro WM8770 and WM8995 codecs, Wolfson Micro WM8958 multi-band compressors, and Wolfson Micro WM8737 analog-to-digital converters.

  • Input: Austria Microsystem AS5011 joysticks.

  • Miscellaneous: NXP Semiconductors PN544 near-field communication chips, Oki Semiconductor ML7213 IOH GPIO controllers, Freescale MC13892 PMIC regulators, Freescale MCF548x watchdog timers, TI TPS6524X Power regulators, AMD/ATI SP5100 TCO timer/watchdog chipsets, Atheros AR71XX/AR724X/AR913X hardware watchdogs, nVidia TCO timer/watchdog devices, Intel EG20T platform controller hubs, and Maxim MAX17042/8997/8966 fuel gauges.

Changes visible to kernel developers include:

• ktest.pl, a script which can automate the process of building, testing, and bisecting kernels, has been added to the tools directory.

• The "%pK" format specifier can be used to print the value of potentially sensitive kernel pointers, especially in places like /proc files. The behavior of this specifier depends on the value of /proc/sys/kernel/kptr_restrict; a value of zero means that kernel pointers will be printed as usual, one causes pointers to be printed as zero for users without CAP_SYSLOG, and two hides the pointers for all users.

• cdev_index() has been removed; since there are no in-kernel users, nobody is likely to notice.

• The new function kref_test_and_get() will take a reference only if the current reference count is not zero.

• Some new dentry operations have been added to support automounters within the VFS.

• The fallocate() filesystem callback has been moved from struct inode_operations to struct file_operations.

With the 2.6.38 feature set complete, the process of stabilizing all of this new code can continue; expect a final 2.6.38 release sometime in late March.

Comments (4 posted)

Transparent huge pages in 2.6.38

The memory management unit in almost any contemporary processor can handle multiple page sizes, but the Linux kernel almost always restricts itself to just the smallest of those sizes - 4096 bytes on most architectures. Pages which are larger than that minimum - collectively called "huge pages" - can offer better performance for some workloads, but that performance benefit has gone mostly unexploited on Linux. That may change in 2.6.38, though, with the merging of the transparent huge page feature.

Huge pages can improve performance through reduced page faults (a single fault brings in a large chunk of memory at once) and by reducing the cost of virtual to physical address translation (fewer levels of page tables must be traversed to get to the physical address). But the real advantage comes from avoiding translations altogether. If the processor must translate a virtual address, it must go through as many as four levels of page tables, each of which has a good chance of being cache-cold, and, thus, slow. For this reason, processors maintain a "translation lookaside buffer" (TLB) to cache the results of translations. The TLB is often quite small; running cpuid on your editor's aging desktop machine yields:

   cache and TLB information (2):
      0xb1: instruction TLB: 2M/4M, 4-way, 4/8 entries
      0xb0: instruction TLB: 4K, 4-way, 128 entries
      0x05: data TLB: 4M pages, 4-way, 32 entries

So there is room for 128 instruction translations, and 32 data translations. Such a small cache is easily overrun, forcing the CPU to perform large numbers of address translations. A single 2MB huge page requires a single TLB entry; the same memory, in 4KB pages, would need 512 TLB entries. Given that, it's not surprising that the use of huge pages can make programs run faster.

The main kernel address space is mapped with huge pages, reducing TLB pressure from kernel code. The only way for user-space to take advantage of huge pages in current kernels, though, is through the hugetlbfs, which was extensively documented here in early 2010. Using hugetlbfs requires significant work from both application developers and system administrators; huge pages must be set aside at boot time, and applications must map them explicitly. The process is fiddly enough that use of hugetlbfs is restricted to those who really care and who have the time to mess with it. Hugetlbfs is often seen as a feature for large, proprietary database management systems and little else.

There would be real value in a mechanism which would make the use of huge pages easy, preferably requiring no development or administrative attention at all. That is the goal of the transparent huge pages (THP) patch, which was written by Andrea Arcangeli and merged for 2.6.38. In short, THP tries to make huge pages "just happen" in situations where they would be useful.

Current Linux kernels assume that all pages found within a given virtual memory area (VMA) will be the same size. To make THP work, Andrea had to start by getting rid of that assumption; thus, much of the initial part of the patch series is dedicated to enabling mixed page sizes within a VMA. Then the patch modifies the page fault handler in a simple way: when a fault happens, the kernel will attempt to allocate a huge page to satisfy it. Should the allocation succeed, the huge page will be filled, any existing small pages in the new page's address range will be released, and the huge page will be inserted into the VMA. If no huge pages are available, the kernel falls back to small pages and the application never knows the difference.

This scheme will increase the use of huge pages transparently, but it does not yet solve the whole problem. Huge pages must be swappable, lest the system run out of memory in a hurry. Rather than complicate the swapping code with an understanding of huge pages, Andrea simply splits a huge page back into its component small pages if that page needs to be reclaimed. Many other operations (mprotect(), mlock(), ...) will also result in the splitting of a page.

The allocation of huge pages depends on the availability of large, physically-contiguous chunks of memory - something which Linux kernel programmers can never count on. It is to be expected that those pages will become available at inconvenient times - just after a process has faulted in a number of small pages, for example. The THP patch tries to improve this situation through the addition of a "khugepaged" kernel thread. That thread will occasionally attempt to allocate a huge page; if it succeeds, it will scan through memory looking for a place where that huge page can be substituted for a bunch of smaller pages. Thus, available huge pages should be quickly placed into service, maximizing the use of huge pages in the system as a whole.

The current patch only works with anonymous pages; the work to integrate huge pages with the page cache has not yet been done. It also only handles one huge page size (2MB). Even so, some useful performance improvements can be seen. Mel Gorman ran some benchmarks showing improvements of up to 10% or so in some situations. In general, the results were not as good as could be obtained with hugetlbfs, but THP is much more likely to actually be used.

No application changes need to be made to take advantage of THP, but interested application developers can try to optimize their use of it. A call to madvise() with the MADV_HUGEPAGE flag will mark a memory range as being especially suited to huge pages, while MADV_NOHUGEPAGE will suggest that huge pages are better used elsewhere. For applications that want to use huge pages, use of posix_memalign() can help to ensure that large allocations are aligned to huge page (2MB) boundaries.

System administrators have a number of knobs that they can tweak, all found under /sys/kernel/mm/transparent_hugepage. The enabled value can be set to "always" (to always use THP), "madvise" (to use huge pages only in VMAs marked with MADV_HUGEPAGE), or "never" (to disable the feature). Another knob, defrag, takes the same values; it controls whether the kernel should make aggressive use of memory compaction to make more huge pages available. There's also a whole set of parameters controlling the operation of the khugepaged thread; see Documentation/vm/transhuge.txt for all the details.

The THP patch has had a bit of a rough ride since being merged into the mainline. This code never appeared in linux-next, so it surprised some architecture maintainers when it caused build failures in the mainline. Some bugs have also been found - unsurprising for a patch which is this large and which affects so much core code. Those problems are being ironed out, so, while 2.6.38-rc1 testers might want to be careful, THP should be in a usable state by the time the final 2.6.38 kernel is released.

Comments (8 posted)

Reworking disk events handling

A look inside any contemporary desktop-oriented system is likely to reveal a process which is steadfastly polling removable drives on the off chance that somebody might have removed or inserted a disk. Indeed, as your editor can attest, it can be hard to turn that polling off; there's little room in the world for strange people who have their own ideas of what they want to happen when they put a disk into a drive. Be that as it may, if the system is going to poll drives, it would be nice to do so in the best way possible. That is not currently the case, but, thanks to a patch by Tejun Heo, drive polling should be better in 2.6.38.

There are a few problems with how polling is done on Linux; these were nicely outlined by Tejun in the patch changelog. Polling on Linux requires opening the device; this is a somewhat heavyweight operation which does not naturally line up with other operations which might wake the processor. Opening the device in this way might interfere with other users; optical disk burning, in particular, is susceptible to this kind of problem. And polling the disk in this way generates a different set of commands than Windows uses; as Linux driver developers have discovered many times, behavior that differs from Windows is not well tested by vendors and tends to have unpleasant bugs. All that notwithstanding, user-space polling works well enough most of the time, but it would still be nice to make it better.

Tejun's patch works by moving the polling into the kernel. That makes the polling more efficient by removing the need to open the device and by allowing the kernel to delay polling wakeups until something else is going on as well. There is a new function added to struct block_device_operations which should be implemented by drivers:

    unsigned int (*check_events) (struct gendisk *disk, unsigned int clearing);

This function should check the device for new events and return a mask of any which were found. Two events are currently defined: DISK_EVENT_MEDIA_CHANGE and DISK_EVENT_EJECT_REQUEST, the latter of which is new. The clearing parameter is a mask of events which should be cleared until they happen again.

The old media_changed() block device operation still exists, but its use has been deprecated; drivers should be updated to use check_events() instead. Drivers should also, before adding a block device, initialize two new struct gendisk fields:

    unsigned int events;
    unsigned int async_events;

A mask of all events which can be reported by the device should be stored in events, while async_events should list the events which can be reported without needing to poll the device.

A new sysctl knob (block.events_dfl_poll_msecs) tells the kernel how often it should (by default) poll block devices. A value of zero (the default, currently) disables polling entirely. Polling intervals for specific devices can be set in their sysfs directories. If a device says that it can report all events asynchronously, and no polling interval has been explicitly set for it, that device will not be polled at all.

Since user space is no longer polling the device with this scheme, it needs a new way to find out when a disk event has happened. These events are now signaled via a uevent, meaning they can be handled via udev or some other utility which is watching those events. Note that any driver which handles asynchronous event reporting must call kobject_uevent_env() itself to send the event to user space. No driver in 2.6.38-rc1 does that; the first developer to add such a call may want to add a helper function to the core block code for that purpose.

Since polling is disabled by default, the kernel will behave the way it always has and existing user space applications will work. Once the user space environments have been changed to take advantage of this feature, they can turn on kernel polling and stop opening the devices themselves. That should lead to better power consumption and more reliable operation, which can only be a good thing.

Comments (2 posted)

Debian "Squeeze" release in sight

Debian 6.0 ("Squeeze") has now gone into "hard freeze," and the release candidate for the installer was announced on January 12. It would seem that a final release of Debian 6 is finally within sight, carrying a new installer, a completely free kernel, and an experimental Debian flavor based on FreeBSD for x86 and AMD64 systems.

The previous stable Debian release, Debian 5.0 or "Lenny," was announced on February 14, 2009. If two years between releases seems long, it's worth remembering that Debian 3.1 ("Sarge") was just under three years in the making. Debian 4.0 ("Etch") and 5.0 were each in development for about 22 months. Debian releases used to come at a much faster pace, but also didn't have to carry the same number of packages or ports. For instance, Debian 1.1 through 2.2 came out at fairly regular intervals, with roughly a year between releases — Debian had two major releases in 1996, and one every year from 1997 through 2000. Hamm (2.0 in 1998) was the first release to carry two ports, Slink (2.1) brought four ports, and Potato (2.2) came with six.

Lenny had official support for twelve processor architectures, but this has been winnowed down a bit in Squeeze. Specifically, it has removed support for Alpha, HP PA-RISC (hppa), and the original ARM port. The ARM EABI (armel) port supersedes ARM in Squeeze, and is described as "the more efficient successor" to ARM. It has quite a bit of overlap in terms of hardware support with the original ARM port — so it's really more of a transition than dropping a port altogether.

That leaves support for x86, AMD64, Itanium, MIPS, PowerPC, and SPARC. 6.0 adds a new port that merges the Debian userland packages with the FreeBSD kernel called Debian GNU/kFreeBSD. It is available in 32- and 64-bit version and, though it's considered a "technology preview", it is usable.

A glance through the "What's new in Debian GNU/Linux 6.0" guide (which is far more comprehensive than one usually sees for distribution releases) will look very familiar in spots. That is because much of what's new in Debian 6.0 is actually a bit old for other Linux distributions. For instance, the default desktop for 6.0 is GNOME 2.30 (with some 2.32 modules). GNOME 2.30 was released in March of 2010. The release also includes KDE 4.4, which is nearly two releases behind, and the default kernel is 2.6.32.5.

Those who'd like slightly less crusty packages will be able turn to the Debian Backports service, which was dubbed an official Debian service in September of 2010. Since Squeeze is not yet released, only Lenny and Etch are currently supported — but packages should start rolling into the backports service shortly after the official Squeeze release. Another addition to 6.0 is the stable-updates repository, which will make updates for non-security updates available before the point updates for Squeeze.

Major changes in Squeeze

Aside from the usual package updates, Squeeze does come with a number of changes over the previous releases. Though many users simply ride dist-upgrade between releases, it is occasionally necessary to use the installer — such as when putting Debian on a new machine. Debian 6.0 brings a new installer with new artwork with a "SpaceFun" theme.

Artwork is not the only improvement to the installer, of course. The new installer now automatically installs packages which are marked as recommended as well as those marked required. The installer now has support for Ext4 and Btrfs, but the default remains Ext3, most likely due to performance problems with dpkg on Ext4 that were still being worked out late in the development cycle. ZFS is an option for installs of Debian kFreeBSD.

The actual practical experience of using the installer is little changed — depending on one's point of view, the installer either offers much more flexibility or much more complexity. It is worth noting that the Debian crew could at least shave a few steps off the installation by allowing users to enter related details during a single step. It seems unnecessary, for example, to have a single screen to enter a user's full name, then a screen for username, then for the user's password.

Another major change in Squeeze is the removal of non-free firmware from the kernel. For free software purists, this is a win. For those who are slightly less stringent about maintaining a 100% free system, it's a bit more of a mixed bag. I tested Debian Squeeze on three machines, two desktops and a laptop, and it had no problem with the desktop systems' hardware. However, it didn't like a USB Wi-Fi adapter that works perfectly with Ubuntu and openSUSE. It also had some problems with Wi-Fi on the test laptop — which is not terribly surprising.

Users with hardware that requires non-free firmware are not out of luck. The Debian project is providing tarballs of non-free firmware and CD images of the net install CD that include non-free firmware. Note that Debian provides several choices of install ISOs — a business card CD that will require Net access for the base installation packages, a net install image that includes base packages and network access for everything else, and full CD/DVD sets that include full sets of recommended packages. Note that only the first CD image for an architecture is necessary — the remainder carry packages that one might wish to add after an install. If you're planning an eclectic install away from network access, a full 56 ISO images are available for the AMD64 architecture alone. Compare this to the first official Debian release, which fit on one CD-ROM and contained a whopping 474 packages.

The project also has a few new "Pure Blends" being introduced with Squeeze. Formerly known as Custom Debian Distributions, the Pure Blends are collections of the more than 22,000 Debian packages for specific audiences. Squeeze adds two new Pure Blends to the mix: Debian GIS for Geographical Information Systems (GIS) applications and users, and Debichem for users of chemical applications in Debian.

On January 18 Neil McGovern announced that a second release candidate is expected by the weekend of January 22nd, with a target date for the final release the weekend of February 5th. Any non-critical fixes that don't make it by this deadline will be held for Debian 6.0.1.

While I tend to change distributions frequently, I've grown used to distributions like Linux Mint, openSUSE, and Ubuntu which attempt to provide a more user friendly experience than Debian. The time I've spent recently running Debian has been by turns refreshing and frustrating. Debian gives the user much more control over configuring a system, and is much closer to upstream projects like GNOME than, say, Ubuntu.

It's also a bit frustrating in the sense that projects like Mint do take the time to do things like tweak font settings so they're not so glaringly ugly. There's also the matter of the default application set — which need a bit more tweaking for this user in Debian than in Mint, Fedora, or Ubuntu. Applications like Dropbox may not have native packages for Squeeze (the Debian packages for Dropbox do not work with Squeeze) so odds are a few packages will require building from source. Others, like Gwibber, are simply hopelessly outdated.

Aside from minor frustrations, though, Debian 6.0 looks to be a solid release. It should be — all of its components have had ample time to settle. For users who want a very stable and basic distribution, this is as good as it gets. Of course, many of Debian's users ride the testing or unstable releases and pay little attention to stable releases. This user, for instance, intends to switch to unstable almost immediately. But Debian's stable releases do have their place — namely, anywhere one would like to set up and then forget (modulo running occasional security updates) a desktop or server.

Comments (10 posted)

Distribution quotes of the week

-- Christian Perrier

-- Alexander Reichle-Schmehl

Comments (none posted)

Red Hat Enterprise Linux 5.6 Now Available

The release of the sixth update for Red Hat Enterprise Linux 5 has been announced. Highlights of this release include support for new hardware, virtualization improvements, and updated DNS packages. See the release notes for details.

Comments (none posted)

Fedora board meeting, January 17

A summary of the January 17 meeting of the Fedora board has been posted. Fedora strategic goals were discussed with FAmSCo.

Comments (none posted)

A note from the openSUSE board

The openSUSE board has sent out a long note explaining and justifying its decision to evict an individual from the openSUSE community. "First of all, the Board never had to deal with a situation like this one before, where many attempts of mediation and dialogue didn't result in the person who violated the Guiding Principles understanding the concerns and going back into behavior that is socially acceptable (because that's, in essence, what the Guiding Principles are about). Hence we, as a Community, didn't have any precedence to look for, nor any mistakes we did in the past to improve the process upon." The board is trying to maintain the privacy of the person involved - an approach which is not going over entirely well in the subsequent decision. Your editor would request, though, that the board's wish be respected in any discussion on this site.

Full Story (comments: 12)

open-slx end-user platform opens

open-slx has announced the launch of a new community platform for the German-speaking openSUSE community. "Starting in June 2010, a team of German openSUSE community members, open-slx employees and from ubuntuusers.de has worked on the introduction of the new community portal, which consists of three parts: Wiki, Forum and Blog aggregation. In the wiki, more than 2000 articles from ubuntuusers.de's knowledge base have been imported, almost 500 of which serve as the base for the new open-slx platform, the team's goal is to finalize all 2000 articles within this year. The webforum can be used to discuss these articles or get in contact with other openSUSE users. The blog aggregation ("Planet") has a news feed which gives insight into newest developments and articles covering more specific topics."

Full Story (comments: none)

openSUSE 11.1 has reached end of Novell support - 11.1 Evergreen goes on

openSUSE 11.1 has reached its end of official support. Evergreen is a community maintainance project aimed at extending support for openSUSE releases.

Full Story (comments: none)

Ubuntu Server Survey 2011 -- How do You Ubuntu?

The Ubuntu Server community and Canonical would like to know how Ubuntu users are using Ubuntu Server Edition, and in what kinds of organizations. "The anonymous survey takes 10 to 20 minutes to complete and is open to anyone deploying Linux servers today, whether or not they use Ubuntu. The Ubuntu Server Community Team will present the results publicly at the beginning of February."

Full Story (comments: none)

Distribution newsletters

• Debian Project News (January 14)
• DistroWatch Weekly, Issue 388 (January 17)
• Fedora Weekly News Issue 258 (January 12)
• openSUSE Weekly News, Issue 158 (January 16)
• Ubuntu Weekly Newsletter, Issue 219 (January 17)

Comments (none posted)

The Arch Way (Linux Journal)

Linux Journal has several good reasons to use Arch Linux. "Speaking of documentation, the Arch Wiki is the most comprehensive repository of Linux information out there. Back when I used Ubuntu, I often found the answer to a tough configuration issue on The Arch Wiki. Yes, the Arch Wiki is tailored to Arch, but it can often help you solve problems with other distributions as well."

Comments (none posted)

Linux Mint Debian Edition 10: Rolling Release Nirvana (Linux Magazine)

Joe "Zonker" Brockmeier likes LMDE. "One of the questions I usually ask when considering a new Linux distribution is what, if anything, it adds to the community. There's a lot of redundancy out there, and while anybody can make their own Linux distribution it doesn't mean they should. I think LMDE does add something to the mix, especially as Ubuntu evolves further and further away from its Debian heritage."

Comments (none posted)

Lumsden: 2011.Q1 - Foreverware

Alasdair Lumsden has a proposal for an OpenIndiana server release in February. "It's also worth keeping in mind that despite warning users that oi_147 and oi_148 were development releases, people are already using it in production environments, myself included, due to a lack of alternatives. The great news is that it has proven to be exceedingly reliable, and I have no hesitation in recommending it for busy workloads. All we need to do is add security updates and critical bug fixes on top and we'll be in a great position. No small feat I grant you, but we can start off small and work our way up. Now is also an opportune time to do this - our next release will be based on Illumos, which has seen rapid development and will involve some integration pain. Some have called for a stable branch after Illumos is integrated, but it could be many months until we have an Illumos dev build suitable for respinning as a stable branch. That's months of lost opportunity."

Comments (none posted)

Lawyers Can Leave Windows for Linux OS – Ubuntu (SEOLawFirm.com)

SEOLawFirm.com provides a perspective on switching to Linux that is a little different than those we often see. After a largely accurate, if a bit simplified, "crash course" on open source, the article shows law firms how they can save money by switching from Windows to Ubuntu—and how to use web services to replace some tools like Quicken and case management applications. "You work with the law on a daily basis. There is a good chance you did not write the law, but by the labors and efforts of others in the past, you now have the law to work with. What if you had the ability to change the law depending on the case you were working on? Does your client need additional rights? Give it to them. Does your client need a second chance? Write a process doing just that. Then contribute those changes to the legal community and other lawyers can then build upon those laws and do the same. [...] It would probably be chaotic if laws were open source, but when it comes to software, open source contributions allow technology to rapidly evolve."

Comments (23 posted)

On the maintainability of Ruby

Lucas Nussbaum, longtime maintainer of Debian's Ruby packages, announced that he was stepping back from the role on January 2, making the future of Ruby packaging on the distribution uncertain. He cited a list of reasons leading to the decision on his blog — an assortment of project management, release process, and packaging concerns with Ruby itself that compounded his frustration to the point where he felt quitting was the best option. The public reaction to the announcement has been sympathetic to the hardships of Debian package maintainer-ship, but it has also sparked an interesting debate about the competing needs of developers, end users, and Linux distributors.

Management

Nussbaum had several criticisms of the overall management of the Ruby project. The most obvious is the confusing array of development branches. He observes that there are currently six branches of the project in active development: 1.8, 1.8.6, 1.8.7, 1.9.1, 1.9.2, and trunk. While the present status of two of them (1.8.6 and 1.9.1) is clearly marked as important bug-fixes only, the rest are unclear. If 1.9.x is the current development branch, he asks, why are there any active branches in 1.8.x? He also asks about the relationship between 1.9.2 and trunk, and about whether or not there is any API or ABI compatibility between the two.

He also observed that there appears to be no "common understanding" of the release expectations of any of the branches. That phrase could be understood in several different ways, but he adds that it "would be fantastic to have something similar to Python Enhancement Proposals" for Ruby, which suggests the lack of a clear roadmap and/or sunset period for older releases is the underlying issue.

Ruby's direction is set primarily through mailing list discussions, which constitutes another issue. Nussbaum laments that the primary discussion list, ruby-dev, is written in Japanese, while a second list for English-speakers, ruby-core, misses out on the discussion and gets out-of-step. Ruby was invented in Japan, of course, and most of its core developers are Japanese, but maintaining two lists divides the community, keeps out most of the world, and even risks the possibility of a project fork somewhere down the line.

Releases

Releases themselves constitute another problem for Nussbaum, who observes that 1.8.7 and 1.9.2 releases were both made on December 25, 2010, apparently without any beta testing period or Release Candidates preceding them.

On a note similar to the language barrier between the lists, he observes that the Ruby project is in the habit of regularly making its major releases on December 25th, which is the traditional Western date for Christmas, and is a day when a large percentage of offices are closed. The result is that users in Europe, North and South America, and Australia are unavailable to test and provide feedback for several days or weeks.

But most importantly, Nussbaum argues that the Ruby platform consists of more than just the standard implementation Ruby interpreter, known as Matz's Ruby Interpreter (MRI). There are an increasing number of alternative interpreters popular with developers, including the Java Virtual Machine-based JRuby, Rubinius, and MacRuby, but there is no clear process for integrating them with the rest of the Ruby ecosystem.

Furthermore, the ecosystem includes large-scale projects and libraries such as Ruby on Rails, on which a great deal of real-world Ruby code depends. The development community, Nussbaum says, should create a process to ensure that new releases of MRI work with those "big players" before making a release.

Ruby's release process has been criticized by others in the past. In 2008, security holes based on integer overflows were uncovered in the released code, which might have been caught by a public beta testing period, and the fixes pushed out by the project broke the Rails framework.

Users, developers, and systems administrators, oh my....

The most contentious issue cited by Nussbaum in his blog post is how Ruby and Ruby libraries are packaged for distribution. The official, preferred method for installing Ruby is Ruby Version Manager (RVM), a command-line tool that allows individual users to install multiple self-contained Ruby environments, including interpreters, libraries, and applications. Ruby libraries are generally distributed through RubyGems, an online distribution network that functions much like Perl's CPAN. There is a command-line client, which fetches and installs individual packages in the Ruby-specific gem format.

This system works quite well for developers, Nussbaum says, but is unnecessarily hard on both system administrators and end users. End users, he says, expect to be able to install a Ruby-based application using the normal system tools (such as apt-get). Requiring them to instead install RVM and compile Ruby from source, then use the RubyGems package manager to install dependencies that live outside the normal package management system is unreasonable. The problems of the end user are compounded for the system administrator, who may be asked to maintain multiple Ruby interpreters over long periods of time, as well as cope with users each installing their own collections of gems.

Nussbaum also cites several examples of negative, offensive behavior in public — some of it directed at him (accusing him of intentionally creating "crippled" Ruby packages for Debian), but more directed at others.

As for the future, he points out that there are two other Ruby packagers for Debian, who may together decide to continue maintaining the package without him, and he wishes them luck. What is less clear is the future of the Ruby-extras package, which includes most of the Ruby libraries. There appears to be no successor for the maintainer's role for that package. On the plus side, he does say that he is open to returning as a Ruby packager in the future, if the landscape improves.

Reaction

Nussbaum's post spawned quite a bit of discussion, both in comments to his blog and on YCombinator. A few commenters on both sites accused him of "imperialism" because of the list language issue, before it was pointed out that Nussbaum himself is a native French speaker. But by and large, the debate swirled around the clash between the Ruby project's goals and the Debian project's goals.

The essential argument seems to be that Ruby, as a language project, should and does seek to make life as easy as possible for Ruby developers. Debian, on the other hand, seeks to package upstream software in a way to make life as easy as possible for end users.

Developers will almost always want to run bleeding-edge code, to make use of the latest and greatest language and interpreter features. Consequently, they want to compile and install the Ruby environment from source, and want to have access to specific versions of Ruby libraries — sometimes multiple versions, as concurrent projects may demand — and they want to manage those dependencies in gem packages, not Debian packages.

Users, in stark contrast, want installation of Ruby applications to be simple, which demands integration with the main distribution package management system, not use of a specialized Ruby tool. Stuck right in the middle are systems administrators, who get asked to support scores of locally-compiled and installed Ruby environments, and distributions, who get asked to package scores of minor releases for Ruby interpreters and libraries.

Commenter Meneer R pointed out that this a common tension when dealing with interpreted languages, because the execution environment and the development environment are the same. Compiled languages always have the relatively easy out of statically linking-in an old library. Meneer R and several others argue that the "Debian philosophy" needs to change; namely that the distribution should accept the Ruby community's packaging and otherwise get out-of-the-way. Developers, he said, target specific versions of the Ruby platform, not specific releases of Debian, and trying to shoehorn the two into one release system only complicates things for Ruby developers using Debian. Several others pointed out that Ruby and Ruby libraries have to work across all operating systems, and that Debian's packaging techniques simply don't apply to Windows, OS X, or other platforms.

Others from the Debian community take the opposite stance. Commenter Np237 argued that allowing multiple versions of one library to co-exist in the system is itself the problem, because:

Debian developer Gunnar Wolf posted a response on his own blog, generally showing support for Nussbaum. Regarding the conflict between Ruby packages and Debian packages, he observed that maintaining multiple versions of the same library on a system in order to support multiple incompatible applications has inherent risks, such as bug fixes not getting back-ported, and security and stability problems developing over time. The need to keep multiple versions of an interpreter or library installed generally means there are big changes between different releases, he said, and that means that the library is immature and not ready for long-term API commitment — which in turn makes it a bad bet for the developer.

Wolf also weighed in on one of the repeated claims in support of Ruby's packaging system: that because the language is primarily used by web developers, "end users" never need to install it anyway. Web frameworks like Rails unduly influence the discussion, he said, because the core packaging issues exist for non-web developers, too. In addition, saying that web applications live by different rules ignores the needs of systems administrators, he said.

Nussbaum is not the only packager to grapple with the difficulty of fitting Ruby and RubyGems into a distribution's package management system. Fedora has a Ruby special interest group (Ruby SIG) which tackled many of the same issues one year ago. Fedora manages multiple concurrent Ruby releases (1.85, 1.8.6, 1.9.1, and 1.9.2) using the alternatives system, and has packaging guidelines that attempt to wrap gems inside RPM files. Nussbaum commented on his blog that alternatives was not the solution for Debian, because by definition it is designed for tools that can be used interchangeably, which different releases of Ruby interpreters cannot.

Despite all of the arguments on both sides of the packaging issue, the discussion did remain completely civil. Nussbaum updated his post to point to two positive steps taken by the Ruby community: a JRuby maintainer outlining his project's vision and goals as relates to the larger Ruby development ecosystem, and a message from a core Ruby developer inviting more discussion on the English-speaking mailing list, and clarifying that there is no intention of keeping the two subscriber bases separate.

Paul Brannan, one of the original authors of RubyGems, even joined in the discussion, explaining that gem packages were designed with the goal of reducing the strain on administrators and users (including the ability to package a gem inside of an RPM or Debian package). He also offered mea culpas for those design decisions that, with hindsight, did not work out so well.

Language maturity

Wolf made an interesting aside in blog post, noting along the way that Ruby is no longer a "new language". That sentiment is clearly present in Nussbaum's retirement announcement, even if he did not explicitly say it. With a young language, people (developers and systems administrators) expect things to change rapidly and sometimes without warning. But Ruby is fifteen years old now. That may be young compared to C and FORTRAN, but it is certainly not a flash-in-the-plan.

Issues like those Nussbaum raised are difficult for projects to deal with. It is awkward to change the existing culture and tell developers that old language features are deprecated. It is particularly awkward to suggest that a project has grown so much that the main mailing list needs to shift to the lingua franca of distributed development, rather than the original authors' language. But like it or not, even if the two mailing lists do not separate developers into first- and second-class community members, they do split the community into two circles that don't work together.

Yet not all of the Nussbaum's issues are that difficult; there are things that Ruby can do to make life easier downstream. There is still no formal language specification, for example. Standardizing that would help unify the competing interpreters. A more formal process for language extension would ease a lot of minds, and a stricter release schedule with targeted betas would make the community feel more involved. Those are goals any open source project would be proud of, all the more so for the language designated as "A programmer's best friend".

Comments (25 posted)

Quotes of the week

-- Matt Mackall

-- Ingo Molnar takes the long view

Comments (5 posted)

Amarok 2.4 released

Version 2.4 of the Amarok media player is available. "This release brings significant performance, usability, and stability improvements: Most of the bugs involving the context pane have been squished, a completely rewritten collection scanner is presented (hint: it's the thing that reads your music files' tags) -- Amarok now has the ability to write your statistics and covers directly to the audio files to keep the information across different devices and should detect compilations a lot better. At long last, we also have an option to hide the menu bar again." Also in this release are new device support, transcoding of tracks, and more.

Comments (1 posted)

A mailing list for Android audio developers

A new mailing list has been created for developers interested in audio issues on the Android platform. "This list is meant to be a place to discuss about audio development, in the context of music and sound applications, but also games and other apps which use audio on Android."

Full Story (comments: none)

FFmpeg turmoil

A group of FFmpeg developers has announced that the project has a new set of maintainers - news which came as a surprise to existing maintainer Michael Niedermayer. Developer Diego Biurrun has posted an explanation of how the coup came to be, but it's clear that not everybody is satisfied. "The discontent reached the point where a fork was being contemplated and then planned, but it turned out that the momentum had soared way past critical mass and turned into a tidal wave of revolution. The focus moved from forking to avoiding a fork if possible. Since git was being set up on videolan.org, setting up an alternative git tree on mphq was the natural choice. With development moving to videolan.org and such a large group of developers already part of the revolution keeping the infrastructure was the logical consequence." (Thanks to Mattias Mattsson).

Comments (34 posted)

MDP release 3.0

Version 3.0 of the Modular Toolkit for Data Processing is out. "MDP is a Python library of widely used data processing algorithms that can be combined according to a pipeline analogy to build more complex data processing software." Changes include Python 3 support, some new extensions, new algorithms, an improved tutorial, and a switch to the BSD license.

Full Story (comments: none)

Parrot 3.0.0 Released

Version 3.0.0 of the Parrot virtual machine is available. "On behalf of the Parrot team and an enthusiastic but undiscriminating dachshund that followed me home last week, I'm proud to announce Parrot 3.0.0, also known as "Beef Stew", or at the insistence of a shadowy government organization, "Snowflake". Parrot (http://parrot.org) is a virtual machine that dreams about running all dynamic languages everywhere, even the one you're think about right now. Parrot has big plans, even if needs a haircut and sometimes goes outside with its shoes untied." This version includes improvements in the core code and the documentation, new language support, and better test coverage.

Full Story (comments: 6)

Xfce 4.8 released

Version 4.8 of the Xfce desktop has been announced. "With Xfce 4.8 our users will be able to browse remote shares using a variety of protocols (SFTP, SMB, FTP and many more). The window clutter has been reduced by merging all file progress dialogs into a single one. Our panel application has been rewritten, thereby improving positioning, transparency, item and launcher management. It also introduces a new menu plugin to view directories. Its plugin framework remains compatible with 4.6 plugins." LWN previewed this release back in December.

Comments (4 posted)

Development newsletters from the last week

• Caml Weekly News (January 18)
• LibreOffice development summary (January 18)
• PostgreSQL Weekly News (January 16)
• Tcl-URL! (January 14)

Comments (none posted)

More about the Chrome HTML Video Codec Change (The Chromium Blog)

Google's Chromium blog has an update on the decision to remove H.264 support from the Chrome/Chromium browser. "We acknowledge that H.264 has broader support in the publisher, developer, and hardware community today (though support across the ecosystem for WebM is growing rapidly). However, as stated above, there will not be agreement to make it the baseline in the HTML video standard due to its licensing requirements. To use and distribute H.264, browser and OS vendors, hardware manufacturers, and publishers who charge for content must pay significant royalties—with no guarantee the fees won't increase in the future. To companies like Google, the license fees may not be material, but to the next great video startup and those in emerging markets these fees stifle innovation."

Comments (70 posted)

The Document Foundation joins OpenDoc Society

The Document Foundation has become an organizational member of the OpenDoc Society. "OpenDoc Society brings together individuals and organizations with a stake or interest in the openness and future of documents, to learn from each other and share knowledge and best practices about core technologies, available tools, policy issues, transition strategies, legal aspects and of course the latest innovations."

Full Story (comments: none)

New Report about Open Data for an Open Society

Marco Fioretti has announced the availability of the Open Data, Open Society report: "The report discusses the current and potential role, in a truly open society, of raw Public Sector Information (PSI) that is really open, that is fully accessible and reusable by everybody. The general characteristics of PSI and the conclusions are based on previous studies and on the analysis of current examples both from the European Union and the rest of the world. [...] The "Open Data, Open Society" project will continue with an online survey that will attempt to assess how many types of raw PSI are already released, in which formats and under which licenses, by the city and regional administration of the EU-15 countries. A final report will summarize and comment the results of the survey."

Full Story (comments: none)

FSF: supporting Google's push for WebM

The Free Software Foundation has sent out a release stating its support for the WebM project. "We want the world to know that we also support WebM: with its developer-friendly patent license and free software reference implementation, it's a good choice to help ensure the Web fulfills its promise of providing a free way for the world to communicate."

Full Story (comments: none)

Google's dropping H.264 from Chrome a step backward for openness (ars technica)

Ars technica has a lengthy criticism of the removal of H.264 support from the Chrome browser. "Google is now building a community around WebM (similar to that around Theora), but it hasn't taken any steps to submit WebM to ISO, ITU, or SMPTE for formal open standardization. The company is preferring to keep it under its own sole control. For Google to claim that it is moving to 'open codecs' is quite absurd: H.264 is very much an open codec. WebM is not."

Comments (63 posted)

PS3 jailbreak prompts restraining order from Sony (CNet)

CNet covers Sony's DMCA suit against George Hotz, who figured out how to jailbreak the PS3 game console. "Sony is seeking the impoundment within the next 10 days of all circumvention technology that Hotz and his team employ. It also wants all mention of the circumvention removed from the Web, and has asked the court to force Hotz to ignore calls for help from others attempting to install packages on the console."

Comments (23 posted)

CES 2011: A Tale of an Android Onslaught (LinuxPlanet)

LinuxPlanet covers the array of Android devices seen at this year's Consumer Electronics Show. "While it seems every Consumer Electronics Show (CES) in recent memory has some theme that never really panned out (see 3-D TV), you might want to rethink the trend when it comes to Android. Google's Android attack was in full force at this year's edition with a vengeance. From smart phones to tablet devices of all sizes to Google TV, you couldn't travel the exhibit floor very far without bumping into something Android. For some companies, like Motorola, Android has fueled impressive comebacks. For others, like Sony, it's new territory."

Comments (none posted)

Firefox 4 Beta 9 Gives Short Shrift to Linux Users (PCWorld)

PCWorld looks at the lack of hardware acceleration in the Linux version of the latest Firefox beta. ""We tried enabling OpenGL on Linux, and discovered that most Linux drivers are so disastrously buggy (think 'crash the X server at the drop of a hat, and paint incorrectly the rest of the time' buggy) that we had to disable it for now," wrote Mozilla developer Boris Zbarsky Friday in a comment on the Mozilla Hacks developer blog. "Heck, we're even disabling WebGL for most Linux drivers, last I checked.""

Comments (39 posted)

Phipps: Apple and Google and ODF

Simon Phipps notes that in Latvia (and other European countries) governments must accept documents in ODF format, and bemoans the lack of ODF support from Apple and Google. "So I remain surprised that neither Apple nor Google are taking ODF support seriously. Apple still don't support ODF in their applications (despite it being available in their TextEdit gadget on Mac OS X) or the iPhone or iPad, and the ODF support in Google Docs is so weak that documents I try to upload from LibreOffice are routinely rejected in ODF and yet accepted if I save the identical document in .doc format. It's ironic that the best proprietary ODF support right now is from Microsoft." (Thanks to Davide Del Vento)

Comments (8 posted)

Upstreaming your code - a primer

Armijn Hemel and Ralf Baechle have written an introduction to upstreaming code to the kernel. It's aimed at chip vendors, but should be applicable to other aspiring kernel hackers. "It is likely that the first submission to a Linux kernel developer of your code will not be accepted. This could be for various reasons: the code does not follow the coding style, code quality might not be as high as is expected, and so on. Do not see this as a rejection of your code, but as a first step for successful inclusion into the mainline kernel. Sometimes developers might react very terse, or seemingly irritated. Once again, do not see this as a rejection, but as a sign that you need to put in a little bit more effort."

Comments (9 posted)

Eloquent JavaScript--New from No Starch Press

No Starch Press has released "Eloquent JavaScript", by Marijn Haverbeke.

Full Story (comments: none)

Gimp 2.6 for Photographers--New from Rocky Nook

Rocky Nook has released "GIMP 2.6 for Photographers", by Klaus Goelker.

Full Story (comments: none)

Programming Python, 4th Edition--New from O'Reilly

O'Reilly Media has released "Programming Python, Fourth Edition", by Mark Lutz.

Full Story (comments: none)

Software Testing Foundations, Third Edition--New from Rocky Nook

Rocky Nook has released "Software Testing Foundations, Third Edition", by Andreas Spillner, Tilo Linz and Hans Schaefer.

Full Story (comments: none)

Linux Foundation Monthly Newsletter: January 2011

The January edition of the Linux Foundation newsletter covers the 2011 Event Schedule, a new fellow, five new members, individual membership drive winners announced, Linux Powering Your Cell Phone Network: A Case Study, and more.

Full Story (comments: none)

Voting for the 2010 LinuxQuestions.org Members Choice Awards is Now Open

LinuxQuestions.org has announced that voting for the 2010 LinuxQuestions.org Members Choice Awards is now open. "Awards will be given out in 31 categories this year, including Server Distribution of the Year, Desktop Distribution of the Year, Mobile Distribution of the Year, Browser of the Year, NoSQL Database of the Year and Open Source Web Framework of the Year. The polls will close on February 17th."

Full Story (comments: none)

Linux Plumbers' Conference 2011: Call for Tracks

The Linux Plumbers' Conference will take place in Santa Rosa, CA, September 7-9, 2011 and the program committee is looking for track proposals. "The ideal track proposal is focused enough to allow progress and attract the right participants, but also spans several components of the Linux technology stack and a set of problems that can specifically benefit from face to face discussion between the various teams involved."

Full Story (comments: none)

NLUUG Spring Conference 2011

NLUUG will be holding their spring conference May 12, 2011. "We are looking for presentations to complete the program for our Spring Conference of 2011. We would like to receive abstracts on technical subjects (40 minutes time slots) and practical experiences (20 minutes) related to the theme "Open is Efficient". With these 20 minutes shorter presentations we hope to achieve that more people contribute to the conference." The proposal deadline is February 12.

Comments (none posted)

Texas Linux Fest 2011 Call for Papers now open

Texas Linux Fest 2011 takes place April 2, in Austin, Texas. The call for papers is open until February 14.

Full Story (comments: none)

The first set of FOSDEM interviews

FOSDEM (Brussels, February 5-6) has posted the first set of its traditional speaker interviews; featured this time are a certain LWN editor, Eben Moglen, James Turnbull, Boudewijn Rempt, and Gratien D'haese. Quoting Eben: "Currently, governments are very eager to control the Net, which is a disaster for freedom. Industry is responding to the mobile computing revolution by trying to lock down the devices people use in their daily lives. And powerful industrial players who make money by surveilling people and data-mining the resulting information are encouraging more people to give up their basic privacy for small, if not meaningless, benefits. All are large threats to freedom, and if our strategies worked against only one or two such threats, we could still fail."

Comments (1 posted)

LCA 2011 will be happening

The linux.conf.au 2011 team has announced that the event will happen as scheduled, despite the flooding in Brisbane which has left them with some problems to work out still. "At this stage the venue location is still to be confirmed as are a number of social events, though there are some backup plans in place."

Full Story (comments: 4)

LCA moves to a new venue

The venue for linux.conf.au, starting January 24 in Brisbane, has been moved to the Kelvin Grove campus of the Queensland University of Technology - a shift of about 4km. The current recommendation seems to be that attendees should keep their current hotel reservations and use the bus service to get to the venue. "We do ask you to be patient as we work through all the logistics - but the conference will go ahead and no miniconfs or conference streams will be canceled." One only hopes that the organizers have planned a big party after the event; they will certainly have earned it.

Comments (5 posted)

Events: January 27, 2011 to March 28, 2011

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
January 24 January 29	linux.conf.au 2011	Brisbane, Australia
January 27 January 28	Southwest Drupal Summit 2011	Houston, Texas, USA
January 27	Ubuntu Developer Day	Bangalore, India
January 29 January 31	FUDCon Tempe 2011	Tempe, Arizona, USA
February 2 February 3	Cloud Expo Europe	London, UK
February 5 February 6	FOSDEM 2011	Brussels, Belgium
February 5	Open Source Conference Kagawa 2011	Takamatsu, Japan
February 7 February 11	Global Ignite Week 2011	several, worldwide
February 11 February 12	Red Hat Developer Conference 2011	Brno, Czech Republic
February 15	2012 Embedded Linux Conference	Redwood Shores, CA, USA
February 25	Build an Open Source Cloud	Los Angeles, CA, USA
February 25 February 27	Southern California Linux Expo	Los Angeles, CA, USA
February 25	Ubucon	Los Angeles, CA, USA
February 26	Open Source Software in Education	Los Angeles, CA, USA
March 1 March 2	Linux Foundation End User Summit 2011	Jersey City, NJ, USA
March 5	Open Source Days 2011 Community Edition	Copenhagen, Denmark
March 7 March 10	Drupalcon Chicago	Chicago, IL, USA
March 9 March 11	ConFoo Conference	Montreal, Canada
March 9 March 11	conf.kde.in 2011	Bangalore, India
March 11 March 13	PyCon 2011	Atlanta, Georgia, USA
March 19	Open Source Conference Oita 2011	Oita, Japan
March 19 March 20	Chemnitzer Linux-Tage	Chemnitz, Germany
March 19	OpenStreetMap Foundation Japan Mappers Symposium	Tokyo, Japan
March 21 March 22	Embedded Technology Conference 2011	San Jose, Costa Rica
March 22 March 24	OMG Workshop on Real-time, Embedded and Enterprise-Scale Time-Critical Systems	Washington, DC, USA
March 22 March 25	Frühjahrsfachgespräch	Weimar, Germany
March 22 March 24	UKUUG Spring 2011 Conference	Leeds, UK
March 22 March 25	PgEast PostgreSQL Conference	New York City, NY, USA
March 23 March 25	Palmetto Open Source Software Conference	Columbia, SC, USA
March 26	10. Augsburger Linux-Infotag 2011	Augsburg, Germany

If your event does not appear here, please tell us about it.