LWN.net Weekly Edition for June 7, 2012

LinuxCon Japan: Making kernel developers less grumpy

Greg Kroah-Hartman is on something of a mission: reducing the grumpiness factor among kernel developers, and maintainers in particular. His keynote at LinuxCon Japan was meant to help the audience understand what the maintainers do, and how contributors' actions can sometimes result in grumpy maintainers. But, if contributors can follow the rules and make things easier on him, there are a number of things that he will promise to do on their behalf.

He called the Linux kernel the "largest software development project ever" and noted that its development pace is "unprecedented". From 3.0 to 3.4, some 2833 developers from at least 373 companies contributed. In that year (from May 2011 to May 2012), the kernel had a change rate of 5.79 changes per hour. But the rate keeps increasing and if you look at just the 3.4 cycle, the rate is 7.21 changes per hour. That is, of course, just patches that are accepted into the mainline, so it doesn't count those patches that are rejected.

Developers typically send their changes to the maintainer of the file that is being changed. Those maintainers, who number around 700, feed those changes up to the 130 subsystem maintainers. From there, the patches make their way into linux-next, then to Linus Torvalds, and, eventually the mainline—if they get accepted at each step along the way.

So, in order to see why some patches might not get accepted, he looked at those that he received in the last two weeks, which coincided with the 3.5 merge window. The merge window is a time when he really shouldn't be getting many patches. He should have received them all earlier in the cycle so that he could potentially pass them on to Torvalds during the merge window. But, he said, he got 487 patches in that two-week period, many with a wide variety of problems, and some of those from core kernel developers who should know better.

Broken patches

With that, he launched into a description of some of the broken patches he got. One patch was labeled "patch 48/48" (i.e. the last patch in a set of 48) but all of the other pieces were missing. He also got a patch series with no order specified, which means that he would have to guess at the order and undoubtedly get it wrong. The alternative is to ignore the patch entirely. He also got a ten-patch set that was missing patch two in the series.

Another patch came in an email with a signature claiming that it was confidential. He actually sees that one a lot, he said, and there is nothing he can do with those kinds of patches. Linux development is done in the open and you can't send a confidential email to mailing lists or get a confidential patch merged. Obviously, it is boilerplate that gets added somewhere in the email process, but it has to be removed before the patch can be used.

There are also malformed patches that end up in his inbox, including those with tabs converted to spaces. Microsoft Exchange does that, he said, so if that's a problem in your environment, do what IBM, Microsoft, and others do: put a Linux box in the corner for the developers to use to send their mail. Sometimes the leading spaces have been stripped off the diff or the diff is not in unified format. Linux developers have gotten good at raw editing diff format, he said, which is scary in itself, but they shouldn't have to do that.

Patches are also created in the wrong directory, like down in a driver directory for example. He got a patch created in /usr/src/linux-2.6.32 and noted that there were multiple things wrong with that, including the age of the source tree and that it implied it was being built by root. The latter is very dangerous as there was a bug in the Linux build process at one point that would delete the entire root filesystem if it was run as root. None of the core developers noticed because they don't build as root. Suggestions that the bug be left in as a deterrent were ignored, but things like that can happen.

In addition, patches came in that were made against a different tree than any he would expect. He got a patch made against the SCSI development tree, for reasons unknown because it had nothing to do with SCSI.

Then there are those that don't have the right coding style. In one case, the coding style was wrong and the developer acknowledged that but wanted him to take the patch anyway. That gives the impression of "we don't care, take our code anyway", he said. There are tools to help find and fix those kinds of problems, so there is no excuse: "send it in the right coding style".

Something he sees much more than he should are patches that don't even compile. The submitter clearly hasn't even built the patch, he said. Or there are patch sets that break the build in 3/6 but then fix it in 6/6. He even got a patch that broke the build in 5/8 but contained a note that sometime in the future the submitter would send changes to fix it. Another patch had obviously wrong kernel-doc in it that would cause failures building the documentation, so it was clear that the contributor had never even tried to run the kernel-doc extraction tool.

One of the patches he got "had nothing to do with me". It was an x86 core kernel patch, which is not an area of the kernel he has ever dealt with. But the patch was sent only to him. "I get odd patches" a lot, he said.

The last patch he mentioned was 450K in size, with 4500 lines added. Somebody suggested that it be broken up, but in the meantime several maintainers actually reviewed it, so the submitter didn't really learn from that mistake.

All of this occurred during a "calm two weeks", he said. These are examples of what maintainers deal with on a weekly basis and explains why they can be grumpy. That said, he did note that this is the "best job I've ever had", but that's not to say it couldn't be improved.

If someone sends him a patch and he accepts it, that means he may have to maintain it and fix bugs in it down the road. So it's in his self interest to ignore the patch, which is an interesting dynamic, he said. The way around that is to "give me no excuse to reject your patch"; it is as simple as that, really.

Rules

Kroah-Hartman then laid out the rules that contributors need to follow in order to avoid the kinds of problems he described. Use checkpatch.pl, he said, because he will run it on your patch and it is a waste of his time to have to forward the results back when it fails. Send the patch to the right people and there is even a script available (get_maintainer.pl) to list the proper people and mailing lists where a patch should be sent.

Send the patch with a proper subject that is "short, sweet, and descriptive" because it is going to be in the kernel changelog. It should not be something like "fix bugs in driver 1/10". In addition, the changelog comment should clearly say what the patch does, but also why it is needed.

Make small changes in patches. You don't replace the scheduler in one patch, he said, you do it over five years. Small patches make it easier for reviewers and easier for maintainers to accept. In a ten-patch series, he might accept the first three, which means that the submitter just needs to continue working on the last seven. The best thing to do is to make the patch "obviously correct", which makes it easy for a maintainer to accept it.

Echoing the problems he listed earlier, he said that patches should say what tree they are based on. In addition, the order of the patches is important, as is not breaking the build. The latter "seems like it would be obvious" but he has seen too many patches that fail that test. To the extent that you can, make sure that the patch works. It is fine to submit patches for hardware that you don't have access to, but you should test on any hardware that you do have.

Review comments should not be ignored, he said. It is simply common courtesy if he takes time to review the code that those comments should be acted upon or responded to. It's fine to disagree with review comments, but submitters need to say why they disagree. If a patch gets resent, it should be accompanied with a reason for doing so. When reviewer's comments are ignored, they are unlikely to review code the next time.

Maintainer's role

When you follow those rules there are certain things you can expect from him, Kroah-Hartman said, and that you should expect from the other maintainers as well. That statement may make other maintainers mad, he joked, but it is reasonable to expect certain things. For his part, he will review patches within one or two weeks. Other maintainers do an even better job than that, he said, specifically pointing to David Miller as one who often reviews code within 48 hours of its submission. If you don't get a response to a patch within a week, it is fine to ask him what the status is.

He can't promise that he will always give constructive criticism, but he will always give "semi-constructive criticism". Sometimes he is tired or grumpy, so he can't quite get to the full "constructive" level. He will also keep submitters informed of the status of their patch. He has scripts that will help him do so, and let the submitter know when the patch gets merged into his tree or accepted into the mainline. That is unlike some other maintainers, he said, where he has submitted patches that just drop into a "big black hole" before eventually popping up in the mainline three months later.

He ended by putting up a quote from Torvalds ("Publicly making fun of people is half the fun of open source programming. ...") that was made as a comment on one of Kroah-Hartman's Google+ postings. The post was a rant about a driver that had been submitted, which even contained comments suggesting that it should not be submitted upstream. He felt bad about publicly posting that at first, but Torvalds's comment made him rethink that.

Because kernel development is done in the open, we are taking "personal pride in the work we do". As the code comment indicated, the driver developer didn't think it should be submitted because they realized the code was not in the proper shape to do so. It is that pride in the work that "makes Linux the best engineering project ever", he said. Sometimes public mocking is part of the process and can actually help instill that pride more widely.

[ The author would like to thank the Linux Foundation for assistance with his travel to Yokohama. ]

Comments (68 posted)

Backing up in trees with Obnam 1.0

Lars Wirzenius's new backup tool Obnam was just declared 1.0. There is no shortage of backup options these days, and in some way Wirzenius's decision to scratch his own itch with the project is par for the course. But the program does offer a different feature set than many of its competitors.

For starters, Obnam makes only "snapshot" backups — that is, every backup looks like a complete snapshot of the system: there are not separate "full" and "incremental" backup options. That obviates the need to separately configure full and incremental backups on different schedules, and it similarly simplifies the restoration process. Any snapshot can be restored, without "walking" a chain of deltas from a full backup starting position. In his 1.0 release announcement, Wirzenius argues that full-plus-incremental backups make sense for tape drives, where sequential access favors adding deltas with incremental changes after an initial full backup, but that hard-disk backups make the incremental delta approach pointless.

But the sneaky part is that under the hood, Obnam's snapshots are all incremental, at least in the sense that each snapshot only records changes since the last. The difference is that they are stored in copy-on-write (COW) b-trees like those Btrfs uses for filesystems. Any snapshot can be reconstructed from the b-tree, and individual snapshots can be removed by deleting their node and re-attaching the sub-trees. To make the COW b-tree approach space-efficient, it uses pervasive automatic data de-duplication. The same chunk of data on disk is re-used — both across multiple files and over multiple snapshot generations. In addition to saving space by not duplicating files that have not changed between snapshots, moving or renaming large files does not result in duplicate copies of the bits. By default, Obnam uses one-megabyte chunks, although this setting is adjustable in Obnam's configuration file.

Obnam sports other features of practical value, such as built-in GnuPG encryption, which Wirzenius cited as a weakness in most rsync-based backup tools. It also works with local disks or over the network, including NFS, SMB, and SFTP. Wirzenius admits that the latter protocol is slow, but that SCP (which should be faster) lacks support for tracking information like file removals, which Obnam depends on. In network backup setups, Obnam supports both push (client-initiated) and pull (server-initiated) backup sessions.

Storing and retrieving

Installation requires several of Wirzenius's other code projects, including his B-tree library larch and terminal status-update library ttystatus, plus paramiko a third-party SSH2 library. Most are packaged for Debian (Wirzenius packages his own projects for Debian), but not all of them are available in downstream derivatives like Ubuntu. He provides an Apt repository for the necessary packages; instructions and a link to the repository's signing key are provided on his Obnam tutorial page.

The tutorial goes into further detail about Obnam's data de-duplication with practical examples. You can create a new backup with

    obnam backup ~/projectfoo

and subsequently back up a parent directory with

    obnam backup ~

Rather than re-save the files from projectfoo, the new backup will point to the copy already on disk. Each backup created with Obnam is specific to a directory; you can exclude specific subdirectories with the --exclude= flag, but you cannot backup several directories in a single command.

The tutorial also explains that Obnam automatically saves checkpoints every 100MB while creating a new backup. This is valuable because the initial snapshot is always akin to a full backup in other tools, and can be large enough to introduce failures. Checkpoints are not guaranteed to preserve the entire data set as are regular snapshots; they only allow an interrupted backup to resume without starting over from scratch.

Obnam's basic usage is straightforward; the same obnam backup ~ command that is used to start a new backup in the above example is used verbatim to perform the subsequent snapshots. You store snapshots on a remote repository by appending --repository=URL, specify a filesystem storage location with --output=PATH, and specify a GnuPG encryption key with --encrypt-with=KEYID.

You can restore a directory from a snapshot with

    obnam restore --to=/mnt/recovery-volume ~

(which will restore the most recent snapshot of your home directory to /mnt/recovery-volume). You can optionally restore just a file or a subdirectory from the snapshot with

    obnam restore ~/importantfiles --to=/mnt/recovery-volume ~

You can also specify a specific intermediate snapshot by appending a --generation=N flag to the restore command; you can get a list of the available snapshots by running obnam generations. The obnam verify command checks snapshot data against the files on disk, and obnam fsck checks the internal consistency of the b-tree.

Forgetfulness

The only real confusing part of working with Obnam is the snapshot retention process. You can tell the program to immediately delete older snapshots by running

    obnam forget --keep=7d

(which will keep the most recent seven days' worth of snapshots), or some variation. The wrinkle is that the 7d attribute will keep only one backup per day for those seven days, even if you run Obnam hourly. To keep seven days' worth of hourly snapshots, you would need to specify --keep=168h.

You can set a snapshot retention policy in your configuration file that uses these rules in combination. You can retain hourly, daily, weekly, monthly, and yearly snapshots by providing a comma-separated list. For example, 12h,7d,3m will keep the last 12 hourly snapshots, the last seven daily snapshots, and the last three monthly snapshots. When the numbers start to converge (such as the last 48 hourly snapshots and last two daily snapshots) is when the potential for miscounting sets in; Wirzenius recommends that you try your retention policy on the command line with the --pretend option to simulate results before deploying them in the real world.

In an email, Wirzenius elaborated a bit on those tricky multi-factor retention policies. Each retention rule (e.g., hour, day, or month) is examined separately by Obnam, he said, and a snapshot is kept if it matches any of the rules. So a 48h,2d policy would match 48 hourly snapshots, then match two additional daily snapshots, for 50 total.

As of the 1.0 release, there are a few areas that need improvement, such as managing multiple clients storing snapshots on one repository; Wirzenius says that further thought is required before implementing a real "server mode." For example, two or more machines can run Obnam and push their backups to the same remote repository, and they will be tagged with the hostname of origin. However, Obnam can also be run from the repository machine and "pull" backups from the two remote sources, but in that case each one needs to specify a client name with the --client-name= flag in order for Obnam to keep their metadata separate.

In practice, my interest in backup utilities stems largely from how rarely I make good backups on a regular basis (i.e., paranoia). I may be atypical in that way, but the primary reasons I have abandoned most of the backup utilities I have test driven in the past are the overhead in keeping track of full and incremental backup schedules and the lack of good tools for rotating old backups out without manual intervention. Obnam scores on both of those metrics. If you have a complicated setup with multiple machines, you may find quirks (such as the client name issue or the speed of SFTP) working against you, but Wirzenius is still at work on the code — and he seems quite happy to take bug reports and questions.

Comments (17 posted)

Fedora, secure boot, and an insecure future

The UEFI secure boot mechanism has been the source of a great deal of concern in the free software community, and for good reason: it could easily be a mechanism by which we lose control over our own systems. Recently, Red Hat's Matthew Garrett described how the Fedora distribution planned to handle secure boot in the Fedora 18 release. That posting has inspired a great deal of concern and criticism, though, arguably, about the wrong things.

On a system with secure boot enabled, the hardware will refuse to run any system that has not been signed by a key it recognizes. Secure boot is meant to be a way to thwart boot-time malware by ensuring that only trusted (and unmodified) software gains control of the system. It is not effective as a digital rights management (DRM) mechanism; if you can gain control of the system, it is relatively easy to fool an operating system into thinking that secure boot is in effect when it is not. Providing the degree of control needed for effective DRM requires a trusted platform module (or similar) and associated software.

Secure boot does offer some hope of preventing a system from booting if its bootloader or kernel have been compromised by malware, though, as the "Flame" malware shows, there are limits to how much one can rely on signatures to keep systems secure. Secure boot could also, unfortunately, be effective in preventing booting if the user has tried to install an operating system of his or her choice.

The Windows 8 logo requirements specify that secure boot must be enabled. After some pushback, the requirements have been amended to also say that it should be possible for the owner of a system to disable secure boot or install new keys. It does not say that these actions need to be easy to carry out, though. Given that changing secure boot is a firmware-level operation, users wanting to make changes will be subjecting themselves to the very best sort of user experience that can be created by BIOS developers. It would be entirely unsurprising, for example, if users were forced to hand-enter new keys as long hex strings. For this to be an unpleasant and error-prone process would not be surprising.

Fedora's plan

Developers in the Fedora camp have evidently come to the conclusion that they do not want to force their users to endure such an experience to be able to install Fedora on their systems. So Fedora has chosen to take a different approach. Availing themselves of the Microsoft developer program, they will purchase a Microsoft-signed key for $99, then use that key to sign a minimal bootloader. UEFI-enabled hardware will then consent to boot that bootloader, which will immediately turn around and boot a special version of the GRUB2 bootloader which, in turn, will boot the Fedora kernel. A Fedora system set up in this mode should boot on a system with secure boot enabled with no changes required.

The appeal of this solution is clear: Fedora will "just work" on UEFI systems without forcing (possibly highly non-technical) users to make scary firmware-level changes. But there is a down side as well. The signed bootloader must ensure that it only runs GRUB2 if the GRUB2 binary has been signed by Fedora (using its own key at this point), and GRUB2 will only boot kernels that have been signed by Fedora. GRUB2 will need to be locked down, and the kernel too; the kernel will, for example, only be able to load modules that bear Fedora's signature. Given that, Red Hat's persistent attempts to get signed module enforcement into the kernel despite some interesting resistance make more sense.

Much of the coverage of this plan in the mainstream media bore headlines like "Red Hat to pay Microsoft for the right to run Linux." Such headlines are not strictly true; the payment ($99 total) evidently goes to Verisign, and what is really being paid for is the ability to boot Linux with a minimum of UEFI-caused user inconvenience. The payment for a Microsoft-signed key raises eyebrows, but it is evidently seen as the best response to a bad situation. And perhaps that is just what it is. But it also raises a number of interesting questions.

A good idea?

For example: what guarantees exist that a Microsoft-signed key will continue to be available in the future for a reasonable price? If secure boot takes over, and the only universally-recognized keys are those signed by Microsoft, then Microsoft will have a monopoly on the right to boot an operating system on future hardware. Corporations are, in general, not known for a principled refusal to exploit that kind of position, and this corporation, in particular, is well known indeed for the opposite sort of behavior. One can only assume that the price of such keys would increase in this situation.

Microsoft will also have the right to revoke keys if they can be said to be a threat to the promises given by the secure boot mechanism. That is why Fedora must be careful to limit anything that enables direct access to the hardware; should somebody be able to get such access, the signed Fedora system could be used to attack Windows systems that have secure boot enabled. In theory, all it would take is a kernel security hole to enable this sort of attack; that could then cause the Fedora key to be revoked. A quick check shows about 20 kernel security updates issued by Fedora since the beginning of this year, with multiple vulnerabilities fixed in most. That could lead to a lot of key churn, especially if, as Alan Cox suggests, every kernel hole will require that its certificate be revoked.

Depending on what software is run on a specific system (if it dual-boots Windows and Linux, for example), a revoked key could find itself into the system's "forbidden signatures" database. That would immediately disable the booting of the signed Fedora image, essentially crippling the machine. The amount of joy resulting from such an outcome can be expected to be small.

Some developers have argued that Fedora's plan is a violation of the GNU General Public License, or, at least, of the Fedora project's own guidelines, despite Fedora's efforts to ensure that users retain as much freedom as possible. GPL enforcement actions in this case seem unlikely; there's no shortage of much more severely locked-down Linux systems out there, and they have not been the target of such actions thus far. But there is a definite risk of damage to the Fedora project's image as users discover that they cannot easily install their own kernels, add third-party modules, or run tools like SystemTap.

Finally, there is the risk that Fedora's plan will legitimize the UEFI secure boot mechanism. For now secure boot can be disabled on x86 systems; what if Microsoft, in the future, points to Fedora 18 as an example of how everybody is able to work within the secure boot system and tries to make secure boot mandatory? Thus, some argue, Fedora is giving aid and comfort to those who would most like to take control of our systems away from us.

Why bother?

Given all of this, one might well wonder why Fedora is pursuing this path. Fedora users are not generally known to clamor for locked-down systems that they cannot easily tweak. Without any inside information whatsoever, your editor suggests that there are two entirely plausible reasons for Fedora's attempt to work with secure boot:

• The Fedora project, like many free software projects, would like to have a wider base of users. It fears that, in the absence of a "just works" experience on upcoming hardware, it will lose users to other distributions that might be more willing to make that effort. Some of those users may be lost to Linux altogether.

• The plan starts with a disclaimer that it is not representative in any way of Red Hat's intentions for its enterprise distribution. But it seems clear that there could be actual customer demand for a version of RHEL that runs in the secure boot environment. If one embraces the sort of restrictions that come with enterprise support, the additional rules imposed by secure boot will have a minimal impact, while the apparent benefits are clear. Fedora's role is, among other things, to test out technologies that might go into RHEL; in this case, Fedora's users get to stumble into the secure boot land mines so RHEL users don't have to.

So Fedora's decision to take this approach is not all that surprising. The project has concluded that it is better to restrict user freedom in certain settings to make their life easier in other ways; as Matthew Garrett put it:

For those who do think that the loss of freedom inherent in the Fedora scheme is unacceptable, the time between the present and when Windows 8 hardware starts shipping would be an ideal opportunity to demonstrate better alternatives. But it's not clear what those would be.

Alternatives?

One could simply ignore secure boot, requiring users to disable it before they can install Linux on their machines. That imposes a potentially scary or difficult task on those users; by the specification, secure boot cannot be disabled by the software directly. There may also be resistance from users who see a switch saying "turn off security" and don't want to flip it. This approach will work fine for hard-core Linux users and developers, but seems certain to lose other kinds of users.

An alternative would be to attempt to gain more control of the situation at the hardware level. An example can be seen in Google, which has made a point of ensuring that unlockable Android handsets exist and are available at a reasonable price. Hardware designed to run ChromeOS also, by design, comes with an easily-toggled physical switch that turns off the boot-time checks for users wanting to install their own software. The level of interest in "jailbreaks" for locked-down handsets shows that a lot of users do see value in having full control over the hardware they own. Open (and "open source") hardware has a following; it may be that the only real way to remain in control is to work to ensure that this kind of hardware continues to exist and has a growing market share. There should be a business opportunity here; projects like the Vivaldi tablet show that some people see that opportunity and are trying to pursue it.

In the absence of open hardware, we will continue to be at the mercy of others whose interests are unlikely to be the same as ours (for just about any value of "ours"). That will leave us in a position where attempts to cope like what we're seeing with Fedora seem like the best options available. That does not seem like the path to freedom; it is not why we have spent decades developing free operating systems. Fedora's secure boot plan may be an effective workaround, but leaves the real bug unfixed.

Comments (122 posted)

Page editor: Jonathan Corbet

Security

Responsible disclosure in open source: The crypt() vulnerability

At the end of May, five separate open source projects released patches to close the same security hole in their software. This coordinated release and vulnerability handling is a demonstration that "responsible disclosure" can work, especially in open source.

Responsible disclosure is the practice of security researchers discovering a vulnerability and contacting the software vendor to give them a reasonable time to fix it before the vulnerability is published. It contrasts with the policy of "full disclosure" in which security people publish the full details of any vulnerability immediately, in order to get information to the public as quickly as possible. Mostly, these two terms have shown up in the media as part of controversies, or even legal battles, which pit security researchers against software companies and each other.

While the inflammatory confrontations gain most of the news headlines, it doesn't have to be that way. In fact, among open source projects, it isn't that way most of the time. The recent multi-product Crypt-DES vulnerability patch shows that responsible disclosure can and does work well in the open source world.

The Crypt-DES vulnerability

Robin Xu and Joseph Bonneau at Cambridge University had been investigating how non-ASCII passwords were handled by various systems for more than a year. Bonneau started on this research because of the massive Gawker security breach in 2010. In the course of investigating that, his team uncovered several issues with non-ASCII passwords in commonly used software. While the one at Gawker was quickly addressed — to some degree — he and Xu began a research project on the insecurities introduced by applying algorithms designed for ASCII to Unicode text.

The version of crypt() using the DES algorithm (hereafter crypt-DES) is a simple irreversible hash designed to prevent storing passwords in plain text. Introduced in old Unix days, it had the advantages of easy implementation, portability between systems and programming languages, computational speed, and is hard enough to crack that dictionary attacks and social engineering were generally easier ways to grab passwords. Given the age and limited computational "strength" of crypt-DES, however, this is no longer true; brute-force computation of crypt() passwords is easily done. Programmers are encouraged to use more modern hashing and encryption algorithms, such as SHA1 and Blowfish. The "extended" DES version was introduced in BSDi in the early 1990's, improving the algorithm to have a larger "salt", more rounds of encryption, and also to support passwords longer than eight characters by "folding" them down to eight 7-bit characters using a first round of DES hashing.

The last improvement is the problem which causes the crypt() vulnerability. Crypt-DES was designed for ASCII characters, and programmers who upgraded systems to support Unicode didn't really check to see how crypt-DES would work with Unicode passwords, since by that point crypt-DES was no longer mainstream. As it turns out, the folding is broken; the algorithm regards characters containing the byte 0x80 as a "stop" character and disregards any parts of the password after that byte. In many Unicode encodings, characters — such as the common character À — can contain a 0x80 byte, causing all characters after that one to be disregarded. This means if your password was Àlbanez60, then crypt-DES would match it with any password beginning with À.

This is also a good illustration of how security is a process and not an end result. Crypt-DES was an adequately secure password hashing approach well into the mid 1990's, which is why people stopped testing it. It was the introduction of popular Unicode-compliant versions of programming languages and databases which has made it less secure than anyone realized.

Contacting projects

Having found this issue, Xu, Bonneau, and other Cambridge graduate students spent several weeks examining some common software and found that the defective version of Crypt-DES was still shipping with several open source software packages, among them PostgreSQL and FreeBSD. Having found the vulnerability, they emailed the private security mailing lists for the affected projects.

The PostgreSQL security team received this email on April 24th:

The Cambridge team had previously contacted a few other projects, including FreeBSD. The FreeBSD and PostgreSQL projects had to decide what to do about patching the vulnerability. For anyone affected by it, an updated version of crypt-DES would require that all affected passwords (ones containing the 0x80 byte) be regenerated. While neither PostgreSQL nor FreeBSD used crypt-DES for system authentication, both supply functions which are used to hash application passwords. Because of this disruption to some users' applications, it couldn't be done casually.

The FreeBSD security team contacted the OpenBSD, NetBSD and DragonflyBSD projects. Rubin Xu's research indicated that PHP's crypt() also had the faulty algorithm, and had attempted to contact the PHP security team without success. Members of FreeBSD contacted them and brought them into the discussion. NetBSD turned out not to be vulnerable.

Coordinating a release

Among the affected projects, this vulnerability was considered moderate in severity, since it only affected a minority of users of each project. Not only did users need to build applications using crypt() with DES, despite other, more modern hashing options being available, but the password vulnerability only affects passwords with Unicode characters including the 0x80 byte. Specifically, the vulnerability was limited to:

• BSD users who used the included crypt() library with DES (the default) to support their applications.
• PostgreSQL users who installed the optional pgCrypto extension and used its crypt() function with the default DES encryption.
• PHP users who used the PHP crypt() function with DES on platforms without native encryption support, most notably Windows.

On the other hand, the vulnerability affects passwords, which means it's specifically a hole in code people have written to secure their systems. That raised this vulnerability from obscure to moderately serious. So FreeBSD filed for a Common Vulnerability and Exposure number (CVE), and the projects began trying to coordinate a release.

From the perspective of the projects, once one project announced a release and CVE-2012-2143 became public, it wouldn't take much cleverness for a even a newbie black hat to figure out the vulnerability in other products. That meant coordinating a release date among five different projects. In surprisingly short order, they reached a compromise date of May 30th, which was the earliest reasonable release date. On that date:

• FreeBSD released a security update.
• OpenBSD updated its hotfixes repository.
• DragonflyBSD released a security update.
• PostgreSQL released a source code patch for pgCrypto, with a full update release to follow on June 4.
• PHP released a new Release Candidate with the fix, although it did not publicize the issue.

The entire timeline from the discovery of the vulnerability to deploying fixes for multiple projects took about three months. The majority of this time (about seven weeks) was taken up by the researchers finding and contacting affected projects. If there's room for improvement in the process of responsible discovery, it's that finding affected projects or products and contacting their security teams is slow and time-consuming. The remaining five weeks is only two weeks longer than the minimum time for most packaged projects to do a release at all, due to packaging, scheduling, testing, and coordination requirements. One could easily argue that immediate disclosure would have gotten the news about the vulnerability out much sooner, but it's not clear how that would have benefited affected users until fixes for their software were available.

In relatively short order, five major open source projects were updated to close it. Nobody was threatened, no single project's users or developers were singled out, the security researchers were thanked for their work, and nobody needed to spend more than a few hours of their time getting the fixes made and released. At least from the perspective of software maintainers and regular users, this episode looks like a success.

Why it worked this time

This whole episode had two important factors to make it a relative success: the security researchers were university staff unmotivated by fame or profit, and the open source projects are community non-profits lacking incentives to defer or deny patching security holes. This meant that everyone involved was motivated to fix the vulnerability in the fastest, most responsible way possible.

This is by no means exceptional in the open source world. On the PostgreSQL project today, as with many other open source projects, companies and academic researchers regularly practice responsible disclosure, letting the developers know about a security issue in a reasonable time to fix it. If anything, this is the rule in the non-profit open source world. So why does disclosure cause friction, user exposure, blog wars, and legal threats in the for-profit world?

Well, when you look at failures of security disclosure, the overwhelming trend is bad faith. Software companies don't want to do expensive releases and get bad press for security issues, so they put off security researchers forever, or even threaten them. Security people or their employers want fame and attention so they publicize security holes as widely as possible without verification, or giving the vendor a chance to patch issues. Or worse, researchers, companies and agencies participate in a marketplace of secret security exploits.

So, while responsible disclosure can and does work in the non-profit open source world, it's not clear how to transfer these practices to the for-profit world, or even if it's possible to do so. Maybe the answer is simply to use more open source software.

[ Note that MITRE has not updated their CVE database. As such, the CVE link for the exploit will still show as "pending". ]

Comments (26 posted)

Brief items

Security quotes of the week

-- Alan Cox

-- Matthew Garrett

-- Erik van Straten

-- Dan Goodin

Comments (none posted)

Critical vulnerability derails Ruby on Rails (The H)

The H reports on a newly-discovered SQL injection vulnerability in Ruby on Rails, affecting the 3.0.x, 3.1.x, and 3.2.x versions. "The vulnerability exists in versions 3.0 and later of Active Record, Rail's database layer, and is exposed when using nested query parameters. Code that directly passes parameters to a where method, is affected. For example, using the common idiom params[:id] can be tricked into returning a crafted hash which causes the generated SQL statement to query an arbitrary table." The Rails team pushed out a fix, but shortly thereafter had to follow it up with another.

Comments (4 posted)

New vulnerabilities

apache-commons-compress: denial of service

Package(s):	apache-commons-compress	CVE #(s):	CVE-2012-2098
Created:	June 4, 2012	Updated:	May 13, 2013
Description:	From the CVE entry: Algorithmic complexity vulnerability in the sorting algorithms in bzip2 compressing stream (BZip2CompressorOutputStream) in Apache Commons Compress before 1.4.1 allows remote attackers to cause a denial of service (CPU consumption) via a file with many repeating inputs. Fixed in version 1.4.1.
Alerts:	Fedora FEDORA-2012-8428 2012-06-02 Fedora FEDORA-2012-8465 2012-06-03 Fedora FEDORA-2013-5548 2013-05-11 Fedora FEDORA-2013-5546 2013-05-11

Comments (none posted)

arpwatch: privilege escalation

Package(s):	arpwatch	CVE #(s):	CVE-2012-2653
Created:	June 4, 2012	Updated:	April 5, 2013
Description:	From the Debian advisory: Steve Grubb from Red Hat discovered that a patch for arpwatch (as shipped at least in Red Hat and Debian distributions) in order to make it drop root privileges would fail to do so and instead add the root group to the list of the daemon uses.
Alerts:	Debian DSA-2481-1 2012-06-02 Debian DSA-2482-1 2012-06-02 Fedora FEDORA-2012-8677 2012-06-20 Fedora FEDORA-2012-8675 2012-06-20 Fedora FEDORA-2012-8702 2012-06-20 Mageia MGASA-2012-0129 2012-06-27 openSUSE openSUSE-SU-2012:0915-1 2012-07-25 Mandriva MDVSA-2012:113 2012-07-26 Mandriva MDVSA-2013:017 2013-04-04 Mandriva MDVSA-2013:030 2013-04-05

Comments (none posted)

bind: multiple vulnerabilities

Package(s):	bind9	CVE #(s):	CVE-2012-1667 CVE-2012-1033
Created:	June 6, 2012	Updated:	August 7, 2012
Description:	From the Dan Luther discovered that Bind incorrectly handled zero length rdata fields. A remote attacker could use this flaw to cause Bind to crash or behave erratically, resulting in a denial of service. (CVE-2012-1667) It was discovered that Bind incorrectly handled revoked domain names. A remote attacker could use this flaw to cause malicious domain names to be continuously resolvable even after they have been revoked. (CVE-2012-1033)
Alerts:	Ubuntu USN-1462-1 2012-06-05 Debian DSA-2486-1 2012-06-05 Red Hat RHSA-2012:0716-01 2012-06-07 Red Hat RHSA-2012:0717-01 2012-06-07 CentOS CESA-2012:0716 2012-06-07 CentOS CESA-2012:0716 2012-06-07 CentOS CESA-2012:0717 2012-06-07 Oracle ELSA-2012-0716 2012-06-08 Oracle ELSA-2012-0716 2012-06-08 Oracle ELSA-2012-0717 2012-06-08 Scientific Linux SL-bind-20120607 2012-06-07 Scientific Linux SL-bind-20120607 2012-06-07 Mandriva MDVSA-2012:089 2012-06-10 openSUSE openSUSE-SU-2012:0722-1 2012-06-11 Fedora FEDORA-2012-8968 2012-06-13 SUSE SUSE-SU-2012:0741-2 2012-06-15 SUSE SUSE-SU-2012:0741-1 2012-06-15 Slackware SSA:2012-166-01 2012-06-14 Fedora FEDORA-2012-8962 2012-06-15 Fedora FEDORA-2012-8946 2012-06-15 SUSE SUSE-SU-2012:0741-3 2012-06-16 SUSE SUSE-SU-2012:0741-4 2012-06-18 SUSE SUSE-SU-2012:0741-5 2012-06-27 openSUSE openSUSE-SU-2012:0863-1 2012-07-11 openSUSE openSUSE-SU-2012:0864-1 2012-07-11 SUSE SUSE-SU-2012:0741-6 2012-07-13 Red Hat RHSA-2012:1110-01 2012-07-23 Oracle ELSA-2012-2028 2012-08-06 Gentoo 201209-04 2012-09-23 Slackware SSA:2012-341-01 2012-12-06 openSUSE openSUSE-SU-2013:0605-1 2013-04-03

Comments (none posted)

drupal7: full path disclosure

Package(s):	drupal7	CVE #(s):	CVE-2012-2922
Created:	June 4, 2012	Updated:	November 2, 2012
Description:	From the CVE entry: The request_path function in includes/bootstrap.inc in Drupal 7.14 and earlier allows remote attackers to obtain sensitive information via the q[] parameter to index.php, which reveals the installation path in an error message.
Alerts:	Fedora FEDORA-2012-8362 2012-06-02 Fedora FEDORA-2012-8360 2012-06-02 Fedora FEDORA-2012-8398 2012-06-03 Mageia MGASA-2012-0319 2012-11-01 Mandriva MDVSA-2013:074 2013-04-08

Comments (none posted)

firefox: multiple vulnerabilities

Package(s):	firefox	CVE #(s):	CVE-2011-3101 CVE-2012-1937 CVE-2012-1938 CVE-2012-1939 CVE-2012-1940 CVE-2012-1941 CVE-2012-1944 CVE-2012-1945 CVE-2012-1946 CVE-2012-1947
Created:	June 6, 2012	Updated:	January 8, 2013
Description:	From the Red Hat advisory: Several flaws were found in the processing of malformed web content. A web page containing malicious content could cause Firefox to crash or, potentially, execute arbitrary code with the privileges of the user running Firefox. (CVE-2011-3101, CVE-2012-1937, CVE-2012-1938, CVE-2012-1939, CVE-2012-1940, CVE-2012-1941, CVE-2012-1946, CVE-2012-1947) Note: CVE-2011-3101 only affected users of certain NVIDIA display drivers with graphics cards that have hardware acceleration enabled. It was found that the Content Security Policy (CSP) implementation in Firefox no longer blocked Firefox inline event handlers. A remote attacker could use this flaw to possibly bypass a web application's intended restrictions, if that application relied on CSP to protect against flaws such as cross-site scripting (XSS). (CVE-2012-1944) If a web server hosted HTML files that are stored on a Microsoft Windows share, or a Samba share, loading such files with Firefox could result in Windows shortcut files (.lnk) in the same share also being loaded. An attacker could use this flaw to view the contents of local files and directories on the victim's system. This issue also affected users opening HTML files from Microsoft Windows shares, or Samba shares, that are mounted on their systems. (CVE-2012-1945)
Alerts:	Red Hat RHSA-2012:0710-01 2012-06-05 CentOS CESA-2012:0710 2012-06-06 CentOS CESA-2012:0710 2012-06-06 Scientific Linux SL-fire-20120606 2012-06-06 Ubuntu USN-1463-1 2012-06-06 Red Hat RHSA-2012:0715-01 2012-06-06 Scientific Linux SL-thun-20120607 2012-06-07 Oracle ELSA-2012-0710 2012-06-06 Oracle ELSA-2012-0715 2012-06-07 CentOS CESA-2012:0715 2012-06-07 Debian DSA-2489-1 2012-06-07 Debian DSA-2488-1 2012-06-07 Mandriva MDVSA-2012:088 2012-06-09 SUSE SUSE-SU-2012:0746-1 2012-06-15 openSUSE openSUSE-SU-2012:0760-1 2012-06-19 Ubuntu USN-1463-3 2012-06-20 Slackware SSA:2012-166-02 2012-06-14 Slackware SSA:2012-166-03 2012-06-14 Slackware SSA:2012-166-04 2012-06-14 Ubuntu USN-1463-4 2012-06-22 Debian DSA-2499-1 2012-06-24 Mandriva MDVSA-2012:088-1 2012-06-23 Ubuntu USN-1463-5 2012-06-26 Ubuntu USN-1463-6 2012-06-26 Mageia MGASA-2012-0135 2012-06-28 Mageia MGASA-2012-0136 2012-06-29 Mageia MGASA-2012-0176 2012-07-21 Gentoo 201301-01 2013-01-07

Comments (none posted)

globus-gridftp-server: privilege escalation

Package(s):	globus-gridftp-server	CVE #(s):
Created:	June 4, 2012	Updated:	June 6, 2012
Description:	GridFTP acts as the wrong user when user doesn't exist. See this globus advisory for details.
Alerts:	Fedora FEDORA-2012-8445 2012-06-03 Fedora FEDORA-2012-8461 2012-06-03 Fedora FEDORA-2012-8488 2012-06-03 Fedora FEDORA-2012-8445 2012-06-03 Fedora FEDORA-2012-8461 2012-06-03 Fedora FEDORA-2012-8488 2012-06-03

Comments (none posted)

kernel: multiple vulnerabilities

Package(s):	kernel	CVE #(s):	CVE-2012-2127 CVE-2012-2319
Created:	June 4, 2012	Updated:	October 3, 2012
Description:	From the SUSE advisory: CVE-2012-2127: Various leaks in namespace handling over fork where fixed, which could be exploited by e.g. vsftpd access by remote users. CVE-2012-2319: A memory corruption when mounting a hfsplus filesystem was fixed that could be used by local attackers able to mount filesystem to crash the system.
Alerts:	SUSE SUSE-SU-2012:0689-1 2012-06-02 Ubuntu USN-1471-1 2012-06-12 Ubuntu USN-1472-1 2012-06-12 Ubuntu USN-1473-1 2012-06-13 Ubuntu USN-1474-1 2012-06-13 Ubuntu USN-1476-1 2012-06-15 SUSE SUSE-SU-2012:0736-1 2012-06-14 Ubuntu USN-1488-1 2012-06-29 Ubuntu USN-1490-1 2012-06-29 Ubuntu USN-1491-1 2012-06-29 Ubuntu USN-1492-1 2012-06-29 Ubuntu USN-1493-1 2012-06-29 Ubuntu USN-1530-1 2012-08-10 Red Hat RHSA-2012:1323-01 2012-10-02 CentOS CESA-2012:1323 2012-10-03 Scientific Linux SL-kern-20121003 2012-10-03 Oracle ELSA-2012-1323 2012-10-03 Oracle ELSA-2012-1323 2012-10-04 Ubuntu USN-1594-1 2012-10-03 Red Hat RHSA-2012:1347-01 2012-10-09 Ubuntu USN-1607-1 2012-10-11

Comments (none posted)

moodle: multiple vulnerabilities

Package(s):	moodle	CVE #(s):	CVE-2012-2353 CVE-2012-2354 CVE-2012-2355 CVE-2012-2356 CVE-2012-2357 CVE-2012-2358 CVE-2012-2359 CVE-2012-2360 CVE-2012-2361 CVE-2012-2362 CVE-2012-2363 CVE-2012-2364 CVE-2012-2365 CVE-2012-2366 CVE-2012-2367
Created:	June 1, 2012	Updated:	August 2, 2012
Description:	From the Fedora advisory: Update Information: CVE-2012-2353 MSA-12-0024: Hidden information access issue CVE-2012-2354 MSA-12-0025: Personal communication access issue CVE-2012-2355 MSA-12-0026: Quiz capability issue CVE-2012-2356 MSA-12-0027: Question bank capability issues CVE-2012-2357 MSA-12-0028: Insecure authentication issue CVE-2012-2358 MSA-12-0029: Information editing access issue CVE-2012-2359 MSA-12-0030: Capability manipulation issue CVE-2012-2360 MSA-12-0031: Cross-site scripting vulnerability in Wiki CVE-2012-2361 MSA-12-0032: Cross-site scripting vulnerability in Web services CVE-2012-2362 MSA-12-0033: Cross-site scripting vulnerability in Blog CVE-2012-2363 MSA-12-0034: Potential SQL injection issue CVE-2012-2364 MSA-12-0035: Cross-site scripting vulnerability in "download all" CVE-2012-2365 MSA-12-0036: Cross-site scripting vulnerability in category identifier CVE-2012-2366 MSA-12-0037: Write access issue in Database activity module CVE-2012-2367 MSA-12-0038: Calendar event write permission issue Correct CAS unbundling. Drop bundled language packs. New upstreams, multiple vulnerabilities.
Alerts:	Fedora FEDORA-2012-8284 2012-06-01 Fedora FEDORA-2012-8325 2012-06-02 Fedora FEDORA-2012-8365 2012-06-02

Comments (none posted)

nut: application crash

Package(s):	nut	CVE #(s):	CVE-2012-2944
Created:	June 1, 2012	Updated:	September 28, 2012
Description:	From the Ubuntu advisory: Nut could be made to crash if it received specially crafted network traffic.
Alerts:	Ubuntu USN-1456-1 2012-05-31 Debian DSA-2484-1 2012-06-02 Mandriva MDVSA-2012:087 2012-06-05 Fedora FEDORA-2012-8747 2012-06-15 Fedora FEDORA-2012-8729 2012-06-15 Fedora FEDORA-2012-8694 2012-06-15 Mageia MGASA-2012-0155 2012-07-10 openSUSE openSUSE-SU-2012:1069-1 2012-08-30 Gentoo 201209-19 2012-09-27

Comments (none posted)

openoffice.org: code execution

Package(s):	openoffice.org	CVE #(s):	CVE-2012-2334
Created:	June 5, 2012	Updated:	June 14, 2012
Description:	From the Red Hat advisory: An integer overflow flaw, leading to a buffer overflow, was found in the way OpenOffice.org processed an invalid Escher graphics records length in Microsoft Office PowerPoint documents. An attacker could provide a specially-crafted Microsoft Office PowerPoint document that, when opened, would cause OpenOffice.org to crash or, potentially, execute arbitrary code with the privileges of the user running OpenOffice.org.
Alerts:	Red Hat RHSA-2012:0705-01 2012-06-05 CentOS CESA-2012:0705 2012-06-05 CentOS CESA-2012:0705 2012-06-05 Scientific Linux SL-open-20120605 2012-06-05 Oracle ELSA-2012-0705 2012-06-05 Debian DSA-2487-1 2012-06-07 Fedora FEDORA-2012-8114 2012-06-13 Mandriva MDVSA-2012:090 2012-06-14 Mandriva MDVSA-2012:091 2012-06-14 Mandriva MDVSA-2012:091 2012-06-15 Ubuntu USN-1495-1 2012-07-02 Ubuntu USN-1496-1 2012-07-02 Mageia MGASA-2012-0253 2012-09-04

Comments (none posted)

php: integer overflow

Package(s):	php5	CVE #(s):	CVE-2012-2386
Created:	June 5, 2012	Updated:	July 16, 2012
Description:	From the openSUSE advisory: Specially crafted tar archives could cause an integer overflow in the phar extension
Alerts:	openSUSE openSUSE-SU-2012:0695-1 2012-06-05 Debian DSA-2492-1 2012-06-10 Mandriva MDVSA-2012:093 2012-06-15 Ubuntu USN-1481-1 2012-06-19 Red Hat RHSA-2012:1046-01 2012-06-27 Red Hat RHSA-2012:1047-01 2012-06-27 CentOS CESA-2012:1047 2012-06-27 Oracle ELSA-2012-1047 2012-06-28 Fedora FEDORA-2012-9490 2012-06-30 Fedora FEDORA-2012-9490 2012-06-30 Oracle ELSA-2012-1046 2012-06-30 Fedora FEDORA-2012-9762 2012-07-02 Fedora FEDORA-2012-9762 2012-07-02 Fedora FEDORA-2012-9762 2012-07-02 SUSE SUSE-SU-2012:0840-1 2012-07-05 Scientific Linux SL-php5-20120705 2012-07-05 Scientific Linux SL-php-20120709 2012-07-09 CentOS CESA-2012:1046 2012-07-10 Slackware SSA:2012-195-01 2012-07-14 Gentoo 201209-03 2012-09-23

Comments (none posted)

postgresql: multiple vulnerabilities

Package(s):	postgresql-8.3, postgresql-8.4, postgresql-9.1	CVE #(s):	CVE-2012-2143 CVE-2012-2655
Created:	June 5, 2012	Updated:	September 28, 2012
Description:	From the Ubuntu advisory: It was discovered that PostgreSQL incorrectly handled certain bytes passed to the crypt() function when using DES encryption. An attacker could use this flaw to incorrectly handle authentication. (CVE-2012-2143) It was discovered that PostgreSQL incorrectly handled SECURITY DEFINER and SET attributes on procedural call handlers. An attacker could use this flaw to cause PostgreSQL to crash, leading to a denial of service. (CVE-2012-2655)
Alerts:	Ubuntu USN-1461-1 2012-06-05 Debian DSA-2491-1 2012-06-09 Mandriva MDVSA-2012:093 2012-06-15 Mandriva MDVSA-2012:092 2012-06-15 Fedora FEDORA-2012-8915 2012-06-15 Fedora FEDORA-2012-8893 2012-06-15 Fedora FEDORA-2012-8924 2012-06-15 Ubuntu USN-1481-1 2012-06-19 Red Hat RHSA-2012:1036-01 2012-06-25 Red Hat RHSA-2012:1037-01 2012-06-25 CentOS CESA-2012:1036 2012-06-25 CentOS CESA-2012:1037 2012-06-25 Oracle ELSA-2012-1036 2012-06-26 Oracle ELSA-2012-1037 2012-06-26 Red Hat RHSA-2012:1046-01 2012-06-27 Red Hat RHSA-2012:1047-01 2012-06-27 CentOS CESA-2012:1047 2012-06-27 Oracle ELSA-2012-1047 2012-06-28 Fedora FEDORA-2012-9490 2012-06-30 Fedora FEDORA-2012-9490 2012-06-30 Oracle ELSA-2012-1037 2012-06-30 Oracle ELSA-2012-1046 2012-06-30 Fedora FEDORA-2012-9762 2012-07-02 Fedora FEDORA-2012-9762 2012-07-02 Fedora FEDORA-2012-9762 2012-07-02 openSUSE openSUSE-SU-2012:0826-1 2012-07-04 SUSE SUSE-SU-2012:0840-1 2012-07-05 Scientific Linux SL-php5-20120705 2012-07-05 Scientific Linux SL-post-20120705 2012-07-05 Scientific Linux SL-post-20120709 2012-07-09 Scientific Linux SL-php-20120709 2012-07-09 CentOS CESA-2012:1037 2012-07-10 CentOS CESA-2012:1046 2012-07-10 Oracle ELSA-2012-1263 2012-09-14 Gentoo 201209-03 2012-09-23 openSUSE openSUSE-SU-2012:1251-1 2012-09-26 Gentoo 201209-24 2012-09-28 openSUSE openSUSE-SU-2012:1288-1 2012-10-04 openSUSE openSUSE-SU-2012:1299-1 2012-10-06

Comments (none posted)

python-crypto: insecure key generation

Package(s):	python-crypto	CVE #(s):	CVE-2012-2417
Created:	June 4, 2012	Updated:	April 10, 2013
Description:	From the Red Hat bugzilla: A security flaw was found in the implementation of ElGamal algorithm of python-crypto, a cryptography library for Python language, in the way how random number 'g' was generated.
Alerts:	Fedora FEDORA-2012-8392 2012-06-03 Fedora FEDORA-2012-8470 2012-06-03 Fedora FEDORA-2012-8490 2012-06-03 Debian DSA-2502-1 2012-06-24 Gentoo 201206-23 2012-06-24 Ubuntu USN-1484-1 2012-06-28 openSUSE openSUSE-SU-2012:0830-1 2012-07-04 Mandriva MDVSA-2012:117 2012-07-27 Mageia MGASA-2012-0194 2012-08-02 Mandriva MDVSA-2013:120 2013-04-10

Comments (none posted)

rubygem-rack-cache: information leak

Package(s):	rubygem-rack-cache	CVE #(s):
Created:	June 6, 2012	Updated:	June 6, 2012
Description:	From the Red Hat bugzilla: The Rack::Cache rubygem has a flaw where it will cache sensitive headers (such as Set-Cookie response headers), which could leak potentially sensitive information.
Alerts:	Fedora FEDORA-2012-8439 2012-06-05

Comments (none posted)

socat: buffer overflow

Package(s):	socat	CVE #(s):	CVE-2012-0219
Created:	June 1, 2012	Updated:	May 29, 2013
Description:	From the Fedora advisory: Fix for CVE-2012-0219 heap-based buffer overflow
Alerts:	Fedora FEDORA-2012-8274 2012-06-01 Fedora FEDORA-2012-8328 2012-06-07 openSUSE openSUSE-SU-2012:0809-1 2012-07-02 Mageia MGASA-2012-0138 2012-07-09 Gentoo 201208-01 2012-08-14 Mandriva MDVSA-2013:127 2013-04-10 Mandriva MDVSA-2013:169 2013-05-29

Comments (none posted)

Page editor: Nathan Willis

Kernel development

Brief items

Kernel release status

The current development kernel is 3.5-rc1, announced on June 3. "It's a pretty normal release - roughly 60% drivers, 20% arch updates, and 20% "all over" - filesystems, documentation, tools, you name it. [...] Depending on what your interest is, you might be excited about the CoDel packet scheduler, or about the GPU driver updates, or the new scsi targets. There is something in here for pretty much anybody."

Stable updates: the 3.0.33, 3.2.19, 3.3.8, and 3.4.1 stable kernels were all released on June 4; each contains a long list of fixes. 3.3.8 is the final update for the 3.3 series.

Comments (none posted)

Quotes of the week

— Al Viro

— Hugh Dickins

— Steven Rostedt

— Peter Zijlstra doesn't accept poorly-described patches before Tuesday

Comments (3 posted)

Kernel development news

The conclusion of the 3.5 merge window

Linus closed the 3.5 merge window on June 2; the announcement for the 3.5 prepatch came one day later. Just under 1,000 patches were pulled into the mainline after the writing of last week's merge window summary, for a total of 9,534 for the merge window as a whole. Some of the more significant changes pulled at the end of the merge window include:

• The HFS filesystem has gained native language support (NLS) capabilities, with codepages added for several languages.

• A process's directory under /proc now includes a children file containing the IDs of its child processes.

• The kcmp() system call has been added. Its purpose is to help user space checkpoint/restore utilities to determine whether two processes share a given resource or not; see this article for a description of the interface.

• Also for checkpoint/restore: the prctl() system call has gained options to set the beginning and end of the argv and environment areas and the executable file a process is running.

• The ext4 filesystem now has support for metadata checksumming, a feature which should help to catch metadata corruption that might otherwise escape notice. This feature requires an on-disk format change (to store the checksum), so it must be turned on explicitly. Once this feature is enabled, the filesystem can only be mounted read-only on older kernels. See this article for more information on the metadata checksum feature.

• A lot of changes have gone in to improve the handling of system-specific features on Sony laptops.

• The new skew_tick= boot option controls whether the system skews timer ticks on a per-CPU basis to minimize contention on the xtime_lock lock. It defaults to off; turning it on can reduce jitter on some workloads, but will also increase power consumption.

• The "frontswap" mechanism, part of the transcendent memory family of technologies, sneaked its way into the mainline just after the -rc1 release.

• The FUSE filesystem API has added an operation to implement the fallocate() system call.

• Two new drivers were added at the end of the merge window; they enable AUO K1900 and K1901 epaper display controllers and Emma Mobile STI timers.

Changes visible to kernel developers include:

• The task_work_add() function, useful for requesting that a function be run in the context of a specific process, has been added. See this article for a description of the task_work_add() API.

• The SUNRPC code has a new utility function:

      int svc_bind(struct svc_serv *serv, struct net *net);
  

  Its purpose is to handle service registration in the given network context; svc_bind() is exported GPL-only.

• struct inode_operations has a new update_time() function whose job is to provide any needed special handling for changes to any of the file timestamps. The file_update_time() prototype has been changed: it now returns an int that can indicate that the operation failed. Failures to update the last-access time are now explicitly ignored; this is done to ensure that atime update failures don't make the filesystem unreadable. This work has been generalized out of the btrfs filesystem; see this article for a description of how atime updates can go wrong otherwise.

At this point, the addition of features for the 3.5 development cycle should be at an end unless something sneaks in before -rc2. If the usual pattern holds, the final 3.5 release can be expected right around the beginning of August.

Comments (none posted)

Generic red-black trees

Red-black trees (or "rbtrees") are used throughout the kernel to maintain a sorted list of related items. For example, the block I/O subsystem uses an rbtree to maintain a list of outstanding requests sorted by sector number, and the scheduler has an rbtree of runnable tasks sorted by a quantity known as "virtual runtime." The rbtree interface was described in this 2006 article; it has since been extended, but the core features of the API remain the same. In particular, users must provide their own functions for inserting nodes into the tree and performing searches; that allows the creation of highly-optimized rbtrees containing arbitrary types of structures.

There is some appeal to being able to hand-code the search and insertion functions, but there would also be value in generic implementations. The amount of code in the kernel would shrink slightly and the task of debugging those functions would, hopefully, only have to be done once. So it is arguably surprising that nobody has proposed a generic rbtree implementation for all these years. Just as surprising is the fact that two independent generic implementations surfaced at about the same time.

The simpler of the two can be found in this patch set from Kent Overstreet. In this version, rbtree users are required to provide a function to compare two rbtree nodes with this prototype:

    typedef int (rb_cmp_t) (struct rb_node *l, struct rb_node *r);

With that in place, the following new functions become available:

    int rb_insert(struct rb_root *root, struct rb_node *new, rb_cmp_t cmp);
    void rb_insert_allow_dup(struct rb_root *root, struct rb_node *new,
			     rb_cmp_t cmp);
    struct rb_node *rb_search(struct rb_root *root, struct rb_node *search,
			      rb_cmp_t cmp);
    struct rb_node *rb_greater(struct rb_root *root, struct rb_node *search,
			       rb_cmp_t cmp);

As can be seen from the prototypes, all of these functions deal directly with rb_node structures. Only the comparison function needs to know about what sort of structure the rb_node is embedded within. There is no compile-time mechanism to ensure that the comparison function expects the actual structures found in the tree, but one assumes any errors along those lines will show themselves quickly at run time.

One potential problem is that rb_search() and rb_greater() need to know what is being searched for; that, in turn, requires creating and passing in one of the structures stored in the tree. In some situations, that structure may need to be created on the stack, which is a clear problem if the structure is large. Unfortunately, in the block subsystem example mentioned above, that structure (struct request) is large indeed. Kent has tried to work around this problem by providing inlined versions (called _rb_search() and _rb_greater()) that, with luck, will cause the stack allocation to be optimized away. That depends on all supported versions of the compiler doing the right thing on all architectures, though, which may make some people nervous.

The alternative patch set, posted by Daniel Santos, is significantly more complicated. It can be thought of as a set of tricky preprocessor macros and inline functions that serve as a template for the creation of type-specific rbtree implementations. Here, too, one starts with the creation of a comparison function:

    typedef long (*rb_compare_f)(const void *a, const void *b);

In this case, the comparison function will be passed pointers to the actual key value stored in the rbtree node. One then defines an "rbtree interface" with this daunting macro:

    RB_DEFINE_INTERFACE(prefix, container_type, root_member,
			left_pointer, right_pointer,
			object_type, rbnode_member, key_member,
			flags, comparison_function, augment_func);

Eleven arguments are a lot to keep track of, so it makes sense to discuss them in the context of an example (taken from Daniel's patch). The CPU scheduler defines a type (struct cfs_rq in kernel/sched/sched.h) to represent a run queue; each run queue contains a red-black tree called tasks_timeline. To optimize the location of the highest-priority task to run, the scheduler keeps a pointer to the leftmost tree node in rb_leftmost. The entries in the red-black tree are of type struct sched_entity (defined in <linux/sched.h>); the embedded rb_node structure is called run_node, and the key used to sort the tree is the u64 value vruntime.

To create the scheduler's tree using the generic mechanism, Daniel adds this declaration:

    RB_DEFINE_INTERFACE(
	fair_tree,
	struct cfs_rq, tasks_timeline, rb_leftmost, /* no rightmost */,
	struct sched_entity, run_node, vruntime,
	0, compare_vruntime, 0)

Here, fair_tree is the "prefix" value used to generate the names of the tree functions—see below. The next line describes the structure containing the tree (struct cfs_rq), the name of the tree, and the name of the leftmost pointer used by the scheduler; there is no rightmost pointer (nobody cares about the lowest-priority task), so that parameter is simply left blank. The line after that describes the structure contained within the tree (struct sched_entity), the name of its embedded rb_node structure, and the name of the key value. Finally, no flags are given, compare_vruntime() is the comparison function, and, since this is not an augmented tree, there is no augmented callback function. Yes, it lacks a semicolon—the macro supplies that itself.

The result is a new set of functions like:

    struct sched_entity *fair_tree_insert(struct cfs_rq runqueue, 
					  struct sched_entity *entity);
    void fair_tree_remove(struct cfs_rq runqueue, struct sched_entity *entity);
    struct sched_entity *fair_tree_find(struct cfs_rq runqueue, u64 *key);
    /* ... */

These functions are all defined with the proper type, so the compiler will ensure that they are always called with the proper argument types. Everything is defined as __always_inline, so the implementations will be inlined at the place where they are called. That should eliminate any performance penalty caused by out-of-line implementations (as seen in Kent's patch), but, so far, nobody seems to have tried to measure that penalty.

There has been little in the way of review of these patches in general. They represent different approaches to the task of creating a generic red-black implementation, one emphasizing simplicity while the other emphasizes explicitness and type safety. Which version might be inserted into the mainline kernel—if either goes in—is entirely unclear at this point.

Comments (16 posted)

Volatile ranges with fallocate()

Last November LWN looked at the volatile ranges patch set from John Stultz. This patch is intended to bring an Android feature into the mainline, but it is a reimplemented feature that is more deeply tied into the memory management subsystem. That patch has now returned, but the API has changed so another look is warranted.

A "volatile range" is a set of pages in memory containing data that might be useful to an application at some point in the future; a key point is that, if the need arises, the application is able to reacquire (or regenerate) that data from another source. A web browser's in-RAM image cache is a classic example. Keeping the images around can reduce net traffic and improve page rendering times, but, should the cached images vanish, the browser can request a new copy from the source. Thus, while it makes sense to keep this data around, it also makes sense to get rid of it if a more pressing need for memory arises.

If the kernel knew about this sort of cached data, it could dump that data during times of memory stress and quickly reclaim the underlying memory. In such a situation, applications could cache more data than they otherwise would, knowing that there are limits to how much that caching can affect the system as a whole. The result would be better utilization of memory and a system that performs better overall.

Google's Robert Love implemented such a mechanism for Android as "ashmem." There is a desire to get the ashmem functionality into the mainline kernel, but the implementation and API were not to everybody's taste. To get around that problem, John took the core ashmem code, reworked the virtual memory integration, and hooked it into the posix_fadvise() system call; that is the version of the patch that was described last November.

Dave Chinner subsequently pointed out that this functionality might be better suited to the fallocate() system call. That call looks like this:

    int fallocate(int fd, int mode, off_t offset, off_t len);

This system call is meant to operate on ranges of data within a file. Of particular interest, perhaps, is the FALLOCATE_FL_PUNCH_HOLE operation, which removes a block of data from an arbitrary location within a file. Declaring a volatile range can be thought of as a form of hole punching, but with a kernel-determined delay. If memory is tight, the hole could be punched immediately; otherwise the operation could complete at some later time, or not at all. Given the similarity between these two operations, it made sense to implement them within the same system call; John duly reworked the patch along those lines.

With the new patch set, to declare that a range of a file's contents is volatile, an application would call:

    fallocate(fd, FALLOCATE_FL_MARK_VOLATILE, offset, len);

Where offset and len describe the actual range to be marked. After the call completes, the kernel is not obligated to keep that range in memory, and is not obligated to write that range to backing store before reclaiming it. The application should not attempt to access that portion of the file while it has been marked volatile, since the contents could disappear at any time. Instead, if and when the data turns out to be useful, a call should be made to:

    fallocate(fd, FALLOCATE_FL_UNMARK_VOLATILE, offset, len);

If the indicated range is still present in memory, the call will return zero and the application can proceed to work with the data. If, instead, any part of the range has been purged by the kernel since it was marked volatile, a non-zero return value will inform the application that it needs to find that data somewhere else.

Any filesystem could conceivably implement this functionality, but, in practice, it only makes sense for a RAM-based filesystem like tmpfs, so it is only implemented there.

The patch is in its third revision as of this writing, having gotten a number of comments in its first two iterations. The number of complaints has fallen off considerably, though, suggesting that most reviewers are happy now. So this feature may just find its way into the 3.6 kernel.

Comments (13 posted)

Patches and updates

Kernel trees

• Linus Torvalds: Linux 3.5-rc1 . (June 3, 2012)

• Greg KH: Linux 3.4.1 . (June 4, 2012)

• Thomas Gleixner: 3.4-rt8 . (June 4, 2012)

• Greg KH: Linux 3.3.8 . (June 4, 2012)

• Ben Hutchings: Linux 3.2.19 . (June 4, 2012)

• Steven Rostedt: 3.2.19-rt30 . (June 6, 2012)

• Greg KH: Linux 3.0.33 . (June 4, 2012)

Core kernel code

• Daniel Santos: [RFC] Generic Red-Black Trees . (June 2, 2012)

• Srivatsa S. Bhat: [PATCH 00/27] Generic framework for SMP booting/CPU hotplug related code . (June 4, 2012)

• Glauber Costa: per cgroup cpu statistics . (June 5, 2012)

• Thomas Gleixner: kthread: Implement park/unpark facility . (June 5, 2012)

Development tools

• Andrew Vagin: [PATCH 0/5] Teach perf tool to profile sleep times (V4) . (June 4, 2012)

• Hui Zhu: KGTP (Linux Kernel debugger and tracer) 20120605 release(user task memory access and user task stack dump available now) . (June 5, 2012)

• Masami Hiramatsu: ftrace,kprobes: Ftrace-based kprobe optimization . (June 5, 2012)

• Steven Rostedt: kprobes/ftrace: Making ftrace usable for kprobes . (June 6, 2012)

Device drivers

• Jonghwa Lee: regulator: MAX77686: Add Maxim 77686 regulator driver . (June 2, 2012)

• Roland Stigge: MTD: LPC32xx SLC NAND driver . (June 4, 2012)

• Mark Brown: clk: wm831x: Add initial WM831x clock driver . (June 4, 2012)

• Jiri Slaby: TTY buffer in tty_port -- prep no. 3 . (June 4, 2012)

• Ezequiel Garcia: media: Add stk1160 new driver . (June 4, 2012)

• Yoshihiro YUNOMAE: ivring: Add IVRing driver . (June 5, 2012)

• Fabio Baltieri: led: add oneshot trigger . (June 6, 2012)

Filesystems and block I/O

• John Stultz: [RFC] Fallocate Volatile Ranges v3 . (June 2, 2012)

• Tejun Heo: block: implement per-blkg request allocation . (June 4, 2012)

• Shaohua Li: raid5: improve write performance for fast storage . (June 4, 2012)

• Richard Weinberger: UBI: Fastmap updates (v7+) . (June 4, 2012)

• Miklos Szeredi: vfs: atomic open v6 (part 2) . (June 5, 2012)

• Cyrill Gorcunov: fcntl: Add F_GETOWNER_UIDS option v3 . (June 5, 2012)

Memory management

• Christoph Lameter: Common [00/20] Sl[auo]b: Common code rework V4 . (June 4, 2012)

• ehrhardt@linux.vnet.ibm.com: swap: improve swap I/O rate - V2 . (June 4, 2012)

Networking

• pablo@netfilter.org: [RFC] new user-space connection tracking helper infrastructure . (June 4, 2012)

Architecture-specific

• Sonic Zhang: crypto: bfin_crc: CRC hardware driver for BF60x family processors. . (June 4, 2012)

• Chris Metcalf: tilegx network driver: initial support . (June 5, 2012)

• Marek Szyprowski: [PATCH/RFC 0/2] ARM: DMA-mapping: new extensions for buffer sharing (part 2) . (June 6, 2012)

Virtualization and containers

• Michael S. Tsirkin: kvm: eoi optimization support . (June 4, 2012)

• Nikunj A. Dadhania: KVM paravirt remote flush tlb . (June 4, 2012)

• Isaku Yamahata: postcopy migration: umem: Linux char device for postcopy . (June 4, 2012)

Page editor: Jonathan Corbet

Distributions

Looking forward to Fedora 18

With Fedora 17 freshly released, the project has turned its attention to deciding on a feature set for Fedora 18. Among the approved changes so far are mounting /tmp in RAM, a better "hotspot" mode for WiFi connection sharing, a rewritten graphical installer, and a reorganization of the package categorization system.

The basics

New Fedora features must be approved by the distribution's Engineering Steering Committee (FESCo), which discusses them during its weekly IRC meetings. The approved list resides on the Fedora wiki. At the moment, it lists 14 items — but there are a few more recent additions, plus some proposals still in limbo. Proponents will have until July to make their cases for additional features.

Several of the items are mundane, such as bumping a major package to the newest upstream version. Individual packages rarely merit a hearing by FESCo, but the group weighs in where it concerns major projects (such as desktop environments) or packages that affect a large number of dependencies (such as programming languages, compilers, or packaging). Among those key package updates for Fedora 18 are Xfce 4.10, the Glasgow Haskell Compiler (GHC) 7.4, Boost 1.50.0, RPM 4.10, and the Perl-Compatible Regular Expression (PCRE) library 8.30.

Other system infrastructure pieces receiving a change in Fedora 18 are Kerberos and Red Hat's GlusterFS filesystem. The Kerberos package will move its cached credentials from /tmp/ to under /run/user/, while GlusterFS will be incremented to version 3.3. GlusterFS is designed to work with Red Hat's OpenShift cloud computing platform, which will also be added to Fedora 18. The package added will be OpenShift Origin, which is the community version of the OpenShift product.

The distribution also wants to make Fedora machines work with Microsoft Active Directory domains out-of-the-box, a project that involves configuration and bug fixes in a variety of packages. Kerberos configuration is among them, but the fixes target SSH, NTP, and an array of other tools.

User-visible desktop features

Fedora's installer is called Anaconda, and a visual and architectural overhaul has been in the works since 2011. According to the wiki, the user interface has not been updated in six to seven years, which makes it badly in need of a new coat of paint, but there are also architectural limitations to the design that make it difficult to add new features (both for Fedora itself and for downstream distributions). The plan is to rework the installer according to a "hub and spoke" design, in which only a few configuration screens are mandatory and demand user input, with optional configuration screens accessible for those who need them. GUI mock-ups and other design documents are on the wiki. A side-effect of the overhaul is that the text-mode version of the installer will be dropped for Fedora 18, and slated to reappear for 19.

A related change is the refactoring of the distribution's package groups. These are the metadata-defined blocks of related packages that users can use to install sets of applications and libraries that serve a related need. Years ago, installing packages in such blocks was the norm in most distributions, but Fedora's group definitions have drifted out of alignment with the components that actually serve as the "building blocks" for Fedora itself and its "spin" derivatives. Refactoring the groups should ease maintenance in the long run, but implementing the change involves a lot of package metadata editing.

A practical change on the feature list is a new, easy-to-use WiFi hotspot mode in NetworkManager. This is a feature used for turning the computer into a WiFi access point to be shared with other devices. The existing version of the hotspot-sharing mode uses WiFi Independent Basic Service Set (IBSS) ad-hoc mode, but in practice ad-hoc mode is not reliable (particularly if WPA or WPA2 is activated). As a result, many consumer devices refuse to connect to ad-hoc networks. The new hotspot mode will in essence let the Fedora box function as a standard WiFi access point (AP). Allowing Fedora to present itself as an AP will make connection sharing work with these devices.

System component updates and migration

Fedora 18 will also move /tmp to a tmpfs filesystem in RAM by default. As we covered last week, there are pros and cons to this approach, but FESCo decided that tmpfs-backed /tmp offered more benefits in the common case than downsides. The option to store /tmp on disk will still be provided. Interestingly, one of the main reasons cited for the Kerberos cache move mentioned earlier was that /run/ is a tmpfs filesystem and will vanish when the machine is rebooted, offering better security — but if /tmp moves to tmpfs, too, users will get the same added security they would from moving Kerberos's cache to /run.

There are other system components being refreshed in the new release. One is migrating access control to privileged operations entirely to PolicyKit. The usermode package, which includes userhelper, userinfo, usermount, and userpasswd, is one of the last holdouts. These tools are setuid root shims around existing system tools, and will be replaced with PolicyKit-managed access to the underlying programs.

The procps tools are also marked for replacement, in this case by the rewritten procps-ng suite. Procps is the suite of utility programs that report on and manage process information: ps, top, kill, free, and so forth. Fedora's "legacy" version of these utilities long ago diverged from upstream with a series of distribution-specific patches, to the point where they can no longer be re-merged. The procps-ng utilities are a rewrite designed to provide better message and error printing, exit codes, and cleaner behavior all around.

Finally, as we discussed in greater depth elsewhere, Red Hat's Matthew Garrett is implementing support for UEFI's secure boot for Fedora 18. The plan involves purchasing a secure boot signing key for the project and making changes to the bootloader and kernel module-loading process to permit only signed binaries to boot on secure boot hardware.

More to come

FESCo recently ratified the release schedule for Fedora 18, which gives contributors until the first week of July to propose new features for inclusion, aiming for a final release at the end of October. There are a few still under consideration that may make the cut before the deadline, including adding ARM as a first-class architecture and a new first-run tool.

Promoting ARM support to the first tier would mean treating ARM equally alongside i686 and x64_64 in the project's build system. Proponents of the feature argue that although ARM devices vary in their capabilities, including many low-power devices without floating-point support, there are enough general-purpose desktop and laptop machines capable of serving as everyday PCs to warrant ensuring that Fedora runs on them. They also point out that ARM-based servers will come to market in the not too distant future. The promotion is a major undertaking, though, including hardware changes in Red Hat's build facility, plus extensive QA work on everything from the kernel to Anaconda. The idea has its supporters, but it could prove too ambitious for this cycle.

The new first-run tool is documented as the Fedora Initial Experience. It encompasses post-installation setup, from license notices and networking setup to online account configuration and a tour of the GNOME 3 desktop. The specification is for some of these Initial Experience steps to replace the normal GDM login after installation. However, there remain issues to be sorted out, including how to make setup tasks optional and how to smoothly transition between the Initial Experience and a regular login session.

With around one month to go, there is still plenty of time to add to the feature list; there may be new surprises or the reanimation long-delayed efforts like Btrfs-by-default or LXC containers. Thus far, Fedora 18 is taking on a smaller set of changes than those tackled in Fedora 17 — though some of them are of greater significance than a single bullet-point on a list suggests. The OpenShift features and ARM support would even the distribution with Ubuntu's offerings in those areas, which is clearly of interest to both the community and Red Hat. On the other hand, the UEFI secure boot feature has already spawned considerable debate, as has the /tmp move to a lesser degree. How those debates shake out will be interesting to watch over the coming six months.

Comments (none posted)

Brief items

Distribution quote of the week

-- Joey Hess

Comments (none posted)

A Gentoo x32 release candidate

The x32 ABI enables the running of processes in 64-bit mode while using 32-bit pointers, hopefully getting the best of both the 32-bit and 64-bit worlds. Users of most distributions will have to wait some time for an x32 version, but Gentoo, as is often the case, is ahead of the curve: there is an x32 release candidate available now. "I call this a release candidate as it is using glibc-2.15 with backported patches. the ABI however should be the same as glibc-2.16, so hopefully the upgrade will go smooth. but i don't want to make an official release on the chance that things aren't smooth, so this is a release candidate."

Full Story (comments: 57)

Debian Diversity statement accepted

The Debian project has accepted a diversity statement which reads:

The Debian Project has a press release available.

Full Story (comments: 11)

Distribution News

Debian GNU/Linux

Bits from the Ruby team: switching to Ruby 1.9 and trasition to new policy

The Debian Ruby team has announced that Ruby 1.9 will be the default Ruby interpreter. Ruby 1.8 will be available in Wheezy, but will be dropped for Wheezy+1. Click below for a look at what that means for Ruby users and package maintainers.

Full Story (comments: none)

Gentoo Linux

Gentoo Council 2012/2013 election

Nominations for the Gentoo Council 2012/2013 are open until June 17, 2012. Anyone can nominate a candidate, but voting is only open to those who were active Gentoo developers as of June 3.

Full Story (comments: none)

Newsletters and articles of interest

Distribution newsletters

• DistroWatch Weekly, Issue 459 (June 4)
• Maemo Weekly News (June 4)
• Ubuntu Weekly Newsletter, Issue 268 (June 3)

Comments (none posted)

Page editor: Rebecca Sobol

Development

Geary: simple email, for now

Yorba released the first usable version of its desktop email client Geary on May 4. Geary is heavily GNOME-based, but is focused on working well with popular web-based email providers out of the box. The initial release boasts considerable UI polish, but only a basic feature set.

Yorba is a relatively young non-profit software project that is developing several GNOME-centric desktop applications. Arguably the most well-known of the bunch is the lightweight photo organizer Shotwell. Geary is the newest offering, and like its sibling projects it is developed in the Vala language and uses SQLite for storage (in Geary's case, that includes offline email cached to enable full-text searching). It also comes with desktop notifications to integrate into the GNOME 3 environment.

On the one hand, a skeptic might asked why yet another desktop email client is necessary for GNOME, when there are so many well-established options — including the "official" client, Evolution. But that might be a question fairly leveled at the other Yorba projects, too, and they have gained popularity among users due to their simplicity. There is considerable bloat and feature-creep in Evolution, Thunderbird, and similar email clients. Geary seems designed to do away with as much of it as possible, starting over with recent GNOME platform components, and targeting GNOME desktop integration.

In the 0.1 release, Geary offers support for IMAP accounts only, and the application is limited to one account. The account data is stored in its own subdirectory under ~/.local/geary/, though, and multiple accounts are on the roadmap. The account setup window also comes pre-configured for GMail and Yahoo Mail accounts, which should suit the needs of a large portion of the user community. It does support SSL/TLS, but without much control over the settings — I was unable to get it to connect to TLS-equipped private mail server, for example.

Read Me

But supporting multiple server features is clearly not the emphasis in this release; a lot of work has gone into the presentation of messages and message conversations themselves. The application window is split into three columns: one for folders, one for the selected folder's message threads, and one for the open thread. Geary mimics GMail's "conversation" view, in which each thread is treated as a unit; Geary opens the entire thread as one item, with each message rendered in order in one long, scrollable pane. This is rapidly becoming the most popular way to render email threads, and it has its advantages. First, when reading long discussions, you can quickly scroll through the entire thread without clicking to load every message. Second, you see more of your inbox in the folder-contents column, by virtue of having each thread occupy only one list element. On the other hand, it is easier to get lost in a long thread, and it is more difficult to leap right to a specific message.

Geary makes this message threading model the only option, and the three-pane view the only window arrangement. More subtle, but equally important, is the work that has gone into displaying the messages themselves. Each email is rendered onto the message canvas like a sheet of paper, with headers and metadata (such as sender's avatar images) set off only by font attributes and spacing. The look is cleaner, and nicer than what Evolution and Thunderbird use, both of which employ separate GTK+ panes for message content and headers.

There are not a lot of other features to explore in this first release; you can read, reply to, and forward email. Message composition is not as smooth as message display — the editor looks like any other email client's composition window, which is not something that would be noteworthy except that it breaks with the slick presentation used for message display. Only HTML email is supported, and the editor sports an uneven set of editing tools. For example, your only font size choices are the vague "Small," "Medium", and "Large," but you have the full RGB / HSV color selector wheel to play with. I doubt I have often needed precise control over my font sizes in outgoing mail, but I have definitely needed it more often than I have needed multicolor text. There is a basic spell-checker built-in, which functions only in as-you-type mode.

Missing miscellany

What you do not get in this release is support for daily email tasks beyond message reading: no attachments, no searching, no filters, and so on. There is also no address book functionality, junk mail filtering, or encryption. It is clear that at least some of these tasks are already on the Yorba project's to-do list (hence the advertisement of SQLite, despite the fact that message searching is not implemented in 0.1). Thus, it is not a criticism of the project to point out that these features are absent, it is merely a warning that the client is not ready for industrial usage.

But I can't help but wonder how much of the "clean and simple" shine that Geary 0.1 enjoys will rub off as the project is forced to add more features to the mix (and to the window). For example, right now the three-pane window is slick, putting message reading front and center with each thread represented by a minimalist "card" showing a short message preview and an unread count. But there are not any columns in that view; it does not take long to realize that there is no way to sort the messages any way other than date, newest messages on top. A lot of users will want to sort and re-sort their mail folders, and it will be challenging to implement without adding considerable complexity to the clean and simple UI design.

That hurdle gets drastically higher when you start talking about address book management, or, worst of all, the pinnacle of usability nightmares that is public key encryption. In short, it is pretty easy to make a clean and simple user interface when you only implement a handful of features. What few projects can do is maintain the high standard of UI design as the feature set expands outward. That is not simply a theoretical challenge; as Shotwell has added more and more features, its menus and sidebar have started to fill up, making the application look more and more like every other photo organizer.

Of course, part of the reason for Yorba's projects is presumably to develop new code-bases built on top of Vala, GObject, and other modern GNOME frameworks, so in the long run feature creep is probably not anathema. At the moment, however, Geary offers a smooth and lightweight email reading experience. You could adapt Thunderbird to mimic many of the new application's UI features (via extensions or optional settings), but of course you do so with a higher memory footprint and more complexity. So perhaps it is a good idea to enjoy the simplicity of Geary while it lasts.

Comments (1 posted)

Brief items

Quotes of the week

— Guido van Rossum

— Lawrence Rosen

Comments (none posted)

Firefox 13 released

The Firefox 13 release is out. New features include improved 'new tab' behavior and better access to utilities (bookmarks, history, etc.) from the default home page. There are plenty of bug fixes, some security related. See the release notes for details.

Update: Thunderbird 13 has also been released. The Thunderbird release notes have the details.

Comments (6 posted)

KDE Announces 4.9 Beta1 and Testing Initiative

KDE has released the first beta for its renewed Workspaces, Applications, and Development Platform. Some highlights of 4.9 include Qt Quick in Plasma Workspaces, improvements in the Dolphin file manager, deeper integration of Activities, plus performance improvements and bug fixes. KDE's Beta Testing Team could always use help finding bugs before the final released. "The Beta Testing Team is a group of people who will do a 'Testapalooza' on the beta releases, catching as many bugs as possible. This is organized in a structured process so that developers know clearly what to fix before the final release. Anyone can help, even people without programming skills."

Full Story (comments: 20)

Obnam 1.0 released

Lars Wirzenius has announced the stable, 1.0 release of his backup tool Obnam. "It's been a long project, much longer than I expected, and much longer than was really sensible. However, it's ready now. It's not bug free, and it's not as fast as I would like, but it's time to declare it ready for general use. If nothing else, this will get more people to use it, and they'll find the remaining problems faster than I can do on my own." Among other features, Obnam makes btrfs-style "snapshot" backups and uses data de-duplication to save space.

Comments (21 posted)

Samba 4.0 beta1 released

Samba 4.0 is getting closer: the first beta release is now available for testing. "We are making this beta release to gain real-world use of the 's3fs' file server combination we will use for the Samba 4.0 release. Users should expect some rough edges."

Full Story (comments: none)

Twisted 12.1.0 released

Version 12.1.0 of the Twisted "networking engine" has been released. There's lots of new reactors for BSD kqueue support, GTK3 and GObject introspection, better file descriptor passing, systemd support, and more; see the NEWS file for details.

Full Story (comments: none)

Newsletters and articles

Development newsletters from the last week

• Caml Weekly News (June 5)
• Perl Weekly (June 3)
• PostgreSQL Weekly News (June 3)

Comments (none posted)

Page editor: Jonathan Corbet

Announcements

Brief items

Samsung becomes a Linux Foundation platinum member

The Linux Foundation has announced that it has signed up Samsung at its highest membership level. "With today’s announcement, Samsung Electronics is reiterating its commitment to Linux and investing its resources in a platform it believes will help sustain its global leadership position. The company will work with The Linux Foundation on streamlining its participation with the kernel community and adopting open source best practices."

Full Story (comments: 9)

Red Hat launches CloudForms platform

Red Hat has announced the availability of Red Hat CloudForms. "CloudForms is an open hybrid cloud management platform built to enable enterprises to create and manage Infrastructure-as-a-Service (IaaS) hybrid clouds with the ability to make self-service computing resources available to users in a managed, governed and secure way."

Full Story (comments: none)

"Linux Tycoon" and "2299" now Free

Bryan Lunduke has released his software under the GPL. "I put a challenge to the community last week: Help me raise enough funds to support me working full time, and I will Open Source (under the GPL) all of my games and software. The community stepped up and we hit that goal in just 8 short days -- helping to prove the viability of supporting Open Source development via Donations." "Linux Tycoon", the Linux-distro-simulation game, and "2299 : THE GAME" are the first of his games to be released under the GPL.

Full Story (comments: none)

Articles of interest

Free Software Supporter -- Issue 50, May 2012

The May issue of the Free Software Supporter covers an FSF statement on Oracle v Google, DRM, a call on the new French president to "Stop software patents", and several other topics.

Full Story (comments: none)

FSFE Newsletter - June 2012

The Free Software Foundation Europe Newsletter for June covers the usage of terms like FOSS, Libre Software, FLOSS; enforcing web standards; French elections and non-free software; and several other topics.

Full Story (comments: none)

Ah! The Morning After... (Groklaw)

In a follow-up to yesterday's coverage of the Oracle v. Google ruling on the copyrightability of APIs, Groklaw has a summary of the reactions from various parties in and around the lawsuit, including interpretations on the case's broader impact. "The ruling totally wipes away the earlier partial ruling by the jury on the API question. Or as lawyers would say, it's now moot. There is no question of fair use for code that can't be copyrighted. You can't infringe something that isn't covered by copyright law, and so you don't need a defense like fair use."

Comments (41 posted)

8 million leaked passwords connected to LinkedIn, dating website (ars technica)

Ars technica reports that a cracker has posted 8 million cryptographic hashes to the Internet that appear to belong to users of LinkedIn and also a popular dating website. "The massive dumps over the past three days came in postings to user forums dedicated to password cracking at insidepro.com. The bigger of the two lists contains almost 6.46 million passwords that have been converted into hashes using the SHA-1 cryptographic function. They use no cryptographic "salt," making the job of cracking them considerably faster. Rick Redman, a security consultant who specializes in password cracking, said the list almost certainly belongs to LinkedIn because he found a password in it that was unique to the professional social networking site. Robert Graham, CEO of Errata Security said much the same thing, as did researchers from Sophos. Several Twitter users reported similar findings." If you have an account on LinkedIn you should probably change your password.

Comments (53 posted)

Education and Certification

LPI Announces Online Resource for Linux Education

The Linux Professional Institute (LPI) has announced an online "one-stop-shopping" solution for learning resources for their Linux Professional Institute Certification (LPIC) program: the LPI Marketplace. "The LPI Marketplace was created for LPI by mindhub(TM) of Pearson VUE and is an online resource center offering a wide variety of learning products designed to help individuals prepare for LPI's industry-recognized certification, enhance their existing skills set and achieve their professional goals. Products in the LPI Marketplace will include a full range of expert reference materials including books, flash cards, practice exams, and video training modules. Content providers presently participating in the LPI Marketplace include such widely-recognized companies as Axzo Press, Element K, ExamForce, MeasureUp, McGraw-Hill, Sybex, Transcender and uCertify."

Full Story (comments: none)

Linup Front GMBH Publishes Free Training Manual

Linup Front GmbH has announced the availability of a free training manual, in German and English, for the new "Linux Essentials" certification published by the Linux Professional Institute (LPI). "The 250-page manual can be downloaded as a PDF file from http://shop.linupfront.com/ and is being published under a Creative Commons license allowing unmodified redistribution. The same content is also being issued as an Android app for tablets and smartphones, which can be obtained free of charge from the Google Play Store."

Full Story (comments: none)

Calls for Presentations

LCA2013 opens call for presentations

Linux.conf.au 2013 returns to the Australian National University in Canberra starting January 28. The call for presentations is now open. "This year, they are focusing on deep technical content, and the things that you think are going to really matter in the future. Do you have something to say about freedom and privacy, open source cloud systems, or energy efficient server farms of the future? Or something else that you are passionate about?" Submission deadline is July 6; see this page for more information.

Full Story (comments: none)

Upcoming Events

MeetBSD California 2012

MeetBSD California 2012 will take place November 3-4, 2012 in Sunnyvale, CA. "It will feature one day of scheduled talks on Saturday followed by one day of unConference-style community-scheduled emergent activities on Sunday. We’re looking forward to another engaging and interesting conference! By combining scheduled talks and community-driven events like lightning talks and hacking sessions, we hope that everyone will get the most out of this year’s activities."

Full Story (comments: none)

Events: June 7, 2012 to August 6, 2012

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
June 6 June 8	LinuxCon Japan	Yokohama, Japan
June 6 June 10	Taiwan Mini DebConf 2012	Hualien, Taiwan
June 7 June 10	Linux Vacation / Eastern Europe 2012	Grodno, Belarus
June 8 June 10	SouthEast LinuxFest	Charlotte, NC, USA
June 9 June 10	GNOME.Asia	Hong Kong, China
June 11 June 15	YAPC North America	Madison, Wisconsin, USA
June 11 June 16	Programming Language Design and Implementation	Beijing, China
June 12	USENIX Cyberlaw '12: 2012 USENIX Workshop on Hot Topics in Cyberlaw	Boston, USA
June 12	WiAC '12: 2012 USENIX Women in Advanced Computing Summit	Boston, USA
June 12	UCMS '12: 2012 USENIX Configuration Management Workshop: Virtualization, the Cloud, and Scale	Boston, USA
June 12 June 13	HotCloud '12: 4th USENIX Workshop on Hot Topics in Cloud Computing	Boston, USA
June 13	WebApps '12: 3rd USENIX Conference on Web Application Development	Boston, USA
June 13 June 14	HotStorage '12: 4th USENIX Workshop on Hot Topics in Storage and File Systems	Boston, MA, USA
June 13 June 15	2012 USENIX Annual Technical Conference	Boston, MA, USA
June 14 June 15	TaPP '12: 4th USENIX Workshop on the Theory and Practice of Provenance	Boston, MA, USA
June 14 June 17	FUDCon LATAM 2012 Margarita	Margarita, Venezuela
June 15	NSDR '12: 6th USENIX/ACM Workshop on Networked Systems for Developing Regions	Boston, MA, USA
June 15 June 16	Nordic Ruby	Stockholm, Sweden
June 15 June 16	Devaamo summit	Tampere, Finland
June 16	Debrpm Linux Packaging Workshop in the Netherlands	The Hague, Netherlands
June 19 June 21	Solutions Linux Open Source	Paris, France
June 20 June 21	Open Source Summit (NASA, State Dept, VA)	College Park, MD, USA
June 26 June 29	Open Source Bridge: The conference for open source citizens	Portland, Oregon, USA
June 26 July 2	GNOME & Mono Festival of Love 2012	Boston, MA, USA
June 30 July 1	Quack And Hack 2012	Paoli, PA, USA
June 30 July 6	Akademy (KDE conference) 2012	Tallinn, Estonia
July 1 July 7	DebConf 2012	Managua, Nicaragua
July 2 July 8	EuroPython 2012	Florence, Italy
July 5	London Lua user group	London, UK
July 6 July 8	3. Braunschweiger Atari & Amiga Meeting	Braunschweig, Germany
July 7 July 8	10th European Tcl/Tk User Meeting	Munich, Germany
July 7 July 12	Libre Software Meeting / Rencontres Mondiales du Logiciel Libre	Geneva, Switzerland
July 8 July 14	DebConf12	Managua, Nicaragua
July 9 July 11	GNU Tools Cauldron 2012	Prague, Czech Republic
July 10 July 11	AdaCamp Washington, DC	Washington, DC, USA
July 10 July 15	Wikimania	Washington, DC, USA
July 11	PuppetCamp Geneva @RMLL/LSM	Geneva, Switzerland
July 11 July 13	Linux Symposium	Ottawa, Canada
July 14 July 15	Community Leadership Summit 2012	Portland, OR, USA
July 16 July 20	OSCON	Portland, OR, USA
July 26 July 29	GNOME Users And Developers European Conference	A Coruña, Spain
August 3 August 4	Texas Linux Fest	San Antonio, TX, USA

If your event does not appear here, please tell us about it.

Page editor: Rebecca Sobol