LWN.net Weekly Edition for November 29, 2012

Open source High Dynamic Range photography in 2012

Back in 2010, we looked at Luminance HDR, a standalone application for creating and editing high dynamic range (HDR) images on Linux systems. At that time, Luminance was a technically capable tool with a byzantine user interface that often got in the user's way. In the intervening two years, the program has made strides, although it has yet to break through the easy-to-grok barrier. But it still remains one of the best applications in a small set of open source options.

In 2010, Luminance was at version 2.0.1. Today, the latest release is 2.3.0, which was released in July 2012. The intervening releases introduced cross-platform support (including Mac OS X, which is a popular request among photographers), 64-bit processor support, and significant speed improvements through harnessing multiple CPU cores. The core of the application has been reworked as well, and it now uses LibRaw for raw photo conversion, Lcms2 for color management, and has separated the user interface out — which allows for a new command line front-end.

The feature set is essentially the same as in earlier versions, however. Users can import a stack of images taken at different exposure levels and blend them into one; the combined image covers a far wider range of light-to-dark than a camera sensor can grab in a single shot. The "exposure-blended" output can be exported either as an HDR image (in OpenEXR or TIFF format), or tone-mapped into a more pedestrian low dynamic range (LDR) format. There is one new tone-mapping algorithm in the latest builds, a size-independent version of the Fattal method, and there is an alignment tool which is supposed to support aligning raw images. The other big news on the feature front is support for batch processing, for those who create a significant number of HDR images.

Underexposed	Exposed	Overexposed

Luminance put to the test

The most visible changes in Luminance 2.3.0 are in the user interface, however. Most notably, the application confines itself to a single window — the old interface popped-up multiple windows and windows-embedded-within-windows, which quickly leads to clutter. There is a "wizard" to guide the user through the process of importing and aligning images. Once the image stack is aligned, the user can click through previews of the output using all of the application's available algorithms, and adjust the settings of each to gauge how they affect the output. This is a big improvement over the old interface, which required generating each output option separately, then eyeballing them side-by-side in preview windows. Toggling back-and-forth between the options brings out the often subtle differences between the various effects.

On the other hand, the application has done little to de-math-nerd-ify the settings and preferences. In 2010, I speculated that there may be a limit to how much simplification can be done — the algorithms are complex equations with a host of esoteric variables, after all. But Luminance still fails to offer the user any real clues, presenting instead a list of preset profiles with the utterly opaque names "Profile 1, Profile 2, Profile 3, Profile 4," (and so on) on the last screen of the project wizard. I could not even find an option to rename the profiles. Being able to tweak each algorithm's settings is important, and seeing the changes applied instantly is a boon, but the generic names create a usability barrier.

For example, one of the algorithms always produces desaturated output, and another produces a significant "black outline" effect. Renaming the profiles "Desaturated" and "Outlined" might not be technically correct, but it might help users learn to work with the available options. Sure, user-assigned profile names will be non-standard, but so are the generic "Profile N" monikers. Does anyone know which of the numbered profiles represents the algorithms used in other tone-mapping programs, like Enfuse?

2.3.0 is easier to navigate than the older releases, but it still has is its share of bugs. In particular, the alignment tool failed completely in all of my tests. The interface allows manual, fine-grained adjustment of the imported images, but the adjustment is immediately lost on the next step. Initially, I thought I had forgotten to click "Apply" or some other such button, but that was not the case. Even if one crops the images after the alignment is perfected, the changes are lost immediately. There are other minor irritations, such as the file-open dialog not remembering the last directory, and Exif rotation not being picked up in the project wizard, but a non-functional alignment tool means Luminance is effectively useless without near-pixel-perfect aligned images.

There are two possibilities for getting there. The "correct" thing to do is probably to take all of one's photos from a sturdy tripod with a cable release, thus eliminating awkward misalignments from the outset. But even then, wind and camera shake will frequently cause small misalignments, which necessitates the other option: aligning the images first using a separate application. Fortunately, there is a good option available.

Hugin

The panorama-generation toolkit Hugin has collected an assortment of valuable accessories over the years, including lens-calibration functions, cloud-detection routines, and more. One of the handiest is the ability to automatically align multiple photographs of the same scene. This is a critical step for stitching multiple images into a panorama, but it can also be used to align a stack of images. Hugin will also correct geometric distortion, which is not mandatory for HDR work, but certainly does not hurt.

In fact, Luminance 2.3.0 can call out to Hugin's automatic image alignment tool. In my own tests, however, working directly in Hugin is far simpler. First, in the event that the alignment routine fails to find where the images line up, you can manually fix problems in Hugin itself. A failure in Luminance leaves you with only the Luminance manual-alignment tool (which, in my tests, never worked). Second, Hugin's ability to fix lens distortion is always applied before the alignment step, which greatly increases the odds of success with wide-angle shots (where there is more distortion). Luminance's call to Hugin's alignment routine does not perform distortion correction.

But Hugin can also perform the exposure-blending step itself, too. It uses the Enfuse utility, and it produces excellent results. The only caveat is that Hugin offers no control over the tone-mapping used to create LDR output. Whatever defaults Enfuse uses are applied to the output when generating an LDR file. It attempts to create sane-looking results: no weird color saturation, no "fringing" or other artifacts one might find in the HDR sections of a photo-sharing web site.

The real question is whether there is any reason to head back to Luminance if the output from Hugin is good enough. One certainly can: Hugin can export OpenEXR or HDR TIFF output too; these files can then be opened in Luminance and tone-mapped with any of the included algorithms. Whether that is worth it is a purely artistic decision. For the most part, the output that Hugin gets from Enfuse is straightforward: balanced color, the full range of contrast. If the point of blending multiple images is simply to capture a scene that does not fit into a normal exposure in the camera's viewfinder, Enfuse will likely suffice. Only if the special-effects look is desirable does Luminance offer much extra functionality.

Comparing the two user interfaces is a bit of a toss-up. Both fall well short of making the task at hand simple to understand. Technically, the best results from Hugin are probably found by making as few adjustments to the tools and options as possible — but that fact is far from discoverable. The Optimizer tab, for instance, offers tantalizing options like "Optimize high dynamic range," but they rarely do what the user expects and are difficult to revert. Luminance may throw a screenful of esoteric research-paper terms at the user, but at least it is easy to click between the alternatives.

The rest

Nevertheless, there are still other options to weigh. The Digikam photo-management suite can also blend multiple pictures into an HDR image. It, too, uses Enfuse for this functionality, so its results will be similar, but it is simpler to use. Hugin's UI is an array of potentially-confusing tabs — most of which hold tools and buttons that have no bearing on blending a stack of images together into an HDR format. Avoiding them entirely may be the best way to get started in HDR images on Linux.

On the other hand, Digikam is extremely anxious to take over one's photo collection and manage it, whether one wants it to or not. Running it for the first time launches an import wizard that wants to index the user's entire home folder; the only way to prevent this is to point it at an empty directory instead.

Another approach might be to use Enfuse directly. Enfuse was designed to be executed from another application (namely Hugin, which the Enfuse manual recommends in its workflow section), but it can be called from the command line as well. The caveats are that Enfuse needs pre-aligned images, and it expects certain things from the input image formats. For instance, it expects input images to have an alpha channel — which they do when they are generated by Hugin — and throws warnings when no alpha channel is found.

The final option to consider is exporting an OpenEXR or HDR TIFF file and using one of the various open source raw converters to make adjustments to it. Darktable, RawTherapee, and Rawstudio each support OpenEXR, and they offer the usual slate of adjustment tools. In short, there is not any reason that an HDR photo needs to be filtered through one of Luminance's tone-mapping algorithms. Those techniques can produce intriguing results, but it would be wrong to regard them as the only "HDR" option.

Luminance HDR	Hugin	Digikam

In that sense, perhaps Luminance deserves a pass for keeping its algorithm-adjustment options complex. Ultimately, the majority of users just want a photo that looks good to the eye. The surreal-looking tone-mapping that first captured the public's attention with the term HDR has its place, but it does not have to be the final word, and it probably will not be. The expanding support for OpenEXR and other natively-HDR formats within the usual photo-editing applications probably means we will see other uses for HDR gaining in popularity.

For that matter, the digital camera market is beginning to see hardware that natively blends together stacks of exposures; HDR is one of the main selling points of replacement camera firmware projects like CHDK and Magic Lantern. Add to that mix the high-bit-depth support coming in GIMP 2.10, and two years from now, the HDR editing scene on Linux — and other platforms — could look drastically different.

Comments (9 posted)

Ubuntu on the Nexus 7

Your editor recently related that, thanks to the generosity of the folks at Google, he had come into possession of a second Nexus 7 tablet. There are many advantages to that state of affairs, not the least of which being that one can install questionable software on one tablet without breaking the other; it's possible to have your tablet and hack it too. Never one to turn down such an opportunity, your editor decided to give the recently announced Ubuntu Nexus 7 port a try.

Why play with Ubuntu on such a device? Even the most ardent Android supporters have sometimes been heard to complain that it's not really very Linux-like above the kernel level. There has been a constant level of interest in more "pure" alternatives like webOS, MeeGo, Nemo, etc., but, so far, none of those alternatives have found any great success in the market. So the availability of Ubuntu 12.10 for a tablet device caught your editor's eye. Might this be a reasonable path to get "real" Linux in a mobile setting?

Like all "Nexus" devices, the Nexus 7 is open from the outset; there is no need to root it via some sort of exploitable vulnerability first. It's a simple matter of plugging the device into the computer and using "fastboot" to unlock it. The unlock operation wipes all the data on the tablet, so, obviously, any needed backups should be made first. The next step is to use fastboot again to flash the "ClockworkMod" recovery image. ClockworkMod allows all kinds of low-level manipulation of the device including backups, operating system installations, and more; it really should be a standard feature of all Nexus devices.

Installation of the Ubuntu port is a straightforward task — assuming one has an Ubuntu desktop system handy. It is just a matter of installing and running the ubuntu-nexus7-installer package. Some rough edges show through quickly enough; the installer cannot figure out the storage capacity of the device and must ask the user to supply that information. More frightening, perhaps, are the scary warnings about not having any other devices attached to the system during the installation; there is, it seems, no way to tell the installer which device to overwrite.

There is another discouraging note during the installation process: the release as a whole is made available under a noncommercial-use license. The reason given in the license notice is proprietary drivers and codecs from Broadcom and NVIDIA. Such restrictions have the potential to raise all kinds of licensing issues. The problem is not created by Ubuntu, though: they are simply using a rebuilt Android kernel and the drivers that came with it. Be that as it may; your editor came to the conclusion that writing a review constituted fair use rather than commercial use.

The use of the Android kernel raises some other interesting questions, since Ubuntu's user space is designed for mainline kernels. Some quick looking around suggests that Ubuntu is not using the Android-specific interfaces; wakelocks have been configured out, for example. Battery life under Ubuntu is claimed to be comparable to what is obtained with Android, but it's being done with Linux-style power management instead of opportunistic suspend. The Nexus 7 thus provides an ideal platform for comparison of the two approaches to power management; this is an area that bears watching.

Once the installation is complete, the tablet reboots and presents the classic Ubuntu screen with the Unity icon bar on the left; there is no login screen. It looks like an interface that was designed for tablets, until one tries to use the tiny icons in the upper right corner. Then, at least for the fat-fingered among us, life starts to get harder. And it doesn't stop there. The simple truth of the matter is that Ubuntu on the Nexus 7 is a painful system to use; it is really only of interest to developers and other masochists at this time.

In fairness, nobody ever claimed otherwise; it is described as an experimental release for those who want to help find and fix problems. So, sure enough, problems do exist. Many of them derive from the fact that the traditional Linux desktop (and Unity remains close enough to "traditional" for the purposes of this discussion) is just not designed around touch-oriented interfaces. Others are simply glitches in the tablet port.

So, for example, one cannot scroll windows with the standard drag gesture; instead, one ends up trying to hit scrollbars in just the right spot. Anything involving a middle or right mouse button requires a complicated dance with the "Onboard" on-screen keyboard. Autocompletion popups swallow keystrokes, so trying to type a URL into Firefox is an exercise in extended pain. The tablet often freezes or goes into a weird unresponsive mode, requiring a reboot — there is a reason that the first entry in the Ubuntu Nexus 7 FAQ is "How do you reset the device when it locks up?". The screen does not auto-rotate (but one can rotate it manually with the xrotate command). Neither Bluetooth nor the camera work. The device often runs out of memory; the known issues page describes the process for configuring zram (an in-memory compression system formerly known as Compcache), which helps a lot. And so on.

On the other hand, there's something refreshing about being able to run multiple windows on a tablet display; as these devices grow in both size and resolution, there really is no justification for forcing every application to run in full-screen mode. It is nice to have a true Linux user space with a complete package repository behind it.

The Unity "dash" is meant to be the way users find applications on the tablet. It remains rather painful to use in the touch environment, though; it is slow and the scrolling is difficult to use. Searching for applications in the main screen quickly turns up unwanted things — the opportunity to buy stuff from Amazon, for example. The interaction between the dash and the onscreen keyboard is problematic; it is often not possible to get both onto the screen at once, and, when that does work out, the keyboard tends to cover the part of the window one is trying to use.

Those difficulties notwithstanding, the onscreen keyboard is, it must be said, one of the best your editor has encountered — at least, for the task of typing at terminal emulators and related applications.

Ubuntu's keyboard lacks the word completion and correction features found on the Android keyboard, but it offers other amenities: easy access to special characters, "control" and "alt" keys, arrow keys, function keys, macros, configurable layouts, themes, and more. Your editor has not attempted to use it with Emacs, but the idea is only mildly irrational.

Some concluding thoughts

In the end, while Ubuntu on tablets is essentially unusable now, that could change in the future. Whether it will change in time to be relevant is not clear, though. Beyond the fundamental issues of making the distribution work on this hardware (and, in particular, within the tablet's memory constraints), there needs to be a set of applications that work well with touchscreens. So it is a little discouraging that Ubuntu has no plans to support Android applications; doing so would help to jump-start the distribution on mobile devices. There is also, according to Mark Shuttleworth (as quoted in this OMG! Ubuntu! article), no plan to improve the interface for the upcoming 13.04 release. So a version of Ubuntu that is actually usable on tablets is, at a bare minimum, a full year away; it may, in fact, take rather longer than that.

The situation isn't helped by Canonical's apparent determination to go it alone in this quest. Rather than pick up a system which has a lot of the basics working now (Nemo or Plasma Active, say), Canonical is trying to build up its own "Unity" shell, and it seems to lack a story altogether when it comes to the development of touch-friendly applications. So it's going to take a while, and that is unfortunate: a year or three in the future may well be too late. There are other tablet-oriented systems out there, mostly of the non-free variety, that are ready and grabbing market share now. By the time Ubuntu gets to be a serious contender, there may be no space for another offering, no matter how nice. Linux on the tablet may repeat the history of Linux on the desktop.

So Ubuntu on the tablet has the look of a cool toy that most of us may never play with. But, then, your editor is highly gifted when it comes to being wrong on the Internet. This distribution is certainly a cool hack, fun to play with, and it might just attract contributors and develop quickly into something people want to use. For now, though, your editor will be putting Android back onto this particular device.

Comments (59 posted)

2012 Linux and free software timeline - Q1

Here is LWN's fifteenth annual timeline of significant events in the Linux and free software world.

2012 has largely been business as usual. Software and distributions continue to be released at an astonishing rate; some distributions have risen in popularity as others fall. Linux continues to appear in more and more places, from embedded to high-performance computing. And, though Linux still has not conquered the desktop, it (in the form of Android) seems to have conquered the phone market and to be well established on tablets. The usual threats to free software continue to lurk: patents and technological measures such as UEFI secure boot. Nevertheless, it has been another good year for Linux, and the prospects for the coming year(s) seem bright.

We will be breaking the timeline up into quarters, and this is our report on January-March 2012. Over the next month, we will be putting out timelines of the other three quarters of the year.

---

This is version 0.8 of the 2012 timeline. There are almost certainly some errors or omissions; if you find any, please send them to timeline@lwn.net.

LWN subscribers have paid for the development of this timeline, along with previous timelines and the weekly editions. If you like what you see here, or elsewhere on the site, please consider subscribing to LWN.

If you'd like to look further back in time, our timeline index page has links to the previous timelines and some other retrospective articles going all the way back to 1998.

January

Linux 3.2 is released (announcement, KernelNewbies summary, LWN merge window summaries: part 1 and part 2).

The Apache Software Foundation releases Hadoop 1.0 (announcement, LWN article).

Microsoft confirms earlier fears about UEFI secure boot by requiring vendors to lock down ARM devices (LWN blurb).

Scribus 1.4 is released (announcement, LWN article).

After nearly two years' work, the Mozilla Project releases the Mozilla Public License 2.0 (announcement, license text).

The Tizen project releases a set of source repositories and an alpha SDK (LWN article).

systemd v38 is released; this is the first release containing "the journal" (announce,ment).

NSA releases security-enhanced Android (LWN blurb).

linux.conf.au is held in Ballarat, Australia, January 16-21 (LWN coverage: A Samba 4 update; Addressing the failure of open source; The past, present, and future of Ubuntu on ARM; Jacob Appelbaum on surveillance and censorship; An LCA 2012 summary; videos).

Southern California Linux Expo (SCALE) 10x is held in Los Angeles, January 20-22 (LWN coverage: Robots rampage; The trickiness of the education market; The road ahead for automotive Linux and open source).

FreeBSD 9.0 is released (announcement, release notes).

An X.Org Server flaw that allows screen-lock programs to be bypassed is disclosed; the bug is quickly fixed (LWN coverage).

GDB 7.4 is released (release announcement).

The KDE project releases KDE Plasma Workspaces, KDE Applications, and KDE Platform 4.8 (LWN blurb, announcement).

HP announces a roadmap to make webOS open source by September (LWN blurb, announcement).

Cinnamon 1.2 is released; this is the first stable release of Linux Mint's fork of the GNOME Shell (LWN article).

February

Greg Kroah-Hartman moves to the Linux Foundation (LWN blurb).

ownCloud 3 is released; LWN looked at this personal cloud system in January.

Linaro Connect Q1.12 is held in Redwood City, California, February 6-10 (LWN coverage of the scheduling minisummit).

FOSDEM '12 is held in Brussels, Belgium, February 4-5 (LWN coverage: Multiarch on Debian and Ubuntu; The Wayland display server).

The Document Foundation announces that it will be based in Berlin (announcement).

Canonical ceases sponsoring one of their employees to work full-time on the Kubuntu project (announcement).

The book Open Advice is published under a CC-BY-SA license; the book consists of a set of essays with advice on contributing to FOSS projects (LWN review).

SUSE, the oldest of the current commercial Linux distributions, turns 20 this year (IT World article, TechWeek Europe article).

Robyn Bergeron becomes the new leader of the Fedora Project, succeeding Jared Smith (LWN interview with Robyn).

LibreOffice 3.5 makes its third stable release, with the project starting to settle into a 6-month release cycle (announcement; LWN article).

Wayland protocol and Weston compositor 0.85.0 are released (announcement, LWN article; The H article).

The Android Builders Summit is held in Redwood Shores, California, February 13-14 (LWN coverage: Android and the kernel mainline).

Canonical announces Ubuntu for Android (LWN article).

VLC 2.0 is released (LWN blurb).

Adobe ends separate distribution of its proprietary Linux Flash plugin used by the Firefox browser (LWN blurb and later article on Flash support on Linux).

Mozilla announces a deal to start shipping HTML5-driven smartphones by the end of 2012 (announcement, LWN article).

The proposed /usr unification causes gnashing of teeth in some quarters (LWN article).

MINIX 3.2.0 is released (LWN article).

March

PHP 5.4.0 is released (LWN blurb; announcement).

The GitHub repository site is compromised (LWN blurb).

Wine 1.4 is released (announcement).

Vagrant 1.0 is released (LWN article).

X.Org Server 1.12 is released (LWN article on the XInput multitouch extension in that release).

The Open Invention Network announces an expansion of the range of software that is covered by the group's patent license agreement. (LWN article).

Google releases the LinSched scheduler-testing framework (release announcement, LWN article).

bzr 2.5.0 is released (announcement).

Gnuplot 4.6 is released (announcement).

Cinnamon 1.4 is released (LWN blurb).

Crossroads I/O 1.0.0 is released (LWN article on this ZeroMQ fork).

The Mozilla project decides to support the H.264 video codec (LWN blurb and article).

Linux 3.3 is released (announcement; KernelNewbies summary; LWN merge window summaries part 1 and part 2; LWN development statistics article).

GCC 4.7.0 is released; the project is now 25 years old (announcement).

LTTng 2.0 "stable" is released (LWN coverage: part 1 and part 2).

The GNU C library steering committee dissolves itself, one of several events that signal a change in the governance of the project (LWN blurb and article).

Version 1 of the Go programming language is released (LWN blurb).

GNOME 3.4 is released; this is the second major update of GNOME 3 (announcement).

Comments (none posted)

Page editor: Jonathan Corbet

Security

Security implications for user interface changes?

Free software users are not generally known for their quiet acceptance of user interface changes. Many changes to the UI of desktop environments or popular applications lead to long and loud threads from users—with some percentage of those users claiming they will move to an alternative rather than "put up" with the change. But what happens if the alternative is to stick with an earlier, unsupported version of the application? That's the question that came up in a short, but interesting, thread on the Mozilla security mailing list.

Plans for Firefox to remove the "tabs on bottom" feature have so incensed a vocal subset of users (see this bug report or this lengthy thread on the mozilla.dev.apps.firefox group) that they don't plan to upgrade the browser once this change is implemented. For many releases now, Firefox has had its tabs below the controls and "awesome bar", which is the behavior called "tabs on bottom". More recent versions have had a "Tabs on top" toggle in the toolbar configuration, which moves the tabs to just below the menu (and above the controls and awesome bar). The toggle is slated for removal, with tabs on top becoming the default. The old behavior will still be available by setting browser.tabs.onTop to false in about:config, but users are concerned that will eventually disappear as well.

The ferocity of the arguments against moving the tabs (and removing the toggle) led Zack Weinberg to suggest keeping the toggle and feature:

Web browsers, by their nature, need frequent updates. Because browsers face the often hostile internet and can provide a portal to users' documents, photos, passwords, and so forth, it is critically important for users to keep up with browser updates. Anything that gets in the way of that process is (and should be) worrisome. That is the main reason that Chrome and Firefox have moved to automatic updates, for example.

But there is a tradeoff to be made here. Mozilla's VP of Firefox Engineering Johnathan Nightingale argues that, over the years, too much attention has been paid to the most vocal user contingent. There is code that is "in desperate need of clean up", he said, so Firefox developers cannot necessarily afford to heed the negative feedback:

Weinberg is not convinced that cleaning up the code base overrides the security issue, however. He is concerned that the "tabs on bottom" issue is really just the straw that broke the camel's back for some segment of users. Even a small percentage of the Firefox install base can make for a rather large problem:

Drawing a clear line is difficult, though. If any change to the UI can be seen as a "security problem" because users might decide not to upgrade, it will be difficult for Firefox to make any changes. Users have to take some responsibility for their choices. As Curtis Koenig put it:

Users will make poor choices at times, and it is certainly possible that some change will drive some of them to make those choices. Is there a "moral responsibility", as Weinberg claimed, for Firefox (and, by implication, other applications, desktops, etc.) to continue to deliver a user experience that its users have become accustomed to? Are UI changes always potential security problems? There are obviously some kinds of UI changes that are security flaws, but simply changing the way the user interacts with the program likely doesn't really reach that level.

Both Koenig and Nightingale do not see the "tabs on bottom" change as a security issue. There may be design or development issues that need to be resolved—though Nightingale seems confident that those have largely been dealt with—but changing some UI elements around is not cause for a security red flag. In fact, Nightingale called the security concern "a red herring (or a slippery slope, take your pick)".

There is only so much that a project can do to protect its users. Part of the problem with this particular case is that the other "major" free alternative, Chrome/Chromium, also has its tabs at the top. One guesses that the uproar would be good deal more subdued if there were an "easy" alternative that behaved the way the "vocal conservatives" want. There may be good reasons to consider leaving the "tabs on bottom" feature alone; security isn't really one of them. But it is always good to see projects thinking about and debating where these lines are.

Comments (57 posted)

Brief items

Security quotes of the week

-- UPI

-- L. Gordon Crovitz in The Wall Street Journal

Comments (2 posted)

Backdoor inserted into Piwik

The Piwik web server analytics package was given an undesirable feature — a backdoor — as the result of a compromise of the piwik.org server. "You would be at risk only if you installed or updated to Piwik 1.9.2 on Nov 26th from 15:43 UTC to 23:59 UTC. If you are not using 1.9.2, or if you have updated to 1.9.2 earlier than Nov 26th 15:40 UTC or from Nov 27th, you should be safe." The announcement has details on the backdoor and how to detect it.

Comments (2 posted)

New vulnerabilities

awstats: unspecified vulnerability

Package(s):	awstats	CVE #(s):	CVE-2012-4547
Created:	November 28, 2012	Updated:	April 8, 2013
Description:	From the CVE entry: Unspecified vulnerability in awredir.pl in AWStats before 7.1 has unknown impact and attack vectors.
Alerts:	Fedora FEDORA-2012-18423 2012-11-28 Mandriva MDVSA-2013:061 2013-04-08

Comments (none posted)

bugzilla: multiple vulnerabilities

Package(s):	bugzilla	CVE #(s):
Created:	November 26, 2012	Updated:	November 28, 2012
Description:	From the Fedora advisory: Update to 4.0.9 Confidential product and component names can be disclosed to unauthorized users if they are used to control the visibility of a custom field. When calling the 'User.get' WebService method with a 'groups' argument, it is possible to check if the given group names exist or not. Due to incorrectly filtered field values in tabular reports, it is possible to inject code which can lead to XSS. When trying to mark an attachment in a bug you cannot see as obsolete, the description of the attachment is disclosed in the error message. A vulnerability in swfstore.swf from YUI2 can lead to XSS.
Alerts:	Fedora FEDORA-2012-18210 2012-11-24 Fedora FEDORA-2012-18224 2012-11-24

Comments (none posted)

firefox: multiple vulnerabilities

Package(s):	firefox	CVE #(s):	CVE-2012-5843 CVE-2012-5836 CVE-2012-4203 CVE-2012-4204 CVE-2012-4205 CVE-2012-4208 CVE-2012-4212 CVE-2012-4213 CVE-2012-4217 CVE-2012-4218 CVE-2012-5838
Created:	November 22, 2012	Updated:	January 8, 2013
Description:	From the Ubuntu advisory: Gary Kwong, Jesse Ruderman, Christian Holler, Bob Clary, Kyle Huey, Ed Morley, Chris Lord, Boris Zbarsky, Julian Seward, Bill McCloskey, and Andrew McCreight discovered multiple memory safety issues affecting Firefox. If the user were tricked into opening a specially crafted page, an attacker could possibly exploit these to cause a denial of service via application crash, or potentially execute code with the privileges of the user invoking Firefox. (CVE-2012-5842, CVE-2012-5843) Jonathan Stephens discovered that combining vectors involving the setting of Cascading Style Sheets (CSS) properties in conjunction with SVG text could cause Firefox to crash. If a user were tricked into opening a malicious web page, an attacker could cause a denial of service via application crash or execute arbitrary code with the privliges of the user invoking the program. (CVE-2012-5836) It was discovered that if a javascript: URL is selected from the list of Firefox "new tab" page, the script will inherit the privileges of the privileged "new tab" page. This allows for the execution of locally installed programs if a user can be convinced to save a bookmark of a malicious javascript: URL. (CVE-2012-4203) Scott Bell discovered a memory corruption issue in the JavaScript engine. If a user were tricked into opening a malicious website, an attacker could exploit this to execute arbitrary JavaScript code within the context of another website or arbitrary code as the user invoking the program. (CVE-2012-4204) Gabor Krizsanits discovered that XMLHttpRequest objects created within sandboxes have the system principal instead of the sandbox principal. This can lead to cross-site request forgery (CSRF) or information theft via an add-on running untrusted code in a sandbox. (CVE-2012-4205) Peter Van der Beken discovered XrayWrapper implementation in Firefox does not consider the compartment during property filtering. An attacker could use this to bypass intended chrome-only restrictions on reading DOM object properties via a crafted web site. (CVE-2012-4208) Abhishek Arya discovered multiple use-after-free and buffer overflow issues in Firefox. If a user were tricked into opening a malicious page, an attacker could exploit these to execute arbitrary code as the user invoking the program. (CVE-2012-4214, CVE-2012-4215, CVE-2012-4216, CVE-2012-5829, CVE-2012-5839, CVE-2012-5840, CVE-2012-4212, CVE-2012-4213, CVE-2012-4217, CVE-2012-4218) Several memory corruption flaws were discovered in Firefox. If a user were tricked into opening a malicious page, an attacker could exploit these to execute arbitrary code as the user invoking the program. (CVE-2012-5830, CVE-2012-5833, CVE-2012-5835, CVE-2012-5838)
Alerts:	Ubuntu USN-1638-1 2012-11-21 Ubuntu USN-1636-1 2012-11-21 Ubuntu USN-1638-2 2012-11-21 openSUSE openSUSE-SU-2012:1583-1 2012-11-28 openSUSE openSUSE-SU-2012:1584-1 2012-11-28 openSUSE openSUSE-SU-2012:1585-1 2012-11-28 openSUSE openSUSE-SU-2012:1586-1 2012-11-28 SUSE SUSE-SU-2012:1592-1 2012-11-29 Ubuntu USN-1638-3 2012-12-03 Fedora FEDORA-2012-18952 2012-12-04 Fedora FEDORA-2012-18931 2012-12-04 Ubuntu USN-1430-5 2012-11-29 Mageia MGASA-2012-0353 2012-12-07 Fedora FEDORA-2012-18661 2012-12-19 Fedora FEDORA-2012-18661 2012-12-19 Fedora FEDORA-2012-18661 2012-12-19 Fedora FEDORA-2012-18661 2012-12-19 Fedora FEDORA-2012-18661 2012-12-19 Gentoo 201301-01 2013-01-07 openSUSE openSUSE-SU-2013:0175-1 2013-01-23

Comments (none posted)

hyper-v: denial of service

Package(s):	Hyper-V	CVE #(s):	CVE-2012-2669
Created:	November 22, 2012	Updated:	November 28, 2012
Description:	From the openSUSE advisory: The source code without this patch caused hv_kvp_daemon to exit when it processed a spoofed Netlink packet which has been sent from an untrusted local user. Now Netlink messages with a non-zero nl_pid source address are ignored and a warning is printed into the syslog. This fixes the previous change from CVE-2012-2669.
Alerts:	openSUSE openSUSE-SU-2012:1526-1 2012-11-22 Ubuntu USN-1719-1 2013-02-12 Ubuntu USN-1720-1 2013-02-12 Ubuntu USN-1726-1 2013-02-14

Comments (none posted)

insight: remote denial of service

Package(s):	insight	CVE #(s):	CVE-2012-3509
Created:	November 26, 2012	Updated:	November 28, 2012
Description:	From the CVE entry: Multiple integer overflows in the (1) _objalloc_alloc function in objalloc.c and (2) objalloc_alloc macro in include/objalloc.h in GNU libiberty, as used by binutils 2.22, allow remote attackers to cause a denial of service (crash) via vectors related to the "addition of CHUNK_HEADER_SIZE to the length," which triggers a heap-based buffer overflow.
Alerts:	Fedora FEDORA-2012-18311 2012-11-24 Fedora FEDORA-2012-18300 2012-11-24

Comments (none posted)

kernel: denial of service

Package(s):	kernel	CVE #(s):	CVE-2012-4461
Created:	November 22, 2012	Updated:	November 28, 2012
Description:	From the Red Hat Bugzilla entry: A flaw has been found in the way Linux kernel's KVM subsystem handled vcpu->arch.cr4 X86_CR4_OSXSAVE bit set upon guest enter. On hosts without the XSAVE feature an unprivileged local user could use this flaw to crash the system.
Alerts:	Fedora FEDORA-2012-18684 2012-11-22 Fedora FEDORA-2012-18691 2012-11-28 Ubuntu USN-1688-1 2013-01-15 Ubuntu USN-1689-1 2013-01-15 Mageia MGASA-2013-0010 2013-01-18 Mageia MGASA-2013-0009 2013-01-18 Mageia MGASA-2013-0011 2013-01-18 Mageia MGASA-2013-0012 2013-01-18 Ubuntu USN-1696-1 2013-01-17 Ubuntu USN-1699-1 2013-01-17 Ubuntu USN-1704-1 2013-01-22 Mageia MGASA-2013-0016 2013-01-24 Ubuntu USN-1699-2 2013-02-01 Ubuntu USN-1696-2 2013-02-01 Ubuntu USN-1704-2 2013-02-01 Red Hat RHSA-2013:0223-01 2013-02-05 CentOS CESA-2013:0223 2013-02-06 Oracle ELSA-2013-0223 2013-02-06 Scientific Linux SL-kern-20130206 2013-02-06 Oracle ELSA-2013-2503 2013-02-07 Oracle ELSA-2013-2507 2013-02-28

Comments (none posted)

libsocialweb: untrusted connection to flickr

Package(s):	libsocialweb	CVE #(s):	CVE-2012-4511
Created:	November 23, 2012	Updated:	November 28, 2012
Description:	From the Fedora advisory: The libsocialweb library is prone to a security vulnerability that allows attackers to perform man-in-the-middle attacks. Remote attackers can exploit this issue to gain access to sensitive information or modify the integrity of user accounts. Other attacks are also possible.
Alerts:	Fedora FEDORA-2012-17746 2012-11-23 Fedora FEDORA-2012-17749 2012-11-23

Comments (none posted)

libssh: code execution

Package(s):	libssh	CVE #(s):	CVE-2012-4559 CVE-2012-4560 CVE-2012-4561 CVE-2012-4562
Created:	November 27, 2012	Updated:	December 6, 2012
Description:	From the Ubuntu advisory: Xi Wang and Florian Weimer discovered that libssh incorrectly handled memory. A remote attacker could use this to cause libssh to crash, resulting in a denial of service, or possibly execute arbitrary code. (CVE-2012-4559, CVE-2012-4560, CVE-2012-4561, CVE-2012-4562)
Alerts:	Ubuntu USN-1640-1 2012-11-26 Fedora FEDORA-2012-18677 2012-11-29 Mandriva MDVSA-2012:175 2012-11-29 Mageia MGASA-2012-0344 2012-11-29 Debian DSA-2577-1 2012-12-01 Fedora FEDORA-2012-18687 2012-12-06 openSUSE openSUSE-SU-2012:1620-1 2012-12-07 openSUSE openSUSE-SU-2012:1622-1 2012-12-07 Slackware SSA:2012-341-02 2012-12-06 openSUSE openSUSE-SU-2013:0130-1 2013-01-23 Mandriva MDVSA-2013:045 2013-04-05

Comments (none posted)

libssh2: multiple integer overflows

Package(s):	libssh2	CVE #(s):	CVE-2012-4562
Created:	November 22, 2012	Updated:	November 29, 2012
Description:	From the SUSE advisory: This update of libssh fixes multiple integer overflows. CVE-2012-4562 has been assigned to this issue.
Alerts:	SUSE SUSE-SU-2012:1520-1 2012-11-21 openSUSE openSUSE-SU-2012:1620-1 2012-12-07 openSUSE openSUSE-SU-2012:1622-1 2012-12-07 Slackware SSA:2012-341-02 2012-12-06 Mandriva MDVSA-2013:045 2013-04-05

Comments (1 posted)

libvoikko: denial of service

Package(s):	libvoikko	CVE #(s):
Created:	November 26, 2012	Updated:	November 28, 2012
Description:	From the Mageia advisory: Version 3.2.1 fixes the handling of embedded null characters in input strings entered through the Python interface. The bug could be used to cause denial of service conditions and possibly other problems. Users of these interfaces are recommended to upgrade to this release. Applications that use the native C++ library directly (this includes all well known desktop applications) are not affected by this bug and no changes to the native library have been made in this release.
Alerts:	Mageia MGASA-2012-0340 2012-11-23

Comments (none posted)

lighttpd: denial of service

Package(s):	lighttpd	CVE #(s):	CVE-2012-5533
Created:	November 23, 2012	Updated:	November 30, 2012
Description:	From the Novell advisory: Specially-crafted HTTP header can cause a Denial of Service (infinite loop) in lighttpd.
Alerts:	openSUSE openSUSE-SU-2012:1532-1 2012-11-23 Mageia MGASA-2012-0345 2012-11-29

Comments (none posted)

mantis: multiple vulnerabilities

Package(s):	mantis	CVE #(s):	CVE-2012-5522 CVE-2012-5523
Created:	November 26, 2012	Updated:	November 28, 2012
Description:	From the CVE entries: MantisBT before 1.2.12 does not use an expected default value during decisions about whether a user may modify the status of a bug, which allows remote authenticated users to bypass intended access restrictions and make status changes by leveraging a blank value for a per-status setting. (CVE-2012-5522) core/email_api.php in MantisBT before 1.2.12 does not properly manage the sending of e-mail notifications about restricted bugs, which might allow remote authenticated users to obtain sensitive information by adding a note to a bug before losing permission to view that bug. (CVE-2012-5523)
Alerts:	Fedora FEDORA-2012-18294 2012-11-24 Fedora FEDORA-2012-18299 2012-11-24

Comments (none posted)

moodle: unintended Dropbox access

Package(s):	moodle	CVE #(s):	CVE-2012-5471
Created:	November 28, 2012	Updated:	November 28, 2012
Description:	From the CVE entry: The Dropbox Repository File Picker in Moodle 2.1.x before 2.1.9, 2.2.x before 2.2.6, and 2.3.x before 2.3.3 allows remote authenticated users to access the Dropbox of a different user by leveraging an unattended workstation after a logout.
Alerts:	Fedora FEDORA-2012-18525 2012-11-28 Fedora FEDORA-2012-18570 2012-11-28

Comments (none posted)

pcp: insecure temporary file use

Package(s):	pcp	CVE #(s):	CVE-2012-5530
Created:	November 23, 2012	Updated:	November 28, 2012
Description:	From the Fedora advisory: A security flaw was found in the way Performance Co-Pilot (PCP), a framework and services to support system-level performance monitoring and performance management, performed management of its temporary files used by various services from the suite. A local attacker could use this flaw to conduct symbolic link attacks (alter or remove different system files, accessible with the privileges of the user running the PCP suite, than it was originally intended).
Alerts:	Fedora FEDORA-2012-18654 2012-11-23 Fedora FEDORA-2012-18686 2012-11-23 SUSE SUSE-SU-2013:0190-1 2013-01-23

Comments (none posted)

perl-CGI: header injection

Package(s):	perl-CGI	CVE #(s):	CVE-2012-5526
Created:	November 28, 2012	Updated:	December 19, 2012
Description:	From the CVE entry: CGI.pm module before 3.63 for Perl does not properly escape newlines in (1) Set-Cookie or (2) P3P headers, which might allow remote attackers to inject arbitrary headers into responses from applications that use CGI.pm.
Alerts:	Fedora FEDORA-2012-18318 2012-11-28 Mageia MGASA-2012-0346 2012-11-29 Ubuntu USN-1643-1 2012-11-29 Debian DSA-2586-1 2012-12-11 Debian DSA-2587-1 2012-12-11 Mandriva MDVSA-2012:180 2012-12-17 Fedora FEDORA-2012-18330 2012-12-18 Fedora FEDORA-2012-18330 2012-12-18 Fedora FEDORA-2012-19282 2012-12-13 Fedora FEDORA-2012-19282 2012-12-13 SUSE SUSE-SU-2013:0441-1 2013-03-13 SUSE SUSE-SU-2013:0442-1 2013-03-13 openSUSE openSUSE-SU-2013:0497-1 2013-03-20 openSUSE openSUSE-SU-2013:0502-1 2013-03-20 Red Hat RHSA-2013:0685-01 2013-03-26 CentOS CESA-2013:0685 2013-03-26 CentOS CESA-2013:0685 2013-03-26 Oracle ELSA-2013-0685 2013-03-26 Oracle ELSA-2013-0685 2013-03-27 Scientific Linux SL-perl-20130327 2013-03-27

Comments (none posted)

rssh: command execution

Package(s):	rssh	CVE #(s):	CVE-2012-2251 CVE-2012-2252
Created:	November 28, 2012	Updated:	November 28, 2012
Description:	From the Debian advisory: James Clawson discovered that rssh, a restricted shell for OpenSSH to be used with scp/sftp, rdist and cvs, was not correctly filtering command line options. This could be used to force the execution of a remote script and thus allow arbitrary command execution.
Alerts:	Debian DSA-2578-1 2012-11-28 Fedora FEDORA-2012-20109 2012-12-19

Comments (none posted)

tomcat: multiple vulnerabilities

Package(s):	tomcat6	CVE #(s):	CVE-2012-2733 CVE-2012-5885 CVE-2012-5886 CVE-2012-5887 CVE-2012-3439
Created:	November 22, 2012	Updated:	January 10, 2013
Description:	From the Ubuntu advisory: It was discovered that the Apache Tomcat HTTP NIO connector incorrectly handled header data. A remote attacker could cause a denial of service by sending requests with a large amount of header data. (CVE-2012-2733) It was discovered that Apache Tomcat incorrectly handled DIGEST authentication. A remote attacker could possibly use these flaws to perform a replay attack and bypass authentication. (CVE-2012-5885, CVE-2012-5886, CVE-2012-5887)
Alerts:	Ubuntu USN-1637-1 2012-11-21 Fedora FEDORA-2012-20151 2012-12-19 openSUSE openSUSE-SU-2012:1701-1 2012-12-27 openSUSE openSUSE-SU-2012:1700-1 2012-12-27 Mandriva MDVSA-2013:004 2013-01-10 Mageia MGASA-2013-0015 2013-01-18 openSUSE openSUSE-SU-2013:0147-1 2013-01-23 Red Hat RHSA-2013:0623-01 2013-03-11 CentOS CESA-2013:0623 2013-03-12 Oracle ELSA-2013-0623 2013-03-11 Scientific Linux SL-tomc-20130312 2013-03-12 Red Hat RHSA-2013:0640-01 2013-03-12 CentOS CESA-2013:0640 2013-03-12 Oracle ELSA-2013-0640 2013-03-13 Scientific Linux SL-tomc-20130312 2013-03-12

Comments (none posted)

unity-firefox-extension: code execution

Package(s):	unity-firefox-extension	CVE #(s):	CVE-2012-0960
Created:	November 22, 2012	Updated:	November 28, 2012
Description:	From the Ubuntu advisory: It was discovered that unity-firefox-extension incorrectly handled certain callbacks. A remote attacker could use this issue to cause unity-firefox-extension to crash, resulting in a denial of service, or possibly execute arbitrary code.
Alerts:	Ubuntu USN-1639-1 2012-11-22

Comments (none posted)

vlc: denial of service

Package(s):	vlc	CVE #(s):	CVE-2012-5470
Created:	November 22, 2012	Updated:	November 28, 2012
Description:	From the Mageia advisory: libpng_plugin in VideoLAN VLC media player 2.0.3 allows remote attackers to cause a denial of service (application crash) via a crafted PNG file (CVE-2012-5470).
Alerts:	Mageia MGASA-2012-0333 2012-11-21

Comments (none posted)

weechat: shell injection

Package(s):	weechat	CVE #(s):	CVE-2012-5534
Created:	November 28, 2012	Updated:	December 3, 2012
Description:	From the openSUSE advisory: added weechat-fix-hook_process-shell-injection.patch which fixes a shell injection vulnerability in the hook_process function (bnc#790217, CVE-2012-5534)
Alerts:	openSUSE openSUSE-SU-2012:1580-1 2012-11-28 Fedora FEDORA-2012-18575 2012-11-29 Fedora FEDORA-2012-18526 2012-11-29 Mageia MGASA-2012-0347 2012-11-30 Fedora FEDORA-2012-19533 2012-12-11 Fedora FEDORA-2012-19538 2012-12-11 Debian DSA-2598-1 2013-01-05 openSUSE openSUSE-SU-2013:0150-1 2013-01-23

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

The current development kernel is 3.7-rc7, released on November 25. "A week ago, I had even considered skipping -rc7 entirely as things had been so calm, but decided that there was little reason to hurry the release. And oh, how sadly right I was." This could prove to be the last one, though, if the next week of testing goes well.

Stable updates: 3.0.53, 3.4.20 and 3.6.8 were released on November 26.

Comments (none posted)

Quotes of the week

— Linus Torvalds's new decision-making process

— Paul McKenney

— Mark Brown

— Andrew Morton

Comments (22 posted)

Kernel development news

Uninitialized blocks and unexpected flags

One often-heard complaint in the early BitKeeper era was that, by letting code reach the mainline without going via a mailing list, BitKeeper made it easy for maintainers to slip surprise changes in underneath the review radar. Those worries have mostly proved unfounded; when surprises have happened, the response from the community has usually helped to ensure that there would be no repeats. But some developers are charging that the 3.7 kernel contains exactly this type of stealth change and are demanding that it be reverted.

Background

The fallocate() system call is meant to be a way for an application to request the efficient allocation of blocks for a file. Use of fallocate() allows a process to verify that the required disk space is available, helps the filesystem to allocate all of the space in a single, contiguous group, and avoids the overhead that block-by-block allocation would incur. In the absence of an fallocate() implementation (each filesystem must implement it independently), the C library will emulate it by simply writing zeroes to the requested block range; that gets the space allocated, but is less efficient than one would like. The implementation of fallocate() within filesystems tries to be more efficient than that; one way to do so is to avoid the process of writing zeroes to the newly-allocated blocks.

Leaving stale data in allocated blocks has obvious security implications: a hostile application could read those blocks in the hopes of finding confidential documents, passwords, or the missing Fedora 18 Beta release announcement. To avoid this exposure, filesystems like ext4 will mark unwritten blocks as being uninitialized; any attempt to read those blocks will be intercepted and just return zeroes. In the normal case, the application will write data to those blocks before ever trying to read them; writing obviously initializes the blocks without the need to write zeroes first. This implementation seems like it should be about optimal.

Except that, seemingly, ext4 marks uninitialized blocks at the extent (group of contiguous blocks) level. So, if an application writes to one uninitialized block, the containing extent must be split and the newly-written block(s) added to the previous extent, if possible. That turns out to be more expensive than some users would like. So a shortcut was attempted.

That shortcut first appeared in April, 2012, in the form of a new fallocate() flag called FALLOC_FL_NO_HIDE_STALE. If fallocate() was called with that flag, the newly-allocated blocks would be marked as being initialized even though the old data remained untouched. That obviously brings the old security issues back; to mitigate the problem, the patch added a mount option making the new functionality available only to members of a specific group. That was deemed to be enough, especially for settings where access to the machine as a whole is tightly controlled.

At least, the authors and supporters of the patch deemed the group check to be enough. The patch was roundly criticized by other filesystem developers; the prevailing opinion appeared to be that it was trying to open up a huge security hole in order to avoid fixing an ext4 performance problem. After that discussion, the patch went away and wasn't heard from again.

A surprise flag

At least, it was not heard from until recently, when some filesystem developers were surprised to discover this commit by Ted Ts'o which found its way into the mainline (via the ext4 tree) during the 3.7 merge window. The patch is small and simple; it simply defines the FALLOC_FL_NO_HIDE_STALE flag, but adds no code to actually implement it. The changelog reads:

Filesystem developer Dave Chinner, at least, does not recall this discussion. His response was to post a patch reverting the change, saying:

It is true that this particular change is a bit abnormal. It changes the core filesystem code but came by way of a filesystem-specific tree with no acks from any other developers. The patch does not appear to have been posted to any relevant mailing list, violating the rule that all patches should go through public review before being pushed toward the mainline. The addition of a flag with no in-kernel users is also contrary to usual kernel practice. It is, in summary, the sort of change that less well-established kernel developers would never get away with making. So it is hard to fault other filesystem developers for being surprised and unhappy.

On the other hand, the change just adds a flag definition; it obviously cannot cause problems for existing code. And there does appear to be a real user community for this feature. Ted justified his action this way:

This explanation does not appear to have satisfied anybody, though. So we have an impasse of sorts; some developers want a flag to control a functionality they need, while others see it as a security problem and the result of an abuse of the kernel's trust system.

Alan Cox suggested that it would be possible to, instead, reserve a set of filesystem-private flags that could be used for any purpose by any filesystem. Dave pointed out, however, that a flag bit that behaved differently from one filesystem to the next is a recipe for trouble. His suggestion, instead, is that this functionality should be implemented via the ioctl() interface, which is where filesystem-specific options usually hide. The ioctl() approach seems like it should be workable, but no patches to that effect have been posted thus far.

As of this writing, Linus has not accepted the revert, so the FALLOC_FL_NO_HIDE_STALE flag can still be found in the 3.7 kernel. He has also remained silent in the discussion. He will have to make a decision one way or the other, though, before the final 3.7 release is made. Once that flag is made available in a stable mainline release, it will be much harder to get rid of, so, if that flag is going to come out, it needs to happen soon.

Comments (5 posted)

The return of loadable security modules?

The idea behind the Linux Security Module (LSM) interface was initially discussed as part of the "NSA Linux" session at the first Kernel Summit back in 2001. The intent was to avoid wiring a particular security solution into the kernel; instead, multiple approaches to security could be built on top of a common kernel API. Originally, as the name implies, the solutions were built as loadable kernel modules, but eventually the "M" in LSM became just a historical artifact as the API was no longer exported to modules (essentially requiring security "modules" to be statically linked into the kernel). But it's possible that may all change again with a recent patch to bring back loadable LSMs.

Some history

A bit of history is probably in order. The LSM API came about specifically because Linus Torvalds didn't want to have to choose between a number of competing access control mechanisms for the kernel. Instead, LSM would provide a way for any of those mechanisms to hook into the kernel and deny access to various kinds of resources (files, devices, tasks, inodes, etc.) based on the security model being implemented. Initially, the LSMs would be implemented as kernel modules that could be loaded at runtime and, in some cases, unloaded.

The LSM interface was released as part of the 2.5 development kernel series in 2002, and was part of the first 2.6 release in December 2003. For several years after that, there was only one in-tree user of the interface: SELinux. That led to a 2005 suggestion to remove the LSM API entirely, effectively just calling SELinux directly. That would turn SELinux into the "one true security solution" for Linux. In 2006, James Morris proposed a patch to move LSM to the "feature removal" list, scheduled for the 2.6.18 kernel, which was roughly two months out at that point.

But, along came Smack, which implemented a "simplified" Mandatory Access Control (MAC) scheme for the kernel. It also used the LSM interface, so, to a certain extent, the decision on whether to merge it hinged on the future of LSM. In October 2007, Torvalds clearly stated his intention to keep LSM in the kernel, thus paving the way for Smack to be merged.

At more or less the same time Smack was merged, another change to LSM was made. First discussed in mid-2007, Torvalds merged a patch for the 2.6.24 kernel that switched LSM to a static interface so that security "modules" needed to be built into the kernel. One could still choose which security module to use with kernel command-line parameters, but dynamic security module loading would no longer be allowed.

There were a number of reasons behind the switch. For one thing, unloading modules was always messy (or impossible), partly because keeping a coherent security state through that process is difficult. In addition, the LSM API is very intrusive, allowing modules to hook nearly any kernel operation, which can be (and was) abused. While the LSM symbols were exported as GPL-only, that didn't stop some proprietary modules from abusing the interface. There were also free software modules that used the interface for non-security purposes (e.g. the realtime "security" module). Those kinds of problems could also be used as arguments against having the LSM API at all, but since Torvalds had already put his foot down on that particular question, removing the ability to load LSMs was seen as a reasonable alternative.

At the time that Torvalds merged the patch that made that switch, he asked for "real world" users of loadable security modules to step forward. There were a few examples of out-of-tree LSMs that were loadable (and, possibly, unloadable), but none that actually seemed to require that ability. The main users of the feature were LSM developers, who might routinely load and unload their LSM during development.

The next few years saw the merging of Smack (2.6.25), TOMOYO (2.6.30), and AppArmor (2.6.38). The latter had been long out of tree; its existence was part of the reason that the LSM interface came about in the first place. There have also been periodic attempts to get smaller, single-purpose security changes into the kernel over the years, but those were always pointed to the LSM interface. There is a problem with that particular suggestion, though, as only one LSM can be active at a time. Most distributions already have their one security module "slot" filled up. Red Hat and Fedora use SELinux, Ubuntu uses AppArmor, while SUSE and openSUSE have both AppArmor and SELinux available. Adding a specialized LSM for additional security protections is generally not possible without removing or disabling the distribution-supplied security solution.

Proposed LSM changes

That "one LSM at a time" problem has led to persistent (if intermittent) calls for ways to stack or chain LSMs. Smack developer Casey Schaufler is the most recent to propose a stacking solution. His patch set has been steadily reviewed on the LSM mailing list since it was first posted in September; it is now up to version 8. That particular version came with an interesting caveat:

Those requests came from the developer of the TOMOYO LSM, Tetsuo Handa. In earlier discussion of Schaufler's stacking patch, Handa advocated a return to allowing loadable LSMs. In fact, he went further than that, proposing a set of patches that would restore the ability to load LSMs as well as converting TOMOYO to use that feature.

Handa lists three reasons for making the change. To start with, any distribution that wants to allow its users to experiment with different LSMs must build all of those LSMs statically into the kernel. That will not only increase the size of the kernel, it will also increase the time it takes to load and boot the kernel. Most of that space (and time) would be completely wasted even for the users who are experimenting. All of that makes it less likely that distributions will actually build kernels that way.

Beyond that, though, many distributions have their preferred LSM, so they don't build extra LSMs into their kernels. That leaves users to build their own kernels, which is generally unacceptable, particularly in enterprise settings. But even if there are other LSMs built into the kernel, it takes a reboot to enable them. Handa notes that he uses a loadable kernel module that implements TOMOYO (called AKARI) to diagnose problems in enterprise systems. In order to access the LSM symbols (which are no longer exported), AKARI must do some kind of runtime address resolution, perhaps using /proc/kallsyms or System.map. But, AKARI is something he can load into running systems when needed—unlike regular LSMs.

One could argue that Handa's use of an LSM for system troubleshooting is a misuse of the interface, but the fact remains that changing LSMs currently requires a reboot. That problem potentially becomes more acute if LSM stacking is merged. One must decide pre-boot which LSMs to enable (and in what order they are consulted). Whatever else can be said, disallowing LSM loading reduces flexibility.

Handa's third reason is a bit more philosophical: "LSM is not the tool for thought control." Essentially, he argues, disallowing LSM loading just makes dealing with LSMs harder for both users and developers. It also means that the more "minor" LSMs (e.g. TOMOYO and Smack) get less exposure because fewer users can actually try them.

While there have been no comments on Handa's patches as yet, there have been expressions of support for loadable LSMs by some. Schaufler, for example, does not seem opposed necessarily. Kees Cook agreed with the need for loadable LSMs, though he was concerned that combining it with the LSM stacking patches would potentially block the progress for stacking. Morris, who authored the original patch to block loadable LSMs, has not yet spoken up one way or the other.

Taking away the ability to load LSMs did not really change the picture for the kinds of abuses that were brought up at the time the change was made. Kernel modules can still abuse the interface, though it may take a bit more work. If binary modules were willing to ignore the GPL-only export of the LSM interface, they are probably willing to ferret out the addresses they need instead. Open source modules can do much the same. At the time of the switch to a static interface back in 2007, Torvalds seemed very open to reverting it if there were real users—perhaps he can still be convinced.

Comments (none posted)

Statistics from the 3.7 development cycle

The 3.7-rc7 prepatch came out on November 25; it may well be the last prepatch for the 3.7 development cycle. 3.7 was one of the more active cycles in recent history, with nearly 12,000 non-merge changesets incorporated by the time of this writing. It's time for our traditional look at what was done during this cycle and where all that work came from.

The 3.7 merge window was especially busy this time around. Here are some counts for recent kernels:

Kernel	-rc1	Total
3.0	7,333	9,153
3.1	7,202	8,693
3.2	10,214	11,881
3.3	8,899	10,550
3.4	9,249	10,899
3.5	9,534	10,957
3.6	8,587	10,247
3.7	10,409	11,815

The 3.7 development cycle, thus, saw the most active merge window in the 3.x era; it is, in fact, the most active merge window ever. Even allowing for the fact that 3.7 will add a few more changesets before final release, the 2.6.25 kernel, at 12,243 changesets total, will probably still hold the record for the most active development cycle ever, but the 2.6.25 merge window only saw 9,450 changesets merged. One could conclude from these numbers that we are getting better at getting our changes in during the merge window — and at having fewer things to fix thereafter.

Nearly 395,000 lines of code were removed from the kernel this time around. That must be balanced against the 719,000 lines that were added, though; the kernel grew by almost 324,000 lines as a result.

1,271 developers contributed to the 3.7 kernel — a relatively high number, but not out of line with previous development cycles. The lists of the most active developers do see some changes this time around, though:

Most active 3.7 developers

By changesets H Hartley Sweeten 417 3.5% Antti Palosaari 216 1.8% Al Viro 167 1.4% Wei Yongjun 145 1.2% Sachin Kamat 138 1.2% Mark Brown 136 1.2% Eric W. Biederman 130 1.1% Daniel Vetter 122 1.0% David Howells 119 1.0% Hans Verkuil 119 1.0% Greg Kroah-Hartman 116 1.0% Arnd Bergmann 112 0.9% Peter Senna Tschudin 104 0.9% Ben Skeggs 97 0.8% Peter Ujfalusi 96 0.8% Ian Abbott 96 0.8% Devendra Naga 90 0.8% David S. Miller 84 0.7% Takashi Iwai 83 0.7% Johannes Berg 78 0.7%

By changed lines David Howells 65206 7.6% Ben Skeggs 50282 5.8% David Daney 46825 5.4% Arnd Bergmann 17505 2.0% Sebastian Andrzej Siewior 16088 1.9% Daniel Cotey 14157 1.6% H Hartley Sweeten 13566 1.6% Catalin Marinas 13519 1.6% Antti Palosaari 12336 1.4% Bill Pemberton 10935 1.3% Dan Magenheimer 10509 1.2% Ezequiel Garcia 10211 1.2% David S. Miller 9258 1.1% Hans Verkuil 8686 1.0% Will Deacon 8404 1.0% Shawn Guo 7464 0.9% Alois Schlögl 7301 0.8% Roland Stigge 6987 0.8% Greg Kroah-Hartman 6920 0.8% Laurent Pinchart 6107 0.7%

In a repeat of his 3.6 performance, H. Hartley Sweeten hit the top of the by-changesets list with a vast number of patches preparing the comedi drivers for graduation from the staging tree (removing over 5000 lines of code in the process). Antti Palosaari did a lot of work on drivers in the Video4Linux2 subsystem. Al Viro continues to refactor and clean up the VFS and core kernel areas with some excursions into most architecture subtrees. Wei Yongjun and Sachin Kamat both did a lot of cleanup work all over the driver tree.

David Howells ended up at the top of the "lines changed" column mostly by virtue of the user-space API header file thrashup, but he also contributed code for module signing and more. Ben Skeggs merged a major reworking of the nouveau driver, David Daney improved support for MIPS OCTEON processors, Arnd Bergmann's many patches were dominated by the removal of the unused mach-bcmring architecture code, and Sebastian Andrzej Siewior did a lot of work on the USB gadget driver subsystem.

Worth noting in passing: Fengguang Wu is credited with 63 bug reports during this cycle, almost 11% of the total. The others with at least ten reports are Dan Carpenter (21), Randy Dunlap (16), Stephen Rothwell (15), Paul McKenney (11), and Alex Lyakas (10). Every one of those reports resulted in a bug that was fixed before this kernel was released in stable form.

An even 200 employers (that we know about) contributed during the 3.7 cycle. The most active of these were:

Most active 3.7 employers

By changesets (None) 1435 12.1% Red Hat 1159 9.8% (Unknown) 843 7.1% Intel 800 6.8% Texas Instruments 597 5.1% IBM 516 4.4% Linaro 509 4.3% Vision Engraving Systems 417 3.5% SUSE 356 3.0% Google 245 2.1% Samsung 198 1.7% Freescale 181 1.5% Oracle 177 1.5% Wolfson Microelectronics 148 1.3% AMD 144 1.2% Trend Micro 144 1.2% Cisco 138 1.2% Linux Foundation 132 1.1% Arista Networks 130 1.1% NVIDIA 123 1.0%

By lines changed Red Hat 157023 18.2% (None) 80191 9.3% (Unknown) 71992 8.3% Cavium 46757 5.4% IBM 39227 4.5% Intel 33381 3.9% Linaro 28900 3.4% Texas Instruments 28493 3.3% ARM 24913 2.9% Oracle 24095 2.8% NVIDIA 19167 2.2% linutronix 17211 2.0% Vision Engraving Systems 14844 1.7% Samsung 14519 1.7% Microtrol S.R.L. 12800 1.5% PHILOSYS Software 10311 1.2% SUSE 10226 1.2% Marvell 10067 1.2% Cisco 9828 1.1% Pengutronix 9793 1.1%

There are few surprises here. Texas Instruments has reached a new high in its contribution volume, a trend which, unfortunately, may not continue after the recent changes play out there. AMD, too, seems unlikely to remain on this list in the future. Meanwhile Red Hat maintains its place at the top of the list, where it has been since we first started generating these statistics.

And that is where things stand as the 3.7 kernel approaches its final release. Things appear to be running smoothly, with most development cycles taking less than 70 days to complete (if there is no 3.7-rc8, this cycle will run about 64 days). Stay tuned for the about-to-begin 3.8 cycle, with a release to be expected in early February, 2013.

Comments (1 posted)

Patches and updates

Kernel trees

• Linus Torvalds: Linux 3.7-rc7 . (November 26, 2012)

• Greg KH: Linux 3.6.8 . (November 26, 2012)

• Greg KH: Linux 3.4.20 . (November 26, 2012)

• Steven Rostedt: 3.4.20-rt31 . (November 28, 2012)

• Greg KH: Linux 3.0.53 . (November 26, 2012)

• Steven Rostedt: 3.0.53-rt77 . (November 28, 2012)

Core kernel code

• Fabio Baltieri: cpufreq: handle SW coordinated CPUs . (November 27, 2012)

• Aristeu Rozanski: devcg: introduce proper hierarchy support . (November 28, 2012)

Development tools

• Anton Vorontsov: KDB: Kiosk (reduced capabilities) mode . (November 24, 2012)

• Jan Kiszka: Add gdb python scripts as kernel debugging helpers . (November 27, 2012)

Device drivers

• Adrian Hunter: Add SDHCI ACPI driver . (November 24, 2012)

• Thierry Reding: iio: adc: Add Texas Instruments ADC081C021/027 support . (November 24, 2012)

• Thierry Reding: rtc: Add NXP PCF8523 support . (November 24, 2012)

• Benjamin Tissoires: Support of dual pen/multitouch and new default for win 7 certified devices . (November 24, 2012)

• Terje Bergstrom: Support for Tegra 2D hardware . (November 26, 2012)

• Laurent Pinchart: Common Display Framework . (November 27, 2012)

• Prabhakar Lad: Media Controller capture driver for DM365 . (November 28, 2012)

Documentation

• Vince Weaver: perf: proposed perf_event_open() manpage . (November 26, 2012)

Filesystems and block I/O

• Dave Kleikamp: loop: Issue O_DIRECT aio using bio_vec . (November 24, 2012)

• Kent Overstreet: AIO performance improvements/cleanups . (November 28, 2012)

Janitorial

• H. Peter Anvin: RFC: Remove 386 support . (November 28, 2012)

Memory management

• Mel Gorman: Automatic NUMA Balancing V5 . (November 24, 2012)

• Ingo Molnar: Latest numa/core release, v17 . (November 24, 2012)

• Ming Lei: solve deadlock caused by memory allocation with I/O . (November 24, 2012)

• Wen Congyang: memory-hotplug: hot-remove physical memory . (November 27, 2012)

• Jeff Moyer: make I/O path allocations more numa-friendly . (November 28, 2012)

• Dave Chinner: Numa aware LRU lists and shrinkers . (November 28, 2012)

• Anton Vorontsov: Add mempressure cgroup . (November 28, 2012)

Networking

• Jesper Dangaard Brouer: [RFC net-next PATCH V1 0/9] net: fragmentation performance scalability on NUMA/SMP systems . (November 27, 2012)

Architecture-specific

• Anton Vorontsov: ARM: KDB FIQ debugger . (November 24, 2012)

• Davide Ciminaghi: enable support for AMBA drivers under x86 . (November 24, 2012)

• Jacob Pan: pm: Intel powerclamp driver . (November 27, 2012)

• Yinghai Lu: x86, boot, 64bit: Add support for loading ramdisk and bzImage above 4G . (November 28, 2012)

Virtualization and containers

• George Zhang: VMCI for Linux upstreaming . (November 24, 2012)

• George Zhang: VSOCK for Linux upstreaming . (November 24, 2012)

• Stanislav Kinsbursky: nfsd: more NFSv4 state containerization . (November 26, 2012)

• Miklos Szeredi: autofs4: allow autofs to work outside the initial PID namespace . (November 27, 2012)

• Jason Wang: Multiqueue virtio-net . (November 27, 2012)

Page editor: Jonathan Corbet

Distributions

Haiku edges toward general release

Haiku, the open source re-creation of BeOS, threatens to become "The Duke Nukem of operating systems," joked long-time contributor Ryan Leavengood. Actually, after eleven years of development, Haiku still falls four years short of Duke Nukem Forever's long delay, but few other projects have been so long in development. However, with the recent release of Alpha 4.1, Haiku is at last nearing general release.

Haiku started in 2001, after Be, Inc. was bought by Palm and development stopped on BeOS. Although never popular, BeOS had developed a cult following due to its rethinking of the assumptions that go into most operating systems, and the resulting simplicity and performance. To this day, some of its features, such as the customizable attributes supported by its filesystem, remain either unimplemented on other operating systems, or else implemented with more overhead and less efficiency by desktops or specialized applications.

Consequently, BeOS retains a certain nostalgic cachet, with at least one distribution, ZevenOS, dedicated to recreating its user experience on Linux. Although Leavengood suggested vaguely that Haiku might interest modern users with older machines or concerns about viruses, more likely the main interest in the finished Haiku will be due to its unique structures and solutions. If nothing else, they retain a historical interest as roads not taken.

Reasons for the delay

Over the years, about twenty active developers have drifted in and out of Haiku, with occasional new contributors being attracted through Google's Summer of Code and Code-In. Along the way, project founder Michael Phipps and several others have had to drop out for personal reasons, but, by now, the core developers have become a close-knit team familiar with each other's preferences and long-accustomed to filling in for each other.

So why has the project taken so long? According to Leavengood, many of the basics, such as the kernel, the API, and most of the subsystems for printing, sound, and other basic functionality — the "kits," in BeOS / Haiku jargon — were largely finished in the first years of the project. He continued:

Moreover, as a small project, Haiku often faces an unresolvable dilemma. On the one hand, struggling to keep pace with a much larger project can be demanding for a small team of hobbyists. For example, Leavengood, who first ported WebKit to Haiku, observed that at times WebKit can have "hundreds of commits" daily.

On the other hand, when a key piece of software is forked, it can fall behind the original version and become increasingly difficult to patch. Haiku ran into this second problem with GCC, finding that the fork of release 2.95 originally used by BeOS needed to be replaced with a much later release in order to build WebKit. What seems expedient in the short term may actually have created more work in the long term.

Now, however, "we are over the hump of the last twenty percent," Leavengood said, "and I think the general release is possible by next year."

A quick tour of the desktop

Exploring Alpha 4.1 makes Leavengood's prediction sound plausible. Considering that it is an Alpha, the release is a relatively mature milestone, with the operating system and many basic utilities complete and stable. (It follows in a matter of weeks after Alpha 4.0, whose release revealed timing and hardware compatibility issues that the project lacked the resources to reproduce without the assistance of more users.) It is available as both a Live image and a virtual machine image for VMware and VirtualBox, and installs in well under ten minutes on most machines.

Overview documents of both the BeOS architecture and the Haiku desktop are available as icons on the desktop, as is a Welcome page. All three are concise and well-written starting points for exploring the system, although basic navigation should offer few challenges for most computer users. The greatest problem may be minor inconsistencies and unexpected usages.

Like BeOS, Haiku is designed for use with a graphical interface and a built-in window manager. Currently, it is a 32-bit, single-user system, although a 64-bit version and multiple user support are in development.

On first impression, Haiku resembles a minimal desktop environment, perhaps slightly archaic in appearance compared to GNOME or KDE, but, like Xfce or LXDE, lightweight and fast to boot or open windows. Desktop icons, virtual workspaces, desktop applets, and limited customization options are all available, although themes are not.

Linux users may note small idiosyncrasies. For example, minimizing or maximizing windows is controlled by the same titlebar button, while multiple windows can be converted into tabs in a single window, as they can be in KDE. Similarly, although the desktop context menu has no item for adding launchers, you can drag and drop items from a file listing directly to the desktop. In many cases, expected functionality is available, but may not be immediately obvious.

The most important of these idiosyncrasies is the deskbar in the upper right corner. Like the launcher in Unity, the deskbar is an unmovable access point for the main menu, taskbar, the file manager Tracker, and system settings. Although unmovable and sometimes requiring users to drill down several layers, on the whole it is an efficient use of desktop space, although more text-based than many modern users are accustomed to seeing.

Currently, the main shortcoming of the Haiku desktop is a lack of applications. Printing to PDF is supported, but the browser does not support Flash, for example. As described on the Welcome page, users can install BeOS applications, but by modern standards, many offer only limited functionality. Similarly, most of the applications installed with Haiku are basic utilities, such as a CD player, a screen capture program, and a media player. Moreover, most of the applications go unmentioned in the help pages and have names that will not only be unfamiliar to Linux users, but provide no clue to their function until opened.

Probably, the most advanced — and most important — application is WebPositive, the WebKit-based browser. Although short of features compared to Chrome or Firefox, WebPositive appears faster than both. Just as importantly, the browser helps to compensate for the lack of productivity tools such as spreadsheets or word processors by providing access to online tools like Google Docs.

Under the hood

Despite Haiku's graphical orientation, some of its most interesting features are best viewed from the terminal. To start with, Haiku's filesystem hierarchy begins with the /boot folder. It has three branches: /boot/system, /boot/common, and /boot/home. The /boot/system branch is written to only by software updates, while as part of the preliminary support for a multi-user system, /boot/common is reserved for shared resources and configuration files, and /boot/home for individual user's resources and configuration files. Eventually, /boot/home will support separate home directories, but for now there is only one.

On each branch, the files for a piece of software have a similarly named path. Although the paths are not identical — for instance, according to the User Guide, /boot/common/add-ons is the equivalent to /boot/home/config/add-ons — the paths are arguably more consistent than, for example, the locations of /bin and /share directories in Linux.

In addition, aside from a few time-honored abbreviations such as /config, the directory names are usually spelled out in full, making them slower to type, but easier to identify at a glance. Such organization makes knowing where to look for a particular resource much easier than in typical Unix-like systems.

Haiku also supports a modified version of Bash that, to judge from the help, seems included largely for scripting. An Introduction to Bash tutorial is available online, and the scripting chapter of the BeOS Bible is included as part of Haiku's installation.

However, probably the most interesting feature of Haiku is the filesystem's file attributes. To start with, files on Haiku have several unique attributes that are built into them. These include flags for setting whether single or multiple instances can be opened, and whether or not a file responds to system messages.

Even more interestingly, the filesystem allows users to create their own attributes for files using Haiku's addattr command. Like a spreadsheet cell, each attribute is a separate field, for which a specific content type is defined, such as string or numeric. Because they are implemented at the filesystem level, these customizable attributes help Haiku to index and search files with considerably less overhead than, for example, KDE's Nepomuk, which operates on the desktop level and uses its own database.

The advantages of custom attributes inspired the Haiku developers to make each email received by the system a separate file, rather than storing it in a specific application's database. As a result, emails can be searched and viewed in Haiku by ordinary file applications rather than by specialized mail browsers, which simplifies development and integrates emails into the filesystem. In effect, customizable attributes eliminate the need for a separate database layer.

Some of these features were present in BeOS. However, Haiku is also going beyond BeOS, and includes features such as font anti-aliasing that BeOS never had. Even in its present incomplete stage, Haiku aspires to be thoroughly modern while being markedly different from any other operating system available today.

Moving closer to the general release

"There are some people who use Haiku daily and get by with it," Leavengood said, but he immediately admitted that "usually they are those who came from BeOS and really like the system."

What is required for the general release is not hardware drivers, as some might expect. Thanks to a compatibility layer, FreeBSD drivers can be easily recompiled to support Haiku. Nor, with the increasing popularity of web applications, are productivity applications the problem they were a decade ago.

Instead, Leavengood listed the remaining necessities as polish, "some kernel bugs to work out," and the need for "a fairly decent package management system." Haiku's software installation is already a straightforward matter of uncompressing zipped files into the correct directory, but the development team has a more ambitious vision for packages:

One of the main reasons that this work remains unfinished, according to Leavengood, is that the two developers doing most of the work on the package manager will not be available for several months to work on it full-time. However, by the time they are, Leavengood said, "we can start working on the betas."

In some ways, Leavengood said, Haiku's decade-long development cycle may improve its chances of a favorable reception. "We're in a different world now from what we were when BeOS was around," he noted. "Windows, while still quite dominant, isn't dominant in quite the same way." Not only has OS X gained in popularity, but the rise of smartphones and tablets means that average users are more used to multiple operating systems.

Probably, Leavengood said, the first release will not be as complete as the development team would prefer. However, so far as he is concerned, the time is rapidly approaching when "we just need to do the best job we can and get it out there for the world."

In today's more diverse market, Leavengood suggested that Haiku can offer a hybrid design philosophy, specifically:

When the general release finally comes, Haiku may not gain many dedicated users. However, at the very least, it does stand a reasonable chance of being installed virtually and on secondary machines by many who are curious and, perhaps looking for fresh perspectives on operating system design. If so, they are unlikely to be disappointed.

Comments (40 posted)

Brief items

Distribution quotes of the week

-- Lennart Poettering

-- Andrew Powell (in the comments)

-- Bjørn Mork

Comments (33 posted)

Fedora 18 Beta is out

After much delay, Fedora 18 "Spherical Cow" beta is available for testing. "Fedora 18 offers a brand-new version of the Gnome desktop, version 3.6, straight from the upstream development process. Updates have also been made to the KDE, XFCE and Sugar desktop environments; additionally, the MATE desktop is available for the first time in Fedora. Fedora's new installer user interface enhances the anaconda installer with improvements in ease of use and installation."

Full Story (comments: 50)

GNU Guix launches

The GNU Guix project has announced its existence; it is an attempt to build a 100% free GNU-oriented system with, inevitably, a new package manager. "In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection."

Full Story (comments: 112)

Distribution News

openSUSE

openSUSE Board official candidates announced

The openSUSE Election Officials committee has announced this year's official list of candidates for the openSUSE 2013 Board. They are: Matt Baringer, Richard Brown, Carl Fletcher, Manu Gupta, Chuck Payne, Robert Schweikert, Stefan Seyfried and Raymond Woonick. Voting begins December 5.

Full Story (comments: none)

Newsletters and articles of interest

Distribution newsletters

• Debian Project News (November 26)
• DistroWatch Weekly, Issue 484 (November 26)
• Maemo Weekly News (November 26)
• Ubuntu Weekly Newsletter, Issue 293 (November 25)

Comments (none posted)

Langasek: Upstart in Debian

Steve Langasek has announced that upstart is available in Debian unstable. "One of the consequences is that it's now possible to do meaningful head-to-head comparisons of boot speed between sysvinit (with startpar), upstart, and systemd. At one time or another people have tested systemd vs. sysvinit when using bash as /bin/sh, and upstart vs. sysvinit, and systemd vs. sysvinit+startpar, and there are plenty of bootcharts floating around showing results of one init system or another on one distro or another, but I'm not aware of anyone having done a real, fair comparison of the three solutions, changing nothing but the init system."

Comments (112 posted)

Jolla offer a first look at their Sailfish smartphone OS (The Verge)

The Verge takes a look at the new Meego-based Sailfish, from the Finnish startup, Jolla. "There are a few interesting UI concepts detailed in the video. First, the homescreen itself acts as a multitasking menu: when in an app you can swipe it away to push it to the homescreen, where it'll act like a large, interactive widget. The application drawer itself lives underneath the homescreen and can be accessed with a swipe up, while notifications are displayed in the upper right. There's an automatic theme generator called "ambience," which creates a look and feel for your device by analyzing a photo. Jolla says many Android apps will run "unaltered" via Myriad's Alien Dalvik software, while others will require some tweaking." (Thanks to Mats)

Comments (22 posted)

Page editor: Rebecca Sobol

Development

Wikipedia's new HTML5 video player

Wikimedia recently rolled out a new feature on Wikipedia and related sites: an HTML5-based media player. Although Wikimedia's version is implemented in an extension for the MediaWiki software, the player itself is based on an existing open source tool, which makes it potentially useful to a much wider audience.

The player is provided in MediaWiki's TimedMediaHandler extension, which is an adaptation of the HTML5 media player created by the Kaltura video platform. Although the HTML5 <video> element garners more attention, the player can handle <audio> content as well. Previously, Wikipedia and its sister sites had embedded multimedia content in pages via an Ogg-only player (Theora for video, Vorbis for audio), which came with its share of difficulties for users whose browsers lacked Ogg support. The new player handles the royalty-free WebM format as well (sporting VP8 video support), but its key features extend beyond codec coverage.

First, the extension is designed to allow page authors to embed video or audio directly into a MediaWiki site using the same syntax used for images. Displaying a video can be as simple as adding [[File:Foo.ogv]] to the page text; the extension handles the rest. As is the case with other MediaWiki markup elements, appending a vertical bar allows the author to access some optional features. The |start=mm:ss argument will start playing the media clip at the time specified; similarly |end=mm:ss indicates when to stop playing the clip. The |thumbtime=mm:ss argument uses the frame at time mm:ss as the thumbnail image for the video.

For site visitors, the player should appear as an embedded video with a strip of controls across the bottom, just like all of the Flash-based players so common today. The Wikimedia bug tracker includes some reports of difficulty with Safari on some versions of Mac OS X, but at the moment there is not a definitive browser compatibility list.

The extension is also designed to handle closed captioning, subtitles, and any other "timed text" associated with an audio or video file. MediaWiki supports the SRT captioning format for timed text, and the player displays a "CC" icon in the media toolbar from which users can activate any available text track. Subtitles can be in any language; after all, one of Wikimedia's goals is to provide translations into as many languages as possible. Consequently, the HTML5 media player integrates with MediaWiki's existing translation tools.

It is also tied in to MediaWiki's media upload system. In a blog post outlining the history of the player, Guillaume Paumier notes that the organization has put effort into re-engineering Wikimedia Commons, the content repository shared by all of Wikimedia's sites. The ability to start and end delivery of multimedia content at specific timestamps is, no doubt, one technique used to remove redundancy: there is no need to store multiple copies of a lengthy video so that specific segments can be played back on different pages.

Paumier also contrasts the playback extension with the services offered by commercial video hosting sites. The MediaWiki extension does not harvest end-user data or report it to anyone else, and it displays authorship information for the content, even when embedded in a third-party page.

Servers, fallback, and more

The server side of the TimedMediaHandler extension also allows the wiki to transcode content on the server side to deliver it at different quality levels and resolutions. It uses FFmpeg to perform the transcoding, from a pre-defined set of basic profiles (160p, 480p, 720p, etc.). Which transcoding profiles are active is configurable in the MediaWiki settings file. Transcoding jobs are not automatically run by the MediaWiki job queue due to their resource-intensive nature; they can be queued up, but the resulting webVideoTranscode queue must be started manually. In the player user interface, the available versions of a file are accessible in a pop-up menu.

On browsers that do not support the player widget (which would include any that have JavaScript switched off, as it uses jQuery), the extension falls back on the Java-based Cortado media player applet. Presumably those who switch off both JavaScript and Java are comfortable missing out on some audio and video content. Interestingly enough, although Vorbis remains the sole audio codec supported in the current release, the documentation notes that MP3 support is in the works, awaiting the expiration of the relevant patents. In fact, the documentation says the patents will expire in 2012, although there are plenty of other dates given in other locations (including Wikipedia itself).

Beyond MP3, there are other codec possibilities (such as WAV, Opus, and FLAC). Multiple codec support is hardly the mission-critical issue for Wikimedia, however. Integration with the existing page creation and editing tools is probably of more practical importance. On that front, there is still plenty of work remaining to be done. In September 2010, TimedMediaHandler developer Michael Dale wrote a blog entry about the Kaltura video "sequencer," an editing widget that Wikimedia was working to integrate into the MediaWiki page editing tools.

The Kaltura sequencer is not yet a full editor; rather than creating a single output file, it creates a tightly-orchestrated sequences of clips. This means there is still a gap in the workflow for open web video, much as there still is on the desktop. Nevertheless, the Wikimedia TimedMediaHandler player is still an important milestone, if for no other reason than because of the vast assortment of MediaWiki-based web sites.

There are multiple alternative HTML5-based media players available; the html5video.org site maintains a list with basic information on each. It is important to note that the site is hosted by Kaltura, but it also embeds each of the listed players, so a side-by-side comparison is comparatively simple. The Kaltura player is clearly one of the most modern options, with support for less common parts of the specification like Media Events (e.g., attributes like onloadedmetadata and onratechange). But Wikimedia has clearly done work to improve it in other areas, such as the timed text support. For the majority of the millions of visitors that frequent Wikipedia and the like on a daily basis, the only important feature is the fact that playback will work. But for open source advocates and other software developers, the availability of a new playback tool is just the beginning.

Comments (3 posted)

Brief items

Quotes of the week

— Eric S. Raymond

— Alan Cox

Comments (53 posted)

GNU Libtasn1 3.1 released

Version 3.1 of GNU Libtasn1, the library for working with X.509 and Kerberos structures, is out. Noteworthy changes include a number of new and renamed types, and more detailed error messages.

Full Story (comments: none)

kdev-python 1.4 stable released

The first stable release of kdev-python is now available. Kdev-python is the Python support plugin for KDE's KDevelop IDE. It supports KDevelop's syntax highlighting, code completion, jump-to-declaration, and all sorts of other IDE goodies.

Full Story (comments: none)

Koha 3.10 released

The Koha community has released version 3.10.0 of its free software library management system. Highlights include Plack compatibility in the staff interface and significant enhancements to the Acquisitions, Cataloging, and Circulation modules.

Comments (none posted)

GStreamer SDK for Android released

Collabora and Fluendo have announced the next release of their jointly-developed GStreamer SDK, this time adding Android support. The Android port is decidedly unofficial, as reflected in the announcement. "The GStreamer SDK for Android is targeted at API Level 9 (Android 2.3.1, Gingerbread) or higher. However, due to some of the restrictions of previous versions of Android, some features such as hardware acceleration are only available from API Level 16 (Android 4.1, Jelly Bean)."

Comments (none posted)

Newsletters and articles

Development newsletters from the past week

• Caml Weekly News (November 27)
• What's cooking in git.git (November 21)
• What's cooking in git.git (November 28)
• Haskell Weekly News (November 21)
• Mozilla Hacks Weekly (November 22)
• Perl Weekly (November 26)
• PostgreSQL Weekly News (November 26)
• Ruby Weekly (November 22)

Comments (none posted)

Esfahbod: High-DPI, Subpixel Text Positioning, Hinting

HarfBuzz maintainer Behdad Esfahbod has published a white paper [Google Doc] of interest to those working on font rendering. Entitled "High-DPI, Subpixel Text Positioning, Hinting," it covers his recent work to catalog the effects of various approaches to hinting, antialiasing, and sub-pixel positioning, particularly when applied to modern high-resolution displays. If reading this on your new monitor hurts your eyes, you should probably take a look.

Full Story (comments: none)

Page editor: Nathan Willis

Announcements

Brief items

Ask OpenOffice -- The Apache OpenOffice project invites your questions

The Apache OpenOffice project is collecting questions about Apache OpenOffice. "Technical questions, questions about the open source project, questions about where we see things going in 5 years, all are good candidates. You submit your questions and help rate questions submitted by others. We'll then collect the top 10 questions and respond to them in a future project blog post (http://blogs.apache.org/ooo/)."

Full Story (comments: none)

Calls for Presentations

Open Source Data Center Conference 2013 - CfP

The Open Source Data Center Conference will take place April 17-18, 2013 in Nuremberg, Germany. The call for papers is open until December 31.

Comments (none posted)

Upcoming Events

linux.conf.au announces Golden Ticket winner

linux.conf.au (LCA) has announced the winner of the Golden Ticket. "Gavin [Jackson] is a Canberra resident who has attended several linux.conf.au conferences previously at his own expense. He says he is 'absolutely thrilled and grateful to be attending the event as a delegate and [I am] looking forward to networking with other members of the LCA community and sharing war stories, tips and beers.'" LCA takes place in Canberra, Australia, January 28-February 2, 2013.

Full Story (comments: none)

Events: November 29, 2012 to January 28, 2013

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
November 29 November 30	Lua Workshop 2012	Reston, VA, USA
November 29 December 1	FOSS.IN/2012	Bangalore, India
November 30 December 2	Open Hard- and Software Workshop 2012	Garching bei München, Germany
November 30 December 2	CloudStack Collaboration Conference	Las Vegas, NV, USA
December 1 December 2	Konferensi BlankOn #4	Bogor, Indonesia
December 2	Foswiki Association General Assembly	online and Dublin, Ireland
December 5	4th UK Manycore Computing Conference	Bristol, UK
December 5 December 7	Qt Developers Days 2012 North America	Santa Clara, CA, USA
December 5 December 7	Open Source Developers Conference Sydney 2012	Sydney, Australia
December 7 December 9	CISSE 12	Everywhere, Internet
December 9 December 14	26th Large Installation System Administration Conference	San Diego, CA, USA
December 27 December 29	SciPy India 2012	IIT Bombay, India
December 27 December 30	29th Chaos Communication Congress	Hamburg, Germany
December 28 December 30	Exceptionally Hard & Soft Meeting 2012	Berlin, Germany
January 18 January 19	Columbus Python Workshop	Columbus, OH, USA
January 18 January 20	FUDCon:Lawrence 2013	Lawrence, Kansas, USA
January 20	Berlin Open Source Meetup	Berlin, Germany

If your event does not appear here, please tell us about it.

Page editor: Rebecca Sobol