The Grumpy Editor's Moblin review

Despite your editor's affection for electronic toys, he has, thus far, managed to avoid cluttering his desk with a netbook system. Until now. It seemed like it was past time for a closer look at Moblin, and it further seemed that a distribution designed for netbooks should be experienced on one. So, it didn't take long for your editor to come into possession of a Dell "Mini 10v", which ships with the Ubuntu Moblin remix preinstalled. The 10v is a cute little system, but it is, alas, saddled with a free-software-unfriendly Broadcom chipset. Needless to say, the version of Ubuntu shipped on the hardware includes the binary driver needed to make that chipset work.

Much hype has been generated about Moblin's extra-fast booting behavior. A quick check with the stopwatch shows that this system requires 27 seconds from when the BIOS completes its power-on ablutions until the login screen appears. That is a definite improvement over a number of other systems, but it's not quite, yet, the five seconds that the Moblin folks have been aiming for. Suspend and resume are both quite fast; opening the lid yields a working system within 2-3 seconds.

The Moblin experience starts at the "MyZone" screen, containing calendar and "to do" items, icons for a few favorite applications, a set of screens from recently-run applications, and an area meant to contain communications from online friends. When an application is running, all of the "MyZone" stuff goes away, leaving the full screen for whatever the user is working on at the time. Screen space is generally at a premium on netbooks, so Moblin goes out of its way to waste as little of it as possible.

A core feature of the Moblin interface is "zones." These are really just the virtual desktops or workspaces that Linux users have been using since before Linux existed. On a small screen, though, there is little value in having more than one application on-screen at a time, so Moblin usually starts each application in its own zone. Switching between applications normally requires moving between zones.

There is a task bar which can be obtained by moving the pointer to the top of the screen. A quick look at this bar is enough to clarify the things that Moblin's designers think netbook users will want to do. Top-level tasks in Moblin include adjusting one's online social networking status, connecting to people, running a web browser, running a media player, and accessing a "pasteboard." There are icons for battery and networking status, one for moving between zones, and one for "applications" which is the path toward any other programs the user might want to run. Users who buy a netbook to support extensive twitter activity, watch videos, and view the occasional web page will be more than pleased with Moblin. Those wanting to do kernel development are likely to find this environment to be somewhat irritating.

Your editor has been using computers for quite some time; the notion that one can get a program into a system without punching it onto cards first still seems novel at times. To your editor's eyes, the Moblin environment has the feel of a toy. Lots of bright pastel colors assault the eye. Picture thumbnails dance around each other before lining up in pretty little rows. Dialog windows bounce on the screen in ways which risk inducing motion sickness. It's all very cute and joyful and social; Moblin is clearly not aimed at a typical longtime desktop Linux user.

Another choice which makes it clear that your editor is not in the target audience: this is the first distribution encountered in years which does not come with an SSH client. This kind of problem is easily fixed - the entire Ubuntu repository is accessible to people who dig far enough into the menus - but it is a bit of a surprise.

This machine arrived with an Ubuntu 9.04-based system running Moblin 2.0. This distribution, it must be said, has some rough edges. OpenOffice.org comes up with a dialog whose buttons are below the bottom of the window, which, in turn, refuses to let the user resize it (see image to the right). The mail client features color choices which sometimes render text unreadable. There are no bookmarks in the web browser; this browser also thinks that users want their searches to go to Yahoo. Windows vanish abruptly from the screen, losing whatever work may be in progress. Dell's page notes that the system is for early adopters; that certainly seems to be the case.

One should note that 9.04 is not the current version of Ubuntu, and 2.0 is not the current version of Moblin. There is a newer version of the Moblin build, based on the 9.10 release. The download page nicely offers a CD image of this release, seemingly unaware of the fact that a lot of netbooks lack CD drives. Ubuntu has a tool (usb-creator) which will create a bootable USB device from a CD image; too bad that its window is much taller than a typical netbook screen, making the crucial buttons unreachable. Your editor finally got past that little problem and was able to create a bootable Ubuntu 9.10 device.

The result was a very sluggish, very brown, but a generally slicker-looking Moblin installation. The software installation feature has been made more prominent, and the list of available applications has grown. Moblin 2.1 lacks support for the Broadcom wireless adapter found in this device, but that is not really Moblin's fault. The web browser still leaves much to be desired - strange, because Moblin 2.1 has made a number of improvements in that area. One other thing your editor noticed with both Ubuntu versions: the power consumption seems high. Running PowerTop shows a steady state of anywhere between 100 and 350 wakeups/second - not the way to get the most from one's battery. Moblin is supposed to be better than that.

Your editor decided to go straight to the source: the Moblin.org download page, which offers an image which works nicely from a USB stick. Some things have not really improved: it still takes 30 seconds to boot the system (though it should be noted that the use of a USB stick will slow things somewhat). But 30 seconds beats the few minutes that USB-based Ubuntu required, and the system is more responsive thereafter as well. And there are some improvements to be seen in this version of the distribution.

For example, the web browser (a Mozilla derivative) is indeed improved: it now has support for bookmarks, extensions, and a full set of preferences to tweak. This version of Moblin comes with its own package installer backed by Moblin's repository; users can install real applications like Thunderbird or AbiWord, but the package selection is far smaller than found with Ubuntu 9.10. Interestingly, OpenOffice.org is not available for this build - a surprise, given how many people your editor has seen running presentations from netbooks over the last year.

The official Moblin build is indeed more power-efficient, though it still runs at 80-90 wakeups/second, which is too many. All told, Moblin feels a little bit like an unfinished product, still.

In general, your editor is not really sold on the netbook idea. The screen is too small to get much serious work done, and the aspect ratio is wrong for any sort of text-oriented work. The keyboard tends to be just big enough to tempt the user to try to really type on it. And, frankly, Moblin-like software just tends to get in the way of a user who is used to the full Linux desktop experience and who does not spend a lot of time on Twitter. Chances are good that this particular netbook will eventually find itself running a more traditional Linux distribution.

But, as has been noted already, your editor is clearly not the market that these systems are aimed at. Not yet, at least. There are some very interesting changes happening in the area of consumer-level computers, where the traditional desktop idea seems to be slowly falling out of favor. Many experiments are underway to come up with something better; in the free software world these experiments have names like Android, Chromium OS, Litl, Maemo, and Moblin. Free software is trying to break new ground here; this is not a case of following somebody else's taillights. So, while your editor does not see Moblin as his system of choice at the moment, he is most interested in seeing where this project goes in the near future.

Comments (44 posted)

Fedora 12 and unprivileged package installation

Fedora 12 was released on November 17 with the usual pile of new packages and features. By the sounds, it is a solid, well-received release. But one feature—unpublicized, undocumented, and turned on by default—has a number of Fedora users up in arms, leading to a huge thread on fedora-devel, in the bugzilla entry, and here at LWN. In short, the problem was that in the Fedora 12 default installation, regular users sitting at the console could install signed packages from any repository that the administrator has enabled.

Since the release, and all of the publicity and complaints, the maintainers of PackageKit have decided to remove the feature. Out of this controversy, though, are lessons for any project regarding security, transparency, and system defaults. There were no real complaints about the existence of the feature, rather it was the choice to make it the default, coupled with a lack of any notice of the change, that led to the outcry.

Unprivileged package installation

Non-root install is a convenience feature, and one that was supported somewhat differently in earlier Fedoras. From F9 through F11, the same effect could be achieved by using PackageKit to install a package, entering the root password, and checking a box to allow that user to install in the future without needing to enter the root password. The key difference is that in F12, no root password is ever required; the checkbox has been removed and is treated as if it was turned on for all users.

PackageKit goes to some lengths to determine that the user is logged on at the physical console of the machine before allowing non-root installs. It only allows installation of packages, not update or removal, and requires that the packages are signed by a key that has been installed by root. The only repositories that are allowed to be used are those that were previously configured and enabled by a root user.

The use case is for single-user (or all trusted user) systems, where the logged-in user is likely to be the same person as the root user. Some people evidently don't like having to enter the root password, or, worse, having to track down the person with the root password, when they install software. It is part of the effort to simplify the desktop experience, with package installation being considered a "routine" task that many users would like to be able to do without the extra password-entering step. But that has serious security implications.

So, why wasn't the previous behavior just propagated into F12? It turns out that the PolicyKit feature that was used (auth_admin_keep_always) was eliminated between PolicyKit 0.9 and 1.0, because it was considered to be a security problem. Because F12 uses PolicyKit 1.0, it makes it difficult to just revert to the old behavior. Instead, PackageKit maintainer Richard Hughes has decided to change the policy such that the root password will always be required for installing packages on F12. An alternative was proposed by Kevin Kofler that may allow the earlier behavior to return without the PolicyKit support, though it is unclear whether it is being considered.

Security ramifications

It didn't take much thought for various folks to come up with security issues with the new feature. Even for the stated use case, allowing unprivileged package installation has some fairly significant implications. The idea that running on the console somehow makes a process trusted is dubious at best. Firefox is an excellent example of a program that regularly has flaws which may lead to arbitrary code execution. That means that attackers on the web may be able to install packages on F12 systems.

But the proponents of this feature insist that there is no risk to installing trusted packages from trusted repositories. There are a number of problems with that thinking, starting with the fact that there are, without question, "trusted" packages in the Fedora repositories today that have privilege escalation and other security flaws. Trusting a repository does not in any way imply trusting every package in it. Administrators may well have added other repositories to pick up a package or two as well, without considering the fact that they have now opened up their systems to all of the packages contained in that, less trusted, repository.

There is also an incident that some are conveniently forgetting. In August 2008, there was some kind of break-in to the Fedora project servers, including the system used for signing packages. There is no evidence that malicious packages were signed at that time, but it is always a possibility in the future. During the time when Fedora was figuring out what happened, and issuing new signing keys, users were warned not to update their systems. But, if console applications can be subverted to do that installation, one can easily see a path to mass compromise of Fedora systems.

Because of the way this was changed, administrators who upgraded to F12 will find that the privileges of the users on their system have suddenly been elevated. Because PackageKit and PolicyKit are relatively new additions, many administrators may be largely unaware of them and their capabilities. Eliminating PackageKit is one way to avoid the current problem, but other, seemingly unrelated packages are dependent on it; setroubleshoot for example. Because PackageKit and PolicyKit function in ways that are very different from the traditional UNIX security model, it is imperative that Fedora protect its users from unexpected security holes, at least in the default installation.

SELinux hacker James Morris has a summary of the problems that can occur, including such things as denial of service from exhausting the disk space on the system. A user could also install an SELinux policy that is weaker than that installed by the administrator, leading to an overall reduction in the security of the system. Overall, the implications of this change were not fully explored before it was added to F12.

Defense

The main defense of the PackageKit behavior seems to be that any attacker who has physical access to the machine has many ways to subvert it. Hughes listed a number of other actions that a normal user can do in F12, some of which could certainly be considered security issues. Those don't make for a valid reason for the PackageKit changes, though, as Rick L. Vinyard Jr. points out: "Perhaps those should also be discussed and analyzed further, but that doesn't serve as a justification for the matter at hand."

There are multiple scenarios where console access does not imply access to the machine. A monitor, keyboard, and mouse are all that is required for a console, not necessarily access to the power button, USB ports, CD drives, and so forth. So, it is not universally true that console access equates to physical access. In addition, various tools like VNC allow remote users to act as if they were on the console. While VNC itself is detected as a non-local console, x11vnc is not.

Normally, one would expect this kind of change to be noticed in Rawhide—Fedora's development distribution—long before the F12 release. That would have allowed the issue to be discussed and resolved well in advance of unsuspecting users upgrading into the new policy. Unfortunately, the Rawhide packages aren't signed, so PackageKit always requires the root password to install them. So the first time the Fedora community saw this change was after they had upgraded to the "real" F12.

One might also expect a change of this magnitude to appear, perhaps quite prominently, in the F12 release notes, but that was not the case at the time of the release. Since that time, a very eye-catching note was added to the security section of the release notes. That could serve as a warning for users that read, or at least skim, those notes.

The other main line of defense is that this behavior is "just" a default, and can be changed by administrators. While that is true, the process to do so is not obvious. It involves mucking about with PolicyKit files, something that many Fedora users probably know little to nothing about. Hughes thinks that users should learn PolicyKit: "If you're deploying F12, then I really think you should know the basics about PolicyKit." But, when Seth Vidal set out to find out how to disable the feature—documented on his blog—asking Hughes did not lead to the solution: "So, if our engineers don't know the basics, how should our users?"

Overall, Hughes's reaction to the problem has been dismissive, bordering on rude:

Based on the above, one could argue with the "trivially", but, more to the point, one must understand an existing security model before changing it. If one myopically focuses on a single use case, and ignores the problems inherent in even that case, concluding that allowing unprivileged users to do package installation might make sense. But, for overall system security—not to mention the reputation of Fedora as a distribution that keeps security in mind—myopia is not a good strategy.

Hughes often refers to the change only being made for the "desktop spin", but that doesn't really make sense as the feature was added to all of Fedora. Certainly, some spins—server, for example—could change this default, but that seems backward. The core should default to secure choices, and allow spins to relax those requirements if they so desire. As Adam Williamson points out:

Larger issues

Regardless of how they feel about the specific feature in question, Fedora developers would like to avoid being blindsided by these kinds of changes in the future. To that end, Chris Adams started a discussion on security policy oversight: "Any package (whether new or an update) that adds/changes PolicyKit, consolehelper, or PAM configuration, and anything that installs new setuid/setgid executables, should require some additional third-party review." As part of that discussion, Hughes seems to be coming around to the majority view: "At the moment we're asking the server spin to essentially close the door, when maybe we should start with a closed door, and be asking the desktop spin to open it up a little more."

There is concern that a package maintainer can change global behavior for the distribution without any notice. Once the change is made, that maintainer can refuse to change it back, requiring the Fedora Engineering Steering Committee (FESCo) to step in and make a decision. In the case of a security problem, one that the maintainer is unwilling to acknowledge, the delay could be a serious problem. Since the change to PackageKit still has not been released as of this writing, there are numerous systems out there that are being installed or upgraded into less security.

FESCo discussed the issue at its November 20 meeting, and Vidal will be putting together a proposal to require the maintainers of critical packages to disclose any changes that might have this kind of impact. From the discussion of Adams's call for more oversight, a need for an overall framework of what users should and should not be able to do was noted. Fedora engineering manager Tom "spot" Callaway has started gathering a list of things that unprivileged users should not be able to do that would presumably factor into such a policy.

There were also some ancillary, but still important, issues. The responses from Hughes and David Zeuthen ("I'm not interested in this bike-shed or what color it is") in the thread angered quite a few. As the discussions grew, several Fedora leaders tried to tamp down the flames, which also didn't sit well with folks. A suggestion to vote on the bug, rather than continue piling on to the discussion was met with opposition as well, leading Jeff Garzik to note:

At one point Red Hat employees started using hidden comments in the bug to create a virtual "executive session", but folks started noticing the skipped message numbers and wondered what was going on. Williamson stepped in quickly to stop that:

The future

The Fedora project has likely learned quite a bit from this particular controversy, and it seems to be taking the right steps to avoid a repeat in the future. For a distribution that went through a great deal of pain to integrate SELinux features in order to increase the security of the system, it is mind-boggling to many that this non-root install feature was added as the default. There were multiple missteps—making it the default, not highlighting it in the release notes, not testing it in Rawhide, and so on—but those can all be corrected. Hopefully, the outcry and publicity will ensure that the word gets out, so that Fedora users will understand the issue and can make the appropriate changes for their systems.

In the meantime, though, other projects—distributions or software packages—would be well-served by studying this episode. Security is hard, and requires great diligence. It is likely that other projects could have hit this same kind of problem, but, hopefully, with this incident as a guide, will avoid doing so in the future.

Comments (51 posted)

The 2009 Linux and free software timeline - Q1

Here is LWN's twelfth annual timeline of significant events in the Linux and free software world for the year.

2009 offered few surprises to those that have been following Linux and free software for as long as we have. As expected, there were new releases of many of the tools and underlying infrastructure that we use on a daily basis. There were also lawsuits over software patents, arguments over licensing, and various security flaws found and fixed. Distributions were packaged up and released, more phones and other devices with Linux and free software were sold, and so forth. All part of the march to "world domination". We look forward to 2010—and beyond.

This year we will be breaking things up into quarters, and this is our report on January-March 2009. Over the next month or so, we will be putting out timelines of the other three quarters of the year.

---

This is version 0.8 of the 2009 timeline. There are almost certainly some errors or omissions; if you find any, please send them to timeline@lwn.net.

LWN subscribers have paid for the development of this timeline, along with previous timelines and the weekly editions. If you like what you see here, or elsewhere on the site, please consider subscribing to LWN.

For those with a nostalgic bent, our timeline index page has links to the previous eleven timelines and some other retrospective articles going all the way back to 1998.

January

One Laptop Per Child (OLPC) restructures, laying off half the staff and "refocusing" in various ways. (OLPC blog)

Valgrind releases version 3.4.0 of the popular program analysis tool for finding memory and other errors. (review).

Nokia announces the release of Qt under LGPLv2.1 for the upcoming 4.5 release. (announcement).

linux.conf.au is held in Hobart, Tasmania. (LWN coverage, 2, 3, 4, and 5)

Red Hat Enterprise Linux 5.3 is released. (announcement)

Moonlight developers work overtime to make President Obama's inauguration viewable on Linux, because the streams were only made available in Silverlight form. (article)

GCC and FSF announce a GPLv3 exception to allow for GCC plugins; the exception is for the GCC runtime library and will allow free software plugins, while preventing proprietary plugins. This particular incarnation of the exception is not adopted. (announcement)

KNOPPIX 6.0 is released. (announcement, review)

KDE 4.2 is released. (announcement)

AMD releases 3D register reference guide for R6xx/R7xx chips, which will help with the development of free software drivers for devices using those chips. (announcement)

The Linux Foundation kicks off the "We're Linux" video contest. (press release)

February

Zope 3.4 is released after two years of development on the Python-based web application server.(announcement)

Red Hat hires former Mandriva community manager Adam Williamson to drive community involvement in Fedora QA. (introduction)

Miro internet TV version 2.0 is released. (announcement)

RPM version 4.6.0 released; the package manager used by Red Hat, Mandriva, SUSE, and others. (announcement)

Debian 5.0 ("Lenny") is released after "22 months of constant development". (announcement) The release is dedicated to Thiemo Seufer, a community member who died in a car accident.

DragonFly BSD 2.2 is released—now with a production-ready HAMMER filesystem. (announcement)

Kurt Roeckx is appointed as Debian project secretary, after the previous secretary resigned in late 2008. (announcement)

Red Hat moves from Xen to KVM for virtualization in future releases, as expected by many after its acquisition of Qumranet. (press release)

Microsoft launches patent suit against TomTom, for patents on the VFAT filesystem among other things. (LWN coverage)

BASH 4.0 is released.; BASH is the Bourne-Again SHell (announcement)

X server 1.6.0 released. (announcement)

March

The Linux Foundation acquires the Linux.com domain, which they will turn into a community news and collaboration site. (announcement)

MontaVista starts Meld community site for embedded Linux developers. (announcement)

The "ext4 data loss" controversy heats up. (first LWN article)

Firefox 3.1 renamed to 3.5 to better reflect the scope of the changes. (announcement)

The Linux kernel gets a new logo for one release; "Tuz" is a reminder of the plight of the Tasmanian devil. (LWN coverage)

GNOME 2.26 released. (announcement)

Parrot 1.0.0 released; Parrot is a "virtual machine aimed at running all dynamic languages". (announcement, LWN article)

Linux 2.6.29 is released with an experimental Btrfs, squashfs, kernel mode setting for Intel graphics hardware, and more. (announcement, KernelNewbies coverage)

SUSE Linux Enterprise 11 is released in both desktop (SLED) and server (SLES) varieties. (press release)

Rails 2.3 released—aka Ruby on Rails, the Ruby-based web framework. (announcement)

GNOME switches to Git, from Subversion, for version control. (announcement)

Microsoft vs. TomTom comes to an end, via a settlement, but, before that, TomTom joins the Open Invention Network and countersues Microsoft. (Groklaw settlement article)

Fedora issues report on August 2008 intrusion, seven months after it occurs. (report)

Python starts switch to Mercurial for distributed version control. (Guido van Rossum's announcement)

Comments (18 posted)

Firefox locks down the components directory

Mozilla announced an important security change in the upcoming Firefox 3.6 that should prevent dangerous behavior caused by external applications bypassing the normal add-on installation process. Starting with 3.6, extensions, plugins, and third-party applications will no longer be able to install code in Firefox's components directory, where much of the browser's own code is housed and loaded automatically. Installing code into the directory was never an approved method, but because the lockdown is a change, Mozilla is providing add-on developers with technical assistance to transition their code to the approved, XPI-based installation scheme. For end users, the change should not adversely affect browsing, but may improve stability and should make for a more secure Firefox experience.

Mozilla's Johnathan Nightingale (whose actual job title is "Human Shield") announced the change on the Mozilla developer center blog on November 16, noting that Firefox is "built around the idea of extensibility" and that although the ability for third-party programs like add-ons to install code in components has been around for a long time, "there are no special abilities that come from doing things this way, but there are some significant disadvantages".

Code installed into the components directory is invisible to end user tools such as the Add-ons Manager, he said, making it impossible to deactivate, and it does not include the version compatibility information that Firefox uses to deactivate deprecated and incompatible add-ons.

Starting with the current beta release of Firefox 3.6, no code except Firefox's known binary components will be loaded from the directory, by checking the file components.list. The change closes bug 519357, the Bugzilla comment thread of which sheds additional light on the implementation. It is still possible for a third-party installer to put binaries or scripts into the components directory, but Firefox will ignore them.

Mozilla developer Vladimir Vukicevic clarified that most extensions and other add-ons are well-behaved in regard to the components directory. "The majority of the problems are actually with third-party apps, such as virus scanners, network optimizers, and similar. These often attempt to integrate with Firefox by dropping a bare component library (a DLL on windows, or a shared library on Linux) into Firefox's component directory." The situation is essentially a non-issue on Linux, but is "pretty prevalent" on Windows. Going forward, he added, third-party applications can hook into Firefox with the same amount of access by providing an extension.

Exactly which legitimate add-ons have been exploiting the components directory loophole is not publicly documented, but one prominent example given in the Bugzilla bug page is Google Desktop Search, which places three files into the directory. Commenters at Slashdot and other web discussion forums pointed fingers at other commercially-developed extensions from Yahoo, Adobe, Sun, and Skype. The mozdev.org plugin documentation for Linux advises installing several older plugins in the components directory, although this is not the recommended approach, and Vukicevic noted that the instructions generally address only XPCOM Type Library (XPT) files, which do not contain any code.

Security with binary and JavaScript add-ons

Vukicevic provided more detail on the issue at his personal blog. Binary code modules in particular took advantage of the components directory to be loaded on application startup, bypassing the version-checking that approved add-ons undergo. Because "the line between supported/frozen and completely unfrozen internal Gecko interfaces is blurred", third-party binaries can cause serious problems when run with a different version of Firefox than the one they were originally written against. Several add-ons taking advantage of the components directory were written for Firefox 3.0, and have not been updated for 3.5, leading to crashes.

Clearly, having executables loaded automatically at application startup simply because they are located in the components directory is a security hole, particularly when they are beyond the reach of Firefox's add-on management interface — some bug reports, such as 503946, suggest isolated incidents of malware attempting to exploit the situation, although no widespread attacks are known. Another risk raised in the Bugzilla comments, however, is the possibility that the legitimate Firefox components could be corrupted or replaced; preventing that attack would require additional measures, like hashing the component contents, which are not part of the current plan. Restricting the components directory to a list of known files is important enough, though, that Vukicevic said it may be backported to 3.5

Vukicevic also hopes that many developers who have relied on binaries in the past — including through the official add-ons approach — will take advantage of another new feature in Firefox 3.6, JSCtypes. JSCtypes is a module that exposes C-compatible external library functions to JavaScript code. This allows add-on authors to call native operating system APIs without writing their add-on as a binary XPCOM component. By keeping as much of the code as possible in JavaScript, Vukicevic said, the chances of breaking the add-on between Firefox revisions is reduced.

To assist affected application and add-on developers to transition away from using the components directory and to standard installation methods, Mozilla has also written a migration document entitled "Migrating raw components to add-ons." The guide covers basic extension authoring, installers, best practices for automatic extension installation, and an introduction to JSCtypes.

Microsoft came under fire earlier in 2009 when its .Net Framework Assistant add-on was automatically installed by Windows Update and tied to a serious security vulnerability. Mozilla used its rarely-triggered add-ons blocklist to disable .Net Framework Assistant. Although it was a high-profile story at the time, the case is noteworthy not because the offending add-on was guilty of exploiting the components directory — it was not — but because the add-on obeyed the rules. Had .Net Framework Assistant dumped unregistered DLLs into the components directory, Mozilla's blocklist feature wold not have been able to automatically deactivate them. Starting with Firefox 3.6, that speculative scenario is no longer a possibility.

Comments (6 posted)

Linux Implements Support For Trusted Computing, Safer Online Transactions (The Gov Monitor)

The Gov Monitor looks at Trusted Computing (TC) features in openSUSE 11.2. "Technikon led a consortium of 23 research and business partners, including AMD, IBM, HP, Infineon and Novell, in developing open source software and applications for TC environments as part of the EU-funded OpenTC project. The group’s implementation of TC support in openSUSE version 11.2 involved building a trusted software stack (TSS) for Linux, developing universal virtualisation layers (including improvements to the Xen hypervisor virtual machine monitor) and creating TC and TPM management software. It constitutes a pioneering implementation of TC technology."

Comments (29 posted)

asterisk: cross-site ajax requests

Package(s):	asterisk	CVE #(s):	CVE-2008-7220
Created:	November 24, 2009	Updated:	June 4, 2010
Description:	From the CVE entry: Unspecified vulnerability in Prototype JavaScript framework (prototypejs) before 1.6.0.2 allows attackers to make "cross-site ajax requests" via unknown vectors.
Alerts:	Gentoo 201006-20 asterisk 2010-06-04 Debian DSA-1952-1 asterisk 2009-12-15 Fedora FEDORA-2009-11126 asterisk 2009-11-06 Fedora FEDORA-2009-11070 asterisk 2009-11-06

Comments (none posted)

bugzilla: information leak

Package(s):	bugzilla	CVE #(s):	CVE-2009-3386
Created:	November 24, 2009	Updated:	November 24, 2009
Description:	From the CVE entry: Template.pm in Bugzilla 3.3.2 through 3.4.3 and 3.5 through 3.5.1 allows remote attackers to discover the alias of a private bug by reading the (1) Depends On or (2) Blocks field of a related bug.
Alerts:	Fedora FEDORA-2009-11919 bugzilla 2009-11-24

Comments (none posted)

gforge: cross-site scripting

Package(s):	gforge	CVE #(s):	CVE-2009-3303
Created:	November 23, 2009	Updated:	November 24, 2009
Description:	From the Debian advisory: It was discovered that gforge, collaborative development tool, is prone to a cross-site scripting attack via the helpname parameter.
Alerts:	Debian DSA-1937-1 gforge 2009-11-21

Comments (none posted)

opera: cross-site scripting

Package(s):	opera	CVE #(s):	CVE-2009-3265 CVE-2009-3266
Created:	November 24, 2009	Updated:	November 24, 2009
Description:	Cross-site scripting (XSS) vulnerability in Opera 9 and 10 allows remote attackers to inject arbitrary web script or HTML via a (1) RSS or (2) Atom feed, related to the rendering of the application/rss+xml content type as "scripted content." NOTE: the vendor reportedly considers this behavior a "design feature," not a vulnerability. (CVE-2009-3265) Opera before 10.01 does not properly restrict HTML in a (1) RSS or (2) Atom feed, which allows remote attackers to conduct cross-site scripting (XSS) attacks, and conduct cross-zone scripting attacks involving the Feed Subscription Page to read feeds or create feed subscriptions, via a crafted feed, related to the rendering of the application/rss+xml content type as "scripted content." (CVE-2009-3266)
Alerts:	SuSE SUSE-SR:2009:019 cups, jetty5, libqt4/dbus-1-qt, opera, puretls/jessie, kdegraphics3-pdf, qemu 2009-11-24

Comments (none posted)

php: multiple vulnerabilities

Package(s):	php	CVE #(s):	CVE-2009-3557 CVE-2009-3558 CVE-2009-3559
Created:	November 23, 2009	Updated:	January 25, 2010
Description:	From the Mandriva advisory: - Fixed a safe_mode bypass in tempnam() identified by Grzegorz Stachowiak. (CVE-2009-3557, Rasmus) - Fixed a open_basedir bypass in posix_mkfifo() identified by Grzegorz Stachowiak. (CVE-2009-3558, Rasmus) - Fixed bug #50063 (safe_mode_include_dir fails). (CVE-2009-3559, Johannes, christian at elmerot dot se)
Alerts:	Slackware SSA:2010-024-02 php 2010-01-25 Gentoo 201001-03 php 2010-01-05 Mandriva MDVSA-2009:324 php 2009-12-07 Ubuntu USN-862-1 php5 2009-11-26 Mandriva MDVSA-2009:303 php 2009-11-28 Mandriva MDVSA-2009:302 php 2009-11-21

Comments (none posted)

php-mail: arbitrary file disclosure

Package(s):	php-mail	CVE #(s):
Created:	November 23, 2009	Updated:	November 24, 2009
Description:	From the VUPEN advisory: A vulnerability has been identified in Debian, which could be exploited by attackers to bypass security restrictions and gain knowledge of sensitive information. This issue is caused by an error in php-mail. The advisory refers to another related advisory, which has more information: A vulnerability has been identified in PEAR Mail, which could be exploited by attackers to bypass security restrictions and gain knowledge of sensitive information. This issue is caused by an input validation error in the "Mail/sendmail.php" script when passing the "form" parameter as an argument to sendmail, which could be exploited by attackers to inject arbitrary arguments and e.g. disclose or copy the contents of arbitrary files.
Alerts:	Debian DSA-1938-1 php-mail 2009-11-23

Comments (none posted)

qemu: arbitrary code execution

Package(s):	qemu	CVE #(s):	CVE-2009-3616
Created:	November 24, 2009	Updated:	February 15, 2010
Description:	From the CVE entry: Multiple use-after-free vulnerabilities in vnc.c in the VNC server in QEMU 0.10.6 and earlier might allow guest OS users to execute arbitrary code on the host OS by establishing a connection from a VNC client and then (1) disconnecting during data transfer, (2) sending a message using incorrect integer data types, or (3) using the Fuzzy Screen Mode protocol, related to double free vulnerabilities.
Alerts:	Pardus 2010-33 qemu 2010-02-14 SuSE SUSE-SR:2009:019 cups, jetty5, libqt4/dbus-1-qt, opera, puretls/jessie, kdegraphics3-pdf, qemu 2009-11-24

Comments (none posted)

snort: denial of service

Package(s):	snort	CVE #(s):	CVE-2009-3641
Created:	November 24, 2009	Updated:	November 24, 2009
Description:	From the CVE entry: Snort before 2.8.5.1, when the -v option is enabled, allows remote attackers to cause a denial of service (application crash) via a crafted IPv6 packet that uses the (1) TCP or (2) ICMP protocol.
Alerts:	Fedora FEDORA-2009-10751 snort 2009-10-27 Fedora FEDORA-2009-10783 snort 2009-10-27

Comments (none posted)

Kernel release status

The current 2.6 development kernel is 2.6.32-rc8, released on November 19. "The way things are going, this will likely be the last -rc. I wish we had more people looking at the regression list, but at some point I'm just going to have to say 'ok, enough is enough'." Details may be found in the full changelog.

There have been no stable kernel updates in the last week.

Comments (none posted)

Quotes of the week

-- Dan Williams

-- Frank Ch. Eigler

-- Matthew Garrett

Comments (9 posted)

LogFS returns

LogFS is a longstanding project by Jörn Engel to create a filesystem for contemporary solid-state storage devices; it was last covered here in May, 2007. Since then, LogFS has mostly disappeared from view. As of November 20, though, LogFS is back and, seemingly, ready for a mainline merge. Jörn says:

Sufficiently crazy users seem to be relatively scarce so far. But having more options for upcoming hardware can only be a good thing; it will be interesting to see what results come out as people start to play with this new filesystem.

Comments (5 posted)

Snapshot merge for the device mapper

Last week, LWN looked at the use of Btrfs snapshots to help system administrators recover from problematic upgrades. Btrfs is not the only snapshot mechanism in the kernel, though; the device mapper layer has had this capability for some time. What is missing from DM is the ability to restore the "origin" (main) device to an earlier state if need be. So the device mapper, in its current form, cannot be used to roll back an unfortunate upgrade without taking the system down and copying data.

That situation could change soon, possibly as early as 2.6.33. Mike Snitzer has posted patches for a snapshot-merge target for DM. This target, simply, merges a snapshot back to the origin device, restoring the state of that device to what it was when the snapshot was taken. So a system administrator could snapshot the device immediately prior to an upgrade, then get back to the pre-upgrade state if things do not go well.

One nice feature is that merging a snapshot preserves the state of all other snapshots on the device. So our system administrator could take another snapshot after the failed upgrade, before returning to the previous state. That post-upgrade snapshot would continue to exist, allowing the cherry-picking of any files with changes that should persist after the system as a whole is rolled back.

DM maintainer Alasdair Kergon has told your editor that he'll be reviewing this code shortly, and that it may find its way into linux-next in the near future.

Comments (6 posted)

Help wanted: kbuild maintainer

Sam Ravnborg, long-time maintainer of the kernel build (kbuild) subsystem, has announced his intention to step down from that role. "I have done this solely on a hobbyist basis and family (3 kids etc) + job require me so the kbuild maintainer job was becoming a duty and not that fun suddenly." It's not clear who the replacement will be. Thanks are due to Sam, who has left the state of kernel building far better than he found it.

Full Story (comments: 1)

Who wrote 2.6.32

As of this writing, the 2.6.32 appears poised for a release right around the beginning of December. That can only mean that the time has come to look at the code which has gone into this kernel and where it came from. It has been another active cycle, with a lot of changes making it into the mainline.

In particular, as of this writing (shortly after the 2.6.32-rc8 release), 2.6.32 is the result of 10,767 non-merge changesets sent in by 1,229 developers. These changes added a total of 1.17 million lines, while removing 611,000 lines, for a net growth of 559,000 lines of code. According to Rafael Wysocki's regression reports, this development cycle introduced a total of 86 regressions into the kernel - slightly fewer than we saw for 2.6.31. As of that posting, the number of unresolved regressions was shrinking quickly, with 25 of them still without a resolution.

So who added all those regressions lines of code? The statistics for this cycle look like this:

Most active 2.6.32 developers

By changesets Greg Kroah-Hartman 202 1.9% Johannes Berg 180 1.7% Bartlomiej Zolnierkiewicz 164 1.5% Mark Brown 154 1.4% Paul Mundt 139 1.3% Takashi Iwai 139 1.3% Alan Cox 129 1.2% Roel Kluin 115 1.1% Luis R. Rodriguez 105 1.0% Dan Williams 86 0.8% Tejun Heo 84 0.8% Herbert Xu 81 0.8% Peter Zijlstra 80 0.7% Ingo Molnar 77 0.7% Julia Lawall 77 0.7% Steven Rostedt 73 0.7% Magnus Damm 72 0.7% Joe Perches 71 0.7% Joerg Roedel 70 0.7%

By changed lines Greg Kroah-Hartman 174427 11.5% Bartlomiej Zolnierkiewicz 108056 7.1% Mauro Carvalho Chehab 62719 5.2% Jing Huang 49189 3.2% Forest Bond 45009 3.0% Ben Hutchings 37418 2.5% Eilon Greenstein 28008 1.8% Mark Brown 24516 1.6% Brian Swetland 22775 1.5% Hank Janssen 19681 1.3% Leo Chen 17458 1.2% Palash Bandyopadhyay 16790 1.1% Alan Cox 16466 1.1% Mithlesh Thukral 15173 1.0% Jerome Glisse 14343 0.9% Michael Chan 13415 0.9% Martyn Welch 12480 0.8% Iliyan Malchev 12172 0.8% Jesse Brandeburg 11051 0.7%

As has become traditional, Greg Kroah-Hartman and Bartlomiej Zolnierkiewicz feature at the top of both lists. Much of Greg's work had to do with the cleaning up of Microsoft's "hv" drivers. His state of mind during this process is best assessed from the commit messages, which tend to read like this one:

Greg also removed some drivers from the staging tree, shrinking the kernel by over 100,000 lines.

The bulk of Bartlomiej's work is also in the staging tree, and that is mostly concerned with fixing up a series of rather unloved wireless network drivers. These patches are somewhat controversial; the wireless developers would rather see that effort going into a different set of non-staging drivers. But those drivers are not yet ready for prime time, and, meanwhile, people are using the staging drivers. Wireless drivers were also the focus of Johannes Berg's work; he has made a long set of improvements to the mac80211 subsystem and its cfg80211 configuration interface. Mark Brown continues to contribute large amounts of code in support of Wolfson Micro's components, and Paul Mundt remains active as the Super-H maintainer.

In the "lines changed" column, Mauro Carvalho Chehab contributed a lot of patches as the Video4Linux2 maintainer. Jing Huang contributed the Brocade BFA FC SCSI driver, and Forest Bond added the VT6656 wireless driver to the staging tree.

Developers working on 2.6.32 were supported by (at least) 196 employers. The most active companies this time around are:

Most active 2.6.32 employers

By changesets (None) 1845 17.1% Red Hat 1028 9.5% (Unknown) 933 8.7% Intel 888 8.2% Novell 662 6.1% IBM 603 5.6% Oracle 319 3.0% Renesas Technology 264 2.5% AMD 251 2.3% Nokia 204 1.9% Fujitsu 201 1.9% Atheros Communications 197 1.8% (Consultant) 195 1.8% (Academia) 167 1.6% Texas Instruments 155 1.4% Wolfson Micro 153 1.4% Broadcom 149 1.4% HP 130 1.2% Analog Devices 124 1.2% Pengutronix 119 1.1%

By lines changed (None) 282017 18.6% Novell 256808 16.9% Red Hat 150781 9.9% Broadcom 84904 5.6% Intel 79267 5.2% (Unknown) 77122 5.1% Brocade 49189 3.2% Logic Supply 45165 3.0% Google 40936 2.7% IBM 29616 2.0% Wolfson Micro 25577 1.7% Texas Instruments 24824 1.6% Renesas Technology 24507 1.6% Nokia 24192 1.6% Microsoft 19696 1.3% Oracle 19410 1.3% (Consultant) 18774 1.2% Conexant 16790 1.1% LinSysSoft Technologies 15173 1.0% GE Fanuc 12495 0.8%

The sharp-eyed reader will notice that Red Hat has fallen below 10% of the total changes - the first time that has happened since the 2.6.21 development cycle in early 2007. The number of changes from Red Hat this time around is only slightly lower than the usual, though; what's happening is that some of the other companies are catching up.

There are a couple of other interesting entries here. Google takes a lot of grief for not contributing back, but that company was the source of a fair amount of code going into 2.6.32. Much of that was support for the HTC "Dream" (aka G1 or ADP1) phone platform, but Google also contributed to control groups, ext4, memory management, IPVS, and libata. And one may have never expected to see Microsoft show up on the list of top kernel contributors, but the hv drivers put it there for 2.6.32.

The numbers for signoffs have not changed much from previous cycles:

Top non-author signoffs in 2.6.32
Individuals David S. Miller 996 10.2% John W. Linville 994 10.2% Greg Kroah-Hartman 788 8.1% Andrew Morton 786 8.1% Ingo Molnar 501 5.1% Mauro Carvalho Chehab 398 4.1% James Bottomley 310 3.2% Len Brown 188 1.9% Paul Mundt 171 1.8% Russell King 165 1.7%	Employers Red Hat 3606 37.1% Novell 1309 13.5% Intel 906 9.3% Google 793 8.2% (None) 445 4.6% IBM 384 3.9% (Consultant) 274 2.8% Renesas Technology 180 1.9% Wolfson Micro 155 1.6% Oracle 138 1.4%

If anything, the subsystem maintainers are concentrating even more than before. Fully 2/3 of the patches going into the mainline kernel pass through the hands of developers working for just four companies.

At the 2009 Kernel Summit, the participants concluded that, while improvements can always be made, the process as a whole is working well. The picture that comes from these numbers suggests the same conclusion: the kernel development machine continues to absorb massive numbers of changes from a wide development community while continuing to produce stable, increasingly functional releases.

Comments (20 posted)

Journal-guided RAID resync

The RAID4, 5, and 6 storage technologies are designed to protect against the failure of a single drive. Blocks of data are spread out across the array and, for each stripe, there is a parity block stored on one of the drives. Should one drive fail, the lost data can be recovered through the use of the remaining drives and the parity information. This mechanism copes less well with system crashes and power failures, though, forcing software RAID administrators to choose between speed and reliability. A new mechanism called journal-guided resyncronization may make life easier, but only if it actually gets into the kernel.

The problem is that data and parity blocks must be updated in an atomic manner; if the two go out of sync, then the RAID array is no longer in a position to recover lost data. Indeed, it could return corrupted data. Expensive hardware RAID solutions use battery backup to ensure that updates are not interrupted partway through, but software RAID solutions often do not have that option. So if the system crashes - or the power fails - in the middle of an update to a RAID volume, that volume could end up being corrupted. Computer users, being a short-sighted kind of people in general, tend to regard this as a Bad Thing.

There are a couple of possible ways of mitigating this risk. One is to perform a full rescan of the RAID volume after a crash, fixing up any partially-updated stripes. The problem here is that (1) the correct fix for an inconsistent stripe may not always be clear, and (2) this process can take a long time. Long enough to cause users to think nostalgically about the days of fast, reliable floppy-disk storage.

An alternative approach is to introduce a type of journaling to the RAID layer. The RAID implementation can set aside some storage where it writes stripes (perhaps not the data, but, perhaps, just the numbers of the affected stripes) prior to changing the real array. This approach works, and it can recover a crashed RAID array without a full rescan, but there is a cost here too: that journaling can slow down the operation of the array significantly. Writes to the journal must be synchronous or it cannot be counted on to do its job, so write operations become far slower than they were before. Given that, it's not surprising that a lot of RAID administrators turn off RAID-level journaling and spend a lot of time hoping that nothing goes wrong.

A few years ago, Timothy E. Denehy, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau published a paper describing a better way, which they called "journal-guided resynchronization." Contemporary filesystems tend to do journaling of their own; why not use the filesystem journal to track changes to the RAID array as well? Running one journal can only be cheaper than running two - especially when one considers that the RAID journal must track, among other things, changes to the filesystem journal. The only problem is that the RAID and filesystem layers communicate through the relatively narrow block-layer API; using filesystem journaling to track RAID-level information has the potential to mix the layers considerably.

Jody McIntyre's journal-guided resync implementation adds a new "declared" mode to the ext3 filesystem. As the journal is being written, a new "declare block" is added describing exactly which blocks are to be written to the storage device. Those blocks are then written with a new BIO flag stating that the filesystem has taken responsibility for resynchronizing the stripe should something go wrong; that lets the storage layer forget about that particular problem. Should the system crash, the filesystem will find those declare blocks in the journal; it can then issue a (new) BIO_SYNCRAID operation asking the storage subsystem to resynchronize the specific stripes containing the listed blocks.

The result should be the best of both worlds. The cost of adding one more block to the filesystem journal is far less than doing that journaling at the RAID layer; Jody claims a 3-5% performance hit, as compared to 30% with the MD write-intent bitmap mechanism. But resynchronization after a crash should be quite fast, since it need only look at the parts of the array which were under active modification at the time. The only problem is that it requires the addition of specific support at the filesystem layer, so each filesystem must be modified separately. How this technique could be used in a filesystem which works without journaling (Btrfs comes to mind) would also have to be worked out.

There's one other little problem as well. This work was done at Sun as a way of improving performance with the Lustre filesystem. But Jody notes:

So this patch series has been abandoned for now. It seems like this functionality should be useful to software RAID users, so, hopefully, somebody will pick them up and carry them forward. In the absence of a new developer, software RAID administrators will continue to face an unhappy choice well into the future.

Comments (22 posted)

Videobuf: buffer management for V4L2 drivers

Video4Linux2 (V4L2) drivers provide access to webcams, TV tuners, and TV output devices, among others. LWN covered much of the V4L2 API in 2007; sadly, like almost any two-year-old kernel documentation, those articles are now somewhat obsolete. One thing that has not changed, though, is that V4L2 drivers tend to be moderately complex beasts; they are usually an assembly of two or three drivers working together to operate hardware with a number of complex operating modes. Despite all that, a V4L2 driver has, at its core, a relatively simple task: fill large buffers in memory with video frames and transfer them between the device and user space. The management of these buffers, while subject to complexities of its own, tends to be quite similar from one driver to the next. It would be nice if there were a support layer which could be used to handle much of this task in a standard way.

The good news is that such a layer does exist; it's called videobuf. The bad news is that the documentation for this code is...not quite what it could be. This article is an attempt to fill that gap; a version of it will eventually be submitted for inclusion into the kernel documentation directory.

The videobuf layer functions as a sort of glue layer between a V4L2 driver and user space. It handles the allocation and management of buffers for the storage of video frames. There is a set of functions which can be used to implement many of the standard POSIX I/O system calls, including read(), poll(), and, happily, mmap(). Another set of functions can be used to implement the bulk of the V4L2 ioctl() calls related to streaming I/O, including buffer allocation, queueing and dequeueing, and streaming control. Using videobuf imposes a few design decisions on the driver author, but the payback comes in the form of reduced code in the driver and a consistent implementation of the V4L2 user-space API.

Buffer types

Not all video devices use the same kind of buffers. In fact, there are (at least) three common variations:

• Buffers which are scattered in both the physical and (kernel) virtual address spaces. All user-space buffers are like this, but it makes great sense to allocate kernel-space buffers this way as well when it is possible. Unfortunately, it is not always possible; working with this kind of buffer normally requires hardware which can do scatter/gather DMA operations.

• Buffers which are physically scattered, but which are virtually contiguous; buffers allocated with vmalloc(), in other words. These buffers are just as hard to use for DMA operations, but they can be useful in situations where DMA is not available but virtually-contiguous buffers are convenient.

• Buffers which are physically contiguous. Allocation of this kind of buffer can be unreliable on fragmented systems, but simpler DMA controllers cannot deal with anything else.

Videobuf can work with all three types of buffers, but the driver author must pick one at the outset and design the driver around that decision.

Data structures, callbacks, and initialization

Depending on which type of buffers are being used, the driver should include one of the following files:

    <media/videobuf-dma-sg.h>
    <media/videobuf-vmalloc.h>
    <media/videobuf-dma-contig.h>

The driver's data structure describing a V4L2 device should include a struct videobuf_queue instance for the management of the buffer queue, along with a list_head for the queue of available buffers. There will also need to be an interrupt-safe spinlock which is used to protect (at least) the queue.

The next step is to write four simple callbacks to help videobuf deal with the management of buffers:

    struct videobuf_queue_ops {
	int (*buf_setup)(struct videobuf_queue *q,
			 unsigned int *count, unsigned int *size);
	int (*buf_prepare)(struct videobuf_queue *q,
			   struct videobuf_buffer *vb,
			   enum v4l2_field field);
	void (*buf_queue)(struct videobuf_queue *q,
			  struct videobuf_buffer *vb);
	void (*buf_release)(struct videobuf_queue *q,
			    struct videobuf_buffer *vb);
    };

buf_setup() is called early in the I/O process, when streaming is being initiated; its purpose is to tell videobuf about the I/O stream. The count parameter will be a suggested number of buffers to use; the driver should check it for rationality and adjust it if need be. As a practical rule, a minimum of two buffers are needed for proper streaming, and there is usually a maximum (which cannot exceed 32) which makes sense for each device. The size parameter should be set to the expected (maximum) size for each frame of data.

Each buffer (in the form of a struct videobuf_buffer pointer) will be passed to buf_prepare(), which should set the buffer's size, width, height, and field fields properly. If the buffer's state field is VIDEOBUF_NEEDS_INIT, the driver should pass it to:

    int videobuf_iolock(struct videobuf_queue* q, struct videobuf_buffer *vb,
			struct v4l2_framebuffer *fbuf);

Among other things, this call will usually allocate memory for the buffer. Finally, the buf_prepare() function should set the buffer's state to VIDEOBUF_PREPARED.

When a buffer is queued for I/O, it is passed to buf_queue(), which should put it onto the driver's list of available buffers and set its state to VIDEOBUF_QUEUED. Note that this function is called with the queue spinlock held; if it tries to acquire it as well things will come to a screeching halt. Yes, this is the voice of experience. Note also that videobuf may wait on the first buffer in the queue; placing other buffers in front of it could again gum up the works. So use list_add_tail() to enqueue buffers.

Finally, buf_release() is called when a buffer is no longer intended to be used. The driver should ensure that there is no I/O active on the buffer, then pass it to the appropriate free routine(s):

    /* Scatter/gather drivers */
    int videobuf_dma_unmap(struct videobuf_queue *q,
    			   struct videobuf_dmabuf *dma);
    int videobuf_dma_free(struct videobuf_dmabuf *dma);

    /* vmalloc drivers */
    void videobuf_vmalloc_free (struct videobuf_buffer *buf);

    /* Contiguous drivers */
    void videobuf_dma_contig_free(struct videobuf_queue *q,
			          struct videobuf_buffer *buf);

One way to ensure that a buffer is no longer under I/O is to pass it to:

    int videobuf_waiton(struct videobuf_buffer *vb, int non_blocking, int intr);

Here, vb is the buffer, non_blocking indicates whether non-blocking I/O should be used (it should be zero in the buf_release() case), and intr controls whether an interruptible wait is used.

File operations

At this point, much of the work is done; much of the rest is slipping videobuf calls into the implementation of the other driver callbacks. The first step is in the open() function, which must initialize the videobuf queue. The function to use depends on the type of buffer used:

    void videobuf_queue_sg_init(struct videobuf_queue *q,
			        struct videobuf_queue_ops *ops,
			 	struct device *dev,
			 	spinlock_t *irqlock,
			 	enum v4l2_buf_type type,
			 	enum v4l2_field field,
			 	unsigned int msize,
			 	void *priv);

    void videobuf_queue_vmalloc_init(struct videobuf_queue *q,
			        struct videobuf_queue_ops *ops,
			 	void *dev,
			 	spinlock_t *irqlock,
			 	enum v4l2_buf_type type,
			 	enum v4l2_field field,
			 	unsigned int msize,
				void *priv);

    void videobuf_queue_dma_contig_init(struct videobuf_queue *q,
				       struct videobuf_queue_ops *ops,
				       struct device *dev,
				       spinlock_t *irqlock,
				       enum v4l2_buf_type type,
				       enum v4l2_field field,
				       unsigned int msize,
				       void *priv);

In each case, the parameters are the same: q is the queue structure for the device, ops is the set of callbacks as described above, dev is the device structure for this video device, irqlock is an interrupt-safe spinlock to protect access to the data structures, type is the buffer type used by the device (cameras will use V4L2_BUF_TYPE_VIDEO_CAPTURE, for example), field describes which field is being captured (often V4L2_FIELD_NONE for progressive devices), msize is the size of any containing structure used around struct videobuf_buffer, and priv is a private data pointer which shows up in the priv_data field of struct videobuf_queue. Note that these are void functions which, evidently, are immune to failure.

The void *dev typing in videobuf_queue_vmalloc_init() is a bit of an anomaly; your editor has submitted a patch to change it to struct device *. The ops pointer also should really be const; that will probably change in 2.6.33.

V4L2 capture drivers can be written to support either of two APIs: the read() system call and the rather more complicated streaming mechanism. As a general rule, it is necessary to support both to ensure that all applications have a chance of working with the device. Videobuf makes it easy to do that with the same code. To implement read(), the driver need only make a call to one of:

    ssize_t videobuf_read_one(struct videobuf_queue *q,
			      char __user *data, size_t count, 
			      loff_t *ppos, int nonblocking);

    ssize_t videobuf_read_stream(struct videobuf_queue *q,
			         char __user *data, size_t count, 
				 loff_t *ppos, int vbihack, int nonblocking);

Either one of these functions will read frame data into data, returning the amount actually read; the difference is that videobuf_read_one() will only read a single frame, while videobuf_read_stream() will read multiple frames if they are needed to satisfy the count requested by the application. A typical driver read() implementation will start the capture engine, call one of the above functions, then stop the engine before returning (though a smarter implementation might leave the engine running for a little while in anticipation of another read() call happening in the near future).

The poll() function can usually be implemented with a direct call to:

    unsigned int videobuf_poll_stream(struct file *file,
				      struct videobuf_queue *q,
				      poll_table *wait);

Note that the actual wait queue eventually used will be the one associated with the first available buffer.

When streaming I/O is done to kernel-space buffers, the driver must support the mmap() system call to enable user space to access the data. In many V4L2 drivers, the often-complex mmap() implementation simplifies to a single call to:

    int videobuf_mmap_mapper(struct videobuf_queue *q,
			     struct vm_area_struct *vma);

Everything else is handled by the videobuf code.

The release() function requires two separate videobuf calls:

    void videobuf_stop(struct videobuf_queue *q);
    int videobuf_mmap_free(struct videobuf_queue *q);

The call to videobuf_stop() terminates any I/O in progress - though it is still up to the driver to stop the capture engine. The call to videobuf_mmap_free() will ensure that all buffers have been unmapped; if so, they will all be passed to the buf_release() callback. If buffers remain mapped, videobuf_mmap_free() returns an error code instead. The purpose is clearly to cause the closing of the file descriptor to fail if buffers are still mapped, but every driver in the 2.6.32 kernel cheerfully ignores its return value.

ioctl() operations

The V4L2 API includes a very long list of driver callbacks to respond to the many ioctl() commands made available to user space. A number of these - those associated with streaming I/O - turn almost directly into videobuf calls. The relevant helper functions are:

    int videobuf_reqbufs(struct videobuf_queue *q,
		         struct v4l2_requestbuffers *req);
    int videobuf_querybuf(struct videobuf_queue *q, struct v4l2_buffer *b);
    int videobuf_qbuf(struct videobuf_queue *q, struct v4l2_buffer *b);
    int videobuf_dqbuf(struct videobuf_queue *q, struct v4l2_buffer *b, 
                       int nonblocking);
    int videobuf_streamon(struct videobuf_queue *q);
    int videobuf_streamoff(struct videobuf_queue *q);
    int videobuf_cgmbuf(struct videobuf_queue *q, struct video_mbuf *mbuf, 
    			int count);

So, for example, a VIDIOC_REQBUFS call turns into a call to the driver's vidioc_reqbufs() callback which, in turn, usually only needs to locate the proper struct videobuf_queue pointer and pass it to videobuf_reqbufs(). These support functions can replace a great deal of buffer management boilerplate in a lot of V4L2 drivers.

The vidioc_streamon() and vidioc_streamoff() functions will be a bit more complex, of course, since they will also need to deal with starting and stopping the capture engine. videobuf_cgmbuf(), called from the driver's vidiocgmbuf() function, only exists if the V4L1 compatibility module has been selected with CONFIG_VIDEO_V4L1_COMPAT, so its use must be surrounded with #ifdef directives.

Buffer allocation

Thus far, we have talked about buffers, but have not looked at how they are allocated. The scatter/gather case is the most complex on this front. For allocation, the driver can leave buffer allocation entirely up to the videobuf layer; in this case, buffers will be allocated as anonymous user-space pages and will be very scattered indeed. If the application is using user-space buffers, no allocation is needed; the videobuf layer will take care of calling get_user_pages() and filling in the scatterlist array.

If the driver needs to do its own memory allocation, it should be done in the vidioc_reqbufs() function, after calling videobuf_reqbufs(). The first step is a call to:

    struct videobuf_dmabuf *videobuf_to_dma(struct videobuf_buffer *buf);

The returned videobuf_dmabuf structure (defined in <media/videobuf-dma-sg.h>) includes a couple of relevant fields:

    struct scatterlist  *sglist;
    int                 sglen;

The driver must allocate an appropriately-sized scatterlist array and populate it with pointers to the pieces of the allocated buffer; sglen should be set to the length of the array.

Drivers using the vmalloc() method need not (and cannot) concern themselves with buffer allocation at all; videobuf will handle those details. The same is true of contiguous-DMA drivers; videobuf will allocate the buffers (with dma_alloc_coherent()) when it sees fit. That means that these drivers may be trying to do high-order allocations at any time, an operation which is not always guaranteed to work. Some drivers play tricks by allocating DMA space at system boot time; videobuf does not currently play well with those drivers.

Filling the buffers

The final part of a videobuf implementation has no direct callback - its the portion of the code which actually puts frame data into the buffers, usually in response to interrupts from the device. For all types of drivers, this process works approximately as follows:

1. Obtain the next available buffer and make sure that somebody is actually waiting for it.

2. Get a pointer to the memory and put video data there.

3. Mark the buffer as done and wake up the process waiting for it.

Step (1) above is done by looking at the driver-managed list_head structure - the one which is filled in the buf_queue() callback. Because starting the engine and enqueueing buffers are done in separate steps, it's possible for the engine to be running without any buffers available - in the vmalloc() case especially. So the driver should be prepared for the list to be empty. It is equally possible that nobody is yet interested in the buffer; the driver should not remove it from the list or fill it until a process is waiting on it. That test can be done by examining the buffer's done field (a wait_queue_head_t structure) with waitqueue_active().

For scatter/gather drivers, the needed memory pointers will be found in the scatterlist structure described above. Drivers using the vmalloc() method can get a memory pointer with:

    void *videobuf_to_vmalloc(struct videobuf_buffer *buf);

For contiguous DMA drivers, the function to use is:

    dma_addr_t videobuf_to_dma_contig(struct videobuf_buffer *buf);

The contiguous DMA API goes out of its way to hide the kernel-space address of the DMA buffer from drivers.

The final step is to set the size field of the relevant videobuf_buffer structure to the actual size of the captured image, set state to VIDEOBUF_DONE, then call wake_up() on the done queue. At this point, the buffer is owned by the videobuf layer and the driver should not touch it again.

Conclusion

This article has covered most aspects of the videobuf API. Developers who are interested in more information can go into the relevant header files; there are a few low-level functions declared there which have not been talked about here. Also worthwhile is the vivi driver (drivers/media/video/vivi.c), which is maintained as an example of how V4L2 drivers should be written. Vivi only uses the vmalloc() API, but it's good enough to get started with. Note also that all of these calls are exported GPL-only, so they will not be available to non-GPL kernel modules.

Comments (2 posted)

Fedora 12: a conversation with Paul Frields

Last week, just before the final release of Fedora 12 was announced, I had the opportunity to speak with Fedora Project Leader Paul Frields. The following article is based on that conversation.

Although F12 had a slightly shorter release cycle than F11, there are more new and improved features in this release. Support has been added for many models of wireless and video cards. Paul was particularly excited about the newly-added support for bluetooth tethering, which allows him to connect his laptop to the internet through his mobile phone.

Of course there are also new desktop productivity applications, and new artwork for this release. Thanks to the Fit and Finish project there are clearer menus, and tooltips and notifications are better positioned and easier to read. PackageKit, a system designed to make installing and updating software on your computer easier, can be used with shell commands or GUIs. It uses PolicyKit for fine-grained permissions.

For Java programmers there's NetBeans 6.7 and SystemTap users will be happy to see it integrated with Eclipse. System administrators and software testers should appreciate the virtualization features. These include the kernel shared memory (KSM) feature, which allows better performance for more guests using less RAM. The introduction of libguestfs and the interactive tool guestfish allow easy access to virtual machine disk images.

We spoke briefly about the What is Fedora? debate and Fedora's target audience. Paul was looking forward to discussing these and other topics at the upcoming FUDCon in Toronto, December 5-7, 2009. Paul went on to say that Fedora already has thousands of contributors, but they are always interested in encouraging more people to contribute. Of course there are many ways to contribute. Developers, artists, translators, bug reporters, wiki editors, etc. should all feel comfortable in contributing to the Fedora Project. The Fedora Project seeks to cultivate a community of contributors with open ideas. By designing a better system for contributors they hope to create a better system for consumers as well.

We also talked about the future of rawhide. Jesse Keating has proposed a significant change to rawhide, which is Fedora's development branch. "So my plans are really funny. I plan to make rawhide more unstable more of the time, and I plan to make "rawhide" more stable more of the time. Crazy eh? How can I do this? By splitting "rawhide" in two."

In this proposal rawhide will not be an installable tree. Instead it will become a repository of developmental and experimental packages. People who want the latest versions of their favorite software can install all the packages they want from rawhide, but the base system will be more stable. On the other hand, there will also be a "pending release" installable tree. The installable tree will spawn F13 alpha, beta, and any snapshots that are released for testing until the final F13 release. Meanwhile developers can work on bringing the latest features to the rawhide versions of these packages, without stopping for feature freezes. There is more information about this proposal on the wiki. There was also a recent meeting about this proposal, and this meeting summary provides an overview with a look at some of the obstacles.

Fedora elections are coming up, the nomination period is over and candidates have started campaigning. Seats are open on the Fedora Advisory Board, FESCo (Fedora Engineering Steering Committee), and FAmSCo (Fedora Ambassadors Steering Committee). A short list of release names for Fedora 13 is out now, and voting will begin soon.

Fedora is free software and it is built using free software. Paul likes to think of it as "drinking your own champagne". Fedora 12 is available now, along with several official spins. Download Fedora 12 from a mirror near you, and check out the new features for yourself.

Comments (2 posted)

Chromium OS source released

Google has posted some information about Chromium OS, along with the current source. "First, it's all about the web. All apps are web apps. The entire experience takes place within the browser and there are no conventional desktop applications. This means users do not have to deal with installing, managing and updating programs." See the Chromium OS page for more information.

Comments (85 posted)

KNOPPIX 6.2 / ADRIANE 1.2

Klaus Knopper has announced the release of KNOPPIX 6.2 and ADRIANE (Audio Desktop Reference Implementation And Networking Environment) 1.2. The release notes contain more information. "The current version 6.2 has been completely updated from Debian/"Lenny", "Testing" and "Unstable", and uses Kernel 2.6.31.6, xorg 7.4 for supporting currentmost computer hardware."

Comments (none posted)

Owl moves to 2.6 kernels, integrates OpenVZ; new ISOs

Openwall GNU/Linux has released fresh ISO images of Owl-current for x86 and x86-64. "These ISOs represent a major development milestone. We have replaced the default kernel with a 2.6 OpenVZ one (featuring optional container-based virtualization), we've integrated OpenVZ tools (vzctl and vzquota packages needed to create, control, examine, and/or destroy OpenVZ containers), and we've dropped support for Linux 2.4 kernels (although they're still supported in the maintained Owl 2.0-stable branch - until our next release)."

Full Story (comments: none)

SimplyMEPIS 8.5 alpha Release

Warren Woodford has announced the release of SimplyMEPIS 8.4.80, the alpha release of MEPIS 8.5. MEPIS is based on Debian Stable source with some newer packages. Warren said: ""In this case, a lot of users have asked that we support the KDE 4.3 desktop, yet other users have asked that we continue to support KDE 3.5.10. So in the spirit of our ongoing updates, we are building 8.5, not as a replacement for 8.0, but rather as an alternative for those users who want KDE 4.3. We will continue to support 8.0 with KDE 3.5.10 in parallel with 8.5 and KDE 4.3.""

Comments (none posted)

New source package formats now available

The Debian dpkg maintainers introduce new source package formats, "3.0 (quilt)" and "3.0 (native)". "The last step for us (dpkg maintainers) in this project is to change dpkg-source to use those new formats by default. However, before we do this we want to ensure that no packages (in sid) will be broken due to this switch and there are quite a few packages left to fix:"

Full Story (comments: 1)

Fedora 12 lets unprivileged users install packages

Fedora bug #534047 contains an interesting Fedora 12 surprise: "PackageKit allows you to install signed content from signed repositories without a password by default. It only asks you to authenticate if anything is unsigned or the signatures are wrong." So any user can install any package found in the official repository. Some Fedora developers, at least, seem to see this as a feature; see this rapidly-growing thread for the discussion.

The bug report contains the incantation needed to disable this behavior:

    pklalockdown --lockdown org.freedesktop.packagekit.package-install

Evidently that is not a long-term solution, though; see this post for a rather more involved fix. Stay tuned: we'll probably post a longer look at this issue in the near future.

Comments (109 posted)

Fedora 12 updates PackageKit

Fedora has released an update to PackageKit to "switch the signed install permission to require the root password". This is the fix for the change that allowed unprivileged package installation, which led to an uproar from some Fedora users and developers. Note that it is a regular update, not a security update. Click below for the update information.

Full Story (comments: 13)

Cooperative Bug Isolation for Fedora 12

The Cooperative Bug Isolation project has been made available for Fedora 12. "CBI is an ongoing research effort to find and fix bugs in the real world. We distribute specially modified versions of popular open source software packages. These special versions monitor their own behavior while they run, and report back how they work (or how they fail to work) in the hands of real users like you. Even if you've never written a line of code in your life, you can help make things better for everyone simply by using our special bug-hunting packages. We currently offer instrumented versions of Evolution, The GIMP, GNOME Panel, Gnumeric, Nautilus, Pidgin, Rhythmbox, and SPIM."

Full Story (comments: 4)

Fedora Board Recap

Click below for a recap of the November 19, 2009 meeting of the Fedora Advisory Board. Topics include Trademark licenses, an update proposal, FUDCon session setup, F12 release, FreeMedia, F13 naming, and more.

Full Story (comments: none)

Nouveau To Enter The Ubuntu 10.04 LTS Kernel (Phoronix)

Phoronix is reporting that Ubuntu 10.04 will ship with the Nouveau driver for NVIDIA graphics hardware. Nouveau is a reverse-engineered driver that is released as free software—unlike the NVIDIA-provided drivers. "With Ubuntu 9.04 it became easier to use the Nouveau driver with Ubuntu Linux as a snapshot of its DDX driver (xf86-video-nouveau) and its kernel DRM code were packaged up and made available through its package repositories, which continues to be the case in Ubuntu 9.10. Though after the Ubuntu Developer Summit last week for 10.04 'Lucid Lynx', Canonical is now going to put Nouveau into the kernel by default. Not too many details beyond that or their intentions are known at this time, but Nouveau developers are currently being asked about the matter."

Comments (6 posted)

Retirement of Ubuntu's lpia architecture

Ubuntu will not support lpia architecture in 10.04 (Lucid Lynx) and beyond. Existing lpia users on stable versions of Ubuntu will be supported through the support period, but upgrades are not recommended. "Recent announcements by Intel indicate that processors supported by the lpia architecture are expected to remain compatible with the IA32 instruction set, so there is a reduced expectation of the need to retain a separate architecture to address future requirements. As of 9.10, lpia and i386 share the kernel configuration, reducing most of the benefits potentially available in previous Ubuntu releases."

Full Story (comments: none)

DistroWatch Weekly, Issue 330

The DistroWatch Weekly for November 23, 2009 is out. "Chromium OS, Google's long-awaited entry into the operating system market, finally made a tentative appearance last week. Although it created much excitement on many technology news sites, the new Linux-based distribution doesn't target general desktop computers; instead it attempts to kick-start a new era of cloud computing on netbooks and other portable devices. Fedora 12, the latest version of the popular distribution and the other big topic of discussion last week, has been hit by an unprivileged package installation controversy, while its unofficial LXDE edition has been withdrawn due to a nasty bug. But other than these two issues, the new release looks great - read our first-look review to find out more. In other news, Ubuntu removes GIMP from default installation, FreeBSD prepares for the imminent arrival of 8.0-RELEASE, openSUSE re-evaluates its decision to remove the DHT technology from the Transmission BitTorrent client, and founders of Qimo 4 Kids, an Ubuntu-based distribution for children, discuss the beginnings of their project and the motivation behind using open source software to start a charity. Also in the news section we link to interviews with Fedora's Paul Frields and Mandriva's Buchan Milne. Finally, this week's Questions and Answers column considers the pros and cons of rolling versus time-based distribution release models. Happy reading!"

Comments (none posted)

Fedora Weekly News 203

The Fedora Weekly News for November 22, 2009 is out. "Fedora 12 "Constantine" was released this past week, and we kick off this week's issue with a sampling of reviews from around the globe. Also in announcements, details on a change in Fedora 12's PackageKit permissions. In news from the Fedora Planet, some details on what's involved with providing delta RPMs, a new feature in Fedora 12, a site visit to the new Red Hat Computing Lab at Carnegie Mellon, and much more from Fedora contributors. Quality Assurance brings us up to date with the recent weekly meetings of the QA team which have focused on F12, with lots of interesting detail behind the scenes! In Translation news, details on updates and errata for Fedora 12 release notes, and a couple translation requests from SSSD and Midnight Commander. Security Advisories keeps us current with security patches for Fedora 10, 11, and 12. In news from the world of Fedora virtualization, coverage of a recent interview with virtualization luminaries, a status report on Fedora virtualization and details on the latest version of libvirt. Enjoy FWN 203!"

Full Story (comments: none)

The Mint Newsletter - issue 97

This issue of the Mint Newsletter covers the first release candidate of Linux Mint 8 "Helena", and several other topics.

Comments (none posted)

OpenSUSE Weekly News/98

This issue of the OpenSUSE Weekly News covers Board Election 2009 Startup, openSUSE 11.2 Launchparties, openSUSE Spotlight: What does the openSUSE Board do?, Uwe Gansert: Interactive AutoYaST Rules, openSUSE Forums: K3B and mp3 support, and more.

Comments (none posted)

Ubuntu Weekly Newsletter #169

The Ubuntu Weekly Newsletter for November 21, 2009 is out. "In this issue we cover: Lucid Ubuntu Developer Summit Videos, New LoCo Council Members, America's Membership Board Meeting: November 18th, 2009, Developer Membership Board public meeting, LoCo Contact Change: Wisconsin LoCo Team, Doctor Mo: Ice Skating at UDS, Matthew Helmke: Heading Home from UDS-L, Joe Baker: An Interview with Richard Johnson (nixternal), Martin Pitt: Nicer Launchpad upstream releases with lp-project-upload, and much, much more!"

Full Story (comments: none)

Interview: Red Hat on Red Hat Enterprise Virtualization (Montana Linux)

Scott Dowdle interviews Red Hat's Andy Cathrow and Jim Brennan about the company's latest Enterprise Virtualization technology. "ML: In the year between the merger with Qumranet and the release of RHEV for Servers, what were the primary changes made to the product? AC: We made many, significant changes. A quick, but not complete list includes: * SAN support - including iSCSI and Fiber channel (previously NFS only) * Multipath I/O * NIC bonding (host) * Multiple nics (guest) * VLANs * High availability * System scheduler (distribution policies, scheduling VMS) * Power Saver * Support for large hosts 96 cores, 1TB RAM * Support for large guests 16 cores, 256GB RAM * Support for managing hosts - including configuration files and software updates".

Comments (8 posted)

Czajkowski: Qimo 4 kids - NGO Interview

Laura Czajkowski has an interview with Michael and Michelle Hall of the Qimo 4 kids project. "QuinnCo is a very small not for profit located in Central Florida. In fact, it's just the two of us, Michelle and I, operating out of our house and garage. We take in second hand computers, fix them up if they aren't working, then put Ubuntu or Qimo on them and give them out to kids and families in need. We given out approx. 50 computers this year."

Comments (none posted)

digiKam approaches 1.0

Digital photographs can be something of a pain. With the storage capacities available today, it is easy to take thousands of pictures, with no regard for the cost. With film cameras, there was an incremental cost for each shot taken and each print made, which tended to reduce—but not eliminate—the problem of organizing a photo collection. With digital photos, though, there are programs like digiKam that can assist in this task. As digiKam approaches its 1.0 release it seems like a good time to see what it can do.

When first starting digiKam (1.0-beta5 from Fedora Rawhide), one is faced with the "First Run Assistant" that allows the user to make some choices on settings for the program. Earlier versions (0.10.0 on Fedora 10) seem to want to index the entire disk or something—perhaps from an errant setting—when they start, leading the user to believe that digiKam has crashed or exited, so the assistant is a much better welcome. Unfortunately, it asks too many questions and, more importantly, several that a new user is unlikely to have a good answer for. Taking the defaults is a reasonable option, but also seems unnecessary; asking for a storage directory and pointing users at the configuration menu item would seem enough to get started.

The program itself has two main sections, the left hand side has photo albums, searches, calendar view, map search view, etc. based on which tool is selected, and the right hand side shows the results of the operation. The results show thumbnails of the images with information on tags, ratings, and creation date. Hovering over an image or thumbnail brings up a box with much more information including EXIF data from the file, image dimensions, and filename.

That's all fairly standard fare for photo organization programs, at least to this untrained observer. The tagging, rating, and searching make things much more interesting. Tags can be applied to photos to characterize them in some way, and photos in multiple albums can carry the same tags. So if one had photos of monkeys from Costa Rica in one album and strange animals and insects seen at home in another, tagging them all with "animal" makes finding them all quite simple. A search of that nature can then be saved and recalled as needed.

Ratings allow the user to apply up to five stars to photos, based on their quality or subject. Advanced searches can then use the ratings as a criteria in the search, allowing for searches like "find all the five star animal pictures". The calendar view (shown at right) shows photos based on when they were taken, which is a nice way to organize pictures from multiple sources of the same trip or event for example. For images tagged with their location, the map searching could be used, though none of the author's pictures were tagged that way (yet, anyway). The map search seems to incorporate the Marble widget for use in selecting geographic regions.

One of the first steps when using a photo organizer is to get some photos into the system. Importing from an Android ADP1 (treated as an external USB device) did not go very well, as digiKam crashed while rooting through the SD card. It seemed unhappy with a Bill Monroe mp3 file, but it wasn't clear why it might be looking at such a thing. In any case, manually moving those images over to a local directory and pointing digiKam at that worked fine. Normally, I would have pointed it at several thousand images on a USB drive, but, the fates conspired to have two identical terabyte drives containing the photos (and a vast quantity of FLACs) stop showing up on the USB bus. Presumably just a temporary glitch, but not one to try to track down under deadline pressure.

But digiKam is not just about organizing photos, it is also targeted at those who want to manipulate the images in various ways. Even the most basic user will want to rotate images or do red-eye removal occasionally and those are, of course, supported, but digiKam goes far beyond that. There is a whole raft of corrections that can be applied to photos in the image editor. The digiKam web site lists various kinds of image processing that can be done, including color management, noise reduction, working with camera raw file formats, and so on.

digiKam also comes with a standalone photo editor, ShowFoto that has all of the same editing capabilities, but does not have the album management and searching that come with digiKam. In addition, digiKam uses the KDE Image Plugin Interface (KIPI), so that KIPI-Plugins can be used to export the digiKam data in a wide variety of formats. KIPI-Plugins exist for various web photo services (Flickr, Picasa, etc.) as well as social networking sites like Facebook.

Exporting an album (or the results of a search) to HTML is also possible for those that want to set up their own simple photo web site. There are multiple theme choices, and the resulting web site is functional but basic—just fine for those who would rather keep their photos on their own site. Exporting to personal photo web site programs, like Gallery, is supported as well.

The author has few real complaints about working with digiKam 1.0, it seems like a fairly solid program with lots of interesting potential. There was some confusion about working with albums and adding new directories of images, but that should be easily overcome by working with it more—something that is very likely to happen. Once those thousands of images are extracted from the recalcitrant USB drives, digiKam seems like the right program to use to organize them. Certainly far better than the ad hoc "organization" there is today.

It probably makes a great deal of sense to photographers, but the most serious complaint I have about digiKam (and especially ShowFoto) is the lack of support for PNG and GIF images. Rather often, manipulating both JPEGs and PNGs is one of the tasks required for putting together a weekly edition. Doing that in one tool would be useful, which is why I use the GIMP for those simple tweaks. But, the tagging and other features available in digiKam could certainly be used for many kinds of graphic images. Perhaps it makes photographers cringe, but it would be valuable to some of the rest of us. [Update: as pointed out by a reader below, this paragraph is entirely bogus and was the result of pilot error. ]

There is lots of documentation that comes with digiKam (in the digikam-doc package, at least for Fedora), including the 300+ page digiKam Handbook [PDF]. If just using it more doesn't answer the album/directory questions, one would guess that the handbook will. A release candidate is due at the end of November, with the final release of 1.0 scheduled for December 20. Based on the beta, it will be an excellent release, and I look forward to using it. Perhaps in that quiet week at the end of the year.

Comments (7 posted)

JACK 1.9.4 released

Version 1.9.4 of the JACK Audio Connection Kit has been announced. "Continuing the JACK2 serie[s]: Jack 1.9.4 is API synched with JACK 0.118.0. Fix a lot of more or less important bugs, especially on OSX with much better support off CoreAudio devices (input/output devices "internally" aggregated, hog mode...etc...)."

Comments (none posted)

mpd 0.15.6 released

Version 0.15.6 of MPD, a server-side application for playing music, has been announced. "This release fixes OggFLAC, some annoyances and a few critical bugs."

Comments (none posted)

PulseAudio 0.9.21 released

Version 0.9.21 of the PulseAudio sound server has been announced. The Change Log states: "This is mostly a bugfix release, and merges Colin Guthrie's device manager module, which should probably be considered experimental at this time and whose API is not stable yet. This will mostly be used by the KDE integration but might be useful elsewhere, too."

Comments (none posted)

announcing Caribou: python migrations for sqlite databases

The Caribou project has been launched. "Caribou is a simple SQLite database migrations library for Python, built primarily to manage the evolut[i]on of client side databases over multiple releases of an application."

Full Story (comments: none)

MySQL Community Server 5.0.88 has been released

MySQL Community Server 5.0.88 has been released. This release includes a security fix along with other bug fixes. "Security Fix: MySQL clients linked against OpenSSL did not check server certificates presented by a server linked against yaSSL."

Full Story (comments: none)

PostgreSQL Weekly News

The November 22, 2009 edition of the PostgreSQL Weekly News is online with the latest PostgreSQL DBMS articles and resources.

Full Story (comments: none)

Exim 4.71 released

Version 4.71 of the Exim mail transfer agent has been announced. "This release is a pure bug fix release over version 4.70."

Full Story (comments: none)

When Designing for Moblin, Think Like a Mobile User (Moblin Zone)

Moblin Zone has a two-part series on user interface design for Moblin (part 1) and (part 2). The article looks at user attention span, limited screen real estate, limited input methods, and so on, that characterize a mobile device, with specific advice on how applications should work based on those constraints. "Consider the warning that you might create if your live application loses its network application. Don’t say 'Cable [Unplugged],' and then 'Lost IP Address,' and then 'Lost Connection to the Internet' and then 'Connection Recovered' and then 'IP Address Acquired'... you get the idea. Tell the user only what he/she truly needs to know. Don't forget that the user might not be looking at the screen at any particular moment."

Comments (12 posted)

lighttpd 1.4.25 released

Version 1.4.25 of lighttpd, a light-weight web server, has been announced. "We did some important bug fixes (some of them new since 1.4.24, and some older bugs). Only 2 small new features: traceback for lua errors and the SSL_CLIENT_* vars export for ssl client cert validation."

Comments (none posted)

ControlTier 3.4.9 released

Version 3.4.9 of the ControlTier business management framework has been announced. "This release has quite a few bug fixes and improvements. It is also the first ControlTier release distributed in RPM packages."

Full Story (comments: none)

GNOME 2.29.2 released

Version 2.29.2 of GNOME has been announced. "So, here's the second development release of GNOME 2.29/2.30 development cycle. I was a bit lazy so this release is a couple hours late. But we're on time for certain timezones so I guess this is ok :-P This release includes the accepted modules proposed for 2.30."

Full Story (comments: none)

GNOME Software Announcements

The following new GNOME software has been announced this week:

• couchdb-glib 0.5.3 (bug fixes and code cleanup)
• Empathy 2.28.1.2 (bug fixes, documentation and translation work)
• GNOME DVB Daemon 0.1.13 (new features, bug fixes and translation work)
• libgdata 0.5.1 (bug fixes)
• Rhythmbox 0.12.6 (bug fixes and translation work)
• Rygel 0.4.6 (new features and bug fixes)
• tracker 0.7.8 (new features and bug fixes)
• Vala Toys for gEdit 0.6.1 (maintenance release)

You can find more new GNOME software releases at gnomefiles.org.

Comments (none posted)

Repositioning the KDE Brand (KDE.News)

KDE.News has announced an effort to reposition the KDE brand. The K Desktop Environment will now be referred to as KDE and starting with version 4.4, the software making up KDE will be known as the KDE Software Compilation. "KDE has changed over the past 13 years. The application framework has grown, matured and gone cross-platform, as have the applications. Strong growth in our community has created an increasingly diverse and large set of high-quality applications. In the process, KDE's identity has shifted from being simply a desktop environment to representing a global community that creates a remarkably rich body of free software targeted for use by people everywhere. KDE is no longer software created by people, but people who create software. To be able to communicate this clearly in our messaging, it is necessary to reposition the KDE brand so that it reflects the reality. We therefore also need distinct brands for the products we produce."

Comments (3 posted)

KDE Software Announcements

The following new KDE software has been announced this week:

• Bazaar-Servicemenu 0.1 (initial release)
• console_reader 1.0 (unspecified)
• komparator4 0.1 (unspecified)
• KTorrent 3.3.1 (bug fixes)
• satyr 0.1beta1 (initial release)
• streamDirctoryModel 1.0 (unspecified)

You can find more new KDE software releases at kde-apps.org.

Comments (none posted)

Xorg Software Announcements

The following new Xorg software has been announced this week:

• libvdpau 0.3 (bug fixes and documentation work)
• pixman 0.17.2 (new features and performance improvements)
• xf86-input-evdev 2.3.1 (bug fixes)
• xf86-input-wacom 0.10.1 (new features, bug fixes and code cleanup)
• xorg-server 1.7.1.902 (bug fixes)

More information can be found on the X.Org Foundation wiki.

Comments (none posted)

Inkscape 0.47 released

Inkscape 0.47 - a massively reworked version of this vector drawing editor, has been released. Beyond improved performance, there's a long list of new features; see the release notes for details. Also released is an updated version of Inkscape: Guide to a vector drawing program, available from your favorite online bookstore or for direct download.

Comments (5 posted)

SfePy 2009.4 released

Version 2009.4 of SfePy has been announced, it adds some new capabilities and bug fixes. "SfePy (simple finite elements in Python) is a software, distributed under the BSD license, for solving systems of coupled partial differential equations by the finite element method. The code is based on NumPy and SciPy packages."

Full Story (comments: none)

pyspread 0.0.12a released

Version 0.0.12a of pyspread has been announced. "Pyspread is getting close to the first Beta. This new release should work with Windows as well as with Linux. Pyspread is a cross-platform Python spreadsheet application. It is based on and written in the programming language Python."

Full Story (comments: none)

KOffice 2.1 released

Version 2.1 of the KOffice office suite has been announced. "The KOffice team is very happy to announce version 2.1.0 of KOffice, 6 months after the platform release 2.0.0. This release brings a number of new features as well as general improvements in the maturity of the individual applications. Importing of documents have also been given an overhaul. The advantages of the clean and well-structured codebase have started to show. Despite a relatively limited developer group, there are a large number of improvements over 2.0. During the development of 2.1, it was also announced that KOffice is going to be used in the Nokia n900 smartphones based on Maemo Linux."

Comments (9 posted)

Real-Time Toolkit 1.10.2 released

Version 1.10.2 of the Real-Time Toolkit from the Open Robotics Control Software project has been announced. "It's a month since last release, and the RTT deserved another one. Thanks to all of you for reporting bugs and providing fixes. Read on below for the release notes and the noteworthy changes. Upgrading is recommended. The Orocos development team is pleased to announce the second bug fix release of the Real-Time Toolkit v1.10, a C++ toolkit for building component based, real-time robotics and machine control applications."

Comments (none posted)

Caml Weekly News

The November 24, 2009 edition of the Caml Weekly News is out with new articles about the Caml language.

Full Story (comments: none)

Haskell Communities and Activities Report

The November, 2009 edition of the Haskell Communities and Activities Report has been published. "This is the 17th edition of the Haskell Communities and Activities Report. As usual, fresh entries are formatted using a blue background, while updated entries have a header with a blue background. The report is thinner/shorter this time, but has a good percentage of blue and semi-blue entries. I have implemented the strategy, outlined in the May edition, of replacing with online pointers to previous versions those entries for which I received a liveness ping, but which have seen no essential update for a while."

Comments (none posted)

Perl 5.11.2 is now available

Version 5.11.2 of Perl has been announced. "This is the third DEVELOPMENT release in the 5.11.x series leading to a stable release of Perl 5.12.0. You can find a list of high-profile changes in this release in the file "perl5112delta.pod" inside the distribution."

Full Story (comments: none)

Rakudo Perl 6 development release #23 (use Perl)

Development release #23 of Rakudo Perl 6 has been announced. "On behalf of the Rakudo development team, I'm pleased to announce the November 2009 development release of Rakudo Perl #23 "Lisbon". Rakudo is an implementation of Perl 6 on the Parrot Virtual Machine (see http://www.parrot.org)."

Comments (none posted)

PHP 5.3.1 released

Version 5.3.1 of PHP has been announced. "This release focuses on improving the stability of the PHP 5.3.x branch with over 100 bug fixes, some of which are security related. All users of PHP are encouraged to upgrade to this release."

Comments (none posted)

2to3c: an implementation of Python's 2to3 for C code

David Malcolm has announced the 2to3c project. "I've written a tool to help people port their C python extensions from Python 2 to Python 3. It uses the Coccinelle tool to apply a series of "semantic patches" to .c files. I also had to code one of the refactorings in python with regular expressions (due to the need to manipulate preprocessor macros containing commas)."

Full Story (comments: none)

AVC 0.8.0 released

Version 0.8.0 of AVC has been announced. "AVC is a multiplatform, fully automatic, live connection among graphical interface widgets and application variables for the python language. AVC supports in a uniform way the most popular widget toolkits: GTK+, Qt3, Qt4, Tk, wxWidgets, Swing."

Full Story (comments: none)

Release 0.19.0 of CodeInvestigator announced

Version 0.19.0 of CodeInvestigator, a tracing tool for Python programs, has been announced. This version adds new functionality and bug fixes.

Full Story (comments: none)

Cython 0.12 released

Version 0.12 of Cython, a language for writing C extensions to Python, has been announced. "This is the culmination of many months of work, including a mergeback of the experimental branch (after much testing) that was started earlier this year."

Full Story (comments: none)

IMDbPY 4.3 released

Version 4.3 of IMDbPY has been announced. "IMDbPY is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies. With this release, a lot of bugs were fixed, and some minor new features introduced."

Full Story (comments: none)

PyGUI 2.1.1 released

Version 2.1.1 of PyGUI has been announced. "This is an emergency bugfix release to repair some major breakage in the gtk version. Also corrects some other problems. PyGUI is a cross-platform GUI toolkit designed to be lightweight and have a highly Pythonic API."

Full Story (comments: none)

Python-URL! - weekly Python news and links

The November 24, 2009 edition of the Python-URL! is online with a new collection of Python article links.

Full Story (comments: none)

Tcl/Tk 8.5.8 released

Version 8.5.8 of Tcl/Tk has been announced. "The Tcl Core Team is pleased to announce the 8.5.8 releases of the Tcl dynamic language and the Tk toolkit. This is the eighth patch release of Tcl/Tk 8.5. More details can be found below."

Full Story (comments: none)

UliPad 4.0 released

Version 4.0 of UliPad has been announced. "UliPad is a flexible editor, based on wxPython. It's has many features, just like:class browser, code auto-complete, html viewer, directory browser, wizard, etc. The main feature is the usage of mixin. This makes UliPad can be extended easily. So you can write your own mixin or plugin, or simple script, these can be easy and seamless integrated with UliPad."

Full Story (comments: none)

yappi 0.3 released

Version 0.3 of yappi has been announced. "yappi(yet another python profiler) is a Python Profiler with multithreading support. This is the last beta version with some major changes and bugfixes".

Full Story (comments: none)

py.test 1.1.1 released

Version 1.1.1 of py.test, an automated testing tool for Python2, Python3 and Jython, has been announced. "This is a compatibility fixing release of pylib/py.test to work better with previous 1.0.x test code bases. It also contains fixes and changes to work with `execnet>=1.0.0`_ to provide distributed testing and looponfailing testing modes. py-1.1.1 moreover introduces a new mechanism for registering plugins via setuptools."

Full Story (comments: none)

tig 0.15 released

Version 0.15 of tig, an ncurses-based text-mode interface for git, has been announced. "After a long time of silence, here is a brand new version of tig with changes that has accummulated since version 0.14.1. It mainly brings minor improvements for tweaking tig usage via keybindings and options."

Full Story (comments: none)

Freedom Food: FSFE to cook for its donors

The Free Software Foundation Europe is holding a feast, the date and location is to be determined. "For this year's round of fund-raising, the Free Software Foundation Europe (FSFE) has cooked up something special. The biggest donor of 2009 will share a cooking session with FSFE's President and other members of the organisation's executive team. FSFE is looking to raise 100,000 Euro during the last quarter of 2009. Free Software and cooking are both about creativity, skill and enjoying yourself. "Recipes can be used for any purpose, including sharing them with others, and making your own improvements. In these respects cooking is just like Free Software - you have the freedom to use, study, share and improve" says FSFE's President Karsten Gerloff."

Full Story (comments: none)

FSF works with PayPal

The Free Software Foundation has announced that it is working with PayPal to the benefit of the free software community. "A large number of people in the free software community feel that PayPal is a convenient way to send money to others. Part of the reason for this is that you can use many of PayPal's services with only free software--they typically don't require special proprietary software, or even JavaScript. However, FSF recently discovered that PayPal had added a proprietary software license to its User Agreement. FSF license compliance engineer Brett Smith explained, "Of course, the FSF couldn't agree to those terms, so as soon as we learned about them, we contacted PayPal to see if we could make other arrangements. The company listened to our concerns, and specifically excepted us from these conditions. But not only that: next year, PayPal is also updating its user agreement to ensure that the free software community can continue to receive and make payments without having to accept a proprietary software license.""

Full Story (comments: 2)

State of Mozilla and 2008 Financial Statements

Mitchell Baker has put up the annual state of Mozilla posting. Mozilla remains nearly unique among free software projects in its ability to complain about the performance of its investment portfolio; the overall picture, though, says that things are going quite well. "The number of people using Firefox on a daily basis increased from 28 million in 2006 to 49 million in 2007. In 2008 we moved up to 75 million daily users. As of November 2009 the daily number has grown to 110 million, bringing the total number of users to approximately 330 million people."

Comments (11 posted)

Open Letter to the EC about Open Standards and Interoperability

The Open Source Software Thematic Group (GTLL) has sent a letter to the EC concerning open standards and interoperability. "Our Open Source Software Thematic Group became deeply concerned two weeks ago after we learned that the European Commission was preparing a new version of the European Interoperability Framework (EIF) with definitions of interoperability, open standards and maybe even open source completely void of any substance, allowing any proprietary software vendor to claim being compatible with the new guidelines."

Full Story (comments: none)

Sony Ericsson releases WebSDK

Sony Ericsson has announced the Sony Ericsson WebSDK, an open source tool created in collaboration with the PhoneGap open source community. "The SDK makes it possible for web and mobile developers to create mobile apps using HTML, CSS and JavaScript programming. A bridge component allows JavaScript to access platform features and data such as accelerometer, GPS, Camera and contacts."

Full Story (comments: none)

Live Blog: The Google Chrome OS Press Event (ReadWriteWeb)

ReadWriteWeb covers the Google Chrome OS press event. "Google has scheduled a press event for 10am PST this morning where the company plans to announce more details about its Linux-based Google Chrome OS. According to the information we received from Google, the company plans to launch Chrome OS next year. We don't expect Google to release an early build of Chrome OS today, but we would be more than happy to be wrong. We do, however, expect to hear more details about the OS and to see a demo of Chrome OS's functionality." (Thanks to Don Marti).

Comments (none posted)

Why Tim O'Reilly Sees Microsoft as a Proponent of the Open Web (eWeek)

eWeek reports on Tim O'Reilly's prediction of a shift towards openness at Microsoft. "At the Web 2.0 Expo, Tim O'Reilly predicts that Microsoft will emerge as a leading proponent of the open Web, despite the company's tradition of fostering its own proprietary operating systems and development languages. O'Reilly says Microsoft's recent deals to index Twitter tweets and use Wolfram Alpha's APIs for computational data show a shift in its willingness to work with other Web companies. Moreover, the Windows Azure cloud computing operating system is designed to work with open-source technology."

Comments (5 posted)

Debug It!--New from Pragmatic Bookshelf

Pragmatic Bookshelf has published the book Debug It! by Paul Butcher.

Full Story (comments: none)

jQuery Cookbook--New from O'Reilly

O'Reilly has published the book jQuery Cookbook, edited by Cody Lindley.

Full Story (comments: none)

Programming Google App Engine--New from O'Reilly

O'Reilly has published the book Programming Google App Engine by Dan Sanderson.

Full Story (comments: none)

Programming Python 3 (Second Edition) now available

Mark Summerfield has announced the publication of the second edition of his book Programming in Python 3.

Full Story (comments: none)

RESTful Java with JAX-RS--New from O'Reilly Media

O'Reilly has published the book RESTful Java with JAX-RS by Bill Burke.

Full Story (comments: none)

GNOME Journal Issue 17 released

Issue 17 of the GNOME Journal is available; the theme of this issue is "Women in open source." Topics covered include Telepathy, the "un-scary screwdriver," a look back at the 2006 women's summer outreach program, GNOME Shell, and more.

Full Story (comments: 1)

2009 Linux Medical News Freedom Award Recipients (LinuxMedNews)

LinuxMedNews has announced the winners of its 2009 Freedom Award. "It was a difficult choice this year among many worthy people and projects but the panel of judges has spoken. The recipients of the 2009 Linux Medical News Freedom Award presented at the American Medical Informatics Association (amia.org) Fall Conference are the open source, US Government NHIN Connect project and Kevin Toppenberg, MD for his invaluable work and use of Veterans Affairs VistA in the private sector."

Comments (none posted)

Students line up for new free software master at open universities (OSOR)

The Open Source Observatory has a brief article on the "Free Technology Academy," a program oriented around free software and open standards beginning at two European universities. "The FTA will begin on 25 January 2010. The first two courses tutored are 'the concepts of Free Software and Open Standards' and 'The GNU/Linux Operating System'. The programme for the second semester lists 'Web applications development', 'Software development' and 'Economical models'."

Comments (none posted)

LPI announces new partner in the Netherlands

The Linux Professional Institute has announced its newest partner. "The Linux Professional Institute, the world's premier Linux certification organization, announced a new sub-affiliate for their program in Central Europe: LPI-The Netherlands".

Full Story (comments: none)

Linux Professional Institute announces new affiliate in Turkey

The Linux Professional Institute has announced a new Turkish affiliate. "LPI-Turkey will be managed by Endersys Consultancy and Software Ltd. Endersys was founded in 2006 and provides Linux and Open Source solutions, services and IT consulting in Turkey."

Full Story (comments: none)

CFP: FOMS 2010

A call for papers has gone out for FOMS 2010, the Foundations of Open Media Software workshop. Submissions are due by November 30. "FOMS 2010 will take place from 13-15th January 2010 in Wellington, NZ, in the week ahead of LCA. If you are an open media software developer, you should really come along - we will have some key developers come from all over the planet. FOMS is free for community members - the attendance fee for professional delegates is A$420."

Full Story (comments: none)

Linux Audio Conference 2010

The Linux Audio Conference 2010 has been announced. " Linux Audio Conference 2010 The conference about Open Source Software for music and audio May 1-4 2010 Hogeschool voor de Kunsten Utrecht (HKU) Utrecht, The Netherlands Registration is open, and so is the call for abstracts and papers."

Full Story (comments: none)

DrupalSouth: Wellington, NZ

DrupalSouth Wellington has been announced. "DrupalSouth Wellington 2010 is the New Zealand Drupal event. It will be NZ's largest ever gathering of Drupal developers, designers, contributers and business folk. DrupalSouth Wellington will be on Saturday and Sunday 23-24 January - the weekend after Linux.conf.au Wellington."

Full Story (comments: none)

PostgreSQL vs MySQL discussion

A web-based discussion on PostgreSQL vs MySQL has been announced, it will take place on December 8. "For years, the common industry perception has been that MySQL is faster and easier to use than PostgreSQL. PostgreSQL is perceived as more powerful, more focused on data integrity, and stricter at complying with SQL specifications, but correspondingly slower and more complicated to use. Like many perceptions formed in the past, these things aren't as true with the current generation of releases as they used to be."

Comments (none posted)

PyCon registration is open

Registration is open for PyCon, the event takes place in Atlanta, Georgia on February 17-25, 2010. "Registering early gets you early-bird registration rates, guarantees you the tutorials you want, and helps the PyCon volunteers plan better."

Full Story (comments: none)

Events: December 3, 2009 to February 1, 2010

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
December 1 December 5	FOSS.IN/2009	Bangalore, India
December 4	Italian PostgreSQL Day 2009	Pisa, Tuscany, Italy
December 5 December 7	Fedora Users and Developers Conference	Toronto, Canada
December 7 December 11	Annual Computer Security Applications Conference	Honolulu, HI, USA
December 7 December 13	Make Art 2009	Poitiers, France
December 12 December 13	Django Development Sprint	Dallas, TX, USA
December 12 December 17	SciPy India 2009	Kerala, India
December 12	BSD community day	Utrecht, The Netherlands
December 19	New Mexico Linux Fest	Albuquerque, NM, USA
December 27 December 30	26th Chaos Communication Congress	Berlin, Germany
January 13 January 15	Foundations of Open Media Software	Wellington, New Zealand
January 15 January 22	Camp KDE 2010	San Diego, CA, USA
January 18 January 23	linux.conf.au	Wellington, New Zealand
January 23	Workshop on GCC Research Opportunities	Pisa, Italy
January 23 January 24	DrupalSouth Wellington 2010	Wellington, New Zealand

If your event does not appear here, please tell us about it.

PyGUI Mailing List announced

A new PyGUI Mailing List has been announced. "PyGUI is a cross-platform GUI toolkit designed to be lightweight and have a highly Pythonic API."

Full Story (comments: none)