LWN.net Weekly Edition for October 08, 2009

X.Org releases: present and future

The X.Org Foundation released xorg-server 1.7 on October 1st, in preparation for the imminent release of X11R7.5. Users can look forward to improvements in display configuration, screen transformation, and input devices, including the much-anticipated Multi-Pointer X (MPX) code that supports multiple independent keyboard focus points and mouse pointers. At the same time, the development team is drawing up plans to adopt a new release process to accommodate a predictable release schedule and better testing.

What's new with 1.7 and 7.5

Lower-level changes in the new release include several new display-oriented technologies: support for Extended Extended display identification data (E-EDID) and an update to the X Resize, Rotate and Reflect Extension (XRandR). Another proposed update, the "Shatter" enhancement to the EXA acceleration architecture, was deferred to a future release.

E-EDID is a revision of the EDID format with which monitors provide a machine-readable list of capabilities to attached graphics cards. E-EDID supports longer strings than EDID, localization of strings, and adds fields for aspect ratio changing and additional timing and frequency formulas. E-EDID will eventually be superseded by a newer format called DisplayID, but is of particular importance to home electronics users because of its usage in HDMI devices.

XRandR 1.3 adds two new capabilities: projective transforms and panning. Projective Transforms allow more generalized transformations of the image buffer than the previously supported rotation and reflection. This will allow transforms to correct for keystoning and other distortion, as well as scaling of the image buffer. If the displayed desktop is smaller than the virtual screen size, enabling Panning will allow the display to follow the cursor automatically.

The deferred project Shatter was one of X.Org's Google Summer of Code projects, and when integrated will allow screens to be split between multiple framebuffers.

Input devices also see changes in this release, most notably with MPX. As the name indicates, MPX allows multiple input devices to be used at once. That does not mean merely the ability to plug in two mice and two keyboards physically; X has supported that for a long time. But without MPX, multiple attached mice both control the same pointer, and multiple keyboards both route keystrokes into the same input stream. MPX allows for multiple, separate cursors, with separate focus behavior. Some X applications and toolkits will require modification to work with MPX, as they hard-code in the assumption that there is only one keyboard and pointer.

MPX is part of a larger revision to the X input system named XInput2. XInput2 builds on the previous XInput API, and adds other features such as Input Device Properties, a mechanism through which generic properties can be attached to input devices to report special characteristics to the X server and client applications. Such properties might include mouse button timeouts, pointer acceleration, or even logical names (such as distinguishing between multiple attached mice).

Other updates to specific subsystems include changes for Mesa, SELinux, and VGA arbitration, enhancements to the XQuartz server designed for Mac OS X, as well as the deprecation of several obsolete and unmaintained modules and extensions.

The process for 1.8

Peter Hutterer proposed reworking the X.Org release process in an email to the xorg-devel mailing list on September 26. He cited three problems with the existing process: an unpredictable schedule, too much development in the git master that frequently leaves it broken and unusable, and a too-short testing cycle that occurs late in the release process. He noted that the three problems were tightly related, and proposed that the project adopt a timed, predictable release schedule with separate windows for feature merging, bug fixing, and final testing.

The proposed process begins with starting separate branches for new features, rather than developing them as patch sets that could disrupt master. For each release cycle, the project would then use a three month merge window to integrate the feature branches into master, then enter into a two-month bug fix window, and finally freeze master for a one-month release window, during which time a release manager is in charge, and only crucial fixes are merged in. The result, argued Hutterer, would be a predictable six-month release cycle, and a much easier environment for testers.

Keith Packard questioned whether 3:2:1 was the best ratio for feature merging, bug fixing, and release freezing, specifically noting that the feature merge window was considerably larger than that used by the Linux kernel team. Hutterer replied that he thought it was a good starting value, particularly due to the fact that the entire process was new, but added that he thought every facet of the process should be reviewed after the 1.8 cycle, including possibly shortening the merge window.

The effect on testing was particularly popular with the other developers on the list during the subsequent discussion. Several contrasted X.Org's differences from the Linux kernel, beginning with the relative scarcity of X.Org testers. The consensus in the thread was that the history of an unstable git master and lack of documentation to guide willing testers in building and testing the code was to blame; a revised release process with a stable master and individual feature branches could go a long way towards building a community of active X.Org testers.

Hutterer made his proposal on the list because he was unable to attend the 2009 X Developers' Conference (XDC), held in Portland September 28-30. The XDC attendees discussed the proposal, after which Daniel Stone posted their decisions to xorg-devel. The group plans to adopt the basic proposed model for the xorg-server 1.8 / X11R7.6 release cycle, with the addition of choosing release managers for each cycle and asking developers to adopt per-subsystem trees in same manner that the Linux kernel developers use for subsystem maintenance.

Stone's email generated its own controversy thanks to its suggestion that if the the new process is a success for the 1.8/7.6 cycle, then the next step would be to merge graphics drivers into the main xorg-server tree for the 1.10/7.7 cycle. The arguments against merging drivers into the main xorg-server code base included license incompatibilities, but ultimately more developers deemed the simplicity of maintaining drivers in the same codebase as the server to be a long-term win. Still, that change in source code management is still just a proposal, and one slated for two release cycles in the future.

Ultimately, the goal of the proposed new release process is to make the main X.Org codebase more stable, more predictable, and as a result, easier to test. As several on the xorg-devel list pointed out, xorg-server is used on just as many systems as the Linux kernel, but has only a fraction of the active testers that help make the kernel so robust. X.Org continues to make improvements and enhancements with every release, and long gone are the naysayers of a decade ago who proposed ditching X altogether. Hopefully, X11R7.6 and xorg-server 1.8 will arrive on schedule six months from now, and will show the fruits of a longer and more determined testing process, too.

Comments (17 posted)

Scenes from the Real Time Linux Workshop

Real time or real fast?

There is a certain amount of confusion surrounding realtime systems; most commonly, people think that it is concerned with speed. The real focus of realtime computing, though, is determinism: the fastest possible response is far less important than knowing that the system will respond within a bounded time period. In fact, realtime is often at odds with speed, especially if speed is measured in system throughput; this conflict was driven home by Paul McKenney's talk titled "Real fast or real time: how to choose." Paul concluded that one should choose the "real fast" option in a number of situations, including those where throughput is the primary consideration, virtualization is in use, or hard deadlines are not present. In other words, if realtime response is not needed, a realtime kernel should not be used - not a particularly surprising conclusion.

Interestingly, though, the "real fast" option may sometimes be best in hard-deadline situations as well. In particular, if the amount of processing which must be done within the deadline is large enough, the performance costs associated with hard realtime systems may become more of an impediment to getting the work done in time than the non-deterministic nature of general-purpose systems. The number Paul put out was 20ms; if the system must do more computing than that within each deadline cycle, it is likely to perform better on "real fast" machines. In other words, after 20ms of computation, a throughput-optimized system will have caught up enough to make up for any extra latency which might delay the start of that computation.

See Paul's paper [PDF] for more details.

Non-deterministic hardware

Determinism is generally seen as a software issue; it is expected that hardware always behaves in a consistent way. Some research [PDF] presented by Peter Okech, though, makes it clear that contemporary hardware is not as deterministic as one might think. Today's computers incorporate a great deal of complexity from many sources: multiple processors, multiple levels of caching, long instruction-processing pipelines, instruction reordering, branch prediction, system management interrupts, etc. From complexity, says Peter, comes randomness. As a demonstration of this fact, his group did extensive timings of simple instruction sequences; even after long "warmup" cycles and with interrupts disabled, these sequences never did reach a point where they would execute in a constant or predictable time.

For added fun, Peter's group coded a random number generator based on hardware non-determinism. The resulting random number sequences were then subjected to all of the tests they could come up with, from basic mean-calculation and compression tests through to full entropy computation. The results came out the same each time: instruction timings on contemporary systems are truly random. There is no real need to buy special-purpose hardware for random number generation; we are already running on such hardware. Needless to say, there are implications for anybody looking for strict determinism from their systems, especially on very small time scales.

Developers and academics

The closing event of the conference was a panel session on the disconnect between academia and the development community; the panelists were James H. Anderson, Thomas Gleixner, Hermann Härtig, Jan Kiszka, Doug Niehaus, Ismael Ripoll, and Peter Zijlstra. The problem statement asked: why are there dozens of papers on deadline schedulers, but no implementation in Linux? How can somebody get a computer science degree without learning about the problems posed by multicore processors? The actual discussion was relatively unstructured, involving numerous members of the audience, and it did not answer those specific questions. But it was interesting nonetheless.

The session opened with an invitation to the panelists to make wishes, with no real concern for practicality. Developers and academics both wished that professors could receive recognition and credit for patches which get merged into an upstream project. The current system rewards the publication of papers while ignoring practical contributions (including little details like teaching) altogether. Without an incentive to get their work upstream, researchers tend to stop working once their research reaches a publishable state.

It was noted that in some companies (Siemens was cited), employees get credit for accepted patches in much the same way they get credit for more traditional publications.

Another wish which was well received on both sides was the idea that developers and researchers should attend each others' conferences. The two groups tend to speak very different languages; for example, academics talk about "deadlines" (a set period after which the work must be done) while developers worry about "latency" (how long it takes the system to respond to an event). Given fundamental concepts that differ in this way, it is not entirely surprising that the two groups do not always communicate well. Going over to the other side and being immersed in the concerns and language found there would be helpful for everybody working in this field.

Developers asked for the publication of papers which are more easily read on their own. It is hard for busy developers to make time to read academic papers; if they have to go look up a dozen other papers to make sense of one, they are likely to just give up. The publication of more survey papers was suggested as one way to help in this area. Another was to read recent dissertations, which tend to start with relatively complete summaries of the current state of academic understanding. The hosting of summary tutorials at conferences was also suggested.

There was a request from academia for more example problems and tasks that students could take on. Also requested was an easier way to hook research code into the kernel and play with it. That might make it easier for academics to push code upstream, but not all developers are convinced that's a good idea. Instead, they say, it may be better if academics remain focused on long-term problems, with the development community adapting the best ideas for implementation and upstream merging.

[PULL QUOTE: The best thing that could happen would be that Linus Torvalds suddenly falls in love with microkernels. END QUOTE] If one gives academics the green light to be impractical, they will rarely miss the opportunity. So, it was suggested, the best thing that could happen would be that Linus Torvalds suddenly falls in love with microkernels. Thomas Gleixner could then become the maintainer of the L4 microkernel system. The underlying motivation here was not just that academics still think microkernels are better (many certainly do); it's also the simple fact that the Linux kernel has become so complex that it's getting hard for researchers to play with.

There was some lamentation that the academic community is not really producing students who are able to work with the development community. They don't know how to get code upstream. Increasingly, it seems, they don't really even know how to program - especially at the operating systems level. The academic system was charged with churning out armies of Java programmers who have little understanding of how computers actually work and have no clue of the costs of things. The result is that they go forth and create no end of highly bloated systems. The really good developers, it was claimed, tend to come from an electrical engineering background - though the prevalence of hardware engineers who churn out bad code was also noted.

Some universities have experimented with "real-world programming" courses. One of the things they have found is that registrations tend to be low - there is not a great deal of interest in taking that kind of class. There was also some special criticism directed toward the "Bologna process," which is trying to harmonize educational offerings across Europe. That process calls for reducing the standard undergraduate program to three years, which is not at all sufficient to teach people what they really need to know.

A suggestion for students who are interested in learning community development was to simply start with mailing list archives and spend some time watching how things are done. Then dive in. The community is making a real effort to avoid flaming people to a crisp these days, so jumping in is safer than it once was. But, in the end, people join the development community because they are interested in doing so; offering netiquette lessons is unlikely to inspire more of them. There are very few students who have the interest and the ability to become competent system-level programmers. It has always been that way; things have not really changed in that regard.

Internships at open source companies were suggested as a way to build both interest and experience. Such internships exist at a number of companies, though they tend to be fairly severely limited in number. What does exist, though, is the Google Summer of Code program, which is, for all practical purposes, an internship program on a massive scale. The problem here is that the kernel and realtime communities are not really organized in a way that lets them sign up to mentor summer of code students - this problem should certainly be solvable.

But none of that will help if students do not want to learn to do real development in the community. As strange as it seems, it appears to not be an entirely attractive profession. It takes years of work to become a competent engineer; many are simply unwilling to put in that time. Whether things have gotten worse because people expect instant gratification now, or whether it has always been this way was a matter of debate. One panelist suggested that things will only get better when good engineers make more money than good lawyers.

Another complaint was that universities have a certain tendency to actively block free software users. Some use proprietary virtual private network technology which is not available to Linux users. Homework submission sites which only work with Internet Explorer were also mentioned.

The session ended with little in the way of specific action items, but there was one: researchers requested a means by which they could easily experiment with new scheduling algorithms in the kernel. It was agreed that some sort of pluggable schedule technology would be added to the realtime tree, which has long served as a sort of playground for interesting new approaches. A pluggable scheduler seems unlikely to make it upstream, but presence in the realtime tree should make it sufficiently available for researchers to make use of.

The conference adjourned with the announcement of the venue for next year's event. The Real Time Linux Workshop has tended to move around more than most conferences; past events have been held all over Europe as well as China, Mexico, and the US. The 2010 Workshop will continue that practice by moving to Nairobi, Kenya, in the latter part of October. That should be an interesting place to discuss what's happening in the rapidly developing realtime Linux area.

Comments (26 posted)

Toward a freer Android

The dust has mostly settled after Google's shutdown of the Cyanogen build for Android phones. Nobody can really dispute Google's core claim that Cyanogen was redistributing proprietary software in ways not allowed by the license. But numerous people have disputed Google's good sense; those applications are freely downloadable elsewhere and can only run on phones which already shipped with a copy. So shutting down their redistribution does Google little (if any) good, but it has had a harsh chilling effect over the enthusiastic communities that were promoting Android and trying to make it better. Now those communities are trying to regroup and continue their work, but the rules of the game have changed.

The most community-friendly representative within Google has long been Jean-Baptiste Queru; he clearly puts quite a bit of time into helping other developers work with Android. He is now at the center of an effort to turn Google's "Android Open Source Project" (AOSP) into something deserving of that name. Jean-Baptiste has (belatedly, one might say) figured out one of the major obstacles to contributing to the platform: the difficulty of actually running one's changes.

Anybody who has tried to build and install Android knows that this "out of the box" experience is certainly not available today. Part of the problem is the massive size and complexity of the Android platform as a whole; there is not a whole lot to be done about that. But even owners of the "Android Developer Phone," who might reasonably expect to be able to develop for their phones, have to locate a set of proprietary components and incorporate them into the build. And then there's the problem of those proprietary applications. A purely-free Android build lacks the maps, gmail, calendar, and market applications - and the synchronization backends which keep things current with the mothership. Such a build does not equip a handset to be "usable as a day-to-day phone."

The first step, according to Jean-Baptiste, is to get to where an Android build just works on the target hardware - the ADP1, naturally. Once the hardware-level hassles have been overcome, it might make sense to talk about filling in the missing applications. But until developers can easily create a build that runs on a real handset, there's not much point in looking at the bigger goals. With the upcoming AOSP 1.0 release, it looks like this preliminary phase is nearing completion.

Solving the rest of the problems should not be all that hard. If the gmail application never becomes available, mail can be read through IMAP instead - and that might just inspire some people to help improve the somewhat painful email application currently shipped with Android. There is a lot of interest in free mapping utilities, including tools like AndNav which have the potential to surpass Google's maps program. AndNav works from OpenStreetMap data and has the ability to do turn-by-turn navigation - something that the Google tool is unlikely to ever be able to do. SlideME has been offered as a free replacement for the market application. And so on.

The harder part might be the tools requiring synchronization with Google's services; those protocols are not always open. It has been made clear that the Android Open Source Project - hosted at Google - is not going to host software developed for reverse-engineered protocols. So, if Google continues to refuse to make the gmail, calendar, and market backends available, those applications simply will not be supported in free builds. There is, of course, nothing preventing the implementation of applications which synchronize to services hosted elsewhere.

The other place where Google will make its presence felt in this project is with licensing:

GPLv2 might be allowed for new components, maybe, but given the extent to which Android has gone out of its way to avoid GPLv2 software as well, it could still be a hard sell.

Those looking for a more independent effort may be interested in the Open Android Alliance, which is working to make a fully-free version of Android outside of Google. The Alliance's project page (on Google Code, ironically) states that new work will be licensed under GPLv3. It looks like the developers behind the Alliance are not strongly tied to that license, but there are certainly developers out there who would like to see some sort of copyleft license used. If Google is going to hold back and make them reimplement applications, they reason, Google should not be able to take the resulting code and distribute it as another proprietary application.

The Open Android Alliance has a number of developers who are said to be working on various aspects of the problem. It does not, however, appear to have a mailing list or any code available for download. This is a newborn project; its long-term viability is yet to be determined.

What is clear is that people take the "open handset" idea seriously. It is not enough to dump a bunch of code into an online git server; many of us actually want to mess with our devices. Google, perhaps, is starting to understand that, even if it is still having a hard time balancing pressures from the development community, wireless carriers, hardware manufacturers, and its own lawyers. It is not yet clear whether that understanding will translate into sufficient openness for the Android project, but it appears that things might be headed in the right direction.

It seems that Linux World Domination in the handset market is within our grasp. But which Linux distributions will participate in that success? There are a number of Android handsets out there, but there are still more based on other Linux distributions and the LiMo platform. Soon (not soon enough, for many of us) there will be Maemo-based handsets to play with, and it would not be entirely surprising to see Moblin-based handsets in the not-too-distant future. Some of these platforms will do better than others in the market. It may well be that the platform which is the most open, and which draws the most developer interest, will win out.

Comments (36 posted)

Page editor: Jonathan Corbet

Security

LPC: Three sessions from the security track

The Linux Plumbers Conference (LPC) had a full-day security track with talks on multiple topics of interest—far too many to adequately cover. So, just a few of the talks will be looked at here. Some of the other presentations will likely serve as the basis for other articles on this page in the future.

SELinux in Ubuntu

Caleb Case reported on the status of SELinux in Ubuntu. Since Ubuntu already uses AppArmor, one of the obvious questions was: why would Ubuntu add SELinux? Case said that users were asking for it and that having more options for running SELinux (beyond Fedora/RHEL) was desirable. Ubuntu has had SELinux available to install since Hardy Heron (8.04), but it has many more policy modules enabled in Jaunty (9.04) and Karmic (soon to be released 9.10).

The SELinux policy "needs work", Case said, and SELinux in Ubuntu is "not nearly as slick" as it is in Fedora, but it is a work in progress. Users can now do an apt-get install selinux, which will pull in everything that is needed and uninstall AppArmor. The installation updates initramfs, installs the policy, and schedules a system relabel.

Policy is loaded from initramfs instead of via a patched init as has been done in the past. The upstart maintainers did not want to carry a patch to do policy loading, as they didn't want to have to patch for each and every Linux Security Module (LSM) that came along. As it turns out, loading from initramfs is becoming the popular option. Fedora is doing that via dracut and someone from the AppArmor team spoke up to note that it had switched over to loading policy from initramfs as well.

In the future, Case would like to see setroubleshoot added to Ubuntu and integrated with the desktop. They would like to enable more policy modules by default, so setroubleshoot would come in handy. Case said that the Ubuntu policy has fewer confined daemons than Fedora does, and that the reference policy has not been changed anywhere near as much as it has for Fedora. He invited the audience to "check it out, [and] see if it works, or doesn't" and joked that bugs should be submitted to Red Hat's Dan Walsh.

Smack and applications

Smack developer Casey Schaufler presented a look at application changes needed to support Smack on Linux. He started with a brief overview of Smack, including some newer information on packet labeling that can be used by Smack to enforce various controls on network traffic.

Not many changes were required to core applications to support Smack. Things like ls, id, and attr needed to change to show the Smack labels, while login required changes to set the Smack label on the user's login shell. mount needed to support some Smack-specific options for setting default labels on filesystems, and a new utility, newsmack—an administrative tool that is used for setting smack labels on processes and files—was added.

For network applications, sshd needed to be changed to handle the labeling of the login shell. To support network services running at different labels, an xinetd-like utility called smackpolyport was created. It listens at the '*' label and can spawn services running with other labels to enforce network access restrictions. There is also work in progress on adding a Smack extension to the X Access Control Extension (XACE). There is more work to be done to integrate Smack into window managers as well as things like D-Bus, he said.

Schaufler has a habit of tweaking the SELinux development community as part of his talks, and he continued that tradition at LPC. He was discussing his work on making Smack work with the Oracle 11gR1 database server, and one of the criteria he noted was that it did not work with SELinux. In fact, the first step in the installation guide is to turn off SELinux. Some grumbling from the SELinux developers was heard in response to that, with the indication that it was possible—perhaps even unofficially working—but there is no public information on how to run Oracle with SELinux. Schaufler then went through the, fairly simple, steps he took to make Oracle and Smack work together.

Someone asked Schaufler if Smack had been integrated into any distributions. He said that Wind River listed Smack in one of its brochures, and someone from Wind River piped up to say that it was in versions 2.0 and 3.0 of its Linux product. Schaufler also noted that Philips televisions are, or will be soon, running Smack.

Why policy is special

Joshua Brindle looked at the interaction between package managers and SELinux policies, noting that installing policies is very different than application installation. There are policies available for more than 290 applications currently that are typically packaged by distributions, often after some customization is done. For rpm-based distributions, policies get loaded via post-script sections, which can lead to problems that require user intervention if the policy module fails to load.

In addition, third-parties (like Oracle) have a hard time supporting policies for their packages, he said. There are "numerous hacks" to support policy loading. In general, policies just do not fit well into the current application installation model.

Policy is different because it potentially affects the entire system, unlike an application. Policies should be loaded before the applications they affect, or else there is a window in which the application is present, but the labels and policies have not been changed. If the policy fails to load, the application should not be installed, but under the current system, there is no way for rpm to roll the installation back if the post-script section fails.

Policies may also affect multiple applications and their interactions. In many cases, the policy should not be removed if the application is, because there may be user data that is protected by that policy. In addition, other applications may require the policies to be present so they can access the data. So, Brindle said, a new approach is needed. The goal of that work is to include the policy with the distribution package such that policies are installed first, "without hacks", and are part of the installation transaction, so they can be rolled back in the case of failure.

Brindle outlined additional goals of this work, which is initially targeted at rpm: supporting various corner cases like cross-installs and bootstrap installs. Helping third-parties distribute policies for their applications is also an explicit goal, so there needs to be support for multiple policies and policy types (e.g. targeted), as well as support for different distributions and releases. Overall, he summed up the goals as trying to "make life with SELinux easier".

The initial patch to rpm adds policy loading support before the transaction. A second patch changes the %Policy directive to support policy renaming as well as allowing policies to obsolete one another. In addition, the changes to the %Policy directive allow for different policies based on the policy type of the system. Additional patches will support bootstrapping and chroot() installations. Those patches will also add the policies to the rpm database, which will allow the user to change the system policy type while giving rpm the information it needs to install the proper policy.

There is more work to be done, of course. One area that needs to be addressed is how to inform the administrator of policy changes that are being done by a package. Packages from dubious sources could install policies that have the effect of disabling some or all SELinux protections, so administrators need to be informed. There may be support added for differing levels of trust based on where the package file came from, so that administrators can enforce restrictions on what kind of policies packages can install.

Other talks

The most popular attendee was clearly the AVC cow, which made an appearance in Eamon Walsh's demo of XACE. The cow popped up whenever there was an AVC denial from SELinux, which led to calls for more violations so the cow would pop up again. As Dan Walsh (no relation) noted in his blog linked above, it is proof that at least some folks at the NSA (where Eamon Walsh works) have a sense of humor.

Other talks in the track were Dan Walsh's presentation on "sandbox -X", a look at the kernel crypto subsystem by Herbert Xu, David Safford on using the Integrity Management Architecture (IMA), James Carter on a new SELinux policy infrastructure, and a discussion of how to make SELinux easier to use led by Bryan Jacobson. The slides for each of the talks are available on the LPC Program page. There was a fair amount of audience participation, both in terms of questions and suggestions, throughout the sessions; very much in keeping with the mission of LPC. Overall, it was a very useful track for anyone trying to keep up with security in Linux.

Comments (28 posted)

Brief items

ClamAV 0.94.x end of life - with prejudice

Full Story (comments: 19)

New vulnerabilities

backuppc: privilege escalation

Comments (none posted)

elinks: off-by-one buffer overflow

Comments (none posted)

kernel: denial of service

Comments (none posted)

kernel: multiple vulnerabilities

1) NET: llc, zero sockaddr_llc struct
sllc_arphrd member of sockaddr_llc might not be changed. Zero sllc before
copying to the above layer's structure.

http://git.kernel.org/linus/3480c63bdf008e9289aab94418f43b9592978fff
http://git.kernel.org/linus/28e9fc592cb8c7a43e4d3147b38be6032a0e81bc
http://milw0rm.com/exploits/9513

Note that LLC sockets are restricted to root since v2.6.25-rc9 (see commit
3480c63b).

2) can: Fix raw_getname() leak
raw_getname() can leak 10 bytes of kernel memory to user

http://git.kernel.org/linus/e84b90ae5eb3c112d1f208964df1d8156a538289

Note that this was introduced in v2.6.25-rc1.

3) irda: Fix irda_getname() leak
irda_getname() can leak kernel memory to user.

http://git.kernel.org/linus/09384dfc76e526c3993c09c42e016372dc9dd22c

4) appletalk: fix atalk_getname() leak
atalk_getname() can leak 8 bytes of kernel memory to user

http://git.kernel.org/linus/3d392475c873c10c10d6d96b94d092a34ebd4791
http://milw0rm.com/exploits/9521

5) netrom: Fix nr_getname() leak
nr_getname() can leak kernel memory to user.

http://git.kernel.org/linus/f6b97b29513950bfbf621a83d85b6f86b39ec8db

6) econet: Fix econet_getname() leak
econet_getname() can leak kernel memory to user.

http://git.kernel.org/linus/80922bbb12a105f858a8f0abb879cb4302d0ecaa

7) rose: Fix rose_getname() leak
rose_getname() can leak kernel memory to user.

http://git.kernel.org/linus/17ac2e9c58b69a1e25460a568eae1b0dc0188c25

CVE request:
http://article.gmane.org/gmane.comp.security.oss.general/2029
http://article.gmane.org/gmane.comp.security.oss.general/2033  

Comments (none posted)

kernel: denial of service

Comments (none posted)

openoffice.org: arbitrary code execution

Comments (none posted)

samba: several vulnerabilities

Comments (none posted)

strongswan: multiple vulnerabilities

Comments (none posted)

wget: man in the middle attack

Comments (1 posted)

xen: guest privilege escalation

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

One other user-visible change: by default, VFAT filesystems are now mounted with shortname=mixed instead of shortname=lower, preventing the downcasing of filenames when filesystems are copied. Also notable is that Btrfs is finally able to handle full-disk situations; we'll have to find something else to give Chris Mason grief about. The short-form changelog is in the announcement, or see the full changelog for all the details.

The current stable kernel is 2.6.31.3, released on October 7. It contains a single fix for a TTY problem that was affecting a number of users.

Previously, 2.6.31.2 was released on October 5. This is a large stable release; from the review announcement:

So expect more than the usual amount of change for a stable update.

2.6.27.36 and 2.6.30.9 were also released on October 5; they contain a somewhat smaller set of fixes. "This is the last release of the 2.6.30-stable series. Everyone should now move to the 2.6.31 kernel tree. If there are any issues preventing people from doing this, please let me know!"

Comments (none posted)

Quotes of the week

Comments (2 posted)

Netconf 2009 minutes available

Comments (none posted)

The CFQ "low latency" mode

The new mode (initially called "desktop" before being renamed "low_latency") is enabled by default; it can be adjusted by setting the iosched/low_latency attribute associated with each block device in sysfs. When set, some of the delays for "synchronous operations" (reads, generally) no longer happen. The result should be more responsive I/O and, one would hope, happier users.

Note: please see the comments for a description of this change which is more, um, accurate. Your editor blames the Death Flu that his kids brought home.

Comments (10 posted)

The ACPI processor aggregator driver

    NACKed-by: Peter Zijlstra <a.p.zijlstra@chello.nl>

The merging of this patch has drawn some complaints: why should it have made it into the mainline when a core developer clearly has problems with it?

The story goes something like this. ACPI provides a mechanism by which it can ask the system to make processors go idle in emergency situations; these can include power problems or an overheating system. The ACPI folks had originally proposed putting some hacks into the scheduler to implement this functionality. These changes, it seems, were little loved; that was the patch that Peter Zijlstra blocked outright.

So Shaohua Li went back and implemented this functionality as a driver instead. If the ACPI hardware starts sounding the red alert, this driver will create a top-priority realtime thread and bind it to the CPU that is to be idled. That thread, when it "runs," will simply put the CPU into a relatively deep sleep state for a while. When the emergency passes, the thread will go away and normal life resumes. It's a bit of a hack, but it gets the job done, and it is not destructive to system state the way hot-unplugging the CPU would be.

The proper fix would be to enhance the scheduler (the right way) to provide this functionality. But that almost certainly requires the intervention of a real scheduler hacker, and they haven't yet gotten around to solving the problem. So the ACPI "driver" is in the mainline for now. And it may stay that way; Linus said:

And that's where things stand. The driver is little loved, but it will also be little used, can be replaced with a better mechanism if the right people care, and, in the mean time, it may solve a real problem for some users.

Comments (2 posted)

Kernel development news

2.6.x-rc0

Responding to the 2.6.32-rc3 announcement, Len Brown noted that, as far as the version number is concerned, there is no difference between (say) 2.6.31 and a kernel checked out near the end of the 2.6.32 merge window, despite the fact that those two kernels differ significantly from each other. Len had a simple request:

Others echoed this request, but Linus made it clear that he was not interested in this idea:

So what is the problem with the -rc0 idea? It turns out there are a few, one of which being that there is already a much more flexible mechanism built into the kernel build system. If the LOCALVERSION_AUTO configuration option is set, the extra version information will be set in a more specific manner. Your editor, who has not been at home long enough to install a new kernel on his desktop for a bit, is currently running a kernel which reports its version as:

    2.6.31-rc5-00002-g3ce001e

It says that the kernel is the one found at git commit ID g3ce001e; the 00002 indicates that it is two commits after 2.6.31-rc5. This version number makes the exact kernel being run clear in a way that a simple makefile tweak would not. Even if -rc0 were really indicative, it would not really say which kernel was being run.

It gets worse than that, though, especially when developers start bisecting kernels to track down bugs. Consider this example: the two post-2.6.31-rc5 commits in your editor's kernel are a pair of BKL-removal patches which fell through the cracks and didn't make the 2.6.32 merge window. Assuming they make it into 2.6.33, the (simplified) git revision history will look something like this:

A developer trying to use bisection to find a problem in 2.6.33-rc1 might well end up at your editor's commit g3ce001e - as a stopping point, of course; that commit could not possibly be the cause of the problem. Should that developer look at the kernel version number at that point, they will not see 2.6.33-rc0 (even if Linus were to make that change) or even 2.6.32 - the version will be 2.6.31-rc5, the version that particular commit is based on. In the git era, kernel development is not a straight-line affair.

What this implies is that anybody who depends on the kernel version number as found in the Makefile is likely to end up confused. There is, of course, one important exception: that number is meaningful only for the actual release it represents. At any other time, it is an unreliable guide.

That doesn't change the fact that people are getting confused by running a kernel which identifies itself as 2.6.x, but which is really closer to 2.6.x+1. So it seems likely that a couple of things will be done to help. One of those is to make the LOCALVERSION_AUTO option enabled by default, and, possibly, difficult to disable. The other is to add some smarts to the build system which tries to check whether the kernel being built differs from the one which was tagged with the official release number. If that is the case, a simple "+" is appended to the version number. So a kernel checked out in the middle of the 2.6.33 merge window would identify itself as 2.6.32+.

Linus doesn't much like that last option (he sees it as losing a lot of information that the full LOCALVERSION_AUTO option provides), but he "doesn't hate" it either. He actually managed to not hate the idea enough to put together a patch implementing it. It has not been merged as of this writing; there is still some discussion happening about possible changes to the LOCALVERSION_AUTO format. But it seems likely that something along these lines will go in during the 2.6.33 merge window, if not before.

Comments (6 posted)

Concurrency-managed workqueues

Of the mechanisms listed above, the most commonly used by far is workqueues. A workqueue makes it easy for code to set aside work to be done in process context at a future time, but workqueues are not without their problems. There is a shared workqueue that all can use, but one long-running task can create indefinite delays for others, so few developers take advantage of it. Instead, the kernel has filled with subsystem-specific workqueues, each of which contributes to the surfeit of kernel threads running on contemporary systems. Workqueue threads contend with each other for the CPU, causing more context switches than are really necessary. It's discouragingly easy to create deadlocks with workqueues when one task depends on work done by another. All told, workqueues - despite a couple of major rewrites already - are in need of a bit of a face lift.

Tejun Heo has provided that face lift in the form of his concurrency managed workqueues patch. This 19-part series massively reworks the workqueue code, addressing the shortcomings of the current workqueue subsystem. This effort is clearly aimed at replacing the other thread pool implementations in the kernel too, though that work is left for a later date.

Current workqueues have dedicated threads associated with them - a single thread in some cases, one thread per CPU in others. The new workqueues do away with that; there are no threads dedicated to any specific workqueue. Instead, there is a global pool of threads attached to each CPU in the system. When a work item is enqueued, it will be passed to one of the global threads at the right time (as deemed by the workqueue code). One interesting implication of this change is that tasks submitted to the same workqueue on the same CPU may now execute concurrently - something which does not happen with current workqueues.

One of the key features of the new code is its ability to manage concurrency in general. In one sense, all workqueue tasks are executed concurrently after submission. Actually doing things that way would yield poor results, though; those tasks would simply contend with each other, causing more context switches, worse cache behavior, and generally worse performance. What's really needed is a way to run exactly one workqueue task at a time (avoiding contention) but to switch immediately to another if that task blocks for any reason (avoiding processor idle time). Doing this job correctly requires that the workqueue manager become a sort of special-purpose scheduler.

As it happens, that's just how Tejun has implemented it. The workqueue patch adds a new scheduler class which behaves very much like the normal fair scheduler class. The workqueue class adds a couple of hooks which call back into the workqueue code whenever a task running under that class transitions between the blocked and runnable states. When the first workqueue task is submitted, a thread running under the workqueue scheduler class is created to execute it. As long as that task continues to run, other tasks will wait. But as soon as the running task blocks on some resource, the scheduler will notify the workqueue code and another thread will be created to run the next task. The workqueue manager will create as many threads as needed (up to a limit) to keep the CPU busy, but it tries to only have one task actually running at any given time.

Also new with Tejun's patch is the concept of "rescuer" threads. In a tightly resource-constrained system, it may become impossible to create new worker threads. But any existing threads may be waiting for the results of other tasks which have not yet been executed. In that situation, everything will stop cold. To deal with this problem, some special "rescuer" threads are kept around. If attempts to create new workers fail for a period of time, the rescuers will be summoned to execute tasks and, hopefully, clear the logjam.

The handling of CPU hotplugging is interesting. If a CPU is being taken offline, the system needs to move all work off that CPU as quickly as possible. To that end, the workqueue manager responds to a hot-unplug notification by creating a special "trustee" manager on a CPU which is sticking around. That trustee takes over responsibility for the workqueue running on the doomed CPU, executing tasks until they are all gone and the workqueue can be shut down. Meanwhile, the CPU can go offline without waiting for the workqueue to drain.

These patches were generally welcomed, but there were some concerns expressed. The biggest complaint related to the special-purpose scheduling class. The hooks were described as (1) not really scheduler-related, and (2) potentially interesting beyond the workqueue code. For example, Linus suggested that this kind of hook could be used to implement the big kernel lock semantics, releasing the lock when a process sleeps and reacquiring it on wakeup. The scheduler class will probably go away in the next version of the patch; what remains to be seen is what will replace it.

One idea which was suggested was to use the preemption notifier hooks which are already in the kernel. These notifiers would have to become mandatory, and some new callbacks would be required. Another possibility would be to give in to the inevitable future when perf counters events will take over the entire kernel. Event tracepoints are designed to provide callbacks at specific points in the kernel; some already exist for most of the interesting scheduler events. Using them in this context would mostly be a matter of streamlining the perf events mechanism to handle this task efficiently.

Andrew Morton was concerned that the new code would take away the ability for a specific workqueue user to modify its worker tasks - changing their priority, say, or having them run under a different UID. It turns out that, so far, only a couple of workqueues have been modified in this way. The workqueue used by stop_machine() puts its worker threads into the realtime scheduling class, allowing them to monopolize the processors when needed; Tejun simply replaced that workqueue with a set of dedicated kernel threads. The ACPI code had bound a workqueue thread to CPU 0 because some operations corrupt the system if run anywhere else; that case is easily handled with the existing schedule_work_on() function. So it seems that, for now at least, there is no need for non-default worker threads.

One remaining issue is that some subsystems use single-threaded workqueues as a sort of synchronization mechanism; they expect tasks to complete in the same order they were submitted. Global thread pools change that behavior; Tejun has not yet said how he will solve that problem.

It almost certainly will be solved, along with the other concerns. David Howells, the creator of the slow work subsystem, thinks that the new workqueues could be a good replacement. In summary, this change looks likely to be accepted, perhaps as early as 2.6.33. Then we might finally have a single thread pool in the kernel.

Comments (11 posted)

Infrastructure unification in the block layer

For nearly as long there have been suggestions that having two frameworks is a waste and that they should be unified. However little visible effort has been made toward this unification and such efforts as there might have been have not yielded any lasting success. The most united thing about the two is that they have a common directory in the Linux kernel source tree (drivers/md); this is more a confusion than a unification. The two subsystems have both seen ongoing development side by side, each occasionally gaining functionality that the other has and so, in some ways, becoming similar. But similarity is not unity, rather it serves to highlight the lack of unity as it is no longer function that keeps the two separate, only form.

Exploring why unification has never happened would be an interesting historical exercise that would need to touch on the personalities of the people involved, the drift in functionality between the two systems which started out with quite different goals, the differing perceptions of each by various members of the community, and the technological differences that would need to be resolved. Not being an historian, your author only feels competent to comment on that last point, and, as it is the one where a greater understanding is most likely to aid unification, this article will endeavor to expose the significant technological issues that keep the two separate. In particular, we will explore the weaknesses in each infrastructure. Where a system has strengths, they are likely to be copied, thus creating more uniformity. Where it has weaknesses, they are likely to be assiduously avoided by others, thus creating barriers.

Trying to give as complete a picture as possible, we will explore more than just DM and MD. Loop, NBD, and DRBD provide similar functionality behind their own single-use infrastructure; exploring them will ensure that we don't miss any important problems or needs.

The flaws of MD

Being the lead developer of MD for some years, your author feels honour bound to start by identifying weaknesses in that system.

One of the more ugly aspects of MD is the creation of a new array device. This is triggered by simply opening a device special file, typically in /dev. In the old days, when we had a fairly static /dev directory, this seemed a reasonable approach. It was simply necessary to create a bunch of entries in /dev (md0, md1, md2, ...) with appropriate major and minor numbers at the same time that the other static content of /dev was created. Then, whenever one of those entries was opened, the internal data structures would spontaneously be created so that the details of the device could be filled in.

However with the more modern concept of a dynamic /dev, reflecting the fact that the set of devices attached to a given system is quite fluid, this doesn't fit very well. udev, which typically manages /dev, only creates entries for devices that the kernel knows about. So it will not create any md devices until the kernel believes them to exist. They won't exist until the device file has been created and can be opened - a classic catch-22 situation.

mdadm, the main management tool for MD, works around this problem by creating a temporary device special file just so that it can open it and thereby create the device. This works well enough, but is, nonetheless, quite ugly. The internal implementation is particularly ugly and it was only relatively recently that the races inherent in destroying an MD device (which could be recreated at any moment by user space) were closed so that MD devices don't have to exist forever.

A closely related problem with MD is that the block device representing the array appears before the array is configured or has any data in it. So when udev first creates /dev/md0, an attempt to open and read from it, (to find out if a filesystem is stored there, for example) will find no content. It is only after the component devices have been attached to the array and it has been fully configured that there is any point in trying to read data from the array.

This initial state, where the device exists but is empty, is somewhat like the case of removable-media devices, and can be managed along the same lines as those: we could treat the array as media that can spontaneously appear. However MD is, in other ways, quite unlike removable media (there is no concept of "eject") and it would generally cause less confusion if MD devices appeared fully configured so they looked more like regular disk drive devices.

A problem that has only recently been addressed is the fact that MD manages the metadata for the arrays internally. The kernel module knows all about the layout of data in the superblock and updates it as appropriate. This makes it easy to implement, but not so easy to extend. Due to the lack of any real standard, there are many vendor-specific metadata layouts that all can be used to describe the same sort of array. Supporting all of those in the kernel would unnecessarily bloat the kernel, and supporting them in user space requires information about required updates to be reported to user space in a reliable way.

As mentioned, this problem has recently been addressed, so it is now quite possible to manage vendor-specific metadata from user space. It is still worth noting, though, as one of the problems that has stood in the way of earlier attempts at DM/MD integration: DM does not manage metadata at all, leaving it up to user-space tools.

The final flaw in MD to be exposed here is the use and nature of the ioctl() commands that are used to configure and manage MD arrays. The use of ioctl() has been frowned upon in the Linux community for some years. There are a number of reasons for this. One is that strace cannot decode newly-defined ioctls, so the use of ioctl() can make a program's behaviour harder to observe. Another is that it is a binary interface (typically passing C structures around) and so, when Linux is configured to support multiple ABIs (e.g. a 32bit and a 64bit version), there is often a need to translate the binary structure from one ABI to the other (see the nearly 3000 lines in fs/compat_ioctl.c).

In the case of MD, the ioctl() interface is not very extensible. The command for configuring an array allows only a "level", a "layout", and a "chunksize" to be specified. This works well enough for RAID0, RAID1, and RAID5, but even with RAID10 we needed to encode multiple values into the "layout" field which, while effective, isn't elegant.

In the last few years MD has grown a separate configuration interface via a collection of attribute files exposed in sysfs. This is much more extensible, and there are a growing number of features of MD which require the sysfs interface. However even here there is still room for improvement. The MD attribute files are stored in a subdirectory of the block device directory (e.g. /sys/block/md0/md/). While this seems natural, it entrenches the above-mentioned problem that the block device must exist before the array can be configured. If we wanted to delay creation of the block device until the array is ready to serve data, we would need to store these attribute files elsewhere in sysfs.

The failings of DM

DM has a very different heritage than MD and, while it shares some of the flaws of MD, it avoids others.

DM devices do not need to exist in /dev before they can be created. Rather there is a dedicated "character device" which accepts DM ioctl() commands, including the command to create a new device. Thus, the catch-22 problem from which MD suffers, is not present in DM. It has been suggested that MD should take this approach too. However, while it does solve one problem, it still leaves the problem of using ioctl(). There doesn't seem a lot of point making significant change to a subsystem unless the result avoids all known problems. So while waiting for a perfect solution, no such small steps have been made to bring MD and DM closer together.

The related issue of a block device existing before it is configured is still present in DM, though separating the creation of the DM device from the creation of the block device would be much easier in DM. This is because, as mentioned, with DM all configuration happens over the character device whereas with MD, the configuration happens via the block device itself, so it must exist before it can be configured.

While DM also uses ioctl() commands, which could be seen as a weakness, the commands chosen are much more extensible than those used by MD. The ioctl() command to configure a device essentially involves passing a text string to the relevant module within DM, and it interprets this string in any way it likes. So DM is not limited to the fields that were thought to be relevant when DM was first designed.

Metadata management with DM is very different than with MD. In the original design, there was never any need for the kernel module to modify metadata, so metadata management was left entirely in user space where it belongs. More recently, with RAID1 and RAID5 (which is still under development), the kernel is required to synchronously update the metadata to record a device failure. This requires a degree of interaction between the kernel and user space which has had to be added.

The main problem with the design of DM is the fact that it has two layers: the table layer and the target layer. This undoubtedly comes from the original focus of DM, which was logical volume management (LVM), and it fits that focus quite well. However, it is an unnecessary layering and just gets in the way of non-LVM applications.

A "target" is a concept internal to DM, which is the abstraction that each different module presents. So striping, raid1, multipath, etc. each present a target, and these targets can be combined via a table into a block device.

A "table" is simply a list of targets each with a size and an offset. This is analogous to the "linear" module in MD or what is elsewhere described as concatenation. The targets are essentially joined end-to-end to form a single larger block device.

This contrasts with MD where each module - raid0, raid1, or multipath, for example - presents a block device. This block device can be used as-is, or it can be combined with others, via a separate array, into a single larger block device.

To highlight the effect of this layering a little more, suppose we were to have two different arrays made of a few devices. In one array we want the data striped across the devices. In the other we lay the data out filling the first device first and then moving on to the next device. With MD, the only difference between these two would be the choice of "raid0" or "linear" as the module to manage them. With DM, the first step would involve including all the devices in a single "stripe" target, and then placing that target as the sole entry in a table. The second would involve creating a number of "linear" targets, one for each device, and then combining them into a table with multiple entries.

Having this internal abstraction of a "target" serves to insulate and isolate DM from the block device layer, which is the common abstraction used by other virtual devices. A good example of this separation is the online reconfiguration functionality that DM provides. The boundary between the table and the targets allows DM to capture new requests in the table layer while allowing the target layer to drain and become idle, and then to potentially replace all the targets with different targets before releasing the requests that have been held back.

Without that internal "target" layer, that functionality would need to be implemented in the block layer on its boundary with the driver code (i.e. in generic_make_request() and bio_endio()). Doing this would be more effort (i.e. DM would not benefit from insulation) and it would then be more generally useful (i.e. DM would not be so isolated). Many people have wanted to be able to convert a normal device into a degraded RAID1 "array" or to enable multipath support on a device without first unmounting the filesystem which was mounted directly from one of the paths. If online reconfiguration were supported at the block layer level, these changes would become possible.

The difference of DRBD

DRBD, the Distributed Replicated Block Device, is the most complex of the virtual block devices that do not aim to provide a general framework. It is not yet included in the mainline, but it could yet be merged in 2.6.33.

Its configuration mechanism is similar to that of DM in a number of ways. There is a single channel which can be used to create and then manage the block devices. The protocol used over this channel is designed to be extensible, though the current definitions are very much focused around the particular needs of DRBD (as would be expected), so how easy it might be to extend to a different sort of array is not immediately clear.

Where DM uses ioctl() with string commands over a dedicated character device, DRBD uses a packed binary protocol over a netlink connection. This is essentially a socket connection between the kernel module and a user-space management program which carries binary encoded messages back and forth. This is probably no better or worse than ioctl(); it is simply different. Presumably it was chosen because there is general bad feeling about ioctl(), but no such bad feeling about netlink. Linus, however, doesn't seem keen on either approach.

DRBD appears to share metadata management between the kernel module and user space. Metadata which describes a particular DRBD configuration is created and interpreted by user-space tools and the information that is needed by the kernel is communicated over the netlink socket. DRBD uses other metadata to describe the current replication state of the system - which blocks are known to be safely replicated and which are possibly inconsistent between the replicas for some reason. This metadata (an activity log and a bitmap) is managed by the kernel, presumably for performance reasons.

This sharing of responsibility makes a lot of sense as it allows the performance-sensitive portions to remain in the kernel but still leaves a lot of flexibility to support different metadata formats. This approach could be improved even more by making the bitmap and activity log into independent modules that can be used by other virtual devices. Each of DM, MD, and DRBD have very similar similar mechanisms for tracking inconsistencies between component devices; this is possibly the most obvious area where sharing would be beneficial.

Loop, NBD and the purpose of infrastructure

Partly to emphasize the fact that it isn't necessary to use a framework to have a virtual block device, loop and NBD (the Network Block Device) are worth considering. While loop doesn't appear to aim to provide a framework for a multiplicity of virtual devices, it nonetheless combines three different functions into one device. It can make a regular file look like a block device, it can provide primitive partitioning of a different block device, and it can provide encryption and decryption so that an encrypted device can be accessed. Significantly, these are each functions that were subsequently added to DM, thus highlighting the isolating effect of the design of DM.

NBD is much simpler in that is has just one function: it provides a block device for which all I/O requests are forwarded over a network connection to be serviced on - normally - a different host. It is possibly most instructive as an example of a virtual block device that doesn't need any surrounding framework or infrastructure.

Two areas where DM or MD devices make use of an infrastructure, while Loop and NBD need to fend for themselves, are in the creation of new devices and the configuration of those devices. NBD takes a very simple approach of creating a predefined number of devices at module initialization time and not allowing any more. Loop is a little more flexible and uses the same mechanism as MD, largely provided by the block layer, to create loop devices when the block device special file is opened. It does not allow these to be deleted until the module is unloaded, usually at system shutdown time. This architecture suggests that some infrastructure could be helpful for these drivers, and that the best place for that infrastructure could well be in the block layer, and thus shared by all devices.

For configuration, both Loop and NBD use a fairly ad hoc collection of ioctl() commands. As we have already observed, this is both common and problematic. They could both benefit from a more standardized and transparent configuration mechanism.

It might be appropriate to ask at this point why there is any need for subsystem infrastructure such as DM and MD. Why not simply follow the pattern seen in loop, NBD and DRDB and have a separate block device driver for each sort of virtual block device? The most obvious reason is one that doesn't really apply any more. At the time when MD and DM were being written there was a strong connection between major device numbers and block device drivers. Each driver needed a separate major number. Loop is 7, NBD is 43, DRBD is 147, and MD is 9. DM doesn't have a permanently allocated number, it chooses a spare number when the module is loaded, so it usually gets 254 or 253.

Furthermore, at that time, the number of available major numbers was limited to 255 and there was danger of running out. Allocating one major number for RAID0, one for LINEAR, one for RAID1 and so forth would have looked like a bit of a waste, so getting one for MD and plugging different personalities into the one driver might have been a simple matter of numerical economy. Today, we have many more major numbers available, and we no longer have a tight binding between major numbers and device drivers - a driver simply claims whichever device numbers it wants at any time, when the module is loaded, or when a device is created.

A second reason is the fact that all the MD personalities envisioned at the time had a lot in common. In particular they each used a number of component devices to create a larger device. While creating a midlayer to encapsulate this functionality might be a mistake, it is a very tempting step and would seem to make implementation easier.

Finally, as has been mentioned, having a single module which defines its own internal interfaces can provide a measure of insulation from other parts of the kernel. While this was mentioned only in the context of DM, it is by no means absent from MD. That insulation, while not necessarily in the best interests of the kernel as a whole, can make life a lot easier for the individual developer.

None of these reasons really stand up as defensible today, though some were certainly valid in the past. So it could be that, rather than seeking unification of MD and DM, we should be seeking their deprecation. If we can find a simple approach to allow different implementations of virtual block devices to exist as independent drivers, but still maintain all the same functionality as they presently have, that is likely to be the best way forward.

Unification with the device model

This brings us to the Linux device model. While there may be no real need to unify DM with MD, the devices they create need to fit into the unifying model for devices which we call the "device model" and which is exposed most obviously through various directory trees in sysfs. The device model has a very broad concept of a "device." It is much more than the traditional Unix block and character devices; it includes busses, intermediate devices, and just about anything that is in any way addressable.

In this model it would seem sensible for there to be an "array" device which is quite separate from the "block" device providing access to the data in the array. This is not unlike the current situation where a SCSI bus has a child which is a SCSI target which, in turn, has a child which is a SCSI LUN (Logical UNit), and that device itself is still separate from the block device that we tend to think of as a "SCSI disk". This separation would allow the array to be created and configured before the block device can come into being, thus removing any room for confusion for udev.

The device model already allows for a bus driver to discover devices on that bus. In most cases this happens automatically during boot or at hotplug time. However, it is possible to ask a bus to discover any new devices, or to look for a particular new device. This last action could easily be borrowed to manage creation of virtual block devices on a virtual bus. The automatic scan would not find any devices, but an explicit request for an explicitly-named device could always succeed by simply creating that device. If we then configure the device by filling in attribute files in the virtual block device, we have a uniform and extensible mechanism for configuring all virtual block devices that fits with an existing model.

Again, the device model already allows for binding different drivers to devices as implemented in the different "bind" files in the /sys/bus directory tree. Utilizing this idea, once a virtual block device was "discovered" on the virtual block device bus, an appropriate driver could be bound to it that would interpret the attributes, possibly create files for extra attributes, and, ultimately, instantiate the block device.

Possibly the most difficult existing feature to represent cleanly in the device model is the on-line reconfiguration that DM and, more recently, MD provide. This allows control of an array to be passed from one driver to another without needing to destroy and recreate the block device (thus, for example, a filesystem can remain mounted during the transition). Doing this exchange in a completely general way would involve detaching a block device from one parent and attaching it to another. This would be complex for a number of reasons, one being the backing_dev_info structure, which creates quite a tight connection between a filesystem and the driver for the mounted block device.

Another weakness in the device model is that dependencies between devices are very limited - a device can be dependent on at most one other device, its parent. This doesn't fit very well with the observation that an array is dependent on all the components of the array, and that these components can change from time to time. Fortunately this weakness has already been identified and, hopefully, will be resolved in a way that also works for virtual block devices.

So, while there are plenty of issues that this model leaves unresolved, it does seem that unification with the device model holds the key to unification between MD and DM, along with any other virtual block devices.

So what is the answer?

Knowing that a problem is hard does not excuse us from solving it. With the growing interest in managing multiple devices together, as seen in DRBD and Btrfs, as well as in the increasing convergence in functionality between DM and MD, now might be the ideal time to solve the problems and achieve unification. Reflecting on the various problems and differences discussed above, it would seem that a very important step would be to define and agree on some interfaces; two in particular.

The first interface that we need is the device creation and configuration interface. It needs to provide for all the different needs of DM, MD, Loop, NBD, DRBD, and probably even Btrfs. It needs to be sufficiently complete such that the current ioctl() and netlink interfaces can be implemented entirely through calls into this new interface. It is almost certain that this interface should be exposed through sysfs and so needs to integrate well with the device model.

The second interface is the one between the block layer and individual block drivers. This interface needs to be enhanced to support all the functionality that a DM target expects of its interface with a DM table, and in particular it needs to be able to support hotplugging of the underlying driver while the block device remains active.

Defining and, very importantly, agreeing on these interfaces will go a long way towards achieving the long sought after unification.

Comments (24 posted)

Patches and updates

Kernel trees

• Linus Torvalds: Linux 2.6.32-rc3 . (October 5, 2009)

• Greg KH: Linux 2.6.31.3 . (October 7, 2009)

• Greg KH: Linux 2.6.31.2 . (October 5, 2009)

• Greg KH: Linux 2.6.30.9 . (October 5, 2009)

• Greg KH: Linux 2.6.27.36 . (October 5, 2009)

• Thomas Gleixner: 2.6.31.2-rt13 . (October 6, 2009)

Core kernel code

• Bharata B Rao: CFS Hard limits - v2 . (October 3, 2009)

• Tejun Heo: workqueue: implement concurrency managed workqueue . (October 3, 2009)

• cl@linux-foundation.org: Introduce per cpu atomic operations and avoid per cpu address arithmetic . (October 3, 2009)

• Jeff Moyer: aio: implement request batching . (October 3, 2009)

• john stultz: time: logrithmic time accumulation . (October 3, 2009)

Development tools

• Masami Hiramatsu: tracing/kprobes, perf: perf probe support take 2 . (October 5, 2009)

• Martin K. Petersen: scsi_debug: Thin provisioning . (October 5, 2009)

• Tom Zanussi: perf trace: support for general-purpose scripting . (October 6, 2009)

• Subrata Modak: The Linux Test Project has been Released for SEPTEMBER 2009 . (October 6, 2009)

Device drivers

• Alok Kataria: SCSI driver for VMware's virtual HBA - V5. . (October 3, 2009)

• Samu Onkalo: LIS3LV02D I2C driver . (October 3, 2009)

• Stephen M. Cameron: Add hpsa driver for HP Smart Array controllers. . (October 3, 2009)

• Dave Hansen: LED driver for Intel NAS SS4200 series (v5) . (October 3, 2009)

• Marek Szyprowski: Global Video Buffers Pool - PMM and UPBuffer reference drivers [RFC] . (October 5, 2009)

• Kyungmin Park: Haptic class support (v2) . (October 6, 2009)

• Mike Frysinger: mfd: ADP5520 Multifunction LCD Backlight and Keypad Input Device Driver . (October 6, 2009)

• Giuseppe CAVALLARO: net: add support for STMicroelectronics Ethernet controllers. . (October 6, 2009)

Documentation

• Michael Kerrisk: man-pages-3.23 is released . (October 3, 2009)

Filesystems and block I/O

• Valerie Aurora: Union mounts/writable overlays design . (October 3, 2009)

• scjody@sun.com: Journal guided resync and support . (October 3, 2009)

• Ryo Tsuruta: I/O bandwidth controller and BIO tracking . (October 3, 2009)

• Edward Shishkin: Reiser4 for Linux-2.6.31 . (October 5, 2009)

• Sage Weil: ceph distributed file system client . (October 6, 2009)

• Nick Piggin: store-free path walking . (October 7, 2009)

Janitorial

• Jean Delvare: Drop generic i2c chip driver module parameters . (October 6, 2009)

Memory management

• Suresh Jayaraman: Swap over NFS -v20 . (October 3, 2009)

• Lee Schermerhorn: hugetlb: V8 numa control of persistent huge pages alloc/free . (October 5, 2009)

• Nick Piggin: Latest vfs scalability patch . (October 6, 2009)

• Gleb Natapov: add MAP_UNLOCKED mmap flag . (October 6, 2009)

• Lee Schermerhorn: hugetlb: V9 numa control of persistent huge pages alloc/free . (October 6, 2009)

• Wu Fengguang: some writeback experiments . (October 7, 2009)

Networking

• Gregory Haskins: net: add dataref destructor to sk_buff . (October 3, 2009)

Security-related

• Tetsuo Handa: Starting TOMOYO 2.3 . (October 5, 2009)

• Serge E. Hallyn: cr: add generic LSM c/r support (v4) . (October 5, 2009)

Virtualization and containers

• KAMEZAWA Hiroyuki: [PATCH 0/2] memcg: improving scalability by reducing lock contention at charge/uncharge . (October 3, 2009)

• Gregory Haskins: net: Add vbus_enet driver . (October 3, 2009)

• Gregory Haskins: KVM: xinterface . (October 3, 2009)

• Jeremy Fitzhardinge: Extending pvclock down to usermode for vsyscall . (October 6, 2009)

Benchmarks and bugs

• Rafael J. Wysocki: 2.6.32-rc1-git2: Reported regressions from 2.6.31 . (October 3, 2009)

• Rafael J. Wysocki: 2.6.32-rc1-git2: Reported regressions 2.6.30 -> 2.6.31 . (October 3, 2009)

Miscellaneous

• Mathieu Desnoyers: Userspace RCU library (liburcu) 0.2 . (October 3, 2009)

• Douglas Gilbert: sg3_utils-1.28 available . (October 3, 2009)

• Douglas Gilbert: sdparm 1.04 available . (October 3, 2009)

• Jayson R. King: BFS backport to 2.6.27 . (October 5, 2009)

• Linus Walleij: CGFreak control group visualizer . (October 7, 2009)

Page editor: Jonathan Corbet

Distributions

News and Editorials

openSUSE Boosters

The openSUSE Boosters' Team held their first meeting after the openSUSE conference. Francis Giannaros introduced the team in a recent blog post.

These people will be working full time on making it easier to contribute to the openSUSE Project and improving communication within and outside the project. Together they aim to make openSUSE the best Linux distribution, with an open, inviting and independent community. They will work to improve the infrastructure and lower the bar for contributors.

For now the Booster project team has been divided into three groups. One group will be organizing all documentation on how to contribute to the project. They will create a single wiki page with links to all such documentation, including video tutorials so that new people can more easily find what they are looking for.

A second group will be doing the same for the infrastructure, integrating all of it under one umbrella so that people can see all the various web services on one web page. Current web services include user forums, download sites, the build service, openFATE feature tracking, blogs, the weekly news, and much more. There will be a consistent look and feel for each web service landing page. The idea is to group these projects on the portal page, with categories for novices, experienced users and computer professionals so that people can figure out what needs work and where they can best contribute.

The third group will be working on making openSUSE's development Factory branch more transparent. There will be a Factory Status overview page that will allow developers and testers to easily see what packages are failing to build, which one are in need of a maintainer and where more testing is needed.

Future goals include adding social collaboration tools to the distribution and developing an Upstream Attraction Program. Overall they will help to improve communication within the project and help to spread the word about the project to potential contributors.

The mailing list has not been set up yet, but watch for it to get started soon. Meanwhile, contact information for the team members can be found on the openSUSE Boosters' Team site.

Comments (none posted)

New Releases

Gentoo Linux - Ten Years Compiling: 1999 - 2009

Comments (none posted)

openSUSE 11.2 Milestone 8 Released

Full Story (comments: none)

Ubuntu 9.10 Beta released

Full Story (comments: none)

Distribution News

Debian GNU/Linux

Debian Release architectures

Full Story (comments: none)

Fedora

Fedora documentation team switching to CC-BY-SA license

Full Story (comments: none)

Red Hat Enterprise Linux

From Code to Community to Enterprise-Ready (Red Hat News)

Comments (none posted)

Slackware Linux

GNOME SlackBuild 2.26.3 GNOME Desktop for Slackware and Slackware64 13.0

Comments (none posted)

Ubuntu family

2009 Ubuntu Community Council vote complete

Full Story (comments: none)

Minutes from the Ubuntu Technical Board meeting

Full Story (comments: none)

Ubuntu DMB public meeting about approval process

Full Story (comments: none)

Other distributions

Arch Linux Handbook in print

Comments (none posted)

New Distributions

ZevenOS

Comments (none posted)

Distribution Newsletters

DistroWatch Weekly, Issue 323

Comments (none posted)

Fedora Weekly News 196

Full Story (comments: none)

Openmoko Community Updates/2009-09-30

Comments (none posted)

OpenSUSE Weekly News/91

Comments (none posted)

Ubuntu Weekly Newsletter #162

Full Story (comments: none)

Distribution reviews

openSUSE 11.2 M8: What a Fine Lookin' Lizard (ZDNet)

Comments (none posted)

Ubuntu's Karmic Koala opens its eyes (The Register)

Comments (none posted)

Page editor: Rebecca Sobol

Development

LPC: The past, present, and future of Linux audio

The history, status, and future of audio for Linux systems was the topic of two talks—coming at the theme from two different directions—at the Linux Plumbers Conference (LPC). Ardour and JACK developer Paul Davis looked at audio from mostly the professional audio perspective, while PulseAudio developer Lennart Poettering, unsurprisingly, discussed desktop audio. Davis's talk ranged over the full history of Linux audio and gave a look at where he'd like to see things go, while Poettering focused on the changes since last year's conference and "action items" for the coming year.

Davis: origins and futures

Davis started using Linux as the second employee at Amazon in 1994, and started working on audio and MIDI software for Linux in 1998. So, he has been working in Linux audio for more than ten years. His presentation was meant to provide a historical overview on why "audio on linux still sucks, even though I had my fingers in all the pies that make it suck". In addition, Davis believes there are lessons to be learned from the other two major desktop operating systems, Windows and Mac OS X, which may help in getting to better Linux audio.

He outlined what kind of audio support is needed for Linux, or, really, any operating system. Audio data should be able to be brought in or sent out of the system via any available audio interface as well as via the network. Audio data, as well as audio routing information, should be able to be shared between applications, and that routing should be able to changed on the fly based on user requests or hardware reconfiguration. There needs to be a "unified approach" to mixer controls, as well. Most important, perhaps, is that the system needs to be "easy to understand and to reason about".

Some history

Linux audio support began in the early 1990s with the Creative SoundBlaster driver, which became the foundation for the Open Sound System (OSS). By 1998, Davis said, there was growing dissatisfaction with the design of OSS, which led Jaroslav Kysela and others to begin work on the Advanced Linux Sound Architecture (ALSA).

Between 1999 and 2001, ALSA was redesigned several times, each time requiring audio applications to change because they would no longer compile. The ALSA sequencer, a kernel-space MIDI router, was also added during this time frame. By the end of 2001, ALSA was adopted as the official Linux audio system in favor instead of OSS. But, OSS didn't disappear and is still developed and used both on Linux and other UNIXes.

In the early parts of this decade, the Linux audio developer community started discussing techniques for connecting audio applications together, something that is not supported directly by ALSA. At roughly the same time, Davis started working on the Ardour digital audio workstation, which led to JACK. The audio handling engine from Ardour was turned into JACK, which is an "audio connection kit" that works on most operating systems. JACK is mostly concerned with the low-latency requirements of professional audio and music creation, rather than the needs of desktop users.

Since that time, the kernel has made strides in supporting realtime scheduling that can be used by JACK and others to provide skip-free audio performance, but much of that work is not available to users. Access to realtime scheduling is tightly controlled, so there is a significant amount of per-system configuration that must be done to access this functionality. Most distributions do not provide a means for regular users to enable realtime scheduling for audio applications, so most users are not benefiting from those changes.

In the mid-2000s, Poettering started work on the PulseAudio server, KDE stopped using the aRts sound server, GStreamer emerged as a means for intra-application audio streaming, and so on. Desktops wanted "simple" audio access APIs and created things like Phonon and libsydney, but meanwhile JACK was the only way to access Firewire audio. All of that led to great confusion for Linux audio users, Davis said.

Audio application models

At the bottom, audio hardware works in a very simple manner. For record (or capture), there is a circular buffer in memory to which the hardware writes, and from which the software reads. Playback is just the reverse. In both cases, user space can add buffering on top of the circular buffer used by the hardware, which is useful for some purposes, and not for others.

There are two separate models that can be used between the software and the hardware. In a "push" model, the application decides when to read or write data and how much, while the "pull" model reverses that, requiring the hardware to determine when and how much I/O needs to be done. Supporting a push model requires buffering in the system to smooth over arbitrary application behavior. The pull model requires an application that can meet deadlines imposed by the hardware.

Davis maintains that supporting push functionality on top of pull is easy, just by adding buffering and an API. But supporting pull on top of push is difficult and tends to perform poorly. So, audio support needs to be based on the pull model at the low levels, with a push-based API added in on top, he said.

Audio and video have much in common

OSS is based around the standard POSIX system calls, such as open(), read(), write(), mmap(), etc., while ALSA (which supports those same calls) is generally accessed through libasound, which has a "huge set of functions". Those functions provide ways to control hardware and software configuration along with a large number of commands to support various application styles.

In many ways, audio is like video, Davis said. Both generate a "human sensory experience" by rescanning a data buffer and "rendering" it to the output device. There are differences as well, mostly in refresh rates and the effect of missing refresh deadlines. Unlike audio, video data doesn't change that frequently when someone is just running a GUI—unless they are playing back a video. Missed video deadlines are often imperceptible, which is generally not true for audio.

So, Davis asked, does anyone seriously propose that video/graphics applications should talk to the hardware directly via open/read/write/etc.? For graphics, that has been mediated by a server or server-like API for many years. Audio should be the same way, even though some disagree, "but they are wrong", he said.

The problem with UNIX

The standard UNIX methods of device handling, using open/read/write/etc., are not necessarily suitable interfaces for interacting with realtime hardware. Davis noted that he has been using UNIX for 25 years and loves it, but that the driver API lacks some important pieces for handling audio (and video). Both temporal and data format semantics are not part of that API, but are necessary for handling that audio/video data. The standard interfaces can be used, but don't promote a pull-based application design.

What is needed is a "server-esque architecture" and API that can explicitly handle data format, routing, latency inquiries, and synchronization. That server would mediate all device interaction, and would live in user space. The API would not require that various services be put into the kernel. Applications would have to stop believing that they can and should directly control the hardware.

The OSS API must die

The OSS API requires any services (like data format conversion, routing, etc.) be implemented in the kernel. It also encourages applications to do things that do not work well with other applications that are also trying to do some kind of audio task. OSS applications are written such that they believe they completely control the hardware.

Because of that, Davis was quite clear that the "OSS API must die". He noted that Fedora no longer supports OSS and was hopeful that other distributions would follow that lead.

When ALSA was adopted, there might have been an opportunity to get rid of OSS, but, at the time, there were a number of reasons not to do that, Davis said. Backward compatibility with OSS was felt to be important, and there was concern that doing realtime processing in user space was not going to be possible—which turned out to be wrong. He noted that even today there is nothing stopping users or distributors from installing OSS, nor anything stopping developers from writing OSS applications.

Looking at OS X and Windows audio

Apple took a completely different approach when they redesigned the audio API for Mac OS X. Mac OS 9 had a "crude audio architecture" that was completely replaced in OS X. No backward compatibility was supported and developers were just told to rewrite their applications. So, the CoreAudio component provides a single API that can support users on the desktop as well as professional audio applications.

On the other side of the coin, Windows has had three separate audio interfaces along the way. Each maintained backward compatibility at the API level, so that application developers did not need to change their code, though driver writers were required to. Windows has taken much longer to get low latency audio than either Linux or Mac OS X.

The clear implication is that backward compatibility tends to slow things down, which may not be a big surprise.

JACK and PulseAudio: are both needed?

JACK and PulseAudio currently serve different needs, but, according to Davis, there is hope that there could be convergence between them down the road. JACK is primarily concerned with low latency, while PulseAudio is targeted at the desktop, where application compatibility and power consumption are two of the highest priorities.

Both are certainly needed right now, as JACK conflicts with the application design of many desktop applications, while PulseAudio is not able to support professional audio applications. Even if an interface were designed to handle all of the requirements that are currently filled by JACK and PulseAudio, Davis wondered if there were a way to force the adoption of a new API. Distributions dropping support for OSS may provide the "stick" to move application developers away from that interface, but could something similar be done for a new API in the future?

If not, there are some real questions about how to improve the Linux audio infrastructure, Davis said. The continued existence of both JACK and PulseAudio, along with supporting older APIs, just leads to "continued confusion" about what the right way to do audio on Linux really is. He believes a unified API is possible from a technical perspective—Apple's CoreAudio is a good example—but it can only happen with "political and social manipulation".

Poettering: The state of Linux audio

The focus of Poettering's talk was desktop audio, rather than embedded or professional audio applications. He started by looking at what had changed since last year's LPC, noting that EsounD and OSS were officially gone ("RIP"), at least in Fedora. OSS can still be enabled in Fedora, but it was a "great achievement" to have it removed, he said.

There were only bugs reported against three applications because of the OSS removal, VMware and quake2 amongst them. He said that there "weren't many complaints", but an audience member noted the "12,000 screaming users" of VMware as a significant problem. Poettering shrugged that off, saying that he encouraged other distributions to follow suit.

Confusion at last year's LPC led him to create the "Linux Audio API Guide", which has helped clarify the situation, though there were complaints about what he said about KDE and OSS.

Coming in Fedora 12, and in other distributions at "roughly the same time", is using realtime scheduling by default on the desktop for audio applications. There is a new mechanism to hand out realtime priority (RealtimeKit) that will prevent buggy or malicious applications from monopolizing the CPU—essentially causing a denial of service. The desktop now makes use of the high-resolution timers, because they "really needed to get better than 1/HZ resolution" for audio applications.

Support for buffers of up to 2 seconds has been added. ALSA used to restrict the buffer size to 64K, which equates to 70ms 370ms of CD quality audio. Allowing bigger buffers is "the best thing you can do for power consumption" as well as dropouts, he said.

Several things were moved into the audio server, including timer-based audio scheduling which allows the server to "make decisions with respect to latency and interrupt rates". A new mixer abstraction was added, even though there are four existing already in ALSA. Those were very hardware specific, Poettering said, while the new one is a very basic abstraction.

Audio hardware has acquired udev integration over the last year, and there is now "Bluetooth audio that actually works". Poettering also noted that audio often didn't work "out of the box" because there was no mixer information available for the hardware. Since last year, an ALSA mixer initialization database has been created and populated: "It's pretty complete", he said.

Challenges for the next year

There were a number of issues with the current sound drivers that Poettering listed as needing attention in the coming year. Currently, for power saving purposes, PulseAudio shuts down devices two seconds after they become idle. That can lead to problems with drivers that make noise when they are opened or closed.

In addition, there are areas where the drivers do not report correct information to the system. Decibel range of the device is one of those, along with the device strings that are either broken or missing in many drivers, which makes it difficult to automatically discover the hardware. The various mixer element names are often wrong as well; in the past it "usually didn't matter much", but it is becoming increasingly important for those elements to be consistently named by drivers. Some drivers are missing from the mixer initialization database, which should be fixed as well.

The negotiation logic for sample rates, data formats, and so on are not standardized. The order in which those parameters are changed can be interpreted differently by each driver which leads to problems at the higher levels, he said. There are also problems with timing for synchronization between audio and video that need to be addressed at the driver level.

Poettering also had a whole slew of changes that need to be made to the ALSA API so that PulseAudio (and others) can get more information about the hardware. Things like the routing and mixer element mappings as well as jack status (and any re-routing that is done on jack insertion) and data transfer parameters such as the timing and the granularity of transfers. Many of the current assumptions are based on consumer-grade hardware which doesn't work for professional or embedded hardware, he said. It would be "great if ALSA could give us a hint how stuff is connected".

There is also a need to synchronize multiple PCM clocks within a device, along with adding atomic mixer updates that sync to the PCM clock. Latency control, better channel mapping, atomic status updates, and HDMI negotiation are all on his list as well.

Further out, there are a number of additional problems to be solved. Codec pass-through—sending unaltered codec data, such as SPDIF, HDMI, or A2DP, to the device—is "very messy" and no one has figured out how to handle synchronization issues with that. There is a need for a simpler, higher-level PCM API, Poettering said, so that applications can use the pull model, rather than being forced into the push model.

Another area that needs work is handling 20 second buffering. There are a whole new set of problems that come with that change. As an example, Poettering pointed out the problems that can occur if the user changes some setting after that much audio data has been buffered. There need to be ways to revoke the data that has been buffered or there will be up to 20 second lags between user action and changes to the audio.

Conclusion

Both presentations gave a clear sense that things are getting better in the Linux audio space, though perhaps not with the speed that users would like to see. Progress has clearly been made and there is a roadmap for the near future. Whether Davis's vision of a unified API for Linux audio can be realized remains to be seen, but there are lots of smart hackers working on Linux audio. Sooner or later, the "one true Linux audio API" may come to pass.

Comments (32 posted)

System Applications

Audio Projects

New Music Player Daemon releases

Comments (none posted)

PulseAudio 0.9.19 released

Comments (none posted)

Database Software

Elixir 0.7.0 released

Full Story (comments: none)

Ingres Database 9.3 released

Full Story (comments: 30)

PostgreSQL Weekly News

Full Story (comments: none)

Interoperability

Samba announces four security releases

Comments (none posted)

Networking Tools

OpenSSH 5.3 released

Full Story (comments: none)

Web Site Development

Grok 1.0 released

Full Story (comments: none)

TurboGears 1.1 final released

Full Story (comments: none)

Miscellaneous

Orocos Bayesian Filtering Library 0.7.0 released

Comments (none posted)

Desktop Applications

Audio Applications

Amarok 2.2 "Sunjammer" released (KDE.News)

Comments (none posted)

Data Visualization

python-graph 1.6.2 released

Full Story (comments: none)

Desktop Environments

Back on GNOME Bugzilla: weekly-bug-summary, describeuser

Full Story (comments: none)

GNOME Software Announcements

• CrunchyFrog 0.4.1 (new features and bug fixes)
• Empathy 2.28.0.1 (new features, bug fixes and translation work)
• gedit 2.28 (stable branch)
• GLib 2.22.2 (bug fixes and translation work)
• GNOME DVB Daemon 0.1.10 (new features and translation work)
• GNOME DVB Daemon 0.1.11 (bug fixes)
• GTK+ 2.18.1 (new features, bug fixes and translation work)
• GTK+ 2.18.2 (bug fixes and translation work)
• gtkmm 2.18.2 (bug fixes)
• gwget 1.0.3 (bug fixes and build change)
• libvtemm 0.20.6 and 0.22.0 (build change and documentation work)
• pygtksourceview 2.28 (stable branch)
• Rygel 0.4.2 (bug fixes)
• Thumbak 0.5 (initial release)
• tracker 0.7.1 (new features and bug fixes)

Comments (none posted)

KDE 4.3.2 is available (KDEDot)

Comments (none posted)

KDE Software Announcements

• Amarok 2.2 (new features)
• Bangarang 1.0 alpha A (initial release)
• Brasillinux Desktop RC6 (unspecified)
• digiKam 1.0.0-beta5 (beta release for KDE4)
• Dikt 1j (new feature)
• Dikt 1.0 (unspecified)
• kcm_touchpad 0.1.0 (initial release)
• XMS 1.0 (unspecified)

Comments (none posted)

Xorg Software Announcements

• compositeproto 0.4.1 (bug fixes)
• dmxproto 2.3 (new macro requirement)
• fixesproto 4.1.1 (bug fixes and documentation work)
• fontcacheproto 0.1.3 (bug fixes and code cleanup)
• inputproto 2.0 (stable release)
• libdmx 1.1.0 /a> (new macro requirement)
• libX11 1.3 (bug fixes)
• libXdamage 1.1.2 (bug fixes and code cleanup)
• libXext 1.1 (new macro requirement)
• libXi 1.3 (bug fixes and documentation work)
• libXinerama 1.1 (new macro requirement)
• libxkbfile 1.0.6 (bug fixes and documentation work)
• libXrender 0.9.5 (bug fixes and documentation work)
• libXtst 1.1.0 (new macro requirement)
• libXv 1.0.5 (bug fixes and documentation work)
• libXvMC 1.0.5 (code cleanup and documentation work)
• libXxf86dga 1.1 (new macro requirement)
• libXxf86dga 1.1.1 (bug fix)
• libXxf86vm 1.1.0 (new macro requirement)
• luit 1.0.4 (new features, bug fixes and documentation work)
• mkcomposecache 1.2.1 (new macro requirement and code cleanup)
• randrproto 1.3.1 (new features and documentation work)
• recordproto 1.14 (new macro requirement)
• windowswmproto 1.0.4 (bug fixes and new macro requirement)
• xeyes 1.0.991 (bug fixes, code cleanup and documentation work)
• xf86dgaproto 2.1 (new macro requirement)
• xf86-input-joystick-1.4.99.2 (bug fixes and code cleanup)
• xf86-input-keyboard 1.4.0 (new macro requirement and code cleanup)
• xf86-input-mouse 1.5.0 (bug fix)
• xf86vidmodeproto 2.3 (new macro requirement)
• xineramaproto 1.2 (new macro requirement)
• xkbcomp 1.1.1 (new feature)
• xmodmap 1.0.4 (new features, bug fixes and documentation work)
• xorg-server 1.6.4.901 (bug fix)
• xorg-server 1.7.0 (new features and bug fixes)
• xscope 1.2 (grand unification release, new features and bug fixes)

Comments (none posted)

Desktop Publishing

AsciiDoc 8.5.0 released

Full Story (comments: none)

Encryption Software

python-gnupg 0.2.2 released

Full Story (comments: none)

Financial Applications

Gnucash 2.3.7 released

Full Story (comments: none)

SQL-Ledger 2.8.25 released

Comments (none posted)

Games

Humerus 2.1 released

Full Story (comments: none)

Mail Clients

Stresstesting IMAP clients (Hackvalue)

Comments (46 posted)

Multimedia

liboggz 1.0.1 released

Full Story (comments: none)

Music Applications

csound ifn parser and tools 1.04 released

Full Story (comments: none)

Office Suites

OpenOffice.org Newsletter

Full Story (comments: none)

Miscellaneous

BleachBit 0.6.5 released

Full Story (comments: none)

Languages and Tools

C

GCC Link Time Optimization branch merged

Comments (2 posted)

GCC 4.5 Status Report

Full Story (comments: none)

Caml

Caml Weekly News

Full Story (comments: none)

Perl

Perl 5.11.0 now available

Full Story (comments: 5)

Python

doit 0.4.0 released

Full Story (comments: none)

M2Crypto 0.20.2 released

Full Story (comments: none)

Python 2.6.3 released

Full Story (comments: 2)

stdeb 0.3.2 and 0.4.1

Full Story (comments: none)

Libraries

Fast decimal arithmetic module released

Full Story (comments: none)

Page editor: Forrest Cook

Announcements

Non-Commercial announcements

FSF offers "GNU Bucks" for finding nonfree works in free distributions

Full Story (comments: 92)

FSFE to EC: Don’t waste an opportunity with a hasty deal

Comments (none posted)

OW2 Consortium and Open Solutions Alliance announce merger

Full Story (comments: none)

Commercial announcements

Netgear's open-source router

Comments (39 posted)

Welte: Netgear trying to fool their users with "Open Source Router"

Comments (18 posted)

Legal Announcements

FSF files brief in Bilski case

Full Story (comments: none)

Red Hat Asks Supreme Ct. to Exclude Software From Patentability (Groklaw)

Comments (26 posted)

Articles of interest

Garmin Takes a New Tack With Linux-Based Nav Phone (LinuxInsider)

Comments (22 posted)

Laptops for all (Economist)

Comments (none posted)

The OpenBlockS 600 is a Linux server that fits in your palm (ComputerWorld)

Comments (14 posted)

Open Source Initiative loses corporate status

Comments (25 posted)

Linux saves Aussie electrical grid (the Inquirer)

Comments (21 posted)

New Books

Programming in Python 3 (Second Edition) released

Full Story (comments: none)

Resources

FOSS: War is over (if you want it) (451 CAOS)

Comments (3 posted)

Linux Gazette #167 is out!

Full Story (comments: none)

Contests and Awards

EFF announces awards

Full Story (comments: none)

Call for award nominations: Government Open Source Conf.

Full Story (comments: none)

Calls for Presentations

CFP reminder: GROW'10 - 2nd Workshop on GCC research opportunities

Full Story (comments: none)

Upcoming Events

Akademy 2010 dates announced (KDEDot)

Comments (none posted)

Linux Foundation announces End User Summit

Comments (none posted)

End Users Meet Year End (Linux Journal)

Comments (none posted)

Come to Italy for the OpenOffice.org Annual Conference

Full Story (comments: none)

SciPy India conference announced

Full Story (comments: none)

YAPC::Brasil 2009 announced (use Perl)

Comments (none posted)

Events: October 15, 2009 to December 14, 2009

If your event does not appear here, please tell us about it.

Page editor: Forrest Cook