LWN.net Weekly Edition for August 13, 2009

The unending story of cdrtools

But these problems are never solved, it seems. In June, Jörg Schilling, the author of cdrtools, wandered into the fedora-legal list with a request for Fedora to resume shipping the "original, legal" cdrtools software. After a discussion of the type that typically follows Jörg around, Tom "spot" Callaway stepped in with a definitive response (short version: "no") which pretty much brought the discussion to an end.

Life got quiet again until early July, when Luis Medinas suggested that openSUSE might want to switch back to cdrtools. That was Jörg's cue to make one of his predictable appearances, inspiring an even longer and stronger version of the kind of discussion that tends to follow him around. This time Jörg made a direct lawsuit threat against SUSE, but showed his forgiving side too:

One might well wonder about the reversal of roles here; now it's Jörg who is complaining about the legality of cdrkit. His complaints have been posted to the web. They include the fact that the "wodim" CD recorder packaged in cdrkit is installed as "cdrecord" (a GPL violation, he says), the lack of detailed change information within the source files, the failure to print a copyright notice "as intended by the original author," an (unspecified) failure to distribute "complete" source, and a couple of alleged violations of German copyright law (which, it seems, forbids any change which Jörg disapproves of). All told, it is a long series of complaints resulting from a simple fork of a GPL-licensed program.

Most observers do not take these claims seriously. The complaint about the cdrecord binary is (somehow) based on the preamble of the GPL - which is not part of the binding terms. Section 2a of the GPL does require dated notifications of changes, but it's a rare project which carries those notifications within the source files themselves, as Jörg is demanding. The complaint about copyright notices is interesting. Cdrecord has traditionally been a verbose utility, and that verbosity has extended to Jörg's thoughts about Linux distributors and kernel developers. For example, version 2.01.01a01 (from 2004) would print things like:

    Warning: Running on Linux-2.6
    There are unsettled issues with Linux-2.5 and newer.
    If you have unexpected problems, please try Linux-2.4 or Solaris.

    SuSE Linux is known to ship bastardized and defective versions of cdrecord.
    SuSE is unwilling to cooperate with the authors.
    If you like to have a working version of cdrtools, get the
    original source from ftp://ftp.berlios.de/pub/cdrecord/

(The current version, 2.01.01a63, has lost some of that language). The removal of some of that verbosity is what he is complaining about. But GPL section 2c only requires the printing of "an appropriate copyright notice" (not any specific notice), and it only applies to programs which read commands interactively, which wodim does not do. So this claim, like the others, has failed to create widespread worry.

In short, many in the community seem to see Jörg as a sort of comic figure, but that should not be allowed to obscure an important fact: there are some points worth noting behind his complaints. These include:

• Jörg alleges that openSUSE is shipping two related, legally problematic packages: vcdimager and libcdio. Both packages are GPL-licensed and hosted with the GNU project, but other distributions have recognized problems with them; Debian has shipped a patched version since 2004, and Fedora users must get it from an external repository. Fedora also does not ship libcdio, which is alleged to have suffered a license change which is not acceptable to the original author of the code.

• Cdrkit is nearly unmaintained. The mailing list for changes is a quiet and lonely place. Jörg states that hundreds of unfixed bugs have been introduced into cdrkit. The reality, as shown by distribution bug trackers, is a bit less spectacular, but it is true that some bugs exist which might not be present in cdrecord - which is actively maintained by Jörg.

The first issue needs to be taken seriously; it is never a good idea to distribute code with problematic or disputed licensing. The fix here is relatively straightforward: stop distributing that code if the license cannot be verified, and, possibly, reimplement it (as Sun is said to have done with libcdio).

The second may be harder. The freedom to fork a package out from under an uncooperative maintainer is one of the fundamental features of free software. But forking is expensive; it only works if somebody else does the work which has been pulled away from that maintainer. An unmaintained fork is just more dead code. If cdrkit reaches a point where it fails to work for users, distributors will be left with an unpalatable choice: continue to ship unmaintained code, or go back to the original, with its difficult maintainer and incompatible licensing. It would be much nicer to find somebody willing to put some time into this important tool. CD recording is a detailed and tricky task, but we have plenty of people in our community with the necessary skills to work in that area.

Comments (46 posted)

KDE struggles with feature requests

Sometimes developers have a prickly relationship with their users. Users may have unrealistic, or overly demanding, requests that can be difficult to respond to. The most vocal of these users are often unwilling to take "no"—or even "not yet"—for an answer. Some KDE developers are currently struggling with that problem, and trying to find ways to smooth the dialog between users and developers.

In a posting to the kde-devel mailing list, Pau Garcia i Quiles wondered where KDE 3 features that were missing from KDE 4 should be collected. He noted that there are various places users were complaining about these missing features (including an openSUSE web page that collects them), but no central location for KDE to track such things. His suggestion: "Can we start something like that in UserBase, for people to tell us what they miss in KDE4 from KDE3? Or have a special category in Bugzilla?"

That set off a bit of a rant from Aaron J. Seigo about user complaints:

Seigo also noted that the openSUSE list doesn't "mention _at all_ the actually useful features that are missing", and, that, when he commented on that wish list item, he "got yelled at by two different people on the report, completely without cause". Frustration is obvious in his posting, and he noted that it was probably not quite the response Garcia expected, but he wanted to make it clear that the current options were not working:

Anne Wilson noted that the users Seigo is referring to are just a "*very* vocal minority" that "can only be ignored". She is concerned with the users who are trying to make a difference with their bug reports and feature requests, only to be treated as if they are part of that loud minority. She disagreed with Seigo's suggestion that users should either write—or pay for—the code, or just be patient:

But, Seigo sees things somewhat differently. He points to this vocal minority as part of the reason that KDE projects aren't "paying much attention to feature requests made on bugs.kde.org". Once again, he places the blame largely at the feet of the user community:

There was some discussion of changing various bug tags, particularly WONTFIX, as it is regularly misinterpreted, to try to alleviate the problem. That is unlikely to mollify the users who are most vocal, though. Trying to ensure that features and bugs closed as WONTFIX get some kind of explanation will probably help with, but not eliminate, the problem, as well. Andreas Pakulat points out that it is a social problem: "people are getting used to be able to shout, rant and moan on the net without ever being held responsible for the possible damage they do with that".

One idea that seems to be gaining some traction is to use KDE Brainstorm, which was suggested as a place to gather features by Stefan Majewsky. Aside from some usability issues that seem like they could be dealt with relatively easily, Brainstorm provides a means to discuss new (or missing KDE 3) features, while allowing users to vote on those they find most important. Seigo sees it as a starting point:

But the problem is more than just work flow. From the postings in the thread, some KDE developers are finding it difficult to work with the user community, largely because of the behavior of a few of its members. Parker Coates is unconvinced that a tool-driven process will eliminate the problem:

Due to the very vocal, and largely negative, reaction to the release of KDE 4 more than a year and a half ago, there is still a great deal of frustration within the project—for both users and developers. While there are certainly some important points in the developers' messages, the tone is such that they also could be taken as an indictment of all users—something that is clearly not intended.

This is a problem that certainly isn't limited to KDE, as other projects have or will run into the same kinds of problems. There is a delicate balance between ignoring the "vocal minority" and ignoring the user community as a whole. The latter could easily lead a project to completely lose touch with the needs of its users, to the point where those users end up walking away. That is an outcome both sides want to—and should—avoid. Finding better ways to handle feature requests, while avoiding the conflicts with the few who will not be civil, is a good step on that path.

Comments (74 posted)

Ubuntu's multisearch surprise

The bug report filed on July 21 had to do with broken functionality. It seems that, when using the version of Firefox distributed with the third Karmic alpha release, typing a search string into the "awesome bar" no longer takes the user directly to the first search result from Google. Instead, users end up at a Google "search partner" page listing the results and, of course, advertisements. Other quick searches, including stock quotes and currency conversions, also break. A related change is that opening a new tab now brings up an Ubuntu search page instead of a blank page - a change that some users find jarring.

It turns out that Ubuntu has placed a new Firefox extension, called "multisearch," into the Karmic alpha release. In essence, multisearch rewires the various search mechanisms built into the browser, causing them all to pass through Ubuntu's partner page. It can be disabled by going into the "Tools->Add-ons" menu, but, by default, it is installed and active on all systems.

So why was this done? Rick Spencer, Ubuntu's desktop engineering manager, explained the reasoning in a fair amount of detail. The "new tab" change is an attempt to improve the user experience - something that Mozilla developers are working on as well. The search change lets Ubuntu know which search mechanisms are being used most; beyond that, he said:

Ubuntu users are not necessarily opposed to the idea of revenue going toward the development of their distribution; it's a "feature" they can support. Many of them are, however, rather less thrilled about their search data being used to that end. Rick's explanation - "it's simply the same data that is already sent to Google and Mozilla: the requested search, and the channel for the search" - does not appear to have made anybody feel any better. As might be imagined, some of the more vocal users are throwing around words like "spyware" and "privacy violations." But even calmer voices are concerned that this "feature" was silently added to their systems, that it is not something they wish to have around, and that there has been little talk of privacy protections for the accumulated data.

Apologies from the Ubuntu side have been few and far between. Ubuntu Mozilla maintainer Alexander Sack justifies the change this way:

Of course, one should bear in mind that default Ubuntu installations are "opted in" to the ubuntu-desktop metapackage; very few users will have deliberately made that choice.

The other thing to bear in mind is that this feature appears in an alpha release - and that users did indeed make a deliberate choice to install that release. It's not uncommon to find unpleasant surprises in alpha-quality distributions, even if it's a bit more uncommon for those surprises to have been introduced deliberately. Alexander says that multisearch "is not intended to stay forever - at least not in its current form." One can interpret that to mean that some of the more annoying failures will be fixed. It's possible that the entire thing will be taken out before the end of the alpha-test period. But nobody from Canonical is saying that now.

A great deal of trust is placed in Linux distributors; they have the ability to inflict all kinds of unpleasant behavior on their users. Distributors seen to abuse that trust are not likely to retain their users for all that long, though. The beauty of free software shows through in a few ways here: undesirable behavior is very hard to hide, it is quite easy to remove, and, if all else fails, one can switch to a different distribution with minimal pain. Ubuntu is probably not losing any users over this episode - yet. But any user of this distribution who is concerned about this behavior may want to watch closely to see what decisions are made between now and the final Karmic Koala release.

(Update: multisearch was removed from Ubuntu on August 11.)

Comments (115 posted)

Page editor: Jonathan Corbet

Security

OpenVAS replacing Nessus in Debian

For many years, the Nessus network vulnerability scanner was a tool in the toolbox of most free-software-oriented administrators. It provided a very useful, GPL-licensed scanner to detect various network vulnerabilities, misconfigurations, and other types of security problems in the network. But, starting in late 2005 that all began to change, when Nessus 3.0 switched licenses, so folks looking for a free software network scanner had to turn elsewhere.

There have been a number of attempts to fork the last GPL version of the Nessus software (2.2), but the most successful to date has been the Open Vulnerability Assessment System (or OpenVAS). The forked scanner has been making great strides to the point where Debian's Nessus maintainer, Javier Fernández-Sanguino Peña, asked that Nessus be removed from the unstable branch in favor of OpenVAS. In his message, he noted:

There are really two parts to a vulnerability scanner, a core scanner and a set of plugins that implement network vulnerability tests (or NVTs). Much like virus scanners, NVTs are constantly being added and updated, and are available via network feeds. For a vulnerability scanner to be really usable, NVTs must be available for older vulnerabilities as well as being developed for new ones as they come along. In the thread on the debian-security mailing list, Tim Brown reports that OpenVAS has reached that point:

Where Debian goes, other distributions are likely to follow, so we may see Nessus removed in favor of OpenVAS elsewhere as well. It is unfortunate that Tenable, the company behind Nessus, was unable to find a way to continue with a GPL-licensed Nessus, but the rise of OpenVAS shows the power of code that is available under a free software license. That is not to say that Tenable did anything wrong, it was their code and thus their choice; in fact, the community should be grateful that they provided the core of a nice tool for as long as they did. But, because the GPL allows forks like OpenVAS, Nessus users still had a free software path to follow once Tenable decided to go in a different direction.

The main stumbling block to getting to this point has been the NVTs released for Nessus. Those are governed by a separate license, that made it somewhat legally dubious, at best, to use them in OpenVAS. So, the OpenVAS developers had to tackle that problem themselves. Based on Brown's message, it would seem they have gotten most of the way there, and have an active community to continue that work into the future.

Comments (1 posted)

Brief items

You Deleted Your Cookies? Think Again (Wired)

Comments (18 posted)

New vulnerabilities

apr: arbitrary code execution

Comments (none posted)

camlimages: arbitrary code execution

Comments (3 posted)

fetchmail: SSL impersonation vulnerability

Comments (none posted)

java-1.6.0-openjdk: multiple vulnerabilities

Comments (none posted)

libvorbis: denial of service

Comments (none posted)

libxml: multiple vulnerabilities

Comments (none posted)

mantis: database credentials leak

Comments (none posted)

memcached: heap-based buffer overflow

Comments (none posted)

squid3: multiple denial of service vulnerabilities

Comments (none posted)

subversion: heap overflows

Comments (none posted)

wireshark: multiple vulnerabilities

Comments (none posted)

wordpress: remote admin password reset

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

Comments (none posted)

Kernel development news

Quotes of the week

Comments (6 posted)

In Brief

What Tux3 needs, it seems, is some new development energy. It could be an interesting project for developers who are wanting to get started in filesystem development.

Resource counters. The resource counter mechanism is built into control groups; it is intended for use by tools like the memory use controller. These counters contain, at their core, a (believe it or not) counter value which tracks the current usage of a resource by a given control group. This counter has run into the same problem which afflicts any frequently-changed global variable: it scales poorly due to cache line bouncing. The usage of some resources (pages of memory, for example) can change frequently, causing the associated counter to be a drag on the system as a whole.

Balbir Singh's scalable resource counters patch aims to fix that situation. With this patch, the single "usage" counter becomes an array of per-CPU counters. Since each processor works with its own copy of the counter, there is no more cache line bouncing and things run faster. The down side is that the count becomes approximate. The per-CPU counters are summed occasionally to keep everything roughly in sync, but keeping exact counts would take away much of the scalability that this patch was meant to provide. The good news is that exact counts are not really needed anyway; as long as the counter reflects something close enough to reality, the system will work essentially as it did before - only a little more quickly.

Inline spinlocks. Once upon a time, spinlocks were implemented with a series of inline functions, on the notion that such a performance-critical primitive would need to be as fast as possible. That changed in 2004, when spinlocks were turned into normal functions. The function call overhead hurt a bit, but moving spinlocks out-of-line made the kernel considerably smaller, which has performance benefits of its own. And that's how spinlocks have been ever since.

The pendulum may be about to swing the other way again, though, at least for the S390 architecture. Heiko Carstens noted that function calls on this architecture are quite expensive. He put together an inline spinlocks patch and measured performance improvements of 1-5%. So he would like to put this patch into the mainline, along with a configuration option allowing each architecture to choose the best way to implement spinlocks. So far, there has been little commentary for or against this idea.

Const seq_operations. James Morris has posted a patch making seq_operations structures constant throughout the kernel. These structures are almost always populated at compile time and never need to change; allowing the function pointers therein to be overwritten can only be useful to those who would like to subvert the kernel. A number of core VFS operations structures have been made const over the years, but seq_operations has not been addressed until now. James says: "This is derived from the grsecurity patch, although generated from scratch because it's simpler than extracting the changes from there."

data=guarded. Back in the middle of the discussion of crash robustness and latency in the ext3 filesystem, Chris Mason came forward with a proposal for a data=guarded mode, which would delay metadata updates when files change size to prevent the disclosure of unrelated information. Since then, the data=guarded patch has disappeared from view. In response to a query from Frans Pop, Chris confirmed that he is still working on that code, and that he plans to get it merged for 2.6.32.

Among those welcoming the news was Andi Kleen, who remarked: "data=writeback already cost me a few files after crashes here." The data=guarded mode may not help with that particular problem, though: it is really meant to combine the security benefits of data=ordered (not disclosing random data, in particular) with the performance benefits of data=writeback. The worst data-loss problems should have already been addressed by the robustness fixes that went into ext3 for 2.6.30.

Comments (4 posted)

Fun with tracepoints

LWN last looked at the tracepoint discussion in April. Since then, the disagreement has returned with little change. The catalyst this time was Mel Gorman's page allocator tracepoints patch, which further instruments the memory management layer. The mainline kernel already contains tracepoints for calls to functions like kmalloc(), kmem_cache_alloc(), and kfree(). Mel's patch adds tracepoints to the low-level page allocator, in places like free_pages_bulk(), __rmqueue_fallback(), and __free_pages(). These tracepoints give a view into how the page allocator is performing; they'll inform a suitably clueful user if fragmentation is growing or pages are being moved between processors. Also included is a postprocessing script which uses the tracepoint data to create a list of which processes on the system are putting the most stress on the memory management code.

As has happened before, Andrew Morton questioned the value of these tracepoints. He tends not to see the need for this sort of instrumentation, seeing it instead as debugging code which is generally useful to a single developer. Beyond that, Andrew asks, why can't the relevant information be added to /proc/vmstat, which is an established interface for the provision of memory management information to user space?

There are a couple of answers to that question. One is that /proc/vmstat has a number of limitations; it cannot be used, for example, to monitor the memory-management footprint of a specific set of processes. It is, in essence, pre-cooked information about memory management in the system as a whole; if a developer needs information which cannot be found there, that information will be almost impossible to get. Tracepoints, instead, provide much more specific information which can be filtered to give more precise views of the system. Mel bashed out one demonstration: a SystemTap script which uses the tracepoints to create a list of which processes are causing the most page allocations.

Ingo Molnar posted a lengthy set of examples of what could be done with tracepoints; some of these were later taken by Mel and incorporated into a document on simple tracepoint use. These examples merit a look; they show just how quickly and how far the instrumentation of the Linux kernel (and associated tools) have developed.

One of the key secrets for quick use of tracepoints is the perf tool which is shipped with the kernel as of 2.6.31-rc1. This tool was written as part of the performance monitoring subsystem; it can be used, for example, to run a program and report on the number of cache misses sustained during its execution. One of the features slipped into the performance counter subsystem was the ability to treat tracepoint events like performance counter events. One must set the CONFIG_EVENT_PROFILE configuration option; after that, perf can work with tracepoint events in exactly the same way it manages counter events.

With that in place, and a working perf binary, one can start by seeing which tracepoint events are available on the system:

    $ perf list
      ...
      ext4:ext4_sync_fs                        [Tracepoint event]
      kmem:kmalloc                             [Tracepoint event]
      kmem:kmem_cache_alloc                    [Tracepoint event]
      kmem:kmalloc_node                        [Tracepoint event]
      kmem:kmem_cache_alloc_node               [Tracepoint event]
      kmem:kfree                               [Tracepoint event]
      kmem:kmem_cache_free                     [Tracepoint event]
      ftrace:kmem_free                         [Tracepoint event]
      ...

How many kmalloc() calls are happening on a system? The question can be answered with:

    $ perf stat -a -e kmem:kmalloc sleep 10

     Performance counter stats for 'sleep 10':

           4119  kmem:kmalloc            

     10.001645968  seconds time elapsed

So your editor's mostly idle system was calling kmalloc() almost 420 times per second. The -a option gives whole-system results, but perf can also look at specific processes. Monitoring allocations during the building of the perf tool gives:

    $ perf stat -e kmem:kmalloc make
      ...
 Performance counter stats for 'make':

           5554  kmem:kmalloc            

  2.999255416  seconds time elapsed

More detail can be had be recording data and analyzing it afterward:

    $ perf record -c 1 -e kmem:kmalloc make
      ...
    $ perf report
    # Samples: 6689
    #
    # Overhead          Command                         Shared Object  Symbol
    # ........  ...............  ....................................  ......
    #
      19.43%             make  /lib64/libc-2.10.1.so                 [.] __getdents64
      12.32%               sh  /lib64/libc-2.10.1.so                 [.] __execve
      10.29%              gcc  /lib64/libc-2.10.1.so                 [.] __execve
       7.53%              cc1  /lib64/libc-2.10.1.so                 [.] __GI___libc_open
       5.02%              cc1  /lib64/libc-2.10.1.so                 [.] __execve
       4.41%               sh  /lib64/libc-2.10.1.so                 [.] __GI___libc_open
       3.45%               sh  /lib64/libc-2.10.1.so                 [.] fork
       3.27%               sh  /lib64/ld-2.10.1.so                   [.] __mmap
       3.11%               as  /lib64/libc-2.10.1.so                 [.] __execve
       2.92%             make  /lib64/libc-2.10.1.so                 [.] __GI___vfork
       2.65%              gcc  /lib64/libc-2.10.1.so                 [.] __GI___vfork

Conclusion: the largest source of kmalloc() calls in a simple compilation process is getdents(), called from make, followed by the execve() calls needed to run the compiler.

The perf tool can take things further; it can, for example, generate call graphs and disassemble the code around specific performance-relevant points. See Ingo's mail and Mel's document for more information. Even then, we're just talking about statistics on tracepoints; there is a lot more information available which can be used in postprocessing scripts or tools like SystemTap. Suffice to say that tracepoints open a lot of possibilities.

The obvious question is: was Andrew impressed by all this? Here's his answer:

He suggested that he would be happier if the new tracepoints could be used to phase out /proc/vmstat and /proc/meminfo; that way there would not be a steadily-increasing variety of memory management instrumentation methods. Removing those files is problematic for a couple of reasons, though. One is that they form part of the kernel ABI, which is not easily broken. It would be a multi-year process to move applications over to a different interface and be sure there were no more users of the /proc files. Beyond that, though, tracepoints are good for reporting events, but they are a bit less well-suited to reporting the current state of affairs. One can use a tracepoint to see page allocation events, but an interface like /proc/vmstat can be more straightforward if one simply wishes to know how many pages are free. There is space, in other words, for both styles of instrumentation.

As of this writing, nobody has made a final pronouncement on whether the new tracepoints will be merged. Andrew has made it clear, though, that, despite his concerns, he's not firmly opposing them. There is enough pressure to get better instrumentation into the kernel, and enough useful things to do with that instrumentation, that, one assumes, more of it will go into the mainline over time.

Comments (15 posted)

clone_with_pids()

As part of the changes to support application checkpoint and restart in the kernel, Sukadev Bhattiprolu has proposed a new system call: clone_with_pids(). When a process that was checkpointed gets restarted, having the same process id (PID) as it had when the checkpoint was done is important to some kinds of applications. Normally, the kernel assigns an unused PID when a new task is started (via clone()), but, for checkpointed processes, that could lead to processes' PIDs changing during their lifetime, which could be an undesirable side effect. So, Bhattiprolu is looking for a way to avoid that by allowing clone() callers to specify the PID—or PIDs for processes in nested namespaces—of the child.

The actual system call is fairly straightforward. It adds an additional pid_set parameter to clone(), to contain a list of process ids; pid_set has the obvious definition:

    struct pid_set {
	   int num_pids;
	   pid_t *pids;
    };

	pid_t pids[] = { 0, 77, 99 };
	struct pid_set pid_set;

	pid_set.num_pids = sizeof(pids) / sizeof(int);
	pid_set.pids = &pids;

	clone_with_pids(flags, stack, NULL, NULL, NULL, &pid_set);

The patchset assumes that being able to set PIDs is desirable, but Linus Torvalds was not particularly in favor of that approach when it was first discussed on linux-kernel back in March. His complaint was that there are far too many stateful attributes of processes to ever be able to handle checkpointing in the general case. His suggestion: "just teach the damn program you're checkpointing that pids will change, and admit to everybody that people who want to be checkpointed need to do work".

Others disagreed—no surprise—but it is unclear that Torvalds has changed his mind. He was also concerned about the security implications of processes being able to request PID assignments: "But it also sounds like a _wonderful_ attack vector against badly written user-land software that sends signals and has small races." That particular concern should be alleviated by the requirement that a process have the CAP_SYS_ADMIN capability (essentially root privileges) in order to use clone_with_pids().

Requiring root to handle restarts, which in practice means that root must manage the checkpoint process as well, makes checkpoint/restart less useful, overall. But there are a whole host of problems to solve before allowing users to arbitrarily checkpoint and restore from their own, quite possibly maliciously crafted, checkpoint images. Even with root handling the process, there are a number of interesting applications.

There is an additional wrinkle that Bhattiprolu notes in the patch. Currently, all of the available clone() flags are allocated. That doesn't affect clone_with_pids() directly, as the flags it needs are already present, but, when adding a system call, it is good to look to the future. To that end, there are two proposed implementations of a clone_extended() system call, which could be added instead of clone_with_pids(), that would allow for more clone() flags, while still supporting the restart case.

The first possibility is to turn the flags argument into a pointer to an array of flag entries, that would be treated like signal() sets, including operations to test, set, and clear flags a la sigsetops():

    typedef struct {
	    unsigned long flags[CLONE_FLAGS_WORDS];
    } clone_flags_t;

    int clone_extended(clone_flags_t *flags, void *child_stack, int *unused,
	    int *parent_tid, int *child_tid, struct pid_set *pid_set);

Another option is to split the flags into two parameters, keeping the current flags parameter as it is, and adding a new clone_info parameter that contains new flags along with the pid_set:

    struct clone_info {
	    int num_clone_high_words;
	    int *flags_high;
	    struct pid_set pid_set;
    }

    int clone_extended(int flags_low, void *child_stack, void *unused,
	    int *parent_tid, int *child_tid, struct clone_info *clone_info);

It is hard to imagine that copying a bit of data from user space will measurably impact a system call that is creating a process, though, so some derivative of the first option would seem to be the better choice. It's also a bit hard to see the need for more than 64 clone() flags, but if that is truly desired, something with a path for compatibility is needed.

There has been no objection to the implementation of clone_with_pids(), but there have been few comments overall. Pavel Machek wondered about the need for setting the PID of anything but the inner-most namespace, but Serge E. Hallyn noted that nested namespaces require that ability: "we might be restarting an app using a nested pid namespace, in which case restart would specify pids for 2 (or more) of the innermost containers".

Machek also thought there should be a documentation file that described the new system call, and Bhattiprolu agreed, but is waiting to see what kind of consensus on either clone_with_pids() or clone_extended() (and which of the two interfaces for the latter) would emerge. So far, no one has commented on that particular aspect.

This is version 4 of the patchset, and the history shows that earlier comments have been addressed. It is still at the RFC stage, or, as Bhattiprolu puts it: "Its mostly an exploratory patch seeking feedback on the interface". That feedback has yet to emerge, however, and one might wonder whether Torvalds will still object to the whole approach. It would seem, though, that there are too many important applications for checkpoint and restart—including process migration and the ability to upgrade kernels underneath long-running processes—for some kind of solution not to make its way into the kernel eventually.

Comments (8 posted)

Interrupt mitigation in the block layer

NAPI is well suited to network drivers, since high packet rates can lead to high interrupt rates, but it has not spread to other parts of the kernel, where interrupt rates are lower. That situation could change in 2.6.32, though, if Jens Axboe follows through with his plan to merge the new blk-iopoll infrastructure into the mainline. In short, blk-iopoll is NAPI for block devices; indeed, some of the core code was borrowed from the NAPI implementation.

Converting a block driver to the blk-iopoll is straightforward. Each interrupting device needs to have a struct blk_iopoll structure defined for it, presumably in the structure which describes the device within the driver. This structure should be initialized with:

    #include <linux/blk-iopoll.h>

    typedef int (blk_iopoll_fn)(struct blk_iopoll *, int);

    void blk_iopoll_init(struct blk_iopoll *iop, int weight, blk_iopoll_fn *poll_fn);

The weight value describes the relative importance of the device; a higher weight results in more requests being processed in each polling cycle. As with NAPI, there is no definitive guidance as to what weight should be; in Jens's initial patch, it is set to 32. The poll_fn() will be called when the block subsystem decides that it's time to poll for completed requests.

I/O polling for a device is controlled with:

    void blk_iopoll_enable(struct blk_iopoll *iop);
    void blk_iopoll_disable(struct blk_iopoll *iop);

A call to blk_iopoll_enable() must be made by the driver before any polling of the device will happen. Enabling polling allows that polling to occur, but does not cause it to happen. There is no point in polling a device which is not doing any work, so the block layer will not actually poll a given device until the driver informs it that there may be a reason to do so.

That normally happens when the device is actually interrupting. The driver can, in its interrupt handler, switch over to polling mode through a three-step process. The first is to check the global variable blk_iopoll_enabled; if it is zero, block I/O polling cannot be used. Assuming polling is enabled, the driver should prepare the blk_iopoll structure with:

    int blk_iopoll_sched_prep(struct blk_iopoll *iop);

In the first version of the patch, a return value of zero means that the preparation "failed," either because polling is disabled or because the device is already in polling mode. In future versions, the sense of the return value is likely to be inverted to the more standard "zero means success" mode. If blk_iopoll_sched_prep() succeeds, the driver can then call:

    void blk_iopoll_sched(struct blk_iopoll *iop);

At this point, polling mode has been entered; the driver need only disable interrupts from its device and return. The "disable interrupts" step should, of course, be done at the device itself; masking the IRQ line would be an antisocial act in a world where those lines are shared.

Later on, the block layer will call the poll_fn() which was provided to blk_iopoll_init(). The prototype for this function is:

        typedef int (blk_iopoll_fn)(struct blk_iopoll *iop, int budget);

The polling function is called (in software interrupt context) with iop being the related blk_iopoll structure, and budget being the maximum number of requests that the poll function should process. In normal usage, the driver's device-specific structure can be obtained from iop with container_of(). The budget value is just the weight that was specified back at initialization time.

The return value should be the number of requests actually processed. If the device consumes less than the given budget, it should turn off further polling with:

    void blk_iopoll_complete(struct blk_iopoll *iopoll);

Interrupts from the device should be re-enabled, since further polling will not happen. Note that the block layer assumes that a driver will not call blk_iopoll_complete() if it has consumed its full budget. If it's necessary to return to interrupt mode despite having exhausted the budget, the driver should either (1) use blk_iopoll_disable(), or (2) lie about the number of requests processed when returning from the polling function.

One might well wonder about the motivation behind all of this work. Block device interrupt handling has not traditionally been a performance bottleneck. The problem is the rapid improvement in solid-state storage devices. It is expected that, before too long, these devices will be operating in the range of 100,000 I/O operations per second - far beyond anything that rotating storage can do. When dealing with that many I/O operations, the kernel must take care to minimize the per-operation overhead in any way possible. As others have observed, the block layer needs to become more like the network layer, with the per-request cost squeezed to a bare minimum. The blk-iopoll code is a step in that direction.

How big a step? Jens has posted some preliminary numbers showing significant reductions in system time on a random-read disk benchmark. More testing will certainly be required; in particular, some developers are concerned about the possibility of increasing I/O latency. But the initial numbers suggest that this work has improved the efficiency of the block subsystem under load.

Comments (5 posted)

Patches and updates

Kernel trees

• John Kacur: linux-2.6.31-rc5-rt1.2 . (August 12, 2009)

Build system

• Andi Kleen: [0/5] kbuild: A couple of Kbuild fixes/improvements . (August 6, 2009)

Core kernel code

• KAMEZAWA Hiroyuki: kcore clean up and enhance . (August 6, 2009)

• Arnd Bergmann: Kill the BKL in compat ioctl handling . (August 6, 2009)

• Sukadev Bhattiprolu: clone_with_pids() system call . (August 7, 2009)

• Heiko Carstens: Allow inlined spinlocks (again) . (August 10, 2009)

• Stefani Seibold: kfifo reorganization for a generic interface . (August 10, 2009)

Development tools

• Amerigo Wang: V2 Implement crashkernel=auto . (August 6, 2009)

• Mel Gorman: Add some trace events for the page allocator v4 . (August 6, 2009)

• Li Zefan: [PATCH v2 0/3] tracing/filters: Support specifying filter hook to a TRACE_EVENT field . (August 7, 2009)

• Frederic Weisbecker: perfcounters: Support for ftrace event records sampling . (August 10, 2009)

• Jason Baron: add syscall tracepoints V3 . (August 11, 2009)

Device drivers

• Zhang Rui: introduce ALS sysfs class . (August 6, 2009)

• Zhang Rui: introduce ACPI ALS device driver . (August 6, 2009)

• Tejun Heo: chrdev: implement __[un]register_chrdev() . (August 6, 2009)

• Rafael J. Wysocki: PM: Introduce core framework for run-time PM of I/O devices (rev. 13) . (August 7, 2009)

• Pavel Machek: HTC Dream for staging: add support for input on GPIO pins . (August 10, 2009)

• Mark Brown: backlight: Add WM831x backlight driver . (August 10, 2009)

• Sascha Hauer: Freescale MC13783 PMIC support . (August 11, 2009)

• Dave Airlie: vga: implements VGA arbitration on Linux . (August 11, 2009)

Filesystems and block I/O

• Sage Weil: ceph distributed file system client . (August 6, 2009)

• Jens Axboe: blk-iopoll, a polled completion API for block devices . (August 10, 2009)

• Christoph Hellwig: cleanup blockdev_direct_IO locking . (August 10, 2009)

Janitorial

• Jonathan Corbet: pm_qos: BKL removal and cleanup . (August 7, 2009)

Memory management

• Jan Kara: [RFC] Fix page_mkwrite for blocksize < pagesize . (August 11, 2009)

• Eric B Munson: Add pseudo-anonymous huge page mappings . (August 12, 2009)

Networking

• Dmitry Eremin-Solenikov: Software implementation of IEEE 802.15.4 . (August 10, 2009)

Architecture-specific

• Andreas Herrmann: [PATCH 0/8 v5] x86: Adapt CPU topology detection for AMD Magny-Cours . (August 7, 2009)

• Feng Tang: x86/RTC: introduce a new generic rtc_ops for x86 . (August 11, 2009)

Security-related

• James Morris: security: constify seq_operations . (August 11, 2009)

Virtualization and containers

• Joerg Roedel: Nested SVM cleanups v2 . (August 7, 2009)

• Michael S. Tsirkin: vhost_net: a kernel-level virtio server . (August 10, 2009)

• Balbir Singh: Help Resource Counters Scale Better (v3) . (August 11, 2009)

• Dan Smith: c/r: Add AF_UNIX support (v9) . (August 12, 2009)

Benchmarks and bugs

• Rafael J. Wysocki: 2.6.31-rc5-git5: Reported regressions from 2.6.30 . (August 10, 2009)

• Rafael J. Wysocki: 2.6.31-rc5-git5: Reported regressions 2.6.29 -> 2.6.30 . (August 10, 2009)

Page editor: Jonathan Corbet

Distributions

News and Editorials

SUSE Studio for Linux appliances

At the end of July, Novell launched SUSE Studio, which it calls a "simple and fast appliance builder". It provides a free and easy-to-use, web-based user interface to roll your own customized (SUSE) Linux distribution. The resulting image can be deployed on bare metal or as a virtual appliance on a hypervisor. Novell has also started a SUSE Appliance Program to support independent software vendors creating Linux-based software appliances.

First, why would one build such a "software appliance" with SUSE Studio? For companies, the advantages are obvious: an appliance tailored to a specific application set makes it easier to maintain and gives it a more predictable behavior. These companies can distribute one integrated package with the operating system and (server) software to their customers. But the same advantages hold for people who want to build a custom Linux distribution to distribute for free, who want do demo some application during a tradeshow, or who want a distribution with their own branding. SUSE Studio developer Cornelius Schumacher lists some concrete use cases:

SUSE Studio is in beta phase and the website says it's "currently available for a limited number of users", but anyone who sends their email address via the SUSE Studio login page will be put on the invitation list. Kindly asking one of the operators in the #susestudio IRC channel for an invite tends to work too, as your author found out. After receiving login information, signing in can be done with with OpenID among other options. SUSE Studio works in Firefox 2 and 3 on Linux, Mac OS X and Windows, and in Safari 3 on Mac OS X.

From template to installable image

After the first login, the system offers some templates to build the appliance upon. For the base system, the website gives the choice among openSUSE 11.1, SUSE Linux Enterprise 10 and SUSE Linux Enterprise 11. There are templates for JeOS (Just enough OS, a minimalist server appliance), Server, GNOME Desktop, KDE 3 or 4 Desktop, and Minimal X. The last one, which uses the IceWM window manager, is especially useful for making a kiosk-like application. There is also a choice between 32 or 64-bit architectures and a text box to give the appliance a descriptive name.

The user navigates through the rest of the steps by clicking on some tabs. In the Software tab, one can add or remove packages and repositories or upload their own rpm files. Some basic knowledge of the available repositories and software packages is needed, but the interface has a helpful search function. SUSE Studio is also integrated well with the openSUSE Build Service (which enables users to create packages for various Linux distributions from source). One can build packages in the openSUSE Build Service, import the repository into SUSE Studio and use the freshly built packages. For example, this makes it possible to create an openSUSE distribution with KDE 4.3, although by default KDE 4.1 is installed.

The "Configure" tab gives a slew of options to configure: locale settings, timezone, network, firewall, users and groups, the look and feel, the default runlevel, a custom license agreement, MySQL configuration, custom boot scripts, and so on. In the "Overlay files" tab one can add overlay files, if, for example, some custom scripts are needed or some configuration files in /etc have to be changed. Clicking on the "Build" tab gives the choice among a disk image, a live cd, a VMware image, or a Xen image. Support for creating images in OVF (Open Virtualization Format, an open standard for packaging and distributing virtual appliances), Microsoft's Hyper-V, and Amazon's EC2 AMI format (including deployment to EC2) is planned. The build process itself takes less than five minutes most of the time, even for a complete desktop system. The result can then be downloaded.

By looking at some details, one can see that SUSE Studio is well-engineered and has undergone a lot of usability tweaking. For example, in the "Software" tab there's a "Recommended" list of applications, which is different depending on the template the user has chosen. Moreover, at any time the current disk footprint of the appliance is shown at the left side of the web page, which is useful while building an image which has to fit on a CD or a USB thumb drive. The left side also shows helpful messages and tips. For example, while building a VMware image, it suggests installing the open-vm-tools package. And, after adding a package, one can view more details about the package, such as which dependencies are installed. If the firewall is enabled in the "Configuration" tab and it is not installed (for example while building upon a JeOS template), then an error message appears with a button to add SuSEfirewall2. There are a lot of these tips, which seem to come from the developers' experiences and feedback by users of the alpha version.

An appliance in your browser

One great feature is the SUSE Studio Testdrive. This allows the user to boot and test their appliance in a browser window without the need to download it first. After the build, clicking on the Testdrive button will launch the virtual machine. This runs on a KVM hypervisor on Novell's servers and exposes the virtual machine framebuffer via VNC to a Flash applet running in the user's browser. There are even buttons to switch to a different virtual console, to press ctrl-alt-del, to press ctrl-alt-backspace, or to change the keyboard layout. Each Testdrive instance gets 512 MB of RAM and an hour to run.

One can also make changes to files in Testdrive, investigate the diffs, and commit those changes to the appliance. The changes will then be implemented when starting a new build. This is really nice, as one can, for example, fire up an interactive installer of an application and commit the changes, so that appliance users don't have to. SUSE Studio creator Nat Friedman explains in his blog how this is implemented with a copy-on-write (COW) disk image for the KVM image. The original image is only used for reading and all writes go to the COW file. Whenever the virtual machine wants to read a block, it first checks for its presence in the COW file, and falls back to the read-only image if not:

Many possibilities

The SUSE Studio website is a closed-source Ruby on Rails web application, but there are plans to open up the code sometime in the future. Later this year, companies can buy SUSE Studio Onsite, which is a version on a server hosted within their own data center. SUSE Studio uses the open source Kiwi build tool as its backend. When the user has created an appliance with SUSE Studio, he can download the appliance description from SUSE Studio and build the appliance later, directly with Kiwi. Some Kiwi recipes can be found in the Kiwi cookbook on openSUSE's wiki. Kiwi also supports generating an EC2 AMI, so this slight detour is a perfect way to generate an AMI for an appliance made by SUSE Studio if you can't wait for the support coming later this year.

A big selling point of SUSE Studio is the documentation on openSUSE's wiki. This varies from a FAQ, a tour, HOWTOs, and the API, to known issues, support channels, and downloads. The last features some appliances that users have built with SUSE Studio, such as a Mono 2.4 appliance, an Elisa Media Centre appliance, openSUSE images for netbooks, and a demo live cd of Marble Desktop Globe.

The HOWTOs seem to be of good quality and guide you through some elaborate tasks. The general HOWTOs are mostly related to firstboot scripts and kiosk-like appliances, while there are more specific KDE and Gnome HOWTOs. Other HOWTOs are related to the type of deployment, such as VMware, VirtualBox (the VMware images run fine in VirtualBox), disk images and live CDs.

An appliance in fifteen minutes

In the last few months, your author has tested some other Linux appliance building tools as well, and can say with confidence that SUSE Studio was the most impressive. While rBuilder Online is also a handy and efficient tool, which supports more Linux distributions than just the SUSE family and has a neat management console for the appliance, SUSE Studio is much more user-friendly. Another solution, VMware Studio 1.0, also doesn't match the user experience of SUSE Studio: the user has to run this appliance builder in VMware and download a whole DVD image for the distribution the appliance will be based on. Moreover, building the image can take hours and has to start over from scratch when the user makes the slightest error, such as a typo in a URI. Advantages of VMware Studio are the support for a lot of Linux distributions and the excellent documentation.

All in all, compared to its competitors, SUSE Studio is without doubt the most easy-to-use, the most well-engineered, and the most efficient appliance builder. Even someone without any previous experience can build a software appliance in fifteen minutes. If it would only offer more distributions than just the SUSE family for the operating system base, it would be even more interesting.

Comments (5 posted)

New Releases

Arch Linux 2009.08 available

Comments (none posted)

openSUSE 11.2 Milestone 5 Released

Full Story (comments: none)

Slackware 13.0 RC2

Full Story (comments: none)

Distribution News

Debian GNU/Linux

Debian Etch and Ubuntu Feisty: a comparison

Update: here's a new analysis from Anthony with more detailed statistics and a look at the "Intrepid" and "Jaunty" releases.

Full Story (comments: 10)

Fedora

Slip of Fedora 12 Alpha by one week

Full Story (comments: none)

Fedora 11 Release Events Contest Winners Announcement

Full Story (comments: none)

Fedora Advisory Board Meeting Summary

Full Story (comments: none)

Gentoo Linux

Gentoo mourns the passing of Ferris McCormick

Full Story (comments: 1)

SUSE Linux and openSUSE

Novell increasing openSUSE support

Full Story (comments: 11)

Unofficial openSUSE KDE 4.3 RPMs and Live CDs

Full Story (comments: none)

Advance notice of discontinuation of openSUSE 10.3

Full Story (comments: none)

Ubuntu family

Ubuntu removes "multisearch"

Comments (59 posted)

Ubuntu Karmic GRUB menu hiding

Full Story (comments: none)

Minutes from the Ubuntu Technical Board meeting

Full Story (comments: none)

Distribution Newsletters

Arch Linux Magazine

Comments (none posted)

DistroWatch Weekly, Issue 315

Comments (none posted)

Fedora Weekly News 188

Full Story (comments: none)

Openmoko Community Updates

Comments (none posted)

OpenSUSE Weekly News/83

Comments (none posted)

Ubuntu Weekly Newsletter #154

Full Story (comments: none)

Distribution reviews

An Early Look at Ubuntu 9.10 Karmic Koala (Tuxarena)

Comments (none posted)

Xandros Presto Linux (PCWorld)

Comments (none posted)

Page editor: Rebecca Sobol

Development

Translating software with Pootle

Pootle is Free/Open Source, web-based software, aimed at making the process of translation of other software simpler. As a typical translation team's workload increases with each new software release, it becomes more and more important to have a smooth workflow, not only for managing the translations, but also for managing the translators as well. Simultaneously, to ensure that translators from all kinds of backgrounds are able to participate, the entire process needs to be kept as accessible and easy as possible. Pootle's goal is to make this a reality.

Pootle is written in Python and is used by FOSS projects such as OpenOffice.org, One Laptop per Child/Sugar Labs, LXDE, Sahana, GNU Mailman, etc. A list of live Pootle installations is maintained on the Pootle wiki. In its backend, Pootle utilizes the Translate Toolkit, which provides an API for a comprehensive set of localization related functions, along with support for multiple translation file formats such as PO, XLIFF, QT .ts, CSV, and Mozilla .properties. Using this backend, Pootle offers a web-based interface for managing translators, adding, modifying and suggesting translations, and support for connecting to a number of version control systems (CVS, SVN, Git, Mercurial, etc).

In a typical Pootle installation the server configuration points to the po/ subdirectory of a project (or an appropriate symlink). The project may be a checkout/clone of a remote repository or standalone. In the former case the language team lead (or other authorized team members) can commit (and push) translations to the remote, upstream repository via the web interface. If the PO files are not part of a repository, the integration step is usually carried out manually. All the PO files (or equivalent translation files) in the po/ directory are exposed via the web UI, and the messages in the files are presented as distinct units to the translators, who can translate them (or edit the translations) using the browser.

Pootle's web-based translation interface lowers the barrier to entry considerably, this is crucial for ensuring success for translation teams (especially those with limited resources and volunteers). Translators often come from non-technical backgrounds, so the web-interface, by abstracting away the underlying complexity of multiple version control systems and the gettext toolchain, makes their work considerably easier.

The web-based interface, despite simplifying things with its ease of use can sometimes exclude people from participating. One barrier is that it relies on continuous Internet access, which may be non-existent or prohibitively expensive in many regions of the world. Fortunately, Pootle provides a feature which allows translators to download PO files, translate offline, and re-upload the files.

This "offline-translation" feature can be highly useful for translation "sprints", where a group of volunteers gather together, translate a set of downloaded PO files using offline tools such as Poedit or Virtaal, then upload back the files to the Pootle server.

However, the overall ease of use brings up yet another potential problem. With a large number of translators from various backgrounds participating in the translation process, quality issues invariably crop up. Such issues can be divided into two broad categories:

• Mistranslations: These can be either plain "wrong" translations, or errors like incorrectly typed format placeholders or an incorrect number of escape sequences (e.g. '\n'). This can be partially addressed by having access control rules in place. Pootle has the provision for different levels of access, where a user can be assigned various rights on the basis of the user's experience level. Using this feature, a user can be allowed to perform any combination of tasks such as translating, suggesting possible translations, reviewing suggested translations, committing translations, etc.

  To ensure that the quality level of a new contributor is up to the mark, many language team coordinators choose to set the default access level to "suggest". When the initial set of suggestions are found to be satisfactory, the access level can be increased to "translate", "review", or even higher.

  Another way of ensuring high quality translation is to use the built-in "checks" feature offered by the software. Currently there is a set of 47 checks in Pootle which test for common errors in translations. Some of these tests, like the capitalization check are language and writing system specific. But others, like the check for translated file paths and function names (normally file paths or function names are kept untranslated) are applicable universally and can be quite useful.

  For translators not sufficiently familiar with English (messages to be translated are almost invariably in English), the ability to view reference translations in a third language often helps. Support for this viewing of an alternate language was added in the last stable release of Pootle (version 1.2). For example, this feature was found to be very useful in OLPC's South American deployments, where translators for languages like Aymara and Quechua wanted to see Spanish translations while translating, to better understand the meaning of the original English strings.

  

  A combination of the above three features, if used properly, can reduce mistranslations to a large extent.

• Inconsistency: In this commonly occurring problem, two or more translators may translate the same term differently. For example, for a certain term, one translator might want to keep an English transliteration, while another may want to use the literal meaning of the term. This almost certainly confuses the end user, and ensuring consistency is important. To address this problem, Pootle provides a glossary, or terminology feature, where a single set of translations for commonly used terms can be stored. During translation, the appropriate entry from this set is provided as a suggestion whenever the translator comes across a message containing a term from the glossary. This particular feature can also be useful if there is government or language-body sanctioned terminology for computer-related vocabulary.

  Though deciding on and creating a glossary of terms manually can be difficult for projects with a large string count, the Translate Toolkit provides a command line tool called poterminology to automatically produce a list of commonly occurring terms from a set of PO or POT (PO Template) files.

With the features outlined above, Pootle can make the life of translators in any FOSS project considerably easier. However, from the author's experience at OLPC/Sugar Labs, there can been a few problems along the road as well. Any project thinking of adopting Pootle should ideally factor in these issues in before taking the plunge:

• Speed issues: Pootle can be quite slow during certain operations involving large PO files (as an example, the largest of the PO files served by the OLPC/Sugar Labs Pootle installation has around 4400 strings). These operations that can make Pootle utilize 100% CPU on the hosting server for a significant amount of time. Operations such as merging two large PO files (which usually happens during uploads), updating a set of PO files from the corresponding set of updated reference POT files, or searching through large files for untranslated strings, are examples of potentially lengthy tasks.

  Newer versions of Pootle (and the Translate Toolkit) have seen the introduction of sqlite-based databases for storing translation statistics and the use of Lucene/Xapian based indexing of PO files. These changes have significantly improved the search performance.

• Communication: Pootle itself does not provide for any mode of communication within teams or between site-administrators and teams. This can be addressed partially by using a mailing list for each translation team, as well as a special "all-hands" mailing list for general announcements and reminders. However, even if mailing lists are used, it becomes increasingly difficult to get in touch and coordinate with individual translators and team leads as the translation community scales up. This can become an especially pressing issue toward the end of a release cycle; often translation leads need to be reminded individually about committing the work they have completed so far.

  On the other hand, the Pootle developers seem to be aware of the urgency of the issue, and there is currently a page in the wiki outlining the various use-cases for this feature. Some of the required features have already been implemented in Pootle's SVN trunk.

• Error checking: The error checking feature in Pootle needs to invoked manually (via the "checks" system mentioned before). This means that there is always a chance that wrongly formatted PO files may get introduced into the software's source tree, causing the build to break in some cases. Of course, this is not an issue if the language team lead always ensures that at least the the critical "checks" pass for the PO file before a commit is made.

Despite the above weaknesses, from an overall perspective Pootle can be a useful component of any FOSS project's localization arm. In fact, it is not only the translators who benefit from the features that Pootle offers, developers only have to grant repository access to one user account (the one used by Pootle) to take care of all translations. The process of providing the requisite repository access to each translator or language team has been a major bottleneck for quite a few FOSS projects in the past.

As of August 2009, Pootle has been undergoing significant changes in both code and visual design. Along with a switch to the Django web framework, the user interface has changed radically. Now, overall statistics for each project and language are viewable from the first page and UI changes are visible in the individual project/language pages. Among the other user-visible changes that are supposed to ship with Pootle 1.3, notable features include support for multiple alternate languages and initial implementation of intra-team communication methods. Also, with the shift to Django, which is much more popular compared to the older jToolkit framework, it should become easier for external developers to extend and customize Pootle.

The code for Pootle 1.3 (pre-release) can be downloaded from the SVN repository of the project. It is pretty much usable in its current form, though probably not recommended for production use. However, the project does need more help with testing, and reports about any bugs in the latest version from SVN are welcome.

With the impending release of Pootle 1.3, along with the fast pace of growth of other similar tools (Transifex, as well as the recently open-sourced Rosetta), things certainly do look bright for FOSS translation tools. In the next few months, we should be able to see significant improvements in the way Open Source translation is done, and through all these changes, the general translation workflow for FOSS will be greatly improved.

Comments (10 posted)

System Applications

Database Software

Firebird 2.5 Beta 2 kits released

Comments (none posted)

PostgreSQL Weekly News

Full Story (comments: none)

SQLite release 3.6.17 released

Comments (none posted)

SQLObject 0.11.0 released

Full Story (comments: none)

Interoperability

Samba 3.0.36 maintenance release available

Comments (none posted)

Web Site Development

Apache HTTP Server 2.2.13 released

Full Story (comments: none)

Apache ODE 1.3.3 announced

Full Story (comments: none)

Desktop Applications

Data Visualization

matplotlib 0.99 released

Comments (none posted)

Desktop Environments

GNOME Software Announcements

• Accerciser 1.7.90 (translation work)
• atk 1.27.90 (translation work)
• ccss 0.5 (new features)
• Cheese 2.27.90 (new features, code cleanup and translation work)
• Clutter 1.0 (code branch)
• Deskbar-Applet 2.27.90 (translation work)
• Evince 2.27.90 (bug fixes and translation work)
• F-Spot 0.6 (unspecified)
• GCalctool 5.27.90 (bug fixes, documentation and translation work)
• GNOME Commander 1.2.8.1 (bug fixes)
• GNOME DVB Daemon 0.1.8.1 (new features and bug fixes)
• gnome-keyring 2.27.90 (bug fixes and translation work)
• GOK 2.27.90 (bug fixes and translation work)
• GParted 0.4.6 (bug fixes and translation work)
• gtk-css-engine 0.3.1 (new features)
• Java ATK Wrapper 0.27.6 (bug fix)
• libchamplain 0.3.90 (bug fix)
• moserial 2.27.4 (bug fixes, documentation and translation work)
• mousetweaks 2.27.90 (translation work)
• Nautilus-Actions 1.12.0 (new features, bug fixes and translation work)
• Orca 2.27.90 (bug fixes and translation work)
• PyClutter 0.9.2 (new features)
• PyGobject 2.19.0 (new features and bug fixes)
• Sabayon 2.27.0 (new features, bug fixes and translation work)
• seahorse 2.27.90 (bug fixes, documentation and translation work)
• Tangerine 0.3.2 (bug fixes)

Comments (none posted)

KDE Software Announcements

• 2ManDVD 0.9.1 (new features and bug fixes)
• 2ManDVD 0.9.2 (bug fixes)
• 2ManDVD 0.9.3 (bug fixes and translation work)
• Choqok 0.6.6 (new features and bug fixes)
• Electronic Business Assistant 1.0.0.1 (unspecified)
• Frescobaldi 0.7.13 (bug fixes and translation work)
• KDevelop 3.5.5 (new features)
• KMidimon 0.7.1 (new features and bug fixes)
• Konversation 1.2-alpha6 (bug fixes)
• KTorrent 3.2.3 and 3.3beta1 (new features and bug fixes)
• ktv 0.1 (initial release)
• 'Q' DVD-Author 1.11.0 (new features and bug fixes)
• QComicBook 0.4.1 (bug fixes and code cleanup)
• qt gui for amule 0.1 (initial mockup release)
• Simple Root Actions Menu 2.2.0 (new feature and translation work)
• Skanlite 4.3 (unspecified)

Comments (none posted)

Xorg Software Announcements

• font-util 1.0.2 (bug fixes and documentation work)
• inputproto 1.9.99.901 (code cleanup and documentation work)
• libSM 1.1.1 (bug fixes, code cleanup and documentation work)
• pixman 0.15.20 (new feature and bug fixes)
• xextproto 7.1.0 (code cleanup)
• xf86-input-evdev 2.2.4 (bug fix)
• xf86-input-vmmouse 12.6.5 (bug fixes and code cleanup)
• xrandr 1.3.1 (bug fixes)

Comments (none posted)

Desktop Publishing

Scribus 1.3.5 released

Comments (none posted)

Encryption Software

gnupg 0.2.1 released

Full Story (comments: none)

Interoperability

Wine 1.1.27 announced

Comments (none posted)

Multimedia

Moovida Media Center 1.0.6 released

Full Story (comments: none)

Office Applications

Roundup Issue Tracker 1.4.9 released

Full Story (comments: none)

Office Suites

OpenOffice.org Newsletter

Full Story (comments: none)

Digital Photography

digiKam digest (KDEDot)

Comments (none posted)

Miscellaneous

New releases of Mozilla Lightning and SOGo

Full Story (comments: none)

Languages and Tools

C

GCC 4.3.4 released

Full Story (comments: none)

Perl

"Useful and usable" Perl 6 release coming in April

Comments (21 posted)

Perl 5.10.1 RC1 released (use Perl)

Comments (none posted)

Test-Run 0.0120 released

Full Story (comments: none)

Python

CodeInvestigator 0.14.0 announced

Full Story (comments: none)

Distribute 0.6 released

Full Story (comments: none)

M2Crypto 0.20 released

Full Story (comments: none)

python-utmp 0.8 released

Full Story (comments: none)

Python-URL! - weekly Python news and links

Full Story (comments: none)

Tcl/Tk

Tcl-URL! - weekly Tcl news and links

Full Story (comments: none)

Test Suites

oejskit 0.8.6 announced

Full Story (comments: none)

Page editor: Forrest Cook

Linux in the news

Recommended Reading

Widenius: thoughts on dual-licensing

Comments (17 posted)

Peters: The Desktop or the Browser: Is the Netbook Escalating the Battle?

Comments (15 posted)

Trade Shows and Conferences

Free Desktop Communities come together at GCDS (KDEDot)

Comments (9 posted)

The SCO Problem

Chapter 11 Trustee to be Appointed; SCO's Sale Motion Denied (Groklaw)

Comments (15 posted)

Companies

Red Hat pushes certification program amid Linux wave (Computerworld)

Comments (6 posted)

Red Hat's JBoss road less traveled (CNet)

Comments (none posted)

National Vision chooses SUSE Linux (Reuters)

Comments (none posted)

Resources

Is Linux Irrelevant? (InformationWeek)

Comments (29 posted)

Reviews

Python Python Python (aka Python 3) (Linux Journal)

Comments (5 posted)

Miscellaneous

Why can't local government and open source be friends? (guardian.co.uk)

Comments (62 posted)

Page editor: Forrest Cook

Announcements

Non-Commercial announcements

EFF on the RealDVD decision

Comments (13 posted)

Legal Announcements

Patent fun: Microsoft Word sales banned in the US

Comments (42 posted)

New Books

O'Reilly publishes book excerpts

Comments (none posted)

Resources

AMD RS780 docs released, coreboot support coming

Full Story (comments: none)

VistA community members propose VistA Installation Spec (LinuxMedNews)

Comments (none posted)

Event Reports

O'Reilly Open Source Convention delivers benefits of open source

Full Story (comments: none)

Calls for Presentations

CFP: FOMS 2010

Full Story (comments: none)

CFP: International workshop on Secure Software Engineering

Full Story (comments: none)

Upcoming Events

2009 Gnome Boston Summit

Full Story (comments: none)

Japan Linux Symposium program announced

Full Story (comments: 2)

LCA2010 announces successful miniconfs

Full Story (comments: none)

Announcing Camp KDE 2010 (KDEDot)

Comments (none posted)

Events: August 20, 2009 to October 19, 2009

If your event does not appear here, please tell us about it.

Page editor: Forrest Cook