LWN.net Weekly Edition for August 13, 2009
The unending story of cdrtools
Certain unwelcome stories seem to never really go away. One may think that an issue has been resolved, only to be attacked by a zombie version years later. It has been almost exactly three years since LWN last wrote about license problems with cdrtools; the combination of GPL- and CDDL-licensed code in that package rendered the whole undistributable. Linux distributors responded by switching to cdrkit - a fork of cdrtools taken from a release prior to the problematic license changes - and it seemed like the problem was solved in an optimal way. The community had eliminated a licensing problem with an important package and disconnected from a difficult upstream maintainer at the same time.But these problems are never solved, it seems. In June, Jörg Schilling, the author of cdrtools, wandered into the fedora-legal list with a request for Fedora to resume shipping the "original, legal" cdrtools software. After a discussion of the type that typically follows Jörg around, Tom "spot" Callaway stepped in with a definitive response (short version: "no") which pretty much brought the discussion to an end.
Life got quiet again until early July, when Luis Medinas suggested that openSUSE might want to switch back to cdrtools. That was Jörg's cue to make one of his predictable appearances, inspiring an even longer and stronger version of the kind of discussion that tends to follow him around. This time Jörg made a direct lawsuit threat against SUSE, but showed his forgiving side too:
One might well wonder about the reversal of roles here; now it's Jörg who is complaining about the legality of cdrkit. His complaints have been posted to the web. They include the fact that the "wodim" CD recorder packaged in cdrkit is installed as "cdrecord" (a GPL violation, he says), the lack of detailed change information within the source files, the failure to print a copyright notice "as intended by the original author," an (unspecified) failure to distribute "complete" source, and a couple of alleged violations of German copyright law (which, it seems, forbids any change which Jörg disapproves of). All told, it is a long series of complaints resulting from a simple fork of a GPL-licensed program.
Most observers do not take these claims seriously. The complaint about the cdrecord binary is (somehow) based on the preamble of the GPL - which is not part of the binding terms. Section 2a of the GPL does require dated notifications of changes, but it's a rare project which carries those notifications within the source files themselves, as Jörg is demanding. The complaint about copyright notices is interesting. Cdrecord has traditionally been a verbose utility, and that verbosity has extended to Jörg's thoughts about Linux distributors and kernel developers. For example, version 2.01.01a01 (from 2004) would print things like:
Warning: Running on Linux-2.6
There are unsettled issues with Linux-2.5 and newer.
If you have unexpected problems, please try Linux-2.4 or Solaris.
SuSE Linux is known to ship bastardized and defective versions of cdrecord.
SuSE is unwilling to cooperate with the authors.
If you like to have a working version of cdrtools, get the
original source from ftp://ftp.berlios.de/pub/cdrecord/
(The current version, 2.01.01a63, has lost some of that language). The removal of some of that verbosity is what he is complaining about. But GPL section 2c only requires the printing of "an appropriate copyright notice" (not any specific notice), and it only applies to programs which read commands interactively, which wodim does not do. So this claim, like the others, has failed to create widespread worry.
In short, many in the community seem to see Jörg as a sort of comic figure, but that should not be allowed to obscure an important fact: there are some points worth noting behind his complaints. These include:
- Jörg alleges that openSUSE is
shipping two related, legally problematic packages: vcdimager
and libcdio.
Both packages are GPL-licensed and hosted with the GNU project, but
other distributions have recognized problems with them;
Debian has shipped a
patched version since 2004, and Fedora users must get it from an
external repository. Fedora also does not ship libcdio, which is
alleged to have suffered a license change which is not acceptable to
the original author of the code.
- Cdrkit is nearly unmaintained. The mailing list for changes is a quiet and lonely place. Jörg states that hundreds of unfixed bugs have been introduced into cdrkit. The reality, as shown by distribution bug trackers, is a bit less spectacular, but it is true that some bugs exist which might not be present in cdrecord - which is actively maintained by Jörg.
The first issue needs to be taken seriously; it is never a good idea to distribute code with problematic or disputed licensing. The fix here is relatively straightforward: stop distributing that code if the license cannot be verified, and, possibly, reimplement it (as Sun is said to have done with libcdio).
The second may be harder. The freedom to fork a package out from under an uncooperative maintainer is one of the fundamental features of free software. But forking is expensive; it only works if somebody else does the work which has been pulled away from that maintainer. An unmaintained fork is just more dead code. If cdrkit reaches a point where it fails to work for users, distributors will be left with an unpalatable choice: continue to ship unmaintained code, or go back to the original, with its difficult maintainer and incompatible licensing. It would be much nicer to find somebody willing to put some time into this important tool. CD recording is a detailed and tricky task, but we have plenty of people in our community with the necessary skills to work in that area.
KDE struggles with feature requests
Sometimes developers have a prickly relationship with their users. Users may have unrealistic, or overly demanding, requests that can be difficult to respond to. The most vocal of these users are often unwilling to take "no"—or even "not yet"—for an answer. Some KDE developers are currently struggling with that problem, and trying to find ways to smooth the dialog between users and developers.
In a posting to the kde-devel mailing list,
Pau Garcia i Quiles wondered where KDE 3 features that were missing from
KDE 4 should be collected. He noted that there are various places users
were complaining about these missing features (including an openSUSE
web page that collects them), but no central location for KDE to track such
things. His suggestion: "Can we start something like that in
UserBase, for people to tell us
what they miss in KDE4 from KDE3? Or have a special category in
Bugzilla?
"
That set off a bit of a rant from Aaron J. Seigo about user complaints:
[...] but i won't go back on various design decisions and throw out all the benefits we're reaping due to those decisions. i refuse to fall into some misguided knee-jerk-to-the-latest-random-user-moaning design "methodology"
Seigo also noted that the openSUSE list doesn't "mention _at all_ the
actually useful features that are missing
", and, that, when he
commented on that wish list item, he "got yelled at by two different
people on the report, completely without cause
". Frustration is
obvious in his posting, and he noted that it was probably not quite the
response Garcia expected, but he wanted to make it clear that the current
options were not working:
Anne Wilson noted that the users Seigo is
referring to are just a "*very* vocal minority
" that
"can only be ignored
". She is concerned with the users who
are trying to make a difference with their bug reports and feature
requests, only to be treated as if they are part of that loud minority.
She disagreed with Seigo's suggestion that users should either write—or
pay for—the code, or just be patient:
But, Seigo sees things somewhat differently. He points to this vocal minority as part of the
reason that KDE projects aren't "paying much attention to
feature requests made on bugs.kde.org
". Once again, he places the
blame largely at the feet of the user community:
There was some discussion of changing various bug tags, particularly
WONTFIX, as it is regularly misinterpreted, to try to alleviate the
problem. That is unlikely to mollify the users who are most vocal, though.
Trying to ensure that features and bugs closed as WONTFIX get some
kind of explanation will probably help with, but not eliminate, the
problem, as well.
Andreas Pakulat points out that it is a
social problem: "people are getting used to be
able to shout, rant and moan on the net without ever being held
responsible for the possible damage they do with that
".
One idea that seems to be gaining some traction is to use KDE Brainstorm, which was suggested as a place to gather features by Stefan Majewsky. Aside from some usability issues that seem like they could be dealt with relatively easily, Brainstorm provides a means to discuss new (or missing KDE 3) features, while allowing users to vote on those they find most important. Seigo sees it as a starting point:
But the problem is more than just work flow. From the postings in the thread, some KDE developers are finding it difficult to work with the user community, largely because of the behavior of a few of its members. Parker Coates is unconvinced that a tool-driven process will eliminate the problem:
Due to the very vocal, and largely negative, reaction to the release of KDE 4 more than a year and a half ago, there is still a great deal of frustration within the project—for both users and developers. While there are certainly some important points in the developers' messages, the tone is such that they also could be taken as an indictment of all users—something that is clearly not intended.
This is a problem that certainly isn't limited to KDE, as other projects have or will run into the same kinds of problems. There is a delicate balance between ignoring the "vocal minority" and ignoring the user community as a whole. The latter could easily lead a project to completely lose touch with the needs of its users, to the point where those users end up walking away. That is an outcome both sides want to—and should—avoid. Finding better ways to handle feature requests, while avoiding the conflicts with the few who will not be civil, is a good step on that path.
Ubuntu's multisearch surprise
If you are a Linux distributor, you have a number of possible ways to upset your user base. Breaking existing, well-established functionality is one of them. Another would be to install software which appears to be monitoring user activity behind their backs. Seeming to make money off of these activities will not help. Extra points are awarded for doing it all as a surprise. Ubuntu has risked all of the above with the "multisearch" Firefox extension included in the current "Karmic Koala" alpha release.The bug report filed on July 21 had to do with broken functionality. It seems that, when using the version of Firefox distributed with the third Karmic alpha release, typing a search string into the "awesome bar" no longer takes the user directly to the first search result from Google. Instead, users end up at a Google "search partner" page listing the results and, of course, advertisements. Other quick searches, including stock quotes and currency conversions, also break. A related change is that opening a new tab now brings up an Ubuntu search page instead of a blank page - a change that some users find jarring.
It turns out that Ubuntu has placed a new Firefox extension, called "multisearch," into the Karmic alpha release. In essence, multisearch rewires the various search mechanisms built into the browser, causing them all to pass through Ubuntu's partner page. It can be disabled by going into the "Tools->Add-ons" menu, but, by default, it is installed and active on all systems.
So why was this done? Rick Spencer, Ubuntu's desktop engineering manager, explained the reasoning in a fair amount of detail. The "new tab" change is an attempt to improve the user experience - something that Mozilla developers are working on as well. The search change lets Ubuntu know which search mechanisms are being used most; beyond that, he said:
Generating revenue that supports the project is a feature, not a bug. However, we are mindful of not throwing the baby out with the bath water. In other words, we must strike the balance of continuing to deliver a top notch user experience while taking advantage of revenue opportunities.
Ubuntu users are not necessarily opposed to the idea of revenue going
toward the development of their distribution; it's a "feature" they can
support. Many of them are, however,
rather less thrilled about their search data being used to that end.
Rick's explanation - "it's simply the same data that is already sent
to Google and Mozilla: the requested search, and the channel for the
search
" - does not appear to have made anybody feel any better. As
might be imagined, some of the more vocal users are throwing around words
like "spyware" and "privacy violations." But even calmer voices are
concerned that this "feature" was silently added to their systems, that it is
not something they wish to have around, and that there has been little talk
of privacy protections for the accumulated data.
Apologies from the Ubuntu side have been few and far between. Ubuntu Mozilla maintainer Alexander Sack justifies the change this way:
Of course, one should bear in mind that default Ubuntu installations are "opted in" to the ubuntu-desktop metapackage; very few users will have deliberately made that choice.
The other thing to bear in mind is that this feature appears in an alpha
release - and that users did indeed make a deliberate choice to install
that release. It's not uncommon to find unpleasant surprises in alpha-quality
distributions, even if it's a bit more uncommon for those surprises to have
been introduced deliberately. Alexander says that multisearch "is not
intended to stay forever - at least not in its current form.
" One
can interpret that to mean that some of the more annoying failures will be
fixed. It's possible that the entire thing will be taken out before the
end of the alpha-test period. But nobody from Canonical is saying that now.
A great deal of trust is placed in Linux distributors; they have the ability to inflict all kinds of unpleasant behavior on their users. Distributors seen to abuse that trust are not likely to retain their users for all that long, though. The beauty of free software shows through in a few ways here: undesirable behavior is very hard to hide, it is quite easy to remove, and, if all else fails, one can switch to a different distribution with minimal pain. Ubuntu is probably not losing any users over this episode - yet. But any user of this distribution who is concerned about this behavior may want to watch closely to see what decisions are made between now and the final Karmic Koala release.
(Update: multisearch was removed from Ubuntu on August 11.)
Security
OpenVAS replacing Nessus in Debian
For many years, the Nessus network vulnerability scanner was a tool in the toolbox of most free-software-oriented administrators. It provided a very useful, GPL-licensed scanner to detect various network vulnerabilities, misconfigurations, and other types of security problems in the network. But, starting in late 2005 that all began to change, when Nessus 3.0 switched licenses, so folks looking for a free software network scanner had to turn elsewhere.
There have been a number of attempts to fork the last GPL version of the Nessus software (2.2), but the most successful to date has been the Open Vulnerability Assessment System (or OpenVAS). The forked scanner has been making great strides to the point where Debian's Nessus maintainer, Javier Fernández-Sanguino Peña, asked that Nessus be removed from the unstable branch in favor of OpenVAS. In his message, he noted:
There are really two parts to a vulnerability scanner, a core scanner and a set of plugins that implement network vulnerability tests (or NVTs). Much like virus scanners, NVTs are constantly being added and updated, and are available via network feeds. For a vulnerability scanner to be really usable, NVTs must be available for older vulnerabilities as well as being developed for new ones as they come along. In the thread on the debian-security mailing list, Tim Brown reports that OpenVAS has reached that point:
AFAIK the only plugins that are in Nessus 2 but not in OpenVAS are those which Tenable have since claimed are not GPL and for these the OpenVAS team are actively developing replacements.
Where Debian goes, other distributions are likely to follow, so we may see Nessus removed in favor of OpenVAS elsewhere as well. It is unfortunate that Tenable, the company behind Nessus, was unable to find a way to continue with a GPL-licensed Nessus, but the rise of OpenVAS shows the power of code that is available under a free software license. That is not to say that Tenable did anything wrong, it was their code and thus their choice; in fact, the community should be grateful that they provided the core of a nice tool for as long as they did. But, because the GPL allows forks like OpenVAS, Nessus users still had a free software path to follow once Tenable decided to go in a different direction.
The main stumbling block to getting to this point has been the NVTs released for Nessus. Those are governed by a separate license, that made it somewhat legally dubious, at best, to use them in OpenVAS. So, the OpenVAS developers had to tackle that problem themselves. Based on Brown's message, it would seem they have gotten most of the way there, and have an active community to continue that work into the future.
Brief items
You Deleted Your Cookies? Think Again (Wired)
Wired looks at the use of Flash cookies implemented by Adobe's browser plugin. "Several services even use the surreptitious data storage to reinstate traditional cookies that a user deleted, which is called re-spawning in homage to video games where zombies come back to life even after being 'killed,' the report found. So even if a user gets rid of a websites tracking cookie, that cookies unique ID will be assigned back to a new cookie again using the Flash data as the 'backup.'" See also this 2008 post from Gnash developer Rob Savoye, as well as an LWN article from last October, for more information on Flash cookies.
New vulnerabilities
apr: arbitrary code execution
| Package(s): | apr | CVE #(s): | CVE-2009-2412 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | August 6, 2009 | Updated: | May 10, 2010 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Mandriva alert: A vulnerability has been identified and corrected in apr and apr-util: Fix potential overflow in pools (apr) and rmm (apr-util), where size alignment was taking place (CVE-2009-2412). | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
camlimages: arbitrary code execution
| Package(s): | camlimages | CVE #(s): | CVE-2009-2660 | ||||||||||||||||||||
| Created: | August 10, 2009 | Updated: | June 1, 2010 | ||||||||||||||||||||
| Description: | From the Debian advisory: Tielei Wang discovered that CamlImages, an open source image processing library, suffers from several integer overflows which may lead to a potentially exploitable heap overflow and result in arbitrary code execution. This advisory addresses issues with the reading of JPEG and GIF Images, while DSA 1832-1 addressed the issue with PNG images. | ||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||
fetchmail: SSL impersonation vulnerability
| Package(s): | fetchmail | CVE #(s): | CVE-2009-2666 | ||||||||||||||||||||||||||||||||||||||||||||
| Created: | August 6, 2009 | Updated: | June 2, 2010 | ||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the slackware alert: This update fixes an SSL NUL prefix impersonation attack through NULs in a part of a X.509 certificate's CommonName and subjectAltName fields. | ||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||
java-1.6.0-openjdk: multiple vulnerabilities
| Package(s): | java-1.6.0-openjdk | CVE #(s): | CVE-2009-2475 CVE-2009-2476 CVE-2009-2625 CVE-2009-2670 CVE-2009-2671 CVE-2009-2672 CVE-2009-2673 CVE-2009-2674 CVE-2009-2675 CVE-2009-2689 CVE-2009-2690 CVE-2009-1896 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | August 7, 2009 | Updated: | November 30, 2009 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Fedora advisory:
CVE-2009-2475 OpenJDK information leaks in mutable variables CVE-2009-2476 OpenJDK OpenType checks can be bypassed CVE-2009-2625 OpenJDK XML parsing Denial-Of-Service CVE-2009-2670 OpenJDK Untrusted applet System properties access CVE-2009-2671 CVE-2009-2672 OpenJDK Proxy mechanism information leaks CVE-2009-2673 OpenJDK proxy mechanism allows non-authorized socket connections CVE-2009-2674 Java Web Start Buffer JPEG processing integer overflow CVE-2009-2675 Java Web Start Buffer unpack200 processing integer overflow CVE-2009-2689 OpenJDK JDK13Services grants unnecessary privileges CVE-2009-2690 OpenJDK private variable information disclosure CVE-2009-1896 openjdk/netx grants privileges for signed jars to bundled unsigned jars | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
libvorbis: denial of service
| Package(s): | libvorbis | CVE #(s): | CVE-2009-2663 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | August 11, 2009 | Updated: | August 17, 2010 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the CVE entry: libvorbis before r16182, as used in Mozilla Firefox before 3.0.13 and 3.5.x before 3.5.2 and other products, allows context-dependent attackers to cause a denial of service (memory corruption and application crash) or possibly execute arbitrary code via a crafted .ogg file. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||
libxml: multiple vulnerabilities
| Package(s): | libxml | CVE #(s): | CVE-2009-2414 CVE-2009-2416 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | August 11, 2009 | Updated: | September 22, 2010 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Red Hat advisory:
A stack overflow flaw was found in the way libxml processes the root XML document element definition in a DTD. A remote attacker could provide a specially-crafted XML file, which once opened by a local, unsuspecting user, would lead to denial of service (application crash). (CVE-2009-2414) Multiple use-after-free flaws were found in the way libxml parses the Notation and Enumeration attribute types. A remote attacker could provide a specially-crafted XML file, which once opened by a local, unsuspecting user, would lead to denial of service (application crash). (CVE-2009-2416) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
mantis: database credentials leak
| Package(s): | mantis | CVE #(s): | |||||
| Created: | August 10, 2009 | Updated: | August 12, 2009 | ||||
| Description: | From the Debian advisory: It was discovered that the Debian Mantis package, a web based bug tracking system, installed the database credentials in a file with world-readable permissions onto the local filesystem. This allows local users to acquire the credentials used to control the Mantis database. | ||||||
| Alerts: |
| ||||||
memcached: heap-based buffer overflow
| Package(s): | memcached | CVE #(s): | CVE-2009-2415 | ||||||||||||||||||||
| Created: | August 7, 2009 | Updated: | December 11, 2009 | ||||||||||||||||||||
| Description: | From the Debian advisory: Ronald Volgers discovered that memcached, a high-performance memory object caching system, is vulnerable to several heap-based buffer overflows due to integer conversions when parsing certain length attributes. An attacker can use this to execute arbitrary code on the system running memcached (on etch with root privileges). | ||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||
squid3: multiple denial of service vulnerabilities
| Package(s): | squid3 | CVE #(s): | CVE-2009-2622 CVE-2009-2621 | ||||||||||||||||||||
| Created: | August 10, 2009 | Updated: | August 18, 2009 | ||||||||||||||||||||
| Description: | From the Mandriva advisory: Due to incorrect buffer limits and related bound checks Squid is vulnerable to a denial of service attack when processing specially crafted requests or responses (CVE-2009-2621). Due to incorrect data validation Squid is vulnerable to a denial of service attack when processing specially crafted responses (CVE-2009-2622). | ||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||
subversion: heap overflows
| Package(s): | subversion | CVE #(s): | CVE-2009-2411 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | August 7, 2009 | Updated: | December 8, 2009 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the subversion advisory: Subversion clients and servers have multiple heap overflow issues in the parsing of binary deltas. This is related to an allocation vulnerability in the APR library used by Subversion. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||
wireshark: multiple vulnerabilities
| Package(s): | wireshark | CVE #(s): | CVE-2009-2560 CVE-2009-2562 CVE-2009-2563 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | August 6, 2009 | Updated: | May 28, 2010 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the National Vulnerability Database entries:
CVE-2009-2560:
"
CVE-2009-2562:
"
CVE-2009-2563:
" | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
wordpress: remote admin password reset
| Package(s): | wordpress | CVE #(s): | |||||||||
| Created: | August 12, 2009 | Updated: | August 12, 2009 | ||||||||
| Description: | From the advisory on full-disclosure: A web browser is sufficient to reproduce this Proof of concept: http://DOMAIN_NAME.TLD/wp-login.php?action=rp&key[]= The password will be reset without any confirmation. An attacker could exploit this vulnerability to compromise the admin account of any wordpress/wordpress-mu <= 2.8.3 | ||||||||||
| Alerts: |
| ||||||||||
Page editor: Jake Edge
Kernel development
Brief items
Kernel release status
The current development kernel remains 2.6.31-rc5; there have been no 2.6.31 prepatches released since July 31. Patches continue to flow into the mainline repository (442 since 2.6.31-rc5, as of this writing) and the 2.6.31-rc6 release can be expected at almost any time.
Kernel development news
Quotes of the week
I'm nervous about ext4 coming into wider use and people finding some of the bits which aren't -quite- ready for prime time yet, and winding up with a disaster.
In Brief
Tux3. The once-noisy Tux3 development community has gone rather quiet in recent months. An inquiry into the status of the project led to one of last week's quotes of the week, wherein developer Daniel Phillips pled a lack of time and expressed regrets at not having merged the code into the mainline months ago. When asked (by Ted Ts'o) for a description of what makes Tux3 interesting, Daniel responded this way:
What Tux3 needs, it seems, is some new development energy. It could be an interesting project for developers who are wanting to get started in filesystem development.
Resource counters. The resource counter mechanism is built into control groups; it is intended for use by tools like the memory use controller. These counters contain, at their core, a (believe it or not) counter value which tracks the current usage of a resource by a given control group. This counter has run into the same problem which afflicts any frequently-changed global variable: it scales poorly due to cache line bouncing. The usage of some resources (pages of memory, for example) can change frequently, causing the associated counter to be a drag on the system as a whole.
Balbir Singh's scalable resource counters patch aims to fix that situation. With this patch, the single "usage" counter becomes an array of per-CPU counters. Since each processor works with its own copy of the counter, there is no more cache line bouncing and things run faster. The down side is that the count becomes approximate. The per-CPU counters are summed occasionally to keep everything roughly in sync, but keeping exact counts would take away much of the scalability that this patch was meant to provide. The good news is that exact counts are not really needed anyway; as long as the counter reflects something close enough to reality, the system will work essentially as it did before - only a little more quickly.
Inline spinlocks. Once upon a time, spinlocks were implemented with a series of inline functions, on the notion that such a performance-critical primitive would need to be as fast as possible. That changed in 2004, when spinlocks were turned into normal functions. The function call overhead hurt a bit, but moving spinlocks out-of-line made the kernel considerably smaller, which has performance benefits of its own. And that's how spinlocks have been ever since.
The pendulum may be about to swing the other way again, though, at least for the S390 architecture. Heiko Carstens noted that function calls on this architecture are quite expensive. He put together an inline spinlocks patch and measured performance improvements of 1-5%. So he would like to put this patch into the mainline, along with a configuration option allowing each architecture to choose the best way to implement spinlocks. So far, there has been little commentary for or against this idea.
Const seq_operations. James Morris has posted a patch making seq_operations structures
constant throughout the kernel. These structures are almost always
populated at compile time and never need to change; allowing the function
pointers therein to be overwritten can only be useful to those who would
like to subvert the kernel. A number of core VFS operations structures
have been made const over the years, but seq_operations
has not been addressed until now. James says: "This is derived from
the grsecurity patch, although generated
from scratch because it's simpler than extracting the changes
from there.
"
data=guarded. Back in the middle of the discussion of crash robustness and latency in the ext3 filesystem, Chris Mason came forward with a proposal for a data=guarded mode, which would delay metadata updates when files change size to prevent the disclosure of unrelated information. Since then, the data=guarded patch has disappeared from view. In response to a query from Frans Pop, Chris confirmed that he is still working on that code, and that he plans to get it merged for 2.6.32.
Among those welcoming the news was Andi Kleen, who remarked: "data=writeback already cost
me a few files after crashes here.
" The data=guarded mode may not
help with that particular problem, though: it is really meant to combine
the security benefits of data=ordered (not disclosing random data, in
particular) with the performance benefits of data=writeback. The worst
data-loss problems should have already been addressed by the robustness
fixes that went into ext3 for 2.6.30.
Fun with tracepoints
Tracepoints are a marker within the kernel source which, when enabled, can be used to hook into a running kernel at the point where the marker is located. They can be used by a number of tools for kernel debugging and performance problem diagnosis. One of the advantages of the DTrace system found in Solaris is the extensive set of well-documented tracepoints in the kernel (and beyond); they allow administrators and developers to monitor many aspects of system behavior without needing to know much about the kernel itself. Linux, instead, is rather late to the tracepoint party; mainline kernels currently feature only a handful of static tracepoints. Whether that number will grow significantly is still a matter of debate within the development community.LWN last looked at the tracepoint discussion in April. Since then, the disagreement has returned with little change. The catalyst this time was Mel Gorman's page allocator tracepoints patch, which further instruments the memory management layer. The mainline kernel already contains tracepoints for calls to functions like kmalloc(), kmem_cache_alloc(), and kfree(). Mel's patch adds tracepoints to the low-level page allocator, in places like free_pages_bulk(), __rmqueue_fallback(), and __free_pages(). These tracepoints give a view into how the page allocator is performing; they'll inform a suitably clueful user if fragmentation is growing or pages are being moved between processors. Also included is a postprocessing script which uses the tracepoint data to create a list of which processes on the system are putting the most stress on the memory management code.
As has happened before, Andrew Morton questioned the value of these tracepoints. He tends not to see the need for this sort of instrumentation, seeing it instead as debugging code which is generally useful to a single developer. Beyond that, Andrew asks, why can't the relevant information be added to /proc/vmstat, which is an established interface for the provision of memory management information to user space?
There are a couple of answers to that question. One is that /proc/vmstat has a number of limitations; it cannot be used, for example, to monitor the memory-management footprint of a specific set of processes. It is, in essence, pre-cooked information about memory management in the system as a whole; if a developer needs information which cannot be found there, that information will be almost impossible to get. Tracepoints, instead, provide much more specific information which can be filtered to give more precise views of the system. Mel bashed out one demonstration: a SystemTap script which uses the tracepoints to create a list of which processes are causing the most page allocations.
Ingo Molnar posted a lengthy set of examples of what could be done with tracepoints; some of these were later taken by Mel and incorporated into a document on simple tracepoint use. These examples merit a look; they show just how quickly and how far the instrumentation of the Linux kernel (and associated tools) have developed.
One of the key secrets for quick use of tracepoints is the perf tool which is shipped with the kernel as of 2.6.31-rc1. This tool was written as part of the performance monitoring subsystem; it can be used, for example, to run a program and report on the number of cache misses sustained during its execution. One of the features slipped into the performance counter subsystem was the ability to treat tracepoint events like performance counter events. One must set the CONFIG_EVENT_PROFILE configuration option; after that, perf can work with tracepoint events in exactly the same way it manages counter events.
With that in place, and a working perf binary, one can start by seeing which tracepoint events are available on the system:
$ perf list
...
ext4:ext4_sync_fs [Tracepoint event]
kmem:kmalloc [Tracepoint event]
kmem:kmem_cache_alloc [Tracepoint event]
kmem:kmalloc_node [Tracepoint event]
kmem:kmem_cache_alloc_node [Tracepoint event]
kmem:kfree [Tracepoint event]
kmem:kmem_cache_free [Tracepoint event]
ftrace:kmem_free [Tracepoint event]
...
How many kmalloc() calls are happening on a system? The question can be answered with:
$ perf stat -a -e kmem:kmalloc sleep 10
Performance counter stats for 'sleep 10':
4119 kmem:kmalloc
10.001645968 seconds time elapsed
So your editor's mostly idle system was calling kmalloc() almost 420 times per second. The -a option gives whole-system results, but perf can also look at specific processes. Monitoring allocations during the building of the perf tool gives:
$ perf stat -e kmem:kmalloc make
...
Performance counter stats for 'make':
5554 kmem:kmalloc
2.999255416 seconds time elapsed
More detail can be had be recording data and analyzing it afterward:
$ perf record -c 1 -e kmem:kmalloc make
...
$ perf report
# Samples: 6689
#
# Overhead Command Shared Object Symbol
# ........ ............... .................................... ......
#
19.43% make /lib64/libc-2.10.1.so [.] __getdents64
12.32% sh /lib64/libc-2.10.1.so [.] __execve
10.29% gcc /lib64/libc-2.10.1.so [.] __execve
7.53% cc1 /lib64/libc-2.10.1.so [.] __GI___libc_open
5.02% cc1 /lib64/libc-2.10.1.so [.] __execve
4.41% sh /lib64/libc-2.10.1.so [.] __GI___libc_open
3.45% sh /lib64/libc-2.10.1.so [.] fork
3.27% sh /lib64/ld-2.10.1.so [.] __mmap
3.11% as /lib64/libc-2.10.1.so [.] __execve
2.92% make /lib64/libc-2.10.1.so [.] __GI___vfork
2.65% gcc /lib64/libc-2.10.1.so [.] __GI___vfork
Conclusion: the largest source of kmalloc() calls in a simple compilation process is getdents(), called from make, followed by the execve() calls needed to run the compiler.
The perf tool can take things further; it can, for example, generate call graphs and disassemble the code around specific performance-relevant points. See Ingo's mail and Mel's document for more information. Even then, we're just talking about statistics on tracepoints; there is a lot more information available which can be used in postprocessing scripts or tools like SystemTap. Suffice to say that tracepoints open a lot of possibilities.
The obvious question is: was Andrew impressed by all this? Here's his answer:
As usual, we're adding tracepoints because we feel we must add tracepoints, not because anyone has a need for the data which they gather.
He suggested that he would be happier if the new tracepoints could be used to phase out /proc/vmstat and /proc/meminfo; that way there would not be a steadily-increasing variety of memory management instrumentation methods. Removing those files is problematic for a couple of reasons, though. One is that they form part of the kernel ABI, which is not easily broken. It would be a multi-year process to move applications over to a different interface and be sure there were no more users of the /proc files. Beyond that, though, tracepoints are good for reporting events, but they are a bit less well-suited to reporting the current state of affairs. One can use a tracepoint to see page allocation events, but an interface like /proc/vmstat can be more straightforward if one simply wishes to know how many pages are free. There is space, in other words, for both styles of instrumentation.
As of this writing, nobody has made a final pronouncement on whether the new tracepoints will be merged. Andrew has made it clear, though, that, despite his concerns, he's not firmly opposing them. There is enough pressure to get better instrumentation into the kernel, and enough useful things to do with that instrumentation, that, one assumes, more of it will go into the mainline over time.
clone_with_pids()
As part of the changes to support application checkpoint and restart in the kernel, Sukadev Bhattiprolu has proposed a new system call: clone_with_pids(). When a process that was checkpointed gets restarted, having the same process id (PID) as it had when the checkpoint was done is important to some kinds of applications. Normally, the kernel assigns an unused PID when a new task is started (via clone()), but, for checkpointed processes, that could lead to processes' PIDs changing during their lifetime, which could be an undesirable side effect. So, Bhattiprolu is looking for a way to avoid that by allowing clone() callers to specify the PID—or PIDs for processes in nested namespaces—of the child.
The actual system call is fairly straightforward. It adds an additional pid_set parameter to clone(), to contain a list of process ids; pid_set has the obvious definition:
struct pid_set {
int num_pids;
pid_t *pids;
};
A pointer to a pid_set is passed as the last parameter to
clone_with_pids(). Each of the PIDs is used to specify
which PID should be assigned at each level of namespace nesting.
The patch that actually implements clone_with_pids() (as opposed
to the earlier patches in the patchset that prepare the way)
illustrates this with an example (slightly
edited for clarity):
pid_t pids[] = { 0, 77, 99 };
struct pid_set pid_set;
pid_set.num_pids = sizeof(pids) / sizeof(int);
pid_set.pids = &pids;
clone_with_pids(flags, stack, NULL, NULL, NULL, &pid_set);
If a target-pid is 0, the kernel continues to assign a pid for the process in
that namespace. In the above example, pids[0] is 0, meaning the kernel will
assign next available pid to the process in init_pid_ns. But kernel will assign
pid 77 in the child pid namespace 1 and pid 99 in pid namespace 2. If either
77 or 99 are taken, the system call fails with -EBUSY.
The patchset assumes that being able to set PIDs is desirable, but
Linus Torvalds was
not particularly in favor of that approach when it was first discussed on linux-kernel back
in March. His complaint was that there are far too many stateful
attributes of processes to ever be able to handle checkpointing in the
general case. His suggestion: "just teach the damn program
you're checkpointing that pids will change, and admit to everybody
that people who want to be checkpointed need to do work
".
Others disagreed—no surprise—but it is unclear that
Torvalds has changed his mind. He was also concerned about the security
implications of processes being able to request PID assignments:
"But it also sounds like a _wonderful_ attack vector against badly
written user-land software that sends signals and has small races.
"
That particular concern should be alleviated by the requirement that a
process have the CAP_SYS_ADMIN capability (essentially root
privileges) in order to use clone_with_pids().
Requiring root to handle restarts, which in practice means that root must manage the checkpoint process as well, makes checkpoint/restart less useful, overall. But there are a whole host of problems to solve before allowing users to arbitrarily checkpoint and restore from their own, quite possibly maliciously crafted, checkpoint images. Even with root handling the process, there are a number of interesting applications.
There is an additional wrinkle that Bhattiprolu notes in the patch. Currently, all of the available clone() flags are allocated. That doesn't affect clone_with_pids() directly, as the flags it needs are already present, but, when adding a system call, it is good to look to the future. To that end, there are two proposed implementations of a clone_extended() system call, which could be added instead of clone_with_pids(), that would allow for more clone() flags, while still supporting the restart case.
The first possibility is to turn the flags argument into a pointer to an array of flag entries, that would be treated like signal() sets, including operations to test, set, and clear flags a la sigsetops():
typedef struct {
unsigned long flags[CLONE_FLAGS_WORDS];
} clone_flags_t;
int clone_extended(clone_flags_t *flags, void *child_stack, int *unused,
int *parent_tid, int *child_tid, struct pid_set *pid_set);
In the proposal, CLONE_FLAGS_WORDS would be set to 1 for 64-bit
architectures,
while on 32-bit architectures, it would be set to 2, thus doubling the
number of available flags to 64. Should the number of clone flags needed
grow, that could be expanded as required, though doing so in a
backward-compatible manner is not really possible.
Another option is to split the flags into two parameters, keeping the current flags parameter as it is, and adding a new clone_info parameter that contains new flags along with the pid_set:
struct clone_info {
int num_clone_high_words;
int *flags_high;
struct pid_set pid_set;
}
int clone_extended(int flags_low, void *child_stack, void *unused,
int *parent_tid, int *child_tid, struct clone_info *clone_info);
There are pros and cons to each approach, as Bhattiprolu points out. The
first requires a copy_from_user() for the flags in all cases
(though 64-bit architectures might be able to avoid that for now), while
the second requires the awkward splitting of the flags, but avoids the
copy_from_user() for calls that don't use the new flags or
pid_sets.
It is hard to imagine that copying a bit of data from user space will measurably impact a system call that is creating a process, though, so some derivative of the first option would seem to be the better choice. It's also a bit hard to see the need for more than 64 clone() flags, but if that is truly desired, something with a path for compatibility is needed.
There has been no objection to the implementation of
clone_with_pids(), but there have been few comments overall.
Pavel Machek wondered about the need for
setting the PID of anything but the inner-most namespace, but
Serge E. Hallyn noted that nested
namespaces require that ability: "we might be restarting an app
using a nested pid namespace, in which case restart would specify pids for
2 (or more) of the innermost containers
".
Machek also thought there should be a documentation file that described the new system call, and Bhattiprolu agreed, but is waiting to see what kind of consensus on either clone_with_pids() or clone_extended() (and which of the two interfaces for the latter) would emerge. So far, no one has commented on that particular aspect.
This
is version 4 of the patchset, and the history shows that earlier comments
have been addressed. It is still at the RFC stage, or, as
Bhattiprolu puts it: "Its mostly an exploratory patch seeking
feedback on the interface
". That feedback has yet to emerge,
however, and one might wonder whether Torvalds will still object to the
whole approach. It would seem, though, that there are too many important
applications for checkpoint and restart—including process migration
and the ability to upgrade kernels underneath long-running
processes—for some kind of solution not to make its way into the
kernel eventually.
Interrupt mitigation in the block layer
Network device drivers have been using the increasingly misnamed NAPI ("new API") interface for some time now. NAPI allows a network driver to turn off interrupts from an interface and go into a polling mode. Polling is often seen as a bad thing, but it's really only a problem when poll attempts turn up no useful work to do. With a busy network interface, there will always be new packets to process; "polling," in this situation, really means "going off to deal with the accumulated work." When there is always work to do, interrupts informing the system of that fact are really just added noise. Your editor likes to compare the situation to email notifications; anybody who gets a reasonable volume of email is quite likely to turn such notifications off. They are distracting, and there is probably always email waiting whenever one gets around to checking.NAPI is well suited to network drivers, since high packet rates can lead to high interrupt rates, but it has not spread to other parts of the kernel, where interrupt rates are lower. That situation could change in 2.6.32, though, if Jens Axboe follows through with his plan to merge the new blk-iopoll infrastructure into the mainline. In short, blk-iopoll is NAPI for block devices; indeed, some of the core code was borrowed from the NAPI implementation.
Converting a block driver to the blk-iopoll is straightforward. Each interrupting device needs to have a struct blk_iopoll structure defined for it, presumably in the structure which describes the device within the driver. This structure should be initialized with:
#include <linux/blk-iopoll.h>
typedef int (blk_iopoll_fn)(struct blk_iopoll *, int);
void blk_iopoll_init(struct blk_iopoll *iop, int weight, blk_iopoll_fn *poll_fn);
The weight value describes the relative importance of the device; a higher weight results in more requests being processed in each polling cycle. As with NAPI, there is no definitive guidance as to what weight should be; in Jens's initial patch, it is set to 32. The poll_fn() will be called when the block subsystem decides that it's time to poll for completed requests.
I/O polling for a device is controlled with:
void blk_iopoll_enable(struct blk_iopoll *iop);
void blk_iopoll_disable(struct blk_iopoll *iop);
A call to blk_iopoll_enable() must be made by the driver before any polling of the device will happen. Enabling polling allows that polling to occur, but does not cause it to happen. There is no point in polling a device which is not doing any work, so the block layer will not actually poll a given device until the driver informs it that there may be a reason to do so.
That normally happens when the device is actually interrupting. The driver can, in its interrupt handler, switch over to polling mode through a three-step process. The first is to check the global variable blk_iopoll_enabled; if it is zero, block I/O polling cannot be used. Assuming polling is enabled, the driver should prepare the blk_iopoll structure with:
int blk_iopoll_sched_prep(struct blk_iopoll *iop);
In the first version of the patch, a return value of zero means that the preparation "failed," either because polling is disabled or because the device is already in polling mode. In future versions, the sense of the return value is likely to be inverted to the more standard "zero means success" mode. If blk_iopoll_sched_prep() succeeds, the driver can then call:
void blk_iopoll_sched(struct blk_iopoll *iop);
At this point, polling mode has been entered; the driver need only disable interrupts from its device and return. The "disable interrupts" step should, of course, be done at the device itself; masking the IRQ line would be an antisocial act in a world where those lines are shared.
Later on, the block layer will call the poll_fn() which was provided to blk_iopoll_init(). The prototype for this function is:
typedef int (blk_iopoll_fn)(struct blk_iopoll *iop, int budget);
The polling function is called (in software interrupt context) with iop being the related blk_iopoll structure, and budget being the maximum number of requests that the poll function should process. In normal usage, the driver's device-specific structure can be obtained from iop with container_of(). The budget value is just the weight that was specified back at initialization time.
The return value should be the number of requests actually processed. If the device consumes less than the given budget, it should turn off further polling with:
void blk_iopoll_complete(struct blk_iopoll *iopoll);
Interrupts from the device should be re-enabled, since further polling will not happen. Note that the block layer assumes that a driver will not call blk_iopoll_complete() if it has consumed its full budget. If it's necessary to return to interrupt mode despite having exhausted the budget, the driver should either (1) use blk_iopoll_disable(), or (2) lie about the number of requests processed when returning from the polling function.
One might well wonder about the motivation behind all of this work. Block device interrupt handling has not traditionally been a performance bottleneck. The problem is the rapid improvement in solid-state storage devices. It is expected that, before too long, these devices will be operating in the range of 100,000 I/O operations per second - far beyond anything that rotating storage can do. When dealing with that many I/O operations, the kernel must take care to minimize the per-operation overhead in any way possible. As others have observed, the block layer needs to become more like the network layer, with the per-request cost squeezed to a bare minimum. The blk-iopoll code is a step in that direction.
How big a step? Jens has posted some preliminary numbers showing significant reductions in system time on a random-read disk benchmark. More testing will certainly be required; in particular, some developers are concerned about the possibility of increasing I/O latency. But the initial numbers suggest that this work has improved the efficiency of the block subsystem under load.
Patches and updates
Kernel trees
Architecture-specific
Build system
Core kernel code
Development tools
Device drivers
Filesystems and block I/O
Janitorial
Memory management
Networking
Security-related
Virtualization and containers
Benchmarks and bugs
Page editor: Jonathan Corbet
Distributions
News and Editorials
SUSE Studio for Linux appliances
At the end of July, Novell launched SUSE Studio, which it calls a
"simple and fast appliance builder
". It provides a free and
easy-to-use, web-based user interface to roll your own customized (SUSE)
Linux distribution. The resulting image can be deployed on bare metal or as
a virtual appliance on a hypervisor. Novell has also started a SUSE
Appliance Program to support independent software vendors creating
Linux-based software appliances.
First, why would one build such a "software appliance" with SUSE Studio? For companies, the advantages are obvious: an appliance tailored to a specific application set makes it easier to maintain and gives it a more predictable behavior. These companies can distribute one integrated package with the operating system and (server) software to their customers. But the same advantages hold for people who want to build a custom Linux distribution to distribute for free, who want do demo some application during a tradeshow, or who want a distribution with their own branding. SUSE Studio developer Cornelius Schumacher lists some concrete use cases:
SUSE Studio is in beta phase and the website says it's "currently
available for a limited number of users
", but anyone who sends their
email address via the SUSE
Studio login page will be put on the invitation list. Kindly asking one
of the operators in the #susestudio IRC channel for an invite tends to work
too, as your author found out. After receiving login information, signing
in can be done with
with OpenID among other options. SUSE Studio works in Firefox 2 and 3 on Linux, Mac OS X and
Windows, and in Safari 3 on Mac OS X.
From template to installable image
After the first login, the system offers some templates to build the appliance upon. For the base system, the website gives the choice among openSUSE 11.1, SUSE Linux Enterprise 10 and SUSE Linux Enterprise 11. There are templates for JeOS (Just enough OS, a minimalist server appliance), Server, GNOME Desktop, KDE 3 or 4 Desktop, and Minimal X. The last one, which uses the IceWM window manager, is especially useful for making a kiosk-like application. There is also a choice between 32 or 64-bit architectures and a text box to give the appliance a descriptive name.
The user navigates through the rest of the steps by clicking on some tabs. In the Software tab, one can add or remove packages and repositories or upload their own rpm files. Some basic knowledge of the available repositories and software packages is needed, but the interface has a helpful search function. SUSE Studio is also integrated well with the openSUSE Build Service (which enables users to create packages for various Linux distributions from source). One can build packages in the openSUSE Build Service, import the repository into SUSE Studio and use the freshly built packages. For example, this makes it possible to create an openSUSE distribution with KDE 4.3, although by default KDE 4.1 is installed.
The "Configure" tab gives a slew of options to configure: locale settings, timezone, network, firewall, users and groups, the look and feel, the default runlevel, a custom license agreement, MySQL configuration, custom boot scripts, and so on. In the "Overlay files" tab one can add overlay files, if, for example, some custom scripts are needed or some configuration files in /etc have to be changed. Clicking on the "Build" tab gives the choice among a disk image, a live cd, a VMware image, or a Xen image. Support for creating images in OVF (Open Virtualization Format, an open standard for packaging and distributing virtual appliances), Microsoft's Hyper-V, and Amazon's EC2 AMI format (including deployment to EC2) is planned. The build process itself takes less than five minutes most of the time, even for a complete desktop system. The result can then be downloaded.
By looking at some details, one can see that SUSE Studio is well-engineered and has undergone a lot of usability tweaking. For example, in the "Software" tab there's a "Recommended" list of applications, which is different depending on the template the user has chosen. Moreover, at any time the current disk footprint of the appliance is shown at the left side of the web page, which is useful while building an image which has to fit on a CD or a USB thumb drive. The left side also shows helpful messages and tips. For example, while building a VMware image, it suggests installing the open-vm-tools package. And, after adding a package, one can view more details about the package, such as which dependencies are installed. If the firewall is enabled in the "Configuration" tab and it is not installed (for example while building upon a JeOS template), then an error message appears with a button to add SuSEfirewall2. There are a lot of these tips, which seem to come from the developers' experiences and feedback by users of the alpha version.
An appliance in your browser
One great feature is the SUSE Studio Testdrive. This allows the user to boot and test their appliance in a browser window without the need to download it first. After the build, clicking on the Testdrive button will launch the virtual machine. This runs on a KVM hypervisor on Novell's servers and exposes the virtual machine framebuffer via VNC to a Flash applet running in the user's browser. There are even buttons to switch to a different virtual console, to press ctrl-alt-del, to press ctrl-alt-backspace, or to change the keyboard layout. Each Testdrive instance gets 512 MB of RAM and an hour to run.
One can also make changes to files in Testdrive, investigate the diffs, and commit those changes to the appliance. The changes will then be implemented when starting a new build. This is really nice, as one can, for example, fire up an interactive installer of an application and commit the changes, so that appliance users don't have to. SUSE Studio creator Nat Friedman explains in his blog how this is implemented with a copy-on-write (COW) disk image for the KVM image. The original image is only used for reading and all writes go to the COW file. Whenever the virtual machine wants to read a block, it first checks for its presence in the COW file, and falls back to the read-only image if not:
Many possibilities
The SUSE Studio website is a closed-source Ruby on Rails web application, but there are plans to open up the code sometime in the future. Later this year, companies can buy SUSE Studio Onsite, which is a version on a server hosted within their own data center. SUSE Studio uses the open source Kiwi build tool as its backend. When the user has created an appliance with SUSE Studio, he can download the appliance description from SUSE Studio and build the appliance later, directly with Kiwi. Some Kiwi recipes can be found in the Kiwi cookbook on openSUSE's wiki. Kiwi also supports generating an EC2 AMI, so this slight detour is a perfect way to generate an AMI for an appliance made by SUSE Studio if you can't wait for the support coming later this year.
A big selling point of SUSE Studio is the documentation on openSUSE's wiki. This varies from a FAQ, a tour, HOWTOs, and the API, to known issues, support channels, and downloads. The last features some appliances that users have built with SUSE Studio, such as a Mono 2.4 appliance, an Elisa Media Centre appliance, openSUSE images for netbooks, and a demo live cd of Marble Desktop Globe.
The HOWTOs seem to be of good quality and guide you through some elaborate tasks. The general HOWTOs are mostly related to firstboot scripts and kiosk-like appliances, while there are more specific KDE and Gnome HOWTOs. Other HOWTOs are related to the type of deployment, such as VMware, VirtualBox (the VMware images run fine in VirtualBox), disk images and live CDs.
An appliance in fifteen minutes
In the last few months, your author has tested some other Linux appliance building tools as well, and can say with confidence that SUSE Studio was the most impressive. While rBuilder Online is also a handy and efficient tool, which supports more Linux distributions than just the SUSE family and has a neat management console for the appliance, SUSE Studio is much more user-friendly. Another solution, VMware Studio 1.0, also doesn't match the user experience of SUSE Studio: the user has to run this appliance builder in VMware and download a whole DVD image for the distribution the appliance will be based on. Moreover, building the image can take hours and has to start over from scratch when the user makes the slightest error, such as a typo in a URI. Advantages of VMware Studio are the support for a lot of Linux distributions and the excellent documentation.
All in all, compared to its competitors, SUSE Studio is without doubt the most easy-to-use, the most well-engineered, and the most efficient appliance builder. Even someone without any previous experience can build a software appliance in fifteen minutes. If it would only offer more distributions than just the SUSE family for the operating system base, it would be even more interesting.
New Releases
Arch Linux 2009.08 available
Arch Linux has announced that 2009.08 snapshot images are available. "As you can see, just like with 2009.02 there have been some large changes in our installation tools. Also the actual building process and environment has been improved a lot since we started the releng project. So it should be more straightforward to build snapshots in the future. As usual, the images come in iso and img flavors, with grub and isolinux for people whose cd-rom drive doesn't work with grub. Due to all mentioned changes, the installation guide has also been overhauled. The new guide is for now only available in English but we'll add other versions when we receive translated versions."
openSUSE 11.2 Milestone 5 Released
openSUSE 11.2 Milestone 5 (M5) is now available for download. "It includes several updates, new features, bugfixes, and other improvements. This milestone includes KDE 4.3 final, a kernel built specifically for desktop systems, and beta 1 of OpenOffice.org 3.1.1."
Slackware 13.0 RC2
The August 6 Slackware current changelog entry announces the release of Slackware 13.0 RC2. "Hi folks! We're going to call this set of updates RC2. There are still README files to be handled (that's the nature of documentation, I guess), as well as some other things remaining on the TODO list, but X seems pretty stable now, and it seems like a release soon would be in order so that we can march right back into development territory with -current again soon. :-) Enjoy, and let us know about any problems you run into!"
Distribution News
Debian GNU/Linux
Debian Etch and Ubuntu Feisty: a comparison
Anthony Towns has posted an interesting comparison of the packages found in the Ubuntu "Feisty" release and the Debian "Etch" release that Feisty was loosely based on. It turns out that 6874 packages have the exact same source, while 2273 appear only in Ubuntu. There are 31 security updates to Etch that Ubuntu didn't get, but none in the core part of the distribution. Anthony concludes: "In any event, seems like there's more room for collaboration there at first glance."
Update: here's a new analysis from Anthony with more detailed statistics and a look at the "Intrepid" and "Jaunty" releases.
Fedora
Slip of Fedora 12 Alpha by one week
The release of Fedora 12 Alpha has slipped a week. "Today in the release engineering meeting, it was decided to enact a one week slip of the Fedora 12 Alpha release date. This is due to remaining bugs on the F12Alpha tracker preventing creation of a release candidate and preventing testing of proposed fixes. We expect to be able to test/clear the list early this week, therefor only a week slip is needed at this time. The new Alpha release date August 25th. As soon as we have a successful Alpha compose we will lift the Alpha freeze and allow rawhide to move forward."
Fedora 11 Release Events Contest Winners Announcement
Fedora Ambassadors are honored for their contributions to the Fedora Project, in particular those that organized great Fedora 11 release events. "It's my pleasure and honour to announce the F11 Release Events Contest winners: Kevin Higgins with the Vancouver, WA (USA) release event, Neville A. Cross with the Managua (NI) release event and, finally, Truong Anh Tuan with the Hanoi (VN) release event."
Fedora Advisory Board Meeting Summary
Click below for a summary of the August 6, 2009 meeting of the Fedora Board. Topics include Russian Fedora initiative, BitTorrent stats counter, Extended Life Cycle, and Thai community request from FAB.
Gentoo Linux
Gentoo mourns the passing of Ferris McCormick
The Gentoo community has sent out an eulogy for Ferris E. McCormick, known as "fmccor," who passed away on August 5. "Ferris joined Gentoo on April 16th 2004 as part of the sparc team and improved sparc support for the entire open source community. Within a year he also joined the Developer Relations team to help with mediation of any issues that might come up between people. As time went on Ferris continued to expand and assist Gentoo in many ways including assisting with the User Relations team and growing to become the Strategic Manager of the sparc project. Finally, he became a trustee and the Vice President of the Foundation assisting in getting the foundation back into good standing." There is a forum where thoughts can be posted.
SUSE Linux and openSUSE
Novell increasing openSUSE support
Novell manager Roland Haidl has sent out a note stating that the company has decided to increase the resources dedicated to the openSUSE project. "It is not longer the 'when time is left, please work in the openSUSE project' thing we often had before, we now have the singular situation to have a team of more than 10 experts in Novell to only work on openSUSE community topics. This is the Novell 'openSUSE Team', and it is there to be a part of the community and make it easier for people to join in, enjoy and contribute."
Unofficial openSUSE KDE 4.3 RPMs and Live CDs
The KDE Project has announced that stable KDE 4.3 is available via One-Click Install for openSUSE Factory, 11.1, 11.0, and 10.3. There is also a live CD using the openSUSE Build Service and KIWI.Advance notice of discontinuation of openSUSE 10.3
SUSE Security has announced that openSUSE 10.3 will be discontinued soon. "Having provided security-relevant fixes for two years, we will stop releasing updates after October 31st 2009."
Ubuntu family
Ubuntu removes "multisearch"
A new comment in Ubuntu's Launchpad system shows that the controversial "multisearch" feature (described in this LWN article) has been removed; it will not appear in the "Karmic Koala" alpha 4 release.Ubuntu Karmic GRUB menu hiding
The GRUB boot menu in Ubuntu's Karmic Koala (9.10) release is hidden by default. "If you're upset by the boot menu being hidden all of a sudden, then you should edit /etc/default/grub, comment out the GRUB_HIDDEN_TIMEOUT line, and set GRUB_TIMEOUT to the timeout you want in seconds (say "10"), then run 'sudo update-grub'."
Minutes from the Ubuntu Technical Board meeting
Click below for the minutes from the August 11, 2009 meeting of the Ubuntu Technical Board. Topics include Review of outstanding actions, Debian TC liaison, Technical Board nominations, Ubuntu security policies, Handling community problems, Investigate alternative to Google CSE, and Statistics gathering policy.
Distribution Newsletters
Arch Linux Magazine
The Arch Linux Magazine for August 2009 is available. Topics include news from Devland, a Featured Interview with Loui Chang, community highlights, Revision Control System, Tips and Tricks, and a software review of video editors.DistroWatch Weekly, Issue 315
The DistroWatch Weekly for August 10, 2009 is out. "Pardus Linux is one of those lesser-known distributions that many people overlook in favour of the usual big names. But this independently-developed project, generously financed by the Turkish government, is a rather surprising package - a user-friendly operating system with an excellent package management system and a wealth of custom utilities. Read our first-look review of the recently released Pardus Linux 2009 to find out more. In the news section, KDE 4.3 makes its first appearance in many popular distributions, Mandriva's "Cooker" continues its rapid march towards the next stable release, Sabayon Linux publishes an update over its next major release, and OpenBSD announces the release date for its upcoming version 4.6. Finally, don't miss the announcement about the new Yellow Dog Linux on a Stick, a 16 GB live USB Flash drive for Sony PLAYSTATION 3. Happy reading!"
Fedora Weekly News 188
The Fedora Weekly News for August 10, 2009 is out. "This week's issue begins with some detail on the recent Fedora Classroom events, and updates on Fedora 12 alpha. In news from the Fedora Planet, a multi-part series on OCaml internals, a few proposals for a new Fedora website design, and coverage of a session on Sanskrit and usage in computing. We're pleased to bring news from the Fedora Marketing team back to you with a new beat member, Mel Chua. In Marketing news, pointers to the latest team meeting log, details about Fedora Insight, and a transition of leadership on the Marketing Team. In Quality Assurance news, details of the upcoming Test Day on NetworkManager, many updates on the weekly meetings and availability of a new Xfce spin for testing. In Translation news, many updates on the progress to Fedora 12 Alpha translation items, a proposal to translate some more pages from docs.fedoraproject.org and the landing of Transifex v0.7, a tool used by the localization teams. This week's issue rounds out with news from the Art/Design team, with more progress on Fedora 12 wallpapers for the F12 Alpha release, and coverage of a discussion about a Design Spin on the Art Team discussion list. These are just a few highlights of this week's FWN. Enjoy!"
Openmoko Community Updates
Openmoko Community Updates are available every two weeks. The August 6 edition covers the QTMOKO and Qalee distributions, plus new applications and other community news.OpenSUSE Weekly News/83
This issue of the openSUSE Weekly News covers a review of KDE 4.3, Marek Stopka: YaST Education module is no more GSoC project, Linux.com/Rob Day: The Kernel Newbie Corner: What's in That Loadable Module, Anyway?, Jeff Jaffe: Cloud Securityv, Nat Friedman: Running Linux in the browser, and more.Ubuntu Weekly Newsletter #154
The Ubuntu Weekly Newsletter for August 8, 2009 is out. "In this issue we cover: Karmic Alpha 4 freeze ahead, Landscape: Canonical Systems Management & Monitoring Tool, ubuntu-ph.org is back in business, First Launchpad community meet-up, Code Hosting quick-start guide, notify-osd 0.9.16 released, Migrating to an Encrypted Home Directory, Touchscreen = fail?, Ubuntu-UK Podcast: Day of Reckoning, Full Circle Magazine #27, and much, much more!"
Distribution reviews
An Early Look at Ubuntu 9.10 Karmic Koala (Tuxarena)
Tuxarena takes an early look at Ubuntu 9.10. "In this article I'll overview the latest update of Karmic, after installing the Alpha 3 and performing a full sudo apt-get update && apt-get dist-upgrade. Currently, Karmic Koala comes with Linux kernel 2.6.31, GNOME 2.27.5 as desktop environment, Empathy as the default chat application, OpenOffice 3.1 as the office suite, GIMP 2.6.6 as image editor. Firefox 3.5 is not yet included in the Karmic repositories, but it will surely be available in the October release, so currently 3.0.13 is the version in the repositories."
Xandros Presto Linux (PCWorld)
PCWorld reviews Xandros Presto Linux. "Xandros Presto Linux is - for want of a better term - a 'lightweight' version of the company's Debian-derived Linux distribution, and no prizes for guessing the thinking behind its name. Chopped down to a basic stub that boots in the presence of Windows, it fires up in seconds, and shuts down equally rapidly. Loading it from a laptop also running Windows XP, it becomes obvious how bloated Windows has become over the years by comparison, even in its supposedly clean XP form."
Page editor: Rebecca Sobol
Development
Translating software with Pootle
Pootle is Free/Open Source, web-based software, aimed at making the process of translation of other software simpler. As a typical translation team's workload increases with each new software release, it becomes more and more important to have a smooth workflow, not only for managing the translations, but also for managing the translators as well. Simultaneously, to ensure that translators from all kinds of backgrounds are able to participate, the entire process needs to be kept as accessible and easy as possible. Pootle's goal is to make this a reality.
Pootle is written in Python and is used by FOSS projects such as OpenOffice.org, One Laptop per Child/Sugar Labs, LXDE, Sahana, GNU Mailman, etc. A list of live Pootle installations is maintained on the Pootle wiki. In its backend, Pootle utilizes the Translate Toolkit, which provides an API for a comprehensive set of localization related functions, along with support for multiple translation file formats such as PO, XLIFF, QT .ts, CSV, and Mozilla .properties. Using this backend, Pootle offers a web-based interface for managing translators, adding, modifying and suggesting translations, and support for connecting to a number of version control systems (CVS, SVN, Git, Mercurial, etc).
In a typical Pootle installation the server configuration points to the po/ subdirectory of a project (or an appropriate symlink). The project may be a checkout/clone of a remote repository or standalone. In the former case the language team lead (or other authorized team members) can commit (and push) translations to the remote, upstream repository via the web interface. If the PO files are not part of a repository, the integration step is usually carried out manually. All the PO files (or equivalent translation files) in the po/ directory are exposed via the web UI, and the messages in the files are presented as distinct units to the translators, who can translate them (or edit the translations) using the browser.
Pootle's web-based translation interface lowers the barrier to entry considerably, this is crucial for ensuring success for translation teams (especially those with limited resources and volunteers). Translators often come from non-technical backgrounds, so the web-interface, by abstracting away the underlying complexity of multiple version control systems and the gettext toolchain, makes their work considerably easier.
The web-based interface, despite simplifying things with its ease of use can sometimes exclude people from participating. One barrier is that it relies on continuous Internet access, which may be non-existent or prohibitively expensive in many regions of the world. Fortunately, Pootle provides a feature which allows translators to download PO files, translate offline, and re-upload the files.
This "offline-translation" feature can be highly useful for translation "sprints", where a group of volunteers gather together, translate a set of downloaded PO files using offline tools such as Poedit or Virtaal, then upload back the files to the Pootle server.
However, the overall ease of use brings up yet another potential problem. With a large number of translators from various backgrounds participating in the translation process, quality issues invariably crop up. Such issues can be divided into two broad categories:
-
Mistranslations: These can be either plain "wrong" translations, or errors like incorrectly typed format placeholders or an incorrect number of escape sequences (e.g. '\n'). This can be partially addressed by having access control rules in place. Pootle has the provision for different levels of access, where a user can be assigned various rights on the basis of the user's experience level. Using this feature, a user can be allowed to perform any combination of tasks such as translating, suggesting possible translations, reviewing suggested translations, committing translations, etc.
To ensure that the quality level of a new contributor is up to the mark, many language team coordinators choose to set the default access level to "suggest". When the initial set of suggestions are found to be satisfactory, the access level can be increased to "translate", "review", or even higher.
Another way of ensuring high quality translation is to use the built-in "checks" feature offered by the software. Currently there is a set of 47 checks in Pootle which test for common errors in translations. Some of these tests, like the capitalization check are language and writing system specific. But others, like the check for translated file paths and function names (normally file paths or function names are kept untranslated) are applicable universally and can be quite useful.
For translators not sufficiently familiar with English (messages to be translated are almost invariably in English), the ability to view reference translations in a third language often helps. Support for this viewing of an alternate language was added in the last stable release of Pootle (version 1.2). For example, this feature was found to be very useful in OLPC's South American deployments, where translators for languages like Aymara and Quechua wanted to see Spanish translations while translating, to better understand the meaning of the original English strings.
A combination of the above three features, if used properly, can reduce mistranslations to a large extent.
-
Inconsistency: In this commonly occurring problem, two or more translators may translate the same term differently. For example, for a certain term, one translator might want to keep an English transliteration, while another may want to use the literal meaning of the term. This almost certainly confuses the end user, and ensuring consistency is important. To address this problem, Pootle provides a glossary, or terminology feature, where a single set of translations for commonly used terms can be stored. During translation, the appropriate entry from this set is provided as a suggestion whenever the translator comes across a message containing a term from the glossary. This particular feature can also be useful if there is government or language-body sanctioned terminology for computer-related vocabulary.
Though deciding on and creating a glossary of terms manually can be difficult for projects with a large string count, the Translate Toolkit provides a command line tool called poterminology to automatically produce a list of commonly occurring terms from a set of PO or POT (PO Template) files.
With the features outlined above, Pootle can make the life of translators in any FOSS project considerably easier. However, from the author's experience at OLPC/Sugar Labs, there can been a few problems along the road as well. Any project thinking of adopting Pootle should ideally factor in these issues in before taking the plunge:
-
Speed issues: Pootle can be quite slow during certain operations involving large PO files (as an example, the largest of the PO files served by the OLPC/Sugar Labs Pootle installation has around 4400 strings). These operations that can make Pootle utilize 100% CPU on the hosting server for a significant amount of time. Operations such as merging two large PO files (which usually happens during uploads), updating a set of PO files from the corresponding set of updated reference POT files, or searching through large files for untranslated strings, are examples of potentially lengthy tasks.
Newer versions of Pootle (and the Translate Toolkit) have seen the introduction of sqlite-based databases for storing translation statistics and the use of Lucene/Xapian based indexing of PO files. These changes have significantly improved the search performance.
-
Communication: Pootle itself does not provide for any mode of communication within teams or between site-administrators and teams. This can be addressed partially by using a mailing list for each translation team, as well as a special "all-hands" mailing list for general announcements and reminders. However, even if mailing lists are used, it becomes increasingly difficult to get in touch and coordinate with individual translators and team leads as the translation community scales up. This can become an especially pressing issue toward the end of a release cycle; often translation leads need to be reminded individually about committing the work they have completed so far.
On the other hand, the Pootle developers seem to be aware of the urgency of the issue, and there is currently a page in the wiki outlining the various use-cases for this feature. Some of the required features have already been implemented in Pootle's SVN trunk.
-
Error checking: The error checking feature in Pootle needs to invoked manually (via the "checks" system mentioned before). This means that there is always a chance that wrongly formatted PO files may get introduced into the software's source tree, causing the build to break in some cases. Of course, this is not an issue if the language team lead always ensures that at least the the critical "checks" pass for the PO file before a commit is made.
Despite the above weaknesses, from an overall perspective Pootle can be a useful component of any FOSS project's localization arm. In fact, it is not only the translators who benefit from the features that Pootle offers, developers only have to grant repository access to one user account (the one used by Pootle) to take care of all translations. The process of providing the requisite repository access to each translator or language team has been a major bottleneck for quite a few FOSS projects in the past.
As of August 2009, Pootle has been undergoing significant changes in both code and visual design. Along with a switch to the Django web framework, the user interface has changed radically. Now, overall statistics for each project and language are viewable from the first page and UI changes are visible in the individual project/language pages. Among the other user-visible changes that are supposed to ship with Pootle 1.3, notable features include support for multiple alternate languages and initial implementation of intra-team communication methods. Also, with the shift to Django, which is much more popular compared to the older jToolkit framework, it should become easier for external developers to extend and customize Pootle.
The code for Pootle 1.3 (pre-release) can be downloaded from the SVN repository of the project. It is pretty much usable in its current form, though probably not recommended for production use. However, the project does need more help with testing, and reports about any bugs in the latest version from SVN are welcome.
With the impending release of Pootle 1.3, along with the fast pace of growth of other similar tools (Transifex, as well as the recently open-sourced Rosetta), things certainly do look bright for FOSS translation tools. In the next few months, we should be able to see significant improvements in the way Open Source translation is done, and through all these changes, the general translation workflow for FOSS will be greatly improved.
System Applications
Database Software
Firebird 2.5 Beta 2 kits released
Version 2.5 Beta 2 of the Firebird DBMS has been announced. "The Firebird Team is pleased to release the second Beta of the new Firebird 2.5 version for field-testing. 32-bit and 64-bit builds for Linux, Windows and MacOSX Intel are available."
PostgreSQL Weekly News
The August 9, 2009 edition of the PostgreSQL Weekly News is online with the latest PostgreSQL DBMS articles and resources.SQLite release 3.6.17 released
Version 3.6.17 of the SQLite DBMS has been announced. "Changes associated with this release include the following: * Expose the sqlite3_strnicmp() interface for use by extensions and applications. * Remove the restriction on virtual tables and shared cache mode. Virtual tables and shared cache can now be used at the same time. * Many code simplifications and obscure bug fixes in support of providing 100% branch test coverage."
SQLObject 0.11.0 released
Version 0.11.0 of SQLObject has been announced, it adds a number of new capabilities. "SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB)."
Interoperability
Samba 3.0.36 maintenance release available
Maintenance release 3.0.36 of Samba has been announced. "This is the latest bug fix release for Samba 3.0 series." Also, the first edition of the Samba Team Blog has been published.
Web Site Development
Apache HTTP Server 2.2.13 released
Version 2.2.13 of Apache has been announced. "This version of Apache is principally a security and bug fix release. Notably, this version bundles the APR Library version 1.3.8 and APR Utility Library version 1.3.9, which address a security concern which may be triggered by some third party modules. We consider this release to be the best version of Apache available, and encourage users of all prior versions to upgrade."
Apache ODE 1.3.3 announced
Version 1.3.3 of Apache ODE has been announced. "I'm pleased to announce the release of ODE 1.3.3, a security release of Apache ODE. It fixes a vulnerability in the process deployment that allowed, using a forged message, to create, overwrite or delete files on the server file system. See the full vulnerability announcement below. Apache ODE is a WS-BPEL compliant web service orchestration engine. It organizes web services calls following a process description written in the BPEL XML grammar. Another way to describe it would be a web-service capable workflow engine."
Desktop Applications
Data Visualization
matplotlib 0.99 released
Version 0.99 of matplotlib, a data visualization package, is out with numerous bug fixes. See the CHANGELOG file for details.
Desktop Environments
GNOME Software Announcements
The following new GNOME software has been announced this week:- Accerciser 1.7.90 (translation work)
- atk 1.27.90 (translation work)
- ccss 0.5 (new features)
- Cheese 2.27.90 (new features, code cleanup and translation work)
- Clutter 1.0 (code branch)
- Deskbar-Applet 2.27.90 (translation work)
- Evince 2.27.90 (bug fixes and translation work)
- F-Spot 0.6 (unspecified)
- GCalctool 5.27.90 (bug fixes, documentation and translation work)
- GNOME Commander 1.2.8.1 (bug fixes)
- GNOME DVB Daemon 0.1.8.1 (new features and bug fixes)
- gnome-keyring 2.27.90 (bug fixes and translation work)
- GOK 2.27.90 (bug fixes and translation work)
- GParted 0.4.6 (bug fixes and translation work)
- gtk-css-engine 0.3.1 (new features)
- Java ATK Wrapper 0.27.6 (bug fix)
- libchamplain 0.3.90 (bug fix)
- moserial 2.27.4 (bug fixes, documentation and translation work)
- mousetweaks 2.27.90 (translation work)
- Nautilus-Actions 1.12.0 (new features, bug fixes and translation work)
- Orca 2.27.90 (bug fixes and translation work)
- PyClutter 0.9.2 (new features)
- PyGobject 2.19.0 (new features and bug fixes)
- Sabayon 2.27.0 (new features, bug fixes and translation work)
- seahorse 2.27.90 (bug fixes, documentation and translation work)
- Tangerine 0.3.2 (bug fixes)
KDE Software Announcements
The following new KDE software has been announced this week:- 2ManDVD 0.9.1 (new features and bug fixes)
- 2ManDVD 0.9.2 (bug fixes)
- 2ManDVD 0.9.3 (bug fixes and translation work)
- Choqok 0.6.6 (new features and bug fixes)
- Electronic Business Assistant 1.0.0.1 (unspecified)
- Frescobaldi 0.7.13 (bug fixes and translation work)
- KDevelop 3.5.5 (new features)
- KMidimon 0.7.1 (new features and bug fixes)
- Konversation 1.2-alpha6 (bug fixes)
- KTorrent 3.2.3 and 3.3beta1 (new features and bug fixes)
- ktv 0.1 (initial release)
- 'Q' DVD-Author 1.11.0 (new features and bug fixes)
- QComicBook 0.4.1 (bug fixes and code cleanup)
- qt gui for amule 0.1 (initial mockup release)
- Simple Root Actions Menu 2.2.0 (new feature and translation work)
- Skanlite 4.3 (unspecified)
Xorg Software Announcements
The following new Xorg software has been announced this week:- font-util 1.0.2 (bug fixes and documentation work)
- inputproto 1.9.99.901 (code cleanup and documentation work)
- libSM 1.1.1 (bug fixes, code cleanup and documentation work)
- pixman 0.15.20 (new feature and bug fixes)
- xextproto 7.1.0 (code cleanup)
- xf86-input-evdev 2.2.4 (bug fix)
- xf86-input-vmmouse 12.6.5 (bug fixes and code cleanup)
- xrandr 1.3.1 (bug fixes)
Desktop Publishing
Scribus 1.3.5 released
Version 1.3.5 of the Scribus desktop publishing application has been announced. "This release is a major development version on the road to milestone 1.4, which will be based on 1.3.5. Following many testing cycles, the Scribus Team considers this version to be fairly stable and ready for many real-world use cases."
Encryption Software
gnupg 0.2.1 released
Version 0.2.1 of gnupg has been announced, it includes minor bug-fixes. "The gnupg module allows Python programs to make use of the functionality provided by the Gnu Privacy Guard (abbreviated GPG or GnuPG). Using this module, Python programs can encrypt and decrypt data, digitally sign documents and verify digital signatures, manage (generate, list and delete) encryption keys, using proven Public Key Infrastructure (PKI) encryption technology based on OpenPGP."
Interoperability
Wine 1.1.27 announced
Version 1.1.27 of Wine has been announced. Changes include: "- New version of the Gecko engine. - New GSM 06.10 codec support. - Improved support for the disk volume APIs. - Support for XShm pixmaps for better performance. - Various bug fixes."
Multimedia
Moovida Media Center 1.0.6 released
Version 1.0.6 of Moovida Media Center has been announced. "This new version fixes many important bugs (see attached release notes) among which crashes on windows (reduced by 35%) and improves the overall performance (the UI is twice as fluid and Moovida uses 10MB less memory)."
Office Applications
Roundup Issue Tracker 1.4.9 released
Version 1.4.9 of Roundup Issue Tracker has been announced. "I'm proud to release version 1.4.9 of Roundup which fixes some bugs".
Office Suites
OpenOffice.org Newsletter
The July, 2009 edition of the OpenOffice.org Newsletter is out with the latest OO.o office suite articles and events.
Digital Photography
digiKam digest (KDEDot)
The August 8, 2009 edition of the digiKam digest has been announced. "This week we saw: fixing of Exif rotation flags; new layout of RemoveRedEyes KIPI plugin; new widget for lists of images for KIPI plugins; refactoring of ICC code for cleaner code-base, preparation for general bug fixing and easier maintenance in future. More screenshots for newest KIPI plugin - Local contrast plugin."
Miscellaneous
New releases of Mozilla Lightning and SOGo
Version 1.0.4 of SOGo and version0.9.6 of Mozilla Lightning "Inverse Edition" have been announced. "SOGo provides a rich AJAX-based Web interface and supports multiple native clients through the use of standard protocols such as CalDAV, CardDAV and GroupDAV. It features a very tight integration with Mozilla Thunderbird and Lightning and enable mobile devices synchronization through the use of the Funambol middleware. Version 1.0.4 provides new features such as Apple iPhone OS 3 support, better Apple iCal 3 support and memcached support for high-scalability. Moreover, Inverse releases Lightning "Inverse Edition" v0.9.6. This is a maintenance release of Mozilla Lightning..."
Languages and Tools
C
GCC 4.3.4 released
Version 4.3.4 of GCC has been announced. "GCC 4.3.4 is a bug-fix release containing fixes for regressions and serious bugs in GCC 4.3.3."
Perl
"Useful and usable" Perl 6 release coming in April
Patrick Michaud, the Rakudo Perl 6 "pumpking," has announced that a usable version of Perl 6 is coming in April, even if the language will not be "finished" by then. "To me, one good answer is to start making releases of Perl 6 that may not implement the entire Perl 6 specification, but that application writers will feel comfortable enough to start using in their projects. I've started to call these 'useful releases' or 'usable releases'. While it might not have every feature described in the Perl 6 synopses, enough features will be present that can make it a reasonable choice for application programs."
Perl 5.10.1 RC1 released (use Perl)
Version 5.10.1 RC1 of Perl has been announced. "This is the first maintenance release of the 5.10 series; it provides numerous bug fixes, improvements and core module updates from CPAN. Read the full list of changes in the perldelta document."
Test-Run 0.0120 released
Version 0.0120 of Test-Run has been announced, it includes some code rewriting. "Test-Run is an improved test harness for TAP based test streams. Originally forked from Test-Harness-2.x, it has been heavily modularised and extended, and was ported to use TAP-Parser. It has been split into a front-end, a back- end, a prove-like script-in-a-module, all with several OOP classes, and has several optional plugins on CPAN for such things as colouring the output, using alternate interpreters for running the TAP scripts and trimming the displayed filenames. More plugins can be written."
Python
CodeInvestigator 0.14.0 announced
Version 0.14.0 of CodeInvestigator, a tracing tool for Python, has been announced. "Changes: Unicode characters. They are handled better. Performance changes: The database is written more efficiently, so that recording takes less time. Performance tip: deleting ALL runs at the same time is fastest; I then initialize the database."
Distribute 0.6 released
Version 0.6 of Distribute has been announced. "Distribute is a friendly fork of the Setuptools project." Click below for change details.
M2Crypto 0.20 released
Version 0.20 of M2Crypto, a Python wrapper for OpenSSL, has been announced. "I am please to announce the M2Crypto 0.20 release, which was in development for over nine months. Over 30 bugs fixed by more than ten people."
python-utmp 0.8 released
Version 0.8 of python-utmp has been announced, the main change is Python 3 support. "python-utmp provides 3 modules to access utmp and wtmp records: * utmpaccess is lowlevel module written in C, wrapping/emulating glibc functions * UTMPCONST provides useful constants * utmp is module build on top of utmpaccess module, providing object oriented interface."
Python-URL! - weekly Python news and links
The August 6, 2009 edition of the Python-URL! is online with a new collection of Python article links.
Tcl/Tk
Tcl-URL! - weekly Tcl news and links
The August 6, 2009 edition of the Tcl-URL! is online with new Tcl/Tk articles and resources.
Test Suites
oejskit 0.8.6 announced
Version 0.8.6 of oejskit has been announced, it includes py.test 1.0 compatibility fixes. "jskit contains infrastructure and in particular a py.test plugin to enable running unit tests for JavaScript code inside browsers. The plugin requires py.test 1.0".
Page editor: Forrest Cook
Linux in the news
Recommended Reading
Widenius: thoughts on dual-licensing
Michael "Monty" Widenius has posted an interesting article on the rather severe restrictions added to Sun's new commercial MySQL license [PDF] and how dual-licensed projects should really work. "I believe one should be very permissive when doing dual licenses with Open Source as otherwise you lose many of the business advantages you get from being Open Source. The Open Source community is a very effective ecosystem and if you allow it to participate with your business you have a better chance to succeed."
Peters: The Desktop or the Browser: Is the Netbook Escalating the Battle?
GNOME Foundation Executive Director Stormy Peters writes about her upcoming presentation at OpenSource World on her blog. She is concerned that moving applications into the browser is limiting users. "One of the things that has worried me is how people are living more and more in their browser. I myself am guilty of this. I use the browser to check my mail, calendar, read news, track my todo list, check my bank account, check on friends, upload pictures ... [...] People doing everything in their browser scares me not because I think everyone should use the desktop but rather because I don't think the browser is the best user tool for doing all those things."
Trade Shows and Conferences
Free Desktop Communities come together at GCDS (KDEDot)
Sebastian Kügler reports on the success of the recent Gran Canaria Desktop Summit. "KDE and GNOME benefit from shared technologies in multimedia, metadata storage, desktop search, application messaging and hardware integration. These shared technologies provide users with improved integration and a consistent user experience. Discussions during the summit resulted in agreements to continue to work on shared technologies, shared interfaces and shared code. In particular, several working sessions around the freedesktop.org initiative resulted in clearer processes for for sharing specifications and technologies which will accelerate the ability of both projects and the greater free desktop community to collaborate and communicate with other projects."
The SCO Problem
Chapter 11 Trustee to be Appointed; SCO's Sale Motion Denied (Groklaw)
Groklaw reports the latest news from the SCO bankruptcy hearing. "The judge in the SCO bankruptcy has ruled at last. SCO's motion to let it sell to unXis is denied. There could be an auction later. The motions to convert to Chapter 7 by IBM, Novell and the US Trustee's Office are also denied, but alternative relief is granted, and there will be a Chapter 11 trustee appointed. IBM and Novell agreed that a Chapter 11 Trustee was appropriate if he did not convert to Chapter 7, and that is what he has done. That means presumably that SCO management no longer run this show."
Companies
Red Hat pushes certification program amid Linux wave (Computerworld)
Computerworld talks to Red Hat's Max McLarenon about the company's expansion of its web-based training program in Australia. "The Academy program is currently available through just over 10 colleges that include TAFE NSW, Canberra Institute of Technology, the Burnie campus in Tasmania and Chisolm TAFE in Victoria. General manager for Red Hat in Australia and New Zealand, Max McLaren, said the internationally available certification is particularly popular in Australia. We have more Red Hat Certified Engineers per capita then any where else in the world and one of the reasons the certification is so successful here is that we have so many colleges to help us offer it, he said. McLaren said that the demand for training is still healthy despite the economic downturn."
Red Hat's JBoss road less traveled (CNet)
Matt Asay looks at Red Hat's JBoss business. "Red Hat has announced its 2009 Innovation Awards, with some impressive finalists making the list. From Whole Foods to Harvard Business School Publishing, major organizations are doing impressive things with Red Hat technology. Interestingly, however, the real "innovation" revealed by these awards is just how much more money Red Hat makes in its JBoss deals than in its Red Hat Enterprise Linux (RHEL) deals."
National Vision chooses SUSE Linux (Reuters)
Reuters reports that National Vision will be deploying POS terminals loaded with SUSE Linux. "Novell today announced that National Vision Inc., one of the largest optical retailers in the United States, plans to use SUSE(R) Linux Enterprise Point of Service to improve the performance, stability and uptime of the network of 5,000 point-of-sale devices within its stores. With more than 500 retail locations in 44 states, including America's Best Contacts & Eyeglasses and Vision Centers at select Wal-Mart stores, SUSE Linux Enterprise Point of Service will provide National Vision with an agile, reliable and cost-effective operating system."
Resources
Is Linux Irrelevant? (InformationWeek)
InformationWeek looks at the relevance of Linux as a "brand". The article is annoyingly broken into five pages, but offers a perspective that is somewhat different than what we normally see. "The end result is that Linux has become less of a developer or even programmer environment than a compilation environment -- a place for source code to be deployed rather than compiled code. A starting point, again, rather than an endpoint. The endpoint, as seen in products like Android, often has little to do with Linux as a distribution: It's a product unto itself with little connection to other things made from Linux, except in the sense you can probably compile the same software there."
Reviews
Python Python Python (aka Python 3) (Linux Journal)
Linux Journal reviews the current state of Python 3. The article looks at the language a bit, but also at the state of support for Python 3 in several Python projects (Django, Twisted, and SciPy/NumPy). How and when to transition large Python projects to Python 3 is a problem that these projects (and lots of others) face. "Such an effort (maintaining a single code-base that supports Python 3 and previous versions) is something that the Python 3 developers disapprove of, but that hasn't stopped the Django developers having a go. It's still in the early days and progress has been slow, but it does bode well for the future. If efforts such as this are given more support within the Django world, perhaps a release 3 compatible version may arrive sooner than we think."
Miscellaneous
Why can't local government and open source be friends? (guardian.co.uk)
Over at guardian.co.uk, there is a report on the failure of the Birmingham, UK government to even consider open source solutions for their new web site. Said web site is now late and 5x over budget. "The trouble is that the website never stood a chance. Nobody seems to have stood up in a meeting and said: 'You know, there's lots of very good open source content management systems (CMS) out there - there's one called Wordpress which is free and eminently customisable.' This is peculiar, as Wordpress was available (and as solid as any CMS) in 2005, runs on MySQL and PHP (which are both free products used by some of the largest companies in the world, such as airlines and Yahoo). And there are pots of programmers around with MySQL and PHP skills." Thanks to Eugene Markow.
Page editor: Forrest Cook
Announcements
Non-Commercial announcements
EFF on the RealDVD decision
The Electronic Frontier Foundation has sent out a release on the RealDVD decision. "The heart of Judge Patel's ruling is her interpretation of the DVD-CCA license agreement, and since large portions of those agreements remain confidential, it is difficult to evaluate the merits of her reasoning. However, she does make the troubling suggestions that fair use is never a defense when you circumvent an "access control" like encryption on DVDs. She also suggests that irreparable harm can be presumed whenever copyright infringement or a DMCA violation is likely..."
Legal Announcements
Patent fun: Microsoft Word sales banned in the US
Here is a press release from legal firm McKool Smith, which is quite proud at having gotten a US court to rule that Word violates patent #5,787,499. "Today's permanent injunction prohibits Microsoft from selling or importing to the United States any Microsoft Word products that have the capability of opening .XML, .DOCX or DOCM files (XML files) containing custom XML." The text of this patent is quite vague; if it stands it could almost certainly be used to make life difficult for free software as well.
New Books
O'Reilly publishes book excerpts
O'Reilly has published the following book excerpts: Shared and Platform-Specific Keywords - SQL in a Nutshell, Installing Git - Version Control with Git and Installing Apache Hadoop - Hadoop: The Definitive Guide.
Resources
AMD RS780 docs released, coreboot support coming
Documentation for the AMD RS780 is now available. "The coreboot community, which includes government organizations, corporations, research labs and individuals from around the world, is very excited to expand on our existing and decade-long collaboration with AMD. This collaboration has, over the years, resulted in the inclusion of coreboot into everything from some of the largest AMD-based supercomputers in the world to some of the smallest embedded systems. Together with the recent SB700/SB710/SB750 documentation release, the Developer Guide release for the RS780 family of Integrated Chipset/ Graphics Processors enables the coreboot community to support any board with AMD chipsets out there, from embedded to enthusiast desktop and high-end server boards."
VistA community members propose VistA Installation Spec (LinuxMedNews)
LinuxMedNews reports on the approval of the VistA Standard Base specification, release candidate 8. "The proposed document is intended to guide installation of Veterans Affairs VistA system on Linux using the Free/Open Source GTM mumps compiler."
Calls for Presentations
CFP: FOMS 2010
A call for papers has gone out for the Foundations of Open Media Software workshop (FOMS). The event takes place on January 13-15 in Wellington, New Zealand, submissions are due by October 15. "Open media is a movement to promote free expression and innovation in online video and audio. Foundations of Open Media Software (FOMS) is a task-oriented developer meeting. At FOMS, developers of open media software gather to collaborate on code and plan future technology to enable a future of open media."
CFP: International workshop on Secure Software Engineering
A call for papers has gone out for the Fourth International Workshop on Secure Software Engineering (SecSE). The event takes place in Krakow, Poland on February 15-18, 2010, submissions are due by September 30.
Upcoming Events
2009 Gnome Boston Summit
The 2009 Gnome Boston Summit till take place on October 10-12 in Cambridge, MA. "As of right now we have funding thanks to Novell to hold one hackfest the week before the Summit. The content of that hackfest is yet to be determined. As always since hackfests are focused on getting specific teams together so that they may plan projects face to face, travel sponsorship will be done via invite and handled by the specific hackfest organizers. If you are a company or organization which wants to organize and sponsor a second or even third hackfest please get in-touch with myself (J5 on irc) or the GNOME Foundation Board."
Japan Linux Symposium program announced
The program for the first Japan Linux Symposium has been posted. "The event, a new annual technical conference designed to provide a collaboration and education space in the Pacific Rim covering all matters Linux, takes place October 21-23, 2009 at Akihabara Convention Hall, Tokyo, Japan. More than fifty technical sessions are included with speakers featuring the top technical talent from across the globe." They have succeeded in attracting an interesting selection of speakers.
LCA2010 announces successful miniconfs
The miniconfs for LCA2010 have been announced. "Here is the full list of accepted Miniconfs: - Arduino (Jonathan Oxer) - Business of Open Source (Martin Michlmayr) - Data Storage and Retrieval (Peter Lieverdink) - Distro Summit (Fabio Tranchitella) - Education (Tabitha Roder) - Free The Cloud! (Evan Prodromou) - Haecksen and Linuxchix (Joh Clarke) - Mini Libre Graphics Meeting (Jon Cruz) - Multicore and Parallel Computing (Nicolas Erdody) - Multimedia (Conrad Parker) - Open and the Public Sector (Daniel Spector) - Open Programming Languages (Christopher Neugebauer) - System Administration (Simon Lyall) - Wave Developers (Shane Stephens)".
Announcing Camp KDE 2010 (KDEDot)
KDE.News has announced Camp KDE 2010. "Camp KDE 2010 will take place at the University of California San Diego (UCSD) in La Jolla, California, USA from January 15th until January 22nd, 2010. The event is free to all participants. UCSD is both our host and a sponsor, and KDE is looking forwards to participation and attendance from the UCSD body of students and faculty. The schedule is currently slated to include presentations, BoFs, hackathons and a day trip."
Events: August 20, 2009 to October 19, 2009
The following event listing is taken from the LWN.net Calendar.
| Date(s) | Event | Location |
|---|---|---|
| August 18 August 23 |
2009 Python in Science Conference | Pasadena, CA, USA |
| August 22 August 23 |
Free and Open Source Conference (FrOSCon) | St. Augustin, Germany |
| August 22 August 23 |
OpenSQL Camp | St. Augustin, Germany |
| August 31 September 4 |
Ubuntu Developer Week | Internet, Internet |
| September 1 September 4 |
JBoss World Chicago | Chicago, IL, USA |
| September 1 September 4 |
Red Hat Summit Chicago | Chicago, IL, USA |
| September 1 September 5 |
DrupalCon | Paris, France |
| September 4 September 5 |
PyCon 2009 Argentina | Buenos Aires, Argentina |
| September 7 September 11 |
XtreemOS summer school | Oxford, UK |
| September 7 September 8 |
FRHACK.ORG IT Security Conference | Besançon, France |
| September 8 September 12 |
DjangoCon '09 | Portland, OR, USA |
| September 10 September 11 |
Fedora Developer Conference 2009 | Brno, Czech Republic |
| September 12 | Evil Robot Conference (Free Conference, Free Software) | Raleigh, NC, USA |
| September 14 September 18 |
Django Bootcamp at the Big Nerd Ranch | Atlanta, Georgia, USA |
| September 15 September 17 |
International Conference on IT Security Incident Management and IT Forensics | Stuttgart, Germany |
| September 17 September 18 |
Internet Security Operations and Intelligence 7 | San Diego, CA, USA |
| September 17 September 20 |
openSUSE Conference | Nuremberg, Germany |
| September 18 September 19 |
BruCON | Brussels, Belgium |
| September 18 September 20 |
EuroBSDCon 2009 | Cambridge, UK |
| September 19 | Atlanta Linux Fest 2009 | Atlanta, Georgia, USA |
| September 19 | Beijing Perl Workshop | Beijing, China |
| September 19 | Software Freedom Day | Worldwide |
| September 20 | SELinux Developer Summit 2009 @ LinuxCon | Portland, Oregon, USA |
| September 21 September 23 |
LinuxCon 2009 | Portland, OR, USA |
| September 21 September 25 |
Ruby on Rails Bootcamp with Charles B. Quinn | Atlanta, USA |
| September 23 September 25 |
Linux Plumbers Conference | Portland, Oregon, USA |
| September 23 September 25 |
Recent Advances in Intrusion Detection | Saint-Malo, Brittany, France |
| September 23 September 25 |
OpenSolaris Developer Conference 2009 | Hamburg, Germany |
| September 23 | Bacula Conference 2009 | Cologne, Germany |
| September 24 September 26 |
Joomla! and Virtue Mart Day Germany | Bad Nauheim, Germany |
| September 25 September 27 |
International Conference on Open Source | Taipei, Taiwan |
| September 25 September 27 |
Ohio LinuxFest | Columbus, Ohio, USA |
| September 26 September 27 |
PyCon India 2009 | Bengaluru, India |
| September 26 | Open Source Conference 2009 Okinawa | Ginowan City, Okinawa, Japan |
| September 26 September 27 |
Mini-DebConf at ICOS | Taipei, Taiwan |
| September 28 September 30 |
Real time Linux workshop | Dresden, Germany |
| September 28 September 30 |
X Developers' Conference 2009 | Portland, OR, USA |
| September 28 October 2 |
Sixteenth Annual Tcl/Tk Conference (2009) | Portland, OR 97232, USA |
| September 30 | HCC!Linux Theme Day | Houten, Netherlands |
| October 1 October 2 |
Open World Forum | Paris, France |
| October 2 October 4 |
7th International Conference on Scalable Vector Graphics | Mountain View, CA, USA |
| October 2 | LLVM Developers' Meeting | Cupertino, CA, USA |
| October 2 October 4 |
Linux Autumn (Jesien Linuksowa) 2009 | Huta Szklana, Poland |
| October 2 October 4 |
Ubuntu Global Jam | Online, Online |
| October 2 October 3 |
Open Source Developers Conference France | Paris, France |
| October 2 | Mozilla Public DevDay/Open Web Camp 2009 | Prague, Czech Republic |
| October 3 October 4 |
T-DOSE 2009 | Eindhoven, The Netherlands |
| October 3 October 4 |
EU MozCamp 2009 | Prague, Czech Republic |
| October 7 October 9 |
Jornadas Regionales de Software Libre | Santiago, Chile |
| October 8 October 10 |
Utah Open Source Conference | Salt Lake City, Utah, USA |
| October 9 October 11 |
Maemo Summit 2009 | Amsterdam, The Netherlands |
| October 10 October 12 |
Gnome Boston Summit | Cambridge, MA, USA |
| October 10 | OSDN Conference 2009 | Kiev, Ukraine |
| October 12 October 14 |
Qt Developer Days | Munich, Germany |
| October 15 October 16 |
Embedded Linux Conference Europe 2009 | Grenoble, France |
| October 16 October 17 |
Pycon Poland 2009 | Ustron, Poland |
| October 16 October 18 |
Pg Conference West 09 | Seattle, WA, USA |
| October 16 October 18 |
German Ubuntu conference | Göttingen, Germany |
| October 18 October 20 |
2009 Kernel Summit | Tokyo, Japan |
If your event does not appear here, please tell us about it.
Event Reports
O'Reilly Open Source Convention delivers benefits of open source
O'Reilly presents an event report for the recent OSCON. "Thousands of independent thinkers gathered at the 11th annual OSCON in San Jose, CA, July 20-24 to hear about the latest solutions and savings that open source technology can deliver. For five full days and nights, the open source convention featured hundreds of inspiring sessions and practical tutorials on a full range of languages and platforms. Faced with a daunting economic climate, OSCON attendees found the tools to give their businesses a competitive edge."
Page editor: Forrest Cook
