Papers from the Real Time Linux Workshop

There are far too many interesting Linux and free software conferences these days, so it would be difficult—really, impossible—to attend them all. Slides and videos of the talks can help fill in the gaps, but, for conferences with a more academic bent, the papers that are the basis of the presentations can give an even more detailed look. The papers from the recently concluded Real Time Linux Workshop are a good example; this article will briefly look at a few of them.

Myths and Realities of Real-Time Linux Software Systems

This paper [PDF] can serve as an introduction to realtime for those who are not familiar with what that means. Author Kushal Koolwal starts with the basics: defining realtime, describing various kinds of latencies, and looking at hard vs. soft realtime, before moving into a few myths. Koolwal then looks at realtime in Linux, focusing on the PREEMPT_RT patchset. In a few short pages, this paper will give the reader a good foundation in realtime and the trade-offs necessary to support it.

Finding origins of latencies using Ftrace

Ftrace developer Steven Rostedt describes how to use ftrace to find unexpected and/or unacceptable latencies, which may be a barrier to realtime processing, in his paper [PDF]. Ftrace is a relatively new tool in the kernel that provides various kinds of tracing information and has some facilities that can be used specifically for tracking down latency issues. Tracers like irqsoff, preemptoff, and wakeup (along with some variants) capture information while the kernel is running in specific modes (i.e. with interrupts disabled, preemption turned off, etc.).

Rostedt's paper gives a fairly detailed look at the tracers, how to enable them, what they do, and the output they produce. While these latency tracers are active, they capture things like kernel functions called or trace event points encountered while looking for the maximum time spent in the latency-causing modes. By looking at what the kernel is doing when the latency has exceeded expectations, it can lead a developer to the specific cause—which may lead to a way to reduce the latency. Rostedt mentions the JACK "audio connection kit" developers as an early adopter of latency tracing, noting that they found both kernel and JACK bugs that were causing excess latency.

Towards Linux as a Real-Time Hypervisor

Jan Kiszka reported [PDF] on experiments using Linux as a hypervisor for realtime processing. Using KVM and QEMU, he measured the latency in both the host and guest operating systems under a number of different scenarios. One of the more obvious means to increase the responsiveness of the guest is to raise the priority of the QEMU threads and to put them into a realtime scheduling class. But that can lead to starving host OS processes that the guest is waiting on, which could lead to deadlock or other undesirable behavior.

The paper reports on the measurements of average and maximum latency, as part of a latency histogram, under different conditions: a baseline test in the host as well as in the guest, applying the priority and scheduling class changes to the guest, lowering the priority on the asynchronous I/O (AIO) QEMU threads, and using PREEMPT_RT kernel on the host. In addition, Kiszka describes a "paravirtualized scheduling" approach that allows the guest to send the host information on spinlock usage that will allow the host scheduler to adjust priorities of the guest processes for more efficient use of the CPUs, while avoiding priority inversions

ARM Fast Context Switch Extension for Linux

The organization of the ARMv5 cache can cause performance problems that may preclude its use for realtime tasks. The cache is based on virtual memory addresses and, since Linux processes share the same range of virtual addresses, each context switch requires invalidating the cache. Depending on the CPU type, memory speed, and the program's data access pattern, the cost of reloading a process's data from main memory can be on the order of 200 microseconds—too much for many time-critical applications.

One alternative is to share a flat address space between all of the processes, but then the memory protection provided by separate address spaces is lost. Gilles Chanteperdrix and Richard Cochran describe [PDF] another approach for doing context switches that preserves the memory protections without sacrificing the cache at every context switch. They use the ARM Fast Context Switch Extension (FCSE) and partition the virtual address space into separate 32MB chunks so that processes do not have overlapping address ranges. This allows for up to 128 processes running in the 3GB available for non-kernel addresses. The translation lookaside buffer (TLB) must still be flushed on context switches to enforce memory protection, but the data and instruction caches are preserved.

The actual implementation required reducing the number of available processes to 95. Either 95 or 128 processes, along with the 32MB address space restriction, were unacceptable for many embedded applications, so the authors added a "best effort mode" that eliminates those restrictions, but cannot guarantee that it won't have to do cache flushes on some context switches. They reported that average latencies for their test cases reduced by roughly half when the "guaranteed" mode was used, and by roughly one-quarter with "best effort" mode, when compared to the standard Linux kernel.

Design and Implementation of Node Order Protocol

Distributed systems often use "time division multiple access" (TDMA) as a means to coordinate access to a shared communications medium (e.g. shared bus or wireless frequencies). But, TDMA requires a reliable means to synchronize the clocks on the various systems and that synchronization uses some of the shared bandwidth simply for timekeeping. The authors, Li Chanjuan, Nicholas McGuire, and Zhou Qingguo, propose [PDF] a different protocol, Node Ordering Protocol (NOP), that avoids much of the complexity and bandwidth waste that occur with TDMA.

As its name implies, NOP relies on a consistent ordering of the nodes in the network. It also requires that nodes monitor the other nodes to determine if a faulty node is not correctly following the ordering scheme. The advantages, according to the authors, are that NOP is much easier to implement and validate than other protocols with complex synchronization requirements, loss of bandwidth due to temporal padding is not required, and that error detection is much simpler and bounded in time.

Use of cookies in real-time system development

One last paper to mention is the scholarly-sounding, if tongue-in-cheek, look at cookie consumption and "the positive impact on the real-time Linux community we were able to observe". The authors, M. Gleixner and M. McGuire, look at various cookie protocols—with code—and conclude that uni-directional protocols are best for real-time Linux development: "Though greedy protocols have been discussed in the past, we found that considering these has negative impacts on developers long term and thus are deprecated."

The slides for some of the presentations are available on the Open Source Automation Development Lab (OSADL) web site. There are quite a few more papers than we looked at here available as well. While the papers can't really replace the experience of attending, there is much of interest for those that are looking for more information on realtime in Linux.

Comments (3 posted)

Maemo Summit 2009: Fremantle, Harmattan, and N900

Nokia's second annual Maemo Summit brought together 400 developers and power users of the Linux-based tablet community in Amsterdam over the October 9-11 weekend. Maemo Community Manager Quim Gil said the primary goal was to bring the community together for the social and interactive benefits, but the three-day program also provided a solid introduction to the new Maemo 5 release, a preview of Maemo 6, and a chance to work with the Nokia N900 — which ships with as close to a standard Linux distribution as the marketplace has seen delivered on a usable mobile phone.

Friday's program was planned by Nokia, and featured several plenary talks delivered by Nokia staff, including the company's vice president of Maemo devices Ari Jaaksi, vice president of marketing Jose-Luis Martinez Merino, and director of Maemo product planning Janne Heikkinen. Session talks covered user interface and user experience design for the new release, application and widget development, and introductions to the changes slated for the next generation of Maemo, Maemo 6 "Harmattan." Saturday and Sunday's program was chosen by the Maemo.org community, and covered development in depth, as well as more user-centric subjects like introductions to the key applications and device customization.

Meet 5, meet 6

The biggest news in the Maemo community is the impending release of the Nokia N900, successor to the 770, N800, and N810 tablets — and the first to include cellular phone support. The N900 uses Maemo 5 "Fremantle," for which the official software development kit (SDK) has been available since December of 2008. Like previous releases, Maemo 5 uses libraries and toolkits common to desktop Linux distributions: Glib, GTK+, Pango, Cairo, GStreamer, BlueZ, D-Bus, Telepathy, etc. The new version adds Clutter, PulseAudio, GUPnP, Tracker, and upstart to that list, among other changes.

Nokia made a surprise announcement on the first day, providing pre-production N900 devices to 300 summit attendees for a six-month loan period (attendance was initially capped at 300; when registration hit the cap, it was expanded to 400 — Nokia assured the attendees that everyone who was not a Nokia employee or Nokia subcontractor would receive a device).

The core N900 applications — media player, calendar, communications tools, and web browser — were built by Nokia, but a considerable number of community-written and third-party applications are already built and available for the platform through the Maemo.org repositories. The Summit program recognized many of these applications with lightning talks or full sessions, and showcased five of them in the Sunday morning keynote slot for "Fremantle Stars." The stars included the microblogging tool Mauku, weather report utility OMWeather, exercise trainer eCoach, OpenStreetMap mapping tool OSM2Go, and liqbase, a suite of widgets and small applications designed for handwriting use, sketching, and other non-keyboard interaction.

The early focus on providing Maemo 5 development tools and support should make the N900 application market full when the phone finally goes on sale. Similarly, Nokia has already started preparing developers for the changes and additions slated for Maemo 6, which is expected to be released sometime in late 2010.

The most talked-about change in Maemo 6 is the adoption of the Qt framework for the default applications. Nokia states several technical reasons for the change, including the desire to use OpenGL ES hardware acceleration for the entire interface, multi-touch and gesture input, and an application state machine API. It is no secret, though, that Nokia has business reasons for the change as well; as the owner of Qt creator Trolltech, the company is pushing the framework on all platforms: Maemo, Symbian, desktop Linux, Windows, and embedded devices. On Friday morning, Nokia announced the immediate availability of an official Qt stack for Maemo 5 — specifically, a "technology preview" release targeting application developers.

When the switch to Qt was initially announced, some questioned whether it was too soon, and would short-circuit developers' desire to build their applications for Maemo 5 with the current framework. Judging by the number and variety of Maemo 5 applications currently available, though, few, if any, developers appear to have chosen to skip Fremantle.

Oh, and did we mention it's a phone?

The majority of the excitement and discussion surrounding Maemo 5 and the N900 focuses on its capabilities as a portable Internet tablet like its predecessors; one could almost be excused for forgetting from time to time that the N900 is "also" a fully-capable cell phone. Perhaps that emphasis is appropriate; Blackberry, Palm, and Apple have placed the bulk of their efforts into highlighting the Internet features of their high-end mobile phone platforms.

Maemo 5 is still a big step forward for Linux on mobile devices, however, precisely because cell phone functionality on the N900 makes it a choice for the average smartphone consumer. More importantly, Maemo 5 is, for all intents and purposes, a standard Linux distribution under the hood — it is not a locked-down, code-signed environment in which all user applications run in an isolated virtual machine environment. An xterm is one of the default N900 applications; any user that wishes to alter system components right down to the kernel can gain root access without tiptoeing through an arcane jailbreaking process.

Consequently, one of the most intriguing talks at Maemo Summit was Saturday's "Maemo and oFono," which detailed Nokia's work on an open source cell phone communication stack. The cellular stack handles a range of functions, from communicating with the cell modem itself, to monitoring and caching network status and selection information, to processing and formatting SMS and MMS messages, to managing SIM card security. Modem management itself is a tricky field, as many cell modems still use the decades-old Hayes (i.e. "AT") command set, while others (including Nokia's) use vendor specific replacements with more features.

oFono uses D-Bus, and has a small core with a plugin system that handles functions like network registration, SMS formatting, and call handling. Intel is also heavily involved in the project, and according to speaker Rémi Denis-Courmont, has made most of the commits, particularly with the hardware drivers. oFono is still in pre-release; Maemo 5 uses a closed-source stack called the cellular service daemon (CSD), and Denis-Courmont said that oFono is unlikely to make it to Maemo 6 by launch time, either.

Free software purists may decry the N900 for its inclusion of proprietary code, including the CSD phone stack. Unlike other closed applications on Maemo devices, however, there has never been a free alternative, so oFono represents a big piece of the "purity" puzzle — even if it is several years from appearing in consumer devices. When asked by the audience why Nokia did not simply open up CSD, Denis-Courmont responded that the company was not satisfied with the code and preferred to write a proper open source solution from scratch, even if that takes time.

In a sense, that is the same story as Maemo itself. Nokia invested several years and several generations of product design building the Maemo Linux platform, and at the same time invested heavily in building an enthusiastic and deep community of Maemo application developers. It even encourages development of competing applications and interfaces, such as the Canola media player, the Mer distribution, and liqbase.

As a result, Nokia now has a solid open source phone platform to show for it, one that (according to Jaaksi) shares approximately 80 percent of its code with the standard desktop and enterprise Linux distributions. More importantly, neither Android nor the LiMo Foundation — which have not made the same investment in community building — has anywhere close to the active community writing open source applications and contributing upstream to the project. Google, for example, places its emphasis on commercial applications, and does not even track open source applications for the platform. On the other end of the spectrum, the OpenMoko project—lofty as its goals are—has not developed into a commercially viable product and gets poor reviews on user experience. The size of the Maemo community does not guarantee success for the platform, but as open source advocates know, it does provide a large advantage.

[Note: Nokia underwrote travel for the author, as well as for several other journalists and around one quarter of the community attendees. The author would like to thank Nokia, Quim Gil, and the Maemo Community Council for the opportunity, for their help, and for being accessible during a very busy event.]

Comments (17 posted)

The Open Web: KDE frees the web from the browser

In recent times, the KDE project has been at the forefront of integrating the web into the desktop. In mid-September, Sebastian Kügler announced Project Silk. Meanwhile, Frank Karlitschek has been working on the Social Desktop for some time now.

On October 29, NLUUG holds its Fall Conference with the theme "The Open Web", where both KDE developers will give a talk. Sebastian will talk about freeing the web from the browser, while Frank's talk will be about the Social Desktop integration of web communities into desktop applications. Your author asked them to give a preview of their talks in an email interview.

Free the web from the browser

Sebastian will talk about the differences between the web as we use it now and what he calls "local clients": "My plan is to show how we can make the web more accessible for a multitude of different devices, and how we can tackle this." This is all part of his Project Silk, which has, as its ultimate goal, a close integration of the content and services of the web into the KDE user experience.

To see why it makes sense to integrate web content into a desktop environment, we have to take a birds-eye view on the evolution of web browsers in the last couple of years. Then, we see that the web browser is now actually a very general platform: it evolved from a simple HTML viewer into a runtime environment for web applications that provide online services. What didn't change is one core assumption, according to Sebastian:

Indeed, if we look at our current use of the internet, it's much more varied than before: from mobile phones and internet tablets to netbooks, laptops, desktops, and media centers. Assumptions that were true when we accessed web services five years ago are simply not true anymore, Sebastian explains:

So in all those new use cases, the assumptions that websites make fall short. Reading small text is often impossible on large but low-resolution TV screens, a remote control or a touch screen behaves completely differently than a mouse or touch pad. Those capabilities of a device are best known to the client, so it's much easier to control these input mechanisms for web applications on the client. But there's more:

Of course using a native client to display online content provides a more coherent user experience when you're on the web from your desktop. Last but not least, Sebastian sees a security advantage: by separating content and client-side application logic from each other, it should be easier to build safer web applications.

Many things in the pipeline

Project Silk doesn't come out of the blue: KDE has already integrated many online sources into applications. For example, Sebastian has worked with Frank and others on components that integrate social networking into KDE's Plasma desktop shell. So we have already a lot of online content, such as email, contacts and social networks, integrated into desktop applications. But Sebastian admits that most of these efforts are rather cluttered and do little in the sense of sharing code, ideas, or infrastructure.

Most KDE applications are already fully network-transparent: they use the KIO API that allows a user to access remote files using the same methods as accessing local files. Moreover, in recent years some interesting services have been built: with Akonadi KDE has central cached storage for PIM data such as emails, contacts and calendaring. Nepomuk provides the basis to handle all kinds of metadata on the KDE desktop. According to Sebastian, these services are a powerful base to build applications that are "vastly superior to JavaScript-driven browser applications."

One of the first components in project Silk that KDE developers have been working on is Selkie, which turns a web application into a first class citizen of the KDE desktop by providing better integration with the window manager, integration with desktop services such as the notification system. and custom, per-web-application actions:

More things in the pipeline include components to integrate online multimedia content into the KDE Media Center components. Sebastian also lists some lower-level activity going on, in the form of QJSon, a small library for the JSon protocol, classes to search and access content from MediaWikis such as Wikipedia, a thumbnailer for web pages, and so on. Because many applications that make KDE "silk" are integrated into the applications or Plasma, there is no central point with Project Silk code, but there is a git repository that collects the bits that aren't integrated yet.

The Social Desktop

Frank's talk at the NLUUG Fall Conference will be about the idea and the status of the Social Desktop in KDE, which he first proposed in his keynote speech at the 2008 Akademy in Belgium. The core idea of this project is to bring communication and collaboration in online communities to desktop applications, taking away the need to surf to forums and social networks in a browser. During his talk, Frank will give an overview of all the new features and possibilities from a developer and a user point of view. There is already work in-progress:

So the point of the Social Desktop is to connect people to each other and bring them closer to the KDE desktop project and related topics. Frank stresses that this is not only useful for developers, but also for regular users: "For example, you can get in contact with other KDE users and can see what your friends are doing. Or you can get help via the Knowledge Base Integration without the need to open up a browser."

To be able to use social networks on the KDE desktop without tying the Social Desktop to one web platform like Facebook or Twitter, the developers worked on support for the open collaboration services (OCS) standard, which is an open standard published on freedesktop.org. According to Frank, several websites have already committed to supporting this specification so that users can choose from different data sources:

Just like Sebastian said, Frank explains that there's nothing wrong with the browser per se, but if you integrate web services directly in desktop applications you can get a richer experience:

Social Desktop Contest

This brings us to the winners of the Social Desktop Contest, which was started on 17 June 2009. The goal of the contest was to foster community development and innovations around the OCS API. The winners of the contest were recently announced. In the announcement, Frank wrote:

The winning submission of the contest is an ExtendedAboutDialog for KDE apps by Téo Mrnjavac. It gets information about the developers of a program — for example contact details, web addresses, and nationality — from opendesktop.org and shows it in the "About" window. This gives the application a human face and enables direct interaction with the development team. The extended about dialog can be used in any KDE application and already ships with Amarok 2.2.

The second place winner is the Knowledge base widget by Marco Martin. This is a plasmoid that lets users query the online opendesktop.org knowledge base without the need to visit a forum or subscribe to a mailing list. The widget will be part of KDE 4.4. The third place is for libopengdesktop by Guido Roberto. This is a simple Glib-based library that gives easy access to Open Collaboration Services providers. It will be useful to bring the Social Desktop to Gnome and XFCE platforms. The fourth place is for PyContent, a plasma widget written in Python to show the newest contents in specific categories from a content provider.

Open projects for an open web

Both projects, the Social Desktop and Project Silk, have a very open attitude. The idea, the server side and the API specifications of the Social Desktop are completely platform independent, Frank stresses:

At this moment, project Silk is still KDE-specific, but this has its reasons, Sebastian explains:

Contribute to an open web

Both projects are also open for contributions. KDE developers and users can get involved easily in the social desktop project. There is a mailing list, a wiki, the specification is available on freedesktop.org and of course all the KDE code is available in the KDE svn. Frank adds that there will be a developer meeting later this year specially for the Social Desktop: "Everybody is welcome. So please send me an email if you want to participate. It isn't important which desktop or technology you use. The Social Desktop is a completely open project."

Project Silk is very much an umbrella for people that work on web integration components, so it's not a closed group of people but, instead, consists of a number of people from all over KDE that have already been working on web integration aspects. That's why it's easy to contribute, as Sebastian notes: "Developers that find these ideas interesting should get in touch with us on the #kde-silk IRC channel on the Freenode.net network, or via the kde-silk@kde.org mailing list. We've spent some time documenting ideas about Project Silk on Techbase, KDE's knowledge base wiki."

All in all, both projects are still in their infancy, which is an interesting opportunity for open source developers who want to do some innovative work. The submissions to the Social Desktop Contest and the Project Silk ideas on Techbase are good sources for inspiration.

Comments (12 posted)

TorProxy and Shadow

Users give up a certain level of anonymity when they browse the web. Not only do things like cookies make them less anonymous, server logs also keep a record of which IP addresses connected to them, and ISPs, companies, and others may record the destination of outbound traffic. Unlike cookies, though, there is nothing a user can do to prevent their address from being captured by endpoints or intervening routers—except by using some kind of proxy. Using Tor, for example, allows users to proxy their request through an anonymizing network so that there is no direct connection between their address and the server they are contacting. Now, through the work of Connell Gauld, Android users can also browse through Tor using TorProxy and Shadow.

There are any number of reasons that someone might want to disguise their web requests: repressive governments, potential embarrassment, hiding illegal activities, and so forth. Tor routes each request that it gets through several, randomly-chosen nodes within its network. The request eventually emerges at an exit node—which, importantly, sees the traffic in the clear—where it is handed off to the destination server.

Essentially, the only information available is that the source node connected to a Tor node, and some time later a different Tor node connected to the destination. With enough monitoring, traffic analysis might be used to determine the correspondence between those two things, but it raises the bar by quite a bit. Cookies and user logins on destination sites can also potentially pierce a user's anonymity, but those are able to be controlled by users.

TorProxy and Shadow are two free software programs for Android mobile phones that give users access to the Tor network. Both can be installed from the Android Market application. As the name implies, TorProxy is the proxy agent that sits between applications that want to anonymously use the network and the network itself, routing the traffic through Tor. Shadow uses the Android browser classes to implement a browser, but routes its requests through TorProxy.

There are some questions (see the update) about the code that underlies TorProxy, so it may not, yet, be suitable for "operational" use. But, the code is free, and there have been successful efforts to get the C version of the Tor client running on Android, so it would seem likely that a secure version of TorProxy will come along.

Once installed, TorProxy can be configured to maintain a Tor connection at all times, or only on demand from applications that specifically request it, such as Shadow. Shadow has a bit of a different look from the standard Android browser, at least on startup, but it functions more or less the same. But, much like desktop Tor usage, it suffers from fairly serious delays.

When first connecting, TorProxy takes roughly 30 seconds to initiate a connection. An onion logo—Tor is sometimes known as "The Onion Router"—with a countdown appears in the Android status bar. Once the connection is established, one can then surf the web. It is something of a nostalgic experience, reminding one of those halcyon days of accessing the net via 9600bps (or worse) modems.

Unfortunately, any serious attempt to anonymize traffic is going to be somewhat slow. Each hop along the way is going to add some time to the process, but each will add a bit more unpredictability as well. For those that need the anonymity that Tor can provide, however, the wait is likely worth it—the wait in a gulag or prison will likely be much longer.

Comments (4 posted)

Walsh: Google Chrome Policy

SELinux hacker Dan Walsh looks at creating policies for the Google Chrome browser on his weblog. His posting is a detailed look at creating SELinux policy for Chrome/Chromium, and, in particular, the Chromium sandbox. "When I write new policy now, I default to permissive domains to make sure I don't blow up the user environment. I usually wait for the next version of the OS to turn permissive domains to enforcing domains. This means I will probably leave chrome_sandbox_t as a permissive domain for all of F12 and turn it enforcing in F13. This allows me to gather lots of AVC's and not force the user to disable SELinux [or] not use chrome. And hopefully allows me to write better policy. You can use the seinfo --permissive command to list all the permissive domains on your system."

Comments (35 posted)

Urgent Django security updates released

The Django project has announced the release of a set of urgent security updates. "This issue was disclosed publicly by a third party on a high-traffic mailing list, and attempts have been made to exploit it against live Django installations; as such, we are bypassing our normal policy for security disclosure and immediately issuing patches and updated releases." The vulnerability (a denial of service problem) affects any Django application running 1.0 or later and using the EmailField or URLField features.

Comments (1 posted)

aria: buffer overflow

Package(s):	aria2	CVE #(s):	CVE-2009-3575
Created:	October 9, 2009	Updated:	January 14, 2010
Description:	From the Red Hat bugzilla: Buffer overflow in DHTRoutingTableDeserializer.cc in aria2 0.15.3, 1.2.0, and other versions allows remote attackers to cause a denial of service (crash) and possibly execute arbitrary code via unknown vectors.
Alerts:	Gentoo 201001-06 aria2 2010-01-13 Debian DSA-1957-1 aria2 2009-12-28 Fedora FEDORA-2009-10344 aria2 2009-10-09

Comments (none posted)

deltarpm: old zlib vulnerability

Package(s):	deltarpm	CVE #(s):
Created:	October 9, 2009	Updated:	October 14, 2009
Description:	deltarpm prior to the current build ships with a bundled copy of zlib. This version of zlib has a known vulnerability with CVE identifier: CAN-2005-1849 This build of deltarpm patches the program to use the system zlib (which was fixed when the vulnerability was first discovered) instead of the bundled copy.
Alerts:	Fedora FEDORA-2009-10262 deltarpm 2009-10-06 Fedora FEDORA-2009-10233 deltarpm 2009-10-03 Fedora FEDORA-2009-10237 deltarpm 2009-10-03

Comments (none posted)

dopewars: denial of service

Package(s):	dopewars	CVE #(s):	CVE-2009-3591
Created:	October 14, 2009	Updated:	October 14, 2009
Description:	Dopewars 1.5.12 has a denial of service vulnerability in the face of a "REQUESTJET" message with an invalid location.
Alerts:	Fedora FEDORA-2009-10385 dopewars 2009-10-14

Comments (none posted)

drupal-service_links

Package(s):	drupal-service_links	CVE #(s):	CVE-2009-3648
Created:	October 14, 2009	Updated:	October 14, 2009
Description:	Drupal's "service links" module does not properly validate user-supplied input, leading to a cross-site scripting vulnerability; see this advisory for more information.
Alerts:	Fedora FEDORA-2009-10466 drupal-service_links 2009-10-14 Fedora FEDORA-2009-10445 drupal-service_links 2009-10-14

Comments (none posted)

graphicsmagick: multiple vulnerabilities

Package(s):	graphicsmagick	CVE #(s):	CVE-2007-1667 CVE-2007-1797 CVE-2007-4985 CVE-2007-4986 CVE-2007-4988 CVE-2008-1096 CVE-2008-3134 CVE-2008-6070 CVE-2008-6071 CVE-2008-6072 CVE-2008-6621 CVE-2009-1882
Created:	October 8, 2009	Updated:	June 1, 2010
Description:	graphicsmagick has a long list of vulnerabilities. From the Debian alert: Several vulnerabilities have been discovered in graphicsmagick, a collection of image processing tool, which can lead to the execution of arbitrary code, exposure of sensitive information or cause DoS. The Common Vulnerabilities and Exposures project identifies the following problems: CVE-2007-1667: Multiple integer overflows in XInitImage function in xwd.c for GraphicsMagick, allow user-assisted remote attackers to cause a denial of service (crash) or obtain sensitive information via crafted images with large or negative values that trigger a buffer overflow. It only affects the oldstable distribution (etch). CVE-2007-1797: Multiple integer overflows allow remote attackers to execute arbitrary code via a crafted DCM image, or the colors or comments field in a crafted XWD image. It only affects the oldstable distribution (etch). CVE-2007-4985: A crafted image file can trigger an infinite loop in the ReadDCMImage function or in the ReadXCFImage function. It only affects the oldstable distribution (etch). CVE-2007-4986: Multiple integer overflows allow context-dependent attackers to execute arbitrary code via a crafted .dcm, .dib, .xbm, .xcf, or .xwd image file, which triggers a heap-based buffer overflow. It only affects the oldstable distribution (etch). CVE-2007-4988: A sign extension error allows context-dependent attackers to execute arbitrary code via a crafted width value in an image file, which triggers an integer overflow and a heap-based buffer overflow. It affects only the oldstable distribution (etch). CVE-2008-1096: The load_tile function in the XCF coder allows user-assisted remote attackers to cause a denial of service or possibly execute arbitrary code via a crafted .xcf file that triggers an out-of-bounds heap write. It affects only oldstable (etch). CVE-2008-3134: Multiple vulnerabilities in GraphicsMagick before 1.2.4 allow remote attackers to cause a denial of service (crash, infinite loop, or memory consumption) via vectors in the AVI, AVS, DCM, EPT, FITS, MTV, PALM, RLA, and TGA decoder readers; and the GetImageCharacteristics function in magick/image.c, as reachable from a crafted PNG, JPEG, BMP, or TIFF file. CVE-2008-6070: Multiple heap-based buffer underflows in the ReadPALMImage function in coders/palm.c in GraphicsMagick before 1.2.3 allow remote attackers to ca use a denial of service (crash) or possibly execute arbitrary code via a crafted PALM image. CVE-2008-6071: Heap-based buffer overflow in the DecodeImage function in coders/pict.c in GraphicsMagick before 1.1.14, and 1.2.x before 1.2.3, allows remote attackers to cause a denial of service (crash) or possibly execute arbitrary code via a crafted PICT image. CVE-2008-6072: Multiple vulnerabilities in GraphicsMagick allow remote attackers to cause a denial of service (crash) via vectors in XCF and CINEON images. CVE-2008-6621: Vulnerability in GraphicsMagick allows remote attackers to cause a denial of service (crash) via vectors in DPX images. CVE-2009-1882: Integer overflow allows remote attackers to cause a denial of service (crash) and possibly execute arbitrary code via a crafted TIFF file, which triggers a buffer overflow.
Alerts:	Oracle ELSA-2012-0301 imagemagick 2012-03-07 Fedora FEDORA-2010-0001 GraphicsMagick 2010-01-02 Fedora FEDORA-2010-0036 GraphicsMagick 2010-01-02 Debian DSA-1903 graphicsmagick 2009-10-07 Mandriva MDVSA-2009:261 graphicsmagick 2009-08-08 Mandriva MDVSA-2009:260 imagemagick 2009-08-08

Comments (none posted)

mimetex: multiple vulnerabilities

Package(s):	mimetex	CVE #(s):	CVE-2009-1382 CVE-2009-2459
Created:	October 8, 2009	Updated:	March 25, 2013
Description:	From the Ubuntu alert: Chris Evans discovered that mimeTeX incorrectly handled certain long tags. An attacker could exploit this with a crafted mimeTeX expression and cause a denial of service or possibly execute arbitrary code. (CVE-2009-1382) Chris Evans discovered that mimeTeX contained certain directives that may be unsuitable for handling untrusted user input. This update fixed the issue by disabling the \input and \counter tags. (CVE-2009-2459)
Alerts:	Fedora FEDORA-2013-3902 mimetex 2013-03-24 Fedora FEDORA-2013-3910 mimetex 2013-03-23 Fedora FEDORA-2010-6546 mimetex 2010-04-14 Fedora FEDORA-2009-10170 mimetex 2009-10-03 Fedora FEDORA-2009-10225 mimetex 2009-10-03 Debian DSA-1917-1 mimetex 2009-10-24 Ubuntu USN-844-1 mimetex 2009-10-08

Comments (none posted)

netpbm: denial of service

Package(s):	netpbm	CVE #(s):	CVE-2008-4799
Created:	October 9, 2009	Updated:	December 7, 2009
Description:	From the Mandriva advisory: pamperspective in Netpbm before 10.35.48 does not properly calculate a window height, which allows context-dependent attackers to cause a denial of service (crash) via a crafted image file that triggers an out-of-bounds read.
Alerts:	Mandriva MDVSA-2009:317 netpbm 2009-12-05 Mandriva MDVSA-2009:262 netpbm 2009-08-09

Comments (none posted)

opensaml2: interpretation conflict

Package(s):	opensaml2 shibboleth-sp2	CVE #(s):
Created:	October 13, 2009	Updated:	October 14, 2009
Description:	From the Debian advisory: In DSA-1895-1, the xmltooling package was updated to address several security issues. It turns out that the change related to SAML metadata processing for key constraints caused problems when applied without the matching changes in the opensaml2 and shibboleth-sp2 packages.
Alerts:	Debian DSA-1895-2 opensaml2 2009-10-09

Comments (none posted)

phpmyadmin: cross-site scripting, SQL injection

Package(s):	phpmyadmin	CVE #(s):
Created:	October 13, 2009	Updated:	October 16, 2009
Description:	From the Mandriva advisory: This is a security release for XSS and SQL injection problems. This upgrade provides phpmyadmin 2.11.9.6 for CS4 and 3.2.2.1 for MES5 which is not vulnerable for these security issues.
Alerts:	Fedora FEDORA-2009-10510 phpMyAdmin 2009-10-15 Mandriva MDVSA-2009:274 phpmyadmin 2009-10-13 Fedora FEDORA-2009-10530 phpMyAdmin 2009-10-15

Comments (none posted)

python-django: directory traversal

Package(s):	python-django	CVE #(s):	CVE-2009-2659
Created:	October 13, 2009	Updated:	December 9, 2009
Description:	From the Mandriva update: The Admin media handler in core/servers/basehttp.py in Django 1.0 and 0.96 does not properly map URL requests to expected static media files, which allows remote attackers to conduct directory traversal attacks and read arbitrary files via a crafted URL.
Alerts:	Mandriva MDVSA-2009:276-1 python-django 2009-12-08 Mandriva MDVSA-2009:276 python-django 2009-10-13 Mandriva MDVSA-2009:275 python-django 2009-10-13

Comments (none posted)

python-django: denial of service

Package(s):	python-django	CVE #(s):	CVE-2009-3695
Created:	October 13, 2009	Updated:	December 9, 2009
Description:	From the Mandriva advisory: Algorithmic complexity vulnerability in the forms library in Django 1.0 before 1.0.4 and 1.1 before 1.1.1 allows remote attackers to cause a denial of service (CPU consumption) via a crafted (1) EmailField (email address) or (2) URLField (URL) that triggers a large amount of backtracking in a regular expression.
Alerts:	Mandriva MDVSA-2009:276-1 python-django 2009-12-08 Debian DSA-1905-1 python-django 2009-10-10 Mandriva MDVSA-2009:276 python-django 2009-10-13

Comments (none posted)

sympa: symlink attack

Package(s):	sympa	CVE #(s):	CVE-2008-4476
Created:	October 9, 2009	Updated:	October 14, 2009
Description:	From the Mandriva advisory: sympa.pl in sympa 5.3.4 allows local users to overwrite arbitrary files via a symlink attack on a temporary file. NOTE: wwsympa.fcgi was also reported, but the issue occurred in a dead function, so it is not a vulnerability.
Alerts:	Mandriva MDVSA-2009:263 sympa 2009-08-09

Comments (none posted)

wireshark: denial of service

Package(s):	wireshark	CVE #(s):	CVE-2009-3241
Created:	October 13, 2009	Updated:	December 1, 2009
Description:	From the Mandriva advisory: Unspecified vulnerability in the OpcUa (OPC UA) dissector in Wireshark 0.99.6 through 1.0.8 and 1.2.0 through 1.2.1 allows remote attackers to cause a denial of service (memory and CPU consumption) via malformed OPCUA Service CallRequest packets (CVE-2009-3241).
Alerts:	Mandriva MDVSA-2009:270 wireshark 2009-10-12 Fedora FEDORA-2009-9837 wireshark 2009-09-24 Debian DSA-1942-1 wireshark 2009-11-29 Gentoo 200911-05 wireshark 2009-11-25 SuSE SUSE-SR:2009:016 silc-toolkit, open-iscsi, strongswan,freeswan,openswan, mutt, openldap2, cyrus-imapd, java-1_6_0-openjdk, postgresql, IBMJava2-JRE/java-1_4_2-ibm, wireshark, freeradius, dovecot 2009-10-13

Comments (none posted)

Kernel release status

The current development kernel is 2.6.32-rc4, released on October 11. It has lots of small fixes and a pair of new SCSI drivers. The short-form changelog is in the announcement, or see the full changelog for all the details.

2.6.32-rc5 is to be expected on October 15, immediately prior to Linus's travel to Tokyo for the 2009 Kernel Summit.

The current stable kernel is 2.6.31.4, released (along with 2.6.27.37) on October 12. These updates contain another set of important fixes for these kernels; this summary by Andy Whitcroft for a bit more information on the changes in 2.6.31.4.

Comments (none posted)

Quotes of the week (Linus special)

-- Linus Torvalds

-- Linus Torvalds

-- Linus Torvalds

Comments (21 posted)

Stanse

Static analysis tools can bring great value to the development process; they often find bugs which escape review and which, potentially, can live in the code base for years. Linux has benefited from bug reports from Coverity's tools, but those tools are proprietary. Unfortunately, free static analysis tools have always lagged the proprietary alternatives.

That won't change overnight, but there is a new contender on the block in the form of Stanse; the 1.0 version was recently announced on the kernel mailing list. Specific problems that Stanse can test for include locking errors, memory leaks, failure to check for memory allocation failures, non-atomic operations in atomic context, and some reference counting errors. A list of kernel bugs found by Stanse has been posted.

Clearly, it would be nice to extend Stanse with more tests. Many kernel developers may balk at doing that, though; Stanse is a Java application, and checker rules must be written in XML. That limits rule additions to those who are both familiar with kernel code and able to work in a Java/XML environment.

Comments (9 posted)

Looking forward to the kernel summit

The 2009 Kernel Summit will be held October 19 and 20 in Tokyo, Japan, immediately prior to the Japan Linux Symposium. This will be the first time that the Summit has been held in Asia. If nothing else, the sight of that many kernel hackers running loose in Akihabara should be amusing.

The draft agenda for the event has been posted; as usual, it gives an insight into the kinds of problems which are seen to be pressing at this time. Following the tradition of the last few years, the Summit is spending a relatively small amount of time on specific technical issues; that kind of problem is usually best solved on the mailing lists and with code. What face-to-face meetings are often best for, instead, is process-oriented issues.

The agenda this time contains a panel consisting of (unnamed, thus far) end users from both the embedded and enterprise communities. Enterprise representatives have been fairly common participants at these meetings, but the presence of the embedded user community is new. With any luck, this panel will encourage the trend whereby embedded systems vendors are participating more in the development process. On the second day, instead, the Summit will hear from a user not normally associated with embedded systems: there will be a session on Google's use of Linux and problems which have been encountered.

Another process-oriented session is the perennial "regressions and kernel quality" topic. A separate session looks at performance regressions in particular; it's likely to follow up on a similar discussion held during the kernel developers' panel at LinuxCon. There's also sessions on how linux-next and the staging tree work, and an open session on improving the development process.

On the technical side, the summit begins with summary reports from a number of recently-held mini-summits. Perf events and tracing occupy a significant chunk of time; some of that will be dedicated to a demonstration of what can be done with perf, ftrace, and timechart. There will be discussions on expanding the use of the device tree abstraction to other architectures, improving generic architecture support, and the merging of the remaining realtime preemption patches. The "hacking hour," introduced last year, is back; there has been a suggestion that the topic this year could be big kernel lock elimination.

As usual, LWN editor Jonathan Corbet will be there to report on the discussion. Reports will be posted as soon as they are available; stay tuned.

Comments (1 posted)

On the driver life cycle

As a general rule, all new features are supposed to be added to the kernel during the two-week merge window. There is an exception of sorts, though, for new device drivers. A well-written driver should not be able to cause regressions anywhere else in the kernel, and there is often value in getting it to users as quickly as possible. So drivers will often make it into the mainline when other large changes are barred.

As the story of the recent SCSI fixes pull request shows, though, there are limits. This request included a pair of new drivers for high-end SCSI storage systems. Linus got grumpy for a few reasons: he would like to see subsystem maintainers try harder to get drivers in during the merge window, he thinks that the "driver exception" is mainly useful for consumer-level devices, and the driver in question here is not small bit of code - it's a 50,000 line monster. In the end, the driver was merged for 2.6.32-rc4, but Linus made it clear that he would rather see this kind of code during the merge window.

The conversation drifted into whether the driver should have gone into the staging tree instead; those who looked at it did not describe it as the best code they had seen that day. SCSI maintainer James Bottomley sees the staging tree mainly as the place where user-space ABI issues are cleaned up. Mere code quality issues, he believes, are better handled directly in the SCSI tree. Others disagree; in the end, it will come down to what specific subsystem maintainers want to do. If the maintainer takes a new driver directly into the subsystem tree, nobody else can force it into staging instead.

The discussion brought out another potential use for the staging tree - as a last resting place for old drivers on their way out of the kernel. Staging maintainer Greg Kroah-Hartman noted:

The idea remains hypothetical, though, until somebody actually uses the staging tree in this manner. It is hard to imagine a demotion to staging that would not be resisted by somebody; the first attempt to do so may well be interesting to watch.

Comments (3 posted)

Char devices for network interfaces

One of the longstanding quirks of BSD-inspired networking is that network interfaces are a strange sort of device. They live in their own namespace, do not appear in /dev, and generally fail to live up to the "everything is a file" idea that drives much of the POSIX API. That said, the Unix way of networking has functioned well for nearly 30 years. It is likely that few people were expecting a serious patch which tries to change things.

This patch from "Narendra K" at Dell does exactly that, though, and in surprising ways. With this patch in place, every network interface gets an associated char device. It is a singularly useless device: any attempt to open it just returns ENOSYS. The only real reason for this device's existence, it turns out, is to generate events for udev which, in turn, can generate alternative names for the interface.

Why this change? System vendors and administrators are getting tired of their network interfaces changing name at each boot. Non-deterministic interface naming is the result of a few factors, including weird BIOS behavior and the way current kernels enumerate devices via a parallel hot-plug approach. When interfaces change names, configuration scripts can get confused; the end result is rarely a working network. Some of this confusion can be avoided by carefully configuring interfaces based on their MAC address, but that, too, can fail in the face of "swap out the entire server" approach to fast failure recovery.

Vendors have tried to work around some of these difficulties at the hardware level. Dell machines are designed to enumerate network interfaces in the same order as often as possible. HP blade servers can configure interface MAC addresses at power-on time. But there are limits to how many hardware hacks the vendors are willing to add to deal with this problem. This message from Matt Domsch is recommended reading for anybody wanting a full understanding of the difficulties here.

Thus the patch, which allows udev to create pseudo-names for each interface based on criteria like the PCI slot number, chassis label, or anything else that seems to make sense. The patch is tied to the libnetdevname library, which maps these pseudo-names into the real interface name, which can then be used with the socket system calls.

The patch has gotten a bit of a rough reception; it looks to some like a hack for problems which can be solved in other ways. The discussion has made it clear that a real problem exists, though, so some sort of solution will likely be applied in the end. Udev seems like the place for this solution to happen - that is how naming has been handled for every other device in the system, after all. So expect something to get in eventually, though the current may evolve somewhat before it is considered ready for merging.

Comments (21 posted)

Fixing kmap_atomic()

Once upon a time, Linux was limited to less than 1GB of physical memory on 32-bit systems. This limit was imposed by two technical decisions: processes run with the same page tables in both kernel and user mode, and all physical memory had to be directly addressable by the kernel. Not changing page tables at every transition between kernel and user space is a significant performance win, but it forces the two modes to share the same 4GB address space. The directly-addressable requirement meant that total physical memory could not exceed the amount of virtual memory address space assigned to the kernel. Indeed, not even the full kernel space was available, due to the need to leave some space for I/O memory, vmalloc(), and so on. The normal split is 3GB for user space and 1GB for kernel space; that limited systems to a bit less than 1GB of physical memory.

The way this problem was fixed was to create the concept of "high memory": memory which is not directly addressable by the kernel. Most of the time, the kernel does not need to directly manipulate much of the memory on the system; almost all user-space pages, for example, are usually only accessed in user mode. But, occasionally, the kernel must be able to reach into any page in the system. Zeroing new pages is one example; reading system call arguments from a user-space page is another. Since high-memory pages cannot live permanently in the kernel's virtual address space, the kernel needs a mechanism by which it can temporarily create a kernel-space address for specific high-memory pages.

That mechanism is called kmap(); it takes a pointer to a struct page and returns a kernel-space virtual address for the page. When the kernel is done with the page, it must use kunmap() to unmap the page and make the address available for other mappings. kmap() works, but it can be slow; it requires translation lookaside buffer flushes and, potentially, cross-CPU interrupts for every mapping. Linus recently commented on the costs of high memory:

All that costly work is done to keep the kernel-space mapping consistent across all processors in the system, even though many of these mappings are used only briefly, and only on a single CPU.

To improve performance, the kernel developers introduced a special version:

    void *kmap_atomic(struct page *page, enum km_type idx);

Atomic kmap slots
KM_BOUNCE_READ
KM_SKB_SUNRPC_DATA
KM_SKB_DATA_SOFTIRQ
KM_USER0
KM_USER1
KM_BIO_SRC_IRQ
KM_BIO_DST_IRQ
KM_PTE0
KM_PTE1
KM_IRQ0
KM_IRQ1
KM_SOFTIRQ0
KM_SOFTIRQ1
KM_SYNC_ICACHE
KM_SYNC_DCACHE
KM_UML_USERCOPY
KM_IRQ_PTE
KM_NMI
KM_NMI_PTE

This function differs from kmap() in some important ways. It only creates a mapping on the current CPU, so there is no need to bother other processors with it. It also creates the mapping using one of a very small set of kernel-space addresses. The caller must specify which address to use by way of the idx argument; these addresses are specified by a set of "slot" constants. For example, KM_USER0 and KM_USER1 are set aside for code called directly from user context - system call implementations, generally. KM_PTE0 is used for page table operations, KM_SOFTIRQ0 is used in software interrupt mode, etc. There are about twenty of these slots defined in current kernels; see the list at the right for the 2.6.32 slots.

The use of fixed slots requires that the code using these mappings be atomic - hence the name kmap_atomic(). If code holding an atomic kmap could be preempted, the thread which takes its place could use the same slots, with unfortunate results. The per-CPU nature of atomic mappings means that any cross-CPU migration would be disastrous. It's worth noting that there is no other protection against multiple use of specific slots; if two functions in a given call chain disagree about the use of KM_USER0, bad things are going to happen. In practice, this problem does not seem to actually bite people, though.

This API has seen little change for years, but Peter Zijlstra has recently decided that it could use a face lift. The result is a patch series changing this fundamental interface and fixing the resulting compilation problems in over 200 files. The change is conceptually simple: the slots disappear, and the range of addresses is managed as a stack instead. After all, users of kmap_atomic() don't really care about which address they get; they just want an address that nobody else is using. The new API does force map and unmap operations to nest properly, but the atomic nature of these mappings means that usage generally fits that pattern anyway.

There seems to be little question of this change being merged; Linus welcomed it, saying "I think this is how we should have done it originally." There were some quibbles about the naming in the first version of the patch (kmap_atomic() had become kmap_atomic_push()), but that was easily fixed for the second iteration.

It is also interesting to look at how this patch series was reworked. The first version was a single patch which did all of the changes at once. In response to reviewers, Peter broke the second version down into four steps:

1. Make sure that all atomic kmaps are created and destroyed in a strictly nested manner. There were a few places in the code where that did not happen; fixing it was usually just a matter of reordering a couple of kunmap_atomic() calls.

2. Switch to the stack-based mode without changing the kmap_atomic() prototype. So, after this patch, kmap_atomic() simply ignores the idx argument.

3. The kmap_atomic() prototype loses the idx argument; this is, by far, the largest patch of the series.

4. Various final details are fixed up.

Doing things this way will make it a lot easier to debug any strange problems which result from the changes. The most significant change in terms of how the kernel works is step 2, so that's the patch which is most likely to create problems. But this organization makes that patch relatively small, so tracking down any residual bugs should be relatively easy. Instead, the really huge patch (part 3) should not really change the binary kernel at all, so the chances of it being problem-free are quite high.

All that remains is getting this change merged. It's too late for 2.6.32, but putting it into linux-next is likely to create large numbers of patch conflicts. That is a common problem with wide-ranging patches like this, though; developers have gotten better over the years at maintaining them against a rapidly-changing kernel

Comments (13 posted)

Ask a kernel developer, part 2

One in a series of columns in which questions are asked of a kernel developer and he tries to answer them. If you have unanswered questions relating to technical or procedural things around Linux kernel development, ask them in the comment section, or email them directly to the author.

How do I open an effective communication channel with a kernel developer to get my issues fixed?

Despite the size of most kernel subsystem maintainer's inbox, this is a question that comes up a lot in conversations with users, so it is good to get it out there.

The easiest way to communicate with a kernel developer about a problem is to write an email and send it to the subsystem list that handles the area in which you are having problems, and to copy the developers as well to make sure that they see the message.

Ah, but how do you figure out what subsystem or mailing list to use? Luckily the kernel contains a list of the mailing lists and the developers responsible for the different kernel subsystems. The file, MAINTAINERS in the Linux kernel source tree, lists all of the different subsystems, the name of the maintainer, the email address, and the mailing list that is the best place to bring up things on. If there is no mailing list specified, then use the default linux-kernel mailing list address.

If you narrow the problem down to a file that you are having questions about, the script scripts/get_maintainer.pl in the kernel source tree can find the proper people responsible for changing it last, as well as any maintainer and mailing lists automatically. For example, suppose you have a problem with the ftdi_sio driver, which is located in drivers/usb/serial/ftdio_sio.c. By running the get_maintainer.pl script with the -f option, you would get the following:

    $ scripts/get_maintainer.pl -f drivers/usb/serial/ftdi_sio.c
    Greg Kroah-Hartman <gregkh@suse.de>
    Alan Cox <alan@linux.intel.com>
    linux-usb@vger.kernel.org
    linux-kernel@vger.kernel.org

Make sure you always send a copy to a development mailing list, do not just email kernel developers privately, as their email load is quite high. By emailing the mailing list, you offer up the ability for anyone to help you out with your question - taking advantage of the large development community - and you avoid overloading the individual maintainers any more than they are already overloaded.

What happens if I get no response from my email?

Be persistent. If you do not hear back within a week, send a friendly "did you miss this email?" type response.

In the BSD world, there is a "security officer." Why is there no "security officer" for the Linux kernel?"

It is true there is no one person responsible for security for the Linux kernel, it is a group of developers who have taken this role on. The email address security@kernel.org goes directly to this group of developers who will quickly respond to any reported problems.

Instructions on how to contact this list, and the rules around which they operate concerning disclosure and amount of time before publicly fixing the problem, can be found in the Linux kernel file Documentation/SecurityBugs. If anyone has any questions about these rules, feel free to contact the security team for clarification.

Do you look at the code of the BSDs in order to find new ideas and concepts, or do you ignore them completely?

This is a personal decision on where to find ideas to implement in Linux. As far as I am concerned, I have not looked at the BSDs in many many years, as I have been busy with lots of Linux-only things (driver model, USB, Linux Driver Project, etc.) But other kernel developers do work with the BSD developers on coming up with solutions to different problems, or to get proper hardware support for types of devices.

Back in the early days of USB support in Linux, I did work with a number of the BSD USB kernel developers to share how specific devices operated so that drivers could be written for both operating systems, and overall, the developers are quite friendly toward each other, as we are working toward solving the same types of problems, but usually in different ways.

Comments (2 posted)

Deadline scheduling for Linux

Much of the realtime scheduling work in Linux has been based around getting the best behavior out of the POSIX realtime scheduling classes. Techniques like priority inheritance, for example, exist to ensure that the highest-priority task really can run within a bounded period of time. In much of the rest of the world, though, priorities and POSIX realtime are no longer seen as the best way to solve the problem. Instead, the realtime community likes to talk about "deadlines" and deadline-oriented scheduling. In this article, we'll look at a deadline scheduler has recently been posted for review and related discussion at the recent Real Time Linux Workshop in Dresden.

Priority-based realtime scheduling has the advantage of being fully deterministic - the highest-priority task always runs. But priority-based scheduling is subject to some unpleasant failure modes (priority inversion and starvation, for example), does not really isolate tasks running on the same system, and is often not the best way to describe the problem. Most tasks are more readily described in terms of an amount of work which must be accomplished within a specific time period; the desire to work in those terms has led to a lot of research in deadline-based scheduling in recent years.

A deadline system does away with static priorities. Instead, each running task provides a set of three scheduling parameters:

• A deadline - when the work must be completed.
• An execution period - how often the work must be performed.
• The worst-case execution time (WCET) - the maximum amount of CPU time which will be required to get the work done.

Deadline-scheduled tasks usually recur on a regular basis - thus the period parameter - but sporadic work can also be handled with this model.

There are some advantages to this model. The "bandwidth" requirement of a process - what percentage of a CPU it needs - is easily calculated, so the scheduler knows at the outset whether the system is oversubscribed or not. The scheduler can (and should) refuse to accept tasks which would require more bandwidth than the system has available. By refusing excess work, the scheduler will always be able to provide the requisite CPU time to every process within the specified deadline. That kind of promise makes realtime developers happy.

Linux currently has no deadline scheduler. There is, however, an implementation posted for review by Dario Faggioli and others; Dario also presented this scheduler in Dresden. This implementation uses the "earliest deadline first" (EDF) algorithm, which is based on a simple concept: the process with the earliest deadline will be the first to run. Essentially, EDF attempts to ensure that every process begins executing by its deadline, not that it actually gets all of its work done by then. Since EDF runs work as early as possible, most tasks should complete well ahead of their declared deadlines, though.

This scheduler is implemented with the creation of a new scheduling class called SCHED_EDF. It does away with the distinction between the "deadline" and "period" parameters, using a single time period for both. The patch places this class between the existing realtime classes (SCHED_FIFO and SCHED_RR) and the normal interactive scheduling class (SCHED_FAIR). The idea behind this placement was to avoid breaking the "highest priority always runs" promise provided by the POSIX realtime classes. Peter Zijlstra, though, thinks that deadline scheduling should run at the highest priority; otherwise it cannot ensure that the deadlines will be met. That placement could be seen as violating POSIX requirements; to that, Peter responds, "In short, sod POSIX."

Peter would also like to name the scheduler SCHED_DEADLINE, for the simple reason that EDF is not the only deadline algorithm out there. In the future, it may be desirable to switch to a different algorithm without forcing applications to change which scheduling class they request. At the moment, the other contender would appear to be "least laxity first" scheduling, which picks the task with the smallest amount of "cushion" time between its remaining compute time and its deadline. Least laxity first tries to ensure that each process can complete its computing by the deadline. It tends to suffer from much higher context-switching rates than EDF, though, and nobody is pushing such a scheduler for Linux at the moment.

One nice feature of deadline schedulers is that no process should be able to prevent another from completing its work before its deadline passes. The real world is messier than that, as we will see below, but, even in the absence of deeper problems, the scheduler can only make that guarantee if every process actually stops running within its declared WCET. The EDF scheduler solves that problem in an unsubtle way: when a process exceeds its bandwidth, it is simply pushed out of the CPU until its next deadline period begins. This approach is simple to implement and ensures that deadlines will be met, but it can be hard on a process which must do a bit of extra computing on occasion.

In the SCHED_EDF patch, processes indicate the end of their processing period by calling sched_yield(). This modification to the semantics of that system call makes some developers uneasy, though; it is likely that the final patch will do something different. There may be a new "I'm done for now" system call added for this purpose.

Peter also gave a talk in Dresden; his was mostly about why Linux does not have a deadline scheduler yet. The "what happens when a process exceeds its WCET" problem was one of the reasons he gave. Calculating the worst-case execution time is exceedingly difficult for any sort of non-trivial program. As Peter puts it, researchers have spent their entire lives trying to solve it. There are people working on automatically deriving WCET from the source, but they are far from being able to do this with real-world systems. So, for now, specification of the WCET comes down to empirical observations and guesswork.

Another serious problem with EDF is that it works much better on single-processor systems than on SMP systems. True EDF on a multiprocessor system requires the maintenance of a global run queue, with all of the scalability problems that entails. One solution is to partition SMP systems, so that each CPU becomes an essentially independent scheduling domain; the SCHED_EDF patch works this way. Partitioned systems have their own problems, of course; the assignment of tasks to CPUs can be a pain, and it is hard (or impossible) to get full utilization if tasks cannot move between CPUs.

Another problem with partitioning is that some scheduling problems simply cannot be solved without occasional process migration. Imagine a two-CPU system running three processes, each of which needs 60% of a single CPU's time. The system clearly has the resources to run those three processes, but not if it is unable to move processes between CPUs. So a partitioned EDF scheduler needs to be able to migrate processes occasionally; the SCHED_EDF developers have migration logic in the works, but it has not yet been posted.

Yet another serious problem, according to Peter, is priority inversion. The priority inheritance techniques used to solve priority inversion are tied to priorities; it is not clear how to apply them to deadline schedulers. But the problem is real: imagine a process acquiring an important lock, then being preempted or forced out because it has exceeded its WCET. That process can then block the execution of otherwise runnable processes with urgent deadlines.

There are a few ways to approach this issue. Simplest, perhaps, is deadline inheritance: lock owners inherit the earliest deadline in the system for as long as they hold the lock. More sophisticated is bandwidth inheritance; in this case, a lock owner which has exhausted its WCET will receive a "donation" of time from the process(es) blocked on that lock. A variant of that technique is proxy execution: blocked processes are left on the run queue, but, when they "run," the lock owner runs in their place. Proxy execution gets tricky in SMP environments when multiple processes are blocked on the same lock; the result could be multiple CPUs trying to proxy-execute the same process. The solution to that problem appears to be to migrate blocked processes to the owner's CPU.

Proxy execution also runs into difficulties when the lock-owning process is blocked for I/O. In that case, it cannot run as a proxy for the original blocked task, which must then be taken off the run queue. That, in turn, forces the creation of a "wait list" of processes which must be returned to a runnable state when a different process (the lock owner) becomes runnable. Needless to say, all this logic adds complexity and increases system overhead.

The final problem, according to Peter, is POSIX, but it's an easy one to solve. Since POSIX is silent on the topic of deadline schedulers, we can do anything we want and life is good. He repeated that SCHED_DEADLINE will probably be placed above SCHED_FIFO in priority. There will be a new system call - sched_setscheduler_ex() - to enable processes to request the deadline scheduler and set the parameters accordingly; the SCHED_EDF patch already implements that call. So many of the pieces for deadline scheduling for Linux are in place, but a number of the details are yet to be resolved.

The bottom line is that deadline schedulers in the real world are a non-trivial problem - something that is true of real-world scheduling in general. These problems should be solvable, though, and Linux should be able to support a deadline scheduler at some point in the future. That scheduler will probably make its first appearance in the realtime tree, naturally, but it could eventually find users well beyond the realtime community. Deadline schedulers are a fairly natural fit for periodic tasks like the management of streaming media, which could profitably make use of deadline scheduling to help eliminate jitter and dropped-data problems. But that remains a little while in the future; first, the code must be made ready for widespread use. And that, as we all know, is a process which recognizes few deadlines.

Comments (40 posted)

Defining the Fedora Project

There has been a discussion on the Fedora advisory board mailing list recently on the subject: "What is the Fedora Project?". John Poelstra started the discussion in an attempt to get this item off the agenda before 2010.

Beside the fact that it's an old agenda item, the Fedora Project continues to grow, and, without direction, that growth could eventually lead to fragmentation and chaos. Some definition of the target audience for the Fedora distribution, and some goals for the project are useful for everyone involved.

Mike McGrath wrote:

While there was general agreement that some kind of focus was needed, Greg DeKoenigsberg wanted to make it clear that competing visions still have a place in the project:

But, Máirín Duffy sees it as more of a positioning and messaging problem, as, currently, there is no coherent story for Fedora:

The project is more than just a distribution, it includes the entire community of contributors and users. The Fedora distribution also includes many spins, each of which has its own target audience, so perhaps the definition of a target audience should only apply to the default spin. Does that default spin contain development tools to appeal to developers? Does it target the lowest common denominator user with software for email, web browsing and an office suite? The project would like the default spin to fit on a single CD, so that it is accessible to people with low bandwidth and older hardware. You can't make a decision about what goes into the default if you don't know your target audience.

Fedora does have a Mission statement: "The Fedora Project's mission is to lead the advancement of free and open source software and content as a collaborative community." It also has defined values, or Foundations.

Fedora needs to define goals that align with its mission and values. No decisions have been made yet, be we look forward to seeing some definition to Fedora's target audience and some goals for the future soon.

Comments (16 posted)

Gentoo Ten Live DVD 10.1

Gentoo has released an update to the Gentoo Ten Live DVD. Version 10.1 contains numerous bug fixes and enhancements.

Comments (none posted)

Mandriva Linux 2010 RC2 is available

Mandriva Linux 2010 RC2, last development version, is available for testing. See the 2010 RC2 wiki page for additional information.

Comments (none posted)

Nexenta Core Platform 3 Alpha1 Released

The Nexenta project has announced the availability of the Nexenta Core Platform 3.0 alpha1. This is the first release towards NCP3.0. "This is an alpha release, and a moving target until we reach the final release. It is in a good shape for developers to tinker with.. and we welcome community efforts to improve NCP3."

Full Story (comments: none)

A report from the RPM summit

An RPM summit, featuring developers from Red Hat and SUSE, was held at the openSUSE conference in September. A report from the event has been posted; it is a sort of near-future roadmap for RPM. "Soft dependencies keywords that are already used by SUSE (Recommends, Suggests, Supplements, Enhances) will be recognized in the future versions of RPM. RPM will not do anything with them except of storing in the database and reporting to higher levels of package management stack. There is an ongoing discussion whether and how to implement soft dependencies in YUM."

Comments (34 posted)

Debian pushes development of kFreeBSD port

The Debian Release Team has announced that squeeze, the upcoming release, will support the FreeBSD kernel. "The kFreeBSD architectures for the AMD64/Intel EM64T and i386 processor architectures are now release architectures. Severe bugs on these architectures will be considered release critical the same way as bugs on other architectures like armel or i386 are. If a particular package does not build or work properly on such an architecture this problem is considered release-critical."

Full Story (comments: none)

Misc Debian developer news (#18)

This issue of miscellaneous developer news covers 3-way merge of debian/changelog files, Pending security issues now shown in PTS pages, DEP-3 updated for compatibility with git format-patch, debian-devel and ITPs, and wiki.debian.org bugstatus.

Full Story (comments: none)

Resignation of Josh Boyer from FESCo

Josh Boyer has resigned from the Fedora Engineering Steering Committee (FESCo). David Woodhouse will replace Josh on FESCo. "Josh will be focusing on the QA of updates to stable Fedora releases, where he will need all the help that he can get. Please reach out to Josh and offer whatever help you can provide him in this effort."

Full Story (comments: none)

Re-introducing Mandriva Linux

Mandriva has introduced a series of articles re-introducing Mandriva Linux. The first article, "Being a Linux distribution publisher", is out. "Let's start with a large overview of the Linux distribution publisher activity for Mandriva. We will deal with the best known, public and freely available versions (Free and One)."

Comments (none posted)

Ubuntu to store copies of all users' address books

Here's an interesting note from Canonical's Elliot Murphy, noting that CouchDB 0.10.0 has been loaded into the nearly-ready "Karmic" release. It seems they have big plans for how they plan to use it: "[B]y the time Ubuntu 9.10 is released on October 29th every single Ubuntu user will have an address book stored in CouchDB that replicates with one.ubuntu.com, and Tomboy notes that are replicated via a web API at the application but then stored in CouchDB and carried along in the CouchDB replication that we have set up. Optionally they can also store all their Firefox bookmarks in CouchDB and have those replicated as well. We'll be doing our best to help teach application developers to use CouchDB in order to 'cloud-enable' their apps."

Comments (64 posted)

FreeBSD Status Reports April - September, 2009

The FreeBSD Quarterly Status Report is available. "This report covers FreeBSD related projects between April and September 2009. During that time a lot of work has been done on wide variety of projects, including the Google Summer of Code projects. The BSDCan conference was held in Ottawa, CA, in May. The EuroBSDCon conference was held in Cambridge, UK, in September. Both events were very successful. A new major version of FreeBSD, 8.0 is to be released soon. If you are wondering what's new in this long-awaited release, read Ivan Voras' excellent summary."

Full Story (comments: none)

DistroWatch Weekly, Issue 324

The DistroWatch Weekly for October 12, 2009 is out. "Ladislav and crew are on vacation this week, but we'll muddle through okay. It was a bit of slow week as several popular distributions are gearing up for their next major releases, but the news has been exciting. Novell got annoyed at Red Hat claiming 75% market penetration and Debian was used to power an underwater vehicle to victory. Then I moved into Sabayon's latest to see if their KDE 4 build could perform any better than others I've tried. All this and more in this week's issue of DistroWatch Weekly - happy reading!"

Comments (none posted)

Fedora Weekly News 197

The Fedora Weekly News for October 11, 2009 is out. "Starting off with announcements, which includes general, development and event announcements, word that the Docs team will be switching to the Creative Commons Attribution-Share Alike 3.0 Unported License (CC-BY-SA), an update on Fedora Engineering Steering Committee (FESCo) leadership, and updates on Fedora 12 milestones. In news from the Fedora Planet, selected posts from the Fedora contributor community that includes discussion on "What is Fedora?", mockups for the fedoraproject.org redesign, and discussion on virt-top. In Ambassador news, detail on the Utah Open Source Conference. Translation brings us notification of new members to the Fedora Localization Project, coverage of some discussion around docs.fedoraproject.org issues, and other issues. In Design Team news, a request for more font packagers, discussion around reuse of Fedora Remix logos, and acceptable use cases. There are a few Fedora 10 and 11 security updates in the Security Advisories beat, and the issue rounds out with virtualization news, including more detail on the new virt-top release, and limiting VNC access to a single guest. Read on, and enjoy!"

Full Story (comments: none)

OpenSUSE Weekly News/92

This issue of the OpenSUSE Weekly News covers openSUSE News: Introducing the 'openSUSE Boosters' Team, The Geek Stuff/Ramesh Natarajan: Unix Sed Tutorial: How To Write to a File Using Sed, Martin Vidner: WebYaST Beta 1, openSUSE Forums: openSUSE 11.2 will support live updates, KDE 4.3.2 Stabilizes Free Desktop, and more.

Comments (none posted)

Ubuntu Weekly Newsletter #163

The Ubuntu Weekly Newsletter for October 10, 2009 is out. "In this issue we cover: 2009 Community Council vote complete, Ubuntu Server Eucalyptus Testers Needed, Developer Membership Board Meeting: New Approval Process, Ubuntu Translation Templates Priority, New MOTU's, LoCo News: Catalan, Copenhagen, & Paris, Bazaar 2.0.0: interview with Martin Pool, Help us improve Launchpad's icons, Ubuntu Forums Interview & Tutorial of the Week, The Planet: Joey Stanford & Roderick Greening, Ubuntu 9.10 - Almost Perfect, Hulu Desktop (Linux), and much, much more!"

Full Story (comments: none)

The 10 Best Linux Distributions of 2009 (DaniWeb)

DaniWeb takes a look at the 10 best distributions for 2009. "It was exactly one year ago today that I published my original "The 10 Best Linux Distributions" and it's time to put forth a new list for this year's best. Without looking at the old list, I've decided to compile this one from scratch. This 2009 list takes several factors into account for placement in the list: Community support, commercial support, software variety, update engine and distribution frequency. Even for old Linux salts, there are a few surprises on this list. For starters, Ubuntu is not number one."

Comments (none posted)

More Linux Distros That Don’t Suck (tech.nocr.at)

tech.nocr.at presents part two in a series on Linux Distros That Don’t Suck. "Following Linux Distros That Don’t Suck from earlier this year here is a comprehensive list of, you guessed it, more Linux Distros that don’t suck. Some of these are more obscure lesser known distros that are quite powerful and very useful. Some of these come from personal use some of them came from the great comments you readers posted the first time I did this."

Comments (none posted)

A strategy for dealing with a flaky computer

Your author has just made it through some rough times in the world of computer reliability and thought he should share his experiences with the LWN readers. It all started a few months back when his primary desktop machine, a 3+ year old Lini box with a 3 Ghz Athlon 64 processor (reviewed here) started to act up.

The machine had been running without too many problems for most of its life, it had served well as a test platform for a variety of hardware configurations and had run many versions of the Ubuntu distribution with much success. A few months back, the machine was upgraded from Ubuntu 8.10 "Intrepid Ibex" to Ubuntu 9.04 "Jaunty Jackalope". There were a few initial and easily solved issues with a runaway chippcardd4 process, the upgrade appeared to work. Except, the machine became noticeably sluggish and the problem was difficult to locate. The top command did not reveal any load problems, /var/log/messages had nothing interesting to show, nor did the dmesg command. The machine seemed slower, but it worked well enough that your author decided to live with it until Ubuntu 9.10 "Karmic Koala" was released.

After several months of using Jaunty Jackalope, things got really strange. While pounding on Firefox, the most used application, the machine's user interface would become locked up. The mouse still moved on the screen, but all keyboard input was frozen. It was still possible to connect to the machine from a remote host via ssh. Doing so and running top showed that the Xorg process was eating up the CPU. The /var/log/Xorg.0.log file revealed nothing. Luckily, it was possible to reboot the machine and avoid a risky crash and a time consuming 1TB fsck operation. This mode of operation continued for a while and your author really started to appreciate Firefox's ability to recover the previous session. It seemed like an opportune time to make a fresh backup of the machine to a remote host, something your author does every 2 weeks or so. That effort succeeded.

At this point, your author started suspecting flaky hardware. The keyboard seemed like a likely, if unusual culprit, it was swapped with no change. The video card was swapped out for a different model to force a change in the video card driver, still no improvement. It had been over a year since the CPU fan had been cleaned, this time, the fan was removed and "detailed" with a damp toothbrush. The CPU was cleaned and new heat sink grease was applied. The dust was blown off of the motherboard and power supply. The memory and I/O cards were cleaned and reseated. The BIOS was checked for proper power supply voltages. Memtest86 was run and the memory checked ok. Still, the machine flaked out under normal use.

At this point, your author was very glad that he had previously set up an old Athlon 1700 machine as an alternate work platform. That machine was used to perform the weekly LWN email and article processing tasks, it was slow, but slow is better than randomly flaky.

Around this time, the beta release of Ubuntu 9.10 "Karmic Koala" was announced. A copy of the beta iso was downloaded on the reliable machine and a disk was burned (and verified). An older 500 GB SATA disk was swapped into the machine and the new operating system was installed on that disk. The old 1TB SATA disk was installed as the secondary drive and all of the data (/home, your author's 250GB music collection and a few /etc config files) were copied to the 500GB drive. The Karmic Koala beta installation was a success, suddenly the Lini became much faster and it has been used heavily for over a week with no lockups. So far, the only problem with the new system has involved some folder refresh issues and key mapping problems with Claws-Mail 3.7.2. Otherwise, the system has been solid.

Upgrading to a newer distribution version, whether as an emergency measure or as a planned job, is much easier to do if you have one or more spare system disk drives. The extra drive can be thought of as a bootable data cartridge. The secondary backup on the remote machine allowed your author to sleep at night and the functional but slow alternate machine allowed work to continue without much interruption. Finally, the old drive with the flaky system is still intact if further investigation is required, it will soon become the next backup target. Your author has always had a preference for full installs over upgrades. It may be superstitious to blame the Intrepid->Jaunty upgrade as the root of the problem with no real data to prove that theory, but a fresh install will always clean out the cobwebs from the old system.

Comments (19 posted)

Firebird 1.5.6 released

Version 1.5.6 of the Firebird DBMS has been announced. "The project is pleased to announce that Firebird 1.5.6 release kits are now available for all the supported main-line platforms (Win32, Linux i86 and MacOS-X/Darwin i86 and PPC). No further sub-releases are planned for the V.1.5.x series."

Comments (none posted)

MySQL Server 5.4.3-beta released

Version 5.4.3-beta of MySQL Server has been announced. "MySQL 5.4 is based on MySQL 5.1 but includes several high-impact changes to address scalability and performance issues in MySQL Server. These changes exploit advances in hardware and CPU design and enable better utilization of existing hardware. MySQL 5.4 currently has Beta status."

Full Story (comments: none)

PostgreSQL Weekly News

The October 11, 2009 edition of the PostgreSQL Weekly News is online with the latest PostgreSQL DBMS articles and resources.

Full Story (comments: none)

libshcodecs 0.9.7 released

Version 0.9.7 of libshcodecs has been announced, it adds Android support. "libshcodecs is a library for controlling SH-Mobile hardware codecs. The [SH-Mobile][0] processor series includes a hardware video processing unit that supports MPEG-4 and H.264 encoding and decoding. libshcodecs is available under the terms of the GNU LGPL."

Full Story (comments: none)

libshveu 1.1.0 released

Version 1.1.0 of libshveu, a driver for the SH-Mobile processor's video engine unit, has been announced. "This is the first public release of libshveu. It contains: * src/libshveu: the libshveu shared library * src/tools: commandline tools".

Full Story (comments: none)

BusyBox 1.15.2 released

Version 1.15.2 of BusyBox, a collection of command line utilities for embedded systems, has been announced. "Bug fix release. 1.15.2 has fixes for ash and hush (`trap` handling), dd (fixed handling of I/O errors), find (fix for -follow and symlinks), pidof (corrected recognition of kernel thread names), sed (SEGV), uniq (memory leak), line editing (Ctrl-D works again), build system."

Comments (none posted)

Free N900 (KDE.News)

KDE.News has a report from the Maemo Summit, where Nokia handed out 300 N900 phones to developers. "However, it is not the hardware which is most interesting to us - it is the software. The N900 runs Maemo, a Debian Linux based operating system for high-end smart phones. Compared to pretty much all competition, the N900 offers a very open software platform and provides a terminal application by default. Most of the GUI is currently build upon GTK, using Clutter on OpenGL and various other FOSS components in the lower stack. But the upcoming release, Maemo 'Harmattan' 6 will receive a new, Qt based GUI. Qt is already available for the N900 and in the first half of next year we can expect an officially supported Qt 4.6 release for Maemo 'Freemantle' 5."

Comments (26 posted)

UIOMux 1.0.0 released

Version 1.0.0 of UIOMux has been announced. "UIOMux is a conflict manager for system resources, including UIO devices. This is the first public release of libuiomux, and targets Renesas SH-Mobile processors."

Full Story (comments: none)

Ardour 2.8.3 released

Version 2.8.3 of Ardour, a multi-track audio workstation, has been announced. "This is primarily a mantainance released, but does include at least 3 major new pieces of functionality. It also includes a suprisingly large number of bug fixes for things varying from clean builds with the latest compiler releases to minor GUI tweaks and the occasional crashing bug."

Comments (none posted)

QjackCtl 0.3.5 released

Version 0.3.5 of QjackCtl, a GUI front-end for the JACK audio connection kit, has been announced. This version adds new capabilities and bug fixes.

Full Story (comments: none)

Timemachine 0.3.3 released

Version 0.3.3 of Timemachine has been announced. "It can now be configure[d] to start recording when the input level rises over some threashold, and stop when it falls below for some specified period of time, making it easier to do hands-free recordings."

Full Story (comments: none)

GNOME Software Announcements

The following new GNOME software has been announced this week:

• Banshee 1.5.1 (new features and bug fixes)
• couchdb-glib 0.5.2 (bug fixes)
• Dogtail 0.7.0 (new features, code cleanup and documentation work)
• Flickr Remote Organizer for GNOME 0.2 (new features and bug fixes)
• gedit-plugins stable branch (Gnome 2.28 branch)
• GParted 0.4.7 (bug fixes and translation work)
• GNOME Do plugins 0.8.2.1 (bug fixes)
• GNOME Shell 2.28.0 (new features, bug fixes and code cleanup)
• Libgda 4.0.5 (new features, bug fixes and documentation work)
• Libgda 4.1.3 (new features, bug fixes and translation work)
• libvtemm 0.22.1 (bug fixes and documentation work)
• tracker 0.7.2 (new features and bug fixes)

You can find more new GNOME software releases at gnomefiles.org.

Comments (none posted)

KDE Software Announcements

The following new KDE software has been announced this week:

• Bibliotehk 0.4 (unspecified)
• Device Information KCM 0.1.0 (initial release)
• Flacon 0.1 (initial release)
• Konversation 1.2 (KDE4 release)
• Krecipes 2.0-alpha3 (bug fixes)
• QOTR-DecoderQUI 0.1.3 (unspecified)

You can find more new KDE software releases at kde-apps.org.

Comments (none posted)

Xorg Software Announcements

The following new Xorg software has been announced this week:

• font-util 1.1.0 (bug fixes and new macro requirement)
• font-util 1.1.1 (bug fix)
• iceauth 1.0.3 (bug fixes, documentation work and new macro requirement)
• kbproto 1.0.4 (bug fix and code cleanup)
• libdrm 2.4.15 (new features, code cleanup and documentation work)
• libWindowsWM 1.0.1 (code cleanup and documentation work)
• libXcomposite 0.4.1 (new macro requirement, code cleanup and documentation work)
• libXfont 1.4.1 (bug fixes, documentation work and new macro requirement)
• libXfixes 4.0.4 (new macro requirement, code cleanup and documentation work)
• libXfontcache 1.0.5 (new macro requirement, code cleanup and documentation work)
• libXft 2.1.14 (new macro requirement, code cleanup and documentation work)
• libXpm 3.5.8 (build and bug fixes)
• libXres 1.0.4 (code cleanup and documentation work)
• libXt 1.0.7 (build fixes and code cleanup)
• libXxf86misc 1.0.2 (code cleanup and documentation work)
• makedepend 1.0.2 (bug fixes and code cleanup)
• mkfontdir 1.0.5 (new macro requirement and documentation work)
• mkfontscale 1.0.7 (new macro requirement and documentation work)
• sessreg 1.0.5 (code cleanup and documentation work)
• X11R7.5 font modules RC1 (new features, bug fixes and documentation work)
• X11R7.5 font modules (new fonts and bug fixes)
• xbitmaps 1.1.0 (new macro requirement and documentation work)
• xev 1.0.4 (bug fixes, code cleanup and documentation work)
• xf86-input-evdev 2.2.99.2 (bug fixes and documentation work)
• xf86-input-synaptics 1.2.0 (code cleanup and documentation work)
• xf86miscproto 0.9.3 (bug fixes and new macro requirement)
• xf86_video-openchrome 0.2.904 (new features and bug fixes)
• xf86-video-radeonhd 1.3.0 (new features, bug fixes and code cleanup)
• xgamma 1.0.3 (new macro requirement and documentation work)
• xhost 1.0.3 (new feature, new macro requirement and documentation work)
• xinput 1.5.0 (code cleanup and documentation work)
• xkill 1.0.2 (new macro requirement, bug fixes and documentation work)
• xlsatoms 1.0.2 (new macro requirement, bug fixes and documentation work)
• xorg-cf-files 1.0.3 (new macro requirement, bug fixes and code cleanup)
• xorg-server 1.6.5 (new feature and bug fix)
• xorg-server 1.7.0.901 (new features and bug fixes)
• xprop 1.1.0 (new features, code cleanup and documentation work)
• xproxymanagementprotocol 1.0.3 (new macro requirement and code cleanup)
• xrdb 1.0.6 (bug fixes, code cleanup and documentation work)
• xwininfo 1.0.5 (new feature, code cleanup and documentation work)
• xwud 1.0.2 (bug fixes and code cleanup)

More information can be found on the X.Org Foundation wiki.

Comments (none posted)

rst2pdf version 0.12 released

Version 0.12 of rst2pdf, a tool to generate PDF files directly from restructured text sources via reportlab, has been announced. "This version includes many bugfixes and **MANY** new features compared to the previous 0.11 version, including but not limited to better styling, integration with `sphinx, a very raw, preliminar graphical frontend called bookrest, and a much more powerful tables implementation."

Full Story (comments: none)

Synfig 0.62.00 released

Version 0.62.00 of Synfig, a 2D animation and design program, has been announced. "The 0.62.00 version comes with a lot of new features and mayor changes: * Migration to new host. * Synfig code is now at sourceforge in git. * Wiki reworking and new website. *New features ** SVG import module. ** Keyframe widget: ** New toggle buttons in the canvas window ** Onion skin past and future are selectable individually ** New Curve Warp Layer. ** Lots of bugs fixed, and much more...."

Full Story (comments: none)

Wine 1.1.31 announced

Version 1.1.31 of Wine has been announced. Changes include: "- Vastly improved monthcal control. - Performance improvements for DIB sections. - Several sound driver fixes. - Beginning of ActiveX support in JScript. - More Direct3D 10 work. - More 16-bit dlls split off to separate modules. - Support for attachments in MAPI. - Various bug fixes."

Comments (none posted)

Claws Mail 3.7.3 unleashed

Version 3.7.3 of Claws Mail has been announced, it includes a number of new capabilities and bug fixes.

Full Story (comments: none)

Claws Mail Extra Plugins 3.7.3 unleashed

Version 3.7.3 of Claws Mail Extra Plugins has been announced. "The claws-mail-extra-plugins-3.7.3 package contains 19 plugins, including 1 new plugin: Python!"

Full Story (comments: none)

Oggz 1.1.0 released

Version 1.1.0 of Oggz, a library which provides commands to inspect, edit and validate Ogg files, has been announced. "This release introduces a new API structure for reporting the byte offsets of the start and end pages for each packet."

Full Story (comments: none)

alsa-midi-latency-test 0.0.2 released

Version 0.0.2 of alsa-midi-latency-test has been announced. "alsa-midi-latency-test measures the roundtrip time of a MIDI message in the alsa subsystem of the linux kernel using a high precision timer. It calculates the worst case roundtrip time of all sent MIDI messages and displays a histogram of the rountrip time jitter."

Full Story (comments: none)

Qtractor 0.4.3 released

Version 0.4.3 of Qtractor, an Audio/MIDI multi-track sequencer, has been announced. "One can also think as the last and stable release before a probable next generation do break all loose. Automation and full MIDI control is popping up over the horizon."

Full Story (comments: none)

Sonic Visualiser 1.7 released

Version 1.7 of Sonic Visualiser, an audio analysis package, has been announced. "This release contains a number of new features, enhancements, and bug fixes."

Full Story (comments: none)

Roundup 1.4.10 released

Version 1.4.10 of Roundup, an issue-tracking system with command-line, web and e-mail interfaces, has been announced. "I'm proud to release version 1.4.10 of Roundup which fixes some bugs".

Full Story (comments: none)

SeaMonkey 2.0 Release Candidate 1 released

Version 2.0 Release Candidate 1 of SeaMonkey, an all-in-one internet application suite, has been announced. "We encourage testers to get involved in discussing and reporting problems as well as further improving the product."

Full Story (comments: none)

KOffice 2.1 Beta 3 released (KDEDot)

KDE.News has announced KOffice 2.1 Beta 3. "The KOffice team is happy to announce the third beta of the upcoming KOffice 2.1. This extra beta has been added to ensure the highest quality for the final 2.1 release. The KOffice team has worked overtime and can show a longer list of fixed bugs than ever."

Comments (none posted)

SyncEvolution 0.9.1 beta 1 released

Version 0.9.1 beta 1 of SyncEvolution has been announced and version 1.0 development branch of SyncML server is also available. "SyncEvolution synchronizes personal information management (PIM) data like contacts, calenders, tasks, and memos using the SyncML information synchronization standard. Up to and including 0.9.x, a third-party SyncML server is required."

Full Story (comments: none)

Caml Weekly News

The October 13, 2009 edition of the Caml Weekly News is out with new articles about the Caml language.

Full Story (comments: none)

cssutils 0.9.6 final released

Version 0.9.6 of cssutils, a Python package for working with Cascading Style Sheets, has been announced. This is a bug fix release.

Full Story (comments: none)

Distribute 0.6.4 released

Version 0.6.4 of Distribute has been announced. "Distribute is a fork of the Setuptools project. Distribute is intended to replace Setuptools as the standard method for working with Python module distributions, on the top of Distutils. - This release is now compatible with zc.buildout".

Full Story (comments: none)

Docutils 0.6 released

Version 0.6 of Docutils has been announced. "Changes are : * Two new writers for ODT and manpage (so there is no excuse for python software not having a manpage anymore). * Python2.2 is no longer supported. Release 0.6 is compatible with Python versions from 2.3 up to 2.6 and convertible to 3.1 code..."

Full Story (comments: none)

gevent 0.11.0 released

Version 0.11.0 of gevent has been announced, it includes bug fixes. "gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of libevent event loop."

Full Story (comments: none)

Testoob 1.15 released

Version 1.15 of Testoob, a Python test suite, has been announced. "Version 1.15 (Oct. 2009) adds better Python 2.6, IronPython, and Jython support, as well as test coverage improvements, better color support, and some new options and bugfixes."

Full Story (comments: none)

GDB 7.0 released

Version 7.0 of the GDB debugger - the first major release since 2003 - has been announced. Some of the major additions include support for scripting the debugger in Python, reverse debugging (which records the state of the program and can step backward through its execution history), non-stop debugging (whereby a single thread in a multi-threaded program can be stopped while the others continue to run), better support for inline functions, and more.

Full Story (comments: 33)

GIT 1.6.5 released

Version 1.6.5 of the GIT distributed version control system has been announced. "This cycle took a bit longer than I hoped, but here it is. We already have some new features cooking in 'next', and I expect we may be able to have 1.6.6 by the end of the year."

Full Story (comments: none)

KDE Community Forums celebrate their first birthday (KDEDot)

KDE.News has announced the first year anniversary of the KDE community forums. "Exactly one year ago, on October 12th, the KDE Community Forums were founded. It was about time to give users a place for discussion and support beside the mailing lists, which were mainly used by developers and other contributors. A lot of time has passed since then, and the forums have grown into a healthy community, contributing to the KDE landscape as a whole. Here people can ask KDE-related questions, help other users, find useful information, or just discuss whatever comes to mind."

Comments (none posted)

X.Org Foundation Board annual elections

The X.Org Foundation Board annual elections have been announced. "The next annual election of the X.Org Foundation Board of Directors will be in November---nominations start right away."

Full Story (comments: none)

Cisco becomes a major Linux server vendor overnight (cnet)

cnet reports on Cisco's entry into the Linux market. "In what might have looked like a publicity stunt around a $100,000 prize for Linux developers, Cisco's Linux development contest was actually a major clue as to just how serious it is about becoming a leading server vendor with a global development community--and soon. Today, Cisco announced the winners of its "Think Inside the Box" contest. The three winning applications are very interesting, but the bigger story here is what Cisco's contest just demonstrated: Most of Cisco's 7 million installed Integrated Services Routers (ISRs) are now servers, for all intents and purposes. The contest proved that server-side Linux developers who know C/C++, Java, or Python can now write applications to Cisco routers with little or no knowledge of routers."

Comments (19 posted)

Hulu Desktop Brings Remote-Controlled Streaming to Linux (Lifehacker)

Lifehacker takes a look at Hulu's new Linux version. "Hulu has taken their designed-for-widescreen, remote-friendly Desktop viewer and released it for Linux-powered systems, giving anyone with an open-source media center or Linux laptop an easier way to watch streaming video clips and movies on their much bigger screen."

Comments (12 posted)

Intel talks Linux, netbooks and rivalry with ARM (ZDNet Asia)

ZDNet Asia presents an interview with Intel's director of Intel's Open Source Technology Center, Imad Sousou. "You're about to release Moblin v2.0. Can you tell us more about that? [As a] release from an open source perspective, it's a milestone more than it is a product release--product releases come from OS vendors and OEMs [manufacturers]. Soon you will start to see OEMs shipping netbooks with Moblin. You will see an announcement over [time]. It's still in beta--right now we're in the very final phase. Very minor but critical bug fixes are being done based on input from OEMs and OS vendors. We don't want to call it a release until someone actually ships a product with it. The final version will be available for download over the next couple of weeks. Once that happens, we will go on a regular cadence of a release every six months… to make it easy for people to be able to have predictability in terms of time schedules."

Comments (16 posted)

Linux Foundation woos with lifetime linux.com handle (The Register)

The Register reports that @linux.com mailing addresses are available from the Linux Foundation for a one-time fee. "Previously, members needed to keep up with their yearly membership payments in order to hold a @linux.com addy. But as Zemlin tells us, folks kept requesting a way to hold on to their "geek street cred" without being tied to their membership status. With the change, current members can now secure the email from a one-time fee of $150. New members will need to pay $250 for a one-year membership along with the lifetime benefit."

Comments (10 posted)

Nokia ports Qt to Maemo 5 (the Inquirer)

The Inquirer covers Nokia's port of Qt to Maemo 5. "According to Nokia, there are currently community created ports available, which the company used to help create the official upcoming port. The port itself will be based on Qt's upcoming version 4.6, a beta version of which is expected to be launched next week at the Qt Developer Days conference in Munich."

Comments (14 posted)

Next-gen Nokia Linux devices will get multitouch and Qt UI (ars technica)

Ars technica looks forward to Maemo 6 and the devices that will use it. "DRM enforcement and access to protected content will only be available on stock ROMs that can be validated with code signing. Users will still be completely free to hack their devices and install custom ROMs, but they won't be able to play protected content if they do so. The specific mechanism has not yet been described, but Nokia says that users will be able to switch between the DRM-mode and the open ROM mode (except, potentially, on carrier-locked devices)."

Comments (18 posted)

EFF Warns Texas Instruments to Stop Harassing Calculator Hobbyists

The Electronic Frontier Foundation is warning TI to stop harassing its calculator customers over a DMCA issue. "The Electronic Frontier Foundation (EFF) warned Texas Instruments (TI) today not to pursue its baseless legal threats against calculator hobbyists who blogged about potential modifications to the company's programmable graphing calculators. TI's calculators perform a "signature check" that allows only approved operating systems to be loaded onto the hardware. But researchers were able to reverse-engineer signing keys, allowing tinkers to install custom operating systems and unlock new functionality in the calculators' hardware."

Full Story (comments: none)

The Book of Xen--New from No Starch Press

No Starch Press has published The Book of Xen by Chris Takemura and Luke S. Crawford.

Full Story (comments: none)

2020 FLOSS Roadmap announced

The 2009 version of the 2020 FLOSS Roadmap has been announced. "This Roadmap is a projection of the influences that will affect FLOSS until 2020, with descriptions of all FLOSS-related trends as anticipated by an international workgroup of 40 contributors over this period of time and highlights 7 predictions and 8 recommendations."

Comments (none posted)

FSFE Newsletter

The October 12, 2009 edition of the FSFE Newsletter is online with the latest Free Software Foundation Europe news. Topics include: "1. FSFE to EC: Don't waste an opportunity with a hasty deal 2. The Fellowship interviews: Andreas Tolf Tolfsen 3. The Second Fellowship Jabber meeting, 22 September 4. Celebrating the Software Freedom Day, 19 September 5. FSFE uses Free Software micro-blogging on Identi.ca 6. Welcome to Hugo Roy and Lena Simon: new Berlin interns".

Full Story (comments: none)

Updated R5xx 3D programming guide available

AMD has released an updated R5xx 3D programming guide. "The new version offers expanded coverage on texture memory layouts, some notes on implementing anti-aliased lines/points/polys, and some additional registers."

Full Story (comments: none)

WikiReader: OpenMoko's "Project B"

The mysterious new project being pursued by OpenMoko founder Sean Moss-Pultz has now been unveiled: it's a small gadget called "WikiReader," which appears to be a simple device containing a full copy of Wikipedia. "WikiReader takes our original ideas of openness and accessibility to an even greater realm. WikiReader is so amazingly simple. There really is no interface. You turn it on and instantly become immersed in the rich world of reading specific topics or the serendipitous pleasure of discovering something by chance." More information on the WikiReader site, but it's sparse. In particular, one might assume it's a Linux-based device, but there's no information on the operating software or how it might be extended.

Full Story (comments: 30)

Open source identity: PulseAudio creator Lennart Poettering (CIO)

CIO interviews Lennart Poettering. "One should never forget what we are doing here. We took an audio system that followed the low-level design that was current in the early '90s and brought it in one big step to what is current today. We inserted an entire new layer into our stack right in the middle, so that we can catch up with the more advanced audio stack that Mac OS X or Windows provide right now. Doing something like this, of course, will trigger problems at many places. Criticism hence must be expected."

Comments (9 posted)

Robotic Submarine Running Debian Wins International Competition (debian-news.net)

A team of 35 students from Cornell University won the recent Autonomous Underwater Vehicle Competition with a submarine running Debian. In addition, Debian was used to design the electronics, and various other free software tools were used for image processing and video capture. "The competition takes place in a large acoustic testing pool operated by the US Navy SPAWAR Systems Center. It calls for entries to pass through a gate, follow a path, ram a submerged buoy, fire through a square target with small torpedoes, drop markers into bins containing simulated targets, recover a PVC target and surface through an octagon shape, all without human intervention."

Comments (10 posted)

The Social Desktop Winners (KDEDot)

KDE.News has announced the winners of the Social Desktop Contest. "The Social Desktop Contest was launched in June with the goal to bring Web 2.0 ideas and our user and developer community closer to the desktop and foster community development and innovations around the OCS API." Winning projects include: "ExtendedAboutDialog for KDE apps" by Téo Mrnjavac, "Knowledge base widget" by Marco Martin, "libopengdesktop" by Guido Roberto and "PyContent" by Ni2c2k.

Comments (none posted)

PyCon 2010 US - Call For Tutorials ending soon

The Call For Tutorials for PyCon 2010 ends October 18. "There is still time for you to get a proposal on your favorite Python topic and teach a 3-hour class (with breaks and refreshments) to your colleagues on the Wednesday or Thursday before the conference ("Tutorial Days")."

Full Story (comments: none)

MOSS 8 Seminar - Tokyo (LinuxMedNews)

The 8th Medical Open Source Software Seminar has been announced. "It is a pleasure for me to announce the 8th Medical Open Source Software Seminar (MOSS8) in Tokyo, Japan as follows:. * Date and Time: October 31, 2009, from 13:00 to 18:00 * Location: Waseda University, Building 19, Room 610".

Comments (none posted)

pyTexas: Regional Conference - Oct 24/25

pyTexas takes place in Ft. Worth, TX on October 24-25. "The format is scheduled talks on Saturday morning followed by open space talks driven by the attendees in the afternoon. On Sunday there will be sprints on various projects and, in parallel, a Python Lab that tests the attendees with interesting programming puzzles."

Full Story (comments: none)

Events: October 22, 2009 to December 21, 2009

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
October 19 October 22	ZendCon 2009	San Jose, CA, USA
October 21 October 23	Japan Linux Symposium	Tokyo, Japan
October 22 October 24	Décimo Encuentro Linux 2009	Valparaiso, Chile
October 23 October 24	Ontario GNU Linux Fest	Toronto, Ontario, Canada
October 23 October 24	PGCon Brazil 2009	Sao Paulo, Brazil
October 24 October 25	PyTexas	Fort Worth, TX, USA
October 24 October 25	FOSS.my 2009	Kuala Lumpur, Malaysia
October 24	Florida Linux Show 2009	Orlando, Florida, USA
October 24	LUG Radio Live	Wolverhampton, UK
October 25	Linux Outlaws and Ubuntu UK Podcast OggCamp	Wolverhampton, UK
October 26 October 28	Techno Forensics and Digital Investigations Conference	Gaithersburg, MD, USA
October 26 October 28	GitTogether '09	Mountain View, CA, USA
October 26 October 28	Pacific Northwest Software Quality Conference	Portland, OR, USA
October 27 October 30	Linux-Kongress 2009	Dresden, Germany
October 28 October 30	Hack.lu 2009	Luxembourg
October 28 October 30	no:sql(east).	Atlanta, USA
October 29	NLUUG autumn conference: The Open Web	Ede, The Netherlands
October 30 November 1	YAPC::Brasil 2009	Rio de Janeiro, Brazil
October 31	Linux theme day with ubuntu install party	Ede, Netherlands
November 1 November 6	23rd Large Installation System Administration Conference	Baltimore, MD, USA
November 2 November 6	ApacheCon 2009	Oakland, CA, USA
November 2 November 6	Ubuntu Open Week	Internet, Internet
November 3 November 6	OpenOffice.org Conference	Orvieto, Italy
November 4 November 5	Linux World NL	Utrecht, The Netherlands
November 5	Government Open Source Conference	Washington, DC, USA
November 6 November 8	WineConf 2009	Enschede, Netherlands
November 6 November 10	CHASE 2009	Lahore, Pakistan
November 6 November 7	PGDay.EU 2009	Paris, France
November 7 November 8	OpenFest 2009 - Biggest FOSS conference in Bulgaria	Sofia, Bulgaria
November 7 November 8	OpenRheinRuhr	Bottrop, Germany
November 7 November 8	Kiwi PyCon 2009	Christchurch, New Zealand
November 9 November 13	ACM CCS 2009	Chicago, IL, USA
November 10 November 11	Linux Foundation End User Summit	Jersey City, New Jersey
November 12 November 13	European Conference on Computer Network Defence	Milan, Italy
November 13 November 15	Free Society Conference and Nordic Summit	Göteborg, Sweden
November 14	pyArkansas	Conway, AR, USA
November 16 November 19	Web 2.0 Expo	New York, NY, USA
November 16 November 20	INTEROP	New York, NY, USA
November 16 November 20	Ubuntu Developer Summit for Lucid Lynx	Dallas, TX, USA
November 17 November 20	DeepSec IDSC	Vienna, Austria
November 19 November 22	Piksel 09	Bergen, Norway
November 19 November 21	Firebird Conference 2009	Munich, Germany
November 19 November 20	CONFIdence 2009	Warsaw, Poland
November 20 November 21	PostgreSQL Conference 2009 Japan	Tokyo, Japan
November 21	Baltic Perl Workshop 2009	Riga, Latvia
November 25 November 27	Open Source Developers Conference 2009	Brisbane, Australia
November 27 November 29	Ninux Day 2009	Rome, Italy
December 1 December 5	FOSS.IN/2009	Bangalore, India
December 4	Italian PostgreSQL Day 2009	Pisa, Tuscany, Italy
December 5 December 7	Fedora Users and Developers Conference	Toronto, Canada
December 7 December 11	Annual Computer Security Applications Conference	Honolulu, HI, USA
December 7 December 13	Make Art 2009	Poitiers, France
December 12 December 13	Django Development Sprint	Dallas, TX, USA
December 12 December 17	SciPy India 2009	Kerala, India
December 12	BSD community day	Utrecht, The Netherlands
December 19	New Mexico Linux Fest	Albuquerque, NM, USA

If your event does not appear here, please tell us about it.

Watch the 2009 Linux Plumbers Conference (Linux.com)

Linux.com has announced that videos of many of the presentations from last month's Linux Plumbers Conference are now available for viewing. "A lot of LinuxCon attendees stuck around for the Plumbers Conference, to delve deep into the kernel, utility, and library programming that surrounds Linux. This is deep-topic stuff, and attendees I spoke with were not disappointed. [...] Now you can view for yourself, as we've just posted the first batch of videos from the Plumbers Conference on the LF Video site. This set of videos includes discussions on SELinux, networking, clustered filesystems, video processing, and more."

Comments (38 posted)