LWN.net Logo

LWN.net Weekly Edition for December 4, 2008

MySQL 5.1 and development models

By Jonathan Corbet
December 3, 2008
The MySQL development team decided to celebrate the (US) Thanksgiving holiday with the release of MySQL 5.1.30, the first "general availability" (read "production-ready") release in the 5.1 series. There is a lot of good stuff in 5.1.30, including table partitioning, row-based replication, a new plugin API, a built-in job scheduler, and more; see the nutshell summary for more information. It's a celebration point for a long development series; the MySQL developers are to be congratulated for what they have accomplished with this release.

Behind the celebration, though, one can hear the grumbling from unhappy developers and users. This release has been a long time in coming; the first 5.0 GA release was in October, 2005 - just over three years ago. The first 5.1 release candidate (5.1.22) came out in September, 2007; seven more "release candidates," many with major changes, were announced over the following 14 months. So the 5.1 production release came rather later than desired, but some developers feel that it was still to soon; the complaints reached a climax in this lengthy posting from Michael "Monty" Widenius, the original creator of MySQL. His point of view, in short, is that this release has fatal bugs, and that these bugs come from a number of flaws in how MySQL development is managed.

Your editor cannot claim to be an expert on the MySQL development community. But Monty, presumably, is an expert on this community, so his observations have a higher than usual likelihood of reflecting something close to reality. Reading various dissenting posts (example) has done little to make your editor feel otherwise. And, in any case, much of what Monty says rings true when compared against experiences from elsewhere in the free software community. As projects grow, they must occasionally revisit their development models. There is little happening here which is truly unique to MySQL.

Monty asserts:

MySQL 5.1 was declared beta and RC way too early. The reason MySQL 5.1 was declared RC was not because we thought it was close to being GA, but because the MySQL manager in charge *wanted to get more people testing MySQL 5.1*. This didn't however help much, which is proved by the fact that it has taken us 14 months and 7 RC's before we could do the current "GA". This caused problems for developers as MySQL developers have not been able to do any larger changes in the source code since February 2006!

Two things jump out of that statement. One is that MySQL apparently suffers from an inadequate testing community. Needless to say, that is not a problem which is unique to this project; testing is a scarce resource throughout our community. MySQL users who are unhappy with the results of the development process might want to ask themselves if they are doing enough to help with the testing process. Like it or not, testing software and finding bugs is one of the costs of "free" (beer) software. If this testing doesn't happen during the development cycle, it will end up happening with the "stable" releases instead.

The other attention-getter above is the statement that MySQL developers have been unable to make major changes since early 2006. One need only think back to the 2.4 kernel days to see the kind of damage that can result from pent up "patch pressure." Developers get frustrated, major changes start to find their way into "release candidate" code, and the number of bugs tends to increase. The existence of a separate MySQL 6 development branch helps, perhaps, in reducing patch pressure, but it can also only serve to distract developers from stabilizing current release candidates.

Related to this is another assertion:

Too many new developers without a thorough knowledge of the server have been put on the product trying to fix bugs. This in combined with a failing review process have introduced of a lot new bugs while trying to fix old bugs.

Review would appear to be a big part of the problem in general. It may well be that a failure of review has caused the introduction of new bugs with fixes. But one could argue that the problem is deeper than that: any code which failed to stabilize over fourteen months of release candidates should, almost certainly, never have been merged into the MySQL trunk to begin with. It seems that there are not enough eyeballs being applied to major new features before they go in.

Your editor has resisted the temptation to make comparisons with other relational database manager projects, but there is value in comparing this state of affairs with the review problems faced by PostgreSQL in recent years. An inability to get additions to PostgreSQL properly reviewed resulted in those additions not being merged. That, in turn, leads to delayed releases with fewer than the desired number of features, neither of which is particularly pleasing for users or developers. But, on the other hand, PostgreSQL does not appear to have the same kind of trouble stabilizing its major releases.

Perhaps the key point to take away from all of this, though, is here:

In addition, the MySQL current development model doesn't in practice allow the MySQL community to participate in the development of the MySQL server.

MySQL is very much a corporate-owned, corporate-driven project, and it has been for a long time. Decisions on what to include are made internally; there is little discussion of development decisions on the project's mailing lists. It is hard to find information on how to contribute to the project; some of the available information still tells prospective contributors to use BitKeeper. All code is copyrighted by MySQL (now Sun), which reserves (and uses) a right to distribute that code under proprietary licenses.

All of the above reflects an arrangement which has worked well for years, and which has produced an immensely valuable database manager used by vast numbers of people. But it is not a community project, so development decisions will not necessarily reflect the best interests of the wider user or developer communities. If, as Monty suggests, those decisions are made in ways which favor features and deadlines over quality, there will be little that the community can do about it.

Comments (11 posted)

KSM runs into patent trouble

By Jake Edge
December 3, 2008

On the kernel page a few weeks ago, we took a look at KSM, a technique to reduce memory usage by sharing identical pages. Currently proposed for inclusion in the mainline kernel, KSM implements a potentially useful—but not particularly new—mechanism. Unfortunately, before it can be examined on its technical merits, it may run afoul of what is essentially a political problem: software patents.

The basic idea behind KSM is to find memory pages that have the same contents, then arrange for one copy to be shared amongst the various users. The kernel does some of this already for things like shared libraries, but there are numerous ways for identical pages to get created that the kernel does not know about directly, thus cannot coalesce. Examples include initialized memory (at startup or in caches) from multiple copies of the same program and virtualized guests that are running the same operating system and application programs.

Unfortunately, as Dmitri Monakhov points out, the KSM technique appears to be patented by VMware. A patent for "Content-based, transparent sharing of memory units" was filed in July 2001 and granted in September 2004. The abstract seems to clearly cover the ideas behind KSM:

[...] The context, as opposed to merely the addresses or page numbers, of virtual memory pages that [are] accessible to one or more contexts are examined. If two or more context pages are identical, then their memory mappings are changed to point to a single, shared copy of the page in the hardware memory, thereby freeing the memory space taken up by the redundant copies. The shared copy is ten preferable [sic] marked copy-on-write. Sharing is preferably dynamic, whereby the presence of redundant copies of pages is preferably determined by hashing page contents and performing full content comparisons only when two or more pages hash to the same key.

It should be noted that the abstract has no legal bearing, that comes from the—always tortuously worded—claims, which can be seen at the link above. In this case, as far as can be determined, the claims and abstract are in close agreement.

The dates above are rather important because there is some "prior art" to consider, namely the mergemem patch first announced in March of 1998. It is substantially the same as the patented idea: it looks for identical "context pages", then changes the memory mappings to point to a single copy-on-write page. This would seem to be a clear example of the idea being implemented well before the patent was filed, so it should invalidate the patent. As with everything surrounding software patents, though, it isn't as easy as that.

In order to invalidate a patent, either a court must rule that way or the patent office must be convinced to re-examine it, then find that the prior art makes it invalid. Both of these methods take time and usually money and lawyers as well. Free software projects may have time, but the other two are typically out of reach. Alan Cox suggests that "perhaps the Linux Foundation and some of the patent busters could take a look at mergemem and re-examination". While that might eventually resolve the problem, it is a multi-year process at best.

The folks behind the KSM project are some of the kvm hackers from Qumranet—which is now part of Red Hat. It is certainly conceivable that VMware might consider kvm a competitor and try to use this patent as a "competitive" weapon. That concern is probably enough to keep KSM out of the mainline until the issue is resolved.

There is a much quicker resolution available should VMware wish to do so. Like IBM has done with the RCU patent, VMware could license its patent for use in GPL-licensed code. There is much to be gained by doing that, at least in terms of positive community relations, and there is little to be lost—unless VMware truly believes that the patent will stand up to scrutiny. Both VMware and its parent, EMC, are members of the Linux Foundation, so one could see a role for the foundation in helping to put that kind of agreement together.

The original mergemem idea did not make into the kernel, but the code is still available for those running Linux 2.2.9. It appears that it was not pushed very hard in the face of some security concerns—which will need to be addressed by KSM as well. Processes could create a page of memory with known contents then, after waiting for the checker process (or kernel thread) to run, see if memory usage has increased. Based on that information, one can determine if other processes have a page with identical values. It would seem rather difficult to exploit, but clearly does allow some information to leak.

It will come as no surprise to most LWN readers that software patents are an increasingly dense minefield that can derail free software projects. Unfortunately, it is the kind of problem that has no solution in the technical domain where such projects excel. The political arena is where any solution will have to come from, though there seems to be some hope that judicial opinions (like the Bilski decision) may limit the scope of the damage. It is a problem that we are likely to see more frequently until there is some kind of resolution.

Comments (43 posted)

A look at free software in Ecuador

December 3, 2008

This article was contributed by Marco Fioretti

I recently spoke at the Congress on Free Software and Democratization of Knowledge hosted in Quito by the Universidad Politecnica Salesiana of Ecuador. My general report about the conference and Free as in Freedom knowledge in that country is at the P2P Foundation blog: the trip, however, was also an excellent occasion to check out the most interesting Free Software projects currently taking place in Ecuador. It turns out that there is a lot of activity at the Government level to promote Free Software, and interesting news from some cool projects developed locally.

FOSS in the Government

A recent presidential decree mandates that most national Public Administrations migrate entirely to Free Software. Ing. Mario Albuja, head of the Subsecretariat for Information Technology of the Presidency of Ecuador, explained during the congress the reasons and the general guidelines of this initiative. Later on, I was able to get more details in a couple of meetings with the members of his staff. Among the most important things going on right now there are the studies and tests for a Government digital signatures application which runs on Gnu/Linux and a unified document management system for 45 central Public Administrations. There is also a field trial of the GPL hospital management software Care2X in the works.

The initial implementation of the digital signature project, which uses Free Software whenever possible, is based on keys and digital certificates stored on SafeNet iKey 2032 USB tokens from Entrust. The first official field test will take place in the next weeks, when President Correa himself will use one such key to sign a decree. The Certificate Authority infrastructure which will issue keys and certificates is the same implemented by Banco Central del Ecuador in November 2007.

The software application, instead, runs inside any browser. A PostgreSQL backend stores all the documents, together with administrative metadata, on a CentOS-based server. The decrees waiting for electronic signature are presented to the user via a simple Apache/PHP front-end. The actual digital signature happens through a Java applet which reads the encrypted key from the USB token thanks to libraries provided by Entrust.

Another big step in the process of freeing Ecuador institutions from proprietary software will be the formal ratification of OpenDocument 1.0 by the Ecuadorian Institute of Standards (INEN). Large-scale usage of this format for public documents should take off right after that, around mid-2009.

All the public officials I talked with really believe in the potential of Free Software for a developing country like Ecuador. This only makes more relevant, and worthy of careful consideration, a comment I got from them: there, they say, is no coordination or common vision among the developers of the several FOSS applications they need to deploy. This was no surprise, of course: people at the Subsecretariat understand how FOSS development works. Nevertheless, the fact that there is no unified, local, reliable source for support, with predictable, if not guaranteed, response times, is creating them more problems than they expected when they began. There may be quite a business opportunity here for local FOSS entrepreneurs.

Talking with hackers

Rafael Bonifaz told me what's new in the Elastix world. In case you never heard of it, Elastix is a specialized GNU/Linux distribution born and (mostly) developed in Ecuador. Its goal is to solve all the communication problems of organizations of any size. Elastix integrates in one easy to administer package all you need to have PBX, VoIP, email, instant messaging, fax and fax/email gateway through Asterisk, Hylafax, Postfix and Openfire for Jabber. You can manage all the PBX functions with a customized version of freepbx. Other tools developed by the Elastix team provide hardware detection, centralized automatic configuration of phones and billing support with a2billing.

Elastix is doing great in Ecuador: RTS and Aerolineas Galapagos (Aerogal), which are respectively one of the most important TV channels and one of the main domestic airlines in Ecuador, are using it. Namely, Aerogal is running its call center off Elastix, which is being deployed also in the Ministry of Public Health.

Rafael, who is the current coordinator of the Elastix Community, is also proud of the fact that Elastix is the only Gnu/Linux distribution for communications which has two manual books, totaling about five hundred pages, freely downloadable from the Internet: Elastix Without Tears [PDF] by Ben Sharif and Unified communications with Elastix [PDF] by Edgar Landivar. The second manual is still a beta version, currently available only in Spanish. There already is, however, a new mailing list devoted to coordinating all the translation efforts for this second book.

Still thanks to Rafael, after knowing about Elastix I met a local group of Java developers who have very recently begun developing a new, interesting content management system called Melenti. Adrian Cadena, member of the Melenti team, explained to me that he and his partners needed a GPL, friendly, easy to use and fast CMS that could scale well from personal web pages to corporate portals. Another must on their requirement list was ease of integration with enterprise software (Java or not) for ERP, CRM and SAP services. That's why, three months ago, after some unsatisfactory experiences with the popular Joomla CMS they started writing Melenti.

One of the main features of Melenti should be performance under high loads. Adrian said they are aiming for something able to handle hundreds of thousands of clicks per second, something which Joomla "simply could not handle, when we tried it". Melenti administrators, instead, would be able to configure load balancing without problems, thanks to an interface based on Jndi and other tools.

Melenti should run on any JEE infrastructure, from Websphere to JBoss, BEA, Oracle AS, Tomcat, Jetty and more. According to Adrian, Melenti will also be much simpler to set up and extend than most other GPL software for Content Management. Installation should be as simple as dropping a .war file into your flavor of JEE container and following the steps of the graphical wizard which will pop up. Writing Melenti "gadgets", that is plugins, should also be easier than with Joomla, Drupal, Php-nuke and similar products. This because, says Adrian, "unlike those products, Java has worldwide standards like Spring, JPA, JSF, GWT and so on: new developers can just take a look at the core Melenti API and start writing their own gadgets in no time."

The first releases of Melenti will support basic CMS functions like management of web pages, images and other files. There will be also interfaces for banner rotation, creation of user polls and a Web Services Creator. The latter is a simple wizard to create Web Services from existing Melenti gadgets. The first alpha version of Melenti has been just uploaded to Sourceforge. You're obviously welcome to have a look at the code and to participate in the development of Melenti.

Let's go back to the reason why I went to Quito now, that is Free Software and Democratization of Knowledge. Quiliro Ordonez, with one friend and other occasional volunteers, is now implementing in the field a project first announced in 2007: placing Free Software in a school of the community of Quilapungo, south of Quito, which serves about 200 students. Thus far, Quiliro has installed 2 servers and 4 thin clients running gNewSense. He chose this distribution because it is "100% free software, without non-free repositories or blobs in the kernel which promote functionality before anything else, as this would weaken our position for freedom." He's also very happy with TCOS, which made setting up the thin clients a breeze. The school staff will use Projecto Alba, a modular administration and planning software for schools first developed in Argentina. While gNewSense worked fine out of the box, Quiliro and his partners had to localize Alba to adapt it to the terminology and procedures adapted in Ecuadorian schools.

Eventually, the school in Quilapungo will have about 40 Gnu/Linux workstations, but Quiliro doesn't plan to stop there. If all goes well, Quilapungo will be presented as a pilot project in a proposal for Free Software deployment in all public schools in Ecuador. Let's wish Quiliro good luck!

Comments (9 posted)

Page editor: Jake Edge

Security

System integrity in Linux

By Jake Edge
December 3, 2008

Ensuring that a Linux system is only running "approved" programs—ones that haven't been maliciously replaced—is one of the goals of the integrity patches currently being proposed for the Linux mainline. With some hardware assistance, in the form of a Trusted Platform Module (TPM) chip, systems will be able to protect against unauthorized binaries as well as attest to other systems that they are only running good code. These patches have been around for a number of years in various forms, but it would seem they are getting close to being merged. Perhaps more interestingly, we are starting to see them be used by various projects.

Over on the kernel page, we have looked at the integrity patches several times, most recently in March 2007. The core idea is to complement mandatory access control (MAC) systems, such as SELinux, by preventing attacks that are made when that system isn't running—the machine has been booted with a different kernel for example. It is generally considered a security truism that physical access to a device moots any security measures, but with a properly outfitted TPM-based system, that is no longer the case.

Conceptually, there are two parts to the integrity feature. One is the extended verification module (EVM) that associates each file with a hash that has been calculated over its contents and metadata. That hash is then signed by the TPM chip ensuring that unauthorized changes will be noticed. The other half is the integrity measurement architecture (IMA) which tracks the use of mmap(). IMA verifies the hashes of files that have been mapped in executable mode and then keeps track of them in a way that the TPM can sign. EVM then provides the protection against tampering with binaries, while IMA can provide a signed attestation of which executables have been run.

Previous incarnations of EVM and IMA used the Linux Security Modules (LSM) interface, but that has a very unfortunate side effect: inability to also run SELinux. LSM code has no way to stack or cooperate, so there can only be one module active at a time. Since integrity and MAC are intended to work together, this was seen as a rather serious impediment, so the most recent versions add in hooks for Linux Integrity Modules (LIM). IMA is then added as a LIM integrity provider rather than as an LSM.

In response to an Andrew Morton query about the need for LIM/IMA (EVM has been incorporated into IMA over time), David Safford listed several users of the code:

LIM/IMA's maintenance of a TPM hardware anchored file measurement list is fundamental to the Trusted Computing Group's standards efforts. Several projects have implemented the TNC (Trusted Network Connect) and PTS (Platform Trust Services) standards (see below). There are three demo packaged distros which have integrated these apps, two of which are government funded (EU and US), with definite customer interest. We are working with the RHEL team to provide a supported, patched kernel for HAP. All of these so far have used the old LSM based IMA, and have asked for a supported, upstreamed implementation, with the ability to work with SELinux.

While that looks a bit like alphabet soup, there is a lot of useful information there (and in his links further down in the post linked above). The biggest news is the three distributions that are implementing "Trusted Computing". The High Assurance Platform (HAP) program is funded by the US National Security Agency (NSA), the folks who brought us SELinux, while the Open Trusted Computing project is funded by the European Commission.

While the security that can be provided by a Trusted Computing platform is useful for some installations, there are some potential pitfalls as well. Systems with TPM hardware can be configured to only run binaries that are signed by some external authority. If manufacturers were to enable that functionality, but only provide the key to "trusted" software companies, it would lead to a horrendous loss of freedom. This is why some have called it "Treacherous Computing".

There are numerous examples of systems that do not necessarily preserve physical security, but that one might want to ensure were running the proper code—voting and cash machines come quickly to mind. For those situations, as well as countless others, Trusted Computing will be a real boon. We just need to be vigilant so that hardware vendors (or, worse yet, governments) don't start restricting what we can run on our own machines.

Comments (9 posted)

New vulnerabilities

clamav: stack overflow

Package(s):clamav CVE #(s):
Created:December 3, 2008 Updated:December 3, 2008
Description:

From the ClamAV bugzilla entry:

There is a recursive stack overflow in clamav 0.93.3 and 0.94 (and probably older versions) in the jpeg parsing code. it scan[]s the jpeg file, and if there is a thumbnail, it'll scan that too. the thumbnail itself is just another jpeg file and the same jpeg scanning function gets called without checking any kind of recur[]sing limit. this can eas[i]ly lead to a recurisive stack overflow.

Alerts:
Ubuntu USN-684-1 2008-12-02

Comments (none posted)

cupsys: arbitrary code execution

Package(s):cupsys CVE #(s):CVE-2008-5286
Created:December 3, 2008 Updated:January 26, 2009
Description:

From the Debian advisory:

An integer overflow has been discovered in the image validation code of cupsys, the Common UNIX Printing System. An attacker could trigger this bug by supplying a malicious graphic that could lead to the execution of arbitrary code.

Alerts:
rPath rPSA-2008-0338-1 2008-12-19
CentOS CESA-2008:1028 2008-12-15
Red Hat RHSA-2008:1028-01 2008-12-15
Gentoo 200812-11 2008-12-10
Debian DSA-1677-1 2008-12-02
Mandriva MDVSA-2009:029 2009-01-24
Mandriva MDVSA-2009:028 2009-01-24
SuSE SUSE-SR:2009:002 2009-01-19
Ubuntu USN-707-1 2009-01-12

Comments (none posted)

flamethrower: temporary file vulnerability

Package(s):flamethrower CVE #(s):CVE-2008-5141
Created:December 2, 2008 Updated:December 3, 2008
Description: Flamethrower suffers from a temporary file vulnerability which can be exploited to create a denial of service situation.
Alerts:
Debian DSA-1676-1 2008-12-01

Comments (none posted)

jailer: denial of service via symlink

Package(s):jailer CVE #(s):CVE-2008-5139
Created:December 1, 2008 Updated:December 3, 2008
Description:

From the Debian advisory:

Javier Fernandez-Sanguino Pena discovered that updatejail, a component of the chroot maintenance tool Jailer, creates a predictable temporary file name, which may lead to local denial of service through a symlink attack.

Alerts:
Debian DSA-1674-1 2008-11-30

Comments (none posted)

kernel: denial of service

Package(s):linux, linux-source-2.6.15/22 CVE #(s):CVE-2007-5498
Created:November 27, 2008 Updated:December 3, 2008
Description: The kernel has a denial of service vulnerability. From the vulnerability database entry: The Xen hypervisor block backend driver for Linux kernel 2.6.18, when running on a 64-bit host with a 32-bit paravirtualized guest, allows local privileged users in the guest OS to cause a denial of service (host OS crash) via a request that specifies a large number of blocks.
Alerts:
Ubuntu USN-679-1 2008-11-27

Comments (none posted)

kernel: denial of service

Package(s):linux, linux-source-2.6.15/22 CVE #(s):CVE-2008-5025
Created:November 27, 2008 Updated:February 10, 2009
Description: The kernel has a denial of service vulnerability. From the vulnerability database: Stack-based buffer overflow in the hfs_cat_find_brec function in fs/hfs/catalog.c in the Linux kernel before 2.6.28-rc1 allows attackers to cause a denial of service (memory corruption or system crash) via an hfs filesystem image with an invalid catalog namelength field, a related issue to CVE-2008-4933.
Alerts:
CentOS CESA-2009:0014 2009-01-15
Mandriva MDVSA-2008:246 2008-12-29
Red Hat RHSA-2009:0264-01 2009-02-10
SuSE SUSE-SA:2009:008 2009-01-29
Debian DSA-1687-1 2008-12-15
Debian DSA-1681-1 2008-12-04
Ubuntu USN-679-1 2008-11-27
SuSE SUSE-SA:2009:004 2009-01-21
SuSE SUSE-SA:2009:003 2009-01-20
Red Hat RHSA-2009:0014-01 2009-01-14

Comments (none posted)

kernel: denial of service

Package(s):linux, linux-source-2.6.15/22 CVE #(s):CVE-2008-5033
Created:November 27, 2008 Updated:August 20, 2009
Description: The kernel has a denial of service vulnerability. From the vulnerability database entry: The chip_command function in drivers/media/video/tvaudio.c in the Linux kernel 2.6.25.x before 2.6.25.19, 2.6.26.x before 2.6.26.7, and 2.6.27.x before 2.6.27.3 allows attackers to cause a denial of service (NULL function pointer dereference and OOPS) via unknown vectors.
Alerts:
SuSE SUSE-SA:2009:045 2009-08-20
Mandriva MDVSA-2008:246 2008-12-29
Ubuntu USN-679-1 2008-11-27

Comments (none posted)

libsamplerate: buffer overflow

Package(s):libsamplerate CVE #(s):CVE-2008-5008
Created:December 2, 2008 Updated:December 5, 2008
Description: Versions of libsamplerate prior to 0.1.4 suffer from a buffer overflow which could be exploited (via a specially-crafted audio file) to execute arbitrary code.
Alerts:
Mandriva MDVSA-2008:238 2008-12-04
Gentoo 200812-05 2008-12-02

Comments (none posted)

mantis: multiple vulnerabilities

Package(s):mantis CVE #(s):CVE-2008-4687 CVE-2008-4688 CVE-2008-4689
Created:December 2, 2008 Updated:December 3, 2008
Description: The mantis bug tracking system has a few bugs of its own, including a failure to sanitize input parameters, information disclosure, and a failure to unset session cookies on logout.
Alerts:
Gentoo 200812-07 2008-12-02

Comments (none posted)

perl: privilege escalation

Package(s):perl CVE #(s):CVE-2008-5302 CVE-2008-5303 CVE-2005-0448 CVE-2004-0452
Created:December 3, 2008 Updated:June 14, 2010
Description:

From the Debian advisory:

Paul Szabo rediscovered a vulnerability in the File::Path::rmtree function of Perl. It was possible to exploit a race condition to create setuid binaries in a directory tree or remove arbitrary files when a process is deleting this tree. This issue was originally known as CVE-2005-0448 and CVE-2004-0452, which were addressed by DSA-696-1 and DSA-620-1. Unfortunately, they were reintroduced later.

Alerts:
Mandriva MDVSA-2010:116 2010-06-11
CentOS CESA-2010:0458 2010-06-12
Red Hat RHSA-2010:0458-02 2010-06-07
SuSE SUSE-SR:2009:004 2009-02-17
Ubuntu USN-700-1 2008-12-24
Debian DSA-1678-2 2008-12-21
rPath rPSA-2009-0011-1 2009-01-20
Debian DSA-1678-1 2008-12-03
Ubuntu USN-700-2 2009-01-15

Comments (none posted)

phpMyAdmin: cross-site scripting

Package(s):phpmyadmin CVE #(s):CVE-2008-4326
Created:December 1, 2008 Updated:February 2, 2009
Description:

From the Debian advisory:

Masako Oono discovered that phpMyAdmin, a web-based administration interface for MySQL, insufficiently sanitises input allowing a remote attacker to gather sensitive data through cross site scripting, provided that the user uses the Internet Explorer web browser.

Alerts:
Debian DSA-1675-1 2008-11-30
SuSE SUSE-SR:2009:003 2009-02-02

Comments (none posted)

samba: bounds checking problem

Package(s):samba CVE #(s):CVE-2008-4314
Created:November 27, 2008 Updated:October 5, 2009
Description: Samba has a bounds checking issue. From the Ubuntu alert: It was discovered that Samba did not properly perform bounds checking in certain operations. A remote attacker could possibly exploit this to read arbitrary memory contents of the smb process, which could contain sensitive information or possibly have other impacts, such as a denial of service.
Alerts:
Fedora FEDORA-2009-10172 2009-10-03
Gentoo 200903-07 2009-03-07
SuSE SUSE-SR:2008:027 2008-12-09
Fedora FEDORA-2008-10612 2008-12-02
Fedora FEDORA-2008-10518 2008-12-02
Fedora FEDORA-2008-10638 2008-12-02
Slackware SSA:2008-333-01 2008-12-01
Ubuntu USN-680-1 2008-11-27
rPath rPSA-2009-0006-1 2009-01-13
SuSE SUSE-SR:2009:001 2009-01-12
Fedora FEDORA-2009-0268 2009-01-07

Comments (none posted)

wordpress: cross-site scripting

Package(s):wordpress CVE #(s):CVE-2008-5278
Created:December 3, 2008 Updated:December 22, 2008
Description:

From the Red Hat bugzilla entry:

Cross-site scripting (XSS) vulnerability in the self_link function in in the RSS Feed Generator (wp-includes/feed.php) for WordPress before 2.6.5 allows remote attackers to inject arbitrary web script or HTML via the Host header (HTTP_HOST variable).

Alerts:
Fedora FEDORA-2008-11104 2008-12-22
Fedora FEDORA-2008-10482 2008-12-03
Fedora FEDORA-2008-10483 2008-12-03
Fedora FEDORA-2008-10468 2008-12-03

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

The current 2.6 development kernel is 2.6.28-rc7, released on December 1. Linus says: "I was gone for a week, and it wasn't quite as quiet as I was hoping for, but there's a new -rc out there now with the merges of the fallout." Along with the usual fixes, 2.6.28-rc7 includes a new set of resource limits intended to prevent excessive kernel memory usage via the epoll_wait() system call.

Details, as usual, can be found in the long-format changelog.

There have been no stable kernel updates over the last week. The 2.6.27.8 update is in the review process as of this writing; this 104-patch monster can be expected sometime on or after December 5.

Comments (none posted)

Kernel development news

Quotes of the week

Too many people seem to think that documentation is the "final" argument. It's not. Not even close. It's a hint and a help, but it's _secondary_ to code. Anybody who doesn't understand that should never be allowed to write code (or documentation, for that matter).
-- Linus Torvalds

Doing

	unsigned i;

is an act of insane vandalism, punishable by spending five additional years coding in fortran.

-- Andrew Morton

Meanwhile, 10 years and counting, the Linux kernel still generates a stupid write IO for every file read that apps do. Fortunately hardware designers will get rid of rotating disks faster than we can fix our glaring process problems in this space - but it's still a bit sad.
-- Ingo Molnar

Comments (none posted)

Sam Leffler releases HAL source

Sam Leffler, maintainer of the Atheros hardware abstraction layer (HAL), has released the source under an ISC license. Previously, this piece of the MadWifi driver for Atheros wireless chipsets was only available in binary form. "In his announcement, Sam states: Coincident with the release of this code I have concluded my agreement with Atheros whereby I had access to information about their devices. This means that in the future all fixes, updates for new chips, etc. will need to be a community effort. Atheros states the Linux platform will be the reference public code base so folks wanting to add support for other platforms will have to scrape the information from there."

Comments (11 posted)

Tux3: the other next-generation filesystem

By Jonathan Corbet
December 2, 2008
There is a great deal of activity around Linux filesystems currently. Of the many ongoing efforts, two receive the most attention: ext4, the extension of ext3 expected to keep that filesystem design going for a few more years, and btrfs, which is seen by many as the long-term filesystem of the future. But there is another project out there which is moving quickly and is worth a look: Daniel Phillips's Tux3 filesystem.

Daniel is not a newcomer to filesystem development. His Tux2 filesystem was announced in 2000; it attracted a fair amount of interest until it turned out that Network Appliance, Inc. held patents on a number of techniques used in Tux2. There was some talk of filing for defensive patents, and Jeff Merkey popped up for long enough to claim to have hired a patent attorney to help with the situation. What really happened is that Tux2 simply faded from view. Tux3 is built on some of the same ideas as Tux2, but many of those ideas have evolved over the eight intervening years. The new filesystem, one hopes, has changed enough to avoid the attention of NetApp, which has shown a willingness to use software patents to defend its filesystem turf.

Like any self-respecting contemporary filesystem, Tux3 is based on B-trees. The inode table is such a tree; each file stored within is also a B-tree of blocks. Blocks are mapped using extents, of course - another obligatory feature for new filesystems. Most of the expected features are present. In many ways, Tux3 looks like yet another POSIX-style filesystem, but there are some interesting differences.

Tux3 implements transactions through a forward-logging mechanism. A set of changes to the filesystem will be batched together into a "phase," which is then written to the journal. Once the phase is committed to the journal, the transaction is considered to be safely completed. At some future time, the filesystem code will "roll up" the journal changes and write them back to the static version of the filesystem.

The logging implementation is interesting. Tux3 uses a variant of the copy-on-write mechanism employed by Btrfs; it will not allow any filesystem block to be overwritten in place. So writing to a block within a file will cause a new block to be allocated, with the new data written there. That, in turn, will require that the filesystem data structure which maps file-logical blocks to physical blocks (the extent) will need to be changed to reflect the new block location. Tux3 handles this by writing the new blocks directly to their final location, then putting a "promise" to update the metadata block into the log. At roll-up time, that promise will be fulfilled through the allocation of a new block and, if necessary, the logging of a promise to change the next-higher block in the tree. In this way, changes to files propagate up through the filesystem one step at a time, without the need to make a recursive, all-at-once change.

The end result is that the results of a specific change can remain in the log for some time. In Tux3, the log can be thought of as an integral part of the filesystem's metadata. This is true to the point that Tux3 doesn't even bother to roll up the log when the filesystem is unmounted; it just initializes its state from the log when the next mount happens. Among other things, Daniel says, this approach ensures that the journal recovery code will be well-tested and robust - it will be exercised at every filesystem mount.

In most filesystems, on-disk inodes are fixed-size objects. In Tux3, instead, their size will be variable. Inodes are essentially containers for attributes; in Tux3, normal filesystem data and extended attributes are treated in almost the same way. So an inode with more attributes will be larger. Extended attributes are compressed through the use of an "atom table" which remaps attribute names onto small integers. Filesystems with extended attributes tend to have large numbers of files using attributes with a small number of names, so the space savings across an entire filesystem could be significant.

Also counted among a file's attributes are the blocks where the data is stored. The Tux3 design envisions a number of different ways in which file blocks can be tracked. A B-tree of extents is a common solution to this problem, but its benefits are generally seen with larger files. For smaller files - still the majority of files on a typical Linux system - data can be stored either directly in the inode or at the other end of a simple block pointer. Those representations are more compact for small files, and they provide quicker data access as well. For the moment, though, only extents are implemented.

Another interesting - but unimplemented - idea for Tux3 is the concept of versioned pointers. The btrfs filesystem implements snapshots by retaining a copy of the entire filesystem tree; one of these copies exists for every snapshot. The copy-on-write mechanism in btrfs ensures that those snapshots share data which has not been changed, so it is not as bad as it sounds. Tux3 plans to take a different approach to the problem; it will keep a single copy of the filesystem tree, but keep track of different versions of blocks (or extents, really) within that tree. So the versioning information is stored in the leaves of the tree, rather than at the top. But the versioned extents idea has been deferred for now, in favor of getting a working filesystem together.

Also removed from the initial feature list is support for subvolumes. This feature initially seemed like an easy thing to do, but interaction with fsync() proved hard. So Daniel finally concluded that volume management was best left to volume managers and dropped the subvolume feature from Tux3.

One feature which has never been on the list is checksumming of data. Daniel once commented:

Having been checksumming filesystem data during continuous replication for two years now on multiple machines, and having caught exactly zero blocks of bad data passed as good in that time, I consider the spectre of disks passing bad data as good to be largely vendor FUD. That said, checksumming will likely appear in the feature list at some point, I just consider it a decoration, not an essential feature.

Tux3 development is far from the point where the developers can worry about "decorations"; it remains, at this point, an embryonic project being pushed by a developer with a bit of a reputation for bright ideas which never quite reach completion. The code, thus far, has been developed in user space using FUSE. There is, however, an in-kernel version which is now ready for further development. According to Daniel:

The functionality we have today is roughly like a buggy Ext2 with missing features. While it is very definitely not something you want to store your files on, this undeniably is Tux3 and demonstrates a lot of new design elements that I have described in some detail over the last few months. The variable length inodes, the attribute packing, the btree design, the compact extent encoding and deduplication of extended attribute names are all working out really well.

The potential user community for a stripped-down ext2 with bugs is likely to be relatively small. But the Tux3 design just might have enough to offer to make it a contender eventually.

First, though, there are a few little problems to solve. At the top of the list, arguably, is the complete lack of locking - locking being the rocks upon which other filesystem projects have run badly aground. The code needs some cleanups - little problems like the almost complete lack of comments and the use of macros as formal function parameters are likely to raise red flags on wider review. Work on an fsck utility does not appear to have begun. There has been no real benchmarking work done; it will be interesting to see how Daniel can manage the "never overwrite a block" policy in a way which does not fragment files (and thus hurt performance) over time. And so on.

That said, a lot of these problems could end up being resolved rather quickly. Daniel has put the code out there and appears to have attracted an energetic (if small) community of contributors. Tux3 represents the core of a new filesystem with some interesting ideas. Code comments may be scarce, but Daniel - never known as a tight-lipped developer - has posted a wealth of information which can be found in the Tux3 mailing list archives. Potential contributors should be aware of Daniel's licensing scheme - GPLv3 with a reserved unilateral right to relicense the code to anything else - but developers who are comfortable with that are likely to find an interesting and fast-moving project to play in.

Comments (53 posted)

Debugfs and the making of a stable ABI

By Jonathan Corbet
December 3, 2008
Remi Colinet recently proposed the addition of a new virtual file, /proc/mempool, which would display the usage of memory pools within the kernel. Nobody really disagreed with the idea of making this information available, but there were some grumbles about putting it into /proc. Once upon a time, just about anything could go into that directory, but, in recent years, there has been a real attempt to confine /proc to its original intent: providing information about processes. /proc/mempool is not about processes, so it was considered procfile-non-grata. It was suggested that another home should be found for this file.

Where that other home should be is not obvious, though. Somewhere like /sys/kernel might seem to make sense, but sysfs has rules of its own. In particular, the one-value-per-file rule makes it hard to create an easy file where developers can simply query the state of a kernel subsystem, so sysfs is not a suitable home for this file either.

The next option is debugfs, which was created in December, 2004. Debugfs is meant to be an aid for kernel developers; it explicitly disclaims any rules on the types of files that can be put there. All rules except for one: debugfs is not a mandatory part of any kernel installation, and nothing found therein should be considered to be a part of the stable user-space ABI. It is, instead, a dumping ground where kernel developers can quickly export information which is useful to them.

Since debugfs is not a part of the user-space ABI, it seems like a poor place to put things that users might depend on. When this was pointed out, it became clear that the non-ABI status of debugfs is not as well established as one might think. Quoting Matt Mackall:

The problem with debugfs is that it claims to not be an ABI but it is lying. Distributions ship tools that depend on portions of debugfs. And they also ship debugfs in their kernel. So it is effectively the same as /proc, except with the 1.0-era everything-goes attitude rather than the 2.6-era we-should-really-think-about-this one.

Pushing stuff from procfs to debugfs is thus just setting us up for pain down the road. Don't do it. In five years, we'll discover we can't turn debugfs off or even clean it up because too much relies on it.

As an example, Matt pointed out the extensively-documented usbmon interface which provides a great deal of information about what's happening on a USB bus. If it is not an ABI, he says, nobody should be upset if he submits a patch which breaks it.

That is a perennial problem with interfaces between the kernel and user space; changing them causes pain for users. That is why incompatible changes to user-space interfaces are almost never allowed; an important goal for the kernel development process is to avoid breaking user-space programs. One might think that this problem could be avoided for a specific interface by explicitly documenting it as an unstable interface. The files in Documentation/ABI/testing are meant to serve that role; anything found there should be considered to be unstable. But, as soon as people start using programs which depend on a specific interface, it has, for all practical purposes, hardened into part of the kernel ABI.

Linus put it this way:

The fact that something is documented (whether correctly or not) has absolutely _zero_ impact on anything at all. What makes something an ABI is that it's useful and available. The only way something isn't an ABI is by _explicitly_ making sure that it's not available even by mistake in a stable form for binary use.

Example: kernel internal data structures and function calls. We make sure that you simply _cannot_ make a binary that works across kernel versions. That is the only way for an ABI to not form.

So a given kernel interface can be kept away from ABI status if it is so hard to get to, and so unstable, that nothing ever comes to depend on it. The kernel module interface certainly fits this bill. Modules must generally be built for the exact kernel they are intended to work with, and they must often be built with the same configuration options and the same compiler. Anybody who has gotten into the dark business of distributing binary-only modules has learned what a challenge it can be.

Debugfs is different, though. It is enabled in a number of distributor kernels, even if, perhaps, it is not mounted by default. Once a set of files gets placed there, their format tends to change rarely. So it is possible for people to write programs which depend on debugfs files. And the end result of that is that debugfs files can become part of the stable kernel ABI. That is generally not a result that was intended by anybody involved, but it happens anyway. The only way to avoid it would be to deliberately shake up debugfs every kernel cycle - and few developers have much desire to do that.

This is a discussion without a whole lot in the way of useful conclusions; it leaves /proc/mempool without a home. ABI design, it turns out, is still hard. In the longer term, dealing with an ABI which was never really designed, but which just sort of settled into being, is even harder. There does not appear to be any substitute for thinking seriously about every interface between kernel and user space, even if it's just for a developer's debugging tool.

Comments (8 posted)

Variations on fair I/O schedulers

December 3, 2008

This article was contributed by Goldwyn Rodrigues

An I/O scheduler is a subsystem of the kernel which schedules I/O operations to the various storage devices to get the best possible throughput from those devices. The algorithm is often reminiscent of the algorithm used by elevators when dealing with requests coming from different floors to go up or down. This is the reason I/O scheduling algorithms are also called "elevators." I/O requests are submitted in an order designed to minimize disk head movement (thus minimizing disk seek times), yet guaranteeing good I/O rates. The next request chosen will be dependent on the current disk head position, in order to service the requests quickly, and spend less time seeking, or moving the disk head. However, algorithms may also consider other aspects such as fairness or time guarantees.

The Completely Fair Queuing (CFQ) I/O scheduler, is one of the most popular I/O scheduling algorithms; it is used as the default scheduler in most distributions. As the name suggests, the CFQ scheduler tries to maintain fairness in its distribution of bandwidth to processes, and yet does not compromise much on the throughput. The elevator's fairness is accomplished by servicing all processes and not penalizing those which have requests far from the current disk head position. It grants a time slice to every process; once the task has consumed its slice, this slice is recomputed and task is added to the end of the queue. The I/O priority is used to compute the time slice granted and the offset in the request queue.

The Budget Fair Queuing scheduler

The time-based allocation of the disk service in CFQ, while having the desirable effect of implicitly charging each application for the seek time it incurs, still suffers from fairness problems, especially towards processes which make the best possible use of the disk bandwidth. If the same time slice is assigned to two processes, they may each get different throughput, as a function of the positions on the disk of their requests. Moreover, due to its round robin policy, CFQ is characterized by an O(N) worst-case delay (jitter) in request completion time, where N is the number of tasks competing for the disk.

The Budget Fair Queuing (BFQ) scheduler, developed by Fabio Checconi and Paolo Valente, changes the CFQ round-robin scheduling policy based on time slices into a fair queuing policy based on sector budgets. Each task is assigned a budget measured in number of sectors instead of amount of time, and budgets are scheduled using a slightly modified version of the Worst-case Fair Weighted Fair Queuing+ (WF2Q+) algorithm (described in this paper [compressed PS]), which guarantees a worst case complexity of O(logN) and boils down to O(1) in most cases. The budget assigned to each task varies over time as a function of its behavior. However, one can set the maximum value of the budget that BFQ can assign to any task.

BFQ can provide strong guarantees on bandwidth distribution because the assigned budgets are measured sectors. There are limits, though: processes spending too much time to exhaust their budget are penalized and the scheduler selects the next process to dispatch I/O. The next budget is calculated on the feedback provided by the request serviced.

BFQ also introduces I/O scheduling within control groups. Queues are collected into a tree of groups, and there is a distinct B-WF2Q+ scheduler on each non-leaf node. Leaf nodes are request queues as in the non-hierarchical case. BFQ supports I/O priority classes at each hierarchy level, enforcing a strict priority ordering among classes. This means that idle queues or groups are served only if there are no best effort queues or groups in the same control group, and best effort queues and groups are served only if there are no real-time queues or groups. As compared to cfq-cgroups (explained later), it lacks per device priorities. The developers however claim that this feature can be incorporated easily.

Algorithm

Requests coming to an I/O scheduler fall into two categories, synchronous and asynchronous. Synchronous requests are those for which the application must wait before continuing to send further requests - typically read requests. On the other hand, asynchronous requests - typically writes - do not block the application's progress while they are executed. In BFQ, as in CFQ, synchronous requests are collected in per-task queues, while asynchronous requests are collected in per-device (or, in the case of hierarchical scheduling, per group) queues.

When the underlying device driver asks for the next request to serve and there is no queue being served, BFQ uses B-WF2Q+, a modified version of WF2Q+, to choose a queue. It then selects the first request from that queue in C-LOOK order and returns it to the driver. C-LOOK is a disk scheduling algorithm, where the next request picked is the one with the immediate next highest disk sector to the current position of the disk head. Once the disk has serviced the maximum sector number in the request queue, it positions the head to the sector number of the request having the lowest sector number.

When a new queue is selected it is assigned a budget, in disk sector units, decremented each time a request from the same queue is served. When the device driver asks for new requests and there is a queue under service, they are chosen from that queue until one of the following conditions is met: (1) the queue exhausts its budget, (2) the queue is spending too much time to consume its budget, or (3) the queue has no more requests to serve

On termination of a request, the scheduler recalculates the budget allocated to each process depending on the feedback it gets. For example, for greedy processes which have exhausted their budgets, the budget is increased, whereas if it has been idle for long, its budget is decreased. The maximum budget a process can get is a configurable system parameter (max_budget). Two other parameters, timeout_sync and timeout_async, control the timeout time for consuming the budget of the synchronous and asynchronous queues respectively. In addition, max_budget_async_rq limits the maximum number of requests serviced from an asynchronous queue.

If a synchronous queue has no more requests to serve, but it has some budget left, the scheduler idles (i.e., it tells to the device driver that it has no requests to serve even if there are other active queues) for a short period, in anticipation of a new request from the task owning the queue.

Test Results

The developers compared six different I/O scheduling algorithms: BFQ, YFQ, SCAN-EDF, CFQ, the Linux anticipatory scheduler, and C-LOOK. They compared a multitude of test scenarios analogous to real-life scenarios, including throughput, bandwidth distribution, latency, and short-term time guarantees. With respect to bandwidth distribution, BFQ can be concluded as the best, and a good algorithm for most scenarios. There were also extensive tests comparing BFQ against CFQ, and the results are available here. The throughput of BFQ is more or less the same as CFQ, but it scores well in distributing I/O bandwidth fairly among the processes, and displays lower latency with streaming data.

Using sector budgets instead of time as a factor of granting slice for fair bandwidth distribution is an interesting concept. The algorithm also employs timeouts to terminate requests of "seeky" processes taking too much time to consume their budget and penalizes them. The feedback from current requests help determine future budgets, making the algorithm self-learning. Such tighter bandwidths distribution would be a requirement for systems running virtual machines, or container classes. However, it depends on how BFQ stands the test of time against the tried-and-tested stable CFQ.

See the BFQ technical report [PDF] for (much) more information.

Expanded CFQ

Control Groups provide a mechanism for aggregating sets of tasks, and all their future children, into hierarchical groups. These groups can be allocated dedicated portions of the available resources, or resource sharing can be prioritized within these groups. Control groups are controlled by the cgroups pseudo-filesystem. Once mounted, the top level directory shows the complete set of existing control groups. Each directory made under the root filesystem makes a new group, and resources can be allocated to the tasks listed in the tasks file in the individual groups directory.

Control groups can be used to regulate access to CPU time, memory, and more. There are also several projects working toward the creation of I/O bandwidth controllers for control groups. One of those is the expanded CFQ scheduler patch for cgroups by Satoshi Uchida. This patch set introduces a new I/O scheduler called cfq-cgroups, which introduces cgroups for the I/O scheduling subsystem.

This scheduler, as the name suggests, is based on Completely Fair Queuing I/O scheduler. It can take advantage of hierarchical scheduling of processes, with respect to the cgroup they belong to, each cgroup having its own CFQ scheduler. I/O devices in a control group can be prioritized. The time slice given to each hierarchical group per device is a function of the device priority. This helps shaping of I/O bandwidth per group, per device.

Usage

To use, cfq-cgroups, select it as a default scheduler at boot by passing elevator=cfq-cgroups as a boot parameter. This can also be dynamically changed for individual devices by writing cfq-cgroups to /sys/block/<device>/queue/scheduler. There are two levels of control: through the cgroups filesystem, for individual groups, and through sysfs, for individual devices.

Like any other control group, cfq-cgroup is managed through the cgroup pseudo-filesystem. To access the cgroups, mount the pseudo cgroups filesystem:

    # mount -t cgroup -o cfq cfq /mnt/cgroup

The cgroup directory, by default, will have a file called cfq.ioprio, which contains the individual priority on a per-device basis. The time slice received per device per group is a function of the I/O priority listed in cfq.ioprio. The tasks file represents the list of tasks in the particular group. To make more groups, create a directory in the mounted cgroup directory:

    # mkdir /mnt/cgroup/group1

The new directories are automatically populated with files, cfq.ioprio, tasks etc, which are used to control the resources in this group. To add tasks in a group, write the process ID of the task to the tasks file:

    #echo <pid> > /mnt/cgroup/group1/tasks

The cfq.ioprio file contains the list of devices and their respective priorities. Each device in the cgroup has a default I/O priority of 3, while the valid values are 0 to 7. To change the priority of a device for the cgroup group1, run:

    # echo 2 > /mnt/cgroup/group1/cfq.ioprio

This would change the priority of the entire group. To change the I/O priority of a specific device:

    # echo 2 sda > /mnt/cgroup/group1/cfq.ioprio

To change the default priority while keeping the priority of the devices unchanged:

    # echo 4 defaults > /mnt/cgroup/group1/cfq.ioprio

The device view shows the list of cgroups and their respective priorities on a per-group basis. This can be changed by:

    # echo 2 group1 > /sys/block/sda/queue/iosched/ioprio

The device view contain other parameters similar to the CFQ scheduler, such as back_seek_max or back_seek_penalty, which are specific to the control of the individual device, same as the traditional CFQ.

Implementation

The patch introduces a new data structure called cfq_driver_data for the control of I/O bandwidth for cgroups. All driver-related data has been moved from the traditional cfq_data structure to cfq_driver_data structure. Similarly, cfq_cgroups is a new data structure to control the cgroup parameters. The organization of data can be assumed as a matrix with cfq_cgroups as rows and cfq_driver_data as columns, as shown in the diagram below.

[cfq-cgroup diagram]

At each intersection, there is a cfqd_data structure which is responsible for all CFQ related queue handling, so that each cfq_data corresponds to one cfq_cgroup and cfq_driver_data combination.

When a new cgroup is created, the cfq_data from the parent cgroup is copied into the new group. While inserting new nodes of cfq_data into the cgroup, the cfq_data structure is initialized with the priority of the cfq_cgroup. This way all data of the parent is inherited by the child cgroup, and shows up in the respective files per group in the cgroup filesystem.

Scheduling of cfq_data within the CFQ scheduler is similar to that of the native CFQ scheduler. Each node is assigned a time slice. This slice is calculated according to the I/O priority of the device, using the per-device base time slice. The time slice offset forms the key of the red-black node to be inserted in the service tree. One cfq_data entry is picked from the start of the red-black tree and scheduled. Once its time slice expires it is added to the tree again, after recalculation of its time slice offset. So, each cfq_data structure acts as a queue node per device, and, within each CFQ data structure, requests are queued as with a regular CFQ queue.

Both BFQ and cfq-cgroups are attempts to bring a higher degree of fairness to I/O scheduling, with "fairness" being tempered by the desire to add more administrative control via the control groups mechanism. They both appear to be useful solutions, but they must contend with the wealth of other I/O bandwidth control implementations out there. Coming to some sort of consensus on which approach is the right one could prove to be a rather longer process than simply implementing these algorithms in the first place.

Comments (6 posted)

Patches and updates

Kernel trees

Core kernel code

Development tools

Device drivers

Documentation

Filesystems and block I/O

Janitorial

Kernel building

Memory management

Networking

Architecture-specific

Security-related

Virtualization and containers

Benchmarks and bugs

Miscellaneous

Page editor: Jake Edge

Distributions

News and Editorials

Packaging qmail for Debian

By Jake Edge
December 3, 2008

An effort to get the qmail mail transfer agent (MTA) into Debian repositories has run aground due to various concerns, but the overriding one seems to be a distaste for qmail itself. Distributions make package availability decisions based on "taste" all the time, but they are generally made strictly on technical grounds, which does not seem to be the case here. While it has its share of detractors, qmail is a relatively popular MTA—with an excellent security track record—and one of the main impediments, its license, has changed in the last year. Because of that, it makes it a bit hard to understand why qmail would be kept out of Debian.

More than six months ago, Gerrit Pape had uploaded qmail and related packages to the ftp-master system, but they have yet to be added to the official Debian archive. He recently outlined his efforts in a post to debian-devel trying to see if he could break a kind of standoff between him and the ftpmasters, who are the folks that decide which packages get moved into the official archives. More than two months after his first upload of the packages, Pape got a reply from Joerg Jaspert outlining multiple technical reasons why the packages were being opposed, but also containing the following disheartening verdict:

Aside from these technical - and possibly fixable - problems, we (as in the ftpteam) have discussed the issue, and we are all of the opinion that qmail should die, and not receive support from Debian. As such we *STRONGLY* ask you to reconsider uploading those packages.

After that, Pape addressed some, but not all, of the technical complaints and uploaded updated packages along with a reply to Jaspert's rejection on September 1. Since that time, there has been no action on the packages nor any further communication from the ftpteam, which is what led to the debian-devel post. Responses there mostly backed the ftpmaster's "decision"; qmail, it seems, is not very popular with many Debian developers.

Unfortunately, some of the complaints are based on old or faulty information. There is a reasonably active upstream and, since Daniel J. Bernstein (aka djb) released the code into the public domain, there is no longer the need to patch qmail to get a sensible MTA. There are some legitimate concerns, in particular the backscatter that gets created by the default qmail configuration, but it is rather disingenuous to list security as one of those problems.

While not as bulletproof as djb would have it, qmail does have a long record of few security problems. In response to claims that the Debian security team would have more work because of qmail's inclusion, Moritz Muehlenhoff makes it clear that the team won't block qmail. Florian Weimer puts it this way:

Like Moritz, I don't see issues with security support, provided that the number of additional patches is rather small. (To my knowledge, badly patched qmail with a SMTP AUTH bypass vulnerability was one of the few MTAs which were actually exploited to send spam in recent times.) I'm also not sure if upstream can be considered dead, and arguments along that line are not very convincing because similar criticism could be brought against our default MTA.

I can understand that people have strong feelings. I'm willing to provide security support, but it's extremely unlikely that I'll run qmail on production MTAs ever again. 8-/

In the end, it comes down to emotions, largely. People generally feel strongly about qmail, either hating it or loving it, with few who know much about it anywhere in between. Clearly the ftpteam has the responsibility to reject packages on technical grounds, but are they the arbiters of taste for Debian as well?

An earlier thread about including qmail, from shortly after djb freed the code, showed a fair amount of interest in qmail, along with some opposition. It is unlikely that all Debian developers are happy with all of the packages currently supported by the distribution, so singling qmail out seems rather arbitrary. As Wouter Verhelst notes:

As long as qmail is free, packaged properly, and integrates well with the rest of Debian, I don't see why anyone should oppose its packaging.

Whether or not it's a good MTA, the fact is that it's a *popular* MTA. That alone should be a good reason to package it.

Installing qmail has always been painful; it is a package that cries out for distribution integration, which Pape is trying to provide. Whether it gets into the official repositories or not, unofficial qmail packages do exist. If the problems with qmail are largely packaging-related, it is hard to see how they will get fixed by staying unofficial. But if the problems are based on an emotional response to qmail itself—whether based in technical concerns or not—it is hard to see how a developer can overcome them.

Comments (20 posted)

New Releases

Fedora 10 Cambridge Spins : Other Planetary Systems

Along with Fedora 10, seven additional spins were released. They are: Fedora Electronic Lab, Fedora KDE, Fedora Edu-Math, Fedora XFCE, Fedora AOS, Fedora Developer, and Fedora BrOffice. "While astronomers took first ever pictures of other planetary systems last month, the fedora community was building its own planetary systems. Some say one planet isn't enough, other strive to enlarge their own planet. While F-10 Cambridge elevates the performance barrier and much more, other planetary spins have been geared to specific applications."

Full Story (comments: none)

FreeBSD 6.4-RELEASE Available

FreeBSD has announced the availability of FreeBSD 6.4, which is expected to be the last of the 6-STABLE releases. New features include a much-improved NFS lock manager, boot loader changes to allow booting from USB, KDE 3.5.10, GNOME 2.22.3, and more. Click below for the full announcement.

Full Story (comments: none)

Development Release: openSUSE 11.1 RC 1 Now Available

The first release candidate for openSUSE 11.1 is available now. "We all want openSUSE 11.1 to be the best release yet, and we need your help to get there. This release is ready for widespread testing, and we're encouraging everyone to download and work with the testing releases to find any critical bugs before release. Please run the release through your usual routine, and let us know about any bugs or other issues that you find. Remember that this is a release candidate, and is not suitable for use on production systems, however. "

Full Story (comments: none)

Distribution News

Debian GNU/Linux

Call for Talks for the Debian Developers' room at FOSDEM

Wouter Verhelst is looking for speakers for the Debian developers' room at FOSDEM, which will be held in Brussels, Belgium on February 7 and 8, 2009. "As has become a habit now over the years, I've again requested a developers' room for the Debian project, and have just received an acceptance of that request. This means we'll be able to hold talks at FOSDEM for one and a half day: during the afternoon on saturday, and the whole day on sunday. So, if you're interested in holding a talk at FOSDEM, now is the time. As usual, talks will be accepted on a first-come, first-serve basis; that is, of course, as long as they are suitable for a Debian venue."

Full Story (comments: none)

Fedora

Fedora 8 end-of-life is January 7

The Fedora Project has announced that support for the Fedora 8 distribution will end on January 7, 2009. That is actually a little later than the support policy allows for, but sticking to the policy would have ended support on Christmas day, which, perhaps, was not the sort of holiday gift Fedora users were looking for. So users of Fedora 8 have a little more than a month to upgrade to a newer release.

Full Story (comments: 3)

Fedora 11 naming process begins

The Fedora project is soliciting suggestions for the release name of Fedora 11. The name must have some relationship to "Cambridge", the release name for Fedora 10. "We're doing the name collection differently this year than in the past. Contributors wishing to make a suggestion are asked to go to the F11 naming wiki page, and add an entry to the suggestion table found there: https://fedoraproject.org/wiki/Name_suggestions_for_Fedora_11 The naming submissions are open starting now until Dec 8. The rest of the schedule is outlined on the wiki page."

Full Story (comments: none)

FUDCon F11 Boston

Paul Frields has an update on the Fedora User and Developer conference (FUDCon) F11 which will be held January 9-11, 2009 at MIT in Cambridge, MA. "And here's some further news to sweeten the pot -- the One Laptop Per Child and SugarLabs communities will be joining us for FUDCon, to address areas of common interest like packaging and building for these unique projects, and to talk to Fedora community members about getting involved. This should make FUDCon a very exciting event and I look forward to seeing everyone there who can make it!" Click below for his full message.

Full Story (comments: none)

Mandriva Linux

Losses at Mandriva

Two Mandriva contractors - Adam Williamson and Oden Eriksson have announced that their contracts are being terminated. These two developers are responsible for a great deal of the work which goes into the Mandriva distribution; as Oden notes: "Someone, or a couple of people will get their hands full (or not) maintaining the 1200+ source rpm packages I currently maintain. This is mostly server related stuff. For example the (L)AMP stack, to my knowledge the most complete on the planet, constantly growing and alive." The Mandriva community seems less than pleased; an online petition has been launched in an effort to get Mandriva to reconsider in Adam Williamson's case.

Comments (68 posted)

Red Hat Enterprise Linux

Performance tuning guides for RHEL

Two performance tuning guides for Red Hat Enterprise Linux 5 are available. One covers I/O tuning, looking at both performance analysis and tuning for the I/O subsystem. The other covers power management: "It discusses different techniques that lower power consumption (for both server and laptop), and how each technique affects the overall performance of your system." (thanks to Komal Shah).

Comments (none posted)

Slackware Linux

Slackware 12.2 is coming

The December 2 Slackware Changelog includes the following: "While there are still some docs that need updating and probably a few more tweaks and updates here and there, everything is mostly in place for the next stable release, Slackware 12.2. The versions of major components like X and the kernel can be considered frozen. Anything major that we haven't done yet will probably wait for the release after this next one." This is, they say, the first release candidate for Slackware 12.2.

Full Story (comments: none)

SUSE Linux and openSUSE

openSUSE Board Meeting Minutes, Nov19

The openSUSE board met on November 19, 2008. Items discussed were adding an advisory board, a board blog, as well as an openSUSE conference. Click below for the minutes of the meeting.

Full Story (comments: none)

New Distributions

Glendix

Glendix is a Plan9/Linux distribution. It combines Plan9 user space with the Linux kernel. This paper (PDF) contains more information. "We believe Plan 9 has a lot to offer in terms of features and functionality to the end-user. However, while the Plan 9 kernel is an excellent example of kernel design, it lacks in terms of device drivers. Plan 9 does not run on several commodity hardware, thereby severely reducing its adoption rate. Most people run Plan 9 in virtual machines, not on actual hardware. Linux, on the other hand has had years of work by thousands of developers put into it. It runs on significantly larger amounts of hardware than the Plan 9 kernel."

Comments (3 posted)

Distribution Newsletters

DistroWatch Weekly, Issue 280

The DistroWatch Weekly for December 2, 2008 is out. "This week's DWW feature story looks at the global economic crisis and its impact on Linux distributions and businesses. It turns out that the release of Fedora 10 isn't the only good news for Red Hat as a major Wall Street analyst sees a bright future for the company. Meanwhile Mandriva is struggling financially and a well-known and well-respected member of the Linux community was axed by the Paris-based company sparking a truly harsh reaction. Novell took a step to address community concerns by changing the openSUSE license. In other news, Phoronix published the results of benchmark tests comparing the performance of the newly released Fedora 10 with Ubuntu 8.10, a new distribution called Glendix brings Plan 9 to Linux, a detailed case study examines Pardus Linux, and the new graphical installer in VectorLinux 6.0 Beta 1 gets a close look. Finally, we get a progress update on FreeBSD 7.1 and some tantalizing hints about the release of Slackware 12.2."

Comments (none posted)

Debian Project News available

The December 2, 2008 issue of Debian Project News is available. It covers the "Etch-and-a-half" installation image update, the GNU Affero GPL is now suitable for Debian "main", the Security Team meeting in Essen, and more.

Comments (none posted)

Fedora Weekly News #154

The November 30, 2008 edition of Fedora Weekly News is available. "This week many of us enjoyed Thanksgiving turkey and we all enjoyed a full helping of Fedora 10 and were left stunned and satisfied. In Announcements the availability of third-party repositories and end-of-life of Fedora 8 are detailed. Developments catches up with 'Power Management and Filesystem Parameters' and a promising initiative to bring the man pages up-to-date. Artwork passes on some kudos for the 'Release Banner for the Website' and the demo of some awesome 'Stickers'. Don't forget to peruse the SecurityAdvisories!"

Full Story (comments: none)

openSUSE Weekly News, Issue 48

This issue of openSUSE Weekly News looks at openSUSE 11.1 RC 1, the YaST mascot winner, bashdb, MonoDevel, and the Linux kernel magic SysRq key. Click below for links to translations into multiple languages.

Full Story (comments: none)

Ubuntu Weekly Newsletter #119

The Ubuntu weekly newsletter for November 29 covers: "Ubuntu Free Culture Showcase II, Archive of Interviews, New MOTU: Onkar Shinde, Ubuntu Tamil Team, Ubuntu Tunisia Team, Ubuntu-NL release party, Launchpad 2.1.11 and Open ID Support, RSS feeds for Ubuntu Forums, Ubuntu Podcast #13, Full Circle Magazine #19, Community Interview of Nicolas Scerpella, Hardening the Linux Desktop, Coming Soon: Ubuntu Pocket Guide and Reference, 50 amazing Ubuntu time-saving tips, 50 of the best looking Gnome/Ubuntu desktop themes, Team meeting summaries, and much, much more!"

Full Story (comments: none)

Distribution reviews

What's new in Fedora 10 (Heise)

Heise has a detailed review of Fedora 10, written by Fedora contributor Thorsten Leemhuis. "Compared with other distributions, Cambridge uses a rather sparingly extended Linux version 2.6.27.5 kernel. The largest of the patches used in the Fedora kernel's source RPM provide support for KMS [kernel mode setting], squashfs and utrace as well as the Atl2, At76, Lirc and Nouveau drivers. In addition, the Fedora developers have updated the Ext4 support to largely match that of the main development branch of Linux which is currently due to produce the 2.6.28 kernel; with this kernel version, the kernel hackers will complete the file system's main development phase."

Comments (none posted)

Fedora 10 Review (Montana Linux)

Scott Dowdle at Montana Linux takes a look at Fedora 10. He installed F10 on multiple computers, trying both the GNOME and KDE desktops. "Fedora 10 has a lot of impressive features and a large library of software. The install is very easy and fast especially from the LiveCD. I believe there are still some compatibility issues with Xorg (that are probably not Linux distribution specific) on some newer hardware so the LiveCD option is great for pain free hardware compatibility testing. Fedora 10 doesn't offer a lot in the way of new major features but offers a lot of pretty impressive minor ones."

Comments (none posted)

Page editor: Rebecca Sobol

Development

Mercurial 1.1 - a major feature release

By Forrest Cook
December 2, 2008

The Mercurial project is described as: "a fast, lightweight Source Control Management system designed for efficient handling of very large distributed projects." The Major Features document presents an overview of Mercurial's capabilities and Understanding Mercurial explains how Mercurial works as a distributed source control system.

Mercurial version 1.1 was announced this week: "This is a major release with numerous new features."

[Mercurial]

The What's New document explains the many changes that were added to Mercurial 1.1. Highlights include a new resolve command for tracking in-progress merges, a new repository format, performance improvements, support for Python 2.6, bug fixes and work on the documentation. The web interface now has a canvas-based repository graph, new themes, improved WSGI compliance, support for the display of nested repositories and other improvements.

The Mercurial commands have gone through numerous improvements and extensions, some bugs have also been fixed. Some new extensions have been added to Mercurial 1.1, including a rebase extension for rebasing changesets, a bookmarks extension for providing git-like branches, a zeroconf extension for publishing repositories and an hgcia extension for communicating with CIA. Some of the existing extensions have undergone a variety of improvements. Version 1.2 of the mercurial plugin for the Eclipse IDE was also announced this week.

According to Wikipedia, Mercurial was started in 2005 and the software is being used by such high profile projects as Mozilla, OpenSolaris and Xen. This latest release shows that the code continues to undergo active development, and holds an important place in the world of source code control systems.

Comments (2 posted)

System Applications

Database Software

MySQL Server 5.1.30 has been released

Version 5.1.30 of MySQL Server has been announced. "We are proud to present to you the MySQL Server 5.1.30 GA release, the first 5.1 production version of the popular open source database. MySQL 5.1.30 is recommended for use on production systems. MySQL 5.1 provides a number of new enhancements including: - Table and index partitioning - Row-based and mixed replication - Built-in job scheduler - Improved XML handling with XPath support - New SQL diagnostic aids and performance utilities - The return of the embedded library (libmysqld)"

Full Story (comments: none)

Monty on MySQL 5.1

Here's an interesting, detailed posting from Michael "Monty" Widenius on the problematic MySQL 5.1 release. "So what went wrong with MySQL 5.1? This is surprisingly not because our developers don't do a good job. On the contrary we have an excellent dedicated team of developers that are very good in what they are doing. However, even an excellent team can't work if the conditions are not right."

Comments (13 posted)

phpMyAdmin: 3.1.0 is released (SourceForge)

Version 3.1.0 of phpMyAdmin has been announced. "phpMyAdmin is a tool written in PHP intended to handle the administration of MySQL over the Web. Currently it can create and drop databases, create/drop/alter tables, delete/edit/add fields, execute any SQL statement, manage keys on fields.Welcome to phpMyAdmin 3.1.0. This version has a new setup mechanism and supports BLOBstreaming and the Swekey hardware authentication."

Comments (none posted)

PostgreSQL Weekly News

The November 30, 2008 edition of the PostgreSQL Weekly News is online with the latest PostgreSQL DBMS articles and resources.

Full Story (comments: none)

SQLite release 3.6.6.2 announced

Version 3.6.6.2 of the SQLite DBMS has been announced, some bugs have been fixed.

Comments (none posted)

Embedded Systems

BusyBox 1.13.1 and 1.12.3 released

Versions 1.13.1 and 1.12.3 of BusyBox, a collection of command line utilities for embedded systems, have been announced. "Bug fix releases. 1.13.1 has fixes for ash, option parsing, id, init, inotifyd, klogd, line editing and modprobe. 1.12.3 has fixes for option parsing and line editing."

Comments (none posted)

Interoperability

Samba 3.2.5 is available

Stable version 3.2.5 of Samba has been announced, this is a security release. See the Change Log for details.

Comments (none posted)

Networking Tools

libnetfilter_conntrack 0.0.98 release

Version 0.0.98 of libnetfilter_conntrack has been announced. "libnetfilter_conntrack is a userspace library providing a programming interface (API) to the in-kernel connection tracking state table. This library requires a linux kernel >= 2.6.18. This release includes one major bugfix, a couple of minor fixes, the new attribute group API and cleanups."

Full Story (comments: none)

Telecom

Linux boots on the iPhone

The iPhone Linux hackers have announced that they have a rudimentary 2.6 kernel running on iPhone devices, and some iPod Touch devices as well. "This is a rough first draft of the port, and many drivers are still missing, but it's enough that a real alternative operating system is running on the iPhone." It looks like there is a lot of work to do for those who would like to help with this project.

Comments (22 posted)

Web Site Development

nginx 0.6.33 is available

Version 0.6.33 of nginx, a light weight web server, has been announced. Some new capabilities and bug fixes have been added, see the Change Log for details.

Comments (none posted)

Desktop Applications

Audio Applications

aTunes: 1.11.0 released (SourceForge)

Version 1.11.0 of aTunes has been announced. "aTunes is a powerful, full-featured, cross-platform player and manager, with audio cd rip frontend. Currently supported formats are mp3, ogg, wav, wma, flac, mp4, ape, mpc, mac, radio streaming and podcasts. The final version of 1.11.0 has been released. It's mainly focused on fixing bugs and improve current features."

Comments (none posted)

JACK 0.115.6 is available

Version 0.115.6 of the JACK Audio Connection Kit is available, the main new feature is support for D-Bus. "D-Bus modifications add optional autodetected support for the D-Bus based server control system. D-Bus is object model that provides IPC mechanism. D-Bus supports autoactivation of objects, thus making it simple and reliable to code a "single instance" application or daemon, and to launch applications and daemons on demand when their services are needed."

Full Story (comments: none)

SLV2 0.6.2 released

Version 0.6.2 of SLV2, a library for accessing LV2 audio plugins, has been announced. "This is a bugfix release. 0.6.1 failed to recognize any LV2 features. Upgrading is strongly recommended. Let us all just pretend that 0.6.1 never existed. 0.6.1 is not to be invited to any parties. It will sit alone for the holidays, cursing haste and drinking alone."

Full Story (comments: none)

Songbird 1.0 is available

Version 1.0 of Songbird has been announced. "We set out to build an open, customizable music player. Today, we’re launching with dozens of integrated services, hundreds of add-ons, and a growing developer community. We’ll be the first to admit that there’s plenty left to do. And, while we’re not ready for everyone, 160k users a month are expressing their vote for an alternative music player. If you think like us, you’ve got huge aspirations and you’re out to change the world. You gotta start somewhere, this is our 1.0."

Comments (none posted)

TimeMachine 0.3.2 released

Version 0.3.2 of TimeMachine has been announced, it includes some new features and bug fixes. "I used to always keep a minidisc recorder in my studio running in a mode where when you pressed record it wrote the last 10 seconds of audio to the disk and then caught up to realtime and kept recording. The recorder died and haven't been able to replace it, so this is a simple jack app to do the same job. It has the advantage that it never clips and can be wired to any part of the jack graph."

Full Story (comments: none)

Data Visualization

Veusz 1.2.1 announced

Version 1.2.1 of Veusz has been announced, it adds a bug fix. "Veusz is a scientific plotting package. It is written in Python, using PyQt4 for display and user-interfaces, and numpy for handling the numeric data. Veusz is designed to produce publication-ready Postscript/PDF output. The user interface aims to be simple, consistent and powerful. Veusz provides a GUI, command line, embedding and scripting interface (based on Python) to its plotting facilities. It also allows for manipulation and editing of datasets."

Full Story (comments: none)

Desktop Environments

GNOME 2.24.2 released

Version 2.24.2 of the GNOME desktop has been announced. "This is the second update to GNOME 2.24. It contains many fixes for important bugs that directly affect our users, documentation updates and also a large number of updated translations. Many thanks to all the contributors who worked hard on delivering those changes in time. We hope it will help people feel better in their daily use of computers!"

Full Story (comments: 31)

GNOME 2.25.2 released

Version 2.25.2 of the GNOME desktop environment has been announced. "Early Xmas present! 2.25.2 is here. This is the second development release towards our 2.26 release that will happen in March 2009. By now, development is well under way, and we've already made good progress on some of the goals that we've set ourselves for 2.26 ( http://live.gnome.org/GnomeGoals). Keep up the good work !"

Full Story (comments: none)

GNOME Software Announcements

The following new GNOME software has been announced this week: You can find more new GNOME software releases at gnomefiles.org.

Comments (none posted)

KDE 4.2 Beta1 Out for Testing (KDE.News)

KDE.News has announced the release of KDE 4.2 Beta1. "Today, the KDE team invites interested testers and reviewers to give KDE 4.2.0-Beta1 a go. The release announcement lists some significant improvements. The purpose of this release is to get feedback from the community, preferably in the form of bugreports on the new bugs.kde.org bugtracker."

Comments (none posted)

KDE Software Announcements

The following new KDE software has been announced this week: You can find more new KDE software releases at kde-apps.org.

Comments (none posted)

Xorg Software Announcements

The following new Xorg software has been announced this week: More information can be found on the X.Org Foundation wiki.

Comments (none posted)

Electronics

pcb-20081128 snapshot announced

Snapshot version 20081128 of PCB, a printed circuit CAD application, has been announced. A number of bugs have been fixed and other improvements have been made.

Comments (none posted)

Games

Cyphesis 0.5.18 released

Version 0.5.18 of Cyphesis has been announced. "Cyphesis is a small to medium scale server for WorldForge games, with builtin AI. This version includes the demo game Mason which is currently in development. This release is intended for server administrators wishing to run a Mason server and World developers developing new worlds or game systems. Major changes in this version: World data is now fully persistent across crashes and restarts. Lots of bug fixes and performance improvements."

Comments (none posted)

Mail Clients

Sylpheed 2.6.0rc announced

Version 2.6.0rc of the Sylpheed mail client has been announced. "2.6.0 final is scheduled for release in December. * The backups for configuration files are kept for four generations now. * Only new messages are counted when incorporating from local mbox. * In the folder selection dialog, only folders which are expanded in the folder view are expanded now..."

Comments (none posted)

Multimedia

Elisa Media Center 0.5.20 released

Version 0.5.20 of Elisa Media Center has been announced. "New features include a mechanism to trust external plugins' integrity when distributed through the plugin repository, and a better integration in Windows. As usual, a bunch of bugs were fixed".

Full Story (comments: none)

Music Applications

Calf Audio Plugin Pack 0.0.17 released

Version 0.0.17 of Calf Audio Plugin Pack has been announced, some new capabilities have been added. "Calf is a set of open source instruments and effects for digital audio workstations. Multiple plugin interfaces are supported: LV2, DSSI, JACK (standalone application) and, for slightly outdated hosts, LADSPA."

Full Story (comments: none)

rakarrack 0.3.0 is released

Version 0.3.0 of rakarrack, a guitar effects processor, has been announced. "This is a bug fix version with few improvements."

Full Story (comments: none)

PianoBooster version 0.5.0 has been released

Version 0.5.0 of PianoBooster has been announced. "PianoBooster version 0.5.0 has just been released, this is the first ever release of PianoBooster. If you dabble in music and have a Midi Piano keyboard lying around then give PianoBooster a go as it actually makes sight reading music fun!"

Full Story (comments: none)

Strasheela 0.9.7 released

Version 0.9.7 of Strasheela a constraint-based music composition system, has been announced. "This release improves Strasheela's export of sound synthesis formats. For example, expressive timing of scores can be defined by time shift functions. Predefined MIDI file export now supports also non-note events like continuous controllers (e.g., pitch bend, aftertouch, CC) and a global tempo curve. Additional examples have been added, and further extensions and bug fixes have been made as well."

Full Story (comments: none)

zynjacku version 2 released

Version 2 of zynjacku, a JACK-based, GTK 2.x host for LV2 synths, is out. "In this release: * zynjacku codebase updated to the latest state of LV2 art. Especially, event MIDI port synths are now supported. * The tarball now contains lv2rack - a host for LV2 effect plugins. * Saving/restoring of presets. Does not work for dynparam plugins yet. * LASH support (autodetected at runtime). * Lot of general fixes that should make zynjacku more usable."

Full Story (comments: none)

Office Applications

eGroupware 1.6 released

Version 1.6 of eGroupware, a calendar, todo and email sharing utility, has been announced. "The eGroupware project is proud to present the 4th major release since the foundation of the project in 2003." (Thanks to David Greaves).

Comments (none posted)

Office Suites

OpenOffice.org Newsletter

The November, 2008 edition of the OpenOffice.org Newsletter is out with the latest OO.o office suite articles and events.

Full Story (comments: none)

Science

HDF5 for Python 1.0 is out

Version 1.0 of HDF5 for Python has been announced. "HDF5 for Python (h5py) is a general-purpose Python interface to the Hierarchical Data Format library, version 5. HDF5 is a versatile, mature scientific software library designed for the fast, flexible storage of enormous amounts of data."

Full Story (comments: none)

Video Applications

KDE 4 Video Editor Kdenlive Released (KDE.News)

KDE.News has a quick look at the 0.7 release of Kdenlive, the first KDE 4 release of the non-linear video editor. "This closes another gap of the free desktop world: a usable open source video editor. Kdenlive has the potential to become the Amarok or K3b of video editors, offering comfort and elegance so far not available in alternative programs. The feature set looks amazingly complete and far exceeds the KDE 3 version already."

Comments (3 posted)

Miscellaneous

Exomind v0.2 is out

Version 0.2 of Exomind has been announced. "Exomind is an experimental Python console and programmatic framework for building decorated graphs and developing open-source intelligence modules and ideas, centered on social network services, search engines and instant messaging."

Full Story (comments: none)

Languages and Tools

C

GCC 4.4.0 Status Report

The November 27, 2008 edition of the GCC 4.4.0 Status Report has been published. "Trunk is in Stage 4 (regression and documentation fixes mode). GCC 4.4 will be branched when there are no open P1 regressions for 4.4 and the total number of P1, P2 and P3 regressions for 4.4 is 100 or below. Trunk will open for Stage 1 for GCC 4.5 immediately after 4.4 branches, and 4.4.0-rc1 will be made from the branch shortly after branching."

Full Story (comments: none)

Caml

Caml Weekly News

The December 2, 2008 edition of the Caml Weekly News is out with new articles about the Caml language.

Full Story (comments: none)

Haskell

Haskell Communities and Activities Report

The November, 2008 edition of the Haskell Communities and Activities Report is online with a new collection of Haskell articles. "This is the 15th edition of the Haskell Communities and Activities Report. There are interesting news on the implementation front, new analysis and transformation tools, many fresh projects, and new developments in established ones. Generic programming is one field that has seen a lot of activity, and you will no doubt identify other recent trends as you go through the report."

Comments (none posted)

JSP

Pyjamas 0.4 released

Version 0.4 of Pyjamas has been announced. "pyjamas is a stand-alone python to javascript compiler, an AJAX framework / library and a Widget set API."

Full Story (comments: none)

Perl

On the future of Perl 5

Here's an interesting discussion from the "Journal of Ovid" on the usePerl site on the (apparently) declining popularity of the Perl language. "When 5.12 hits we have a chance of winning programmers back, but the language has so much baggage -- both technical and social -- that I don't think it will win anyone back. We're losing because we didn't need marketing when we were the duct tape of the internet and now we sneer at it. We're losing because we rested on our laurels and scoffed at the up-and-comers. We're losing because older programmers ignored the concerns of the younger programmers. We're losing for the same reason that there are still people who object to images on Web pages, much less Flash and Javascript."

Comments (93 posted)

Python

lockfile 0.4 announced

Version 0.4 of lockfile (for Python) has been announced. "The lockfile module exports a FileLock class which provides a simple API for locking files. Unlike the Windows msvcrt.locking function, the Unix fcntl.flock, fcntl.lockf and the deprecated posixfile module, the API is identical across both Unix (including Linux and Mac) and Windows platforms. The lock mechanism relies on the atomic nature of the link (on Unix) and mkdir (On Windows) system calls."

Full Story (comments: 1)

PyAlsaAudio 0.4 released

Version 0.4 of PyAlsaAudio, an ALSA wrapper for Python, has been announced. "The list of changes is: - Support for Python 3.0 - API changes: mixers() and Mixer() now take a card index instead of a card name as optional parameter. - Documentation converted to reStructuredText; use Sphinx instead of LaTeX. - added cards() - added PCM.close() - added Mixer.close() - added mixer.getenum()"

Full Story (comments: none)

urllib3 0.2 announced

Version 0.2 of urllib3 has been announced. "What's wrong with urllib and urllib2? There are two critical features missing from the Python standard library: Connection re-using/pooling and file posting. It's not terribly hard to implement these yourself, but it's much easier to use a module that already did the work for you."

Full Story (comments: none)

Python-URL! - weekly Python news and links

The December 3, 2008 edition of the Python-URL! is online with a new collection of Python article links.

Full Story (comments: none)

Tcl/Tk

Tcl-URL! - weekly Tcl news and links

The November 26, 2008 edition of the Tcl-URL! is online with new Tcl/Tk articles and resources.

Full Story (comments: none)

Tcl-URL! - weekly Tcl news and links

The December 2, 2008 edition of the Tcl-URL! is online with new Tcl/Tk articles and resources.

Full Story (comments: none)

Page editor: Forrest Cook

Linux in the news

Recommended Reading

Linux Evolution Reveals Origins of Curious Mathematical Phenomenon (PhysOrg)

PhysOrg.com summarizes a scientific paper describing how investigators used the Debian package history to verify Zipf's Law. "Using the data, they showed that the growth rates of connectivities between packages are proportional to the degree of connectivity between packages. In addition, they showed empirically that the average growth rate of the total number of links to a given package over a time interval is proportional to that time interval. Further, the variability of the total number of links to a given package increases proportionally to the square-root of time, providing a crucial test of the mechanism of stochastic proportional growth of connectivity between packages. Altogether, these characteristics are responsible for the universal distribution pattern of Zipf's law."

Comments (6 posted)

Resources

StudioDave Does A Hardware Review And Meets Ubuntu 8.10 (Linux Journal)

Over at Linux Journal, Dave Phillips picks a laptop and distribution for his mobile audio needs with some fairly specific requirements. He looks at problems that he ran into along with solutions that he found. "I had specific intentions for this machine. Above all it had to run AVSynthesis, which meant that it would need accelerated 3D graphics capability along with support for high-quality realtime audio. I also wanted to install a complete environment for building a specific version of Csound 5.09 and for compiling Ardour 3 from its SVN source code, which meant that I would need a relatively up-to-date Linux distribution. Other required components included a recent version of JACK and the latest Java SDK."

Comments (none posted)

Reviews

Go fish! (developerWorks)

developerWorks examines the fish shell. "Thankfully, fish--the Friendly Interactive Shell--swims upstream against the tide of obfuscation, providing streamlined syntax and a much-improved user experience. Like other shells, fish provides redirection, shortcuts, globbing (that is, expansion of wildcards), subshells, tab completion, and variables. Unlike alternatives, however, fish also provides color-coded CLIs, an extensive command-line editor, and rich documentation."

Comments (15 posted)

Miscellaneous

Linux readied for DOD IPv6 certification (GCN)

Government Computer News reports on the recent IPv6 Special Interoperability Certification of the Linux kernel. "The Linux kernel has been brought into full compliance with the Defense Information System Agency's IPv6 Special Interoperability Certification, the Linux Foundation announced this week. As a result, all Linux distributions that use the latest kernel and enhancements should be able to pass certification. In 2007, the Defense Department's Office of the Chief Information Officer and the chief assistant secretary of Defense for networks and information integration developed an IPv6 Master Test Plan to ensure all DOD equipment using IPv6 would be able to interoperate."

Comments (3 posted)

Page editor: Forrest Cook

Announcements

Commercial announcements

New MySQL Enterprise with Query Analyzer boosts DBMS performance

Sun Microsystems has announced a new version of its MySQL Enterprise subscription service. "MySQL Enterprise subscribers now have access to Sun's new MySQL Query Analyzer tool for monitoring query performance to accurately pinpoint and correct problem SQL code -- in addition to MySQL Enterprise's regular software updates and 24x7 production technical support."

Comments (none posted)

Create your perfect island paradise with My Tribe!

Grubby Games has announced the availability of the game "My Tribe". "Grubby Games is pleased to announce "My Tribe", an island simulation game with near infinite replayability! Shipwrecked on a beautiful tropical island, your brave little tribespeople must build a new home. With your guidance, the tribe will uncover mysterious secrets, research new technologies, and construct massive wonders. Come back often to feed your tribe, clothe them, and look after the babies and children!"

Full Story (comments: none)

New Books

Enterprise Recipes with Ruby and Rails--New from Pragmatic Bookshelf

Pragmatic Bookshelf has published the book Enterprise Recipes with Ruby and Rails by Maik Schmidt.

Full Story (comments: none)

Resources

Linux Gazette #157

Issue #157 of the Linux Gazette has been published. "Linux Gazette is a volunteer-run monthly web magazine dedicated to two simple ideas: making Linux a little more fun, and sharing ideas and discoveries."

Full Story (comments: none)

RSS Feed for the Python Job Board

An online Python job board has been announced. "The Python.org job board ( http://www.python.org/community/jobs/) now has an RSS feed which can be found at http://www.python.org/community/jobs/jobs.rss which list the 20 most recent postings in summary form along with a link to the posting on the jobs page."

Full Story (comments: none)

Education and Certification

LPI expands training partner program

The Linux Professional Institute has announced an expansion of its training partner program. ""Over the past few years we have continued to evolve our training partner program resulting in a substantial increase in the number of countries where we have representation--this demonstrates both the growing adoption of Open Source Software and the need world-wide for vendor-neutral Linux training," said Jim Lacey, President and CEO of LPI. Mr. Lacey noted that due to this demand and increasing requests from employers, educational institutions and exam candidates for high quality training options LPI is adding new improvements and resources to the training partner program throughout 2009."

Full Story (comments: none)

Calls for Presentations

Call for papers: Open Source in Data Mining workshop

A call for papers has gone out for the Open Source in Data Mining workshop. The event takes place on April 27, 2009 in Bangkok, Thailand. Submissions are due by December 22.

Full Story (comments: none)

Upcoming Events

Events: December 11, 2008 to February 9, 2009

The following event listing is taken from the LWN.net Calendar.

Date(s)EventLocation
December 5
December 13
International Joint Conferences on Computer, Information, and Systems Sciences, and Engineering Online,
December 7
December 12
Computer Measurement Group Conference 2008 Las Vegas, NV, USA
December 8
December 12
Ubuntu Developer Summit Mountain View, CA, USA
December 10
December 11
First Workshop on I/O Virtualization San Diego, CA, USA
December 13 NLLGG meeting/BSD Community Day Utrecht, The Netherlands
December 27
December 30
Chaos Communication Congress Berlin, Germany
January 8
January 11
Consumer Electronics Show Las Vegas, NV, USA
January 9
January 11
Fedora User and Developer Conference Boston, USA
January 15
January 16
Foundations of Open Media Software 2009 Hobart, Tasmania, Australia
January 17
January 23
Camp KDE 2009 Negril, Jamaica
January 19
January 24
linux.conf.au - penguins march south Hobart, Australia
January 25
January 29
Ruby on Rails Bootcamp with Charles B. Quinn Atlanta, GA, USA
January 25
January 28
GCC Research Opportunities Paphos, Cyprus
January 31 Greater London Linux Users Group meeting London, UK
January 31
February 3
Black Hat Briefings DC Arlington, VA, USA
February 4
February 5
DC BSDCon 2009 Washington, D.C., USA
February 4
February 6
Money:Tech 2009 New York, NY, USA
February 5
February 9
German Perl Workshop Frankfurt, Germany
February 7 Frozen Perl 2009 Minneapolis, MN., USA
February 7
February 8
FOSDEM 2009 Brussels, Belgium

If your event does not appear here, please tell us about it.

Page editor: Forrest Cook

Copyright © 2008, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds