LWN.net Weekly Edition for March 12, 2009

Puppets, chefs, and community competition

There are many criticisms that one can make of the applications offered by the free software community, but lack of choice is generally not one of them. Our community thrives on competition while our licensing makes it hard to keep secrets from competitors. A recent episode in the Puppet community shows that, while this competition can sometimes take unwelcome forms, there is often little to do but to welcome it anyway.

Puppet is an automated configuration management system intended to make life easier for system administrators; it can be seen as a competitor to venerable tools like cfengine. Over time, Puppet has attracted an active community of users and developers; it would appear to be a tool which is growing in capability and popularity. Puppet is managed by Reductive Labs, which has a clear commercial interest in providing training and support services for Puppet users.

Recently (January, 2009), a project named Chef announced its existence. Chef's developers, who have previously worked with the Puppet code, set out to solve a similar problem. Chef is not a fork of Puppet, though; it's a new project developed from the beginning. Among other things, the Chef developers decided to use Ruby as the configuration language and they chose the Apache License (Puppet, instead, is distributed under the GPL). This project claims to be in active, production use, but its community, at this point, is clearly small. As of this writing, the chef-dev mailing list shows a total of four messages over its entire history.

Initially, the Puppet developers responded confidently to the Chef announcement:

More recently, though, Puppet developer Luke Kanies posted to the project's user list that Chef wasn't competing entirely fairly:

In particular, it is said that one developer from the Chef project has been sending private mail to Puppet users - especially those experiencing problems with Puppet - suggesting that they should switch to Chef. Luke, clearly, sees this activity as a threat to his livelihood; every Puppet user who deserts is one less potential customer. Even without that incentive, though; it can be hard to stand by and watch as others try to woo users away from your project. One need only think back to the days when "Ubuntu is better" posts were a semi-regular feature of the Fedora mailing lists to see how galling it can be.

In this case, a cooler perspective quickly won over and it became clear that there was little to be done. If nothing else, the objectionable messages were private email; there is little that the project could do to stop them even if it wanted to. Beyond that, though, certain things are inherent in the running of a free software project, including:

• There will be competition, in some form or other. Somebody, somewhere, is sure to decide to scratch an itch, even if that itch is no more substantial than "I want to run my own project." This is both a strength and a weakness in our community. The ability for new and different ideas to develop into functioning projects is the source for much of the great software we now have, but it also leads to a certain amount of duplication of effort and confusion of users.

• Some Puppet users expressed dissatisfaction that the Chef developers had clearly drawn a lot of inspiration and knowledge from the Puppet project. But, again, that's how our community works. Anybody who wants to hide the ideas that go into an application would be well advised to keep their software proprietary and closed. In the free software world we learn from each other - at least some of the time.

• In a community which values freedom, attempts to silence or banish inconvenient characters will not get very far. When inappropriate or unethical behavior is seen (and spamming users of a competing project to urge them to switch is certainly pushing the boundary), shining light on that behavior is usually the best thing to do. In this case, the discussion made it clear that this email campaign did not inspire respect; it would not be surprising to learn that the pro-Chef emails have already stopped.

Andrew Shafer summed up the situation nicely:

Projects which are focused on "awesome" tend, over the long term, to be rather more successful than projects which worry about what others might be saying about them. They are also likely to be more successful than projects which put their effort into trying to poach another project's users. Puppet appears to have good code and an active and engaged user community. If it can stay focused on that code and that community, this project need not fear what its competitors are doing.

(Thanks to Friedrich Clausen for calling our attention to this discussion).

Comments (32 posted)

OpenStreetMap: the data behind the maps

In my last article on OpenStreetMap I looked at the recent mass imports of public data — everything from British oil wells to the entire road network for the United States. But for those interested in more than an alternative to Google Maps, the ability to extract or add data to the project is what really makes OpenStreetMap shine. Whether you want to get an SVG of a campus map or import a local government's database of every building in the city, Linux users will find plenty of tools that cater to their needs.

The export tab on the web site provides the most simple way to access data. Users can draw an area on the main map view and then grab an image (in PNG, JPEG, PDF or PS formats); some HTML to embed the map into your web site; or the raw XML data. To further modify the data, either in the OpenStreetMap database or a local copy (stored as an XML .osm file on your disk) download the data using an editor like JOSM (the 'Java OpenStreetMap editor'). To make life easier when selecting the area to download, open up the preferences dialog and install the namefinder and slippy_map_chooser plugins.

Grabbing larger amounts of data would be difficult, slow and clumsy with these methods. More advanced users can get data directly through the API. Check the latitude and longitude coordinates for the area you want — an easy method for this is to use the export tab to draw an area, then note down the coordinates it records — then fire up wget or curl and download the data:

    wget http://api.openstreetmap.org/api/0.5/trackpoints?bbox=left,bottom,right,top

The main api only lets you grab 5,000 points per request; you have to page the request to get the additional data. To pull out a really large chunk of data, or to filter it (for example to just download all the pubs in the city) use the extended OSM API (XAPI, or 'zappy'). Access to really enormous amounts of data, such as the entire planet or a country, can be found in the frequently updated dumps listed on the Planet.osm wiki page.

Once you have the data there are all manner of uses - your GPS navigation device, rendering your own maps for the web or print, or converting the data into another standard GIS format with tools like the Ruby osmlib. The documentation for each tool various enormously, but the toolchains tend to be relatively straight forward.

Of course, extracting data is only half the story. Not only should all good open source citizens be contributing back, but you will get the most value from the data if you collaborate with others in developing a rich data set that will lead to tools and use cases you can later replicate.

OpenStreetMap abounds with methods and tools for entering data. You might like the "old school" method of tracing a breadcrumb GPS trail — much more fun in the early days when I mapped much of Reading with some friends from a completely blank slate. Many mappers have traced basic road layouts and buildings from aerial imagery donated from Yahoo! so that others can go in and identify street names and points of interest. The main editing tools are Potlatch, a flash interface on the main web site (just click on the 'Edit' tab once you're zoomed into your local area), and the previously-mentioned JOSM. The wiki has plenty of guidance.

When importing large sets of existing data, things get a little more complicated. The first step is to step back and have a good think. Imports can cause two kinds of headaches for other contributors if done wrong: you might put a load of new data over the top of somebody else's efforts and make a complete mess in the process; or worse, you might import data without proper permission, causing legal difficulties for the project and technical difficulties in taking the data back out again.

It's always best to begin by asking a few questions on the relevant mailing list; there are localized lists for many areas, a general (high traffic) "talk" list, and a "legal-talk" list for legal issues such as licensing for imports. It's especially important to avoid convenient interpretations of web site notices regarding copyright and database rights when deciding if you can import the data. You need to get written confirmation so that the OpenStreetMap project is immune from legal attacks. There are some nice general guidelines on the wiki, which are worth a read.

If you have data with written permission to use it, you can begin the import process. The first, and most laborious, step is to map out the data against standard OSM tags, as in this UK public transport example or this really comprehensive exercise for CanVec data. You'll notice that oftentimes source-specific data (like unique IDs for features and really niche data) is retained in a namespace like "CanVec:FID" and "naptan:StopAreaCode". This can also be useful where you don't want the data to appear until volunteers have gone through checking it against existing data in the database, for example to merge two bus stops (one crowdsourced, the other from the import).

For large chunks of data, importers have tended to write custom scripts to then bring the data in. If the data is in the OpenStreetMap format, and it is in a state suitable to go straight into the database, this bulk import script makes the process quick and painless. The Canvec2osm code shows how to pull in more complicated data; this converts 11 different shape files into themed osm files with correct tagging, which can then be worked into a suitable state for importing.

A more cautious approach can be appropriate in areas with a lot of existing data. One quite technically challenging route is to set-up your own Web Map Service (WMS) using a tool like mapserver, and then set-up the JOSM WMS plugin to pull those maps in as a layer underneath your map data so it can be traced. This Map Warper tool is in beta and tries to make this process easier. If the data is quite simple you could just put the source and editor side-by-side on your screen and use your judgement to copy over points of interest.

However you want to proceed, you're probably best off getting in touch with some local or more experienced community members. Interested people could even just lobby local government officers and public institutions to get the data, then pass it along to somebody with more of an appetite for the technical stage. Given 6 months to study, process, and import the data, you should find richly detailed maps and underlying data available under a Creative Commons BY-SA license; the license, incidentally, may soon change to one more suitable for databases. Whatever you do, just remember to have fun.

Comments (35 posted)

Interview: Ciaran O'Riordan of End Software Patents

Software patents were rejected several years ago in the European Union (EU) and undermined last year by the Bilski case in the United States. Under these circumstances, what direction should anti-software patent activism take? Ciarán O'Riordan, the newly-appointed director of the End Software Patents (ESP) campaign, answers that now is the time to organize the arguments and legal documents used in the past so that they can be used to fight the next software patent battles around the world. This material might be useful not only in the EU and USA should the status of software patents change in either jurisdiction, but also in the rest of the world.

O'Riordan began his career as a software developer with a strong interest in free software. In fact, he has membership card #8 in the Free Software Foundation (FSF), which indicates that he was one of the first to take out membership when it was offered. Moving from Ireland to Brussels in 2003, he found night time work in a bar. Increasingly, however, he found his days being filled by lobbying members of the European parliament as the debate over whether to allow software patents in the EU intensified.

"It was very strange," O'Riordan recalls. "In Europe we had the habit of reading Slashdot, and reading about all the crazy patents in the USA, and we all had a good laugh. Then, very suddenly, we were faced with our own software patent problem."

At first, O'Riordan's lobbying was volunteer work, in which he was simply "looking for the most important thing to work on." However, several months before the European parliament rejected the idea of software patents, he was hired as a lobbyist by Free Software Foundation Europe (FSFE), a separate organization from the FSF.

After the vote in parliament, he continued to lobby for FSFE whenever an issue emerged. The work, he says, "was very interesting and very important, and I found it wasn't very difficult. There was a bit of a power vacuum in the European Parliament, because people in Europe are not very interested in European politics. So when I asked politicians if I could talk to them, they were very available. So I was able to talk to various politicians, and I was able to get deeply involved in the topic, despite not having a background in patents."

Recently, O'Riordan has been studying law at Facultés universitaires Saint-Louis in Brussels and taking a leave of absence from his FSFE work. But when offered the position at ESP by the FSF, the campaign's major sponsor, he jumped at it. "Since it's a legal topic and the FSF is a good institution, I decided to give it a try," he says.

Phase 2 of the ESP Campaign

As the new director, O'Riordan replaces Ben Klemens, who was hired in November 2007 when ESP was first organized, and quietly departed in spring 2008 after preparing an amicus curiae brief in the Bilski hearing. "When the Bilski case was over, there wasn't a similar case in sight, so I guess that at that point he decided to move on," O'Riordan says, although he has yet to talk to Klemens directly.

O'Riordan now refers to Klemens's time as director as "the first phase" of ESP. In discussing the directions in which he might take the campaign, O'Riordan concluded that "in the next phase it would be a good idea to document what happened in the EU before all the documents completely disappear, and then do the same for the Bilski case. The Bilski case did its job in terms of influencing the court's decision. but it can also do a second job of aiding people all around the world who are working on similar projects. It seemed that an obvious Phase 2 would be to move from the specific to the general, and try to turn the previous campaigns into a base for future campaigns."

O'Riordan argues that such cataloging is badly needed:

Applying a global perspective

Admittedly, law can differ greatly between jurisdictions. All the same, O'Riordan suggests that ESP's new direction will be useful because most laws that concern software patents are based on international treaties. In Europe, for instance, most countries' patent laws are specific implementations of the European Patent Convention.

Similarly, given that patent law in most countries is often written ambiguously — it often pre-dates software — and is ill-equipped to deal with it, interpretation is essential. Most of the time, O'Riordan observes, interpretation is based on the question "'how do we harmonize with the rest of the world?'" — which, given the historical American dominance in trade, usually means "'how do we harmonize with the USA?'"

Even when laws and circumstances differ, O'Riordan adds, a global viewpoint can put matters into perspective. For example, the tendency of small and medium-sized American companies to support software patents — perhaps because they "are afraid of angering their mega-corporation business partners" — might be countered by pointing out that "small and medium enterprises aren't using software patents in Europe, Canada, or Australia. If we can build a picture from other countries, sometimes that can fill the gaps in the argument in one country like the USA."

In addition, O'Riordan hopes that ESP can provide a more accurate perspective. For instance, during the campaign in Europe, the fact that 77% of software patent applications in the EU were by American companies caused some observers to view the issue of software patents as a matter of American domination. However, if you take ESP's estimate on its home page that software patents cost the United States $11.2 billion, then you can establish that "it's not a case of one country taking over the world; it's a cost to everyone, and it's slowing down innovation. A lot of these arguments are actually improved by putting all the information together."

Looking ahead

To help with Phase 2, O'Riordan plans to extend the ESP's repository of information beyond the United States and Europe through a wiki that should be ready in the next few weeks.

"The first thing will be to find out what's happening already in people's countries," O'Riordan says. "For example, in the Philippines, does the patent office give out software patents? Well, I don't know. Who can I ask? So, in some cases, we're going to document what's not known, or at least what people or legal authorities or organizations we know in an area. We'll start with that and, when we have time to dig into each jurisdiction, we can start asking them questions."

As O'Riordan points out, there is no way of knowing beforehand what information might be found:

Other content for the ESP site might include advice about how to conduct a campaign and lobby politicians. "There are certain ways to talk to politicians," O'Riordan says. "They like hearing about studies, and they like hearing about legislation, legal wordings, and comparisons between other countries. They like hearing about these things, but, if people start without having these resources, then sometimes they can get off on the wrong foot."

O'Riordan also points out that politicians are not just a source of support, but also of advice about how to conduct a campaign. For example, in his own lobbying, the Green Party's explanation of how the European legislature worked was as important as the eventual votes of its members.

O'Riordan does not rule out ESP's involvement in specific campaigns. Recently, for instance, O'Riordan and other activists distributed a one page letter about Microsoft's patent case against TomTom at the company's Innovation and Growth Day in Brussels. "This is just a small way to keep the topic alive and always remind everyone that there are people against software patents."

However, ESP's main focus for now will remain education and gathering of information. Although the issue of software patents is relatively quiet now, O'Riordan does not assume that it will remain so. "The European Commission [the EU executive] will change in November , and the European patent office is having a consultation about this topic, so there's a chance that the topic will come back on the table. There's also a small chance that the [American] Supreme Court will review the Bilski decision. So now is a good time to talk stock and to prepare for possible new campaigns."

Moving into Phase 2, O'Riordan counts on the support of the free software community. "The free software community tends to understand these issues very quickly, so it's very useful, because these people get active a lot easier than people who are new to the topics of freedom and software." At the same time, though, he stresses that ESP is not directly connected to the FSF, nor aimed only at free software users. The goal of ESP, O'Riordan says, is "to build a real coalition, to really convince the politicians that this is something that effects everyone — every computer user, and every business." And, for now, the best way to reach this goal, according to O'Riordan, is to prepare the ammunition for the next campaign.

Comments (8 posted)

Page editor: Jonathan Corbet

Security

Fusil: a Python fuzzing library

When applications receive inputs they do not expect, they generally fail, hopefully with an error message of some kind—indicating that the programmer anticipated that type of bad input. But sometimes, programs crash when they receive bad input, which can lead a researcher—or attacker—to an exploitable vulnerability. Testing applications by feeding them bad data is known as "fuzzing", and there are numerous toolkits and frameworks available to help with such a task. One of those is Fusil, a Python library which can be used to write fuzzing programs.

The basic idea behind Fusil is that it will start the targeted program in a limited environment, create bad input to feed to it, and watch for various events that would indicate a program crash. Fusil monitors the process exit code, stdout and stderr for patterns that might indicate a crash, as well as keeping track cpu usage and run time to look for infinite loops and the like. It runs the process as a separate user ("fusil") to try to avoid any adverse effects to the user's environment from any crashes that result.

Fusil's most recent version is 1.2, released in early February, which comes with more than a dozen fuzzing programs for standard applications and libraries. There are fuzzers for firefox, clamav, python, and mplayer for example, along with ones for libraries like gettext and for printf() in libc. There is also a rather impressive list of crashes found by Fusil, including several that became CVE entries.

Getting started using Fusil is fairly straightforward when following the usage guide, though the author ran into a number of problems when trying to run as a non-root user. Running the fusil-python fuzzer did produce a crash ("unexpected exception during garbage collection"), which needs to be looked into further.

When it crashes an application, Fusil creates a script that will reproduce the error along with various files to help diagnose the problem. The output and a core file from the application are stored with the replay.py script. The data file and a log of the session are stored there as well. One can re-run the failing process inside gdb or valgrind by passing the appropriate option (i.e. --gdb or --valgrind) to replay.py.

There is also a document on how to write fuzzers using Fusil. It starts with the traditional "hello world" program using echo—not much fuzzing going on there—and moves into a more real-world echo fuzzer. Fusil provides various ways to randomize the data that gets handed to the application. Then there are mechanisms available to inject bad data via the command line, environment variables, data files, or the network.

Overall, Fusil looks like an interesting tool. It has already been used to find crashes in various applications and libraries, and it has the capability to be extended to many more. If you are in need of a framework to fuzz test your application, Fusil is worth a look. If more projects made use of tools like Fusil, we would probably see fewer exploitable vulnerabilities caused by unexpected input.

Comments (1 posted)

Security reports

Risk report: Four years of Red Hat Enterprise Linux 4 (Red Hat Magazine)

Mark Cox, Red Hat's director of security response, has released another of his annual reports on the security risks associated with Red Hat Enterprise Linux. It would be nice to see more distributions doing this kind of reporting on the number of vulnerabilities handled, what their severity was, and how quickly they were addressed. "The aim of this report was to get a measure of the security risk to users of Red Hat Enterprise Linux 4 during the first four years since release. We’ve shown that although on the surface it looks like Red Hat released a large number of security advisories, many of them do not apply to usual or default installations, and only a very small subset are a high risk."

Comments (1 posted)

New vulnerabilities

dash: privilege escalation

Package(s):	dash	CVE #(s):	CVE-2009-0854
Created:	March 10, 2009	Updated:	March 11, 2009
Description:	From the Ubuntu advisory: Wolfgang M. Reimer discovered that dash, when invoked as a login shell, would source .profile files from the current directory. Local users may be able to bypass security restrictions and gain root privileges by placing specially crafted .profile files where they might get sourced by other dash users.
Alerts:	Ubuntu USN-732-1 2009-03-10

Comments (none posted)

firefox: multiple vulnerabilities

Package(s):	firefox	CVE #(s):	CVE-2009-0771 CVE-2009-0772 CVE-2009-0773 CVE-2009-0774 CVE-2009-0775 CVE-2009-0776 CVE-2009-0777
Created:	March 5, 2009	Updated:	July 13, 2009
Description:	Firefox has multiple vulnerabilities. From the Red Hat alert: Several flaws were found in the processing of malformed web content. A web page containing malicious content could cause Firefox to crash or, potentially, execute arbitrary code as the user running Firefox. (CVE-2009-0040, CVE-2009-0771, CVE-2009-0772, CVE-2009-0773, CVE-2009-0774, CVE-2009-0775) Several flaws were found in the way malformed content was processed. A website containing specially-crafted content could, potentially, trick a Firefox user into surrendering sensitive information. (CVE-2009-0776, CVE-2009-0777)
Alerts:	Debian DSA-1830-1 2009-07-12 CentOS CESA-2009:0258 2009-05-19 SuSE SUSE-SA:2009:023 2009-04-20 Mandriva MDVSA-2009:083 2009-04-01 Fedora FEDORA-2009-3161 2009-03-30 Fedora FEDORA-2009-3101 2009-03-30 Red Hat RHSA-2009:0258-01 2009-03-24 Fedora FEDORA-2009-2882 2009-03-21 Fedora FEDORA-2009-2884 2009-03-21 Debian DSA-1751-1 2009-03-22 Ubuntu USN-741-1 2009-03-19 SuSE SUSE-SA:2009:012 2009-03-16 Mandriva MDVSA-2009:075 2008-03-13 Slackware SSA:2009-069-02 2009-03-11 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 CentOS CESA-2009:0315 2009-03-06 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Fedora FEDORA-2009-2422 2009-03-08 Fedora FEDORA-2009-2421 2009-03-08 Ubuntu USN-728-3 2009-03-06 Ubuntu USN-728-2 2009-03-06 Ubuntu USN-728-1 2009-03-05 CentOS CESA-2009:0325 2009-03-05 Red Hat RHSA-2009:0325-01 2009-03-04 Red Hat RHSA-2009:0315-00 2009-03-04 Gentoo 201301-01 2013-01-07

Comments (none posted)

irrlicht: arbitrary code execution

Package(s):	irrlicht	CVE #(s):	CVE-2008-5876
Created:	March 9, 2009	Updated:	March 11, 2009
Description:	From the Gentoo advisory: An unspecified component of the B3D loader is vulnerable to a buffer overflow due to missing boundary checks. A remote attacker could entice a user to open a specially crafted .irr file, possibly resulting in the execution of arbitrary code with the privileges of the user running the application, or a Denial of Service (crash).
Alerts:	Gentoo 200903-10 2009-03-07

Comments (none posted)

kernel: denial of service

Package(s):	kernel	CVE #(s):	CVE-2009-0675
Created:	March 11, 2009	Updated:	June 9, 2009
Description:	The skfp driver will allow an unprivileged user to reset the device statistics, thus losing the relevant information.
Alerts:	SuSE SUSE-SA:2009:031 2009-06-09 Debian DSA-1794-1 2009-05-06 Debian DSA-1787-1 2009-05-02 Ubuntu USN-752-1 2009-04-07 Ubuntu USN-751-1 2009-04-07 SuSE SUSE-SA:2009:017 2009-04-03 SuSE SUSE-SA:2009:015 2009-04-03 CentOS CESA-2009:0326 2009-04-01 Red Hat RHSA-2009:0326-01 2009-04-01 Red Hat RHSA-2009:0360-01 2009-03-26 Debian DSA-1749-1 2009-03-20 Mandriva MDVSA-2009:071 2009-03-10

Comments (none posted)

kernel: information disclosure

Package(s):	kernel	CVE #(s):	CVE-2009-0676
Created:	March 11, 2009	Updated:	August 20, 2009
Description:	The kernel socket code fails to properly initialize an internal data structure, allowing local users to obtain information via the getsockopt() system call.
Alerts:	SuSE SUSE-SA:2009:045 2009-08-20 SuSE SUSE-SA:2009:031 2009-06-09 SuSE SUSE-SA:2009:030 2009-06-08 Debian DSA-1794-1 2009-05-06 Debian DSA-1787-1 2009-05-02 CentOS CESA-2009:0459 2009-05-01 Red Hat RHSA-2009:0459-01 2009-04-30 SuSE SUSE-SA:2009:021 2009-04-16 Ubuntu USN-752-1 2009-04-07 Ubuntu USN-751-1 2009-04-07 SuSE SUSE-SA:2009:017 2009-04-03 SuSE SUSE-SA:2009:015 2009-04-03 CentOS CESA-2009:0326 2009-04-01 Red Hat RHSA-2009:0326-01 2009-04-01 Red Hat RHSA-2009:0360-01 2009-03-26 Debian DSA-1749-1 2009-03-20 Mandriva MDVSA-2009:071 2009-03-10

Comments (none posted)

libsndfile: arbitrary code execution, denial of service

Package(s):	libsndfile	CVE #(s):	CVE-2009-0186
Created:	March 6, 2009	Updated:	December 3, 2009
Description:	From the Mandriva advisory: Crafted data - channels per frame value - in CAF files enables remote attackers to execute arbitrary code or denial of service via a possible integer overflow, leading to a possible heap overflow.
Alerts:	Fedora FEDORA-2009-11618 2009-11-16 Fedora FEDORA-2009-11499 2009-11-16 Gentoo 200904-16 2009-04-17 Ubuntu USN-749-1 2009-03-30 Debian DSA-1742-1 2009-03-16 Mandriva MDVSA-2009:067 2008-03-05

Comments (none posted)

mahara: insufficient input sanitising

Package(s):	mahara	CVE #(s):	CVE-2009-0660
Created:	March 11, 2009	Updated:	March 11, 2009
Description:	The mahara portfolio manager is susceptible to cross-site scripting attacks.
Alerts:	Debian DSA-1736-1 2009-03-10

Comments (none posted)

mpfr: denial of service

Package(s):	mpfr	CVE #(s):	CVE-2009-0757
Created:	March 9, 2009	Updated:	May 8, 2009
Description:	From the Gentoo advisory: Multiple buffer overflows have been reported in the mpfr_snprintf() and mpfr_vsnprintf() functions. A remote user could exploit the vulnerability to cause a Denial of Service in an application using MPFR via unknown vectors.
Alerts:	Ubuntu USN-772-1 2009-05-07 Gentoo 200903-13 2009-03-09

Comments (none posted)

openswan: insecure tmp file usage

Package(s):	openswan	CVE #(s):	CVE-2008-4190
Created:	March 9, 2009	Updated:	April 9, 2009
Description:	From the Gentoo advisory: Dmitry E. Oboukhov reported that the IPSEC livetest tool does not handle the ipseclive.conn and ipsec.olts.remote.log temporary files securely. A local attacker could perform symlink attacks to execute arbitrary code and overwrite arbitrary files with the privileges of the user running the application.
Alerts:	CentOS CESA-2009:0402 2009-04-09 Debian DSA-1760-1 2009-03-30 Red Hat RHSA-2009:0402-01 2009-03-30 Gentoo 200903-18 2009-03-09

Comments (none posted)

openttd: arbitrary code execution

Package(s):	openttd	CVE #(s):	CVE-2008-3547 CVE-2008-3576 CVE-2008-3577
Created:	March 9, 2009	Updated:	March 11, 2009
Description:	From the Gentoo advisory: Multiple buffer overflows have been reported in OpenTTD, when storing long for client names (CVE-2008-3547), in the TruncateString function in src/gfx.cpp (CVE-2008-3576) and in src/openttd.cpp when processing a large filename supplied to the "-g" parameter in the ttd_main function (CVE-2008-3577). An authenticated attacker could exploit these vulnerabilities to execute arbitrary code with the privileges of the OpenTTD server.
Alerts:	Gentoo 200903-09 2009-03-07

Comments (none posted)

pdfjam: multiple vulnerabilities

Package(s):	pdfjam	CVE #(s):	CVE-2008-5843 CVE-2008-5743
Created:	March 9, 2009	Updated:	March 13, 2009
Description:	From the Gentoo advisory: * Martin Vaeth reported multiple untrusted search path vulnerabilities (CVE-2008-5843). * Marcus Meissner of the SUSE Security Team reported that temporary files are created with a predictable name (CVE-2008-5743). A local attacker could place a specially crafted Python module in the current working directory or the /var/tmp directory, and entice a user to run the PDFjam scripts, leading to the execution of arbitrary code with the privileges of the user running the application. A local attacker could also leverage symlink attacks to overwrite arbitrary files.
Alerts:	Fedora FEDORA-2009-2655 2009-03-13 Fedora FEDORA-2009-2651 2009-03-13 Gentoo 200903-05 2009-03-07

Comments (none posted)

php: web site intrusion

Package(s):	php	CVE #(s):	CVE-2009-0754
Created:	March 6, 2009	Updated:	January 6, 2010
Description:	From the Mandriva advisory: PHP 4.4.4, 5.1.6, and other versions, when running on Apache, allows local users to modify behavior of other sites hosted on the same web server by modifying the mbstring.func_overload setting within .htaccess, which causes this setting to be applied to other virtual hosts on the same server.
Alerts:	Gentoo 201001-03 2010-01-05 Fedora FEDORA-2009-3768 2009-04-21 Fedora FEDORA-2009-3848 2009-04-21 Debian DSA-1789-1 2009-05-04 Ubuntu USN-761-1 2009-04-20 Red Hat RHSA-2009:0350-01 2009-04-14 CentOS CESA-2009:0338 2009-04-07 CentOS CESA-2009:0337 2009-04-06 Red Hat RHSA-2009:0337-01 2009-04-06 Red Hat RHSA-2009:0338-01 2009-04-06 Mandriva MDVSA-2009:066 2008-03-05 Mandriva MDVSA-2009:065 2009-03-05

Comments (none posted)

poppler: denial of service

Package(s):	poppler	CVE #(s):	CVE-2009-0755 CVE-2009-0756
Created:	March 6, 2009	Updated:	December 1, 2009
Description:	From the Mandriva advisory: A crafted PDF file that triggers a parsing error allows remote attackers to cause denial of service. This bug is consequence of a wrong processing on FormWidgetChoice::loadDefaults method (CVE-2009-0755). A crafted PDF file that triggers a parsing error allows remote attackers to cause denial of service. This bug is consequence of an invalid memory dereference on JBIG2SymbolDict::~JBIG2SymbolDict destructor when JBIG2Stream::readSymbolDictSeg method is used (CVE-2009-0756).
Alerts:	Ubuntu USN-850-1 2009-10-21 SuSE SUSE-SR:2009:012 2009-07-03 Debian DSA-1941-1 2009-11-25 rPath rPSA-2009-0059-1 2009-04-17 Mandriva MDVSA-2009:068-1 2009-03-07 Mandriva MDVSA-2009:068 2008-03-06

Comments (none posted)

roundup: privilege escalation

Package(s):	roundup	CVE #(s):
Created:	March 11, 2009	Updated:	April 10, 2009
Description:	Any authenticated roundup users who is able to create and edit queries is able to edit any queries on the system, regardless of ownership. See this bug report for more information.
Alerts:	Debian DSA-1754-1 2009-04-09 Fedora FEDORA-2009-2583 2009-03-11 Fedora FEDORA-2009-2591 2009-03-11

Comments (none posted)

websvn: multiple vulnerabilities

Package(s):	websvn	CVE #(s):	CVE-2008-5918 CVE-2008-5919
Created:	March 9, 2009	Updated:	March 11, 2009
Description:	From the Gentoo advisory: James Bercegay of GulfTech Security reported a Cross-site scripting (XSS) vulnerability in the getParameterisedSelfUrl() function in index.php (CVE-2008-5918) and a directory traversal vulnerability in rss.php when magic_quotes_gpc is disabled (CVE-2008-5919). A remote attacker can exploit these vulnerabilities to overwrite arbitrary files, to read changelogs or diffs for restricted projects and to hijack a user's session.
Alerts:	Gentoo 200903-20 2009-03-09

Comments (none posted)

xerces-c: denial of service

Package(s):	xerces-c	CVE #(s):	CVE-2008-4482
Created:	March 9, 2009	Updated:	March 11, 2009
Description:	From the Gentoo advisory: Frank Rast reported that the XML parser in Xerces-C++ does not correctly handle an XML schema definition with a large maxOccurs value, which triggers excessive memory consumption during the validation of an XML file. A remote attacker could entice a user or automated system to validate an XML file using a specially crafted XML schema file, leading to a Denial of Service (stack consumption and crash).
Alerts:	Gentoo 200903-19 2009-03-09

Comments (none posted)

znc: privilege escalation

Package(s):	znc	CVE #(s):	CVE-2009-0759
Created:	March 9, 2009	Updated:	March 11, 2009
Description:	From the Gentoo advisory: cnu discovered multiple CRLF injection vulnerabilities in ZNC's webadmin module. A remote authenticated attacker could modify the znc.conf configuration file and gain privileges via newline characters in e.g. the QuitMessage field, and possibly execute arbitrary code.
Alerts:	Debian DSA-1735-1 2009-03-10 Gentoo 200903-02 2009-03-06

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

The current 2.6 development kernel remains 2.6.29-rc7; no new prepatches have been released over the last week. About 160 fixes have been merged into the mainline since the 2.6.29-rc7 release; a -rc8 prepatch is likely sometime in the very near future.

The current stable 2.6 kernel remains 2.6.28.7; no stable updates have been released since February 20.

Comments (1 posted)

Kernel development news

Quotes of the week

-- Matthew Garrett

-- Anthony Liguori

-- Ted Ts'o

Comments (1 posted)

ext4 and data loss

The ext4 filesystem offers a number of useful features. It has been stabilizing quickly, but that does not mean that it will work perfectly for everybody. Consider this example: Ubuntu's bug tracker contains an entry titled "ext4 data loss", wherein a luckless ext4 user reports:

Your editor had not intended to write (yet) about this issue, but quite a few readers have suggested that we take a look at it. Since there is clearly interest, here is a quick look at what is going on.

Early Unix (and Linux) systems were known for losing data on a system crash. The buffering of filesystem writes within the kernel, while being very good for performance, causes the buffered data to be lost should the system go down unexpectedly. Users of Unix systems used to be quite aware of this possibility; they worried about it, but the performance loss associated with synchronous writes was generally not seen to be worth it. So application writers took great pains to ensure that any data which really needed to be on the physical media got there quickly.

More recent Linux users may be forgiven for thinking that this problem has been entirely solved; with the ext3 filesystem, system crashes are far less likely to result in lost data. This outcome is almost an accident resulting from some decisions made in the design of ext3. What's happening is this:

• By default, ext3 will commit changes to its journal every five seconds. What that means is that any filesystem metadata changes will be saved, and will persist even if the system subsequently crashes.

• Ext3 does not (by default) save data written to files in the journal. But, in the (default) data=ordered mode, any modified data blocks are forced out to disk before the metadata changes are committed to the journal. This forcing of data is done to ensure that, should the system crash, a user will not be able to read the previous contents of the affected blocks - it's a security feature.

• The end result is that data=ordered pretty much guarantees that data written to files will actually be on disk five seconds later. So, in general, only five seconds worth of writes might be lost as the result of a crash.

In other words, ext3 provides a relatively high level of crash resistance, even though the filesystem's authors never guaranteed that behavior, and POSIX certainly does not require it. As Ted put it in his excruciatingly clear and understandable explanation of the situation:

Accidental or not, the avoidance data loss in a crash seems like a nice feature for a filesystem to have. So one might well wonder just what would have inspired the ext4 developers to take it away. The answer, of course, is performance - and delayed allocation in particular.

"Delayed allocation" means that the filesystem tries to delay the allocation of physical disk blocks for written data for as long as possible. This policy brings some important performance benefits. Many files are short-lived; delayed allocation can keep the system from writing fleeting temporary files to disk at all. And, for longer-lived files, delayed allocation allows the kernel to accumulate more data and to allocate the blocks for data contiguously, speeding up both the write and any subsequent reads of that data. It's an important optimization which is found in most contemporary filesystems.

But, if blocks have not been allocated for a file, there is no need to write them quickly as a security measure. Since the blocks do not yet exist, it is not possible to read somebody else's data from them. So ext4 will not (cannot) write out unallocated blocks as part of the next journal commit cycle. Those blocks will, instead, wait until the kernel decides to flush them out; at that point, physical blocks will be allocated on disk and the data will be made persistent. The kernel doesn't like to let file data sit unwritten for too long, but it can still take a minute or so (with the default settings) for that data to be flushed - far longer than the five seconds normally seen with ext3. And that is why a crash can cause the loss of quite a bit more data when ext4 is being used.

The real solution to this problem is to fix the applications which are expecting the filesystem to provide more guarantees than it really is. Applications which frequently rewrite numerous small files seem to be especially vulnerable to this kind of problem; they should use a smarter on-disk format. Applications which want to be sure that their files have been committed to the media can use the fsync() or fdatasync() system calls; indeed, that's exactly what those system calls are for. Bringing the applications back into line with what the system is really providing is a better solution than trying to fix things up at other levels.

That said, it would be nice to improve the robustness of the system while we're waiting for application developers to notice that they have some work to do. One possible solution is, of course, to just run ext3. Another is to shorten the system's writeback time, which is stored in a couple of sysctl variables:

    /proc/sys/vm/dirty_expire_centisecs
    /proc/sys/vm/dirty_writeback_centisecs

The first of these variables (dirty_expire_centiseconds) controls how long written data can sit in the page cache before it's considered "expired" and queued to be written to disk; it defaults to 30 seconds. The value of dirty_writeback_centiseconds (5 seconds, default) controls how often the pdflush process wakes up to actually flush expired data to disk. Lowering these values will cause the system to flush data to disk more aggressively, with a cost in the form of reduced performance.

A third, partial solution exists in a set of patches queued for 2.6.30; they add a set of heuristics which attempt to protect users from being badly burned in certain situations. They are:

• A patch adding a new EXT4_IOC_ALLOC_DA_BLKS ioctl() command. When issued on a file, it will force ext4 to allocate any delayed-allocation blocks for that file. That will have the effect of getting the file's data to disk relatively quickly while avoiding the full cost of the (heavyweight) fsync() call.

• The second patch sets a special flag on any file which has been truncated; when that file is closed, any delayed allocations will be forced. That should help to prevent the "zero-length files" problem reported at the beginning.

• Finally, this patch forces block allocation when one file is renamed on top of another. This, too, is aimed at the problem of frequently-rewritten small files.

Together, these patches should mitigate the worst of the data loss problems while preserving the performance benefits that come with delayed allocation. They have not been proposed for merging at this late stage in the 2.6.29 release cycle, though; they are big enough that they will have to wait for 2.6.30. Distributors shipping earlier kernels can, of course, backport the patches, and some may do so. But they should also note the lesson from this whole episode: ext4, despite its apparent stability, remains a very young filesystem. There may yet be a surprise or two waiting to be discovered by its early users.

Comments (114 posted)

A superficial introduction to fsblock

Many kernel developers may work through their entire career without encountering a buffer_head structure. But the buffer head (often called "bh") sits at the core of the kernel's memory management and filesystem layers. Simply put, a bh maintains a mapping between a specific page (or portion thereof) in RAM and its corresponding block on disk. In the 2.4 days, the bh structure was also a key part of the block I/O layer, but 2.6 broke that particular association. That notwithstanding, the lowly, much-maligned bh still plays a crucial role in contemporary kernels.

Why "much-maligned"? Buffer heads are difficult to manage, to the point that they can create significant memory pressure on some systems. They deal in very small units of I/O (512 bytes), so you need a pile of them to represent even a single page. And there is a certain sense of antiquity that one encounters when dealing with them; the buffer head code is some of the oldest code in the core kernel. But it is important and tricky code, so few developers dare to try to improve it.

Nick Piggin is the daring type. But Nick, too, is not trying to improve the bh layer; instead, he would like to replace it outright. The result is an intimidating set of large patches known as "fsblock." This code was first posted in 2007, making it fairly young by the standards of memory-management patches. This patch set was reposted in early March; it has shown a number of improvements on the way. Nick says "I'm pretty intent on getting it merged sooner or later," so we'll likely be seeing more of this code in the future.

The core data structure is struct fsblock, which represents one block:

    struct fsblock {
	unsigned int	flags;
	unsigned int	count;

    #ifdef BDFLUSH_FLUSHING
	struct rb_node	block_node;
    #endif
	sector_t	block_nr;
	void		*private;
	struct page	*page;
    };

This structure, notes Nick, is about 1/3 the size of struct buffer_head, but it serves roughly the same purpose: tracking the association between an in-memory block (found in page) and its on-disk version, indexed by block_nr. The flags field describes the state of this block: whether it's up-to-date (memory and disk versions match), locked, dirty, in writeback, etc. Some of these flags (the dirty state, for example) match the state stored with the in-memory page; the fsblock layer (unlike the buffer_head code) takes great care to keep those flags in sync.

There are a couple of interesting flags in the fsblock structure which one does not find associated with buffer heads. One of them is not a flag at all: BL_bits_mask describes a subfield giving the size of the block. In fsblock, "blocks" are not limited to the standard 512-byte sector size; they can, in fact, even be larger than a page. These "superpage" blocks have been on some filesystem developers' wish lists for some time; they would make it easy to create filesystems with large blocks which, in turn, would perform better in a number of situations. But the superpage feature may be removed for any initial merge of fsblock in an attempt to make the code easier to understand and review. Besides, large blocks are a bit of a controversial topic, so it makes sense to address that issue separately.

The flags field also holds a flag called BL_metadata; this flag indicates a block which holds filesystem metadata instead of file data. In this case, the block is actually part of a larger structure which (edited slightly) looks like this:

    struct fsblock_meta {
	struct fsblock block;
	union {
    #ifdef VMAP_CACHE
	    /* filesystems using vmap APIs should not use ->data */
	    struct vmap_cache_entry *vce;
    #endif
	    /*
	     * data is a direct mapping to the block device data, used by
	     * "intermediate" mode filesystems.
	     */
	    char *data;
	};
    };

In short, this structure makes it easy for filesystem code to deal directly with metadata blocks. Finally, the fsblock_sb structure ties a filesystem superblock into the fsblock subsystem.

A filesystem can, at mount time, set things up with a call to:

    int fsblock_register_super(struct super_block *sb, 
                               struct fsblock_sb *fsb_sb);

The superblock can then be read in with a call to sb_mbread():

    struct fsblock_meta *sb_mbread(struct fsblock_sb *fsb_sb, 
                                   sector_t blocknr);

There's only one little problem: before fsblock can perform block I/O operations, it must have access to the superblock. So, thus far, filesystems which have been converted to fsblock must still use the buffer head API to read the superblock. One assumes that this little glitch will be taken care of at some point.

A tour of the full fsblock API would require a few articles - it is a lot of code. Hopefully a quick overview will provide a sense for how it all works. To start with, blocks are reference-counted objects in fsblock, so there is the usual set of functions for incrementing and decrementing the counts:

    void block_get(struct fsblock *block);
    void block_put(struct fsblock *block);
    void mblock_get(struct fsblock_meta *block);
    void mblock_put(struct fsblock_meta *block);

There's a whole set of functions for performing I/O on blocks and metadata blocks; some of these are:

    struct fsblock_meta *mbread(struct fsblock_sb *fsb_sb, sector_t blocknr, 
    	   		        unsigned int size);
    int mblock_read_sync(struct fsblock_meta *mb);
    int sync_block(struct fsblock *block);

Note that, while there are a number of functions for reading blocks, there are fewer write functions. Instead, code will use a function like set_block_dirty() or mark_mblock_dirty(), then leave it up to the memory management code to decide when the actual I/O should take place.

There is a lot more than this to fsblock, including functions to lock blocks, look up in-memory blocks, perform page I/O, truncate pages, implement mmap(), and more. One assumes that Nick will certainly write exhaustive documentation for this API sometime soon.

Beyond that little documentation task, there are a few other things to do, including supporting direct I/O and fixing a number of known bugs. But, even now, fsblock seems to have a lot of potential; it updates the old buffer head API in a way which is more efficient and more robust. It also appears to perform better with the ext2 filesystem - a fact which appears to be surprising to Nick. So something like fsblock will almost certainly be merged sooner or later. A lot could happen in the mean time, though. Core memory-management-related patches like this are notoriously slow to get through the merging process, and, despite its age, fsblock has not seen a great deal of review to date. So there's likely to be plenty of time and opportunity for other developers to find things to disagree with before fsblock hits the mainline.

Comments (1 posted)

Linux and 4K disk sectors

As storage devices become bigger and bigger in capacity, the areal density (number of bits packed per physical square inch) increases; hard drives are now hitting the limits. Hard drive manufacturers are now pushing to increase the basic unit of data transfer in hard drives - physical sector size - from 512 bytes to 4096 bytes (or 4KB) to improve storage efficiency and performance. However, there are a lot of subsystems affected by this change that are currently not ready to accept a 4K sector size.

The first hard drive, the RAMAC, was shipped on September 13, 1956. It weighed 2,140 pounds and held a total of 5 megabytes (MB) of data on fifty 24-inch platters. It was available for lease for $35,000 USD, the equivalent of approximately $300,000 in today's dollars.

We have come a long way since then. Hard drive capacities are now measured in terabytes, but some legacy parameters, such as the sector size, have remained unchanged. The sector size is wired into a lot of data structures in the kernel, for example, the i_blocks field of struct inode stores the number of 512-byte physical blocks it occupies on the media. Even though the core kernel deals with 512-byte sectors, the block layer is capable of handling hardware with different length sector sizes.

Why the Change?

Any sort of data communication must contend with noise. This noise is also present during the data transfer from the magnetic surface of the physical hard drive platter to the head of the hard drive. Noise can be introduced by physical defects on the hard drive platter. Noise such as this is measured with respect to the signal strength, more commonly known as Signal to Noise Ratio (SNR). As disk drive areal density increases, the signal to noise ratio decreases, thereby creating increased sensitivity to defects.

Hard Disk Drives have special reserved bits in addition to the packed data, called the Error-Correcting Code (ECC) bits. Each physical data byte sector block is followed by, besides other bytes, the ECC bytes on the physical medium. ECC is responsible for the reliability of the data transferred. Usually the Reed-Solomon Algorithm is used to compute the ECC bits; to detect and to a certain extent, correct the errors read; it is an efficient algorithm to correct errors which come in bursts. The ECC bits are placed immediately after the data bytes (as shown in the diagram below), so the error, if any, can be corrected as the disk spins. Besides the ECC, the disk also has bits reserved before the data bits, for the preamble, data sync mark; and the Inter Sector Gap (ISG) after the ECC bits.

With the increase in areal density, more bits are packed in a square inch of physical surface. A physical defect of, say 100 nanometers, would require more ECC bits to correct than is needed at lower densities. The physical defect induces more noise than signal hence the SNR decreases. This requires more bytes packed in ECC fields of the sector to compensate for the decrease in SNR and ensure the reliability of the data stored on the disk. For example: on disks with a density of 215 kbpi (kilo bytes per square inch), a 512-byte data sector requires 24 bytes of ECC; a format efficiency (number of user data bytes vs total number of bytes on disk) of 92%. With an increase of areal density to 750 kbpi, each 512-byte sector requires 40 bytes per sector to achieve the same level of disk reliability. The format efficiency of such a drive is 87%.

A sector size of 4096 bytes requires 100 bytes for ECC to maintain the same level of reliability at an areal density of 750kbpi; that yields a format efficiency of 96%. As areal densities in disk drives continue to increase, the physical size of each sector on the surface of the disk become smaller. If the mean size and number of disk defects and scratches does not scale at the same rate, then we expect more sectors to be corrupted, and we expect the resulting burst errors to more easily exceed the error correction capability of each sector. Having larger sectors, would enable such burst errors to be detected for larger sectors, hence decreasing the total ECC overhead. Besides the ECC, the disk also has bits reserved before the data bits, for the preamble, data sync mark, and the Inter Sector Gap (ISG). Increasing the sector size to 4K from 512 bytes, would decrease the occurrences of these fields, thus improving the format efficiency further.

For all of these reasons, the storage industry wants to move to larger sector sizes. The IDEMA International Disk Drive Equipment and Materials Association (IDEMA) was formed to increase co-operation among competing hard drive brands. IDEMA is responsible for the smooth transition of sector size from 512 bytes to 4Kbytes. Also, bigsector.org was set up to maintain documentation of the transition. The documentation section of bigsector.org contains more information about the transition.

Transition

This change affects a lot of areas in the storage system chain: from the drive interface, the host interface, BIOS, OS to applications such as partition managers. A change affecting so many subsystems might not be readily acceptable to the market. To make a smooth transition, the following stages are planned:

1. 512 byte logical with 512 byte physical. This is the current state of hard drives

2. 512-byte logical with 4096-byte physical sector size. This would facilitate a smooth transition from 512-byte to 4096-byte sector sizes.

3. 4096-byte logical with 4096-byte physical sectors. This would be done once all hardware and software would be aware of the underlying change and geometry with respect to sector size. This change would first be seen in SCSI devices and later in ATA devices.

During the transition phase (step 2), drives are planned to use 512 byte emulation, known as read-modify write (RMW). Read-modify-write is a technique used to emulate 512-byte sector size over a 4K physical sector size. Written data which does not correspond to full 4K sectors would result in the drive first reading the existing 4K sector, modifying the part of data which changed, and writing the 4K sector data back to the drive. More information on RMW and its implementation can be found in this set of slides. Needless to say, RMW decreases the throughput of the device, though the shorter ECC will compensate by giving an overall better performance (hopefully). Such drives are expected to be commercially available in the first quarter of 2011.

Matthew Wilcox recently posted a patch to support 4K sectors according to the ATA-8 standard (PDF). The patch adds an interface function by the name sector_size_supported(). Individual drivers are required to implement this function and return the sector size used by the hardware. The size returned is stored in the sect_size field of the ata_device structure. This function returns 512 if the device does not recognize the ATA-8 command, or the driver does not implement the interface. The sect_size is used instead of ATA_SECT_SIZE when the data transfer is a multiple of 512-byte sectors.

The partitioning system and the bootloader will also require changes because they rely on the fact that partitions start from the 63rd sector of the drive, which is misaligned with the 4K sector boundary. This problem will be solved, in the short term, by using the 4K physical - 512 byte logical drives. The 512-byte sectors are aligned in a way that the 1st logical sector starts from the 1st octant of the physical 1st 4K sector, as shown below.

This scheme to coincide the logical and physical sectors to optimize data storage and transfer is known as odd-aligned physical/logical sectors. It can lead to other problems though: odd-aligned sectors might misalign the data with respect to filesystem blocks. Assuming a 4K page size, a random read would require two 4K sector reads. This is the reason, applications such as bootloaders and partitioning systems should be ready for 4K sector size hard drives (step 3), for overall throughput efficiency.

An increased sector size is required by hard drives to break the current limits of hard drive capacity while minimizing the overhead of error checking data. However, a smooth transition will decide the acceptability of these drives in the market. The previous transition, which broke the 8.4GB limit using Large Block Access (LBA), was easily accepted. However, with so many drives in use currently, the transition would be determined by the co-operation of various subsystems of the data supply chain, such as filesystems and applications dealing with hard drives.

Comments (20 posted)

Patches and updates

Kernel trees

• Stephen Rothwell: linux-next: state of the trees. (March 6, 2009)

Core kernel code

• Gautham R Shenoy: sched: Extend sched_mc/smt_power_savings framework. (March 9, 2009)

• Andrea Arcangeli: fork vs gup(-fast) fix. (March 11, 2009)

Development tools

• Akinobu Mita: generic debug pagealloc (-v2). (March 5, 2009)

• Mathieu Desnoyers: LTTng 0.105 core for Linux 2.6.27-rc9. (March 6, 2009)

• Joerg Roedel: DMA-API debugging facility v4. (March 6, 2009)

• Frederic Weisbecker: Binary ftrace_printk. (March 6, 2009)

• Frederic Weisbecker: Syscalls tracing. (March 8, 2009)

• Larry Woodman: mm tracepoints. (March 9, 2009)

• Dan Carpenter: smatch 1.51 released. (March 10, 2009)

Device drivers

• Sven-Thorsten Dietrich: [ANNOUNCE] USB genirq infrastructure for threaded interrupt handlers V2. (March 5, 2009)

• Matthew Garrett: toshiba_acpi: Add full hotkey support. (March 6, 2009)

• Rohit Hagargundgi: [MTD] Flex-OneNAND support. (March 8, 2009)

• Alex Chiang: PCI core learns hotplug. (March 9, 2009)

• Harry Ciao: Add AMD8111 and AMD8131 EDAC drivers. (March 9, 2009)

• Geert Uytterhoeven: Generic RTC class driver. (March 9, 2009)

• graff.yang@gmail.com: [net/irda]: new Blackfin on-chip SIR IrDA driver. (March 10, 2009)

• Daniel Mack: Added driver for ISL29003 ambient light sensor. (March 10, 2009)

• Lennert Buytenhek: mwl8k: Marvell TOPDOG wireless driver. (March 10, 2009)

• graff.yang@gmail.com: [net/irda]: new Blackfin on-chip SIR IrDA driver. (March 11, 2009)

• Ilya Yanok: dnet: Dave DNET ethernet controller driver. (March 11, 2009)

• Rafael J. Wysocki: PM: Rework suspend-resume ordering to avoid problems with shared interrupts (updated). (March 11, 2009)

Documentation

• Jeff Moyer: man-pages: add documentation about the memlock implications of io_setup. (March 9, 2009)

• Michael Kerrisk: MAINTAINERS: downgrade support for man-pages. (March 11, 2009)

Filesystems and block I/O

• J. R. Okajima: Aufs source files. (March 9, 2009)

• Ryusuke Konishi: nilfs2 settle matters related to disk format. (March 9, 2009)

• Sage Weil: ceph: Ceph distributed file system client. (March 10, 2009)

• Ryusuke Konishi: Asking for inclusion of nilfs2 in the mainline kernel. (March 10, 2009)

Janitorial

• Thomas Gleixner: Remove obsolete genirq defines and typedefs. (March 11, 2009)

• H. Peter Anvin: x86: remove zImage support. (March 11, 2009)

Memory management

• Balbir Singh: Memory controller soft limit patches (v4). (March 6, 2009)

• Tejun Heo: percpu: use dynamic percpu allocator as the default percpu allocator. (March 10, 2009)

Networking

• Luis R. Rodriguez: cfg80211: send regulatory beacon hint events to userspace. (March 10, 2009)

Architecture-specific

• Tejun Heo: x86, percpu: implement and use reserved percpu alloc. (March 6, 2009)

• James Bottomley: convert voyager over to the x86 quirks model. (March 8, 2009)

Security-related

• etienne: SMACK : add logging support V1. (March 8, 2009)

Virtualization and containers

• Dave Hansen: track files for checkpointability. (March 5, 2009)

• Bharata B Rao: cpuacct: per-cgroup utime/stime statistics - v1. (March 10, 2009)

Miscellaneous

• Neil Brown: ANNOUNCE: mdadm 2.6.9 - A tool for managing Soft RAID under Linux. (March 10, 2009)

• Neil Brown: ANNOUNCE: mdadm 3.0-devel3 - A tool for managing Soft RAID under Linux. (March 10, 2009)

Page editor: Jonathan Corbet

Distributions

News and Editorials

Mer: Remastering Maemo

Mer is an outgrowth of Nokia's Maemo environment, designed to flesh out the tablet-centric operating system into a full-fledged Linux distribution suitable for embedded and desktop systems of all description. The project's genesis was an effort to back port the upcoming Maemo 5.0 release to no-longer-supported Nokia N800 and N810 tablets, but it has subsequently evolved to run on BeagleBoards, embedded navigation devices like the Pocket LOOX, and standard x86 hardware.

Nokia released the first Maemo 5.0 SDK alpha this month, building towards beta and final releases before the end of 2009. Codenamed Fremantle, 5.0 will be the first major upgrade since June of 2008. Although much online discussion has centered on speculative hardware devices that might accompany the release, the more significant changes in 5.0 are under the hood: Nokia's concerted effort to synchronize the platform with standard PC Linux distributions. Fremantle will use technologies like Upstart, PulseAudio, Open Hardware Manager, and more.

Nokia announced in late 2008 that the 5.0 release would target OMAP3 processors, meaning that OMAP2 devices such as the existing Maemo tablets would be unsupported. The Maemo developer community soon embarked on a "Maemo Reconstructed" campaign to build and maintain the free portions of the software for the older devices. That project eventually grew into Mer. Developer Carsten Munk described the effort as a proof-of-concept operating system initially, but added that it became viable for day-to-day use as well. Part of the credit belongs to the Maemo community, he said, and part belongs to Nokia itself, which has cooperated fully, offered to relicense components wherever possible, and even provide firmware images for closed-source drivers such as the tablets' power management system.

The result, as Munk put it, is that the Mer project can focus on building a "proper" distribution — eventually incorporating package repositories and regular releases. Like Nokia, the project has decided to align its base system with mainstream desktop Linux. Mer will do more than just track the underlying components, however, and will base its system on Ubuntu. That means building the same packages included in the desktop distribution, rather than low-resource alternatives like BusyBox. Although the present system makes heavy use of the Hildon application toolkit developed for Maemo tablets, Munk says to expect GTK+ and Qt support as well.

0.9 Dream

The team has been working in two-week-long development sprints since January of 2009, focusing its efforts. The most recent release is 0.9, from March 2. Flashable firmware images are available for all three Nokia tablets, as are builds for the BeagleBoard, the Pocket LOOX 720, generic x86 machines, and a bootable VMWare disk image.

Mer 0.9 runs kernel 2.6.28 and ships with a working Hildon desktop environment. Many of the desktop applications and home screen applets pre-loaded on Nokia's tablets are not installed by default, but you can add them through the package manager. Mer currently uses its own package repositories for the base system, but the tablet builds should be compatible with standard Maemo .ipkg packages. The Maemo Extras repository is enabled, opening the door to dozens of third-party packages built by the larger Maemo community, but some of the available applications fail to install due to missing dependencies not yet provided by Mer.

The core of the operating system is stable: input, display, and networking all work without trouble. Mer 0.9 is not yet usable as an everyday tablet operating system, however, due to lack of applications. The WebKit-based Midori browser is provided and works fine — but email, PIM, and instant messaging are not yet available. Munk said that the plan is to build open source applications provided in Nokia's Maemo releases; in the meantime some users may want to try the Modest email client. Maemo developers would do well to test their applications on Mer, but end users need to know that it is not ready to replace official Nokia firmware.

Nokia, GNOME, Ubuntu, et al.

In fact, Nokia's open source spokesman Quim Gil said he hopes that application developers will take a look at Mer, because having two Maemo operating systems is better for the platform. Feedback is one thing, "but it's something different if someone takes some parts of your platform, makes some changes and comes back with a proof of concept that such changes might be better for your own platform."

Mer helps "make Maemo's long tail longer and stronger," Gil said. "In order to get their work done they need to look at our code and they do file bugs and enhancement requests against platform components and with a platform integration mentality. This is useful feedback because it comes soon (and sometimes often) and also because it complements well the kind of feedback we get from users and application developers."

Gil describes the relationship between the Mer project and Nokia's Maemo team as mutually beneficial, noting that Mer makes concrete requests for licensing and redistribution changes, which are far better than blanket requests to free everything. "From our point of view it is much easier and sensible to react to specific requests with a concrete output (e.g. 'please allow the redistribution of these Nokia binaries so we can try to deliver a Maemo 5 community edition for the N800/N810'). This is also true for platform components that are actually not owned by Nokia, for instance TI's graphics acceleration drivers for OMAP2, where we are trying also to help getting a 'community edition' of such drivers."

The project is already working on a proposal to include community editions of these closed-source drivers, including firmware images fully installable on existing Nokia tablets — although it is unknown when the first such images would become available. The next Mer sprint ends on March 16, and the corresponding 0.10 release should include improvements to battery management, theme support, and wireless networking.

Mer is far from being the only Linux distribution aimed at mobile devices. Some even use many of the same stack components, such as Ubuntu's Mobile Internet Device (MID) Edition or the GNOME Mobile platform. According to Munk, Mer is different in that it is community-owned and not primarily a platform for sale to vendors. Not that it is unsuitable, he added, noting "it would be trivial to take a typical Mer image, put it on an OMAP3 board with touchscreen, put your Map software on top of it and then you have a GPS gadget ... and that's how easy it ought to be."

Maemo has been highly respected and successful on Nokia's tablet hardware, including the original Hildon interface and UI toolkits and well-integrated components from upstream Linux. As the first independent, noncommercial deployment of Maemo, Mer, if successful, could anticipate further blurring of the lines between handheld devices and mainstream distributions.

Comments (3 posted)

New Releases

Mandriva Linux 2009 Spring RC1

Mandriva Linux 2009 Spring RC1 (code name pomerol) is available in the following isos: Free DVD edition (x86-32 and x86-64 architectures), One KDE (x86-32 architecture only), One GNOME (x86-32 architecture only) and Dual arch (both x86-32 and x86-64 architectures).

Full Story (comments: none)

Tin Hat 20090309 released

Version 20090309 of the Tin Hat distribution has been announced. Tin Hat is an interesting, RAM-only, Gentoo-based distribution. "Tin Hat was conceived as a challenge to the old mantra that physical access to a system means full access to the data. This is certainly true in the case of unencrypted file systems, and at least potentially true in the case of encrypted. Rather, Tin Hat aims towards the ideal of guaranteeing zero information loss should the attacker physically acquire the box --- either the adversary is faced with no file system to even begin cracking, or if any non-ephemeral memory is found, the adversary should not be able to tell if he is looking at encrypted data or random noise."

Comments (8 posted)

New TurnKey releases

TurnKey Linux announced new versions of its PostgreSQL appliance and the LAPP (Linux, Apache2, PostgreSQL, PHP/Perl/Python) appliance.

Comments (none posted)

Distribution News

Debian GNU/Linux

Debian Project Leader Election 2009

Two candidates have emerged for the upcoming Debian project leader (DPL) election: Stefano Zacchiroli and current DPL Steve McIntyre. The campaigning period has started and will run until the voting starts on March 29. Click below for the full announcement.

Full Story (comments: none)

Debian's Google Summer of Code 2009

Debian is looking for mentors for the 2009 Summer of Code. Steve McIntyre has already volunteered to be an admin and to submit the application to Google.

Full Story (comments: none)

New Security Team Members

The Debian Project has announced the addition of Nico Golde and Steffen Joeris as full members of the security team. "Both developers have worked on testing-security before and are extending their work to the old and current stable releases of Debian GNU/Linux."

Full Story (comments: none)

Fedora

Fedora revises its trademark guidelines

Fedora project leader Paul Frields writes about changes to the Fedora trademark guidelines in his blog. "So back in June and July of 2008 we had numerous discussions about the issues and I drafted a set of use cases we wanted to cover. From there, we figured out how we could enable as many of those use cases as possible, while still making sure that the Fedora trademark retained its particular identity and value. [...] The result is our new trademark guidelines. These have actually been about 99% unchanged for the last couple of months. A few minor tweaks and the removal of a draft notice later, voila!"

Comments (none posted)

Approved spins for Fedora 11

The following spins have been approved for Fedora 11: AOS Spin, BrOffice.org Spin, Education Spin, Electronic Lab Spin, Games Spin and XFCE Spin. A spin contains only those packages available in the Fedora repository, bundled together in a convenient DVD or CD format. Click below for addition information.

Full Story (comments: none)

Fedora Classroom - March 2009 completed

IRC logs are available for Fedora Classroom sessions. These sessions include: Introduction to bash shell scripting - Scott McBrien, Training & Tips for Fedora Ambassadors -- Max Spevack, Fedora Trademarks -- Paul W. Frields, Perl Basics -- Doran Barton, Using the Windows cross-compiler - Richard Jones, and Introduction to mock, the chroot package builder -- Kevin Fenzi.

Full Story (comments: none)

SUSE Linux and openSUSE

A proposed openSUSE 11.2 Roadmap

A proposal for the openSUSE 11.2 roadmap has been posted. It calls for the release in November; it would feature KDE 4.3, GNOME 2.28, a 2.6.30 kernel, and possibly ext4 as the default filesystem. "To give us something to plan around, we would like to propose a fixed release schedule. As a six-month release schedule is not something we consider feasible to maintain high-quality standards, we are proposing a fixed eight-month schedule."

Full Story (comments: 3)

openSUSE: Gearing Up for Google Summer of Code

The openSUSE Project is looking for a few good mentors for the Google Summer of Code 2009. The openSUSE Project was a mentoring organization last year, and they are hoping to participate again in 2009. Organization sign up started March 9, and prospective mentors can sign up the following week as well.

Comments (none posted)

download.opensuse.org broken

The download server at download.opensuse.org will be unavailable for some time. Its storage array is broken, and there is no backup. Sponsors for hardware would be gladly accepted to create a backup server. Joe "Zonker" Brockmeier has posted an update.

Full Story (comments: none)

Distribution Newsletters

Arch Linux Newsletter for March, 2009

The Arch Linux Newsletter March 2009 edition is out. "Welcome to yet another release of the Arch Linux Newsletter. This month is a very special one for the Arch Linux Newsletter. We have an interesting interview with Pierre Schmitz, the developer in charge of KDE. In the Community Highlights section we take a look at the contributions of many Arch Linux users, highlighting the contributions of nsf, an outstanding Arch Linux user and contributor. As well, we discover who is the winner of the Screenshot of the Month title."

Comments (none posted)

DistroWatch Weekly, Issue 293

The DistroWatch Weekly for March 9, 2009 is out. "Have you ever run out of space on your hard drive or had to fiddle with shifting data around? Well, Logical Volume Management (LVM) could be the answer for you! It is supported in most Linux distributions and this week we take a look at what it can offer. In the news section, openSUSE publishes new trademark guidelines, Ubuntu releases mainline kernel updates, the Debian project goes to the polls to elect a new project leader, and ULTILEX delivers a brilliant live CD that offers a collection of minimalist Linux distributions and several useful utilities. Finally, while the week has brought precious little in terms of interesting new releases, we took the time to add four new distributions to the DistroWatch database; that includes the purposely insecure Damn Vulnerable Linux, the Slackware-based Parslinux with KDE 4, the ultra minimalist Tiny Core Linux, and the energy-saving wattOS."

Comments (none posted)

Fedora Weekly News #166

The Fedora Weekly News for March 8, 2009 is out. "A small sample of this issue's stories reflects the imminent release of Fedora 11! Announcements lists the freeze dates and upcoming Fedora events. PlanetFedora rounds up essential blog reading including a piece by Thomas Vander Stichele on "meltdown analysis". Marketing cheers for "One Million New Fedora 10 Installations". In QualityAssurance a reminder that the next of the "Test Days" is of interest to Intel video users is just one of the items reflecting a massive amount of QA activity. Ambassadors relates some OLPC news from Rochester Institute of Technology. Developments explains why "Orphans are Purged" and asks are we "Ready for a New RPM Version?". Translation highlights a "Study about FLP". Artwork stares at the wallpaper while "Preparing for the Beta Release". SecurityAdvisories lists stuff to help you avoid a rooting. Virtualization pops some salient items out of the development maelstrom including a "New Release of libvirt-0.6.1" and SELinux "sVirt Support Committed". There's a lot more, so keep reading!"

Full Story (comments: none)

openSUSE Weekly News Issue 62

This issue of the covers openSUSE Trademark Guidelines Released, Ken Yap: How to clone a VirtualBox Linux VM to a real machine (v0.9), 11.2 Roadmap and Fixed Release Cycle for openSUSE, Rupert Horstkotter: OSF Status Report #3 and much more.

Comments (none posted)

Ubuntu Weekly Newsletter #132

The Ubuntu Weekly Newsletter for the week ending March 7, 2009 is out. "In this issue we cover: Karmic Koala release schedule, QA Team: Next testing day, Hug Day: March 12th, Americas Board: New Ubuntu Members, LoCo Directory Moves Forward, Ubuntu Tunisia: Migration Project, Ubuntu in the Cloud, Community Interview: Michael Godawski, Simplifying Forums Categories, Mark a thread as Solved, mail Stack Improvements in Ubuntu 9.04, Ubuntu Encrypted home with 2 factor authentication, Ubuntu Drupal: Planet Module, Introducing Tarmac, TurnKey: 12 new Ubuntu-based server appliances released, Technical Board Meeting, Server Team Meeting Minutes: March 3rd, and much, much more!"

Full Story (comments: none)

Distribution meetings

Registration for DebConf9 opened

Registration is open for DebConf9, which will take place in Cáceres, Extremadura, Spain, July 24 - 30, 2009. Click below for the announcement. The call for papers, presentations, discussion sessions and tutorials is also open, until April 15, 2009. "This year submission of a formal written paper for the conference proceedings is again optional, though encouraged. Providing a written paper in advance means that interested people can attend your session ready with ideas for discussion, and especially helps those who find it hard to follow rapid English speech."

Full Story (comments: none)

Newsletters and articles of interest

Installing KVM Guests With virt-install On Ubuntu 8.10 Server (HowtoForge)

HowtoForge shows one way of installing KVM Guest on a Ubuntu 8.10 server. "Unlike virt-manager, virt-install is a command line tool that allows you to create KVM guests on a headless server. You may ask yourself: "But I can use vmbuilder to do this, why do I need virt-install?" The difference between virt-install and vmbuilder is that vmbuilder is for creating Ubuntu-based guests, whereas virt-install lets you install all kinds of operating systems (e.g. Linux, Windows, Solaris, FreeBSD, OpenBSD) and distributions in a guest, just like virt-manager. This article shows how you can use it on an Ubuntu 8.10 KVM server."

Comments (2 posted)

Page editor: Rebecca Sobol

Development

The Lucene Search Suite

The Lucene project lets you index the documents on your filesystem or web server so you can run combined full text and metadata searches. A full text search takes one or more words of a human language as a query and should return documents which are the "most relevant" for those words. Web searches are a classic example of full text searches. Metadata searches should be familiar to anyone who has used the find command; for example, looking for all files that have been modified in the last week.

The primary goal of Lucene is to provide a fast index and query implementation and to specify an interface to the index implementation -- how to send queries to it and get your results back as fast as possible. Lucene is not, by itself, designed to be a complete user-facing index solution but rather to provide the heart of such a system. There are also higher level projects which use one of the Lucene implementations to provide search capabilities, for example, KDE4's strigi desktop search. If you just want to add a search capability to something then you might like to explore these higher level tools to see if you can save the time of writing a program that uses the Lucene API directly.

It is tempting to think of adding full text to an index as just a filesystem traversal where you read each file and shove the byte contents into the index. Normally you want to extend this to allow conversions too, such as extracting the plain text of PDF files and indexing the extracted human readable text instead of the bytes that comprise the PDF file. The metadata associated with a document is entirely up to you, for example, extracting the Vorbis artist, album and track comments from FLAC audio files and adding them as metadata.

Using Lucene to index your Web site lets you offer a text search feature - like a Google search box - for servicing searches like "Wakelocks embedded". This is only the beginning though, because you can also offer advanced searches by combining metadata into the search. If you build a Lucene index for each registered user, the personalized search you can offer is hard to beat. For example, finding pages about "locking" that contain a link to a specific web site in the article comments. Or any article on "locking" that contains a comment by any one of your friends.

Lucene is actually an umbrella project which has many implementations in Java, C++, Ruby and PHP among others. Probably the most widely known implementation of Lucene is the original one that is done in Java. In recent times, implementations in C++ (CLucene) and PHP (Zend_Search_ Lucene) have become available. There are also implementations in Perl and Ruby, see the full list for details. The CLucene page states that its primary goal is to be faster than the Java version. It would appear that the PHP implementation was primarily driven by the desire to be homogeneous with the PHP environment.

The implementation of these full text and metadata search types normally call for different queries and thus different implementations to best resolve the queries. For example, it might be quite common to want to search for a range in a metadata query, like all the documents added to the index in December, whereas a full text query might demand ranking of documents that contain the strings "DDR3" and "latency".

You don't really need to know what Lucene does on its side of the API to build and search indexes with it, though a high level knowledge of what happens in the implementation can help you understand how to make efficient use of the API.

Abstractly, a Lucene index consists of many Document objects, each of which contains one or more fields. A field is a key-value pair, for example, the key of "indexed-on" and a value "Wed Dec 17, 2008 @ 3:58 PM". The full text content of a document is also added to the Lucene index as a field property of a document.

Fields can be stored verbatim in the index, or have an index created for them, or both. You might want to index and store the URL that a document was retrieved from, but might want to only index the document text because storing it verbatim might make the index too large for your application. An index on the contents of a file is likely to be much smaller than the file itself. If you have access to the original file you don't really want to store it in the Lucene index verbatim too. A field can also be tokenized or stored atomically (a so called keyword). You would want to tokenize the text content of a file but probably want the date it was indexed to remain an atomic value.

Normally you would have Lucene tokenize the text of a file and build an inverted file arrangement for the tokens. For example, the word "token" would have a list of which document numbers contain that word along with other metadata relating to how often that term appears in each document relative to the length of the document. This way queries looking for "token" and "lucene" can be resolved by merging the two lists for each token.

A great deal of attention has been paid to not locking data in the index with Lucene. This way, the index can undergo updating in the background while it is actively being used to service searches. This eliminates the need to wait on the background process. You can only have a single update running for an index at any time, but many clients can be reading the index while that update is occurring.

A Lucene index is made up of one or more segments. Each segment is fully independent of any other segment and is stored in one or more files. Concurrency without locking is achieved by writing any new or changed data to a new Segment. One way to speed up indexing documents and create fewer segments is to have Lucene cache as many of the added documents in RAM and flush out a single, large segment on a less frequent basis

For Java Lucene the setRAMBufferSizeMB is used to set how much RAM can be used before a new segment is written, its default is only 16Mb. Creating larger segments during indexing means it will take slightly longer before clients can see new documents (because the new segment is not written and is thus not accessible) but will make for fewer, larger segments and thus less need to merge segments later.

Instead of flushing a new segment when enough RAM has been used, you can force a segment to be flushed every X documents with setMaxBufferedDocs. By default, flushing is done when the buffered RAM size is reached and there is no default maximum number of documents before a flush.

Segments are merged either periodically during the adding of documents or by calling one of many optimize methods. If an index is to remain constant for a period of time it is a good idea to optimize it so that multiple segments are converted into a single segment. Optimization has the additional side benefit that if your filesystem is not full, writing a new single-segment Lucene index should also mean that the index is stored in a single filesystem extent.

Adding segments and merging segments are very similar operations. To merge segments, all of the data is copied from the old segments into a new segment and the old segments are then discarded. The currently active segments are listed in the "segments" file. Depending on how the implementation of Lucene you are using operates, the segments file might use a commit lock to protect it while it is being updated. At any rate, as the segments file just lists the file names and other metadata about segments, it can be updated very quickly.

I mentioned at the outset that Lucene specializes in full text indexing. There are some issues when using Lucene for numerical and date metadata which make using those datatypes a more complex task than just shoving full text into the index.

Knowing the Lucene API and how to include and search for information in a Lucene index can allow you to develop many applications. Hopefully the glimpse behind the API that I've included can help you get started writing applications that use Lucene efficiently. Because there are implementations of Lucene in PHP, C++, C#, Java and other languages you can apply general knowledge of Lucene to applications ranging from Web development to embedded coding.

Comments (none posted)

System Applications

Database Software

Firebird 2.1.2 RC2 released

Version 2.1.2 release candidate 2 of the Firebird DBMS has been announced. "This is the second release candidate of the Firebird version 2.1.2 patch release. It is a BETA whose purpose is for FIELD TESTING. It is recommended that you test it before deploying it into production."

Comments (none posted)

MySQL Community Server 5.1.32 released

Version 5.1.32 of MySQL Community Server has been announced. "MySQL Community Server 5.1.32, a new version of the popular Open Source Database Management System, has been released. MySQL 5.1.32 is recommended for use on production systems."

Full Story (comments: none)

PostgreSQL Weekly News

The March 8, 2009 edition of the PostgreSQL Weekly News is online with the latest PostgreSQL DBMS articles and resources.

Full Story (comments: none)

Embedded Systems

BusyBox 1.13.3 released

Version 1.13.3 of BusyBox, a collection of command line utilities for embedded systems, has been announced. "1.13.3 is a bug fix release. It has fixes for awk, depmod, init, killall, mdev, modprobe, printf, syslogd, tar, top, unzip, wget."

Comments (none posted)

New Online Community for Developers of Embedded Linux Devices (LinuxElectrons)

LinuxElectrons looks at Meld, a new on-line community for embedded Linux. Meld, which is sponsored by MontaVista, takes some of the ideas of social networks and applies them to help embedded Linux developers collaborate. "'Linux is based on the idea of sharing knowledge, and there are strong underpinnings of this throughout the Linux community, yet there isn't a place for embedded Linux developers to go to collaborate and experience that sense of community,' said Joerg Bertholdt, Vice President of Marketing at MontaVista Software. 'Now, through Meld we want all embedded Linux device developers to come together to share their knowledge, collaborate with one another, and speed the design of innovative, commercial solutions running on embedded Linux. A strong community benefits all of its members and we believe this forum will allow Linux to grow and prosper in embedded devices.'"

Comments (11 posted)

Networking Tools

libnetfilter_log 0.0.16 released

Version 0.0.16 of libnetfilter_log has been announced. "libnetfilter_log is a userspace library providing interface to packets that have been logged by the kernel packet filter. It is is part of a system that deprecates the old syslog/dmesg based packet logging."

Full Story (comments: none)

libnetfilter_queue 0.0.17 released

Version 0.0.17 of libnetfilter_queue has been announced. "The netfilter project proudly presents: libnetfilter_queue-0.0.17 is a userspace library providing an API to packets that have been queued by the kernel packet filter. It is is part of a system that deprecates the old ip_queue / libipq mechanism."

Full Story (comments: none)

libnfnetlink 0.0.41 released

Version 0.0.41 of libnfnetlink has been announced. "libnfnetlink is the low-level library for netfilter related kernel/userspace communication. It provides a generic messaging infrastructure for in-kernel netfilter subsystems (such as nfnetlink_log, nfnetlink_queue, nfnetlink_conntrack) and their respective users and/or management tools in userspace."

Full Story (comments: none)

ulogd 2.0.0 beta 3 released

Version 2.0.0 beta 3 of ulogd has been announced. "ulogd is a userspace logging daemon for netfilter/iptables related logging. This includes per-packet logging of security violations, per-packet logging for accounting purpose as well as per-flow logging. "

Full Story (comments: none)

Virtualization Software

ConVirt: goes 1.0 (SourceForge)

Version 1.0 of ConVirt has been announced. "ConVirt is an intuitive, graphical management tool providing comprehensive life cycle management for Virtual Machines. We are extremely pleased to announce the immediate availability of ConVirt v1.0. This critical milestone comes after many months of development, bug-fixing and hard-earned validation in data centers, all of which was made possible by the invaluable feedback, encouragement and contributions from the ConVirt Community."

Comments (none posted)

Web Site Development

lighttpd 1.4.22 released

Version 1.4.22 of the lighttpd web server has been announced. "And here we are again… we had some bad regressions, so 1.4.22 was needed earlier than we expected and spawn-fcgi is still included in this release."

Comments (none posted)

LimeSurvey: 1.80 full release (SourceForge)

Version 1.80 of LimeSurvey has been announced. "LimeSurvey (formerly PHPSurveyor) is a PHP survey software to create online surveys. Features open/closed surveys, branching, participant administration, quotas, WYSIWYG HTML editor, email invitations & reminders, assessments, basic statistics and more. The LimeSurvey 1.80 release marks the end of four release candidates and five month of work on this new release."

Comments (none posted)

Miscellaneous

Python process utility 0.1.1 released

Version 0.1.1 of psutil has been announced. "psutil is a module providing an interface for retrieving information on running processes in a portable way by using Python. It currently supports Linux, OS X, FreeBSD and Windows. Aside from fixing some bugs psutil 0.1.1 includes the following major enhancements: * FreeBSD support has been added * Support for determining process's UID and GID has been added * Support for determining parent PID of a process * A process_iter() function to iterate over processes as Process objects with a generator has been added * Process objects can now also be compared with == operator for equality (PID, name, command line are compared). As of now psutil is released to the general public, and should be considered a beta release implementing basic functionality."

Full Story (comments: none)

Desktop Applications

Audio Applications

First version of jackpanel (0.0.1) released

The initial release of jackpanel has been announced. "jackpanel is a graphical frontend for the JACK audio server, emphasizing simplicity, good look and feel and GNOME integration. Realtime switch, latency and samplerate can be changed with one or two mouse clicks. It comes in two flavors: A GNOME panel applet and standalone. X-runs are displayed and can be reset with a mouse click."

Full Story (comments: none)

Jajuk: 1.7 'Firestarter' made available (SourceForge)

Version 1.7 of Jajuk has been announced. "Jajuk 1.7 comes with major performance enhancements and a brand new rating system. Jajuk is a Java music organizer for all platforms. The main goal of this project is to provide a fully-featured application to advanced users with large or scattered music collections."

Comments (none posted)

Taglib extension library and tools: First release (SourceForge)

The first release of Taglib extension library and tools has been announced. "The libtagext0 library is a short hack to provide extended reading and writing meta tags for several audio files as an extension to Scott Wheeler's "TagLib" library."

Comments (none posted)

Desktop Environments

GNOME 2.26.0 release candidate (2.25.92) released

GNOME 2.26.0, release candidate 2.25.92 has been announced. "My friends, we're nearly there! 2.26.0 will be out in two weeks. Yes, it will! I tell you so. And it will be a milestone in our history. Sure, it will! You don't doubt it. Because it's looking quite good. It definitely does! Ask around you to check. And people will love it. That's for sure! Make people try it. But we can still work a bit more on polishing GNOME for the prime time. In the next ten days, we should all try to focus on the list of showstoppers [1] and try to close as many of them as possible."

Full Story (comments: none)

GNOME Software Announcements

The following new GNOME software has been announced this week:

• Banshee 1.4.3 (bug fixes)
• Empathy 2.25.92 (new features, bug fixes and translation work)
• gnome-netstatus 2.26.0 (bug fixes, code cleanup and translation work)
• GNOME Power Manager 2.25.92 (bug fixes and translation work)
• GPointingDeviceSettings 1.0.0 (unspecified)
• Libgda 3.99.13 (bug fixes)
• Peggy 0.01 (initial release)
• PyGTK 2.14.1 (bug fix and documentation work)
• Pygtksourceview 2.5.0 (new feature)
• Tomboy 0.13.6 (new features, bug fixes and translation work)

You can find more new GNOME software releases at gnomefiles.org.

Comments (none posted)

KDE 4.2.1 released

Version 4.2.1 of KDE has been announced. "KDE Community Ships First Translation and Service Release of the 4.2 Free Desktop, Containing Numerous Bugfixes, Performance Improvements and Translation Updates".

Full Story (comments: none)

KDE Software Announcements

The following new KDE software has been announced this week:

• 2ManDVD 0.5.3 (bug fixes and translation work)
• Amarok 2.0.2 (bug fixes)
• Babiloo 2.0.3.1 (new features and translation work)
• FlvToMp3 1.1.2 (translation work)
• GamCat 0.27 (unspecified)
• kopete-thinklight KDE4 0.50 (unspecified)
• KRadio4 for KDE4.x snapshot-2009-03-05 (new features, bug fixes and KDE4 support)
• KRadio4 for KDE 4.2 snapshot-2009-03-08 (bug fixes)
• QtiPlot 0.9.7.5 (new features, bug fixes and translation work)
• QUfw 0.1 alpha 2 (unspecified)
• Simutrans Starter 1.0.5 (bug fixes)
• SMILE 0.9.0 (bug fixes)
• Speaker-Phone Plasma Applet 0.30 (for KDE4.2) (new feature)
• Speaker-Phone Plasma Applet 0.90-rc (for KDE4.2) (new features)
• yape 2.1 (new feature and license change)

You can find more new KDE software releases at kde-apps.org.

Comments (none posted)

Xorg Software Announcements

The following new Xorg software has been announced this week:

• applewmproto 1.2.0 (bug fixes and code cleanup)
• libAppleWM 1.2.0 (new features, bug fixes and code cleanup)
• libXrandr 1.3.0 (bug fixes, code cleanup and documentation work)
• randrproto 1.3.0 (new features, bug fixes and code cleanup)
• xf86-input-evdev 2.2.0 (bug fixes and build improvements)
• xf86-input-synaptics 1.0.99.4 (bug fix)
• xf86-input-synaptics 1.1.0 (documentation work)
• xf86-video-intel 2.6.99.901 (new features, bug fixes and code cleanup)
• xf86-video-intel 2.6.99.902 (build fix)

More information can be found on the X.Org Foundation wiki.

Comments (none posted)

Desktop Publishing

Inforama: Community Edition 1.2 beta 2 Available (SourceForge)

Version 1.2 beta 2 of Inforama Community Edition has been announced. "Inforama - Document Automation. Document templates, generation and distribution. Create letter templates using OpenOffice and import existing Acrobat forms. Merge data to produce high quality PDF documents and automatically email, print and view. Inforama version 1.2 beta 2 has been released. We didn't announce the beta 1 release as we found some significant bugs which we wanted to fix - hence the jump straight to beta 2."

Comments (none posted)

Mail Clients

Claws Mail 3.7.1 unleashed

Version 3.7.1 of Claws Mail has been announced. "New in this release: * Spell Checking has been added to the Subject in the Compose window. * The 'Quotation characters' option has been moved from the Compose/ Writing page of the preferences to the /Message View/Text Options page, where it should be. * When replying to signed and/or encrypted mail and the preference to sign and/or encrypt is set, the original mail's privacy system is automatically used, if available. * If a text/calendar attachment is present in a message it is automatcally selected if a suitable plugin (i.e. vCalendar) is available. * /Tools/List URLs now shows both the link title and URI if possible. * A URI appearing in the statusbar is now only trimmed if necessary. * When using /Tools/Create filter|procesing rule/Automatically the List-Id header is preferred to X-* headers..."

Full Story (comments: none)

Multimedia

Elisa Media Center 0.5.31 released

Version 0.5.31 of Elisa Media Center has been announced. "Elisa is a cross-platform and open-source Media Center written in Python. It uses GStreamer for media playback and pigment to create an appealing and intuitive user interface. This release is a "light weight" release, meaning it is pushed through our automatic plugin update system."

Full Story (comments: none)

Music Applications

Strasheela 0.9.9 released

Version 0.9.9 of Strasheela has been announced. "Strasheela is a highly expressive constraint-based music composition system. Users declaratively state a music theory and the computer generates music which complies with this theory. A theory is formulated as a constraint satisfaction problem (CSP) by a set of rules (constraints) applied to a music representation in which some aspects are expressed by variables (unknowns). Music constraint programming is style-independent and is well-suited for highly complex theories (e.g. a fully-fledged theory of harmony). Results can be output into various formats including MIDI, Lilypond, and Csound. This release brings many small-scale improvements and extensions to Strasheela."

Full Story (comments: none)

Office Suites

KOffice 2.0 Beta 7 Released (KDEDot)

Version 2.0 Beta 7 of KOffice has been announced. "The KOffice developers have released their seventh beta for KOffice 2.0. This release may be the last of the many betas. A decision on whether there will be another beta or if the next version will be the first Release Candidates will be made next week. The list of changes is longer than ever. For this release we have concentrated on crashes, data loss bugs and ODF saving and loading."

Comments (none posted)

OpenOffice.org Newsletter

The February, 2009 edition of the OpenOffice.org Newsletter is out with the latest OO.o office suite articles and events.

Full Story (comments: none)

Video Applications

FFmpeg 0.5 released

It has been a long time since we have seen an FFmpeg release, but version 0.5 is now out. As one might expect, the changes are extensive and are mostly in the form of new codecs. More information can be found on the web site and in the version 0.5 changelog. "It is codenamed 'half-way to world domination A.K.A. the belligerent blue bike shed' to give an idea where we stand in the grand scheme of things and to commemorate the many fruitful discussions we had during its development."

Comments (6 posted)

Web Browsers

Firefox 3.0.7 is available

Version 3.0.7 of the Firefox web browser has been announced. "As part of Mozilla Corporation's ongoing security and stability update process, Firefox 3.0.7 is now available for Windows, Mac, and Linux for free download from http://getfirefox.com/. We strongly recommend that all Firefox users upgrade to this latest release." Several security fixes are included, see the release notes for more information.

Full Story (comments: 8)

Firefox 3.1 becoming Firefox 3.5

Firefox version 3.1 will be renumbered to version 3.5. "As was discussed at the delivery meeting yesterday, we're proposing to change the version number of Shiretoko from 3.1 to 3.5. The increase in scope represented by TraceMonkey and Private Browsing, plus the sheer volume of work that's gone into everything from video and layout to places and the plugin service make it a larger increment than we believe is reasonable to label ".1"."

Full Story (comments: 2)

Languages and Tools

Caml

Caml Weekly News

The March 10, 2009 edition of the Caml Weekly News is out with new articles about the Caml language.

Full Story (comments: none)

Perl

POE::Component::IRC 6.00 is here

Version 6.00 of POE::Component::IRC has been announced. "For the uninitiated, POE::Component::IRC is an event-driven IRC client library built on top of POE. People mostly use it to write bots. Some have made that even easier by creating a simpler interface suited to that task (see Bot::BasicBot). I became involved in the project about 14 months ago, fixing bugs and adding features. There've been about 50 releases during that time, so there's something for everybody."

Comments (none posted)

Python

Jython 2.5 beta 2 released

Version 2.5 beta 2 of Jython, an implementation of Python in Java, has been announced. "Unless a severe bug is found, this will be the last beta before we start putting out release candidates. The modjy project has been pushed into the core, there have been many bugfixes. I attempted to get all of the bugfixes out of the tracker and into the NEWS file. Hopefully we can get more disciplined about change logs in the future."

Full Story (comments: none)

Jython 2.5 beta 3 released

Version 2.5 beta 3 of Jython has been released. "When I released Beta 2 this Saturday, I said it would be the last beta unless a severe bug was found. Well, a severe bug was found. Under certain circumstances Jython Beta 2 would not start on Windows."

Full Story (comments: none)

Python 3.1 alpha 1 released

Version 3.1 alpha 1 of Python has been announced. "Python 3.1 focuses on the stabilization and optimization of features and changes Python 3.0 introduced. The new I/O system has been rewritten in C for speed. Other features include a ordered dictionary implementation and support for ttk Tile in Tkinter. Please note that these are alpha releases, and as such are not suitable for production environments."

Full Story (comments: none)

Python-URL! - weekly Python news and links

The March 11, 2009 edition of the Python-URL! is online with a new collection of Python article links.

Full Story (comments: none)

Tcl/Tk

Tcl-URL! - weekly Tcl news and links

The March 5, 2009 edition of the Tcl-URL! is online with new Tcl/Tk articles and resources.

Full Story (comments: none)

XML

cssutils 0.9.6a2 released

Version 0.9.6a2 of cssutils has been announced. The software is: "A Python package to parse and build CSS Cascading Style Sheets." Changes include bug fixes, an API change and some new capabilities.

Full Story (comments: none)

Cross Compilers

PyMite release 07 announced

Release 07 of PyMite has been announced, it includes new features and bug fixes. "PyMite is a flyweight Python interpreter written from scratch to execute on 8-bit and larger microcontrollers with resources as limited as 64 KB of program memory (flash) and 4 KB of RAM. PyMite supports a subset of the Python 2.5 syntax and can execute a subset of the Python 2.5 bytecodes. PyMite can also be compiled, tested and executed on a desktop computer."

Full Story (comments: none)

IDEs

eric 4.3.1 released

Version 4.3.1 of eric, an IDE for Python and Ruby, has been announced. "I just uploaded eric 4.3.1. It is a maintenance release fixing some bugs."

Full Story (comments: none)

Test Suites

Linux Desktop Testing Project 1.5.1 released

Version 1.5.1 of the Linux Desktop Testing Project, a test automation framework for desktop applications, has been announced. "This release features number of important breakthroughs in LDTP as well as in the field of Test Automation. This release note covers a brief introduction on LDTP followed by the list of new features and major bug fixes which makes this new version of LDTP the best of the breed."

Full Story (comments: none)

Version Control

Mercurial 1.2 released

Version 1.2 of the Mercurial source code management system has been announced. "This is a larger feature release."

Full Story (comments: none)

TopGit 0.7 announced

Version 0.7 of TopGit has been announced, it adds new features and bug fixes. "The most useful new feature (in my opinion) is a new export method that provides your patches as a linear history in a regular git branch for pulling by your upstream. TopGit aims to make handling of large amount of interdependent topic branches easier. In fact, it is designed especially for the case when you maintain a queue of third-party patches on top of another (perhaps Git-controlled) project and want to easily organize, maintain and submit them - TopGit achieves that by keeping a separate topic branch for each patch and providing few tools to maintain the branches".

Full Story (comments: none)

Page editor: Forrest Cook

Linux in the news

Recommended Reading

How To Successfully Compete With Open Source Software (MicroISV)

The MicroISV blog has some interesting thoughts on competing with open source. The author makes a closed-source application for teachers and the article looks at six areas where proprietary applications can better their open source competition. While his focus is on proprietary application developers, there is much for open source developers to consider. "However, relatively few people in the discussion mentioned B2C (Business To Consumer — you know, the stuff that isn’t paid for by an expense account) software, which people often tell me is doomed, doomed, doomed. Seeing as how I run a small B2C software business, and am experiencing a crushing shortage of doom, I thought I would explain why this is possible." (thanks to Patrick Spinler).

Comments (40 posted)

Linux companies sign Microsoft patent protection pacts (LinuxWorld)

Steven J. Vaughan-Nichols investigates possible GPL violations hidden by secret Microsoft FAT patent licenses in a ComputerWorld blog. "So, while we now know there are at least 18 FAT LFN licensees, we still don't know which companies have signed such deals. This information is kept secret by Microsoft and these companies are well-aware of the open-source and legal backlash that could result from admitting to these patent deals. The most important reason why the specifics of these deals are under NDA is that any company doing a patent cross license without covering its downstream recipients, i.e. users, is a direct violation of GPLv2 section 7, and is even more explicitly a GPLv3 violation."

Comments (14 posted)

Trade Shows and Conferences

First Free Software Conference Held in Nigeria (KDEDot)

Jonathan Riddell covers the recent Nigerian FOSS conference. "The first Nigerian conference on Free and Open Source Software was held this week in Kano, Nigeria. The conference featured local speakers, consultants, network engineers, system administrators and academics, and international guests from KDE for three days at Bayero University of Kano. Over 500 students and professionals attended, filling the hall to capacity."

Comments (none posted)

Companies

Linux Foundation Forges Deal, Takes the Wheel at Linux.com (LinuxInsider)

LinuxInsider looks at a recent deal between the Linux Foundation and SourceForge. "Both companies emphasized that the sale involved only the "Linux.com" name, not the business or its other activities conducted by SourceForge. "This is a transfer of the URL. There's some collaboration and some use of SourceForge content on the site. We are representing the site in media sales. This sits very naturally, and we are very sensitive to the community. The Linux Foundation will have day-to-day responsibility for running the Linux.com site," Jon Sobel, group president of Media for SourceForge, told LinuxInsider. The transfer of the URL is just one part of the whole relationship between the two companies."

Comments (none posted)

Wind River reports strong year, lowers guidance (LinuxDevices)

LinuxDevices takes a look at Wind River's fourth quarter results. "Linux sales and wins remained strong, said [CEO Ken] Klein. In addition to the 48 percent fourth-quarter growth in bookings over the previous 4Q, Linux revenues totaled 14 million, a 27 percent increase year over year. Yearly sales involving Linux were said to have totaled $65 million."

Comments (none posted)

Linux at Work

Midland Memorial Hospital improves operations with OpenVista

Medsphere has sent out a press release concerning a successful deployment of the open-source OpenVista EHR system. "Since the implementation of Medsphere's OpenVista electronic health record (EHR), Midland Memorial Hospital has realized a host of improved clinical results, including fewer patient deaths and medical errors and decreased infection rates, an independent case study confirms. The 2008 study was initiated by Medsphere to evaluate the effectiveness of OpenVista and was conducted by a third-party organization without Medsphere involvement." (Found on LinuxMedNews).

Comments (none posted)

Resources

Little Boxes: Audio Production Hardware At Studio Dave (Linux Journal)

Over at Linux Journal, Dave Phillips catalogs some of the audio hardware that he uses with Linux in his studio. "Some caveats: the gear described here is oriented toward music production, not consumer audio requirements. However, Studio Dave is hardly what I would call a professional studio, so please understand that when I use the term "professional", I'm referring to the equipment, not my studio's physical plant. Also, bear in mind that features common to consumer-grade devices might not be found on equipment designed for use in audio production, so if you're looking for the best soundcard for playing MP3s and DVD surround-sound audio, these devices are not likely to be your best solutions."

Comments (1 posted)

Virtual Users And Domains With Postfix, Courier, MySQL And SquirrelMail (HowtoForge)

HowtoForge presents a tutorial on setting up a mail server with virtual domains. "This document describes how to install a Postfix mail server that is based on virtual users and domains, i.e. users and domains that are in a MySQL database. I'll also demonstrate the installation and configuration of Courier (Courier-POP3, Courier-IMAP), so that Courier can authenticate against the same MySQL database Postfix uses."

Comments (1 posted)

Reviews

Cisco's PostPath to Linux powered hosted email (InternetNews.com)

Sean Michael Kerner takes a look at a Linux powered hosted email service from Cisco. "It will be interesting to see how the PostPath technology furthers Cisco's Linux interest as well since Cisco tends not to do things on a small scale. A large Linux powered hosted email system will no doubt result in scalability and performance improvement that could well extend behind the confines of Cisco itself and benefit the broader open source ecosystem."

Comments (none posted)

Miscellaneous

Bruce Perens: Is Open Source Capitalist or Communist? (Datamation)

Over at Datamation, Bruce Perens takes a look at whether open source is capitalist or communist, coming to the conclusion that it is both. "First, is business capitalistic? Well, sure, you'd say. But the truth is that business rarely operates under a pure capitalist model. And especially not now. Under such a model, a bankrupt or uncompetitive business would be allowed, indeed encouraged, to die in a sort of Social Darwinism."

Comments (61 posted)

Page editor: Forrest Cook

Announcements

Non-Commercial announcements

European Open Source Foundation Introduces First SOA for Cloud Computing

The European Open Source Foundation has introduced an SOA for Cloud Computing that creates a secure bridge between Java and .NET. ""Cloud computing is the future," said Andreas Hartl, head of the OSBF Interoperability project group and Director of Platform Strategy at Microsoft Germany. "That's why, when we launched our group in summer 2008, we defined two projects ISB and Identity Network Service (INS) both of which focus squarely on customer requirements. The ISB makes it possible for services of different developers in the cloud to communicate with one another. And the INS allows users to access a number of combined services with a single sign-on to the cloud.""

Full Story (comments: none)

Free Software Foundation Europe Celebrates Eighth Birthday

The Free Software Foundation Europe has announced its Eighth Birthday. "For eight years now, the Free Software Foundation Europe has been working tirelessly for basic rights and freedoms in an increasingly software-driven society. The 11th of March 2009 sees another major milestone passed, with its 2^3 (eighth) birthday being celebrated by its friends, Fellows and associates."

Full Story (comments: none)

Introducing the Open Source Hardware Central Bank

Here is a proposal for the "Open Source Hardware Central Bank", an organization dedicated to easing the problems associated with bringing open source hardware projects to a successful conclusion. "The Open Source Hardware Bank will work to eliminate the scaling and quantity pricing problem for OSHW projects by funding the build of 2x the quantity of any Open Source Hardware product. That means, if a project has found a way to find 10 potential buyers, the bank will put down the money needed to fund 10 more, for a total of 20 products. If a project has found 25 community members to buy in, the bank will fund another 25, to bring the total quantity down to 50. This should reduce the unit costs by around 10-30% of any hardware project, and in the case of the Illuminato, it'll reduce costs by almost 40%!"

Comments (6 posted)

New Books

Security Monitoring - New from O'Reilly

O'Reilly has published the book Security Monitoring by Chris Fry and Martin Nystrom.

Full Story (comments: none)

Surveys

Python packaging survey

A Python packaging survey has been announced. "The Python Langage Summit is coming up. To prepare this event, I have put online a survey you can take to tell us a bit more about you and how you package your Python applications. * Who should take the survey : any Python developer that packages and distributes his code, no matter how."

Full Story (comments: none)

Calls for Presentations

Final Call for Papers on Cyber Warfare

The final call for papers has gone out for the Conference on Cyber Warfare, which will take place in Tallinn, Estonia on June 17-19, 2009. "Authors should send a one-page abstract to cfp (at) ccdcoe.org by March 15, 2009; the Selection Committee will notify authors of its decisions ASAP following submission but NLT April 1. Final papers are due May 15, 2009."

Full Story (comments: none)

kernel conf australia 2009 call for papers

A call for papers has gone out for kernel conf australia 2009. The conference takes place on July 15-17, 2009 in Brisbane, Australia. Submissions are due by May 1.

Comments (none posted)

LinuxCon CFP submission deadline - April 1st

A call for papers has gone out for LinuxCon 2009. "Reminder - CFP submissions for LinuxCon 2009 are due by Wednesday, April 1st, 2009. LinuxCon is taking place September 21-23, 2009 in Portland, OR and is co-located with the Linux Plumbers Conference. LinuxCon will provide an unmatched collaboration and education space covering all matters Linux, and including everyone in the Linux community including developers, end users, sys admins, community and more."

Full Story (comments: none)

Upcoming Events

FSF announces LibrePlanet 2009 speakers

The Free Software Foundation has announced the latest speaker lineup for the LibrePlanet 2009 conference. The event will take place in Cambridge, MA on March 21-22, 2009. "The conference, to be attended by GNU/Linux users, free software activists, and programmers from around the world, stresses three themes: strengthening global free software activism, addressing the threats posed to free software users by moves toward "cloud computing" and "software as a service," and advancing the projects on the FSF's High Priority Projects list."

Full Story (comments: 3)

Events: March 19, 2009 to May 18, 2009

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
March 16 March 20	Android Bootcamp with Mark Murphy	Atlanta, USA
March 16 March 20	CanSecWest Vancouver 2009	Vancouver, BC, Canada
March 21 March 22	Libre Planet 2009	Cambridge, MA, USA
March 23 March 27	iPhone Bootcamp	Atlanta, Georgia, USA
March 23 March 27	ApacheCon Europe 2009	Amsterdam, The Netherlands
March 23 April 3	Google Summer of Code '09 Student Application Period	online, USA
March 24 March 26	UKUUG Spring 2009 Conference	London, England
March 25 March 29	PyCon 2009	Chicago, IL, USA
March 27 March 29	Free Software and Beyond The World of Peer Production	Manchester, UK
March 28	Open Knowledge Conference 2009	London, UK
March 31 April 2	Solutions Linux France	Paris, France
March 31 April 3	Web 2.0 Expo San Francisco	San Francisco, CA, USA
April 3 April 4	Flourish Conference	Chicago, IL, USA
April 3 April 5	PostgreSQL Conference: East 09	Philadelphia, PA, USA
April 6 April 7	Linux Storage and Filesystem Workshop	San Francisco, CA, USA
April 6 April 8	CELF Embedded Linux Conference	San Francisco, CA, USA
April 8 April 10	Linux Foundation Collaboration Summit	San Francisco, CA, USA
April 14	OpenClinica European Summit	Brussels, Belgium
April 15	Linuxwochen Österreich - Krems	Krems, Austria
April 16 April 17	Nordic Perl Workshop 2009	Oslo, Norway
April 16 April 18	Linuxwochen Austria - Wien	Wien, Austria
April 16 April 19	Linux Audio Conference 2009	Parma, Italy
April 20 April 23	MySQL Conference and Expo	Santa Clara, CA, USA
April 20 April 24	samba eXPerience 2009	Göttingen, Germany
April 20 April 24	Perl Bootcamp at the Big Nerd Ranch	Atlanta, GA, USA
April 20 April 24	Cloud Slam '09	Online, Online
April 22 April 25	ACCU 2009	Oxford, United Kingdom
April 23	Linuxwochen Austria - Linz	Linz, Austria
April 23 April 24	European Licensing and Legal Workshop for Free Software	Amsterdam, The Netherlands
April 23 April 26	Liwoli 2009	Linz, Austria
April 25	Linuxwochen Austria - Graz	Graz, Austria
April 25	Festival Latinoamericano instalación de Software libre	All Latin America, All Latin America
April 25	Grazer Linux Tage 2009	Graz, Austria
April 25 April 26	LinuxFest Northwest 2009 10th Anniversary	Bellingham, Washington, USA
April 25 May 1	Ruby & Ruby on Rails Bootcamp	Atlanta, Georgia, USA
April 27	OSDM 2009	Bangkok, Thailand
May 4 May 6	EuroDjangoCon 2009	Prague, Czech Republic
May 4 May 6	SYSTOR 2009---The Israeli Experimental Systems Conference	Haifa, Israel
May 4 May 7	RailsConf 2009	Las Vegas, NV, USA
May 4 May 8	JavaScript/Ajax Bootcamp at the Big Nerd Ranch	Atlanta, Georgia, USA
May 5	Linuxwochen Austria - Salzburg	Salzburg, Austria
May 6 May 8	Embedded Linux training	Maynard, USA
May 6 May 9	Libre Graphics Meeting 2009	Montreal, Quebec, Canada
May 7	NLUUG spring conference	Ede, The Netherlands
May 8 May 9	Linuxwochen Austria - Eisenstadt	Eisenstadt, Austria
May 8 May 9	Erlanger Firebird Conference 2009	Erlangen-Nürnberg, Germany
May 8 May 10	PyCon Italy 2009	Florence, Italy
May 11	The Free! Summit	San Mateo, CA, USA
May 13 May 15	FOSSLC Summercamp 2009	Ottawa, Ontario, Canada
May 15	Firebird Developers Day - Brazil	Piracicaba, Brazil
May 15 May 16	CONFidence 2009	Krakow, Poland
May 16 May 17	YAPC::Russia 2009	Moscow, Russia

If your event does not appear here, please tell us about it.

Mailing Lists

OpenOffice.org announces new security alerts mailing list

The OpenOffice.org project has a new security alerts mailing list. "The OpenOffice.org Security Team publishes details of security vulnerabilities in their Security Bulletin on the OpenOffice.org web site: http://www.openoffice.org/security/bulletin.html. As an additional service, the Team will now also publish these alerts via a dedicated mailing list security-alerts@openoffice.org. This mailing list will not be used for any other purpose."

Full Story (comments: none)

Audio and Video programs

New netlabel GOSUB10 announces the Substrate release

GOSUB10 has announced the Substrate music compilation. "Dedicated to innovative music and audio/visuals, the GOSUB10 label will feature an eclectic group of musicians drawn together by their shared use of Free/Libre/Open Source Software (FLOSS). Freely distributed by stream, download and special DVD releases, and made available through an open license, GOSUB10 is run by the GOTO10 collective, an international group of artists, musicians and programmers, dedicated to FLOSS and digital arts."

Full Story (comments: none)

Page editor: Forrest Cook