Amarok weathers its own storm of reactions

"It's our KDE 4.2," said Amarok developer Jeff Mitchell. He was referring to version 2.1 of the popular media player, including the recent 2.1.1 maintenance release. He was implying, too, that the Amarok 2.0's reception mirrored that of KDE 4, provoking hostile early reactions that were only quelled as later releases answered the worst of the complaints. It's an analogy that seems fully borne out by a hands-on look at 2.1.1's basic interface and customization features.

The analogy is to the hostile reception that KDE 4.0 received when released in January 2008. Despite warnings that the release was not intended for general use, distributions packaged KDE 4.0 as soon as possible. Almost immediately, users began complaining about the changes in the interface and the lack of features found in earlier releases. These complaints subsided only when the 4.1 and 4.2 releases re-introduced missing features and more customization, although you can still hear rumblings whenever KDE 4 is mentioned online.

"The development process of Amarok 2 has followed a similar path," said Mitchell. Like KDE, "We had visions for where we could go with Amarok, and a code base that had sprawled from Amarok's much more limited and simple beginnings and was in no shape to take us there. So, much like KDE 4, Amarok 2 was not a simple porting of Amarok to Qt4 and KDE4libs; it was an almost total rewrite."

A major part of this rewrite was a transition to the new technologies of KDE 4.0. The switch to Phonon, KDE's multimedia API, allowed Amarok to stop maintaining its own media engines and, in 2.1, to add a graphic equalizer to the settings dialogue. In much the same way, Amarok now includes an embedded version of Plasma, KDE's core desktop technologies. This change allows the context features in Amarok's middle pane — for instance, the lyrics and artist information drawn from the web — to be written as Plasma applets. The integration of such KDE 4 technologies improves performance and reduces the code base that needs to be maintained while adding new functionality to the KDE 4 technologies themselves.

In total, Amarok 2 added some 200,000 lines of code. Since neither the project, nor any distributions, packaged the pre-releases, Mitchell admitted that the additions were not thoroughly tested. However, the project decided to release in December 2008, mainly because "motivation really tends to falter when you develop with no end in sight. Releasing 2.0 invigorated us, since we could finally get out of our endless feature freeze and commit huge new chunks of functionality, [and] get more bug reports and patches sent in."

The downside of releasing 2.0 was a reception almost identical to that received by KDE 4.0 — a similarity that commenters picked up on almost immediately. If you do an Internet search, you will find no shortage of complaints about the change of interface from the previous 1.4 release. Complaints about a missing playlist editor, statistics, sound equalizer, and other features were almost as common.

This reception was not completely unexpected by the project, according to Mitchell. Released on 4 June 2009, version 2.1 was intended "to be our first user release," Mitchell explained. "So far, this prediction has come true. We, the developers, feel it is far more polished and that it is ready for general consumption; users have responded in kind, reversing the scads of negative 2.0 reviews with (as far as we've seen) generally very positive 2.1 reviews."

He admitted that the interface continues to receive complaints and might need work. On the whole, though, Mitchell described the Amarok team as already looking towards new developments, such as using Solid, KDE's device integration framework, for detecting portable media, and integrating with Strigi, KDE's desktop search daemon. Other upcoming features will include breadcrumb navigation and new context applets for YouTube and Flickr.

The new interface

For those who want to test Mitchell's assertions for themselves, Amarok 2.1.1 is already available for most major distributions and many minor ones. In many cases, though, you need to look outside the official repositories to find it. It is available, for example, from Launchpad for Kubuntu, and the Experimental repository for Debian.

Those used to the 1.4 Amarok window are likely to find the changes in the interface overwhelming at first. Rather than displaying sources and auxiliary information in a single pane on the left, Amarok 2.1.1 now spreads the same information over the first two panes on the left, leaving only the third for playlist tracks.

This layout is not ideal from one perspective. In the playlist, the artist and album are listed as a header, rather than repeated for each track, which can be inconvenient in a long list. Similarly, track details and collection sorting are available only from context menus that at first you might miss.

However, from another perspective, the layout removes unnecessary information while making it still available. I am assuming, of course, that most people are already familiar with the artist and album, and only occasionally want to look at cover art.

Moreover, the middle context pane, with its applets in tabs along the bottom, delivers far more information than 1.4, including lyrics and Wikipedia information. The applets are not always perfect — for example, if the Wikipedia applet hits a disambiguation page first, clicking the correct link opens it in your web browser rather than in Amarok. But the applets are welcome all the same, and with these additions, some information had to give way.

On the whole, I think that Amarok has prioritized the information correctly, even if you do risk occasionally overlooking a feature. Besides, if you are uninterested in the context information, you can always adjust the size of the panes so that the middle one is hidden and Amarok looks more as it did in version 1.4

Advanced and customizing features

One feature that you might overlook at first is the PopUp Dropper (PUD). If you drag a track from the collection pane to the playlist, PUD appears as the cursor moves across the middle pane. If you drop the track on Append to Playlist or Replace Playlist, that action is immediately carried out. However, you can also hover the cursor over More, and PUD's menu expands to reveal less common options. Although the feature may not be strictly necessary, it's a convenience that I wouldn't mind seeing in ordinary file managers — especially when using a small screen like those on a netbook, where it could eliminate the need to scroll.

Other features that are tidied away until you discover them expand Amarok's basic functionality. Depending on how your distribution ships Amarok, you might be able to enable more Internet Services from Settings -> Configure Amarok -> Internet Services. These services are more numerous and more varied than in version 1.4.

For still more variety, you can check Tools -> Script Manager to add extra features to the basic installation. The available scripts include one to to stream audio books from Librivox, as well as a service that lists Internet radio stations. Click the Get More Scripts button, and you can download additional scripts, such as local audio streams, as well as applets or extra functionality, such as a musical alarm and a random album player. Activating these scripts will generally require restarting Amarok.

A new feature in 2.1.1 is the ability to created biased playlists from a random collection. Using tracks' metatags, you can set the proportion of certain characteristics that you want. For example, you might want 25% of the playlist to include a certain artist, or 10% of a particular album. Alternatively, you might use the Fuzzy Bias feature so that the list includes only songs that are less three minutes long, or were released in a certain year. With such features, Amarok seems more customizable than ever.

In a nut shell

While some features from earlier releases are still missing in 2.1.1, whether you miss them is largely a matter of preference. Some might miss the setup wizard. Others might miss the ability to use a database other than MySQL, although, since the rest of KDE uses MySQL, the restriction seems natural enough when you consider the latest version's tighter integration into KDE. As for the ability to use a credit card to buy tracks from Magnatune, that is a service discontinued by Magnatune, not Amarok. Probably other features are missing as well, but, if there are any major ones, I failed to notice them in five days of living with the new release.

What I did notice was that the new interface contains more information, and makes controls more noticeable. An obvious example is the enlargement and movement of the basic playing controls to the upper left corner from the lower right.

I also appreciate the context information applets, and the way that advanced functionality and customization options are readily available, but not immediately obvious. With this arrangement, you can discover advanced features gradually, instead of being overwhelmed by complexity when you open the application for the first time.

Being asked to adjust to the unfamiliar is always difficult. Probably most users will have one or two complaints about the new Amarok. Mine are the notification window that pops up when a new track starts and Amarok does not have the focus, and the noticeable lag before the Stop button has an effect.

However, if you can put aside the preconceptions based on earlier versions, then you will probably conclude that Amarok's developers are right: With version 2.1.1, Amarok really has left most of its problems behind. Another release or two, and it should regain its former popularity, and relegate the often hostile reception of version 2.0 to a fading memory.

Comments (17 posted)

Valencia moves to free software

Over the last decade or so there have been multiple reports of governments or agencies making the switch to free software. Some have been relatively successful, like Munich—though not without some bumps along the way—others have been less so. A recent report [PDF], from a ministry of the Valencia Autonomous Community in Spain, provides a nice look inside the transition to free software. It will be a helpful guide for other organizations who are thinking about making the switch.

The Ministry of Infrastructure and Transport (CIT) started to look into free software in 2003: "This change in commercial software sales strategy, together with the CIT's policy of having legal licences for all our users, meant a considerable increase in licence costs which became unsustainable as the majority of our budget went on acquiring these licences." So, the Organization and Computing Department—headed by Martín García Hernández, who wrote the report's introduction—made a proposal to switch all of CIT's systems to free software. The proposal was accepted in September of 2003, and, after a feasibility study, the gvPONTIS project was launched in January 2004.

While it is not an enormous organization, CIT does employ around 1000 people, in a variety of tasks, with 600 administrative employees and 400 engineers and architects. This diversity meant that there were various kinds of software, from office applications to GIS and CAD systems, that required migration. In the process, gvPONTIS also created two new free software projects, gvSIG and MOSKitt, that clearly demonstrate its understanding of the advantages of free software.

Hernández noted that the biggest problem gvPONTIS faced was the "fear of change". This is a common problem encountered when switching users from a familiar environment to something new—not just for software. But, the project had a plan: "In our case, we have faced up to the challenge with well-laid plans, training and an alternative plan of action just in case."

The report is quite detailed in the steps gvPONTIS had to take, the applications and infrastructure that it needed to migrate, as well as the tools it used to get the job done. While the specifics of CIT's environment are unlikely to be replicated elsewhere, the decisions and thought processes that went into the migration will be applicable to other organizations considering a transition to free software.

Moving from existing Access and Oracle databases was eased by using UnixODBC to connect from applications, such as OpenOffice, to those existing databases. But, new development was targeted for a free software alternative. After comparing Interbase, MySQL, and PostgreSQL, it was determined that the latter best fit the needs of the project. The report has a detailed account of the switch to PostgreSQL and the difficulties encountered. There are, of course, still problem areas, which the report clearly indicates, for example:

The project took a pragmatic approach, by continuing to use the existing applications (many of which ran on Windows or other proprietary systems, like Oracle) as it worked towards alternative free software solutions—even if it had to develop its own. Virtualization with VMWare was used to provide access to some Windows applications, for example.

After looking at SUSE 9.0 for the office desktops, the project decided to use the LliureX distribution for which there was local support. LliureX is an education-oriented distribution that was created by the Valencia government and is based on Ubuntu. Standard free software tools replaced much of the proprietary applications for web browsing, email, and office tasks.

One of the biggest successes of the project is the gvSIG GIS/CAD package that was developed for the project. By having a free software solution—not licensed per seat—it allowed more users to access the GIS and CAD capabilities. Instead of 90 GIS/CAD users, there are 400 now. The project has also gained quite a following outside of Valencia, so several pages of the report outline the spread of gvSIG usage throughout the world.

There is great deal more to the report, and it is well worth a read for anyone interested in free software migration. There is information on the networking environment, servers, document management system, CASE modeling tools, and more, from the perspective of moving to free solutions. It clearly shows what a dedicated organization can accomplish if it is truly important to migrate to free software. It undoubtedly had its bumps as well, but Hernández sums it up nicely:

As long as proprietary software companies cling to per-seat licensing and proprietary data formats, there is likely to be more of this kind of migration over time. Organizations that are interested in doing so need to realize that it cannot happen overnight. It took decades of building up their in-house systems, so it will certainly take some time to migrate away from them as well. It would seem that Valencia has provided a nice road map for one way to get there.

[ Thanks to Ismael Olea for pointing us to this story. ]

Comments (4 posted)

Why OpenAerialMap failed where OpenStreetMap succeeded

OpenStreetMap (OSM) is an ambitious project: creating a crowd-sourced, user-editable, free vector map of the world's roads, footpaths, and trails. Despite the scale of the task it has been remarkably successful, with more than 100,000 registered users and more than 29,000,000 ways. In 2007, a group of interested OSM users set out to do the same thing for raster-based satellite and aerial photography that OSM did for road data — but that project, OpenAerialMap (OAM), never reached the same level of success, and closed its doors in December of 2008. Looking back, OAM's creator finds both technical and social causes for the project's fate, and lessons for the future.

Christopher Schmidt launched OAM in hopes of building a free replacement for the copyrighted satellite image layer used by Google Maps. The OSM project had struck a deal with Yahoo to permit usage of Yahoo's aerial image layer in the Potlatch map editor, but only for the purpose of editing GPS traces against a satellite background image: the data was to remain Yahoo's and it could not be displayed in the OSM web map. A sister project, OpenTopoMap, attempted to do the same thing for topographic map data, using a similar methodology, but Schmidt put the greater effort into OAM due to its presumably wider user-appeal.

OAM's initial map set was a Landsat layer donated by i-Cubed. Where aerial photography was available, OAM would support that instead thanks to its greatly increased resolution. The project chose to store its map tiles as uncompressed GeoTIFF images in the standardized EPSG:4326 map projection; that way, although more storage space was required than for a compressed format, the openaerialmap.org server could serve up map tiles without worrying about the CPU overhead of converting every tile served.

By the middle of 2008, however, things had started to slow down. Although there were scores of free, often public-domain aerial data sets, they were not organized or easy to work with: many in the U.S. belonged to individual state or county government projects, running on legacy systems — if online at all — in different projections, and in inconvenient file formats like the proprietary MrSID. On top of that, the data began to grow faster than the disk space on the donated server could keep up with.

The technical dimension

By the time Schmidt decided to pull the plug on the project, he had uploaded ten terabytes of map data, which was only a fraction of what was available — the U.S. Geological Survey (USGS) generously offered to donate its entire aerial data set of the United States, which would have taken up more than 100 terabytes of compressed storage.

It is possible that a benefactor could have been found to donate storage hardware, but that was not the only technical hurdle. Converting the data sets from their original form into the standard adopted by OAM was a time-consuming process, and one that varied from data set to data set, Schmdit said. "We wanted to have large mosaics, EPSG:4326 projected, as GeoTIFFs. Building that kind of tileset out of hundreds, or thousands, of small MrSID files requires significant resources. Just loading MrSID itself can be a beast since MrSID is closed source and not built into many tools."

MrSID (mutli-resolution seamless image database) is a wavelet-based geographic image file format, created and patented by map software company LizardTech. Each file can be extracted in multiple resolutions and quality levels, and sub-blocks of the image can be extracted without reading the entire file. The format is popular in commercial applications, but there is no open source implementation. Several free tools can decode MrSID to convert it to other formats.

In contrast to the process required to convert a MrSID archive to GeoTIFF, OpenStreetMap map-making requires the comparatively straightforward conversion of GPS traces to OSM ways. From the beginning of OAM, Schmidt undertook the lion's share of this "technical gruntwork" himself, rather than building software tools to automate the process for end users. In retrospect, he reflected, that decision severely hindered the development of a community around the project. "The system was simply not designed for users, because it was kind of a one man show. My lack of delegation skills is apparent in many of the things I do :)." A handful of users got up to speed on the conversion process, he added, but most simply gave pointers to available data sets and waited for someone else to take care of them.

The social dimension

Without easy-to-use conversion tools, the number of contributors to the project remained low, but, Schmidt said, the task of building a raster graphics map did not lend itself towards community participation either. In OSM, large data set imports are rare; the majority of the contributions come from individual users collecting GPS traces of their local surroundings, and editing their local map.

In spite of having similar goals, OAM found itself in almost the exact opposite situation. "Aerial imagery gathering doesn't lend itself to crowdsourcing the same way vector data gathering does," Schmidt said, "Almost all of your data is going to be big dumps from state agencies and the like." A few users collected their own aerial photos with kites or drones, he said, but their images amounted to just a tiny fraction of the total.

"I think the important thing is to let people take ownership of what they're doing," Schmidt remarked. Without the level of direct participation found in the OSM project, users did not feel like they "owned" the final product. "Looking back, I can see a lot of places to build community, but at the time, I wasn't really interested in building a community, and by the time that I realized I was burnt out, there was already an expectation that things would just 'magically happen'." Consequently, when Schmidt decided he was burnt out, there was no one else ready to take on the task of stewarding the project.

The next level

Although he switched off the openaerialmap.org tile server in December 2008, Schmidt has not stopped thinking about OAM, and how to grow a community around it. "The desire for easy to find an access aerial imagery is definitely there. The continued interest in OAM even after the web frontend essentially stopped working speaks to that, and the continued attempts to participate even with terrible documentation, poor setup, etc."

He is considering re-launching the project with a different set of goals and different ways of attracting user participation, placing less emphasis on serving as a single-source repository of image tiles. Instead, OAM could function like a directory for the existing aerial imagery map sets hosted elsewhere, like the approach taken by Web Map Service (WMS) directory wms-sites.com. Such a service would be almost as useful as serving the tiles themselves, he said, since so much of the difficulty now is in finding the data set for a particular area.

Building a wiki-like directory that points to data hosted elsewhere might also allow users to build "ownership" of their contributions, Schmidt added. "The imagery is only a first step towards the end result. Does the imagery have metadata about it? Is it aligned properly? Do people know how to find it?" Most often, he said, the only way to find a data set is search the web, and then the results have no user interface, "just an open FTP site linked from one page under 'GIS Data'."

The mapping agencies are usually helpful, he said, but do not have the manpower or IT resources to run a WMS or tile server. "This is why I'm thinking a catalog approach might work: if you let the people who maintain this imagery at places like MassGIS contribute to a site information *about* the imagery — concentrating on things like quality, capture dates, etc. — you can have them have a product they can feel like they're the owner of."

Although he has not taken any formal steps toward this goal, it is undeniable that there is latent interest in resurrecting the OAM project in some form. The map server is down, but the OAM wiki remains up, and other users have started a discussion page to brainstorm ideas for the future of the project. Its main archive is off-line, but people continue to find the mailing list via third-party sites and write in to ask questions.

Rarely does an open source software project that has failed to get off the ground take a serious look at why — although the benefits of doing so are clear. Schmidt's assessments of what went wrong with the first incarnation of OAM will help make the next effort more technically prepared, and better matched to the user community it needs in order to build up critical mass and thrive. Similarly, looking back at how the practical differences between raster and vector mapping led OAM down a completely different path than OSM (in spite of their seemingly similar goals) is an object lesson other projects would be wise to study as well.

Comments (1 posted)

Apache attacked by a "slow loris"

The slow loris is an exotic animal of southeast Asia that is best known for its slow, deliberate movements. This characterizes the technique used by a new Denial of Service (DoS) tool that has been named after the animal. Slowloris was released to the public by security researcher "RSnake" on June 17. Unlike previously utilized DoS methods, slowloris works silently. Still, it results in a quick and complete halt of the victim's Apache web server.

The release of slowloris was only done after RSnake had contacted the Apache security team. Their response, while quick, was not quite what he expected:

RSnake commented that this response misses the point completely and that the security tips advertised are of no help. Subsequently, he released the slowloris script, which was followed by a confusing discussion that ranged over multiple blog postings, comments on the postings, as well as various mailing lists. On one side are those hard-boiled experts that say they knew about this technique for years and that it is nothing new. On the other side are those who think this is genuinely new or at least new to the public, and that it could have a devastating effect on the internet as a whole or at least on the half of the world wide web that runs on Apache. Another Internet Storm Center (ISC) post provides more context, along with some useful comments.

However, the majority seems to be stunned by the simplicity of the attack and the fatal effect of it, as well as being puzzled by the reaction of the Apache security team. The team's response makes it seem as if the slowloris attack is well-known, leaving Apache installations vulnerable to DoS by script kiddies, and that there is nothing the Apache developers can do to prevent it. Consequently, they also closed the bugzilla report.

One particular commenter expressed his concern on the full disclosure mailing list with the following words:

It is not entirely clear which web servers have the means to defend against the attack, but there is general agreement that there is no way for Apache to completely defend against it, and that IIS is not vulnerable to the slowloris technique.

Looking more closely at the slowloris script provides an overview of the technique used. Slowloris gives the attacker a simple way to open an HTTP session with a server and to keep it open for a very long time — a lot longer than it would usually stay open: minutes or even hours.

The way the script achieves this goal can be likened to a person at a checkout lane in a store. Everyone has encountered someone paying the cashier, one by one — literally — in pennies. This can take time; often it feels like it is taking forever.

To the company that has a chain of several stores, this random person does not affect its business. For an online shop, however, there is a single URL and slowloris unleashes hundreds if not thousands of these people approaching the checkout lane with an endless supply of pennies ready to block the queue. For HTTP, slowloris uses HTTP headers instead of pennies, and it is ready to add a new header every 5, 10, or 299 seconds.

Unfortunately, the Apache at the cashier has no memory. With every penny dropped in its hands, it resets the timeout counter. With this technique it is rather simple to block every server thread or prefork process and bring the web server to a complete halt. Because the default timeout setting for Apache is 300 seconds, each header added can stretch things out for that much longer.

An unfortunate side effect of this attack method is that the access log of the web server will not show it is under attack. Also, the messages in the error log are likely to be sparse. The CPU will be idle, no disk IO will be done, and there will also be hardly any network traffic to be seen. All you can observe is a large number of open network connections in the ESTABLISHED state.

Obviously, this is an application level attack. In their book on Internet Denial of Service, Mirkovic/Dietrich et. al noted that application level DoS is difficult to handle: "[...] many defenses are not able to help you defend against this kind of attack".

So we are back to what the Apache Security team concluded: This is an inherent problem for servers. If you want to serve, then you have to accept clients, and, if they intend to block you, so be it.

But, let's not give up so fast. Obviously, if the well-known proprietary alternative from Microsoft, IIS, is not affected by this problem, there are other solutions. What IIS does differently, is in the way it handles incoming requests: There is no static tie between a worker thread and a network socket in IIS. Rather, the workers are organized in a pool where they wait for incoming TCP packets (rather than TCP connections as Apache does). These packets are then assigned dynamically to threads. So, an idle connection occupies a socket, but it does not block an entire thread. Thus the web need not be shut down by penny-wielding customers or slowloris.

Some developers pointed out that the AcceptFilter directive used in conjunction with FreeBSD's kernel http_accept filter should be ported to Linux in order to help with the defense. But, the http_accept filter only works with clear text traffic, so this defense will not work for business-critical services running over SSL (i.e. HTTPS).

It has already been noted that you can catch the single attacking IP address with netstat and block it via your firewall. Or you can use any of the Apache modules that limit the number of sockets allowed for a single IP address. Mod_qos seems best suited for this purpose. But this could block proxies or NAT routers that bundle multiple clients onto a single IP address. A threshold of 30 or even 100 sockets is nothing that should pose any problems to the clients behind these proxies, unless those sites are truly huge. Limiting connections to some threshold would help guard the server. But slowloris is only a proof of concept of a HTTP DoS. It could easily be extended into a HTTPS Distributed Denial of Service (DDoS) attack of a far nastier nature.

So it seems slowloris does not use the full potential of the attack method. The method can also be used more generally, which is why the name does not quite fit. The slow loris is an exotic animal, but the technique the script uses is not really exotic. It is very natural indeed: it attempts to delay an attack from the client side like slow internet connections used to do. So, if there is a term that describes the general form of the attack, then it should be "Request Delaying Attack".

In a follow-up blog post, RSnake has extended the concept to use keep-alive requests, and then delaying any subsequent requests. Other techniques are possible. They are described in great detail in a message to the ModSecurity mailing list from 2006.

Still, the attack has not yet been publicly observed in the wild, and there are still many experts that consider DDoS a non-issue.

RSnake did not claim to have invented the idea, in fact he points out that Ivan Ristić had described it in his book on Apache Security in 2005. Furthermore, posts in the various mailing lists suggest the concept has been around since the 90s, but RSnake has given this simple technique a wider audience.

Now that there is an audience, it is disturbing to see that the Apache community has so little to say about possible defenses. There has been some discussion on how to handle it, but overall the market leader seems rather complacent these days.

Comments (68 posted)

amule: insufficient input sanitizing

Package(s):	amule	CVE #(s):	CVE-2009-1440
Created:	June 23, 2009	Updated:	September 9, 2009
Description:	From the Debian advisory: Sam Hocevar discovered that amule, a client for the eD2k and Kad networks, does not properly sanitise the filename, when using the preview function. This could lead to the injection of arbitrary commands passed to the video player.
Alerts:	Gentoo 200909-06 amule 2009-09-09 Debian DSA-1821-1 amule 2009-06-22

Comments (none posted)

ctorrent: buffer overflow

Package(s):	ctorrent	CVE #(s):	CVE-2009-1759
Created:	June 18, 2009	Updated:	November 20, 2013
Description:	ctorrent has a buffer overflow vulnerability. From the Debian alert: Michael Brooks discovered that ctorrent, a text-mode bittorrent client, does not verify the length of file paths in torrent files. An attacker can exploit this via a crafted torrent that contains a long file path to execute arbitrary code with the rights of the user opening the file.
Alerts:	Gentoo 201311-11 ctorrent 2013-11-20 Fedora FEDORA-2009-8897 ctorrent 2009-08-25 Fedora FEDORA-2009-8969 ctorrent 2009-08-25 Debian DSA-1817-1 ctorrent 2009-06-17

Comments (none posted)

gforge: multiple vulnerabilities

Package(s):	gforge	CVE #(s):
Created:	June 24, 2009	Updated:	June 24, 2009
Description:	Laurent Almeras and Guillaume Smet have discovered a possible SQL injection vulnerability and cross-site scripting vulnerabilities in gforge, a collaborative development tool. Due to insufficient input sanitising, it was possible to inject arbitrary SQL statements and use several parameters to conduct cross-site scripting attacks.
Alerts:	Debian DSA-1818-1 gforge 2009-06-18

Comments (none posted)

mahara: insufficient input sanitization

Package(s):	mahara	CVE #(s):
Created:	June 23, 2009	Updated:	June 24, 2009
Description:	From the Debian advisory: It was discovered that mahara, an electronic portfolio, weblog, and resume builder is prone to several cross-site scripting attacks, which allow an attacker to inject arbitrary HTML or script code and steal potential sensitive data from other users.
Alerts:	Debian DSA-1822-1 mahara 2009-06-23

Comments (none posted)

moin: XSS vulnerability

Package(s):	moin	CVE #(s):	CVE-2008-3381
Created:	June 18, 2009	Updated:	June 24, 2009
Description:	moin has cross site scripting vulnerabilities. From the National Vulnerability Database entry: Multiple cross-site scripting (XSS) vulnerabilities in macro/AdvancedSearch.py in moin (and MoinMoin) 1.6.3 and 1.7.0 allow remote attackers to inject arbitrary web script or HTML via unspecified vectors.
Alerts:	Fedora FEDORA-2009-6557 moin 2009-06-18 Fedora FEDORA-2009-6559 moin 2009-06-18

Comments (none posted)

pcsc-lite: world writable subdirectory

Package(s):	pcsc-lite	CVE #(s):
Created:	June 19, 2009	Updated:	June 24, 2009
Description:	pcscd creates a world writable subdirectory. See the Red Hat bugzilla for details.
Alerts:	Fedora FEDORA-2009-6695 pcsc-lite 2009-06-19

Comments (none posted)

ruby: denial of service

Package(s):	ruby	CVE #(s):	CVE-2009-1904
Created:	June 22, 2009	Updated:	February 16, 2010
Description:	From the CVE entry: The BigDecimal library in Ruby 1.8.6 before p369 and 1.8.7 before p173 allows context-dependent attackers to cause a denial of service (application crash) via a string argument that represents a large number, as demonstrated by an attempted conversion to the Float data type.
Alerts:	Fedora FEDORA-2010-0533 ruby 2010-01-14 Fedora FEDORA-2009-13066 ruby 2009-12-11 Mandriva MDVSA-2009:325 ruby 2009-12-07 Ubuntu USN-900-1 ruby1.9 2010-02-16 Debian DSA-1860-1 ruby1.8 2009-08-12 Mandriva MDVSA-2009:177 ruby 2009-07-29 Mandriva MDVSA-2009:160 ruby 2009-07-27 Ubuntu USN-805-1 ruby1.8, ruby1.9 2009-07-20 CentOS CESA-2009:1140 ruby 2009-07-02 Red Hat RHSA-2009:1140-02 ruby 2009-07-02 Gentoo 200906-02 ruby 2009-06-28 Slackware SSA:2009-170-02 ruby 2009-06-22

Comments (none posted)

vlc: integer overflows

Package(s):	vlc	CVE #(s):	CVE-2008-4686
Created:	June 18, 2009	Updated:	June 24, 2009
Description:	vlc has several integer overflow vulnerabilities. From the National Vulnerability Database entry: Multiple integer overflows in ty.c in the TY demux plugin (aka the TiVo demuxer) in VideoLAN VLC media player, probably 0.9.4, might allow remote attackers to execute arbitrary code via a crafted .ty file, a different vulnerability than CVE-2008-4654.
Alerts:	Debian DSA-1819-1 vlc 2009-06-18

Comments (none posted)

Kernel release status

The current 2.6 development kernel is 2.6.31-rc1, released by Linus on June 24. "There's a lot in there, but let me say that as far as the whole merge window has gone, I've seldom felt happier merging stuff from people. I'm really hoping it isn't just a fluke, but people kept their git trees clean, and while there were a couple of times that I said "no, I'm not going to merge that", on the whole this was a really painless merge window for me." Linus did say that he is likely to still merge the S+Core architecture (score) as it only touches the MAINTAINERS file outside of its tree. The significant changes for 2.6.31 include performance counters, character devices in user space (CUSE), kernel modesetting for Radeon hardware, kmemleak and kmemcheck, fsnotify (which provides a common implementation for dnotify and inotify), along with a vast quantity of drivers of various sorts. See the long-format changelog for all the details.

There have been no stable updates in the last week, nor are there any stable patches out for review.

Comments (none posted)

Quotes of the week

-- Matthew Garrett (the entire post is worth reading)

-- Greg Kroah-Hartman gives an update on the -staging tree

-- Linus Torvalds

Comments (1 posted)

2.6.31 merge window, week 2

There have been around 2050 non-merge changesets merged into the mainline since last week's article, bringing the total to 8288 changes merged for 2.6.31. The merge window has closed, so all major features of 2.6.31 (with the possible exception of the S+Core architecture) will have been merged. Interesting changes since last week that are user visible include:

• The MIPS architecture has added support for hugetlbfs, as well as hibernation support (but only for uni-processor systems).

• The SLUB page allocator has added new diagnostic information printk()s for debugging OOM conditions.

• A /proc/softirqs file has been added to show the number of software interrupts for each CPU. Also, a "softirq" line has been added to /proc/stat.

• The gcov profiling infrastructure, for code coverage testing, has been merged. It adds functions needed by the profiling code, kbuild support for building kernels with gcov profiling, along with a debugfs interface to retrieve the profiling data.

• An API for pulse-per-second (PPS) devices has been added. These are devices which provide a high-precision signal that can be used to adjust system clock time.

• The EXT4_IOC_MOVE_EXT ioctl() has been added to support ext4 online defragmentation.

• A sysfs interface to add I2C devices, which takes the place of various force_* module parameters, has been merged.

• A command stream checker for Radeon r3xx-r5xx hardware has been added to stop user-space processes from accessing memory outside of what they own.

• The perf tool has added multiple features, including raw data output as well as call graph profiling.

• The PowerPC architecture has added support for software performance counters.

• PCI end-to-end CRC checking (ECRC) can now be enabled or disabled with the ecrc boot parameter.

• A PCI Express Advanced Error Reporting (AER) software error injector has been merged.

• NFS version 4.1 client support has been added as an experimental feature. Server support for 4.1 is, as yet, not merged.

• Firewire (IEEE 1394) now has support for IPv4 networking.

• New device drivers

  • Architectures/processors/systems: Keymile KMETER1 PPC boards, X-ES Freescale MPC85xx-based single-board computers, Palm Treo 680 smartphones, Openmoko GTA02 / Freerunner phones, MINI2440 ARM-based development boards.

  • Network: Xtensa S6105 GMAC ethernet devices.

  • Input devices: TI DaVinci DM355 EVM keypads and IR remotes, TWL4030 Power buttons, WM97xx Atmel accelerated touchscreens, LM8323 keypad chips, W90P910 touchscreens, EETI touchscreen panels, Synaptics I2C touchpads,

  • Miscellaneous: Toshiba TXx9 SoC DMA controllers, TX4939 hardware random number generators, ST-Ericsson AB3100 Mixed Signal circuits (core functionality needed for other AB3100 devices), PCAP ASIC for EZX phones (needed to support other devices), Epson RX-8025SA/NB real-time clocks, IBM CPC925 PPC Memory Controller, PrimeCell PL061 GPIO devices, TI DaVinci DM355 EVM Keypad and IR remote devices, VIA SD/MMC card readers, MSM7K onboard serial devices, NAND Flash devices for OMAP2 and OMAP3, Broadcom BCM47xx watchdog timers, PNX833x hardware watchdog timers, TWL4030 watchdog timers, ST-Ericsson COH 901 327 watchdog timers, Freescale STMP3XXX watchdog timers, FibreChannel ELS/CT pass-thru support, Synopsys DesignWare I2C adapters, Maxim MAX17040 Fuel Gauge batteries.

  • Staging: Cavium Networks Octeon ethernet ports, CPC CAN USB driver, USB Quatech ESU-100 8 Port Serial Driver (as serqt_usb2, replacing the obsolete serqt_usb staging driver), RDC_17F3101X IDE devices, Displaylink USB framebuffer devices, Realtek RTL8192 USB wifi devices.

Changes visible to kernel developers include:

• Quite a bit of Big Kernel Lock (BKL) removal code has been merged in the fs/ tree. Now, all of the super_operations and address_space_operations are called without holding the BKL.

• IRQF_SAMPLE_RANDOM, which governs whether a driver's interrupts are used as an entropy source, has been added to the feature-removal-schedule.

• The memory debugging infrastructure for DRM has been removed. "It hasn't been used in ages, and having the user tell your how much memory is being freed at free time is a recipe for disaster even if it was ever used."

• David Miller is now the IDE subsystem maintainer, taking over from Bartlomiej Zolnierkiewicz, in a friendly handoff. Miller plans to put IDE into maintenance-only mode.

• The SCSI device information matching has added support for multiple blacklist tables.

• The instrumentation of jbd2 and ext4 has been converted from kernel markers to tracepoints.

• OCFS2 has added support for lockdep, by adding the proper lockdep annotations for all of the cluster locks except those that are acquired for a node, rather than a process.

• Access control list (ACL) information is now cached in struct inode for some filesystems (jfs, ext2, ext3, ext4, jffs2, btrfs, reiserfs, nilfs2, xfs).

Since the merge window has closed, the next step is stabilization. Something approaching 3000 more changes will likely make their way into the mainline before the 2.6.31 release, which should happen in late August or early September.

Comments (5 posted)

Protected RAMFS

Many embedded systems have a block of non-volatile RAM (NVRAM) separate from normal system memory. A recent patch, posted by Marco Stornelli, is a filesystem for these kinds of NVRAM devices, where the device could store frequently accessed data (such as the address book for a cellphone). Protected RAMFS (PRAMFS) protects the NVRAM-based filesystem from errant or stray writes to the protected portion of the RAM caused by kernel bugs. Because it is stored in the NVRAM, the filesystem can survive a reboot, and hence can also be used to keep important crash information.

Basic Features

PRAMFS is robust in the face of errant writes to the protected area, which could arise due to kernel bugs. The page table entries that map the backing-store RAM are marked read-only on initialization. Write operations to the filesystem temporarily mark the pages to be written as writable, the write operation is carried out with locks held, and then the pte is marked read-only again. This limits the writes to the filesystem in the window when the locks are held. The write-protection feature can be disabled by the kernel config option CONFIG_PRAMFS_NOWP.

PRAMFS forces all files to use direct-IO. The filp->f_flags is set to O_DIRECT when the files are opened. Opening all files as O_DIRECT avoids page caching, and data is written immediately to a storage device. This is nearly equal to the speed of the system RAM, but it forces applications to do block-aligned I/O.

PRAMFS does not have recovery facilities, such as journaling, to survive a crash or power failure during a write operation. The filesystem maintains checksums for the superblock and inode to check the validity of the stored object. An inode with an incorrect checksum is marked as bad, which may lead to data loss in case of power failure during a write operation.

PRAMFS also supports execute in place (XIP), which is a technique that executes programs directly from the storage instead of copying it into RAM. For a RAM filesystem, XIP makes sense since the system can execute from the storage device as fast as it can from the system RAM, and it does not make a duplicate copy in RAM.

Usage

There is no mkfs utility to create a PRAMFS. The filesystem is automatically created when the filesystem is mounted with the init option. The command to create and mount a PRAMFS is:

    # mount -t pramfs -o physaddr=0x20000000,init=0x2F000,bs=1024 none /mnt/pram

This command creates a filesystem of 0x2F000 bytes, with a block size of 1024 bytes, and locates it at the physical address 0x20000000.

To retrieve an existing filesystem, mount the PRAMFS with the physaddr parameter that was used in the previous mount. The details of the filesystem such as blocksize and filesystem size are read from the superblock:

    # mount -t pramfs -o physaddr=0x20000000 none /mnt/pram

Other filesystem parameters are:

• bpi: specifies the bytes-per-inode ratio. For every bpi bytes in the filesystem, an inode is created.

• N: specifies the number of inodes to allocate in the inode table. If the option is not specified, the bytes-per-inode ratio is used to calculate the number of inodes.

If the init option is not specified, the bs, bpi, or N options are ignored by the mount, since this information is picked up from the existing filesystem. When creating the filesystem, if no option for the inode reservation is specified, by default 5% of the filesystem space is used for the inode table.

To test the memory protection of PRAMFS, the developers have written a kernel module that attempts to write within the PRAMFS memory with the intention of corrupting the memory space. This causes a kernel protection fault, and, after a reboot, you may re-mount the filesystem to find that the test module was not capable of corrupting the filesystem.

Filesystem Layout

PRAMFS has a simple layout, with the super-block in the first 128 bytes of the RAM block, followed by the inode table, the block usage map, and finally the data blocks. The superblock is 128 bytes long and contains all of the important information, such as filesystem size, block size, etc., needed to remount the filesystem.

The inode table consists of the inodes required for the filesystem. The number of inodes are computed when the filesystem is initialized. Each inode is 128 bytes long. Directory entry information, such as filename and owning inode, are contained within the inode. This presents a problem for hard links because a hard link requires two directory entries under different directories for the same inode. Hence, PRAMFS does not support hard links. The inode format also limits the filename to 48 characters. The inode number is the absolute offset of that inode from the beginning of the filesystem.

Regular PRAMFS file inodes contain the i_type.reg.row_block field, which points to a data block which contains doubly-indirect pointers to the file's data blocks. This is similar to the double indirect block field of the ext2 filesystem inode. But, that means that a file smaller than 1 block will require 3 blocks to store it.

Inodes within a directory are linked together in a doubly-linked list. The directory inode stores the first and last inode in the directory listing. The previous entry of the first inode and the next entry of the last inode are null terminated.

Write Protection

PRAMFS utilizes the system's paging unit by mapping its RAM initially as read-only. Writes to data objects first mark the corresponding page table entries as writable, perform the write and then mark them read-only again. This operation is done atomically by holding the page-table spin-lock with interrupts disabled. Following a write, stale entries in the system TLB are flushed. Write locks are held at the superblock, inode, or block level, depending on the granularity of modification.

Since PRAMFS attempts to avoid filesystem corruption caused because of kernel bugs, shared mmap() regions can only be read. Dirty pages in the page cache cannot be written back to the filesystem. For this reason, PRAMFS defines only the readpage() member of struct address_space_operations; the writepage() entry is declared as NULL.

Acceptance

This is the second attempt to get PRAMFS in the mainline. The previous attempt was done in 2004 by Steve Longerbeam of Montavista.

The home page of PRAMFS claims the filesystem to be fully-featured. But, as part of the linux-kernel discussion, Henrique de Moraes Holschuh strongly disagreed:

There are not enough performance benchmarks information against other filesystems, yet, to form an opinion. Performance tests done while adding Execute in Place (XIP) reveal a performance as low as 13Mbps for per-character writes and 35Mbps for block writes using bonnie. Pavel Machek considers these numbers to be pretty low, especially for a RAM-based filesystem:

No tests have been performed using existing solutions, such as ramdisk on the same hardware, to compare apples with apples. The low performance is attributed to the excessive locking done for writes. Pavel believes the developers of PRAMFS are confused regarding the goals of the filesystem, and whether they are designing for speed, completeness, or robustness.

PRAMFS is a niche filesystem, mostly for embedded devices with NVRAM, and hence lacks important features, such as hard links and shared mmap()s. However, for quite a number of situations an entire filesystem seems like overkill. Pavel suggests a special NVRAM-based block device with a traditional filesystem or a filesystem based on Solid State Device (SSD) filesystems would be a better option. With the current number of objections, PRAMFS is unlikely to go into the mainline. However, Marco plans to further improve the code with more features, and to update the PRAMFS homepage to better reflect the filesystem's goals.

Comments (5 posted)

Linux kernel design patterns - part 3

In this final article we will be looking at just one design pattern. We started with the fine details of reference counting, zoomed out to look at whole data structures, and now move to the even larger perspective of designing subsystems. Like every pattern, this pattern needs a name, and our working title is "midlayer mistake". This makes it sounds more like an anti-pattern, as it appears to describe something that should be avoided. While that is valid, it is also very strongly a pattern with firm prescriptive guides. When you start seeing a "midlayer" you know you are in the target area for this pattern and it is time to see if this pattern applies and wants to guide you in a different direction.

In the Linux world, the term "midlayer" seems (in your author's mind and also in Google's cache) most strongly related to SCSI. The "scsi midlayer" went through a bad patch quite some years ago, and there was plenty of debate on the relevant lists as to why it failed to do what was needed. It was watching those discussions that provided the germ from which this pattern slowly took form.

The term "midlayer" clearly implies a "top layer" and a "bottom layer". In this context, the "top" layer is a suite of code that applies to lots of related subsystems. This might be the POSIX system call layer which supports all system calls, the block layer which supports all block devices, or the VFS which supports all filesystems. The block layer would be the top layer in the "scsi midlayer" example. The "bottom" layer then is a particular implementation of some service. It might be a specific system call, or the driver for a specific piece of hardware or a specific filesystem. Drivers for different SCSI controllers fill the bottom layer to the SCSI midlayer. Brief reflection on the list of examples shows that which position a piece of code takes is largely a matter of perspective. To the VFS, a given filesystem is part of the bottom layer. To a block device, the same filesystem is part of the top layer.

A midlayer sits between the top and bottom layers. It receives requests from the top layer, performs some processing common to the implementations in the bottom layer, and then passes the preprocessed requests - presumably now much simpler and domain-specific - down to the relevant driver. This provides uniformity of implementation, code sharing, and greatly simplifies that task of implementing a bottom-layer driver.

The core thesis of the "midlayer mistake" is that midlayers are bad and should not exist. That common functionality which it is so tempting to put in a midlayer should instead be provided as library routines which can used, augmented, or ignored by each bottom level driver independently. Thus every subsystem that supports multiple implementations (or drivers) should provide a very thin top layer which calls directly into the bottom layer drivers, and a rich library of support code that eases the implementation of those drivers. This library is available to, but not forced upon, those drivers.

To try to illuminate this pattern, we will explore three different subsystems and see how the pattern specifically applies to them - the block layer, the VFS, and the 'md' raid layer (i.e. the areas your author is most familiar with).

Block Layer

The bulk of the work done by the block layer is to take 'read' and 'write' requests for block devices and send them off to the appropriate bottom level device driver. Sounds simple enough. The interesting point is that block devices tend to involve rotating media, and rotating media benefits from having consecutive requests being close together in address space. This helps reduce seek time. Even non-rotating media can benefit from having requests to adjacent addresses be adjacent in time so they can be combined into a smaller number of large requests. So, many block devices can benefit from having all requests pass through an elevator algorithm to sort them by address and so make better use of the device.

It is very tempting to implement this elevator algorithm in a 'midlayer'. i.e. a layer just under the top layer. This is exactly what Linux did back in the days of 2.2 kernels and earlier. Requests came in to ll_rw_block() (the top layer) which performed basic sanity checks and initialized some internal-use fields of the structure, and then passed the request to make_request() - the heart of the elevator. Not quite every request went to make_request() though. A special exception was made for "md" devices. Those requests were passed to md_make_request() which did something completely different as is appropriate for a RAID device.

Here we see the first reason to dislike midlayers - they encourage special cases. When writing a midlayer it is impossible to foresee every possible need that a bottom level driver might have, so it is impossible to allow for them all in the midlayer. The midlayer could conceivably be redesigned every time a new requirement came along, but that is unlikely to be an effective use of time. Instead, special cases tend to grow.

Today's block layer is, in many ways, similar to the way it was back then with an elevator being very central. Of course lots of detail has changed and there is a lot more sophistication in the scheduling of IO requests. But there is still a strong family resemblance. One important difference (for our purposes) is the existence of the function blk_queue_make_request() which every block device driver must call, either directly or indirectly via a call to blk_init_queue(). This registers a function, similar to make_request() or md_make_request() from 2.2, which should be called to handle each IO request.

This one little addition effectively turns the elevator from a midlayer which is imposed on every device into a library function which is available for devices to call upon. This was a significant step in the right direction. It is now easy for drivers to choose not to use the elevator. All virtual drivers (md, dm, loop, drbd, etc.) do this, and even some drivers for physical hardware (e.g. umem) provide their own make_request_fn().

While the elevator has made a firm break from being a mid-layer, it still retains the appearance of a midlayer in a number of ways. One example is the struct request_queue structure (defined in <linux/blkdev.h>). This structure is really part of the block layer. It contains fields that are fundamental parts of the block interface, such as the make_request_fn() function pointer that we have already mentioned. However many other fields are specific to the elevator code, such as elevator (which chooses among several IO schedulers) and last_merge (which is used to speed lookups in the current queue). While the elevator can place fields in struct request_queue, all other code must make use of the queuedata pointer to store a secondary data structure.

This arrangement is another tell-tale for a midlayer. When a primary data structure contains a pointer to a subordinate data structure, we probably have a midlayer managing that primary data structure. A better arrangement is to use the "embedded anchor" pattern from the previous article in this series. The bottom level driver should allocate its own data structure which contains the data structure (or data structures) used by the libraries embedded within it. struct inode is a good example of this approach, though with slightly different detail. In 2.2, struct inode contained a union of the filesystem-specific data structure for each filesystem, plus a pointer (generic_ip) for another filesystem to use. In the 2.6 kernel, struct inode is normally embedded inside a filesystem-specific inode structure (though there is still an i_private pointer which seems unnecessary).

One last tell-tale sign of a midlayer, which we can still see hints of in the elevator, is the tendency to group unrelated code together. The library design will naturally provide separate functionality as separate functions and leave it to the bottom level driver to call whatever it needs. The midlayer will simply call everything that might be needed.

If we look at __make_request() (the 2.6 entry point for the elevator), we see an early call to blk_queue_bounce(). This provides support for hardware that cannot access the entire address space when using DMA to move data between system memory and the device. To support such cases, data sometimes needs to be copied into more accessible memory before being transferred to the device, or to be copied from that memory after being transferred from the device. This functionality is quite independent of the elevator, yet it is being imposed on all users of the elevator.

So we see in the block layer, and its relationship with the elevator a subsystem which was once implemented as a midlayer, but has taken a positive step away from being a midlayer by making the elevator clearly optional. It still contains traces of its heritage which have served as a useful introduction to the key identifiers of a midlayer: code being imposed on lower layer, special cases in that code, data structures storing pointers to subordinate data structures, and unrelated code being called by the one support function.

With this picture in mind, let us move on.

The VFS

The VFS (or Virtual File System) is a rich area to explore to learn about midlayers and their alternatives. This is because there is a lot of variety in filesystems, a lot of useful services that they can make use of, and a lot of work has been done to make it all work together effectively and efficiently. The top layer of the VFS is largely contained in the vfs_ function calls which provide the entry points to the VFS. These are called by the various sys_ functions that implement system calls, by nfsd which does a lot of file system access without using system calls, and from a few other parts of the kernel that need to deal with files.

The vfs_ functions fairly quickly call directly in to the filesystem in question through one of a number of _operations structures which contain a list of function pointers. There are inode_operations, file_operations, super_operations etc, depending on what sort of object is being manipulated. This is exactly the model that the "midlayer mistake" pattern advocates. A thin top layer calls directly into the bottom layer which will, as we shall see, make heavy use of library functions to perform its task.

We will explore and contrast two different sets of services provided to filesystems, the page cache and the directory entry cache.

The page cache

Filesystems generally want to make use of read-ahead and write-behind. When possible, data should be read from storage before it is needed so that, when it is needed, it is already available, and once it has been read, it is good to keep it around in case, as is fairly common, it is needed again. Similarly, there are benefits from delaying writes a little, so that throughput to the device can be evened out and applications don't need to wait for writeout to complete. Both of these features are provided by the page cache, which is largely implemented by mm/filemap.c and mm/page-writeback.c.

In its simplest form a filesystem provides the page cache with an object called an address_space which has, in its address_space_operations, routines to read and write a single page. The page cache then provides operations that can be used as file_operations to provide the abstraction of a file that must be provided to the VFS top layer. If you look at the file_operations for a regular file in ext3, we see:

    const struct file_operations ext3_file_operations = {
	.llseek		= generic_file_llseek,
	.read		= do_sync_read,
	.write		= do_sync_write,
	.aio_read	= generic_file_aio_read,
	.aio_write	= ext3_file_write,
	.unlocked_ioctl	= ext3_ioctl,
    #ifdef CONFIG_COMPAT
	.compat_ioctl	= ext3_compat_ioctl,
    #endif
	.mmap		= generic_file_mmap,
	.open		= generic_file_open,
	.release	= ext3_release_file,
	.fsync		= ext3_sync_file,
	.splice_read	= generic_file_splice_read,
	.splice_write	= generic_file_splice_write,
    };

Eight of the thirteen operations are generic functions provided by the page cache. Of the remaining five, the two ioctl() operations and the release() operation require implementations specific to the filesystem; ext3_file_write() and ext3_sync_file are moderately sized wrappers around generic functions provided by the page cache. This is the epitome of good subsystem design according to our pattern. The page cache is a well defined library which can be used largely as it stands (as when reading from an ext3 file), allows the filesystem to add functionality around various entry points (like ext3_file_write()) and can be simply ignored altogether when not relevant (as with sysfs or procfs).

Even here there is a small element of a midlayer imposing on the bottom layer as the generic struct inode contains a struct address_space which is only used by the page cache and is irrelevant to non-page-cache filesystems. This small deviation from the pattern could be justified by the simplicity it provides, as the vast majority of filesystems do use the page cache.

The directory entry cache (dcache)

Like the page cache, the dcache provides an important service for a filesystem. File names are often accessed multiple times, much more so than the contents of file. So caching them is vital, and having a well designed and efficient directory entry cache is a big part of having efficient access to all filesystem objects. The dcache has one very important difference from the page cache though: it is not optional. It is imposed upon every filesystem and is effectively a "midlayer." Understanding why this is, and whether it is a good thing, is an important part of understanding the value and applicability of this design pattern.

One of the arguments in favor of an imposed dcache is that there are some interesting races related to directory renames; these races are easy to fail to handle properly. Rather than have every filesystem potentially getting these wrong, they can be solved once and for all in the dcache. The classic example is if /a/x is renamed to /b/c/x at the same time that /b/c is renamed to /a/x/c. If these both succeed, then 'c' and 'x' will contain each other and be disconnected from the rest of the directory tree, which is a situation we would not want.

Protecting against this sort of race is not possible if we only cache directory entries at a per-directory level. The common caching code needs to at least be able to see a whole filesystem to be able to detect such a possible loop-causing race. So maintaining a directory cache on a per-filesystem basis is clearly a good idea, and strongly encouraging local filesystems to use it is very sensible, but whether forcing it on all filesystems is a good choice is less clear.

Network filesystems do not benefit from the loop detection that the dcache can provide as all of that must be done on the server anyway. "Virtual" filesystems such as sysfs, procfs, ptyfs don't particularly need a cache at all as all the file names are in memory permanently. Whether a dcache hurts these filesystems is not easy to tell as we don't have a complete and optimized implementation that does not depend on the dcache to compare with.

Of the key identifiers for a midlayer that were discussed above, the one that most clearly points to a cost is the fact that midlayers tend to grow special case code. So it should be useful to examine the dcache to see if it has suffered from this.

The first special cases that we find in the dcache are among the flags stored in d_flags. Two of these flags are DCACHE_AUTOFS_PENDING and DCACHE_NFSFS_RENAMED. Each is specific to just one filesystem. The AUTOFS flag appears to only be used internally to autofs, so this isn't really a special case in the dcache. However the NFS flag is used to guide decisions made in common dcache code in a couple of places, so it clearly is a special case, though not necessarily a very costly one.

Another place to look for special case code is when a function pointer in an _operations structure is allowed to be NULL, and the NULL is interpreted as implying some specific action (rather than no action at all). This happens when a new operation is added to support some special-case, and NULL is left to mean the 'default' case. This is not always a bad thing, but it can be a warning signal.

In the dentry_operations structure there are several functions that can be NULL. d_revalidate() is an example which is quite harmless. It simply allows a filesystem to check if the entry is still valid and either update it or invalidate it. Filesystems that don't need this simply do nothing as having a function call to do nothing is pointless.

However, we also find d_hash() and d_compare(), which allow the filesystem to provide non-standard hash and compare functions to support, for example, case-insensitive file names. This does look a lot like a special case because the common code uses an explicit default if the pointer is NULL. A more uniform implementation would have every filesystem providing a non-NULL d_hash() and d_compare(), where many filesystems would choose the case-sensitive ones from a library.

It could easily be argued that doing this - forcing an extra function call for hash and compare on common filesystems - would be an undue performance cost, and this is true. But given that, why is it appropriate to impose such a performance cost on filesystems which follow a different standard?

A more library-like approach would have the VFS pass a path to the filesystem and allow it to do the lookup, either by calling in to a cache handler in a library, or by using library routines to pick out the name components and doing the lookups directly against its own stored file tree.

So the dcache is clearly a midlayer, and does have some warts as a result. Of all the midlayers in the kernel it probably best fits the observation above that they could "be redesigned every time a new requirement came along". The dcache does see constant improvement to meet the needs of new filesystems. Whether that is "an effective use of time" must be a debate for a different forum.

The MD/RAID layer

Our final example as we consider midlayers and libraries, is the md driver which supports various software-RAID implementations and related code. md is interesting because it has a mixture of midlayer-like features and library-like features and as such is a bit of a mess.

The "ideal" design for the md driver is (according to the "midlayer mistake" pattern) to provide a bunch of useful library routines which independent RAID-level modules would use. So, for example, RAID1 would be a standalone driver which might use some library support for maintaining spares, performing resync, and reading metadata. RAID0 would be a separate driver which use the same code to read metadata, but which has no use for the spares management or resync code.

Unfortunately that is not how it works. One of the reasons for this relates to the way the block layer formerly managed major and minor device numbers. It is all much more flexible today, but in the past a different major number implied a unique device driver and a unique partitioning scheme for minor numbers. Major numbers were a limited resource, and having a separate major for RAID0, RAID1, and RAID5 etc would have been wasteful. So just one number was allocated (9) and one driver had to be responsible for all RAID levels. This necessity undoubtedly created the mindset that a midlayer to handle all RAID levels was the right thing to do, and it persisted.

Some small steps have been made towards more of a library focus, but they are small and inconclusive. One simple example is the md_check_recovery() function. This is a library function in the sense that a particular RAID level implementation needs to explicitly call it or it doesn't get used. However, it performs several unrelated tasks such as updating the metadata, flushing the write-intent-bitmap, removing devices which have failed, and (surprisingly) checking if recovery is needed. As such it is a little like part of a mid-layer in that it imposes that a number of unrelated tasks are combined together.

Perhaps a better example is md_register_thread() and friends. Some md arrays need to have a kernel thread running to provide some support (such as scheduling read requests to different drives after a failure). md.c provides library routines md_register_thread() and md_unregister_thread(), which can be called by the personality as required. This is all good. However md takes it upon itself to choose to call md_unregister_thread() at times rather than leaving that up to the particular RAID level driver. This is a clear violation of the library approach. While this is not causing any actual problems at the moment, it is exactly the sort of thing that could require the addition of special cases later.

It has often been said that md and dm should be unified in some way (though it is less often that the practical issues of what this actually means are considered). Both md and dm suffer from having a distinct midlayer that effectively keeps them separate. A full understanding of the fact that this midlayer is a mistake, and moving to replace it with an effective library structure is likely to be an important first step towards any sort of unification.

Wrap up

This ends our exploration of midlayers and libraries in the kernel -- except maybe to note that more recent additions include such things as libfs, which provides support for virtual filesystems, and libata, which provides support for SATA drives. These show that the tendency away from midlayers is not only on the wishlist of your author but is present in existing code.

Hopefully it has resulted in an understanding of the issues behind the "midlayer mistake" pattern and the benefits of following the library approach.

Here too ends our little series on design patterns in the Linux kernel. There are doubtlessly many more that could be usefully extracted, named, and illuminated with examples. But they will have to await another day.

Once compiled, such a collection would provide invaluable insight on how to build kernel code both effectively and uniformly. This would be useful in understanding how current code works (or why it doesn't), in making choices when pursuing new development, or when commenting on design during the review process, and would generally improve visibility at this design level of kernel construction. Hopefully this could lead, in the long term, to an increase in general quality.

For now, as a contribution to that process, here is a quick summary of the Patterns we have found.

• kref: Reference counting when the object is destroyed with the last external reference

• kcref: Reference counting when the object can persist after the last external reference is dropped

• plain ref: Reference counting when object lifetime is subordinate to another object.

• biased-reference: An anti-pattern involving adding a bias to a reference counter to store one bit of information.

• Embedded Anchor: This is very useful for lists, and can be generalized as can be seen if you explore kobjects.

• Broad Interfaces: This reminds us that trying to squeeze lots of use-cases in to one function call is not necessary - just provide lots of function calls (with helpful and (hopefully) consistent names).

• Tool Box: Sometimes it is best not to provide a complete solution for a generic service, but rather to provide a suite of tools that can be used to build custom solutions.

• Caller Locks: When there is any doubt, choose to have the caller take locks rather than the callee. This puts more control in that hands of the client of a function.

• Preallocate Outside Locks: This is in some ways fairly obvious. But it is very widely used within the kernel, so stating it explicitly is a good idea.

• Midlayer Mistake: When services need to be provided to a number of low-level drivers, provide them with a library rather than imposing them with a midlayer.

Exercises

1. Examine the "blkdev_ioctl()" interface to the block layer from the perspective of whether it is more like a midlayer or a library. Compare the versions in 2.6.27 with 2.6.28. Discuss.

2. Choose one other subsystem such as networking, input, or sound, and examine it in the light of this pattern. Look for special cases, and imposed functionality. Examine the history of the subsystem to see if there are signs of it moving away from, or towards, a "midlayer" approach.

3. Identify a design pattern which is specific to the Linux kernel but has not been covered in this series. Give it a name, and document it together with some examples and counter examples.

Comments (26 posted)

GRUB 2 becomes the default bootloader in Ubuntu 9.10

At the Ubuntu Developer Summit, some discussions took place about the status of GRUB 2, and in early June Canonical's Colin Watson announced that GRUB 2 would be the default boot loader for new Ubuntu installations, starting with Ubuntu 9.10 scheduled for release later this year. GRUB 2 will replace GRUB, which has been used for many years for the task of selecting different kernel images or other operating systems to boot from, both in Ubuntu and other Linux distributions.

GRUB is the first thing a user sees after the computer's BIOS has initialized the PC. In recent times Linux distributions have had the tendency to hide GRUB's menu, so one might wonder why users should care about it. Nevertheless, Ubuntu's switch to GRUB 2 will bring internationalization support, greater flexibility and many other improvements to the boot loader. As changing the boot loader is an inherently risky operation, upgrades from existing Ubuntu installations will not replace GRUB by GRUB 2.

Problems with legacy GRUB

Although GNU GRUB is used by virtually all Linux distributions, it has a couple of problems. First, official development for the most used version, called GRUB Legacy, was stopped. The developers still accept bug fixes but don't add new features. They have refocused on GRUB 2, a complete rewrite of the boot loader. This leaves Linux distributions with a maintenance problem, according to Watson in an email interview with your author:

Because the GRUB developers don't accept new features, each distribution has actually forked the package to add the features it needs. Ubuntu, for example, added support for UUIDs (a "Universally Unique Identifier" to identify a partition uniquely, even when the boot order and hence the name changes) and for the Ext4 file system. But this is not the preferred way to go, Watson admits:

There are also a variety of technical problems without much hope of a good resolution. For example, the mapping of Linux device names to and from GRUB device names, which on a PC-class system ought to be derived from the BIOS boot order, has long been problematic. In principle the primary master hard disk is known to GRUB as hd0, but some BIOSes let the user configure the secondary master hard disk to become the first one to boot, which confuses GRUB. BIOS Enhanced Disk Drive (EDD) Specification support helps when it works, but hardware support for it is rather limited: BIOSes older than a few years don't have it and even newer BIOSes do not implement it consistently or completely. Ubuntu has papered over this with the use of UUIDs, but this only works for some parts of the boot chain. For example, it doesn't help GRUB find its own stage 1.5, the stage that understands a file system. Another problem is the format of the configuration file /boot/grub/menu.lst, which Watson describes as "a historical disaster".

New features of GRUB 2

So which problems would GRUB 2 solve? First, GRUB 2 adds built-in UUID support, so different distributions no longer have to maintain their own hacked-up versions for this. The boot loader has also refactored its multi-stage load process, which according to Watson "is likely to be less of a practical problem than the current load process". Stage 1.5 was eliminated.

Another change is the new configuration file format, which has been entirely overhauled. It has more extensive scripting support, with conditionals, loops, variables and functions, allowing the user to express some more complex booting logic. However, according to Watson this complexity doesn't make it more complex to use these features:

Other interesting new features are the graphical boot menu, internationalization (including support for non-ASCII character codes, fonts and message catalogs like gettext), a modular framework and the possibility of dynamic loading of modules at run time. Beyond that, GRUB 2 also supports non-x86 platforms, such as PowerPC.

What will Ubuntu do with GRUB 2?

Now the question is what GRUB 2 will bring for Ubuntu's users. The more reliable configuration handling should let the developers do some more interesting things at the desktop level, which according to Watson have been difficult in practice before, "as inserting the right options ran a significant risk of stomping over people's local changes." One example he gives is to offer a straightforward interface to reboot into a specific operating system at the desktop level. The user doesn't have to first reboot and then pick that operating system from GRUB's boot menu.

The use of a graphical menu means that, in principle, distributions have a chance of integrating fairly smoothly with kernel mode setting even when a boot menu is displayed. Watson adds:

There's a Google Summer of Code project to add graphical menu support to GRUB 2, although it hasn't landed yet. Even though Ubuntu will be aiming for a so-called "quiet-by-default" boot loader, which doesn't show a boot menu by default, Watson expects from the sheer volume of requests that many Ubuntu users will be interested in a graphical boot menu that they can tweak with their own theme.

Why switch now?

The switch to GRUB 2 doesn't come out of the blue. According to Watson, Ubuntu has been keeping half an eye on GRUB 2 for some time:

This situation has changed considerably in recent times. The pace of development over the last year has picked up markedly, and there's now quite an active GRUB 2 community in place. Then, pressure to add proper support for things like EFI and boot menu internationalization in Ubuntu has been growing, so the developers thought they'd take the opportunity to have another look at the new boot loader:

Ubuntu is not alone in this task. As Watson says, Debian makes a big difference to Ubuntu too:

With all this in mind, the Ubuntu developers felt that switching over was much more feasible than they'd previously thought, and that the risk of there still being a couple of missing features was tolerable given the pace of upstream development. Watson considers the time ripe now for GRUB 2 in Ubuntu:

No switch to GRUB 2 in openSUSE 11.2

While Ubuntu is switching later this year, openSUSE currently has no plans to switch to GRUB 2 in its next release, version 11.2. Andreas Jaeger explains this:

Jaeger is right that GRUB 2 is not entirely ready. For example, some GRUB Legacy functionality isn't yet supported in GRUB 2: it doesn't have boot password support yet, and the savedefault functionality which saves the current menu entry as the default entry doesn't work properly.

According to Jaeger, the openSUSE developers have so far not received a single request to move to GRUB 2. But if somebody volunteers to maintain the package and integrate it in openSUSE, he welcomes it to give it some testing and then decide if it can be used for future releases. He adds that support for GRUB 2 doesn't mean only a new package, but also changes the way the boot loader setup is automatically generated by openSUSE's system management tool YaST.

Fedora

The Fedora developers also have no concrete plans to use GRUB 2 as their default boot loader. However, Fedora's GRUB 2 maintainer Lubomir Rintel sees the current situation with GRUB clearly as unsatisfactory:

GRUB Legacy has posed some real problems for Fedora. For example, the support for Ext4 didn't land in the version shipped with Fedora 11, because the patches weren't ready in time for the beta. This means that Fedora 11 users can't boot from an Ext4 file system, something that is possible in Ubuntu 9.04.

Rintel sees the active upstream community of GRUB 2 as a nice advantage, especially now that Ubuntu is going to use it. He expects the community around it to grow and the quality of the GRUB 2 code to increase rapidly, increasing chances of inclusion in other distributions, including Fedora. In the meantime, he will ensure that GRUB 2 is up-to-date with upstream code and easily installable for anyone interested in using it in Fedora. The short runway for Fedora 12 means that there's really not that much time to get GRUB 2 in shape for Fedora's next release, but according to Fedora's wiki the integration is planned for Fedora 13.

With respect to the technical qualities of GRUB 2, Rintel is mostly impressed about GRUB 2's modular architecture. He calls other features such as the graphics subsystem with mouse support capable of rendering antialiased Unicode glyphs "hardly things that would convince anyone to switch to GRUB 2".

An interesting adventure

It remains to be seen which hurdles Ubuntu has to take for the switch from the stable legacy GRUB to the new GRUB 2. Adventurous Ubuntu users can already test GRUB 2 in Jaunty Jackalope and are invited to report any bugs. If this experiment turns out successfully, other distributions could very well follow Ubuntu's lead.

Comments (24 posted)

Mandriva Linux 2010 Alpha 1 and 2010 specifications

Mandriva Linux 2010 Alpha 1 has been announced. "Mandriva Linux 2010 Alpha 1 is now available on public mirrors. This first alpha is available only through Free version, 32 and 64 bits DVDs. This development release is the first one realized without mkcd, our historical build tool, but using bcd available also on Mandriva svn. This new tool should improve global quality of our release and make tests much easier and efficient."

Comments (none posted)

June 2009 Fedora Board election results

The election results for the Fedora Advisory Board are available. Tom Callaway, Mike McGrath and Dennis Gilmore have been elected to the Fedora Board for a complete two release cycle term. Click below for details.

Full Story (comments: none)

FESCo June 2009 election results

The Fedora Engineering Steering Committee election results are in. Bill Nottingham, Seth Vidal, Kevin Fenzi, Kevin Kofler and Dennis Gilmore have been elected to FESCo for a full, 2 release cycle term. Click below for details.

Full Story (comments: none)

Fedora 11 s390x preview

The Fedora s390x team has announced a first preview of Fedora 11 for s390x. "We currently have ~11600 binary packages of Fedora 11/s390x and are working on getting real boot images."

Full Story (comments: none)

Gentoo Council meeting summary

Click below for a summary of the Gentoo Council meeting of June 11, 2009. Topics include Short discussion of EAPI 3 progress, Default contents of ACCEPT_LICENSE(license filtering), BASH 4 in EAPI 3, and Define EAPI development/deployment cycles.

Full Story (comments: none)

openSUSE Factory is Now Open

Joe 'Zonker' Brockmeier covers changes to the openSUSE development Factory. "openSUSE development is now even _more_ open than before. Factory development is changing, and we're making it easier for contributors to take responsibility for packages and to contribute directly to openSUSE. This means contributors will be able to be directly responsible for packages, without having to go through a Novell employee to make changes."

Full Story (comments: 4)

One Hundred Paper Cuts

A project led by Canonical's Design and User Experience team aims to improve the user experience in Ubuntu by identifying 100 small points of pain for users, or "paper cuts", and healing them. "A paper cut is a minor usability annoyance that an average user would encounter on his/her first day of using a new installation of Ubuntu 9.10"

Comments (none posted)

Announcing the openArtist linux distribution

openArtist is a ready to use Ubuntu derivative built for creative people. Based on Ubuntu Hardy, openArtist incorporates packages from Intrepid, Jaunty, PureDyne, Debian, Mint, and RPM based distributions. There are also many self made packages, and some freeware. It can be used as live CD or installed to a hard drive. Software packages for 2D, 3D, Audio, Video, VJ, Hardware Interfacing, Programming and Collaboration have been integrated.

Full Story (comments: none)

DistroWatch Weekly, Issue 308

The DistroWatch Weekly for June 22, 2009 is out. "The Slackware-based VectorLinux project has been around for almost a decade, making regular releases of the fast, desktop-friendly Linux distribution. Robert Lange, the project's co-founder and lead developer, was kind enough to agree to an interview. How did it all start? And what does the future hold for the project? Read on to find the answers to these and other questions. In the news section, the Fedora community announces a new live CD featuring the lightweight LXDE desktop environment, Mandriva prepares for its 2010 release with major changes to the development branch, and Ubuntu gets set to tackle minor annoyances which impact negatively on user experience. Also in the news, we link to an interview with Linux Mint's Clement Lefebvre and O'Reilly publishes answers provided by developers about the recently released OpenBSD 4.5. Finally, don't miss the "New Distributions" section, with no fewer than seven new additions to the DistroWatch's waiting list. Happy reading and happy testing!"

Comments (none posted)

Fedora Weekly News 181

The Fedora Weekly News for June 21, 2009 is out. "Here are a few highlights from this week's issue. Announcements starts us off with a reminder that Fedora 9 end-of-life is July 10, and an update from the Fonts SIG, including many Fedora 11 items, along with coverage of the Fedora Activity Day recently held. From the Fedora Planet, details on a new Fedora Community Portal that opened to much excitement. The QA beat turns its sights to Fedora 12, with details on schedule, installer and rawhide testing plans. In Fedora Ambassador, news on the SouthEast Linux Fest last week in Clemson, SC. Translation news brings us word of many new members to the localization project, and translation updates for Fedora web for Dutch and Hebrew. The Artwork & Design beat completes the issue with coverage of discussion on Fedora 12 theming."

Full Story (comments: none)

OpenSUSE Weekly News/77

This issue of the OpenSUSE Weekly News covers openSUSE Factory is now open, GSoC Status Reports, Lubos Lunak: OpenOffice.Org KDE4 Integration, openSUSE 11.2 Launch Planning, Linux.com/Rocky: The Plasma desktop shell of KDE 4, and more.

Comments (none posted)

Ubuntu Weekly News #147

The Ubuntu Weekly Newsletter for June 21, 2009 is out. "In this issue we cover Ubuntu Free Culture Showcase competition, 3 New Members of the Americas Region Membership Board, Bootchart testing for UNR, Empathy to replace Pidgin in Karmic Koala, Ubuntu Global Jam 2nd - 4th October 2009, New freenode webchat (and why to use it), Ubuntu Stats, In the Press & Blogosphere, Upcoming Meetings & Events, Updates & Security, and much, much more!"

Full Story (comments: none)

5 Ways to Decide on a Linux Distribution (DaniWeb)

DaniWeb looks at five things to consider when choosing a Linux distribution for corporate needs. "Prejudices and opinions aside, at some point in your career you'll be asked to select a viable Linux distribution for your corporate network. How will you choose? Will you use the same distribution that you use at home or will you do some research and find something that's corporate-ready? Are you up to the task? Do you know what to look for in a distribution to support a corporate environment?"

Comments (none posted)

The beginner's guide to Slackware Linux (TechRadar)

TechRadar UK takes a look at Slackware. "[Slackware is] not trying to win enormous desktop market share, nor is it loaded with blinking lights, hold-your-hand graphical wizards and package managers that change with every release. Slackware is about as pure a GNU/Linux system as you can get - at least, without all the arduous leg work of Linux From Scratch."

Comments (none posted)

Interview with Frank Wierzbicki from the Jython project

Jython is a version of the Python language that is written in Java while standard Python (CPython) is written in C. From the project's V2.2 web site: "Jython is an implementation of the high-level, dynamic, object-oriented language Python written in 100% Pure Java, and seamlessly integrated with the Java platform. It thus allows you to run Python on any Java platform."

We conducted an email interview with Jython developer Frank Wierzbicki.

Greetings, Could you tell us about yourself and your role with the Jython project?

How many team members are on the project, and what are their functions?

What was the motivation behind the creation of Jython?

Do any companies provide financial backing for Jython?

What kinds of of applications would be better to write in Jython instead of C Python?

How does the memory usage and speed of Jython compare with C Python?

Is Jython more popular with users of Linux, other types of Unix or Windows?

Could you tell us about the enhancements that have been added to the recently released Jython 2.5.0?

What are the plans for upcoming releases of Jython?

Is there anything else you would like to share with our readers about Jython?

Comments (4 posted)

dbf converter 0.02 released

Version 0.02 of dbf converter has been announced. "dbfconverter.py is platform independent python script. It converts database with dbf files or single dbf file into sql code, wich you can load into any sql database."

Full Story (comments: 1)

PostgreSQL Weekly News

The June 21, 2009 edition of the PostgreSQL Weekly News is online with the latest PostgreSQL DBMS articles and resources.

Full Story (comments: none)

BusyBox 1.14.2 released

Version 1.14.2 of BusyBox, a collection of command line utilities for embedded systems, has been announced. "Bug fix release. Contains fixes in ash ('.' builtin), ftpd, httpd, modprobe (better exit code compatibility), readlink (more options supported), telnetd (now closes file descriptors in childred, it was forgetting to do so)."

Comments (none posted)

Four new versions of Samba available

Versions 3.3.6 (security fix), 3.2.13 (security fix), 3.0.35 (security fix) and 3.4.0rc1 (release candidate) of Samba have been announced.

Comments (none posted)

libnfnetlink 1.0.0 released

Version 1.0.0 of libnfnetlink has been announced, it includes new features and bug fixes. "libnfnetlink is the low-level library for netfilter related kernel/userspace communication. It provides a generic messaging infrastructure for in-kernel netfilter subsystems (such as nfnetlink_log, nfnetlink_queue, nfnetlink_conntrack) and their respective users and/or management tools in userspace."

Full Story (comments: none)

Ada and LEGO MINDSTORMS NXT

The Ada language is being used for an educational robotic software control system. ""GNAT for LEGO MINDSTORMS NXT is a GPL port for the GNAT compilation system to the LEGO NXT robotic platform, designed to offer an educational platform for teaching and learning embedded systems development."

Full Story (comments: none)

Announcing ConnMan.net

Intel and Nokia have jointly launched the ConnMan project, an open source project to accelerate and expand development of Internet connection management for Linux Devices. "Connman.net is now the place to bring developers together who are interested in furthering the development of Internet connection management within Linux. Review the source code for more information. ConnMan is licensed under GPLv2, and provides a daemon for managing Internet connections within Linux devices."

Full Story (comments: 48)

Karrigell 3.0 released

Version 3.0 of Karrigell, a Python web framework, has been announced. "Karrigell has been around since 2002 and after numerous versions, until 2.4.0 last year, it needed a complete rewriting. Some modules had been written long ago, using out-of-date patterns ; the global architecture needed cleaning up, to replace an accumulation of modules added one release after the other ; most important, the previous versions were unable to run reliably in a multithreaded environment, which was a serious limitation for professional web developement ".

Full Story (comments: none)

nginx 0.6.38 announced

Version 0.6.38 of the nginx web server has been announced. The CHANGES document says: "Feature: the "keepalive_requests" directive."

Comments (none posted)

GNOME 2.27.3 released

Version 2.27.3 of GNOME has been announced. "This is the third development release towards the wonderful 2.28 release! Various bug fixes and nice improvements in several modules. Fun, fun, fun!"

Full Story (comments: none)

GNOME Software Announcements

The following new GNOME software has been announced this week:

• Clutter 0.9.4 and 1.0.0rc1 (new features, bug fixes and code cleanup)
• Clutter-GTK 0.9.2 (new features and bug fixes)
• Glade 3.6.6 (bug fixes)
• Gnome Subtitles 0.9 (new features, bug fixes and translation work)
• gwget 1.0.2 (bug fixes and code cleanup)
• JSON-GLib 0.7.2 (new features, bug fixes and code cleanup)
• PyGTK 2.15.2 (new features, bug fixes and documentation work)
• rep-gtk 0.18.6 (bug fixes)
• Yelp 2.27.1 (documentation work)

You can find more new GNOME software releases at gnomefiles.org.

Comments (none posted)

KDE Software Announcements

The following new KDE software has been announced this week:

• ATI Overclocking Utility X32 1.0.0 (new features)
• ATI Overclocking Utility X64 1.0.0 (new features)
• Choqok 0.6.1 (bug fix)
• Cirkuit 0.1 (initial release)
• Cirkuit 0.1.1 (new feature and bug fix)
• GimpBrushThumbs 1.0 (unspecified)
• Kalternatives 0.12 (new features and KDE4 port)
• KMidimon 0.7.0 (new features, documentation and translation work)
• KpGen 1.2 Beta 2 (new features)
• mediainfo-qt 0.1 (initial release)
• mtvcgui 0.3 (new features and bug fixes)
• PokerTH 0.7.1-beta (new features, bug fixes and code cleanup)
• QtiPlot 0.9.7.8 (new features, bug fixes and translation work)
• Simutrans Starter 2 1.99.3 (new features, bug fixes and translation work)
• SixA 0.3 (unspecified)
• Soprano 2.2.4 (bug fixes)
• Soprano 2.2.69 RC1 (new features)

You can find more new KDE software releases at kde-apps.org.

Comments (none posted)

Blog series on XInput 2

Peter Hutterer has published a three part blog series (part 1, part 2 and part 3) on X11's XInput 2. "XI2 is now merged into master. Over the next couple of days, I will post various recipes on how to deal with the new functionality. The examples here are merely snippets, full example programs to summarize each part are available here." (Thanks to Paul Wise).

Comments (13 posted)

Xorg Software Announcements

The following new Xorg software has been announced this week:

• inputproto 1.5.1 (bug fix)
• inputproto 1.9.99.12 (new features and bug fixes)
• util-macros 1.2.2 (bug fixes)
• xfs 1.1.0 (new features, bug fixes and code cleanup)
• xinput 1.4.99.1 (new features, bug fixes and code cleanup)

More information can be found on the X.Org Foundation wiki.

Comments (none posted)

rst2pdf 0.11 is out

Version 0.11 of rst2pdf has been announced. "Rst2pdf is a tool to generate PDF files directly from restructured text sources via reportlab. This version includes many bugfixes and some new features compared to the previous 0.10.1 version, including but not limited to embedding PDF images, much improved image sizing, nicer list layouts, better styling, page backgrounds, and more than 15 bugs fixed."

Full Story (comments: none)

Sugar on a Stick Strawberry released

Sugar Labs has announced the release of the Sugar on a Stick Strawberry learning platform. "The Sugar on a Stick Strawberry release is based Fedora 11 and includes the latest updates as of June 22. It also features the last Sugar desktop environment, namely version 0.84, and additional activities to enrich the user experience."

Comments (none posted)

Freecell Solver 2.32.0 released

Version 2.32.0 of Freecell Solver has been announced, it includes several new capabilities and bug fixes. "Freecell Solver is an open-source (MIT/X11 Licensed) ANSI C-based library and some standalone command-line tools for solving Freecell and similar Solitaire variants such as Baker's Game, Seahaven Towers and Eight Off as well as Simple Simon."

Full Story (comments: none)

Wine 1.1.24 announced

Version 1.1.24 of Wine has been announced. Changes include: "Support for freedesktop file associations. Support for exception handling on 64-bit. Improved ARB shaders. Fixes for the FBO mode. Many listview improvements. Various bug fixes."

Comments (none posted)

Thunderbird 2.0.0.22 available for download

Version 2.0.0.22 of Thunderbird has been announced. "We strongly recommend that all Thunderbird users upgrade to this latest release. If you already have Thunderbird 2.0.0.x, you will receive an automated update notification within 24 to 48 hours. This update can also be applied manually by selecting "Check for Updates?" from the Help menu."

Full Story (comments: none)

ride tab editor 2.0 released

Version 2.0 of ride tab editor has been announced. "ride tab editor is a freeware multiplatform tab editor for bass, guitar and drums. included is a csound tool released under a dual license (free for non-profit the qt license in other words). to convert the drum tabs into csound scores".

Full Story (comments: none)

ride tab editor 2.02 released

Version 2.02 of ride tab editor has been announced, it includes some GUI improvements. "ride tab editor is a tab editor created in 2004 that originally allowed guitar, bass and drums. ride tab editor 2 is started to address problems people had of finding tab editors for other instruments by allowing them to create custom format definitions. It is also very useful to create text based piano rolls for text based music programming languages such as csound. two examples are provided."

Full Story (comments: none)

Leo 4.6 beta 2 released

Version 4.6 beta 2 of Leo has been announced. "Leo is a text editor, data organizer, project manager and much more."

Full Story (comments: none)

The initial publication of about:mozilla

The about:mozilla blog site has been announced. "I’m in the process of re-re-evaluating the purpose and utility of this blog, so there may be new activity here soon (once I sort out what I want to do with it). In the meantime, I’ve figured out how to autogenerate a list of all the about:mozilla archives! You can find every issue we’ve ever published on the Newsletter archives page. Enjoy!" (Thanks to Joe Drew).

Comments (none posted)

SeaMonkey 1.1.17 released

Version 1.1.17 of SeaMonkey has been announced. "Today, the SeaMonkey project released a new version of its all-in-one Internet suite. SeaMonkey 1.1.17 closes several security vulnerabilities and fixes a few smaller problems found in previous versions. With that, SeaMonkey stays at the same level of security as its sibling Firefox 2, which is issuing updates for the same problems this week as well."

Full Story (comments: none)

Caml Weekly News

The June 23, 2009 edition of the Caml Weekly News is out with new articles about the Caml language.

Full Story (comments: none)

PHP 5.3.0RC4 released

Release candidate 4 of PHP 5.3.0 has been announced. "The PHP development team is proud to announce the fourth release candidate of PHP 5.3.0 (PHP 5.3.0RC4). This RC focuses on bug fixes and stability improvements, and we hope only minimal changes are required for the next candidate or final stable releases. PHP 5.3.0 is a newly developed version of PHP featuring long-awaited features like namespaces, late static binding, closures and much more. "

Comments (none posted)

Python-URL! - weekly Python news and links

The June 20, 2009 edition of the Python-URL! is online with a new collection of Python article links.

Full Story (comments: none)

CodeInvestigator 0.13.0 released

Version 0.13.0 of CodeInvestigator, a tracing tool for Python, has been announced. "Bug fix: Changes to imported files were not taken into account. This version only works with Python 2.6. Some modules disappear in Python 3.0 and this version has these modules replaced."

Full Story (comments: none)

cx_Logging 2.0 announced

Version 2.0 of cx_Logging has been announced, it includes numerous improvements. "cx_Logging is a Python extension module which operates in a fashion similar to the logging module that ships with Python 2.3 and higher. It also has a C interface which allows applications to perform logging independently of or in tandem with Python."

Full Story (comments: none)

ErrorHandler 1.0.0 released

Version 1.0.0 of ErrorHandler has been announced. "I'm pleased to finally get around to announcing the release of ErrorHandler. This is a handler for the python standard logging framework that can be used to tell whether messages have been logged at or above a certain level. This can be useful when wanting to ensure that no errors have been logged before committing data back to a database."

Full Story (comments: none)

HDF5 for Python 1.2 announced

Version 1.2 of HDF5 for Python, a general-purpose Python interface to the Hierarchical Data Format library, has been announced. "I'm pleased to announce the availability of HDF5 for Python 1.2 final! This release represents a significant update to the h5py feature set."

Full Story (comments: none)

MailingLogger 3.3.0 released

Version 3.3.0 of MailingLogger has been announced. "Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process."

Full Story (comments: none)

Numexpr 1.3.1 released

Version 1.3.1 of Numexpr has been announced. "Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. This is a maintenance release. On it, support for the `unit32` type has been added (it is internally upcasted to `int64`), as well as a new `abs()` function (thanks to Pauli Virtanen for the patch)."

Full Story (comments: none)

TestFixtures 1.6.0 released

Version 1.6.0 of TestFixtures has been announced. "I'm pleased to announce the first advertised release of TestFixtures. This is a collection of helpers and mock objects that are useful when writing unit tests or doc tests."

Full Story (comments: none)

Tcl-URL! - weekly Tcl news and links

The June 20, 2009 edition of the Tcl-URL! is online with new Tcl/Tk articles and resources.

Full Story (comments: none)

Tcl-URL! - weekly Tcl news and links

The June 24, 2009 edition of the Tcl-URL! is online with new Tcl/Tk articles and resources.

Full Story (comments: none)

PuDB 0.92.1 - a console-based visual debugger

Version 0.92.1 of PuDB has been announced. "PuDB is a full-screen, console-based visual debugger for Python. Its goal is to provide all the niceties of modern GUI-based debuggers in a more lightweight and keyboard-friendly package. PuDB allows you to debug code right where you write and test it--in a terminal. If you've worked with the excellent (but nowadays ancient) DOS-based Turbo Pascal or C tools, PuDB's UI might look familiar."

Full Story (comments: none)

bzr 1.16 released

Version 1.16 of the bzr adaptive version control system has been announced. "This version of Bazaar contains the beta release of the new ``2a`` repository format, suitable for testing by fearless, advanced users. This format or an updated version of it will become the default format in Bazaar 2.0. Please read the NEWS entry before even thinking about upgrading to the new format. Also included are speedups for many operations on huge projects, a bug fix for pushing stacked new stacked branches to smart servers and the usual bevy of bug fixes and improvements."

Full Story (comments: none)

GIT 1.6.3.3 released

Version 1.6.3.3 of the GIT distributed version control system has been announced, it includes numerous bug fixes.

Full Story (comments: none)

libfiu: fault injection in userspace

Version 0.1.1 of libfiu, a C library for fault injection, has been announced. "It comes with some tools that can be used to simulate failures in the POSIX functions without having to modify the application's source code. The main application (or at least the one I had in mind when writing it) is to simplify automated testing of software behaviour at failure scenarios, for software which is supposed to be fault-resistant or at least gracefully cope with failures."

Full Story (comments: 3)

Does the Linux Desktop Innovate Too Much? (Datamation)

Bruce Byfield wonders if the Linux desktop is evolving too quickly. "Is there a compelling argument for innovation? Or has the free desktop reached a point where it satisfies most users and any attempt to change its current state is going to be regarded as an unwarranted intrusion on the average person's activities?"

Comments (101 posted)

Why Parrot is Important (Linux Magazine)

Allison Randal describes the motivations behind Parrot in this Linux Magazine article. "Good tools lower the risk of trying out new ideas. Spending 10 years implementing a radical new concept in programming languages, alien to users of existing languages, is a daunting prospect. Spending a few weeks, on the other hand, is quite tolerable. Successful experiments can quickly go on to solve more problems, while the unsuccessful ones can be quickly set aside, applying the lessons learned to the next language."

Comments (17 posted)

KOffice Developers At The First ODF Plugfest (KDEDot)

KDE.News covers the First ODF Plugfest. "Dutch minister for Foreign Trade Frank Heemskerk held an opening speech, outlining why the Dutch government pushes the usage of the three Os: Open Standards, Open Content and Open Source Software. Interoperability brings a better market and thus leads to better quality, lower prices, more innovation and vendor independence. The Netherlands wants to join the Scandinavian countries in becoming one of the leading countries in adopting ODF and Open Source software. Unfortunately, as we learned later on, the adoption of ODF is progressing very slowly."

Comments (none posted)

OpenSource World Unlocks the Word on Keynote Speakers (Linux Journal)

Linux Journal looks forward to the OpenSource World conference, previously known as LinuxWorld. "Keynote speakers are always a highlight of any conference, and OpenSource World is no exception. The expo's main speaker will be California Secretary of State Debra Bowen, who is known to the Open Source community for understanding and advocating Open Source software. Additionally, there will be a keynote panel, "Assessing the Real Market Opportunities and Obstacles for Making Cloud Computing Mainstream," lead by CloudWorld conference chairman Jeffrey Kaplan and including discussion and debate by panelists Joe Weinman of AT&T Business Solutions, Sam Charrington of Appistry, and James Urquhart of Cisco."

Comments (none posted)

Open Video Conference an Amazing Step Forward (J5's Blog)

J5's Blog has a review of the Open Video Conference. "I was mainly there looking to see what video producers wanted from FOSS application developers and to support the PiTiVi/GStreamer teams on behalf of the GNOME Foundation. It is amazing to see the PiTiVi non-linear video editing app at such a usable state. While Edward Hervey gave his mini presentation on PiTiVi I was busy hacking up a “How To Make Chocolate Truffles” video from pictures and clips I had laying around. Afterwards I showed him some of the bugs I encountered in the 0.13.1 release and he just rattled off, fixed in git, fixed in git, fixed in git…etc." (Thanks to Paul Wise).

Comments (none posted)

More on the Bankruptcy Hearing - Who Exactly Is The Proposed Buyer? (Groklaw)

Groklaw attempts to uncover the buyer of bankrupt SCO. "According to our update 5, Stephen Norris signed the agreement, but now the question is, for which entity? And when was the entity formed? Recently? Is that the sound of wings I hear? It makes one wonder who exactly is the final buyer, particularly because the Berger Singerman lawyer who worked out the deal, Mr. Kaplan, reportedly testified at the hearing that he wasn't clear what Mr. Norris' role was. According to update 5, he testified like this: "The principal businessman is in London. The principal is in London. I am not certain of Norris' role. I have not been aware of Mr. Norris." So, the only possible response to that is, what's up with that? He is referring to Eric LeBlan, presumably, or at least according to our eyewitnesses' reports, as the UK businessman. But you might want to mentally put a marker here, to make sure to watch this issue as more information arrives. At some point SCO will have to file with the SEC presumably. And as you know, Darl McBride testified under oath in a prior court hearing that he always tells the SEC the truth."

Comments (none posted)

Red Hat profit rises, bucks tech industry trend (Reuters)

Reuters reports on Red Hat's latest financial results. "Software company Red Hat Inc (RHT.N) reported a 7 percent rise in quarterly profit onWednesday, bucking an industry trend of declining earnings, as margins widened under the scrutiny of its cost-conscious CEO. Operating margin rose to 23.4 percent from 21.8 percent a year earlier, after excluding stock compensation and amortization expenses. That was better than the 23 percent the company projected three months ago."

Comments (1 posted)

Linux on Netbooks: The Smoking Gun (Groklaw)

Groklaw covers reports from Computex in Taiwan, regarding the death of Linux netbooks. "Is there no regulatory body that can get Microsoft's fat fanny off of Linux so it can get some air? Instead the DOJ are investigating *Google*? What Microsoft is reportedly doing is a pimple on the antitrust regulators' noses. We see it. Why can't you? Where are you? Please don't wait until Linux is totally crushed."

Comments (8 posted)

The triumph of Linux as a supercomputer OS (Royal Pingdom)

Royal Pingdom analyzes the results of a recent supercomputer survey. "Operating systems on supercomputers used to be custom-made affairs, but this has changed. These days, Linux has become a popular choice for supercomputers. But how popular? You may be surprised. Top500.org maintains a list of the fastest supercomputers in the world. A new list was published yesterday (it happens twice a year), so we took the opportunity to go through the list and find out what OS the top 20 supercomputers are using."

Comments (28 posted)

ScummVM GPL conflict with Atari

The ScummVM project announced a GPL conflict involving Atari and Mistic Software Inc. The issue has been settled. "In December 2008, members of the ScummVM team discovered that three games for the Nintendo Wii console ("Freddi Fish: The Case of the Missing Kelp Seeds", published in Germany under the name "Fritzi Fisch und der verschwundene Schatz"; "Pajama Sam: No Need to Hide When It's Dark Outside", published in Germany under the name "Pyjama Pit: Keine Angst im Dunkeln"; and "Spy Fox: Dry Cereal", published in Germany under the name "Spy Fox in: Das Milchkartell") made use of ScummVM, without complying with the terms of the GPL license. They sent a warning letter to the German distributor of these games, Atari Deutschland GmbH, who was not aware that ScummVM was used in the creation of the games. Atari Deutschland GmbH established contact with Mistic Software Inc., the developers of the games." (Thanks to Armijn Hemel).

Comments (2 posted)

Benchmarks Of Fedora 9 Through 11 (Phoronix)

Phoronix performs a series of benchmark tests on Fedora 9, 10, 11 and Fedora Rawhide. "When testing out each of the Fedora releases (except for Rawhide where we had upgraded on top of a Fedora 11 installation), we had performed a clean installation of the given Fedora release using the x86_64 DVD and used the default file-system layout that occupied the entire SATA HDD. When each Fedora release was installed, the system was left with its stock configuration/settings, which does include the use of SELinux, etc."

Comments (2 posted)

T-Mobile Phones Home, Again (Linux Journal)

Linux Journal takes a look at the next generation of the Android phone. "The T-Mobile myTouch 3G with Google is lighter than its predecessor, a reported six hours of battery life, and will come in a designer "Merlot" color as well as the less chic black and white. The feature — or rather, lack thereof — drawing the most attention, however, is the departure from the physical keyboard considered a strong selling feature of the G1."

Comments (none posted)

IDE rev'd for improved multi-core debugging (LinuxDevices)

LinuxDevices reviews the latest version of Enea's IDE. "Swedish telecom software firm Enea announced a new version of its Linux-compatible Eclipse-based integrated development environment (IDE). Optima 2.1 adds enhanced system level debugging functionality via upgraded versions of the Enea BlackBox Recorder and Optima Log Analyzer, with a special focus on debugging multi-core and multi-processor applications, the company says."

Comments (none posted)

Teaching Math with the KDE Interactive Geometry Program (Linux Journal)

Linux Journal takes a look at the KDE Interactive Geometry (Kig) program to teach mathematics. "Kig allows you to use various tools to diagram and demonstrate different mathematical concepts. With Kig, you can draw points, lines, line segments, half lines, vectors, circles and various other conic sections. When Kig refers to a "half line", it means what I was taught was a ray—essentially a line with one endpoint. Drawing hyperbolic curves on the computer sure beats getting dry-erase marker all over yourself or sneezing because of chalk dust. Even more important though, Kig diagrams are interactive, which means that once you create a diagram, you can move various elements around and see how they behave."

Comments (none posted)

Software as an abstract gamespace that is not about Software (michaeldehaan.net)

Michael Dehaan muses about the future of open source software. "In the future 1000 years from now, was it more important to have worked on Web App X or Database Engine Y? Neither. Because that can't be what matters. Theory: The power of OSS tech is not in technology, it is that it crosses boundaries. It is a system, an ideal. The tech does not matter. OSS is not about software. Software is an abstract playing field in which we teach ourselves how to collaborate. One of many such fields. Maybe not even the most efficient. Computer Science is just about logic anyway. It was never about computers." (Thanks to Paul Wise).

Comments (4 posted)

EFF demands public release of FBI surveillance rules

The Electronic Frontier Foundation has announced the filing of a lawsuit against the US DOJ. "The Electronic Frontier Foundation (EFF) filed suit against the Department of Justice today, demanding the public release of the surveillance guidelines that govern investigations of Americans by the Federal Bureau of Investigation (FBI). The FBI's Domestic Investigative Operational Guidelines went into effect in December of 2008 and detail the Bureau's procedures and standards for implementing the Attorney General's Guidelines on approved surveillance strategies."

Full Story (comments: none)

Introducing FSFE's new president, vice president and executive team

The Free Software Foundation Europe has announced its new leaders. "During its General Assembly in Miraflores de la Sierra, Spain, the members of FSFE elected new coordinators for several of the organisation's activities, including strategy, legal and executive coordination. "The new team will continue FSFE's highly successful work with the same long-term vision. We will work hard to justify the trust which the Free Software community is putting into FSFE", says the new president, Karsten Gerloff."

Full Story (comments: none)

Wireless-N Linux Router from Cisco

Cisco has announced a new Wireless-N Router that uses Linux. "Cisco® today announced a new Linux powered router, the Linksys by Cisco Wireless-N Broadband Router with Storage Link (WRT160NL). The new model complements the existing Linksys by Cisco consumer router line-up and is essentially the next generation of the popular WRT54GL. The design of the product is similar to other Linksys by Cisco N-routers, but has integrated connectors for external antennae. Consumers that prefer external aerials can now enjoy the new Linksys by Cisco router design because of the integrated R-SMA antenna connectors. The integrated Storage Link functionality lets consumers connect their USB storage device to the router to create a powerful media sharing solution that enables video, photo and music sharing through the integrated media server."

Full Story (comments: none)

Intel and Nokia announce strategic mobile computing relationship

Intel and Nokia have announced a new strategic mobile computing relationship involving Linux. "Further uniting the Internet with mobile phones and computers, Intel Corporation and Nokia today announced a long-term relationship to develop a new class of Intel® Architecture-based mobile computing device and chipset architectures which will combine the performance of powerful computers with high-bandwidth mobile broadband communications and ubiquitous Internet connectivity. To realize this shared vision, both companies are expanding their longstanding relationship to define a new mobile platform beyond today's smartphones, notebooks and netbooks, enabling the development of a variety of innovative hardware, software and mobile Internet services."

Comments (3 posted)

Hadoop: The Definitive Guide - New from O'Reilly

O'Reilly has published the book Hadoop: The Definitive Guide by Tom White.

Full Story (comments: none)

Mercurial: The Definitive Guide--New from O'Reilly

O'Reilly has published the book Mercurial: The Definitive Guide by Bryan O'Sullivan.

Full Story (comments: none)

Programming Clojure--New from Pragmatic Bookshelf

Pragmatic Bookshelf has published the book Programming Clojure by Stuart Halloway.

Full Story (comments: none)

Python Essential Reference, 4th Edition

Python Essential Reference, 4th Edition by David Beazley has been published.

Full Story (comments: none)

Free Software Awards - Trophees du Libre 2009

The winners of the Trophees du Libre 2009 Free Software Awards have been announced. "On June 5th and 6th, 2009 the fifth édition of the Trophées du Libre 2009 took place at the IUT de Cuffies-Soissons. The event was organized by the CETRIL association in partnership with the European Communinity, the France State, the Picardy region and the Grand Soissons council. All the 23 finalist projects were awarded in each of the seven categories : Security, Administration, Education, Professionnel, Multi-media technology, Leasure and Sciences - plus three special prizes : the jury special prize, the prize for a large potential software usage and the most innovative project."

Full Story (comments: none)

PSF Community Awards go to Stephan Deibel and Sean Reifschneider

Stephan Deibel and Sean Reifschneider have received the PSF Community Awards. "Stephan was last year's outgoing chairman after four years in harness. This year Stephan has stepped down as a director, after helping to ensure that the Foundation's bylaws were reorganized. Stephan developed pythonology.com to promote Python, and his work as founder of Wingware (wingware.com) and a developer of the Wing IDE has also had a significant impact. Sean has master-minded the PyCon networking every time it's worked, and without the support of this always helpful and reliably competent tummy.com director our conferences simply would not have been the same."

Full Story (comments: none)

Social Desktop Contest (KDEDot)

KDE.News has announced a Social Desktop Contest. "Have you already played with the idea of developing an application for the Social Desktop? Are you interested in programming a clean and easy REST API? You have already heard about the Open Collaboration Services API and wanted to join in? Now is your chance to do so and to win great prizes for your creation! We and the openDesktop.org community are searching for the best and most creative contest submission around the Social Desktop/OCS API! The goal of the Social Desktop Contest is to foster community development and innovations around the OCS API. Prize winners will be selected by the community and a jury. Entrants are encouraged to submit entries that encourage community participation." The contest will run until August 25.

Comments (none posted)

Ruby shines in North American developer survey (The Register)

The Register reports on the rising popularity of Ruby. "Ruby use is up 40 per cent amongst North American software developers since 2008, according to a new study from Evans Data. Despite the jump in popularity, Ruby still occupies a relatively small niche in the developer community as a whole, the company says. Only 14 per cent of developers polled in North America use Ruby at least some of the time, up from 10 per cent in a poll last year. About 20 per cent, however, told Evans they expect to use Ruby in the coming year."

Comments (none posted)

Perl 6 Design Minutes

The Perl 6 Design Minutes for June 6, 2009 are available. "The Perl 6 design team met by phone on 06 May 2009. Larry, Allison, Patrick, Nicholas, and chromatic attended."

Comments (none posted)

CHASE 2009 Lahoe Pakistan call for papers

A call for papers has gone out for CHASE 2009. The event takes place in Lahore, Pakistan on November 6-10 2009, submissions are due by September 4. "CHASE is a unique information and network security event of its kind being organized in Pakistan since 2006. In addition to presentations and talks, CHASE-2009 will include gaming competition and four tracks of trainings. Limited travel funds are vailable for speakers coming outside of Pakistan."

Full Story (comments: none)

InsideMobile Conference registration and call for papers

O'Reilly has announced the call for papers and registration for the InsideMobile Conference. "Interested in learning more about the growing mobile industry? Looking for information on developing for mobile platforms? Wanting a way to get ahead in your field? If so, the newly announced InsideMobile Conference, co-produced by O'Reilly Media and 360Conferences, on July 26-27, 2009, in San Jose, CA, is a must-attend event."

Full Story (comments: none)

Internet Security Operations and Intelligence CFP

A call for papers has gone out for the Internet Security Operations and Intelligence conference. "The 7th ISOI (Internet Security Operations and Intelligence) will take place on September 17th and 18th in San Diego, California."

Full Story (comments: none)

X Developers' Conference 2009

Talks are still needed for the upcoming X Developer's Conference. "A reminder that this year's X Developer's Conference is now just 3 months away, but the wiki pages for attendees and talks are still bare. If you're planning on attending, please add yourself to the attendees wiki. If you've got something to talk about, please add it to the program wiki page. The 2009 X Developers' Conference will be held at Portland State University (PSU) in Portland, Oregon, from Monday September 28 through Wednesday September 30."

Full Story (comments: none)

LCA2010 announces first keynote speaker: Benjamin Mako Hill

The first LCA2010 keynote speaker has been announced. "Our first speaker has a distinguished reputation in the free software world, with significant contributions to Debian, Ubuntu and OLPC across a variety of areas including development, policy, advocacy, activism, education and writing. He is an active board member with several non-profit organisations including The Free Software Foundation and Software Freedom International, and he is involved in the global discussion of copyright, software patents and free culture." LCA2010 will take place in Wellington, New Zealand on January 18-23, 2010.

Full Story (comments: 1)

OpenSource World announces keynote speakers (Linux-Watch)

Linux-Watch has announced the keynotes for OpenSource World, which will take place on August 12-13 in San Francisco, CA. "California Secretary of State Debra Bowen, will offer the keynote address to OpenSource World 2009, says IDG World Expo. Bowen is said to have been been recognized in the open source industry for her familiarity with the benefits of open source. The conference will also feature a keynote panel called "Assessing the Real Market Opportunities and Obstacles for Making Cloud Computing Mainstream." The panel will feature: * Jeffrey M. Kaplan -- conference chairman for CloudWorld and managing director for THINKstrategies, Inc. * Sam Charrington -- VP of product management and marketing, Appistry, Inc. * James Urquhart -- market manager, cloud computing and virtualization, Cisco Systems * Joe Weinman -- strategy and business development, AT&T Business Solutions".

Comments (none posted)

UKUUG Summer 2009 Conference and Tutorials

The UKUUG Summer 2009 Conference and Tutorials will be held in Birmingham, UK on August 7-9, 2009. Early bird registration is open until July 6.

Full Story (comments: none)

Events: July 2, 2009 to August 31, 2009

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
June 28 July 4	EuroPython 2009	Birmingham, UK
July 1 July 3	OSPERT 2009	Dublin, Ireland
July 1 July 3	ICOODB 2009	Zurich, Switzerland
July 2 July 5	ToorCamp 2009	Moses Lake, WA, USA
July 3 July 11	Gran Canaria Desktop Summit (GUADEC/Akademy)	Gran Canaria, Spain
July 3	PHP'n Rio 09	Rio de Janeiro, Brazil
July 4	Open Tech 2009	London, UK
July 6 July 10	Python African Tour : Sénégal	Dakar, Sénégal
July 7 July 11	Libre Software Meeting	Nantes, France
July 13 July 17	(Montreal) Linux Symposium	Montreal, Canada
July 15 July 17	Kernel Conference Australia 2009	Brisbane, Queensland, Australia
July 15 July 16	NIT Agartala FOSS and GNU/Linux fest	Agartala, India
July 18 July 19	Community Leadership Summit	San Jose, CA, USA
July 19 July 20	Open Video Conference	New York City, USA
July 19	pgDay San Jose	San Jose, CA, USA
July 20 July 24	2009 O'Reilly Open Source Convention	San Jose, CA, USA
July 24 July 30	DebConf 2009	Cáceres, Extremadura, Spain
July 25 July 30	Black Hat Briefings and Training	Las Vegas, NV, USA
July 25 July 26	EuroSciPy 2009	Leipzig, Germany
July 25 July 26	PyOhio 2009	Columbus, OH, USA
July 26 July 27	InsideMobile	San Jose, CA, USA
July 31 August 2	FOSS in Healthcare unconference	Houston, TX, USA
August 3 August 5	YAPC::EU::2009	Lisbon, Portugal
August 7 August 9	UKUUG Summer 2009 Conference	Birmingham, UK
August 7	August Penguin 2009	Weizmann Institute, Israel
August 10 August 14	USENIX Security Symposium	Montreal, Quebec, Canada
August 11 August 13	Flash Memory Summit	Santa Clara, CA, USA
August 11	FOSS Dev Camp - Open Source World	San Francisco, CA, USA
August 12 August 13	OpenSource World Conference and Expo	San Francisco, CA, USA
August 12 August 13	Military Open Source Software	Atlanta, Georgia, USA
August 13 August 16	Hacking At Random 2009	Vierhouten, The Netherlands
August 18 August 23	2009 Python in Science Conference	Pasadena, CA, USA
August 22 August 23	Free and Open Source Conference (FrOSCon)	St. Augustin, Germany
August 22 August 23	OpenSQL Camp	St. Augustin, Germany

If your event does not appear here, please tell us about it.

FreedomHEC Taipei 2009 videos

Videos from FreedomHEC Taipei 2009 are available. "Slides and ogg Theora encoded videos of the talks are now linked from the schedule below!" (Thanks to Scott Tsai).

Comments (none posted)

Video: John D. Hunter on Matplotlib

Videolectures.net presents a video lecture with John D. Hunter on Matplotlib. "Matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts, the python and ipython shell (ala matlab or mathematica), web application servers, and works with six graphical user interface toolkits. matplotlib tries to make easy things easy and hard things possible. You can generate plots, histograms, power spectra, bar charts, error charts, scatter plots, etc, with just a few lines of code. For the power user, you have full control of line styles, font properties, axes properties, etc, via an object oriented interface or via a handle graphics interface familiar to matlab users."

Comments (none posted)