LWN.net Weekly Edition for November 12, 2009

Python moratorium and the future of 2.x

On November 9, Python BDFL ("Benevolent Dictator For Life") Guido van Rossum froze the Python language's syntax and grammar in their current form for at least the upcoming Python 2.7 and 3.2 releases, and possibly for longer still. This move is intended to slow things down, giving the larger Python community a chance to catch up with the latest Python 3.x releases.

The idea of freezing the language was originally proposed by Van Rossum in October on the python-ideas list and discussed on LWN. There are three primary arguments for the freeze, all described in the original proposal:

• Letting alternate implementations, IDEs, catch up:
  [...] frequent changes to the language cause pain for implementors of alternate implementations (Jython, IronPython, PyPy, and others probably already in the wings) at little or no benefit to the average user [...]

• Encouraging the transition to Python 3.x:
  The main goal of the Python development community at this point should be to get widespread acceptance of Python 3000. There is tons of work to be done before we can be comfortable about Python 3.x, mostly in creating solid ports of those 3rd party libraries that must be ported to Py3k before other libraries and applications can be ported.

• Redirecting effort to the standard library and the CPython implementation:
  Development in the standard library is valuable and much less likely to be a stumbling block for alternate language implementations. I also want to exclude details of the CPython implementation, including the C API from being completely frozen — for example, if someone came up with (otherwise acceptable) changes to get rid of the [Global Interpreter Lock] I wouldn't object.

The proposal turned into PEP 3003, "Python Language Moratorium", which is more definite about what cannot be changed:

Adding a new method to a built-in type will still be open for consideration, and so is changing language semantics that turn out to be ambiguous or difficult to implement. Python's C API can be changed in any way that doesn't impose grammar or semantic changes, and the modules in the standard library are still fair game for improvement.

The duration of the freeze is given in the PEP as "a period of at least two years from the release of Python 3.1." Python 3.1 was released on June 27 2009, so the freeze would extend until at least June 2011. Van Rossum later clarified the duration on python-dev, writing "In particular, the moratorium would include Python 3.2 (to be released 18-24 months after 3.1) but (unless explicitly extended) allow Python 3.3 to once again include language changes."

Most responses to the moratorium idea were favorable, but those who had objections felt those objections very strongly. Steven D'Aprano wrote:

The 2-to-3 transition

One of the reasons for the moratorium is the developers' increasing concern at the slow speed of the user community's transition away from Python 2.x. The moratorium thread led to a larger discussion of where Python 3.x stands.

Progress on the transition can be roughly measured by looking at the third-party packages available for Python 3.x. Only about 100 of the 8000 packages listed on the Python Package Index claim to be compatible with Python 3, and many significant packages have not yet been ported (Numeric Python, MySQLdb, PyGTk), making it impossible for users to port their in-house code or application. Few Linux distributions have even packaged a Python 3.x release yet.

For the Python development community, it's tempting to nudge the users toward Python 3 by discouraging them from using Python 2. The Python developers have been dividing their attention between the 2.x and 3.x branches for a few years now, and a significant number of them would like to refocus their attention on a single branch. Given the slow uptake of Python 3, though, it's difficult to know when Python 2 development can stop. The primary suggestions in the recent discussion were:

1. Declare Python 2.6 the last 2.x release.
2. Declare Python 2.7 the last 2.x release.
3. After Python 2.7, continue with a few more releases (2.8, 2.9, etc.).
4. Declare the 3.x branch an experimental version, call it dead, and begin back-porting features to the 2.x branch.

Abandoning the 3.x branch had very few supporters. Retroactively declaring 2.6 the final release was also not popular, because people have been continuing to apply and backport improvements on the assumption that there was going to be a 2.7 release.

As Skip Montanaro phrased it:

A significant amount of work has already been committed to the 2.7 branch, as can be seen by reading "What's New in Python 2.7" or the more detailed NEWS file. New features include an ordered dictionary type, support for using multiple context managers in a single with statement, more accurate numeric conversions and printing, and several features backported from Python 3.1.

Clearly a 2.7 release will happen, and manager Benjamin Peterson's draft release schedule projects a 2.7 final release in June 2010. There's no clear consensus on whether to continue making further releases after 2.7. Post-2.7 releases could continue to bring 2.x and 3.x into closer compatibility and improve porting tools such as the 2to3 script, while keeping existing 2.x users happy with bugfixes and a few new features, but this work does cost effort and time. Brett Cannon stated his case for calling an end with 2.7:

Raymond Hettinger argued that imposing an end-of-life is unpleasant for users:

Hettinger is unmoved by the argument that maintaining 2.x takes up a lot of time, arguing that backporting a feature is relatively quick compared to the time required to implement it in the first place. He's also concerned that 3.x still needs more polishing, and concludes:

Assessment

Declaring such a long-term freeze on the language's evolution is a surprising step, and not one that developer groups often choose. Languages defined by an official standard, such as C, C++, or Lisp, are forced to evolve very slowly because of the slow standardization process, but Python is not so minutely specified. D'Aprano makes a good point that the developers are already pretty conservative; most suggestions for language changes are rejected. On the other hand, switching to Python 3.x is a big jump for users and book authors; temporarily halting further evolution may at least give them the sense they're not aiming for a constantly shifting target.

It's probably premature to call the transition to Python 3.x a failure, or even behind schedule. These transitions invariably take a lot of time and proceed slowly. Many Linux distributions have adopted Python for writing their administrative tools, making the interpreter critical to the release process. Distribution maintainers will therefore be very conservative about upgrading the Python version. It's a chicken-and-egg problem; third-party developers who stick to their distribution's packages can't use Python 3 yet, which means they don't port their code to Python 3, which gives distributions little incentive to package it. Eventually the community will switch, but it'll take a few years. The most helpful course for the Python developers is probably to demonstrate and document how applications can be ported to Python 3, as Martin von Löwis has done by experimentally porting Django to Python 3.x, and where possible get the resulting patches accepted by upstream.

It remains to be seen if a volunteer development group's efforts can be successfully redirected by declaring certain lines of development to be unwelcome. Volunteers want to work on tasks that are interesting, or amusing, or relevant to their own projects. The moratorium may lead to a perception that Python development is stalled, and developers may start up DVCS-hosted branches of Python that contain more radical changes, or move on to some other project that's more entertaining.

The nearest parallel might be the code freezes for versions 2.4 and 2.6 of the Linux kernel. The code freeze for Linux 2.4 was declared in December 1999, and 2.5.0 didn't open for new development until November 2001, nearly two years later. The long duration of the freeze led to a lot of pressure to bend the rules to get in one more feature or driver update.

Python's code freeze will be of similar length and there may be similar pressure to slip in just one little change. However, freezing the language still leaves lots of room to improve the standard library and the CPython implementation, enhance developer tools, and explore other areas not covered by the moratorium. Perhaps these tasks are enough of an outlet for creative energy to keep people interested.

Comments (50 posted)

Switching desktops surprisingly painless

Busy days are not uncommon here at LWN, but Tuesdays and Wednesdays are particularly full with writing and editing tasks for the weekly edition. Any kind of computer problem is most unwelcome on such days, whether that is because your ISP has decided to route your packets to Borneo, or because some critical piece of the desktop goes out to lunch. So, trying to wrestle with a Plasma—KDE 4's desktop shell—crash while under deadline pressure made for a rather stressful few hours.

The point here is not to bash on KDE, or Fedora, which is the distribution I tend to use, as the fault may well be my own. But, in the process of getting back to a working state, I made a surprising (to me, anyway) discovery: switching from KDE to GNOME was completely painless. The desktop wars would lead one to believe that there are such fundamental differences between the two dominant desktops that switching between them would be "entering a world of pain". Not so, at least for this relatively unsophisticated user.

Generally, I try and avoid dipping into the Fedora 10 update stream in the early part of any week, just for stability purposes. I also tend to ignore updates for several weeks at a time, which is part of what made the Plasma crash so surprising—I hadn't updated for roughly two weeks. It may well have been that I didn't restart the desktop after the last update, I often do yum update in some random Konsole and forget about it. So, when Plasma tried to restart, perhaps there was a newer version of it or libraries it depended on.

Back in January, I wrote an article about the KDE 4 transition and, in it noted that I had gone through the transition with few problems. In some ways, I was mystified by all of the uproar about KDE 4 as it more or less worked for me. Oddly, I think I now have a better perspective on what folks suffered at that time. If Plasma won't start, you don't really have a desktop; maybe there are various workarounds to switch workspaces or to particular windows (especially iconified windows) using just the keyboard, but I didn't have any time to figure that out.

Over the last year and a half or so—since switching to KDE 4—I had been having infrequent Plasma crashes. Roughly once a month, the desktop would briefly seize up, with the Panel disappearing and windows rearranging themselves, and then it would all come back. Sometimes, the KDE bug reporting tool would come up, but, since I don't run debug versions of the desktop applications, the tool made it clear that the reports weren't wanted. In fact, as I found out yesterday, reporting a bug through the tool requires having a bugs.kde.org login, which is an exceedingly annoying—unlikely to be surmounted by many—requirement.

This time, things were different, as Plasma went away and didn't come back. Since I had a lot of state in various workspaces (i.e. virtual desktops) that I didn't want to lose, I poked around on the net for ways to restart Plasma. None of those worked, so I saved off what I could and killed the X server with ctrl-alt-backspace, fully believing that logging in anew would clear up the problem. Not so; new logins would either just return to the login screen (after loading most of KDE based on the progress bar), or get to a screen with KDE bug tool reporting that Plasma had crashed.

So, it seemed like it might be time to upgrade or downgrade KDE to get back to a working state. I didn't have a lot of patience, so I just did:

    yum remove kde*

    yum groupinstall "KDE (K Desktop Environment)"

For a moment, I was kind of at a loss, trying to figure out the optimal approach for tracking down the breakage and somehow downgrading to a working version, or deciding to switch to the laptop—ironically running Rawhide without any problems—for the next few days. At that point I realized there was another option, switch over to using GNOME.

Now that probably is an obvious choice, in hindsight it certainly is, but when you are focused on a particular desktop, the alternative doesn't even enter your mind right away. Once I decided to do that, I immediately began to dread the amount of configuration and hassle that I expected it to take. I hadn't run GNOME in many years, perhaps as many as ten. I know lots of folks who run GNOME and had no reason to believe it was inferior in any way, but I did think, or led myself to believe, that it was different.

It took roughly five minutes of working with GNOME before I was largely unaware that I had switched. It may be that my use of the desktop is relatively minimal, though it is in use for many hours a day. I tend to have multiple Konsole windows (each with multiple tabs), a browser, the Gimp, claws-mail, and emacs up all the time, scattered around multiple workspaces. The only departures from standard GNOME were to move the menu bar to the bottom of the screen and to make windows auto-raise when the mouse pointer moves into them (something that has to be done for KDE as well).

I imagine I will revert back to KDE once I resolve whatever problem Plasma is having (and it may be as simple as yum update in a day or two), but I certainly won't look at the competing desktops for Linux in quite the same way. For any hardcore advocates of one desktop or the other, I would seriously encourage giving the other a try for a few hours some day. You may very well be surprised at how little difference, in terms of actually getting work done, there really is.

There are certainly lessons here for both Fedora and KDE, even though the problem may be partially of my own doing. It is hard to see how removing KDE and reinstalling from the F10 repositories didn't clear up this problem, unless there is some configuration in my home directory that causes it. Perhaps it is something in the Plasmoids (I did fiddle with the World Clock widget an hour or two before the crashes started) or Panel widgets that I have running. It is not easy to tell, nor is it particularly easy to Google for.

Plasma should (obviously) try to be more robust, and provide some better idea of where things might be going wrong—and what to do about it—but, certainly KDE should not be placing barriers between its users and its bug reporting system. I suspect it is an attempt to reduce some kind of bug report spam, but requiring a username/password for some "random" site is something that will stop bug reports from lots of users, including me.

Fedora has been concerned recently about the stability of upgrades. A recent issue with a Thunderbird upgrade, which caused much pain for users, is just one of a number of Fedora upgrade woes. This particular Plasma problem may be another, and, if so, will likely cause the project to focus on the upgrade stability issue even more.

Software has bugs, but desktop environments—much like the kernel—occupy a special place. If those pieces stop working, there is little the user can do to work around that fact. Usually, downgrading the kernel to the previously functioning one is an option, and it may be for desktops as well, but another alternative that free software brings is to switch to a different desktop (or even kernel, FreeBSD anyone?) entirely. That's a freedom worth having.

Comments (28 posted)

Nepomuk: sharing application metadata

The Nepomuk project has the potential to unlock the data from its originating application so that it can used by other applications on the desktop. If Nepomuk becomes pervasive, history logs, bookmarks, file metadata, email, instant messages, photo tags, or other metadata will be shared between various desktop applications. Why should music metadata like track length or artist and song title be locked away in an index created and used explicitly by a music playing application?

Consider a download assistant such as kget. The subversion branch of kget recently got the ability to store its download history using Nepomuk. kget could already save the transfer history in XML or SQLite. The advantage of using Nepomuk is that other desktop tools can easily see where a file was downloaded from and when; the information is unlocked from just kget. With Nepomuk, other applications don't need to know where the SQLite file is, or find and parse an XML file. All of the sudden, the file manager can let you know where this file came from so you can easily return for newer versions, or a desktop search can reveal all the files downloaded last year from http://example.com.

To allow data to be stored, exchanged, and understood by many applications, Nepomuk uses the same underlying technology that the Semantic Web is designed around. The Semantic Web tries to separate the data from the presentation in a way that allows for both humans and computers to inspect and digest the data. At the base of the Semantic Web is Resource Description Framework (RDF) which aims to allow metadata to be exchanged in an unambiguous, machine-processing-friendly format.

There are many who dismiss the Semantic Web as an ivory tower pipe dream. Various concerns are cited as reasons that RDF will not be adopted: it takes extra time to generate RDF data, it allows for automated comparisons which will make companies uncomfortable, and there will no agreement between companies on which schemas to use, etc.

Nepomuk and RDF have a huge potential on the Free and Open Source Software (FOSS) desktop because application developers have no vested interest in locking their data away, and due to the nature of free software, one can patch in RDF and/or Nepomuk support into projects. The latter problem about projects designing their own schema is still present for FOSS, but, luckily, schema mismatches in and of themselves are not a show stopper for RDF adoption. By definition, once data is in RDF it can be processed automatically by a computer, so the machine, rather than the human, can always work around schema differences.

RDF tries to capture information in the form of triples. The classic examples are relationships and ownerships, for example: "Mary knows Mark" and "dog has tail". To avoid name clashes for things described in RDF, longer URL style identifiers are used for the three pieces of information. To get back to smaller text strings for these URLs, prefixes are used in the style of XML namespaces. For example, foaf:name could be used for a human name which expands to the URL http://xmlns.com/foaf/0.1/name. This way, individual things can still be described concisely, but they should also have globally understood meaning. A foaf:name is a person's name, whereas a toolshed:name might name a screwdriver.

Below is an example of using Nepomuk from the command line to create and list an RDF file:

    $ sopranocmd --backend redland add \
	"<http://onto.libferris.com/things/1234>" "<http://onto.libferris.com/price>" "30"
    $ sopranocmd --backend redland add \
	"<http://onto.libferris.com/things/1234>" "<http://onto.libferris.com/title>" \
 	"super crazy magical item"

    $ sopranocmd --backend redland list
    <http://onto.libferris.com/things/1234> <http://onto.libferris.com/price> \
	"30"^^<http://www.w3.org/2001/XMLSchema#int> (empty)
    <http://onto.libferris.com/things/1234> <http://onto.libferris.com/title> \
	"super crazy magical item"^^<http://www.w3.org/2001/XMLSchema#string> (empty)
    Total results: 2
    Execution time: 00:00:00.1

While an RDF repository can be used to just store, update, and query these triples, a schema can also be imposed so that applications know what to expect. For example, that the foaf:homepage is a link to a web site with certain constraints. Examples of constraints include the type of data stored (integer, date, etc), how many times a property can appear (only one homepage), and so on.

The SPARQL query language can be used to join together the triples and select the information of interest. While SPARQL uses familiar SQL, like the SELECT, WHERE, ORDER BY, and LIMIT keywords, joining triples is a bit different than with SQL. For example, the query below grabs the price and title for "something". We don't particularly care what the something is, as long as the same something has a title and a price of less than 30.5.

    SELECT  ?title ?price
    WHERE   { ?x ns:price ?price .
	      FILTER (?price < 30.5)
	      ?x dc:title ?title . }

With all this talk of RDF, triples, and ivory towers, one might think that using Nepomuk and RDF will be painful and have an extremely long learning curve. Below are a few examples of using Nepomuk in a KDE application to quell those fears. Nepomuk makes using RDF simple because it provides a code generator that makes native C++ classes to allow interaction with the RDF store:

    Nepomuk::File f( "/home/foo/bar.txt" );
    f.setAnnotation( "This is just a test file that contains nothing of interest." );

The above is much neater than thinking in terms of the triples shown below which might be stored to represent it. In this case X will really be a persistent unique identifier used to identify the file, similar to the device number and inode in the kernel. The type, file etc will of course be longer URIs in the real RDF store.

    X type        file
    X url         "/home/foo/bar.txt"
    X annotation  "This is just a..."

The above example which uses setAnnotation() takes advantage of a schema for annotating and tagging files which comes with Nepomuk itself. The kget program mentioned earlier in the article is a good example of not using a standard schema. In the sources of kget, the transferhistorystore.cpp file manages the XML, SQLite, and Nepomuk representations of download history. At the end of transferhistorystore.cpp file, there is the following code:

    void NepomukStore::saveItem(const TransferHistoryItem &item)
    {
      Nepomuk::HistoryItem historyItem(item.source());
      historyItem.setDestination(item.dest());
      historyItem.setSource(item.source());
      historyItem.setState(item.state());
      historyItem.setSize(item.size());
      historyItem.setDateTime(item.dateTime());
    }

    void NepomukStore::deleteItem(const TransferHistoryItem &item)
    {
      Nepomuk::HistoryItem historyItem(item.source());
      historyItem.remove(); 
      ...

The HistoryItem class is generated by Nepomuk using the custom schema file kget_history.trig, part of which is repeated below. While the schema language that kget_history.trig is using may be unfamiliar, it should still be clear that there is a ndho:HistoryItem which has properties of various types with various restrictions on them, such as a destination property which can appear zero or one times and is a string. Given the below schema file, Neopmuk can generate the C++ class Nepomuk::HistoryItem needed to allow the above C++ code compile.

    <http://nepomuk.kde.org/ontologies/2008/10/06/ndho> {

	ndho:HistoryItem
	a rdfs:Class ;
	rdfs:comment "A kget history item." ;
	rdfs:label "application" ;
	rdfs:subClassOf rdfs:Resource .

	ndho:destination
	a rdf:Property ;
	rdfs:comment "Destination of the download." ;
	rdfs:domain ndho:HistoryItem ;
	rdfs:label "source" ;
	rdfs:range xsd:string ;
	nrl:maxCardinality "1" . 
	...

At the base of the Nepomuk project is the Soprano library and command line tools which depend only on QtCore, making them a useful RDF library for use on both desktop and mobile platforms. The Nepomuk libraries build on Soprano to make writing KDE applications using RDF simple. One of the great things about the design of Soprano is that there are multiple backends which can store and query RDF. So there can be a memory mapped implementation for a mobile device, or a full-blown database server for a LAN, and applications still use the same API.

For a long time Soprano has had two main backends: Redland and Sesame2. The former is a C library for RDF and the latter a Java implementation. While Sesame2 is written in Java it can deliver better query performance than Redland. This left KDE4 in the predicament that it required Java to achieve good RDF performance. To solve this issue the new Virtuoso backend was created and is getting to the point where it is now stable. As I discovered recently, the main impediment to developing a backend for soprano is implementing SPARQL.

Adoption still remains the major hurdle for Nepomuk and Soprano. With the host of persistence options available, the first thing that comes to an application developer's mind might be flat files, MySQL, Sqlite, Berkeley DB, or some generic relational database library, when wanting to store and retrieve data. However, when storing data that might be of interest to other applications, using Nepomuk or Soprano has the potential to unlock an application's data. As can be seen above, the main thing to learn is a bit about the schema language and then native C++ objects can be used to interact with Nepomuk from an application.

Comments (8 posted)

Page editor: Jonathan Corbet

Security

What lessons can be learned from the iPhone worms?

Word spread quickly earlier this week when the first known worms targeting the Apple iPhone surfaced days apart in Australia and the Netherlands. Neither inflicts serious harm on devices — though perhaps the Australian "Ikee" worm is guilty of crimes against good taste, as it swaps out the iPhone's wallpaper for a vintage photo of Rick Astley. The events are notable, however, because they only affect jailbroken iPhones, raising questions over whether the iPhone jailbreaking community behaves responsibly when it frees the devices from Apple's factory restrictions.

Ikee was created on November 4 by Ashley Towns, a programmer from Wollongong, Australia. The worm propagates by scanning IP ranges in the blocks used by the iPhone's Australian carrier, checking for iPhone OS fingerprints, and looking for a running SSH daemon on any iPhones it finds. Because all iPhones ship from the factory with the same default root password, "alpine", the worm can connect, copy itself over to the new device, install its signature wallpaper, and repeat. Ikee also deactivates SSHd on the host phone as part of its payload, but it does not change the root password. Thus, restarting SSH makes the phone vulnerable to reinfection.

It attracted considerably less public attention than Ikee, but on November 2, a worm surfaced in the Netherlands using the exact same attack vector: IP range scanning of the approved 3G carrier, OS fingerprinting, and connecting via SSH using the default password. The Dutch worm lacked the campy sensibility of Ikee; rather than Rickrolling the iPhone's wallpaper, it popped-up a message telling the user that the iPhone was insecure and asking €4.95 for instructions on how to secure it. That same day, however, the author changed his mind and posted both an apology and free instructions for securing the phone on the web site to which the worm pointed its victims.

Jailbreaks are risky. Much like in real life.

"Jailbreaking" refers to hacks that enable full access to the iPhone's operating system, and is distinct from SIM unlocking, which removes carrier-restrictions that limit the device to registering on only approved 3G networks. The iPhone has multiple layers of security built into the OS, including everything from boot loader restrictions to application code signing. The term "jailbreak" originates from the fact that all iPhone applications run inside a chroot jail; when connecting the iPhone to a Mac or PC only the contents of the chroot jail are accessed by the iTunes application for installing and removing media content and applications.

Consequently, breaking out of the chroot jail is required to perform any real OS customization, and the popular jailbreaking utilities set up niceties that most hackers will expect on a Unix-like system like OS X. That includes SSH, which obviates the need to keep the phone attached to a PC by cable while working on it.

It is Apple's decision to ship all models of its iPhone with the same root password — a tactic common to embedded device makers, if not particularly secure. The point of debate is whether the security hole left open by the combination of a default root password and a running SSH daemon is Apple's fault, the jailbreaking tool authors', or simply the users'.

Searching for solutions

Changing the password is simple enough; should PwnageTool, PurpleRa1n, and the other jailbreak utilities do so upon installation, either by automatically generating a new password or by prompting the user? An Australian blogger who interviewed Towns about Ikee thinks so; he posted an opinion piece following up on the interview in which he asks the jailbreaking tool developers to to do just that.

From reading the various news sites' discussions, it is clear that plenty of others disagree, noting that the responsibility for securing the device stops with the end user. But is there any substantial risk to changing the root password on the device, when measured against the risks of a default password exploit? Default password exploits are well-known enough that most end-user applications have moved away from them; other than the fact that the iPhone is an embedded system, why should it be measured by different standards than a desktop or server, or for that matter, a PayPal account?

For comparison's sake, Android phones do not include a password-enabled root account, and although changing this is well-documented, it is a complicated process even for the comparatively unlocked Android Dev Phone 1. Similarly, installing a working SSH daemon is not a straightforward process even on a rooted phone. On the other hand, Maemo offers OpenSSH (both client and server) as a one-click install, but installing it automatically prompts the user to change the root password from its default.

ComputerWorld's Robert McMillan speculates that Apple cares little about the issue — jailbroken phones are not covered by warranty, so there is no incentive for the company to close its part of the security hole by changing its root password policy. In fact, it may tacitly approve of the negative press generated around jailbreaking.

On the other hand, as the author of the Dutch worm put it, "the way I got access to your iPhone can be used by thousands of others. And they can send text messages from your number (like I did..), use it to call (or record your calls), and actually whatever they want [...]" The next exploit taking advantage of the iPhone's default root password may not require a jailbroken device. Apple will surely sit up and take notice then. In the meantime, though, the question remains: should the people who care about freeing a device from the factory's proprietary lock-downs also care about putting it into a secure state once it is free?

While some FOSS-based phones may not require jailbreaking — or not enable SSHd as part of the process — there are warnings here for Google, Nokia, Palm, and others. By forcing users to perform dodgy operations on their phones, in order to enable the functionality they want, phone makers may very well be putting those users at risk. Whether the underlying code is open source or proprietary doesn't alter that risk.

Comments (15 posted)

Brief items

Vixie: What DNS Is Not

Comments (none posted)

New vulnerabilities

alienarena: buffer overflow

Comments (none posted)

apache: man-in-the-middle/SSL injection

Comments (none posted)

cups: cross-site scripting

Comments (none posted)

drupal6: multiple vulnerabilities

Comments (none posted)

horde: cross-site scripting

Comments (none posted)

java: multiple vulnerabilities

530053 - CVE-2009-3873 OpenJDK JPEG Image Writer quantization problem (6862968)
530057 - CVE-2009-3875 OpenJDK MessageDigest.isEqual introduces timing attack vulnerabilities
(6863503)
530061 - CVE-2009-3876 OpenJDK ASN.1/DER input stream parser denial of service (6864911)
CVE-2009-3877
530062 - CVE-2009-3869 OpenJDK JRE AWT setDifflCM stack overflow (6872357)
530063 - CVE-2009-3871 OpenJDK JRE AWT setBytePixels heap overflow (6872358)
530067 - CVE-2009-3874 OpenJDK ImageI/O JPEG heap overflow  (6874643)
530098 - CVE-2009-3728 OpenJDK ICC_Profile file existence detection information leak (6631533)
530173 - CVE-2009-3881 OpenJDK resurrected classloaders can still have children (6636650)
530175 - CVE-2009-3882 CVE-2009-3883 OpenJDK information leaks in mutable variables
(6657026,6657138)
530296 - CVE-2009-3880 OpenJDK UI logging information leakage(6664512)
530297 - CVE-2009-3879 OpenJDK GraphicsConfiguration information leak(6822057)
530300 - CVE-2009-3884 OpenJDK zoneinfo file existence information leak (6824265)
532904 - CVE-2009-3729 JRE TrueType font parsing crash (6815780)
532906 - CVE-2009-3872 JRE JPEG JFIF Decoder issue (6862969)
532914 - CVE-2009-3886 JRE REGRESSION:have problem to run JNLP app and applets with signed Jar
files (6870531)
533211 - CVE-2009-3865 java-1.6.0-sun: ACE in JRE Deployment Toolkit (6869752)
533212 - CVE-2009-3866 java-1.6.0-sun: Privilege escalation in the Java Web Start Installer
(6872824)
533214 - CVE-2009-3867 java-1.5.0-sun, java-1.6.0-sun: Stack-based buffer overflow via a long file:
URL argument (6854303)
533215 - CVE-2009-3868 java-1.5.0-sun, java-1.6.0-sun: Privilege escalation via crafted image file
due improper color profiles parsing (6862970)

Comments (none posted)

kernel: integer overflow

Comments (none posted)

libvorbis: arbitrary code execution

Comments (none posted)

qtwebkit: arbitrary local file access

Comments (none posted)

typo3: multiple vulnerabilities

Comments (none posted)

wordpress-mu: denial of service

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

The current stable kernel is 2.6.31.6, released (along with 2.6.27.39) on November 9. Both kernels contain a long list of important fixes, including some which are security-related.

Comments (none posted)

Quotes of the week

Comments (1 posted)

The block I/O controller

This first small step is a 20-part patch series implementing a proportional-weight I/O controller within the CFQ I/O scheduler. A new control group (cgroup) class (called "blkio") is created, with each group given an I/O weight between 100 and 1000. The available I/O bandwidth is then handed out to groups as determined by their configured weight.

Working within CFQ gives the I/O controller fine-grained control over how much bandwidth each group gets. On the other hand, CFQ is poorly positioned to identify block I/O traffic which is not directly generated by user space (writeback, for example) or to throttle activity at the virtual memory level. Achieving that will require the integration of a controller which hooks into the system at a much higher level. The plan is to write such a controller and have it work transparently within the same blkio cgroup. But a couple more small steps will probably be required before that is ready to go in.

Comments (1 posted)

Removing binary sysctl

The problem with removing sysctl(), of course, is that it is part of the kernel ABI. As long as the possibility of broken applications exists, this ABI cannot be removed. So it continues to sit in the kernel, despite the fact that its absence would be noted by few people.

Eric Biederman has come up with a new approach to the problem. His patch set removes the current sysctl() implementation, getting rid of a few thousand lines of unloved code. He then adds back a new wrapper which emulates the sysctl() ABI by way of /proc/sys. So any applications using sysctl() should continue to work, but the code dedicated to making it work is much reduced from what was there before.

The patch set still concerns some developers. The compatibility wrapper has its own configuration option, leading some to worry that distributions might disable it and cause obscure things to break. Going through /proc/sys will make access to these variables much slower than it was before. That should not really be a problem: access to sysctl variables is not normally a performance-critical operation. So there does not appear to be any sort of real obstacle to the merging of these patches; maybe, someday, binary sysctl() will truly vanish into the past.

Comments (4 posted)

Next-generation Linux file systems: NiLFS(2) and exofs (developerWorks)

Comments (8 posted)

Kernel development news

Supporting transactions in btrfs

Btrfs has tried to make this capability available to user space by way of the BTRFS_IOC_TRANS_START and BTRFS_IOC_TRANS_END ioctl() calls. There are some real problems with this approach, though. They operate as a pair of system calls, with any operations between the two being treated as a transaction within the filesystem. But if something fails, or if the application never quite gets around to ending the transaction, things will eventually come to a halt. That is why the btrfs code includes this comment:

It is, in other words, a dangerous capability which cannot be made generally available.

Sage Weil has posted a patch taking a rather different approach to the problem. The key idea is to avoid the problem of never-completed transactions by encapsulating the entire thing within a single system call. The result is a new BTRFS_IOC_USERTRANS command for ioctl(); chances are it will require a bit of work yet, but it could be the base for user-space transactions in the future.

This command takes a structure which looks something like the following:

    struct btrfs_ioctl_usertrans {
	__u64 num_ops;
	struct btrfs_ioctl_usertrans_op *ops_ptr;
	__u64 num_fds;
	__u64 data_bytes, metadata_ops;
	__u64 flags;
	__u64 ops_completed;
    };

The ops_ptr argument points to an array of num_ops individual operations to complete:

    struct btrfs_ioctl_usertrans_op {
	__u64 op;
	__s64 args[5];
	__s64 rval;
	__u64 flags;
    };

Here, op describes an individual operation. It can be BTRFS_IOC_UT_OP_OPEN (open a file), BTRFS_IOC_UT_OP_CLOSE (close a file), BTRFS_IOC_UT_OP_PWRITE (write to a file), BTRFS_IOC_UT_OP_UNLINK (unlink a file), BTRFS_IOC_UT_OP_LINK (make a link to a file), BTRFS_IOC_UT_OP_MKDIR (create a directory), BTRFS_IOC_UT_OP_RMDIR (remove a directory), BTRFS_IOC_UT_OP_TRUNCATE (truncate a file), BTRFS_IOC_UT_OP_SETXATTR (set an extended attribute), BTRFS_IOC_UT_OP_REMOVEXATTR (remove an extended attribute), or BTRFS_IOC_UT_OP_CLONERANGE (copy a range of data from one file to another). For each operation, the args field contains a set of arguments similar to what one would see for the corresponding system call. One interesting difference is that there are no hard-coded file descriptor numbers; instead, the transaction gets a new file descriptor table and all operations work with indexes into that table. Essentially, transactions work within a file descriptor space separated from that used by the calling process.

The flags field describes how the return value from each operation should be interpreted. It can be contain any of: BTRFS_IOC_UT_OP_FLAG_FAIL_ON_NE, BTRFS_IOC_UT_OP_FLAG_FAIL_ON_EQ, BTRFS_IOC_UT_OP_FLAG_FAIL_ON_LT, BTRFS_IOC_UT_OP_FLAG_FAIL_ON_GT, BTRFS_IOC_UT_OP_FLAG_FAIL_ON_LTE, and BTRFS_IOC_UT_OP_FLAG_FAIL_ON_GTE. In each case, the flag causes the return value to be compared against the passed-in rval field; if the comparison is successful, the transaction will fail.

What happens if the transaction fails? The partially-completed transaction will not be rolled back; btrfs, not being a database, is not really set up to do that. Instead, the number of successfully-completed operations will be passed back to user space. Optionally, the application can provide the BTRFS_IOC_UT_FLAG_WEDGEONFAIL flag, causing btrfs to leave the transaction open, locking up the filesystem until the system is rebooted. This may seem like a rather antisocial approach to transaction atomicity, but, if failure is both highly unlikely and highly problematic, that might be the right thing to do.

A patch like this raises a lot of questions. The first obstacle may be the fact that it requires exporting a number of system call implementations to modules, a change which has been resisted in the past. Kernel code need not normally call functions like sys_mkdir(), but this patch does exactly that. Calling system call implementations directly can be a bit tricky on some architectures, and there are good reasons for not making these functions available to modules in general.

Another problem is that the filesystem has no real way of determining whether a transaction will succeed before jumping into it; the best it can do is reserve some metadata space in accordance with an estimate provided by the application. If transactions are allowed to complete partially, they are no longer transactions. But the alternative of locking up the system can only leave people wondering if there isn't a better way.

Then, there is a question which was raised on the list: btrfs provides cheap snapshots, why not use them to implement transactions? Using a snapshot would take advantage of existing functionality and should make proper rollback possible. The problem would appear to be performance: btrfs snapshots are not quite that cheap, especially when one considers the need to exclude other activity on the filesystem while the transaction is active. So Chris Mason has suggested that the old standby - write-ahead logging - would be preferable because it will perform much better. But, he thinks, the multi-operation ioctl() could maybe perform better yet.

Finally, there would appear to be some real similarities between this API and the rather more general syslets mechanism. Syslets have been on the back burner for some time now, but they could come back forward if they seemed like a good way to solve this problem.

Clearly, like much of btrfs, this new ioctl() is a work in progress. If it gets into the mainline, it will be likely to have changed quite a bit on the way. But the problem this patch is trying to solve is real; it's clearly an issue which is worth thinking about.

Comments (18 posted)

Another new ABI for fanotify

The first obstacle has been more-or-less overcome. Even developers who think that malware scanning is the worst sort of security snake oil can agree that having these utilities use a well-defined kernel interface is better than having them employ nasty tricks like hooking into the system call table. ABI difficulties can be harder to overcome, though. With the latest fanotify posting, developer Eric Paris may have resolved this issue for at least a portion of the fanotify functionality.

The new version does away with the novel interface using setsockopt() in favor of a couple of new system calls. The first of these is fanotify_init():

    int fanotify_init(unsigned int flags, unsigned int event_f_flags,
		      int priority);

This system call initializes the fanotify subsystem, returning a file descriptor which is used for further operations. There are two flags values implemented: FAN_NONBLOCK creates a nonblocking file descriptor, and FAN_CLOEXEC sets the close-on-exec flag. Currently, event_f_flags and priority are unused; they should be set to zero.

Management of notification events is then done with fanotify_mark():

    int fanotify_mark(int fanotify_fd, unsigned int flags,
		      int dfd, const char *pathname, u64 mask,
		      u64 ignored_mask);

This call is used to "mark" specific parts of the filesystem hierarchy, indicating an interest in events involving those files. fanotify_fd is the file descriptor returned by fanotify_init(). The flags parameter must be one of FAN_MARK_ADD or FAN_MARK_REMOVE, indicating whether this call adds new marks or removes existing ones; there are also a couple of flags to control following of symbolic links and the marking of directories (without their contents).

The file(s) to be marked are determined by dfd and pathname; these parameters work much like in any of the *at() system calls. If dfd is AT_FDCWD, the pathname is resolved using the current working directory. If, instead, dfd points to a directory, the pathname lookup starts at that directory. If pathname is null, though, then dfd is interpreted as the actual object to mark.

Finally, mask and ignored_mask control which events are reported. To generate a specific event, a file must have the appropriate flag set in mask and clear in ignored_mask. The flags are FAN_ACCESS (file access), FAN_MODIFY (a file is modified), FAN_CLOSE_WRITE (a writable file has been closed), FAN_CLOSE_NOWRITE (a read-only file has been closed), FAN_OPEN (a file has been opened), and FAN_EVENT_ON_CHILD (events on children of a directory). There is also a FAN_Q_OVERFLOW event for event queue overflows, but that is not currently implemented.

Once files have been marked, the application can simply read from the fanotify file descriptor to get events. The events look like:

    struct fanotify_event_metadata {
	__u32 event_len;
	__u32 vers;
	__s32 fd;
	__u64 mask;
    };

Here, event_len is the length of the structure, vers indicates which version of fanotify generated the structure, fd is an open file descriptor for the object being accessed, and mask describes what is actually happening.

There is one crucial component missing in these patches: there is no way for the fanotify user to react to these events. In particular, the ability to block an open() call, a core part of the malware-scanning process, is missing. That, presumably, is to be added in a future revision. Meanwhile, Eric has requested permission to put the notification code into linux-next, presumably with a 2.6.33 merge in mind. As of this writing, objections have not been forthcoming.

Comments (11 posted)

A futex overview and update

Futexes are the basis of several mutual exclusion constructs commonly used in threaded programming. These include pthread mutexes, condvars, semaphores, rwlocks, and barriers. They have been through a lot of reconstructive and cosmetic surgery over the last several years, and are now more efficient, more functional, and better documented than ever before.

Overview

While few application developers will use futexes directly, a cursory knowledge of how to do so is necessary to appreciate the improvements presented a bit later. By way of a simple example, futexes can be used to store the state of a lock and provide a kernel waitqueue for tasks blocking on the lock. To minimize syscall overhead, this state should allow for atomic lock acquisition when the lock is uncontended. The state could be defined as:

0. unlocked
1. locked

In order to acquire the lock, an atomic test-and-set instruction (such as cmpxchg()) can be used to test for 0 and set to 1. In this case, the locking thread acquires the lock without involving the kernel (and the kernel has no knowledge that this futex exists). When the next thread attempts to acquire the lock, the test for zero will fail and the kernel needs to be involved. The blocking thread can then use the futex() system call with the FUTEX_WAIT opcode to put itself to sleep on the futex, passing the address of the futex state variable as an argument. To release the lock, the owner changes the lock state to zero (unlocked) and issues the FUTEX_WAKE opcode, which will wake the blocked thread to return to user space and try to acquire the lock (as described above). This is a an obviously trivial example with lots of room for optimization. Ulrich Drepper's "Futexes are Tricky" [PDF] is still the undisputed reference for using futexes to build locking primitives such as mutexes. It explores the many race conditions involved with using futexes as well as optimizations to improve on the example given here.

When the user threads call into the kernel with the futex() system call, they pass the address of the futex state (uaddr), the opcode to perform (op), and various other arguments. The uaddr is used by the kernel to generate a unique "futex_key" to reference the futex. When a thread requests to block on a futex, as with FUTEX_WAIT, a "futex_q" is created and queued in the "futex_queues" hash table. There is one futex_q for every task blocked on a futex, possibly many futex_q's per futex. The futex_queues themselves (the hash table lists, not the "futex_q's") are shared among futexes, since multiple futex_keys will hash to the same queue. These relationships are illustrated below:

In most cases, there is no policy defining how the user space state variable is to be used (despite what the futex man pages may or may not say). The application (or a library such as glibc) uses this value to define the state of the locking construct being implemented. This can be as simple as a boolean variable (as in the example above), however optimized implementations and other locking mechanisms require more complex state values.

In addition to the simple FUTEX_WAIT and FUTEX_WAKE operations, the kernel also manages special operations that require more knowledge of the locking construct's state than can be had in user space, most notably the priority inheritance (PI) chains and robust lists. PI and robust futexes are exceptions to the user-defined-policy rule regarding the state variable. Their state depends not only on the locked state of the mutex, but also on the identity of the owner and whether or not there are waiters. As such, the futex value is defined as the thread identifier (TID) of the owner and a bit to indicate pending owners. This policy still allows for user space atomic operations to avoid calling into the kernel in the uncontended case.

Improvements

Futexes have seen numerous improvements from a handful of developers since their debut appearance in 2.5.7. Some notable improvements include priority based wake-up for real-time tasks (by Pierre Peiffer) and robust and PI futexes (by Ingo Molnar and Thomas Gleixner). These latter features have been in use for some time and have been adequately covered here on LWN.net as well as in the excellent discussions on the kernel mailing list. Your author's foray into futexes picks up here, about two and a half years ago. Aside from several fixes to address rare corner cases and race conditions, the futex code has seen several functional and performance improvements since those earlier contributions.

Significant effort went into reducing futex overhead. Eric Dumazet introduced private futexes as an optimization for PTHREAD_PROCESS_PRIVATE pthread mutexes. Private futexes can only be used by threads of the same process. They are distinguishable from each other simply by their virtual address, while shared futexes have different virtual addresses in each process, requiring the kernel to lookup their physical address for unique identification. This optimization eliminates the use of the mmap_sem semaphore for private futexes, reducing system-wide contention. It also eliminates the atomic operations used in reference counting for private futexes, resulting in less cache-line bouncing on SMP machines. Glibc now uses private futexes by default.

Peter Ziljstra further reduced the futex dependency on mmap_sem by using get_user_pages_fast() in the fast paths, making use of get_user_pages(), and pushing the mmap_sem locks down tightly around the slow paths (September 2008). These changes had the added benefit of removing much of the virtual memory related logic from kernel/futex.c, simplifying the code considerably. Due to their dependence on user space addresses, futexes are burdened with several possible fault points. Holding mmap_sem complicated the fault logic since it had to be released prior to calling get_user(). With mmap_sem usage reduced, your author greatly simplified the fault logic (March 2009), resulting in far more legible code.

Bitset conditional wakeup was added by Thomas Gleixner (February 2008) in order to enable optimized rwlock implementations in glibc. FUTEX_WAIT_BITSET and FUTEX_WAKE_BITSET allow the user to specify a bitmask which limits the woken tasks to those which specified the same bitset (or a superset, such as FUTEX_BITSET_MATCH_ANY) at wait time.

Since the introduction of PI futexes, the glibc condvar implementation of pthread_cond_broadcast() (with a PI mutex) has been forced to wake all waiters, rather than take advantage of FUTEX_REQUEUE, due to the lack of support for requeueing to PI futexes. This leads to a wake-up storm as all the waiters race back to user space to contend for the lock. It also fails to ensure that the highest priority task acquires the lock first. Recent kernels (2.6.31-rt* and 2.6.32-rc*) now have your author's FUTEX_CMP_REQUEUE_PI patches (April 2009) which provide the kernel-side support for requeueing waiters from a non-PI futex to a PI futex. With glibc patches in the works by Dinakar Guniguntala, real-time applications will soon be able to use pthread condition variables with guaranteed wake-up order and fewer overall wake-ups.

Now What?

While there are several items that futex developers may consider in the future, they are hopeful that kernel/futex.c and all its brain-bending, liver-killing insanity can be put to rest for at least a little while. However, since no article is complete without a list of next steps, the following items may receive some attention in the future:

Man pages: The current man pages do not include some of the new futex operations. They suggest a policy for the value of the futex which has led to some confusion regarding usage of futexes. Worst of all, the user space futex() definition has been removed from /usr/include/linux/futex.h, rendering the man pages not only incomplete, but also inaccurate. Users of futexes must use the syscall interface directly.

Adaptive futexes: It is possible that some of the scheduling overhead of futexes can be reduced by some optional amount of spinning prior to going to sleep in the kernel. However, as futexes expose their state to user space, this spinning can also be done in user space, as is done with adaptive mutexes in glibc now, albeit without the knowledge of whether the owner is running, so spinning is reduced to a simple maximum-retries loop.

Interruptible futexes: There is some interest in interruptible blocking lock operations from large proprietary software projects. Futex operations currently restart themselves in the event of a signal, rather than returning -EINTR to user space. Futexes could be flagged with FUTEX_INTERRUPTIBLE which would be checked on signal-induced wakeup to determine if the syscall should be restarted or if -ECANCELED should be returned to user space. Exposing such a feature in the pthread locking primitives would involve non-POSIX compliant changes to the pthread library, but this is not without precedent.

Scalability enhancements: There has been some discussion on LKML regarding private as well as NUMA-optimized hash tables. The futex hash table is shared across all processes and is protected by spinlocks which can lead to real overhead, especially on large systems. This overhead is not serving any useful purpose if these systems are partitioned on NUMA nodes, or even for processes that use private futexes exclusively.

Futex test suite: Your author has been compiling a list of requirements for an exhaustive test suite to validate futex functionality. This test suite would serve as a regression suite for future development. The many corner cases and misuse cases possible with futexes complicate the test suite and present a challenge to its design.

Acknowledgements

I would like to extend my thanks to John Stultz, Will Schmidt, Paul McKenney, Nivedita Singhvi, and, of course, Jon Corbet, whose reviews have made this article far more legible and complete than it would have been otherwise.

Comments (6 posted)

Patches and updates

Kernel trees

• Greg KH: Linux 2.6.31.6 . (November 10, 2009)

• Thomas Gleixner: 2.6.31.5-rt17 . (November 6, 2009)

• Greg KH: Linux 2.6.27.39 . (November 10, 2009)

• Willy Tarreau: Linux 2.4.37.7 . (November 7, 2009)

Build system

• nir.tzachar@gmail.com: nconfig v5 . (November 6, 2009)

• Bernhard Kaindl: user remarks for config symbols . (November 7, 2009)

• Albin Tonnerre: Add support for LZO-compressed kernel images . (November 11, 2009)

Core kernel code

• Sukadev Bhattiprolu: Implement clone_with_pids() system call . (November 5, 2009)

• Sukadev Bhattiprolu: Implement eclone() syscall . (November 11, 2009)

• Eric W. Biederman: generic compat_sys_sysctl . (November 7, 2009)

• Bharata B Rao: CFS Hard limits - v3 . (November 9, 2009)

Development tools

• Hitoshi Mitake: Adding general performance benchmarking subcommand to perf. . (November 5, 2009)

• Frederic Weisbecker: hw-breakpoints: Rewrite on top of perf events v5 . (November 6, 2009)

Device drivers

• Tomi Valkeinen: OMAP: DSS2 v5 intro . (November 9, 2009)

• Linus Walleij: Add COH 901 318 DMA block driver v4 . (November 11, 2009)

• Paul Fertser: power: implement platform battery driver . (November 11, 2009)

Filesystems and block I/O

• Miklos Szeredi: vfs: new O_NODE open flag . (November 5, 2009)

• Boaz Harrosh: exofs: Multi-device mirror support . (November 11, 2009)

• Sage Weil: btrfs: big fat transaction ioctl . (November 11, 2009)

Janitorial

• Thomas Gleixner: Remove old style lock initializers . (November 7, 2009)

• Eric W. Biederman: Removal of binary sysctl support . (November 9, 2009)

Memory management

• Christoph Lameter: Accessors for mm locking . (November 5, 2009)

• Hugh Dickins: mm: prepare for ksm swapping . (November 11, 2009)

Networking

• Tom Herbert: rps: Receive packet steering . (November 11, 2009)

Architecture-specific

• Wu Zhangjin: ftrace for MIPS . (November 9, 2009)

Security-related

• John Johansen: AppArmor security module . (November 11, 2009)

• Tetsuo Handa: TOMOYO Linux version 1.7.1 released. . (November 11, 2009)

Virtualization and containers

• Gregory Haskins: AlacrityVM v0.2 is released . (November 6, 2009)

• Patrick Mullaney: vbus: venet macvlan support . (November 11, 2009)

Miscellaneous

• Zheng, Jeff: wifi-test 1.0 released . (November 11, 2009)

• Kay Sievers: udev 147 release . (November 11, 2009)

• Jozsef Kadlecsik: ipset 4.0 released . (November 11, 2009)

Page editor: Jonathan Corbet

Distributions

News and Editorials

Mandriva 2010

Mandriva 2010.0 is the latest achievement of the Mandriva community. According to the features page it is faster and sexier than ever. Released earlier this month, Mandriva 2010.0 has some interesting new features including Moblin, Sugar, and the so-called "Smart Desktop".

As usual, Mandriva is shipped in three flavors - One, Free and PowerPack. The Flash drive edition was not available when this article was written, but should be available soon. While PowerPack and One contain some non-free packages like drivers or patent encumbered multimedia codecs, Free is 100% freedom software. Mandriva recommends the free edition to advanced users, so that is what we reviewed.

Mandriva Free is available as an installation DVD for 32 (i586) and 64 bit architectures, together with a dual arch DVD which seems to be the same as a 32 bit one. At the time this article was written the reviewers guide had no information about this.

Mandriva has a tradition of being user friendly, with a touch of simplicity. The installer is no exception. It offers a partition setup based on the available free space in case there are other operating systems and data. It is also possible to configure a custom disk setup. KDE is chosen by default, with GNOME, other desktops and all available software on the DVD as an option. We chose all the defaults and the installation finished quickly. Advanced options are there for the users who need them.

Since the 100% free software edition was used, no non-free drivers or firmware were installed. Therefore my Thinkpad T61, which was used for the tests, was left with no hardware acceleration for the Nvidia Quadro graphics and no wireless connection with its Intel 4965 chip. After the non-free repository was added to rpmdrake, the necessary drivers and firmware became available for installation. A few clicks and an X restart made the hardware usable. Rpmdrake was very responsive on my system, starting and performing very quickly.

No more KDE3

KDE3 and its applications, such as Quanta Plus, have been discontinued in Mandriva 2010. The default setup should be familiar to KDE3 users, however, with the desktop set to Folder View, and the menu in the classic layout. While the default setup provides a safe landing for KDE3 users, the full KDE4 desktop is only a few clicks away.

The Smart Desktop is one of the most interesting features in Mandriva 2010. In a nutshell, the Smart Desktop tries to build a semantic, task based desktop on top of KDE's Nepomuk service. In order to be usable, Smart Desktop requires both Nepomuk and Stringi to be installed and running. Interaction with the user happens through the Task Management Widget and a program called Tasktop.

The concept is simple—files and information are gathered around a common context, called a task. The task is created through the widget or Tasktop, and allows the user to associate contacts, files, emails and web pages to the task. The service is integrated with the Dolphin file manager, which associates the files to the task through the Actions menu. Dolphin can list all the files associated with a certain task, and can add files to a task with a right click. Tasktop also has an 'Open in Dolphin' option, which opens a file manager window with all of the task's associated files, contacts, emails and web pages. Open/Save dialogs are also integrated into the Smart Desktop's task based semantic mode. For example you can save a file to a task using Dolphin.

The theory behind the Smart Desktop is very interesting. In practice the service is very new and other KDE applications have yet to be integrated. It would be nice to have an option to associate a contact to the task directly from the Kontact, or to do the same with a bookmark from Konqueror, but it's not there yet. It would be even nicer if this support were integrated into other applications such as GNOME PIM or Mozilla.

Sweet and mobile

Mandriva 2010.0 is a multi-purpose desktop. Beside the popular desktops like KDE and GNOME, Mandriva ships Sugar and Moblin desktops, available for installation through the task-sugar and task-moblin packages, according to the 2010 tour. Due to a conflict with the etoys package, task-sugar wasn't immediately available for installation.

Fortunately, task-moblin installed the Moblin GUI without any problems. Moblin is designed for netbooks, but the interface can be used on other hardware. Mandriva also claims to support a wide range of netbooks, so netbook users should have no problem using Moblin on Mandriva.

Other

The Mandriva Control Center has been updated. The network profiles management tool was redesigned and net_monitor was rewritten from scratch. Package management and updating is more usable and configurable now. The package search function is now able to use full names, versions and releases. The update system offers more configurable frequency settings. The Tomoyo security framework replaces AppArmor and is configurable with the Tomoyo-gui. New partitions are formatted as Ext4. Plymouth provides a pretty boot process, and if you like to benchmark you can install the phoronix-test-suite package.

Mandriva actively advertises Moovida, the renamed Elisa multimedia center with a new graphical user interface. Mandriva positions itself as a home multimedia center capable distribution, too.

Mandriva 2010.0 features the latest versions of many popular applications. KDE 4.3, GNOME 2.28, OpenOffice.org 3.1.1, Firefox 3.5.5, and Linux 2.6.31 are just some of the examples.

Conclusion

We experienced some small glitches during initial setup, the desktop froze during font configuration and there was an offer to upgrade to 2009.1, but these glitches didn't persist after the initial setup. Mandriva manages to pack a high quality desktop. Their careful approach with KDE4 settings will hopefully be enough compromise for those who dislike it, while the performance and responsiveness bring nothing but joy. The Smart Desktop is certainly a innovation and the most interesting part of this release.

Comments (4 posted)

New Releases

Moblin v2.1 project release for netbooks and nettops

Full Story (comments: none)

Distribution News

Fedora

Fedora's CC license changeover complete

Full Story (comments: none)

Nominations now open for December Fedora Elections

Full Story (comments: none)

F13 Naming: Leonidas -> Constantine -> <New Name>?

Full Story (comments: none)

Fedora Board Recap 2009-11-05

Full Story (comments: none)

Ubuntu family

Debian and Ubuntu now use the same patch tagging guidelines

Full Story (comments: none)

Lucid translation imports are now active

Full Story (comments: none)

Distribution Newsletters

DistroWatch Weekly, Issue 328

Comments (none posted)

Fedora Weekly News 201

Full Story (comments: none)

Ubuntu Weekly Newsletter #167

Full Story (comments: none)

Page editor: Rebecca Sobol

Development

]Project Open[ for Enterprise Project Management

]Project Open[, or ]po[ is a cross-platform project management tool that is integrated within an enterprise resource planning system. It can be used for project planning, tracking, controlling and invoicing. ]po[ is designed in a modular fashion, individual packages can be installed and removed at runtime. The ]Project Open[ Company is located in Spain and is under German management. From the in a Nutshell document:

The depth of functionality provided by ]Project Open[ is shown by the extensive list of modules that can be used for performing the many operations. Key module categories include Projects, Finance, Customers, Collaboration, Human Resources, Providers, Knowledge Management, Reporting and Translation. The ]po[ Architecture FAQ explains the project's organization relative to the required open-source project dependencies. ]po[ requires the AOLServer web server with the OpenACS libraries. It does not work with Apache and there are no plans for Apache support in the future. PostgreSQL is used as the underlying DBMS. The Architecture Intro document explains the relationships of the various subcomponents within ]po[.

]po[ is released under a mixed-source model of licensing including GPL, the ]project-open[ Free License (FL) and the ]project-open[ Commercial License (CL). The model is explained this way:

The "boring" nature of ERP software hasn't prevented the community from producing a variety of offerings, though.

The ]po[ Free License is unusual:

The license text is a bit more clear: redistribution of the software requires a separate license which may be had by contacting the project and, presumably, meeting whatever terms they may require. The "Free License" is a "free beer" license which happens to include source code.

Version 3.4 of ]po[ was released on November 5, 2009.

If your small or medium sized business needs a Linux-compatible business management system and you don't mind working with a mixed-source product, ]Project Open[ appears to be capable of providing a wide range of capabilities.

Comments (4 posted)

System Applications

Database Software

MySQL Connector/ODBC 5.1.6 is available

Full Story (comments: none)

PostgreSQL Weekly News

Full Story (comments: none)

SQLite 3.6.20 released

Comments (none posted)

Mail Software

Exim 4.70 RC 3 released

Comments (none posted)

Printing

CUPS 1.4.2 released

Comments (none posted)

Web Site Development

Pyjamas 0.7pre1 released

Full Story (comments: none)

Desktop Applications

CAD

ForcePAD 2.4.0 released

Comments (none posted)

Desktop Environments

GNOME 3.0 in September 2010

Full Story (comments: 49)

New module decisions for GNOME 2.30

Full Story (comments: none)

GNOME Software Announcements

• f-spot 0.6.1.5 (bug fixes)
• gbrainy 1.1 (new features, bug fixes and translation work)
• gedit 2.28.1 (new features, bug fixes and translation work)
• GENIUS 1.0.8 (bug fixes and documentation work)
• gnoMint 1.1.0 (bug fixes and translation work)
• Nautilus-Actions 1.12.3 (bug fixes and code cleanup)
• OCRFeeder 0.4 (new features)
• Postr 0.12.4 (bug fixes)
• tracker 0.7.6 (bug fixes and code cleanup)
• Vala 0.7.8 (new features and bug fixes)

Comments (none posted)

KDE Software Announcements

• digiKam 1.0.0-beta6 (new features and bug fixes)
• KTorrent 3.3 (new features and bug fixes)

Comments (none posted)

Xorg Software Announcements

• xgc 1.0.2 (bug fixes and code cleanup)
• xkbprint 1.0.2 (bug fixes, code cleanup and documentation work)
• xorg-server 1.7.1.901 (bug fixes)

Comments (none posted)

Financial Applications

SQL-Ledger 2.8.27 released

Comments (none posted)

Imaging Applications

pymos 0.6 introduction

Full Story (comments: none)

Office Suites

OpenOffice.org Newsletter

Full Story (comments: none)

Web Browsers

Firefox 3.5.5 now available for download

Full Story (comments: 3)

Miscellaneous

CollectionSpace 0.2 released

Full Story (comments: none)

Languages and Tools

Caml

Caml Weekly News

Full Story (comments: none)

Java

IcedTea6 1.5.3 and 1.6.2 released

Full Story (comments: none)

Python

eGenix pyOpenSSL Distribution 0.9.0-0.9.8l released

Full Story (comments: none)

ErrorHandler 1.1.0 released

Full Story (comments: none)

Fabric 0.9 released

Full Story (comments: none)

Python-URL! - weekly Python news and links

Full Story (comments: none)

Tcl/Tk

Tcl-URL! - weekly Tcl news and links

Full Story (comments: none)

Profilers

smem 0.9 released

Full Story (comments: 2)

Miscellaneous

Google's new "Go" language

Comments (98 posted)

Page editor: Forrest Cook

Announcements

Non-Commercial announcements

FSFE in battle for European interoperability

Full Story (comments: none)

Commercial announcements

Cavium Networks to acquire MontaVista Software

Comments (11 posted)

Red Hat expands Mission-Critical program

Comments (none posted)

Samsung's new smartphone platform

Comments (7 posted)

Three Ulteo news announcements

Full Story (comments: none)

Legal Announcements

SFLC summary of Bilski v. Kappos

Comments (2 posted)

Analysis: The "Lorenzo Jones" case emerges (SCOTUS Blog)

Comments (9 posted)

Court Gets A Torrent-full About Linux (Linux Journal)

Comments (9 posted)

Articles of interest

The State of Open Source on Firefox's Fifth Birthday (PC Magazine)

Comments (4 posted)

Microsoft, Novell say alliance still bearing fruit (ComputerWorld)

Comments (7 posted)

Novell cuts 3 percent of its workforce (cnet)

Comments (14 posted)

New Books

Grails: A Quick-Start Guide--New from Pragmatic Bookshelf

Full Story (comments: none)

Resources

Let's Act on ACTA Before it's Too Late (Linux Journal)

Comments (18 posted)

Checking in on CodePlex (The Standards Blog)

Comments (none posted)

Learning with Gcompris (Linux Journal)

Comments (none posted)

Contests and Awards

FSFE to award 36 Fellowship grants

Full Story (comments: none)

Matthias Ettrich receives German Federal Cross of Merit (KDE.News)

Comments (none posted)

Surveys

Linux owns 32 percent of netbook market, says study (DesktopLinux.com)

Comments (28 posted)

Event Reports

Linux-Kongress 2009 (The H)

Comments (none posted)

Calls for Presentations

ELC 2010 Call for Presentations

Full Story (comments: none)

SCALE 8X Call For Papers closes December 1th

Full Story (comments: none)

Upcoming Events

CONFidence 2.0 schedule online, registration closing

Full Story (comments: none)

FLOSS in health care track at Med-e-Tel 2010 (LinuxMedNews)

Comments (none posted)

Events: November 19, 2009 to January 18, 2010

If your event does not appear here, please tell us about it.

Miscellaneous

Perl far from dead, more popular than you think (Royal Pingdom)

Comments (95 posted)

Page editor: Forrest Cook