One more hop for Kermit

For many systems in those days - especially systems without a government budget behind them - the "network interface" was the serial port. That's how we connected to The Computer at work; that's also how we got onto the early services which were available in those days. The first thing any RS232 networking user needed was a way to move keystrokes between one interface (where they sat) and the interface connected to the modem. Unix systems often came with a tool called "cu" for that purpose, but, even on those systems, users tended to gravitate toward a newish tool called Kermit instead.

Kermit has its roots at Columbia University, where it was developed as a way to communicate between computers on a heterogeneous network. It would fit easily onto a floppy and was (unlike cu) easy to set up and use; one just needed to figure out how many data bits, how many stop bits, and what kind of parity to use (RS232 is a fun "standard"), type an appropriate ATD command at the modem, and go. Kermit could even handle things like translating between different character sets; talking to that EBCDIC mainframe was not a problem.

Over a short period of time, Kermit developed some reasonable file transfer capabilities. Its protocol was efficient enough, but it was also designed to deal with serial ports which might interpret control characters in unexpected ways, turn eight-bit data into seven-bit data, and more. The robustness of the protocol meant it "just worked" between almost any two arbitrary types of machines. So it's not surprising that, in those pre-Internet days, Kermit became a popular way of moving files around.

Columbia distributed a number of versions of Kermit in source form; it could be had from bulletin board sites, DECUS tapes, and more. The program was never released as free software, though. When Kermit was first coming into use, free software licenses as such didn't exist yet. The university considered releasing the code into the public domain but decided that it wasn't a good idea:

The license for Kermit varied over time, but was always of the "you may make noncommercial use of this program" variety. The final version of the Kermit license allowed bundling the program with free operating systems, but did not allow modifications to the source without permission. As a result, despite the fact that Kermit's license allowed distribution with systems like Linux, most distributors eventually shied away from it because it was not truly free software.

Anybody other than free operating system projects wanting to distribute Kermit commercially had to buy per-seat licenses (at a cost of $3-10 each) from Columbia.

Kermit has proved remarkably durable over the years. During Kermit's lifespan, the Internet has taken over, even our phones can run ssh, and RS232-based communications have mostly fallen by the wayside. Kernel developers are still known to use serial consoles for some types of debugging, but your editor would predict that a substantial portion of the rest of LWN's readership has never had to wonder where that null modem cable went to. Kermit's user base must certainly be shrinking, but it is still being maintained and sold to those who need it.

Or, it least, it was; in March the university announced that it was unable to continue supporting the Kermit project. One assumes that commercial license sales had finally dropped to the point where they weren't worth the administrative overhead of dealing with them. As of July 31, 2011, the university will put no further development effort into Kermit and will no longer provide support and maintenance services. A three-decade project is coming to a close.

Columbia University is doing one more thing with Kermit before the end, though - it is releasing the software under the BSD license. C-Kermit 9.0 will carry that license; the first beta release was made on June 15. The 9.0 release will also support the FORCE-3 packet protocol ("for use under severe conditions"), improved scripting, various fixes, and more. So the 9.0 release, presumably scheduled for sometime close to the July 31 deadline, will not just have new features; it will be free software for the first time.

As a result of this change, Kermit may soon show up in a distribution repository near you; most Linux users are, at this point, unlikely to care much. But, for many of us, there will yet come a time when the only way to talk to a system of interest is through a serial connection. Kermit is far from the only option we have at this point, needless to say, but it's a good one. Kermit's hop into the the free software community is more than welcome.

Comments (20 posted)

On keys and users

Secure communication requires encryption keys, but key management and distribution are difficult problems to solve. They are also important problems to solve; solutions that can be easily used by those who are not technically inclined are especially needed. Turning encrypted communications into the default, rather than an option used mostly by the technically savvy, will go a long way toward protecting users from criminals or repressive regimes. Even if these secure communications methods are only used by a subset of internet users, that subset truly needs these tools.

HTTPS

For most users, the only encrypted communication they use is SSL/TLS for accessing web pages served over HTTPS. The keys used for HTTPS are normally stored on the server side in the form of certificates that are presented to the browser when the connection is made. In order to establish the identity of the remote server, to avoid phishing-style attacks, those certificates are signed by certificate authorities (CAs) such that the browser can verify the key that it receives.

This scheme suffers from a number of problems, but it works well enough in practice so that, by and large, users can safely enter credit card and other personal information into the sites. It relies on centralized authorities in the form of the CAs, however, which can be a barrier to web site owners or users who might otherwise be inclined to serve content over HTTPS. In addition, trusting CAs has its own set of dangers. But, contrary to some of the big web players' beliefs, the web is not the only form of communication.

Other protocols

For the most part, things like instant messages, email, peer-to-peer data transfers, and voice over IP (VoIP) are all transmitted over the internet in the clear. Solutions exist to encrypt those communications, but they are not widely used, at least partly because of the key problem. Each protocol generally has its own idea of key formats and storage, as well as how to exchange those keys.

For efforts like the Freedom Box, it will be extremely important to find a solution to this key problem, so that users can more-or-less effortlessly communicate in private. If multiple applications that used keys could agree on common key types and formats, it would reduce the complexity. That may be difficult for a number of reasons, some technical, others social, so perhaps finding a way to collect up all of a user's keys into a single "key bundle" makes sense.

Generally, encryption these days uses public key cryptography, which means that there are actually two keys being used. One is the public key that can be widely disseminated (thus its name), while the other is a private key that must be kept secure. A key bundle would thus have two (or more) parts, the bundle that's shown to the rest of the world for communication and authentication, and the other that's kept secret. This private bundle would require the strongest protections against loss or theft.

Generating the keys, and agreeing upon some kind of bundle format, are fairly straightforward technical problems. Those can be solved rather easily. But the real problems will lie in educating users about keys, the difference between public and private keys, and how to properly protect their private keys. User interfaces that hide most of that complexity will be very important, but there will be things that users need to understand (chief among them will be the importance of private bundle safety).

Storing the keys

It would be a simpler problem to solve if the private bundle could be kept, securely, in one location, but that's not really a workable scenario. Users will want (or need) to communicate from a wide variety of devices, desktops, laptops, tablets, mobile phones, etc., and will want to maintain their identities (i.e. keys) when using any or all of them.

Storing the bundles at some "trusted" location on the internet ("in the cloud" in the parlance of our times) might be possible but it would also centralize their location. If some entity can cut off access to your key bundle, it leaves you without a way to communicate securely. Spreading the bundles out to various locations, and caching them locally, would reduce those problems somewhat. Obviously, storing the bundles anywhere (including locally) would require that the bundles themselves be encrypted—using a strong password—which leads to another set of potential problems, of course.

Since these private key bundles will be so important, there will need to be some way to back them up, which would be an advantage of the cloud scenario. Losing one's well-established identity could be a very serious problem. A worse problem would be if someone of malicious intent were to gain control over the keys. For either of those problems, some kind of revocation mechanism for the corresponding public keys would need to be worked out.

There are already some solutions to these problems, but, like much of the rest of the key management landscape, they tend to be very key and application-specific. For example, GNU Privacy Guard (GPG) public keys are often registered at a key server so that encrypted email can be decrypted and verified. Those keys can also be revoked by registering a revocation certificate with the key server. But, once again, key servers centralize things to some extent. Likewise, SSL/TLS certificates can be revoked by way of a certificate revocation list that is issued by a CA.

Public keys

Most of the above is concerned with the difficulties in maintaining the private bundle, but there are problems to be solved on the public key side as well. Gathering a database of the public keys of friends, family, colleagues, and even enemies will be important in order to communicate securely with them. But, depending on how that public key is obtained, it may not necessarily correspond to the individual in question.

It is not difficult to generate a key that purports to be associated with any arbitrary person you choose. The SSL/TLS certificate signing process is set up to explicitly deal with that problem by having the CAs vouch that a given certificate corresponds to a particular site. That kind of centralized authority is not a desirable trait for user-centric systems, however, so something like GPG's (and others') "web of trust" is a better model. Essentially, the web of trust allows the user to determine how much trust to place in a particular key, but it requires a fair amount of user knowledge and diligence that may make it too complex for non-technical users.

Free software can make a difference

As can be seen, there are a large number of hurdles that need to be cleared in order to make secure communication both ubiquitous and (relatively) simple to use. Key management has always been the achilles heel of public key cryptography, and obvious solutions are not ready to hand. But they are important problems to solve, even though they may only be used by a minority of internet users. For many, the additional hassles required to securely communicate may not be overcome by the concern that their communications may be intercepted. For others, who are working to undermine repressive regimes for example, making it all Just Work will be very important.

This is clearly an area where free software solutions make sense. Proprietary software companies may be able to solve some of these problems, but the closed-source nature of their code will make it very worrisome to use for anyone with a life-and-death need for it. There have just been far too many indications that governments can apply pressure to have "backdoors" inserted into communications channels. With open standards and freely available code, though, activists and others can have a reasonable assurance of privacy.

These problems won't be solved overnight, nor will they all be solved at once. Public key cryptography has been with us for a long time without anyone successfully addressing the key management problem. Recent events in the Middle East and elsewhere have shown that internet communication can play a key role in thwarting repressive regimes. Making those communications more secure will further those aims.

Comments (11 posted)

Karen Sandler on her new role at GNOME

Eight months after Stormy Peters left the post to join Mozilla, the GNOME Foundation has chosen Karen Sandler as its new executive director. Sandler is leaving a position with the Software Freedom Law Center (SFLC) as general counsel and starting with the Foundation June 21, but will be working part-time for each organization during the transition.

Prior to joining the SFLC, Sandler was an associate with Gibson, Dunn & Crutcher LLP in New York and Clifford Chance in New York and London. Taking up a position as executive director seems a slight departure from practicing law, even if focused on free software — so what made Sandler choose to pursue working with the GNOME Foundation? Sandler says the appeal is GNOME itself: "It's an incredibly important and impressive software that's been entering a critical time. I can't wait to be a part of that and assist the GNOME community to develop and grow."

She does acknowledge that it's a departure from focusing on legal issues, but says that she's looking forward to the change. "As a lawyer you're generally working to avoid pitfalls and anticipate the worst case scenarios and I'm excited to help much more proactively than that."

So what will Sandler actually be doing as executive director? When Peters held the role, she says that she started by "asking everyone in the GNOME community what they thought I should do." During her time, Peters says that she ran the Foundation's day-to-day operations, served as a contact point for questions, helped build the marketing team and travel committee, and "helped make events happen" though she says events mostly happened thanks to the GNOME community.

Sandler, taking a cue from Peters says that she'll ask what people think she'll be working on, but hopes to spend at least some time on advocacy:

She notes that this isn't dissimilar to what she's been doing for the SFLC already:

Sandler also points out that there are likely to be a lot of housekeeping tasks that have gathered dust since Peters left for Mozilla and her first days will be "spent getting up to speed, getting to know people, taking care of the administrative backlog and ramping up for the Desktop Summit (and of course following up on items raised during the Summit). I'll also look to renew relationships and generally try to immerse myself in all things GNOME."

Peters left in November 2010, and the search committee for the executive director was announced at the end of December. The committee included Peters, IBM's Robert Sutor, GNOME Foundation director Germán Póo-Caamaño, Kim Weins of OpenLogic, Jonathan Blandford of Red Hat, Luis Villa of Mozilla, former GNOME board member Dave Neary, and Bradley Kuhn of the Software Freedom Conservancy (SFC). However, Kuhn says that he stepped down from the committee once Sandler emerged as a serious candidate, as they had worked closely together at the SFLC and SFC; Kuhn also considers her a personal friend.

One thing Sandler won't be doing is driving the technical direction of GNOME. Sandler says that, like Peters, she has "a limited role" in the technical direction of GNOME, and says "I'd support whatever the community and release team decided."

Another large part of the role is fundraising. The executive director is the point person for the advisory board and works to encourage members to sign up and donate not only the advisory board fees, but also to contribute to specific events like GNOME Hackfests.

Financially, the GNOME Foundation is doing well enough. In April 2009 there was a concern that the foundation would be unable to continue the executive director position due to lack of funds. In that message, John Palmieri said that Peters had managed to recruit a number of new corporate sponsors, but "we are still projecting that without a significant influx of steady contributions we will be unable to keep an Executive Director on the payroll without cutting into the activities budget." The foundation started leaning heavily on its Friends of GNOME program for individual donations, and doubled advisory board fees for corporate members from $10,000 per year to $20,000.

Despite the increased fees and additional income from Friends of GNOME, the budget for 2011 shows a decline in income of about $52,000, while the proposed expenditures are higher by about $64,000. Though it's worth noting the expenditures will likely be lower than planned, as it appears the budget was prepared with the expectation an executive director would be hired by March.

The GNOME Foundation budget for 2011 is $518,000 — with $145,510 earmarked for employees, and the executive director position is the largest part of that budget. (The GNOME Foundation also has a part-time system administrator.) So, is the executive director position the best use of GNOME Foundation resources? Sandler says yes:

Sandler is joining the GNOME Foundation at an interesting time. The GNOME community is looking a bit fragmented at the moment. The GNOME 3.0 release has gotten mixed reviews, some users are feeling dissatisfied with the lack of ability to modify GNOME to suit their needs, and the relationship between GNOME and Canonical is strained at best. The GNOME Shell has not been widely embraced — Fedora 15 has shipped GNOME Shell, but Canonical has gone its own way with Unity, and other GNOME-centric distributions like Linux Mint chose to sit out the GNOME 3.0 release and ship GNOME 2.32.

In short, some would say that GNOME as a project has seen better days. Sandler is not convinced of that:

Whether GNOME 3 has time to evolve is another question. The Linux desktop, on traditional PCs and laptops, simply is not gaining much traction beyond its current audience. But Linux is being used in a number of mobile systems that address end users, but GNOME in its entirety is not yet there. Sandler says that she believes GNOME 3 will make GNOME more relevant on mobile devices:

Sandler is also unwilling to give up on the PC desktop just now. "It's also probably worth noting that desktop computing is still how most people use their computers (I think people forget that sometimes)!"

As for GNOME 3's slow adoption and potential fragmentation, Sandler says that the transition is "still underway and it will take time to see how things really shake out." She says that "strong decisions" will always alienate some people, but she hopes that GNOME can restore relationships. "I think that ultimately good software and good community will fuel increased participation rather than the fragmentation that seems [to] be arising now."

The upcoming Desktop Summit may be a good opportunity to mend some fences, says Sandler. "The GNOME board tells me that there's already a full list of sponsors and attendees for the Desktop Summit (and that Canonical in particular is planning to attend and sponsor the event). I believe that developers in the different distros are all talking to GNOME and I hope that we'll only see more cooperation going forward."

Ultimately, Sandler says she's optimistic about GNOME: "Coming to GNOME is obviously a vote of confidence from me personally. I love my work at SFLC and am only persuaded to leave because I think it's a great opportunity to be a part of a great organization."

One thing is certain, working with GNOME at this stage is likely to be an interesting job. We wish Sandler the best in her new role, and thank her for taking the time to talk with LWN ahead of the announcement.

Comments (19 posted)

A hole in crypt_blowfish

A longstanding bug that was recently found in the crypt_blowfish password hashing library highlights the problems that can occur when a bug is found in a widely used low-level library. Because crypt_blowfish has been around for so long (this bug is said to go back to 1998 or possibly 1997), it has been used by various other packages (PHP for example) as well as some Linux distributions. The security impact is not likely to be huge, because it only affects passwords with somewhat uncommon characteristics, but the impact on those who have stored hashed passwords generated using the library may be a bit more painful.

Password hashing is a standard technique that is used by authentication mechanisms (for logging into a system or web application for example), so that the plaintext password does not need to be stored. Instead, something derived from the plaintext password is stored. Typically, one-way cryptographic hash functions like SHA-1 or the Blowfish cipher are used for that purpose. When the user presents the password, the same function is applied to that input and compared to what is stored. The idea is that even if an attacker gets access to the password database, they would need to "crack" the hashed values stored there. As its name implies, crypt_blowfish implements password hashing based on Blowfish.

The bug itself is quite simple, and the change to fix it will likely make the problem obvious to C programmers:

    tmp |= *ptr;

    tmp |= (unsigned char)*ptr;

The problem was actually discovered in the John the Ripper (JtR) password cracking program. As part of creating a test suite, "magnum" was attempting to crack a password that consisted of a single non-ASCII character (£ or 0xa3), which was hashed using a library other than crypt_blowfish. Because JtR and crypt_blowfish share their implementation of the Blowfish encryption algorithm, tests using both would pass, but independent implementations would generate the correct hash, thus failing to match what was generated by JtR.

JtR and crypt_blowfish developer Alexander Peslyak (aka Solar Designer) analyzed the effects of the bug and found that some password pairs would hash to the same value with only minimal differences (e.g. "ab£" hashed to the same value as "£"), which would make password cracking easier. A further analysis shows that some characters appearing just before one with the high bit set may be effectively ignored when calculating the hash. That would mean that a simpler password than that given by the user could be used and would still be considered valid—a significant weakening of the user's password.

It should be noted that Solar Designer has been very forthcoming with details of the problem and its effects. His CVE request is quite detailed, and he takes full responsibility for the error. Other projects who find security holes in their code would do well to follow his lead here.

There is no real problem for JtR, as it can get the updated code so that it can crack passwords with high-bit-set characters in them. In addition, it could continue to use the broken code to crack passwords that were hashed incorrectly. But for applications that use crypt_blowfish, it's a different story. Passwords with the high bit set may be fairly uncommon—at least for sites with typically ASCII-only users—but there is no easy way to determine whether stored hashes are invalid or not.

The safest solution for administrators with potentially bad password hashes (which could include those running Openwall Linux (Owl), SUSE, and ALT Linux which can use crypt_blowfish for hashing the password database) is to invalidate all passwords and have their users input new ones. That could prove to be a logistical nightmare, however, depending on how easily, and securely, users can set new passwords without authenticating with their existing password. Requiring that users change their existing passwords after logging in is an alternative, but one that might give attackers a window in which to operate. It also leaves passwords unchanged for any users that do not log in.

The risks of attackers using this flaw to log in to a site are likely to be fairly small, but they do exist. If a site's login process is susceptible to brute force attacks, this bug makes it somewhat easier to do so, at least for specific password types. On the other hand, if the password database has been exposed, and some passwords were not crackable (at least for attackers using cracking programs other than JtR), this information would give them the means to crack those passwords. In the end analysis, it is an exploitable hole, but not the kind to send administrators into panic mode.

It is somewhat amazing that this bug has existed for 13+ years in a fairly widely used library. Up until now, no one has tested it in this way, at least publicly, which should serve as something of a warning to those who are using well-established software, both free and proprietary. With free software (crypt_blowfish has been placed into the public domain which may be a bit legally murky but is in keeping with free software ideals) there are more and easier opportunities to examine and test the code, but that's only effective if the testing is actually done.

There are, without doubt, bugs still lurking in various security-oriented libraries that we depend on every day, so testing those libraries (as well as code that is not being used for security purposes) regularly and systematically can only help find them. While it took more than a decade for this bug to come to light, it's worth pointing out that it certainly could have been discovered by others in the interim. Attackers clearly do their own testing, and are generally not inclined to release their results. That may not be the case here, but one should always recognize that public disclosure of a vulnerability is not necessarily tied to its discovery.

Comments (54 posted)

Security quotes of the week

Comments (9 posted)

The story behind the mysterious CyanogenMod update

Comments (50 posted)

ffmpeg mplayer: arbitrary code execution

Comments (none posted)

groff: insecure tmp file usage

Comments (none posted)

kernel: multiple vulnerabilities

Comments (none posted)

klibc: command injection

Comments (none posted)

libvirt: arbitrary host file access

Comments (none posted)

libxml2: code execution

Comments (none posted)

moodle: multiple vulnerabilities

Comments (none posted)

movabletype-opensource: multiple vulnerabilities

Comments (none posted)

Mozilla products: multiple vulnerabilities

Comments (none posted)

nagios: cross-site scripting

Comments (none posted)

pam_ssh: privilege escalation

Comments (none posted)

php: code execution

Comments (none posted)

SUSE Manager: multiple vulnerabilities

Comments (none posted)

SUSE Manager Proxy: insecure SSL settings

Comments (none posted)

torque: remote code execution

Comments (none posted)

unixODBC: buffer overflow

Comments (none posted)

Kernel release status

Stable updates: there have been no stable updates released in the last week.

Comments (none posted)

Quotes of the week

Comments (9 posted)

Netconf 2011

All told, it looks like an interesting and productive gathering; if past patterns hold, we will get a more thorough summary at the 2011 Kernel Summit in October.

Comments (none posted)

User-friendly disk names

Device names, particularly for disks, can be confusing to Linux administrators because they get assigned at boot time based on the order in which the disks are discovered. So the same physical disk can be assigned a different device name (in /dev) on each boot, which means that kernel log messages and the output of various utilities may not correspond with the administrator's view of the system. A recent patch set looks to change that situation, but it is meeting some resistance from kernel hackers who think it should be handled by user space.

The patches posted by Nao Nishijima are pretty straightforward. They just add a preferred_name entry to struct device, which can be updated via sysfs. The patches then change some SCSI log messages and /proc/partitions output to use the preferred name if it has been set. Greg Kroah-Hartman expressed concerns about changing /proc/partitions as various tools parse that file so it is part of the kernel's user-space interface. Adding the preferred name as a new field on each line might very well confuse utilities that parse the file.

More importantly, though, he notes that one could just change the tools so that they use the names as arguments or in their output. Any scheme that would map preferred names to specific disks requires some kind of mapping file, so tools that wanted to use these preferred names (things like mount, smartd, and other disk-related tools) could do so using that mapping without involving the kernel at all:

While the patches only use preferred_name for disk devices, the idea is to allow them to be added to any device (and then change log messages and utilities to use them). It is modeled after the ifalias entry that was added for network devices back in 2008, but some don't see that as something to emulate. Allowing only one alias for network devices is generally not enough "because people want not a single but multiple names at the same time", Kay Sievers said, so ifalias is only used by some SNMP utilities. Currently, udev maintains a set of links in /dev/disk/by-* that relate disks to kernel devices by a variety of characteristics (ID, label, path, and UUID). James Bottomley would like to see that be extended for the preferred names:

But there are problems inherent in that idea. In order for udev to know that the preferred name was set, a uevent would have to be generated. That could be done, but it leads to other problems, as Sievers points out (instead of by-preferred, he uses by-pretty):

    /dev/disk/by-pretty/foo

Essentially, udev keeps track of the devices present in the system (and their attributes like, potentially, preferred name), but doesn't have any concept of tracking "no longer valid names" as Sievers puts it. That means that udev can't just leave older entries around when user space changes the preferred name: "We can not just add stuff to /dev without a udev database entry, it would never get removed on device unplug and leave a real mess behind."

One possible solution for the renaming problem is to only allow one write to preferred_name, so that, once established, those aliases couldn't be changed without a reboot. udev could set up the proper links, and various tools could use the aliases as needed. That would solve the renaming problem at the cost of some flexibility. In general, no one was really opposed to the idea of some kind of more-mnemonic name for disks, it is more of a question of how to get there.

Sievers proposed adding a way for udev to list all of the symlinks that it creates during device discovery. Anyone (or any tool) that needed to associate an alias with a particular disk could use that output to determine the current device being used (based on the UUID for example), then make the substitution as appropriate. That would work, in general, but Bottomley sees it as overly complex for users:

But Sievers and driver core maintainer Kroah-Hartman see it as papering over more substantial issues. Sievers, at least, would like to see text-file-style debug and error message output replaced (or supplemented) with something more structured:

But, from the user's perspective, disks may already have names (with labels on the enclosures themselves for example) and it would be quite convenient for the kernel's messages to reflect them. In the end, Sievers isn't opposed to a disk-specific (rather than for all devices) solution, though he thinks it isn't really the right direction to go. Kroah-Hartman agrees and is adamant that this change not go into the driver core. Based on that, Nishijima plans to redo the patches, moving the name to struct gendisk, renaming the field to alias_name (rather than "preferred") to better reflect its purpose, and generating a uevent when the name changes.

But, following the lead of the network ifalias will add to the kernel's user-space interface, only this time for disks. While it may solve an immediate problem for administrators, it will also leave behind some legacy code when, or if, a better solution comes around. That's unfortunate, but, since it solves a real problem, and the change is restricted to subsystem whose maintainer (Bottomley) is in favor of it, it may well turn up in the mainline before long. Any change to system error and debug logging along the lines of what Sievers described is certainly quite a ways off, though there have long been calls for structured kernel output. Sometimes it is just easier to make a change like this in one place, rather than trying to identify and fix all of the places outside of the kernel that would need it.

Comments (30 posted)

The platform device API

Alas, life is not so simple; there are plenty of devices which are still not discoverable by the CPU. In the embedded and system-on-chip world, non-discoverable devices are, if anything, increasing in number. So the kernel still needs to provide ways to be told about the hardware that is actually present. "Platform devices" have long been used in this role in the kernel. This article will describe the interface for platform devices; it is meant as needed background material for a following article on integration with device trees.

Platform drivers

A platform device is represented by struct platform_device, which, like the rest of the relevant declarations, can be found in <linux/platform_device.h>. These devices are deemed to be connected to a virtual "platform bus"; drivers of platform devices must thus register themselves as such with the platform bus code. This registration is done by way of a platform_driver structure:

    struct platform_driver {
	int (*probe)(struct platform_device *);
	int (*remove)(struct platform_device *);
	void (*shutdown)(struct platform_device *);
	int (*suspend)(struct platform_device *, pm_message_t state);
	int (*resume)(struct platform_device *);
	struct device_driver driver;
	const struct platform_device_id *id_table;
    };

At a minimum, the probe() and remove() callbacks must be supplied; the other callbacks have to do with power management and should be provided if they are relevant.

The other thing the driver must provide is a way for the bus code to bind actual devices to the driver; there are two mechanisms which can be used for that purpose. The first is the id_table argument; the relevant structure is:

    struct platform_device_id {
	char name[PLATFORM_NAME_SIZE];
	kernel_ulong_t driver_data;
    };

If an ID table is present, the platform bus code will scan through it every time it has to find a driver for a new platform device. If the device's name matches the name in an ID table entry, the device will be given to the driver for management; a pointer to the matching ID table entry will be made available to the driver as well. As it happens, though, most platform drivers do not provide an ID table at all; they simply provide a name for the driver itself in the driver field. As an example, the i2c-gpio driver turns two GPIO lines into an i2c bus; it sets itself up as a platform device with:

    static struct platform_driver i2c_gpio_driver = {
	.driver		= {
		.name	= "i2c-gpio",
		.owner	= THIS_MODULE,
	},
	.probe		= i2c_gpio_probe,
	.remove		= __devexit_p(i2c_gpio_remove),
    };

With this setup, any device identifying itself as "i2c-gpio" will be bound to this driver; no ID table is needed.

Platform drivers make themselves known to the kernel with:

    int platform_driver_register(struct platform_driver *driver);

As soon as this call succeeds, the driver's probe() function can be called with new devices. That function gets as an argument a platform_device pointer describing the device to be instantiated:

    struct platform_device {
	const char	*name;
	int		id;
	struct device	dev;
	u32		num_resources;
	struct resource	*resource;
	const struct platform_device_id	*id_entry;
	/* Others omitted */
    };

The dev structure can be used in contexts where it is needed - the DMA mapping API, for example. If the device was matched using an ID table entry, id_entry will point to the specific entry matched. The resource array can be used to learn where various resources, including memory-mapped I/O registers and interrupt lines, can be found. There are a number of helper functions for getting data out of the resource array; these include:

    struct resource *platform_get_resource(struct platform_device *pdev, 
					   unsigned int type, unsigned int n);
    struct resource *platform_get_resource_byname(struct platform_device *pdev,
					   unsigned int type, const char *name);
    int platform_get_irq(struct platform_device *pdev, unsigned int n);

The "n" parameter says which resource of that type is desired, with zero indicating the first one. Thus, for example, a driver could find its second MMIO region with:

    r = platform_get_resource(pdev, IORESOURCE_MEM, 1);

Assuming the probe() function finds the information it needs, it should verify the device's existence to the extent possible, register the "real" devices associated with the platform device, and return zero.

Platform devices

So now we have a driver for a platform device, but no actual devices yet. As was noted at the beginning, platform devices are inherently not discoverable, so there must be another way to tell the kernel about their existence. That is typically done with the creation of a static platform_device structure providing, at a minimum, a name which is used to find the associated driver. So, for example, a simple (fictional) device might be set up this way:

    static struct resource foomatic_resources[] = {
	{
		.start	= 0x10000000,
		.end	= 0x10001000,
		.flags	= IORESOURCE_MEM,
		.name	= "io-memory"
	},
	{
		.start	= 20,
		.end	= 20,
		.flags	= IORESOURCE_IRQ,
		.name	= "irq",
	}
    };

    static struct platform_device my_foomatic = {
	.name 		= "foomatic",
	.resource	= foomatic_resources,
	.num_resources	= ARRAY_SIZE(foomatic_resources),
    };

These declarations describe a "foomatic" device with a one-page MMIO region starting at 0x10000000 and using IRQ 20. The device is made known to the system with:

    int platform_device_register(struct platform_device *pdev);

Once both a platform device and an associated driver have been registered, the driver's probe() function will be called and the device will be instantiated. Registration of device and driver are usually done in different places and can happen in either order. A call to platform_device_unregister() can be used to remove a platform device.

Platform data

The above information is adequate to instantiate a simple platform device, but many devices are more complex than that. Even the simple i2c-gpio driver described above needs two additional pieces of information: the numbers of the GPIO lines to be used as i2c clock and data lines. The mechanism used to pass this information is called "platform data"; in short, one defines a structure containing the specific information needed and passes it in the platform device's dev.platform_data field.

With the i2c-gpio example, a full configuration looks like this:

    #include <linux/i2c-gpio.h>

    static struct i2c_gpio_platform_data my_i2c_plat_data = {
	.scl_pin	= 100,
	.sda_pin	= 101,
    };

    static struct platform_device my_gpio_i2c = {
	.name		= "i2c-gpio",
	.id		= 0,
	.dev = {
		.platform_data = &my_i2c_plat_data,
	}
    };

When the driver's probe() function is called, it can fetch the platform_data pointer and use it to obtain the rest of the information it needs.

Not everybody in the kernel community is enamored with platform devices; they seem like a bit of a hack used to encode information about specific hardware platforms into the kernel. Additionally, the platform data mechanism lacks any sort of type checking; drivers must simply assume that they have been passed a structure of the expected type. Even so, platform devices are heavily used, and that's unlikely to change, though the means by which they are created and discovered is changing. The way of the future appears to be device trees, which will be described in the following article.

Comments (5 posted)

Platform devices and device trees

The current platform device mechanism is relatively easy to use for a developer trying to bring up Linux on a new system. It's just a matter of creating the descriptions for the devices present on that system and registering all of the devices at boot time. Unfortunately, this approach leads to the proliferation of "board files," each of which describes a single type of computer. Kernels are typically built around a single board file and cannot boot on any other type of system. Board files sort of worked when there were relatively small numbers of embedded system types to deal with. Now Linux-based embedded systems are everywhere, architectures which have typically depended on board files (ARM, in particular) are finding their way into more types of systems, and the whole scheme looks poised to collapse under its own weight.

The hoped-for solution to this problem goes by the term "device trees"; in essence, a device tree is a textual description of a specific system's hardware configuration. The device tree is passed to the kernel at boot time; the kernel then reads through it to learn about what kind of system it is actually running on. With luck, device trees will abstract the differences between systems into boot-time data and allow generic kernels to run on a much wider variety of hardware.

This article is a good introduction to the device tree format and how it can be used to describe real-world systems; it is recommended reading for anybody interested in the subject.

It is possible for platform devices to work on a device-tree-enabled system with no extra work at all, especially once Grant Likely's improvements are merged. If the device tree includes a platform device (where such devices, in the device tree context, are those which are direct children of the root or are attached to a "simple bus"), that device will be instantiated and matched against a driver. The memory-mapped I/O and interrupt resources will be marshalled from the device tree description and made available to the device's probe() function in the usual way. The driver need not know that the device was instantiated out of a device tree rather than from a hard-coded platform device definition.

Life is not always quite that simple, though. Device names appearing in the device tree (in the "compatible" property) tend to take a standardized form which does not necessarily match the name given to the driver in the Linux kernel; among other things, device trees really are meant to work with more than one operating system. So it may be desirable to attach specific names to a platform device for use with device trees. The kernel provides an of_device_id structure which can be used for this purpose:

    static const struct of_device_id my_of_ids[] = {
	{ .compatible = "long,funky-device-tree-name" },
	{ }
    };

When the platform driver is declared, it stores a pointer to this table in the driver substructure:

    static struct platform_driver my_driver = {
	/* ... */
	.driver	= {
		.name = "my-driver",
		.of_match_table = my_of_ids
 	}
    };

The driver can also declare the ID table as a device table to enable autoloading of the module as the device tree is instantiated:

    MODULE_DEVICE_TABLE(of, my_of_ids);

The one other thing capable of complicating the situation is platform data. Needless to say, the device tree code is unaware of the specific structure used by a given driver for its platform data, so it will be unable to provide that information in that form. On the other hand, the device tree mechanism is equipped to allow the passing of just about any information that the driver may need to know. Making use of that information will require the driver to become a bit more aware of the device tree subsystem, though.

Drivers expecting platform data should check the dev.platform_data pointer in the usual way. If there is a non-null value there, the driver has been instantiated in the traditional way and device tree does not enter into the picture; the platform data should be used in the usual way. If, however, the driver has been instantiated from the device tree code, the platform_data pointer will be null, indicating that the information must be acquired from the device tree directly.

In this case, the driver will find a device_node pointer in the platform devices dev.of_node field. The various device tree access functions (of_get_property(), primarily) can then be used to extract the needed information from the device tree. After that, it's business as usual.

In summary: making platform drivers work with device trees is a relatively straightforward task. It is mostly a matter of getting the right names in place so that the binding between a device tree node and the driver can be made, with a bit of additional work required in cases where platform data is in use. The nice result is that the static platform_device declarations can go away, along with the board files that contain them. That should, eventually, allow the removal of a bunch of boilerplate code from the kernel while simultaneously making the kernel more flexible.

Comments (16 posted)

Mageia bootstraps its package update policy

Mageia, the community-driven fork of Mandriva Linux, has made its initial release, just eight months after the project's inception. It is a major accomplishment, but the development team has already turned its attention to the next pressing challenge: how to set up and manage package updates, including how to handle security fixes, how to integrate backports and missing packages, and how to bootstrap a QA process.

The release of Mageia 1 marks a turning point for the distribution, because end users are now encouraged to install and use it on a daily basis — and up until now, most of those users were still running Mandriva, which continued to receive regular package updates.

A user switching to Mageia is likely to encounter several distinct package update scenarios: security updates to close specific vulnerabilities, incremental bugfix updates, and new packages that for one reason or another were not ready for inclusion with the initial release of Mageia 1. Each scenario has its peculiarities, although for simplicity's sake, it is in the project's interest to provide as uniform a testing and update process as possible.

Mageia's effort to hash out an update process is not simply a start-up problem. Many on the team were unhappy with Mandriva's process over the years. As Michael Scherer explained:

The Mageia team, therefore, has set goals for its own process to be more streamlined, more scalable, and to take security vulnerabilities more seriously. Many of the core team members are former Mandriva contributors (and quite a few were Mandriva employees), so the process includes their opportunity to learn directly from Mandriva's mistakes.

Thus far, Mageia has not pushed out any updated packages to the updates repository, although a June 19 message indicates that the first package updates have at least reached testing. Meanwhile, the outstanding list of "missing package" backport requests, which includes security updates made by Mandriva, stands at over a dozen. So, since the release of Mageia 1, security updates have languished along with other updates, though it appears that will be changing soon.

Updates (security and otherwise)

The discussion over how to structure the update process began on the mageia-dev mailing list in early June. As is the case for Mandriva, Mageia plans to run separate RPM repositories for main releases and package updates, plus testing repositories for use by developers and packagers. The draft update policy draws the distinction between the "updates" repository (which is to be used to provide bug-fix or security-fix updates to existing packages), and the "backports" repository (which is to be used to provide new releases of packages).

Nevertheless, there are a handful of exceptions. First, new versions of packages that require an update in order to work with a remote online service are allowed in the updates repository (the examples include networked games and virus scanners). Second, new versions of applications are permitted when the old version is no longer supported by the upstream project itself (a rule that primarily encompasses applications with a short lifecycle like the Chromium and Firefox web browsers).

The update release process follows the same broad plan as used by many other distributions: anyone can file a bug against a package; once the bug is triaged and assigned, the package maintainer can apply the relevant patches and rebuild the package. The maintainer is then responsible for submitting the new build to the updates_testing repository and re-assigning the bug to the QA team. The QA team tests the package to ensure that it fixes the bug, that it both installs and successfully updates the previous version of the package, and (in the case of security fixes) that it closes the security vulnerability.

A few obstacles remain to implementing this process. The first is that, as a new distribution, very few packages have official maintainers. Mageia may assign bugs to package committers as an initial step, but the understanding is that maintainership of a package is a voluntary task. The second obstacle is setting up QA communication channels, to ensure that QA team members do not accidentally step on each other's toes. Whether that takes the form of a read-only mailing list or a web announcement mechanism has yet to be determined.

The third obstacle is that while the QA team is significant in size (and growing), the security team is still much smaller, and less formalized. Conventional wisdom states that the security team has the time-consuming task of monitoring security vulnerability lists and databases, as well as the difficult job of assessing that a rebuilt package fixes the security vulnerability, as opposed to simply verifying a specific "normal" bug fix.

That is not to diminish the amount of work that goes into the QA process. Mageia, like Mandriva before it, prides itself on providing well-tested packages that both integrate with free software applications and play well with third-party (potentially proprietary) offerings. It supports both the GNOME and KDE desktop environments when many distributions choose either one or the other, and is available for both 32-bit and 64-bit architectures.

Backports and support

That desire to offer well-tested packages comes to the forefront in the project's discussion over how to handle "missing packages" and backports. The root of the issue is that Mageia 1 lacks several packages that were available in previous Mandriva releases. Due to personnel constraints, it simply isn't possible to package and test every application and library — so the project did its best with the Mageia 1 release, and is attempting to craft a policy for providing the missing pieces to users between now and the release of Mageia 2.

A case could be made that missing packages are essentially no different than other bugs, and if a package can be tested and added to the repository, it should go in the updates repository. The alternate viewpoint is that updates is by definition reserved for fixes to existing packages, and that the backports repository exists for the specific purpose of allowing users early access to packages slated for the next release — early access at their own risk.

But Mageia 1 is a special circumstance; many of the missing packages have long existed for Mandriva users, and in some cases are even stable and "expected" in a standard distribution (examples from the mailing list include passwd-gen and dd_rescue). Forcing users in need of those packages to enable an unstable repository like backports (which could inadvertently update other packages to new, un-QA-validated versions) puts them at an unacceptable risk. Yet treating backports as a fully-supported repository just to enable the addition of missing packages has its own risks, namely far more work for the QA and security teams.

Scherer argued that if the root cause of the trouble was that backports has historically been synonymous with "bug-filled," then fixing the backports process was the solution. Others suggested that the real pain-point was the notion that backports is "unsupported" — a term casually tossed around in many distributions, but rarely if ever defined. After all, in most cases, a user reporting a problem with a backports package still receives help fixing it from the community. Complicating the issue further is that Mandriva's package management tool RPMDrake allowed users to access specific packages from Mandriva's backports repository, but without enabling the repository as a whole — a behavior that carries its own set of QA and security trade-offs.

Samuel Verschelde suggested a plan of attack for bugs filed against backports, that differentiates them from bugs in updates by establishing a "maintainer first, then upstream" process that relieves the regular QA team from the burden of integrating the fixes. Christiaan Welvaart recommended a way to tie bugs in the backports repository more closely in with the existing QA process by providing better step-by-step guidance to bug reporters.

Eventually, the team came to a consensus around a plan that makes a special exception for the transitional period between Mageia 1 and the release of Mageia 2. During the transition period, missing packages can be added to the updates repository provided that they follow the same QA process as every other update, and that there is a bug report filed formally requesting the package. After Mageia 2 is released, that process will be closed, and missing packages should go through the backports repository, or simply wait for the next release cycle.

Of course, Mageia is also in the process of working out the details of its release cycle, so the proposed backports plan has its small share of uncertainty. At the moment, the distribution has a bleeding-edge distribution named "Cauldron" that operates as a rolling release, but stable end-user releases are tentatively planned to follow the six-month cycle established by Mandriva.

Bubble bubble, hopefully with minimal toil and trouble

Bootstrapping a Linux distribution is always a tremendous amount of work, even for teams like Mageia's, with its healthy number of ex-Mandriva engineers and community contributors. But as many old Linux veterans will tell you, getting the first release out the door is not nearly as taxing as setting up a smooth development, testing, and release process that is sustainable over a multi-year period. Even established distributions struggle with QA and package testing issues from time to time.

The Mageia backports discussion touched on an intriguing issue: what precisely it means when an open source project declares something "supported" or "unsupported." It appears as if that thread has fallen by the wayside, and the team has moved on to the more practical issues of fleshing out the backports process and handling the corner-case of assisting Mandriva users as they transition to the first ever Mageia release. But perhaps Mageia's unusual position as a brand-new distribution staffed by team members that moved over collectively from Mandriva gives it a unique chance to address the supported/unsupported question. It has already allowed them to produce a full-featured distribution in well under a year's time.

Comments (4 posted)

Distribution quotes of the week

Comments (6 posted)

AV Linux 5.0 Released

Full Story (comments: none)

openSUSE releases milestone 2

Comments (none posted)

Scientific Linux 5.6

Comments (none posted)

USU Linux and Jeoss Linux

USU Linux hails from Bulgaria. It's Ubuntu based and comes in three editions. The mini edition is suitable for home or business use, the desktop edition has a strong focus on educational uses, and the netbook edition is specially created for little mini portable computers. (Thanks to Tsvetomir Parvanov)

Jeoss Linux is a compact install-everywhere Ubuntu based server oriented distribution. It is directly-installable even on legacy, limited resource, and embedded x86 platforms. (Thanks to Patrick Masotta)

Comments (none posted)

Planet Debian Derivatives

Comments (none posted)

New round of Debian IRC Training Sessions

Full Story (comments: none)

Voting for the Gentoo Council 2011-2012 Election begins

Full Story (comments: none)

Distribution newsletters

• DistroWatch Weekly, Issue 410 (June 20)
• openSUSE Weekly News, Issue 180 (June 18)
• Ubuntu Weekly Newsletter, Issue 221 (June 20)

Comments (none posted)

Tiny Core Linux 3.7 brings "multi-Core" ISOs (The H)

Comments (none posted)

Making static analysis easier

The key enabling changes have been happening in the compiler suites. Compilers must already perform a detailed analysis of the code in order to produce optimized binaries; it makes sense to make use of that analysis for other purposes as well. Some of that already happens in the compiler itself; GCC and LLVM can produce a much wider set of warnings than was once possible. These warnings are a good start, but much more can be done. That is especially true if projects can create their own analysis tools for project-specific checks; projects of any size tend to have invariants and rules of their own which go beyond the rules for the source language in general.

The FSF was, for years, hostile to the idea of making it easy to plug analysis modules into GCC, fearing that a plugin mechanism would enable the creation of proprietary modules. After some years of deliberation, the FSF rewrote the license exception for its runtime modules in a way that addressed the proprietary module worries; since then, GCC has had plugin module support. The use of that feature has been relatively low, so far, but there are signs that the situation may be beginning to change.

An early user of the plugin mechanism was the Mozilla project, which created two modules (Dehydra and Treehydra) to enable the writing of analysis code in JavaScript. These tools have seen some use within Mozilla, but development there seems to have slowed to a halt. The mailing list is moribund and the software does not appear to have seen an update in some time.

An alternative is GCC MELT. This project provides a fairly comprehensive plugin which allows the writing of analysis code in a Lisp-like language. This code is translated to C and turned into a plugin which can be invoked by the compiler. MELT is extensively documented; there are also slides from a couple of tutorials on its use.

MELT seems to be a capable system, but there do not appear to be a lot of modules written for it in the wild. One does not need to look at the documentation for long to understand why; the "basic hints" start with: "You first need to grasp GCC main internal representations (notably Tree & Gimple & Gimple/SSA)." MELT author Basile Starynkevitch's 130-slide presentation on MELT [PDF] does not get past the introductory GCC material until slide 85. MELT, in other words, requires a fairly deep understanding of GCC; it's not something that an outsider can pick up quickly. The lack of easy examples to work from is not helpful either.

More recently, David Malcolm has announced the release of a new framework which enables the creation of plugins as Python scripts which run within the compiler. His immediate purpose is to create tools for the development of the Python system itself; the most significant checker in his code tries to ensure that object reference counts are managed properly. But he sees the tool as potentially being useful for a number of other projects and even for prototyping new features for GCC itself.

At a first glance, David's gcc-python-plugin mechanism suffers from the same difficulty as MELT - the initial learning curve is steep. It is also a very young and incomplete project; David has, by his own admission, only brought out the functionality he had immediate need for. The analysis code seems more approachable, though, and the mechanism for running scripts directly in the compiler seems more natural than MELT's compile-from-Lisp approach. It may be that this plugin will attract more users and developers than MELT as a result.

Or it may just be that your editor, being rather more proficient in Python than in Lisp, naturally likes the Python-based solution better.

In any case, one conclusion is clear: writing static analysis plugins for GCC is currently too hard; even capable developers who approach the problem will need to dedicate a significant chunk of time to understanding the compiler before they can begin to achieve anything in this area. The efforts described above are a big step in the right direction, but it seems clear that they are the foundations upon which more support code must be built. It's hard to say when it will reach the tipping point that inspires a flood of new analysis code, but it's easy to say that we are not there yet.

GCC is not where all the action is, though; there is also an interesting static analysis tool which has been built with the LLVM clang compiler. Documentation of this tool is scarce, but it appears to be capable of detecting some kinds of memory leaks, null pointer dereferences, the computation of unused values, and more. Some patches have been posted to add a plugin feature to this tool, but they do not seem to have proceeded very far yet.

Back in May, John Smith ran the checker on several open source projects to see what kind of results would emerge. Those results have been posted on the net; they show the kind of potential problems that can be found and the nice HTML output that the checker can create. Some of the warnings are clearly spurious - always a problem with static analysis tools - but others seem worth looking into. In general, the clang static analyzer seems, like the other tools mentioned here, to be in a relatively early state of development. Things are moving fast, though; this tool is worth keeping an eye on.

Actually, that is true of the static analysis area in general. The lack of good analysis tools has been a bit of a mystery - given the number of developers we have, one would think that a few would scratch that particular itch. Your editor would not have minded living in a world with one less version control system but with better analysis tools. But the nature of free software development is that people work on problems that interest them. As the foundations of our static analysis tools get better, one can hope that more developers will find those foundations interesting to build on. The entire development community will benefit from the results.

Comments (28 posted)

Quotes of the week

Comments (18 posted)

The Document Foundation ponders certification

Comments (18 posted)

MediaWiki 1.17.0 released

Full Story (comments: none)

Mozilla Delivers New Version of Firefox

Comments (14 posted)

pari-2.5.0

Full Story (comments: none)

Pyro 4.7 released

Full Story (comments: none)

Harmattan Python is available

Full Story (comments: none)

Tornado 2.0

Comments (none posted)

Development newsletters from the last week

• Caml Weekly News (June 21)
• LibreOffice development summary (June 21)
• Linux Audio Monthly Roundup (June)
• PostgreSQL Weekly News (June 19)
• Tahoe-LAFS Weekly News (June 17)

Comments (none posted)

An Early Expedition with Media Explorer on Linux (Linux.com)

Comments (none posted)

Phillips: Qi Hardware Releases Free Wireless Hardware

Comments (none posted)

FSFE on AVM v. Cybits

Comments (28 posted)

Nokia's N9 handset launched

Comments (41 posted)

Formally Launching the MeeGo Community Device Program

Comments (none posted)

Karen Sandler has been named GNOME foundation executive director

Comments (none posted)

Oracle v. Google - Updating the Reexams (Groklaw)

Comments (5 posted)

Keeping the Desktop Dream Alive (LinuxInsider)

Comments (97 posted)

Keeping the Desktop Dream Alive, Part 2 (TechNewsWorld)

Comments (none posted)

HTML5 & CSS3 For The Real World--New from SitePoint

Full Story (comments: none)

Call for Participation: Workshops and BoFs at the Desktop Summit 2011

Full Story (comments: none)

PyCon Finland 2011 Call For Proposals

Full Story (comments: none)

2nd Call For Papers, 18th Annual Tcl/Tk Conference 2011

Full Story (comments: none)

LinuxCon North America schedule released

Comments (2 posted)

Events: June 30, 2011 to August 29, 2011

If your event does not appear here, please tell us about it.