LWN.net Weekly Edition for July 19, 2012

Oracle takes aim at CentOS

CentOS is the most popular of the free RHEL clones; it is widely offered to customers by hosting providers. It has become the default option for anybody wanting to run a RHEL-like system without actually paying for it. There can be no doubt that some sites would decide to pop for a real RHEL subscription if a system like CentOS were not available. At the same time, there must certainly be a steady stream of customers who started with CentOS, only to decide that Red Hat's support would be a worthwhile upgrade.

Oracle clearly has its eyes on that stream of customers. The plan seems to be to make it easy for CentOS users to switch a running system over to Oracle's distribution. And easy it is, if Oracle's instructions are to be believed; one need only download a shell script from Oracle's server and feed it, unread, to a root shell. The script will tweak some repository pointers and install a few packages, but it leaves most of the existing CentOS (or Scientific Linux) system as-is until the next update.

Why would CentOS users, who are benefiting from the efforts of a free software project, want to switch to Oracle's offering? Oracle is clearly trying to take advantage of the security update difficulties experienced by CentOS in 2011. The page reads:

Things have improved in the CentOS camp since the 2011 difficulties. The project has changed its workflow and found the sponsorship to hire a couple of developers; the recent CentOS 6.3 release surprised almost everybody with its promptness. But CentOS remains a project with limited resources and a lot of tedious work to do; it's always possible that things could fall behind again. CentOS users who were left without security updates in 2011—at least, those who are concerned about the security of their systems—cannot entirely eliminate that fear from the backs of their minds, even if things look better now.

So it is possible that Oracle is on to something here. Some CentOS users may well jump at the chance to switch to a free RHEL clone with big-company support behind it. And, when some of those users decide that paid support is worth their while, Oracle will naturally be the first provider to come to mind. This little initiative might well translate into some extra revenue for Oracle.

Of course, there could be some costs. The CentOS project is unlikely to be strengthened by having some of its users defect to Oracle. In the worst (presumably unlikely) case, CentOS could be fundamentally damaged if vast numbers of users were to vote with their feet and leave. That would leave the community with one less free enterprise distribution project. There have been a lot of complaints that CentOS is far from a truly open, community-oriented project. But anybody concerned about those issues is unlikely to find Oracle's distribution more to their liking. Oracle does make some good contributions, but community-oriented development is not, in general, among the company's greatest strengths.

Also worth keeping in mind is the fact that Oracle is making no promises that it will provide this free service for any period of time. If this effort fails to provide the desired financial results, Oracle could pull the plug on it at any time—as it did with OpenSolaris. That would leave ex-CentOS users with the choice of somehow migrating back to CentOS (assuming CentOS is still there and healthy) or becoming paid Oracle customers in a hurry. One could argue that any free (beer) distribution poses such a hazard, but a corporate-controlled distribution can only be doubly hazardous.

So this initiative by Oracle looks like it could be either a positive or a negative thing. It could increase the choices for users looking for a well-supported, highly stable, free-of-charge distribution and increase competition in the enterprise distribution space in general. Or it could just be a cynical attempt by a large corporation to profit from a free software project's success and deprive its main competitor of a potential revenue stream. Enterprise distribution users will have to make their own choice as to where their best interests lie.

Comments (25 posted)

Akademy: Freedom and the internet

Mathias Klang opened this year's Akademy with a keynote look at freedom and the internet. It was something of a cautionary tale that outlined the promises that technology brings, while noting that the dangers are often being overlooked. Klang comes from an academic and legal background—he is currently a researcher and senior lecturer at the University of Göteborg in Sweden—which gives him something of a different perspective on technology issues.

Klang's talk was titled "Expressions in Code and Freedom", but he came up with a different title the night before the talk: The TiVo-ization of everyday life. That title is "silly", but it does reflect some of the dangers he sees. He noted that he is not a programmer, but is surrounded by them, and they "put up with my stupidity". His background in the law means that he "likes reading licenses" and thinks everyone should. His current research is looking into social media, particularly in the area of control by the providers.

There have been multiple revolutions in communication over the years, with writing only coming about 6000 years ago or so. Punctuation did not arise until 200 BC and putting spaces between words is only 1000 years old. Gutenberg (who Klang called "The Steve Jobs of his day") revolutionized writing once again with the printing press, but the digitization of information was arguably the biggest revolution.

Once information has been digitized we can start connecting up the devices that store that data, which leads to the internet. The internet is not a bad thing, per se, but it is set up for control. The promise of the open web ("so wonderfully open, so wonderfully free") is great, but that openness invites people to come in and start closing it down in various ways.

The web started as an open platform, but that "wild web" is becoming an endangered species. For example, he said, we don't actually publish our own links anymore, instead we use various social media services to send each other links. That leaves us more and more dependent on the people who collect and store our data. It is becoming rare for people to create their own permanent web sites to store their data as it is largely being stored under the control of social media service providers.

"What would newspapers write about if we didn't have Facebook?", Klang asked. Perhaps they would write about the euro crisis instead, he joked. More seriously, social change is happening and much of it is being brought about by technology.

For example, he noted that online Scrabble games are all the rage right now. Two years ago, you wouldn't go to the pub and brag about playing Scrabble. But in Sweden (at least), people are constantly posting their high scores and such to Facebook.

Social media is set up to "create a performance lifestyle", he said. The whole idea behind it is to have an audience, but the tools used to reach that audience are controlled by the providers. Another example is Klang's Facebook post of a picture of his morning coffee as "my amazing coffee". He gets comments from people all over the world who are "lurking around my digital life". It is a bit creepy, overall. The things he routinely does online today would have been considered stalking ten years ago, but "now I'm Facebooking".

The walled gardens and information silos that typify many internet services are a threat. The service providers ensure that they "keep us entertained so we will supply them more data", he said. But, without access to the underlying code and data, we are totally at their mercy.

Klang gave more examples of how technology, social media in particular, is worming its way into everyday life. "People say that if you want to start a revolution, use Facebook", he said, and they generally point to the recent events in Egypt as an example. In Sweden, educators are asking "should we be teaching Facebook in school?" and "how do I use Facebook" as an educational tool?

Beyond that, even police departments are going online. The Swedish police now have a Facebook presence and have even had crimes reported to them via that mechanism. There was recently a "wonderful or sad" twitter message (i.e. "tweet") about a man lying unconscious in Göteborg. Klang does not think that's a good way to report such things, "but the police think it is and that's sad".

Certainly social media sites increase our ability to talk to one another, which is good. But much of that communication is being forced into these walled gardens, he said, "and that's scary".

"It's only technology" is something that is heard a lot, but that's something of a slippery slope. As an example, he pointed to tubular anti-homeless benches in Tokyo. Instead of passing a law against sleeping in parks or putting up a sign, the benches make it almost impossible to sleep on them. This is an example of TiVo-ization in real life, he said. If we create a law against sleeping on benches, there will be complaints about human rights, but creating a technological measure avoids those problems. "Design choices have consequences", Klang said.

"The more technology we embed into our lives, the less freedom we have", he said. We should all "love technology", but recognize that every piece of it has an effect on our lives. That includes all kinds of technology, not just gadgets and web sites, but things like chairs, desks, and carpets as well.

One of the problems is that the educational system teaches students how to use technology, "but we don't teach them code", Klang said. Sweden, for example, has been focusing on the use of various gadgets in schools, but you don't have to be "vaguely technical to use an iPhone or iPad". Educators are asking how to use the iPad in the classroom, rather than asking whether they should use the device.

He referenced Douglas Adams's notion of "digital natives", that those under a certain age (15, say) natively understand technology changes while those over a certain age (e.g. 35) will always be immigrants and lack that understanding. Klang would like to see everyone become a digital native so that the understanding of technology and the consequences of technological change become widespread.

He had several suggestions toward that goal. To begin with, we should all try to "hack society for openness". Our infrastructure remains open, so far, but much of what runs atop it isn't. Richard Stallman, was "not being friendly" when he started the free software movement; "he was being right", Klang said.

"Be that guy", he suggested, and tell people what their information habits are doing to their (and other people's) lives. He likened it to getting a PhD, where you do research that "you and four other people in the world care about", but when people ask, you explain what it is and why it's important. In this case, it is necessary to make people aware of the problems that arise when "going from an information deficit to an information circus", which is what we have seen over the last decade or more. He also said that we should read all of the end-user license agreements (EULAs) and terms of service that are presented to us, but "I know you won't".

He closed with the idea that developers should at least think about what their code does, and "how you are affecting other people". All of the different gadgets out there "manipulate lives", but who decides how they do that, he asked. Everything we do with technology has effects on others, so he encouraged developers to think about those effects. He was clear that he wasn't advocating not building new devices and technologies, only asking that developers think about how those technologies might be used—or abused.

[ The author would like to thank KDE e.V. for travel assistance to Tallinn for Akademy. ]

Comments (5 posted)

Akademy: Contour and Plasma Active

Contour was a project to create a mobile, touch-friendly user experience atop Qt and KDE that started in 2010. It eventually became part of the KDE's Plasma Active effort so Eva Brucherseifer came to Akademy to recount how and why that came about. She also discussed how the Contour design process worked, along with the process of integrating Plasma Active with various device platforms.

Brucherseifer is a longtime KDE community member, going back to the 1990s. She was elected to the KDE e.V. board in 2002 and served as its president for three years, which is what happens "if you talk too much about what should be done". She started the embedded services company basysKom in 2003 and serves as its managing director.

Contour's history and goals

In the (northern hemisphere) summer of 2010, there were lots of ideas floating around that factored into the ideas behind Contour, she said. Using context and semantic data to provide recommendations for users was one idea. There was also a lot of talk about the mobile space, and back then people were anticipating numerous Linux devices to ship with Qt because of MeeGo. The idea of "daily use" devices and the need for an improved user interface to support that use case was another consideration.

People at basysKom were thinking about these ideas, which led to Contour. Later in 2010 the project was formed based on Qt and KDE. Contour project members wrote a proposal and the project was granted funding by the German Bundesministerium für Wirtschaft und Technologie (Federal Ministry of Economics and Technology). So, basysKom and the German government each funded half of the project.

The core idea behind Contour is to be "information-centric" rather than "app-centric" as the iPhone is. The goal was to create a new user paradigm where context and usage information are used to adapt the device's behavior to the user and their needs. A "learning system" would be used to try to derive patterns of use so that the device could anticipate and facilitate the user's task. There were ideas on how to do that, she said, but it was unclear if they would work.

For Contour, using KDE's "activities" made sense as a way to group the information around specific tasks that users do. By using contexts, such as what time of day it is and whether the user is at home or work, the interface can have an idea of the "things you do at those times". Deriving the usage patterns will help make it easier for the user, she said.

Contour is a user experience (UX) layer on top of the Plasma shell. Other UXes are possible, including ones targeted at set-top boxes or automobiles.

Designing the interface

The Contour interface evolved as the project tried out various ideas. Brucherseifer put up images of different parts of the interface (e.g. activity switcher, activity screen) as they changed from the initial prototype to the final design (which can be seen in her slides [PDF]). The activity switcher started out as a stacked set of activities, with a slider to activate the switch. That moved through a rotating "wheel" of activities that kept the slider to the final version which kept the wheel idea, but made the activity thumbnails completely visible and eliminated the slider. The three versions from her slides are shown at right.

Similarly, the activity screen that shows the files, applications, and other information associated with an activity went through a number of iterations. It moved from a tree-like structure to something more like a standard desktop. When she shows the interface to customers as an example of what can be done with Qt and QML, "they love it".

There are still more things to do, including an application launcher and a task switcher. WebKit integration is still lacking, but is needed because HTML 5 will be important, she said. Private, password-protected activities are another feature that will be added. Some of these features will be needed for a real product, but the project couldn't get to all of them.

In addition to a tablet UX, basysKom created an in-vehicle infotainment (IVI) UX as an internal project. It took two person-months to implement, she said, and it still needs some polish. It was done as an experiment to see how long it would take.

Designing the UXes went through several phases. It started with sketches, which were then turned into wireframes using Photoshop. Promising versions were then implemented using QML. Multiple iterations back and forth between those steps were required, she said.

Getting the code onto devices was challenging. The project started basing itself on top of MeeGo, because it believed that the two big companies behind the distribution would make for a stable platform. That didn't work out, of course, so a switch to Mer was eventually made.

The project "learned a lot" in getting its code running on two different devices: the WeTab/ExoPC and the Archos G9. There was a need to create binary RPMs for all of the packages as well as a single binary image that could be installed onto the devices. Some of those things are not necessarily easy to accomplish using volunteers in an open community.

Plasma Active

In March 2011, Plasma Active was announced. A workshop was held in Darmstadt, Germany where basysKom joined forces with Sebastian Kügler of open-slx and Aaron Seigo of Coherent Theory to create Plasma Active. Contour was adopted as the "activities and recommendations" piece of Plasma Active. Several coding sprints were held and two releases of Plasma Active have been made so far.

There were multiple reasons behind basysKom's decision to contribute Contour to KDE. The company could have held on to Contour and sold it to customers, but if it wanted to develop the technology upstream the code needed to be free. Trying to set up a "joint process" between the community and companies was another consideration. Plasma Active is a chance for KDE to succeed in the mobile space, which also factored into the decision. She (and, by extension, basysKom) cares about KDE, so it made sense to contribute Contour to the effort.

There were a few challenges that Plasma Active has faced over the last year or so. The cooperation between the community and companies can be hard at times. Volunteers often work all day then work on Plasma Active at night, which can make it hard to hit deadlines. The embedded development process is "not there yet", she said, and encouraged anyone interested to help out with that. In addition, desktop technologies are "too large and slow" for embedded devices; Plasma Active is still too large for many devices.

On the other hand, a lot of things have gone well. There were lots of KDE frameworks that could be reused, which made it relatively easy and quick to get something working. It surprised her how quickly things could come together. The project also produced highly motivated people; there were basysKom employees who wanted to continue working on Plasma Active even after the company needed to scale back its commitment, for example.

Plasma Active is not done yet; what has been produced so far is a "starting point", Brucherseifer said. There is currently no way to manage multiple open applications, for example, which is "probably very solvable", but needs to be done. More automation is needed in creating images for devices and there is a need for release management and QA as well. Those who are interested should get an Archos G9 and an image from basysKom to get started. Support for the WeTab/ExoPC is still available, but those devices are no longer on the market, so the project is focused on the G9 for now.

An audience member asked about how to get the design process used by Contour and Plasma Active into KDE. Brucherseifer said that the most important part is that developers need to be open to suggestions. basysKom hired two UX people to work on the project, but they "didn't really enjoy it". The developers had very different opinions from the designers, which made things difficult at times. There is a need for a different culture between developers and designers, "if both could give a little bit, it would be helpful", she said.

At the end of the talk, Brucherseifer demonstrated the interface, including showing how the recommendation system worked. The UX looks quite usable at this point, though there are still things to do as she noted. It will be interesting to see real devices shipped with Plasma Active (such as the Vivaldi tablet) down the road.

[ The author would like to thank KDE e.V. for travel assistance to Tallinn for Akademy. ]

Comments (none posted)

Page editor: Jonathan Corbet

Security

The ups and downs of strlcpy()

Adding the strlcpy() function (and the related strlcat() function) has been a perennial request (1, 2, 3) to the GNU C library (glibc) maintainers, commonly supported by a statement that strlcpy() is superior to the existing alternatives. Perhaps the earliest request to add these BSD-derived functions to glibc took the form of a patch submitted in 2000 by a fresh-faced Christoph Hellwig.

Christoph's request was rejected, and subsequent requests have similarly been rejected (or ignored). It's instructive to consider the reasons why strlcpy() has so far been rejected, and why it may well not make its way into glibc in the future.

A little prehistory

In the days before programmers considered that someone else might want to deliberately subvert their code, the C library provided just:

    char *strcpy(char *dst, const char *src);

with the simple purpose of copying the bytes from the string pointed to by src (up to and including the terminating null byte) to the buffer pointed to by dst.

Naturally, when calling strcpy(), the programmer must take care that the bytes being copied don't overrun the space available in the buffer pointed by dst. The effect of such buffer overruns is to overwrite other parts of a process's memory, such as neighboring variables, with the most common result being to corrupt data or to crash the program.

If the programmer can with 100% certainty predict at compile time the size of the src string, then it's possible (if unwise) to preallocate a suitably sized dst buffer and omit any argument checks before calling strcpy(). In all other cases, the call should be guarded with a suitable if statement to check the size of its argument. However, strings (in the form of input text) are one of the ways that humans interact with computers, and thus quite commonly the size of the src string is controlled by the user of a program, not the program's creator. At that point, of course, it becomes essential for every call to strcpy() to be guarded by a suitable if statement:

    char dst [DST_SIZE];
    ...
    if (strlen(src) < DST_SIZE)
        strcpy(dst, src);

(The use of < rather than <= ensures that there's at least one byte extra byte available for the null terminator.)

But it was easy for programmers to omit such checks if they were forgetful, inattentive, or cowboys. And later, other more attentive programmers realized that by carefully controlling what was written into the overflowed buffer, and overrunning into more exotic places such as function call return addresses stored on the stack, they could do much more interesting things with buffer overruns than simply crashing the program. (And because code tends to live a long time, and the individual programmers creating it can be slow to to learn about the sharp edges of the tools they use, even today buffer overruns remain one of the most commonly reported vulnerabilities in applications.)

Improving on strcpy()

Prechecking the arguments of each call to strcpy() is burdensome. A seemingly obvious way to relieve the programmer of that task was to add an API that allowed the caller to inform the library function of the size of the target buffer:

    char *strncpy(char *dst, const char *src, size_t n);

Although choosing a suitable value for n ensures that strncpy() will never overrun dst, it turns out that strncpy() has problems of its own. Most notably, if there is no null terminator in the first n bytes of src, then strncpy() does not place a null terminator after the bytes copied to dst. If the programmer does not check for this event, and subsequent operations expect a null terminator to be present, then the program is once more vulnerable to attack. The vulnerability may be more difficult to exploit than a buffer overflow, but the security implications can be just as severe.

One iteration of API design didn't solve the problems, but perhaps a further one can… Enter, strlcpy():

    size_t strlcpy(char *dst, const char *src, size_t size);

strlcpy() is similar to strncpy() but copies at most size-1 bytes from src to dst, and always adds a null terminator following the bytes copied to dst.

Problems solved?

strlcpy() avoids buffer overruns and ensures that the output string is null terminated. So why have the glibc maintainers obstinately refused to accept it?

The essence of the argument against strlcpy() is that it fixes one problem—sometimes failing to terminate dst in the case of strncpy(), buffer overruns in the case of strcpy()—while leaving another: the loss of data that occurs when the string copied from src to dst is truncated because it exceeds size. (In addition, there is still an unusual corner case where the unwary programmer can find that strlcat(), the analogous function for string concatenation, leaves dst without a null terminator.)

At the very least, (silent) data loss is undesirable to the user of the program. At the worst, truncated data can lead to security issues that may be as problematic as buffer overruns, albeit probably harder to exploit. (One of the nicer features of strlcpy() and strlcat() is that their return values do at least facilitate the detection of truncation—if the programmer checks the return values.)

All of which brings us full circle: to avoid unhappy users and security exploits, in the general case even a call to strlcpy() (or strlcat()) must be guarded by an if statement checking the arguments, if the state of the arguments can't be predicted with certainty in advance of the call.

Where are we now?

Today, strlcpy() and strlcat() are present on many versions of UNIX (at least Solaris, the BSDs, Mac OS X, and Irix), but not all of them (e.g., HP-UX and AIX). There are even implementations of these functions in the Linux kernel for internal use by the kernel code. Meanwhile, these functions are not present in glibc, and were rejected for inclusion in the POSIX.1-2008 standard, apparently for similar reasons to their rejection from glibc.

Reactions among core glibc contributors on the topic of including strlcpy() and strlcat() have been varied over the years. Christoph Hellwig's early patch was rejected in the then-primary maintainer's inimitable style (1 and 2). But reactions from other glibc developers have been more nuanced, indicating, for example, some willingness to accept the functions. Perhaps most insightfully, Paul Eggert notes that even when these functions are provided (as an add-on packaged with the application), projects such as OpenSSH, where security is of paramount concern, still manage to either misuse the functions (silently truncating data) or use them unnecessarily (i.e., the traditional strcpy() and strcat() could equally have been used without harm); such a state of affairs does not constitute a strong argument for including the functions in glibc.

The appearance of an embryonic entry on this topic in the glibc FAQ, with a brief rationale for why these functions are currently excluded, and a note that "gcc -D_FORTIFY_SOURCE" can catch many of the errors that strlcpy() and strlcat() were designed to catch, would appear to be something of a final word on the topic. Those that still feel that these functions should be in glibc will have to make do with the implementations provided in libbsd for now.

Finally, in case it isn't obvious by now, it should of course be noted that the root of this problem lies in the C language itself. C's native strings are not managed strings of the style natively provided in more modern languages such as Java, Go, and D. In other words, C's strings have no notion of bounds checking (or dynamically adjusting a string's boundary) built into the type itself. Thus, when using C's native string type, the programmer can never entirely avoid the task of checking string sizes when strings are manipulated, and no replacements for strcpy() and strcat() will ever remove that need. One might even wonder if the original C library implementers were clever enough to realize from the start that strcpy() and strcat() were sufficient—if it weren't for the fact that they also gave us gets().

Comments (95 posted)

Brief items

Security quotes of the week

Comments (17 posted)

Systemd gets seccomp filter support

Comments (24 posted)

Android Security Overview

Comments (3 posted)

New vulnerabilities

automake: code execution

Comments (none posted)

exif: information leak

Comments (none posted)

extplorer: cross-site request forgery

Comments (none posted)

glibc: multiple vulnerabilities

Comments (none posted)

gypsy: multiple vulnerabilities

Comments (none posted)

libexif: multiple vulnerabilities

Comments (none posted)

libxslt: denial of service

Comments (none posted)

libytnef: buffer overflow

Comments (none posted)

mono: cross-site scripting

Comments (none posted)

mozilla: multiple vulnerabilities

Comments (none posted)

mozilla: denial of service

Comments (none posted)

nova: denial of service

Comments (none posted)

openldap: weaker than expected encryption

Comments (none posted)

puppet: multiple vulnerabilities

Comments (none posted)

rhythmbox: code execution

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

Stable updates: the 3.0.37 and 3.4.5 updates were released on July 16; 3.2.23 was released on July 13. The 3.0.38 and 3.4.6 updates are in the review process as of this writing; they can be expected on or after July 19.

Comments (none posted)

Quotes of the week

-#define HV_LINUX_GUEST_ID_HI		0xB16B00B5
+#define HV_LINUX_GUEST_ID_HI		0x0DEFACED

Comments (11 posted)

30 Linux Kernel Developers in 30 Weeks: Dave Jones (Linux.com)

Comments (2 posted)

Kernel development news

Random numbers for embedded devices

Computing systems traditionally do not have sources of true randomness built into them. So they have operated by attempting to extract randomness from the environment in which they operate. The fine differences in timing between a user's keystrokes are one source of randomness, for example. The kernel can also use factors like the current time, interrupt timing, and more. For a typical desktop system, such sources usually provide enough randomness for the system's needs. Randomness gets harder to come by on server systems without a user at the keyboard. But the hardest environment of all may be embedded systems and network routers; these systems may perform important security-related tasks (such as the generation of host keys) before any appreciable randomness has been received from the environment.

As Zakir Durumeric, Nadia Heninger, J. Alex Halderman, and Eric Wustrow have documented, many of the latter class of systems are at risk, mostly as a result of keys generated with insufficient randomness and predictable initial conditions. They write: "We found that 5.57% of TLS hosts and 9.60% of SSH hosts share public keys in an apparently vulnerable manner, due to either insufficient randomness during key generation or device default keys." They were also able to calculate the actual keys used for a rather smaller (but still significant) percentage of hosts. Their site includes a key checker; concerned administrators may point it at their hosts to learn if their keys are vulnerable.

Fixes for this problem almost certainly need to be applied at multiple levels, but kernel-level fixes seem particularly important since the kernel is the source for most random numbers used in cryptography. To that end, Ted Ts'o has put together a set of patches designed to improve the amount of randomness available in the system from when it first boots. Getting there involves making a number of changes.

One of those is to fix the internal add_interrupt_randomness() function, which is used to derive randomness from interrupt timing. Use of this function has been declining in recent years, as a result of both its cost and concerns about the actual randomness of many interrupt sources. Ted's patch set tries to address the cost by batching interrupt-derived randomness on a per-CPU basis and only occasionally mixing it into the system-wide entropy pool. That mixing is also done with a new, lockless algorithm; this algorithm contains some small race conditions, but those could be seen to make the result even more random. An attempt is made to increase the amount of randomness obtained from interrupts by mixing in additional data, including the value of the instruction pointer at the time of the interrupt. After this change, adding randomness from interrupts should be fast and effective, so it is done by default for all interrupts; the IRQF_SAMPLE_RANDOM interrupt flag no longer has any effect.

Next, the patch set adds a new function:

    void add_device_randomness(const void *buf, unsigned int size);

The purpose is to allow drivers to mix in device-specific data that, while not necessarily random, is system-specific and unpredictable. Examples include serial, product, and manufacturer information from attached USB devices, the "write counter" from some realtime clock devices, and the MAC address from network devices. Most of this data should be random from the point of view of an attacker; it should help to prevent the situation where multiple, newly-booted devices generate the same keys.

Finally, Ted's patch set also changes the use of the hardware random number generator built into a number of CPUs. Rather than return random numbers directly from the hardware, the code now mixes hardware random data into the kernel's entropy pool and generates random numbers from there. His reasoning is that using hardware random numbers directly requires placing a lot of trust in the manufacturer:

Mixing hardware random data into the entropy pool helps to mitigate that threat. The first time this patch came around, Linus rejected it, saying "It would be a total PR disaster for Intel, so they have huge incentives to be trustworthy." That opinion was not universally shared, though, and the patch remains in the current set. Chances are it will be merged in its current form.

An important part of the plan, though, is to get these patches into the stable updates despite their size. Then, with luck, device manufacturers will pick them up relatively quickly and stop shipping systems with a known weakness. Even better would be, as Ted suggested, to make changes at the user-space levels as well. For example, delaying key generation long enough to let some randomness accumulate should improve the situation even more. But making things better at the kernel level is an important start.

Comments (24 posted)

TCP small queues

A number of bloat-fighting changes have gone into the kernel over the last year. The CoDel queue management algorithm works to prevent packets from building up in router queues over time. At a much lower level, byte queue limits put a cap on the amount of data that can be waiting to go out a specific network interface. Byte queue limits work only at the device queue level, though, while the networking stack has other places—such as the queueing discipline level—where buffering can happen. So there would be value in an implementation that could limit buffering at levels above the device queue.

Eric Dumazet's TCP small queues patch looks like it should be able to fill at least part of that gap. It limits the amount of data that can be queued for transmission by any given socket regardless of where the data is queued, so it shouldn't be fooled by buffers lurking in the queueing, traffic control, or netfilter code. That limit is set by a new sysctl knob found at:

    /proc/sys/net/ipv4/tcp_limit_output_bytes

The default value of this limit is 128KB; it could be set lower on systems where latency is the primary concern.

The networking stack already tracks the amount of data waiting to be transmitted through any given socket; that value lives in the sk_wmem_alloc field of struct sock. So applying a limit is relatively easy; tcp_write_xmit() need only look to see if sk_wmem_alloc is above the limit. If that is the case, the socket is marked as being throttled and no more packets are queued.

The harder part is figuring out when some space opens up and it is possible to add more packets to the queue. The time when queue space becomes free is when a queued packet is freed. So Eric's patch overrides the normal struct sk_buff destructor when an output limit is in effect; the new destructor can check to see whether it is time to queue more data for the relevant socket. The only problem is that this destructor can be called from deep within the network stack with important locks already held, so it cannot queue new data directly. So Eric had to add a new tasklet to do the actual job of queuing new packets.

It seems that the patch is having the intended result:

He also ran some tests over a 10Gb link and was able to get full wire speed, even with a relatively small output limit.

There are some outstanding questions, still. For example, Tom Herbert asked about how this mechanism interacts with more complex queuing disciplines; that question will take more time and experimentation to answer. Tom also suggested that the limit could be made dynamic and tied to the lower-level byte queue limits. Still, the patch seems like an obvious win, so it has already been pulled into the net-next tree for the 3.6 kernel. The details can be worked out later, and the feature can always be turned off by default if problems emerge during the 3.6 development cycle.

Comments (2 posted)

Kernel configuration for distributions

Configuring a kernel was once a fairly straightforward process, only requiring knowledge of what hardware needs to be supported. Over time, things have gotten more complex in general, but distributions have added their own sets of dependencies on specific kernel features—dependencies that can be difficult for regular users to figure out. That led Linus Torvalds to put out an RFC proposal to add distribution-specific kernel configuration options.

The problem stems from distributions' user space needing certain configuration options enabled in order to function correctly. Things like tmpfs and devtmpfs support, control groups, security options (e.g. SELinux, AppArmor), and even raw netfilter table support were listed by Torvalds as "support infrastructure" options that are required by various distributions. But, in addition to being hard to figure out, those options tend to change over time, so a configuration that worked for Fedora N may not work for Fedora N+1. The resulting problems can be hard to find as Torvalds pointed out: "There's been several times when I started with my old minimal config, and the resulting kernel would boot, but something wouldn't quite work right, and it can be very subtle indeed."

So, he suggested adding distribution-specific Kconfig files:

There are others ways to get there, of course, but they leave something to be desired, Torvalds said. Copying the distribution config file would work, but would bring along a bunch of extra options that aren't really necessary for the proper operation of the distribution. Using make localmodconfig (which selects all of options from the running kernel) suffers from much the same problem, he said. The ultimate goal is to have more people able to build kernels:

In general, the idea was met with approval on linux-kernel. There were concerns about how the distribution-specific files would be maintained, and that sometimes they might get out of sync with the distribution's requirements. Dave Jones noted that he sometimes gets blindsided by Fedora kernel requirements (and he is the Fedora kernel maintainer).

Torvalds is pretty explicitly not looking for a perfect solution, however, just one that is better: "even a 'educated guess' config file is better than what we have now". In that message, he outlines two requirements that he sees for the feature. The first is that each configuration option that is selected for a particular distribution version come with a comment explaining why it is needed. The second is that the configuration options be the minimum required to make the system function properly—not that it "grow to contain all the options just because somebody decided to just add random things until things worked".

Commenting the options may be difficult even for those who work directly on distribution kernels though. Ben Hutchings (who maintains the Debian kernel) pointed out that he sometimes does not know the reason that a particular option is needed, particularly at some later point: "just because an option was requested and enabled to support some bit of userland, doesn't mean I know what's using or depending on it now".

Other kinds of configuration options are possible, of course. In his original message, Torvalds mentioned configurations for "common platforms", such as a "modern PC laptop" that would choose options typically required for those (USB storage, FAT/VFAT, power management, etc.). He specifically said that platform configuration should be considered an entirely separate feature from the distribution idea.

KVM (and other virtualization) users were also interested in creating an option that would select all of the drivers and other options needed for those kernels. Currently "you need to hunt through 30+ different menus in order to find what you need to run in a basic KVM virtual machine", as Trond Myklebust put it. There was a lot of discussion (and much agreement) on the need for better configuration options for virtualization, but some of that got rather far afield from Torvalds's original proposal.

Unsurprisingly, kernel developers started thinking about how they could use the feature. There was concern that choosing a particular distribution and its dependencies would make it harder for kernel developers to further customize the configuration. David Lang had some specific complaints about the approach suggested in the RFC, noting that it would be hard to choose a Fedora kernel without getting SELinux for example. He also was concerned about the amount of churn these defconfig-like files might cause (referencing the movement to reduce the number of defconfigs in the ARM tree). But Torvalds makes it clear that Lang and other kernel hackers are not the target of the feature:

There may be ways to satisfy both camps—Lang seemed to think so anyway—but until someone actually posts some code, it's hard to say. While there was general agreement that the feature would be useful, so far no one has stepped up to do the work. Whether Torvalds plans to do that or was just floating a trial balloon and hoping someone else would run with it is unclear, but it does seem like a feature worth having.

Comments (22 posted)

Patches and updates

Kernel trees

• Linus Torvalds: Linux 3.5-rc7 . (July 15, 2012)

• Greg KH: Linux 3.4.5 . (July 16, 2012)

• Ben Hutchings: Linux 3.2.23 . (July 13, 2012)

• Steven Rostedt: 3.2.23-rt36 . (July 18, 2012)

• Greg KH: Linux 3.0.37 . (July 16, 2012)

Core kernel code

• Michel Lespinasse: rbtree updates . (July 13, 2012)

• Frederic Weisbecker: rcu: Userspace RCU extended quiescent state v2 . (July 16, 2012)

• Thomas Gleixner: Per cpu thread hotplug infrastructure - V3 . (July 16, 2012)

• Michael Wang: sched: dynamically schedule domain configuration . (July 16, 2012)

• Tejun Heo: workqueue: reimplement CPU hotplug to keep idle workers . (July 17, 2012)

• Aristeu Rozanski: cgroup: add xattr support . (July 18, 2012)

Development tools

• Anton Vorontsov: KGDB/KDB FIQ (NMI) debugger . (July 13, 2012)

• David Herrmann: fblog: Framebuffer kernel log driver v3 . (July 16, 2012)

• Andrea Arcangeli: AutoNUMA benchmark 0.1 . (July 16, 2012)

Device drivers

• G.Shark Jeong: leds: Add LED driver for lm3554 chip . (July 13, 2012)

• Jon Mason: PCI-Express Non-Transparent Bridge Support . (July 16, 2012)

• Peter Ujfalusi: MFD/GPIO: Add twl6040 GPO driver . (July 18, 2012)

• Or Gerlitz: Add Ethernet IPoIB driver . (July 18, 2012)

• Hans Verkuil: Add adv7604/ad9389b drivers . (July 18, 2012)

• Javier Martin: media: coda: Add driver for Coda video codec. . (July 18, 2012)

Documentation

• Laurent Pinchart: Documentation: DocBook DRM framework documentation . (July 16, 2012)

Filesystems and block I/O

• Jonathan Brassow: DM RAID: Add support for MD RAID10 . (July 16, 2012)

Memory management

• Rob Clark: dma-fence: dma-buf synchronization . (July 12, 2012)

• Mel Gorman: Swap-over-NBD without deadlocking V15 . (July 12, 2012)

• Mel Gorman: Swap-over-NFS without deadlocking V9 . (July 12, 2012)

• Rafael Aquini: make balloon pages movable by compaction . (July 17, 2012)

• Yasuaki Ishimatsu: memory-hotplug : hot-remove physical memory . (July 18, 2012)

Networking

• Yuchung Cheng: TCP Fast Open client . (July 16, 2012)

Architecture-specific

• Domenico Andreoli: ARM: Decompressor multiplatform support . (July 16, 2012)

• Florian Fainelli: ARM: support for Moschip MCS814x SoCs . (July 16, 2012)

Virtualization and containers

• Asias He: Add vhost-blk support . (July 13, 2012)

• Alex Williamson: kvm: level irqfd and new eoifd . (July 16, 2012)

Miscellaneous

• Ben Hutchings: ethtool 3.4.2 released . (July 18, 2012)

Page editor: Jonathan Corbet

Distributions

Left by Rawhide

Rawhide, as it happens, is older than Fedora; it was originally launched in August, 1998—almost exactly 14 years ago. Its purpose was to give Red Hat Linux users a chance to test out releases ahead of time and report bugs; it could also have been seen as an attempt to attract users who would otherwise lean toward a distribution like Debian unstable. Rawhide was not a continuously updated distribution; it got occasional "releases" on a schedule determined by Red Hat. One could argue that Fedora itself now plays the role that Red Hat had originally envisioned for Rawhide. But Rawhide persists for those who find Fedora releases to be far too stable, boring, and predictable.

The Rawhide distribution does provide occasional surprises, to the point that any rational person should almost certainly not consider running it on a machine that is needed for any sort of real work. But, at its best, Rawhide is an ideal tool for LWN editors, a group that has not often been accused of being overly rational. Running Rawhide provides a front-seat view into what the development community is up to; fresh software shows up there almost every day. And it can be quite fresh; Fedora developers will often drop beta-level software into Rawhide with the idea of helping to stabilize it before it shows up in finished form as part of a future Fedora release. With Rawhide, you can experience future software releases while almost never having to figure out how to build some complex project from source.

Rawhide also helps one keep one's system problem diagnosis and repair skills up to date—usually at times when one would prefer not to need to exercise such skills. But that's just part of the game.

In the early days of Fedora, Rawhide operated in a manner similar to Debian unstable, but with a shorter release cycle. When a given Fedora release hit feature freeze, Rawhide would freeze and the flow of scary new packages into the distribution would stop. Except, of course, when somebody put something badly broken in anyway, just to make sure everybody was still awake. While the Fedora release stabilized, developers would accumulate lots of new stuff for the next release; it would all hit the Rawhide repository shortly after the stable release was made. One quickly learned to be conservative about Rawhide updates during the immediate post-release period; things would often be badly broken. So it seemed to many that Rawhide was a little too raw during parts of the cycle while being too frozen and boring at other times.

Sometime around 2009, the project came up with the "no frozen Rawhide" idea. The concept was simple: rather than stabilize Fedora releases in the Rawhide repository, each stable release would be branched off Rawhide around feature-freeze time. So Rawhide could continue forward in its full rawness while the upcoming release stabilized on a separate track. It was meant to be the best of both worlds: the development distribution could continue to advance at full speed without interfering with (or getting interference from) the upcoming release. It may be exactly that, but this decision has changed the nature of the Rawhide distribution in fundamental ways.

In May, 2011, Matthew Miller asked the fedora-devel list: "is Rawhide supposed to be useful?" He had been struggling with a problem that had bitten your editor as well: the X server would crash on startup, leaving the system without a graphical display. The fact that Rawhide broke in such a fundamental way was not particularly surprising; Rawhide is supposed to break in horrifying ways occasionally. The real problem is that Rawhide stayed broken for a number of weeks; the responsible developer, it seems, had simply forgotten about the problem. Said developer had clearly not been running Rawhide on his systems; this was the sort of problem that tended to make itself hard to forget for people actually trying to use the software.

So your editor asked: could it be that almost nobody is actually running Rawhide anymore? The fact that it could be unusably broken for weeks without an uproar suggested that the actual user community was quite small. One answer that came back read: "In the week before F15 change freeze, are you really surprised that nobody's running the F16 dumping ground?" At various times your editor has, in response to Rawhide bug reports, been told that running Rawhide is a bad idea (example, another example, yet another example). There seems to be a clear message that, not only are few people running Rawhide, but nobody is really even supposed to be running it.

The new scheme shows its effects in other ways as well. Bug fixes can be slow to make it into Rawhide, even after the bug has been fixed in the current release branch. Occasionally, the "stable" branch has significantly newer software than Rawhide does; Rawhide can become a sort of stale backwater at times. It is not surprising that Fedora developers are strongly focused on doing a proper job with the stable release; that bodes well for the project as a whole. But this focus has come at the expense of the Rawhide branch, which is now seen, by some developers at least, as a "dumping ground."

Recently, your editor applied an update that brought about the familiar "GNOME just forgot all your settings" pathology, combined with the apparent loss of the ability to fix those settings. It was necessary to return to xmodmap commands to put the control key where $DEITY (in the form of the DEC VT100 designers) meant it to be, for example. Some time had passed before this problem was discovered, so the obvious first step was to update again, get current, and see if the problem had gone away. Alas, that was just when Rawhide exploded in a fairly spectacular fashion, with an update leaving the system corrupted and unable to boot. Not exactly the fix that had been hoped for. Fortunately, many years of experience have taught the value of exceptionally good backups, but the episode as a whole was not fun.

But what was really not fun was the ensuing discussion. Chuck Forsberg made the reasonable-sounding suggestion that perhaps developers could be bothered to see if their packages actually work before putting them into Rawhide. Adam Williamson responded:

This, in your editor's eyes, is not the description of a distribution that is actually meant to be used by real people.

The interesting thing is that Fedora developers seem to be mostly happy with how Rawhide is working. It gives them a place to stage longer-term changes and see how they play with the rest of the system. Problems can often be found early in the process so that the next Fedora development cycle can start in at least a semi-stable condition. By looking at Rawhide occasionally, developers can get a sense for what their colleagues are up to and what they may have to cope with in the future.

In other words, Rawhide seems to have evolved into a sort of distribution-level equivalent to the kernel's linux-next tree. Developers put future stuff into it freely, stand back, and watch how the monster they have just created behaves for a little while. But it is a rare developer indeed who actually does real work with linux-next kernels or tries to develop against them. Producing kernels that people actually use is not the purpose of linux-next, and, it seems, producing a usable distribution is not Rawhide's purpose.

This article was meant to be a fierce rant on how the Fedora developers should never have had the temerity to produce a development distribution that fails to meet your editor's specific needs. But everybody who read it felt the need to point out that, actually, the Fedora project is not beholden to those needs. If the current form of Rawhide better suits the project's needs and leads to better releases, then changing Rawhide was the right thing for the project to do.

Your editor recognizes that, and would like to express his gratitude for years of fun Rawhide roller coaster rides. But it also seems like time to move on to something else that better suits current needs. What the next distribution will be has yet to be decided, though. One could just follow the Fedora release branches and get something similar to old-style Rawhide with less post-release mess, but perhaps it's time for a return to something Debian-like or to go a little further afield. However things turn out, it should be fun finding a new distribution to get grumpy about.

Comments (96 posted)

Brief items

Distribution quote of the week

Comments (none posted)

openSUSE 12.2 RC1 available for testing

Comments (1 posted)

Distribution News

Debian GNU/Linux

Bits of Debian Med team: report from LSM, Geneva 2012

Full Story (comments: none)

DebConf makes in-roads in Central America

Full Story (comments: none)

Gentoo Linux

Gentoo Council Elections Results for term 2012/2013

Full Story (comments: none)

Newsletters and articles of interest

Distribution newsletters

• Debian Project News (July 12)
• DistroWatch Weekly, Issue 465 (July 16)
• Maemo Weekly News (July 16)
• Ubuntu Weekly Newsletter, Issue 274 (July 15)

Comments (none posted)

31 Flavors of Linux (OStatic)

Comments (none posted)

Page editor: Rebecca Sobol

Development

John the Ripper

John the Ripper (JtR) is a password-cracking utility developed at Openwall. The recently-released 1.7.9-jumbo-6 version lands a number of important features, such as the ability to unlock RAR, ODF, and KeePass files, the ability to crack Mozilla master passwords, and the ability to speed up cracking by using GPUs — for some, but not all, tasks.

Get crackin'

Despite the heavy dose of crypto-speak in the documentation, in practice JtR is a straightforward-to-use tool with which you can recover lost passwords, open locked files, or test users' password strength from the command line — recovering a password can be as simple as running:

    john thepasswdfile

JtR can be used to crack a single password or hash value on its own, or it can be deployed against a file full of passwords, logging its successes. There are switches to automatically skip accounts without a shell or by group membership, and utilities to perform related tasks such as emailing users with weak passwords. By default, JtR uses its own encryption routines when cracking a password, but it can also call the system's crypt(3) function, which may be helpful for auditing password hash formats not yet supported by the program.

Not that there are a lot of unsupported formats; JtR tackles many different encryption and hashing algorithms — around 30 in this release. But the main program uses the same "batch cracking" methodology regardless of the underlying format being cracked. Most of the new formats are implemented as plugins, and indeed many of the additions in this latest release were contributed by the JtR community. Considerable effort is also expended on optimizing JtR's performance, which naturally involves squeezing every available advantage out of the architecture. As a result, the optimizations available vary depending on the file format and the processor.

New and improved

The latest release is named JtR 1.7.9-jumbo-6; the "jumbo" indicates that it incorporates community-contributed code. For the sake of comparison, the most recent non-jumbo release is 1.7.9, from November 2011. Openwall also sells a "pro" version of JtR for Linux and Mac OS X, which is currently at 1.7.3, and rolls in a few additional hash types, plus binary packages and a hefty multi-lingual wordlist file. From what I can tell, the community-driven jumbo-6 packages now implement most of the additional features and optimizations in the "pro" version, but of course you get no company-provided support. If compiling from source is too much of a headache, there are also community-contributed builds for Linux (32- and 64-bit), Solaris, OS X, and Android.

According to the release announcement, 1.7.9-jumbo-6 adds 40,000 additional lines of code (that is, not counting changed lines) over the previous release. New hash types supported in this version include IBM's Resource Access Control Facility (RACF), GOST, SHA512-crypt, SHA256-crypt, and several SHA-512 or SHA-256 derivatives (such as those used by DragonFly BSD, EPiServer, and Drupal 7). Several other web application password formats are on the list as well, including Django 1.4, the forum package WoltLab Burning Board 3, and the flavor of SHA1 used by LinkedIn (which reminds one of LinkedIn's recent password troubles).

Just as interesting are the "non-hash" functions, which include a number of encrypted file formats and authentication methods — specifically, message authentication codes and challenge-response protocols (which first require capturing the challenge-response packets using Wireshark or another network sniffer). New in this category are several password-storage formats (Mac OS X keychains, KeePass 1.x files, Password Safe files, and Mozilla master-password files) and general file types (OpenDocument files, Microsoft Office 2007/2010 files, and RAR archives encrypted with the -p option, which leaves metadata in plaintext). The authentication methods added include WPA-PSK, VNC, SIP, and various flavors of HMAC.

GPU-assisted cracking

There are other "assorted enhancements" discussed in the announcement, but the most interesting is GPU-based parallel processing. There are two flavors supported: NVIDIA's Compute Unified Device Architecture (CUDA) and the cross-platform OpenCL. Not every hash or algorithm handled by JtR's normal CPU techniques has support for either, and few have support for both. Some of the GPU-assisted cracking code is marked as "inefficient" in the notes, and some have limitations on the specific graphics chips required. The notes also caution that some ATI cards can hang when running recent drivers.

As for whether or not CUDA and OpenCL result in faster password cracking, the answer at the moment is mixed. It depends largely on the hash or algorithm being cracked; for some (such as bcrypt), the project reports that running solely on the GPU is slower than running solely on the CPU. In addition, the benchmarks included in the announcement note that the GPU still loses on price/performance ratio when compared to CPUs. That may not matter if you are interested in using JtR to spy on your corporate foes, but for standard system administration tasks, it is an important factor. Yet even in those CPU-dominated circumstances, piling on the GPU in addition to the CPU should improve cracking times.

JtR already supports parallel processing with OpenMP for a far larger set of hashes and file formats. All of the new non-hash file formats supported in 1.7.9-jumbo-6 support OpemMP. The new release also includes many new SIMD CPU optimizations, for SSE, XOP, AVX, and even MMX. As a result, sorting out which options to use on which task may be a complicated affair; fortunately when several days of processing time may be required, a few minutes of research is comparatively small potatoes.

Openwall's Alexander Peslyak (who goes by the moniker "Solar Designer") wrote in the announcement that the GPU support "is just a development milestone, albeit a desirable one" for the time being, and that further optimization in future releases will improve its performance. GPU support is not a silver bullet, though. Like any task, password-cracking will always have bottlenecks — in JtR's case, having the main process generate and distribute the candidate passwords is frequently a bottleneck that GPUs cannot overcome. But as Peslyak wrote in 2011, even the question of whether or not it helps to move the candidate-generation to the GPU depends largely on the algorithm or hash. Password cracking is "is not so much about cryptography as it is about optimization of algorithms and code," he said. In that context, then, being able to make use of GPUs that would otherwise sit idle is a tool that needs to be exploited, even if it will not reduce the task to triviality.

Since I do not manage multi-user machines, it is difficult to weigh in on JtR's password-auditing features. But for password- or file-cracking, it is simple to get started and well-documented, which is about all that one could ask for. Password recovery falls into the category of "tools you hope you will never need," and when you find yourself recovering a password, you are not likely to enjoy the process. At least JtR makes it relatively painless — for you, although maybe not for your hardware.

Comments (none posted)

Brief items

Quotes of the week

Comments (6 posted)

eGenix PyRun - One file Python Runtime 1.0.0

EGenix has announced the release of its one-file Python interpreter PyRun, which is designed to provide "an almost-complete Python standard library" in a relocatable binary that does not demand system-wide installation. "Compared to a regular Python installation of typically 100MB on disk, this makes eGenix PyRun ideal for applications and scripts that need to be distributed to many target machines, client installations or customers."

Full Story (comments: none)

Firebug 1.10.0 released

Comments (3 posted)

Firefox 14 is now available

Full Story (comments: 2)

Lantz: automation and instrumentation in Python

Full Story (comments: none)

Redphone released as open source

Comments (none posted)

Newsletters and articles

Development newsletters from the last week

• Caml Weekly News (July 17)
• What's cooking in git.git (July 13)
• Haskell Weekly News (July 11)
• Perl Weekly (July 16)
• PostgreSQL Weekly News (July 15)
• Ruby Weekly (July 12)
• Tahoe-LAFS Weekly News (July 16)

Comments (none posted)

Berkholz: How to recruit open-source contributors

Comments (1 posted)

Jones: Bugzilla 4.2 – what’s new – searching

Comments (1 posted)

Page editor: Nathan Willis

Announcements

Brief items

An overview of small Linux systems

Comments (22 posted)

OSI Announces Individual Membership

Comments (none posted)

Articles of interest

How free is my phone? (The H)

Comments (2 posted)

Hardware Hacks: The Raspberry Pi is everywhere (The H)

Comments (23 posted)

Valve: Steam’d Penguins

Comments (57 posted)

New Books

Ubuntu Made Easy--New from No Starch Press

Full Story (comments: none)

Calls for Presentations

Embedded Linux Conference Call for Presentations

Full Story (comments: none)

Upcoming Events

Speakers set for Texas Linux Fest 2012

Full Story (comments: none)

Events: July 19, 2012 to September 17, 2012

If your event does not appear here, please tell us about it.

Page editor: Rebecca Sobol