LWN.net Weekly Edition for October 27, 2011

LCE2011: Kernel developer panel

In what is becoming a tradition at Linux Foundation conferences, a handful of kernel developers participated in a hour-long discussion of various topics at the first LinuxCon Europe (LCE), which is being held in Prague on October 26-28. The format typically doesn't change, with four kernel hackers and a moderator, but the participants do regularly change. The LCE edition featured Linus Torvalds, Paul McKenney, Alan Cox, Thomas Gleixner, and Lennart Poettering as the moderator. A number of timely topics were discussed (security, ARM, control groups), as well as a few timeless ones (user-space interfaces, aging kernel hackers, future challenges).

While Torvalds really doesn't need to introduce himself, he did so by saying that he writes very little code anymore and just acts as a central point to gather up the work of the other kernel hackers. He is "sadly, mainly just a manager these days", he said. McKenney maintains the read-copy-update (RCU) code in the kernel, Cox has done many things over the years but mostly works on System-on-Chip (SoC) support for Intel these days, while Gleixner works on "parts of the kernel nobody wants to touch" and maintains the realtime patch set.

User-space interfaces

Poettering said that he wanted to start things off with something a bit controversial, so he asked about user space compatibility. Clearly trying to evoke a response, he asked if it was hypocritical that certain user space interfaces have been broken in the past without being reverted. One of his examples was the recent version number change (from 2.6.x to 3.y) which broke a number of user space tools yet wasn't reverted.

The version number change is a perfect example, Torvalds said. Because it broke some (badly written) user-space programs, code was added to the kernel to optionally report the version as 2.6.x, rather than 3.y (where x = 40 + y). It is "stupid of us to add extra code to lie about our version", but it is important to do so because breaking the user experience is the biggest mistake that a program can make, he said.

From the very beginning, Torvalds's mantra has been that "breaking the user experience is a big no-no". The most important aspect of a program is how useful it is to users and "no project is more important than the user". Sometimes user space does get broken by mistake because someone writes code they think is making an improvement but breaks something. It is unavoidable and he feels bad about it when it happens.

Not only does Torvalds feel bad, but he makes sure that the other kernel developers feel bad about it as well, Gleixner said. Torvalds readily agreed that he is quite vocal about telling other kernel developers that they messed up when there is breakage of user space. "There is a saying that 'on the internet, no one can hear you being subtle'", so he doesn't even try, he said. Cox noted that the current unhappiness with GNOME 3 is a demonstration of why you don't suddenly change how things work for users.

There is pressure from hardware and software makers to constantly upgrade, but Cox said that if you ask users, they just want what worked before to continue working. Torvalds noted that he used to have some old a.out binaries that he would periodically test on new kernels to ensure they worked. One of them was an old shell that used deprecated system calls, but he wanted to ensure that even though he had moved on to newer shells, the old stuff would still work. It's fine to add new things, he said, as long you "make sure that the old ways of working still work".

The sysfs interface has been changed many times in ways that could have broken user-space programs, according to Poettering, and he wondered why that was allowed to happen. Cox said that it comes down to a question of whether users care. If no one reports the problem, then no one really sees it as one.

McKenney noted that his approach is to "work on things so far from user space that I can't even see it", but that he was concerned about adding tracepoints to the RCU code. Maintaining the exact same tracepoints could make it difficult to change RCU down the road.

An audience member asked if it was worth it to continue maintaining backward compatibility "forever", and wondered if it doesn't lead to more kernel complexity. Torvalds was unconcerned about that and said that the complexity problems in the kernel rarely arise because of backward compatibility. Open source means that the code can be fixed, he said. Cox echoed that, saying that if the kernel developers give user space better interfaces, the programs will eventually switch, and any compatibility code can be deprecated and eventually removed.

Aging hackers

After noting that many of the Kernel Summit (KS) participants were in bed by 9 or 10, Poettering asked whether the kernel community is getting too old or, as he put it, whether it has become an old man's club. Cox said that he saw a lot of fresh blood in the community, more than enough to sustain it. Torvalds noted, though, that the average age at the summit has risen by one year every year. It's not quite that bad, he said, but he also believes that the KS attendees are not an accurate reflection of the community as a whole. It tends to be maintainers that attend the summit, while many of the younger developers have not yet become maintainers.

Part of the problem is one of perception, according to Torvalds. The Linux kernel crowd used to be notably young, because the older people ignored what those crazy Linux folks were doing, he said. The kernel hackers had a reputation of being ridiculously young, but many of those same people are still around, and are just older now. Cox noted that the kernel is now a stable project and that it may be that some younger folks are gravitating to other projects that are more exciting. Those projects will eventually suffer the same fate, he said.

ARM

In response to an audience question, Torvalds wanted to clear something up: "everyone seems to think I hate ARM", but he doesn't, he said. He likes the architecture and instruction set of the ARM processor, but he doesn't like to see the fragmentation that happens because there is no standard platform for ARM like there is for x86. There are no standard interrupt controllers or timers, which is a mistake, he said. Because of the lack of a standard platform, it takes ten times the code to support ARM vs. x86.

ARM is clearly the most important architecture other than x86, he said, and some would argue that the order should be reversed. The good news is that ARM Linux is getting better, and the ARM community seems to be making progress, so he is much happier with ARM today than he was six months ago. It's not perfect, and he would like see more standardization, but things are much better. Torvalds said that he doesn't necessarily think that the PC platform is wonderful, but "supporting only a few ways to handle timers rather than hundreds is wonderful".

Security

Poettering asked if the panel would feel comfortable putting their private key on a public Linux machine that had other users. In general, the clear consensus was that it would be "stupid" (in Torvalds's words) to put private keys on a system like that. What you want, Torvalds said, is layers of security, and the kernel does a reasonable job there. "But there will always be problems", he said, so it is prudent to have multiple layers and realize that some of the layers will fail. He noted that there are three firewalls between his development machine and the internet, none of which accept incoming ssh.

Cox agreed, saying that in security, depth is important. You might have really good locks on the doors of your house, but you still keep your money in the bank, he said. His primary worry is other local users on the system, rather than remote attackers.

Challenges

Torvalds and Cox disagreed a bit about the biggest challenge facing the kernel in the coming years; Torvalds worries about complexity, while Cox is concerned about code quality. The kernel has always been complicated, Torvalds said, but it is not getting less complicated over time. There is a balancing act between the right level of complexity to do the job right and over-complicating things.

Cox believes that the complexity problem is self-correcting for the most part. Once a subsystem becomes too complicated, it eventually gets replaced with something simpler. But code quality is a problem our industry has faced for sixty years. It is not just an open source problem, but the whole industry struggles with it. When better solutions are found, it may well require language changes which would have an enormous impact on Linux.

Control groups

Poettering next asked about control groups: are they irretrievably broken or can they be fixed? Cox thought that the concept behind cgroups was good, but that some of the controller implementations are terrible. The question in his mind is whether or not the controllers can be fixed and still continue to manage the resource they are meant to. To Torvalds, controllers are a necessary evil because they provide a needed way to manage resources like memory, CPU, and networking across processes. But it is a "fundamentally hard problem to dole out resources when everyone wants everything", he said.

Cox agreed, noting that managing resources at a large scale will introduce inefficiencies, and that's the overhead that some are complaining about. Those who don't use the controllers hate the overhead, but those who need them don't care about the impact, Torvalds said. Gleixner agreed that some kind of resource management is needed.

According to Torvalds, part of the problem is that the more heavily used a feature is, the more its impact is noticed. At one point, people said that caches were bad because they impose some costs, but we got past that, he said. Cox pointed to SMP as another area where the changes needed added costs that some were not interested in paying. Torvalds noted that when Cox started working on SMP for Linux, he thought it was an interesting project but didn't have any personal interest as he couldn't even buy the machines that Cox was using to develop the feature.

Conclusion

It is always nice to get a glimpse inside of the thinking of Torvalds and some of his lieutenants, and these kernel developer panels give people just that opportunity. It is one of the few forums where folks outside of the kernel community can interact with Torvalds and the others in a more or less face-to-face way.

[ I would like to thank the Linux Foundation for supporting my travel to Prague. ]

Comments (11 posted)

An update on UEFI secure boot

The "secure boot" feature that will soon be appearing in PC firmware—partly due to a mandate from Microsoft—has garnered a number of different reactions. On one hand, you have the Free Software Foundation (FSF) calling for signatures to "stand up for your freedom to install free software". On the other hand, you have pundits accusing "Linux fanatics" of wanting to make Windows 8 less secure. The truth of the unified extensible firmware interface (UEFI) secure boot situation lies somewhere in between those two extremes, as might be guessed.

The problem started to come into focus earlier this year. The UEFI specification provides for an optional "secure boot" feature that makes use of a "platform key" (PK) and "key exchange keys" (KEKs) stored in a database in firmware. The corresponding private keys would be used to sign bootloaders, drivers, and operating systems loaded from disk, removable media, or over the network. The database could also contain blacklisted keys, in the event previously valid keys are revoked because they've fallen into malicious hands.

The idea is that only signed drivers and operating systems could be loaded. This could prove a useful feature, given that it could prevent malware from infecting the signed components — but it has also been seen as a threat against open source operating systems (like Linux) and could thwart attempts to jailbreak those systems.

When LWN covered the feature in June, there was concern that "a fair amount of pressure" would be applied by proprietary OS makers to enable this feature. That concern was realized when Microsoft announced that it intended to require secure boot to award the Windows 8 logo to OEMs. Since most OEMs are going to want to qualify for this, and the marketing funds that likely accompany the program, Microsoft requiring secure boot makes it more or less mandatory.

Problems

The first and most obvious problem with secure boot is that it has the potential to only allow Microsoft operating systems to boot. As Matthew Garrett wrote in September, "A system that ships with Microsoft's signing keys and no others will be unable to perform secure boot of any operating system other than Microsoft's. No other vendor has the same position of power over the hardware vendors."

But it's more complex than simply preventing non-Microsoft operating systems from booting on hardware with UEFI secure boot enabled. In all likelihood, users will be able to boot OSes of choice for the foreseeable future. But will they actually control their own hardware?

On October 19th, Garrett provided a follow-up to his earlier posts on secure boot, and said that the problem is whether the end user will have the ability to manage their own keys:

As Garrett explains, the workaround is to turn off secure boot, but that's not very palatable:

He suggested that, rather than requiring secure boot to be disabled, it would be better to work on how the feature could be properly supported for Linux installations.

Solutions

The solution? Garrett says that a proposal has been put forward to the UEFI Forum that would allow users to install their own keys from removable media. According to Garrett, this avoids problems that would have associated with booting untrusted binaries. Because it requires removable media, malware won't be able to trigger the key installation. Instead, the system with secure boot would simply refuse to boot and fall back to system recovery procedures.

It is certainly possible that malware could infect USB keys or other removable media, but part of allowing users control is accepting some risk.

Meanwhile at the UEFI Forum

Mark Doran, president of the UEFI Forum, says that he's "trying to work with some folks in the Linux community, the Linux Foundation board for example, to see how Linux and other operating systems can participate in using this facility." He says that there are "a couple of ways that we could do that," but didn't offer specifics. Doran also said that, despite Microsoft being out front pushing for adoption, it was "absolutely intended for it to be implemented and implementable with any operating system."

Doran was also emphatic that he thought it unlikely that "many" systems would ship without the ability to turn off secure boot.

The reason is pretty simple, says Doran. Too many vendors have to support legacy Windows. At least for the foreseeable future, none of the OEMs are likely to ship systems that can't boot older versions of Windows.

This echoes what Ed Bott reported when slinging mud at "Linux fanatics". Bott asked "a spokesperson for AMI" and the response was "AMI will advise OEMs to provide a default configuration that allows users to enable / disable secure boot, but it remains the choice of the OEM to do (or not do) so." So everyone reports that it's "unlikely" that systems will ship that don't have a disable feature — but that's not quite the same as guaranteeing that users will be able to control their systems. Being able to disable secure boot is better than nothing, but that feature, alone, does not enable Linux to make use of the secure boot capability.

History Repeating Itself?

The UEFI secure boot situation is not unlike the once-feared trusted platform module (TPM) or "trusted computing" features. When first introduced, the concept for TPM was that it could provide additional system security by verifying that only authorized code could run on a system. At first glance, this seemed like a good idea — except that it could be misused to secure a system against its owner.

Microsoft and a number of other companies argued strongly in favor of TPM, while digital rights advocates argued against it. The EFF's Seth Schoen wrote in 2003:

As Schoen noted then, the EFF and others lobbied for user override, but it was initially resisted by the Trusted Computing Group (TCG). This time around, Microsoft and the UEFI Forum are saying that it's in the hands of the vendors to decide whether to implement a bypass to UEFI secure boot, and trying to downplay concerns about the implementations and the effects on users of other platforms.

Ultimately, trusted computing did not turn out to be the nightmare that many feared. We can probably thank the EFF and other organizations that lobbied against TPM that we aren't seeing widespread misuse of TPM against users.

Don't panic... yet

The worst-case scenario — a flood of "restricted boot" systems hitting the market that are incapable of booting Linux or anything other than signed Windows 8 — does seem unlikely. But we're a long way from Garrett's proposal as well. Users interested in complete control of their systems will need to keep an eye on this process, and make sure that OEMs are aware that having the ability to disable secure boot is not enough. In order to truly control your system, you must have a way to install your own trusted keys as well.

Comments (23 posted)

GStreamer 1.0 and 0.10

GStreamer has come a long way in the ten-plus years it has been in existence. The push is on to finally put out a 1.0 release, possibly before the end of this year, while work has not stopped on the existing 0.10.x code base. In two keynotes from the second-ever GStreamer conference, which was held in Prague October 24-25, Wim Taymans outlined the future, while Tim-Philipp Müller looked at developments in 0.10.x.

History and background

Taymans started his talk with a bit of history and background. GStreamer is a library that makes it easy to create multimedia applications. It does so by providing a pipeline architecture with multiple components that can be plugged together in many different ways, which is what provides GStreamer with its "power and flexibility", he said.

Applications that use GStreamer have generally set the tone for the direction of the framework. GStreamer came into the GNOME project in 2002 with the Rhythmbox music player, and the audio capabilities worked well at that time. Video was a different story, but by 2004, the Totem video player ensured that the video in GStreamer worked, Taymans said.

Since GStreamer is a library, it can be integrated with many different applications, frameworks, toolkits, development environments, and so on. Whatever gets integrated with GStreamer gains "many new features". That has led to browsers that use GStreamer for playing content in <video> and <audio> tags. In addition, GStreamer is integrated with (and ported to) Android to bring its capabilities to that mobile platform. Integration is the "strong point of GStreamer", he said.

Taymans then went through all of the different types of applications that GStreamer is being used for, some where it shines and other areas where it still needs some work. For example, transcoding is in good shape; he pointed to Transmageddon as one example. Communication tools that provide voice and video calls, such as Empathy, are another area where GStreamer works well.

On the other hand, complicated audio production, like what Buzztard is doing, is an area that needs more work. Streaming and media distribution are areas for improvement as well. Various companies are using parts of GStreamer to do media distribution, but it is "not quite there yet", he said. Overall, GStreamer is "everywhere" in the multimedia space, but there is plenty more to be done.

Getting to 1.0

Taymans then turned to the future and wanted to "look at what needs to be done to get to 1.0". A 1.0 release has been discussed since 2007 or 2008, he said, but there has been a problem: 0.10 (and earlier) has been "doing things very well", so there was a great temptation to just keep extending that code.

But there are "new challenges", Taymans said. There are things that can be done with 0.10 but aren't easy or optimal, partly because they weren't envisioned back when the project started. Embedded processors aren't really a new challenge, but GStreamer could be better on today's embedded hardware, especially in the areas of power consumption and offloading work to other specialized processors (like DSPs) that are often available.

Video cards with graphical processing units (GPUs) are another resource that could be used better by the framework. Moving video decoding to GPUs when possible would help with power consumption as well. It can be done with 0.10, but "it's hard to do", Taymans said. Other things like video effects could be off-loaded to the GPU. Effects can be done on the main CPU, and are today, but it consumes more power and cannot handle higher resolution video.

Memory management is another area where improvements are needed. GPUs have their own memory, so the challenge is to avoid copying memory "from the CPU to GPU and back again and again". It is important for performance and GStreamer doesn't currently handle the situation well, Taymans said.

The formats of video in memory can't be specified flexibly enough to match what the hardware is producing or consuming. That means that GStreamer elements have to copy the data to massage it into the right format. Better memory management leads of better integration with the hardware and increased performance, all of which leads to better battery life, he said.

Dynamic pipelines, where the elements in a multimedia pipeline are removed or changed, are another feature that is possible with 0.10, but difficult to do correctly. Examples that Taymans gave include the Cheese webcam application, which can apply various effects to the video data and effects can be added or changed on the fly. Another example is the PulseAudio pass-through element, which may be decoding mp3 data in software when a Bluetooth headset is plugged in. The headset can do the decoding in hardware, so the pipeline should be changed to reflect that. There are many more examples of dynamic pipelines, he said, but making them work right has been difficult for application developers—something that should change in 1.0.

The 0.11 branch (which will eventually become 1.0) opened after last year's GStreamer conference (in Cambridge, UK in October 2010). By June it had "everything needed" to start porting elements (aka plugins) and applications to use the new features. A 0.11 release was made in August, followed by 0.11.1 in September.

There are "many many cleanups" in the 0.11 branch, Taymans said, "too many to list". Over the years, the API has accumulated lots of "crazy" things that needed changing. Methods were removed, signatures changed, parameters removed, and so on. In fact, Taymans said that they "implemented everything we said we would do at the last GStreamer conference—more actually". Part of the "more" was the memory management changes which were not fully thought out a year ago.

The core, base, and FFmpeg parts of GStreamer are 100% working and passing the unit tests in 0.11. The "good", "bad", and "ugly" sets of plugins are respectively 68%, 76%, and 11% ported to the new API. The video mixer is not yet ported as the team "focused first on [video] playback so we can start shipping something". Lots of other applications have been ported, but there is still plenty to do. Many of the changes, both for plugins and applications, can be done semi-automatically, Taymans said.

All of that puts the project on track for a 1.0 release later this year possibly, though that will all be worked out at the conference, he said. All of the elements will not be ported in that time frame, as doing one per day would finish the job "sometime in June". So there will need to be a prioritization of the plugins and applications that get ported before the release. "People who want their application to work are welcome to help porting", Taymans said.

The plan is for 0.10 and 1.0 to co-exist; both can be installed and applications will choose the right one for their needs. That will allow for a gradual transition to 1.0 over "half a year or maybe a year". Taymans would like to see 1.0 make it into the next releases of the major distributions along with some applications ported to 1.0, specifically mentioning Cheese and Totem.

Back on the 0.10 branch

On day two of the conference, Tim-Phillip Müller updated attendees on what has been going on in the stable branch and noted that "almost everything applies to the new branch" because it will all be merged over. By way of introduction, Müller said that his work for the project has been "managing releases — or not — trying anyway". He also likened GStreamer to Lego, noting that it allows applications to connect up all kinds of complicated "multimedia handling" pieces to provide the right experience for its users.

The 0.10 series has been API stable for five or six years, he said, but 0.11 changes the API to fix some longstanding problems. Both can be installed together with some applications using the old and some the new, which "works fine". There have been "some possibly interesting changes" that have gone on in the 0.10 series and since everyone can't follow bugzilla, mailing lists, and IRC, Müller's talk was meant to fill them in.

He presented some statistics on the development of 0.10 since last year's conference. There were 962 bugzilla bugs fixed and 2747 files changed. That resulted in 350K lines inserted and 100K deleted in a code base of 1.4 million lines of C and C++ code. Müller said that he was surprised to find that there had been 201 unique patch contributors in that time.

Since Taymans has been working on the new development branch, development in the core has slowed down, Müller said, but there are some significant new core library features. The gst_pad_push() function (which pushes out a buffer to the next stage in the pipeline) has been made lockless by using atomic operations. Previously, multiple locks were taken.

Progress and quality of service (QoS) messages have been added so that elements can post status information to applications. Progress messages could be the status of a data transfer or a mounting operation. QoS messages allow the internal QoS events (which indicate if a frame was dropped, rendered too late, or too early) to propagate out to applications, which will allow them to display statistics.

There have been new base classes added, for the obvious advantage of code reuse, he said. There is a need for APIs that work for all of the different use cases and are reasonably CPU-efficient, rather than having each plugin or element write its own. The reason that it makes sense to add new base classes so close to the end-of-life for 0.10 is that it will make porting to 1.0 easier because some of the new code can be used early, then you can "go to 1.0 for free", he said.

GstBaseParse is a new class to move the code that parses the "caps" (capabilities of elements), which has been repeated in many different places. The quality of the parsers has increased dramatically because of the change, but they have already seen some drawbacks in the new base class. That may result in a change to the class or adding a new GstBaseParse2.

GstVideoEncoder and GstVideoDecoder are new base classes to move some of the repeated code into one place. Right now, they live in the "bad" plugins, and need some API changes before moving into "good". Likewise, GstCollectPads2 (for notification of buffer availability) and GstAudioVisualizer are new base classes.

There are some base classes that may be added in the future including GstMuxer and GstDemuxer to encapsulate multiplexing (and de-multiplexing) functionality. Some "proper filter base classes" and a N-to-1 synchronization base class (for syncing various streams like video and a text overlay) are also possibly on the horizon.

There is also a "new and improved high-level API", Müller said. The playbin2 high-level playback API has some additional features including progressive download, new buffering methods for network streams, and support for non-raw streams. camerabin2 replaces camerabin as the camera input high-level API. The latter "wasn't bad, but was tricky to use". camerabin2 is a new design with new features including support for things like viewfinder streams, snapshotting, and other new hardware capabilities.

GstDiscoverer is a new "fast(ish)" metadata extraction API. It allows queueing files of interest and will give the stream topology, which allows better decisions regarding decoding, transcoding, and transmuxing. There are still some issues with it, he said, including getting an accurate duration for a stream without decoding it. There is also no way yet to tell an application whether the duration reported is an estimate and how good of an estimate it is.

Müller also mentioned the GStreamer Editing Services (g-e-s), which is a "very nice API" that is used by the PiTiVi video editor. The GStreamer RTSP server (which is really a library, he said) that allows you to "share your video camera or stream your files". There are Python and Vala bindings for the library and it comes with an example program that means that you can do those things "really easily".

The "middleware" layer (which is the "plumbing" of GStreamer) has also seen improvements, especially in the areas of parsing (using the new base class) and moving parsing out of decoders and into parsers that get plugged in automatically to handle metadata extraction. The PulseAudio pass-through was mentioned again as part of the middleware changes. In addition, Orc code has been added to produce "just in time" machine code for things like video scaling and conversion.

Replacing the FFmpeg color space code is currently a work in progress. It was forked from the FFmpeg code six years ago and GStreamer should probably apologize to the project because the fork gives FFmpeg "a bit of a bad name" because of how old it is, he said. Supporting the VA API and VDPAU video acceleration APIs is also under development and will eventually automatically plug in on systems that support video cards with those APIs. Lastly, the wrapper for OpenMAX (which is multimedia framework for embedded devices including Android) is being worked on. Müller said that no matter what GStreamer folks might think of OpenMAX, "it's there and people want to use it".

There were a number of other future features that he listed. Adding support for playlist handling, so that each application doesn't need to do its own, proper chapter and track handling, and device discovery and probing were all on that list. 3D video is something that can currently be done with 0.10, but it doesn't integrate well with hardware and other components and will likely be worked on for 1.0.

Müller wrapped up his talk with a vote of confidence in both the 0.10 and 0.11 branches. He thinks that people will fairly quickly be switching away from 0.10 once 1.0 is out, which is in keeping with what was seen at the last release which changed the API (from 0.08 to 0.10). He said that Taymans had "undersold 1.0 yesterday" in his talk, "it's gonna rock". All in all, it seems like that the GStreamer project is very healthy and making strides in lots of interesting directions.

[ I would like to thank the Linux Foundation for travel assistance to attend "conference week in Prague". ]

Comments (5 posted)

Page editor: Jonathan Corbet

Security

STEED: End-to-end email encryption

Is end-to-end email encryption a lost cause, or do the existing protocols simply fail to meet the ease-of-use requirements needed for widespread adoption? That is a question the security community has debated for years without arriving at a consensus. Two key developers from the GnuPG project have outlined a new approach to email encryption that they call STEED, for "Secure Transmission of Encrypted Electronic Data." It is backwards compatible with existing OpenPGP and S/MIME standards, but seeks to smooth out the stumbling blocks that "casual" and non-technical users typically find onerous.

STEED is outlined in an October 17 white paper [PDF] co-authored by Werner Koch, co-founder of FSF Europe and lead developer of GNU Privacy Guard (GnuPG aka GPG), and Marcus Brinkmann, longtime Debian and GNU/Hurd developer and Koch's partner at security consulting firm g10code. The paper outlines the overall security approach taken by OpenPGP and S/MIME — including the setup, configuration, and maintenance tasks expected of the user, as well as the trust model that each scheme employs. S/MIME's trust model is called PKIX, for "public-key infrastructure for X.509," and encompasses the existence and operation of trusted certificate authorities (CAs). In contrast, OpenPGP's trust model is the web-of-trust (WoT), which depends on a global, decentralized reputation system.

Koch and Brinkmann argue that email encryption has failed to take off both because of "usability pitfalls" in OpenPGP and S/MIME approaches, and because the PKIX and WoT trust models fail to match up with user expectations and the "mental model" users associate with email. Ultimately, STEED is a set of practices that, if implemented in mail user agents (MUAs), mail servers, and encryption utilities, would put secure-by-default choices into play with a minimum of user involvement. The resulting system amounts to a different trust model, which the authors feel would seem saner to end users. The paper merely outlines the pieces that make up STEED; the authors are seeking feedback and input, but plan to push the work forward with actual specifications in the future, and code targeting Thunderbird, Outlook, Kmail, and a webmail client.

Components

STEED is comprised of three functional techniques: automatic key generation, automatic key distribution via DNS, and opportunistic encryption.

With automatic key generation, the MUA would automatically create an OpenPGP key-pair or a self-signed S/MIME certificate whenever creating a new mail account. The name and email address associated with the key or certificate are taken directly from the account set-up data, and cryptographic parameters are pre-configured in the MUA using sane defaults. S/MIME, like TLS, is intended to work with CA-signed certificates, but self-signed certificates are cryptographically equivalent — the difference is that self-signed certificates do not follow the PKIX trust model, which STEED is intending to supplant anyway.

If there is anything more confusing to new users than public-private key-pairs, the authors suggest, then it is the "unpractical" methods used to store, distribute, and retrieve keys. Their solution is to replace dedicated keyservers with a DNS-based distribution method. DNS is decentralized, widely available, and the fact that email addresses include a domain name makes it fit naturally into the structures of existing DNS records. Obtaining an encryption key would be matter of querying the DNS server at the relevant person's email domain and asking for the record associated with that person's user name. An added bonus is that piggybacking on DNS automatically benefits from other security advances like DNSSEC. The white paper does not recommend a specific DNS resource record type, but does suggest that the DNS records store a key or certificate fingerprint and an associated URL linking to the full key.

Opportunistic encryption is, quite simply, encrypting messages by default. Encryption is intended to happen in the MUA, which will be left to implement its own policies regarding message headers, options presented to the user, and when to fall back to sending a message in the clear.

Trust

If everyone has a key, and other users' keys are easily retrievable via a DNS query performed transparently by the MUA, then email encryption and digital signatures would work smoothly. The remaining problem in the scheme is how to authenticate the key or email address of a remote person — particularly one that has not made contact in the past. After all, an attacker could intercept DNS queries and spoof an identity or perform a man-in-the-middle attack against a legitimate-looking contact.

The existing email encryption schemes tackle this problem with PKIX and WoT. But the authors cite a PGP usability study that indicates that these trust models are confusing to users:

Furthermore, they add, neither trust model matches up with users' natural expectations when using email, because both defer trust decisions to someone else. PKIX defers all trust judgments to an external authority, while WoT defers it to peer recommendations. In both cases, a binary trust determination is made before the communication is even read: "Neither system utilizes the users own experience with the peer in the context of the communication happening over time."

STEED's trust model is "trust upon first contact" (TUFC), which accepts the certificate or key of the remote party upon first contact, but persists and tracks it for the user. This is the trust model used by SSH, the authors note, and is what "virtually all users do anyway, when faced with the task to make a trust decision that interrupts their line of work." In other words, TUFC exists outside of an external "trust infrastructure," and leaves it up to the user to verify suspicious first contacts through other means (in person, phone calls, etc.). After the first contact, the system helps the user by flagging changed or revoked keys.

Challenges

Implementing STEED would require changes to the popular MUAs and encryption frameworks (including GnuPG), which the authors discuss, but there are also other challenges that cannot be solved by hard work alone. For example, automatic key generation is fine, except that users will either be interrupted to create (and then memorize) a passphrase, or else the passphrase must be stored somewhere. The paper suggests relegating passphrase storage to a "PIM service" — although the authors do not go into any significant detail. They appear to expect such a service to be network-accessible, and acknowledge that it must be able to communicate with MUAs (and other applications) from more than one platform. At one point, they refer to the service having a "passphrase vault," which implies encryption. Perhaps such a service is to protected by a user's existing account credentials, or else it would be imposing yet-another-password-to-remember, when one of STEED's goals is to spare users from choosing and remembering an encryption passphrase.

Since many email users today are content to store even sensitive correspondence with commercial SaaS companies, there might not be an elevated privacy concern with tossing a passphrase into the mix, but it is also just one more piece of information that could be stolen. On the other hand, STEED makes room for savvy users to create their own keys or use existing keys and security devices, so perhaps the extra-cautious could take that route. STEED's goal is to lower the bar for participation in secure email — at some point surely the benefits of preventing eavesdropping outweigh the risk of losing a passphrase.

A bigger issue is that STEED would require email providers to implement a key system in their DNS infrastructure, and a way for users to store and update their keys. That may not present them with a technical challenge, but convincing providers to change their policies and DNS infrastructure would be a huge undertaking. In addition to that, so many users today use webmail services like GMail that are available over SSL/TLS, which makes implementing a "new" security system an even harder sell.

There is also the issue of how to properly handle user mobility, particularly when users wish to access the same mail account from multiple devices. The authors do not offer easy solutions to any of these questions in the paper, but there is a sizable STEED discussion thread running on the GnuPG mailing list.

Jerome Baum suggested bypassing email providers' DNS and including a URL (privately-hosted or on a keyserver) for the sender's encryption key as a mail header instead. That idea has the benefit of being "in-band" and not requiring email provider support, though it still requires changes to the MUA. Peter Lebbing suggested using Bittorrent or another peer-to-peer system for hosting and retrieving keys.

Mark H. Wood said that attempting to bypass the passphrase-management problem is asking for trouble. "To be secure without being involved in the process is an unreasonable expectation which can never be met." Better, he said, to focus on generating passphrases that can be easily memorized, than to suggest storing them off-site.

Perhaps the thorniest problem was raised by Tom Ritter, who observed that supporting multiple devices also means supporting temporary usage of any random computer. "There's the very practical problem of _sometimes_ I do need to read my mail from a machine that is not my own. As a security person I almost never do it. But 'the young crowd' do it all the time. You have to satisfy that requirement also." Of course, most temporary email access of this type is done through a web mail interface, which has its own share of problems: either implementing the cryptographic functions in JavaScript or another client-side technology, or decrypting messages on the remote server. Either option carries risks. In addition, doing the encryption/decryption on the server breaks the "end-to-end encryption" principle.

Koch is scheduled to speak about STEED in November at the 2011 Free Society Conference and Nordic Summit (FSCONS); there may be changes to the proposal or further details on the protocols at that time. Until then, despite all of the debate, the consensus view on the GnuPG list is that STEED is at least taking on a very important problem: most people do not use email encryption because it is too difficult to configure and manage. Taking most of the decisions out of the user's hands and settling them in the MUA may not be perfect, but it would make for a vast improvement.

Even that, however, would not tackle the biggest obstacle to email encryption of all — the naive assumption by many users that their email is already secure because the lock icon is visible in the browser's location bar. At its heart, STEED is an attempt to make MUAs "do the right thing" more and rely less on the user, because the technical underpinnings of end-to-end encryption are already well-understood by the developer community. STEED is completely compatible with user-driven OpenPGP and S/MIME. But whether or not it could push the apathetic masses into end-to-end encryption remains to be seen. Perhaps nothing can, but making it simpler is certainly better than nothing.

Comments (15 posted)

Brief items

Security quotes of the week

-- Steven Bellovin

-- Ross Anderson

Comments (4 posted)

New vulnerabilities

asterisk: multiple vulnerabilities

Package(s):	asterisk	CVE #(s):	CVE-2011-2536 CVE-2011-2665 CVE-2011-2666 CVE-2011-4063
Created:	October 25, 2011	Updated:	June 13, 2012
Description:	From the CVE entries: chan_sip.c in the SIP channel driver in Asterisk Open Source 1.4.x before 1.4.41.2, 1.6.2.x before 1.6.2.18.2, and 1.8.x before 1.8.4.4, and Asterisk Business Edition C.3.x before C.3.7.3, disregards the "alwaysauthreject' option and generates different responses for invalid SIP requests depending on whether the user account exists, which allows remote attackers to enumerate account names via a series of requests. (CVE-2011-2536) reqresp_parser.c in the SIP channel driver in Asterisk Open Source 1.8.x before 1.8.4.3 allows remote attackers to cause a denial of service (NULL pointer dereference and daemon crash) via a SIP packet with a Contact header that lacks a < (less than) character. (CVE-2011-2665) The default configuration of the SIP channel driver in Asterisk Open Source 1.4.x through 1.4.41.2 and 1.6.2.x through 1.6.2.18.2 does not enable the "alwaysauthreject" option, which allows remote attackers to enumerate account names by making a series of invalid SIP requests and observing the differences in the responses for different usernames, a different vulnerability than CVE-2011-2536. (CVE-2011-2666) chan_sip.c in the SIP channel driver in Asterisk Open Source 1.8.x before 1.8.7.1 and 10.x before 10.0.0-rc1 does not properly initialize variables during request parsing, which allows remote authenticated users to cause a denial of service (daemon crash) via a malformed request. (CVE-2011-4063)
Alerts:	Fedora FEDORA-2011-14480 2011-10-18 Fedora FEDORA-2011-14538 2011-10-18 Gentoo 201110-21 2011-10-24 Debian DSA-2493-1 2012-06-12

Comments (none posted)

clamav: recursion level crash

Package(s):	clamav	CVE #(s):	CVE-2011-3627
Created:	October 24, 2011	Updated:	November 10, 2011
Description:	ClamAV 0.97.3 fixes a recursion level crash.
Alerts:	Ubuntu USN-1258-1 2011-11-10 Fedora FEDORA-2011-15119 2011-10-29 Fedora FEDORA-2011-15076 2011-10-29 Gentoo 201110-20 2011-10-23 openSUSE openSUSE-SU-2011:1177-1 2011-10-24

Comments (none posted)

cyrus-imapd: denial of service

Package(s):	cyrus-imapd	CVE #(s):	CVE-2011-3481
Created:	October 24, 2011	Updated:	March 23, 2012
Description:	From the CVE entry: The index_get_ids function in index.c in imapd in Cyrus IMAP Server before 2.4.11, when server-side threading is enabled, allows remote attackers to cause a denial of service (NULL pointer dereference and daemon crash) via a crafted References header in an e-mail message.
Alerts:	Scientific Linux SL-cyru-20111201 2011-12-01 Oracle ELSA-2011-1508 2011-12-01 Oracle ELSA-2011-1508 2011-12-01 Oracle ELSA-2011-1508 2011-12-01 CentOS CESA-2011:1508 2011-12-01 CentOS CESA-2011:1508 2011-12-01 Red Hat RHSA-2011:1508-01 2011-12-01 openSUSE openSUSE-SU-2011:1170-1 2011-10-24 Gentoo 201110-16 2011-10-22 Debian DSA-2377-1 2012-01-01 Mandriva MDVSA-2012:037 2012-03-23

Comments (none posted)

dbus: symlink attack

Package(s):	dbus	CVE #(s):	CVE-2011-2533
Created:	October 24, 2011	Updated:	October 26, 2011
Description:	From the CVE entry: The configure script in D-Bus (aka DBus) 1.2.x before 1.2.28 allows local users to overwrite arbitrary files via a symlink attack on an unspecified file in /tmp/.
Alerts:	Gentoo 201110-14 2011-10-21

Comments (none posted)

etherape: denial of service

Package(s):	etherape	CVE #(s):	CVE-2011-3369
Created:	October 24, 2011	Updated:	August 20, 2012
Description:	From the CVE entry: The add_conversation function in conversations.c in EtherApe before 0.9.12 allows remote attackers to cause a denial of service (NULL pointer dereference and application crash) via an RPC packet, related to the get_rpc function in decode_proto.c.
Alerts:	openSUSE openSUSE-SU-2011:1176-1 2011-10-24 Mageia MGASA-2012-0224 2012-08-18

Comments (none posted)

fail2ban: insecure temp files

Package(s):	fail2ban	CVE #(s):	CVE-2009-5023
Created:	October 24, 2011	Updated:	October 26, 2011
Description:	fail2ban IP used insecure temporary files when unbanning an IP address
Alerts:	openSUSE openSUSE-SU-2011:1175-1 2011-10-24

Comments (none posted)

freetype2: code execution

Package(s):	freetype2	CVE #(s):	CVE-2011-3256
Created:	October 21, 2011	Updated:	December 28, 2012
Description:	From the Mandriva advisory: FreeType allows remote attackers to execute arbitrary code or cause a denial of service (memory corruption) via a crafted font (CVE-2011-3256).
Alerts:	SUSE SUSE-SU-2011:1307-1 2011-12-08 SUSE SUSE-SU-2011:1306-1 2011-12-08 Fedora FEDORA-2011-15956 2011-11-15 Ubuntu USN-1267-1 2011-11-18 Fedora FEDORA-2011-14749 2011-10-22 CentOS CESA-2011:1402 2011-11-09 Scientific Linux SL-free-20111025 2011-10-25 CentOS CESA-2011:1402 2011-10-26 Red Hat RHSA-2011:1402-01 2011-10-25 Debian DSA-2328-1 2011-10-24 Mandriva MDVSA-2011:157 2011-10-21 openSUSE openSUSE-SU-2012:0015-1 2012-01-05 Gentoo 201201-09 2012-01-23 Red Hat RHSA-2012:0094-01 2012-02-02 Oracle ELSA-2012-0467 2012-04-12 SUSE SUSE-SU-2012:0553-1 2012-04-23 Mageia MGASA-2012-0369 2012-12-27

Comments (none posted)

java-1.6.0-sun: multiple unspecified vulnerabilities

Package(s):	java-1.6.0-sun	CVE #(s):	CVE-2011-3389 CVE-2011-3516 CVE-2011-3545 CVE-2011-3546 CVE-2011-3549 CVE-2011-3550 CVE-2011-3555 CVE-2011-3561
Created:	October 20, 2011	Updated:	February 6, 2013
Description:	From the Red Hat advisory: 737506 - CVE-2011-3389 HTTPS: block-wise chosen-plaintext attack against SSL/TLS (BEAST) 747203 - CVE-2011-3516 Oracle/IBM JDK: unspecified vulnerability fixed in 6u29 (Deployment) 747191 - CVE-2011-3545 Oracle/IBM JDK: unspecified vulnerability fixed in 6u29 (Sound) 747205 - CVE-2011-3546 Oracle/IBM JDK: unspecified vulnerability fixed in 6u29 (Deployment) 747198 - CVE-2011-3549 Oracle/IBM JDK: unspecified vulnerability fixed in 6u29 (Swing) 747200 - CVE-2011-3550 Oracle/IBM JDK: unspecified vulnerability fixed in 6u29 (AWT) 747206 - CVE-2011-3555 Oracle/IBM JDK: unspecified vulnerability fixed in 6u29 (JRE) 747208 - CVE-2011-3561 Oracle/IBM JDK: unspecified vulnerability fixed in 6u29 (Deployment)
Alerts:	Fedora FEDORA-2011-17400 2011-12-23 Fedora FEDORA-2011-17400 2011-12-23 Fedora FEDORA-2011-17400 2011-12-23 Fedora FEDORA-2011-17400 2011-12-23 Fedora FEDORA-2011-17400 2011-12-23 Debian DSA-2358-1 2011-12-05 SUSE SUSE-SU-2011:1298-1 2011-12-02 Debian DSA-2356-1 2011-12-01 Red Hat RHSA-2011:1478-01 2011-11-24 Ubuntu USN-1263-1 2011-11-16 Fedora FEDORA-2011-15555 2011-11-07 Mandriva MDVSA-2011:170 2011-11-11 Scientific Linux SL-java-20111019 2011-10-19 Gentoo 201111-02 2011-11-05 openSUSE openSUSE-SU-2011:1196-1 2011-10-28 CentOS CESA-2011:1380 2011-10-19 Red Hat RHSA-2011:1384-01 2011-10-19 Red Hat RHSA-2012:0006-01 2012-01-09 Red Hat RHSA-2012:0034-01 2012-01-18 SUSE SUSE-SU-2012:0114-1 2012-01-23 SUSE SUSE-SU-2012:0122-1 2012-01-26 Fedora FEDORA-2012-1690 2012-02-15 SUSE SUSE-SU-2012:0122-2 2012-02-23 SUSE SUSE-SU-2012:0114-2 2012-03-06 Red Hat RHSA-2012:0508-01 2012-04-23 SUSE SUSE-SU-2012:0602-1 2012-05-09 Fedora FEDORA-2012-16351 2012-10-18 Gentoo 201301-01 2013-01-07 Fedora FEDORA-2013-1898 2013-02-05

Comments (none posted)

kernel: denial of service

Package(s):	linux-lts-backport-maverick	CVE #(s):	CVE-2011-3363
Created:	October 25, 2011	Updated:	November 28, 2011
Description:	From the Ubuntu advisory: Yogesh Sharma discovered that CIFS did not correctly handle UNCs that had no prefixpaths. A local attacker with access to a CIFS partition could exploit this to crash the system, leading to a denial of service.
Alerts:	Scientific Linux SL-kern-20111129 2011-11-29 CentOS CESA-2011:1479 2011-11-30 Oracle ELSA-2011-1479 2011-11-30 Red Hat RHSA-2011:1479-01 2011-11-29 Oracle ELSA-2011-1465 2011-11-28 Ubuntu USN-1281-1 2011-11-24 Scientific Linux SL-kern-20111122 2011-11-22 Red Hat RHSA-2011:1465-01 2011-11-22 Ubuntu USN-1256-1 2011-11-09 openSUSE openSUSE-SU-2011:1222-1 2011-11-08 openSUSE openSUSE-SU-2011:1221-1 2011-11-08 SUSE SUSE-SA:2011:042 2011-10-28 SUSE SUSE-SU-2011:1195-1 2011-10-28 Ubuntu USN-1244-1 2011-10-25 Ubuntu USN-1243-1 2011-10-25 Ubuntu USN-1242-1 2011-10-25 Red Hat RHSA-2012:0010-01 2012-01-10 Oracle ELSA-2012-0150 2012-03-07

Comments (none posted)

kernel: multiple vulnerabilities

Package(s):	kernel	CVE #(s):	CVE-2009-4067 CVE-2011-2942 CVE-2011-3209 CVE-2011-3347
Created:	October 20, 2011	Updated:	March 29, 2012
Description:	From the Red Hat advisory: A flaw in the auerswald USB driver could allow a local, unprivileged user to cause a denial of service or escalate their privileges by inserting a specially-crafted USB device. (CVE-2009-4067, Low) RHSA-2011:1065 introduced a regression in the Ethernet bridge implementation. If a system had an interface in a bridge, and an attacker on the local network could send packets to that interface, they could cause a denial of service on that system. Xen hypervisor and KVM (Kernel-based Virtual Machine) hosts often deploy bridge interfaces. (CVE-2011-2942, Moderate) A flaw in the kernel's clock implementation could allow a local, unprivileged user to cause a denial of service. (CVE-2011-3209, Moderate) Non-member VLAN (virtual LAN) packet handling for interfaces in promiscuous mode and also using the be2net driver could allow an attacker on the local network to cause a denial of service. (CVE-2011-3347, Moderate)
Alerts:	Scientific Linux SL-Kern-20111206 2011-12-06 Oracle ELSA-2011-2037 2011-12-15 Ubuntu USN-1294-1 2011-12-08 Red Hat RHSA-2011:1530-03 2011-12-06 Fedora FEDORA-2011-15856 2011-11-13 Ubuntu USN-1256-1 2011-11-09 Ubuntu USN-1268-1 2011-11-21 Red Hat RHSA-2011:1419-01 2011-11-01 Red Hat RHSA-2011:1418-01 2011-11-01 SUSE SUSE-SA:2011:042 2011-10-28 SUSE SUSE-SU-2011:1195-1 2011-10-28 Scientific Linux SL-kern-20111020 2011-10-20 Ubuntu USN-1236-1 2011-10-20 CentOS CESA-2011:1386 2011-10-21 Red Hat RHSA-2011:1386-01 2011-10-20 Red Hat RHSA-2012:0116-01 2012-02-15 Oracle ELSA-2012-0150 2012-03-07 Ubuntu USN-1409-1 2012-03-27 Ubuntu USN-1405-1 2012-03-27 Ubuntu USN-1404-1 2012-03-27 Ubuntu USN-1412-1 2012-03-29 openSUSE openSUSE-SU-2012:0781-1 2012-06-22 openSUSE openSUSE-SU-2012:0812-1 2012-07-03

Comments (none posted)

kfreebsd-8: code execution

Package(s):	kfreebsd-8	CVE #(s):	CVE-2011-4062
Created:	October 24, 2011	Updated:	October 26, 2011
Description:	From the Debian advisory: Buffer overflow in the "linux emulation" support in the FreeBSD kernel allows local users to cause a denial of service (panic) and possibly execute arbitrary code by calling the bind system call with a long path for a UNIX-domain socket, which is not properly handled when the address is used by other unspecified system calls.
Alerts:	Debian DSA-2325-1 2011-10-23

Comments (none posted)

nova: information leak

Package(s):	nova	CVE #(s):
Created:	October 26, 2011	Updated:	October 26, 2011
Description:	From the Ubuntu advisory: An information leak was discovered in Nova. An attacker with access to a valid EC2_ACCESS_KEY could obtain the corresponding EC2_SECRET_KEY for that user.
Alerts:	Ubuntu USN-1247-1 2011-10-25

Comments (none posted)

open-iscsi: code execution

Package(s):	open-iscsi	CVE #(s):	CVE-2009-1297
Created:	October 21, 2011	Updated:	April 10, 2013
Description:	From the Ubuntu advisory: Colin Watson discovered that iscsi_discovery in Open-iSCSI did not safely create temporary files. A local attacker could exploit this to to overwrite arbitrary files with root privileges.
Alerts:	Ubuntu USN-1235-1 2011-10-20 Mageia MGASA-2012-0241 2012-08-26 Mandriva MDVSA-2013:109 2013-04-10

Comments (none posted)

opera: code injection

Package(s):	opera	CVE #(s):
Created:	October 24, 2011	Updated:	October 26, 2011
Description:	From this article at The H. Security expert José A. Vázquez has released details of a critical security hole in the Opera browser that can be exploited to inject malicious code.
Alerts:	openSUSE openSUSE-SU-2011:1172-1 2011-10-24

Comments (none posted)

pam: arbitrary code execution

Package(s):	pam	CVE #(s):	CVE-2011-3148 CVE-2011-3149
Created:	October 24, 2011	Updated:	March 11, 2013
Description:	From the Debian advisory: Kees Cook of the ChromeOS security team discovered a buffer overflow in pam_env, a PAM module to set environment variables through the PAM stack, which allowed the execution of arbitrary code. An additional issue in argument parsing allows denial of service.
Alerts:	Fedora FEDORA-2011-16365 2011-11-25 Fedora FEDORA-2011-16390 2011-12-04 SUSE SUSE-SU-2011:1218-1 2011-11-04 SUSE SUSE-SU-2011:1207-1 2011-11-03 SUSE SUSE-SU-2011:1205-1 2011-11-03 SUSE SUSE-SU-2011:1209-1 2011-11-03 openSUSE openSUSE-SU-2011:1204-1 2011-11-03 openSUSE openSUSE-SU-2011:1208-1 2011-11-03 Ubuntu USN-1237-1 2011-10-24 Debian DSA-2326-1 2011-10-24 Gentoo 201206-31 2012-06-25 Red Hat RHSA-2013:0521-02 2013-02-21 Oracle ELSA-2013-0521 2013-02-25 Scientific Linux SL-pam-20130228 2013-02-28 CentOS CESA-2013:0521 2013-03-09

Comments (none posted)

phpMyAdmin: multiple vulnerabilities

Package(s):	phpmyadmin	CVE #(s):	CVE-2011-3646 CVE-2011-4064
Created:	October 21, 2011	Updated:	November 14, 2011
Description:	From the Mandriva advisory: When the js_frame parameter of phpmyadmin.css.php is defined as an array, an error message shows the full path of this file, leading to possible further attacks (CVE-2011-3646). Crafted values entered in the setup interface can produce XSS; also, if the config directory exists and is writeable, the XSS payload can be saved to this directory (CVE-2011-4064).
Alerts:	Fedora FEDORA-2011-15460 2011-11-05 Fedora FEDORA-2011-15472 2011-11-05 Fedora FEDORA-2011-15469 2011-11-05 Mandriva MDVSA-2011:158 2011-10-21 Gentoo 201201-01 2012-01-04

Comments (none posted)

puppet: man-in-the-middle attack

Package(s):	puppet	CVE #(s):	CVE-2011-3872
Created:	October 25, 2011	Updated:	November 23, 2011
Description:	From the Ubuntu advisory: It was discovered that Puppet incorrectly handled the non-default "certdnsnames" option when generating certificates. If this setting was added to puppet.conf, the puppet master's DNS alt names were added to the X.509 Subject Alternative Name field of all certificates, not just the puppet master's certificate. An attacker that has an incorrect agent certificate in his possession can use it to impersonate the puppet master in a man-in-the-middle attack.
Alerts:	openSUSE openSUSE-SU-2011:1288-1 2011-11-29 Debian DSA-2352-1 2011-11-22 Fedora FEDORA-2011-15000 2011-10-27 Fedora FEDORA-2011-14994 2011-10-27 Fedora FEDORA-2011-14880 2011-10-25 Ubuntu USN-1238-2 2011-10-25 Ubuntu USN-1238-1 2011-10-24 Gentoo 201203-03 2012-03-05

Comments (none posted)

xen: denial of service

Package(s):	xen	CVE #(s):	CVE-2011-3346
Created:	October 24, 2011	Updated:	October 26, 2011
Description:	A buffer overflow flaw was found in the Xen hypervisor SCSI subsystem emulation. An unprivileged, local guest user could provide a large number of bytes that are used to zero out a fixed-sized buffer via a SAI READ CAPACITY SCSI command, overwriting memory and causing the guest to crash.
Alerts:	Scientific Linux SL-xen-20111024 2011-10-24 Red Hat RHSA-2011:1401-01 2011-10-24 CentOS CESA-2011:1401 2011-10-24

Comments (none posted)

XTier: code execution

Package(s):	Novell XTier framework	CVE #(s):	CVE-2011-1710
Created:	October 26, 2011	Updated:	October 26, 2011
Description:	From the SUSE advisory: Several user supplied header length variables for the HTTP server component in the Novell XTier framework were not size limited, allowing integer overflow attacks to crash the service or potentially execute code.
Alerts:	SUSE SUSE-SU-2011:1185-1 2011-10-26

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

The 3.1 kernel is out, released by Linus on October 24. Headline features in this somewhat delayed release include improved Xen memory management, enhancements to process tracing (the PTRACE_SEIZE command), enhancements to lseek() to ease the finding of holes in files, OpenRISC architecture support, and more.

As of this writing, some 4400 patches have been pulled into the mainline for the 3.2 release. Trees pulled thus far include networking, USB, staging, and security; a full merge window summary will appear in next week's edition.

Stable updates: 3.0.8 was released on October 25 with the usual pile of important fixes.

Comments (none posted)

Quotes of the week

-- Linus Torvalds

-- Neil Brown

Comments (2 posted)

Kernel development news

The 2011 realtime minisummit

The realtime Linux minisummit was held on October 22 in Prague on the third day of the 13th Realtime Linux Workshop. This year's gathering featured a relatively unstructured discussion that does not lend itself to a tidy writeup. What has been attempted here is a report on the most important themes, supplemented with material from some of the workshop talks.

Per-CPU data

One of the biggest challenges for the realtime preemption patch set from the beginning has been the handling of per-CPU data. Throughput-oriented developers are quite fond of per-CPU variables; they facilitate lockless data access and improve the scalability of the code. But working with per-CPU data usually requires disabling preemption, which runs directly counter to what the realtime preemption patch set has been trying to do. It is a classic example of how high throughput and low latency can sometimes be incompatible goals.

The way the realtime patch set has traditionally dealt with per-CPU data is to put a lock around it and treat it like any other shared resource. The get_cpu_var() API has supported that mode fairly well. In more recent times, though, that API has been pushed aside by a new set of functions with names like this_cpu_read() and this_cpu_write(). The value of these functions is speed, especially on the x86 architecture where most of the operations boil down to a single machine instruction. But this new API has also caused some headaches, especially (but not exclusively) in the realtime camp.

The first problem is that there is no locking mechanism built into this API; it does not even disable preemption. So a value returned from this_cpu_read() may not correspond to the CPU that the calling thread is actually running on by the time it does something with the value it reads. Without some sort of external preemption disable, a sequence of this_cpu_read() followed by this_cpu_write() can write a value back on a different CPU than where the value was read, corrupting the resulting data structure; this kind of problem has already been experienced a few times.

The bigger problem, though, is that the API does not in any way indicate where the critical section involving the per-CPU data lies in the code. So there is no way to put in extra protection for realtime operation, and there is no way to put in any sort of debugging infrastructure to verify that the API is being used correctly. There have already been some heated discussions on the mailing list about these functions, but the simple fact is that they make things faster, so they are hard to argue against. There is real value in the this_cpu API; Ingo Molnar noted that the API is not a problem as such - it is just an incomplete solution.

Incomplete or not, usage of this API is growing by hundreds of call sites in each kernel release, so something needs to be done. A lot of time was spent talking about the naming of these functions, which is seen as confusing. What does "this CPU" mean in a situation where the calling thread can be migrated at any time? There may be an attempt to change the names to discourage improper use of the API, but that is clearly not a long-term solution.

The first step may be to figure out how to add some instrumentation to the interface that is capable of finding buggy usage. If nothing else, real evidence of bugs will help provide a justification for any changes that need to be made. There was also talk of creating some sort of "local lock" that can be used to delineate per-CPU critical sections, disable migration (and possibly preemption) in those sections, and, on realtime, to simply take a lock. Along with hopefully meeting everybody's goals, such an interface would allow the proper user of per-CPU data to be verified with the locking checker.

Meanwhile, the realtime developers have recently come up with a new approach to the management of per-CPU data in general. Rather than disabling preemption while working with such data, they simply disable migration between processors instead. If (and this is a bit of a big "if") the per-CPU data is properly protected by locking, preventing migration should be sufficient to keep processes from stepping on each other's toes while preserving the ability for high-priority processes to preempt lower-priority processes when the need arises.

The fear that comes with disabling migration is that processes could get stuck and experience long latencies even though there are free CPUs available. If a low-priority process disables migration, then is preempted, it cannot be moved to an available processor. So that process will simply wait until the preempting process either completes or gets migrated itself. Some work has been done to try to characterize the problem; Paul McKenney talked for a bit about some modeling work he has done in that area. At this point, it's really not clear how big the problem really is, but some developers are clearly worried about how this technique may affect some workloads.

Software interrupts

For a long time, the realtime preemption patch set has split software interrupt ("softirq") handling into a separate thread, making it preemptable like everything else. More recent patches have done away with that split, though, for a couple of reasons: to keep the patches closer to the mainline, and to avoid delaying network processing. The networking layer depends heavily on softirqs to keep the data moving; shifting that processing to a separate thread can add latency in an unwanted place. But there had been a good reason for the initial split: software interrupts can introduce latencies of their own. So there was some talk of bringing the split back in a partial way. Most softirqs could run in a separate thread but, perhaps, the networking softirqs would continue to run as "true" software interrupts. It is not a solution that pleases anybody, but it would make things better in the short term.

There appears to be agreement on the form of the long-term solution: software interrupts simply need to go away altogether. Most of the time, softirqs can be replaced by threaded interrupt handlers; as Thomas Gleixner put it, the networking layer's NAPI driver API is really just a poor-man's form of threaded IRQ handling. There is talk of moving to threaded handlers in the storage subsystem as well; that evidently holds out some hope of simplifying the complex SCSI error recovery code. That is not a trivial change in either storage or networking, though; it will take a long time to bring about. Meanwhile, partially-split softirq handling will probably return to the realtime patches.

An interesting problem related to threaded softirq handlers is the prevalence of "trylock" operations in the kernel. There are a number of sites where the code will attempt to obtain a lock; if the attempt fails, the IRQ handler will stop with the idea that it will try again on the next softirq run. If the handler is running as a thread, though, chances are good that it will be the highest-priority thread, so it will run again immediately. A softirq handler looping in this manner might prevent the scheduling of the thread that actually holds the contended lock, leading to a livelock situation. This problem has been observed a few times on real systems; fixing it may require a restructuring of the affected code.

Upstreaming

There was a long discussion on how the realtime developers planned to get most or all of the patch set upstream. It has been out of tree for some time now - the original preempt-rt patch was announced almost exactly seven years ago. Vast amounts of code have moved from the realtime tree into the mainline over that time, but there are still some 300 patches that remain out of the mainline. Thomas suggested a couple of times that he has maintained those patches for about long enough, and does not want or plan to do it forever.

Much of what is in the realtime tree seems to be upstreamable, though possibly requiring some cleanup work first. There are currently about 75 patches queued to go in during the 3.2 merge window, and there is a lot more that could be made ready for 3.3 or 3.4. The current goal is to merge as much code as possible in the next few development cycles; that should serve to reduce the size of the realtime patch set considerably, making the remainder easier to maintain.

There are a few problem areas. Sampling of randomness is disabled in the realtime tree; otherwise contention for the entropy pool causes unwanted latencies. It seems that contribution of entropy is on the decline throughout the kernel; as processors add hardware entropy generators, it's not clear that maintaining the software pool has value. The seqlock API has a lot of functions with no users in the kernel; the realtime patch set could be simplified if those functions were simply removed from the mainline. There are still a lot of places where interrupts are disabled, probably unnecessarily; fixing those will require auditing the code to understand why interrupts were disabled in the first place. Relayfs has problems of its own, but there is only one in-tree user; for now, it will probably just be disabled for realtime. The stop_machine() interface is seen to be a pain, and is probably also unnecessary most of the time.

And so on. A number of participants went away from this session with a list of patches to fix up and send upstream; with luck, the out-of-tree realtime patch set will shrink considerably.

The future

Once the current realtime patch set has been merged, is the job done? In the past, shrinking the patch set has been made harder by the simple fact that the developers continue to come up with new stuff. At the moment, though, it seems that there is not a vast amount of new work on the horizon. There are only a couple of significant additions expected in the near future:

• Deadline scheduling. That patch is in a condition that is "not too bad," but the developer has gotten distracted with new things. It looks like the deadline scheduler may be picked up by a new developer and progress will resume.

• CPU isolation - the ability to run one or more processors with no clock tick or other overhead. Fredéric Weisbecker continues to work on those patches, but there is a lot to to be done and no pressing need to merge them right away. So this is seen as a long-term project.

Once upon a time, the addition of deterministic response to a general-purpose operating system was seen as an impossible task: the resulting kernel would not work well for anybody. While some people were saying that, though, the realtime preemption developers simply got to work; they have now mostly achieved that goal. The realtime work may never be "done" - just like the Linux kernel as a whole is never done - but this project looks like one that has achieved the bulk of its objectives. Your editor has learned not to make predictions about when everything will be merged; suffice to say that the developers themselves appear to be determined to get the job done before too long. So, while there may always be an out-of-tree realtime patch set, chances are it will shrink considerably in the coming development cycles.

Comments (8 posted)

The 2011 Kernel Summit

The 2011 Kernel Summit was held in Prague on October 23-25. The organization of the event was changed somewhat this year; the first day was dedicated to a small number of minisummits. We do not currently have coverage from those events - that is a gap we hope to fill in the near future.

Day 1

The Kernel Summit proper started on October 24 with the traditional closed session. The topics discussed there were:

• Kernel.org report; including some more information about the compromise along with what is being done to secure this critical infrastructure going forward.

• Tracing for large data centers looked at Google's needs in the tracing area, but then the discussion moved into whether moving the tools into the kernel tree will help alleviate some of the problems with ABIs and backward compatibility.

• Structured error logging; a presentation (only partly delivered) on improving kernel error logging.

• Coming to love cgroups; while love for control groups may not have been the outcome, some more understanding of cgroups, controllers, and the problems with the latter did.

• Memory management issues; a session on the progress the subsystem has made over the last year along with a list of patches that are in need of review and rework with an eye to eventually merging them.

• Preemption disable and verifiable APIs discussed calls to the this_cpu*() family as well as preempt_disable() calls multiplying throughout the kernel, which cause problems, and not just for the realtime kernel.

• Scheduler testing; a reworked version of the LinSched scheduling simulator was discussed and proposed to be added to the kernel tools/ directory. It may provide the long-sought ability to more reliably test scheduler changes.

• Patch review; in a wide-ranging discussion, various problems in the area of patch review were covered. The discussion eventually turned to Android's kernel patches and, in particular, suspend blockers with a, perhaps, surprising conclusion.

• Development process issues; there were fewer problems in this area than there have been in recent years. Linus is happy with how things are going, overall, though the growth of complexity in the kernel is somewhat worrisome.

Day 2

The second day of the 2011 Kernel Summit featured a changed format: the session was open to all attendees. Coverage for this day has been split into two parts:

• Morning: reports from a large number of minisummits, more on kernel.org security and the web of trust, and regression tracking.

• Afternoon: shared libraries, failure handling, the media controller, the kernel build and configuration subsystem, and the future of the event itself.

Needless to say, coverage would not be complete without a picture of the expanded group arranged into a rather compressed space:

(Your editor would like to thank the Linux Foundation for supporting his travel to Prague for this event).

Comments (5 posted)

Patches and updates

Kernel trees

• Linus Torvalds: Linux 3.1 . (October 24, 2011)

• Greg KH: Linux 3.0.8 . (October 25, 2011)

Core kernel code

• Rafael J. Wysocki: PM: Generic PM domains and device PM QoS (v2) . (October 25, 2011)

• Ohad Ben-Cohen: Introducing a generic AMP framework . (October 26, 2011)

Development tools

• Hui Zhu: KGTP (Linux Kernel debugger and tracer) 20111016 release (a gui tool available) . (October 25, 2011)

Device drivers

• Javier Martinez Canillas: Input: Cypress TTSP device driver . (October 25, 2011)

• Sangbeom Kim: mfd: S5M core driver initial release . (October 25, 2011)

• Ricardo Ribalda Delgado: Add spi support for CMA3000 driver and new driver CMR3000 . (October 25, 2011)

• Jonathan Cameron: IIO in kernel interfaces . (October 25, 2011)

• Konrad Rzeszutek Wilk: TTM DMA pool v2.1 . (October 25, 2011)

Filesystems and block I/O

• Aneesh Kumar K.V: New ACL format for better NFSv4 acl interoperability . (October 25, 2011)

• Tejun Heo: rescue cfq from RCU death sprial :) . (October 26, 2011)

• Tao Ma: ext4: Add inline data support. . (October 26, 2011)

Architecture-specific

• Daniel J Blueman: Add Numachip APIC support . (October 26, 2011)

Miscellaneous

• NeilBrown: lsusd - The Linux SUSpend Daemon . (October 25, 2011)

• Greg KH: usbutils 005 release . (October 25, 2011)

Page editor: Jonathan Corbet

Distributions

Creating a Fedora ARM distribution part 2: Bootstrapping

[Editor's note: this is the second of a two-part series on the creation of a Fedora distribution for the ARM architecture, contributed by Red Hat developer Jon Masters. Part 1 covers the history leading up to the current effort; this part will look at how the Fedora ARM distribution was bootstrapped and where things will go from here.]

Like most distributions, Fedora uses binary software packages (RPMs in this case) to manage installed software. These packages are built using complex sets of build dependencies (other software packages), some of which are not explicit dependencies but rather implied through their fixed presence in the standard "buildroots" (chroot environments containing a basic set of packages) used in the Fedora build infrastructure. All of these build dependencies have to initially come from somewhere. Ordinarily, the build system takes care of managing this, picking up dependencies as required, but in the case of a complete bootstrap there are no pre-built dependencies, so they must be created from scratch. Not only that, but build dependencies must be built using a minimal "bootstrap" configuration that avoids circular source dependencies on packages not built yet, and also provides enough basic functionality to rebuild the full set of dependencies using standard tools.

Currently, there is no convenient way to automatically bootstrap Fedora from scratch. Individual packages lack the level of data that would be required, such as a special set of configuration options sufficient to build a minimal "bootstrap" version of the core packages (free of circular dependencies) that could then be used to rebuild the normal configuration. Fedora is not alone in missing support for automated bootstrap, which is currently being investigated by a number of distributions. One of the problems has been that a full bootstrap remains such a rare event that it is difficult to justify the level of effort required to maintain support for automatically bootstrapping in the future (as opposed to making a large number of package changes for one bootstrap effort). Fedora may well eventually support automated bootstrap (a topic to be fully explored). In the meantime, bootstrapping for ARMv7 involved many successive stages.

The first involved building a limited cross-compiling Fedora 15 gcc toolchain for use on an x86 host and targeting the "hard float" ARMv7 ABI. This toolchain was then be used to build a native ARMv7 "hard float" toolchain and associated minimal dependencies necessary to run that toolchain within a chroot environment on an ARM system. The output of stage 1 was a minimal root filesystem, sufficient for use as a chroot (although not bootable), along with a set of bootstrap scripts that can be used in the future should it ever become necessary to repeat this exercise. DJ Delorie wrote the majority of the stage 1 bootstrap scripts, which are available in Fedora's ARM git repository at git://fedorapeople.org/~djdelorie/bootstrap.git. These bootstrap scripts have since been used by other distributions and developers to experiment with ideas of their own around bootstrap, including several comparative studies of the different toolchain options that might be used.

Stage 2 of the bootstrap process used the minimal chroot environment from stage 1 (which contained a native "hard float" gcc toolchain) to build up a small set of packages sufficient to run a native "hard float" version of rpmbuild, a tool used to build RPM packages from source. The goal was not to get a version of rpmbuild sufficient to build every package in the distribution, but rather sufficient to rebuild the packages from this and the previous stage as "real" RPMs. The choice to build only a minimally useful version of rpmbuild was important. It meant that it was possible to reduce the number of build-time dependency packages that would have to be built manually from source to a minimal set. Further, the extensive use of GNU autotools in core distribution packages meant that many of those dependencies would automatically configure away advanced features not necessary at this stage. Stage 2 resulted in the creation of a directory containing a number of "recipe" scripts, one per package dependency, that could run in order using a numbered script prefix.

Using git as a filesystem

Perhaps the most novel thing about stage 2 was the use of git early on. This author had an idea that, since git supported distributed development, more Fedora developers could collaborate to build a minimal root filesystem sufficient for building further bootstrap packages if that filesystem were itself a git repository, along with the scripts used to create it. A git repository was created that initially contained the stage 1 filesystem contents (the "hard float" toolchain components). To this was added a "/stage2" directory, containing a "recipe.d" subdirectory. The latter housed a number of individual scripts necessary to build particular packages from sources also contained within the same filesystem. The use of git to manage the entire work-in-progress chroot meant that all prospective developers would need to do would be to clone the filesystem, and then make it available to an ARM target system as a chroot (typically by having a second root-owned clone that was NFS-exported to a suitable development system). It was ugly, but it worked.

Each time a developer added a new recipe for an RPM build requirement in "/stage2/recipe.d", they were able to commit that recipe along with all of the sources for a given package, its populated build directory after running the build for the package, and the subsequent changes to the filesystem after installation. Each git commit thus represented one more step toward having a usable set of dependencies to build rpmbuild. In cases where there were not direct build interdependencies between the basic set of packages, developers working on stage 2 were able to parallelize the effort by building packages in their own git branches prior to issuing a standard "pull" request to the maintainer of the filesystem. This had the benefit of providing a git history for general posterity. The output of stage 2 was a working rpmbuild.

Once an rpmbuild binary was available and it became possible to build RPMs, it was on to the next stage. Stage 3 involved taking the filesystem from stage 2 (which now contained enough binaries and dependencies to support running minimal RPM builds of the software that had been built from source up until that time) and rebuilding its contents as regular RPM packages, then adding sufficient additional dependency packages to build the standard "yum" and "mock" Fedora packaging tools. A new top level directory, "/stage3" was created and within it various builds were performed using rpmbuild, with the output source and binary packages preserved. After each build successfully completed, its build directory was removed (it was not necessary to preserve since the source and binary RPMs contained enough data to reproduce it if that became necessary). The resulting package was installed into the same filesystem, overwriting any files that had previously been manually built and installed during stage 2 with real RPM versions. The resulting changes to the filesystem then formed yet another git commit for easy tracking.

During stage 3, developers could once again parallelize various builds that were not inter-dependent, commit the result of those builds into their own git branches, and then submit pull requests for their branches to be merged into the filesystem. The whole git approach would at first appear to be error prone, but it did not prove to be that way in reality since the packages being built were largely independent and did not modify the filesystem in ways that could not be trivially resolved during a git merge. Eventually, the total size of the git repository containing the root filesystem exceeded 4.6GB (which git handled with ease) and all of the necessary packages sufficient to build and use the "yum" and "mock" utilities were available. This included both perl and python as well as many plugins. Yum is a utility used to manage installing software from repositories of packages. It can be driven by Mock, which is a tool that creates chroots of installed packages and then uses those to build other packages in a trusted and reproducible fashion.

The git approach to bootstrapping seemed to pay off. It was retired following the completion of stage 3 because there were sufficient RPM packages available and infrastructure to use yum and mock directly to build other packages. In addition, at this stage it became possible for Dennis Gilmore to use the mock support to build a few more packages and to then create a minimal bootable Fedora 15 "hard float" image for installation onto a few PandaBoards. Images were also created with alternative kernels for other ARM systems, including the ComuLab TrimSlice (based on the Nvidia Tegra-2). These images were made available to developers who then had a choice of booting the images or using them as chroots on an existing Fedora 13 installation (the result being a chroot within a chroot for mock builds done on such latter systems). The images were collectively then used for stage 4 of the bootstrap.

The stage 4 images included a set of scripts written by Dennis Gilmore and subsequently enhanced by many others, including Henrik Nordström. These scripts implemented a poor man's distributed build system (for various reasons, it was not possible to run a full Koji build system yet). A dedicated virtual server machine was created to host various source and successfully built packages in a "stage4" yum repo. Then, each developer with a build system running the stage 4 image was able to automate the process of retrieving a package from the server, building it in mock on their ARM system, and contributing the results back to the overall repository of built stage 4 packages. Using this approach, over 13,000 binary "armv7hl" (Fedora's name for the ARMv7 hard-float, little-endian architecture) packages were built over a period of a couple of weeks. These were combined with the existing "noarch" (meaning architecture independent) packages from the primary architecture (x86) to create an initial set of Fedora 15 ARM packages.

Stage 4 was the largest stage by far since it targeted generally building the entire distribution set of packages. At this point, it was discovered that many source packages required some tweaking. Some did not build for ARM at first, requiring usually trivial fixes and the occasional patch be committed into the Fedora git repositories and built first for the primary architecture. In the case that ARM-specific changes that were not yet in the official Fedora git repositories, special package versions were built containing obvious and compatible changes to the release number (such as appending ".arm0") in such a way that updates later built from the fixed git repository would not conflict. A tool was later written by DJ Delorie to automatically track down those packages that required fixes yet to be committed in order to ensure all Fedora ARM packages were fully in sync with the original primary architecture set.

During stage 4, it had been decided generally to build the Fedora 15 package set as it was on the day Fedora 15 was released for the primary architecture, not including all of the updates that might have been built in the interim (unless those updates contained necessary ARM or non-ARM fixes). This is because the intent was to first bootstrap the entire distribution to the same point as Fedora 15 had been at release on the primary architecture. This known combination of packages could then be used to rebuild the entire package set in the stage 5 final Koji mass rebuild. Updates would then be built from the primary architecture using the known good Fedora 15 base. Henrik Nordström in particular assisted greatly at this stage in building packages and fixing problems with the stage 4, even though at times he did not always agree with the pedantry of targeting non-updates packages first.

And that brings the reader up to the present day. As of this writing, a core set of Fedora 15 packages has been built and the distribution is usable on ARMv7 systems (at least experimentally). The final stage is to rebuild everything one more time using the standard build infrastructure in a known good configuration, using the official ARM Koji system. Fedora protocol is that each package be built exactly once (for official builds) for a given release and at that time it should be built for all variants of the particular architecture. This means that it is necessary to wait for a corresponding initial set of ARMv5 packages to be built prior to completing the stage 5 final mass rebuild of Fedora 15. Since ARMv5 is not a new architecture, and is not changing ABI, all that is required there is a stage 4-like rebuild using mock. This is currently being performed using many of the same builders that bootstrapped ARMv7, under the control of a new set of scripts named "moji", written by Jon Chiappetta of Seneca College.

Once a mass rebuild of the entire Fedora 15 package set has been completed using Koji, an initial release of Fedora 15 will be made available. It is hoped that this will happen prior to the final release of Fedora 16 on the primary architecture (in early November), or at least soon thereafter. It should be noted that until Fedora ARM is a primary architecture, and while it is playing a constant game of catch-up, no release will be entirely finished. Instead, it will be good enough for the criteria established for that particular release. This means that there will be some (non essential) packages that do not make the initial release of Fedora 15. These will follow the release as there is time to work on making them available.

Following the Fedora 15 release, attention will turn to the next phase. It has been proposed that the team skip Fedora 16 and target getting rawhide (the development build of Fedora) built as the next step, using a tool known as koji-shadow to automatically rebuild all rawhide package builds within a reasonable period of their being built on the primary architecture. Whether "reasonable" is one day or several days will have to be decided, but it is clear that, by targeting rawhide as quickly as possible, Fedora ARM can benefit from following along closely as Fedora 17 and 18 are released.

The future of Fedora ARM

The Fedora ARM project has gained greater cohesion over the past 6 months as more involvement from many of the parties involved has lead to the solving of a number of challenging problems around package building, architecture bootstrap, and so on. Clearly, one of the primary (pun intended) goals of Fedora ARM is to reach primary architecture status as effectively as possible. That won't happen over night, and, in fact, no secondary architecture has ever been promoted to primary yet, so there is a lot left to determine around what exactly the requirements will be and how that will be done.

Primary architecture status would afford many benefits to Fedora ARM, such as requiring all Fedora packagers to support ARM (rather than relying on good will in fixing bugs - although the assistance given so far has generally been excellent), and being able to define blockers and targets for individual Fedora releases. As a secondary architecture, Fedora ARM must play a constant game of catch-up, rebuilding packages that have already been built on the primary and hoping that not too many issues will arise to slow down the overall release. As a primary, the entire distribution will wait if an issue arises that causes a package somewhere to fail to build on ARM. At the same time, primary architecture status brings benefits to Fedora in terms of having more than one first-class architecture and will allow Fedora to better compete for the growing numbers of users who have ARM based systems.

Getting to primary architecture status will take some time. At a minimum the process will likely require that Fedora ARM very closely track the building of every primary architecture package and produce a build shortly thereafter (and eventually in parallel). Tools such as koji-shadow are already being deployed to automatically track primary architecture builds and initiate secondary ones in this fashion. In addition, the systems used to build packages will need to be more rugged than those that are available today. The current builder hardware is sufficient for a lab environment, but it is occasionally plagued with issues such as faulty SD cards, overheating boards, and so forth. It is hoped that newer, more suitable hardware will appear over the coming months to solve that issue.

Beyond a plan for becoming a primary architecture, a number of specific recommendations are likely to feed out of the current efforts of Fedora ARM project back into the planning around general Fedora releases. They are likely to include a call for mandatory mass rebuilds every one or two Fedora releases. A mass rebuild means that every package in the entire distribution is rebuilt from scratch, and it makes the life of secondary architectures (or new architectures) much easier because those secondary architectures need build only one release worth of package dependencies.

More about Fedora ARM

To find out more about the Fedora ARM project and to get involved, visit the Fedora ARM secondary architecture wiki pages. Here you can find directions on joining the "arm" Fedora mailing list, and joining the #fedora-arm IRC channel on the Freenode IRC network.

[ Jon Masters is a Principal Software Engineer at Red Hat, where he works on the Fedora ARM project. Jon is co-author of Building Embedded Linux Systems, and is currently writing a book on porting Linux to new architectures. ]

Comments (27 posted)

Brief items

Distribution quotes of the week

-- Colin Watson

-- Michael Hall

Comments (7 posted)

OpenELEC 1.0 Released

After more than two years of development, OpenELEC 1.0 has been released. OpenELEC is a Linux distribution for home theater PCs, and is based on XBMC. "OpenELEC is an embedded Linux distribution that aims to allow people to use their Home Theatre PC (HTPC) in the same manner as any other device attached to your TV - like a DVD player or Sky box. Turn on your box, and OpenELEC is ready and waiting in less than 10 seconds - as fast as some DVD players. A simple remote control is all you will need. On top of that, one need not worry about updating OpenELEC, as the software will self-update automatically when connected to the Internet and an update is available. Installing OpenELEC takes less than 5 minutes and requires zero Linux experience. After that, one need merely add one's media files and preferred addons, and OpenELEC will 'just work.'"

Comments (4 posted)

openSUSE 12.1 RC1 available

The openSUSE project has announced the first release candidate for openSUSE 12.1. "The next release of openSUSE is expected to bring a large number of improvements and changes. Many of these are the 'usual' updates any Linux distribution offers. These include the latest Firefox, GNOME 3.2 and KDE's Plasma Workspace 4.7. Under the hood, we have Linux kernel 3.1 and we expect to be the first to ship Google's new programming language Go. We also overhauled our boot procedure introducing systemd and Grub2 (still in testing) and of course we'll ship the latest developer tools and libraries as well as all the sysadmin goodies openSUSE is known for!"

Full Story (comments: none)

Canonical announces five years of support for Ubuntu 12.04 LTS desktops

Instead of the usual three-year support for a long-term support (LTS) release of Ubuntu on the desktop, Canonical has announced that 12.04 LTS ("Precise Pangolin") will be supported for five years. That is in keeping with the support period for the server version of an LTS release. "April 2012 will see the fourth LTS release of Ubuntu. LTS releases have become particularly popular with Ubuntu business users. Canonical's own survey data shows over 70% of server users are deployed on LTS versions of the product. Bringing this extended support to the desktop is a response to similar popularity in businesses of the desktop LTS releases. [...] The first two years of the LTS period will benefit businesses by including hardware updates (through regular point releases) allowing them to keep up to date with the latest hardware upgrades. Maintenance updates will continue for a further three years. Businesses can now rely on always running an LTS version regardless of their hardware refresh rate."

Comments (25 posted)

Yocto 1.1 released

The Yocto project has sent out a press release announcing the availability of version 1.1. New features include multilib support, an improved graphical configuration system, initial support for the x32 ABI, Btrfs support, and more. The Yocto project will also be taking over management of the eglibc C library fork.

Comments (3 posted)

Distribution News

Fedora

Upcoming elections for FAmSCo, FESCo, and the Fedora Board

It's election time for the Fedora Project, with seats open on the Ambassadors Steering Committee, the Engineering Steering Committee, and the advisory board. Nominations are open until November 5, 2011.

Full Story (comments: none)

Newsletters and articles of interest

Distribution newsletters

• DistroWatch Weekly, Issue 428 (October 24)
• Maemo Weekly News (October 24)
• openSUSE Weekly News Issue 198 (October 22)
• Ubuntu Weekly Newsletter Issue 238 (October 23)

Comments (none posted)

Desktop dreams: Ubuntu 11.10 reviewed (ars technica)

Ars technica has a review of Ubuntu 11.10. "Ubuntu 11.10, codenamed Oneiric Ocelot, prowled out of the development forest earlier this month. In our review of Ubuntu 11.04, released back in April, we took a close look at the strengths and weaknesses of the new Unity shell and compared it with GNOME 3.0. In this review, we're going to revisit Unity to see how much progress it has made over the past six months. We will also take a close look at the updated Software Center user interface and the transition from Evolution to Thunderbird."

Comments (none posted)

Andy Rubin: Android 4.0 to be open sourced by year end (The H)

The H reports that the source code for Android 4.0 ("Ice Cream Sandwich") will be released a few weeks after the recently announced Galaxy Nexus smartphone starts shipping, according to Google's Andy Rubin. "Code-named 'Ice Cream Sandwich' (ICS), Android 4.0 was first revealed yesterday (19 October) alongside the new Nexus device, at a joint Google and Samsung event. The new version of the mobile operating system includes features from both the current phone version, 2.3.x "Gingerbread', and the tablet version, 3.x 'Honeycomb'. While it will reportedly work on both large- and small-screen devices, it was only demonstrated on the Galaxy Nexus smartphone."

Comments (45 posted)

Puppy Linux 5.3 "Slacko" based on Slackware 13.37 (The H)

The H takes a look at the latest release of Puppy Linux.

Comments (none posted)

7 Years of Ubuntu: Ubuntu Then and Now (Linux.com)

Joe 'Zonker' Brockmeier looks at how Ubuntu shaped the Linux landscape over the last 7 years. "Today marks seven years since Mark Shuttleworth announced Ubuntu 4.10. Dubbed "Warty Warthog," Ubuntu 4.10 was a major departure from other Linux distros at the time. Seven years, and 15 releases, has brought a lot of change. Let's take a look at how Ubuntu has grown from the new kid on the block to one of the most popular Linux distributions ever. It seems like just yesterday when Ubuntu made its debut. Today, it's hard to imagine a time when Ubuntu wasn't a major part of the Linux landscape. When Ubuntu first appeared, though, it was not obvious to all that it would succeed."

Comments (4 posted)

Page editor: Rebecca Sobol

Development

A quick sketch of Krita 2.4

The painting application Krita is nearing its next point-release, 2.4. Because Krita is a component of the KDE-based office suite Calligra, its numbering scheme is synchronized to the larger project's. But despite the small increment it inherits this time, the new release incorporates a sizable number of important changes, including new artists' tools and a built-in network for sharing resources with other Krita users.

Thus far, the beta releases of Krita 2.4 have been made in source code packages only. However, there are volunteer-created builds for a variety of platforms and Linux distributions. When Krita and Calligra reach the 2.4 final stage, it is expected that most distributions will push out an updated package. I tested nightly binary builds from project NEON on Ubuntu 11.04, primarily because the machine in question is the one to which my Wacom tablet is attached.

This is vital, because Krita is increasingly geared towards tablet usage. While other graphics applications offer solid tablet support — pressure and tilt sensitivity for starters, button and key assignments in the advanced cases — Krita is steeped in tablet culture. The UI has been redesigned in recent development cycles to make it more touch-friendly (with fewer menus and more button and slider controls), and there are an increasing number of painting parameters that can be bound to tablet digitizer sensors. In fact, if you use Krita without a tablet these days, you almost risk being left out of important functionality. Tablet PCs with touch-screens make for a partial substitute, but do not offer as many touch parameters or levels of sensitivity.

Brushes and tools

Many of Krita's recent advances have been in the building new paintbrush engines, each of which simulates a different drawing tool or painting medium — paint, ink, chalk, etc. Obviously Krita is not bound by existing physical media, so some of the new and innovative brush engines are hard to describe.

2.4's "curve brush" is like this; the brush engine creates its own fills and flairs as you draw, which introduces some randomness into the final shapes, much like you would experience with natural media. An earlier incarnation of this technique debuted in 2.3, but this revision is based on an update of the original code, and in my experience is easier to work with. The "sketch brush" is also intended to emulate natural-media randomness, but the results are more straightforward (simulating pencil or ink drawing), as is the "color smudge" brush, which allows you to realistically smear and munge color already on the canvas while painting new strokes. All brushes have tablet sensor support, which allows you to (for example) vary the line width and "ink" density in response to pressure, tilt, and acceleration.

The list of paintbrush engines is always growing, but brushes are certainly not the only tools. New in 2.4 is a "multi-hand tool," which is an extension of the automatic mirroring introduced earlier this year. With the multi-hand tool, you can duplicate your brush strokes on the canvas as you paint, with your choice of reflections and planar translations applied. A few other tools have picked up additional features, including the line tool, which now has adjustable parameters (width, opacity, etc) that can be bound to tablet sensor values just like the paintbrushes.

Another new addition is called "pipe mode" for the text-painting tool. It allows you to "paint" text onto the canvas — you select the source text first, then as you draw, the text is rendered onto the canvas fitted to the curves of the stroke that you draw. It beats the alternative methods for splaying text in a creative fashion: creating "normal" blocks of text and subsequently trying to distort them with perspective adjustments.

Beyond the tools themselves, there is a substantial list of new "blending modes" supported in Krita 2.4's layers. The blending mode is the property of a layer that tells Krita how to combine its pixels with the pixels underneath it — which is particularly important for alpha-transparent regions. You may be familiar with Photoshop or GIMP's standard options, such as "Normal," "Multiply," "Subtract," "Hard Light," "Soft Light," and so on. Some these are simple descriptions of what math is applied to the pixels (e.g., "Multiply"), while others are designed to emulate real world media, such as transparency film physically overlaid.

The new Krita release adds more than twenty modes of both types; from "Geometric Mean" to "Grain Extract." Many of them are blending modes that artists are already familiar with from Photoshop (including Grain Extract), and Krita's implementations are identical. Ensuring this compatibility also entailed adjustments to some of the existing blending modes, which could theoretically be an annoyance to an artist with existing Krita documents making use of them, but is a far better approach in the long run.

In another multiple-tool change, all of the painting tools can now take the source color that they paint with not just from a single, pre-selected color value, but also from gradients, pattern images, or a randomized color source. That allows you to define more complex brush strokes that change color along the way — as well as to paint with random colors, although I have yet to see a truly awe-inspiring work of randomly-colored art.

Pre-sets: you versus the world

A lot of work has gone into enhancing the pre-set system in this new release, perhaps making it a more visible improvement for the average working artist than is any individual brush engine. The issue at hand is that all of Krita's brushes offer a glut of user-tweakable settings: there are always multiple tablet sensor mappings (pressure, tilt, etc), plus brush size, shape, and opacity, blending mode and color source, and there are parameters unique to each individual engine. There is no "best" combination of settings, even for a particular project. In practice, artists want to adjust the settings to fit the situation, and save the most useful combinations of settings for later use.

2.4 makes this substantially easier, with a "preset editor." Saved presets are accessible in the paintbrush drop-down menu at the top of the canvas, as well as in a dockable tool palette that you can place anywhere on the screen. You can assign names, keyword tags, and a short description to each pre-set to assist your memory later. The pre-set editor also has a small sample canvas onto which you can paint, either to test your settings as you make adjustments, or to save as a thumbnail for the pre-set browser.

Pre-sets are an invaluable feature — if you have ever worked with the dodge and burn tools in GIMP trying to freshen up a photograph, you may notice that it is easy to forget exactly what settings worked best. With the ability to save pre-sets, you have a recourse. But what is more intriguing is that Krita pre-sets can be shared with other users via the Get Hot New Stuff (GHNS) protocol. GHNS is supported in some other KDE applications, but its appearance in Krita is a first for the open source creative suite. Other applications have a web-based interface to GHNS "resource repositories," but Krita integrates GHNS brush pre-sets into the existing pre-set framework. You can choose to upload your own pre-sets and browse or download those shared by others.

It has long been possible to share brushes, gradients, and other resources between the major open source graphics applications, but it used to require downloading separate files and dropping them into the appropriate hidden directories in your home folder. GHNS offers is a far better experience — akin to Inkscape's built-in Open Clip Art Library downloader. I like the trend quite a bit; Michael Terry's AdaptableGIMP project (which we covered in May 2011) has a similar feature for retrieving GIMP task-sets (essentially macros), and Krita's menus have space for macros and tutorials, too. Together all of these features make a good argument that in-browser "web apps" are not the only viable approach to collaborative editing, and I hope that we will see more.

Finally, although it is not fully integrated with the brush pre-set system, 2.4 also introduces the ability to save and recall workspace layouts: task-centric arrangements of the Krita interface and its various docks and windows. That hints at more customizability in releases still to come.

UI and core changes

Speaking of such layouts, Krita adds several new dockable utility palettes in this release. Some, like the document undo/redo history and the channel mixer, are standard offerings in other raster editors, but others are more original. For example, there is an "image docker" in which you can open another image file to keep on-screen as a reference — think painting-from-a-photograph. There are also two new color-selectors, continuing Krita's tradition of offering artists multiple ways to approach paint color selection. The new selectors include an artist's color-wheel (which, despite being circular, is not identical to the triangle-inscribed-in-a-circle selector you have seen before), and a "specific color selector" for directly inputting hex color values.

A handful of other UI changes are designed to simply make Krita easier to use. This includes adding opacity, flow, and brush size sliders to the top toolbar, where the user can have one-click access to changing them with minimal interruption. There is also a new full-screen mode, which can be very helpful for tablet users. The active area of a graphics tablet is typically mapped to either the active window or to the entire X display; removing the non-image UI elements from the screen effectively removes dead zones from the outer borders of the tablet's active surface area.

Krita has always fired up a "new image wizard" whenever it is launched without opening an existing file. The wizard lets you choose the dimensions and color space of a new file, or to open up a template. I found several new templates in this release of Krita, although it is possible that some of them have been around longer and I just failed to find them before (perhaps due to my distribution's packaging). In any case, the new templates show off some interesting options, and highlight what Krita templates can do beyond standard flat images. Several are "comic" templates of differing sizes, and they are actually multi-layer documents, with masks and borders to mark off comic panels, and separate grayscale "ink" and RGB "paint" layers underneath, to reproduce a typical comic workflow.

Showtime

As always, there is a long, long list of smaller changes in the new version — better support for popular file formats (including OpenRaster, DNG, GIMP's XCF, and Photoshop's PSD), the ability to do incremental saves (automatically appending a version number to each save), and so on. It is hard to make a case for any particular new feature as a reason to rush out and upgrade, but that is probably not what most users are after anyway. Suffice it to say that the incremental improvements in the UI and painter's tools have made each Krita release better than the last for several development cycles in a row — and that kind of track record demonstrates that the team is paying close attention to what its users need. When you look at the Krita site now, it is filled with drawings and painted artwork, not test images or artificial samples to demonstrate features.

In the future, I am most interested to see where GHNS heads. Thanks to compatibility with GIMP, MyPaint, and other tools, brush pre-sets are not inherently limited to Krita, and sharing other types of content could prove valuable for many other applications. Consider the ability to directly retrieve task-sets and tutorials within applications — interesting in its own right; far more interesting when you imagine tutorials that span multiple applications and are accessible within each at the appropriate step.

There is also some interesting work coming to the Krita core in future releases, such as OpenCL support, which will allow offloading some operations to the GPU. Animation support has also been in the works for a while, through a plug-in to manage multiple frames. But big bullet-point features are always welcome, while adapting to better fit the users' needs is historically the bigger challenge for open source projects. It is good to see Krita managing both.

Comments (4 posted)

Brief items

Quotes of the week

-- Jean-Baptiste Queru clarifies the Android source release plan

-- Rob Weir

Comments (none posted)

The Android source is back online

The Android source repository has been offline since the kernel.org compromise; it has now returned on a new site. Services like Gerritt will take a little longer still. "To reiterate, these servers contain only the 'gingerbread' and 'master' branches from the old AOSP servers. We plan to release the source for the recently-announced Ice Cream Sandwich soon, once it's available on devices."

Comments (14 posted)

The Qt project launches

The Qt project, which is an effort to have open governance and development for the Qt cross-platform application and UI framework, has announced its launch. "All development will happen in one central location, with access for everyone at the same time. No more separate code flow for 'Nokians vs others', and no more time delays! What you see is what we see. Discussions, decisions and roadmapping will all happen in the community, by the community, for the community. Anyone can be a Contributor, and even an Approver or Maintainer if they show enough merit."

Comments (4 posted)

Newsletters and articles

Development newsletters from the last week

• Caml Weekly News (October 25)
• Perl Weekly (October 24)
• PostgreSQL Weekly News (October 23)
• Tahoe LAFS Weekly News (October 25)

Comments (none posted)

Page editor: Jonathan Corbet

Announcements

Articles of interest

Creator Of Lisp, John McCarthy, Dead At 84 (TechCrunch)

TechCrunch reports that John McCarthy has died. "McCarthy believed AI should be interactive, allowing for a give and take similar to AI simulators like Eliza and, more recently, Siri. His own labs were run in an open, free-wheeling fashion, encouraging exploration and argument. He won the Turing Award from the Association for Computing Machinery in 1972 and the National Medal of Science in 1991." (Thanks to Norman Gaywood)

Comments (15 posted)

FSFE: Fellowship interview with Rikard Fröberg

The Free Software Foundation Europe has published an interview with Rikard Fröberg. "Rikard Fröberg used to work in the IT industry as a consultant and teacher, but in the late 90s changed his focus to Free Software and free culture. He helped start up Gnutiken, and has helped organise FSCONS for the last three years. Currently employed by The Society for Free Culture and Software as a project manager, Rikard is managing a project for increasing accessibility at events."

Comments (none posted)

New Books

Droid X2 and Galaxy Tab Missing Manuals--New from O'Reilly Media

O'Reilly Media has released "Droid X2: The Missing Manual, Second Edition" and "Galaxy Tab: The Missing Manual", both by Preston Gralla.

Full Story (comments: none)

Calls for Presentations

LAC 2012: the Linux Audio Conference - Call for Participation

The Linux Audio Conference 2012 will take place April 12-15 in Stanford, California. The call for participation will be open until January 11. "We invite submissions of papers addressing all areas of audio processing based on Linux and open source software. Papers can focus on technical, artistic or scientific issues and can target developers or users. We are also looking for music that has been produced or composed entirely or mostly using Linux and other Open Source music software."

Full Story (comments: none)

Submit your SCALE paper yet?

The Southern California Linux Expo (SCALE) will take place January 20-22, 2012 in Los Angeles, California. The deadline for abstract and proposal submissions is November 17.

Full Story (comments: none)

Upcoming Events

Events: November 3, 2011 to January 2, 2012

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
October 31 November 4	Ubuntu Developer Summit	Orlando, FL, USA
October 31 November 4	Linux on ARM: Linaro Connect Q4.11	Orlando, FL, USA
November 1 November 3	oVirt Workshop and Initial Code Release	San Jose, CA, USA
November 1 November 8	2011 Plone Conference	San Francisco, CA, USA
November 4 November 6	Fedora Users and Developer's Conference : India 2011	Pune, India
November 4 November 6	Mozilla Festival -- Media, Freedom and the Web	London, United Kingdom
November 5 November 6	Technical Dutch Open Source Event	Eindhoven, The Netherlands
November 5 November 6	OpenFest 2011	Sofia, Bulgaria
November 7 November 11	ApacheCon NA 2011	Vancouver, Canada
November 8 November 12	Grace Hopper Celebration of Women in Computing	Portland, Oregon, USA
November 10 November 12	Clojure/conj 2011	Raleigh, NC, USA
November 11 November 12	Zentyal Summit	Zaragoza , Spain
November 11 November 13	Free Society Conference and Nordic Summit 2011	Gothenburg, Sweden
November 12	London Perl Workshop 2011	London, United-Kingdom
November 12	Emacsforum 2011	Copenhagen, Denmark
November 14 November 17	SC11	Seattle, WA, USA
November 14 November 18	Open Source Developers Conference 2011	Canberra, Australia
November 17 November 18	LinuxCon Brazil 2011	São Paulo, Brazil
November 18	LLVM Developers' Meeting	San Jose, CA, USA
November 18 November 20	Foswiki Camp and General Assembly	Geneva, Swizerland
November 19 November 20	MediaWiki India Hackathon 2011 - Mumbai	Mumbai, India
November 20 November 22	Open Source India Days 2011	Bangalore, India
November 24	verinice.XP	Berlin, Germany
November 28	Automotive Linux Summit 2011	Yokohama, Japan
December 2 December 4	Debian Hildesheim Bug Squashing Party	Hildesheim, Germany
December 2 December 4	Open Hard- and Software Workshop	Munich, Germany
December 4 December 7	SciPy.in 2011	Mumbai, India
December 4 December 9	LISA ’11: 25th Large Installation System Administration Conference	Boston, MA, USA
December 27 December 30	28th Chaos Communication Congress	Berlin, Germany

If your event does not appear here, please tell us about it.

Page editor: Rebecca Sobol