Refactoring Roundcube

The Roundcube web-based email client has been around for nearly ten years now. Over that time, it has become quite popular, but it has built up a fair amount of technical debt that the project would like to erase. To that end, it has started a crowdfunding drive targeted at rearchitecting the code with a relatively modest goal (at least compared to the sums collected by some projects). But there's more to it than that; the project is making a real effort to widen its community by bringing in more participants—for both code and ideas.

Thomas Brüderli started Roundcube in 2005 as an alternative to the SquirrelMail and Horde Internet Messaging Program (IMP) webmail clients, which were showing their age. By 2009, though, Brüderli didn't have much free time to contribute to the project. The Kolab groupware project wanted to adopt Roundcube as its email client and he was contacted by Kolab project member Georg Greve, who was in the process of starting Kolab Systems to support Kolab. Those discussions led to Kolab Systems helping to fund Roundcube development, which allowed Brüderli and others to implement a wide variety of new features for Roundcube—all of which were released as free software.

As Greve noted in a blog about the Roundcube Next crowdfunding effort, the collaboration between Roundcube and Kolab Systems has worked well. Roundcube has been "a great success", but this hasn't translated to a larger development community:

Back in September, Brüderli put together a description of the changes that were needed for Roundcube Next. Many of the technical decisions made when the project was young need rethinking, he said. The architecture of Roundcube comes from a different era of the web "when browsers weren’t as capable and performant as they are today and when the term AJAX has not yet been known nor did we have nifty libraries such a jQuery or Backbone.js at hand".

To start with, Brüderli said, Roundcube should do more in JavaScript on the client side, rather than generating HTML pages in PHP on the server that get loaded into an iframe. A widget-based user interface based on Backbone.js should be created that would take into account mobile and high-DPI devices; it should allow customization (i.e. "skinning") through CSS exclusively. Beyond that, the project would "use more 3rd party libraries like require.js, moment.js, jQuery or PHPMailer, Monolog or Doctrine ORM". The kinds of changes he described may "sound like a buzzword bingo from a web developers conference", but they are important changes moving forward.

The Indiegogo crowdfunding campaign started at the beginning of May with a goal of $80,000; as of this writing, it has gotten a little over $31,000 with another month to run. The campaign provides the usual perks—stickers and T-shirts, for instance—but it also provides ways for larger donors to get more involved with the project, especially during the refactoring and rearchitecting that embodies Roundcube Next. For example, donations of $75 or more get access to the "backstage pass":

Donations of more than $5,000 add the "advisor" perk, which gives a seat on the Roundcube Advisory Committee; so far, two donations of that amount or more have been pledged (in fact, one is at the $10,000 "deployer" level that adds ten hours of consulting with the Roundcube team). The campaign is of the "all or nothing" variety (also known as "Fixed Funding"), which means that if it doesn't reach the $80,000 goal, the funds will be returned to the donors. It is hard to say for sure, but the pace of contributions suggests that the campaign may struggle to reach the goal, which is both somewhat surprising and sad.

That would not stop the efforts by Kolab Systems to push the project forward, though it would likely slow progress somewhat. As Greve noted in his post, a community-wide effort is much more desirable:

Perhaps there are other up-and-coming projects that are garnering more attention than Roundcube; Brüderli mentioned RainLoop and Mailpile (which we looked at back in September) in his post. Both of those may offer compelling user interfaces, "which make Roundcube look like the old dinosaur", but they lag behind the feature set of Roundcube. It's also not clear that either of those projects is having any better success at attracting a community for what should be an important component of a free and open internet.

Even if the campaign were to fail (and, given the amount of money targeted, that is a bit hard to understand), it would be nice to at least see more of a community spring up around the project. It is, evidently, quite popular with various webmail providers, but apparently falls short in gaining contributions and suggestions from those users. Perhaps the campaign can serve as a wakeup call of sorts to help build a Roundcube community—both for the Next project and for the future beyond that. Unlike some other fundraising campaigns, the Roundcube Next campaign would seem to be as much as about raising a community as it is about raising money.

Comments (11 posted)

Ambient light adjustment on the desktop

Many current smartphones and tablets are able to automatically adjust their screen brightness in response to ambient lighting conditions. Doing so is often to help reduce battery usage, but it also provides a more eye-pleasing (or, to put it another way, a less eye-straining) user experience. Both issues are of concern to laptop users as well, but desktop Linux environments have not historically supported such a feature. Recently, Richard Hughes released a small, inexpensive hardware add-on that make automatic backlight adjustment possible.

The device is called the ColorHug ALS (for "ambient light sensor"). Hughes announced it in March 2015, with an initial batch of just 42 units. The ALS uses the same light sensor that Hughes has previously employed in his first-generation ColorHug devices. Around the same time, he announced the first test units of a significant update to the ColorHug that makes use of a more sophisticated light sensor. Consequently, the ColorHug ALS may be, in part, a way to make use of unused stock from the old model—in which case it is possible that, when the first batch is gone, a second batch may never appear (although the designs and schematics are freely available to anyone wishing to make their own).

On the other hand, the ColorHug ALS is used with the same client software package (colorhug-client) as the ColorHug colorimeter, which could perhaps be repurposed as an ambient light sensor should the user be so inclined. Hardware details aside, though, it is the software side of the project that is the most interesting. The device is physically tiny; it plugs into a USB-A port and less than half of the circuit board sticks out beyond the edge of the port—just enough for the light sensor and a status LED that blinks when inserted.

Version 0.2.6 and newer of colorhug-client includes a tool called colorhug-backlight that polls the device for an ambient-light reading, then adjusts the display backlight setting in order to compensate for any changes since the last measurement. The user needs to manually adjust the brightness to their personal preference; the utility just makes adjustments to keep it at the same relative level as conditions change. The tool also provides a time-series plot of the readings over the past few minutes, should one desire to see some context.

I tested out the device and tool on a laptop (running GNOME 3.14, which is the minimum required version, although I did have to manually update the colord and gusb packages to the versions that were included on the USB stick that shipped with the device). Under normal conditions, the brightness adjustments are subtle enough that one might not even notice them happening. A sudden, drastic change might make for a different experience, of course. Users whose computing environments include strobe lights might find the device's response time less satisfactory, but those users clearly have bigger issues to worry about.

Moving forward, though, requiring the installation and usage of a custom utility to maintain proper backlight levels is hardly the ideal solution. To that end, Hughes noted in his original blog post that the goal was to get the backlight-control framework moved upstream into GNOME control center.

The next big improvement on that front came on May 27, when he announced a new version of the ALS firmware. This firmware revision is the work of Benjamin Tissoires, and it implements the Sensor Human Interface Device (HID) API that Microsoft is promoting to Windows 8 equipment manufacturers. Consequently, the same code (presumably GNOME control center, at some reasonably near point in the future) would be able to monitor the ColorHug ALS or the built-in light sensors of many recent laptops. Thus, a great many more users will be able to take advantage of automatic brightness adjustments.

Users on desktop machines, though, appear to be out of luck for the time being. Obviously battery life is not of concern to anyone running on wall power, but lighting conditions can and do change in offices and homes as well. Unfortunately, external displays do not provide the same sort of access to backlight adjustments that laptop screens do—so, even with an ambient light reading, other steps would be required to keep the brightness level consistent. Hughes noted this in an email to the colorhug-users mailing list. In a follow-up message, he added that supporting external monitors under X11 would require "several hundred hours" of additional work. The task would be far simpler under Wayland, he said, but at present there are not enough Wayland-only systems to justify the effort.

On May 22, Bastien Nocera announced the 1.0 release of his iio-sensor-proxy project, which provides a general framework for monitoring a variety of environmental sensors (as distinct from internal hardware health sensors like CPU thermometers, which have been well-supported on Linux systems for quite some time). Ambient light sensors are among the first sensors supported by iio-sensor-proxy, and Nocera's announcement included a preview image of how the support may look in GNOME 3.18: a simple toggle switch labeled "Automatic brightness" underneath the existing screen-brightness slider.

Widespread support for this sort of feature may someday be taken for granted. Until then, the ColorHug ALS remains an inexpensive (£20) and simple way to keep one's screen brightness even. It will be interesting to see how much further the project can go, though: the light sensor in the ColorHug ALS reports RGB color readings as well as overall luminosity; one could conceivably use it to make adjustments that take the color of the room light into account as well. There have been allusions to such uses on the colorhug-users mailing list; suffice it to say that a number of people are interested in having their Linux distribution respond more readily to the real-world conditions surrounding the user—which is always a positive step.

Comments (11 posted)

V2X networking and Linux

Despite the prominence that Linux plays in automotive consortia like GENIVI and the long list of car-industry heavyweights involved in projects like Automotive Grade Linux, there are still plenty of areas in the automotive market where Linux and open-source software are forced to play catch-up. One of these areas is so-called V2X networking—a blanket term that encapsulates a handful of related initiatives, namely vehicle-to-vehicle (V2V) communication and vehicle-to-infrastructure (V2I) communication. At the 2015 Automotive Linux Summit in Tokyo, Hong-Jong Jeong from ZENOME gave a talk on the basics of current V2X networking projects and some insights into how Linux developers can get started.

The main barrier to participating in V2X development, he explained, is that the companies spearheading the work have only released hardware in "black box" form: proprietary, undocumented radio modules, often in sealed enclosures. The literal black-box approach makes developing device drivers virtually impossible, of course, and it does not help that the devices themselves tend to be prohibitively expensive as well.

Nevertheless, there is published standards work to draw on, and progress is being made. ZENOME has been concentrating its efforts on V2V. As of today, there are three main standards in use for V2V networking that are designed to function together, and they are close to final approval in the US and in Europe. The lowest layer is 802.11p, an extension to the existing WiFi standard. On top of this is the IEEE 1609 family, which defines a multi-layer stack for the Wireless Access in Vehicular Environments (WAVE) architecture. Finally, the top level is defined in SAE J2735, the Dedicated Short Range Communications (DSRC) Message Set Dictionary.

Broadly speaking, the standards fit together as follows. V2V exchanges are, due to the nature of moving vehicles, short and transient. So 802.11p sets out some exceptions to the normal 802.11 WiFi physical and access-control layers: links between vehicles take place Outside the Context of a BSS (basic service set), so they are referred to as "OCB." A BSS would be the network defined by an access point or the ad-hoc network created in WiFi ad-hoc mode; these conventions are not established between vehicles. There are no authentication or association steps at the access-control level: all participants transmit a wildcard BSSID. At the physical layer, 802.11p standard defines seven 10MHz-wide channels in the 5.9GHz range for V2V networking, with two reserved for emergency and safety messages.

At the level above 802.11p, the WAVE specification defines a connectionless, UDP-like message protocol. The header includes a service provider field, but no IP address, since the participants may have no Internet gateway at all. The top of the stack is the DSRC Message Set, which defines a set of well-known messages that vehicles may transmit and are expected to understand. Jeong discussed several of the messages, which include emergency vehicle alerts, road-side incident alerts, and map data. The most common message type, however, is the "basic safety message," which is a general status report vehicles are expected to emit periodically.

As far as Linux support is concerned, Jeong noted that, as of 3.19, there is an OCB mode patch for 802.11p in the mainline kernel. A team at the Czech Technical University in Prague and developers at Volkswagen have written patches adding the necessary support to mac80211, cfg80211, and iw. With those patches, he said, it is possible to implement raw socket support.

Acquiring a radio device capable of transmitting or receiving 802.11p has been more difficult. Through some trial and error, ZENOME has found two 802.11a chipsets with open-source drivers that can be modified to work with 802.11p's narrower channels. Both are mini-PCIe cards from Atheros, and the company has been testing them with the Ventana GW5100 Single Board Computer from Gateworks.

Using that hardware and software combination, it is possible to set up a machine that captures WAVE packets and decodes DRSC messages. ZENOME maintains a Yocto layer called meta-v2x with the configuration to receive WAVE packets and hand them up to an (included) user-space daemon that decodes the DSRC message content. At the moment, he emphasized, it is still a work in progress. But the team has tested it against four models of V2V black-box radio from the two leading manufacturers, capturing traffic with tcpdump and Wireshark.

The next steps that the team is exploring include a kernel-space module for handling WAVE messages, a priority queue (which would sort incoming messages based on their emergency status), and some sort of API for applications.

So there is clearly quite a way to go. Even with ZENOME's work, it is only possible to receive V2V messages, and only on two specific hardware devices. Transmitting V2V is yet to come, as is V2I networking and further V2X ideas—Jeong mentioned one recent addition to the V2X fold: V2P networking, for vehicle-to-pedestrian messaging. There is a lot of interest in these networking systems, both from the "intelligent road infrastructure" side, which envisions signs and highway markers that constantly update passersby about nearby conditions, and from the autonomous-vehicle side, which is focused on safety and collision avoidance.

It is, perhaps, unfortunate that the Linux community is forced to hack together compatible radios from other networking equipment, but if experience teaches anything, it is that Linux will always catch up if the need is real enough. Given the time and resources being poured into automotive Linux efforts at present, it is probably only a matter of time before our favorite operating system is on par with the proprietary V2X vendors' products.

[The author would like to thank the Linux Foundation for travel assistance to attend ALS 2015.]

Comments (none posted)

Linux/Moose: Interesting but ineffective

The Linux/Moose worm is not particularly innovative, nor does it exploit new holes, but it does highlight a problem that is likely to only increase over time. In fact, the hubbub around the Internet of Things (IoT) reminds us that plans are afoot to put more and more devices—undoubtedly some with default or easily guessed passwords—onto the net. Taking over a device with a default administrative password is not particularly difficult, but the consequences for the device owner can be rather severe, though they generally aren't for Moose.

The Moose worm was described in great detail in a 54-page PDF by Olivier Bilodeau and Thomas Dupuy of the security software firm ESET. The report is replete with various moose jokes along with an extremely detailed look at how the worm operates. The name stems from the name of the malware binary that gets installed on infected systems: elan2. Élan is French for moose.

At its heart, Moose is a form of malware that targets home routers, including those that Internet Service Providers (ISPs) make available to their customers. It spreads via telnet using access credentials from a list of common or default usernames and passwords stored in the binary. Once it has broken in, it starts scanning for new victims—on the internet at large, in the ISP's range of IP addresses, and on the local net behind the firewall maintained by the router. It can also cause several other kinds of mayhem including rerouting DNS traffic, performing social network fraud using hijacked accounts, and eavesdropping on other devices that are using the router.

Beyond that, there are a set of "command and control" (C&C) servers that Moose communicates with to get its marching orders, report "interesting" traffic to, or proxy requests to hosts both inside and outside the firewall, which allows bypassing network address translation (NAT). Once Moose gains access to another device by having its credentials accepted, it contacts its "report C&C" server to give it information it has gathered about the device (IP address, which credentials were used, CPU type, etc.). That server will send back obfuscated commands to be run on the victim. Those commands will typically result in the malware binary being executed on the victim, causing it to join the Moose "network".

Once a system has been infected, it talks to a configuration C&C server chosen at random from a list in the binary. That server will provide the newly infected device with the IP addresses for the other two C&C servers it should use, one for reporting and one to relay traffic from. For relaying, Moose listens on port 10073 but will only connect with IP addresses from a hard-coded whitelist in the binary. For any successful connection, Moose will set up a SOCKS or HTTP proxy based on the value of the first byte sent. Those proxies can then be used to bypass NAT or to send web requests to arbitrary hosts—generally social networking sites.

Moose also sniffs the traffic that is traversing the router, looking for "interesting" strings that the configuration C&C server supplied. Those strings turn out to be the HTTP cookies for sites like Twitter, Facebook, Instagram, Google, YouTube, and so on. The values get reported back to the C&C servers, which can then use them to perpetrate fraud on those social networks—adding "likes", followers, and such for anyone willing to pay for that kind of "service".

There is more to Moose, of course, which is described in the report. The researchers tried to estimate how many Moose-infected routers there are, but found it difficult to do so because of the way the malware operates. It seems clear that Moose has been operating for roughly a year; ESET has been looking at it since July 2014.

One "feature" that Moose is missing is persistence. A reboot of an affected device will remove the malware, though it may just get installed again if the password is not changed. Even for its seeming mission, though, Moose is relatively ineffective. Most of the social networks (with Instagram evidently being one exception) have moved to HTTPS-only access, which means that Moose can't passively sniff the cookies. It also relies on telnet to propagate, which is becoming less and less popular—at least hopefully.

The basic infrastructure that Moose sets up could be used for other nefarious activities, however. Distributed denial of service, spamming, more active HTTPS interception (since most users will simply click through any browser warnings), or other schemes are all possible. In addition, the report notes that IoT or other non-router devices connected to the internet could be affected in a kind of collateral damage. It is possible that Moose's activity on, say, an internet-attached medical device could interfere with its normal operation—yet another good reason not to make such connections.

As it stands, Moose doesn't pose that much of a threat. It is a clever use of well-known techniques and flaws; it could presumably be "upgraded" into something far more dangerous. Adding SSH, for example, would result in more opportunities to propagate, especially in the future, and wouldn't reduce the effectiveness of the weak-password attack. But it is good to learn as much as we can from Moose, to help recognize and repel its descendants (and siblings) down the road.

The vast array of poorly configured home routers seems like a ticking time bomb of sorts. Even those that were configured with good passwords and sensible access policies may never have been updated since they were installed. Thus they are susceptible to vulnerabilities found in the meantime. Moose is simply another reminder of that threat.

Comments (none posted)

Security quotes of the week

— Bruce Schneier

— David Burge

— A headline from The Onion

Comments (5 posted)

Emergency security band-aids with Systemtap

Here's an article on the Red Hat security blog on the use of Systemtap to apply emergency security fixes. "With the vulnerability-band-aid approach chosen, we need to express our intent in the systemtap scripting language. The model is simple: for each place where the state change is to be done we place a probe. In each probe handler, we detect whether the context indicates an exploit is in progress and, if so, make changes to the context. We might also need additional probes to detect and capture state from before the vulnerable section of code, for diagnostic purposes."

Comments (none posted)

apache2: multiple vulnerabilities

Package(s):	apache2	CVE #(s):
Created:	June 2, 2015	Updated:	June 3, 2015
Description:	From the Ubuntu advisory: As a security improvement, this update makes the following changes to the Apache package in Ubuntu 12.04 LTS: Added support for ECC keys and ECDH ciphers. The SSLProtocol configuration directive now allows specifying the TLSv1.1 and TLSv1.2 protocols. Ephemeral key handling has been improved, including allowing DH parameters to be loaded from the SSL certificate file specified in SSLCertificateFile. The export cipher suites are now disabled by default.
Alerts:	Ubuntu USN-2625-1 apache2 2015-06-02

Comments (none posted)

fusionforge: code execution

Package(s):	fusionforge	CVE #(s):	CVE-2015-0850
Created:	June 1, 2015	Updated:	June 11, 2015
Description:	From the Debian advisory: Ansgar Burchardt discovered that the Git plugin for FusionForge, a web-based project-management and collaboration software, does not sufficiently validate user provided input as parameter to the method to create secondary Git repositories. A remote attacker can use this flaw to execute arbitrary code as root via a specially crafted URL.
Alerts:	Fedora FEDORA-2015-9324 fusionforge 2015-06-10 Debian DSA-3275-1 fusionforge 2015-05-30 Fedora FEDORA-2015-9128 fusionforge 2015-06-10

Comments (none posted)

kernel: denial of service

Package(s):	kernel	CVE #(s):
Created:	June 2, 2015	Updated:	June 3, 2015
Description:	From the Fedora advisory: The 4.0.4-303 update contains a fix for a namespace crash issue. [ 1 ] Bug #1220519 - kernel/sched/core.c:7291 __might_sleep+0x87/0x90() [ 2 ] Bug #1200353 - regression in ktime.h breaks infrared IR blaster/transmitter after upgrade to Fedora 21 [ 3 ] Bug #1214474 - Mouse capture issues when running in vmware
Alerts:	Fedora FEDORA-2015-9127 kernel 2015-06-01 Fedora FEDORA-2015-9227 kernel 2015-06-01

Comments (none posted)

kernel: privilege escalation

Package(s):	kernel	CVE #(s):	CVE-2015-1805
Created:	June 3, 2015	Updated:	July 7, 2015
Description:	From the Red Hat advisory: It was found that the Linux kernel's implementation of vectored pipe read and write functionality did not take into account the I/O vectors that were already processed when retrying after a failed atomic access operation, potentially resulting in memory corruption due to an I/O vector array overrun. A local, unprivileged user could use this flaw to crash the system or, potentially, escalate their privileges on the system.
Alerts:	Ubuntu USN-2967-2 linux-ti-omap4 2016-05-09 Ubuntu USN-2968-2 linux-lts-trusty 2016-05-09 Ubuntu USN-2967-1 kernel 2016-05-09 Ubuntu USN-2968-1 kernel 2016-05-09 Oracle ELSA-2016-3528 kernel 3.8.13 2016-03-24 Oracle ELSA-2016-3528 kernel 3.8.13 2016-03-24 CentOS CESA-2016:0494 kernel 2016-03-23 Debian DSA-3503-1 kernel 2016-03-03 Debian-LTS DLA-439-1 linux-2.6 2016-02-29 Red Hat RHSA-2016:0103-01 kernel 2016-02-02 Oracle ELSA-2015-2152 kernel 2015-11-25 Oracle ELSA-2015-3098 kernel 3.8.13 2015-11-13 Oracle ELSA-2015-3098 kernel 3.8.13 2015-11-13 SUSE SUSE-SU-2015:1611-1 kernel 2015-09-23 SUSE SUSE-SU-2015:1592-1 kernel 2015-09-22 SUSE SUSE-SU-2015:1324-1 kernel 2015-07-31 SUSE SUSE-SU-2015:1491-1 kernel 2015-09-04 SUSE SUSE-SU-2015:1490-1 kernel 2015-09-04 SUSE SUSE-SU-2015:1488-1 kernel 2015-09-04 SUSE SUSE-SU-2015:1478-1 kernel 2015-09-02 Ubuntu USN-2679-1 linux-ti-omap4 2015-07-23 Ubuntu USN-2680-1 linux-lts-trusty 2015-07-23 Ubuntu USN-2681-1 kernel 2015-07-23 Ubuntu USN-2678-1 kernel 2015-07-23 SUSE SUSE-SU-2015:1489-1 kernel 2015-09-04 SUSE SUSE-SU-2015:1487-1 kernel 2015-09-04 Oracle ELSA-2015-1221 kernel 2015-07-14 SUSE SUSE-SU-2015:1224-1 kernel 2015-07-10 Red Hat RHSA-2015:1211-01 kernel 2015-07-07 Red Hat RHSA-2015:1199-01 kernel 2015-06-30 Scientific Linux SLSA-2015:1137-1 kernel 2015-06-25 Red Hat RHSA-2015:1190-01 kernel 2015-06-25 Oracle ELSA-2015-1137 kernel 2015-06-23 CentOS CESA-2015:1137 kernel 2015-06-24 Red Hat RHSA-2015:1138-01 kernel-rt 2015-06-23 Red Hat RHSA-2015:1139-01 kernel-rt 2015-06-23 Red Hat RHSA-2015:1137-01 kernel 2015-06-23 CentOS CESA-2015:1081 kernel 2015-06-10 Scientific Linux SLSA-2015:1081-1 kernel 2015-06-09 Oracle ELSA-2015-1081 kernel 2015-06-09 Red Hat RHSA-2015:1082-01 kernel 2015-06-09 Red Hat RHSA-2015:1081-01 kernel 2015-06-09 Debian DSA-3290-1 kernel 2015-06-18 Debian-LTS DLA-246-1 linux-2.6 2015-06-17 Scientific Linux SLSA-2015:1042-1 kernel 2015-06-02 CentOS CESA-2015:1042 kernel 2015-06-03 Red Hat RHSA-2015:1120-01 kernel 2015-06-16 Oracle ELSA-2015-1042 kernel 2015-06-03 Red Hat RHSA-2015:1042-01 kernel 2015-06-02 Debian-LTS DLA-246-2 linux-2.6 2015-06-17

Comments (none posted)

libinfinity: incorrect validation of certificates

Package(s):	libinfinity	CVE #(s):	CVE-2015-3886
Created:	June 2, 2015	Updated:	June 3, 2015
Description:	It was reported that libinfinity did not correctly validate certificates: Debian bug report Red Hat bugzilla entry
Alerts:	Fedora FEDORA-2015-8345 libinfinity 2015-06-01 Fedora FEDORA-2015-8396 libinfinity 2015-06-01 Fedora FEDORA-2015-8316 libinfinity 2015-06-01

Comments (none posted)

mysql-connector-java: information disclosure

Package(s):	mysql-connector-java	CVE #(s):	CVE-2015-2575
Created:	June 1, 2015	Updated:	September 14, 2016
Description:	From the openSUSE advisory: Difficult to exploit vulnerability allows successful authenticated network attacks via multiple protocols. Successful attack of this vulnerability can result in unauthorized update, insert or delete access to some MySQL Connectors accessible data as well as read access to a subset of MySQL Connectors accessible data.
Alerts:	openSUSE openSUSE-SU-2016:2304-1 mysql-connector-java 2016-09-14 Debian DSA-3621-1 mysql-connector-java 2016-07-18 Debian-LTS DLA-526-1 mysql-connector-java 2016-06-25 Mageia MGASA-2015-0255 mysql-connector-java 2015-07-05 openSUSE openSUSE-SU-2015:0967-1 mysql-connector-java 2015-05-29

Comments (none posted)

netty: HttpOnly cookie bypass

Package(s):	netty	CVE #(s):	CVE-2015-2156
Created:	June 1, 2015	Updated:	June 5, 2015
Description:	From the Red Hat bugzilla: A flaw was found in the way Netty’s CookieDecoder method validated cookie name and value characters. An attacker could use this flaw to bypass the httpOnly flag on sensitive cookies.
Alerts:	Fedora FEDORA-2015-8684 netty 2015-05-30 Fedora FEDORA-2015-8713 netty 2015-06-04

Comments (none posted)

nss: cipher-downgrade attacks

Package(s):	nss	CVE #(s):	CVE-2015-4000
Created:	June 2, 2015	Updated:	September 8, 2016
Description:	From the CVE entry: The TLS protocol 1.2 and earlier, when a DHE_EXPORT ciphersuite is enabled on a server but not on a client, does not properly convey a DHE_EXPORT choice, which allows man-in-the-middle attackers to conduct cipher-downgrade attacks by rewriting a ClientHello with DHE replaced by DHE_EXPORT and then rewriting a ServerHello with DHE_EXPORT replaced by DHE, aka the "Logjam" issue.
Alerts:	Debian DSA-3688-1 nss 2016-10-05 openSUSE openSUSE-SU-2016:2267-1 libtcnative-1-0 2016-09-08 Gentoo 201701-46 nss 2017-01-19 Arch Linux ASA-201601-33 lib32-openssl 2016-01-29 Arch Linux ASA-201601-32 openssl 2016-01-29 SUSE SUSE-SU-2016:0262-1 openldap2 2016-01-27 openSUSE openSUSE-SU-2016:0261-1 openldap2 2016-01-27 openSUSE openSUSE-SU-2016:0255-1 openldap2 2016-01-27 SUSE SUSE-SU-2016:0224-1 openldap2 2016-01-25 openSUSE openSUSE-SU-2016:0226-1 openldap2 2016-01-25 Gentoo 201512-10 firefox 2015-12-30 Debian-LTS DLA-507-1 nss 2016-06-07 Gentoo 201603-11 oracle-jre-bin 2016-03-12 openSUSE openSUSE-SU-2016:0483-1 socat 2016-02-17 openSUSE openSUSE-SU-2016:0478-1 socat 2016-02-16 Oracle ELSA-2015-2617 openssl 2015-12-14 openSUSE openSUSE-SU-2015:1684-1 apache2 2015-10-06 SUSE SUSE-SU-2015:1663-1 haproxy 2015-10-01 SUSE SUSE-SU-2015:1581-1 openssh 2015-09-21 Debian DSA-3339-1 openjdk-6 2015-08-19 SUSE SUSE-SU-2015:1375-1 java-1_7_0-ibm 2015-08-12 Ubuntu USN-2706-1 openjdk-6 2015-08-06 SUSE SUSE-SU-2015:1345-1 java-1_6_0-ibm 2015-08-05 Red Hat RHSA-2015:1544-01 java-1.5.0-ibm 2015-08-04 Scientific Linux SLSA-2015:1526-1 java-1.6.0-openjdk 2015-08-03 Oracle ELSA-2015-1526 java-1.6.0-openjdk 2015-07-31 Debian DSA-3324-1 icedove 2015-08-01 SUSE SUSE-SU-2015:1331-1 java-1_7_1-ibm 2015-07-31 SUSE SUSE-SU-2015:1329-1 java-1_7_1-ibm 2015-07-31 SUSE SUSE-SU-2015:1320-1 java-1_7_0-openjdk 2015-07-30 SUSE SUSE-SU-2015:1319-1 java-1_7_0-openjdk 2015-07-30 Oracle ELSA-2015-1526 java-1.6.0-openjdk 2015-07-30 Oracle ELSA-2015-1526 java-1.6.0-openjdk 2015-07-30 CentOS CESA-2015:1526 java-1.6.0-openjdk 2015-07-30 CentOS CESA-2015:1526 java-1.6.0-openjdk 2015-07-30 Red Hat RHSA-2015:1526-01 java-1.6.0-openjdk 2015-07-30 Debian-LTS DLA-303-1 openjdk-6 2015-08-28 Ubuntu USN-2696-1 openjdk-7 2015-07-30 openSUSE openSUSE-SU-2015:1289-1 java-1_8_0-openjdk 2015-07-26 openSUSE openSUSE-SU-2015:1288-1 java-1_7_0-openjdk 2015-07-26 Mageia MGASA-2015-0280 java-1.8.0-openjdk 2015-07-27 Debian DSA-3316-1 openjdk-7 2015-07-25 Red Hat RHSA-2015:1488-01 java-1.7.0-ibm 2015-07-23 Mageia MGASA-2015-0277 java-1.7.0-openjdk 2015-07-23 Red Hat RHSA-2015:1485-01 java-1.7.1-ibm 2015-07-22 Red Hat RHSA-2015:1486-01 java-1.6.0-ibm 2015-07-22 openSUSE openSUSE-SU-2015:1277-1 libressl 2015-07-22 Arch Linux ASA-201507-16 jre7-openjdk 2015-07-22 SUSE SUSE-SU-2015:1509-1 java-1_6_0-ibm 2015-09-08 SUSE SUSE-SU-2015:1449-1 MozillaFirefox, mozilla-nss 2015-08-28 Ubuntu USN-2673-1 thunderbird 2015-07-20 SUSE SUSE-SU-2015:1268-2 firefox, nspr, nss 2015-07-20 SUSE SUSE-SU-2015:1268-1 firefox, nspr, nss 2015-07-20 SUSE SUSE-SU-2015:1269-1 firefox, nspr, nss 2015-07-20 openSUSE openSUSE-SU-2015:1266-1 firefox, thunderbird 2015-07-18 Oracle ELSA-2015-1230 java-1.7.0-openjdk 2015-07-16 Red Hat RHSA-2015:1241-01 java-1.8.0-oracle 2015-07-17 Red Hat RHSA-2015:1242-01 java-1.7.0-oracle 2015-07-17 Red Hat RHSA-2015:1243-01 java-1.6.0-sun 2015-07-17 Scientific Linux SLSA-2015:1228-1 java-1.8.0-openjdk 2015-07-15 Scientific Linux SLSA-2015:1229-1 java-1.7.0-openjdk 2015-07-15 Scientific Linux SLSA-2015:1230-1 java-1.7.0-openjdk 2015-07-15 CentOS CESA-2015:1228 java-1.8.0-openjdk 2015-07-15 CentOS CESA-2015:1228 java-1.8.0-openjdk 2015-07-15 CentOS CESA-2015:1230 java-1.7.0-openjdk 2015-07-15 CentOS CESA-2015:1229 java-1.7.0-openjdk 2015-07-15 CentOS CESA-2015:1229 java-1.7.0-openjdk 2015-07-15 Ubuntu USN-2656-2 firefox 2015-07-15 Red Hat RHSA-2015:1228-01 java-1.8.0-openjdk 2015-07-15 Red Hat RHSA-2015:1230-01 java-1.7.0-openjdk 2015-07-15 Red Hat RHSA-2015:1229-01 java-1.7.0-openjdk 2015-07-15 SUSE SUSE-SU-2015:1177-2 MySQL 2015-07-10 openSUSE openSUSE-SU-2015:1229-1 Firefox, nss 2015-07-13 Ubuntu USN-2656-1 firefox 2015-07-09 openSUSE openSUSE-SU-2015:1216-1 MariaDB 2015-07-09 openSUSE openSUSE-SU-2015:1209-1 mysql-community-server 2015-07-08 SUSE SUSE-SU-2015:1181-2 OpenSSL 2015-07-03 SUSE SUSE-SU-2015:1184-2 OpenSSL 2015-07-03 SUSE SUSE-SU-2015:1183-2 OpenSSL 2015-07-03 SUSE SUSE-SU-2015:1182-2 OpenSSL 2015-07-03 Mageia MGASA-2015-0268 firefox 2015-07-05 Mageia MGASA-2015-0260 filezilla 2015-07-05 SUSE SUSE-SU-2015:1185-1 OpenSSL 2015-07-03 SUSE SUSE-SU-2015:1184-1 OpenSSL 2015-07-03 SUSE SUSE-SU-2015:1183-1 OpenSSL 2015-07-03 SUSE SUSE-SU-2015:1182-1 OpenSSL 2015-07-03 SUSE SUSE-SU-2015:1181-1 OpenSSL 2015-07-03 SUSE SUSE-SU-2015:1177-1 MySQL 2015-07-02 CentOS CESA-2015:1197 openssl 2015-07-02 Scientific Linux SLSA-2015:1197-1 openssl 2015-06-30 Oracle ELSA-2015-1197 openssl 2015-06-30 Scientific Linux SLSA-2015:1185-1 nss 2015-06-29 Red Hat RHSA-2015:1197-01 openssl 2015-06-30 SUSE SUSE-SU-2015:1143-1 openssl 2015-06-25 SUSE SUSE-SU-2015:1150-1 compat-openssl098 2015-06-26 Oracle ELSA-2015-1185 nss 2015-06-25 Oracle ELSA-2015-1185 nss 2015-06-25 openSUSE openSUSE-SU-2015:1139-1 openssl 2015-06-25 CentOS CESA-2015:1185 nss 2015-06-25 CentOS CESA-2015:1185 nss 2015-06-25 CentOS CESA-2015:1185 nss 2015-06-25 CentOS CESA-2015:1185 nss 2015-06-25 Red Hat RHSA-2015:1185-01 nss 2015-06-25 Gentoo 201506-02 openssl 2015-06-22 Mageia MGASA-2015-0246 openssl 2015-06-19 Debian DSA-3287-1 openssl 2015-06-13 Fedora FEDORA-2015-9130 nss-softokn 2015-06-01 Red Hat RHSA-2015:1072-01 openssl 2015-06-04 Ubuntu USN-2624-1 openssl 2015-06-01 Debian DSA-3300-1 iceweasel 2015-07-04 Fedora FEDORA-2015-9130 nss-util 2015-06-01 Fedora FEDORA-2015-9130 nss 2015-06-01 Fedora FEDORA-2015-9161 nss-util 2015-06-14 Fedora FEDORA-2015-9161 nss-softokn 2015-06-14 Fedora FEDORA-2015-9161 nss 2015-06-14 Arch Linux ASA-201506-3 openssl 2015-06-12 Oracle ELSA-2015-1072 openssl 2015-06-04 Oracle ELSA-2015-1072 openssl 2015-06-04 CentOS CESA-2015:1072 openssl 2015-06-04 Fedora FEDORA-2015-9048 nss-util 2015-06-02 Debian-LTS DLA-247-1 openssl 2015-06-17 Scientific Linux SLSA-2015:1072-1 openssl 2015-06-05 CentOS CESA-2015:1072 openssl 2015-06-04 Fedora FEDORA-2015-9048 nss-softokn 2015-06-02 Fedora FEDORA-2015-9048 nss 2015-06-02

Comments (none posted)

python-django: incorrect session flushing

Package(s):	python-django	CVE #(s):	CVE-2015-3982
Created:	June 1, 2015	Updated:	June 3, 2015
Description:	From the Red Hat bugzilla: The following flaw was found in Django 1.8: A change to session.flush() in the cached_db session backend in Django 1.8 mistakenly sets the session key to an empty string rather than None. An empty string is treated as a valid session key and the session cookie is set accordingly. Any users with an empty string in their session cookie will use the same session store. session.flush() is called by django.contrib.auth.logout() and, more seriously, by django.contrib.auth.login() when a user switches accounts. If a user is logged in and logs in again to a different account (without logging out) the session is flushed to avoid reuse. After the session is flushed (and its session key becomes '') the account details are set on the session and the session is saved. Any users with an empty string in their session cookie will now be logged into that account.
Alerts:	Fedora FEDORA-2015-8691 python-django 2015-05-30

Comments (none posted)

symfony: restriction bypass

Package(s):	symfony	CVE #(s):	CVE-2015-4050
Created:	June 1, 2015	Updated:	June 8, 2015
Description:	From the Debian advisory: Jakub Zalas discovered that Symfony, a framework to create websites and web applications, was vulnerable to restriction bypass. It was affecting applications with ESI or SSI support enabled, that use the FragmentListener. A malicious user could call any controller via the /_fragment path by providing an invalid hash in the URL (or removing it), bypassing URL signing and security rules.
Alerts:	Fedora FEDORA-2015-9039 php-symfony 2015-06-06 Fedora FEDORA-2015-9025 php-symfony 2015-06-05 Fedora FEDORA-2015-9034 php-symfony 2015-06-06 Debian DSA-3276-1 symfony 2015-05-31

Comments (none posted)

tomcat6: Security Manager bypass

Package(s):	tomcat6	CVE #(s):	CVE-2014-7810
Created:	May 29, 2015	Updated:	March 24, 2016
Description:	From the Apache advisory: Malicious web applications could use expression language to bypass the protections of a Security Manager as expressions were evaluated within a privileged code section. This issue only affects installations that run web applications from untrusted sources.
Alerts:	Scientific Linux SLSA-2016:2046-1 tomcat 2016-10-11 CentOS CESA-2016:2046 tomcat 2016-10-11 Oracle ELSA-2016-2046 tomcat 2016-10-10 Red Hat RHSA-2016:2046-01 tomcat 2016-10-10 Debian DSA-3447-1 tomcat7 2016-01-17 Debian DSA-3530-1 tomcat6 2016-03-25 CentOS CESA-2016:0492 tomcat6 2016-03-23 Scientific Linux SLSA-2016:0492-1 tomcat6 2016-03-23 Oracle ELSA-2016-0492 tomcat6 2016-03-22 Red Hat RHSA-2016:0492-01 tomcat6 2016-03-22 Debian DSA-3428-1 tomcat8 2015-12-18 Ubuntu USN-2654-1 tomcat7 2015-06-25 Ubuntu USN-2655-1 tomcat6 2015-06-25 Debian-LTS DLA-232-1 tomcat6 2015-05-28

Comments (none posted)

virtio-win: denial of service

Package(s):	virtio-win	CVE #(s):	CVE-2015-3215
Created:	June 3, 2015	Updated:	June 3, 2015
Description:	From the Red Hat advisory: It was found that the Windows Virtio NIC driver did not sufficiently sanitize the length of the incoming IP packets, as demonstrated by a packet with IP options present but the overall packet length not being adjusted to reflect the length of those options. A remote attacker able to send a specially crafted IP packet to the guest could use this flaw to crash that guest.
Alerts:	Red Hat RHSA-2015:1043-01 virtio-win 2015-06-03 Red Hat RHSA-2015:1044-01 virtio-win 2015-06-03

Comments (none posted)

Kernel release status

The current development kernel is 4.1-rc6, released on May 31. As Linus put it: "things look normal".

Stable updates: none have been released in the last week.

Comments (none posted)

Linux support for digital video broadcasting

Mauro Carvalho Chehab, the maintainer of the kernel's media subsystem, has posted the first two in a series of articles on digital video broadcasting support in Linux. Part 1 gives an overview of how the devices and protocols work, while part 2 looks at digital TV network interface use. "Supporting embedded Digital TV hardware is complex, considering that such hardware generally has multiple components that can be rewired in runtime to dynamically change the stream pipelines and provide flexibility for things like recording a video stream, then tuning into another channel to see a different program. This article describes how the DVB pipelines are setup and the needs that should be addressed by the Linux Kernel."

Comments (76 posted)

4.1 development statistics

The 4.1-rc6 prepatch is out, and things look on track for an on-schedule 4.1 release — unless, of course, Linus's vacation plans get in the way. But the kernel development community doesn't stop, even when Linus does, so it seems like about time to have a look at some development statistics for this kernel cycle. 4.1 was a fairly typical development cycle with few surprises.

With 11,664 changesets merged as of 4.1-rc6, this cycle is a little bit slower than most that have been seen in the last year, though it is significantly busier than 4.0, which finished with 10,346 changesets. The number of developers involved, at 1,492, comfortably exceeds 4.0; indeed, it currently ties 3.15 for the title of the most developers ever. Chances are good that, by the time it is released, the 4.1 kernel cycle will be the first to see the participation of over 1,500 developers. Of those over 1,500 developers, 270 have contributed their first patch ever in this cycle — a fairly typical number for recent years.

Those developers have added 486,000 lines of code and removed 286,000 lines, for a net growth of 200,000 lines this time around. The most active developers doing this work were:

Most active 4.1 developers

By changesets Ian Abbott 129 1.1% Takashi Iwai 121 1.0% Hans Verkuil 119 1.0% Marcel Holtmann 117 1.0% Aya Mahfouz 107 0.9% Geert Uytterhoeven 105 0.9% Laurent Pinchart 102 0.9% Richard Weinberger 95 0.8% Joe Perches 92 0.8% Eric Dumazet 92 0.8% Al Viro 90 0.8% Krzysztof Kozlowski 77 0.7% Fabian Frederick 77 0.7% Benjamin Romer 74 0.6% Jiri Olsa 73 0.6% Denys Vlasenko 72 0.6% Mauro Carvalho Chehab 67 0.6% Nicholas Mc Guire 66 0.6% Guenter Roeck 65 0.6% Lars-Peter Clausen 65 0.6%

By changed lines Jie Yang 20194 3.4% Stephen Boyd 13536 2.3% Sudip Mukherjee 10198 1.7% Chanwoo Choi 8571 1.5% Heiko Carstens 8239 1.4% Tomeu Vizoso 7647 1.3% Hongzhou Yang 7391 1.3% Joe Perches 7135 1.2% Laurent Pinchart 6589 1.1% J. German Rivera 6359 1.1% Takashi Iwai 6173 1.0% Magnus Damm 6082 1.0% Mathieu Poirier 5915 1.0% Michael Ellerman 5874 1.0% Ray Jui 5362 0.9% Andy Shevchenko 4857 0.8% Hai Li 4487 0.8% Andrew Bresticker 4252 0.7% Markus Stockhausen 4221 0.7% James Hogan 4172 0.7%

Hartley Sweeten no longer tops out the list with his Comedi driver work but, never fear, Ian Abbott contributed the most 4.1 changesets by working on, yes, the Comedi drivers. Remember, though, that Hartley contributed 463 Comedi changes in 3.19, so it may well be that work on this code is finally slowing down, though there doesn't appear to be a plan to move it out of the staging tree quite yet. Takashi Iwai is the sound subsystem maintainer, so most of his work concentrates in that area; for similar reasons, it is unsurprising that Hans Verkuil's changes were all found within the media subsystem and Marcel Holtmann's patches were applied to the Bluetooth code. Aya Mahfouz, instead, is an intern in the Outreachy project's current round; she has clearly gotten off to a strong start with a lot of cleanup patches applied to staging drivers.

On the "lines changed" side, Jie Yang's work was mostly a reorganization of the Intel audio driver code. Stephen Boyd removed some old ARM drivers, making him the developer having removed the most code in this development cycle. Sudip Mukherjee did a bunch of cleanupv work on a number of staging drivers, Chanwoo Choi worked mostly on the Samsung Exynos clock drivers, and Heiko Carstens removed a bunch of relatively obscure S/390 code — such as 31-bit support.

There were (at least) 215 employers supporting work on the 4.1 kernel, the most active of which were:

Most active 4.1 employers

By changesets Intel 1308 11.2% Red Hat 1069 9.2% (None) 1055 9.0% (Unknown) 950 8.1% SUSE 437 3.7% Linaro 387 3.3% IBM 385 3.3% Outreachy 381 3.3% Google 362 3.1% Samsung 340 2.9% Renesas Electronics 279 2.4% (Consultant) 258 2.2% Texas Instruments 217 1.9% Broadcom 162 1.4% Oracle 155 1.3% Imagination Technologies 151 1.3% Cisco 150 1.3% Freescale 134 1.1% MEV Limited 129 1.1% ARM 129 1.1%

By lines changed Intel 74566 12.6% Red Hat 41496 7.0% (None) 40119 6.8% IBM 39301 6.7% (Unknown) 31558 5.3% Linaro 29588 5.0% Code Aurora Forum 23495 4.0% Samsung 22175 3.8% Google 21588 3.7% Renesas Electronics 17548 3.0% SUSE 16830 2.9% Broadcom 15202 2.6% Freescale 15156 2.6% Imagination Technologies 10935 1.9% VECTOR Institute 10742 1.8% Nokia 9829 1.7% MediaTek 9582 1.6% Texas Instruments 8843 1.5% Collabora Multimedia 8621 1.5% (Consultant) 8312 1.4%

As usual, there are few surprises here, with the possible exception of the 3.3% of the changesets contributed in this cycle by current and aspiring Outreachy interns.

The Signed-off-by tags in a patch provide a picture of who handled the patch on its way into the appropriate subsystem maintainer's tree. In particular, if one looks at tags applied by developers other than the author of each patch, the result gives a view of who the gatekeepers are. For the 4.1 development cycle, the numbers look like this:

Most non-author signoffs in 4.1
By Developer Greg Kroah-Hartman 1544 13.8% David S. Miller 1067 9.6% Ingo Molnar 407 3.6% Mark Brown 405 3.6% Andrew Morton 404 3.6% Daniel Vetter 360 3.2% Mauro Carvalho Chehab 342 3.1% Ralf Baechle 263 2.4% Arnaldo Carvalho de Melo 242 2.2% Kalle Valo 210 1.9%	By employer Red Hat 2249 20.2% Linux Foundation 1568 14.1% Intel 1327 11.9% Linaro 981 8.8% Google 621 5.6% Samsung 521 4.7% SUSE 375 3.4% (None) 316 2.8% (Unknown) 314 2.8% IBM 286 2.6%

While this development cycle is the result of the work of 1,500 developers and over 200 companies, at the subsystem maintainer level things are, as always, far more concentrated: over 60% of the changes going into this kernel passed through the hands of developers working for just five companies. This concentration reflects a simple fact: while many companies are willing to support developers working on specific tasks, the number of companies supporting subsystem maintainers is far smaller. Subsystem maintainership is also, increasingly, not a job for volunteer developers.

And that is the story for the 4.1 development cycle. The kernel-development machine continues to hum along, integrating the work of thousands of developers and producing the kernels that run Linux systems worldwide. There are not many surprises to be seen here, but, for such an important piece of software, "not many surprises" is generally deemed to be a good thing.

Comments (5 posted)

Reinventing the timer wheel

The kernel's "timer wheel" data structure has served it well for some time; it has changed little since it was described in this article in 2005. There are, however, some shortcomings in its original design that have become more costly over time, and the timer wheel has not adapted well to other changes in the scheduler code. So, after many years, this venerable data structure may soon be replaced with a variant that runs more efficiently, but loses some accuracy in timekeeping in the process.

The kernel maintains two types of timers with two distinct use cases. The high-resolution timer ("hrtimer") mechanism provides accurate timers for work that needs to be done in the near future; hrtimer use is relatively rare, but, when hrtimers are used, they almost always run to completion. "Timeouts," instead, are normally used to alert the kernel to an expected event that has failed to arrive — a missing network packet or I/O completion interrupt, for example. The accuracy requirements for these timers are less stringent (it doesn't matter if an I/O timeout comes a few milliseconds late), and, importantly, these timers are usually canceled before they expire. The timer wheel is used for the latter variety of timers.

Here is the 2005 diagram showing the design of the timer wheel:

This data structure is indexed by the kernel's low-resolution "jiffies" clock; one jiffy corresponds to something between 1ms and 10ms, depending on how the kernel is configured. Once every jiffy, the kernel processes any expired timers. That is done by taking the lowest eight bits of the jiffies variable and using them to index into the rightmost array in the above diagram; the result will be a linked list of timer events that expire at the current time.

Every 256 jiffies (in most configurations) the kernel will hit the end of that array; at that point it is necessary to perform a "cascade" operation. Each entry in the next higher array contains 256 jiffies worth of events; the timer code will select the correct entry (by using the next six bits of the jiffies value as an index), collect all of the timer entries found there, and distribute them across the 256 entries of the first array according to their expiration times. When the second level is exhausted, it is refilled by cascading down entries from the third level, and so on.

There are a number of advantages to this data structure, including the ability to immediately locate expired entries and quick addition and removal of events. But it has some downsides as well. The cascade operation can be expensive, and the time required is, to a first approximation, unpredictable; that can lead to unwanted latencies elsewhere in the system. The cascade operation is not particularly cache-friendly. There is also no way to quickly determine when the next timer expiration will happen; that requires searching through the wheel to actually find that event. The presence of deferrable timers, which do not have to expire in any sort of timely manner, makes the identification of the next event that actually does have to expire at the requested time harder yet. For these reasons and more, developers have talked about replacing the timer wheel for years.

The new timer wheel

Thomas Gleixner has now posted a first draft of a reinvented timer wheel. It does away with the costly cascade operations (almost all the time) and handles deferrable timers in a much more straightforward manner. These gains come from the realization that not all timers have to be handled with the same level of accuracy.

At a superficial level, the new data structure is quite similar to the old. There is still a hierarchy of arrays containing lists of timer events. In this case, though, the arrays are all the same size (32 entries), and there are eight levels of them. The lowest array contains events with single-jiffy resolution as before, so any new timeout expiring less than 32 jiffies in the future will be placed in this array.

The next array is a little different, though; each entry represents eight jiffies worth of future timer events. Since there are 32 entries in this level as well, it can represent events up to 256 jiffies into the future. Entries in the third level each hold 64 jiffies worth of events; in the fourth level, they hold 512 jiffies worth, and so on. So each level covers a time period eight times longer than the level below it. The numbers are different from the old implementation, but the concept is the same, so far.

The old timer wheel would, each jiffy, run any expiring timers found in the appropriate entry in the highest-resolution array. It would then cascade entries downward if need be, spreading out the timer events among the higher-resolution entries of each lower-level array. The new code also runs the timers from the highest-resolution array in the same way, but the handling of the remaining arrays is different. Every eight jiffies, the appropriate entry in the second-level array will be checked, and any events found there will be expired. The same thing happens in the third level every 64 jiffies, in the fourth level every 512 jiffies, and so on.

The end result is that the timer events in the higher-level arrays will be expired in place, rather than being cascaded downward. This approach, obviously, saves all the effort of performing the cascade. But it also means that any timeout that is more than 31 jiffies in the future will be run with lower accuracy. For example, a timeout that is 36 jiffies in the future will be put in the next higher eight-jiffy slot — 40 jiffies in the future. So that event will expire four jiffies later than requested. As timeouts are placed further into the future, the accuracy of their expiration will decline accordingly. The seventh level in this scheme will hold timeouts that are at least 1,048,576 jiffies in the future with 262,144-jiffy resolution. On a 1000HZ system, that corresponds to timeouts at least 17 minutes in the future; they will expire with a resolution of four minutes.

The old implementation was not subject to this loss of accuracy; even timeouts days in the future would expire at "exactly" the right time, for a one-jiffy value of "exactly." So one could argue that the replacement timer wheel does not work as well. But, first, one should remember that (1) almost all timeouts are set for the near future, (2) almost all timeouts are canceled before expiration, and (3) timeouts indicate that something went wrong and do not need to be delivered with a high degree of accuracy. So sacrificing some of that accuracy for higher timer-wheel performance would appear to be a good tradeoff.

Thomas's patch also dispenses with the timer slack mechanism. Timer slack allows the expiration of timeouts to be deferred; it is intended to cause timeouts to be executed together and reduce the number of times the system wakes up. The new timer wheel batches things naturally for anything but the shortest of timeouts, so there is arguably no longer a need for a separate "slack" mechanism.

Deferrable timers are a bit different though; they can be deferred indefinitely if need be. They usually correspond to some sort of cleanup work that must be done eventually, but with no particular urgency. If the CPU is running in the tickless mode, those timeouts should be deferred for as long as it takes to avoid interrupting the running application. In Thomas's patch, deferrable timers are stored in a separate, parallel timer wheel; this gets them out of the way and eases the task of figuring out when the next timer interrupt should be scheduled.

The new timer wheel code maintains a bitmap with a bit corresponding to each entry in the timer arrays; if there are timeouts stored in that entry, that bit is set. Finding the first array entry with an outstanding timeout is thus a simple matter of finding the first set bit in the bitmap — a fast operation. Then, since the expiration time of each array entry is known, the time of the next expiring timeout can be calculated without actually needing to look at the timeout entry. Placing deferrable timeouts in their own array makes it easy to simply avoid looking at them when checking for this next expiring timeout, speeding the operation further.

This code is all new and untried; Thomas warns that "it might eat your disk, kill your cat and make your kids miss the bus". That would suggest that it is certainly not considered to be 4.2 material. But, with some time and testing, it could likely be ready for a development cycle shortly after that. Then the kernel will, at last, have a shiny, new, faster timer wheel.

Comments (6 posted)

A tour of /sys/devices

Modern Linux systems have a directory tree at /sys/devices that contains information about all of the "devices" represented in the device model. Having clarified exactly what each of "devices", "buses", and "classes" are (or maybe what they aren't), I am now in a position to address a shortcoming I had to admit in a recent article where I stated that /sys/devices "has a structure that, in all honesty, is rather hard to describe".

/sys/devices is part of the sysfs virtual filesystem; it presents devices as directories arranged in a hierarchy. It includes various files to allow details of each device to be examined and sometimes to be changed. Seeing how they all plug together can give insights into the device model that are not evident by just examining individual devices. To describe this directory structure in detail there are three particular concepts we need to examine beyond the device, bus, and class that we met last time. We need to explore parentage, attributes, and namespace management.

Parenthood

Each device (or "thing") in the Linux device model can have a "parent" device. In the examples from last week, the "tca6507" device was the parent of some "leds" devices, and itself had a parent that was an i2c adapter device, such as "i2c-1". The parent of the device representing a partition on a hard drive is the device representing the hard drive as a whole. "Workqueue" devices do not have parents. This variety leads to the question of what, exactly, is a "parent". There are several different, though related, concepts that are tied up in this idea of "parenthood".

Parent as connection point or service provider

Firstly there is the idea of "addressability" and the related idea of "connectedness". When a device represents a piece of hardware, the parent often represents a connection to that device. This parent knows how to interpret the "address" of the device, and how to send instructions to that address, or how to transfer data to or from that address. The parent doesn't know much about what those instructions or data might do; that knowledge is in the device itself (and its driver).

Closely related to connection is the idea of service provision. When a device represents a piece of hardware, the primary service it needs from a parent device is communication with the CPU. When a device represents something else, the primary service requirement might be different.

Many devices that are implemented by classes do not exactly represent hardware but instead represent an abstraction. Devices in the "disk" class represent anything that can provide read or write access to addressable blocks of storage. Any device that can facilitate that access might reasonably find itself as the parent of a "disk" device. Sometimes the thing that provides service is not a device — it might be a file or a network connections or a set of devices. In those cases the disk device won't have a parent.

I like to picture these two complementary ideas (connection and service provision) by imagining the path a request takes from user space all the way to ultimate physical reality. A request such as "turn the LED on" or "read that block of data", enters the device hierarchy at one of the leaf devices, possibly by a write() or ioctl() on a device node in /dev or by a write to a sysfs attribute file. The request then propagates up the tree, being translated at each stage as is appropriate to each level. Some levels might add extra detail to the address, some levels might wrap "protocol headers" around the message, or divide the message up into components. Another level might add a retry mechanism or might impose a media-access protocol.

Once the message gets to the top of the hierarchy it is transmitted on a core system bus (probably via writing to one or more memory addresses), at which point the request travels through the hardware from controller to controller in a rough analog of traveling back down the tree of devices and out to the physical leaves that are the hardware that will actually pass current through an LED, or read the data off some sort of media.

To make this more concrete, consider an leds device with the "default-on" trigger selected, and then consider writing "100" to the "brightness" file, which implies a request to set the brightness to 100/255 of the maximum brightness (assuming that max_brightness is 255). The full path to that file might be:

    /sys/devices/platform/omap_i2c.2/i2c-2/2-0045/leds/gta04:green:power/brightness  

The trigger driver leaves that request for the leds core to handle; the core determines whether one of two possible interfaces (brightness_set() or brightness_set_sync()) is available and calls it if so; otherwise it returns an error. When an leds device is registered by a call to led_classdev_register(), a struct led_classdev must be passed; it contains various function pointers and other settings that provide access to the underlying mechanism. So the leds driver turns the "set brightness" request into one of two possible function calls into the parent device.

If that parent device is the TCA6507 we discussed previously (named "2-0045" above) then tca6507_brightness_set() is called. The TCA6507 does not provide completely independent control of the seven output pins. Apart from fully on and fully off, there are at most three distinct brightness levels from a set of 16 possible levels. If the request level matches a level already configured, that can be assigned to the appropriate pin. If not, and there are fewer than three in use, then the new level can be programmed to a free slot. If all brightness slots are in use, the closest to the requested value is used.

In any case, the decisions need to be sent to the TCA6507 which, like most simple I2C devices, appears to contain a set of registers. So the leds-tca6507 driver converts the brightness_set() request into one or more "write to register" requests. Those requests must, in turn, be passed to the device via an I2C bus.

When an I2C driver is attached to a device, its probe() function is called with a struct i2c_client parameter. When the driver wants to write to a register it calls i2c_smbus_write_byte_data() or a similar function, giving it the same i2c_client handle, the register address, and the data. "smbus" refers to "System Management Bus" which is a subset of the I2C protocol — drivers that only need that subset are encouraged to use the smbus interfaces.

The i2c/smbus code (implementing device "i2c-2") takes the "register write" request and converts that into a message to transmitted on the i2c bus; it contains a command, an address, and data. The underlying I2C adapter driver can provide a couple of different functions, such as smbus_xfer() or master_xfer(), the latter being a lower-level function that needs more support in the I2C bus code. These functions may be called repeatedly to effect retry on certain failure modes, and are only called after i2c_lock_adapter() has successfully gained exclusive access to the bus.

So now those "write to register" requests have become "message transfer" calls with locking and retry.

If that underlying I2C adapter is the OMAP I2C adapter ("omap_i2c.2"), it will ensure the clocks for the adapter are enabled, make sure it isn't being held in reset, and wait to make sure the bus isn't being used by some other bus master. The driver will then load the size and address of each message into the DMA engine of the adapter so that it can copy the messages out onto the bus. This is all done by writing directly to registers in the I2C adapter using the system memory bus.

At this point the message hasn't quite reached the top of the /sys/devices hierarchy; that step brings in the "platform" device, which represents the whole platform and the main system memory bus, used by the I2C driver to talk to the I2C adapter. The memory bus driver doesn't get involved in copying bytes out, but it was involved when the devices were registered in that it effectively told the I2C driver what address to use to talk to its hardware.

The memory bus carries these writes to the I2C adapter, which fetches the messages from memory and serializes them out onto the I2C bus. The TCA6507 collects the bits off the bus, places them in the addressed register, and then loads internal counters as appropriate so that it can turn the particular pin on and off with the right duty cycle to achieve the requested brightness (or something close to it). This pulls current through the LED so that it generates light some of the time, and doesn't for the rest of the time. Our eyes, which naturally integrate those high frequency changes, see something close to the requested brightness level. It seems like a long journey, but it gets from:

    echo 100 > brightness

all the way to my eyes faster than I can blink.

While this is a helpful picture, it is a somewhat idealized picture. It is important to be aware that the "primary" service provider is not necessarily the only service provider. A specific LED can be configured to blink whenever some specific battery other than the primary is charging, for example. So both the LED and the charge monitoring hardware provide services to the "power supply" device.

In most cases where there are multiple services being provided, there will be one that is fairly obviously primary. Other times there is not. A RAID array receives service of equal value from a number of different block devices. And it could stop receiving that service at any moment if one fails. So there is no clear primary service provider for a RAID array, and indeed md/raid devices do not have a parent.

Parent as discoverer

The third idea is again related to the others but provides a helpful alternate perspective. It is "discovery". Every device must be deliberately added to the device model; this process is referred to a "device discovery". When driver code, acting on behalf of a particular device, discovers another device, it needs to assign a parent to the new device; it will typically use the current device as that parent.

Sometimes this discovery is performed by probing a (physical) bus to see what devices are connected to it. A similar situation happens when a bus supports hotplug and the bus controller reports that a new device has been attached. In these cases, the device that detects the new device is almost certainly the device that can address and send commands to the new device, so it is logical for it to be the parent.

Other times, this discovery happens by examining a configured description of available devices. This might be based on information extracted via a BIOS service such as ACPI, it might be encoded using a device tree, or it could be encoded directly in the kernel using arrays of "struct platform_device" and calls to platform_add_devices(), though that approach is deprecated. When ACPI is used, it will provide a top-level device similar to "platform", which is parent to the devices that are discovered and managed using ACPI, even though ACPI is not exactly a physical connection.

Parent as power source

Finally there is the idea of power management. As power management was apparently one of the driving forces that led to the creation of the device model and /sys/devices, you might expect that power management would be fairly central. The reality is ... more complex.

There are two sides to power management. One is system suspend or system hibernate, where the focus is putting as many devices as possible to the lowest power state. The other is runtime power management, where the focus is placing individual devices into a low power state whenever they aren't actively in use. It might seem reasonable to follow parent-child links in a depth-first order, turning off children and then their parents. But that isn't quite what is done.

In the case of system suspend, there is actually a completely separate structure that is used to determine the order for shutting down devices. As I described in 2012, the power-management code maintains a linear list of devices called "dpm_list" that is used to sequence suspend and resume. This list is created roughly in order of device discovery, which is roughly how the tree of devices grows out to the leaves. So processing dpm_list in order will normally visit parents before children. However power often isn't managed in exactly the same way as device addressing, so the power management system needs a little more control, which it gets by having its own list.

Runtime power management does make some use of parent/child relationships, but not always. Possibly not even often.

For connections that are designed to carry both power and control, such as USB, it makes lots of sense for the bus controller to remain powered on (and providing power) whenever an attached device is powered on. However it may well make sense for the bus controller to enter a low-power state while the device continues to do useful work, particularly if the device can trigger some sort of wakeup event.

For buses that do not carry power, such as i2c, it makes no sense to directly link the power state of the bus controller with the power state of the attached device. Typically an i2c controller will power down (stopping its clock) whenever it doesn't have any commands to send. When a device driver needs to talk to its device it will wake up the controller (its parent), send the message, and let the controller go back to sleep, even while the device itself is fully active (blinking those LEDs).

In the runtime power management system the default behavior is that, when a child device is powered on, its parent must be powered on as well. This default can be overridden by a call to pm_suspend_ignore_children(). Given that many buses do not carry power, I was at first surprised at how few calls there are to this function: in 4.1-rc1 there are only 14 calls that enable the ignoring of children.

Part of the reason for this small number is that the default as stated misses an important detail. It only applies to devices that actually support different runtime power-management states. If both a parent and child support runtime power management, then the default will keep the parent powered when the child is active. If either does not support runtime power management, then no connection is implied. This means that when a device wakes up, that wakeup will only propagate up the tree until it reaches a device that sets "ignore_children" or a device that doesn't support runtime power management at all. Then the propagation will stop.

There are actually quite a lot of devices that do not support runtime power management. Most devices implemented by classes do not.

    $ cat /sys/class/*/*/power/runtime_status | sort | uniq -c
    318 unsupported

Many devices implemented by buses do not either:

    $ cat /sys/bus/*/devices/*/power/runtime_status | sort | uniq -c
     57 active
     11 suspended
    274 unsupported

So while runtime power management can make use of the parent link, it doesn't to a great extent. The parent very often is not the source of power, only of addressing and control.

Despite the apparent richness of meaning we find in parenthood, the device model doesn't really use it much. As a message passes "up" the hierarchy, it doesn't follow the "->parent" links of the device model, but uses references that were provided to each device when it was initialized. The runtime power management code does make direct use of these ->parent links, but not very much. It could just as easily make use of explicit dependencies, just as it does when there are dependencies that are not reflected in the hierarchy.

Possibly the biggest user of these parent links is the "udev" utility, which responds to device addition and removal events in a highly configurable way. This configuration can depend on information about arbitrarily distant ancestors. On the other hand, it cannot depend at all on related devices that are not direct ancestors.

What role for attributes?

Attributes are arbitrary details about a given device; they are presented via small files in /sys/devices known as "attribute files". The particular content and use of these files was discussed here some years ago and has not significantly changed since so will not be addressed further.

Each device is presented by a directory in /sys/devices; most attribute files appear as files directly in those directories. There are exceptions, though, and while they are not particularly interesting, they need to be understood. These exceptions collect a group of attributes together into a subdirectory. The most prevalent such subdirectory is power, which contains attributes relating to the power status of the device. Every device has this subdirectory.

The second most prevalent on the machine on which I am typing are the capabilities and id subdirectories that various devices in the input class contain. Additionally, cpu devices can have a two-level attribute tree with state directories in a cpuidle directory. The important point here is that, when you find a directory in the devices tree, it might just be a collection of attributes. The reason why they have been collected will vary from case to case.

Note that "classes" and "buses" can also have attributes. These affect the whole subsystem and so are somewhat similar to module parameters. They do not appear in "/sys/device", only in "/sys/bus" or "/sys/class", so they are out of scope for this article.

Managing namespaces

While Linux does have the Linux Assigned Names And Numbers Authority, that authority does not oversee the assignment of names to classes, buses, devices, or attributes. In generally we just have to "play nice", try to catch inappropriate duplication during review, and fall back on the kernel complaining if a name gets used twice in one namespace.

Within /sys/devices there can often be names from distinct namespaces appearing in the one directory, so it is worth knowing what namespaces there are and what support there is for conflict avoidance. Firstly, there is a namespace for buses (/sys/bus) and another for classes (/sys/class). These are currently kept separate and are largely disjoint anyway. On my laptop I find:

    $ comm -12 <(ls /sys/bus) <(ls /sys/class)
    mei  

so only one name in common — which is more than I expected.

Each bus and each class defines a namespace for devices that are implemented by that bus or class. Typically a simple syntactic rule will distinguish between different types of devices in the bus. So in the i2c bus, devices that represent an adapter are "i2c-%d" while devices that represent attached hardware are "%d-%04x".

Each bus or class, and to some extent each driver, also defines a namespace of attributes. A driver must clearly not define attributes that conflict with names used by its bus, but different drivers on the same bus could safely use the same name (possibly for different purposes, though that wouldn't be wise). There is also a namespace of system-imposed attributes, of which we have already seen one example: the "power" set of attributes that is imposed on every device. Clearly subsystems and drivers must avoid these.

Reserving the single name "power" for all power management attributes makes it practical to add new power management attributes, such as pm_qos_latency_tolerance_us which was added a little over a year ago. This approach of reserving just one name as a directory to minimize namespace pollution is also used to some extent to reduce the risk of a device name in one subsystem conflicting with an identically named device in another subsystem. When this is done, a directory named for the subsystem is used to contain all relevant devices of that subsystem. But this is only a sometimes thing, as we will shortly see.

Bringing it all together

It seems easiest to describe the whole tree by starting in the middle. Every directory under /sys/devices that contains a file called "uevent" represents a device. This file can be written to in order to synthesize "ADD" events, "REMOVE" events, or other events that can be processed by udev. It can be read to show context information that accompanies those events.

In each device directory there is also a "power" directory of attributes, as we have already seen, and often a "subsystem" symlink that points to the owning subsystem in either /sys/bus or /sys/class.

While any device must be implemented by some subsystem, it may not actually be linked into the list of devices for that subsystem. For example, the "usb" subsystem creates "endpoint" and "port" devices but does not identify them as belonging to the "usb" subsystem; as a result they don't get the "subsystem" symbolic link and do not appear in "/sys/bus/usb/devices".

A device may also have a "device" link which is of legacy interest only and usually points to a recent parent, and (in Linux 4.1) an "of_node" link that, if the device was described by a device tree, will be a link to the relevant location in /sys/firmware/devicetree. Other files and symbolic links will be device-specific or subsystem-specific attributes. Other directories can be one of three sorts of things.

Subdirectories of a device directory may also be device directories if the parent is a "class" or "bus" device. This case is easily recognized by looking for "uevent" in the child directory.

Subdirectories of a device directory may be grouping directories for all children of a particular "class", but only when the parent device is a "bus" device. So the bus device for the TCA6507 device has a directory called .../i2c-2/2-0045, and it contains a grouping directory leds that contains a further subdirectory for each leds device. This case is also easily recognized as the child has the same name as a class, and its children contain uevent files.

Finally, subdirectories of a device directory can be grouping directories for attributes and can contain arbitrary attribute files, symbolic links, and further sub-directories, but no device directories.

When a device has a parent in the device model, the parent of the device directory will either be the device directory of the parent, or will be an intermediate grouping directory named after the class of this device. When a device does not have a parent, there are a few different possibilities.

When a bus is created, it can indicate whether parentless devices should be placed in /sys/devices/virtual, or /sys/devices/system, or directly in /sys/devices. Classes don't have that flexibility: all parentless class devices are placed in /sys/devices/virtual. Being class devices, they have an intermediate directory named after the class, and the device has its device directory attached within that intermediate directory. So md RAID block devices appear as /sys/devices/virtual/block/md0 for example.

Summing up

"Entities must not be multiplied beyond necessity" is a popular formulation of Occam's Razor, but could equally be used as advice for software development. Unfortunately the current state of /sys/devices doesn't seem to meet that goal very well as it seems filled with multiplicity, though that is doubtless due to baggage that has been carried along as the design evolved.

Some of the multiplicities are quite valuable: having directories used for both the actual devices and for namespace control seems very sensible, though it is unfortunate that the namespace control is a little ad hoc. Similarly, having attributes that can be imposed system-wide (power, subsystem, of_node), subsystem-wide, or per-device is a fairly natural choice.

Other multiplicities are largely a matter of interpretation such as the various ways of looking at the meaning of parenthood. It could be that the concept of "parent" isn't really well defined enough to deserve to be enshrined in the hierarchy, but it does seem to be useful to some degree.

And finally there are the multiple subsystems ("class" and "bus") and the multiple roots (/sys/devices, /sys/devices/virtual, and /sys/devices/system) that don't appear to add much value. The fact that there are so many places for parentless devices lends weight to the possibility that parents aren't really as important as we might imagine.

While it might be nice to have more simplicity, the more important thing is understanding what we do have. There is certainly a lot of value in /sys/devices, even if some aspects are not as valuable as others. Understanding what goes where, which parts are important, and which parts don't deserve so much attention, is really where it is most useful to focus. So while it might have been hard to describe, for myself at least it was worth describing.

Comments (1 posted)

Automotive Grade Linux and a distribution for cars

At the 2015 Automotive Linux Summit in Tokyo, Dan Cauchy from the Linux Foundation (LF) kicked off the first day's program with an announcement: that the LF's Automotive Grade Linux (AGL) workgroup has decided to build its own Linux distribution, which it plans to run as an ongoing, long-term project. While the desire for a workgroup to create a distribution tailored to its needs is nothing new, the announcement had several in the crowd wondering what this decision meant for Tizen IVI—which, up until now, has served as the reference distribution for AGL. Tizen, of course, is also an LF-hosted project, and it has made in-vehicle infotainment (IVI) one of its high-priority use cases.

The creation of the new distribution project was the second of Cauchy's announcements during the opening session; the first was the 1.0 release of the AGL specification, which defines a Linux-based platform for use in vehicles. The specification sets out a list of required components. Many are standard modules in any modern distribution, such as BlueZ, GStreamer, systemd, and so on. Others are more peculiar to the automotive use-case, such as a "lifecycle manager" that governs the systematic startup and shutdown of components in a data-preserving manner—which is important considering that a car's driver can choose to switch the system off at any moment.

After the announcements, Cauchy said that Tizen IVI had served well as the jumping off point for AGL, and that the AGL membership has had a good working relationship with the Tizen IVI team. Notably, AGL members had contributed code that Tizen IVI developers merged into the distribution—work that includes, for example, several of the components detailed in the AGL 1.0 specification.

Using Tizen IVI as an upstream project allowed AGL to gain momentum quickly and to bring new developers on board in member companies, Cauchy said. But, he continued, the AGL membership now felt that Tizen IVI was not advancing past the "proof of concept" stage, and the membership was eager to start working on a software platform that it could actually deploy.

As those who follow Tizen will know, the distinction between those two possible states is certainly true. Tizen (in all its various flavors: smartphone, smart TV, IVI, and others) has always been marketed as a base platform on which consumer electronics makers could build a full-fledged product, not as a deployable finished product itself. Tizen IVI is not easy for developers to install, nor is it a plug-and-play solution for most cars. Project co-sponsor Samsung has been using Tizen in its latest generation of products, but the project itself is intentionally not designed to be a drop-in system for commercial vendors. Cauchy also noted that Tizen IVI is focused on supporting Tizen's HTML5 application framework, while many automotive suppliers want support for native applications as well.

Despite the decision to move away from Tizen as a formal upstream, though, Cauchy added that the AGL distribution (while still a long way from its eventual release) will draw heavily on Tizen IVI's components. He showed a slide indicating that the new distribution would incorporate software modules from both Tizen IVI and from the GENIVI Alliance. "In the past," he said, "these have been the three big players. Our goal now is to unite them."

Furthermore, he said, while both GENIVI and Tizen IVI are focused solely on the infotainment platform—roughly defined as the software running on the in-dash head unit—AGL has broader goals. In the future, carmakers will be running Linux on embedded telematics (data-collection and analysis) systems, engine-control units, instrument clusters, advanced driver-assistance systems (ADAS), and other computers in the car, he said, and AGL wants to be the distribution running on all of those devices.

The same approach—standardizing on a single distribution across all vendors—has worked well in other "highly vertical" markets, Cauchy said. It is the tactic undertaken by telecommunications companies in Carrier Grade Linux and by financial institutions. The automotive sector is noticeably behind the times in this regard, he said.

Despite Cauchy's explanation about the background of the decision, a number of people in the audience clearly found the decision to break with Tizen IVI to be a surprise. One of the first questions from the audience at the end of the session was whether or not AGL and Tizen IVI sponsor Intel were discontinuing their collaboration. Cauchy replied that the relationship with Intel remained strong, and that several of the software projects started in Tizen IVI were expected to be key pieces of the AGL distribution, too. Despite these assurances, though, it was hard not to notice that, in a change from the past, Intel and Tizen had little to no presence at the event itself this year.

ARM work

A session later in the day may have shed additional light on the reasoning that went into the decision. Hisao Munakata from ARM vendor Renesas presented [PDF] the results of a recent study that assessed Tizen IVI's suitability for usage in real-world automotive systems based on ARM hardware. ARM support is critical, Munakata said, because many carmakers and equipment suppliers want to take advantage of low-cost system-on-chip (SoC) hardware.

The study identified a few problem areas that perhaps stem from Tizen IVI's historical emphasis on Intel hardware. Intel and ARM processors have different power-management requirements, for example; while most of the engineers working on Tizen IVI came from Intel, the ARM side simply got less testing. But the biggest distinction between the platforms was not the CPU itself—it was the graphics stack. Current automotive UIs, including Tizen IVI's, rely heavily on OpenGL and hardware acceleration. Intel GPUs are (thanks to Intel's open-source drivers) better supported in these areas than are the GPUs found on most ARM SoCs. Closing that gap will require additional engineering effort.

The study also looked at how several Tizen IVI components measured up with respect to automotive-specific use cases. It singled out Automotive Message Broker (AMB) and Crosswalk as components that were positioned to do what carmakers want, but that still need additional work. AMB is a message-passing bus used for low-level interprocess communication (IPC). Munakata said that it needed to be extended to cover several other IPC domains, such as message-passing between discrete engine-control units (ECUs). Crosswalk is Tizen's HTML5 application runtime. Munakata said that Crosswalk performed well on most points in ARM SoCs, but needed additional work on WebGL and Video4Linux2 support.

In both cases, Munakata said, he has presented his findings to the respective Tizen IVI team and they have been well-received. So it may be that the automotive-specific software projects initiated inside Tizen IVI will get more attention moving forward, even if the AGL distribution takes over Tizen's place as the central reference platform.

What next

The distinction between proof-of-concept code and deployable product would appear to be the root issue. But it is still a major undertaking to start a new distribution project. AGL may become an industry standard—it certainly has the backing of major players—but it has a lot of ground to cover.

For the developers who have been contributing to Tizen IVI, the news of a new automotive distribution from AGL may not be all bad news. There is clearly still interest in the work they have done, and there is (thus far) no real overlap between the software projects hosted at Tizen IVI and those at AGL and at GENIVI.

The more difficult question is assessing whether or not the announcement should be taken as a serious setback for Tizen. But, in reality, Tizen IVI has always been substantially different from the other Tizen device profiles: a smart watch may have fewer system resources than a smartphone, but it still has one screen, one simultaneous user, and one battery, for example. Between the differing hardware requirements, the automotive-specific layers, and the regulatory issues that are critical in automotive, perhaps maintaining a shared code base between IVI and other Tizen device profiles is not entirely realistic.

Perhaps it could be argued, though, that as Tizen matures, its various device profiles have diverged to the point where ensuring that they all offer the same stack is a significant strain. If so, though, providing a compatible application-level API through Crosswalk may be the only factor that genuinely needs to unite Tizen IVI and the other Tizen device profiles.

Time will tell how the Tizen project weighs these issues, of course. In the meantime, the prospect of an ongoing automotive distribution project managed by AGL certainly means there will be renewed development on a number of automotive software projects. With any luck, the result will also include something that more Linux users and developers can install, use, and contribute to.

[The author would like to thank the Linux Foundation for travel assistance to attend ALS 2015.]

Comments (8 posted)

Distribution quote of the week

-- Kevin Fenzi

Comments (none posted)

Fedora 22 for POWER and IBM z Systems

Fedora 22 has been released for POWER and IBM z Systems. The POWER port is available in Cloud and Server editions. The IBM z Systems port is available in the Server edition.

Comments (none posted)

Distribution newsletters

• DistroWatch Weekly, Issue 612 (June 1)
• Tumbleweed - Review of the week (May 29)
• Ubuntu Weekly Newsletter, Issue 419 (May 31)

Comments (none posted)

Chromixium Adds Polish to Chrome (LinuxInsider)

LinuxInsider reviews Chromixium. "Chromixium sole developer Rich Jack's innovative version 1.0, released last month, is a Chrome OS clone that runs on nearly any aging or newer computer with the user's choice of the fully open-sourced Chromium Web browser or Google's Chrome browser. Either way, Chromixium gives you a complete Chromebook experience on the hardware of your choice. The innovation is quite impressive. Chromixium also runs software from the Ubuntu distro ecosystem."

Comments (1 posted)

Improving performance in Python 2.7

Backporting a major performance improvement from Python 3 to Python 2 might seem to be in the "no-brainer" category, but things are not quite that simple. Python 2 (in the form of Python 2.7.x) is in a "no new features" mode that would normally preclude large changes, but 2.7 will be around for a lot longer than was previously envisioned. That makes the core development team more willing to consider this kind of patch, especially since it seems to come with a promise of more contributions. As with any topic that touches on the 2 vs. 3 question, though, it led to a long discussion and some dissent.

Vamsi Parasa of the Server Scripting Languages Optimization team at Intel posted a patch to change the switch statement that executes Python bytecode in the CPython interpreter to use computed gotos instead. GCC has support for the feature, which has already been used as an optimization in Python 3. As Eli Bendersky explained in a 2012 blog post, the enormous (2000+ line) switch statement for interpreting Python bytecodes can be made 15-20% faster by changing it to use computed gotos. There are two reasons for that: computed gotos avoid a bounds check that is required by the C99 standard for switch and, perhaps more significantly, the CPU can do better branch prediction, which reduces expensive pipeline flushes.

Several core developers spoke up in favor of the patch, but Berker Peksağ was not so sure. He complained that "performance improvements are not bug fixes" and that it would be better to spend time making Python 3 better. But there is more to this patch than meets the eye, Nick Coghlan explained:

As was noted in an email exchange tacked onto Parasa's post, Intel has hired Python core developer David Murray's company to help out navigating the Python development process with contributions it wants to make. But those contributions "came with one string attached: that the Python 2.7 branch be opened up for performance improvements in addition to bug fixes", Coghlan said. Because the proposal is "backed by a credible offer of ongoing contributions to CPython maintenance and support", that makes it different than other performance enhancements (or features) offered up for Python 2.7 along the way, he continued.

But accepting the offer does signal something of a shift in the way Python 2.7 will be maintained going forward. Encouraging more developers who are being paid to do the "boring" parts of Python maintenance (bug fixes and performance enhancements for 2.7, mostly) will allow volunteers to concentrate on the more interesting bits. Coghlan again:

Python's benevolent dictator for life (BDFL) Guido van Rossum was "strongly in favor" of the patch noting that it could save companies like his employer, Dropbox, "a lot of money". Dropbox has been slow to move to Python 3 but regularly updates to the latest Python 2.7 release, he said. But Victor Stinner wondered if it made more sense to put that effort into Python 3 in the hopes of getting more migration to that version of Python.

Van Rossum was clearly not happy with the idea of crippling Python 2 to somehow promote Python 3:

As Larry Hastings pointed out, though, the question had already become moot. Python 2.7 release manager Benjamin Peterson had merged the patch for 2.7.11, which is slated for release in December.

The rate of Python 3 adoption and the best way to encourage it are often somewhat contentious subjects in Python circles. Though many have bemoaned the lack of new features in Python 2.7, a large part of the decision to do that has been driven by a lack of core developers interested in continuing to work on that branch. Adding more paid developers should help alleviate some of those concerns. It seems unlikely to lead to any wholesale changes to 2.7, but there may be enhancements that can be made in the next few years. Meanwhile, new features like async/await, type hints, and others will continue to be added to Python 3 to hopefully provide a carrot to draw more and more to that version of the language. Or at least that is the plan ...

Comments (14 posted)

Quotes of the week

— Nick Coghlan

— Peter Hutterer (thanks to Josh Triplett)

Comments (none posted)

Firefox 38.0.5

Firefox 38.0.5 has been released. This version introduces Pocket, which helps you keep track of articles and videos. Clean formatting for articles and blog posts with Reader View is also a new feature. See the release notes for more information.

Comments (76 posted)

SfePy 2015.2 released

Version 2015.2 of the SfePy library has been released. SfePy is a Python package "for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support)." The new version marks the release of some major code-simplification work and reorganization of the installation files and examples. Also included are several new features, such as the ability to use time stepping solvers in interactive mode.

Full Story (comments: none)

Buildroot 2015.05 available

Version 2015.05 of Buildroot has been released. The announcement notes that this was the busiest development cycle yet; support was added for the steamroller architecture as well as the corei7-avx and core-avx2 x86 variants, libatomic is now supported for both internal and external toolchains, and close to 100 new packages have been added.

Full Story (comments: none)

Virtuozzo source code available

The OpenVZ team has announced the open source code release of several Virtuozzo userspace utilities. The utilities include prlctl, a unified command line tool to manage virtual machines and containers; libprlsdk, Virtuozzo API C++ and python libraries, used for local and remote communications with a dispatcher management service; prl-disp-service, a primary Containers and Virtual machines management service; libvzctl, a low-level library for Containers management; libvzevent, a low-level library for Containers life-cycle notifications from the kernel; vzctl, a utility to control a Containers; and vztt, a utility for Containers templates management.

Comments (7 posted)

Development newsletters from the past week

• What's cooking in git.git (May 29)
• What's cooking in git.git (June 2)
• LLVM Weekly (June 1)
• OCaml Weekly News (June 2)
• OpenStack Weekly Newsletter (May 22)
• Perl Weekly (June 1)
• PostgreSQL Weekly News (June 1)
• Ruby Weekly (May 28)
• Tor Weekly News (June 3, issue #100)
• Wikimedia Tech News (June 1)

Comments (none posted)

Announcing GitTorrent: A Decentralized GitHub

At his blog, Chris Ball announces "GitTorrent," his new project designed to let developers host Git repositories on BitTorrent. The system takes advantage of Git's ability to run over arbitrary network protocols. "We ask for the commit we want and connect to a node with BitTorrent, but once connected we conduct this Smart Protocol negotiation in an overlay connection on top of the BitTorrent wire protocol, in what’s called a BitTorrent Extension. Then the remote node makes us a packfile and tells us the hash of that packfile, and then we start downloading that packfile from it and any other nodes who are seeding it using Standard BitTorrent. We can authenticate the packfile we receive, because after we uncompress it we know which Git commit our graph is supposed to end up at; if we don’t end up there, the other node lied to us, and we should try talking to someone else instead." The project is, obviously, a new one that still has important ground to cover—such as dealing with comments or pull requests—but there are interesting ideas to consider already.

Comments (18 posted)

First Open Automotive Grade Linux Spec Released (Linux.com)

Linux.com talks with Dan Cauchy, general manager of automotive at the Linux Foundation, about the release of the AGL Requirements Specification. "In July 2014, AGL released its first AGL reference platform built on the Tizen IVI platform running HTML5 apps. The new release instead details precise specifications and requirements for any AGL-compliant IVI stack. For the first time, automakers, automotive suppliers, and open source developers can collaborate on refining the spec -- the first draft of a common, Linux-based software stack for the connected car."

Comments (none posted)

Conservancy Seeks Your Questions on GPL Enforcement

Software Freedom Conservancy has announced a long-term campaign to increase education and understanding about community-driven GPL enforcement processes. "Conservancy invites developers and other Open Source and Free Software contributors to email their questions on GPL enforcement to <enforcement-questions@sfconservancy.org>. Conservancy cannot promise to answer every question; Conservancy will use the collected questions over the coming months to provide more educational and informational materials about GPL enforcement, and in particular about Conservancy's GPL Compliance Project for Linux Developers."

Comments (8 posted)

The Document Foundation 2014 Annual Report

The Document Foundation (TDF) has announced its 2014 Annual Report is available. "TDF Annual Report starts with a Review of 2014, with highlights about TDF and LibreOffice, and a summary of financials and budget."

Full Story (comments: none)

Free Software Supporter - Issue 86, June 2015

This month the Free Software Foundation's newsletter covers The International Day Against DRM (2015/2016), the Licensing and Compliance Lab interviews François Marier, Sysadmin adventures, Asking Obama to protect encryption, FSF announces deputy director search, LibrePlanet videos, FSF is hiring, and several other topics.

Full Story (comments: none)

Final Call for DebConf15 Proposals

The deadline for DebConf15 proposals is June 15. Some have already been approved. DebConf15 will take place August 15-22 in Heidelberg, Germany.

Full Story (comments: none)

linux.conf.au 2016 Geelong - Call for Papers

The 2016 edition of linux.conf.au will take place February 1-5 in Geelong, Australia. The theme for the conference is *Life is better with Linux.* The call for papers closes July 6.

Full Story (comments: none)

PyTexas Call For Proposals

PyTexas will take place September 25-27 in College Station, TX. "We love both experienced and first time speakers, so no matter what your age or background, summon your inner honey badger and submit your talk ideas." The deadline is July 19.

Full Story (comments: none)

XDC2015: Call for papers

The X.org Developer Conference will be held September 16-18 in Toronto, Canada. "As usual, we are open to talks across the layers of the graphics stack, from the kernel to desktop environments / graphical applications and about how to make things better for the developers who build them." The CfP deadline is August 19.

Full Story (comments: none)

CFP Deadlines: June 4, 2015 to August 3, 2015

The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.

Deadline	Event Dates	Event	Location
June 5	August 20 August 21	Linux Security Summit 2015	Seattle, WA, USA
June 6	September 29 September 30	Open Source Backup Conference 2015	Cologne, Germany
June 11	June 25 June 28	Linux Vacation Eastern Europe 2015	Grodno, Belarus
June 15	August 15 August 22	DebConf15	Heidelberg, Germany
June 15	September 24	PostgreSQL Session 7	Paris, France
June 15	November 17 November 18	PGConf Silicon Valley	San Francisco, CA, USA
June 17	October 5 October 7	LinuxCon Europe	Dublin, Ireland
June 17	October 5 October 7	Embedded Linux Conference Europe	Dublin, Ireland
June 19	August 20	Tracing Summit	Seattle, WA, USA
June 28	August 28 September 3	ownCloud Contributor Conference	Berlin, Germany
June 28	November 11 November 13	LDAP Conference 2015	Edinburgh, UK
June 30	November 16 November 19	Open Source Monitoring Conference 2015	Nuremberg, Germany
July 5	October 27 October 29	Open Source Developers' Conference	Hobart, Tasmania
July 15	October 8 October 9	CloudStack Collaboration Conference Europe	Dublin, Ireland
July 15	October 27 October 30	OpenStack Summit	Tokyo, Japan
July 15	November 6 November 8	Jesień Linuksowa 2015	Hucisko, Poland
July 17	October 2 October 3	Ohio LinuxFest 2015	Columbus, OH, USA
July 19	September 25 September 27	PyTexas 2015	College Station, TX, USA
July 24	September 22 September 23	Lustre Administrator and Developer Workshop 2015	Paris, France
July 31	October 26 October 28	Kernel Summit	Seoul, South Korea
July 31	October 24 October 25	PyCon Ireland 2015	Dublin, Ireland
July 31	November 3 November 5	EclipseCon Europe 2015	Ludwigsburg, Germany
August 2	February 1 February 5	linux.conf.au	Geelong, Australia
August 2	October 21 October 22	Real Time Linux Workshop	Graz, Austria
August 2	August 22	FOSSCON 2015	Philadelphia, PA, USA

If the CFP deadline for your event does not appear here, please tell us about it.

EuroPython 2015: More than 200 sessions waiting for you

The call for papers is over and 205 sessions were chosen for EuroPython 2015, which will take place July 20,26 in Bilbao, Spain.

Full Story (comments: none)

Events: June 4, 2015 to August 3, 2015

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
June 3 June 5	LinuxCon Japan	Tokyo, Japan
June 3 June 6	Latin American Akademy	Salvador, Brazil
June 5 June 7	PyCon APAC 2015	Taipei, Taiwan
June 8 June 10	Yet Another Perl Conference 2015	Salt Lake City, UT, USA
June 10 June 13	BSDCan	Ottawa, Canada
June 11 June 12	infoShare 2015	Gdańsk, Poland
June 12 June 14	Southeast Linux Fest	Charlotte, NC, USA
June 16 June 20	PGCon	Ottawa, Canada
June 22 June 23	DockerCon15	San Francisco, CA, USA
June 23 June 25	Solid Conference	San Francisco, CA, USA
June 23 June 26	Open Source Bridge	Portland, Oregon, USA
June 23 June 26	Red Hat Summit	Boston, MA, USA
June 25 June 26	Swiss PostgreSQL Conference	Rapperswil, Switzerland
June 25 June 28	Linux Vacation Eastern Europe 2015	Grodno, Belarus
June 26 June 28	FUDCon Pune 2015	Pune, India
June 26 June 27	Hong Kong Open Source Conference 2015	Hong Kong, Hong Kong
July 3 July 5	SteelCon	Sheffield, UK
July 4 July 10	Rencontres Mondiales du Logiciel Libre	Beauvais, France
July 6 July 12	SciPy 2015	Austin, TX, USA
July 7 July 10	Gophercon	Denver, CO, USA
July 15 July 19	Wikimania Conference	Mexico City, Mexico
July 18 July 19	NetSurf Developer Weekend	Manchester, UK
July 20 July 24	O'Reilly Open Source Convention	Portland, OR, USA
July 20 July 26	EuroPython 2015	Bilbao, Spain
July 25 July 31	Akademy 2015	A Coruña, Spain
July 27 July 31	OpenDaylight Summit	Santa Clara, CA, USA
July 30 July 31	Tizen Developer Summit	Bengaluru, India
July 31 August 4	PyCon Australia 2015	Brisbane, Australia

If your event does not appear here, please tell us about it.