Community support for free hardware

The "Spark" tablet (later renamed "Vivaldi") was first announced at the beginning of 2012; it was meant to be a fully open device that would be "designed by and usable by us on our terms". By 2014 standards the specifications and price (€200) look a bit dated, but they were not totally out of line at the time. The software stack was to be based on the Mer distribution and the Plasma Active interface. Behind it all would be an application distribution environment that would encourage free software while allowing developers to monetize their work. By all appearances, it was set to be an interesting device.

Improv followed toward the end of 2013. It was a reasonably well-equipped development board that, once again, would run Mer. The schematics were licensed under the GPL, and the anticipated profits were destined to support community resources like the Mer open build service. Interested people were encouraged to order without delay, because "we expect the first lot to sell out quickly".

Your editor, who duly put in an order for an Improv board, recently received an email saying that this board would never arrive. Instead, the developers would be making a partial refund of the cost of the board ($52 of the $75 original cost, as it turns out). In the end, there were not enough orders to cover the cost of making that first lot of boards, so the whole project has been canceled. The message also relayed the unsurprising news that the Vivaldi project, which had long been stalled and silent, would also be wound down. The project appears to have failed completely, at considerable cost to its backers.

Why might this be? According to the above-mentioned email:

Free and open hardware is indeed important, but it is worthwhile to consider "free" and "open" independently. Open hardware is fully documented and unlocked; users have the ability to put their own software onto it if they so desire. Free hardware has all of that; in addition, the designs (including, preferably, the files used as input to the manufacturing process) are made available under a free license, allowing others to modify or extend the design.

There can be little doubt of the value of open hardware. The ability to understand what the hardware does and to change how it is used (by changing its operating software) brings a great deal of freedom to users. It allows that hardware to be used in settings and for purposes that its designers could never have envisioned. One might argue that open hardware will not exist if users are unwilling to put their money where their interests are and buy that hardware. It is also worth noting that much hardware that was not intended to open tends to be forced open by determined users. The apparent ease with which much closed hardware can be jailbroken might serve to reduce the demand for truly open hardware somewhat.

That said, there are, in fact, some signs that this demand does indeed exist. The success of Arduino, or of other development boards similar to Improv, is one case in point. Google's "Nexus" program offers hardware that, while not being quite as open as we might like, is far better than what many of us would have expected just a few years ago; customers are willing to pay the substantial cost premium to buy Nexus devices rather than use the "$0" devices from carriers. The Open Compute Project has drawn substantial industry support and has published a number of open hardware designs. So the interest is certainly there.

There is, perhaps, less interest in free hardware — hardware with designs published under a free license. Free hardware might be just as important as free software, but there is a crucial difference: almost anybody with basic skills can take advantage of the freedoms offered by free software. One might "fork" a free hardware design, but, for most of us, the ability to realize any changes to that design in real hardware is beyond reach. Perhaps, someday, it will be easy to render designs into working hardware in small quantities for a reasonable cost, but that day is not here yet. Until that day comes, it should not be surprising that the issue of hardware freedom tends to get an apathetic reaction. It just doesn't seem as relevant to many in our community as software freedom does.

Meanwhile, given that there does seem to be a market for open hardware, why is it that projects like Vivaldi and Improv struggle? It is probably a simple issue of money. Creating an interesting piece of hardware, getting it built, and selling it to users is a cash-intensive business. A handful of free software developers attempting such a project funded from their savings and using a personal weblog as the primary marketing channel will probably have a hard time. Serious funding does not guarantee success — well funded products from established manufacturers fail regularly — and a lack of that funding does not guarantee failure. But, while a new software project can be started with almost no budget at all, trying to create a new piece of hardware on a shoestring budget is sailing against the wind.

So the failure of Improv and Vivaldi is not necessarily a failure on the part of the community to support an important principle. It is probably better described as yet another startup company with some interesting ideas that never quite managed to take off. Eventually somebody will likely succeed with a fully free hardware project — as Arduino has done — and they may well benefit from some of the groundwork that was done by the developers behind Improv and Vivaldi. But they will have to succeed as a business, and not just as a piece of interesting hardware design.

Comments (15 posted)

The demise of Freecode

Many long-time members of the free software community were surprised on June 18 to see the sudden announcement, seemingly without warning signs, that Freecode.com had discontinued operations. The site, which between 1997 and 2011 operated under the name Freshmeat.net, served as an index of free and open-source software projects and provided a steady news feed composed of the aggregated release announcements of thousands of indexed projects.

The shutdown notice makes it clear that the index will remain up and accessible for future reference, which has its benefits, but the news feed has been discontinued and no new updates will be made to existing entries:

It is perhaps easy to forget how different the FOSS landscape looked in 1997. Certainly it was smaller, and it was also less connected. RSS was not created until 1999, and, naturally, took some time to rise in popularity and become a common method to publish news of any sort, release announcements included. Consequently, Freshmeat's ability to stay on top of releases from a wide variety of FOSS projects filled an important niche—and was certainly more convenient than subscribing to dozens or hundreds of mailing lists.

So, too, did Freshmeat's core index, which provided a searchable, categorized database of open-source projects, including the canonical home page for each project, basic information (such as the languages used and the platforms the project was created to run on), and a history of its previous releases. If today it sounds hard to believe that Freshmeat predated the advent of RSS, it is arguably more difficult to grasp that Freshmeat predated, at least by a small margin, the rise of Google.

Google's novel approach to indexing web content eventually made it relatively reliable to find the real home page for a software project of interest, but prior to Google PageRank, the other major search engine companies still relied largely on human-moderated indexes. Thus, they served as gatekeepers, and new sites had a very different path to follow from obscurity to fame. For a niche like free software, a dedicated index was far more accurate than a general-purpose portal like Yahoo's.

But Google did come along, and with it the way that people searched for and kept track of free software projects changed. At the same time, how and where those projects maintained their presence on the web has changed many times over as well. Whereas in the 1990s, SourceForge.net was one of the only (if not the only) hosting sites dedicated to open source software projects, today there are many more options, including Google Code, GitHub, Gitorious, and a wide range of self-hosting packages for popular web frameworks—in addition, of course, to the umbrella projects like GNU that host a large portion of the community's free software still.

One might think that the dispersal of project hosting services would serve to make a dedicated index like Freshmeat more valuable, but there were other forces at work at the same time that had the opposite effect. Package management for Linux distributions changed with the creation of Apt, yum, and related tools; users could increasingly rely on their Linux distribution to not only provide installable packages of relative freshness, but for the distribution's package index to serve as its own, ad-hoc index of the available packages themselves. And there are clearly far more ways to get the word out about a new release than there were in the late 1990s, perhaps more effectively. It may even be faster to spread the news via social media than to push an announcement out to a mailing list's subscribers.

Regardless of how the technical playing field has changed, though (and as everyone knows, it continues to change, arguably faster than ever), Freshmeat also had to function as a profitable business in order keep operating. Perhaps no one will ever know whether business calculations played a significant part in the decision to stop updating the site, but Freshmeat did change hands several times over the course of its lifespan. Dice.com acquired the site (at that point, renamed to Freecode) in 2012, after a lengthy tenure at Geeknet, the company that was formerly the corporate parent of SourceForge, Slashdot, and quite a few other related FOSS properties.

It is certainly possible that Dice.com simply did not know what to do with the site, or that its staff found the arrangement unsatisfactory. In his personal retrospective after the shutdown, longtime co-maintainer Jeff Covey noted that founder Patrick Lenz had left a year ago, after having been moved over to SourceForge operations for the preceding 18 months. Covey himself had been laid off in 2010, he said, brought back as a contractor, then voluntarily quit for personal reasons about three weeks before the shutdown. One of the other two remaining maintainers also left at the same time.

Covey notes with some regret that the departures seem to have precipitated the shutdown, which came as a surprise to him:

Certainly that series of events suggests that the decision for the shutdown had at least something to do with the interest level of its owners, rather than, say, friction with the maintainers. Nevertheless, people will inevitably speculate as to whether the root cause of the shutdown was a business issue or the many changes in the way that free software is deployed and discussed online over the past 17 years.

Has the FOSS community itself already moved on? That is difficult to assess quantitatively, at least from the outside. Certainly some people still feel that Freecode or a site like it is an important resource. On June 21, Eric S. Raymond posted a blog entry proposing a replacement site be created. He cites Freecode's independence from Linux distributions and project-hosting services, as well as its searchable index of project metadata, as vital features. Some of the same functionality is provided in other places, such as Ohloh, he notes, but none replicate the essential indexing and release-news features.

Time will tell if Raymond's proposal bears fruit; it certainly spawned a lengthy and detailed discussion among commenters. But even a full, functional replacement will only be a recreation of the now-gone original—and will, no doubt, have to cope with the same set of challenges as that original. Coping with them and delivering a valuable service for as long as Freshmeat/Freecode did would be a tall order to fill.

Comments (8 posted)

Where does the RHEL 7 source code live?

When the much-anticipated Red Hat Enterprise Linux (RHEL) 7 release was made on June 10, it started the process for various RHEL-rebuild distributions to get their new releases out. Previously, the starting point had been source RPM (SRPM) files that Red Hat used to release. Things have changed a bit this time, which caused some initial consternation among some rebuild developers, but it eventually turned out that it would not to be too difficult to get what is needed from the new Git-based repository.

The expected location for the SRPMS directory for RHEL 7 just has a README file that points to git.centos.org/project/rpms and the Red Hat Customer Portal as locations for the source code. The latter is, as its name would imply, only available to Red Hat customers. That means that all of the other rebuilds need to pick up the source code from a CentOS repository, which seems a little strange—at least at first.

One of the big changes between the release of RHEL 6 and that of RHEL 7 is that CentOS, arguably the leading RHEL-rebuild distribution, joined Red Hat. That would seem to put it in the pole position for creating a RHEL 7 rebuild, but CentOS won't have any real advantages over the others. It will be using exactly the same RHEL code as all of the other distributions. Beyond that, there is a firewall between RHEL and CentOS developers. In addition, CentOS is developing and building CentOS 7 fully in the open, so anyone interested can follow along in the mailing list and Git repositories.

The README led to queries on the centos-devel mailing list (from Morten Stevens and Connie Sieh) wondering about the change in format and location for the RHEL 7 source code. The biggest concern was how to get unmodified source code that corresponds to the RHEL 7 packages and how to verify that the code had not been modified along the way. Previously, the SRPMs that were signed with Red Hat's key would suffice, but those are only available to customers through the portal now—though they may eventually "leak" since most simply contain free software.

The terse message in the README file seemingly did not go far enough to alleviate concerns about exactly what source was stored in the Git repositories. In addition to the mailing list threads, a bug was filed over the lack of SRPMs, but it was quickly closed as NOTABUG. The comment on the closing did, however, seem to clear up any confusion about the origin of the code in the CentOS Git repository—Red Hat put the source code from RHEL 7 there as a starting point for CentOS and anyone else.

Getting Red-Hat-signed SRPMs is still possible for customers, but for non-customers looking for signed SRPMs, CentOS will be providing its SRPMs once it completes the CentOS 7 development process. Those will only minimally differ from RHEL 7, mostly just removing the Red Hat branding, so they make for a better starting point for a rebuild distribution. It is certainly different than what was available before, but not necessarily worse.

It is also possible to generate SRPMs that correspond to the original RHEL 7 code from the Git repository, though those SRPMs won't be signed. There are a number of scripts hosted in the centos-git-common repository that will help perform various tasks using the Git repositories. For example, pointing into_srpm.sh at a commit or branch of interest will use the .spec file found there to create an SRPM. One can also use show_possible_srpms.sh to list the SRPMs that can be built from a particular repository; the -r option will restrict the list to those that come from the Red Hat RHEL 7 commits. There is also information on the CentOS wiki on how to work with the myriad repositories at Git.CentOS.Org.

There are concerns expressed in the bug and mailing list threads about ensuring the authenticity of the code retrieved, but Git, coupled with SSL/TLS, should be enough to alleviate that concern. Using the tools to recreate the SRPMs from the Red Hat commits to the repository, all of which is done over HTTPS (i.e. SSL/TLS), should be safe. Rebuilding the SRPMs, rather than just downloading them, may make it a little more difficult, but it does not fundamentally change what the RHEL rebuilds can get. And having the code available in Git provides an API to do things that would not have been possible with the simple "directory full of SRPMs" approach used previously. As Chris St. Pierre put it: "You no longer have to diff the ftp 'ls' output -- there are actual machine-consumable feeds and APIs to use."

There is clearly some trepidation among the non-CentOS consumers of RHEL source code, as evidenced by the thread and bug comments. But the biggest issue, in the end, seems to be more confusion than anything else. While Red Hat did mention this change in a FAQ about the CentOS merger, it probably could have done a lot more to publicize—and explain—the change. In the final analysis, it may seem a little odd to get RHEL source code from a CentOS repository, but it shouldn't pose any real barrier to anyone getting their work done.

Comments (12 posted)

The WebCrypto API

The World Wide Web Consortium (W3C) is developing a JavaScript API that encompasses a number of useful cryptographic operations for use in situations ranging from signature verification to end-to-end data encryption. Called the Web Cryptography API (or WebCrypto for short), the standard is currently at "Last Call" working draft status with the Web Cryptography Working Group. There are a few lingering issues that may still result in some changes before the API advances to the next stage in the W3C recommendation process, but WebCrypto also faces fundamental criticisms; for instance, that it does not specify those elements on which users actually need standardization—namely, algorithms. And there is always the concern that such an API is risky because the security functions it offers may be inherently untrustable in precisely those scenarios where they are most needed.

The most recent draft of WebCrypto is available in raw form from the W3C version control system. This version reflects some changes since the Last Call was was made on March 25. The Last Call resulted in a number of new issues being created; at this point the working group appears to be aiming at resolving all of those by the middle of July.

The WebCrypto draft begins by setting out its anticipated use cases, most of which are standard web application concerns: client-to-server authentication, sending and storing encrypted files (including those encrypted by the server and those encrypted on the client side), plus making and checking digital signatures. These standard operations are the type of functions that web applications already implement, but with each implementor reinventing the wheel in its own JavaScript code. If browsers offer key generation, signature checking, and data encryption, the thinking goes, there will less duplication of effort and (hopefully) higher-quality code.

Derived from its basic use cases, the draft defines four main APIs and an algorithm dictionary that, together, address the essential cryptographic primitives needed by those use cases. The APIs are RandomSource, which exposes a cryptographically strong pseudorandom number generator (PRNG); CryptoKey, which is an abstraction covering various types of cryptographic keys (including public, private, and shared secret keys, of a variety of algorithms); SubtleCrypto, which is the low-level API for accessing functionality like encryption, decryption, key generation, hash generation, and verification; and WorkerCrypto, which is the interface for Web Worker background scripts. A higher-level Crypto API is also defined, which wraps together SubtleCrypto and RandomSource. The Algorithm dictionary stores the list of supported algorithms available in the browser for various functions.

From an interface standpoint, tracking the keys on the system is a relatively straightforward affair. Each public/private key pair, for instance, is represented by a pair of CryptoKey objects and an entry in the CryptoKeyPair dictionary indicating that the two go together. The specification allows each key to be designated for certain operations (e.g., encryption, decryption, signatures, verification). There is also a boolean attribute, extractable, that allows the user agent to designate that the key's raw material may be exported (for example, to back up the key elsewhere, perhaps even offline).

In contrast, the SubtleCrypto interface occupies the larger portion of the specification, encompassing a dozen operations. But even these "low-level" cryptographic methods ultimately fall back on the underlying algorithms for meaning; after all, calling the digest() method to check the integrity of a block of data is risky if one specifies a broken hash function (like SHA-1) as a parameter. Therein lies the source of many of the complaints about WebCrypto: the working group is explicitly not setting out to recommend a set of safe, reliable algorithms to use. Rather, it is attempting to define an API for calling and working with those algorithms.

As a result, WebCrypto includes a sizable list of algorithm descriptions, but none of them are mandatory to implement. Worse yet, according to some critics, the specification does not provide implementors with guidelines as to what options and parameters are safe to use. There are, for instance, known practical attacks [PDF] against some older algorithms such as RSA PKCS#1 v1.5. WebCrypto includes RSA PKCS#1 v1.5 in its list of supported algorithms, even including a basic mapping of the algorithm onto WebCrypto's methods and attributes.

Several working group members also expressed dissatisfaction at the absence of some newer algorithms, such as Daniel J. Bernstein's Curve25519. But WebCrypto co-editor Ryan Sleevi argues that the specification's role should be limited to describing the functionality that is already available in web browsers, rather than blessing algorithms that, while mathematically desirable, have yet to be implemented.

Curve25519, he notes, is not something that a web application developer can count on being available:

Of course, deciding on a suite of recommended cryptographic algorithms would be a difficult task for any standards body, since it is difficult to agree on what constitutes "safe" algorithms and parameters. For instance, when the Edward Snowden leaks demonstrated that the Dual_EC_DRBG PRNG from the National Institute of Standards and Technology (NIST) had been compromised with an NSA backdoor, some felt that other NIST recommendations, such as its elliptic curves, were surely compromised as well. But that has yet to be proven.

At present, the working group seems ready to compromise on the Curve25519 issue by drafting a complementary specification that describes Curve25519 (as well as another option promoted by Microsoft, known as the "MSR" curves) that could later be merged into an update to WebCrypto. That approach would allow the current WebCrypto specification to continue its progress through the W3C recommendation process.

There are, however, also lingering criticisms of WebCrypto that take issue with a more fundamental facet of the specification: the fact that it describes cryptography that will ultimately be implemented in JavaScript. As Tony Arcieri put it in a blog post, "the convenience of the web stems from the fact it’s a frictionless application delivery platform. Unfortunately, it does not rely on a comprehensive cryptographically secure signature system to determine content is authentic, but instead just trusts whatever is sitting around on the server at the time you access it."

In other words, web-based cryptography should not be trusted to protect the user's interests. It may be useful for web application developers, but the browser is not a trustworthy execution environment for end-to-end encryption of users' secrets. While there are valid uses for WebCrypto, Arcieri says, he fears that it will also be used to implement applications that ostensibly protect the user's data. For instance, he notes the possibility of secure-storage services akin to MEGA, which advertise features like cryptographically secure cloud storage, but are in actuality susceptible to multiple attack vectors.

One might answer Arcieri's charge by pointing out that services like MEGA exist already, and that existing services simply implement their own encryption stack rather than relying on a standard, hopefully well-vetted API. But the underlying concern remains: application code that is delivered to the browser over the web should not be treated with the same level of trust as locally installed and verified packages. A web application that uses the WebCrypto APIs may indeed be free from man-in-the-middle attackers; the risk is simply that the user cannot know one way or other for certain. Regardless of the algorithms employed, that lack of verification places an upper bound on how much trust should be extended.

[Thanks to Paul Wise.]

Comments (none posted)

Security quotes of the week

Comments (5 posted)

Decades-Old Vulnerability Threatens 'Internet Of Things' (Dark Reading)

Comments (17 posted)

asterisk: multiple vulnerabilities

Comments (none posted)

freerdp: two vulnerabilities

Comments (none posted)

kdelibs: information disclosure

Comments (none posted)

kernel: multiple vulnerabilities

Comments (none posted)

libemail-address-perl: denial of service

Comments (none posted)

novell-qtgui, novell-ui-base: privilege escalation

Comments (none posted)

openfire: multiple vulnerabilities

Comments (none posted)

openstack-keystone: privilege escalation

Comments (none posted)

openstack-neutron: denial of service

Comments (none posted)

php: multiple vulnerabilities

Comments (none posted)

phpmyadmin: cross-site scripting

Comments (none posted)

python: information disclosure

Comments (none posted)

python-simplejson: information disclosure

Comments (none posted)

sos: password disclosure

Comments (none posted)

swift: cross-site scripting

Comments (none posted)

wireshark: code execution

Comments (none posted)

zabbix: local file inclusion

Comments (none posted)

Kernel release status

Stable updates: 3.15.2, 3.14.9, 3.10.45, and 3.4.95 were released on June 26, followed by 3.15.3, 3.14.10, 3.10.46, and 3.4.96 on June 30.

Comments (none posted)

Quote of the week

Comments (none posted)

Capsicum for Linux

Capsicum is built around a concept called "capabilities," which, naturally enough, is entirely different from Linux capabilities (and from POSIX capabilities as well). In the Capsicum world, capabilities are attached to file descriptors to regulate which operations can be performed on those descriptors. So, for example, a file descriptor can only be read if the CAP_READ capability is present. Access to lseek() is controlled by CAP_SEEK, memory mapping has a set of capabilities (CAP_MMAP_W to create a writable mapping, for example), truncation is controlled by CAP_FTRUNCATE, and so on. There are two special capabilities for ioctl() and fcntl() that restrict those calls to specific subcommands.

By default, open file descriptors are unrestricted. The normal mode of operation is that a process will apply restrictions to itself using the new cap_rights_limit() system call:

    int cap_rights_limit(unsigned int fd, struct cap_rights *new_rights,
    			 unsigned int new_fcntls, int nioctls, unsigned int *new_ioctls);

After this call, operations on fd will be limited to those listed in new_rights. If those rights include CAP_FCNTL, then new_fcntls limits the set of fcntl() commands available. Similarly, if the capabilities on the file descriptor include CAP_IOCTL, the new_ioctls array (of length nioctls) provides the set of allowed ioctl() commands. Multiple calls to cap_rights_limit() can be made for the same file descriptor, but those calls can only remove capabilities, never add them.

There is also a cap_rights_get() call to query the set of capabilities attached to a given file descriptor.

Needless to say, restrictions on file descriptors are of limited value if the constrained process can simply open new descriptors on the same objects. To prevent that from happening, Capsicum implements a "capability mode" entered via cap_enter(). Once that mode has been entered, access to most global namespaces is curtailed, preventing the opening of new files. A process can still open a file with openat() if it has a directory file descriptor (and, of course, the relevant capabilities are present). Such files are constrained to be underneath that directory, though — use of absolute pathnames or "../" is not allowed.

(As an aside, the "can only open files below this directory" functionality was deemed to be sufficiently useful that David pulled it out into a separate patch and made it available independently of Capsicum. This patch adds a new O_BENEATH_ONLY flag for calls like openat(). Once a directory has been opened with this option, the resulting file descriptor can only be used to open files that exist below that directory in the filesystem hierarchy.)

That said, the patch set as posted does not provide an implementation of cap_enter(). Also missing is the entire "process capabilities" mechanism, which represents specific processes as file descriptors so that the relevant system calls (wait(), kill(), etc.) can be controlled. The patch set is described as being "part 1," so, one assumes, the remaining pieces will come later.

Within the kernel, system call implementations typically start by converting passed-in file descriptors to struct file with calls to fdget(). This is the point where David decided to apply the capability checks. When file descriptors are restricted with Capsicum, the normal file structure is replaced by a wrapper structure containing the rights information. Every fdget() call in the kernel (there are about 100 of them) must be replaced with a call to:

    struct file *fdgetr(unsigned int fd, int caps ...);

Where caps is a variable-length list of capabilities that must be present for the operation to succeed. Callers must also be changed to deal with an "error pointer" return value; fdget() in current kernels can return NULL but not a specific error value. The result is that the patch set is somewhat invasive; that may be a cause of resistance should the patch set reach a point where it is being seriously proposed for inclusion.

The patch set currently works by creating a pair of new Linux security module (LSM) hooks to do the actual capability checks. David wonders, though, whether that is the right approach, since Capsicum is not a complete security module. If the kernel implemented stacked security modules, perhaps Capsicum could be run in this mode alongside another, more complete module. But stacking does not look like it will be supported in the kernel anytime soon. So Capsicum may well be better off implemented outside of the LSM framework.

There is another question that is worth considering here. The kernel's secure computing (seccomp) subsystem allows the loading of programs (written for the BPF virtual machine) that can, in theory, implement all of the restrictions found in Capsicum, especially if the recently proposed BPF changes are merged. It might not be easy, but it should be possible. Somebody is bound to ask whether the kernel needs another sandbox-creation mechanism with similar capabilities.

In general, the addition of new security-related subsystems can be a hard sell; many developers see little value for a lot of cost in these subsystems. But there is value in the ability to reduce the damage that can be done by a compromised process, and FreeBSD's use of Capsicum means that some applications have already had the necessary code added. Adding the same API to Linux would allow that work to be reused. So Capsicum seems worth considering, even if it will likely have some obstacles to overcome before merging is a possibility.

Comments (none posted)

Extending extended BPF

The first thing this patch series does is to move the BPF interpreter out of the networking subsystem. BPF can already be used with non-networking parts of the kernel, and the plans are for such uses to grow over time. So the BPF support code will move into a new subdirectory (kernel/bpf) and be maintained independently from the networking code.

Over the past few development cycles, Alexei has introduced a variant of BPF called "extended BPF" (eBPF) which adds a number of capabilities and performance improvements. Thus far, though, eBPF has only been used within the kernel itself; the existing BPF users load "classic" BPF programs into the kernel which are then translated to eBPF prior to execution. With this patch series, though, eBPF will be made available for direct use from user space. Among other things, that means that the eBPF instruction set will, once users pick it up, become difficult to change. There has been relatively little review of the instruction-set changes so far; anybody who has an interest in how this (significant) addition to the kernel's user-space ABI is defined might want to take a close look in the near future.

Loading programs

The patch series adds a new system call named, simply, bpf(); it is a multiplexor for a range of different operations. Alexei also supplies a wrapper library to present those operations as a set of independent functions. Multiplexed system calls have not always been popular with reviewers in the past; if that pattern holds, we may see the multiplexed interface taken out and the various functions implemented directly as separate system calls.

So, for example, user space can load an eBPF program into the kernel with a call to:

    int bpf(BPF_PROG_LOAD, int prog_id, enum bpf_prog_type type, struct nlattr *prog,
            int len);

Or, using the wrapper function:

    int bpf_prog_load(int prog_id, enum bpf_prog_type prog_type, 
                      struct sock_filter_int *insns, int prog_len, const char *license);

In either case, prog_id is a number used to identify the program; these numbers exist in a single, global namespace. There is currently only one possible value (BPF_PROT_TYPE_UNSPEC) possible for type. In the actual system call, the BPF program is found in prog; the networking roots of BPF show here, where a netlink attribute is used to hold the code. The length of the attribute array is passed in len. The wrapper, instead, hides the nlattr structure, but exposes a struct sock_filter_int structure (which will likely be renamed in the future) to hold the program. The license parameter will be discussed below.

Naturally, adding the ability to run programs within the kernel brings up a number of interesting security issues. So it not surprising that the biggest part of the patch set is a "verifier" that attempts to ensure that eBPF programs cannot harm the running system. The verifier simulates the execution of the program, looking for problematic behavior. Should something suspect turn up, the program will not be loaded.

The verifier looks for a number of things. It tracks the state of every eBPF register and will not allow their values to be read if they have not been set. To the extent possible, the type of the value stored in each register is also tracked. Load and store instructions can only operate with registers containing the right type of data (a "context" pointer, for example), and all operations are bounds-checked. The verifier also disallows any program containing loops, thus ensuring that all programs will terminate.

In this patch set, the CAP_SYS_ADMIN capability is required to use any of the bpf() system call functions. That restriction may limit interesting future uses of eBPF, but there are a number of potential issues (such as the single global ID namespace and resource use limits) that would have to be dealt with before that restriction could be lifted.

Licensing issues

The bpf_prog_load() wrapper also has a license parameter; the value passed there is stored in the nlattr array prior to the bpf() call. It is used to provide a string specifying the license that applies to the eBPF program to be loaded; if that license is not GPL-compatible, the kernel will refuse to load the program. This behavior already appears to be somewhat controversial; reviewers noted that full-blown kernel modules can be loaded (albeit with reduced access) without a GPL-compatible license declaration. It strikes some of them as strange to apply stricter rules to eBPF programs. In response, Alexei has said that future revisions might move to a module-like scheme where any program can be loaded but access to some functions might be restricted to GPL-compatible programs.

There could be some interesting implications from this type of restriction. BPF programs are often generated by other programs; the original BPF, after all, was meant to be emitted by the tcpdump tool. One might well wonder what the "source" of such a program actually is. If the Chromium browser generates an eBPF script to define a sandbox for a plugin module, which parts of Chromium, if any, are part of the source for that script? One can imagine that the discussion of this issue could go on for a long time indeed.

Maps

The other significant addition in this patch set is "maps." A map is a simple key/value data store that can be shared between user space and eBPF scripts and is persistent within the kernel. As an example of their use, consider this simple program included with the patch set. It creates a map with two entries, indexed by IP protocol type; an eBPF script then inspects passing packets and increments the appropriate entry for each. The program in user space can then query those entries to get a sense for what kind of traffic is passing through the system.

Maps can only be created or deleted from user space; eBPF programs do not have that capability. Maps are created and deleted with:

    int bpf_create_map(int map_id, int key_size, int value_size, int max_entries);
    int bpf_delete_map(int map_id);

As with program IDs, the namespace for the map_id is shared across the entire system; there is no mechanism to specify which maps a given eBPF (or user-space) program may access. To store values into and retrieve values from maps, user space can call:

    int bpf_update_elem(int map_id, void *key, void *value);
    int bpf_lookup_elem(int map_id, void *key, void *value);
    int bpf_delete_elem(int map_id, void *key);
    int bpf_get_next_key(int map_id, void *key, void *next_key);

Once again, these are the wrapper functions; the actual operations are done with the bpf() system call. On the eBPF side, access to maps is provided with a set of external functions. Interestingly, each place where use of eBPF programs is enabled (see below) must explicitly set up access to the map functions; this access is not provided to eBPF programs by default. Maps, in the end, function both as a persistent data store for eBPF programs and a means for communication with user space.

Running eBPF programs

There is one operation that is conspicuous by its absence in the discussion thus far: the ability to actually run an eBPF program. There is little point in running an eBPF program on demand from user space; there is not much that it could do that couldn't be more easily accomplished directly. Instead, eBPF programs are meant to respond to events within the kernel.

One common event, of course, is the receipt of a packet from the net. The patch set adds a new form of access to the socket filtering mechanism, allowing a program to directly attach an eBPF program to an open socket:

    setsockopt(sock, SOL_SOCKET, SO_ATTACH_FILTER_EBPF, &prog_id, sizeof(prog_id));

Here, prog_id must be the ID number of a program previously loaded into the kernel with the bpf() system call.

eBPF programs can also be attached to tracepoints; such a program will be run every time that the tracepoint fires. This attachment is done by writing the string "bpf_ID" to the appropriate filter file in the tracing debugfs filesystem; once again, ID is the ID number of a loaded eBPF program. Missing, thus far, is a way to use eBPF directly with the secure computing (seccomp) mechanism; one assumes that will follow at some point.

All told, this patch set represents a significant addition to the BPF virtual machine. It is also a large addition to the kernel's user-space ABI; that suggests that it needs a rather higher level of review than it has seen so far. Once that review happens, the shape of the patch set may well change from what has been described here. But there seems to be little disagreement that the kernel can benefit from a more capable virtual machine that can be used in a number of contexts. So, sooner or later, some version of these patches will probably go in.

Comments (6 posted)

Control groups, part 1: On the history of process grouping

While it may not be the most controversial feature ever added to Linux, there is little difficulty in finding mailing lists or Internet forums containing heated arguments about the merits of control groups — or even downright denials that the feature has any merit at all. Being bereft of a personal agenda on the matter or any deep understanding of the issues, I find it very hard to choose a side in these debates, which seriously lessens the enjoyment I can receive from them. As synthesizing a deep understanding is, I find, much more noble than synthesizing a personal agenda, and as having a discerning audience is an excellent motivation for thorough research, these articles are intended to help me and, hopefully, other readers to develop the deep understanding necessary to truly enjoy an informed debate on Linux control groups, which are also known as "cgroups".

To gain this understanding we will need both a broad perspective and some detailed analysis. The first two articles in this series will try to provide some perspective by first exploring the history of Unix to see what questions it raises about process groups, and then looking at hierarchies, both within and without the Unix family, to give us some yardsticks to measure the hierarchical aspects of cgroups.

Subsequent articles will then delve into the nitty gritty details of cgroups and its various control subsystems and attempt to relate what we find to the questions and metrics the broader perspective gave us.

Sixth Edition Unix

Unix has some history with process groupings and, more significantly, some evolution. Observing this change can help us to see important details. While it would be nice to start at the very beginning, a more practical starting point is the Sixth Edition of Unix, known hereafter as "V6 Unix".

V6 Unix dates from the mid-1970s and was the first edition to get much exposure outside of Bell Labs. It supports two different groupings of processes, though to justify that we should first clarify what we mean by "a grouping of processes".

As in number theory, not every set is a group. The set of processes with a prime identifying number, for example, is certainly a set. However there is no mechanism in Unix (then or now) to distinguish these processes in any way from those with composite ID numbers. The remaining set of processes, with neither a prime nor a composite ID number, does have a distinctive behavior. As it contains only PID 1, though, it is hardly worth considering as a group.

A number-theoretic group includes an operation that operates on members of the group with particular rules for what an "operation" is. For process groups, we will accept a much more vague concept and a different role for an "operation", but still there must be some operation within Unix which can affect, or be affected by, a particular process group.

A less facetious set than the "prime PID" set would be the set of processes owned by a given user ID (or "UID"). We won't consider this to be a group in V6 Unix because while there are operations (e.g. kill) that will affect processes in one group differently from processes in another group, there is no way to interact with the group as a whole.

The first set that really forms a meaningful group is the set of children of some given process. The only operation in V6 Unix which recognizes this group is the wait() system call and it can only detect if the group is empty or not empty. If wait() returns with error ECHILD, then the group is empty. If it returns without an error, or doesn't return, then the set wasn't empty when the call was made (though it might be empty when the call completes).

The same operation could be interpreted as applying to the set of descendants of the given process — that is, the children and any children of those children, etc. ECHILD is returned if and only if this set is empty too. This group has a significantly different behavior, though. In the group of children, a process cannot escape the group except by exiting. In the group of descendants, a process can escape, if it is not an immediate child, when any ancestor of it exits.

Whether the ability to escape is a valuable property of groups or not depends, somewhat, on use-cases and expectations. In V6 Unix, the descendants of PID 1 (that set with a unity ID number) cannot escape but descendants of any other process can. This remained the case for variants of Unix and into Linux until Linux 3.4, when the PR_SET_CHILD_SUBREAPER option for prctl() was added. This allows a process to declare its group of descendant processes as closed so processes cannot escape. If any descendant dies, then all its children are inherited by the process which set this option.

The other, possibly more interesting, process grouping present in V6 Unix is determined by the p_ttyp field in the process structure (defined in proc.h), which is described as the "controlling tty". Whenever a process opens a "tty" device (see dhopen() in dh.c), which would be a serial data connection to a teletype or similar terminal, then if this field is not already set, it will be set to point to the newly opened device. The field is also inherited over a fork() or exec(), so once a process gained a controlling tty, that would continue to apply to the process and all of its future descendants.

One effect of p_ttyp is that any I/O to /dev/tty will go to the controlling tty, but this doesn't really qualify as a "group" operation, as it affects individual processes separately. The "group" operations for controlling ttys involve the delivery of signals (see signal() in sig.c). If a DEL or FS (control-\) character is typed on a tty, then the signal SIGINT or SIGQIT is sent to all processes in the group that have that tty as their controlling tty. Similarly if a disconnect event is detected (like a modem hanging up), a SIGHUP is sent to the same group of processes. Signals can also be sent with the kill() system call. An attempt to send a signal to PID 0 will send it to every process with the same controlling tty as the sending process.

It is quite reasonable to think of this grouping as a prototype of cgroups. It is clearly about the grouping of processes and clearly about controlling those processes — though only through sending a signal. These groups are created automatically, based on behavior, and are permanent — once in a group, the process cannot escape. It appears that they were not perfect, though. The next edition brought changes.

Seventh Edition Unix

While V6 Unix supported process groups, it did not use that terminology. V7 Unix did, and had a richer concept of group. The p_ttyp field still exists, though its role was restricted to managing /dev/tty access. It was renamed to u_ttyp and moved to struct user (user.h) — a structure that could be swapped out to disk with the rest of the process. struct proc (proc.h) instead had a new p_pgrp field to manage process groups. It was set on the first open() of a tty and used for delivering SIGINT, SIGQUIT (which has now gained a 'U'), and SIGHUP, and for delivering signals sent to PID 0. But V7 also brought more flexibility.

The key change was that process groups now had an independent identity and an independent name — independent of the tty, at least. When a process without a controlling tty first opened a tty, a new process group would be created with an ID number matching the process ID number of that process. Though the ID was copied, it really was a new ID for a new object. The group can continue to exist even if the original process exits. Any remaining children will keep the group active and prevent the ID number from being reused, either as a process-group ID or as a process ID.

One consequence of this is that if you log off a tty and log back on again, you get a new process group, and the t_pgrp field in the struct tty structure will be changed. Unlike the situation in V6 Unix, a signal sent to a process group will never go to a process from a previous login on the same tty.

Another consequence is that process groups could be used for more than just ttys. Seventh Edition Unix had a "multiplexor driver" (mpxchan in mx1.c and mx2.c) which, though short-lived, still leaves a legacy in the current stat() manual page:

3000   S_IFMPC         030000   multiplexed character special (V7)
[...]
7000   S_IFMPB         070000   multiplexed block special (V7)

The multiplexor worked a little bit like a socket interface and allowed different processes to connect to each other. An interface was available to form a separate process group for several interconnecting processes, so the master could send a signal to all other members of the group.

V7 Unix process groups were still closed, with processes generally unable to leave them. mpxchan does appear to allow a process to leave its original process group to join a group for a multiplexed channel, but it isn't clear that this was an intended consequence.

Fourth Berkeley Software Distribution

It is a bit of a large jump from V7 to 4BSD, having at least Unix 32v and 3BSD in the meantime. But this is, to some extent, a personal journey and 4.3BSD was the next release that I used.

In 4BSD, we find that a lot has happened with process groups. In 4.3BSD, the set of processes with the same UID has become a group, in that a signal can be sent to all processes in that set (see kill() in kern_sig.c). Sending a signal to a PID of -1 will deliver it to all processes with the same UID as the sending process (though, if sent from a privileged process, the signal will be sent to every process regardless of UID). More significant is that by 4.4BSD there was now a limited hierarchical structure to process groups.

One of the many innovations in the Berkeley versions of Unix was "job control". A "job" here refers to one or more processes working together on a particular task. Unix already had the ability to put some jobs in the "background", but it was implemented in a fairly ad hoc manner. Such processes would be told to ignore any signals from the user (SIGINT and SIGQUIT would both be set to SIG_IGN before starting the process) and the shell would simply not wait for those processes to finish. This mostly worked well, but once a job was in the background, it had to stay there. Also, if such a process wrote to the terminal, its output could get mingled with output from foreground processes, resulting in a mess.

With BSD "job control", each job is placed into its own process group and the shell can tell the terminal to change its idea of which is the current foreground job (and so would receive signals and input and could generate output), and which jobs are in the background so they should be isolated.

The pre-existing concept that process groups were essentially per-login was still important, if only to provide a degree of compatibility with "System V" Unix, a separate path of development from AT&T. In 4.4BSD, these per-login process groups were re-introduced as "sessions". Each process (proc.h) was (potentially) a member of a process group. Each process group was a member of a session. Each terminal (tty.h) had a foreground process group, t_pgrp, and a controlling session, t_session.

Sessions were, and are, much like the V7 Unix process groups, though there are differences. One is that it is not possible to send a signal to all processes in a given session: that functionality only works for process groups, which are now per-job. Another is that a process can leave its session and create a new one by simply calling the setsid() system call.

Either of these are sufficient to frustrate the task of killing all processes at logout — as local policy required in student labs a long time ago in a career far away. A frustration which was, at the time, unfixable due to a dependence on closed-source kernels.

On a modern, windowed desktop, these sessions and process groups are still present, but don't mean quite what they once meant. It is fairly easy to see how session IDs and process-group IDs are assigned by displaying the sess and pgrp fields with ps, as follows:

    ps -axgo comm,sess,pgrp

There is no longer a well defined process grouping for a login session. Instead, each terminal window gets its own session, as do various other applications if they were written to request one. Each job started from the shell prompt still gets its own process group, but there is much less need to start and stop these jobs — rather than suspending the currently running job in one terminal window, it is just as easy to pop up another window and run some new command there.

To properly represent the groupings of processes relevant for a modern desktop, we really need a deeper hierarchy. One level would represent the login sessions, one would represent the applications running in those sessions, and one could be used for jobs within an application. The sessions and process groups that Linux inherits from 4.4BSD can give us only two of those levels. Maybe we can look to cgroups for the third.

Issues

Reflecting on these changes and experiences with process groups, there are a number of issues that may be worth considering when trying to form an opinion on the more modern form of cgroups:

• Names for groups: In V6 Unix the only name was that of the associated resource: a tty. This changed to be an ID number in the same namespace as process ID numbers. In retrospect this sharing of namespaces might seem a little clumsy, though it was clearly convenient. As the kernel was solely responsible for allocating names (another noteworthy feature), any clumsiness remained safely inside that kernel.

• Overlapping uses. The same mechanism was originally used to guide both the delivery of signals and the processing of I/O to /dev/tty. These were quickly separated since they are clearly related, but are not identical.

• Should a process be able to escape its containing group? We have seen a progression in the answer to this, from "no" to "yes". Having the flexibility can be useful in some cases, but having control can be useful in others. Being able to enter a different job under the same session is easy to defend. Being able to create a new session is not so obviously useful for an unprivileged process.

• What role does a hierarchy play? Process groups have only gained even a limited hierarchy toward the end of their development. Is this important? How can it be used?

That last point, hierarchy, certainly is important. A lot of the recent changes in cgroups, and a significant part of the disagreements, relate to hierarchy. While the history of process groups has given us a glimpse of hierarchy it is not enough to develop any real understanding. For that we will need to look elsewhere. In the next installment we will examine a few different "elsewheres" to develop a perspective on hierarchy that we will then take to the inner details of cgroups to see if the former can help us to better understand the latter.

Comments (13 posted)

Fedora and DNF

Fedora has relied on the Yum package manager since its inception, but that won't last much longer. Last month, the Fedora Engineering Steering Committee (FESCo) approved using the DNF package manager by default for Fedora 22. Since I am a Fedora user who mostly manages the installation, updating, and removal of software through Yum on the command line, this upcoming drastic change (albeit one coming nearly a year from now) piqued my interest. With the current version of DNF available in Fedora 20, I decided to investigate DNF to see how well it could replace Yum as default package manager, and how the recent 0.5.2 release and Core DNF Plugins stable release compared to the state of DNF when LWN last looked in on it this past January.

While it was originally released for testing purposes in January 2013 for Fedora 18, DNF development began in 2011. It started a couple of years after Red Hat employee Aleš Kozumplík's work on the Anaconda system installer led him to discover a number of technical quirks with the interaction between the installer and Yum. DNF, which famously doesn't stand for anything, offers a cleaner code base than Yum: in particular, it has an API in the process of stabilizing that will allow developers to write Python plugins to extend its functionality. DNF relies upon two libraries for dependency resolution: hawkey, which provides a Python and C API to libsolv, a library developed by openSUSE for use with its zypper package manager. Libsolv uses a Boolean satisfiability-solving algorithm to resolve dependencies.

When the issue of replacing Yum with DNF came up at the FESCo meeting on June 18, the committee members debated the merits of DNF. The IRC transcript reveals conflicting ideas about how to deal with naming issues; Fedora Project Leader Matthew Miller suggested DNF should be renamed as Yum for Fedora 22, while others debated the merits of users facing a "nag script" to stop using Yum, or automatically running the transaction through DNF when users try to run Yum transactions in Fedora 22. After about 40 minutes of lively discussion, FESCo voted unanimously in favor of using DNF as the default package manger for Fedora 22. However, deciding the details of its implementation, including naming issues, was put off for future meetings.

Controversy about DNF as default dates back to at least this past January, when Harald Reindl raised concerns about missing functionality and system-breaking behavior. Following FESCo's decision, Reindl again raised concerns that some transactions will offer to uninstall core parts of the operating system when certain dependencies are removed. This can include systemd, RPM itself, or the running kernel. That issue is being addressed: Kozumplík suggested the issue could be addressed through a plugin that checks for "protected" packages.

The newly stable Core Plugins package, which is an official project to provide a set of extended functionality to DNF that began in late 2013, offers welcome utility; this will likely go some way toward addressing past complaints of missing features. For example, dnf builddep foo will install needed dependencies based on descriptions in a .spec or .src.rpm file. Users will also welcome the copr plugin, which simplifies managing Copr repositories; Copr provides a system for building unofficial third-party packages for Fedora users, similar to Ubuntu's Personal Package Archive (PPA) feature. Another, non-interactive plugin, generate-completion-cache, speeds up performance by producing data to accelerate command-line completion.

Speed is definitely the most visible improvement for end users familiar with Yum. It was surprising to see how fast DNF resolved dependencies on Fedora 20. For example, DNF resolved dependencies for installing VirtualBox in about 0.5 seconds, while Yum took 2.9 seconds to do so. This speed is probably thanks to libsolv, which will also likely benefit from shared development between the Fedora and openSUSE communities. This speedup alone can be enough to convince one to dnf install foo rather than yum install foo going forward.

DNF's behavior does, of course, differ from Yum, in ways which may surprise veteran Fedora users. For example, DNF doesn't save aborted transactions, while Yum automatically generates a .yumtx file which can be run with yum load-transaction /tmp/filename.yumtx. One of the bigger surprises comes from dnf update, which has completely different output from yum update: it lacks dependency processing information. This is also not a bug, since printing those details would be quite lengthy, as it is with Yum.

The DNF documentation also criticizes the dependency output behavior in Yum for potentially bewildering users if an update turns out to be a sizable transaction. This is not a particularly convincing rationale to me; most users who would be confused by printed dependency resolution output are probably using a graphical front-end like GNOME Software for package management. For those interested in knowing which packages are tied to particular dependencies, the absence of this output may be frustrating, although one can include "-v" with one's transactions to see that output.

DNF and Yum also have different policies for when to schedule metadata updates. This has confused some users in the past. The DNF developers prefer longer periods between metadata updates, largely for resource efficiency; forcing DNF to run a metadata update for a particular transaction can be done by running dnf clean all to clean out the existing metadata before the transaction.

DNF is promising, which probably explains why FESCo approved the still-in-development project for future inclusion as default in Fedora. Its dependency resolution speed makes Yum seem sluggish. That alone may be enough for longtime Yum users to incorporate DNF as part of their package management rhythm. However, certain issues (particularly potential system-breaking through dnf remove foo) have also kept me using Yum, mostly for uninstallation transactions. While DNF does have a number of remaining problems to fix and bugs to squash, the development team does appear to be on top of those issues. I expect that DNF will be a positive (if not universally embraced) change to Fedora's package management system when Fedora 22 is released.

Comments (12 posted)

Distribution quote of the week

Comments (none posted)

Red Hat Enterprise Linux 6.3 Extended Update Support Retirement Notice

Full Story (comments: none)

Ubuntu 14.10 (Utopic Unicorn) alpha-1 released

Full Story (comments: none)

Distribution newsletters

• DistroWatch Weekly, Issue 565 (June 30)
• Five things in Fedora this week (June 25)
• Five things in Fedora this week (July 1)

Comments (none posted)

Combating fragmentation in automotive Linux

Day one of the 2014 Automotive Linux Summit (ALS) in Tokyo concluded with a panel discussion that took aim squarely at one of the well-known but rarely discussed challenges of Linux adoption in the industry: preventing fragmentation. In particular, the automotive Linux marketplace has seen the formation of multiple industry consortia and alliances in recent years, and those groups seem to have overlapping—if not outright competing—goals. Representatives from two of the largest such groups, the GENIVI Alliance and Automotive Grade Linux, sat down for the panel session to discuss how they see their projects co-existing and to announce the beginning of some formal collaboration on software development projects.

The panel included Rudolf Streif from the Linux Foundation (which serves as host of the AGL project), Dan Cauchy from AGL, Dean Mars from GENIVI, and Hisao Munakata from AGL-and-GENIVI-member Renesas. The moderator was automotive industry analyst Roger Lanctot, who started off the session by commenting on how the various industry groups perceive their work in relation to Google's announcement the previous week that it was making a concerted push to enter the automotive software market.

Google is "an uninvited guest in our industry," Lanctot said, and one that has a very different idea of what collaboration means. The Google strategy is to offer its partners what it thinks is "an offer you can't refuse" in the style of The Godfather, he said: it has hundreds of millions of dollars of marketing opportunities to offer in exchange for full access to customers' data. And it also wants automakers to hand total control over the user interface and system-level components to Google, he added, who seems to be planning to deploy Android in car head units just as it has in handsets and tablets.

But the terms of that offer are not, Lanctot thinks, likely to gain much traction. Automakers have strict system-level requirements that they know Android cannot meet, he said—including fast boot time, low power consumption, privacy protection, and integration with unique data services (such as traffic information). He also commented that automakers have always sought to differentiate their products from each other's through customizing the user interface, which makes Google's "one UI fits all" approach a non-starter.

But the most fundamental problem, he said, is that Google seems to be saying that it can impose a single regime on the automotive software industry and, thus, prevent fragmentation. The industry itself prefers to avoid fragmentation by forming collaborative alliances—and the number of companies who participate in GENIVI and AGL dwarfs the number who have agreed to work with Google.

Of course, that still leaves open the fact that GENIVI and AGL have similar (though not identical) goals, which could lead to fragmentation on its own. Lanctot asked the panelists to explain how the two projects differ and are alike. His own assessment, he said, is that GENIVI wants to create a set of packages that carmakers can add to their own Linux distributions, whereas AGL wants to build a full Linux distribution of its own.

The panel more or less concurred with that assessment; Cauchy (who commented that before he worked with AGL he had been in charge of compliance efforts at GENIVI) contended that both approaches are needed. Some OEMs will always prefer to build more of their system in-house, after all. Mars noted that the origins of the organizations were quite different as well. GENIVI is an industry alliance created with the concept of member companies sharing assets, while AGL was envisioned as a "do everything in the open" collaborative project. Where they overlap, he said, GENIVI has started to open up its assets, but it still defaults to working within the confines of its own program.

Lanctot asked Streif whether or not these different approaches would create friction. Streif's answer was that the distinction was mainly one of communication styles, not opposing views of end products. He has worked on a large number of embedded Linux projects over the years, he said, and everyone knows that creating a product is a lot of work. AGL wants to create a baseline that you can "download and use" right away, he said, which is something that has become more difficult to do in recent years as embedded Linux distributions have grown. But he compared AGL to Debian or Fedora: anyone can download and use them as-is, but that does not prohibit the possibility of commercial vendors like Red Hat and Canonical building businesses on top of them.

The collaboration between AGL and GENIVI will begin in earnest with the Media Manager project. Media Manager is a component for providing access to multimedia files on connected consumer electronics devices (e.g., phones and tablets) to applications running on the car's in-vehicle infotainment (IVI) head unit. The goal is to enable any media player application to browse, search, and play back content (including access to metadata like artist information and remaining play time) found on a connected device, regardless of the device specifics.

As the panel indicated, however, they do not intend for Media Manager to be a one-shot collaboration. GENIVI has had a head start of several years, and is already shipping in several BMW vehicles. AGL has only recently set out to create formal specification documents, but those specifications will standardize aspects of AGL's "preview" software releases, which have long included GENIVI software components. Streif pointed out that automotive, like every vertical market, struggles with Not Invented Here (NIH) syndrome, but he said that Linux inevitably overcomes NIH. Munakata concurred on that point; he said that it was important to note that AGL was only launched the same day that GENIVI was releasing its first software projects. Nevertheless, history indicates that "going it alone" can never match the speed and development power of a collaborative project.

The panel touched on several other interesting topics, such as the impact that Tesla Motors has had. The panelists agreed that its disruptive effect is highly beneficial, as, for example, its radically different IVI design demonstrates that new things are possible, whereas other carmakers have often just used Linux to re-implement the same systems that they have been shipping in cars for 20 years. They also discussed when US automakers will get on board with automotive Linux collaborations, predicting that it will be soon, since many of them are already exploring Linux individually. In response to an audience question, the panel commented on the value of building IVI systems that consumers can customize and modify themselves; all agreed that it is valuable, and said OEMs are supportive of it.

Historically, how GENIVI, AGL, and other automotive Linux efforts interrelate has been a subject of some confusion; the fact that GENIVI and AGL are committed to working together on projects moving forward may not simplify matters, but it undoubtedly makes it clear that the various players involved are all working toward the same goal.

[The author would like to thank The Linux Foundation for travel assistance to attend ALS 2014.]

Comments (none posted)

Quotes of the week

Comments (none posted)

PyOpenGL 3.1.0 available

PyOpenGL 3.1.0 has been released. Python 2.5 support has been dropped, experimental support for GLES and EGL has been added, and there have been several additional changes—such as "better isolation and pervasive lazy-loading behaviour to prevent loading unused libraries (e.g. GLUT in non-GLUT contexts or GLES in OpenGL contexts)" and more automated generation of wrappers.

Full Story (comments: none)

Kcov v19 available

Version 19 of the code-coverage collection utility kcov has been released. Most notable in this new release is support for analyzing bash and sh shell scripts.

Comments (none posted)

Development newsletters from the past week

• What's cooking in git.git (June 26)
• LLVM Weekly (June 30)
• OCaml Weekly News (July 1)
• OpenStack Community Weekly Newsletter (June 27)
• Perl Weekly (June 30)
• PostgreSQL Weekly News (June 29)
• Python Weekly (June 26)
• Ruby Weekly (June 26)
• This Week in Rust (June 30)
• Tor Weekly News (July 2)
• Wikimedia Tech News (June 30)

Comments (none posted)

Where KDE is going - Part 1 (KDE.news)

Comments (none posted)

Where KDE is going - Part 2 (KDE.news)

Comments (21 posted)

Day: In praise of Jim Hall

At his blog, Allan Day points GNOME users and developers to some new usability research about GNOME 3.10 and 3.12 conducted by Jim Hall, a graduate student at the University of Minnesota. Day has started filing a number of bug reports based on Hall's findings, including problems with Nautilus bookmarking and confusion over the purpose of the GNOME Software application. The full data set is not yet available online, but Hall is scheduled to present it at GUADEC in July.

Comments (39 posted)

Free Software Supporter - Issue 75, June 2014

Full Story (comments: none)

Nelson: The new 501(c)(3) and the future of free software in the United States

Comments (19 posted)

Embedded Linux Conference Europe - Call for Participation

Full Story (comments: none)

CFP Deadlines: July 3, 2014 to September 1, 2014

If the CFP deadline for your event does not appear here, please tell us about it.

Events: July 3, 2014 to September 1, 2014

If your event does not appear here, please tell us about it.