The Internet of bricks

One of the promises of the "Internet of things" is that it gives us greater control over our homes, gadgets, and more. Free software also offers that sort of promise, along with the idea that, if necessary, we can support our own gadgetry when the manufacturer moves on to some new shiny object. The currently unfolding story of the Revolv hub shows that, in many cases, these promises are empty. The devices we depend on and think we own can, in fact, be turned into useless bricks at the manufacturer's whim.

The Revolv "M1" home-automation hub was one of many products designed to bring home control to the Internet. It is able to control lights, heating, and more, all driven by smartphone-based applications. The product was sufficiently successful to catch the eye of the business-development folks at Nest, who acquired the company; Nest was acquired in turn by Google, and is now a separate company under the "Alphabet" umbrella.

The acquisition of Revolv seemed to be driven primarily as a shopping expedition for engineering talent. Revolv's engineers were quickly moved over to Nest products, with little of interest happening with their own product line. But, as can be seen on the cheery note on the revolv.com front page, the distraction of Revolv's engineers goes beyond neglect of their older products:

Purchasers of Revolv hubs, who might have been under the mistaken impression that they owned their devices, may be less than incredibly excited to learn that, in just over one month, they will be left with inactive electronic waste. They may have deployed these devices in distant homes that will be difficult to get to in time, or they may have just had the devices to play around with. Either way, it is understandable that they would feel like they have been unfairly dealt with.

One might be tempted to point out that abruptly shutting down programs and services is becoming a standard Google technique. The "MyTracks" Android app was once allegedly open source; Google has pulled the source off the net and pronounced that, as of April 30, it will simply cease to work. It is one thing to decide to stop providing a free network service; it's another to vandalize an app (or a device) that should be fully capable of running on its own. But, in any case, this problem goes beyond Google (or, more correctly, Alphabet).

Much of the Revolv system is built on free software; the company's (still available) open-source licenses page suggests that the whole thing is based on Ångström Linux. If Revolv customers actually owned their devices, they would be able to keep that base system running themselves, even if the (presumably) proprietary higher-level software was remotely disabled. But, on a locked-down system, the freedom that comes from using free software has been taken away. Customers are in possession of a box that runs Linux and has home-automation capabilities, but those capabilities are no longer available to them. Free software has not prevented the bricking of this device. Similarly, the release of MyTracks as free software has not kept it from being remotely disabled.

The real problem with both Revolv and MyTracks is arguably the network-service component. But neither should need that component to be able to perform most of its functions. The Revolv hub should be able to carry out its basic functions without recourse to any sort of Internet service at all; how else is it going to work if the network connection is down? About the only thing that might be at all challenging is remote control from an app that is not on the local network. On a free device, there would be a way to replace that service, but that is not the situation here.

The situation with MyTracks is similar; there is no need for a network service to collect location data. Displaying it uses the same mapping APIs many other apps use. There is no reason why it had to be remotely shut down. Truly free software would not support that sort of antifeature, but, when one runs proprietary software, that level of control is lost. The surprise, perhaps, was that MyTracks, as found in the "Google Play Store," was still a proprietary product, even if it was allegedly free software.

What can we learn from this experience? We didn't really need reminders that proprietary software can act against the interests of its users, but we have one here anyway. Maybe, someday, there will have been enough cases of bad behavior in proprietary software to create a market for consumer devices that are fully open, but that day does not appear to be coming soon. Until then, people with the necessary skills can often create their own systems, but that is not an option for the bulk of people who do not have the skills, the time, or the interest to do that.

Given that devices running proprietary software can so easily go dark (or turn actively malicious), it would seem that a great deal of caution toward "Internet of things" gadgetry is warranted. Toys are fun, but perhaps one's cookware or toothbrush do not need to be connected to the net at this time. For any such device, one should ponder what would happen if it abruptly ceased functioning, or all data that it is able to report were to end up in hostile hands. Do not assume that the presence of free software makes a device more free, trustworthy, or supportable.

It would not be surprising if Alphabet/Nest were to back down, temporarily at least, on the Revolv shutdown; this is the kind of thing that attracts class-action lawsuit lawyers in the US. But this action is a clear example of what it means to lack control over devices that you have purchased; when that control lies elsewhere, others will inevitably abuse it. We know what is needed to build a network full of devices that we can at least hope to trust — openness and freedom — but we have not yet succeeded in making that point outside of a narrow technical community. Unfortunately, it may take a number of high-profile incidents before the world understands.

Comments (51 posted)

Why Zephyr?

When the Zephyr project was announced in February, many members of the Linux community seemed to find it puzzling. Although Zephyr is hosted by the Linux Foundation, it is an entirely separate operating system that incorporates no Linux code. It also targets small hardware devices—from Arduinos to ARM system-on-chips (SoCs)—even though there are several other open-source OS projects (including Linux for many ARM SoCs) that also address that device class. At the 2016 Embedded Linux Conference in San Diego, the Zephyr team was on hand to make the case for why the new project is necessary and, hopefully, interesting to developers.

OSes of Things

Intel's Anas Nashif presented an overview of Zephyr in his talk, describing both its background within Intel and its capabilities. The bottom-line goal of the project is to provide an open-source, realtime operating system (RTOS) for "sub-Linux devices." Beyond that, however, Intel looked at the existing RTOS offerings on the market and found them lacking in several respects. First, many of them focus only on one architecture, which limits their appeal.

Second, many of them are "roll your own" projects developed for research or maintained by a single person. That seems to correlate with a lack of security work, which is a serious drawback. In addition, some of these smaller RTOS projects use peculiar software licenses, which also inhibits their adoption within the open-source community. Finally, many of the existing projects are too limited in functionality to serve as a general-purpose OS for Internet of Things (IoT) devices, which Intel sees as the primary use case for Zephyr.

Put all of those factors together, and one has the Zephyr product brief: an open-source RTOS that supports multiple architectures, has a broad contributor base, and offers "more than just a scheduler"—specifically, good networking support and a real security framework. Intel happened to acquire the rights to Wind River's microcontroller RTOS, decided it could be developed into a good fit, and set about to clean up the code and prepare it for release. That RTOS is now the open-source Zephyr, although it also serves as the core of Wind River's "Rocket" RTOS product. Intel is currently leading the development effort.

By building on the existing Wind River code base, Nashif said, Zephyr can start with a battle-tested code base used for more than a decade in real-world conditions. Expanding the original code involved several key steps. First, it was restructured to be more modular. At compile time, developers can include or exclude whichever subsystems they need in order to fit the hardware. The hardware in question includes what he called "the very lowest end of the sub-Linux device spectrum", "where the cost of the silicon is minimal." He showed a slide listing the initial hardware products on which Zephyr runs, which includes the Arduino 101 and Intel Quark D2000 at the low end, and goes all the way up to the second-generation Intel Galileo at the high end. But, he added, the Galileo is a development platform only; it is fully capable of running Linux, so he expected no one would use it for a real Zephyr deployment.

Second, the Zephyr team added networking. It wrote a Bluetooth Low Energy (BLE) implementation from scratch, and ported the IP stack from the open-source Contiki RTOS. IPv6 is supported, include the low-energy 6LoWPAN. The project is still looking at other network stacks; Near-Field Communication (NFC) is likely; ZigBee and WiFi are also possibilities if there is demand.

The third addition to the code base was a cryptographic library based on TinyCrypt. In a separate session, security developer Constanza Heath described TinyCrypt's set of random-number generation (RNG), cipher, and key-exchange primitives, as well as Zephyr's general approach to security. A Zephyr image runs only a single application, which is statically linked in at compile time. There is a single address space and no loadable kernel modules are supported, which reduces the attack surface considerably. In brief, she said, with a single, statically linked process running, if there is malicious code running on your Zephyr hardware, then it was there at compile time. The project thus skips some other security features that larger OSes include to protect against security exploits, though she said the team is still looking into adding some protections like address-space randomization. For most security features, the question at hand is the security-versus-size trade-off.

Features

When it comes to size, Zephyr can run comfortably in 8KB of RAM, and can even run in a minimum of 2KB of RAM, although Nashif said that at that size it provides nothing but a "Hello World" demo. The Zephyr kernel can be configured in either one of two modes. The smallest is "nanokernel" mode, which provides a basic multi-threaded execution environment, inter-thread synchronization services (including semaphores and mutexes), message queues, and interrupt services. Nanokernel mode is expected to be used for the smallest and cheapest devices, he said, such as environmental sensors, which need to do little processing beyond gathering and sending data.

A step up from nanokernel mode is microkernel mode, which offers a superset of nanokernel mode's features, including more sophisticated message queuing, additional memory allocation services, preemptive tasks, and round-robin time slicing. This mode is expected to be used on slightly larger IoT nodes, such as sensor hubs that may need to process and format data collected from sensors before forwarding it on to a remote server. Nashif pointed out that networking was available in both nanokernel and microkernel mode, although configuring a nanokernel build for the full IP networking stack could potentially make it hard to fit Zephyr into a sub-10KB RAM device.

Nashif noted Zephyr also makes use of some tooling from Linux. Configuration is done using the Linux kernel's kconfig, and Zephyr builds are done using kbuild. But the similarities stop with those tools, he said—a Zephyr image is configured and built for a single application. "We're just trying to learn from Linux, not to be Linux." The project offers a software development kit (SDK) that supports five compilers and provides tools for flashing images onto devices and for debugging. In addition to running on Linux, Windows, and Mac OS X, the SDK can run in Docker containers.

Intel sees Zephyr as a strategic investment, Nashif said. It saw gaps in the RTOS market, particularly where IoT devices are concerned, and took the opportunity to try building a project to fill the gap. It hopes to win over the community, and hopes to work with other open-source projects wherever possible. Nashif added that he knows the IoT space is a crowded one; he has seen two new IoT OSes announced since Zephyr was released, and does not expect that trend to slow down.

Whoever establishes dominance in the IoT market will have to overcome quite a bit of competition, both proprietary and open source. It is quite early in Zephyr's lifespan, even though the code base has a longer history through its days at Wind River. Wherever it goes from here, it will be interesting to watch.

[The author would like to thank the Linux Foundation for travel assistance to attend ELC 2016.]

Comments (28 posted)

Redox: a Rust-based microkernel

Creating a new operating system from scratch is a daunting task—witness the slow progress of GNU Hurd, for example. But it would be hard to argue that the existing systems are the be-all and end-all of how we interact with computer hardware. At least some of the deficiencies in today's offerings can arguably be traced to the language used to implement them; C, for all its power, has some fairly serious flaws that lead to bugs and security holes of various sorts. So it is interesting to see a new entrant that uses the Rust language, which is focused on memory safety. The result is Redox, which is still far from ready for everyday use, but may provide an interesting counterpoint to Linux, the BSDs, OS X, Windows, and others.

As laid out in the Redox book (which is also a work in progress), Rust was chosen for its memory and type safety. Some of those features were described in an LWN article back in 2013. The Redox developers see these safety features as a critical part of a modern operating system, even one that is based on the much-older Unix model:

So the goal is to eliminate whole classes of problems that plague C programs, but the focus on security doesn't stop there. Unlike Linux, Redox has a microkernel design, where much of the code that resides in a monolithic kernel like Linux is moved to user space. A microkernel simply provides resource management, interprocess communication (IPC), process/thread management and scheduling, system calls, and little else. The result is a much smaller kernel (16,000 lines of code for Redox and plans to reduce that further) with a concomitant reduction in the attack surface, as well as a much smaller amount of critical kernel code to audit.

In Redox, as in MINIX that it is partly patterned after, drivers live in user space so they can do far less damage when they fail. In monolithic kernels, a bad driver can crash the whole system because it runs in the kernel's address space and with its privilege level. So, for Redox, filesystems, device drivers, networking, and so on are all implemented in user space.

Beyond that, Redox has taken the Unix "everything is a file" philosophy (which was further expanded by Plan 9) and taken it a step further. In Redox, everything is a URL (or, really, a URI), which means that the API is based on using URLs to access local and remote resources: files, networking protocols and sites, devices, and so forth.

The "scheme" portion of a URI (e.g. the "http:" in a web URL) is used to identify the type of the resource being accessed. These are implemented as Redox schemes, which can be implemented in user space or the kernel, though user space is preferred. Schemes are effectively virtual filesystems with a set of operations made available that are appropriate to the type. As the book notes: "'Everything is a scheme, identified by an URL' is more accurate, but not very catchy."

The Redox wiki goes into a bit more detail of how URLs and schemes are used. For example, tcp://10.11.12.13:80 would be used for a remote TCP connection, while file:/apps/terminal/example.txt refers to a local file. In the networking wiki page, there is mention of using a URL like tcp:///80 for a web server or ethernet://12:34:56:78:9A:BC/800 to receive or send IPv4 (type 800) packets for the specified MAC address.

Underneath the schemes are resources, which come in two flavors: file-like (i.e. blocking and buffered) and socket-like (i.e. non-blocking and stream-like). Resources are closely related to schemes, as the book notes:

While the kernel is the main focus of the project currently, there are some side projects of interest as well. To start with, ZFS is planned as the filesystem of choice for Redox, which means writing a compatible implementation in Rust or adapting the existing open-source version (OpenZFS). There is a minimal graphics server, orbital, that is based on the Wayland protocol. There is also a vi-like editor (Sodium), games, and more.

The code is available on GitHub and has been released under the MIT license. The explanation for choosing the MIT license has been somewhat controversial among some GPL advocates, but the developers clearly want Redox to spread far and wide—proprietary forks and extensions are explicitly condoned. ISO images for various targets are available, though they are yet another work in progress. The book also has some instructions (starting here) for those who want to build Redox for themselves.

A Discourse forum is provided to discuss Redox and its development. There are contribution guidelines and ideas, a best practices page, and the inevitable issues list as well. Those interested can keep up with the project by checking out the "This Week in Redox" newsletters, though they are rather more sporadic than the name would imply. The Code of Conduct is fairly extensive and clearly shows an intent to foster an inclusive community. In short, there is quite a bit of information available about Redox, though it is somewhat scattered.

One question that many will ask is "why a new operating system?" The Redox developers have an answer that boils down to dissatisfaction with Linux, the BSDs, and, even, MINIX (which is "the most in line with Redox's philosophy"). There is an admitted "Not Invented Here" sentiment within the project. But one thing that is explicitly not a goal for the project is replacing Linux. A one-word answer ("No") to the "will Redox replace Linux?" question is contained in the book.

That's probably a good thing for a number of reasons. For one thing, hardware support is a huge task that is unlikely to be surmounted without an enormous effort from lots of companies—something that the MIT license might make difficult. Like it or not, the reciprocal nature of the GPL certainly seems to be a driving force for the corporate involvement in Linux. It is also an area where more permissively licensed alternatives (the BSDs and MINIX) have run aground to some extent.

But there are other hurdles as well. Redox has deliberately chosen to implement a minimal subset of the Linux (and POSIX) system calls. That will make it more difficult to port existing applications to Redox. That is partly by design, however, as a Rust-based user-space ecosystem is envisioned, but the huge number of existing freely licensed applications, which may be difficult to port, could be a big impediment to Redox adoption. Longstanding issues with the performance of microkernels (which require many more context switches than monolithic kernels do) may also make Redox a hard sell for the more lucrative server segment—though performance for desktops may turn out to be reasonable, especially if the security gains are substantial.

It is clearly an ambitious project—perhaps even overly ambitious; only time will tell. It does bear watching, though, as an experiment in several dimensions. The microkernel design and Rust-based implementation will provide answers (or at least evidence in one direction or another) to some questions that have been floating around for some time. We clearly don't have all the operating system answers for the future, so more data is called for. Redox might just provide some of that data.

Comments (6 posted)

The LoadPin security module

The merging of security-module stacking was supposed to jump-start the creation of a whole set of small, special-purpose security modules. By making it possible for security modules to play well together, stacking enables modules that solve one little problem rather than implementing an entire security policy. Thus far, the flood of new modules has yet to begin, but the LoadPin module from Kees Cook demonstrates the sort of problem that little security modules might be written to solve.

Security-conscious developers have long worried about code that is loaded into the kernel at run time; such code could clearly, if malicious, subvert the security of the entire system. Mechanisms like the signing of kernel modules are designed to prevent the loading of such code. In the end, though, a signature on a loadable module really only guarantees one thing: where that module came from. If the provenance of kernel modules (or other files loaded into the kernel) could be ascertained by other means, it would not be necessary to sign them.

ChromeOS keeps its security-sensitive files, including kernel modules, in read-only storage; that storage is verified as a whole before the system boots. In such a setting, the loading of modules from the read-only partition is safe regardless of whether they have been signed; they are known to come from Google and cannot have been modified since. So a reasonable security policy for ChromeOS systems might well be to load modules (without signature checking) from the secure partition, while rejecting the loading of modules from anywhere else.

That is the policy that LoadPin was created to implement. It takes advantage of the relatively new kernel file-loading mechanism to intercept all attempts to load a file into the kernel; these include loading kernel modules, reading firmware, loading a security policy, or loading an image for kexec(). In each case, the filesystem containing the file to be loaded is compared against the filesystem used in the first load operation after boot; if the two don't match, the attempt is rejected. All that is required is to build the security module into the kernel; there is no other configuration needed or possible.

The resulting mechanism is, obviously, quite simple in nature. Administrators who want to enable file loading from multiple filesystems, or who want to change the allowed filesystem during the lifetime of the system will be out of luck. But, for the simple case where the system boots from a trusted, read-only image, LoadPin does all that is needed.

In truth, the interface isn't quite that simple. If (and only if) the first load operation is satisfied by a writable filesystem, the LoadPin module will log a warning and establish a sysctl knob (kernel/load_pinning) that can be used to turn the feature on or off. This, of course, is a debugging mechanism; normally one would not want to be able to write to the filesystem containing the trusted modules. After all, the immutability of the trusted filesystem is the foundation on which the trust in its contents is being built. On a production system, the ability to disable pinning would also be undesirable.

It would not be surprising if other developers eventually showed up wanting to add more complex configuration options to this module. One could imagine wanting to apply different policies to different types of files (firmware or kexec() images), for example. The current module is also likely to run into trouble on systems that boot with an initramfs image; the first modules will almost certainly be loaded from that image (that's why it exists, usually), causing loads to be pinned to a temporary filesystem that will go away at the end of the bootstrap process. In the current patch, if the filesystem to which loading is pinned disappears, loading of files will be disabled entirely — behavior that makes sense, but which may not lead to the desired results in an initramfs setting.

But that is work for later, should somebody decide that it is needed. For now, the simple security module would appear to be enough for Google's needs. Indeed, the copyright dates in the code suggest that it has been in use there since 2011.

As Casey Schaufler suggested in his talk on writing one's own security module, the stacking mechanism makes it possible to implement a wide variety of possible policies. LoadPin, perhaps, will be the start of a series of modules from developers who have extended the kernel's security mechanisms in novel ways. The potential is there for some interesting ideas to be contributed back to the kernel. Given that relatively few people seem to think that we have solved all of the security problems with what we have now, that seems like it should be a step in the right direction.

Comments (none posted)

Security quotes of the week

— Ricky Lawshae on a vulnerability in HID door controllers in the Trend Micro blog

— Matthew Garrett reacts to the HID vulnerability

— Juliette Kayyem, former assistant secretary at the US Department of Homeland Security

Comments (22 posted)

OSVDB: FIN

The Open Sourced Vulnerability Database (OSVDB) has been shut down. "This was not an easy decision, and several of us struggled for well over ten years trying to make it work at great personal expense. The industry simply did not want to contribute and support such an effort. The OSVDB blog will continue to be a place for providing commentary on all things related to the vulnerability world." (Thanks to Paul Wise)

Comments (none posted)

apache-commons-collections: code execution

Package(s):	apache-commons-collections	CVE #(s):	CVE-2015-8103
Created:	April 4, 2016	Updated:	April 14, 2016
Description:	From the CVE entry: The Jenkins CLI subsystem in CloudBees Jenkins before 1.638 and LTS before 1.625.2 allows remote attackers to execute arbitrary code via a crafted serialized Java object, related to a problematic webapps/ROOT/WEB-INF/lib/commons-collections-*.jar file and the "Groovy variant in 'ysoserial'".
Alerts:	Mageia MGASA-2016-0137 apache-commons-collections 2016-04-13 Fedora FEDORA-2016-dc3740c56e apache-commons-collections 2016-04-02 Fedora FEDORA-2016-0c5838abc5 apache-commons-collections 2016-04-02

Comments (none posted)

Chromium: denial of service

Package(s):	Chromium	CVE #(s):	CVE-2016-3679
Created:	April 1, 2016	Updated:	April 6, 2016
Description:	From the CVE entry: Multiple unspecified vulnerabilities in Google V8 before 4.9.385.33, as used in Google Chrome before 49.0.2623.108, allow attackers to cause a denial of service or possibly have other impact via unknown vectors.
Alerts:	Ubuntu USN-2955-1 oxide-qt 2016-04-27 openSUSE openSUSE-SU-2016:1059-1 Chromium 2016-04-16 openSUSE openSUSE-SU-2016:0930-1 Chromium 2016-04-01 openSUSE openSUSE-SU-2016:0929-1 Chromium 2016-04-01

Comments (none posted)

file: buffer over-write

Package(s):	file	CVE #(s):
Created:	April 6, 2016	Updated:	April 6, 2016
Description:	From the Mageia advisory: The file command was vulnerable to a buffer over-write with a malformed magic file.
Alerts:	Mageia MGASA-2016-0132 file 2016-04-06

Comments (none posted)

kernel: timing side channel vulnerability

Package(s):	kernel	CVE #(s):	CVE-2016-2085
Created:	April 6, 2016	Updated:	April 6, 2016
Description:	From the Ubuntu advisory: Xiaofei Rex Guo discovered a timing side channel vulnerability in the Linux Extended Verification Module (EVM). An attacker could use this to affect system integrity.
Alerts:	Oracle ELSA-2016-3596 kernel 4.1.12 2016-08-26 Oracle ELSA-2016-3596 kernel 4.1.12 2016-08-26 Mageia MGASA-2016-0233 kernel-tmb 2016-06-22 Mageia MGASA-2016-0232 kernel-linus 2016-06-22 Mageia MGASA-2016-0225 kernel 2016-06-13 Ubuntu USN-2948-2 linux-lts-utopic 2016-04-11 Ubuntu USN-2947-3 linux-raspi2 2016-04-06 Ubuntu USN-2947-2 linux-lts-wily 2016-04-06 Ubuntu USN-2949-1 linux-lts-vivid 2016-04-06 Ubuntu USN-2948-1 linux-lts-utopic 2016-04-06 Ubuntu USN-2946-2 linux-lts-trusty 2016-04-06 Ubuntu USN-2946-1 kernel 2016-04-06 Ubuntu USN-2947-1 kernel 2016-04-06

Comments (none posted)

kubernetes: improper admission check control

Package(s):	kubernetes	CVE #(s):
Created:	April 1, 2016	Updated:	April 6, 2016
Description:	From the Red Hat bug report: It was found that patch will check admission control with an empty object and if it passes, then will proceed to update the object with the patch. Admission control plugins don't get a chance to see/validate what is actually going to be updated.
Alerts:	Fedora FEDORA-2016-a89f5ce5f4 kubernetes 2016-04-01

Comments (none posted)

lhasa: code execution

Package(s):	lhasa	CVE #(s):	CVE-2016-2347
Created:	April 4, 2016	Updated:	April 15, 2016
Description:	From the Debian advisory: Marcin Noga discovered an integer underflow in Lhasa, a lzh archive decompressor, which might result in the execution of arbitrary code if a malformed archive is processed.
Alerts:	Arch Linux ASA-201604-8 lhasa 2016-04-14 openSUSE openSUSE-SU-2016:1027-1 lhasa 2016-04-13 openSUSE openSUSE-SU-2016:1029-1 lhasa 2016-04-13 Debian DSA-3540-1 lhasa 2016-04-03

Comments (none posted)

libebml: use-after-free vulnerability

Package(s):	libebml	CVE #(s):	CVE-2015-8789
Created:	March 31, 2016	Updated:	April 6, 2016
Description:	From the Debian advisory: Context-dependent attackers could trigger a use-after-free vulnerability by providing a maliciously crafted EBML document.
Alerts:	Debian DSA-3538-1 libebml 2016-03-31

Comments (none posted)

libqt4: unsafe SSL ciphers

Package(s):	libqt4	CVE #(s):
Created:	March 31, 2016	Updated:	August 31, 2016
Description:	From the openSUSE advisory: Various unsafe SSL ciphers have been disabled in the standard SSL classes. Also the RC4 based ciphers have been disabled.
Alerts:	openSUSE openSUSE-SU-2016:2199-1 libqt4 2016-08-30 openSUSE openSUSE-SU-2016:0922-1 libqt4 2016-03-31

Comments (none posted)

mercurial: three vulnerabilities

Package(s):	mercurial	CVE #(s):	CVE-2016-3630 CVE-2016-3068 CVE-2016-3069
Created:	April 4, 2016	Updated:	May 3, 2016
Description:	From the Slackware advisory: This update fixes security issues and bugs, including remote code execution in binary delta decoding, arbitrary code execution with Git subrepos, and arbitrary code execution when converting Git repos.
Alerts:	Scientific Linux SLSA-2016:0706-1 mercurial 2016-05-02 Oracle ELSA-2016-0706 mercurial 2016-05-02 CentOS CESA-2016:0706 mercurial 2016-05-02 Red Hat RHSA-2016:0706-01 mercurial 2016-05-02 openSUSE openSUSE-SU-2016:1073-1 mercurial 2016-04-17 Mageia MGASA-2016-0138 mercurial 2016-04-13 openSUSE openSUSE-SU-2016:1016-1 mercurial 2016-04-12 SUSE SUSE-SU-2016:1011-1 mercurial 2016-04-12 SUSE SUSE-SU-2016:1010-1 mercurial 2016-04-12 Fedora FEDORA-2016-79604dde9f mercurial 2016-04-07 Fedora FEDORA-2016-b7f1f8e3bf mercurial 2016-04-07 Arch Linux ASA-201604-6 mercurial 2016-04-06 Debian DSA-3542-1 mercurial 2016-04-05 Slackware SSA:2016-092-01 mercurial 2016-04-01 Gentoo 201612-19 mercurial 2016-12-07

Comments (none posted)

networkmanager: multiple vulnerabilities

Package(s):	NetworkManager	CVE #(s):
Created:	April 4, 2016	Updated:	April 6, 2016
Description:	From the Fedora advisory: [ 1 ] Bug #1268617 - NetworkManager: get_ip_iface_identifier(): NetworkManager killed by SIGABRT [ 2 ] Bug #1270247 - NetworkManager: nm_device_get_unmanaged_flag(): NetworkManager killed by SIGSEGV [ 3 ] Bug #1241198 - NetworkManager: nm_supplicant_interface_get_scanning(): NetworkManager killed by SIGSEGV [ 4 ] Bug #1298007 - NetworkManager: nm_supplicant_interface_get_scanning(): NetworkManager killed by SIGSEGV
Alerts:	Fedora FEDORA-2016-8201e3fefa NetworkManager 2016-04-03

Comments (none posted)

oar: privilege escalation

Package(s):	oar	CVE #(s):	CVE-2016-1235
Created:	April 6, 2016	Updated:	April 6, 2016
Description:	From the Debian advisory: Emmanuel Thome discovered that missing sanitising in the oarsh command of OAR, a software used to manage jobs and resources of HPC clusters, could result in privilege escalation.
Alerts:	Debian DSA-3543-1 oar 2016-04-05

Comments (none posted)

optipng: code execution

Package(s):	optipng	CVE #(s):	CVE-2016-2191
Created:	April 5, 2016	Updated:	June 27, 2016
Description:	From the Arch Linux advisory: An invalid write may occur in optipng before version 0.7.6 while processing bitmap images due to `crt_row' being (inc|dec)remented without any boundary checking when encountering delta escapes. This issue can possibly be used to execute arbitrary code.
Alerts:	Gentoo 201608-01 optipng 2016-08-10 Fedora FEDORA-2016-b8f91621c7 optipng 2016-06-25 Ubuntu USN-2951-1 optipng 2016-04-18 openSUSE openSUSE-SU-2016:1082-1 optipng 2016-04-17 openSUSE openSUSE-SU-2016:1078-1 optipng 2016-04-17 Mageia MGASA-2016-0135 optipng 2016-04-13 Debian DSA-3546-1 optipng 2016-04-07 Arch Linux ASA-201604-5 optipng 2016-04-04

Comments (none posted)

php: multiple vulnerabilities

Package(s):	php	CVE #(s):	CVE-2015-8865 CVE-2016-4070 CVE-2016-4071 CVE-2016-4072 CVE-2016-4073 CVE-2016-8866 CVE-2016-8867
Created:	April 4, 2016	Updated:	December 22, 2016
Description:	PHP 5.6.20 fixes numerous bugs. See the PHP changelog for details.
Alerts:	Red Hat RHSA-2016:2750-01 rh-php56 2016-11-15 Oracle ELSA-2017-3511 docker-engine docker-engine-selinux 2017-01-17 Oracle ELSA-2017-3511 docker-engine docker-engine-selinux 2017-01-17 Gentoo 201701-42 file 2017-01-17 SUSE SUSE-SU-2016:1638-1 php53 2016-06-21 SUSE SUSE-SU-2016:1581-1 php53 2016-06-14 Debian-LTS DLA-499-1 php5 2016-05-31 openSUSE openSUSE-SU-2017:0023-1 ImageMagick 2017-01-04 Ubuntu USN-2984-1 php5, php7.0 2016-05-24 openSUSE openSUSE-SU-2016:1373-1 php5 2016-05-20 SUSE SUSE-SU-2016:1277-1 php5 2016-05-11 openSUSE openSUSE-SU-2016:1274-1 php5 2016-05-11 Debian-LTS DLA-460-1 file 2016-05-07 SUSE SUSE-SU-2016:3258-1 ImageMagick 2016-12-23 openSUSE openSUSE-SU-2016:3233-1 ImageMagick 2016-12-22 openSUSE openSUSE-SU-2016:3238-1 GraphicsMagick 2016-12-22 Debian-LTS DLA-756-1 imagemagick 2016-12-21 Debian DSA-3560-1 php5 2016-04-27 Fedora FEDORA-2016-9282d83bee php 2016-04-09 Fedora FEDORA-2016-1cf1b49047 php 2016-04-09 Mageia MGASA-2016-0131 php 2016-04-06 Slackware SSA:2016-092-02 php 2016-04-01 openSUSE openSUSE-SU-2016:3009-1 containerd docker runc 2016-12-05 Gentoo 201611-22 php 2016-12-01 Fedora FEDORA-2016-15cf686c8d docker 2016-11-19 Fedora FEDORA-2016-8e1558d1c6 docker 2016-11-19

Comments (none posted)

squid: denial of service

Package(s):	squid	CVE #(s):	CVE-2016-3947
Created:	April 4, 2016	Updated:	April 6, 2016
Description:	From the Arch Linux advisory: Due to incorrect bounds checking Squid is vulnerable to a denial of service attack when processing HTTP responses. This problem allows a malicious client script and remote server delivering certain unusual HTTP response syntax to trigger a denial of service for all clients accessing the Squid service.
Alerts:	SUSE SUSE-SU-2016:2089-1 squid3 2016-08-16 openSUSE openSUSE-SU-2016:2081-1 squid 2016-08-16 SUSE SUSE-SU-2016:1996-1 squid3 2016-08-09 Fedora FEDORA-2016-b3b9407940 squid 2016-07-13 Fedora FEDORA-2016-95edf19d8a squid 2016-07-12 Gentoo 201607-01 squid 2016-07-09 Ubuntu USN-2995-1 squid3 2016-06-09 Mageia MGASA-2016-0133 squid 2016-04-06 Arch Linux ASA-201604-4 squid 2016-04-02

Comments (none posted)

squid: denial of service

Package(s):	squid	CVE #(s):	CVE-2016-3948
Created:	April 6, 2016	Updated:	November 11, 2016
Description:	From the Mageia advisory: Due to incorrect bounds checking, Squid before 3.5.16 is vulnerable to a denial of service attack when processing HTTP responses.
Alerts:	Oracle ELSA-2016-2600 squid 2016-11-10 Red Hat RHSA-2016:2600-02 squid 2016-11-03 SUSE SUSE-SU-2016:2089-1 squid3 2016-08-16 openSUSE openSUSE-SU-2016:2081-1 squid 2016-08-16 SUSE SUSE-SU-2016:1996-1 squid3 2016-08-09 Fedora FEDORA-2016-b3b9407940 squid 2016-07-13 Fedora FEDORA-2016-95edf19d8a squid 2016-07-12 Gentoo 201607-01 squid 2016-07-09 Scientific Linux SLSA-2016:2600-2 squid 2016-12-14 Mageia MGASA-2016-0133 squid 2016-04-06

Comments (none posted)

thunderbird: unspecified vulnerabilities

Package(s):	thunderbird/thunderbird-l10n	CVE #(s):
Created:	April 1, 2016	Updated:	April 14, 2016
Description:	According to the Mageia bug report: Mozilla released Thunderbird 38.7.1 on March 25: https://www.mozilla.org/en-US/thunderbird/38.7.1/releasenotes/ The only change is that they disabled the Graphite2 font shaping library, presumably because of multiple security issues, deciding that disabling the library was more efficient than attempting to fix multiple, potentially unknown vulnerabilities.
Alerts:	Mageia MGASA-2016-0139 thunderbird 2016-04-13 Mageia MGASA-2016-0129 thunderbird/thunderbird-l10n 2016-03-31

Comments (none posted)

vtun: denial of service

Package(s):	vtun	CVE #(s):
Created:	April 5, 2016	Updated:	April 25, 2016
Description:	From the Red Hat bugzilla: A vulnerability was found in the vtun package. When you send a SIGHUP to a vtun client process and it cannot connect to the remote server, vtun tries to reconnect without sleep between each attempt. In result, the vtun process uses a lot of CPU, and writes to syslog without limit.
Alerts:	Mageia MGASA-2016-0146 vtun 2016-04-25 Fedora FEDORA-2016-06e4c327b7 vtun 2016-04-05 Fedora FEDORA-2016-0fb6577f07 vtun 2016-04-05

Comments (none posted)

xchat-gnome: man-in-the-middle attack

Package(s):	xchat-gnome	CVE #(s):	CVE-2013-7449
Created:	April 5, 2016	Updated:	April 7, 2016
Description:	From the Ubuntu advisory: It was discovered that XChat-GNOME incorrectly verified the hostname in an SSL certificate. An attacker could trick XChat-GNOME into trusting a rogue server's certificate, which was signed by a trusted certificate authority, to perform a man-in-the-middle attack.
Alerts:	Ubuntu USN-2945-1 xchat-gnome 2016-04-04

Comments (none posted)

xen: multiple vulnerabilities

Package(s):	xen	CVE #(s):	CVE-2012-6030 CVE-2012-6031 CVE-2012-6032 CVE-2012-6033 CVE-2012-6034 CVE-2012-6035 CVE-2012-6036
Created:	April 5, 2016	Updated:	April 6, 2016
Description:	From the CVE entries: The do_tmem_op function in the Transcendent Memory (TMEM) in Xen 4.0, 4.1, and 4.2 allow local guest OS users to cause a denial of service (host crash) and possibly have other unspecified impacts via unspecified vectors related to "broken locking checks" in an "error path." NOTE: this issue was originally published as part of CVE-2012-3497, which was too general; CVE-2012-3497 has been SPLIT into this ID and others. (CVE-2012-6030) The do_tmem_get function in the Transcendent Memory (TMEM) in Xen 4.0, 4.1, and 4.2 allow local guest OS users to cause a denial of service (CPU hang and host crash) via unspecified vectors related to a spinlock being held in the "bad_copy error path." NOTE: this issue was originally published as part of CVE-2012-3497, which was too general; CVE-2012-3497 has been SPLIT into this ID and others. (CVE-2012-6031) Multiple integer overflows in the (1) tmh_copy_from_client and (2) tmh_copy_to_client functions in the Transcendent Memory (TMEM) in Xen 4.0, 4.1, and 4.2 allow local guest OS users to cause a denial of service (memory corruption and host crash) via unspecified vectors. NOTE: this issue was originally published as part of CVE-2012-3497, which was too general; CVE-2012-3497 has been SPLIT into this ID and others. (CVE-2012-6032) The do_tmem_control function in the Transcendent Memory (TMEM) in Xen 4.0, 4.1, and 4.2 does not properly check privileges, which allows local guest OS users to access control stack operations via unspecified vectors. NOTE: this issue was originally published as part of CVE-2012-3497, which was too general; CVE-2012-3497 has been SPLIT into this ID and others. (CVE-2012-6033) The (1) tmemc_save_get_next_page and (2) tmemc_save_get_next_inv functions and the (3) TMEMC_SAVE_GET_POOL_UUID sub-operation in the Transcendent Memory (TMEM) in Xen 4.0, 4.1, and 4.2 "do not check incoming guest output buffer pointers," which allows local guest OS users to cause a denial of service (memory corruption and host crash) or execute arbitrary code via unspecified vectors. NOTE: this issue was originally published as part of CVE-2012-3497, which was too general; CVE-2012-3497 has been SPLIT into this ID and others. (CVE-2012-6034) The do_tmem_destroy_pool function in the Transcendent Memory (TMEM) in Xen 4.0, 4.1, and 4.2 does not properly validate pool ids, which allows local guest OS users to cause a denial of service (memory corruption and host crash) or execute arbitrary code via unspecified vectors. NOTE: this issue was originally published as part of CVE-2012-3497, which was too general; CVE-2012-3497 has been SPLIT into this ID and others. (CVE-2012-6035) The (1) memc_save_get_next_page, (2) tmemc_restore_put_page and (3) tmemc_restore_flush_page functions in the Transcendent Memory (TMEM) in Xen 4.0, 4.1, and 4.2 do not check for negative id pools, which allows local guest OS users to cause a denial of service (memory corruption and host crash) or possibly execute arbitrary code via unspecified vectors. NOTE: this issue was originally published as part of CVE-2012-3497, which was too general; CVE-2012-3497 has been SPLIT into this ID and others. (CVE-2012-6036)
Alerts:	Gentoo 201604-03 xen 2016-04-05

Comments (none posted)

Kernel release status

The current development kernel is 4.6-rc2, released on April 3. Linus said: "You all know the drill by now - another week, another rc. I'd say that things look fairly normal at this point: it's not a big rc2, but that's been true lately (rc3 tends to be a bit bigger - probably just because it takes time for people to start noticing issues)."

Stable updates: none have been released since March 16.

Comments (none posted)

Quote of the week

— Linus Torvalds

Comments (none posted)

QOTW2: Busted

— "Bastien Philbert"

    Received: from [192.168.0.11]
        (CPEbc4dfb2691f3-CMbc4dfb2691f0.cpe.net.cable.rogers.com. [99.231.110.121])
        by smtp.gmail.com with ESMTPSA id w69sm1687054qhw.3.2016.04.06.10.23.24
    	(version=TLSv1/SSLv3 cipher=OTHER); Wed, 06 Apr 2016 10:23:25 -0700 (PDT)

— from the message headers

    Received: from [192.168.0.11]
 	(CPEbc4dfb2691f3-CMbc4dfb2691f0.cpe.net.cable.rogers.com. [99.231.110.121])
	 by smtp.googlemail.com with ESMTPSA id o201sm11982708ioe.15.2016.02.22.12.12.53
	 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 22 Feb 2016 12:12:53 -0800 (PST)

— from an earlier message from Nick

Comments (14 posted)

Analyzing the patchiness of vendor kernels

A great many devices in the embedded Linux world run vendor-supplied kernels. These kernels are not necessarily dangerous simply because they contain code not found in the mainline kernel releases, but they are still a cause for concern. They introduce a maintenance burden at the very least, as users (either end users or downstream commercial partners) must work to apply out-of-tree patches when they migrate to newer kernel releases. At worst, the vendor's out-of-tree patches can include code that introduces security worries. At the 2016 Embedded Linux Conference in San Diego, Hisao Munakata presented a recent effort he led to systematically measure and assess the kernels that ship in vendor board support packages (BSPs).

The full title of Munakata's talk was "Digitization of kernel diversion from the upstream to minimize local code modifications." As the latter half of that title suggests, Munakata believes that measuring the number and riskiness of out-of-tree patches is only the first step; SoC vendors and board manufacturers can and should seek to reduce their kernels' divergence from mainline over time. He works for the embedded Linux device vendor Renesas and tested the measurement approach against the company's own R-Car boards, which are designed for use in automotive projects. Most of the problems that embedded developers encounter come from working with BSPs, he said, so assessing a BSP's "health" is a task all vendors would do well to consider.

Unlike those working on desktop and server Linux projects, Munakata said, embedded developers do not have a turnkey Linux distribution that they can rely on to provide hardware support for the boards or system-on-chips (SoCs) they use. Instead, they almost always rely on the board or SoC vendor's BSP and its included kernel, which they combine in-house with other components.

Vendors pick "random kernels" to ship, he said, while the board is in development. The Long-Term Support Initiative (LTSI) has helped standardize such selections to a degree, but not entirely. Because BSP kernels are generally developed by the vendor while the product is also in development, the hardware is often revised several times prior to release. Thus, the drivers and other patches are often written and tested rapidly and may include patches that are little more than "quick and dirty hacks." Those patches may work for the board in question but break for other use cases, so they will never get merged—assuming they are ever sent upstream.

For users, these BSP kernels cause two main problems: difficulty when migrating to a new kernel releases and difficulty applying security patches. Migration to a new release is sometimes required, even when it is arduous, to make use of a significant new feature or API. Vendor patches that touch the kernel outside of device drivers make this process more complicated. The security-patch issue is similar, except that a security fix backported to an older kernel release also adds a sense of urgency.

Considering these issues, Munakata said, he decided it would be useful to have a "sanity assessment" check for BSP kernels. Ideally, he said, a vendor kernel would include a human-readable "certification of contents" file (in its board's bill of materials or BOM) that captured where the kernel differs from the mainline kernel. The file would describe the purpose of each patch, the size and location of each patch in the tree, and some sort of qualitative measurement of each patch's riskiness. He proposed a simple, three-class system for categorizing riskiness: "clean" for patches that merely enable support for new hardware features or backport a relatively self-contained feature, "safe" for patches that implement minor fixes, and "dirty" for patches that either rewrite or outright break functionality from the mainline kernel.

Considering the size of contemporary kernels, he said, such an assessment has to be generated programmatically. Initially, he tried counting the matches and mismatches of SHA-256 file hashes, using the Yaminabe tool. That method provided insufficient detail, however, because trivial and major changes both trigger a hash mismatch. He also tried tracing patches by Git commit IDs, which he said was better at determining how many in-house patch sets have been applied to a kernel, but still fell far short of the goal, since it does not provide any way to measure the riskiness of a particular patch. Naturally, that approach is also limited to those vendors that manage their kernel patches in Git.

The solution he finally settled on was combining Yaminabe's simple hash-mismatch hits with a second scan using a locality-sensitive hash, a hash function that gives similar hashes for similar files and progressively diverging hashes the more that the files compared differ. The Trend Micro Locality Sensitive Hash (TLSH) is such a hash available in open-source form. The result, built in collaboration with Armijn Hemel, is Yaminabe2. It can compare two Git trees, first weeding out identical files by comparing SHA-256 hashes. For the mismatches, it computes the TLSH hashes and reports a "distance" score for each pair.

Transforming those distance scores into the "clean," "safe," and "dirty" assessments desired took a bit of investigative work. Running Yaminabe2 against various revisions of the R-Car BSP, the distance scores reported ranged from less than 10 (for trivial differences) up to nearly 400. After several rounds of testing, he settled on some cut-off points. A distance score less than 60 can be marked as "clean;" a distance between 60 and 150 can be marked as "safe," and any distance above 150 likely indicates that a patched file is "dirty." Naturally, such cut-off points can catch some false positives, and any such assessment also needs to take a file's location in the kernel source tree into account (e.g., whether it is a device driver or not). But they serve as a good starting point for further exploration.

Munakata showed sample output from his R-Car BSP tests. For the sake of speed, Yaminabe2 uses a pre-computed database of SHA-256 and TLSH hash scores for the comparison kernel (that is, the mainline) and only processes the "target" Git tree for each run. The terminal version of the comparison tool reports the total number of mismatched files, then lists each file and its distance score, followed by the total accumulated distance for the entire tree. For the R-Car second-generation board, based on the 3.10 LTSI kernel, the first BSP release tested (version 1.1.0) had a total distance of 63,616, while the last (version 1.9.6) had a total distance of 72,242. That increase reflects the growing number of bug fixes applied. Over time, he said, he hopes to track such total distance scores as aide to decreasing the R-Car BSP's divergence from mainline.

The third-generation board, which was released in December 2015, is currently working to keep pace with the current kernel, he said, because the company wants to settle on the 2017 LTSI kernel for the board's long-term support. Interestingly enough, he added, the total distance reported by Yaminabe2 is presently higher for the R-Car third generation, which he finds perplexing. But there are still many opportunities to tune Yaminabe2: it could be taught to ignore files for irrelevant architectures or features disabled by the BSP kernel configuration, for example. Perhaps one of those factors accounts for the unexpected results for the new R-Car BSP.

The biggest improvement, he said, would be for other vendors to test Yaminabe2 against their own BSPs, and for companies buying boards or SoCs to run tests against the BSPs provided by their vendors. This would be particularly useful as a way to further refine the risk-scoring system, which at present is quite simple. Munakata added that he also hopes to improve Yaminabe2's reporting features, and to simplify the setup process. At the moment, each Yaminabe2 installation requires a fair amount of customization to tune the TLSH parameters to fit the hardware available (since TLSH is quite CPU-intensive; a full scan can take 12 or more hours).

The code is currently available only as a tar archive on the Yaminabe2 eLinux wiki page, while TLSH must be retrieved from that project's GitHub page. Yaminabe2 is a Python application, and the bundle includes everything users will need to scan and compare any two arbitrary Git trees. But Munakata advised interested parties to start with the (multi-gigabyte) pre-built SHA-256 and TLSH databases also linked to from the wiki page, since scanning and hashing the mainline kernel is a multi-hour operation even on a fast machine.

There are many ways one could gauge the riskiness of vendor-supplied kernel. Munakata and Hemel's work on Yaminabe2 offers just one approach, and it certainly needs more widespread testing. But it may prove to be a good start toward solving an under-addressed problem: helping embedded users get a handle on precisely where and how their BSP kernel diverges from the mainstream.

[The author would like to thank the Linux Foundation for travel assistance to attend ELC 2016.]

Comments (1 posted)

Improvements in CPU frequency management

A few years ago, Linux scheduler maintainer Ingo Molnar expressed a strong desire that future improvements in CPU power management should integrate well with the scheduler and not try to work independently. Since then, there have been improvements to the way the scheduler estimates load — per-entity load tracking in particular — and some patch sets circulating that aim to link these improved estimates into the CPU frequency scaling, but the recently released 4.5 kernel still does CPU power management as it has done it for years. This situation appears to be about to change with some preliminary work already merged into 4.6-rc1 and some more significant improvements being prepared by power management maintainer Rafael Wysocki for likely inclusion in 4.7: the schedutil cpufreq governor.

The focus of these patch sets is just to begin the process of integration. They don't present a complete solution, but only a couple of steps in the right direction, by creating interfaces for linking the scheduler more closely with CPU power management.

CPU power management in Linux 4.5

Linux support for power management of CPUs roughly divides into cpuidle, which guides power usage when the CPU is idle, and cpufreq, which governs power usage when it is active. cpufreq drivers divide into those, like intel_pstate and the Transmeta longrun, that choose the CPU frequency themselves using hardware-specific performance counters or firmware functionality, and those that need to be told which power level, from a given range of options, is appropriate at any given time. This power level is nominally a CPU frequency but, as it can be implemented by scaling a high-frequency clock, it is sometimes referred to as a frequency scaling factor. It is not uncommon to scale down the voltage together with the frequency so you can also see terms like DVFS for "Dynamic Voltage and Frequency Scaling", or the more generic OPP for "Operating Power Point". For the most part, we will just refer to frequency changes here.

The cpufreq governors that choose a frequency for the second group of drivers are also divided into two groups: the trivial and the non-trivial. There are three trivial frequency governors: performance, which always chooses the highest frequency, powersave, which always chooses the lowest, and userspace, which allows a suitably privileged user-space process to make the decision by writing to a file in sysfs. The non-trivial governors vary the CPU frequency based on the apparent system load. This is quite similar to the approach used by intel_pstate, except that they determine load from generic information known to the kernel, rather than hardware-specific counters.

The statistics used by these governors do come from the scheduler, but only in a roundabout way that does not reflect the sort of integration that Molnar hopes for. The governors set a timer with a delay of some tens or hundreds of milliseconds and use the CPU-usage statistics (visible in /proc/stat) to determine the average load over that time, which is the ratio of busy time for the CPU to total time. There is some subtlety to exactly how this number is used but, roughly, the CPU frequency is increased when the load is above a threshold and decreased if it is below another threshold. The two non-trivial frequency governors differ in their response to increasing load: on-demand will immediately jump to the highest frequency and then possibly back off as the idle time increases, making it suitable for interactive tasks, while the conservative governor will scale up more gradually as is fitting for background jobs.

Problems with the current approach

There are various reasons for dissatisfaction with the current governors. Probably the easiest to identify is the one that caused the Android developers to write their own governor: interactive responsiveness. An idle CPU will likely be at the lowest frequency setting. When a user starts interacting with the device the CPU will continue at that setting for many milliseconds until the timer fires and the new load is assessed. While the on-demand governor does go straight to the maximum frequency, it doesn't do so immediately, and the delay is both noticeable and undesirable.

The interactive governor for Android is designed to detect this transition out of idle and to set a much shorter timeout, typically under ten milliseconds ("1-2 ticks"). A new frequency is then chosen based on the load over this short time range rather than the normal longer interval. The result is improved interactive response without the loss of stability that is likely if all samples were over short time periods.

Another reason for dissatisfaction is that resetting those timers frequently on every CPU is far from elegant. Thomas Gleixner, maintainer of the timer code, is known not to like them and Wysocki noted that "getting rid of those timers allows quite some irritating bugs in cpufreq to be fixed".

Finally, the information used to guide frequency choice is based entirely on recent history. While it is hard to get reliable information about the future, there is information about the present that can be useful. To understand this it is helpful to consider the different classes of threads as they are seen by the scheduler, which can be divided into realtime, deadline, or normal.

Of these, threads configured for deadline scheduling provide the most information about the future. Such threads must specify a worst-case CPU time needed to achieve their goal and a period indicating how often it will be needed. From this, it is possible to calculate the minimum CPU frequency that will allow all deadline threads to be serviced. This can be done without any reference to history.

Realtime threads also provide information about the future, though it is not so detailed. When a realtime thread is ready to run, "the only possible choice the kernel has", as Peter Zijlstra put it, "is max OPP". When there is a realtime thread, but it is currently waiting for something, there are two options. If switching to top speed can be done quickly, then it is safe to ignore the thread until it is runnable, and then instantly switch to maximum frequency as it starts running. If switching the CPU frequency takes a bit longer, then the only really safe thing to do is to stay at maximum CPU frequency whenever there are any realtime threads on that CPU.

Lastly, there are the normal threads, those managed by the Completely Fair Scheduler (CFS). While the primary source of information available to CFS is historical, it is more fine-grained than the information currently in use. Using per-entity load tracking, CFS knows the recent load characteristics of every thread on each CPU and knows immediately if a thread has exited, a new one has been created, or if a thread has migrated between CPUs. This allows a more precise determination of load, which is already being used for scheduling decisions and could usefully be used for CPU frequency scaling decisions.

CFS can have a little bit more information than historical usage. It is possible for a maximum CPU bandwidth to be imposed on a process or process group. CFS could use this to determine an upper limit for the load generated by those processes and may usefully be able to provide that information to cpufreq.

Challenges

Even if all this information were readily available, and some of it is, making use of it effectively is not necessarily straightforward. This particularly seems to be the case when working in and around the scheduler, as that code is quite performance-sensitive.

One challenge that cpufreq must face is how exactly to change the CPU frequency once a decision has been made. As was hinted at above, some platforms may allow fast frequency changes but, while that is true, cpufreq doesn't know anything about it. A cpufreq driver has a single interface, target_index(), to set the frequency, which may take locks or block for other reasons, so it cannot be depended on to be fast. An optional non-blocking interface has been proposed, but it will still be necessary to work with cpufreq drivers that need to be called from "process context". This currently means scheduling a worker thread to effect the frequency change.

There is a question whether a regular workqueue thread is really sufficient. It seems possible that on a busy system, such a thread might not get scheduled for a while, so at those times, when a switch to high frequency is most important, it will be most delayed. The interactive governor used by Android has addressed this issue by creating a realtime thread to perform all frequency changes. While this works, it does not seem like an ideal solution for the longer term. As noted above, it may be sensible to keep the CPU at maximum speed whenever there are realtime threads. In that case, the existence of a realtime thread for setting the frequency would prevent it from ever being asked to set a lower frequency.

This is currently an open issue. Wysocki is not convinced that this is a real problem in practice, so he is not keen on anything beyond the current simple approach. Those who feel otherwise will need to provide concrete evidence of a problem, which is often a valuable prerequisite for such changes.

Two steps forward

There are two patch sets that have been prepared by Wysocki and look likely to reach the mainline soon; one has already been included in 4.6-rc1. There are a number of details in which they differ from others that have been proposed, such as scheduler-driven frequency selection originally by Michael Turquette, but possibly the most important is that they are incremental steps with modest goals. They address the issues that can easily be addressed and leave other more difficult issues for further research.

The first of these patch sets removed the timers that Gleixner found so distasteful. Instead of being triggered by a timeout, re-evaluation of the optimal setting for cpufreq is now triggered by the scheduler whenever it updates load or runtime statistics. This will doubtless be called more often than frequency changes are really wanted, and possibly more often than it is possible to update the frequency choice, so the sampling_rate number that previously set the timer is now used to discard samples until an appropriate interval since the last update has passed.

The new frequency choice is still calculated the same way and it happens about as often, but one noteworthy change is that the updates are no longer synchronized with the scheduler "tick" that timers use. This change has resulted in one measurable benefit for the intel_pstate driver, which occasionally made poor decisions due to this synchronization.

The second patch set, some of which is up to its eighth revision, makes two particular changes. A new, optional fast_switch() interface is added to cpufreq drivers so that fast frequency switching can be used on platforms that support it. If provided, this must be able to run from "interrupt context" meaning that it must be fast and may not sleep. As already discussed, there are times when this can be quite valuable.

The other important change is to introduce a new CPU frequency-scaling governor, schedutil, that makes decisions based on utilization as measured by the scheduler. The cpufreq_update_util() call that the scheduler makes whenever it updates the load average already carries information about the calculated load on the current CPU, but no governor uses that information. schedutil changes that. It doesn't change much though.

schedutil still only performs updates at the same rate as the current code, so it doesn't try to address the interactive responsiveness problem, and doesn't try to be clever about realtime or deadline threads. All it does is use the load calculated by the scheduler instead of the average load over the last little while, and optionally imposes that frequency change instantly (directly from the scheduler callback) if the driver supports it.

This is far from a complete solution for power-aware scheduling, but looks like an excellent base on which to make cpufreq more responsive to sudden changes in load, and more aware of some of the finer details that the scheduler can, in theory, provide.

It appears that the long-term goal is to get rid of the selectable governors completely and just have a single governor that handles all cases correctly. It would need to respond correctly to realtime tasks, deadline tasks, interactive tasks, and background tasks. These are all concepts that the scheduler must already deal with, so it is quite reasonable to expect that cpufreq can learn to deal with them too. It will clearly take a while longer to reach the situation that Molnar desires, but it seems we are well on the way.

Comments (1 posted)

Early packet drop — and more — with BPF

The Berkeley packet filter (BPF) mechanism has been working its way into various kernel subsystems since it was rewritten and extended in 2014. There is, it turns out, great value in an in-kernel virtual machine that allows for the implementation of arbitrary policies without writing kernel code. A recent patch set pushing BPF into networking drivers shows some of the potential of this mechanism — and the difficulty of designing its integration in a way that will stand the test of time. If it is successful, it may change the way high-performance networking is done on Linux systems.

Early drop

This patch set from Brenden Blanco is, in one sense, a return to the original purpose of BPF: selecting packets for either acceptance or rejection. In this case, though, that selection is done at the earliest possible moment: in the network-adapter device driver, as soon as the packet is received. The intent is to make the handling of packets destined to be dropped as inexpensive as possible, preferably before doing any protocol-processing work, such as setting up a sk_buff structure (SKB), for those packets.

BPF programs, as loaded by the bpf() system call, have a type associated with them; that type is checked before a program can be loaded for a specific task. Brenden's patch set starts by defining a new type, BPF_PROG_TYPE_PHYS_DEV, for programs that will do early packet processing. Each program type includes a "context" for information that is made available when the program runs; in this case, the context needs to include information about the packet under consideration. Internally, that context is represented by struct xdp_metadata; it contains only the length of the packet in this version of the patch set.

The next step is to add a new net_device_ops method that drivers can supply:

	int (*ndo_bpf_set)(struct net_device *dev, int fd);

A call to ndo_bpf_set() tells the driver to install the BPF program indicated by the provided file descriptor fd; the program should replace the existing program, if any. A negative fd value means that any existing program should be removed. There is a new netlink operation allowing user space to set a program on a given network device.

The driver can use bpf_prog_get() to get a pointer to the actual BPF program from the file descriptor. When a packet comes in, the BPF_PROG_RUN() macro can be used to run the program on the packet; a non-zero return code from the program indicates that the packet should be dropped.

Just a starting point

The interface for the running of the BPF program is where the disagreement starts. The driver must clearly give information about the new packet to the program being run; that is done by passing an SKB pointer to BPF_PROG_RUN(). The internal machinery hides the creation of the xdp_metadata information from the passed-in SKB. The mechanism seems straightforward enough, and it takes advantage of the existing BPF functionality for working with SKBs, but there are a couple of objections. The first of those is that the whole point of the early-drop mechanism is to avoid the overhead of packet processing on packets that will be dropped anyway; the initial, and not insignificant, part of that overhead is the creation of the SKB structure. Creating it anyway would appear to be defeating the purpose.

In truth, the one driver (mlx4) that has been modified to implement this mechanism doesn't create a full SKB; instead, it puts the minimal amount of information into a fake, statically allocated SKB. That avoids the overhead, but at the cost of creating an SKB that isn't really an SKB. The amount of information that needs to go into this fake SKB will surely grow over time — there is surprisingly little call for the ability to drop packets using their length as the sole criterion. Whenever new information is needed, every driver will have to be tweaked to provide it, and, over time, the result will look increasingly like a real SKB with the associated overhead.

The other potential problem is that there is a fair amount of interest in eventually pushing the BPF programs (possibly after a translation pass) into the network adapter itself. That would allow packets to be dropped before they come to the kernel's attention at all, optimizing the process further. But the hardware is not going to have any knowledge of the kernel's SKB structure; all it can see is what is in the packet itself. If BPF programs are written to expect an SKB to be present, they will not work when pushed into the hardware.

There is a larger issue, though: quickly dropping packets is a nice capability, but high-performance networking users want to do more than that. They would like to be able to load BPF programs to do fast routing, rewrite packet contents at ingress time, perform decapsulation, coalesce large packets, and so on. Indeed, there is a whole vision for the "express data path" (or "XDP") built around low-level BPF packet processing; see these slides [PDF] for an overview of what the developers have in mind. In short, they want to provide the sort of optimized processing performance that attracts users to user-space networking stacks while retaining the in-kernel stack and all its functionality.

If the mechanism is to be extended beyond drop/accept decisions, the information and functionality available to BPF programs will clearly have to increase, preferably without breaking any existing users. As Alexei Starovoitov put it: "We have to plan the whole project, so we can incrementally add features without breaking abi". The current patch set does not reflect much planning of this type; it is, instead, a request-for-comments posting introducing the mechanism that the XDP developers want to build on.

So, clearly, this code will not be going into the mainline in its current form. But it has had the desired effect of getting the conversation started; there is, it would seem, a lot of interest in adding this feature. If the XDP approach is able to achieve its performance and functionality goals, it should give user-space stacks a run for their money. But there is some significant work to be done to get to that point.

Comments (2 posted)

Deciding between Buildroot & Yocto

Often, two or more free-software projects that have similar goals prefer to ignore the fact that they compete for users and adoption. The developers and advocates choose to let the rivalry remain the unaddressed elephant in the room whenever they cross paths in public. But that was not the case in at least one session at the 2016 Embedded Linux Conference in San Diego, when a developer from Yocto and one from Buildroot shared a podium for a discussion of where the two projects overlapped and where they differed. Both are used to build an embedded Linux system from scratch (or, at least, from the source code up) and both are popular, but that is about all that they have in common.

Before diving into the details, the two speakers introduced themselves. Alexandre Belloni is a Yocto developer and trainer, maintaining the Yocto layers for several boards and contributing elsewhere (including to the kernel, where he is co-maintainer of the real-time clock subsystem). Thomas Petazzoni is an embedded Linux engineer who works on the Buildroot project. Both are employed by Free Electrons, which consults on Yocto and Buildroot projects for various customers.

The great divide

At the basic level, Petazzoni began, Yocto and Buildroot can both give you the same end product: a root filesystem image for your embedded device, a kernel, a bootloader, and a compatible toolchain. But, while many people have subjective opinions on which projects has the better approach, there are more than enough objective differences in how the two projects function to discuss.

Buildroot, he said, focuses on simplicity. The core Buildroot tool is kept as small and as simple as possible, which makes it easy to use and understand. All special cases are handled in extensions, and Buildroot re-uses existing tools like kconfig wherever possible. Thus, he said, learning Buildroot means learning tools that are applicable in other scenarios. The OS image Buildroot creates is also minimal by default, making builds fast, and delivering a "purpose agnostic" system, not one tailored to any particular use case.

In contrast, Belloni continued, Yocto tries to be versatile and support a wide range of embedded systems. It does that by defining builds in recipes, which specify what software to build and how to build it, and by supporting layers, which are recipe collections written and maintained by the development community. Layers allow the community to support new boards or architectures, define new application stacks, or support entirely new use cases. By relying on cleanly defined layers, the default system image defined can remain small.

The projects provide a tangible difference in what they generate, Petazzoni said. Buildroot's output is a root filesystem image, nothing more; that has even led some to call it a "firmware generator." Whenever you need to update the system, you must regenerate the entire image, or somehow build your own update mechanism (such as a binary diff system). This restriction is by choice, he said; the project believes embedded systems should not be updated like desktops and servers are, since the outcome of a failed or partial update can be disastrous.

Yocto's output is "a distribution," Belloni said. Specifically, it is a package feed, although providing a full-scale package-management system for the target device is optional and generating full disk images is certainly supported. But individual packages can be rebuilt as needed, individual packages can be updated or removed, and updating one package (say, to apply an OpenSSL security fix) is faster than a full rebuild. Yocto can also build a software development kit (SDK) to let application developers build and test their software for the device without rebuilding the OS image.

Builds made with Buildroot and Yocto are configured in differing manners as well, which Petazzoni called one of the clearest distinctions between the two. Buildroot puts all configuration information in one file, which can be edited using any of the interfaces from the kernel's kconfig tool (e.g., xconfig or menuconfig). That file specifies the architecture, kernel, bootloader, and user-space packages to include. He likes to describe the build process as "make menuconfig; make; profit." The drawback is that similar machines cannot share any configuration information: one must build every target separately.

Yocto, Belloni said, keeps configuration information in multiple parts. The distribution configuration dictates general packages, toolchains, and libraries; the machine configuration describes the architecture and hardware features; the image recipe lists which packages should be pre-installed; the local configuration controls how the build is performed. This approach means one may have to gather a lot of layers together before beginning a build, but it also makes it easy to build variations with little additional work. Building for variants of the same board can require only a small tweak.

Yocto, as described earlier, defines a build with a collection of layers. Buildroot has historically eschewed them, Petazzoni said, largely because the out-of-tree configurations the project sees have tended to be of mediocre quality only suitable for single use cases—and, thus, not suitable for sharing. But, recently, a number of companies have started pushing for a Buildroot feature to let them keep certain components outside of their main configuration, and the project has added a feature to support that behavior. It is called the BR2_EXTERNAL option, and it allows a build to incorporate "secret sauce" for package definitions, configurations, or other artifacts that will not be shared. BR2_EXTERNAL is limited, he said: a configuration can only use the feature one time, and it can only be used to add (not override) settings.

Yocto layers, Belloni said, can add or modify packages or even recipes. Layers enable a strict separation between the core of the build, the board support package (BSP), and custom modifications; that allows a configuration to scale properly. A third party can modify an existing layer by adding a layer of its own that enables support for new hardware, without rewriting the original layer. That said, he noted, there are many layers published in the OpenEmbedded Metadata Index (much of Yocto being based on work that originated in OpenEmbedded), but it is up to the user to look at them and notice if they are unmaintained or contradict another layer.

The complexity of the two systems is quite different. Buildroot strives to remain simple; Petazzoni said every change proposed for the core is analyzed in terms of its "usefulness to complexity ratio." The core logic is implemented entirely in Makefiles totaling under 1000 lines of code. That amount of code can be read and understood by an individual, he said. Yocto, Belloni conceded, has a bit more challenging of a learning curve. The main utility, BitBake, is a 60,000-line Python program, and recipes are written in a combination of Python, shell scripts, and a BitBake-specific language. Still, BitBake does provide logging and debugging aides, which are useful. The project's complexity is a known issue, he said: "best practices" are not always easy to find or understand, and many people seem to find the terminology (such as the distinctions between Yocto, OpenEmbedded, Poky, and BitBake) hard to grasp.

Lesser distinctions

The differences discussed so far encompass the major, large-scale distinctions between Yocto and Buildroot, but there are still quite a few important details that differ as well, which Petazzoni and Belloni turned to next.

First, Buildroot currently supports a smaller set of available packages—a little over 1800. The list includes all of the core options one would expect, from graphics layers to application toolkits to multimedia frameworks and programming languages. But it is a curated list of packages and, notably, it does not include any development tools (e.g., compilers and linkers) to run on the target system. Buildroot targets are meant to be the end product, not the development system. Yocto, however, does provide target toolchains—and much more. About 8400 packages are available in total, including some of questionable value for current systems, like Qt 3. But there are packages for more programming languages, including some like Go and Rust that are not available for Buildroot, and there are layers available for many containerization and virtualization frameworks (use cases that are outside of Buildroot's scope).

Dependency handling differs as well. Buildroot again takes a minimalist approach: whenever a new package is added, it is built with as many features disabled as possible. That allows the creation of small footprints: the minimal image is just 2MB. But Buildroot does include a number of automatic dependencies, Petazzoni said. For instance, adding OpenSSL to the configuration will automatically add SSL support for other packages that include an SSL option. In Yocto, package configuration is done at the distribution layer: enabling SSL support enables it for every package by default, although it can still be disabled for individual packages (or enabled for individual packages, if it is set to be off by default). These dependencies can be overridden at the machine-configuration level, but that is considered bad practice.

The two projects differ somewhat in their update schedules and security practices. Buildroot releases on a three-month cycle; each release includes package updates, new releases, and security updates. There are no "long-term support" releases with guarantees of security updates; users must handle that on their own. Yocto releases every six months, with four milestones in between each stable release. The current release and the previous release both have officially appointed maintainers and receive security and major updates. In addition, various community members make updates for older releases.

One final distinction to be aware of is that Buildroot provides no mechanism to detect configuration changes between builds. All rebuilds are full-image rebuilds, Petazzoni noted, so, by design, all rebuilds must be manually initiated. In contrast, BitBake maintains a shared cache between builds, so Yocto can skip rebuilding individual packages if they have not changed. Consequently, Belloni said, building slight variants for several different machines can be quite fast.

There are number of factors where Buildroot and Yocto offer more-or-less comparable features. The toolchains they support are essentially the same, utilizing GCC and the user's choice of C libraries (glibc, uClibc, and musl). They support the same processor architectures, they both provide license-compliance tools, and they produce minimal builds of fairly similar specifications. Buildroot's default build is 2.2MB and can be completed in 15 minutes, the speakers said, while Yocto's default is 4.9MB and can be completed in 50 minutes from scratch or in just over one minute by using the package-build cache.

Ultimately, they said, the decision of which system to use is not a simple (much less single-issue) choice. There are a number of factors one should consider when choosing. The interesting criteria, as they saw matters, include whether or not the architecture needed is something strange or needs a very small root filesystem (factors that Buildroot is better suited for), whether or not you must support several machine configurations or need to provide package-management and updates (scenarios that Yocto is better for). The size of the development project also influences the choice: Buildroot is good for small teams with limited resources, while Yocto is good for large systems composed of many components. In the end, one size does not fit all where embedded Linux build systems are concerned.

[The author would like to thank the Linux Foundation for travel assistance to attend ELC 2016.]

Comments (8 posted)

Distribution quotes of the week

-- Debian Publicity Team (April 1st announcement) (Thanks to Paul Wise)

-- Matthew Garrett

Comments (none posted)

Debian 7.10 and 8.4

The Debian project has released Debian 7.10 (wheezy) and Debian 8.4 (jessie). These updates contain security updates, along with a few adjustments for serious problems.

Comments (none posted)

FreeBSD 10.3

FreeBSD 10.3 has been released. This version features an improved UEFI boot loader, support for the CAM Target Layer ctl(4) in High Availability setups, improvements to the Linux compatibility layer, and more. The release notes contain more details.

Comments (none posted)

openSUSE on IBM z Systems

openSUSE Factory now builds for the IBM z Systems architecture in a special ports project. "The project openSUSE:Factory:zSystems has been setup several weeks ago as a new project in the Open Buildservice. The packages in this project are inherited from openSUSE:Factory. Most of the packages already build without issues. This project now has been made publicly accessible."

Full Story (comments: none)

Distribution newsletters

• DistroWatch Weekly, Issue 655 (April 4)
• openSUSE Tumbleweed – Review of the Week (April 1)
• Ubuntu Weekly Newsletter, Issue 460 (April 3)

Comments (none posted)

Garrett: There's more than one way to exploit the commons

Matthew Garrett's take on the Debian-XScreenSaver disagreement is worth a read. "Free software doesn't benefit from distributions antagonising their upstreams, even if said upstream is a cranky nightclub owner. Debian's users are Debian's highest priority, but those users are going to suffer if developers decide that not using free licenses improves their quality of life. Kneejerk reactions around specific instances aren't helpful, but now is probably a good time to start thinking about what value Debian bring to its upstream authors and how that can be increased."

Comments (122 posted)

Python and to_file()

Python is used in a wide variety of circumstances, by people coming from different backgrounds and with various needs. A recent thread on the python-ideas mailing list thread started with a thought about a quick and easy way to write a string to a file, much like is done in some other, specialized languages (e.g. R, MATLAB). It soon expanded in several directions, partly into a philosophical consideration of the role of the language—and how best to accommodate those coming to Python from those other languages.

Nick Eubank kicked off the discussion by noting that he is social scientist trying to help his colleagues move away from the specialized languages to Python. "One of the behaviors I've found unnecessarily difficult to explain is the 'file.open()/file.close()' idiom (or, alternatively, context managers)." In other settings, saving to a file is a one-step operation, he said, so there is a conceptual hurdle for those new to Python:

While there were objections that a single line "write string to named file" operation was an unneeded niche function that would lead to bad habits and slower code, there was also some support for the idea. Andrew Barnert said that he often uses the feature in other languages but, given Python's feature set, he would rarely need it in Python: "But for users migrating to Python from another language, or using Python occasionally while primarily using another language, I can see it being a lot more attractive." On the other hand, Chris Barker was confused by the need for it at all:

Barker went on to note that NumPy offers ways to directly read and write arrays to and from files. There is also a one-liner for reading a string back in:

    string = open(path).read()

It does suffer from the same problem Barker mentioned for Python implementations that do not use reference-counted garbage collection, however.

Nick Coghlan chimed in with the idea that Python is used for both scripting and application development, but its tutorials and such typically focus on the application side, where "relying on the GC for external resource cleanup isn't a great habit to get into". Thus that introductory material will show the "deterministic cleanup form" for writing to a file. That means that the user has to expand their mental model:

Eubank strongly agreed with Coghlan's formulation and added:

In his post, Coghlan also suggested a "radical notion" that would create some kind of save/load function that would automatically use UTF-8 and JSON. That would mean it could be used for objects more complicated than strings and would create files in a well-defined format. He noted that there would be some benefits to that approach:

Barker suggested that perhaps the language definition could expand to require immediate garbage collection when it is known that the created object is no longer used (as in his one-liner save above). Since the result is never assigned to a variable, the file object created goes immediately out of scope, thus it could be reclaimed. The CPython reference implementation already does this, but as Brett Cannon pointed out, mandating that is not something the core developers are likely to want to do:

There was also discussion of where such a convenience function should live. Eubank's original idea of adding a to_file() method to the string type was not particularly popular. Eubank had noted that the pathlib module has functionality that is similar to what he was asking for, but "can't imagine anyone looking in the Path library" if they just want to write a string to a file. Koos Zevenhoven pointed out that the needed functions (write_text() and read_text()) had only been added in Python 3.5, which was only released back in September, "so there has not really been time for their adoption". Others noted that there are issues with converting strings to Paths and vice versa, which would open up another can of worms for users who were just looking for a simple way to write their string.

Coghlan suggested that the io module might be the right place, but there are still some fundamental issues. For one thing, as Victor Stinner noted, the simple one-liner "solution" does raise a warning (ResourceWarning when the -Wd flag is used) because the state of the file on disk is unknown; it may or may not have been flushed. There is a longstanding bug regarding a portable way to ensure an atomic write to a file, which Stinner also referred to. While a simple wrapper, using a with context manager, could ensure the file gets closed, it cannot ensure the data reaches the disk, which makes it something of an unsafe operation.

So, what seemed a simple idea at the start ballooned into a much more complex question. Part of the problem is that specialized languages can make lots of assumptions about how they will be used, which allows them to "hide more of the complexities from [their] users", as Coghlan put it. But a language like Python can't do that, except in domain-specific modules such as NumPy or Pandas. There is a question of how to choose the right defaults:

While there seemed to be general agreement that the io module would be the right place to put a convenience function of this sort, it is not clear if there is enough support to do so. The alternatives are reasonably readable and understandable; even the proper with form is not that hard to grasp. There will be hurdles when moving from a specialized to a general-purpose language—the advantages of the latter should make clearing the hurdles worth it, at least for some. The conversation did provide an interesting look into the thinking process that goes on in Python circles, though.

Comments (8 posted)

Quotes of the week

— Dirk Hohndel at ELC 2016, in reply to Linus Torvalds's comment that he would start a fourth software project "when something irritates me and I think 'I can do that in two weeks.'"

— Matt Mackall

— Raj Talluri at ELC 2016, explaining his presentation slide that predicted "the future of IoT" as "25B devices" strong.

Comments (none posted)

Mono Relicensed MIT

At the Mono Project blog, Miguel de Icaza announced that the Mono runtime has been relicensed, moving from a dual-license slate (with LGPLv2 and proprietary optiona) to the MIT license. The Mono compiler and class libraries were already under the MIT license and will remain so. "Moving the Mono runtime to the MIT license removes barriers to the adoption of C# and .NET in a large number of scenarios, embedded applications, including embedding Mono as a scripting engine in game engines or other applications." De Icaza notes that Xamarin (which was recently acquired by Microsoft) had developed several proprietary Mono modules in recent years; these will also now be released under the MIT license.

Comments (168 posted)

Discourse 1.5 released

Version 1.5 of the Discourse open-source discussion-and-commenting system has been released. Significant work went into rewriting the top-level "topics" page, resulting in a five-fold speed increase. Administrators can now change and customize every object label used in the interface. "Want topics to be 'threads'? Users to be 'funkatrons'? Like to be 'brofist'? Well, Discourse is your huckleberry." Support for email comments has also been improved, and user groups can now exchange private messages. The badge system, which is used to denote user roles and to mark popular posts, received a visual refresh and new documentation; user summary pages were also refreshed.

Comments (12 posted)

Exim 4.87 Released

Version 4.87 of the Exim mail transfer agent has been released. Several formerly experimental features are now marked as fully supported, including internationalized mail addressing, SOCKS support, REDIS support, and events. There are also many new expansion variables available, and improvements to the regular-expression support in ACLs.

Full Story (comments: none)

LXC 2.0 released

Version 2.0 of the LXC containerization system has been released. Among the changes are more reliable checkpoint and restore, improved control-group handling, and many bug fixes. Also of note is that LXC 2.0 is designated a long-term support release; backported security updates and bugfixes will be provided for the next five years.

Full Story (comments: none)

Rkt 1.3.0 released

Version 1.3.0 of the rkt container system has been released. "rkt version 1.3.0 improves handling of errors within app containers, tightens security for rkt’s modular stage1 images, and provides a more compatible handling of volumes when executing Docker container images rather than rkt’s native ACI image format. This release further develops the essential support for rkt as a component of the Kubernetes cluster orchestrator."

Comments (none posted)

Development newsletters from the past week

• What's cooking in git.git (April 1)
• What's cooking in git.git (April 4)
• OCaml Weekly News (April 5)
• OpenStack Developer Digest (April 1)
• Perl Weekly (April 4)
• PostgreSQL Weekly News (April 3)
• Python Weekly (March 31)
• Ruby Weekly (March 31)
• This Week in Rust (April 4)
• Tahoe-LAFS Weekly News (April 5)
• Wikimedia Tech News (April 4)

Comments (none posted)

KDE Presents its Vision for the Future

The KDE project has released a vision statement, a single sentence that sums up what the project would like to achieve: A world in which everyone has control over their digital life and enjoys freedom and privacy. "Our vision unites KDE in common purpose. It sets out where we want to get to, but it provides no guidance on how we should get there. After finalizing our vision (the "what"), we have immediately started the process of defining KDE's Mission Statement (the "how"). As with all things KDE, you are invited to contribute. You can easily add your thoughts on our mission brainstorming wiki page." (Thanks to Paul Wise)

Comments (8 posted)

New Linux-based effort to support global civil infrastructure demands

The Linux Foundation has announced the Civil Infrastructure Platform, "an open source framework that will provide the software foundation needed to deliver essential services for civil infrastructure and economic development on a global scale." Civil infrastructure systems deliver critical services such as electric power, oil and gas, water, health care, communications, transportation and more. "The Civil Infrastructure Platform will aim to work upstream with the Linux kernel and other open source projects to establish a “base layer” of industrial-grade software. This base layer will enable the use of software building blocks that meet safety, security, reliability and other requirements that are critical to industrial and civil infrastructure projects."

Full Story (comments: 36)

FSFE Newsletter - April 2016

This edition of the Free Software Foundation Europe newsletter covers: EU Radio Directive threatens software freedom, US government demands publicly financed software to be published as Free Software, and more.

Full Story (comments: none)

Free Software Supporter Issue 96, April 2016

The Free Software Foundation's newsletter for April covers: Free Software Awards winners, protesting DRM, LibrePlanet recap, Licensing and Compliance Lab interviews Matt Lee, and more.

Full Story (comments: none)

1st Call For Papers - Tcl/Tk Conference

The 23rd Annual Tcl/Tk Conference (Tcl'2016) will be held November 14-18 in Houston, Texas. The call for papers deadline is September 12.

Full Story (comments: none)

CFP Deadlines: April 7, 2016 to June 6, 2016

The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.

Deadline	Event Dates	Event	Location
April 7	April 8 April 10	mini Linux Audio Conference 2016	Berlin, Germany
April 8	August 2 August 5	Flock to Fedora	Krakow, Poland
April 15	June 27 July 1	12th Netfilter Workshop	Amsterdam, Netherlands
April 15	June 22 June 26	openSUSE Conference 2016	Nürnberg, Germany
April 24	August 20 August 21	Conference for Open Source Coders, Users and Promoters	Taipei, Taiwan
April 26	August 22 August 24	LinuxCon NA	Toronto, Canada
April 27	August 12 August 14	GNOME Users and Developers European Conference	Karlsruhe, Germany
April 30	June 11	TÜBIX 2016	Tübingen, Germany
April 30	October 27 October 28	Rust Belt Rust	Pittsburgh, PA, USA
April 30	August 25 August 26	The Prometheus conference	Berlin, Germany
May 1	August 24 August 26	KVM Forum 2016	Toronto, Canada
May 2	June 24 June 25	devopsdays Silicon Valley 2016	Mountain View, CA, USA
May 6	October 26 October 27	All Things Open	Raleigh, NC, USA
May 6	July 13 July 15	LinuxCon Japan	Tokyo, Japan
May 8	August 12 August 16	PyCon Australia 2016	Melbourne, Australia
May 15	July 2 July 9	DebConf16	Cape Town, South Africa
May 15	September 1 September 8	QtCon 2016	Berlin, Germany
May 15	June 11 June 12	Linuxwochen Linz	Linz, Austria
May 16	October 31 November 2	O’Reilly Security Conference	New York, NY, USA
May 23	October 17 October 19	O'Reilly Open Source Convention	London, UK
May 23	August 20 August 21	FrOSCon - Free and Open Source Software Conference	Sankt-Augustin, Germany
May 24	August 18 August 21	Camp++ 0x7e0	Komárom, Hungary
May 24	November 9 November 11	O’Reilly Security Conference EU	Amsterdam, Netherlands
May 25	October 5 October 7	International Workshop on OpenMP	Nara, Japan
May 29	September 20 September 23	PyCon JP 2016	Tokyo, Japan
May 30	September 13 September 16	PostgresOpen 2016	Dallas, TX, USA
June 3	June 24 June 25	French Perl Workshop 2016	Paris, France
June 4	July 30 July 31	PyOhio	Columbus, OH, USA
June 5	September 26 September 27	Open Source Backup Conference	Cologne, Germany
June 5	September 9 September 10	RustConf 2016	Portland, OR, USA

If the CFP deadline for your event does not appear here, please tell us about it.

miniLAC 2016 - Linux Audio Conference

There will be a mini Linux Audio Conference April 8-10 in Berlin, Germany.

Full Story (comments: none)

FSF: International Day Against DRM

The tenth International Day Against DRM will be May 3. "Bring the International Day Against DRM to your community. We support event planners of all experience levels by supplying printable materials, putting you in touch with other activists near you and online, and publicizing your event for maximum impact. Check out the amazing list of events from 2015's Day for inspiration and dive into the event planning guide."

Full Story (comments: none)

systemd.conf 2016

This year's systemd.conf will take place September 28-October 1 in Berlin, Germany. The Call for Presentations will open soon.

Full Story (comments: none)

Events: April 7, 2016 to June 6, 2016

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
April 4 April 8	OpenFabrics Alliance Workshop	Monterey, CA, USA
April 5 April 7	Lustre User Group 2016	Portland, OR, USA
April 7 April 8	SRECon16	Santa Clara, CA, USA
April 8 April 10	mini Linux Audio Conference 2016	Berlin, Germany
April 9 April 10	OSS Weekend	Bratislava, Slovakia
April 11 April 13	O’Reilly Software Architecture Conference	New York, NY, USA
April 15 April 18	Libre Graphics Meeting	London, UK
April 15 April 17	PyCon Italia Sette	Firenze, Italia
April 15 April 17	Akademy-es 2016	Madrid, Spain
April 16	15. Augsburger Linux Info Tag	Augsburg, Germany
April 18 April 19	Linux Storage, Filesystem & Memory Management Summit	Raleigh, NC, USA
April 18 April 20	PostgreSQL Conference US 2016	New York, NY, USA
April 20 April 21	Vault 2016	Raleigh, NC, USA
April 21 April 24	GNOME.Asia Summit	Delhi, India
April 23	DevCrowd 2016	Szczecin, Poland
April 23 April 24	LinuxFest Northwest	Bellingham, WA, USA
April 25 April 29	OpenStack Summit	Austin, TX, USA
April 25 April 27	Cuba International Free Software Conference	Havana, Cuba
April 26 April 28	Open Source Data Center Conference	Berlin, Germany
April 26	Open Source Day 2016	Warsaw, Poland
April 28 April 30	Linuxwochen Wien 2016	Vienna, Austria
April 28 May 1	Mini-DebCamp & DebConf	Vienna, Austria
April 30	Linux Presentation Day 2016.1	many cities, Germany
May 1 June 29	Open Source Innovation Spring	Paris, France
May 2 May 5	FOSS4G North America	Raleigh, NC, USA
May 2 May 3	PyCon Israel 2016	Tel Aviv, Israel
May 9 May 13	ApacheCon North America	Vancouver, Canada
May 10 May 12	Samba eXPerience 2016	Berlin, Germany
May 14 May 15	Community Leadership Summit 2016	Austin, TX, USA
May 14 May 15	Open Source Conference Albania	Tirana, Albania
May 16 May 19	OSCON 2016	Austin, TX, USA
May 17 May 21	PGCon - PostgreSQL Conference for Users and Developers	Ottawa, Canada
May 24 May 25	Cloud Foundry Summit	Santa Clara, CA, USA
May 26	NLUUG - Spring conference 2016	Bunnik, The Netherlands
May 28 June 5	PyCon 2016	Portland, OR, USA
June 1 June 2	Apache MesosCon	Denver, CO, USA
June 4 June 5	Coliberator 2016	Bucharest, Romania

If your event does not appear here, please tell us about it.