LWN.net Weekly Edition for March 26, 2015
A trademark battle in the Arduino community
The Arduino has been one of the biggest success stories of the open-hardware movement, but that success does not protect it from internal conflict. In recent months, two of the project's founders have come into conflict about the direction of future efforts—and that conflict has turned into a legal dispute about who owns the rights to the Arduino trademark.
The current fight is a battle between two companies that both bear the Arduino name: Arduino LLC and Arduino SRL. The disagreements that led to present state of affairs go back a bit further.
The Arduino project grew out of 2005-era course work taught at the Interaction Design Institute Ivrea (IDII) in Ivrea, Italy (using Processing, Wiring, and pre-existing microcontroller hardware). After the IDII program was discontinued, the open-hardware Arduino project as we know it was launched by Massimo Banzi, David Cuartielles, and David Mellis (who had worked together at IDII), with co-founders Tom Igoe and Gianluca Martino joining shortly afterward. The project released open hardware designs (including full schematics and design files) as well as the microcontroller software to run on the boards and the desktop IDE needed to program it.
Arduino LLC was incorporated in 2008 by Banzi, Cuartielles, Mellis, Igoe, and Martino. The company is registered in the United States, and it has continued to design the Arduino product line, develop the software, and run the Arduino community site. The hardware devices themselves, however, were manufactured by a separate company, "Smart Projects SRL," that was founded by Martino. "SRL" is essentially the Italian equivalent of "LLC"—Smart Projects was incorporated in Italy.
This division of responsibilities—with the main Arduino project handling everything except for board manufacturing—may seem like an odd one, but it is consistent with Arduino's marketing story. From its earliest days, the designs for the hardware have been freely available, and outside companies were allowed to make Arduino-compatible devices. The project has long run a certification program for third-party manufacturers interested in using the "Arduino" branding, but allows (and arguably even encourages) informal software and firmware compatibility.
The Arduino branding was not formally registered as a trademark in the early days, however. Arduino LLC filed to register the US trademark in April 2009, and it was granted in 2011.
At this point, the exact events begin to be harder to verify, but the original group of founders reportedly had a difference of opinion about how to license out hardware production rights to other companies. Wired Italy reports that Martino and Smart Projects resisted the other four founders' plans to "internationalize" production—although it is not clear if that meant that Smart Projects disapproved of licensing out any official hardware manufacturing to other companies, or had some other concern. Heise Online adds that the conflict seemed to be about moving some production to China.
What is clear is that Smart Projects filed a petition with the US Patent and Trademark Office (USPTO) in October 2014 asking the USPTO to cancel Arduino LLC's trademark on "Arduino." Then, in November 2014, Smart Projects changed its company's name to Arduino SRL. Somewhere around that time, Martino sold off his ownership stake in Smart Projects SRL and new owner Federico Musto was named CEO.
Unsurprisingly, Arduino LLC did not care for the petition to the USPTO and, in January 2015, the company filed a trademark-infringement lawsuit against Arduino SRL. Confusing matters further, the re-branded Arduino SRL has set up its own web site using the domain name arduino.org, which duplicates most of the site features found on the original Arduino site (arduino.cc). That includes both a hardware store and software downloads.
Musto, the new CEO of the company now called Arduino SRL, has a bit of a history with Arduino as well. His other manufacturing business had collaborated with Arduino LLC on the design and production of the Arduino Yún, which has received some criticism for including proprietary components.
Hackaday has run a two-part series (in February and March) digging into the ins and outs of the dispute, including the suggestion that Arduino LLC's recent release of version 1.6.0 of the Arduino IDE was a move intended to block Arduino SRL from hijacking IDE development. Commenter Paul Stoffregen (who was the author of the Heise story above) noted that Arduino SRL recently created a fork of the Arduino IDE on GitHub.
Most recently, Banzi broke his silence about the dispute in a story published at MAKEzine. There, Banzi claims that Martino secretly filed a trademark application on "Arduino" in Italy in 2008 and told none of the other Arduino founders. He also details a series of unpleasant negotiations between the companies, including Smart Projects stopping the royalty payments it had long sent to Arduino LLC for manufacturing devices and re-branding its boards with the Arduino.org URL.
Users appear to be stuck in the middle. Banzi says that several retail outlets that claim to be selling "official" Arduino boards are actually paying Arduino SRL, not Arduino LLC, but it is quite difficult to determine which retailers are lined up on which side, since there are (typically) several levels of supplier involved. The two Arduino companies' web sites also disagree about the available hardware, with Arduino.org offering the new Arduino Zero model for sale today and Arduino.cc listing it as "Coming soon."
Furthermore, as Hackaday's March story explains, the recently-released Arduino.cc IDE now reports that boards manufactured by Arduino SRL are "uncertified." That warning does not prevent users from programming the other company's hardware, but it will no doubt confuse quite a few users who believe they possess genuine Arduino-manufactured devices.
The USPTO page for Arduino SRL's petition notes pre-trial disclosure dates have been set for August and October of 2015 (for Arduino SRL and Arduino LLC, respectively), which suggests that this debate is far from over. Of course, it is always disappointing to observe a falling out between project founders, particularly when the project in question has had such an impact on open-source software and open hardware.
One could argue that disputes of this sort are proof that even small projects started among friends need to take legal and intellectual-property issues (such as trademarks) seriously from the very beginning—perhaps Arduino and Smart Projects thought that an informal agreement was all that was necessary in the early days, after all.
But, perhaps, once a project becomes profitable, there is simply no way to predict what might happen. Arduino LLC would seem to have a strong case for continual and rigorous use of the "Arduino" trademark, which is the salient point in US trademark law. It could still be a while before the courts rule on either side of that question, however.
Mapping and data mining with QGIS 2.8
QGIS is a free-software geographic information system (GIS) tool; it provides a unified interface in which users can import, edit, and analyze geographic-oriented information, and it can produce output as varied as printable maps or map-based web services. The project recently made its first update to be designated a long-term release (LTR), and that release is both poised for high-end usage and friendly to newcomers alike.
The new release is version 2.8, which was unveiled on March 2. An official change log is available on the QGIS site, while the release itself was announced primarily through blog posts (such as this post by Anita Graser of the project's steering committee). Downloads are available for a variety of platforms, including packages for Ubuntu, Debian, Fedora, openSUSE, and several other distributions.
![[QGIS main interface]](https://static.lwn.net/images/2015/03-qgis-map-sm.png)
As the name might suggest, QGIS is a Qt application; the latest release will, in fact, build on both Qt4 and Qt5, although the binaries released by the project come only in Qt4 form at present. 2.8 has been labeled a long-term release (LTR)—which, in this case, means that the project has committed to providing backported bug fixes for one full calendar year, and that the 2.8.x series is in permanent feature freeze. The goal, according to the change log, is to provide a stable version suitable for businesses and deployments in other large organizations. The change log itself points out that the development of quite a few new features was underwritten by various GIS companies or university groups, which suggests that taking care of these organizations' needs is reaping dividends for the project.
For those new to QGIS (or GIS in general), there is a detailed new-user tutorial that provides a thorough walk-through of the data-manipulation, mapping, and analysis functions. Being a new user, I went through the tutorial; although there are a handful of minor differences between QGIS 2.8 and the version used in the text (primarily whether specific features were accessed through a toolbar or right-click menu), on the whole it is well worth the time.
QGIS is designed to make short work of importing spatially oriented data sets, mining information from them, and turning the results into a meaningful visualization. Technically speaking, the visualization output is optional: one could simply extract the needed statistics and results and use them to answer some question or, perhaps, publish the massaged data set as a database for others to use.
But well-made maps are often the easiest way to illuminate facts about populations, political regions, geography, and many other topics when human comprehension is the goal. QGIS makes importing data from databases, web-mapping services (WMS), and even unwieldy flat-file data dumps a painless experience. It handles converting between a variety of map-referencing systems more or less automatically, and allows the user to focus on finding the useful attributes of the data sets and rendering them on screen.
Here be data
The significant changes in QGIS 2.8 fall into several categories. There are updates to how QGIS handles the mathematical expressions and queries users can use to filter information out of a data set, improvements to the tools used to explore the on-screen map canvas, and enhancements to the "map composer" used to produce visual output. This is on top of plenty of other under-the-hood improvements, naturally.
![[QGIS query builder]](https://static.lwn.net/images/2015/03-qgis-query-sm.png)
In the first category are several updates to the filtering tools used to mine a data set. Generally speaking, each independent data set is added to a QGIS project as its own layer, then transformed with filters to focus in on a specific portion of the original data. For instance, the land-usage statistics for a region might be one layer, while roads and buildings for the same region from OpenStreetMap might be two additional layers. Such filters can be created in several ways: there is a "query builder" that lets the user construct and test expressions on a data layer, then save the results, an SQL console for performing similar queries on a database, and spreadsheet-like editing tools for working directly on data tables.
All three have been improved in this release. New are support for if(condition, true, false) conditional statements, a set of operations for geometry primitives (e.g., to test whether regions overlap or lines intersect), and an "integer divide" operation. Users can also add comments to their queries to annotate their code, and there is a new custom function editor for writing Python functions that can be called in mathematical expressions within the query builder.
It is also now possible to select only some rows in a table, then perform calculations just on the selection—previously, users would have to extract the rows of interest into a new table first. Similarly, in the SQL editor, the user can highlight a subset of the SQL query and execute it separately, which is no doubt helpful for debugging.
There have also been several improvements to the Python and Processing plugins. Users can now drag-and-drop Python scripts onto QGIS and they will be run automatically. Several new analysis algorithms are now available through the Processing interface that were previously Python-only; they include algorithms for generating grids of points or vectors within a region, splitting layers and lines, generating hypsometric curves, refactoring data sets, and more.
Maps in, maps out
![[QGIS simplify tool]](https://static.lwn.net/images/2015/03-qgis-simplify-sm.png)
The process of working with on-screen map data picked up some improvements in the new release as well. Perhaps the most fundamental is that each map layer added to the canvas is now handled in its own thread, so fewer hangs in the user interface are experienced when re-rendering a layer (as happens whenever the user changes the look of points or shapes in a layer). Since remote databases can also be layers, this multi-threaded approach is more resilient against connectivity problems, too. The interface also now supports temporary "scratch" layers that can be used to merge, filter, or simply experiment with a data set, but are not saved when the current project is saved.
For working on the canvas itself, polygonal regions can now use raster images (tiled, if necessary) as fill colors, the map itself can be rotated arbitrarily, and objects can be "snapped" to align with items on any layer (not just the current layer). For working with raster image layers (e.g., aerial photographs) or simply creating new geometric shapes by hand, there is a new digitizing tool that can offer assistance by locking lines to specific angles, automatically keeping borders parallel, and other niceties.
There is a completely overhauled "simplify" tool that is used to reduce the number of extraneous vertices of a vector layer (thus reducing its size). The old simplify tool provided only a relative "tolerance" setting that did not correspond directly to any units. With the new tool, users can set a simplification threshold in terms of the underlying map units, layer-specific units, pixels, and more—and, in addition, the tool reports how much the simplify operation has reduced the size of the data.
![[QGIS style editing]](https://static.lwn.net/images/2015/03-qgis-style-sm.png)
There has also been an effort to present a uniform interface to one of the most important features of the map canvas: the ability to change the symbology used for an item based on some data attribute. The simplest example might be to change the line color of a road based on whether its road-type attribute is "highway," "service road," "residential," or so on. But the same feature is used to automatically highlight layer information based on the filtering and querying functionality discussed above. The new release allows many more map attributes to be controlled by these "data definition" settings, and provides a hard-to-miss button next to each attribute, through which a custom data definition can be set.
QGIS's composer module is the tool used to take project data and generate a map that can be used outside of the application (in print, as a static image, or as a layer for MapServer or some other software tool, for example). Consequently, it is not a simple select-and-click-export tool; composing the output can involve a lot of choices about which data to make visible, how (and where) to label it, and how to make it generally accessible.
The updated composer in 2.8 now has a full-screen mode and sports several new options for configuring output. For instance, the user now has full control over how map axes are labeled. In previous releases, the grid coordinates of the map could be turned on or off, but the only options were all or nothing. Now, the user can individually choose whether coordinates are displayed on all four sides, and can even choose in which direction vertical text labels will run (so that they can be correctly justified to the edge of the map, for example).
There are, as usual, many more changes than there is room to discuss. Some particularly noteworthy improvements include the ability to save and load bookmarks for frequently used data sources (perhaps most useful for databases, web services, and other non-local data) and improvements to QGIS's server module. This module allows one QGIS instance to serve up data accessible to other QGIS applications (for example, to simply team projects). The server can now be extended with Python plugins and the data layers that it serves can be styled with style rules like those used in the desktop interface.
QGIS is one of those rare free-software applications that is both powerful enough for high-end work and yet also straightforward to use for the simple tasks that might attract a newcomer to GIS in the first place. The 2.8 release, particularly with its project-wide commitment to long-term support, appears to be an update well worth checking out, whether one needs to create a simple, custom map or to mine a database for obscure geo-referenced meaning.
Development activity in LibreOffice and OpenOffice
The LibreOffice project was announced with great fanfare in September 2010. Nearly one year later, the OpenOffice.org project (from which LibreOffice was forked) was cut loose from Oracle and found a new home as an Apache project. It is fair to say that the rivalry between the two projects in the time since then has been strong. Predictions that one project or the other would fail have not been borne out, but that does not mean that the two projects are equally successful. A look at the two projects' development communities reveals some interesting differences.
Release histories
Apache OpenOffice has made two releases in the past year: 4.1 in April 2014 and 4.1.1 (described as "a micro update" in the release announcement) in August. The main feature added during that time would appear to be significantly improved accessibility support.
The release history for LibreOffice tells a slightly different story:
Release Date 4.2.3 April 2014 4.1.6 April 2014 4.2.4 May 2014 4.2.5 June 2014 4.3 July 2014 4.2.6 August 2014 4.3.1 August 2014 4.3.2 September 2014 4.2.7/4.3.3 October 2014 4.3.4 November 2014 4.2.8 December 2014 4.3.5 December 2014 4.4 January 2015 4.3.6 February 2015 4.4.1 February 2015
It seems clear that LibreOffice has maintained a rather more frenetic release cadence, generally putting out at least one release per month. The project typically keeps at least two major versions alive at any one time. Most of the releases are of the minor, bug-fix variety, but there have been two major releases in the last year as well.
Development statistics
In the one-year period since late March 2014, there have been 381 changesets committed to the OpenOffice Subversion repository. The most active committers are:
Most active OpenOffice developers
By changesets Herbert Dürr 63 16.6% Jürgen Schmidt 56 14.7% Armin Le Grand 56 14.7% Oliver-Rainer Wittmann 46 12.1% Tsutomu Uchino 33 8.7% Kay Schenk 27 7.1% Pedro Giffuni 23 6.1% Ariel Constenla-Haile 22 5.8% Andrea Pescetti 14 3.7% Steve Yin 11 2.9% Andre Fischer 10 2.6% Yuri Dario 7 1.8% Regina Henschel 6 1.6% Juan C. Sanz 2 0.5% Clarence Guo 2 0.5% Tal Daniel 2 0.5%
By changed lines Jürgen Schmidt 455499 88.1% Andre Fischer 26148 3.8% Pedro Giffuni 23183 3.4% Armin Le Grand 11018 1.6% Juan C. Sanz 4582 0.7% Oliver-Rainer Wittmann 4309 0.6% Andrea Pescetti 3908 0.6% Herbert Dürr 2811 0.4% Tsutomu Uchino 1991 0.3% Ariel Constenla-Haile 1258 0.2% Steve Yin 1010 0.1% Kay Schenk 616 0.1% Regina Henschel 417 0.1% Yuri Dario 268 0.0% tal 16 0.0% Clarence Guo 11 0.0%
In truth, the above list is not just the most active OpenOffice developers — it is all of them; a total of 16 developers have committed changes to OpenOffice in the last year. Those developers changed 528,000 lines of code, but, as can be seen above, Jürgen Schmidt accounted for the bulk of those changes, which were mostly updates to translation files.
The top four developers in the "by changesets" column all work for IBM, so IBM is responsible for a minimum of about 60% of the changes to OpenOffice in the last year.
The picture for LibreOffice is just a little bit different; in the same one-year period, the project has committed 22,134 changesets from 268 developers. The most active of these developers were:
Most active LibreOffice developers
By changesets Caolán McNamara 4307 19.5% Stephan Bergmann 2351 10.6% Miklos Vajna 1449 6.5% Tor Lillqvist 1159 5.2% Noel Grandin 1064 4.8% Markus Mohrhard 935 4.2% Michael Stahl 915 4.1% Kohei Yoshida 755 3.4% Tomaž Vajngerl 658 3.0% Thomas Arnhold 619 2.8% Jan Holesovsky 466 2.1% Eike Rathke 457 2.1% Matteo Casalin 442 2.0% Bjoern Michaelsen 421 1.9% Chris Sherlock 396 1.8% David Tardon 386 1.7% Julien Nabet 362 1.6% Zolnai Tamás 338 1.5% Matúš Kukan 256 1.2% Robert Antoni Buj Gelonch 231 1.0%
By changed lines Lionel Elie Mamane 244062 12.5% Noel Grandin 238711 12.2% Stephan Bergmann 161220 8.3% Miklos Vajna 129325 6.6% Caolán McNamara 97544 5.0% Tomaž Vajngerl 69404 3.6% Tor Lillqvist 59498 3.1% Laurent Balland-Poirier 52802 2.7% Markus Mohrhard 50509 2.6% Kohei Yoshida 45514 2.3% Chris Sherlock 36788 1.9% Peter Foley 34305 1.8% Christian Lohmaier 33787 1.7% Thomas Arnhold 32722 1.7% David Tardon 21681 1.1% David Ostrovsky 21620 1.1% Jan Holesovsky 20792 1.1% Valentin Kettner 20526 1.1% Robert Antoni Buj Gelonch 20447 1.0% Michael Stahl 18216 0.9%
To a first approximation, the top ten companies supporting LibreOffice in the last year are:
Companies supporting LibreOffice development (by changesets) Red Hat 8417 38.0% Collabora Multimedia6531 29.5% (Unknown) 5126 23.2% (None) 1490 6.7% Canonical 422 1.9% Igalia S.L. 80 0.4% Ericsson 21 0.1% Yandex 18 0.1% FastMail.FM 17 0.1% SUSE 7 0.0%
Development work on LibreOffice is thus concentrated in a small number of companies, though it is rather more spread out than OpenOffice development. It is worth noting that the LibreOffice developers with unknown affiliation, who contributed 23% of the changes, make up 82% of the developer base, so there would appear to be a substantial community of developers contributing from outside the above-listed companies.
Some conclusions
Last October, some concerns were raised on the OpenOffice list about the health of that project's community. At the time, Rob Weir shrugged them off as the result of a marketing effort by the LibreOffice crowd. There can be no doubt that the war of words between these two projects has gotten tiresome at times, but, looking at the above numbers, it is hard not to conclude that there is an issue that goes beyond marketing hype here.
In the 4½ years since its founding, the LibreOffice project has put together a community with over 250 active developers. There is support from multiple companies and an impressive rate of patches going into the project's repository. The project's ability to sustain nearly monthly releases on two branches is a direct result of that community's work. Swearing at LibreOffice is one of your editor's favorite pastimes, but it seems clear that the project is on a solid footing with a healthy community.
OpenOffice, instead, is driven by four developers from a single company — a
company that appears to have been deemphasizing OpenOffice work for some
time. As a result, the project's commit rate is a fraction of what
LibreOffice is able to sustain and releases are relatively rare. As of
this writing, the OpenOffice
blog shows no posts in 2015. In the October discussion, Rob said that "the dogs may
bark but the caravan moves on.
" That may be true, but, in this
case, the caravan does not appear to be moving with any great speed.
Anything can happen in the free-software development world; it is entirely possible that a reinvigorated OpenOffice.org may yet give LibreOffice a run for its money. But something will clearly have to change to bring that future around. As things stand now, it is hard not to conclude that LibreOffice has won the battle for developer participation.
Security
Toward secure package downloads
Kali Linux is a Debian-based distribution developed specifically for use in penetration testing applications. It packages a set of over 600 security-related tools, most of which seem to be oriented toward finding ways to break security schemes. Kali has clear value for people working in the security field — whether they are trying to improve security or defeat it. So it may have come as a surprise when the March 19, 2015 Risks Digest included a message claiming that Kali Linux security is "a joke." The resulting discussion, most of which played out on the oss-security list, suggests that no distribution has yet come up with model for packaging that is entirely lacking in joke-like qualities.The immediate complaint was that Kali serves its distribution without signatures, via plain HTTP. In fact, though the Kali download page is served encrypted with TLS, the actual download links are unencrypted and, thus, the downloaded data would be relatively easy to modify via a man-in-the-middle (MITM) attack. The images are checksummed, though, and the files containing the checksums are signed by the Kali Linux private key, so there is protection there for those who are willing and able to verify the signature and the checksums.
Still, there are a couple of problems with this arrangement, starting with the fact that most users — even, one would guess, users of a distribution like Kali — are not going to actually go through that verification process. For such users, files served with TLS are the only line of defense against MITM attacks. But even users who will verify signatures must get the relevant public key from somewhere. In almost all cases, the only option is to download that key from the distributor's web site — hopefully protected by TLS. So protecting downloaded files with TLS would appear to be an important part of keeping a user's systems secure.
There is still a little problem here, though: the TLS certificate system and the certificate authorities (CAs) that support it have not earned a lot of respect over the years. As Russ Allbery put it:
Even in the absence of falsified certificates, it is easy to put too much trust into TLS. As Daniel Micay noted, TLS is not necessarily a good fit to how software is actually distributed:
Solar Designer summarized things concisely by
saying "I find it ridiculous if we primarily complain that some site
serves downloads over http, and I find it ridiculous if we say they fixed
'the problem' when they move to https.
" As in many other
situations, it seems that TLS on its own provides little in the way of
actual security.
That leaves open the question of whether there is any hope of securing distribution downloads, or whether we are all at the mercy of the next MITM attacker who comes along. The (partial) answer, of course, is to use end-to-end verification: the distributor attaches a signature to packages that are then verified on the user's system. Most distributors now use such a scheme; once the keys are properly installed on a system, that system can verify the provenance any packages before allowing them to be installed. Signatures can be applied to individual packages (as RPM-based distributions tend to do) or to the state of the repository as a whole (as Debian does); there are advantages and disadvantages either way, but either approach can harden the package-delivery path against attack. Though, as the long history of Red Hat bug #998 shows, getting to that point can take some time.
One problem remains: getting the correct public key onto the user's system in the first place. If that step can be subverted, there is no security in anything that follows. That is where one might hope that TLS would come to the rescue; if the user can be sure that they are connecting to the right site, they can (in the absence of a compromise of the distributor's site) assume that the keys they download are legitimate. But, as was mentioned above, confidence in the ability of TLS to protect these downloads is not high.
One possibility for improving the situation is certificate pinning — verifying a certificate through a trusted third party. The fingerprints of pinned certificates are stored in a database somewhere; should a site offer a certificate that does not agree with the pinned version, the download will fail (or, at least, put up a warning). Daniel suggested that distributors of any significance should be able to get their certificates added to the Google Chromium set; since pinning in Firefox starts by importing the Chromium set, any pins would move over there as well. Getting their certificates added should help distributors defend against MITM attacks on their keys, but it's an additional bureaucratic process that, so far, few distributors have bothered with.
Pinning only works with TLS-protected downloads, though. A quick check shows that Debian, Fedora, openSUSE, and Ubuntu all offer their installation-image downloads over unencrypted HTTP connections. Additionally, these images can be downloaded from mirror sites or via BitTorrent — bypassing the distributor's site entirely and rendering any certificate protection moot. Since the installation image is generally the source for the distribution's public keys, the potential for tampering is real.
So what we have in the end is a software-distribution mechanism that mostly works, but only if the initial download of the distribution (or its keys) is not compromised. That appears to be good enough in almost all situations; reports of systems compromised by malicious installation images are rare. Still, it is easy to argue that distributors should do better. But getting there involves finding a solution to the key-distribution problem — a problem that has resisted easy solutions for many years now.
Brief items
Security quote of the week
Docker security in the future (Opensource.com)
Over at Opensource.com, Daniel Walsh writes about applying various Linux security technologies to Docker containers. In the article, he looks at using user namespaces and seccomp filters to provide better security for Docker. "One of the problems with all of the container separation modes described here and elsewhere is that they all rely on the kernel for separation. Unlike air gapped computers, or even virtual machines, the processes within the container can talk directly to the host kernel. If the host kernel has a kernel vulnerability that a container can access, they might be able to disable all of the security and break out of the container. The x86_64 Linux kernel has over 600 system calls, a bug in any one of which could lead to a privilege escalation. Some of the system calls are seldom called, and should be eliminated from access within the container."
Van de Ven: Deprecating old crypto
Worth a read: this post from Arjan van de Ven on the difficulty of removing old, insecure cryptographic algorithms from a Linux distribution. "But more, and this is a call to action: If you're working on an open source project that uses crypto, please please don't opencode crypto algorithm usage. The algorithm may be outdated at any time and might have to go away in a hurry."
Google: Maintaining digital certificate security
It seems it was about time for another certificate authority horror story; the Google Online Security Blog duly delivers. "CNNIC responded on the 22nd to explain that they had contracted with MCS Holdings on the basis that MCS would only issue certificates for domains that they had registered. However, rather than keep the private key in a suitable HSM, MCS installed it in a man-in-the-middle proxy. These devices intercept secure connections by masquerading as the intended destination and are sometimes used by companies to intercept their employees’ secure traffic for monitoring or legal reasons. The employees’ computers normally have to be configured to trust a proxy for it to be able to do this. However, in this case, the presumed proxy was given the full authority of a public CA, which is a serious breach of the CA system."
New vulnerabilities
batik: information leak
Package(s): | batik | CVE #(s): | CVE-2015-0250 | ||||||||||||||||||||||||||||||||||||
Created: | March 25, 2015 | Updated: | June 5, 2015 | ||||||||||||||||||||||||||||||||||||
Description: | From the Ubuntu advisory:
Nicolas Gregoire and Kevin Schaller discovered that Batik would load XML external entities by default. If a user or automated system were tricked into opening a specially crafted SVG file, an attacker could possibly obtain access to arbitrary files or cause resource consumption. | ||||||||||||||||||||||||||||||||||||||
Alerts: |
|
chromium: denial of service
Package(s): | chromium | CVE #(s): | CVE-2015-1232 | ||||||||
Created: | March 23, 2015 | Updated: | April 1, 2015 | ||||||||
Description: | From the CVE entry:
Array index error in the MidiManagerUsb::DispatchSendMidiData function in media/midi/midi_manager_usb.cc in Google Chrome before 41.0.2272.76 allows remote attackers to cause a denial of service or possibly have unspecified other impact by leveraging renderer access to provide an invalid port index that triggers an out-of-bounds write operation, a different vulnerability than CVE-2015-1212. | ||||||||||
Alerts: |
|
compat-openssl098: denial of service
Package(s): | compat-openssl098 | CVE #(s): | CVE-2009-5146 | ||||||||
Created: | March 20, 2015 | Updated: | March 25, 2015 | ||||||||
Description: | From the SUSE advisory: A memory leak in the TLS hostname extension was fixed, which could be used by remote attackers to run SSL services out of memory. | ||||||||||
Alerts: |
|
csync2: file checksum collision
Package(s): | csync2 | CVE #(s): | CVE-2014-8242 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | March 20, 2015 | Updated: | October 15, 2015 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | From the Fedora advisory: librsync previously used a truncated MD4 "strong" check sum to match blocks. However, MD4 is not cryptographically strong. It's possible that an attacker who can control the contents of one part of a file could use it to control other regions of the file, if it's transferred using librsync/rdiff. For example this might occur in a database, mailbox, or VM image containing some attacker-controlled data. To mitigate this issue, signatures will by default be computed with a 256-bit BLAKE2 hash. Old versions of librsync will complain about a bad magic number when given these signature files. Backward compatibility can be obtained using the new `rdiff sig --hash=md4` option or through specifying the "signature magic" in the API, but this should not be used when either the old or new file contain untrusted data. Deltas generated from those signatures will also use BLAKE2 during generation, but produce output that can be read by old versions. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
drupal: multiple vulnerabilities
Package(s): | drupal | CVE #(s): | CVE-2015-2559 | ||||||||||||||||||||||||||||||||||||
Created: | March 23, 2015 | Updated: | March 30, 2015 | ||||||||||||||||||||||||||||||||||||
Description: | From the Drupal advisory:
Access bypass (Password reset URLs - Drupal 6 and 7) Password reset URLs can be forged under certain circumstances, allowing an attacker to gain access to another user's account without knowing the account's password. Open redirect (Several vectors including the "destination" URL parameter - Drupal 6 and 7) Drupal core and contributed modules frequently use a "destination" query string parameter in URLs to redirect users to a new destination after completing an action on the current page. Under certain circumstances, malicious users can use this parameter to construct a URL that will trick users into being redirected to a 3rd party website, thereby exposing the users to potential social engineering attacks. | ||||||||||||||||||||||||||||||||||||||
Alerts: |
|
gdm: screen lock bypass
Package(s): | gdm | CVE #(s): | |||||
Created: | March 19, 2015 | Updated: | March 25, 2015 | ||||
Description: | From the SUSE bugzilla entry:
Using the latest openSUSE Factory snapshot (also present in openSUSE 13.2 RC1) GNOME automatically unlocks if fprintd is present
The user never gets asked for a password journal shows fprintd starting each time the lock screen is activated, but no errors or warnings to imply it's misbehaving Removing fprintd 'resolves' the issue but disables fingerprint authentication | ||||||
Alerts: |
|
gnutls: potenially invalid certificates
Package(s): | gnutls26, gnutls28 | CVE #(s): | CVE-2014-8155 | ||||||||||||||||
Created: | March 24, 2015 | Updated: | March 25, 2015 | ||||||||||||||||
Description: | From the Ubuntu advisory:
It was discovered that GnuTLS did not perform date and time checks on CA certificates, contrary to expectations. This issue only affected Ubuntu 10.04 LTS. | ||||||||||||||||||
Alerts: |
|
kernel: two vulnerabilities
Package(s): | kernel | CVE #(s): | CVE-2015-2150 CVE-2015-2042 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | March 23, 2015 | Updated: | April 23, 2015 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | From the CVE entry:
Xen 3.3.x through 4.5.x and the Linux kernel through 3.19.1 do not properly restrict access to PCI command registers, which might allow local guest users to cause a denial of service (non-maskable interrupt and host crash) by disabling the (1) memory or (2) I/O decoding for a PCI Express device and then accessing the device, which triggers an Unsupported Request (UR) response. (CVE-2015-2150) From the Red Hat bugzilla: A flaw was found in the method that the linux kernel handles userspace tuning of the Reliable Datagram Sockets (RDS) system settings. The incorrect handling allowed a trusted user to set multiple RDS sysctls for RDS with specially formatted data. Reading from these files also returned data from other sysctl settings that would be exposed via the same permissions to this user. (CVE-2015-2042) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
krb5: denial of service
Package(s): | krb5 | CVE #(s): | CVE-2014-5355 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | March 20, 2015 | Updated: | March 29, 2015 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | From the openSUSE advisory: krb5: denial of service in krb5_read_message. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
less: information leak
Package(s): | less | CVE #(s): | CVE-2014-9488 | ||||||||||||||||||||
Created: | March 25, 2015 | Updated: | June 11, 2015 | ||||||||||||||||||||
Description: | From the openSUSE advisory:
Malformed UTF-8 data could have caused an out of bounds read in the UTF-8 decoding routines, causing an invalid read access. | ||||||||||||||||||||||
Alerts: |
|
mono: three SSL/TLS vulnerabilities
Package(s): | mono | CVE #(s): | CVE-2015-2318 CVE-2015-2319 CVE-2015-2320 | ||||||||||||||||
Created: | March 19, 2015 | Updated: | April 16, 2015 | ||||||||||||||||
Description: | From the Debian-LTS advisory:
CVE-2015-2318: Mono's implementation of the SSL/TLS stack failed to check the order of the handshake messages. Which would allow various attacks on the protocol to succeed. ("SKIP-TLS") CVE-2015-2319: Mono's implementation of SSL/TLS also contained support for the weak EXPORT cyphers and was susceptible to the FREAK attack. CVE-2015-2320: Mono contained SSLv2 fallback code, which is no longer needed and can be considered insecure. | ||||||||||||||||||
Alerts: |
|
moodle: multiple vulnerabilities
Package(s): | moodle | CVE #(s): | CVE-2015-2266 CVE-2015-2267 CVE-2015-2268 CVE-2015-2269 CVE-2015-2270 CVE-2015-2271 CVE-2015-2272 CVE-2015-2273 | ||||||||||||
Created: | March 19, 2015 | Updated: | April 6, 2015 | ||||||||||||
Description: | From the Mageia advisory:
In Moodle before 2.6.9, by modifying URL a logged in user can view the list of another user's contacts, number of unread messages and list of their courses (CVE-2015-2266). In Moodle before 2.6.9, authentication in mdeploy can be bypassed. It is theoretically possible to extract files anywhere on the system where the web server has write access. The attacking user must know details about the system and already have significant permissions on the site (CVE-2015-2267). In Moodle before 2.6.9, a non-optimal regular expression in the "Convert links to URLs" filter could be exploited to create extra server load or make particular pages unavailable (CVE-2015-2268). In Moodle before 2.6.9, it is possible to create HTML injection through blocks with configurable titles, however this could only be exploited by users who are already marked as XSS-trusted (CVE-2015-2269). In Moodle before 2.6.9, for the custom themes that use blocks regions in the base layout the blocks for inaccessible courses could be displayed together with sensitive course-related information. Most of the themes, including all standard Moodle themes, are not affected (CVE-2015-2270). In Moodle before 2.6.9, users without proper permission are able to mark tags as inappropriate. Since this capability is given to authenticated users by default, this is not an issue for most sites (CVE-2015-2271). In Moodle before 2.6.9, even when user's password is forced to be changed on login, user could still use it for authentication in order to create the web service token and therefore extend the life of the temporary password via web services (CVE-2015-2272). In Moodle before 2.6.9, Quiz statistics report did not properly escape student responses and could be used for XSS attack (CVE-2015-2273). | ||||||||||||||
Alerts: |
|
mozilla: multiple vulnerabilities
Package(s): | iceweasel firefox thunderbird seamonkey | CVE #(s): | CVE-2015-0817 CVE-2015-0818 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | March 23, 2015 | Updated: | April 1, 2015 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | From the Debian advisory:
CVE-2015-0817: ilxu1a reported a flaw in Mozilla's implementation of typed array bounds checking in JavaScript just-in-time compilation (JIT) and its management of bounds checking for heap access. This flaw can be leveraged into the reading and writing of memory allowing for arbitrary code execution on the local system. CVE-2015-0818: Mariusz Mlynski discovered a method to run arbitrary scripts in a privileged context. This bypassed the same-origin policy protections by using a flaw in the processing of SVG format content navigation. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
openssl: multiple vulnerabilities
Package(s): | openssl | CVE #(s): | CVE-2015-0209 CVE-2015-0286 CVE-2015-0287 CVE-2015-0288 CVE-2015-0289 CVE-2015-0292 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | March 20, 2015 | Updated: | May 5, 2015 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | From the Debian advisory: CVE-2015-0286 Stephen Henson discovered that the ASN1_TYPE_cmp() function can be crashed, resulting in denial of service. CVE-2015-0287 Emilia Kaesper discovered a memory corruption in ASN.1 parsing. CVE-2015-0289 Michal Zalewski discovered a NULL pointer dereference in the PKCS#7 parsing code, resulting in denial of service. CVE-2015-0292 It was discovered that missing input sanitising in base64 decoding might result in memory corruption. CVE-2015-0209 It was discovered that a malformed EC private key might result in memory corruption. CVE-2015-0288 It was discovered that missing input sanitising in the X509_to_X509_REQ() function might result in denial of service. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
openssl: denial of service
Package(s): | openssl | CVE #(s): | CVE-2015-0293 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | March 20, 2015 | Updated: | May 5, 2015 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | From the OpenSSL advisory: A malicious client can trigger an OPENSSL_assert (i.e., an abort) in servers that both support SSLv2 and enable export cipher suites by sending a specially crafted SSLv2 CLIENT-MASTER-KEY message. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
php: code execution
Package(s): | php5 | CVE #(s): | CVE-2015-2331 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | March 23, 2015 | Updated: | April 22, 2015 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | From the Debian advisory:
Emmanuel Law discovered an integer overflow in the processing of ZIP archives, resulting in denial of service or potentially the execution of arbitrary code. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
phpZendFramework2: cross-site forgery
Package(s): | php-ZendFramework2 | CVE #(s): | |||||||||||||
Created: | March 20, 2015 | Updated: | March 25, 2015 | ||||||||||||
Description: | From the Zend advisory: Zend\Validator\Csrf, starting in the Zend Framework 2.3 series, was not correctly identifying null or mal-formatted token identifiers, leading to false positive validations, and thus potentially allowing for Cross-Site Request Forgery vectors. | ||||||||||||||
Alerts: |
|
python-django: two vulnerabilities
Package(s): | python-django | CVE #(s): | CVE-2015-2316 CVE-2015-2317 | ||||||||||||||||||||||||||||||||
Created: | March 24, 2015 | Updated: | June 19, 2015 | ||||||||||||||||||||||||||||||||
Description: | From the Ubuntu advisory:
Andrey Babak discovered that Django incorrectly handled strip_tags. A remote attacker could possibly use this issue to cause Django to enter an infinite loop, resulting in a denial of service. This issue only affected Ubuntu 14.04 LTS and Ubuntu 14.10. (CVE-2015-2316) Daniel Chatfield discovered that Django incorrectly handled user-supplied redirect URLs. A remote attacker could possibly use this issue to perform a cross-site scripting attack. (CVE-2015-2317) | ||||||||||||||||||||||||||||||||||
Alerts: |
|
qt-creator: fails to verify SSH host key
Package(s): | qt-creator | CVE #(s): | |||||||||||||
Created: | March 23, 2015 | Updated: | April 10, 2015 | ||||||||||||
Description: | From the Red Hat bugzilla:
It was reported that Qt Creator does not verify SSH host key when using built-in SSH client. | ||||||||||||||
Alerts: |
|
tor: denial of service
Package(s): | tor | CVE #(s): | CVE-2015-2688 CVE-2015-2689 | ||||||||||||||||||||||||||||
Created: | March 23, 2015 | Updated: | April 6, 2015 | ||||||||||||||||||||||||||||
Description: | From the Debian advisory:
Several denial-of-service issues have been discovered in Tor, a connection-based low-latency anonymous communication system.
CVEs were not available when this entry was created, and were added later. See the Tor release announcement for details. | ||||||||||||||||||||||||||||||
Alerts: |
|
xen: two vulnerabilities
Package(s): | xen | CVE #(s): | CVE-2015-2152 CVE-2015-1563 | ||||||||||||||||||||||||||||
Created: | March 23, 2015 | Updated: | March 25, 2015 | ||||||||||||||||||||||||||||
Description: | From the CVE entries:
Xen 4.5.x and earlier enables certain default backends when emulating a VGA device for an x86 HVM guest qemu even when the configuration disables them, which allows local guest users to obtain access to the VGA console by (1) setting the DISPLAY environment variable, when compiled with SDL support, or connecting to the VNC server on (2) ::1 or (3) 127.0.0.1, when not compiled with SDL support. (CVE-2015-2152) The ARM GIC distributor virtualization in Xen 4.4.x and 4.5.x allows local guests to cause a denial of service by causing a large number messages to be logged. (CVE-2015-1563) | ||||||||||||||||||||||||||||||
Alerts: |
|
xerces-c: denial of service
Package(s): | xerces-c | CVE #(s): | CVE-2015-0252 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | March 23, 2015 | Updated: | June 30, 2015 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | From the Debian advisory:
Anton Rager and Jonathan Brossard from the Salesforce.com Product Security Team and Ben Laurie of Google discovered a denial of service vulnerability in xerces-c, a validating XML parser library for C++. The parser mishandles certain kinds of malformed input documents, resulting in a segmentation fault during a parse operation. An unauthenticated attacker could use this flaw to cause an application using the xerces-c library to crash. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
Page editor: Jake Edge
Kernel development
Brief items
Kernel release status
The current development kernel is 4.0-rc5, released on March 22. Linus said: "There's nothing particularly worrisome going on, although I'm still trying to think about the NUMA balancing performance regression. It may not be a show-stopper, but it's annoying, and I want it fixed. We'll get it, I'm sure."
Stable updates: none have been released in the last week. The 3.19.3, 3.14.37, and 3.10.73 updates are in the review process as of this writing; they can be expected on or after March 26.
Quotes of the week
Kernel development news
NFS performance
On day two of the 2015 Linux Storage, Filesystem, and Memory Management Summit, Chuck Lever led a discussion on NFS performance. There are some bottlenecks to look at, and suggestions were made on ways to avoid some of them.
The transport_lock is a spinlock used by the Remote Procedure Call (RPC) layer. It is a bit like the Big Kernel Lock (BKL), Lever said, in that it protects all of the transport data on a per-socket basis. It is used as a queueing mechanism to prevent RPCs from interleaving on the wire. He is looking for ways to break up that lock, much as the BKL-removal work did with the BKL.
![Chuck Lever [Chuck Lever]](https://static.lwn.net/images/2015/lsf-lever-sm.jpg)
Currently, a thread is woken up to copy the received data, but it might make more sense to do that work in software interrupt (softirq) context, Jeff Layton said. That is how remote DMA (RDMA) does things, Lever said. Layton said you could start by simply doing copies out of the socket buffer from the softirq, but eventually using splice() might provide even better performance.
Lever said that there is also a proposal to make incoming data be page-aligned. Andreas Gruenbacher said that the idea was to use large network frames and to receive them into page-aligned buffers.
Dave Chinner said that will require the sending side to be aware of that setting so that it can form its TCP packets in large frames. Bruce Fields said that the networking developers didn't like the change. Chinner said that he was not surprised, as messing with segment boundaries is always tricky. Gruenbacher noted that it required using the new huge frames to get enough data into one packet, as doing page-aligned receives on small packets will just waste space.
One of the two data copies that are currently being done could be saved if the softirq code changed to look inside the RPC packets, Fields said. By figuring out what the packet contains, the RPC code could route it to the right place, sometimes using splice(). Lever said that RDMA solves the copying problem nicely, but that it is a niche use case and likely to remain that way.
Another area of performance improvement is to use NFS compounds, which allow multiple read and write operations in a single NFS transaction. Lever said that Fields has been working on support for that as part of the NFS 4.2 support in Linux.
In addition, Lever said, there is a new operation in 4.2 called READ_PLUS that will assist when clients are reading sparse files. That operation allows the server to report the holes optimally. There was concern that rematerializing the holes on the client might be expensive, but that turned out not to be the case.
Fields said that he used SEEK_HOLE and SEEK_DATA flags to lseek() to add the holes to the files on the client side. But Chinner cautioned that there is no way of atomically finding holes and returning data beyond them, as it will always race with other operations that are happening on the file.
Lever said that NFS delegations, which are a kind of file lock, would be required from the server when the READ_PLUS operation is used. That will only be granted by the server if no one has the file open for writing. However, delegation is not enabled on all NFS servers. And that is where the conversation kind of trailed off.
[I would like to thank the Linux Foundation for travel support to Boston for the summit.]
Filesystem defragmentation
Dmitry Monakhov prefaced his 2015 LSFMM Summit session on filesystem defragmentation with a statement that the "problem is almost already solved". His session turned into a largely informational description of the status of a defragmentation tool that he has been working on.
Over time, filesystems change and cannot avoid fragmentation issues, he said. For example, extracting a Linux source tree results in many small files that filesystem tries to allocate close to each other. Building in the tree results in lots of temporary files that get removed, so the filesystem gets fragmented.
Beyond appearing in regular filesystems, these fragmentation problems show up in thin provisioning systems, as well as for shingled magnetic recording (SMR) devices, he said. In addition, to make boot times shorter, it would be best to lay out all the needed files sequentially on the disk, which may require defragmentation.
The fragmentation problem is already solved for large files. Btrfs, XFS, and ext4 all have tools for doing defragmentation on files. But there is no solution for directory fragmentation. The filesystems try to put files that are in the same directory close to each other on the disk, but as files get deleted or moved, fragmentation of the directory occurs.
To perform defragmentation, it is often necessary to copy file data from one place to another. Monakhov suggested that a checksum could be calculated on the data when doing that copy, which could then be stored in a "trusted" extended attribute (xattr). He noted that overlayfs uses the "trusted.overlay" xattr, which can only be modified by processes with CAP_SYS_ADMIN, so a "trusted.sha1" (or or other hash) could be calculated and stored when copying data for defragmentation.
Executable files could then have their contents checked and compared to the hash value before being executed. He proposed adding that capability to his tool, but it seemed to be something of an aside. It is not clear how it relates to the integrity measurement architecture (IMA), for example.
He has been working on a tool called e4defrag2 (developed in a branch of e2fsprogs) that will perform defragmentation. It is mostly independent of the filesystem type. It uses the same block scanning code to find fragmentation, but ext4 and XFS have a different ioctl() name for their defragmentation operations.
The result is a "giant utility that works for everything", Monakhov said. The filesystem-dependent part is roughly 100 lines of code. This "universal defragmenter" will be released soon.
Ted Ts'o asked what would be needed to eliminate the 100 lines. He asked if wiring up the XFS ioctl() name into ext4 would help. Monakhov said that the tool needs to get the block bitmap from the filesystem, which is also different between the filesystems. Ts'o and Dave Chinner indicated that they would attempt to provide the same interfaces. Chinner did caution that XFS cannot defragment a range in a file, only the whole file. That is different than ext4, Monakhov said.
[I would like to thank the Linux Foundation for travel support to Boston for the summit.]
UID/GID identity and filesystems
"User namespaces only solve half the problem", Andy Lutomirski said to start off his session at the 2015 LSFMM Summit. User namespaces remap user IDs (UIDs) and group IDs (GIDs) in the running kernel, but they don't do anything for the UID and GID values stored in filesystems. Those IDs are simply integers stored in the filesystem metadata.
Lutomirski noted that when inserting a USB stick with a "real filesystem, not FAT" on it, the mounted filesystem will have UIDs and GIDs that are likely to be wrong. It would be nice, he said, if instead the files showed up as being owned by the user's UID.
This is also a problem for both NFS and FUSE filesystems, he continued. There is a partial solution in that mounting a FUSE filesystem inside a user namespace will map the UIDs inside the namespace before writing them to the filesystem. NFS has a solution as well. He wondered if there could be a more general approach.
Dave Chinner pointed out that some filesystems have mount options to do simple UID remapping. Those options might simply squash all UID/GIDs on the filesystem into a single UID/GID. An option like that could be added to the virtual filesystem (VFS) layer so that all filesystems had access to it.
That might be a reasonable way to approach the problem, Lutomirski said. Obviously NFS has already solved it, he said, though he had not looked to see what it does. Jeff Layton said that NFS has traditionally mapped UIDs and GIDs between the server and the client. That was originally done using strings for the user and group names, which would get mapped at the other end to integers. The current NFS solution is more complicated, Bruce Fields said, involving LDAP lookups, which is probably not what Lutomirski is looking for.
For his use case, squashing to a single UID would be sufficient, Lutomirski said. Handling Linux Security Module (LSM) contexts is trickier, but that could perhaps be added later. There was some discussion of the different ways that filesystems interpret the uid= and gid= mount options; he would like to see there be some uniformity, which would might require an entirely new mount option (possibly something like vfs_uid=).
[I would like to thank the Linux Foundation for travel support to Boston for the summit.]
Issues with epoll()
In a filesystem session at the 2015 LSFMM Summit, Jason Baron led a discussion about the epoll() system call. He and others have observed some performance problems with epoll(), especially for large sets of monitored file descriptors. There are two problems that Baron is trying to address: the "thundering herd" problem on wakeups and the use of global locks when manipulating the epoll() sets. He has posted patches for both, but they haven't really been commented on, he said. He also noted that Fam Zheng has posted some patches that add new system calls for epoll().
![Jason Baron [Jason Baron]](https://static.lwn.net/images/2015/lsf-baron-sm.jpg)
The thundering herd problem occurs when there are multiple threads that share a wakeup source in their epoll() sets. When that file descriptor becomes ready, all of the threads waiting wake up, even though only one of them is needed to service the event. One solution that had been suggested was to have a single epoll() queue, with all events being taken off that single queue. But that is not optimal for what he is trying to do, he said.
His patches simply wakeup the first idle thread that is waiting, then round-robin through the threads on subsequent wakeups. Some suggested using CPU affinity to wake up the thread on the CPU where the interrupt has come in. But epoll() doesn't currently have access to that kind of information. Baron has "heard vaguely" that some people are doing this, but he hasn't seen any patches. He would like to explore the idea further.
His initial proposal was to simply wake up one thread waiting on the epoll() set, but there was concern that might break programs that were expecting the current behavior. The wait queue used is associated with a file descriptor, so it is local to the process (and its threads), rather than global. A flag passed to epoll() could change the behavior for a program without affecting other programs that might also be waiting.
Another option that he has tried is to change the wakeup behavior in the scheduler, though he was worried that the scheduler developers would be unhappy with a change like that. When he posted it, though, there was no feedback of that sort. Still, avoiding changes to the wakeup code is desirable.
But epoll() has the ability to nest the file descriptors it is monitoring. That means a set of file descriptors can be constructed that contains descriptors returned from other epoll_create1() calls. In the past, loops could be created that way, though that has been fixed. One could use the nesting capability, coupled with a new flag to epoll_create1() to add the round-robin feature, but restrict the changes to the epoll() code instead of changing the wakeup code.
Jeff Layton asked if there would be two flags, one to request the CPU affinity mode and one for the round-robin behavior. But Baron did not think both would be needed. The CPU affinity mode could simply fall back to round-robin behavior if the interrupt did not come in on a CPU that was running a thread waiting on the event.
He moved on to locking, which has shown up in some profiles of epoll() performance. Akamai (where Baron works) has not necessarily run into it, but people don't like global locks, in general, he said. Part of the problem is that the kernel does not know when the sets have file descriptors in common, so it locks everything when manipulating them.
The idea is to break up the locks in the classic way, he said, so that operations are serialized only for sets with common file descriptors. He posted patches a few months ago, but they added three pointer fields to struct file, which was not something other developers were happy with. He plans to switch to only adding a single pointer that points to a structure to hold anything that epoll() needs. It would be allocated when the epoll() file descriptor is created.
In addition, his patches eliminate the runtime checking for loops and too deep of nesting in the file descriptor sets. Right now those checks are done when calling epoll_wait(), but his patches do that checking when file descriptors are added to the set in epoll_ctl().
Layton asked if all of this work meant that Baron was volunteering to be the epoll() maintainer. Baron was non-committal, but Chris Mason suggested (with a chuckle) that if these patches were accepted, that would more or less happen by default.
Mason said that Facebook is hitting some of these problems, as is Google. Someone said that GlusterFS is hitting them too. Baron said that Akamai would be using his patches in production, so they should get lots of testing.
There are other epoll() patches out there, including those for new system calls from Zheng. Others include a patch that would add a lockless way to enqueue and dequeue events and one that would optimistically wait (briefly) in the kernel for another event rather than immediately go to sleep. The person working on the latter patches, which were targeted at networking, is now working on other things, Matthew Wilcox said, so they could be taken over by someone else if that was of interest.
It would seem that scalability problems with epoll() are cropping up in a number of places, so some fixes are needed. Baron's patches are not running into much in the way of opposition, at least from the assembled filesystem developers, which means they may make their way into the mainline before long.
[I would like to thank the Linux Foundation for travel support to Boston for the summit.]
Copy offload
In the final combined storage-and-filesystem session at the 2015 LSFMM Summit, Zach Brown and Martin Petersen teamed up to describe the state of and plans for supporting copy offload, which is a way of handing the work of copying a file to a filesystem or lower-level storage device, where the task can often be optimized. The functionality has been available in storage devices for eight years or so, Brown said.
The current strategy is to add a new system call, copy_file_range(), that takes two file descriptors with pointers to offsets and lengths, Brown said. As the later discussion indicated, those file descriptors could be for files on different filesystems, but some feel that they should be restricted to a single filesystem. The big difference from earlier proposals is that callers are now required to create the destination file. That avoids some race conditions in the virtual filesystem (VFS) layer.
![Zach Brown [Zach Brown]](https://static.lwn.net/images/2015/lsf-brown-sm.jpg)
The remaining contentious parts for the system call are minor, he continued. For example, a flag value for the length could indicate that the entire source file should be copied. There is a "whole world of shit we can argue about", he said, since there are 32 bits worth of flag values available. The contentious piece is on the block side, he said. Petersen has added support, but the device mapper developers did not like the approach he took.
For Btrfs, the system call is a wrapper around the existing ioctl(), though there are some alignment issues still to be worked out. Chris Mason said that for Btrfs there are different options for doing copy offload. Creating a directory subvolume is a constant-time operation that can make a copy of an entire file (using copy on write or COW). Making a file copy directly, which could support a range in the file (again, using COW), is proportional to the number of extents in the file. Brown suggested that under the covers Btrfs could implement the copy as a subvolume creation if the copy is for a whole file.
Ric Wheeler seemed to sum up the feeling of many when he said that "anything that works is better than years of nothing" for copy-offload support.
Petersen said that SCSI support for copy offload has advanced since last year, even though he had said it was done then. It now supports more features. There are some patches that add copy-offload support to the device mapper kcopyd (dm-kcopyd), though he "did not agree with the approach exactly". He has also added support for token-based copy offload, where device-generated tokens are used to identify the data of interest at the storage level. The block and SCSI support for copy offload has just been waiting for a user other than dm-kcopyd, he said.
![Martin Petersen [Martin Petersen]](https://static.lwn.net/images/2015/lsf-petersen-sm.jpg)
Brown noted that callers of copy_file_range() could perhaps get an error return if the underlying storage did not support copy offload. That way the caller could decide whether to fall back to a regular copy or not. A flag could be added to the call to do that fallback in the kernel, too.
The new system call would allow copying between files between two different mounted filesystems as long as both support copy offload, at least conceptually, but Christoph Hellwig thought that should be left for an add-on patch. All of the existing system calls will only work within a single mountpoint, he said, so making an exception needs to be considered carefully. Wheeler said that being able to do copies between mountpoints is a powerful feature, but Hellwig thought it should wait until someone actually needs that functionality and can provide a good implementation. It is never a problem to relax restrictions on system calls, Hellwig said.
The cross-filesystem copying feature is most important for network filesystems, Hellwig said. Wheeler disagreed, saying that it is also important for local filesystems. Hellwig said there needs to be a well-thought-out interface, so that users don't get locked into ioctl()-based mechanisms. Block-based filesystems could defer to the lower-level copy-offload support, he suggested. There is "more than one way to skin the cat; we just have to find a cat that we can skin", Dave Chinner said with a chuckle.
Step one should be to get the single-mountpoint system call implementation in, Hellwig said. Getting the block-layer support in should be step two. "Anything more fancy can follow". He also thought that token-based copies "make zero sense" from a user-interface perspective. That should be hidden in the lower levels. Finally, there should be an asynchronous interface with a notification when the operation completes.
The sense in the room was that copy-offload support is nearing inclusion after being discussed for several years at LSFMM. We will have to wait and see what gets into the mainline or whether copy offload will be on the agenda at next year's summit in Raleigh, North Carolina.
[I would like to thank the Linux Foundation for travel support to Boston for the summit.]
Patches and updates
Kernel trees
Architecture-specific
Core kernel code
Device drivers
Device driver infrastructure
Filesystems and block I/O
Memory management
Networking
Virtualization and containers
Miscellaneous
Page editor: Jonathan Corbet
Distributions
A look at Debian's 2015 DPL candidates
Debian's technical committee may have attracted quite a bit of attention in recent months, but most of the day-to-day governance tasks in the project are the purview of the Debian Project Leader (DPL). Debian elects a new DPL every year, and the 2015 election is right around the corner. This year, there are three candidates, each of whom has offered a different take on what their term as DPL would mean for Debian and for the world outside the project.
Lucas Nussbaum, who has served as DPL for the past two years, decided not to run again in 2015. Three candidates did step forward: Mehdi Dogguy, Gergely Nagy, and Neil McGovern. As is customary, each candidate has written a campaign platform detailing his background with Debian, his vision for the future of the project, and his goals for the coming year as DPL.
All three candidates have a lengthy history with Debian, having served on a number of the project's teams or in other significant roles. Both McGovern and Nagy ran for DPL in 2014, although Nagy withdrew his nomination before the voting began.
Platforms
Dogguy's platform
focuses on " To address this concern, Dogguy says he will conduct a review of
Debian's tools, mechanisms, and processes, so that he can identify
" In more concrete terms, he says that Debian should publish a public
roadmap that covers a time frame longer than the goals
established for individual releases, and that he as DPL will work to
make sure that progress is made. Dogguy also notes several major
changes undertaken by Debian (including the init-system change, the
Code of Conduct, and the transition away from 1024-bit PGP keys), but
says " In addition, Dogguy says that Debian should start a recruitment and
mentoring program designed to familiarize new contributors with
Debian's community, processes, and workflow. He also suggested that Debian should
devote some effort to making the distribution available through
non-traditional installation methods, such as cloud and
virtual-machine images.
Nagy's platform
makes a point of declining to state a " Doing so, he explains, means the DPL should remove barriers and
empower people to pursue their passion. This, he says, means putting
other project members first:
Nagy concludes by saying he wishes to be the DPL that no one
remembers. " McGovern's platform
also describes a commitment to " He also adds several specific efforts he would undertake as
DPL. They include deploying a personal package archive (PPA) system,
modernizing Debian's build system and related infrastructure, and
promoting the non-packaging portions of contributing to Debian.
In addition, he says he will continue the daily " Project members can ask all of the candidates questions via the
debian-vote mailing list. The 2015 edition of the question-and-answer
process elaborates on a few platform points and addresses some other
issues.
Dogguy elaborated on his roadmap idea in one response, saying that he does not see
it as a plan for the DPL to draw up, but as " McGovern, likewise, was asked to elaborate on the DPL's role in deploying
a PPA system. He responded that:
McGovern was also asked about his "spend
some money" statement. He replied
that he thinks Debian should spend money on booth paraphernalia,
hosting meetings, and " Additional questions about finance occupied much of the discussion
thread. Martin Krafft asked the candidates
whether they think Debian should offload accounting and finance work
to a hired professional. Dogguy said
no, while McGovern a relatively straightforward
yes and Nagy offered a more tepid
yes.
All of the candidates advocated the idea of Debian funding a small
number of Outreachy students (from two to four), although McGovern said that Debian should do fundraising
specifically to underwrite those students.
In regard to fundraising itself, McGovern later noted that the DPL can act as a project
fundraiser. Nagy suggested that
Debian should try to minimize the number of fundraising campaigns it
runs, due to their unpredictability, and should instead focus on
finding long-term sponsors.
The other major topic in the discussion so far was whether or not
Debian should relax its acceptance of non-free software.
Zacchiroli asked about section 5 of
the Debian Social
Contract (DSC), which provides for the contrib and non-free package archives,
supports their usage by users, and permits their access to the Debian bug
tracker and other infrastructure tools. Zacchiroli asked if the candidates thought the
time was " Nagy replied that dropping section
5 from the DSC but retaining contrib and non-free does not make sense:
Dogguy seemed generally in favor of retaining contrib and non-free,
noting that they increase user awareness of the concept of free
software:
McGovern responded that it was not
time to remove section 5, arguing that taking an " This doesn't mean we shouldn't strive to make §5 obsolete! Great work
has been done to try and remove non-free blobs from the kernel, for
example. I would love to run Debian on all systems without the need for
firmware on open hardware, but that day has not yet come. Until it
does, we should keep section 5.
The question-and-answer period for the candidates is still in full
swing. As of today, there are several interesting questions that are
still pending responses from some (or all) of the candidates.
Nussbaum, for instance, asked where
each candidate sees Debian fitting into the free-software ecosystem
five years from now. Anthony Towns asked the rather open question
" Voting itself will commence April 1, continue through April 14, and the
winner will be announced on April 15. The new DPL's term begins
on April 17.
the complexity of collaboration inside
Debian;" he notes that the project has been having difficulty
making:
non-trivial bottlenecks
", smooth communication between
teams, reduce the complexity of Debian's processes, and provide a
"
single coherent strategy
" everyone can share.
their implementation was a real pain
". Averting
such pain is another campaign plank; Dogguy says he will "be
present during preparation of important changes (be them technical,
social, financial or political) to ensure implementation details have
been studied.
"
grand vision
".
Rather, he says, the DPL's primary purpose is "to be an enabler:
the Project Leader is not a front runner to lead the herd to victory,
but a gentle shepherd to make them happy.
"
I'd rather see people remember all the great things the Project - as a whole - accomplished, for there are many.
"
support and enable
"
project members to do their own work by removing "blockers
".
DPL
log
" started by former DPL Stefano Zacchiroli, as well as
monthly email reports. Finally, he says he will "spend some
money we have horded, noting that Debian has several hundred
thousand in the bank, and that "
we should spend it to make the project more successful.
"
The questions
a process which
will enable us (DDs) to give some visibility to our individual
plans
". The roadmap would allow Debian developers to find
other teams and individuals with similar goals, he said, from which
shared priorities would emerge.
actively
recruiting people
".
ripe
" to drop that section from the DSC, either
removing contrib and non-free entirely, or simply refraining from
publicly sanctioning them.
ideologically
purist view
" does a disservice to users:
To the polls
where should the innovation come from?
" Users and fans
of Debian would be well advised to follow the discussion in earnest
over the coming week.
Distribution News
Debian GNU/Linux
Bits from the DPL -- January to mid-March 2015
Debian Project Leader Lucas Nussbaum looks at upcoming events and other Debian business. Topics include DebConf sponsorship deadline, Outreachy and GSOC deadlines, hosting offers for Debian development, Call for help: partners program, Paul Tagliamonte elected to the OSI board of directors, DPL election, delegations and appointments, and more.Bits from the dpkg project: 1.16.x series
The Debian dpkg maintainers present an update on dpkg development for the 1.16.x series, which is the version in Debian 7 "wheezy". These bits contain a broad summary of new features (mostly interface additions) and significant changes.
Fedora
Fedora seeks a diversity advisor
The Fedora project is looking for somebody to become its diversity advisor. "The Fedora Diversity Advisor will lead initiatives to assess and promote equality and inclusion within the Fedora contributor and user communities, and will develop project strategy on diversity issues. The Diversity Advisor will also be the point of contact for Fedora’s participation in third-party outreach programs and events." You have to get to the bottom of the announcement to read that this is a volunteer position, though they hope to change that someday.
Newsletters and articles of interest
Distribution newsletters
- DistroWatch Weekly, Issue 602 (March 23)
- 5 things in Fedora this week (March 25)
- Ubuntu Weekly Newsletter, Issue 409 (March 22)
Page editor: Rebecca Sobol
Development
An introduction to GlusterFS
Vijay Bellur, who is the co-maintainer of GlusterFS, gave a presentation at the first-ever Vault conference with an introduction to the filesystem and a look at where it is headed. GlusterFS is a distributed filesystem that will aggregate storage to provide a unified namespace for users' files. That data is then accessible via a wide variety of mechanisms.
Bellur began with a brief explanation of the need for GlusterFS (or simply Gluster). It comes down to the amount of data that is being generated these days—on the order of 2.5 exabytes (which is 2500 petabytes or 2.5 million terabytes) daily. In fact, 90% of the data ever generated by humans has been created in the last two years. All of that data must be stored somewhere and that storage should be commoditized and democratized, he said.
![Vijay Bellur [Vijay Bellur]](https://static.lwn.net/images/2015/vault-bellur-sm.jpg)
Gluster is a scale-out distributed storage system that collects up a variety of storage devices that are spread out across the network to present a global namespace for users. Gluster uses regular Linux filesystems that support extended attributes (e.g. ext4, XFS, Btrfs) to store the data. It provides file, object, and block interfaces to access the data.
All of Gluster is implemented as software that runs on commodity hardware, he said. It can run in virtual machines and may be able to be run in containers some day. Traditionally, distributed filesystems rely on metadata servers, but Gluster does away with those. Metadata servers are a single point of failure and can be a bottleneck for scaling. Instead, Gluster uses a hashing mechanism to find data.
Storage elasticity is another attribute of Gluster. It can scale out or scale down as needed. It is based on a modular architecture that is extensible. Most of it is implemented in user space, Bellur said.
Gluster concepts
A Gluster volume is a logical collection of exports from various storage servers, which are called "bricks". Volumes have an administrative name associated with them; users access a volume or part of a volume for their file operations (i.e. create, read, update, and delete, or CRUD).
There are several different types of volumes that are supported by Gluster. The first is a distributed volume that distributes files across the bricks in the volume. When the file is created, a hash is calculated from the file name; that determines which brick it will be placed on. Different clients will calculate the same hash value so they can find the right brick to access the file.
Another volume type is the replicated volume. As the name implies, it makes multiple copies of the file and stores those copies on separate bricks. The number of copies is set at volume-creation time.
A distributed replicated volume is the one used by most Gluster deployments, he said. In those volumes, multiple copies of a file are stored within a replicated volume and distributed across those replicated volumes. It provides high availability while also allowing the storage to grow as needed. More distributed volumes can simply be added to the filesystem as needed.
A new type of volume is the dispersed volume, which became available with Gluster 3.6. It provides RAID 5 over the network using erasure coding, which reduces the amount of storage needed for replication while still providing redundancy. It disperses the file's data across multiple bricks. The algorithm used is Reed-Solomon with a non-systematic erasure coding. All of the encoding and decoding is done on the client side.
Access
Gluster has multiple mechanisms available for clients to access the data stored in the filesystem. The first that was developed is the Filesystem in Userspace (FUSE) implementation that uses the GlusterFS protocol to access the data in the bricks. Much of the functionality in Gluster is client-based, including replication and erasure coding. The FUSE filesystem talks directly to the servers and has built-in failover, so an additional high-availability solution is not needed.
But FUSE is not available on all platforms and it is more mature on Linux than on other operating systems, so NFSv3 access was added. Gluster created its own NFS client in user space that talks NFS to the servers. In that model, distribution and replication are done by the servers.
A representational state transfer (REST) access method was also created, which allows access using web protocols. It uses the OpenStack Swift object storage API as its REST interface. Any combination of access methods can be used interchangeably; files could be created using FUSE, then accessed via REST, for example.
For those wanting to do data analysis using the data in a Gluster filesystem, there is a Hadoop Distributed File System (HDFS) support. Hadoop worker processes are run on the bricks and use FUSE to access the data on that server.
There is also a libgfapi that applications can use to bypass the other access methods and talk to Gluster directly. It is good for workloads that are sensitive to context switches or copies from and to kernel space. Integration with the NFS-Ganesha user-space NFS server is done using libgfapi. That allows using NFSv4 or Parallel NFS (pNFS) for Gluster file access. SMB is supported in a similar way. There is also experimental iSCSI support.
Features
Beyond being a scalable storage system that provides elasticity and quotas, it also provides data protection and recovery features. Volume and file-level snapshots are available and those snapshots can be requested directly by users, which means users won't have to bother administrators to create them. Archiving is supported with both read-only volumes and write once read many (WORM) volumes.
For multi-tenancy support, Gluster has encryption for data at rest and TLS/SSL for its data connections. For better performance, Gluster does caching of data, metadata, and directory entries for readdir(). There are built-in I/O statistics and a /proc-like interface for introspection of the filesystem state.
For provisioning servers with Gluster, there is puppet-gluster. It is also integrated with the oVirt virtualization manager as well as the Nagios monitor for servers. In fact, the sheer number of open-source projects that Gluster interfaces with is rather eye-opening.
Implementation
Gluster is implemented as a series of "translators", which are shared libraries that handle some piece of the functionality. Translators are self-contained units that can be stacked to enable multiple features. For example, distribution is a translator, as is replication; stacking the two of them provides the distributed replicated behavior for those types of volumes.
Translators can be deployed on the server, client, or both because they are "deployment agnostic". There are translators to handle protocols, performance features (e.g. caching, readahead), statistics gathering, access control, and so on. During development, swapping translators in and out of the stack can usually narrow down problems to a particular translator for further debugging.
A user survey in 2014 showed the main Gluster use cases. The two biggest are file synchronization/sharing and virtual machine image storage. After those two, backup and web content delivery network (CDN) uses were the next biggest, though other uses, especially for media files, also showed up in the survey.
Future
Gluster 3.5 was released in April 2014, followed by 3.6 in October 2014. The next release, 3.7, is currently in development and is planned for release in April 2015. The project is moving to a model with two major releases per year, Bellur said.
New features coming in 3.7 include "data tiering", which is a way to provide policies for moving data to and from hot and cold storage tiers based on access patterns. For example, the hot tier could consist of SSD storage while the cold tier is on spinning disks.
Bitrot detection is another feature bound for 3.7. The idea is to detect corruption while the data is at rest. A checksum is added to each object asynchronously and will be checked during periodic data scrubbing operations. Bitrot will also be detected when files are accessed.
A new sharding volume type is being added. Those volumes will split the data in files across multiple bricks. It will help reduce fragmentation in Gluster volumes as well as provide more parallelism for large-file workloads.
The netgroups feature that was developed at Facebook will appear in 3.7. It adds a more advanced configuration and authentication interface for NFS that is similar to /etc/exports. The patches were forward-ported from Gluster 3.4 for the upcoming release.
There are improvements to the support for NFS-Ganesha coming too, including high-availability support based on Pacemaker. Many performance improvements have been made, especially for small-file workloads. There is a TrashCan translator being added to protect from "fat finger" deletions and truncations. It also will capture deletions from system operations like self-healing (automatically resolving synchronization problems) and rebalancing (shuffling files around the bricks when new storage is added to the filesystem).
Another replication mode, arbiter replication, will keep two copies of the data and three copies of the metadata. The third metadata copy can be used to arbitrate in a "split-brain" scenario, where the two file copies get out of sync. In addition, administrative policies to resolve split-brains are coming in 3.7. The current behavior is to simply return an EIO for those files, but users will now be able to view the file versions and resolve the split-brain. There is a laundry list of other improvements coming in 3.7, including the inevitable "loads of bug fixes".
For releases beyond 3.7, the project is looking at a number of different features, including compression of data at rest and deduplication. A translator that provides overlay functionality is in the idea stage. REST interfaces for Gluster management are being planned, as is more integration with OpenStack and containers.
Gluster nodes that can also provide virtualization are on the horizon as well. This "hyperconvergence" is based on oVirt and KVM. There are also plans for a native Gluster driver for OpenStack Manila, which will provide "filesharing as a service" capabilities.
There is a long way to go before it gets there, but the project is already thinking about Gluster 4.0, Bellur said. The key things that will be addressed in that release are features meant to make the filesystem able to scale to larger systems. Currently there are limitations in the management framework that stop Gluster filesystems from growing beyond a certain size. Supporting a thousand nodes or more is part of those plans.
Beyond those features, the project would like to support heterogeneous environments better. Environments with multiple operating systems, many different types of storage, and multiple networks are being targeted. There are also plans to increase the flexibility that deployments have in choosing replication options, erasure codes, and more. There is a new style of replication being looked at, too, which is completely handled by the servers without clients being involved at all.
The feature set for Gluster 4.0 is still up in the air, though implementation of a few key features has already started. New feature ideas can still be submitted and there are plans to vote on which features will be included as part of a Gluster design summit that is tentatively planned for May 2016.
In answer to a question from the audience, Bellur gave a comparison between Gluster and the Ceph distributed filesystem. The architecture of Ceph is quite different than that of Gluster, since Ceph started as an object store and built file storage on top of that, while Gluster did the reverse. Thus file access is more flexible from Gluster, while object or block access is more flexible from Ceph. Gluster may be a better choice for systems that will start relatively small and possibly grow from there, while Ceph may be a good choice when the system is known to need to be huge from the outset.
It would seem that the overarching advantage that Gluster provides is its flexibility in terms of volume types, access methods, and integration with various other tools. It certainly appears to be an active project with lots of interesting plans for the future.
[I would like to thank the Linux Foundation for travel support to Boston for Vault.]
Brief items
Quotes of the week
But the people who don’t see a personal value in free software are missing a larger, more important freedom. One implied by the first four, though not specifically stated. A fifth freedom if you will, which I define as:
- Freedom 4: The freedom to have the program improved by a person or persons of your choosing, and make that improvement available back to you and to the public.
Most of the days however, I tear my hair when fixing bugs, or I try to rephrase my emails to not sound old and bitter (even though I can very well be that) when I once again try to explain things to users who can be extremely unfriendly and whining. I spend late evenings on curl when my wife and kids are asleep. I escape my family and rob them of my company to improve curl even on weekends and vacations. Alone in the dark (mostly) with my text editor and debugger.
There’s no glory and there’s no eternal bright light shining down on me. I have not climbed up onto a level where I have a special status. I’m still the same old me, hacking away on code for the project I like and that I want to be as good as possible.
Firefox 36.0.4
Firefox 36.0.4 has been released. This update includes security and bug fixes, support for the full HTTP/2 protocol, and more. The release notes contain the details.Newscoop 4.4 released
Version 4.4 of the news-site content-management system Newscoop has been released. Updates in this version include a framework for attaching editorial notes to in-process articles, support for "featured article" lists, and a refactored topic-management interface.
GTK+ 3.16.0 released
GTK+ version 3.16 is now available. Major changes include GDK support for rendering windows with OpenGL, a completely overhauled implementation of scrolling (including support for overlayed scrollbars), and an experimental Mir backend. New widgets include GtkGLArea, GtkStackSidebar, GtkModelButton, and GtkPopoverMenu, all of which seem to implement more or less what their names indicate. Interested developers would still be advised to read the documentation, however.
GNOME 3.16 released
The GNOME 3.16 release is out. "This is another exciting release for GNOME, and brings many new features and improvements, including redesigned notifications, a new shell theme, new scrollbars, and a refresh for the file manager. 3.16 also includes improvements to the Image Viewer, Music, Photos and Videos. We are also including three new preview apps for the first time: Books, Calendar and Characters." See the release notes for more information.
LibreOffice Online announced
The LibreOffice project has announced the accelerated development of a new online offering. "Development of LibreOffice Online started back in 2011, with the availability of a proof of concept of the client front end, based on HTML5 technology. That proof of concept will be developed into a state of the art cloud application, which will become the free alternative to proprietary solutions such as Google Docs and Office 365, and the first to natively support the Open Document Format (ODF) standard." The current effort is supported by IceWarp and Collabora; see this FAQ and Michael Meeks's posting for more information. For those wanting to download it, though, note the "
the availability of LibreOffice Online will be communicated at a later stage."
Newsletters and articles
Development newsletters from the past week
- What's cooking in git.git (March 20)
- What's cooking in git.git (March 23)
- LLVM Weekly (March 23)
- OCaml Weekly News (March 24)
- OpenStack Community Weekly Newsletter (March 20)
- Perl Weekly (March 23)
- PostgreSQL Weekly News (March 22)
- Python Weekly (March 19)
- Ruby Weekly (March 19)
- This Week in Rust (March 24)
- Tor Weekly News (March 25)
- Wikimedia Tech News (March 23)
Snellman: On open sourcing existing code
Juho Snellman has an interesting treatise on the oft-overlooked challenges that face developers attempting to release an existing, proprietary codebase under open-source terms. "As soon as you get outside of the "one self-contained file or directory" level of complexity, the threshold for releasing code becomes much higher. And likewise every change to a program that was made in order to open source it will make it less likely that the two versions can really be kept in sync in the long term. In this case the core code is maybe 2k-3k lines and won't require much work. It's all the support infrastructure that's going to be an issue.
" Snellman also reflects on possible strategies for writing internal code that may some day be released to the public.
Windows 10 to make the Secure Boot alt-OS lock out a reality (Ars Technica)
Ars Technica is one of several news outlets to report on a change announced in Microsoft's Windows 10 plans. Though the headlines (including Ars Technica's) paint a rather bleak scenario, the details are not as clear-cut. The UEFI "Secure Boot" mechanism was introduced with Windows 8, at which time Microsoft's OEM-certification rules mandated that hardware must include a means for the local user to disable Secure Boot. The Windows 10 certification rules does not include the mandated disable switch. Writes Peter Bright: "Should this stand, we can envisage OEMs building machines that will offer no easy way to boot self-built operating systems, or indeed, any operating system that doesn't have appropriate digital signatures. This doesn't cut out Linux entirely—there have been some collaborations to provide Linux boot software with the 'right' set of signatures, and these should continue to work—but it will make it a lot less easy.
" Note, also, that the only source for this story appears to be a presentation from a Microsoft event in Shenzhen, China. Bright adds that he has contacted Microsoft seeking clarification, but has so far received no reply.
Page editor: Nathan Willis
Announcements
Brief items
A Turing award for Michael Stonebraker
The ACM has announced that the 2014 A. M. Turing award has gone to Michael Stonebraker. Among many other things, he was the original creator of the database management system now known as PostgreSQL.Sébastien Jodogne, ReGlue are Free Software Award winners
Free Software Foundation executive director John Sullivan has announced the winners of the FSF's annual Free Software Awards. The Award for the Advancement of Free Software went to Sébastien Jodogne for his work on free software medical imaging. The Award for Projects of Social Benefit was given to Reglue, which gives GNU/Linux computers to underprivileged children and their families in Austin, TX.Kat Walsh joins FSF board of directors
The Free Software Foundation has announced the addition of Kat Walsh to its board of directors. "A lawyer with extensive background in the free culture movement, Walsh brings a wealth of experience with law and licensing to the FSF board. In particular, her skills will help support and oversee the FSF's licensing work on the GNU General Public License (GPL) as well as the LGPL and GFDL. Kat worked as a staff lawyer at Creative Commons, where she was on the team that drafted the last major revision to the family of Creative Commons licenses, completed in November 2013 with the release of the 4.0 licenses." (Thanks to Jim Garrison)
Articles of interest
Meet Cyanogen, The Startup That Wants To Steal Android From Google (Forbes)
Forbes takes a look at Cyanogen, and its prospects in the phone market. "Cyanogen has a chance to snag as many as 1 billion handsets, more than the total number of iPhones sold to date, according to some analysts. Fifty million people already run Cyanogen on their phones, the company says. Most went through the hours-long process of erasing an Android phone and rebooting it with Cyanogen. [Kirt] McMaster is now persuading a growing list of phone manufacturers to make devices with Cyanogen built in, rather than Google’s Android. Their phones are selling out in record time. Analysts say each phone could bring Cyanogen a minimum of $10 in revenue and perhaps much more."
LibrePlanet 2015: Highlights and what comes next
The Free Software Foundation looks back on LibrePlanet 2015: Free Software Everywhere, which happened March 21-22. "With approximately 350 people in attendance, we kept pace with last year—with a few improvements. In particular, our tech team did a fantastic job improving our video feeds, with peak usage at around 300 simultaneous views. We also worked hard to streamline the registration process, reducing time spent waiting in the registration line before Richard Stallman's Saturday morning keynote, with few hiccups." Videos will be available at the LibrePlanet video archive.
FSFE: Worldwide more than 50 events about Open Standards
The Free Software Foundation Europe has a reminder that Document Freedom Day is happening from March 24 12:00 UTC until March 26 12:00 UTC. "Document Freedom Day is the global campaign for document liberation by local groups throughout the world. So far more than 50 groups registered their events in over 25 countries ranging from Asia, Europa, Africa, to South and North America."
New Books
New Release by Rocky Nook – Advanced Software Testing, Volume 3, 2nd Edition
Rocky Nook has released "Advanced Software Testing, Volume 3, 2nd Edition" by Rex Black and Jamie Mitchell.
Calls for Presentations
Call for submissions: Libre Graphics magazine 2.4
Libre Graphics magazine is a print publication devoted to showcasing and promoting work created with Free/Open Source Software. The magazine is seeking submissions for it's 2.4 issue on capture. "We’re looking for work, both visual and textual, exploring the concept of capture, as it relates to or is done with F/LOSS art and design. All kinds of capture, metaphorical or literal, are welcome. Whether it’s a treatise on the politics of photo capture in public places, a series of photos taken using novel F/LOSS methods, documentation of a homebrew 3D scanner, any riff on the idea of capture is invited. We encourage submissions for articles, showcases, interviews and anything else you might suggest." The submission deadline is May 11.
CFP Deadlines: March 26, 2015 to May 25, 2015
The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.
Deadline | Event Dates | Event | Location |
---|---|---|---|
March 31 | July 25 July 31 |
Akademy 2015 | A Coruña, Spain |
March 31 | May 4 May 5 |
CoreOS Fest | San Francisco, CA, USA |
April 3 | May 2 May 3 |
Kolab Summit 2015 | The Hague, Netherlands |
April 4 | May 30 May 31 |
Linuxwochen Linz 2015 | Linz, Austria |
April 6 | May 20 May 22 |
SciPy Latin America 2015 | Posadas, Misiones, Argentina |
April 14 | April 14 April 15 |
Palmetto Open Source Software Conference | Columbia, SC, USA |
April 15 | June 12 June 14 |
Southeast Linux Fest | Charlotte, NC, USA |
April 17 | June 11 June 12 |
infoShare 2015 | Gdańsk, Poland |
April 28 | July 20 July 26 |
EuroPython 2015 | Bilbao, Spain |
April 30 | August 7 August 9 |
GNU Tools Cauldron 2015 | Prague, Czech Republic |
May 1 | August 17 August 19 |
LinuxCon North America | Seattle, WA, USA |
May 1 | September 10 September 13 |
International Conference on Open Source Software Computing 2015 | Amman, Jordan |
May 1 | August 19 August 21 |
KVM Forum 2015 | Seattle, WA, USA |
May 1 | August 19 August 21 |
Linux Plumbers Conference | Seattle, WA, USA |
May 2 | August 12 August 15 |
Flock | Rochester, New York, USA |
May 3 | August 7 August 9 |
GUADEC | Gothenburg, Sweden |
May 3 | May 23 May 24 |
Debian/Ubuntu Community Conference Italia - 2015 | Milan, Italy |
May 8 | July 31 August 4 |
PyCon Australia 2015 | Brisbane, Australia |
May 15 | September 28 September 30 |
OpenMP Conference | Aachen, Germany |
May 17 | September 16 September 18 |
PostgresOpen 2015 | Dallas, TX, USA |
May 17 | August 13 August 17 |
Chaos Communication Camp 2015 | Mildenberg (Berlin), Germany |
May 23 | August 22 August 23 |
Free and Open Source Software Conference | Sankt Augustin, Germany |
May 23 | May 23 May 25 |
Wikimedia/MediaWiki European Hackathon | Lyon, France |
If the CFP deadline for your event does not appear here, please tell us about it.
Upcoming Events
Two microconferences accepted for the Linux Plumbers Conference
The 2015 Linux Plumbers Conference (LPC) has announced that two microconferences have been accepted for the event, which will be held August 19-21 in Seattle. The Checkpoint/Restart and Energy-aware scheduling and CPU power management microconferences will be held at LPC. Registration for the conference will open on March 27 and it will be co-located with LinuxCon North America, which will be held August 17-19.openSUSE Conference and Kolab Summit
openSUSE Conference 2015 will be held with Kolab Summit 2015 May 1-4 in The Hague, Netherlands. Registration is open and the call for papers ends March 31. "The conferences will bring together a wide variety of Free & Open Source contributors to collaborate on one of the major Linux distribution and FOSS projects, as well as showcasing the possibilities of free open source software to enterprise."
Events: March 26, 2015 to May 25, 2015
The following event listing is taken from the LWN.net Calendar.
Date(s) | Event | Location |
---|---|---|
March 24 March 26 |
FLOSSUK DevOps Conference | York, UK |
March 25 March 27 |
PGConf US 2015 | New York City, NY, USA |
March 26 | Enlightenment Developers Day North America | Mountain View, CA, USA |
March 28 March 29 |
Journées du Logiciel Libre | Lyon, France |
April 9 April 12 |
Linux Audio Conference | Mainz, Germany |
April 10 April 12 |
PyCon North America 2015 | Montreal, Canada |
April 11 April 12 |
Lyon mini-DebConf 2015 | Lyon, France |
April 13 April 17 |
SEA Conference | Boulder, CO, USA |
April 13 April 17 |
ApacheCon North America | Austin, TX, USA |
April 13 April 14 |
AdaCamp Montreal | Montreal, Quebec, Canada |
April 13 April 14 |
2015 European LLVM Conference | London, UK |
April 14 April 15 |
Palmetto Open Source Software Conference | Columbia, SC, USA |
April 16 April 17 |
Global Conference on Cyberspace | The Hague, Netherlands |
April 17 April 19 |
Dni Wolnego Oprogramowania / The Open Source Days | Bielsko-Biała, Poland |
April 21 | pgDay Paris | Paris, France |
April 21 April 23 |
Open Source Data Center Conference | Berlin, Germany |
April 23 | Open Source Day | Warsaw, Poland |
April 24 | Puppet Camp Berlin 2015 | Berlin, Germany |
April 24 April 25 |
Grazer Linuxtage | Graz, Austria |
April 25 April 26 |
LinuxFest Northwest | Bellingham, WA, USA |
April 29 May 2 |
Libre Graphics Meeting 2015 | Toronto, Canada |
May 1 May 4 |
openSUSE Conference | The Hague, Netherlands |
May 2 May 3 |
Kolab Summit 2015 | The Hague, Netherlands |
May 4 May 5 |
CoreOS Fest | San Francisco, CA, USA |
May 6 May 8 |
German Perl Workshop 2015 | Dresden, Germany |
May 7 May 9 |
Linuxwochen Wien 2015 | Wien, Austria |
May 8 May 10 |
Open Source Developers' Conference Nordic | Oslo, Norway |
May 12 May 13 |
PyCon Sweden 2015 | Stockholm, Sweden |
May 12 May 14 |
Protocols Plugfest Europe 2015 | Zaragoza, Spain |
May 13 May 15 |
GeeCON 2015 | Cracow, Poland |
May 14 May 15 |
SREcon15 Europe | Dublin, Ireland |
May 16 May 17 |
11th Intl. Conf. on Open Source Systems | Florence, Italy |
May 16 May 17 |
MiniDebConf Bucharest 2015 | Bucharest, Romania |
May 18 May 22 |
OpenStack Summit | Vancouver, BC, Canada |
May 18 May 20 |
Croatian Linux User Conference | Zagreb, Croatia |
May 19 May 21 |
SAMBA eXPerience 2015 | Goettingen, Germany |
May 20 May 22 |
SciPy Latin America 2015 | Posadas, Misiones, Argentina |
May 21 May 22 |
ScilabTEC 2015 | Paris, France |
May 23 May 24 |
Debian/Ubuntu Community Conference Italia - 2015 | Milan, Italy |
May 23 May 25 |
Wikimedia/MediaWiki European Hackathon | Lyon, France |
If your event does not appear here, please tell us about it.
Page editor: Rebecca Sobol