LWN.net Weekly Edition for March 10, 2016
The first public release of OpenShot 2.0
The OpenShot video-editor project recently released builds from the long-awaited 2.0 series. Although they are still tagged as "betas," they offer a glimpse at what the development team has been up to since its previous stable update in late 2012. Many fans of OpenShot have learned to be patient, but the 2.0 is worth a look even for those with no prior experience.
The first beta release was announced on January 16, with binary builds provided only to the project's Kickstarter supporters (source code, of course, was available in the repositories). In February, the third beta (version 2.0.6) was made available to the public. Linux, Windows, and Mac OS X binaries are available for download, as are source code bundles.
The previous stable release was version 1.4.3 from October 2012. At that time, plans for 2.0 were already in discussion, and there were some overly optimistic estimates for when it might be released. The lengthy delay was the result of lead developer Jonathan Thomas's decision at that time to stop and rewrite nearly the entire application. OpenShot 2.0 is built on top of a new video-editing library (libopenshot) developed in-house to replace the Media Lovin' Toolkit (MLT) library used in the 1.x releases. In addition, 2.0 drops GTK+ for a completely rewritten PyQt5 user interface, and adds support for both Mac and Windows machines.
Considering the scope of the work involved, the repeatedly pushed-back release date for the 2.0 series is not surprising. In March 2013, Thomas ran a successful Kickstarter campaign to fund his own development time (predicting a December 2013 release date). Despite the slipping schedule, though, he has regularly posted progress updates about libopenshot and the application—no doubt contributing to the continued goodwill that the user community still expresses toward the project in forum posts and comments on the blog.
With the new beta releases, that community can finally put the new codebase to the test—and, because the 2.0 builds include all of the editing and export features found in 1.x (plus a few additions), users can try out the new release on projects similar in scope to those handled by the old OpenShot.
Users coming from other video editors will find the basic interface familiar enough to get started right away with little need for introduction. The main window provides an editing timeline where users can add video and audio tracks, a video player for previewing edits, and a tabbed browser that provides quick access to imported video clips, effects, and transitions. Clips are added to the timeline with drag-and-drop, and most of the additional tools for manipulating a clip are found in the right-click context menu. This includes fade-in and fade-out settings, audio volume, rotation, and even the "split" tool used to cut a clip into segments.
In fact, so many tools and settings have been moved into the context menu that users might be alarmed when they cannot find buttons or menu items for the features they need. In previous releases, for example, several of these settings were found in the "Properties" dialog for each clip. Now, the "Properties" entry in the context menu opens up a sidebar next to the panel that shows details like the start and end times, but leaves out most of those user-configurable properties of a clip now distributed among the context-menu entries. Such rearrangements can take some getting used to.
How transitions function is another noticeable change. In the past, OpenShot's UI for adding a transition between two clips was rather distinctive. The first clip and the second clip had to be on separate tracks in the timeline, and the "transition" element was a box that straddled the two tracks and had an arrow indicating the direction of the change. It was not hard to figure out, but it certainly violated the established rules of "how timelines work in a video editor" obeyed by, well, every other video-editing application.
In the new release, the user drags the two clips in question onto the same track so that they overlap. The transition element then gets planted on top of both. It is hard to say whether or not this is an improvement; nothing seems to be floating in the middle (as was the case in the old UI), but when the clips and transitions are stacked on top of each other, they can be hard to see. Since video effects (masks, blurs, brightness adjustments, and so on) are also stackable elements, one can easily end up with a half dozen translucent rectangles and text labels sitting right on top of each other on the timeline.
The other major UI component is the animated title editor, which pops up in a window of its own, and is rather straightforward to use. OpenShot 2.0 comes with 24 animated title templates, including everything from simple "words slide in from the side of the screen" effects to complicated 3D elements like a Star-Wars–style opening crawl and map flyover similar to those seen in Indiana Jones. Each template has its own battery of settings; the user adjusts them as desired (using a preview window to double-check the output), then OpenShot renders the template into a video clip to add to the timeline.
Apart from the basics, the release notes highlight several new UI features in the 2.0 series. One is a "multiple clip add" tool; for users importing a large collection of clips into a project, this tool can be used to automatically add them all to the timeline at once, with an optional gap added in between. Similarly, there is also a "Split Clip" tool with which the user can mark multiple sections within a longer clip and instantly extract each section into a clip of its own. This is handy for quickly cutting a long recording down into the just the interesting parts.
The new release also sports a larger selection of transitions (well
over 300), and it even allows users to create and load transitions of
their own. There is also a sizable assortment of newly added static (i.e.,
non-animated) title templates. Finally, the new release adds an
auto-save feature and automated backup creation, which anyone who has
invested serious time in a video project will probably be thankful for
at some point.
For the average user, the OpenShot 2.0 builds may appear to be little more than a release on par with contemporary open-source video editors like Pitivi or Kdenlive. But it is clear that Thomas and the other developers have put in a substantial amount of work on the non-user-visible aspects of the new release, too. There are fewer application crashes now, since the front-end catches most exceptions. Libopenshot is making better use of multi-core machines via OpenMP, and offering smoother animations and audio processing. The library is also better at coping with odd or corrupted video files (where, in the past, it might crash), and the project file structure has been reworked to be completely portable across operating systems.
The OpenShot 2.0 rewrite was a massive undertaking, so the development team deserves a lot of credit for getting it done and providing users with a stable end product. The bigger question will be whether or not the reworked interface and brand-new editing library allow the project to push forward. After all, fans of video editors on Linux desktops have seen rewrites and forks announced many times in the past. Some, like Cinelerra and Lumiera, never seem to actually reach the light of day. Others, like Pitivi, are more successful, but progress on new features always seems to be slow. In many respects, that is due to the inherent complexity of video editing. Hopefully, then, the OpenShot 2.0 series and new libopenshot library will prove not only to be useful for users, but to be easier for the developers to work with as time goes by.
Outreachy: an intern's perspective
Last year, I had the privilege of participating as an intern in the Outreachy program. In gratitude, I am sharing my experiences and those of my fellow Outreachy participants (the worms' eye view), as well as a bird's eye view provided by Karen Sandler, one of Outreachy's prime motivators, who generously gave her time to answer my questions.
Background
Outreachy began, under the auspices of GNOME, as a program to promote women in technology. It arranged paid internships for women, with mentors provided, so that they might gain experience in the open-source world. Today, its primary goal is to increase participation of under-represented populations in the open-source community. Last year, it expanded to include transgendered individuals and, this year, to "Black/African American, Hispanic/Latin@, American Indian, Alaska Native, Native Hawaiian, and Pacific Islander" peoples in the US.
Since the statistics for participation by women in the open-source world are abysmal (1-11% of all participants), and are similar for these other groups, Outreachy provides a needed service. The reason the statistics are variable is due to different studies and to the methodologies used: whether only developers are counted or if documentation and other contributors are also included, for example.
For those unfamiliar with the Outreachy application process, it is not simply a matter of filling out a form. Rather, Outreachy publishes a list of projects, a brief description of each project, the project contacts, the mentors, and a desired set of skills. It is the participant's responsibility to research those that look interesting, to ensure that they have the requisite skill set, and to communicate with the mentors to determine if the project will meet the participant's goals.
Once a mentor and a participant have been matched, the two choose a small project that the participant can work on. This small project must be completed and accepted into the open-source software project before the Outreachy application is complete. This year, the application period runs until March 22, so it is definitely not too soon to begin the process, nor is it too late for mentors to volunteer. Outreachy recently sent out a plea for more mentors.
To try to get a sense for how well the program is working, I emailed the 244 Outreachy alumni to ask how many were working in the field. I received over 70 responses: most are working as developers, about 2/3 in open-source, 1/3 in the proprietary world, with a smattering in mixed environments. A few work as technical writers, one is a project manager, a few have other internships or are still students, and one is pursuing a PhD in physics. Two wrote that their companies are actively looking for women and other minorities to hire, and encouraged other past participants to apply.
Bird's eye view
Sandler has helped nurture this program since its early days. She acknowledged that she measures success in absolute numbers (41 Outreachy alumni have given presentations at conferences; fifteen have subsequently participated in Google's Summer of Code), but most of her comments focused on less-tangible impacts. She has surveyed the entire open-source world and is seeing greater numbers of contributions made by women, she said. She also is heartened to note that more women are contributing to blogs, which is a sign that women are more willing to be visible. She said that one of the advantages of this increased participation is that, as time goes on, a "critical mass" can be achieved, where members of under-represented populations will feel more comfortable.
Any member of a minority will acknowledge that when you are the only one of a group, there is a major difficulty in being visible. Any flaws or mistakes tend to be magnified, are remembered longer, and are unfairly applied to the group as a whole. Even though members of the majority may make the same mistakes, newly visible minority members often take more than their share of such criticism.
Sandler also noted that some former participants have become mentors themselves. While the numbers are still small, these "grandmentors" demonstrate that not only are some Outreachy participants getting jobs in the open-source world, they have progressed to a point where they can lend a hand to those coming behind them. Over time, this will help "change the tenor of the community", she said.
The grandmentors point to another benefit accruing to Outreachy participants: the community of alumni that is forming as a result of past participation. I regularly receive emails notifying me of other internship opportunities, as well as additional resources. To compile impressions from past participants for this article, I sent a separate email to the group. The response was heartening; sixteen people took the time to share their goals, dreams, successes, and failures.
Participants speak
The most commonly expressed goal was to become involved with open-source software. Most of the participants were still in college or had recently graduated. However, several were undergoing major career transitions; one was transitioning from the proprietary to the open-source world. Other stated goals were for exposure to a diversity of groups and people in the field and to the international nature of open source. Several noted a desire to work on something bigger than themselves, to contribute, or to gain experience, and several mentioned that the money was appreciated. One woman was working on an OpenStack project and realized that she could get paid for what she was already doing.
All the women expressed high levels of satisfaction with their experiences, although many asked to remain anonymous, so I am extending that courtesy to all of them. The most commonly mentioned successes were the amount learned, the challenges, the opportunities for collaboration, networking opportunities, the increase in confidence and of "hireability," and the impact of their work ("affecting millions," one woman wrote). All were enthusiastic about their mentors who were described as kind, patient, helpful, and supportive.
One particularly eloquent participant described how she had gone through a career transition, had taken a few on-line classes offered by major universities ("Massive Open Online Courses" or MOOCs) on JavaScript, and used her internship to gain experience in what she'd learned. She was delighted that, unlike most internships, "I didn't have to be actively enrolled and pursuing a degree to be eligible." She now credits this program with her current employment as a JavaScript developer. "I advocate at my company to put effort into recruiting in a way that encourages women to apply; to establish work policies that attract women; etc.," she wrote. This attitude is what will achieve the critical mass that Sandler spoke of.
Two women expressed some disappointment that there was not enough coding; a third, who was not coding, was delighted that she was doing usability testing. This speaks to the importance of matching one's needs and goals to the projects available, which should happen during the application process.
Some of the projects that participants worked on included writing GNOME application user help documentation, writing small test-programs for FFmpeg's API, testing new applications, and porting a program from C to C++. One participant was doing usability testing for GNOME; she was excited, because usability was the aspect of computer science that she was studying. Others wrote drivers, made sophisticated (though fairly small) kernel modifications, wrote semantic patches for Coccinelle, performed statistical analysis, worked on aspects of Air Mozilla (the multimedia arm of Mozilla), and so on.
Some participants shared links to their projects. For example, Frances Hocutt wrote the description of the "gold standard" for MediaWiki API client libraries. Lisa Fresh provided two links, one to the presentation she made at the end of her internship with Mozilla and another to the code she worked on.
My Outreachy experience
In my case, I only learned about Outreachy two days before the application deadline was closed. I had a vague notion of becoming involved in Linux kernel development. I wrote this to Marina Zhurakhinskaya, and my project found me—in the person of Lars Kurth of the Xen project. My mentors were Wei Liu and Julien Grall, who were unfailingly kind and patient with me. The application project was to provide functions to compute logical operations on two bitmaps.
My Outreachy project was to take the 9P protocol and to create Xen front- and back-ends to implement it by building on the work that had already been done with 9P in virtio. This was a difficult project, made more so by my own unique circumstances. Despite thirty years of experience, only three of them were in the Unix world and those were back in the early 1980s. I had not worked in a number of years, and during my hiatus, had suffered a minor head injury that affected my ability to read and learn. My personal goals were to determine at what level I was functioning as a contributor, where I was going to have problems in any job, and how to compensate for any deficits I might encounter.
A month into the project, my mentors asked me if the project was too hard. At that point the answer was, "no." Two weeks earlier, I would have had to say, "yes." My mentors supported me as I struggled to set up my environment, learn the tools, and come up to speed on Xen. For learning the virtio implementation, I was pretty much on my own. Learning all of that took longer than I would have liked. However, when it came to modifying virtio code, and writing Xen-specific code, I found myself discovering a joy in, and a facility for, software development that I'd forgotten I had.
My project was described variously as a proof of concept and as a prototype by my mentors. As the first, it was a definite success, because I demonstrated how this could be done and that it could work. This involved showing how few virtio files needed to be modified (they were very modular) and getting a subset of the project working. Since I never got the full finished project working, though, it was something of a failure as a prototype.
The support experienced in Outreachy made participants feel valued. Thus it was a sad postscript to my internship when someone offered to hire me to build on my work. However, since he lacked the funds to pay me, he felt I should get my work up to production quality for a proprietary product, for free, as I was "supposed" to last summer. Then he "might" hire me for add-on work. I politely refused, but I wonder how many young women would have accepted those terms. It undercut the hard work that Outreachy and its representatives put into making participants feel valued and worthwhile. This reflects a larger problem in the open-source world, where at least one developer expressed concern that he was being used, and his work misused.
That experience notwithstanding, Outreachy has been greatly appreciated by its participants, me included. I encourage all eligible people to look into it, and hope more mentors volunteer. In addition to the satisfaction of helping increase diversity, the experience of mentoring will stretch you professionally and personally. Thanks are due to all involved.
Security
CVE woes lead some to seek alternatives
A recent discussion of problems with the Common Vulnerabilities and Exposures (CVE) system for security vulnerabilities—coupled with some efforts to supplement or supplant it—has led some to wonder about its future. The problems stem from the difficulty—sometimes impossibility—of getting CVEs assigned for real vulnerabilities. That, in turn, has led some to stop even requesting CVE numbers for vulnerabilities that they find, which further reduces their usefulness.
The system of assigning CVE IDs has been around since 1999—run by the MITRE Corporation. It was set up to provide a way for researchers, users, and others to track specific vulnerabilities, but has been seen as declining in quality over the years. For example, there are many CVE entries that have been assigned but simply show "RESERVED" with no details, which makes those entries less than entirely helpful.
The IDs do provide a way to refer to specific vulnerabilities, but that only works if the CVEs get assigned properly. As Kurt Seifried, who is part of the Red Hat security response team, put it on the oss-security mailing list, there is reason to worry about that:
Beyond that, though, he has heard "about people that may have given up asking for
CVEs and publicizing their work at all
". He also pointed to a
message from Hanno Böck showing his inability to get a CVE assigned for a
flaw
in the implementation of a cipher used in TLS 1.2.
The request was denied because it was "outside the scope of CVE's
published priorities
".
Seifried concluded
his message by noting that this new CVE
policy that was cited in the message to Böck has two levels of coverage
for the priority of CVE assignments, which is
something he
called "tiered coverage
". That policy may lead in undesirable
directions, he said:
Others who posted in the thread largely agreed with Seifried's concerns. Others echoed Böck's experience and noted that they had given up even trying to get CVEs for the vulnerabilities that they find. The process to request a CVE ID is in place, but it isn't smoothly functioning these days. Adam Caudill summarized some of the problems:
Caudill (and others) called for a new system of some sort. A participant known as "Tim" posted some detailed thoughts on what a new system might look like. As it turns out, though, Alexander Peslyak (better known as "Solar Designer") had created a bare-bones system to assign OVE IDs (OVE presumably stands for "Openwall Vulnerability Entries"). This assignment mechanism does no verification and simply assigns an ID; it is meant to be a friction-free mechanism to get an ID without needing vetting of any kind.
While the benefit of being able to get IDs quickly and easily was of interest, several in the thread were looking for more. Features such as allowing entries to be updated with more information or be curated in some fashion were mentioned. Solar Designer said that he didn't want to add more features, but encouraged others to pursue that path. As part of that discussion, Tim noted a major problem that he has seen: link rot for advisories and other detailed information. As he put it:
That led "halfdog" to propose a blockchain-based database to contain the vulnerability information, with "proof of work" providing a means to allocate the IDs. In the proposal for a "Distributed Cryptoenhanced Vulnerability Enumeration" (DCVE), there are provisions for storing some information with each entry, though it would not seem to completely eliminate the problem that Tim described.
There was also a pointer in the thread to the Open Vulnerability ID (OVI) system, which is another ID-assigning mechanism.
It's not clear how much traction any of those systems mentioned would actually get, at least partly because they are entirely separate from the CVE ID namespace. Another project, which was announced by Seifried on the list, is the Distributed Weakness Filing (DWF) system. It directly incorporates CVE IDs into its database, so that DWF-2016-NNNN is the same as CVE-2016-NNNN. Other vulnerabilities that did not have CVEs could use numbers outside of the CVE numeric range (which is now arbitrarily long, but has never gone beyond four digits—yet). The DWF project, Seifried noted, is not part of his Red Hat duties; it is a side project that he and others are working on.
DWF provides a number of different ways to get an ID, starting with just
trying to get
a CVE number as usual, through requesting one from a DWF Number
Authority (DNA), to changing the project's database file and making a pull request on GitHub—or by simply emailing the project. As he said in the announcement:
"I want to reduce the time and effort needed to
get identifiers, something best achieved by pushing assigning out to as
close to the vulnerability discover/handling as possible
".
So far, there has been little public reaction to the DWF announcement, but it certainly seems to be the furthest along in terms a new system. It complements the existing CVE system, while also setting up something that could replace it someday if the CVE system continues down its current path.
It seems clear that the CVE system is breaking down in various ways. The tiered coverage approach that MITRE has chosen does not look likely to help it back onto a path where it will be the definitive resource for vulnerability IDs. It's also unclear whether any of the proposed alternatives will truly take off, but the free-software community would seem to be a likely candidate to start heading toward some alternative. If independent security researchers make the same choice, we could be seeing the beginning of the end for CVEs—though it is hard to imagine that system fading away completely.
New vectors for fingerprinting Tor Browser
Browser fingerprinting enables a web site to differentiate between anonymous visitors, capturing enough minor differences between visiting browsers to uniquely associate each browser with a specific profile. The technique lets sites track individual users across multiple visits and IP addresses, which is problematic enough for any user who is concerned about privacy. But fingerprinting that can identify users coming through the Tor anonymizing network is even more worrisome. Recently, research uncovered several new fingerprinting vectors that work against Tor Browser—and, by extension, most other web browsers. How significant the vectors are in practical terms, however, remains to be seen.
Jose Carlos Norte posted his findings at his blog on March 6. In the post, Norte describes five experiments he has run attempting to collect entropy (in the information-theory sense) data about visiting web browsers, which helps uniquely identify users. The tests were conducted with Tor Browser on multiple machines, although under controlled conditions.
Prints
All of the tests involve JavaScript (which is enabled by default in Tor Browser). The first test captures the deltas sent by the mouse wheel's scroll events: a physical wheel will send a "3" for every delta, while a trackpad will send a variable stream of deltas that depends on the characteristics of the trackpad and the user's action.
The second test is related; it captures the scroll speed of the mouse, which Norte notes is linked to both the operating system's settings and the capabilities of the hardware. The third test is the speed of the mouse's movement which, as with the scroll speed, depends at least in part on OS and hardware factors.
The fourth test is a CPU benchmark. A CPU-intensive script is run in a Web Worker and the execution time recorded as the result. Norte pointed out in the post that Tor Browser already implements an anti-fingerprinting countermeasure by limiting the clock resolution of JavaScript's Date.getTime() function to 100ms. He was able to bypass this limitation by incrementing a counter once every millisecond with WindowTimers.setInterval(), though, and speculated that there are other ways to work around the clock-resolution problem.
The fifth test is akin to the canvas fingerprinting techniques already explored in existing research. To recap, canvas fingerprinting involves drawing a known shape into an HTML5 <canvas> element, then hashing the pixels of the resulting image. A number of factors result in small differences in how the shape is rendered, meaning considerable entropy can be collected about the specific machine in use.
Norte's test also exploits differences in the on-screen contents rendered by the browser, but it does not require any special element to be drawn onto the screen. Rather, the script queries the Element.getClientRects() function, which returns the precise pixel positions of each CSS border box for a given page element. That function can be called on any page element; Norte's tests call it on a text block, the size and position of which vary with window size, font configuration, and other factors.
Impact
The blog post links to a test page that users can access to see what data their browsers generate. Norte himself reports data only for two machines, which is far from enough to be meaningful. In particular, it is easy to see for oneself that the mouse-speed, scroll-speed, and CPU-benchmark numbers can vary considerably on a single machine between two test runs.
It might be possible to collect multiple samples across time and compute an average, but the value of doing so is likely to be small. Too many external factors can affect the CPU benchmark, and a fingerprinting technique needs to be reproducible to track users. The scrolling behavior recorded is, at the fundamental level, a measurement of user activity, not of the browser.
On the other hand, there is certainly some entropy to be found in determining whether the visiting machine uses a trackpad or a mouse with a physical scroll wheel. By far the most interesting test, though, is the getClientRects() fingerprint. Tor Browser now takes steps to alert the user whenever a script tries to read data from a <canvas> element, specifically so that users can block canvas-fingerprinting attempts.
It is hard to say how big the entropy contribution of either test is in the wild. The entropy numbers published by projects like the Electronic Frontier Foundation (EFF)'s Panopticlick count on millions of users contributing to the test data; the proportion of users that produce the same result on a particular measurement is what determines the entropy reported in Panopticlick's results. So a scroll-wheel delta of "3" (corresponding to any mouse with a physical wheel) does not contribute much, while the deltas sent by a truly strange trackpad would contribute noticeably more.
Various fingerprinting measurements can also overlap with each other, which reduces the entropy contributed by each metric. For instance, there was a bug opened against Tor Browser in 2015 that speculated on several additional JavaScript-based techniques to detect information about the attached pointing device—which may already be in use by real-world browser fingerprinters. It is not clear that Norte's techniques contribute additional information not already detectable through these other means. For its part, Panopticlick does not yet record pointer-related information about browsers except for whether or not the browser has touchscreen support. The techniques used to detect touchscreen support (such as the MouseEvent.mozInputSource property) may also overlap with Norte's tests for many devices.
Norte posted about his findings on the Tor Browser bug-discussion list. The limitations of the Date.getTime() countermeasure were already a known issue, it seems, but a new bug was opened to discuss the getClientRects() issue. Tor's George Kadianakis noted in the bug report that Tor Browser already takes some steps to sanitize the JavaScript functions used in Norte's test; the packages now bundle in a set of default fonts (so variations in system fonts should not impact how an element is rendered) and the browser rounds off the dimensions it reports for window sizes to the nearest 200 pixels.
Naturally, finding and circumventing fingerprinting vectors in Tor Browser is an ongoing process. The project will, no doubt, implement fixes for the open issues as soon as it is able. If further investigation indicates that the getClientRects() vector is indeed a significant entropy source, that could require more subtle adjustments, since getClientRects() has legitimate uses.
For those users running other browsers, however, Norte's findings are simply more bullet points in the already lengthy list of ways that a web site can track people across sessions. Purists will point out that these fingerprinting vectors, like so many others, require JavaScript to be enabled. But, as an increasing percentage of web sites rely on JavaScript for even basic functionality, Tor Browser's approach—which tries to balance practical function with a measure of anonymity—looks more and more like the only way to avoid being fingerprinted.
Brief items
Security quotes of the week
And saving it is dangerous because failing to secure it is damaging. It will reduce a company's profits, reduce its market share, hurt its stock price, cause it public embarrassment, and -- in some cases -- result in expensive lawsuits and occasionally, criminal charges.
All this makes data a toxic asset, and it continues to be toxic as long as it sits in a company's computers and networks. The data is vulnerable, and the company is vulnerable. It's vulnerable to hackers and governments. It's vulnerable to employee error. And when there's a toxic data spill, millions of people can be affected.
Like climate change, the privacy catastrophes of the next two decades are already inevitable. The problem we face is preventing the much worse catastrophes of the following the decades.
And as computers are integrated into the buildings and vehicles and cities we inhabit, as they penetrate our bodies, the potential harms from breaches will become worse.
They just want to pay their damned bills. And while we can assume that AT&T will eventually update their certs, good luck finding the person at AT&T in charge of actually doing that. Sure, call up customer service and try get them to fix it. Go ahead. Everyone should have a hobby.
New vulnerabilities
botan: two vulnerabilities
| Package(s): | botan | CVE #(s): | CVE-2015-5726 CVE-2015-5727 | ||||||||||||
| Created: | March 8, 2016 | Updated: | March 9, 2016 | ||||||||||||
| Description: | From the Mageia advisory:
The BER decoder would crash due to reading from offset 0 of an empty vector if it encountered a BIT STRING which did not contain any data at all. This can be used to easily crash applications reading untrusted ASN.1 data, but does not seem exploitable for code execution (CVE-2015-5726). The BER decoder would allocate a fairly arbitrary amount of memory in a length field, even if there was no chance the read request would succeed. This might cause the process to run out of memory or invoke the OOM killer (CVE-2015-5727). | ||||||||||||||
| Alerts: |
| ||||||||||||||
chromium: multiple vulnerabilities
| Package(s): | chromium | CVE #(s): | CVE-2016-1630 CVE-2016-1631 CVE-2016-1632 CVE-2016-1633 CVE-2016-1634 CVE-2016-1635 CVE-2016-1636 CVE-2016-1637 CVE-2016-1638 CVE-2016-1639 CVE-2016-1640 CVE-2016-1641 CVE-2016-1642 | ||||||||||||||||||||||||||||||||||||
| Created: | March 4, 2016 | Updated: | March 9, 2016 | ||||||||||||||||||||||||||||||||||||
| Description: | From the Arch Linux advisory: CVE-2016-1630: Same-origin bypass in Blink. Credit to Mariusz Mlynski. CVE-2016-1631: Same-origin bypass in Pepper Plugin. Credit to Mariusz Mlynski. CVE-2016-1632: Bad cast in Extensions. CVE-2016-1633, CVE-2016-1634: Use-after-free in Blink. Credit to cloudfuzzer. CVE-2016-1635: Use-after-free in Blink. Credit to Rob Wu. CVE-2016-1636: SRI Validation Bypass. Credit to Ryan Lester and Bryant Zadegan. CVE-2016-1637: Information Leak in Skia. Credit to Keve Nagy. CVE-2016-1638: WebAPI Bypass. Credit to Rob Wu. CVE-2016-1639: Use-after-free in WebRTC. Credit to Khalil Zhani. CVE-2016-1640: Origin confusion in Extensions UI. Credit to Luan Herrera. CVE-2016-1641: Use-after-free in Favicon. Credit to Atte Kettunen of OUSPG. CVE-2016-1642: Various fixes from internal audits, fuzzing and other initiatives. | ||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||
exiv2: denial of service
| Package(s): | exiv2 | CVE #(s): | |||||||||||||
| Created: | March 4, 2016 | Updated: | March 9, 2016 | ||||||||||||
| Description: | From the Red Hat bug report: Exempi contains code to protect against a denial-service-attack related to XML entity expansion ("billion laughs attack"), but it is not compiled into the Fedora package because BanAllEntityUsage is not defined when the package is compiled. | ||||||||||||||
| Alerts: |
| ||||||||||||||
ffmpeg: denial of service
| Package(s): | libav ffmpeg | CVE #(s): | CVE-2016-2326 | ||||||||||||
| Created: | March 7, 2016 | Updated: | March 9, 2016 | ||||||||||||
| Description: | From the CVE entry:
Integer overflow in the asf_write_packet function in libavformat/asfenc.c in FFmpeg before 2.8.5 allows remote attackers to cause a denial of service or possibly have unspecified other impact via a crafted PTS (aka presentation timestamp) value in a .mov file. | ||||||||||||||
| Alerts: |
| ||||||||||||||
gimp: command execution
| Package(s): | gimp | CVE #(s): | CVE-2012-4245 | ||||
| Created: | March 7, 2016 | Updated: | March 9, 2016 | ||||
| Description: | From the CVE entry:
The scriptfu network server in GIMP 2.6 does not require authentication, which allows remote attackers to execute arbitrary commands via the python-fu-eval command. | ||||||
| Alerts: |
| ||||||
jasper: multiple vulnerabilities
| Package(s): | jasper | CVE #(s): | CVE-2016-1577 CVE-2016-2116 | ||||||||||||||||||||||||||||||||||||
| Created: | March 4, 2016 | Updated: | March 9, 2016 | ||||||||||||||||||||||||||||||||||||
| Description: | From the Ubuntu advisory: Jacob Baines discovered that JasPer incorrectly handled ICC color profiles in JPEG-2000 image files. If a user were tricked into opening a specially crafted JPEG-2000 image file, a remote attacker could cause JasPer to crash or possibly execute arbitrary code with user privileges. (CVE-2016-1577) Tyler Hicks discovered that JasPer incorrectly handled memory when processing JPEG-2000 image files. If a user were tricked into opening a specially crafted JPEG-2000 image file, a remote attacker could cause JasPer to consume memory, resulting in a denial of service. (CVE-2016-2116) | ||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||
kernel: multiple vulnerabilities
| Package(s): | kernel | CVE #(s): | CVE-2015-8816 CVE-2015-8830 CVE-2016-2543 CVE-2016-2544 CVE-2016-2545 CVE-2016-2546 CVE-2016-2547 CVE-2016-2548 CVE-2016-2549 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | March 4, 2016 | Updated: | March 9, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Debian advisory: CVE-2015-8816 - A use-after-free vulnerability was discovered in the USB hub driver. This may be used by a physically present user for privilege escalation. CVE-2015-8830 - Ben Hawkes of Google Project Zero reported that the AIO interface permitted reading or writing 2 GiB of data or more in a single chunk, which could lead to an integer overflow when applied to certain filesystems, socket or device types. The full security impact has not been evaluated. CVE-2016-2543 - Dmitry Vyukov found that the core sound sequencer driver (snd-seq) lacked a necessary check for a null pointer, allowing a user with access to a sound sequencer device to cause a denial-of- service (crash). CVE-2016-2544, CVE-2016-2546, CVE-2016-2547, CVE-2016-2548 - Dmitry Vyukov found various race conditions in the sound subsystem (ALSA)'s management of timers. A user with access to sound devices could use these to cause a denial-of-service (crash or hang) or possibly for privilege escalation. CVE-2016-2545 - Dmitry Vyukov found a flaw in list manipulation in the sound subsystem (ALSA)'s management of timers. A user with access to sound devices could use this to cause a denial-of-service (crash or hang) or possibly for privilege escalation. CVE-2016-2549 - Dmitry Vyukov found a potential deadlock in the sound subsystem (ALSA)'s use of high resolution timers. A user with access to sound devices could use this to cause a denial-of-service (hang). | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
kernel: denial of service
| Package(s): | kernel | CVE #(s): | CVE-2016-2550 | ||||||||||||||||||||||||||||||||||||||||
| Created: | March 3, 2016 | Updated: | March 9, 2016 | ||||||||||||||||||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla entry:
The commit referenced in the Fixes tag incorrectly accounted the number of in-flight fds over a unix domain socket to the original opener of the file-descriptor. This allows another process to arbitrary deplete the original file-openers resource limit for the maximum of open files. | ||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||
mozilla: multiple vulnerabilities
| Package(s): | firefox seamonkey | CVE #(s): | CVE-2016-1952 CVE-2016-1954 CVE-2016-1957 CVE-2016-1958 CVE-2016-1960 CVE-2016-1961 CVE-2016-1962 CVE-2016-1964 CVE-2016-1965 CVE-2016-1966 CVE-2016-1973 CVE-2016-1974 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | March 9, 2016 | Updated: | April 18, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Red Hat advisory:
Several flaws were found in the processing of malformed web content. A web page containing malicious content could cause Firefox to crash or, potentially, execute arbitrary code with the privileges of the user running Firefox. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
mozilla: multiple vulnerabilities
| Package(s): | firefox | CVE #(s): | CVE-2016-1953 CVE-2016-1955 CVE-2016-1956 CVE-2016-1959 CVE-2016-1963 CVE-2016-1967 CVE-2016-1968 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | March 9, 2016 | Updated: | April 25, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Ubuntu advisory:
Bob Clary, Christoph Diehl, Christian Holler, Andrew McCreight, Daniel Holbert, Jesse Ruderman, Randell Jesup, Carsten Book, Gian-Carlo Pascutto, Tyson Smith, Andrea Marchesini, and Jukka Jylänki discovered multiple memory safety issues in Firefox. If a user were tricked in to opening a specially crafted website, an attacker could potentially exploit these to cause a denial of service via application crash, or execute arbitrary code with the privileges of the user invoking Firefox. (CVE-2016-1952, CVE-2016-1953) Muneaki Nishimura discovered that CSP violation reports contained full paths for cross-origin iframe navigations. An attacker could potentially exploit this to steal confidential data. (CVE-2016-1955) Ucha Gobejishvili discovered that performing certain WebGL operations resulted in memory resource exhaustion with some Intel GPUs, requiring a reboot. If a user were tricked in to opening a specially crafted website, an attacker could potentially exploit this to cause a denial of service. (CVE-2016-1956) Looben Yang discovered an out-of-bounds read in Service Worker Manager. If a user were tricked in to opening a specially crafted website, an attacker could potentially exploit this to cause a denial of service via application crash, or execute arbitrary code with the privileges of the user invoking Firefox. (CVE-2016-1959) It was discovered that Firefox crashes when local files are modified whilst being read by the FileReader API. If a user were tricked in to opening a specially crafted website, an attacker could potentially exploit this to execute arbitrary code with the privileges of the user invoking Firefox. (CVE-2016-1963) Jordi Chancel discovered a same-origin-policy bypass when using performance.getEntries and history navigation with session restore. If a user were tricked in to opening a specially crafted website, an attacker could potentially exploit this to steal confidential data. (CVE-2016-1967) Luke Li discovered a buffer overflow during Brotli decompression in some circumstances. If a user were tricked in to opening a specially crafted website, an attacker could potentially exploit this to cause a denial of service via application crash, or execute arbitrary code with the privileges of the user invoking Firefox. (CVE-2016-1968) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
nss: code execution
| Package(s): | nss | CVE #(s): | CVE-2016-1950 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | March 9, 2016 | Updated: | April 6, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Red Hat advisory:
A heap-based buffer overflow flaw was found in the way NSS parsed certain ASN.1 structures. An attacker could use this flaw to create a specially crafted certificate which, when parsed by NSS, could cause it to crash, or execute arbitrary code, using the permissions of the user running an application compiled against the NSS library. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
openstack-nova: information exposure
| Package(s): | openstack-nova | CVE #(s): | CVE-2016-2140 | ||||||||||||||||
| Created: | March 9, 2016 | Updated: | March 9, 2016 | ||||||||||||||||
| Description: | From the Red Hat advisory:
An information-exposure flaw was found in the OpenStack Compute (nova) resize and migrate functionality. An authenticated user could write a malicious qcow header to an ephemeral or root disk, referencing a block device as a backing file. With a subsequent resize or migration, file system content on the specified device would be leaked to the user. Only setups using libvirt with raw storage and "use_cow_images = False" were affected. | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
pcs: two vulnerabilities
| Package(s): | pcs | CVE #(s): | CVE-2016-0720 CVE-2016-0721 | ||||||||||||||||
| Created: | March 3, 2016 | Updated: | December 15, 2016 | ||||||||||||||||
| Description: | From the Red Hat bugzilla entries:
CVE-2016-0720: The pcsd web UI is vulnerable to Cross-Site Request Forgery (CSRF). A remote attacker could provide a specially crafted web page that, when visited by a user with a valid pcsd session, would allow the attacker to trigger requests on behalf of the user, for example removing resources, restarting/removing nodes, etc. Each request includes 'X-Requested-With: XMLHttpRequest' but this header is not checked server side. CVE-2016-0721: The session cookie is not invalidated on the server side when logging out of the web UI. It is invalidated in the browser's cookie cache but it still works on requests sent by hand. The cookie is also not invalidated when that same user log in again, meaning all cookies are stored in a pool of valid sessions that are only invalidated when the cookies expires (1 day). This can allow attackers to perform session fixation attacks. | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
phpmyadmin: cross-site scripting
| Package(s): | phpmyadmin | CVE #(s): | CVE-2016-2560 CVE-2016-2561 | ||||||||||||||||||||||||||||||||||||||||
| Created: | March 3, 2016 | Updated: | March 9, 2016 | ||||||||||||||||||||||||||||||||||||||||
| Description: | From the Mageia advisory:
Multiple cross-site scripting (XSS) issues in phpMyAdmin before 4.4.15.5 (CVE-2016-2560, CVE-2016-2561). | ||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||
roundup: information leak
| Package(s): | roundup | CVE #(s): | CVE-2014-6276 | ||||
| Created: | March 4, 2016 | Updated: | March 9, 2016 | ||||
| Description: | From the Debian advisory: Ralf Schlatterbeck discovered an information leak in roundup, a web-based issue tracking system. An authenticated attacker could use it to see sensitive details about other users, including their hashed password. | ||||||
| Alerts: |
| ||||||
samba: two vulnerabilities
| Package(s): | samba | CVE #(s): | CVE-2015-7560 CVE-2016-0771 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | March 8, 2016 | Updated: | March 29, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Ubuntu advisory:
Jeremy Allison discovered that Samba incorrectly handled ACLs on symlink paths. A remote attacker could use this issue to overwrite the ownership of ACLs using symlinks. (CVE-2015-7560) Garming Sam and Douglas Bagnall discovered that the Samba internal DNS server incorrectly handled certain DNS TXT records. A remote attacker could use this issue to cause Samba to crash, resulting in a denial of service, or possibly obtain uninitialized memory contents. This issue only applied to Ubuntu 14.04 LTS and Ubuntu 15.10. (CVE-2016-0771) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
squid: denial of service
| Package(s): | squid | CVE #(s): | CVE-2016-2570 | ||||||||||||||||||||||||||||||||||||
| Created: | March 7, 2016 | Updated: | November 11, 2016 | ||||||||||||||||||||||||||||||||||||
| Description: | From the CVE entry:
The Edge Side Includes (ESI) parser in Squid 3.x before 3.5.15 and 4.x before 4.0.7 does not check buffer limits during XML parsing, which allows remote HTTP servers to cause a denial of service (assertion failure and daemon exit) via a crafted XML document, related to esi/CustomParser.cc and esi/CustomParser.h. | ||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||
tomcat: session hijacking
| Package(s): | tomcat | CVE #(s): | CVE-2015-5346 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | March 3, 2016 | Updated: | March 9, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Mageia advisory:
Session fixation vulnerability in Apache Tomcat 7.x before 7.0.66, when different session settings are used for deployments of multiple versions of the same web application, might allow remote attackers to hijack web sessions by leveraging use of a requestedSessionSSL field for an unintended request, related to CoyoteAdapter.java and Request.java (CVE-2015-5346). | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||
wireshark: multiple dissector crashes
| Package(s): | wireshark | CVE #(s): | CVE-2016-2522 CVE-2016-2523 CVE-2016-2524 CVE-2016-2525 CVE-2016-2526 CVE-2016-2527 CVE-2016-2528 CVE-2016-2529 CVE-2016-2530 CVE-2016-2531 CVE-2016-2532 | ||||||||||||||||||||||||||||||||
| Created: | March 3, 2016 | Updated: | March 14, 2016 | ||||||||||||||||||||||||||||||||
| Description: | From the Mageia advisory:
ASN.1 BER dissector crash (CVE-2016-2522). DNP dissector infinite loop (CVE-2016-2523). X.509AF dissector crash (CVE-2016-2524). HTTP/2 dissector crash (CVE-2016-2525). HiQnet dissector crash (CVE-2016-2526). 3GPP TS 32.423 Trace file parser crash (CVE-2016-2527). LBMC dissector crash (CVE-2016-2528). iSeries file parser crash (CVE-2016-2529). RSL dissector crash (CVE-2016-2530, CVE-2016-2531). LLRP dissector crash (CVE-2016-2532). | ||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||
xen: denial of service
| Package(s): | xen | CVE #(s): | CVE-2015-0268 | ||||
| Created: | March 7, 2016 | Updated: | March 9, 2016 | ||||
| Description: | From the CVE entry:
The vgic_v2_to_sgi function in arch/arm/vgic-v2.c in Xen 4.5.x, when running on ARM hardware with general interrupt controller (GIC) version 2, allows local guest users to cause a denial of service (host crash) by writing an invalid value to the GICD.SGIR register. | ||||||
| Alerts: |
| ||||||
Page editor: Jake Edge
Kernel development
Brief items
Kernel release status
The current development kernel is 4.5-rc7, released on March 6. Linus said: "So things have finally calmed down this past week, and I think we'll end up with a normal release where rc7 is the last rc."
Stable updates: 4.4.4, 3.14.63, and 3.10.99 were released on March 3. The 4.4.5, 3.14.64, and 3.10.100 updates are in the review process as of this writing; they can be expected on or after March 10.
Some departures from linux-next
In his role as the maintainer of the linux-next tree, Stephen Rothwell has a comprehensive view of development activity across the kernel. He has recently announced his intention to drop a number of repositories that have been inactive for a year or more from linux-next. The list is interesting, as it highlights areas of past (and possibly present) interest where development activity has come to a halt.The trees being removed are:
- APM (the obsolete power-management mechanism).
- The bcm2835 driver.
- kgdb: merged, and, presumably, not needing further work.
- lblnet (labeled networking, also merged).
- llvmlinux (building the kernel with the LLVM compiler).
- mips-fixes; MIPS patches follow other paths now.
- MMC: MultiMediaCard device support. MMC patches seem to go via other trees now.
- Random-number generation.
- rpmsg: the remote processor messaging subsystem.
- The squashfs compressed filesystem.
- Tiny, the home of the kernel tinification project.
So far, none of the maintainers who have responded have objected to the removal of their trees. That does not necessarily mean that the projects are dead; as Stephen pointed out, adding the trees back to linux-next is easily done should they become active again.
Kernel development news
Some 4.5 Development statistics
When Linus released the 4.5-rc7 prepatch, he indicated that it would probably be the last one prior to the official 4.5 release. That means we're running a bit late for the traditional article full of statistics for this development cycle. So, without further ado, here is a look at the changes that came in during the 4.5 cycle and where they came from.As of this writing, just over 12,000 non-merge changesets have landed in the mainline repository for 4.5. That makes 4.5 one of the quieter development cycles in the last year; less than 4.2 and 4.4 (both over 13,000 changesets) but approximately equal to 4.1 and 4.3. All things are relative, of course; not that long ago, 12,000 changes would have been one of the busiest cycles. Even when the kernel community slows down a bit, there is a lot going on.
These changes were contributed by 1,528 developers — short of the 1,575 seen in 4.4 or the 1,625 in 4.3, but, once again, a fair crowd of contributors. The most active of these developers were:
Most active 4.5 developers
By changesets Linus Walleij 236 2.0% Arnd Bergmann 226 1.9% Leo Kim 210 1.7% Mauro Carvalho Chehab 169 1.4% Geert Uytterhoeven 159 1.3% Ville Syrjälä 126 1.0% Kuninori Morimoto 112 0.9% Takashi Iwai 108 0.9% Jiri Olsa 104 0.9% Christoph Hellwig 102 0.8% Julia Lawall 101 0.8% Glen Lee 101 0.8% Javier Martinez Canillas 89 0.7% Geliang Tang 89 0.7% Dan Carpenter 85 0.7% Daniel Vetter 81 0.7% Boris Brezillon 80 0.7% Alex Deucher 80 0.7% Kirill A. Shutemov 77 0.6% Thierry Reding 74 0.6%
By changed lines Doug Ledford 53086 7.7% Tomi Valkeinen 36631 5.3% Eric Huang 22714 3.3% Alex Deucher 16604 2.4% yanyang1 11129 1.6% Igal Liberman 10569 1.5% Thierry Reding 9842 1.4% Bard Liao 9762 1.4% Christoph Hellwig 9680 1.4% Arnd Bergmann 9233 1.3% Geert Uytterhoeven 8325 1.2% Stephen Boyd 8183 1.2% Paul E. McKenney 7485 1.1% Rex Zhu 7382 1.1% The etnaviv authors 7238 1.1% Jammy Zhou 7175 1.0% Mauro Carvalho Chehab 6473 0.9% Eric Anholt 6234 0.9% Maruthi Srinivas Bayyavarapu 5239 0.8% Adam Thomson 5153 0.7%
Linus Walleij topped the by-changesets list with a lot of low-level work, mostly near the GPIO subsystem and drivers that use it. Arnd Bergmann works all over the tree, mostly dealing with build problems and improving ARM multiplatform support. Leo Kim worked exclusively on cleaning up the wilc1000 driver in the staging tree, Mauro Carvalho Chehab made many improvements as the maintainer of the media subsystem, and Geert Uytterhoeven did a lot of work in the ARM and related driver subsystems.
In many development cycles, this list has been dominated by developers working in the staging subsystem, but 4.5 is an exception: only two of the developers in the by-changesets column had any significant work in the staging tree at all. Both of them, as it turns out, were working on the wilc1000 driver.
In the lines-changed column, longtime contributor Doug Ledford got to the top with three changesets removing three unloved staging drivers, deleting 53,000 lines of code. Tomi Valkeinen did a lot of work with the TI OMAP subarchitecture, while Eric Huang, Alex Deucher, and "yanyang1" all added functionality to the AMD graphics drivers. Further down that list, "the etnaviv authors" is an alias that was used for a single patch adding the Etnaviv graphics driver; it represents the work of Christian Gmeiner, Russell King, and Lucas Stach.
Work on the 4.5 kernel was supported by just over 200 companies that we could identify — a typical number. The employers supporting the most work this time were:
Most active 4.5 employers
By changesets Intel 1734 14.4% (Unknown) 975 8.1% Red Hat 732 6.1% Linaro 723 6.0% (None) 628 5.2% Samsung 513 4.3% SUSE 382 3.2% Atmel 380 3.2% Renesas Electronics 360 3.0% IBM 346 2.9% AMD 283 2.4% Mellanox 275 2.3% (Consultant) 245 2.0% Broadcom 208 1.7% Oracle 179 1.5% 160 1.3% Texas Instruments 152 1.3% Huawei Technologies 141 1.2% NVidia 137 1.1% ARM 127 1.1%
By lines changed Red Hat 83657 12.1% Intel 80160 11.6% AMD 74673 10.8% Texas Instruments 41808 6.1% (Unknown) 27958 4.1% IBM 25433 3.7% Linaro 22198 3.2% (None) 21929 3.2% Mellanox 19558 2.8% Samsung 19190 2.8% Renesas Electronics 17964 2.6% (Consultant) 15593 2.3% NVidia 15038 2.2% Freescale 13964 2.0% Code Aurora Forum 13514 2.0% Atmel 10845 1.6% Realtek 10090 1.5% Rockchip 9735 1.4% Huawei Technologies 7992 1.2% Broadcom 7930 1.2%
Intel is, by now, the dominant contributor; it would have been at the top of both lists except for the aforementioned drivers removed by Doug Ledford. Red Hat, which once reliably sat at the top of the list, may soon be overshadowed by companies working in the mobile and embedded space. In general, though, this table looks much like it has for some time.
If we look at non-author signoffs — the addition of Signed-off-by tags to patches by developers other than the author — the story looks just a little different:
Most non-author signoffs in 4.5
Developers Greg Kroah-Hartman 1009 9.0% David S. Miller 950 8.5% Mark Brown 585 5.2% Andrew Morton 451 4.0% Martin K. Petersen 264 2.4% Arnaldo Carvalho de Melo 263 2.3% Mauro Carvalho Chehab 235 2.1% Glen Lee 210 1.9% Rafael J. Wysocki 205 1.8% Kalle Valo 193 1.7%
Companies Red Hat 1981 17.8% Intel 1459 13.1% Linux Foundation 1029 9.2% Linaro 1003 9.0% 632 5.7% Samsung 447 4.0% (None) 361 3.2% Oracle 295 2.7% IBM 288 2.6% SUSE 265 2.4%
To a first approximation, this table represents the most active subsystem maintainers — the developers who make the decision to accept any given patch. While the more traditional, enterprise-oriented companies remain at the top of this list, the curve has been flattening over time as more companies take responsibility for the maintenance of parts of the kernel.
Finally, it has been a while since we looked at what the most active companies are most interested in. That is a simple matter of picking out the patches contributed by a given company's developers and noting which files were touched. So, for example, here is where Intel works:
Intel % Subsystem Notes 67% drivers/ 29% gpu, 15% net, 5% staging 12% include/ 9% sound/ 5% net/ 5% arch/ 3.3% x86 4% kernel/ 4% mm/
Intel, clearly, is focused on drivers for its hardware and CPUs, as one might expect. The picture for Red Hat is a bit different:
Red Hat % Subsystem Notes 29% drivers/ 6% gpu, 5% net, 3% tty 22% tools/ 19% perf 17% fs/ 4% xfs, 3% nfs, 3% gfs2, 2% namei.c, 1% btrfs, 1% ceph, 1% f2fs, 1% ext4 14% include/ 13% arch/ 4% x86, 2% arm, 2% s390, 1% powerpc, 1% sparc 7% kernel/ 5% net/ 3% crypto/
Red Hat's contribution to the tools directory (and the perf tool in particular) has increased over the years, but the company still works all over the kernel, putting a significant part of its effort into the core kernel code.
What about Linaro, which has been increasing its contributions over the years?
Linaro % Subsystem Notes 66% drivers/ 17% gpio, 7% clocksource, 7% pinctrl, 4% mfd, 4% staging, 4% net 25% arch/ 19% arm, 4% arm64, 1% mips 6% include/ 3% sound/ 3% Documentation/
Linaro is all about hardware enablement, and its work shows that. Even the work in the documentation directory is aimed that way: almost all of it happened in the devicetree subdirectory. (Lest anybody worry that the numbers add up to over 100%, remember that many patches touch more than one subdirectory, and are thus counted more than once).
Many of the other companies on the list have similar patterns; they tend to be interested in support for their own hardware, so that is where their work is done. Something slightly different can be seen if one skips down the list and looks at Google, though:
% Subsystem Notes 29% drivers/ 9% net, 8% input, 5% usb, 4% md, 3% pci 31% net/ 15% ipv4, 9% core, 8% ipv6 17% include/ 15% arch/ 10% x86, 3% powerpc, 2% arm, 2% arm64 6% fs/ 3% ext4
Many of the improvements to the networking subsystem have, in recent years, come from Google; the numbers here show that Google is still interested in making Linux networking better.
All of this activity is the result of around 200 companies and numerous individuals, all working in pursuit of their own interests without any sort of overall control. As one might expect, the outcome can be a bit patchy at times; less energy goes into areas like documentation and security than one might like. But we still get a quickly evolving, highly capable kernel out of it, and that doesn't look like it will change anytime soon.
In-band deduplication for Btrfs
"In-band deduplication" is the process of detecting and unifying duplicate data blocks as files are being written, rather than at some later time. Btrfs support for this feature has been under development since at least early 2013. Quite recently it reached the point where developer Qu Wenruo thought it was sufficiently ready to send Btrfs maintainer Chris Mason a pull request (as yet unanswered) hoping that it might be added to the kernel during the 4.6 merge window. While this is far from a magic bullet that will suddenly remove all the waste in your filesystem that is caused by duplicate data, there are use cases where it could bring real benefits.
Offline and in-band
It has been possible to unify duplicated blocks in Btrfs since Linux 3.12 when the BTRFS_IOC_FILE_EXTENT_SAME ioctl() command was added (it has since been renamed FIDEDUPERANGE when the basic ioctl() handling was moved to the VFS, so it could someday be used by other filesystems). This ioctl() is given a range of bytes in the file (which must be aligned to the filesystem block size) and a list of offsets in other files (or possibly the same file). This identifies two or more ranges of the same size that are claimed to be identical. Btrfs reads all the blocks, checks that they are in fact identical, and then changes its internal bookkeeping so that all files point to one set of blocks on the storage medium. The space used by the other copies of the data will typically become free space available for reallocation.
To understand how this works it is necessary to understand how Btrfs makes use of "extents". An extent is a contiguous set of blocks on the storage device and also a contiguous set of blocks in a file (at least initially). It has a "logical address" that is mapped to a physical location on one, or perhaps more, of the underlying devices. An extent can become part of several different files, either through snapshots, reflinks, or deduplication. To allow this storage space to ultimately be reused, Btrfs maintains a reference count for each extent and will not re-use any of the space until that count becomes zero.
It is not generally possible to split an extent. If a single block is written into the middle of a large extent, the "copy on write" design of Btrfs requires that either the whole extent be copied or that the file's indexing information be changed to point to the first half of the extent, then the new block, then the remainder of the extent. Btrfs takes the latter approach.
When FIDEDUPERANGE is used, the file indexes will be updated to point to the relevant extents or partial extents of the source file, and the reference counts on the various extents will be increased or decreased as appropriate. This may not release quite as much space as expected since there may have been extents that were only partially identical between two files. The identical part in one extent may have no files including it any more, but the space will not be freed until the whole extent is no longer referenced.
duperemove is a tool that can be used to examine a set of files,
look for duplicated regions, and call the ioctl to remove that
duplication from the underlying storage. This can be useful anywhere
that files are likely to be completely or largely the same, but where
they need to be kept separate for administrative or other practical reasons. A common use case is filesystem images used for
virtualization or the multiple similar-but-not-identical file sets
used by containers. Running duperemove from time to time
could save
a lot of disk space.
This functionality was referred to in the initial commit as "offline deduplication", which is a little confusing since "offline" in the context of filesystems usually implies that the filesystem isn't mounted, and that is not the case here. It is more like "on-demand" deduplication and is sometimes referred to as "batch" deduplication. This contrasts with the new work that could be called "time-of-write" or transparent deduplication, but is called "in-band" deduplication in this patch set.
When duperemove runs, it computes a hash value for every
block in
every file and then compares these, looking for ranges of matching
blocks. When it finds suitable ranges, it requests the deduplication.
In-band deduplication uses the same idea of hashing blocks of data,
but in many other respects it is quite different.
In-band duplication works in units of extents of a specific size — a deduplication block size of up to 8MB can be configured; the default is 128KB. When data is written out to storage, blocks are gathered into extents of this size whenever possible. If there are only sufficient blocks being written for a smaller extent, that extent will be written without deduplication. For every extent of the target size, the hash (currently SHA-256, though possibly configurable in the future) is calculated. If another extent can be found with the same hash, then a new reference is added to that extent and the new data is not written. If no match is found, the data is written as normal and the hash is stored together with the logical address of the extent so that it is available for future matching.
This process can be enabled for a whole Btrfs filesystem using the "btrfs dedup enable" command and then disabled for individual files or directories by using the "btrfs property set" command to set the "dedup" property to "disable". Setting this on a directory causes all files or directories created within that directory to inherit the setting. This means that enabling deduplication on only a subset of a filesystem is possible, but a little bit clumsy.
Two back-ends
There are two separate back-ends for storing the mapping between hashes and extents; the fact that the implementation of these back-ends is kept cleanly separate from their use is rightfully highlighted as a strength of this patch set.
One of the two back-ends is an in-memory mapping. Two red-black trees are created when the filesystem is mounted; whenever new data is written to a file on which deduplication is enabled, new entries are added to the trees. One tree maps from hash to logical address and is used to see if a suitable extent exists that already stores the required data. The second provides a reverse mapping that allows hash entries to be deleted when an extent is freed after its reference count reaches zero. The size of these trees is limited by a configurable entry count that defaults to 32768. It would not be surprising to see that replaced or augmented with a "shrinker" callback from the memory management subsystem, so that it could shrink only when memory is tight.
The second mechanism stores these mappings in the filesystem. Btrfs has flexible data structures for storing all sorts of metadata and data on disk using a number of distinct B-trees, one for each subvolume, one for the extent reference counts, one for storing checksums, etc. The in-band deduplication patchset adds another B-tree that has two types of keys: one for lookup by hash and another for lookup by extent address.
Looking up the hash in the B-tree is not quite so straightforward as one might hope, since Btrfs has a fixed format for the lookup key: a 64-bit object-ID, an eight-bit type, and a 64-bit offset. It is not possible to store the full 256-bit hash in the key and even storing 128 bits in the two 64-bit fields would be problematic. Btrfs requires that all keys be unique, so that approach would effectively limit the hash size to 128 bits.
The approach chosen is to store 64 bits of the hash in the object-ID, and the logical address of the extent in the offset field. Each key is accompanied by a variable-length data field and this is used to store the full hash. If the hashes of two extents collide in those 64 bits, the offsets will still be different, so the keys will be unique.
To handle these collisions a lookup first performs a regular B-tree search for a key with the appropriate hash bits in the object-ID and U64_MAX in the offset field. This will provide the last possible location where the target hash could be stored. A linear search is then performed searching backward and comparing the full hash until a match is found or there are no more keys with the required hash fragment.
Unlike with in-memory lookup there is no mechanism to limit the number of hash entries stored, beyond normal filesystem-full checks that might prevent a new extent from being written.
One difference between this in-band deduplication and the on-demand
approach that is worth highlighting is the dependence placed on the
hash. duperemove uses a hash only to guide the search for duplicate
blocks — the hash is not authoritative and Btrfs will not allow the
deduplication to happen if the identified regions are not
byte-for-byte identical. In-band duplication as currently implemented
does not perform that comparison. If the hash matches, then the
extents are assumed to match.
This means the correctness of the
deduplication is completely dependent on the uniqueness of the hash,
so the extra effort to make use of all 256 bits is easy to justify.
Whether that complete dependence is itself justified is not an easy
question to answer. It is certainly extremely unlikely for two
distinct blocks to have the same hash, but it is also certainly quite
possible. It would only need to cause corruption once to be
extremely embarrassing. Adding byte-for-byte comparison is on the
planned feature list but, according to Wenruo, "not anytime soon".
In-band deduplication brings the benefit of being automatic, but has a cost
that it will probably miss duplication that duperemove could find.
Different alignment of extents between files would completely defeat
the duplicate detection, as would creating files before deduplication
was enabled. The former could be improved to some degree with a
smaller extent size, though that would brings costs of its own. As is so
often the case, finding the most effective solution — which could include a
mix of offline and in-band deduplication — will be highly
dependent on each particular use case.
Test, test, and test
Wenruo assures us that this patch set has seen quite a lot of testing and that it has been some time since the last of the bugs found by that testing was fixed, which is encouraging. Some new tests have been submitted for the xfstests test suite specifically to exercise the deduplication and to check for some of the bugs that have been found and fixed.
One aspect of testing that seemed strangely absent from the pull request was any hint of how in-band deduplication affects performance. It is to be expected that this sort of functionality would slow writes down, particularly when the table of hashes is not kept in memory. It could also speed up some writes if lots of duplicate extents are found. Some indication of the sort of performance change experienced would certainly help to complete the picture.
But maybe the hope is to crowdsource that testing. There are so many different possible hardware configurations and usage scenarios to test that it is hard for one developer to even begin to give meaningful results. A large community, on the other hand, can try lots of things in parallel. As Wenruo noted in his pull request, there is still work to be done, but it should be quite ready for people to test.
Trying it out requires some patches
to the btrfs-progs along with the Git
tree from the pull request, but that should be no challenge for those
who enjoy compiling their own kernels. I'm sure additional results would be
most welcome.
Architecture emulation containers with binfmt_misc
Containers bring a lot of advantages in the areas of security and systems administration; they can be used to run lightweight virtual "systems" in isolation from each other and the host system. Since containers lack their own kernel — they run directly on the host-system kernel — one does not ordinarily expect them to be built for an architecture other than the host they are running on. But it turns out that there are use cases for such containers, and that they can be run using the somewhat obscure "binfmt_misc" kernel mechanism if a small patch set is applied.
binfmt_misc
When an executable file is provided to one of the exec*() system calls, the kernel normally expects to find a native binary for the system it is running on. The kernel has long had a mechanism by which it can recognize other executable-file formats and run them, though. The classic example is the module that looks for a file that begins with "#!" — the marker for a shell script. When such a file is recognized, the name of the interpreter for the script will be read from the first line of the file; the interpreter will then be run with the file as its standard input.
But one can imagine many other possible formats for executable files. These could be binaries built for a different operating system (DOS binaries that could be run with DOSEMU, for example) or byte-code binaries that need to run on a specific machine (such as Java byte code). One could try to code awareness of all these formats into the kernel, but that gets unwieldy after a while. It also lacks flexibility, which is unfortunate; the kernel developers are never going to know about all of the possible executable formats that might be of interest.
The obvious solution is to allow user space to describe new executable formats to the kernel; that is the role of the binfmt_misc mechanism. If this feature is configured into the kernel (as it usually is), a system administrator can add a new executable format by writing a special string to /proc/sys/fs/binfmt_misc/register. That string includes:
- A way for the kernel to recognize the new format. It can either be
a particular file extension, or a "magic number" found near the
beginning of the file.
- The name of the interpreter that is to be run to execute
files with this format.
- Some flags that control how the argv array is created and, essentially, whether files in this format can be setuid or not.
The full details of how it all works can be found in Documentation/binfmt_misc.txt in the kernel source tree.
It is not hard to see how binfmt_misc can be used to run binaries built for a different architecture. It is a simple matter of describing those binaries and naming an emulator (QEMU, for example) that is able to run the binaries. That works well for binaries to be run directly on the host system, but it can be a bit more challenging to run a container that is built for another architecture.
Architecture emulation in containers
The problem, as James Bottomley pointed out in this brief patch set, is that binfmt_misc has to locate and run the interpreter binary at the time that the foreign binary is invoked. This invocation happens within the container, so the interpreter has to be visible in the container as well, but, likely as not, the container is running within a namespace intended to keep it from seeing the rest of the system. As a result, the interpreter must be placed inside the container itself. That complicates what would otherwise be containerized system built entirely for the emulated architecture. It also forces any orchestration system to be aware of the emulation within the container and set things up accordingly, making emulated containers less transparent than they would otherwise be.
The solution is to add a new mode for binfmt_misc wherein the interpreter binary is opened by the kernel when the new format is initially set up. When a binary in that format is encountered, the already-opened interpreter can be run, rather than seeking out and opening the interpreter at that time. This mechanism will work inside a container that otherwise has no access to the interpreter; the kernel already has the interpreter open, so it can run it directly.
This mode is set up by using the new "F" flag when describing the format to binfmt_misc. Once the kernel has opened the interpreter file, it will keep it open until the format is removed. That means that updates to the interpreter binary will not take effect unless the format is removed and reestablished. That should not ordinarily be a problem, but it could be a surprise for system administrators who are not aware of this behavior.
The patch set received a small number of generally favorable reviews. If it is merged, as seems likely, it will make it easier to run containers built for a number of machine architectures on the same host, making Linux containers more flexible in general.
Patches and updates
Kernel trees
Architecture-specific
Build system
Core kernel code
Development tools
Device drivers
Device driver infrastructure
Filesystems and block I/O
Memory management
Networking
Security-related
Virtualization and containers
Miscellaneous
Page editor: Jonathan Corbet
Distributions
Subgraph OS, a new security-centric desktop distribution
Security-conscious Linux users have a new distribution offering to consider: Subgraph OS, a Debian-based operating system that implements system-hardening measures at a number of different levels. The kernel, filesystem, applications, and network configuration are all tweaked to isolate processes and connections and to safeguard the system against attackers. Although the project is a new one and not all of the promised features have been implemented yet, it appears to be well worth watching.
At the project's home page, Subgraph OS is billed as a "difficult to
attack
" operating system built with "system hardening and
a proactive, ongoing focus on security and attack resistance.
"
There is a set of pages documenting
the project's approach. That said, users should be aware that, at
present, the only installable releases are "pre-alpha" ISOs for x86_64
machines. The latest
is from February 22. Although the project describes Subgraph OS
as a lightweight offering (in particular, shipping only a small
assortment of applications), the ISO is a not-insignificant 1.5GB. It
can be run in live-DVD mode or be used to install Subgraph OS onto a
hard disk.
This release is based on the current Debian "testing" distribution (which will become Debian 9 "stretch"). Some of the application-level features are still unstable enough that they are likely to produce more crashes than successes, but there are many other interesting choices implemented in Subgraph OS that make it worth a look.
Security measures
The kernel is version 4.4.2, but with a few notable changes. First, the grsecurity patches have been applied; the project makes particular note of its usage of PaX to guard against memory-corruption vulnerabilities. The paxrat utility is used to set the PaX flags for individual applications; these flags control the use of secure memory protections, address space layout randomization (ASLR), trampoline emulation, and non-executable pages, among others.
The full kernel configuration used is available on GitHub, where several other changes are documented. Most notable is the deactivation of multiple network protocols (such as IPX, IrDA, X.25, and CAN Bus). IPv6 is among those deactivated protocols, which the notes on the GitHub page chalk up to the fact that Subgraph OS uses Tor for all network connections, and so far Tor does not sufficiently support IPv6. Tor's IPv6 support is an ongoing effort, and it is certainly true that few existing Tor nodes support IPv6—perhaps few enough to make relying on it a gamble.
Full-disk encryption using LVM and LUKS is mandatory in the Subgraph OS installer—the option to install without disk encryption is not even presented to the user. Network usage is restricted by two separate means. First, the Metaproxy redirector transparently redirects all network connections to Tor. Second, a firewall is configured to only permit a whitelisted set of applications to access the network. At present, the firewall rules must be manually adjusted to allow a new application access to the network, although the project advertises plans to integrate this process into the desktop. There are (currently non-functional) menu entries to monitor firewall connection requests and firewall settings, and the documentation cites plans to pop up a dialog box asking the user to confirm or deny network requests from non-whitelisted applications.
Interestingly enough, the Subgraph OS installer and the live-DVD session both start with all networking disabled; the user must even switch on wired networking through an option in the GNOME Shell system menu. There is also a built-in MAC address spoofer, which is run at start-up, can also be re-run at will, and can be configured to automatically run whenever a new network device is added.
Tor is, perhaps, a given for security-focused distributions these days. But Subgraph OS goes further than some. Each of the major applications (browser, email client, etc.) is isolated in a separate Tor circuit, and the GNOME Shell user menu includes a button to drop the basic Tor circuit (that is, the one connected through the general-purpose Tor endpoint to which other traffic is redirected by Metaproxy) and establish a new one. The Tor package also includes support for pluggable transports, which can be used to hide tor traffic within other connection types, thus providing better resistance to deep packet inspection.
Application security
At the application level, all of the major applications are started inside a sandboxing system that the distribution calls Oz. The system uses standard containment methods like namespaces, capabilities, and seccomp filters, plus the Xpra X11 forwarding server. On top of that, the latest ISOs have added AppArmor profiles for all "at risk" applications (a designation that includes network-capable applications plus any that might need to open files from an untrusted source—such as the PDF viewer or LibreOffice).
The Oz sandboxes are quite restrictive by default; applications cannot see any files in the user's home directory. To get around that access control, the user must use the Oz GNOME Shell extension. This provides a menu from which the user can manually grant access to a specific file, as well as perform some utility functions like opening a terminal within the sandbox.
The applications themselves are standard fare: Tor Browser, LibreOffice, the VLC media player, IRC and instant-messaging clients, a terminal emulator, and the Icedove email client. Interestingly enough, although a Tor Browser launcher is included on the panel, the first time it is launched it proceeds to download and install the latest Tor Browser release. This is a handy workaround for coping with how often Tor Browser receives important updates, although the initial wait time can be irritating.
The project site highlights a new email client in the works called Subgraph Mail, although it has not yet been released. Perhaps the most intriguing aspect of Subgraph Mail is that it is said to eschew traditional identity-verification systems like the OpenPGP "web of trust," and rely on a verification service run by Subgraph as a Tor hidden service. That service will certainly be worth investigating when it becomes available.
Looking forward
It is somewhat disappointing to see that the Subgraph project speaks of Subgraph Mail and the desktop firewall tools as current features when they are not yet available, but all free-software projects are subject to similar limitations in time and resources, so one must make allowances. Icedove will no doubt suffice for many early adopters, and the command-line firewall-control tools are there for those who truly need them.
There are other pressing issues to be dealt with before Subgraph OS is ready for mainstream adoption. Perhaps the biggest is that the distribution currently uses the Debian package repositories—although, happily, it does access them over Tor. Moving forward, the project will be managing its own repositories (no doubt periodically re-synchronizing with Debian); the web site notes that it plans to implement reproducible package builds as an additional security measure. In the long run, the project will have to address other issues—such as issuing security updates. Developing on top of Debian no doubt makes that an easier job, but not a trivial one.
In design, Subgraph OS is most similar to Tails, although Tails is explicitly designed to run in live mode from removable media. Another clear comparison could be made to Qubes OS, which is similarly security-focused and takes measures to isolate applications. But Qubes OS uses a distinctive system architecture, with the Xen hypervisor managing several virtual machines to hold various applications. Subgraph OS is a more lightweight solution, which might make it more appealing to users already familiar with desktop distributions like Debian and with the various sandboxing and containerization efforts underway in the community. When security is the goal, some additional competition is always welcome; Subgraph OS looks to be a well-thought-out distribution that the security-minded would do well to explore.
Brief items
Distribution quotes of the week
Looks like we finally have the proof that Debian helped a lot in the detection of gravitational waves :)
Announcing the KDE community's Distribution Outreach Program (KDE.news)
KDE.news has an announcement of a new program to foster better cooperation between KDE and distributions. "KDE is distro-agnostic. We do not prefer any distributions over others, and want our software to run everywhere. This extends beyond Linux; we want our software to work for our users on Windows, Mac, BSD and Android as well. Our focus is always on our users having the best experience possible. We are aware that the more closely we cooperate, the better the experience for all, including those who package our software, and we think that open and free communication is the best way to cooperate. KDE developers should be able to tell distributions what our software needs from a distribution in order to work best. And in turn, distributions should be able to tell us what makes our software easy to distribute." A new mailing list has been created to host these conversations.
Distribution News
Debian GNU/Linux
Debian "Stretch" release delayed slightly
The Debian "Stretch" release isn't expected for more than a year, but it just has been pushed back a couple of months, with the full freeze now scheduled for February 5 of next year. The reason is to be able to ship with the first kernel of the year (expected to be 4.10) that, by current plans, should be a long-term support release. "For the avoidance of doubt, this change is a one-off to align with an expected release of Linux only. We aren't in a position to try and accommodate other projects, however much we'd like to be able to."
Debian Project Leader Elections 2016: Call for nominations
Nominations are open for Debian Project Leader elections until March 12. "Prospective leaders should be familiar with the constitution, but just to review: there's a one week period when interested developers can nominate themselves and announce their platform, followed by a three week period intended for campaigning, followed by two weeks for the election itself."
Newsletters and articles of interest
Distribution newsletters
- DistroWatch Weekly, Issue 651 (March 7)
- openSUSE Tumbleweed – Review of the week (March 4)
- Tails report (February)
- Ubuntu Weekly Newsletter, Issue 457 (March 6)
elementary OS: A Distro that Dreams of Disrupting the Linux Desktop (Linux.com)
Swapnil Bhartiya takes a look at elementary OS on Linux.com. "elementary OS is trying to disrupt the desktop Linux world in a couple of ways. It’s aim is to create a distribution that approaches the desktop from a design point of view instead of the crude “there-is-a-command-for-everything” approach that’s popular in the Linux world. So far, the company has been successful in achieving this goal. The latest release of elementary OS, Freya, is one of the most polished and good-looking distributions I’ve seen. One reason behind such this design-centric focus is that the founder of the project comes from a design background."
If You Like Fedora, You'll Love Korora (LinuxInsider)
LinuxInsider reviews the Fedora remix, Korora 23. "Korora's use-it-out-of-the-box philosophy is one of the reasons the distro keeps getting better. If you want a better, more user-friendly Linux distro that reaches beyond Fedora's enterprise appeal, you can't go wrong with any of Korora's five desktop versions. It leaves little for users to desire and makes choosing another distro unnecessary to get your preferred interface. Korora stays true to its mission. It promised an easier Linux for new users without sacrificing either power or features for seasoned Linuxers."
Page editor: Rebecca Sobol
Development
Improving writing with proselint
Here at LWN, we put a lot of effort into the quality of our writing, so our attention is naturally drawn to tools that might help the writing process. Recently, links to the proselint tool made the rounds, so it made sense to take a look. What we found was a tool that shows some potential, but quite a bit of work will be needed to get it to where it aspires to be.What is this project's goal? The word from the front page of its web site is:
Installation of proselint is simple; like many Python utilities, it seems to have bypassed the ordinary distribution channels and can be installed directly with pip. It is a command-line tool; running it on a text file will, by default, result in any warnings being written to the standard output. This is clearly not the expected use, though; it is more likely that proselint will be integrated into other tools that can present its advice in a more friendly manner. To help make this integration easy, proselint can be asked to format its output as JSON rather than plain text.
As a result, it is easy to use proselint as part of the Emacs Flycheck mode, for example; it is a simple matter of adding a few lines to the .emacs file. Once a buffer has been put into flycheck-mode, the text therein will be occasionally checked for problematic English, and the offending text will be highlighted; hovering over that text with the pointer will yield the associated message.
Your editor went searching for some high-quality text to test this tool against. After pondering the classics from Project Gutenberg, the GNU Manifesto, and Trump speech transcripts, the search settled a little closer to home — the kernel's documentation directory. Playing with the Flycheck integration in Emacs quickly yielded results like this:
That is good advice indeed, but, in truth, most of what comes out of
proselint is less helpful. By far the bulk of the warnings are either
(1) using both one and two spaces after a period, or (2) failure
to use curly quotes. Neither warning is all that helpful for a writer who
was hoping for, as the web site promises, "the world’s greatest
writers and editors by your side, where they whisper suggestions on how to
improve your prose
". Instead, it feels a bit more like one of the
world's worst elementary-school teachers criticizing you in front of the
entire class about irrelevant details.
But, then, proselint is a new tool, so there is hope for the future. The success of such a tool depends on two things: the rules it uses to analyze text, and the flexibility of the engine that interprets those rules. The rules themselves are written as Python code; here, for example, is the core of the rule that generated the message shown above:
@memoize
def check_very_damn(text):
err = "twain.damn"
msg = ("Substitute 'damn' every time you're "
"inclined to write 'very;' your editor will delete it "
"and the writing will be just as it should be.")
regex = "very"
return existence_check(text, [regex], err, msg, max_errors=1)
There are various other functions for performing checks on text, but they all have the same core nature: they are applying regular expressions to the text as a way of triggering rules. That makes rules relatively easy to write, but also limits how smart the tool can actually be. It can readily pick out specific words or combinations of words, but will be hard pressed to offer advice that requires a deeper understanding of the subject text.
A look at the code shows that some thought has been put into making the core engine smarter; there are stubs for functions that can extract specific amounts of context or try to determine what the topic of a range of text is, for example. But stubs is all they are at this point; a lot of work will need to be filled in before this tool can come close to the ambitious goals its developers have set.
One can only wish them luck. There is a lot of advice for writers out there, but it can be hard to even read it all, much less keep it all in mind when actually writing or editing a piece of text. If it were possible to distill a significant part of that advice into a form where it could be applied automatically, the result might be better writing all around. That is a result that one could be damn happy about.
Brief items
Quotes of the week
Mozilla unveils Firefox OS based IoT projects (LinuxGizmos.com)
Over at LinuxGizmos, Eric Brown notes some new "Internet of Things" (IoT) projects from Mozilla that were described in a recent blog post by Ari Jaaksi, Mozilla Senior VP for Connected Devices. "The first projects include a Project Start Home framework for a home automation system, as well as a Project Link personal user agent and Vaani voice interface that would work within such a framework. Finally, there’s a crowdsourced Project SensorWeb for tracking air pollution. Interestingly, the term “Firefox OS” is not used in the latest announcement, despite the reference to Firefox OS Connected Devices in the previous post. Still, all the projects appear to use Firefox OS or Mozilla’s underlying Boot to Gecko (b2g) codebase. Mozilla is seeking testers, developers, and advisers, for all these open source projects."
ownCloud 9.0 released
Version 9.0 of ownCloud has been released, with many performance improvements, bug fixes, cleanup, and new features. "[Full Federation] is one of the main goals of ownCloud, since the beginning, to enable everyone to run their own server but still collaborate and share with others. Sharing between different ownCloud servers is possible for a while now. But this is easier than ever with cross-server user name auto complete, trusted servers and more. Once you’ve shared with another ownCloud server, it will be added as a trusted server, exchanging user names. This will enable ownCloud to auto complete names from users of all shared servers. Admins have control over these features, so they can add trusted servers manually and disable the automatic addition."
LLVM 3.8 released
Version 3.8 of the LLVM compiler suite has been released. "This release contains the work of the LLVM community over the past six months: deprecated autoconf build, shrink-wrapping on by default, overhauled MSVC-compatible exception handling, updated Kaleidoscope tutorial, emutls, OpenMP supported by default, as well as improved optimizations, many bug fixes, and more." See the LLVM release notes and the Clang release notes for lots of details.
Firefox 45.0 released
Firefox 45.0 has been released. This release features instant browser tab sharing through Hello, Tabs synced via Firefox Accounts from other devices are now shown in dropdown area of Awesome Bar when searching, Synced Tabs button in button bar, introduces a new preference (network.dns.blockDotOnion) to allow blocking .onion at the DNS level, and more. See the release notes for details.
Newsletters and articles
Development newsletters from the past week
- What's cooking in git.git (March 4)
- LLVM Weekly (March 7)
- OCaml Weekly News (March 8)
- OpenStack Developer Digest (March 4)
- Perl Weekly (March 7)
- PostgreSQL Weekly News (March 6)
- Python Weekly (March 3)
- Ruby Weekly (March 3)
- This Week in Rust (March 7)
- Tahoe-LAFS Weekly News (March 7)
- Wikimedia Tech News (March 7)
Borg, Omega, and Kubernetes (ACM Queue)
Five Google developers share the lessons from ten years of container development in this ACM Queue article. "To cope with these kinds of requirements, configuration-management systems tend to invent a domain-specific configuration language that (eventually) becomes Turing complete, starting from the desire to perform computation on the data in the configuration (e.g., to adjust the amount of memory to give a server as a function of the number of shards in the service). The result is the kind of inscrutable 'configuration is code' that people were trying to avoid by eliminating hard-coded parameters in the application's source code. It doesn't reduce operational complexity or make the configurations easier to debug or change; it just moves the computations from a real programming language to a domain-specific one, which typically has weaker development tools (e.g., debuggers, unit test frameworks, etc)."
Page editor: Nathan Willis
Announcements
Brief items
Microsoft announces SQL Server for Linux
For all of you who have been wanting SQL Server for Linux, the long wait is over, or will be around the middle of 2017 (a preview is available now). "'SQL Server’s proven enterprise experience and capabilities offer a valuable asset to enterprise Linux customers around the world,' said Paul Cormier, President, Products and Technologies, Red Hat. 'We believe our customers will welcome this news and are happy to see Microsoft further increasing its investment in Linux.'"
MAME is now Free and Open Source Software
The MAME (Multiple Arcade Machine Emulator) project has announced a license change,
moving from the old, unique "MAME License" to the GNU GPLv2-or-later
for the full codebase, with many individual components available under
the 3-clause BSD License. The announcement notes that a considerable
effort went into the relicensing process: "We have spent the
last 10 months trying to contact all people that contributed to MAME
as developers and external contributors and get information about
desired license.
" The old license
[Wayback link] had prohibited commercial sale and use.
FSF submits comment to Copyright Office with over 1200 co-signers
The Free Software Foundation submitted a comment along with over 1200 co-signers to the U.S. Copyright Office in response to its *Section 1201 Study: Notice and Request for Public Comment* in regards to the Digital Millennium Copyright Act's (DMCA) anti-circumvention provisions. "The FSF's comment detailed the damaging effects of DRM on all users, and called for the repeal of the DMCA's anti-circumvention provisions and to bring an end to the exemptions process. The comment explained how DRM is about the restriction of computer-users, not enforcing rights. DRM does not respect the rights of users, and gives the DRM implementer a degree of power not envisioned even under copyright law. It permits companies and governments to spy on users, prevent them from controlling their own computing, and abrogates users' rights to even legally permitted copying and modification of copyrighted materials. It interferes with the work of academics and librarians, preventing them from studying and archiving works. It prevents security researchers from discovering vulnerabilities and fraud, as was the case when Volkswagen used software to defeat emission control tests."
FSF: Stop DRM in HTML
The Free Software Foundation's Defective by Design campaign targets "media and proprietary software companies who want to weave Digital Restrictions Management into the HTML standard that undergirds the Web." The DRM proposal, known as EME (Encrypted Media Extensions), will be discussed by World Wide Web Consortium March 20-22. People are urged to sign a petition and/or visit a W3C office in your country.
Articles of interest
Open source math software competes in the classroom (Opensource.com)
William Stein introduces SageMath, a mathematical software system for researchers, teachers, computer programmers, and engineers. "I wanted SageMath to be a powerful tool for my students. It wasn't initially intended to be something hundred of thousands of people used! But as I began building the project, and as more professors and students started contributing to it, I realized these were problems many others were striving to solve as well. SageMath was desperately needed, and that wide interest became the driving force behind getting it up and off the ground. Over 500 contributors have participated and help make SageMath a real solution [available] to students and teachers around the world."
When selling a site means selling a community (Opensource.com)
In the first of two parts Opensource.com talks with Frank Karlitschek about the sale of his network of more than 30 community sites to Blue Systems. "Although he doesn't know much about the future plans Blue Systems has for the websites, Karlitschek says the new owner is interested in continuing to run the websites and develop them in a direction that makes sense for the users and the ecosystem."
Part 2 looks
at the SourceForge and Slashdot communities and how the different
owners have interacted with them. "[Logan] Abbott got off on the
right foot with open source developers with his SourceForge post in early
February, SourceForge Acquisition and Future Plans. "Our first order of business was to terminate the 'DevShare' program," he wrote. " ... We want to restore our reputation as a trusted home for open source software, and this was a clear first step towards that. We're more interested in doing the right thing than making extra short-term profit," he added.
"
Calls for Presentations
CFP Deadlines: March 10, 2016 to May 9, 2016
The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.
| Deadline | Event Dates | Event | Location |
|---|---|---|---|
| March 10 | May 14 May 15 |
Open Source Conference Albania | Tirana, Albania |
| March 12 | April 26 | Open Source Day 2016 | Warsaw, Poland |
| March 15 | April 28 May 1 |
Mini-DebCamp & DebConf | Vienna, Austria |
| March 20 | April 28 April 30 |
Linuxwochen Wien 2016 | Vienna, Austria |
| March 25 | July 11 July 17 |
SciPy 2016 | Austin, TX, USA |
| April 1 | May 26 | NLUUG - Spring conference 2016 | Bunnik, The Netherlands |
| April 2 | May 2 May 3 |
PyCon Israel 2016 | Tel Aviv, Israel |
| April 7 | April 8 April 10 |
mini Linux Audio Conference 2016 | Berlin, Germany |
| April 8 | August 2 August 5 |
Flock to Fedora | Krakow, Poland |
| April 15 | June 27 July 1 |
12th Netfilter Workshop | Amsterdam, Netherlands |
| April 15 | June 22 June 26 |
openSUSE Conference 2016 | Nürnberg, Germany |
| April 24 | August 20 August 21 |
Conference for Open Source Coders, Users and Promoters | Taipei, Taiwan |
| April 26 | August 22 August 24 |
LinuxCon NA | Toronto, Canada |
| April 27 | August 12 August 14 |
GNOME Users and Developers European Conference | Karlsruhe, Germany |
| April 30 | June 11 | TÜBIX 2016 | Tübingen, Germany |
| April 30 | October 27 October 28 |
Rust Belt Rust | Pittsburgh, PA, USA |
| April 30 | August 25 August 26 |
The Prometheus conference | Berlin, Germany |
| May 1 | August 24 August 26 |
KVM Forum 2016 | Toronto, Canada |
| May 2 | June 24 June 25 |
devopsdays Silicon Valley 2016 | Mountain View, CA, USA |
| May 6 | October 26 October 27 |
All Things Open | Raleigh, NC, USA |
| May 6 | July 13 July 15 |
LinuxCon Japan | Tokyo, Japan |
| May 8 | August 12 August 16 |
PyCon Australia 2016 | Melbourne, Australia |
If the CFP deadline for your event does not appear here, please tell us about it.
Upcoming Events
EuroPython 2016: Financial Aid Available
The EuroPython team has announced a financial aid program to help more people attend the conference. EuroPython will take place July 17-24 in Bilbao, Spain.Events: March 10, 2016 to May 9, 2016
The following event listing is taken from the LWN.net Calendar.
| Date(s) | Event | Location |
|---|---|---|
| March 8 March 10 |
Fluent 2016 | San Francisco, CA, USA |
| March 9 March 11 |
18th German Perl Workshop | Nürnberg, Germany |
| March 10 March 12 |
Studencki Festiwal Informatyczny (Students' Computer Science Festival) | Cracow, Poland |
| March 11 March 13 |
Zimowisko Linuksowe TLUG | Puck, Poland |
| March 11 March 13 |
PyCon SK 2016 | Bratislava, Slovakia |
| March 14 March 17 |
Open Networking Summit | Santa Clara, CA, USA |
| March 14 March 18 |
CeBIT 2016 Open Source Forum | Hannover, Germany |
| March 16 March 17 |
Great Wide Open | Atlanta, GA, USA |
| March 18 March 20 |
FOSSASIA 2016 Singapore | Singapore, Singapore |
| March 19 March 20 |
LibrePlanet | Boston, MA, USA |
| March 19 March 20 |
Chemnitzer Linux Tage 2016 | Chemnitz, Germany |
| March 23 | Make Open Source Software 2016 | Bucharest, Romania |
| March 29 March 31 |
Collaboration Summit | Lake Tahoe, CA, USA |
| April 1 | DevOps Italia | Bologna, Italy |
| April 4 April 6 |
OpenIoT Summit | San Diego, CA, USA |
| April 4 April 8 |
OpenFabrics Alliance Workshop | Monterey, CA, USA |
| April 4 April 6 |
Web Audio Conference | Atlanta, GA, USA |
| April 4 April 6 |
Embedded Linux Conference | San Diego, CA, USA |
| April 5 April 7 |
Lustre User Group 2016 | Portland, OR, USA |
| April 6 | PostgreSQL and PostGIS, Session #8 | Lyon, France |
| April 7 April 8 |
SRECon16 | Santa Clara, CA, USA |
| April 8 April 10 |
mini Linux Audio Conference 2016 | Berlin, Germany |
| April 9 April 10 |
OSS Weekend | Bratislava, Slovakia |
| April 11 April 13 |
O’Reilly Software Architecture Conference | New York, NY, USA |
| April 15 April 17 |
Akademy-es 2016 | Madrid, Spain |
| April 15 April 17 |
PyCon Italia Sette | Firenze, Italia |
| April 15 April 18 |
Libre Graphics Meeting | London, UK |
| April 16 | 15. Augsburger Linux Info Tag | Augsburg, Germany |
| April 18 April 19 |
Linux Storage, Filesystem & Memory Management Summit | Raleigh, NC, USA |
| April 18 April 20 |
PostgreSQL Conference US 2016 | New York, NY, USA |
| April 20 April 21 |
Vault 2016 | Raleigh, NC, USA |
| April 21 April 24 |
GNOME.Asia Summit | Delhi, India |
| April 23 April 24 |
LinuxFest Northwest | Bellingham, WA, USA |
| April 23 | DevCrowd 2016 | Szczecin, Poland |
| April 25 April 27 |
Cuba International Free Software Conference | Havana, Cuba |
| April 25 April 29 |
OpenStack Summit | Austin, TX, USA |
| April 26 | Open Source Day 2016 | Warsaw, Poland |
| April 26 April 28 |
Open Source Data Center Conference | Berlin, Germany |
| April 28 April 30 |
Linuxwochen Wien 2016 | Vienna, Austria |
| April 28 May 1 |
Mini-DebCamp & DebConf | Vienna, Austria |
| April 30 | Linux Presentation Day 2016.1 | many cities, Germany |
| May 1 June 29 |
Open Source Innovation Spring | Paris, France |
| May 2 May 3 |
PyCon Israel 2016 | Tel Aviv, Israel |
| May 2 May 5 |
FOSS4G North America | Raleigh, NC, USA |
If your event does not appear here, please tell us about it.
Page editor: Rebecca Sobol
