LWN.net Weekly Edition for October 13, 2016
A tale of two conferences
Your editor just spent two weeks in Europe attending two technical conferences. Numerous articles from specific sessions appear in this week's edition, with a few more yet to come. But there is value in pondering for a moment on the nature of the two events themselves; both have some things to say about where our community is going.
LinuxCon Europe
LinuxCon Europe was held in Berlin this year, colocated with ContainerCon and who knows what else (it's worth noting that these conferences will soon be rebranded into the "Open Source Summit"). It was a large event, with well over 1,000 attendees. As is generally the case with Linux Foundation events, it was impeccably organized and generally busy, with nearly a dozen tracks running simultaneously.
Early LinuxCon events featured a fair amount of low-level technical content, and there was a strong focus on kernel-related topics. Kernel developers were well represented in the audience and tended to receive a fair amount of attention. At LinuxCon Europe 2016, instead, there was no kernel panel, relatively little kernel-related content, and it seems fair to say that most of the attendees didn't know who the few kernel developers present at the event were. Their attention was elsewhere.
Eleven of the talks at this event featured "orchestration" in their titles; 39 mentioned containers, 30 mentioned Docker, and there were nine on Kubernetes. It seems pretty clear that containers are where a lot of the action is at the moment. Perhaps this is a way of saying that, to a great extent, the problems at the lower levels of the system have been solved, so the interesting things to work on are higher up the stack. Or, at least, there is perceived to be more money higher up the stack.
Meanwhile, 17 talks mentioned the kernel, so all is not lost for those of us who are more drawn to lower-level code. One high point was the "Outreachy internship report", where Shraddha Barke, Ioana Ciornei, Cristina Moraru, Ksenija Stanojević, and Janani Ravichandran presented the work they had done during their internships. It was an exercise in optimistic and youthful development energy in general. But your editor also realized, while sitting in the room, that he had never before been in the presence of that many female kernel developers at the same time. That is a sad reflection of the state of the kernel development community, but also a hopeful sign that, maybe, things can get better.
All told, LinuxCon was an intensive and interesting event and a worthwhile snapshot of where at least a part of our community is heading. But, while one is in the middle of the crowds, the overtly commercial keynotes, the show floor, etc., it's sometimes hard not to miss the kind of event we used to have. Linux conferences were once less slick and more focused on the code. What we have in LinuxCon is good, but there is more to our community than what is on offer there.
Kernel Recipes
The week prior to LinuxCon, your editor was fortunate enough to attend Kernel Recipes in Paris. This is the fifth year that this event has been run, but the first time that LWN has been able to be there. This event is a breath of fresh air for anybody who finds the LinuxCon scene to be a bit overwhelming at times.
Kernel Recipes is, at its heart, a three-day gathering for developers to sit down and talk about kernel-related topics. While LinuxCon had a dozen tracks, Kernel Recipes features exactly one. Everybody is in the same room, and, from what your editor saw, all the attendees made a point of being there for the entire event. The conference has a limit of 100 attendees — a limit that was hit in less than two days this year. Each session was a discussion, with wide participation throughout the room. The overall level of engagement was high.
It seems certain that new and interesting work will be inspired by the discussions that happened here. So in that sense, if no other, Kernel Recipes must be seen to be a successful event. The keys to this kind of success would appear to be keeping the size small and a relentless focus on bringing in high-quality talks. Your editor, who sometimes needed prodding to confirm and prepare for his presence there, can attest to the relentless part.
The "small" criterion can be a bit of a problem since it, naturally, limits the number of people who can participate in this kind of event. The Linux Plumbers Conference (now just a few weeks away) is always trying to find the right balance between size and quality of the event, and there, too, tickets tend to sell out quickly. The nice thing about an event like Kernel Recipes, though, is that it ought to be reproducible in other parts of the world. We have a ready supply of good speakers and interesting things to talk about in our community, and it doesn't take that many speakers to make an event like this work.
In the end, it was a privilege to be able to attend both events. Your editor's only regret was being unable to stay in Berlin for the Embedded Linux Conference Europe the following week. Conferences are an opportunity to get a sense for what is happening in our community and to renew one's enthusiasm and energy; both LinuxCon and Kernel Recipes succeeded on all of those fronts. A diverse community needs a diverse range of events; happily, that is just what was in store in Europe during these weeks.
An introduction to color spaces
The Kernel Recipes conference is, unsurprisingly, focused on kernel-related topics, but one of the potentially most useful talks given there was only marginally about the kernel. Applications that deal with the acquisition or display of video data must be aware of color spaces, but few developers really understand what color spaces are or how they work. Media subsystem maintainer Hans Verkuil sought to improve this situation with an overview of the color-space abstraction.His slides started with the v4l2_pix_format structure, which describes the pixel format of the data returned by a video capture device (such as a webcam). The colorspace member of that structure, in particular, identifies the color space in which the pixel data is expressed. Developers of applications (and drivers) for media devices must specify an appropriate color space, but few developers, Verkuil said, understand what that field really means.
Color, he said, can be thought of as a signal consisting of light at one or more specific frequencies and powers. The human eye, though, does not detect all of those frequencies directly. Instead, it has three types of "cones" with sensitivities centered around three specific frequencies — nominally red, green, and blue. Light at a specific frequency and power will generate a certain level of signal from each type of cone; the three-value tuple that results is how color is signaled to the brain. As it happens, there is an infinite set of frequency/power distributions that can result in the same three values. Reproducing a color, as far as the brain is concerned, is just a matter of reproducing a specific color tuple. Photographs and video displays take full advantage of that fact; the colors they produce are not the original colors, but they are able to fool the eye into seeing the the original colors.
Color spaces
When dealing with visual data, we need a way to uniquely identify colors; that is where a color space comes in. Back in the 1920s, the CIE (Commission Internationale de L'Éclairage) performed a set of studies mapping wavelengths onto the RGB values that replicate them. Those values became the CIE RGB color space. A simple linear transformation turns CIE RGB into the CIE XYZ color space, which has a couple of practical advantages: it allows all colors to be represented using positive values, and the Y value describes the overall brightness (luminance) of the color. Among other things, it turns the range of possible colors (at a given luminance) into a two-dimensional quantity. All other color spaces are based on CIE XYZ — which was developed in the 1920s from measurements on a pool of 17 people.
In general, a color space defines three "primaries" that can be thought of
as the red, green, and blue colors, though they don't always correspond to
those colors. Each color space also has a "white point" describing the
![[Hans Verkuil]](https://static.lwn.net/images/conf/2016/recipes/HansVerkuil-sm.jpg "Hans Verkuil")
maximum output value for each primary. Once upon a time, color spaces
corresponded to the physical properties of the phosphors found in CRT
screens, but that is no longer the case.
Color spaces as described thus far are linear, with values corresponding directly to the light levels of the primaries. The human eye does not respond to light linearly, though; a doubling of the light level does not look twice as bright. If (say) eight bits are used to represent a primary value in a color space, many of the 256 available values will be wasted on tiny differences between the brightest values, while the resolution is too coarse at the dim end of the scale. So colors are often represented in nonlinear color spaces that better match how the eye responds.
If a linear color space has RGB values, then a nonlinear equivalent can be obtained by applying a "transfer function" yielding a new set of values, called R'G'B'. The primes should be used for nonlinear color spaces, but almost everybody leaves them out, with the result that nobody ever knows which kind of color space is being talked about. The transfer function is often called a "gamma function", Verkuil said, but that is not quite correct. The screen will typically apply an inverse transfer function to color values to get the actual intensities to display; needless to say, the transfer function and its inverse need to match or colors will not be displayed correctly. OpenGL programmers need to be aware that textures use linear RGB values by default, not nonlinear R'G'B'.
Video applications often deal with colors in the Y'CbCr (or YUV) "color space", but it is not actually a separate color space. Y'CbCr is derived directly from R'G'B' via a matrix multiplication; it is simply a different representation for the same color space. Even so, it seems that a color space can define its own matrix (or even more than one) for this transformation.
Verkuil went quickly through some of the more prominent standards in this area.
- The Rec. 709
color space is for high-definition television; it is, he said, "nicely
done."
- The best-known color space, perhaps, is sRGB, which is typically used
for computer graphics. It has the same chromaticities (primaries) as
Rec. 709, but the transfer function is different.
- SMPTE 170M is the
color space for standard-definition TV; it has the same transfer
function as Rec. 709, but the chromaticities are different.
- BT.2020 is for ultra-high-definition television with at least ten bits for each color component. There are two separate Y'CbCr encodings defined for this color space.
Once the color space and encoding have been figured out, there is one more complication in the form of limited-range encoding. Normally, eight-bit R'G'B' color values use the full 0..255 range, but colors in the Y'CbCr encoding are compressed to fit in the narrower 16..235 range. Limited-range R'G'B' does exist in the wild, though, as does full-range Y'CbCr. The limited-range encoding is a holdover from the old analog television days, when the margin at either end was needed to handle errors. Everything is digital now, but we are still stuck with limited range encoding in a number of situations. Some transports use the out-of-range values as sentinel values for in-band signaling.
After spending a lot of time gaining a better understanding of color spaces, Verkuil added some additional fields to the v4l2_pix_format structure:
__u32 ycbcr_enc; /* enum v4l2_ycbcr_encoding */
__u32 quantization; /* enum v4l2_quantization */
__u32 xfer_func; /* enum v4l2_xfer_func */
These fields describe which Y'CbCr encoding is in use (if any), whether full-range or limited-range quantization is in use, and which transfer function has been applied. Now it is possible for user space to learn everything it needs to handle color spaces correctly — but user-space developers still ignore it all, he said. There is one exception to that, actually: the GStreamer developers have worked hard to get their color-space handling right.
When things go wrong
What happens if you don't put in that effort and don't get things right? There are a number of problems that can result, and it's not all the developers' fault. The names for the color spaces are confusing (CIE XYZ is not the same as CIE xyz or CIE Yxy), conversion matrices can be buggy, and, of course, there is the full range of exciting surprises that originate in hardware implementations.
One thing that often goes wrong is a confusion between SMPTE 170M and Rec. 709. These two color spaces have different primaries, leading to slight color differences. Those differences are indeed slight, though, to the point that only an expert is likely to notice them; most developers can safely ignore this particular difference. It is slightly visible on LCD screens, but completely disappears when projectors are in use.
Things go a bit further amiss when the Rec. 709 and sRGB transfer functions are confused. The differences here are more noticeable, especially toward the black end of the scale. Using the wrong Y'CbCr encoding is quite a bit more obvious; that's something that customers will notice. Using limited-range quantization when full-range is expected (or the reverse) is also quite evident. This one tends to manifest itself when somebody is displaying an Excel spreadsheet; the slight color difference between adjacent rows will vanish if a full-range signal is interpreted as limited-range.
The slides from the presentation give examples of the visible differences resulting from the above problems. The media subsystem documentation has information for developers wanting to learn more about using color spaces with video acquisition devices.
[Your editor would like to thank Kernel Recipes for supporting his travel to the event.]
OpenSSL after Heartbleed
Rich Salz and Tim Hudson started off their LinuxCon Europe 2016 talk by stating that April 3, 2014 shall forever be known as the "re-key the Internet date." That, of course, was the day that the Heartbleed vulnerability in the OpenSSL library was disclosed. A lot has happened with OpenSSL since that day, to the point that, Salz said, this should be the last talk he gives that ever mentions that particular vulnerability. In the last two years, the project has recovered from Heartbleed and is now more vital than ever before.
Ever since Heartbleed, every severe vulnerability has had to feature its
own catchy name, web site, and logo. It is good, in that it has forced
security researchers to develop some artistic sense. Seriously, though,
![[Rich Salz]](https://static.lwn.net/images/conf/2016/lceu/RichSalz-sm.jpg "Rich Salz")
short names for vulnerabilities are useful. Saying that a system "is
susceptible to Heartbleed" makes sense to almost everybody. That is much
less true if one talks about "susceptibility to CVE-whatever" instead.
Heartbleed was also the first general defect that made it onto the front
pages of the mainstream press. Salz suggested that it may end up
outlasting the Kardashians.
At its core, Heartbleed was a simple bug, a missing buffer-length check. That resulted in the disclosure of some arbitrary memory that could be used to extract session and key information from a system running OpenSSL. This bug had been in place for three years when it was disclosed; nobody had managed to see it. The OpenSSL developers missed it, as did external security reviewers and code analysis tools (though, Salz noted, those tools all caught it three days after its disclosure). It was an ordinary bug, not as bad as some of its contemporaries ("goto fail" for example), but it got a lot of attention, perhaps because it, unlike many bugs, affected both server and client systems.
OpenSSL asleep
At the time of Heartbleed, the OpenSSL project had gone into a nearly moribund state. There were no policies for dealing with issues, and releases were not announced. There was no recognition that users might have to respond to releases containing security fixes. The source was complex and arcane; it "made procmail look pretty." There were almost no comments; indeed, there are still almost no comments, though it is starting to get better. The code was hard to maintain and hard to contribute to, especially for developers in the US due to crypto export issues. There were two overworked developers doing almost all of the work and making little money in the process; donations to the project were less than $2,000/year.
How did things get to this point? The project had put almost no time into building its community. It used mailing lists based on an old Majordomo server; they were unsearchable among other problems. Maintainership was static, with a great deal of wariness around the idea of letting anybody else in. The existing developers were strongly driven by the need to obtain consulting money, which caused them to focus on FIPS 140 certification. The project had no ability to make, much less adhere to, plans. The developers had fallen into a mindset that the easiest way to avoid breaking things was to not change them in the first place, so changes of any type were discouraged.
The advent of Heartbleed led immediately to a lot of negative feedback and hard questions. How can the project be trusted after such a vulnerability? When will the next one hit? These are, Salz said, good questions to ask.
After Heartbleed
One of the first after-effects of the Heartbleed disclosure was a wider recognition of the existence of critical, but underfunded projects. In response, the Linux Foundation launched the Core Infrastructure Initiative (CII) as a way of getting resources to those projects. That has helped OpenSSL, among others, to get back into a healthy state.
Before April 2014, OpenSSL had two primary developers, both of whom were volunteers, and no decision-making process. As of December of that year, the project had 15 members, two of whom are paid full-time by CII and two others who are paid from donations made directly to the project. There is a set of "very formal" decision processes in place now.
The group had its first in-person meeting that year in Düsseldorf, allowing
them to get to know each other — a critical part of making a group
functional. They drafted a set of major policies, covering release
![[Tim Hudson]](https://static.lwn.net/images/conf/2016/lceu/TimHudson-sm.jpg "Tim Hudson")
strategies, security, coding style, and more. The project has become much
more transparent, in contrast to the insularity that prevailed before. Now,
Salz said, only the OpenSSL data structures are opaque. Much of the
project's work runs on GitHub now; email traffic on the lists has increased
and has also become much more useful.
In 2016 (so far), the project has seen 3,246 commits. It has put out one major release, 15 bug-fixing releases, and responded to 29 CVE numbers. In general, it shows a lot more activity from many more people and has a far better sense of community. Developers have gotten serious about closing issues reported against the project which, Salz said, is an important step toward building a community. If issues go without responses, users will eventually stop reporting them. The goal is to respond to all reports within four days; the actual response time remains about three times that number, but it's getting closer.
The support policies for OpenSSL release are now well defined. The 1.1.0 release will receive support through September 30, 2018, while 1.0.2, the project's first long-term supported release, will be maintained through the end of 2019. Support for 1.0.1 is limited to security fixes now, and will end entirely at the end of 2016. All older versions are unsupported now. It is important, they said, to drop support for older releases, giving the project the resources to move forward.
Along with more development, the project has more people who are actively looking for problems. There is more fuzz testing going on, and the static analysis tools have been updated; indeed, there is more automated testing in general. A recently added tool will modify the code to invert the sense of the conditions in if statements to see whether the test suite catches the resulting bug; if it doesn't, there is a coverage gap in the test suite. All code must be formally reviewed before being committed.
Future plans and lessons learned
There is now a posted project roadmap to work from. Currently, some 1,100 forks of the project exist on GitHub. At the top of the list for future development is support for TLS 1.3. The project plans to move to the Apache2 license, though they couldn't say just when that will happen. More testing is in the plans.
Then there is the ongoing issue of FIPS 140 certification. This certification is mandatory for most vendors selling software to the United States government; meeting this need essentially funded the project for five years. The validation process is, they said, a time-consuming and irritating thing to have to go through; it takes a couple of years and is expensive. It is an ongoing process as well, in order to keep up with changes in the software; FIPS 140 certification is not something that can be done once and forgotten about.
The OpenSSL FIPS 2.0 certification module applies to the 1.0.x release series. There is a new FIPS module coming, supported by SafeLogic; it will work with the 1.1.0 release. In general, the project is working to make the FIPS-related changes less intrusive in the code base as a whole.
What was learned from the whole Heartbleed experience? No project can rely on any one individual to continually perform superhuman feats; that will always lead to disappointment in the end. Code reviewers have to actually look at the code; they can't just hope the the community will somehow perform a proper review. A project also cannot rely just on tools, they will never do the full job. Proper code review takes a lot of time, and it needs to be done by experienced developers.
Needless to say, OpenSSL is looking for contributors. Beyond contributing patches, interested developers can test the pre-releases, report bugs, and help to close bugs. The presenters concluded by saying that they would like users to get in touch, especially those who are distributing OpenSSL further downstream.
Security
Spam reduction with greylisting
Running your own email server involves trying to keep back the tide of unsolicited email often known as spam, and a good way to do this is to understand that most spam comes from botnets rather than real mail servers. Greylisting is a lightweight and elegant technique for distinguishing between the two sources of email and, when we look at the numbers, it turns out to be surprisingly effective.A common body of opinion says that there is no good reason to run your own mail server. Organizations such as Google will, for not much money, allow you to run your entire corporate email as a hosted service. You can use the mail client of your choice, thanks to services like IMAP and SMTP AUTH decoupling the client experience from the back-end provider; there's also an excellent webmail interface for when you're out and about. Nevertheless, my experience is that, sooner or later, some shortcoming proves to be an inconvenience to the business. Perhaps the webmail site is blocked by some client network you're visiting, or important list emails aren't reaching your inbox and you can't work out why. Maybe the hosting provider has made some arbitrary decisions about content and some of your new employees find their names are unacceptable and they cannot sign up. When this happens, and if your provider is unhelpful or unresponsive, you will find yourself revisiting the decision to run your own server.
Should you decide to run your own, however, you will lose all the beneficial content filtering that your provider was doing — specifically, slimming the torrent of unsolicited commercial email that bedevils us all. There are many content filters, some commercial but many free, which can be inserted into the delivery pipeline of most mail servers to examine each incoming mail. This, however, is the point where you discover that the internet can deliver rubbish to your server more quickly than you can easily and cheaply scan it, and the departmental budget is carefully examined to see if it can accommodate a couple of big compute servers to run the spam filtering.
The cheapest, easiest tests are those that are done during the early phases of the SMTP transaction, specifically the EHLO, MAIL FROM, and RCPT TO phases, before the DATA phase is entered and the body of the email is transmitted. Any rejection that can be done at this early stage is cheap, both in network terms (because the message has not yet been transmitted) and in compute terms (because processing the few tens of bytes the client has thus far sent generally requires less hardware than processing an 8MB MIME-encoded email). There are tests that can be done at the EHLO phase, such as those relating to the resolvability and validity of the client hostname, and at the MAIL FROM phase, such as checking the client IP address against any Sender-Policy-Framework records advertised for the sender's declared domain name. Other tests can be done here, such as realtime blackhole list (RBL) lookups; I do not use those, since I've found no RBL to be sufficiently reliable for use as a bright-line test for acceptance (rather than as contributors to a total spam score in the content-based filtering system that comes later in the pipeline).
But at this stage you have three pieces of information — the sender's IP address, and the envelope sender and recipient addresses — so another elegant test can now be applied. To understand this test, it is necessary to appreciate that much of the spam email on the internet does not come from expensively-hosted bulk email senders, though those do exist. Instead, perhaps as much as 85% comes from botnets — distributed networks of compromised computers, each under the control of a single malicious actor. Many such actors use their botnets to send large quantities of unsolicited commercial email, usually for pay on behalf of third-parties.
In a perfect world, SMTP would contain an exchange where the server asked Are you a real mail server?, and the client would reply "Yes, definitely", or "No, I'm part of a botnet". The client would tell the truth, and the server would accept mail only from clients that were themselves real mail servers. Sadly, we're not there yet. Until we are, one excellent way to distinguish botnet members from real mail servers is to ask the client to behave like a real mail server. One thing real mail servers are required to do is to cope with temporary disappointment, such as the receiving server not having enough free disk space or the user's mailbox being temporarily locked. When a receiving server says it has a temporary problem by means of a 400-class return code, the sender should wait for a while, then retry the transaction. Most botnet clients are senders of a type known as fire-and-forget; they just want to zip through the fifty-million recipient addresses they've been given. If the one they're on right now has a temporary problem, well, that's just too bad; they drop that email on the floor and go on to the next one.
Greylisting
Greylisting takes advantage of this difference in behavior. A receiving server can require a sending client to prove its bona fides by temporarily refusing an email on first delivery attempt while making a note of the triplet consisting of the sender's IP address, the envelope sender, and envelope recipient. When a client arrives with an email with a matching triplet, the server assumes that this is a redelivery attempt and accepts the email. A well-behaving greylist will then add the triplet to a whitelist; having proved itself once, should the same client try to send other mail from the same sender to the same recipient, there's no reason not to accept it straightaway, at least for a while. Some implementations will extend this courtesy to any mail from that client. To prevent botnet members from simply trying each delivery twice, back-to-back, most implementations will require a certain time to elapse between the first delivery attempt, and the redelivery attempt that is permitted to succeed.More subtlety is possible. In the simplest case, IP addresses under one's direct control can be configured as automatically whitelisted. Since few botnet mailers implement SMTP STARTTLS, any email that arrives under cover of SMTP encryption can be accepted immediately. Any mail that is submitted after SMTP AUTH succeeds is clearly from a legitimate local user, and can again be accepted immediately.
Greylisting is not without its downsides. The biggest problem is that much email will be delayed, and the size of this delay is not under the control of the receiver. Although most greylisting implementations will print a human-readable message indicating how long the sending server must wait before redelivery can succeed, there is as yet no standards-based way for the sender to recognize this, and wait only that long before retrying. Instead, the client will have its own configuration telling it how often to retry and, if this does not match the greylist's expectations, severely delayed deliveries can result. Although email delivery has never, in principle, been guaranteed nor instantaneous, some users have developed expectations outside the service guarantees. As Wikipedia dryly notes:
Problems arise with service providers that maintain a bank of sending servers, and move queued emails between these servers after each delivery attempt. Since the triplet includes the sender IP, this means each redelivery attempt is seen by greylisting as being a new email; much time can elapse before an email is lucky enough to be tried a second time from a given sending server. This is not, in my experience, common, but it does happen; all one can then do is add the whole bank of servers to an IP whitelist.
How effective is it?
My mail server is not a big one, and the community it serves is fairly small. Nonetheless, it's been on the internet a long time, and I see no reason to think it non-representative. I looked at the logs for incoming mail for September 2016, and this broke down as follows, approximately:- There were 101,900 inbound delivery attempts
- Of these 101,900, 3,370 were SPF failures, 1,300 were from invalid sender hostnames, and 3,800 were to invalid recipients, leaving 93,400
- Of those 93,400, 71,100 attempted no redelivery and so failed greylisting, leaving 22,300 which passed greylisting and went on to local delivery
- Of those 22,300, 7,800 were list emails filed via procmail, leaving 14,500 which went to the content-based spam filtering system
- Of those 14,500, 3,600 were identified as spam by content-based filtering
Wikipedia says that greylisting typically delays email by about 15 minutes. Looking at my body of emails from September 2016, where greylisting delayed delivery of an inbound email the median delay was 900 seconds. The mean, however, was around 8,000s, due to a long-tailed distribution (a comparatively small number of very large delays). Nonetheless, assuming that all the mail that didn't pass greylisting was spam, greylisting is the single most effective antispam technique on my mail server.
If you're of a mind to give greylisting a try on your mail server, Sendmail does it most easily through the milter interface. My Sendmail uses milter-greylist, which works well and painlessly for me; the project has instructions on configuring it with Sendmail. The milter-greylist developers also note that it works with Postfix, but those instructions seem to be a lot thinner, and other implementations like postgrey may work better. Ubuntu seems to recommend postgrey for Postfix, and the CentOS people also have a HOWTO for it. Exim has always been a bit of a mystery for me, but greylisting.org has a number of suggestions for greylisting with Exim. It would be nice if greylisting capabilities shipped as a standard part of all major mail transport agents (MTAs), but until then it should be fairly easy to glue it onto your MTA of choice.
Brief items
Security quotes of the week
New vulnerabilities
apache: denial of service
| Package(s): | apache httpd | CVE #(s): | CVE-2016-1546 | ||||
| Created: | October 7, 2016 | Updated: | October 12, 2016 | ||||
| Description: | From the CVE entry:
The Apache HTTP Server 2.4.17 and 2.4.18, when mod_http2 is enabled, does not limit the number of simultaneous stream workers for a single HTTP/2 connection, which allows remote attackers to cause a denial of service (stream-processing outage) via modified flow-control windows. | ||||||
| Alerts: |
| ||||||
bind: denial of service
| Package(s): | bind | CVE #(s): | CVE-2016-6170 | ||||||||||||||||
| Created: | October 12, 2016 | Updated: | October 12, 2016 | ||||||||||||||||
| Description: | From the CVE entry:
ISC BIND through 9.9.9-P1, 9.10.x through 9.10.4-P1, and 9.11.x through 9.11.0b1 allows primary DNS servers to cause a denial of service (secondary DNS server crash) via a large AXFR response, and possibly allows IXFR servers to cause a denial of service (IXFR client crash) via a large IXFR response and allows remote authenticated users to cause a denial of service (primary DNS server crash) via a large UPDATE message. | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
ffmpeg: not well specified
| Package(s): | ffmpeg | CVE #(s): | |||||
| Created: | October 6, 2016 | Updated: | October 12, 2016 | ||||
| Description: | The openSUSE advisory has the following information:
ffmpeg was updated to 2.8.8 to fix the following issues, both bugs and security issues: * avformat/oggparsevp8: fix pts calculation on pages ending with an invisible frame * avcodec/mjpegdec: Do not try to detect last scan but apply idct after all scans for progressive jpeg * avformat/oggparseopus: Check that granule pos is within the supported range * avformat/utils: Check bps before using it in a shift in ff_get_pcm_codec_id() * ffmpeg: Check that r_frame_rate is set before attempting to use it * avformat/utils: Do not compute the bitrate from duration == 0 * avformat/utils: Check negative bps before shifting in ff_get_pcm_codec_id() * avformat/avidec: Detect index with too short entries * avformat/oggparseopus: Fix Undefined behavior in oggparseopus.c and libavformat/utils.c * avformat/allformats: Making av_register_all() thread-safe. * avcodec/vp9_parser: Check the input frame sizes for being consistent * avformat/oggdec: Fix integer overflow with invalid pts * avcodec/ffv1enc: Fix assertion failure with non zero bits per sample * avcodec/diracdec: Check numx/y * avformat/avidec: Fix infinite loop in avi_read_nikon() | ||||||
| Alerts: |
| ||||||
freeimage: code execution
| Package(s): | freeimage | CVE #(s): | CVE-2016-5684 | ||||||||||||||||||||||||||||||||
| Created: | October 6, 2016 | Updated: | October 13, 2016 | ||||||||||||||||||||||||||||||||
| Description: | From the Debian-LTS advisory:
It was discovered that there was an out-of-bounds write vulnerability in the XMP image handling functionality in freeimage, a support library for various graphics image formats. A specially crafted XMP file can cause an arbitrary memory overwrite resulting in code execution. | ||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||
ghostscript: multiple vulnerabilities
| Package(s): | ghostscript | CVE #(s): | CVE-2013-5653 CVE-2016-7976 CVE-2016-7977 CVE-2016-7978 CVE-2016-7979 CVE-2016-8602 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | October 12, 2016 | Updated: | February 22, 2017 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Debian advisory:
Several vulnerabilities were discovered in Ghostscript, the GPL PostScript/PDF interpreter, which may lead to the execution of arbitrary code or information disclosure if a specially crafted Postscript file is processed. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
graphicsmagick: multiple vulnerabilities
| Package(s): | graphicsmagick | CVE #(s): | CVE-2016-7996 CVE-2016-7997 CVE-2016-7800 | ||||||||||||||||||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | October 12, 2016 | ||||||||||||||||||||||||||||||||||||||||
| Description: | From the Mageia advisory:
Unsigned underflow leading to heap overflow when parsing 8BIM chunk (CVE-2016-7800). Two issues in the WPG reader (CVE-2016-7996, CVE-2016-7997). | ||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||
imagemagick: two vulnerabilities
| Package(s): | imagemagick | CVE #(s): | CVE-2016-7799 CVE-2016-7906 | ||||||||||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | December 1, 2016 | ||||||||||||||||||||||||||||||||
| Description: | From the Arch Linux advisory:
CVE-2016-7799 (denial of service): A buffer over-read vulnerability was found in ImageMagick. A malicious file could cause the application to crash. CVE-2016-7906 (arbitrary code execution): An attacker is able to trigger a use-after-free when providing a crafted image to ImageMagick's mogrify function. | ||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||
irssi: information disclosure
| Package(s): | irssi | CVE #(s): | CVE-2016-7553 | ||||||||||||||||||||||||
| Created: | October 11, 2016 | Updated: | November 28, 2016 | ||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
An information disclosure vulnerability was found in the buf.pl core script for irssi. Other users on the same machine may be able to retrieve the whole window contents after /UPGRADE when the buf.pl script is loaded. Furthermore, this dump of the windows contents is never removed afterwards. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
kcoreaddons: HTML injection
| Package(s): | kcoreaddons | CVE #(s): | CVE-2016-7966 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | October 24, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Arch Linux advisory:
Through a malicious URL that contained a quote character it was possible to inject HTML code in KMail's plain text viewer. Due to the parser used on the URL it was not possible to include the equal sign (=) or a space into the injected HTML, which greatly reduces the available HTML functionality. Although it is possible to include an HTML comment indicator to hide content. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
kde-cli-tools5: code execution
| Package(s): | kde-cli-tools5 | CVE #(s): | CVE-2016-7787 | ||||||||
| Created: | October 12, 2016 | Updated: | October 12, 2016 | ||||||||
| Description: | From the openSUSE advisory:
A user could sneak an unicode string terminator in the kdesu invocation. | ||||||||||
| Alerts: |
| ||||||||||
kernel: stack corruption
| Package(s): | kernel | CVE #(s): | CVE-2016-7039 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | October 11, 2016 | Updated: | October 14, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Red Hat advisory:
Linux kernel built with the 802.1Q/802.1ad VLAN(CONFIG_VLAN_8021Q) OR Virtual eXtensible Local Area Network(CONFIG_VXLAN) with Transparent Ethernet Bridging(TEB) GRO support, is vulnerable to a stack overflow issue. It could occur while receiving large packets via GRO path as an unlimited recursion could unfold in both VLAN and TEB modules leading to a stack corruption in the kernel. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
libdwarf: out-of-bounds read
| Package(s): | libdwarf | CVE #(s): | CVE-2016-7410 | ||||
| Created: | October 10, 2016 | Updated: | October 12, 2016 | ||||
| Description: | From the Red Hat bugzilla:
An out-of-bounds read vulnerability was found in libdwarf-20160613. | ||||||
| Alerts: |
| ||||||
libgd: denial of service
| Package(s): | libgd | CVE #(s): | CVE-2016-7568 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | October 12, 2016 | Updated: | October 24, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the CVE entry:
Integer overflow in the gdImageWebpCtx function in gd_webp.c in the GD Graphics Library (aka libgd) through 2.2.3, as used in PHP through 7.0.11, allows remote attackers to cause a denial of service (heap-based buffer overflow) or possibly have unspecified other impact via crafted imagewebp and imagedestroy calls. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
libXfixes: integer overflow
| Package(s): | libXfixes | CVE #(s): | CVE-2016-7944 | ||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | December 9, 2016 | ||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
It was found that when receiving a response from the server protocol data is not validated sufficiently. The 32 bit field "rep.length" is not checked for validity, which allows an integer overflow on 32 bit systems. A malicious server could send INT_MAX as length, which gets multiplied by the size of XRectangle. In that case the client won't read the whole data from server, getting out of sync. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
libXi: insufficient validation
| Package(s): | libXi | CVE #(s): | CVE-2016-7945 CVE-2016-7946 | ||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | November 11, 2016 | ||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
It was found that when receiving a response from the server protocol data is not validated sufficiently. This results in integer overflows. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
libXrandr: insufficient validation
| Package(s): | libXrandr | CVE #(s): | CVE-2016-7947 CVE-2016-7948 | ||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | November 2, 2016 | ||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
CVE-2016-7947: It was found that when receiving a response from the server protocol data is not validated sufficiently. This results in integer overflows. CVE-2016-7948: It was found that when receiving a response from the server protocol data is not validated sufficiently. This results in various data mishandlings. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
libXrender: insufficient validation
| Package(s): | libXrender | CVE #(s): | CVE-2016-7949 CVE-2016-7950 | ||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | December 7, 2016 | ||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
CVE-2016-7949: It was found that when receiving a response from the server protocol data is not validated sufficiently. Individual lengths inside received server data can overflow the previously reserved memory. CVE-2016-7950: It was found that when receiving a response from the server protocol data is not validated sufficiently. The memory for filter names is reserved right after receiving the reply. After that, filters are iterated and each individual filter name is stored in that reserved memory. The individual name lengths are not checked for validity, which means that a malicious server can reserve less memory than it will write to during each iteration. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
libXtst: insufficient validation
| Package(s): | libXtst | CVE #(s): | CVE-2016-7951 CVE-2016-7952 | ||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | December 7, 2016 | ||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
CVE-2016-7951: It was found that when receiving a response from the server protocol data is not validated sufficiently. This results in integer overflows. CVE-2016-7952: It was found that when receiving a response from the server protocol data is not validated sufficiently. This results in various data mishandlings. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
libXv: insufficient validation
| Package(s): | libXv | CVE #(s): | CVE-2016-5407 | ||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | December 7, 2016 | ||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
It was found that when receiving a response from the server protocol data is not validated sufficiently. The Xv query functions for adaptors and encodings suffer from out of boundary accesses if a hostile X server sends a maliciously crafted response. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
libXvMC: insufficient validation
| Package(s): | libXvMC | CVE #(s): | CVE-2016-7953 | ||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | December 7, 2016 | ||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
It was found that when receiving a response from the server protocol data is not validated sufficiently. If an empty string is received from an x-server, the buffer might underrun by accessing "rep.nameLen - 1" unconditionally, which could end up being -1. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
mat: information leak
| Package(s): | mat | CVE #(s): | |||||||||
| Created: | October 10, 2016 | Updated: | November 8, 2016 | ||||||||
| Description: | From the Debian LTS advisory:
An implementation flaw was discovered in mat, the metadata anonymisation toolkit. The implementation of PDF support lacks support to anonymize the metadata in embedded images. As there is no easy fix for this flaw, it was decided that PDF support will be removed altogether from mat for the time being. | ||||||||||
| Alerts: |
| ||||||||||
messagelib: two vulnerabilities
| Package(s): | messagelib | CVE #(s): | CVE-2016-7967 CVE-2016-7968 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | October 12, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Arch Linux advisory:
CVE-2016-7967 (cross-site scripting): KMail since version 5.3.0 used a QWebEngine based viewer that had JavaScript enabled. Since the generated html is executed in the local file security context by default access to remote and local URLs was enabled. CVE-2016-7968 (insufficient validation): KMail since version 5.3.0 used a QWebEngine based viewer that had JavaScript enabled. HTML Mail contents were not sanitized for JavaScript and included code was executed. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
mujs: two vulnerabilities
| Package(s): | mujs | CVE #(s): | CVE-2016-7563 CVE-2016-7564 | ||||||||||||||||
| Created: | October 11, 2016 | Updated: | October 12, 2016 | ||||||||||||||||
| Description: | From the Red Hat bugzilla:
1. CVE-2016-7563:
mujs str Out-of-Bound read 1 byte in function chartorune.
2. CVE-2016-7564:
mujs "char *s" Heap overflow in Fp_toString at jsfunction.c:72 | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
nodejs4: two vulnerabilities
| Package(s): | nodejs4 | CVE #(s): | CVE-2016-5325 CVE-2016-7099 | ||||||||||||||||||||||||||||
| Created: | October 7, 2016 | Updated: | October 28, 2016 | ||||||||||||||||||||||||||||
| Description: | From the SUSE bug reports:
CVE-2016-5325: An unspecified low-severity Node.js HTTP processing vulnerability was found and will be fixed in latest update. Details are currently embargoed until new releases are available. CVE-2016-7099: This is a high severity defect that would allow a malicious TLS server to serve an invalid wildcard certificate for its hostname and be improperly validated by a Node.js client. This is due to a flaw in the validation of *. in the wildcard name string. | ||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||
nsd: denial of service
| Package(s): | nsd | CVE #(s): | CVE-2016-6173 | ||||
| Created: | October 12, 2016 | Updated: | October 12, 2016 | ||||
| Description: | From the Red Hat bugzilla:
It was found that NSD does not implement reasonable restrictions for zone sizes. This allows an explicitly configured primary DNS server for a zone to crash a secondary DNS server, affecting service of other zones hosted on the same secondary server. | ||||||
| Alerts: |
| ||||||
ntp: privilege escalation (distribution-specific)
| Package(s): | ntp | CVE #(s): | CVE-2016-0727 | ||||
| Created: | October 6, 2016 | Updated: | October 12, 2016 | ||||
| Description: | From the Ubuntu advisory:
It was discovered that the NTP cronjob incorrectly cleaned up the statistics directory. A local attacker could possibly use this to escalate privileges. (CVE-2016-0727) | ||||||
| Alerts: |
| ||||||
oxide-qt: denial of service
| Package(s): | oxide-qt chromium | CVE #(s): | CVE-2016-7549 | ||||
| Created: | October 7, 2016 | Updated: | October 12, 2016 | ||||
| Description: | From the CVE entry:
Google Chrome before 53.0.2785.113 does not ensure that the recipient of a certain IPC message is a valid RenderFrame or RenderWidget, which allows remote attackers to cause a denial of service (invalid pointer dereference and application crash) or possibly have unspecified other impact by leveraging access to a renderer process, related to render_frame_host_impl.cc and render_widget_host_impl.cc, as demonstrated by a Password Manager message. | ||||||
| Alerts: |
| ||||||
python-pillow: memory disclosure
| Package(s): | python-pillow | CVE #(s): | |||||||||
| Created: | October 10, 2016 | Updated: | October 13, 2016 | ||||||||
| Description: | From the Fedora advisory:
This update backports an overflow fix. ---- Backport fix for three memory disclosure/corruption bugs from insufficient parameter validation leading to integer overflow. | ||||||||||
| Alerts: |
| ||||||||||
qemu: three vulnerabilities
| Package(s): | qemu | CVE #(s): | CVE-2016-7161 CVE-2016-7170 CVE-2016-7908 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | October 11, 2016 | Updated: | October 12, 2016 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Debian LTS advisory:
CVE-2016-7161: Heap-based buffer overflow in the .receive callback of xlnx.xps-ethernetlite in QEMU (aka Quick Emulator) allows attackers to execute arbitrary code on the QEMU host via a large ethlite packet. CVE-2016-7170: The vmsvga_fifo_run function in hw/display/vmware_vga.c in QEMU (aka Quick Emulator) is vulnerable to an OOB memory access. CVE-2016-7908: The mcf_fec_do_tx function in hw/net/mcf_fec.c in QEMU (aka Quick Emulator) does not properly limit the buffer descriptor count when transmitting packets, which allows local guest OS administrators to cause a denial of service (infinite loop and QEMU process crash) via vectors involving a buffer descriptor with a length of 0 and crafted values in bd.flags. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
systemd: denial of service
| Package(s): | systemd | CVE #(s): | CVE-2016-7796 | ||||||||||||||||||||
| Created: | October 10, 2016 | Updated: | January 4, 2017 | ||||||||||||||||||||
| Description: | From the SUSE advisory:
A zero-length message received over systemd's notification socket could make manager_dispatch_notify_fd() return an error and, as a side effect, disable the notification handler completely. As the notification socket is world-writable, this could have allowed a local user to perform a denial-of-service attack against systemd. | ||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||
tomcat: privilege escalation
| Package(s): | tomcat | CVE #(s): | CVE-2016-5425 CVE-2016-6325 | ||||||||||||||||||||||||||||||||||||||||
| Created: | October 11, 2016 | Updated: | November 7, 2016 | ||||||||||||||||||||||||||||||||||||||||
| Description: | From the Red Hat advisory:
It was discovered that the Tomcat packages installed configuration file /usr/lib/tmpfiles.d/tomcat.conf writeable to the tomcat group. A member of the group or a malicious web application deployed on Tomcat could use this flaw to escalate their privileges. (CVE-2016-5425) It was discovered that the Tomcat packages installed certain configuration files read by the Tomcat initialization script as writeable to the tomcat group. A member of the group or a malicious web application deployed on Tomcat could use this flaw to escalate their privileges. (CVE-2016-6325) | ||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||
tracker: denial of service
| Package(s): | tracker | CVE #(s): | |||||||||
| Created: | October 12, 2016 | Updated: | January 2, 2017 | ||||||||
| Description: | From the Ubuntu advisory:
It was discovered that Tracker incorrectly handled certain malformed GIF images. If a user or automated system were tricked into downloading a specially-crafted GIF image, Tracker could crash, resulting in a denial of service. | ||||||||||
| Alerts: |
| ||||||||||
xen: information disclosure
| Package(s): | xen | CVE #(s): | CVE-2016-6835 | ||||||||||||||||||||||||
| Created: | October 7, 2016 | Updated: | October 12, 2016 | ||||||||||||||||||||||||
| Description: | From the SUSE bug report:
Quick Emulator(Qemu) built with the VMWARE VMXNET3 NIC device support is vulnerable to an OOB read access. In that it does not check if packet headers does not check for IP header length. It could lead to a OOB access when reading further packet data. | ||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||
zendframework: SQL injection
| Package(s): | zendframework | CVE #(s): | CVE-2016-4861 | ||||||||||||||||
| Created: | October 6, 2016 | Updated: | October 24, 2016 | ||||||||||||||||
| Description: | From the Debian-LTS advisory:
CVE-2016-4861: The implementation of ORDER BY and GROUP BY in Zend_Db_Select remained prone to SQL injection when a combination of SQL expressions and comments were used. This security patch provides a comprehensive solution that identifies and removes comments prior to checking validity of the statement to ensure no SQLi vectors occur. | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
Page editor: Jonathan Corbet
Kernel development
Brief items
Kernel release status
The 4.9 merge window remains open; see the separate article below for a summary of the work merged so far.Stable updates: 4.8.1, 4.7.7, and 4.4.24 were released on October 7.
Quotes of the week
I will have to ask around the security people to see what they think.
Kernel development news
4.9 Merge window part 2
As of this writing, Linus has pulled 13,488 non-merge changesets into the mainline repository for the 4.9 development cycle. That suggests that not only will 4.9 be the busiest cycle in the kernel's history, but that it will surpass the previous record (3.15, at 13,722 changesets) before the merge window closes. The merging of the greybus driver code has a lot to do with that but, even without greybus, there is a lot going on this time around.Among the user-visible changes merged since last week's summary are:
- The system calls for the memory protection keys feature have been
merged. The pkey_alloc(), pkey_free(), and
pkey_mprotect() calls are as described in this article, but the pkey_set()
and pkey_free() calls, which can be implemented purely in
user space, were not included. See Documentation/x86/protection-keys.txt for
details.
- The bottleneck bandwidth and RTT (BBR)
congestion control algorithm has been merged.
- The BATMAN mesh networking subsystem has a new netlink-based
configuration mechanism that will, over time, supersede and replace
the older, debugfs-based interface.
- The netfilter module supports a new "quota" mechanism designed to
enable the enforcement of byte quotas. There's also a new
random-number generation module intended to enable the random
distribution of packets (across multiple queues, for example).
- There is a new just-in-time BPF compiler that can be used to load BPF
programs for execution within Netronome network interfaces. In 4.9,
only the cls_bpf classifier module will take advantage of
this capability.
- The filesystems in user space (FUSE) module now supports POSIX
access-control lists.
- The Greybus subsystem has been
merged. This bus was intended for the "Project Ara" phone, which has
since been canceled, but Greg Kroah-Hartman successfully argued for its inclusion
anyway. This merge includes the entire development history
for this code, some 2,400 changesets in total.
- There is a new set of resource limits controlling how many namespaces
may be created within any given user namespace. See Documentation/sysctl/user.txt for
details.
- The hardware latency tracer (which seeks to flush out latencies caused
by the hardware itself) has moved into the mainline from the realtime
tree. See Documentation/trace/hwlat_detector.txt
for details and usage information.
- The ubifs filesystem now supports overlayfs and the O_TMPFILE
file-creation option.
- New hardware support includes:
- Systems and processors:
Broadcom BCM53573-based processors.
- Audio:
Nuvoton NAU8810 audio codecs,
Realtek RT5660/RT5663/RT5668 audio codecs,
X-Powers AC100 audio codecs, and
Samsung Exynos SoC low power audio subsystems.
- Industrial I/O:
Maxim thermocouple sensors,
Measurement Computing CIO-DAC digital-to-analog converters,
Asahi Kasei AK8974 3-axis magnetometers,
Domintech DMARD05/DMARD06/DMARD07 accelerometers,
Texas Instruments ADC161S626 1-channel differential
analog-to-digital converters,
Texas Instruments' ADC12130/ADC12132/ADC12138 analog-to-digital
converters,
MediaTek mt65xx analog-to-digital converters,
Linear Technology LTC2485 analog-to-digital converters,
Analog Devices AD8801/AD8803 digital-to-analog converters,
Apex Embedded Systems STX104 analog-to-digital converters,
mCube MC3230 digital accelerometers, and
Murata ZPA2326 pressure sensors.
- Media:
Atmel image sensor controllers,
Analog Devices AD5820 lens voice coils,
Techwell TW5864 video/audio grabber/encoders,
STMicroelectronics HVA multi-format video encoders,
STMicroelectronics STiH4xx HDMI CEC interfaces, and
Gennum GS1662 HD/SD-SDI serializers.
- Miscellaneous:
Rockchip RK818 power-management chips,
Elan eKTF2127 touchscreen controllers,
Microsemi PQI SCSI controllers,
Intel integrated sensor hubs,
Cavium ThunderX I2C buses,
Cavium ThunderX random number generators,
APM X-Gene SoC performance monitoring units,
Qualcomm external bus interfaces (version 2),
JDI LT070ME05000 WUXGA DSI panels, and
Amlogic Meson PWM controllers.
- Networking:
Microsemi VSC85xx PHYs,
Amazon Elastic Network adapters,
Thunder RGX/RGMII MAC interfaces,
Chelsio crypto coprocessors,
Qualcomm EMAC gigabit Ethernet controllers, and
Qualcomm Atheros QCA8K Ethernet switches.
- Pin Control / GPIO:
Aspeed G4/G5 pin and GPIO controllers,
NextThing GR8 pin controllers,
X-Powers AXP209 PMIC GPIO controllers,
Intel Whiskey Cove PMIC GPIO controllers,
Diamond Systems GPIO-MM controllers,
Technologic Systems FPGA I2C GPIO controllers, and
TI LP873X PMIC GPIO controllers.
- Thermal: Qualcomm TSENS temperature sensors, QorIQ thermal monitoring units, and Intel Broxton PMIC thermal monitors.
- Systems and processors:
Broadcom BCM53573-based processors.
Changes visible to kernel developers include:
- The handling of messages printed with printk() has changed for
the case of single-line messages created with multiple
printk() calls. The rule has long been that the continuation
lines should be marked with the KERN_CONT pseudo log level,
but that requirement has not been enforced for several years. As of
this
commit, the use of KERN_CONT is again mandatory; without
it, output will be garbled. Many places in the kernel will need
fixing; for the short term, expect some ugly output from 4.9-rc
kernels.
- The "kthread_worker" API has seen a number of changes. These include
the renaming
of most functions to start with "kthread_"
(e.g. init_kthread_worker() becomes
kthread_init_worker()), the addition of kthread_create_worker()
and kthread_destroy_worker(),
support for delayed kthread
work, and support for freezable
kthreads.
- The network subsystem has added a module called "strparser"; its job
is to parse (in-kernel) application-layer protocol messages from a TCP
connection. See Documentation/networking/strparser.txt
for details.
- The handling of extended attributes in filesystems has changed.
Filesystems that support extended attributes should create an
xattr_handlers structure with its low-level methods and
attach it to the superblock structure. The
setxattr(), getxattr() and removexattr()
inode operations are no longer used and have been removed.
- The rename() inode operation has gained a flags
argument. In truth, rename() was removed and the
rename2() operation was, well, renamed; all in-kernel filesystems
have been updated to reflect the change.
- The new function current_time() returns the current time at the proper resolution for storage in a specific filesystem; it replaces the old CURRENT_TIME() macro. Among other things, the new API is year-2038 safe.
At this point, it seems likely that things will slow down considerably as the 4.9 merge window approaches its scheduled closing on October 16.
On Linux kernel maintainer scalability
LWN's traditional development statistics article for the 4.6 development cycle ended with a statement that the process was running smoothly and that there were no process scalability issues in sight. Wolfram Sang started his 2016 LinuxCon Europe talk by taking issue with that claim. He thinks that there are indeed scalability problems in the kernel's development process. A look at his argument is of interest, especially when contrasted with another recent talk on maintainer scalability.
Beyond changesets merged
Sang's core point is that looking at the number of patches merged only tells part of the story; it says nothing about what had to happen to get those patches into the mainline. Looking at the last few years' worth of development cycles, he noted that relatively few patches carry tags beyond the Signed-off-by applied by the developer and the committer. In particular, around the 3.0 days, only about 20% of the patches in the mainline had an Acked-by, Reviewed-by, or Tested-by tag indicating that anybody other than the maintainer had seriously looked at them. That number is closer to 40% in current kernels, he said; it is a clear improvement, but still does not make him happy. For a properly scalable kernel process, he said, we should have much higher levels of review by developers who are not the subsystem maintainer.
Another metric one can look at is the time difference between the date on
the patch and the date on which it was first committed to a git tree. The
Ethernet driver maintainers, he said, are heroes: 80% of all the patches
were accepted within two weeks. A number of other subsystems do not do
![[Wolfram Sang]](https://static.lwn.net/images/conf/2016/lceu/WolframSang-sm.jpg "Wolfram Sang")
anywhere near as well, and some have gotten significantly worse. I2C,
Sang's own subsystem, has stayed about the same over the last three years,
which surprised him. As the workload has increased, it has come to feel
like things are getting much worse.
The time-to-commit metric may be useful, but it is not without its flaws. The final version of a patch may have been committed fairly quickly, but previous versions could have languished without review for a long time. Patches that are rejected or that get lost are not considered at all.
One way to try to get a better handle on things is to look at the Patchwork systems for the subystems that use it, and, in particular, to look at the backlog of patches found there. For I2C, it shows a relatively low backlog until about 3.16, when he gave up on trying to keep up with the flow and fell behind. The ACPI subsystem has an amazing backlog of zero. The relevant maintainer (Rafael Wysocki) was in the room; he noted that it depends on how a subsystem uses Patchwork. He said that he quickly marks a lot of patches as inapplicable; Sang replied that he doesn't even have the time to do that. The ext4 filesystem shows a linear growth in its backlog, up to about 800 patches currently. The numbers for several other subsystems were shown; almost all of them are going up.
The problem, Sang said, is that the number of committers is not scaling to match the growing number of contributors to the kernel. We are getting more reviewers, but they are coming in slowly and are not anywhere near enough. As a result, the number of unprocessed patches is on the increase.
How can this problem be addressed? Users can help by commenting on and, especially, testing patches. Developers need to be aware that sloppiness is often a problem; they should acknowledge when they have done suboptimal work. Developers need to take part in reviewing; if nothing else, they should review their own patches. For maintainers, working harder is not generally the solution; that just leads to burnout. They should get their tools in order and automate tasks whenever possible; looking at what other maintainers are using can be helpful. Companies should allow and encourage their developers to spend time reviewing patches.
What he does not want to see is a "kernel infrastructure initiative". The Core Infrastructure Initiative, run by the Linux Foundation as a way to channel resources to important but underfunded projects, is a good thing, but it is a reaction to a problem that got out of control. Things had to go wrong first. Sang would rather see action now to keep things from getting to that state.
For I2C, Sang intends to step back a bit. He will become one of the I2C developers, one of its architects, and one of its reviewers, but he will not be the only one. That may slow things down in the short term, since he will be doing less patch review. The advantage is that he will stay sane, and will have the time and energy to try to address the problem on higher levels.
The maintainer as bottleneck
While Sang intends to step back on patch review, his plan still calls for him to be the sole committer of patches for the I2C subsystem. In this context, it is interesting to look at another talk, given at Kernel Recipes one week earlier by i915 graphics driver maintainer Daniel Vetter. He, too, made the point that maintainers don't scale, but he would rather see maintainers get help at all levels.
One year ago, he would have said that there was no problem in the i915 subsystem. Applying patches was relatively easy, after all. He had never reviewed the majority of the patches there; i915 has a number of developers who can do that. But, as the single maintainer, he gave the subsystem "a bus factor of one"; when he wasn't available for any reason, things simply came to a stop.
At the 2015 Kernel Summit, Linus Torvalds said
that he has come to
like the group maintainer model, where more than one person takes
responsibility for a given subsystem. Vetter wanted to give that a try,
but he quickly ran into a problem: nobody was willing to sign up as the
co-maintainer for the i915 subsystem. He was, however, able to find
developers who were willing to commit patches for i915; indeed, he signed
up 15 of them. He figured he would experiment with the multiple-committer
![[Daniel Vetter]](https://static.lwn.net/images/conf/2016/recipes/DanielVetter-sm.jpg "Daniel Vetter")
model for one release cycle. After all, nobody had ever really tried this
before in the kernel, so it must be a stupid idea.
That was one year ago, he said, and disaster has failed to materialize. Instead, he has "seriously happy contributors," and a whole set of reviewers who can apply the patches they look at. He is now "a bored maintainer," and all of the nagging and begging to get code merged has gone away. He has found that commit rights are a strong carrot that can be used to get developers and companies to contribute — and to be careful about the work they do. It also leads to "distributed conflict management" that makes life easier.
So what does he do anymore? His main job at this point, as "the" maintainer for i915, is communications with the outside, including any work that requires coordination with other subsystem trees. He connects developers with the appropriate reviewers, and puts together the pull requests to send work upstream. And, of course, he "takes the blame for everything".
To make this model work, he said, a subsystem clearly needs a team of developers, and non-maintainer reviews must be the norm. The group should be consistent, with developers who stay around; otherwise, enforcement via social feedback will not work well. Good documentation and tools are necessary; i915 has a set of process documents on this page. When somebody makes a mistake, if possible, a check should be put into the tools to keep it from happening again.
Good testing is crucial to this model. A multi-committer tree can never be rebased, so there is no way to remove embarrassing mistakes. They really need to be avoided in the first place; that requires good pre-commit testing to ensure that the obscure corner cases do not break.
The rough consensus model works best for a group like this. The default on any patch is "no action", so a developer's full disagreement will stop things. What's important, he said, is to have agreement on the goals for the subsystem; disagreement on the path taken toward those goals is acceptable. A good rule of thumb is "if you push a patch and there's screaming on IRC, you shouldn't have done it."
In general, he said, the kernel could probably benefit from more maintainer groups like this. It is a more efficient way to maintain busy subsystems, especially those that currently have a lot of submaintainer trees.
Meanwhile in Berlin
Fast-forward one week; your editor raised this idea in Sang's talk and asked whether the single-committer model might be part of the scalability problems raised there. The developers in that room tended toward skepticism over whether the idea could work outside of the i915 tree. Wysocki, in particular, seemed to feel that there were relatively few submaintainers who could be trusted with full commit access. These maintainers push patches that must be rejected fairly often, so they should not be able to commit directly to the subsystem tree.
Perhaps these developers, too, would be pleasantly surprised if they were to run an experiment with more widely distributed commit rights. In any case, it seems likely that growing numbers of developers and patches will put more stress on subsystem maintainers. If those maintainers are not to become a choke point for kernel development, ways to spread the work they do will be required.
[Your editor thanks both the Linux Foundation and Kernel Recipes for supporting his travel to these events.]
Patches and updates
Kernel trees
Architecture-specific
Core kernel code
Device drivers
Device driver infrastructure
Documentation
Filesystems and block I/O
Memory management
Security-related
Virtualization and containers
Miscellaneous
Page editor: Jonathan Corbet
Distributions
Supporting UEFI secure boot in Debian
The Debian project can be accused of many things, but jumping too quickly on leading-edge technology is not one of them. That can be seen in, among other things, the fact that there is still not a version of the distribution that supports the UEFI secure boot mechanism. But, as Ben Hutchings explained during his 2016 Kernel Recipes talk, such support is in the works, and it will be implemented in a uniquely Debian-like manner.
Hutchings started by reviewing how the secure boot technology works. On
systems with secure boot, the UEFI firmware is able to validate the
system's bootloader by checking its signature against a list of trusted
![[Ben Hutchings]](https://static.lwn.net/images/conf/2016/recipes/BenHutchings-sm.jpg "Ben Hutchings")
keys; it can also validate UEFI drivers and firmware updates. This
checking is intended to defend the system against persistent malware
by ensuring that the system's low-level software has not been tampered
with. Support for secure boot is mandatory for systems that will run
Windows, so all desktop systems sold since 2012 have it. It is possible to
build
in trust for multiple signing keys but, in practice, the one key that is
trusted by everybody is Microsoft's key.
To boot Linux on a system with secure boot enabled, the first stage bootloader must have a Microsoft signature; in the Linux world, that is usually the special "shim" utility. Distributors create their own signing key, which they then build into shim; that key is then used to verify the GRUB installation. GRUB, in turn, verifies the signature for the kernel image; the kernel must validate any modules that it loads. By checking for proper signatures at every stage of the boot process, the system can be protected against hostile software modifications.
Supporting secure boot requires some fundamental changes in how Linux systems work. Traditionally, the root account can run arbitrary code, but that cannot be allowed under secure boot; even root should be unable to fully take over the system. To keep that from happening, the module loader must check signatures on kernel modules. The "kexec" mechanism must be prevented from booting into an arbitrary, unsigned image. Various I/O devices that are capable of modifying arbitrary memory must be disabled. The hibernation mechanism must be disabled entirely, though future work could, by signing and verifying the hibernation image, make that work again. Debian will be using Matthew Garrett's patches that make these changes; the patches are still not upstream and may never be, though, since certain high-profile kernel developers dislike them.
The Debian way
The commonly-used approach of signing the kernel image creates some problems for Debian, though. The project's practice with signatures has been to sign metadata describing software, never the code itself. Debian does not want to put signing keys onto its "buildd" systems; those systems are distributed around the globe and present any number of ways in which the keys could be exposed. Debian is also committed to reproducible builds, which cannot depend on secrets (or they would no longer be reproducible).
As a result, Debian cannot automatically build signed kernel binaries in a single step. Instead, the process starts with the creation of an unsigned binary. The signature for that binary is then created offline and a second source package is created to contain it. A second-stage build then rebuilds the kernel binary and applies the signature to it, resulting in a new package with a fully signed kernel binary. As long as the build is truly reproducible, the signature will still apply to the recreated binary. This mechanism allows anybody to create a signed Debian kernel image — as long as it precisely matches the kernel that was created by Debian itself.
Some changes to the Debian Archive Kit (dak) software, which signs archive metadata, will be needed, since it now needs to be able to publish detached signatures too. It may be necessary to publish the signature without publishing the accompanying binary immediately; this could happen in response to an embargoed security issue, for example. There is a script that turns a tarball of EFI binaries (and kernel modules) into a tarball of signatures, which are then installed into the archive metadata. The source package for the initial build is changed to include this tarball, along with a preparation script used by the second-stage build.
Some might wonder whether Ubuntu had already done this sort of work. Ubuntu's solution doesn't work for Debian, though, for a number of reasons, starting with the fact that Ubuntu's package builder doesn't use dak. Ubuntu's signing script only covers EFI binaries, leaving modules unsigned; it also creates signed binaries directly instead of using the two-stage signing process that Debian requires. Ubuntu only supports the amd64 architecture; Debian wants to support all architectures where secure boot is relevant. One other little complication is that the Ubuntu and Debian kernel packages have diverged significantly over the last ten years. The two projects do share some of the utilities, though, including shim and the grub-signed bootloader.
What about the ARM architecture? The upstream kernel has EFI stub support
for both 32-bit and 64-bit ARM systems. Debian has out-of-tree securelevel
patches that work for arm64; other tools, including sbsigntool,
![[Julian Cristau]](https://static.lwn.net/images/conf/2016/recipes/JulienCristau-sm.jpg "Julien Cristau")
shim, and the GRUB EFI code, also work there. So it should be possible to
support arm64 systems under secure boot, at least. The smaller ARM systems
that feature secure boot are locked down and will only run Microsoft
software, but that should be less of a problem in the server space.
Julien Cristau stepped up briefly to talk about how signing keys will be managed as this whole thing goes into production. Once again, the Ubuntu example is only so helpful for Debian. Ubuntu has its own data center, under its control, with the result that it has relatively good security for its machines. Debian lacks this data center, depending instead on machines hosted in various places around the world; it can thus never have as good a handle on the security of those machines. Signing keys will need to be kept in a hardware token, such as a YubiKey device. Patches are in the works to make all of this work in a secure manner.
Where things stand
Hutchings came back to summarize the status of this work; the current state of affairs can also be found in the Debian bug tracker as bug #820036. The changes to dak are in progress, and can be expected soon. The shim package is done, and is in review; there is still a need to put together a signed GRUB build. The signed build for the kernel is ready to go, but a signed build for the fwupdate utility is not ready yet. The installer and live images are ready to work with signed binaries, as is the support to actually install it all. The target for this work is the Debian 9 ("Stretch") release, which goes into freeze in January. More information can be found on the Debian secure boot wiki page.
The first question from the audience had to do with the UEFI signing key: would it be possible to use a separate key and avoid the need for Microsoft's signature? There is some appeal to that approach, but it has a big disadvantage: users would would have to install the new key in their firmware. There are a lot of different firmware implementations out there, each of which has its own horrible interface for key installation. Creating documentation to get Debian users through that process would not be fun.
Why is Debian only coming out with secure boot support now? Hutchings responded that the project has wanted this support all along, but has obviously been slow in implementing it. There has been a lot of talk about how to do it, but nobody actually got the work done. As a result, it's still not possible to install Debian without disabling secure boot in the firmware. That is about to change, though, now that support for secure boot is almost ready.
The final question suggested that, if Debian were to continue to not support secure boot, that might force vendors to make the feature easier to do without. Hutchings's response there was a dry "it's still not the year of the Linux desktop"; he does not believe that Linux distributors can push desktop hardware vendors around in that way.
[Your editor thanks Kernel Recipes for supporting his travel to this event.]
Brief items
Distribution quotes of the week
[1] Not actually true.
[2] Naturally, this is a metaphor for doing something useful, and having a real build system to make it possible.
FreeBSD 11
FreeBSD 11.0 has been released. This version features new architecture support, performance improvements, toolchain enhancements, and support for contemporary wireless chipsets. See the release notes for more information.Announcing the release of Fedora 25 Beta
The Fedora Project has released Fedora 25 Beta. "Fedora's journey is not simply about updating one operating system with the latest and greatest packages. It's also about innovation for the many different platforms represented in the Fedora Project: Workstation, Server, Atomic, and the various Spins. Coordinating the efforts across the many working groups is no small task, and serves as a testament to the talent and professionalism found within the Fedora community."
Newsletters and articles of interest
Distribution newsletters
- DistroWatch Weekly, Issue 682 (October 10)
- Lunar Linux weekly news (October 7)
- openSUSE news (October 5)
- openSUSE Tumbleweed – Review of the Week (October 7)
- Tails report (September)
Page editor: Rebecca Sobol
Development
An update on input
Peter Hutterer gave an update on the input stack at the 2016 X.Org Developers Conference (XDC). A lot has been accomplished, but there is, naturally, more to do—especially as more and more quirky (or buggy) input hardware is released. But, overall Hutterer painted a picture of a mature subsystem that is largely feature-complete at this point.
Touchpads
Hutterer started with touchpads, saying that too many of them are still using the PS/2 protocol in the kernel. That protocol is limited to two touches and has limited bandwidth for reporting events. Over time the PS/2 firmware in those devices has gotten less and less testing as well.
The way forward is the RMI4 [129-page PDF] protocol for Synaptics touchpads for hardware that supports it. RMI4 is roughly six years old, so Linux is a bit behind the curve by not supporting it yet. The protocol allows for multiple fingers and has more accurate axis ranges and resolutions. The RMI4 core is in the mainline for I2C and SPI transport layers. RMI4 over HID devices is a work in progress that is just waiting for review, though some are already using RMI4 on HID over I2C. SMBus support is slated for 4.10.
Someone in the audience wondered how you would know if you need RMI4 support. Hutterer said that if you have a Synaptics touchpad, you will likely want RMI4 support. One side effect of these changes is that the touchpad device names will change. Anything that matches on the touchpad name in user space will end up with a "mute touchpad", he said.
Some new udev properties have been added for touchpads; one to identify trackball/touchpad combinations and another to identify touchpads that are integrated with the keyboard (as in a laptop), which are called "internal", versus those that are external. One use of the property is that only the internal touchpads will be disabled while the user is typing.
There have also been updates to the udev hardware database to make axis range corrections for some devices. The resolution in the database sometimes "does not match the real world". RMI4 will invalidate some of the database entries, as well, which he hopes is not a problem since RMI4 is much better protocol.
The goal for touchpad support in libinput is for it to be better than the xf86-input-synaptics driver. This is not universally true yet. One of the areas that needs work is pointer acceleration. Another is to make libinput responsive without using SIGIO, as had been done in the server in the past; more details about that would come later in the talk, he said.
The libinput developers tried to remove the touchpad hysteresis feature, which requires that a finger move a minimum distance before deciding that it actually is a movement, but that led to a whack-a-mole exercise trying to eliminate wobbly cursors for various devices. So hysteresis was eventually turned on again by default.
Another problem was that libinput would have cursor jumps that did not occur with the Synaptics driver. It turns out that changing between gestures (from two fingers to one for example) would cause the hardware to terminate the wrong finger "slot" so it would appear that the remaining finger jumped. The Synaptics driver is single-touch, so it did not experience the problem. Libinput was effectively penalized for trying to do something fancier.
Hutterer listed some quirky hardware that libinput has had to work around. Some Lenovo trackpoints send random events that trigger palm detection. Other Lenevo hardware randomly filters slower motions, which causes the cursor to stall. That was fixed by setting the acceleration slower on those devices so that users move their fingers faster and don't hit the firmware problem. It is one way to "train the users", an audience member said.
Tablets
Wacom still provides good support for its new tablets, he said. Other tablet support is "more random". It will not help that the DIGImend project, which targets better graphics tablet support for Linux, is shutting down because the maintainer doesn't have time to work on it.
The Wacom kernel driver (wacom.ko) has undergone a massive cleanup, Hutterer said. The biggest change that is visible to user space is that the LEDs are now controlled by the kernel through the LED subsystem. The driver used to require changes to support new devices, but it has switched to using HID descriptors. That allows integrated touch devices (screens and pads) to work out of the box. With luck, it will also allow future Wacom tablets to simply be plugged in and work, without needing an updated kernel for driver changes.
For libinput, tablet support is pretty much feature-complete these days. There is support for the tools, pads, and pad modes (including reporting the modes via the LEDs). Touch support has been changed so that the tablet is either treated as a giant touchpad or touchscreen. Tablet tools (e.g. stylus) are tracked with a serial number, which would allow them to be used on multiple tablets if the application supports it. The pads get their own event nodes, as do the various "rings", "strips", and buttons that are available on tablets.
For Wayland, the tablet protocol is now up to version 2. It is large, just because there is a lot of different kinds of data that is received from the tablets, but it is fairly straightforward, Hutterer said. The GTK+ tablet support is feature-complete at this point. There are still some GNOME configuration pieces that are partially missing, but otherwise there is nearly a full stack for tablets on GNOME.
Mice
He then turned to mice. The battery status handling for Logitech unifying receiver devices is being moved into the kernel. That will standardize the interface for GUIs. It is a work in progress at this point.
Libinput now provides the physical angle for mouse wheels as part of the event data. There is at least one mouse out there that has a different click angle for the horizontal wheel than it does for the vertical one—the horizontal wheel does not even have a way to do a click, but still sends those events at times. As with other devices, mice require some fixes for hardware quirks.
For Wayland, relative pointer information has been added to the protocol that provides both accelerated and unaccelerated motion data. It is something that was wanted for games, particularly "shooter games". There is also a protocol addition for pointer confinement—restricting the pointer to a particular surface (window). Both of those are not yet stable and need more testing. For XWarpPointer() support, which can't be done using the core Wayland protocol, there is a proposal to add it into XWayland.
The libratbag library, which was introduced at XDC 2015 as a library to configure gaming mice, has taken a back seat over the last year, he said. It supports quite a few different devices, with profile support so that configurations can be saved on the devices. As a follow-up to a question from last year, Hutterer told Keith Packard that the most expensive mouse the project has worked with cost $159 (up from roughly $100 last year, which Packard was incredulous about).
There is now a D-Bus daemon for libratbag (ratbagd), which will allow applications to configure these mice through the library. There is a GUI application called Piper, which is a standalone program, not part of GNOME Control Center or any other settings application. It is a rather specialized function that shouldn't be required frequently, he said. It doesn't store the configuration locally, it only changes the settings on the device itself.
One side effect of the libratbag work is a library to access Logitech unifying receivers called liblur. The idea is to hopefully replace the various home-cooked solutions out there, Hutterer said. There are "five, six, or even ten or more" of those available and there is really only a need for one library to handle operations like pairing a mouse or keyboard with a receiver.
Keyboards
The slide said "page intentionally left blank". Hutterer added: "We didn't do anything for keyboards because nothing needed to be done."
X server
Up through X server 1.18, all input events were handled using SIGIO. The user-space drivers would register a file descriptor with the server and install a SIGIO handler. Events would then trigger the signal, which would be processed in the signal handler. But there are lots of operations that cannot be done from a signal handler, memory allocation in particular.
This signal approach would not work for libinput as it needs to be able to allocate memory at times. For example, graphics tablet drivers cannot preallocate memory for all of the tools that might appear. Handling cursor movement, pointer acceleration, and other operations in the signal handler made event processing seem responsive. Not using that approach meant that libinput event handling would sometimes stall when the server was busy, which was especially noticeable when scrolling.
Thanks to Packard, there is a new input thread in the server. It simply waits for one of the input file descriptors to be ready, which makes libinput feel much more responsive. It did lead to some race conditions that needed to be tracked down, but most have been fixed at this point. The libinput test suite helped to find many of those, Hutterer said.
There is a plan to retire various specialized input drivers (e.g. Synaptics, Wacom) in favor of libinput. Traditionally, the evdev driver has been the catch-all driver and there have been some specialized drivers loaded for specific types of devices. Now, though, libinput is also a catch-all driver, but should be preferred over the specialized drivers. The X.Org configuration mechanism (xorg.conf.d) is not flexible enough to support this, however. It is based solely on the sort order of the files in the directory.
So, in server 1.19, there is now support for a new NoMatch directive that will only match a specialized driver if libinput has not already been attached to the device. In addition, the X server now has a built-in fallback driver to use if the driver module chosen is not available. In hindsight, he said, that should have been added in "2007 or so".
There has been some reshuffling of the entries in xorg.conf.d so they sort differently, as well. Instead of being near the bottom priority-wise (i.e. 98), libinput has been moved to 40, which puts it just below evdev and above all of the other specialized drivers. He noted that caused problems for desktop environments that do not support libinput (e.g. Cinnamon and MATE) and he strongly recommended that desktops move to support the library.
In summary, Hutterer said, input developers "are reaching peak boredom". There are mostly just little tweaks for hardware quirks and so on; there are no major features on the horizon. "That's a good thing", he said, since most "sane people" don't want their touchpads, mice, and other input devices to be exciting. For more details, interested readers can consult the slides or YouTube video from the talk.
[I would like to thank the X.Org Foundation for sponsoring my travel to Helsinki for XDC.]
Brief items
Development quotes of the week
As technologists, we tend to be overly enamored of our technology. For us, the device too often becomes the end goal, and we lose track of the genuine human need that we might otherwise meet. We talk about “technology solutions” but lose track of the problem we’re trying to solve (or we create entirely new ones). A subtle shift in perspective will help us build the right things: let’s stop aiming to invent science fiction and start making magic instead.
Dontbug: A reversible debugger for PHP
Sidharth Kshatriya has announced the public release of Dontbug. "Dontbug allows you to run PHP scripts forwards and backwards in the debugger. Practically speaking, you can step over/out backwards, step backwards, run backwards, run to cursor backwards, set breakpoints in the past and so forth (apart from the usual debugging capabilities when running forward)."
GDB 7.12 released
Version 7.12 of the GDB debugger is out. The biggest changes this time around appear to be support for the Andes NDS32 architecture and the ability to debug programs written in the Rust language.Kexi 3.0.0 and frameworks released
Kexi 3.0.0 and frameworks have been released. "For benefit and convenience of fellow software engineers, we're sharing frameworks originally developed within the Kexi Project (that is KDb, KProperty, KReport). There are marked with the same version 3.0.0 and are prepared for general, standalone usage. So this combined release marks a major milestone for two efforts: porting to Qt 5 started in 2014, development of the three frameworks started in 2008."
Newsletters and articles
Development newsletters
- Emacs News (October 10)
- What's cooking in git.git (October 6)
- What's cooking in git.git (October 11)
- This Week in GTK+ (October 11)
- OCaml Weekly News (October 11)
- Perl Weekly (October 10)
- PostgreSQL Weekly News (October 9)
- Python Weekly (October 6)
- Ruby Weekly (October 6)
- Wikimedia Tech News (October 10)
Bassi: Who wrote GTK+ 3.22
On the GTK+ Development Blog, Emmanuele Bassi looks at some statistics on the development of GTK+ 3.22 and GLib contributions during the same cycle (that resulted in GLib 2.50.0). He looks at which developers contributed the most change sets and changed lines of code, as well as how many change sets and hackers there are for each component by company affiliation. "During the 3.22 development cycle, GLib saw a total of 14119 lines added, 2031 removed, for a net gain of 12088 lines [...] GTK+, instead, saw a total of 46581 lines added, 19163 removed, for a net gain of 27418 lines". Those numbers do not include the translation work that was done for 3.22.
Page editor: Rebecca Sobol
Announcements
Brief items
The FSF seeks nominations for the annual Free Software Awards
The Free Software Foundation and the GNU Project are asking for nominations for the 19th annual Free Software Awards. The Award for the Advancement of Free Software will be presented to "an individual who has made a great contribution to the progress and development of free software, through activities that accord with the spirit of free software" and the Award for Projects of Social Benefit will be presented to "
the project or team responsible for applying free software, or the ideas of the free software movement, in a project that intentionally and significantly benefits society in other aspects of life." The deadline for nominations is November 6.
Announcing Google Code-in 2016 and Google Summer of Code 2017
The Google Open Source Programs Office has announced Google Code-in 2016 and Google Summer of Code 2017. Google Code-in is for students from 13-17 years of age who would like to explore open source. "Students will find opportunities to learn and get hands on experience with tasks from a range of categories. This structure allows students to stretch themselves as they take on increasingly more challenging tasks." Students will begin on November 28.
Student applications for Google Summer of Code (GSoC) open on March 20,
2017. Applications for interested open source organizations open on
January 19. GSoC "provides university students from around the world with an opportunity to take their skills and hone them by contributing to open source projects during their summer break from university.
"
Articles of interest
FSFE Newsletter - October 2016
This edition of the Free Software Foundation Europe newsletter covers internship opportunities, FSFE Summit retrospective, and several other topics.Here's Why Software Patents Are in Peril (Fortune)
Fortune covers a ruling [PDF] by the U.S. Court of Appeals for the Federal Circuit that invalidates three patents asserted against anti-virus companies Symantec and Trend Micro. "The most important part of the decision, which has created a stir among the patent bar, is a concurrence by Circuit Judge Haldane Mayer. In striking down a key claim from U.S. Patent 5987610, which claims a monopoly on using anti-virus tools within a phone network, Mayer says it is time to acknowledge that a famous Supreme Court 2014 decision known as “Alice” basically ended software patents altogether."
FSF: Tim Berners-Lee just gave us an opening to stop DRM in Web standards
The Free Software Foundation's Defective By Design campaign reports that Tim Berners-Lee decided not to exercise his power to extend the development timeline for the Encrypted Media Extensions (EME) Web technology standard. "Berners-Lee made his surprising decision on Tuesday, as explained in an email announcement by W3C representative Philippe Le Hégaret. Instead of granting a time extension — as he has already done once — Berners-Lee delegated the decision to the W3C's general decision-making body, the Advisory Committee. The Advisory Committee includes diverse entities from universities to companies to nonprofits, and it is divided as to whether EME should be part of Web standards. It is entirely possible that the Advisory Committee will reject the time extension and terminate EME development, marking an important victory for the free Web."
Calls for Presentations
CFP Deadlines: October 13, 2016 to December 12, 2016
The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.
| Deadline | Event Dates | Event | Location |
|---|---|---|---|
| October 19 | November 19 | eloop 2016 | Stuttgart, Germany |
| October 25 | May 8 May 11 |
O'Reilly Open Source Convention | Austin, TX, USA |
| October 26 | November 5 | Barcelona Perl Workshop | Barcelona, Spain |
| October 28 | November 25 November 27 |
Pycon Argentina 2016 | Bahía Blanca, Argentina |
| October 30 | February 17 | Swiss Python Summit | Rapperswil, Switzerland |
| October 31 | February 4 February 5 |
FOSDEM 2017 | Brussels, Belgium |
| November 11 | November 11 November 12 |
Linux Piter | St. Petersburg, Russia |
| November 11 | January 27 January 29 |
DevConf.cz 2017 | Brno, Czech Republic |
| November 13 | December 10 | Mini Debian Conference Japan 2016 | Tokyo, Japan |
| November 15 | March 2 March 5 |
Southern California Linux Expo | Pasadena, CA, USA |
| November 15 | March 28 March 31 |
PGConf US 2017 | Jersey City, NJ, USA |
| November 18 | February 18 February 19 |
PyCaribbean | Bayamón, Puerto Rico, USA |
| November 20 | December 10 December 11 |
SciPy India | Bombay, India |
| November 21 | January 16 | Linux.Conf.Au 2017 Sysadmin Miniconf | Hobart, Tas, Australia |
| November 21 | January 16 January 17 |
LCA Kernel Miniconf | Hobart, Australia |
| November 28 | March 25 March 26 |
LibrePlanet 2017 | Cambridge, MA, USA |
| December 1 | April 3 April 6 |
‹Programming› 2017 | Brussels, Belgium |
| December 10 | February 21 February 23 |
Embedded Linux Conference | Portland, OR, USA |
| December 10 | February 21 February 23 |
OpenIoT Summit | Portland, OR, USA |
If the CFP deadline for your event does not appear here, please tell us about it.
Upcoming Events
LibrePlanet 2017: "The Roots of Freedom"
LibrePlanet will take place March 25-26, 2017 in Boston, MA. The FSF has a reminder that registration is open and the proposal deadline is November 14.Events: October 13, 2016 to December 12, 2016
The following event listing is taken from the LWN.net Calendar.
| Date(s) | Event | Location |
|---|---|---|
| October 11 October 13 |
Embedded Linux Conference Europe | Berlin, Germany |
| October 13 October 14 |
Lua Workshop 2016 | San Francisco, CA, USA |
| October 13 | OpenWrt Summit | Berlin, Germany |
| October 17 October 19 |
O'Reilly Open Source Convention | London, UK |
| October 18 October 20 |
Qt World Summit 2016 | San Francisco, CA, USA |
| October 21 October 23 |
Software Freedom Kosovo 2016 | Prishtina, Kosovo |
| October 22 October 23 |
Datenspuren 2016 | Dresden, Germany |
| October 22 | 2016 Columbus Code Camp | Columbus, OH, USA |
| October 25 October 28 |
OpenStack Summit | Barcelona, Spain |
| October 26 October 27 |
All Things Open | Raleigh, NC, USA |
| October 27 October 28 |
Rust Belt Rust | Pittsburgh, PA, USA |
| October 28 October 30 |
PyCon CZ 2016 | Brno, Czech Republic |
| October 29 October 30 |
PyCon.de 2016 | Munich, Germany |
| October 29 October 30 |
PyCon HK 2016 | Hong Kong, Hong Kong |
| October 31 | PyCon Finland 2016 | Helsinki, Finland |
| October 31 November 1 |
Linux Kernel Summit | Santa Fe, NM, USA |
| October 31 November 2 |
O’Reilly Security Conference | New York, NY, USA |
| November 1 November 4 |
PostgreSQL Conference Europe 2016 | Tallin, Estonia |
| November 1 November 4 |
Linux Plumbers Conference | Santa Fe, NM, USA |
| November 3 | Bristech Conference 2016 | Bristol, UK |
| November 4 November 6 |
FUDCon Phnom Penh | Phnom Penh, Cambodia |
| November 5 | Barcelona Perl Workshop | Barcelona, Spain |
| November 5 November 6 |
OpenFest 2016 | Sofia, Bulgaria |
| November 7 November 9 |
Velocity Amsterdam | Amsterdam, Netherlands |
| November 9 November 11 |
O’Reilly Security Conference EU | Amsterdam, Netherlands |
| November 11 November 12 |
Seattle GNU/Linux Conference | Seattle, WA, USA |
| November 11 November 12 |
Linux Piter | St. Petersburg, Russia |
| November 12 November 13 |
T-Dose | Eindhoven, Netherlands |
| November 12 November 13 |
Mini-DebConf | Cambridge, UK |
| November 12 November 13 |
PyCon Canada 2016 | Toronto, Canada |
| November 13 November 18 |
The International Conference for High Performance Computing, Networking, Storage and Analysis | Salt Lake City, UT, USA |
| November 14 November 16 |
PGConfSV 2016 | San Francisco, CA, USA |
| November 14 November 18 |
Tcl/Tk Conference | Houston, TX, USA |
| November 14 | The Third Workshop on the LLVM Compiler Infrastructure in HPC | Salt Lake City, UT, USA |
| November 16 November 17 |
Paris Open Source Summit | Paris, France |
| November 16 November 18 |
ApacheCon Europe | Seville, Spain |
| November 17 | NLUUG (Fall conference) | Bunnik, The Netherlands |
| November 18 November 20 |
GNU Health Conference 2016 | Las Palmas, Spain |
| November 18 November 20 |
UbuCon Europe 2016 | Essen, Germany |
| November 19 | eloop 2016 | Stuttgart, Germany |
| November 21 November 22 |
Velocity Beijing | Beijing, China |
| November 24 | OWASP Gothenburg Day | Gothenburg, Sweden |
| November 25 November 27 |
Pycon Argentina 2016 | Bahía Blanca, Argentina |
| November 29 December 2 |
Open Source Monitoring Conference | Nürnberg, Germany |
| November 29 November 30 |
5th RISC-V Workshop | Mountain View, CA, USA |
| December 3 | NoSlidesConf | Bologna, Italy |
| December 3 | London Perl Workshop | London, England |
| December 6 | CHAR(16) | New York, NY, USA |
| December 10 | Mini Debian Conference Japan 2016 | Tokyo, Japan |
| December 10 December 11 |
SciPy India | Bombay, India |
If your event does not appear here, please tell us about it.
Page editor: Rebecca Sobol