LWN.net Weekly Edition for October 19, 2017

Welcome to the LWN.net Weekly Edition for October 19, 2017

This edition contains the following feature content:

• KRACK, ROCA, and device insecurity: reports from a bad week for security systems, with a special focus on KRACK.
• The trouble with text-only email: getting email delivered is hard enough to drive a privacy-oriented organization toward user tracking.
• A comparison of cryptographic keycards: which keycard is best for everyday crypto operations?
• unsafe_put_user() turns out to be unsafe: a dangerous in-kernel API leads to a vulnerability.
• Achieving DisplayPort compliance: a report from the X.Org Developers Conference.
• Point releases for the GNU C Library: should there be a glibc 2.26.1 release, or does glibc not need point releases at all?

This week's edition also includes these inner pages:

• Brief items: Brief news items from throughout the community.
• Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

KRACK, ROCA, and device insecurity

Monday October 16 was not a particularly good day for those who are even remotely security conscious—or, in truth, even for those who aren't. Two separate security holes came to light; one probably affects almost all users of modern technology. The other is more esoteric at some level, but still serious. In both cases, the code in question is baked into various devices, which makes it more difficult to fix; in many cases, the devices in question may not even have a plausible path toward a fix. Encryption has been a boon for internet security, but both of these vulnerabilities have highlighted that there is more to security than simply cryptography.

The ROCA vulnerability affects keys that have been generated on keycards, which is decidedly not an activity the general public has ever heard of, much less knowingly used. There are digital IDs that use keys generated that way, however, so some are being unknowingly exposed to the flaw. The other vulnerability has far more widespread applicability, however: KRACK is a flaw in the WPA2 encryption protocol that is used to secure most WiFi conversations. Both bugs are serious, but here we will focus on KRACK and leave ROCA to another day.

Key reinstallation attacks

Mathy Vanhoef discovered the WPA2 flaws, which center around a way to cause clients to reinstall the keys used to encrypt the traffic; thus the name comes from key reinstallation attacks, yielding KRACK. As is de rigueur for security vulnerabilities these days, KRACK has its own logo and web page.

When a client connects to an access point (AP) using WPA2, there is a four-way handshake that is done to establish a key to be used for the session. That handshake is encrypted using the shared secret (i.e. the WPA2 password) so a man in the middle should not be able to see the contents (which include the session key). It turns out, though, that the attacker doesn't need to see inside the packet; instead they can simply cause part of the handshake to be replayed, which will allow breaking the encryption from the client.

If the acknowledgment for the third message of the handshake gets lost (or is blocked by an attacker), the AP will resend that third message. That will cause the client to reinstall the key and, crucially, to reset its initialization vector (or "nonce") back to zero. The ciphers used by WPA2 are susceptible when nonce values are reused, so the protocol arranges that each packet gets a new nonce by incrementing a packet counter. Unfortunately, that packet counter gets reset to zero when the "new" key is installed. Worse yet, in some implementations, including all Android versions since 6.0, the client's copy of the key will have been cleared after it was installed into the network interface. The replayed third message will then result in the cleared key replacing the previously installed key, causing subsequent encryption to be done with an all-zeroes key.

As detailed in Vanhoef's paper [PDF], the problem is in the WPA2 protocol itself and is not a bug in the implementations. Even the key clearing in wpa_supplicant versions 2.4 and 2.5 (which ended up in Android) "appears to be caused by a remark in the Wi-Fi standard that suggests to clear the encryption key from memory once it has been installed for the first time", he said. One of the root causes of the problem is a poorly specified protocol that was created behind closed doors. As cryptographer Matthew Green put it:

He went on to point out that the IEEE has taken a few steps in the right direction to fix the problem, but that those steps are "hyper-timid incrementalist bullshit". The specifications are now available once they have been public for six months, "coincidentally, about the same time it takes for vendors to bake them irrevocably into their hardware and software". Beyond that, the standards are poorly specified and lack a formal description of the handshake state machine, which requires developers to "implement their code using scraps of pseudocode scattered around the standards document". That pseudocode leads directly to being vulnerable to KRACK.

Green also points out that those implementers bear some of the responsibility, particularly for the key clearing in wpa_supplicant:

Even though the four-way handshake and encryption scheme have been "proven" under various conditions as far back as 2005, Green said, those analyses did not take into account the integration of the handshake and the encryption. Each was separately proven to do the right thing, but the two were not analyzed together. He likened it to the "two unit tests, zero integration tests" meme.

Effects

With KRACK, a suitably positioned attacker could read the traffic being sent by a client to the AP. That traffic could contain various kinds of sensitive information but, in practice, it probably doesn't have all that much of interest. The ubiquity of HTTPS, SSL/TLS, and other layers of encryption (SSH, for example) for much of the traffic carried on networks these days makes KRACK somewhat less dangerous than it might first appear. Though, as Vanhoef notes, there have been various attacks against HTTPS along the way—there are sure to be more down the road.

Other attacks are possible, as well. A TCP connection could be hijacked and data could be injected into the session, for one thing. There are also three different handshakes that can be attacked under various circumstances. One of those can be used to get an AP to reinstall its key, which could then be used to decrypt traffic from the AP to the client. Injecting traffic in that direction could allow all kinds of nastiness.

KRACK is clearly a critical vulnerability that requires updating affected systems as quickly as possible. There is some traffic that is not further encrypted being carried on most WPA2 sessions; DNS queries, for example, might well contain information a user would rather not disclose. Injecting bogus DNS replies could also be highly problematic. Updating should be relatively straightforward for users of various desktop operating systems—wait for the vendor to release a fix (which most have done) and install it. The same goes for most "recent" mobile phones. Upgrading other WiFi devices may not be as easy—or even possible—for millions of devices.

Untold numbers of mobile phones, Internet of Things (IoT) devices, and other gadgets use WiFi (and WPA2) to communicate. Phone handsets typically have the best story regarding upgrades, in that they at least have the ability to be upgraded, but many existing phones are no longer receiving updates because their manufacturers deem them to be too old. Other devices may not even have a way to inform their owners that an upgrade is needed—or have no easy way for the owner to do an upgrade. As a Wired article said, it will take decades to clean all of this up.

It is a fairly bleak picture from a number of different viewpoints. One almost amusing outcome of this mess is contained near the end of Vanhoef's KRACK web page. He notified OpenBSD of the flaw in mid-July with an embargo (at the time) until the end of August. OpenBSD leader Theo de Raadt complained about the length of the embargo, so Vanhoef allowed OpenBSD to silently patch the flaw. "In hindsight this was a bad decision, since others might rediscover the vulnerability by inspecting their silent patch. To avoid this problem in the future, OpenBSD will now receive vulnerability notifications closer to the end of an embargo." That might not quite be the outcome De Raadt was hoping for with his (quite reasonable) complaint, especially given that Vanhoef strongly hints that there are other WiFi vulnerabilities in the pipeline.

Comments (29 posted)

The trouble with text-only email

Mozilla's manifesto commits the organization to a number of principles, including support for individual privacy and an individual's right to control how they experience the Internet. As a result, when Mozilla recently stated its intent to remove the "text only" option from its mailing lists — for the purpose of tracking whether recipients are reading its emails — the reaction was, to put it lightly, not entirely positive. The text-only option has been saved, but the motivation behind this change is indicative of the challenges facing independent senders of email.

The announcement that the text-only option would be removed was made (on the mozilla-governance list) by Michele Warther in late September. It quickly became clear that the community viewed this idea with a bit of skepticism, leading Warther to explain the reasoning behind the change:

The spam-ridden nature of the Internet forces anybody receiving email to be somewhat selective about what is allowed through. Only the most obscure and privately held email accounts can be exposed to an unfiltered mail stream without driving the owner insane. Much of the filtering applied is content-based, but much of it, especially at certain large web-based email providers, also takes into account the "reputation" of the sending site.

Some aspects of email reputation are straightforward. An IP address that is observed to be the source of volumes of spam will quickly find its way onto various online blacklists. When an IPv6 address is involved, the resulting block can cover a significant part of the address space, causing considerable collateral damage; this is why LWN's server is configured to send email via IPv4 whenever possible. IP addresses known to be used for residential Internet access are often penalized — if they are allowed to originate mail at all. Anybody who maintains their own email system can attest that reputation scoring also seems to have a significant random factor.

One metric that some sites evidently use is email sent to accounts that are known to be inactive, which is seen as a sign of a spammy originator. This, seemingly, is where Mozilla has run into trouble. One way to avoid this problem is to track which recipients are actually reading their email; any recipient who doesn't look at any messages for a period of time can then be unsubscribed. Then, in theory, email providers can see that the emails from a given source are actually of interest and refrain from putting up obstacles in their path.

The problem, of course, is that this tracking requires the "feedback loops" mentioned in Warther's message. These loops tend to take the form of tracking images that are fetched from a server belonging to the sender. The privacy implications of this kind of tracking are obvious: not everybody wants email senders to know when their mail was read and where the reader was at the time. Requiring this sort of disclosure would seem to run afoul of Mozilla Manifesto #4: "Individuals’ security and privacy on the Internet are fundamental and must not be treated as optional." But the alternative, Warther said, is an ongoing series of delivery problems for Mozilla's email in general.

There are other problems with tracking images and related mechanisms, starting with the fact that people who are paying attention tend to disable the loading of such images. Your editor recently received a complaint from a financial company that its emails were not being read; those emails were indeed read, they just weren't allowed to phone home and report that fact. Chances are good that this kind of blocking will increase in the future; not everybody wants to be a part of an unrequested "feedback loop".

In this particular case, it would seem that an acceptable compromise has been found, and the text-only option will remain. But, once a year, those subscribers will get a message asking them to click on a link to confirm their continued interest in remaining on the list. That should allow Mozilla to prune its inactive readers — a useful activity even without the reputation issues — without the need for involuntary tracking.

The fact that even principled organizations like Mozilla feel the need to employ tracking says something discouraging about the state of email, though, not that there was really any need for more evidence that the email system is broken. As the reputational checks become harder to pass, more users will be forced to use one of a small number of huge webmail providers (which have no trouble with "feedback loops") just to get their work done. Every kernel merge window features one or more developers having trouble getting their pull requests through to Linus Torvalds's Gmail account, for example. It's not clear what the solution to the email problem is, but the need is obvious.

Comments (119 posted)

A comparison of cryptographic keycards

An earlier LWN article showed that private key storage is an important problem to solve in any cryptographic system and established keycards as a good way to store private key material offline. But which keycard should we use? This article examines the form factor, openness, and performance of four keycards to try to help readers choose the one that will fit their needs.

I have personally been using a YubiKey NEO, since a 2015 announcement on GitHub promoting two-factor authentication. I was also able to hook up my SSH authentication key into the YubiKey's 2048 bit RSA slot. It seemed natural to move the other subkeys onto the keycard, provided that performance was sufficient. The mail client that I use, (Notmuch), blocks when decrypting messages, which could be a serious problems on large email threads from encrypted mailing lists.

So I built a test harness and got access to some more keycards: I bought a FST-01 from its creator, Yutaka Niibe, at the last DebConf and Nitrokey donated a Nitrokey Pro. I also bought a YubiKey 4 when I got the NEO. There are of course other keycards out there, but those are the ones I could get my hands on. You'll notice none of those keycards have a physical keypad to enter passwords, so they are all vulnerable to keyloggers that could extract the key's PIN. Keep in mind, however, that even with the PIN, an attacker could only ask the keycard to decrypt or sign material but not extract the key that is protected by the card's firmware.

Form factor

The four keycards have similar form factors: they all connect to a standard USB port, although both YubiKey keycards have a capacitive button by which the user triggers two-factor authentication and the YubiKey 4 can also require a button press to confirm private key use. The YubiKeys feel sturdier than the other two. The NEO has withstood two years of punishment in my pockets along with the rest of my "real" keyring and there is only minimal wear on the keycard in the picture. It's also thinner so it fits well on the keyring.

The FST-01 stands out from the other two with its minimal design. Out of the box, the FST-01 comes without a case, so the circuitry is exposed. This is deliberate: one of its goals is to be as transparent as possible, both in terms of software and hardware design and you definitely get that feeling at the physical level. Unfortunately, that does mean it feels more brittle than other models: I wouldn't carry it in my pocket all the time, although there is a case that may protect the key a little better, but it does not provide an easy way to hook it into a keyring. In the group picture above, the FST-01 is the pink plastic thing, which is a rubbery casing I received along with the device when I got it.

Notice how the USB connectors of the YubiKeys differ from the other two: while the FST-01 and the Nitrokey have standard USB connectors, the YubiKey has only a "half-connector", which is what makes it thinner than the other two. The "Nano" form factor takes this even further and almost disappears in the USB port. Unfortunately, this arrangement means the YubiKey NEO often comes loose and falls out of the USB port, especially when connected to a laptop. On my workstation, however, it usually stays put even with my whole keyring hanging off of it. I suspect this adds more strain to the host's USB port but that's a tradeoff I've lived with without any noticeable wear so far. Finally, the NEO has this peculiar feature of supporting NFC for certain operations, as we previously covered, but I haven't used that feature yet.

The Nitrokey Pro looks like a normal USB key, in contrast with the other two devices. It does feel a little brittle when compared with the YubiKey, although only time will tell how much of a beating it can take. It has a small ring in the case so it is possible to carry it directly on your keyring, but I would be worried the cap would come off eventually. Nitrokey devices are also two times thicker than the Yubico models which makes them less convenient to carry around on keyrings.

Open and closed designs

The FST-01 is as open as hardware comes, down to the PCB design available as KiCad files in this Git repository. The software running on the card is the Gnuk firmware that implements the OpenPGP card protocol, but you can also get it with firmware implementing a true random number generator (TRNG) called NeuG (pronounced "noisy"); the device is programmable through a standard Serial Wire Debug (SWD) port. The Nitrokey Start model also runs the Gnuk firmware. However, the Nitrokey website announces only ECC and RSA 2048-bit support for the Start, while the FST-01 also supports RSA-4096. Nitrokey's founder Jan Suhr, in a private email, explained that this is because "Gnuk doesn't support RSA-3072 or larger at a reasonable speed". Its devices (the Pro, Start, and HSM models) use a similar chip to the FST-01: the STM32F103 microcontroller.

Nitrokey also publishes its hardware designs, on GitHub, which shows the Pro is basically a fork of the FST-01, according to the ChangeLog. I opened the case to confirm it was using the STM MCU, something I should warn you against; I broke one of the pins holding it together when opening it so now it's even more fragile. But at least, I was able to confirm it was built using the STM32F103TBU6 MCU, like the FST-01.

But this is where the comparison ends: on the back side, we find a SIM card reader that holds the OpenPGP card that, in turn, holds the private key material and does the cryptographic operations. So, in effect, the Nitrokey Pro is really a evolution of the original OpenPGP card readers. Nitrokey confirmed the OpenPGP card featured in the Pro is the same as the one shipped by the Free Software Foundation Europe (FSFE): the BasicCard built by ZeitControl. Those cards, however, are covered by NDAs and the firmware is only partially open source.

This makes the Nitrokey Pro less open than the FST-01, but that's an inevitable tradeoff when choosing a design based on the OpenPGP cards, which Suhr described to me as "pretty proprietary". There are other keycards out there, however, for example the SLJ52GDL150-150k smartcard suggested by Debian developer Yves-Alexis Perez, which he prefers as it is certified by French and German authorities. In that blog post, he also said he was experimenting with the GPL-licensed OpenPGP applet implemented by the French ANSSI.

But the YubiKey devices are even further away in the closed-design direction. Both the hardware designs and firmware are proprietary. The YubiKey NEO, for example, cannot be upgraded at all, even though it is based on an open firmware. According to Yubico's FAQ, this is due to "best security practices": "There is a 'no upgrade' policy for our devices since nothing, including malware, can write to the firmware."

I find this decision questionable in a context where security updates are often more important than trying to design a bulletproof design, which may simply be impossible. And the YubiKey NEO did suffer from critical security issue that allowed attackers to bypass the PIN protection on the card, which raises the question of the actual protection of the private key material on those cards. According to Niibe, "some OpenPGP cards store the private key unencrypted. It is a common attitude for many smartcard implementations", which was confirmed by Suhr: "the private key is protected by hardware mechanisms which prevent its extraction and misuse". He is referring to the use of tamper resistance.

After that security issue, there was no other option for YubiKey NEO users than to get a new keycard (for free, thankfully) from Yubico, which also meant discarding the private key material on the key. For OpenPGP keys, this may mean having to bootstrap the web of trust from scratch if the keycard was responsible for the main certification key.

But at least the NEO is running free software based on the OpenPGP card applet and the source is still available on GitHub. The YubiKey 4, on the other hand, is now closed source, which was controversial when the new model was announced last year. It led the main Linux Foundation system administrator, Konstantin Ryabitsev, to withdraw his endorsement of Yubico products. In response, Yubico argued that this approach was essential to the security of its devices, which are now based on "a secure chip, which has built-in countermeasures to mitigate a long list of attacks". In particular, it claims that:

While I understand those concerns, they eventually come down to the trust you have in an organization. Not only do we have to trust Yubico, but also hardware manufacturers and designs they have chosen. Every step in the hidden supply chain is then trusted to make correct technical decisions and not introduce any backdoors.

History, unfortunately, is not on Yubico's side: Snowden revealed the example of RSA security accepting what renowned cryptographer Bruce Schneier described as a "bribe" from the NSA to weaken its ECC implementation, by using the presumably backdoored Dual_EC_DRBG algorithm. What makes Yubico or its suppliers so different from RSA Security? Remember that RSA Security used to be an adamant opponent to the degradation of encryption standards, campaigning against the Clipper chip in the first crypto wars.

Even if we trust the Yubico supply chain, how can we trust a closed design using what basically amounts to security through obscurity? Publicly auditable designs are an important tradition in cryptography, and that principle shouldn't stop when software is frozen into silicon.

In fact, a critical vulnerability called ROCA disclosed recently affects closed "smartcards" like the YubiKey 4 and allows full private key recovery from the public key if the key was generated on a vulnerable keycard. When speaking with Ars Technica, the researchers outlined the importance of open designs and questioned the reliability of certification:

This issue with open hardware designs seems to be recurring topic of conversation on the Gnuk mailing list. For example, there was a discussion in September 2017 regarding possible hardware vulnerabilities in the STM MCU that would allow extraction of encrypted key material from the key. Niibe referred to a talk presented at the WOOT 17 workshop, where Johannes Obermaier and Stefan Tatschner, from the Fraunhofer Institute, demonstrated attacks against the STMF0 family MCUs. It is still unclear if those attacks also apply to the older STMF1 design used in the FST-01, however. Furthermore, extracted private key material is still protected by user passphrase, but the Gnuk uses a weak key derivation function, so brute-forcing attacks may be possible. Fortunately, there is work in progress to make GnuPG hash the passphrase before sending it to the keycard, which should make such attacks harder if not completely pointless.

When asked about the Yubico claims in a private email, Niibe did recognize that "it is true that there are more weak points in general purpose implementations than special implementations". During the last DebConf in Montreal, Niibe explained:

Niibe recommends to "build the firmware yourself", also saying the design of the FST-01 uses normal hardware that "everyone can replicate". Those advantages are hard to deny for a cryptographic system: using more generic components makes it harder for hostile parties to mount targeted attacks.

A counter-argument here is that it can be difficult for a regular user to audit such designs, let alone physically build the device from scratch but, in a mailing list discussion, Debian developer Ian Jackson explained that:

Since that, as far as we know, the STM microcontrollers are not backdoored, I would tend to favor those devices instead of proprietary ones, as such a backdoor would be more easily detectable than in a closed design. Even though physical attacks may be possible against those microcontrollers, in the end, if an attacker has physical access to a keycard, I consider the key compromised, even if it has the best chip on the market. In our email exchange, Niibe argued that "when a token is lost, it is better to revoke keys, even if the token is considered secure enough". So like any other device, physical compromise of tokens may mean compromise of the key and should trigger key-revocation procedures.

Algorithms and performance

To establish reliable performance results, I wrote a benchmark program naively called crypto-bench that could produce comparable results between the different keys. The program takes each algorithm/keycard combination and runs 1000 decryptions of a 16-byte file (one AES-128 block) using GnuPG, after priming it to get the password cached. I assume the overhead of GnuPG calls to be negligible, as it should be the same across all tokens, so comparisons are possible. AES encryption is constant across all tests as it is always performed on the host and fast enough to be irrelevant in the tests.

I used the following:

• Intel(R) Core(TM) i3-6100U CPU @ 2.30GHz running Debian 9 ("stretch"/stable amd64), using GnuPG 2.1.18-6 (from the stable Debian package)
• Nitrokey Pro 0.8 (latest firmware)
• FST-01, running Gnuk version 1.2.5 (latest firmware)
• YubiKey NEO OpenPGP applet 1.0.10 (not upgradable)
• YubiKey 4 4.2.6 (not upgradable)

I ran crypto-bench for each keycard, which resulted in the following:

Algorithm	Device	Mean time (s)
ECDH-Curve25519	CPU	0.036
	FST-01	0.135
RSA-2048	CPU	0.016
	YubiKey-4	0.162
	Nitrokey-Pro	0.610
	YubiKey-NEO	0.736
	FST-01	1.265
RSA-4096	CPU	0.043
	YubiKey-4	0.875
	Nitrokey-Pro	3.150
	FST-01	8.218

There we see the performance of the four keycards I tested, compared with the same operations done without a keycard: the "CPU" device. That provides the baseline time of GnuPG decrypting the file. The first obvious observation is that using a keycard is slower: in the best scenario (FST-01 + ECC) we see a four-fold slowdown, but in the worst case (also FST-01, but RSA-4096), we see a catastrophic 200-fold slowdown. When I presented the results on the Gnuk mailing list, GnuPG developer Werner Koch confirmed those "numbers are as expected":

And yes, the FST-01 is really fast at doing ECC, but it's also the only keycard that handles ECC in my tests; the Nitrokey Start and Nitrokey HSM should support it as well, but I haven't been able to test those devices. Also note that the YubiKey NEO doesn't support RSA-4096 at all, so we can only compare RSA-2048 across keycards. We should note, however, that ECC is slower than RSA on the CPU, which suggests the Gnuk ECC implementation used by the FST-01 is exceptionally fast.

In discussions about improving the performance of the FST-01, Niibe estimated the user tolerance threshold to be "2 seconds decryption time". In a new design using the STM32L432 microcontroller, Aurelien Jarno was able to bring the numbers for RSA-2048 decryption from 1.27s down to 0.65s, and for RSA-4096, from 8.22s down to 3.87s seconds. RSA-4096 is still beyond the two-second threshold, but at least it brings the FST-01 close to the YubiKey NEO and Nitrokey Pro performance levels.

We should also underline the superior performance of the YubiKey 4: whatever that thing is doing, it's doing it faster than anyone else. It does RSA-4096 faster than the FST-01 does RSA-2048, and almost as fast as the Nitrokey Pro does RSA-2048. We should also note that the Nitrokey Pro also fails to cross the two-second threshold for RSA-4096 decryption.

For me, the FST-01's stellar performance with ECC outshines the other devices. Maybe it says more about the efficiency of the algorithm than the FST-01 or Gnuk's design, but it's definitely an interesting avenue for people who want to deploy those modern algorithms. So, in terms of performance, it is clear that both the YubiKey 4 and the FST-01 take the prize in their own areas (RSA and ECC, respectively).

Conclusion

In the above presentation, I have evaluated four cryptographic keycards for use with various OpenPGP operations. What the results show is that the only efficient way of storing a 4096-bit encryption key on a keycard would be to use the YubiKey 4. Unfortunately, I do not feel we should put our trust in such closed designs so I would argue you should either stick with 2048-bit encryption subkeys or keep the keys on disk. Considering that losing such a key would be catastrophic, this might be a good approach anyway. You should also consider switching to ECC encryption: even though it may not be supported everywhere, GnuPG supports having multiple encryption subkeys on a keyring: if one algorithm is unsupported (e.g. GnuPG 1.4 doesn't support ECC), it will fall back to a supported algorithm (e.g. RSA). Do not forget your previously encrypted material doesn't magically re-encrypt itself using your new encryption subkey, however.

For authentication and signing keys, speed is not such an issue, so I would warmly recommend either the Nitrokey Pro or Start, or the FST-01, depending on whether you want to start experimenting with ECC algorithms. Availability also seems to be an issue for the FST-01. While you can generally get the device when you meet Niibe in person for a few bucks (I bought mine for around $30 Canadian), the Seeed online shop says the device is out of stock at the time of this writing, even though Jonathan McDowell said that may be inaccurate in a debian-project discussion. Nevertheless, this issue may make the Nitrokey devices more attractive. When deciding on using the Pro or Start, Suhr offered the following advice:

So far, I have created a signing subkey and moved that and my authentication key to the YubiKey NEO, because it's a device I physically trust to keep itself together in my pockets and I was already using it. It has served me well so far, especially with its extra features like U2F and HOTP support, which I use frequently. Those features are also available on the Nitrokey Pro, so that may be an alternative if I lose the YubiKey. I will probably move my main certification key to the FST-01 and a LUKS-encrypted USB disk, to keep that certification key offline but backed up on two different devices. As for the encryption key, I'll wait for keycard performance to improve, or simply switch my whole keyring to ECC and use the FST-01 or Nitrokey Start for that purpose.

[The author would like to thank Nitrokey for providing hardware for testing.]

Comments (29 posted)

unsafe_put_user() turns out to be unsafe

When a veteran kernel developer introduces a severe security hole into the kernel, it can be instructive to look at how the vulnerability came about. Among other things, it can point the finger at an API that lends itself toward the creation of such problems. And, as it turns out, the knowledge that the API is dangerous at the outset and marking it as such may not be enough to prevent problems.

Back in late 2015, kernel developers were looking for ways to speed up the movement of data between user space and kernel space. Accessing user data from the kernel, whether for reading or writing, has clear security implications, so the kernel must ensure that the requested access is something that the requester is allowed to do. That involves checking that the intended area of memory is indeed accessible; on newer hardware, it can also require temporarily disabling mechanisms like supervisor mode access prevention (SMAP). Those preparations are expensive relative to a simple data copy, and they can add up if the kernel must perform a sequence of several reads and/or writes to user space.

The obvious solution is to do the preparations once for a series of operations, then perform the operations themselves directly; that amortizes the cost of a single set of checks across multiple accesses. To that end, Linus Torvalds added a new set of access functions for the 4.5 kernel:

    unsafe_get_user(value, source);
    unsafe_put_user(value, destination);

These "functions" are actually macros that expand differently depending on the type of source or destination. If, for example, source is a pointer to a u16 value, then unsafe_get_user() will fetch an unsigned 16-bit value from that location and store it in value. These functions are thus like the traditional get_user() and put_user() macros, but with one change: they dispense with the normal permission checking.

The lack of checks is the clear motivation for the "unsafe" naming, even if, according to the comments in the code, that term does not really apply:

    /*
     * The "unsafe" user accesses aren't really "unsafe", but the naming
     * is a big fat warning: you have to not only do the access_ok()
     * checking before using them, but you have to surround them with the
     * user_access_begin/end() pair.
     */

The required access_ok() call ensures that the pointer involved refers to a user-space address; it is there to prevent user space from asking the kernel to overwrite itself. The user_access_begin() and user_access_end() calls, instead, disable and enable SMAP. The whole idea behind the "unsafe" functions is that access_ok() and (especially) user_access_begin() can be called once before several user-space accesses, thus speeding up the code overall.

During the 4.13 merge window, Al Viro reworked the implementation of the waitid() system call to use the "unsafe" functions. This variant of wait() requests that the kernel fill in a siginfo_t structure in user space with information on how the waited-for process died; doing so requires writing several values to user space from the kernel. The new code switched to unsafe_put_user() to speed these writes. Viro duly called user_access_begin() and user_access_end(), but left out the access_ok() calls. As a result, a call to waitid() could ask that the siginfo_t structure be stored in kernel space, overwriting a piece of kernel memory. Local attackers tend to be most pleased by this sort of unintended functionality.

The result is deemed CVE-2017-5123; it only affects the 4.13 and 4.14-rc kernels. It is fixed by this commit in the mainline, and will be fixed in the 4.13.7 stable update. The good news is that it was caught early enough that the number of machines that are exposed to the problem should be quite small.

The bad news, of course, is that it happened at all. It turns out that this was the first use of unsafe_put_user() in the mainline kernel; this use was written by a developer who understands the issues involved, and he still got it wrong. Even better, the patch tickled a bug in that until-then unused function, with the result that Torvalds looked at the specific patch in question, but the problem still was not noticed. It would seem that, contrary to the comments, the "unsafe" functions are well named indeed, and that truly unsafe use does not jump out at people looking at the code — even if the functions proclaim themselves to be unsafe.

Hopefully reviewers in the future will be better tuned into the fact that there are two necessary preconditions to the use of those functions. Even better would be some code that runs in kernels configured for debugging that would detect a lack of checking and raise the alarm. It might well be possible to apply some sort of static checking to the problem as well. Without such assistance, it seems likely that this type of bug will find its way into the kernel again in the future; it may not be so quickly detected next time.

Comments (17 posted)

Achieving DisplayPort compliance

At the X.Org Developers Conference, hosted by Google in Mountain View, CA September 20-22, Manasi Navare gave a talk about her journey learning about kernel graphics on the way to achieving DisplayPort (DP) compliance for Intel graphics devices. Making that work involved learning about DP, the kernel graphics subsystem, and how to do kernel development, as well. There were plenty of details to absorb, including the relatively new atomic mode setting support, the design of which was described in a two-part LWN article.

Navare has been at Intel for three years; she started on the Chrome media team, but moved to the kernel graphics team after about a year. She has been working on the i915 driver, mostly on DP compliance. It has been a lot of fun learning about all of the different pieces that were required, she said. Most in the room responded positively to her query about whether they had DP, mini-DP, or USB Type-C connectors on their laptops. Many of those also agreed that they had seen the dreaded "random black screen" when connecting displays using those connectors. Those failures generally do not come with any information in the kernel logs, which is frustrating. She said the goal was to get rid of those random failures and the make the graphics hardware and software DP compliant.

Under the bus

She was "completely new" to the subject when she was "thrown under the bus" on this project, she said with a laugh. She was tasked with fixing the driver and making it 100% DP compliant but, being new, she didn't know where to start or even who to ask. Luckily, she had "amazing colleagues and a very supportive community" to help. She decided to break the problem down into "baby steps" that could be solved one at a time.

The first question she had was: "what is DP compliance?" VESA has an elaborate test procedure with an exhaustive set of tests to ensure that a DP driver is compliant with the specification. That is the test suite that was targeted.

One of the first steps she took was to understand what happens when a monitor is connected to a PC or other DP source. Connecting the cable causes the monitor to send a hotplug detect signal to the source. The source then reads the DP configuration data (DPCD) from the monitor to determine the DP capabilities supported by the device. At that point, the source and the monitor (also known as the DP sink) negotiate on the parameters to be used, then data can be sent.

Another thing she needed to understand is DP link training, which is what happens after the hotplug detect signal is received. The source will know about the maximum number of lanes and the maximum link rate supported by the cable, but it will need to determine the quality of the channels between source and sink. The source uses known training patterns to determine the link rate and how many lanes can be supported. Once the channels have been characterized, the link training is done and the two devices are able to exchange data.

She also needed to determine how to actually test for compliance. Her team chose a third-party device, the DPR-120, that has been certified by VESA to execute the compliance test suite. It sits between the source and sink, requesting various tests that display as test patterns on the sink. It compares the patterns to its reference patterns to determine whether the test passes or fails.

Next up was figuring which pieces of hardware and code were responsible for DP compliance. At the lowest layer is the GPU, above that is the i915 driver, which is controlled by the direct rendering manager (DRM) subsystem in the kernel. That exposes an API to user-space components so that programs can send their graphics requests to the hardware. All of those pieces are involved in DP compliance and must be considered in the process.

Mode setting is the process of setting up the scanout buffers, CRTC ("cathode ray tube controller", which is a historical name), and encoders to send the proper display information to the connectors, thus to the sink. That is the responsibility of the kernel. Over the last few years, though, atomic mode setting has been added to make those changes in a single operation in order to prevent partial updates, which can result in display glitches.

When user space requests a set of properties for the display device, atomic mode setting will build the corresponding DRM objects internally and validate them based on its knowledge of the hardware. If the atomic_check() validation step passes, then the atomic_commit() can be done to actually update the hardware with the new configuration. Inherent in that model is the idea that atomic_commit() cannot fail if the check passes, she said.

Handling link failure

When she reached that level of understanding, she believed that atomic mode setting would just make things work and that DP compliance was simply a matter of running the tests. But that turned out not to be the case; some of the compliance tests failed. The problem is that the driver has to do the link training after the commit; if the link training fails at that point, it results in a black screen.

What needs to happen when there is a link failure is for the display pipeline to be set up again with different parameters, but that can't happen until user space requests a new mode. The fix was for the kernel to reduce its saved information about the maximum values for link rate or number of lanes and to send an event to user space informing it of the link training failure. User space can then request a new mode, which results in a new atomic mode setting operation; if it succeeds, the mode set is complete, if it fails, the kernel falls back again and sends another event. That may result in a lower resolution display, but it does not result in the dreaded black screen.

Some lessons were learned in the process; "failure is always an option", she said. The atomic_check() guarantees that the requested mode is possible for the GPU parameters, but cannot guarantee the DP cable and link. Failure of the link can also occur asynchronously; a previously working link can fail due to electrical properties of the connection. It is also possible to do a non-blocking commit, where the mode set returns to user space before the commit has been completed, so there needs to be a way to alert user space to these kinds of problems. Being able to handle asynchronous link failures is an important component of the solution.

There was a bit of a chicken-and-egg problem once the kernel solution was found, though. She made changes to the graphics driver, but without changes to the X server, the tests still would not pass. And an updated X server running on a kernel without the changes would also fail. It required coordinated changes in both to get to a point where the DP compliance could be tested.

Going forward, Intel wants to ensure that any patches made to the driver do not result in regressions to the DP compliance, Navare said. To that end, she has added a DP compliance tool to the IGT test suite that will handle the link failures and request a new mode setting. That way, the DPR-120 can be used to test changes to the graphics driver to catch regressions.

For the future, the plan is to move away from the DPR-120 and to instead use Google's open hardware Chamelium board for display testing. That will allow testing more connector types than just DP—HDMI for example. In order to use it, the DP compliance test suite needs to be ported to the Chamelium and the IGT DP compliance tool will need to be added to the Chamelium testing tool. With that in place, DP compliance testing could be added to the pre-merge continuous-integration tests that Intel is running for its graphics stack.

The whole process "definitely felt like a journey of a thousand miles", she said. There was an enormous amount to learn and it took a fair bit of time to get there. She started working on it at the beginning of 2016 and got the kernel patches merged into 4.12 in mid-2017. She was hesitant at first to post her patches to the mailing list, but learned that doing so was a great way to get good feedback on what she was trying to do. The community was helpful and the feedback was always constructive, she said.

There was some discussion in the Q&A about whether link training could be done at hotplug time so that the X server or other user-space components would not need to handle link failures. It was something that was tried along the way, Navare said, but it can't really work for a number of reasons. In addition, there is always the possibility of a link failure after the mode set call has returned to user space, so the link-failure event handling is still needed.

Interested readers may wish to view the YouTube video of the talk or check out Navare's slides [PDF].

[I would like to thank the X.Org Foundation and the Linux Foundation for travel assistance to Mountain View for XDC.]

Comments (5 posted)

Point releases for the GNU C Library

The GNU C Library (glibc) project produces regular releases on an approximately six-month cadence. The current release is 2.26 from early August; the 2.27 release is expected at the beginning of February 2018. Unlike many other projects, though, glibc does not normally create point releases for important fixes between the major releases. The last point release from glibc was 2.14.1, which came out in 2011. A discussion on the need for a 2.26 point release led to questions about whether such releases have a useful place in the current software-development environment.

The glibc 2.26 release is generally only found in relatively fast-moving distributions at this point. For most users, 2.26 has been without problems, but that is not true for everybody. There have been a few significant regressions in this release that have required fixes; one of those was seen as important enough that the question of creating a 2.26.1 point release was raised. Romain Naour subsequently brought that discussion to the libc-alpha mailing list. Having a point release containing important fixes would be helpful to downstream distributors that want to incorporate those fixes, he said.

Like many projects, glibc fixes bugs first in its repository trunk — the branch that will eventually become 2.27 at the moment. The project does not make point releases, but that does not mean it leaves users of its previous releases entirely out in the cold. A branch in the repository is created for each release and, at the discretion of the manager for that particular release, important fixes are backported and added to that "maintenance branch" post-release. So distributors have a stream of nicely backported fixes for each release; what they don't have are point releases that contain a well-defined set of those fixes.

Some glibc developers strongly question the need for those releases. Creating a formal release is a significant amount of work, involving running tests, writing release notes, creating tarballs, etc. A major release is an indication that a point of stability has been reached and the code is in a state that is meant to be ready for production use. The maintenance branches, instead, are meant to always be in such a state. A point release would just be an arbitrary marker placed in the stream of maintenance patches, so one might as well just use "git describe" output to name that point. Thus, Florian Weimer observed that "the difference between glibc 2.26.5 and glibc 2.26-40 seems rather minor to me". He suggested that all that was really needed was a special tag at the beginning of each release branch that would show up in the output of a git describe command.

Representatives from a few distributions made it clear that they felt differently, though. Arjan van de Ven noted that a release tag helps distributors and users to know what is in a given package:

Andreas Huettel, representing Gentoo, agreed that there is value in having a well-known reference point. Yann Morin, of Buildroot, also agreed, noting that these reference points would be especially helpful to smaller distributors that do not have the bandwidth to follow the maintenance branch closely.

In response to the discussion, Siddhesh Poyarekar, the 2.26 release manager, said that he would probably do a 2.26.1 release in the near future. He explicitly did not commit to doing any point releases beyond that one, though. He also said that a point release, if he creates one, should not be seen as a commitment by the project as a whole to create such releases in the future. Carlos O'Donell stressed that last point, saying that "we don't want to set a precedent that the glibc release manager is always going to do a point release".

O'Donell went on to say that the glibc community in general lacks the resources needed to put together regular point releases. But, he said, downstream distributors might be able to create those releases with "a little help" from the glibc developers. He has promised a proposal describing how that cooperation might work, but it has not been posted as of this writing. He has, however, come out in favor of Weimer's special-tag proposal.

One might be tempted to conclude from this discussion that the need for our release-based software-distribution conventions is waning over time. When every point in a project's history already has a unique name (its Git commit ID), and when growing use of continuous-integration techniques leads to a situation where repository contents can be run at arbitrary points with relatively high confidence, perhaps the release rigmarole can be done away with, at least for point releases. But anybody thinking so should consider the ways in which glibc is unique.

The C library sits between most applications and the kernel. While many users install their own application releases, and a certain number are happy to replace the kernel, it is relatively rare to install a new glibc release. The glibc project moves slowly, new releases tend to have little in the way of exciting new features, and glibc is a rather difficult component to replace so, for the most part, whatever is on the system is good enough for most users. Few people feel the need to be running whatever the latest release is, so they tend to care rather less about glibc point releases than about releases of other components in their systems.

Distributors, instead, pay a lot of attention to the C library. Once they have installed a specific version, they are likely to scrutinize any follow-on patches and will not necessarily accept every change that appears in the glibc maintenance branch. So some distributors, at least, may see little value in point releases for glibc, while others (the smaller ones, perhaps) see some use there.

In other words, glibc may be unique in its relative lack of need for regular point releases; the rest of the community is unlikely to want to trade in concise version numbers for random-seeming commit IDs. But, even in the corner of the community where glibc lives, some users would rather have the occasional well-defined point release, with the result that glibc 2.26.1 may make an appearance in the near future.

Comments (18 posted)