The first half of the 4.17 merge window

Benefits for LWN subscribers

The primary benefit from subscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today!

By Jonathan Corbet
April 5, 2018

As of this writing, 5,392 non-merge changesets have been pulled into the mainline repository for the 4.17 release. The 4.17 merge window is thus off to a good start, but it is far from complete. The changes pulled thus far cover a wide part of the core kernel as well as the networking, driver, and filesystem subsystems.

Some of the more significant changes merged so far are:

Core kernel

The ever-expanding perf_event_open() system call has gained the ability to place a kprobe or uprobe and create an event associated with it; the probe continues to exist for the life of the resulting file descriptor. Probes created this way will not be visible in the tracefs virtual filesystem. This interface was created as a way of ensuring that probes will be cleaned up when the process that created them exits.
The scheduler's load estimation code has been improved, especially for mobile and embedded workloads.
The new BPF_RAW_TRACEPOINT command for the bpf() system call attaches a BPF program to a tracepoint but performs no processing of the tracepoint arguments before calling that program. That enables tracing with minimal overhead, but requires more awareness when writing BPF programs. See this commit for a terse overview of the feature.

Architecture-specific

Support for the Andes Technologies NDS32 architecture has been added.
As described in this article, support for the blackfin, cris, frv, m32r, metag, mn10300, score, and tile architectures has been removed; it seems that none of them had any remaining users. The merge commit for this removal shrinks the kernel by almost 470,000 lines of code.
The SPARC "application data integrity" feature is now supported; it allows the application of tags to virtual-memory addresses. The tag is a four-bit value that is placed in the upper bits of the address; any reference lacking the proper tag will generate a trap. See this commit for details.

Filesystems

The XFS filesystem now supports the lazytime mount option.
The Btrfs filesystem has long supported a set of ioctl() operations to control transactions. It has been concluded that these operations are unused, so they have been removed for 4.17.
The CIFS filesystem now supports SMB 3.1.1 pre-authentication integrity. SMB 3.1.1 is also no longer marked "experimental".
The ext4 filesystem has been made more robust against maliciously crafted filesystem images. Maintainer Ted Ts'o warns "I still don't recommend that container folks hold any delusions that mounting arbitrary images that can be crafted by malicious attackers should be considered sane thing to do, though!"

Networking

The reliable datagram socket protocol now supports zero-copy operation.
It is now possible to apply BPF scripts to filter traffic sent by the sendmsg() and sendfile() system calls. See this commit for details.
A receive-side implementation of the TLS protocol has been added. This adds to the existing transmit implementation to give full in-kernel TLS protocol support.
A set of control-group-specific BPF hooks has been added to the bind() and connect() system calls; attached programs can modify how those calls work. Some information can be found in this commit.

Hardware support

Graphics: ARM Versatile panels, Allwinner DesignWare HDMI phys, AMD Vega12 GPUs, and Raydium RM68200 720x1280 DSI panels.
Media: NXP TDA1997x HDMI receivers, Omnivision ov2685 and ov5695 sensors, Renesas capture engine units, Sony CXD2880 DVB-T2/T tuner/demodulators, and SoundGraph iMON receivers.
Miscellaneous: Marvell 88PG86X voltage regulators, Aspeed KCS IPMI interfaces, and Amiga Gayle PATA controllers.
Networking: NXP MCR20AVHM transceivers, Microchip LAN743x gigabit Ethernet interfaces, and Intel Ethernet connection E800 series interfaces.
Pin control: Qualcomm SDM845 pin controllers, NXP IMX6SLL pin controllers, MediaTek MT2712 pin controllers, and Allwinner H6 pin controllers.

Miscellaneous

The perf script command now supports Python 3 scripts.
The Linux kernel memory model is now a part of the kernel proper. It includes a formal description of how memory coherency works in the kernel, along with an extensive set of tests to prove adherence to the model. See this commit for an overview of what has been merged.

Internal kernel changes

Building the kernel for x86 now requires a compiler with asm goto support. In practice that means that GCC 4.5 or later is now needed. It also rules out the use of Clang on x86 until that compiler gains asm goto support.
wait_on_atomic_one() has been replaced by wait_event_var(), a more general interface; see this article for details.
A massive kernel-wide cleanup has removed almost all direct invocations of system-call implementations from within the kernel. This is useful for a number of reasons: it adds flexibility to the system-call interface and makes it easier to remove set_fs() calls, among other things. A significant change to the x86 system-call mechanism is in the works for the near future.

If this turns out to be a normal two-week merge window, it can be expected to remain open until April 15, with the final 4.17 release happening in early June. The remainder of the changes merged for this release will be summarized once the merge window closes.

Index entries for this article
Kernel	Releases/4.17

(Log in to post comments)

The first half of the 4.17 merge window

Posted Apr 6, 2018 10:09 UTC (Fri) by grawity (subscriber, #80596) [Link]

Maintainer Ted Ts'o warns "I still don't recommend that container folks hold any delusions that mounting arbitrary images that can be crafted by malicious attackers should be considered sane thing to do, though!"

So in other words, it's okay if Linux has even bigger security holes than Windows' much-derided autorun.inf, because the user is to blame if they don't epoxy their USB ports and someone connects a malicious ext4 USB stick.

The first half of the 4.17 merge window

Posted Apr 6, 2018 11:26 UTC (Fri) by knan (subscriber, #3940) [Link]

Troll somewhere else, or fix it yourself.

The first half of the 4.17 merge window

Posted Apr 6, 2018 14:24 UTC (Fri) by sorokin (subscriber, #88478) [Link]

When I was told that all mounted drives are considered trusted by Linux file-system developers I was surprised. This was a few years ago and it still amazes me. I would say that calling grawity a troll is a bit unjustified. The fact that kernel can be broken with a disk image is very unintuitive and unexpected.

I mean as a programmer I realize that it is extremely difficult to handle all corner cases for what can be written on disk in the place of the file-system data structures. But what about flash drives or CD-images? They are widely used. Can we have at least one filesystem, say FAT or ISO-9660, that can be used for untrusted media? Sounds like a good application for fuzz-testing.

The worst thing is when an attacker can make its own device. The rogue disk can alter the data while the disk is being used. Who knows what kind of confusion this can create when kernel works with this disk.

The first half of the 4.17 merge window

Posted Apr 7, 2018 15:40 UTC (Sat) by hmh (subscriber, #3838) [Link]

Bugs are fixed when found, and actively fuzzed for. This covers untrusted data as in corrupted-on-purpose filesystem images. It is still a large surface where bugs can stay undetected for quite a while when one is not lucky.

However, if you want to not trust ACLs, owners, modes, device inodes, etc. then you are to mount the filesystem with the proper options. It is userspace's responsibility.

User-mounted Vfat, udf and iso filesystems are usually handled as unsafe well enough. Ext, btrfs, xfs... it depends. Again, for the better or for the worse, it is on userspace to select the appropriate mount policy.

And the kernel does allow per-usb-port activation control, including whitelist-based. Again, you need userspace to set this policy...

The first half of the 4.17 merge window

Posted Apr 6, 2018 15:00 UTC (Fri) by ebiederm (subscriber, #35028) [Link]

Ted has committed to fixing any known issues, and from what I have seen has been doing a good job with ext4.

The problem is that ext4 is a large code base and on the disk side has a large attack surface and was never built
to withstand attack from a malicious block device. Given how long I have been working on issues so that fuse
(which was built to be safe) is actually safe to mount unprivileged I expect it will take years of work before
there is confidence that ext4 has no exploitable holes if used on a malicious block device.

For dealing with the issue I recommend using fuse and running your filesystem drivers in userspace for
dealing with untrusted/external media.

The first half of the 4.17 merge window

Posted Apr 6, 2018 15:18 UTC (Fri) by epa (subscriber, #39769) [Link]

I guess some filesystems should by default be disabled for removable devices.

The first half of the 4.17 merge window

Posted Apr 6, 2018 15:54 UTC (Fri) by pizza (subscriber, #46) [Link]

For most block devices, the "removable/not-removable" distinction is purely mechanical in nature, and not something that software has any hope of determining.

The first half of the 4.17 merge window

Posted Apr 6, 2018 16:49 UTC (Fri) by droundy (subscriber, #4559) [Link]

True, but the software that decides to automatically mount a USB stick or DVD when it is inserted knows that this is an external device.

The first half of the 4.17 merge window

Posted Apr 6, 2018 17:20 UTC (Fri) by k8to (subscriber, #15413) [Link]

I'd be perfectly happy with the system as a whole simply never mounting a filesystem without explicit user authorization.
Some kind of fstab entry is reasonable for stable filesystems as an indication of authorization, but for random USB insertions, why not ask the live user if it's OK to mount?

Specialized devices without a traditional user interface might have to get a little creative, of course.

Automounting

Posted Apr 6, 2018 21:36 UTC (Fri) by corbet (editor, #1) [Link]

In fact, some of us turn off the "helpful" desktop code that wants to just go ahead and mount anything that shows up. Automatic mounting is not an inherent Linux feature.

Automounting

Posted Apr 8, 2018 0:05 UTC (Sun) by k8to (subscriber, #15413) [Link]

Indeed. It was a comment on "recent" trends in Linux desktop implementations. I try to disable all of these, but sometimes I don't get full freedom to reconfigure my laptop before beginning work.

The first half of the 4.17 merge window

Posted Apr 7, 2018 7:11 UTC (Sat) by karkhaz (✭ supporter ✭, #99844) [Link]

The idea that filesystems are always auto-mounted is totally new to me, none of the laptops or workstation that I own do this. I don't think I did anything special to prevent it, either.

Where does this functionality come from? It's certainly not the kernel that does this. Maybe distributions patch it in, or maybe some of the more heavyweight desktop environments do this on the user's behalf by default?

The first half of the 4.17 merge window

Posted Apr 7, 2018 13:30 UTC (Sat) by pizza (subscriber, #46) [Link]

> Where does this functionality come from? It's certainly not the kernel that does this. Maybe distributions patch it in, or maybe some of the more heavyweight desktop environments do this on the user's behalf by default?

The latter, working in combination with the likes of udisks to detect the media and muck with permissions so that the user can do anything with it.

At least in GNOME's case, it's also completely configurable, including being able to disable it entirely.

The first half of the 4.17 merge window

Posted Apr 8, 2018 12:10 UTC (Sun) by niner (subscriber, #26151) [Link]

FWIW at least KDE requires you to actually click on "Open in File Manager" in the device notifier or on the device in the file manager to actually mount a removable device. So it behaves as you described it should do. I wonder which systems actually do mount automatically without any user participation?

The first half of the 4.17 merge window

Posted Apr 6, 2018 17:30 UTC (Fri) by pizza (subscriber, #46) [Link]

Okay, that's fair, but what exactly is udisks/etc supposed to do instead of mounting the drive? It's not like the user will ever answer "no" to the question "do you want to access the usb stick you just plugged in?"

Granted, udisks/etc could be set up to automatically run fsck on the new filesystems prior to mounting them, but that carries its own set of risks and implications (eg a mechanism to report failures, interactive prompts to allow fixing problems, etc etc...)

Mounting an external drive

Posted Apr 6, 2018 21:42 UTC (Fri) by ebiederm (subscriber, #35028) [Link]

For mounting the drive I would recommend running a file system driver in user space with fuse. I have seen fuse drivers available for most filesystems. That will be quite a bit more robust than using a kernel driver, and the exploit would be less severt. Especially if the filesystem process is sandboxed from the rest of the system.

It definitely makes sense to wait until a user asks. Some devices present as multiple usb devices, sometimes you get confused and plug the wrong device. Waiting for the user to say mount the drive now limits nefarious actions to when a user is actively watching which is more difficult to hide.

So even with in-kernel filesystem drivers that have not been built to be robust against malicious usb sticks there are things you can do that will make an attackers life more difficult.

Mounting an external drive with FUSE

Posted Apr 7, 2018 16:03 UTC (Sat) by alison (subscriber, #63752) [Link]

What is the best way to mount a ext4- or FAT-formatted USB stick with FUSE? 'man mount' and 'man fuse' offer few clues, as does /etc/fuse.conf. 'mount -t fuse /dev/sdc1 /mnt' doesn't work.

I gather that mounting block devices 'noexec' is in sufficient? If the filesystem metadata is, for example, designed to create a stack overflow and 'mount' is executed as root, it's easy to see why there could be a problem.

Mounting an external drive with FUSE

Posted Apr 8, 2018 0:04 UTC (Sun) by pabs (subscriber, #43278) [Link]

There are at least two projects that allow the use of Linux kernel code as user-space shared libraries, perhaps they could lead to the use of Linux kernel filesystem drivers in userland via FUSE.

https://www.phoronix.com/scan.php?page=news_item&px=I...
https://www.phoronix.com/scan.php?page=news_item&px=L...

Mounting an external drive with FUSE

Posted Apr 8, 2018 21:51 UTC (Sun) by OttoErickson (guest, #122996) [Link]

Tanenbaum would be so proud.

Mounting an external drive

Posted Apr 12, 2018 15:14 UTC (Thu) by bfields (subscriber, #19510) [Link]

"For mounting the drive I would recommend running a file system driver in user space with fuse. I have seen fuse drivers available for most filesystems. That will be quite a bit more robust than using a kernel driver, and the exploit would be less severe. Especially if the filesystem process is sandboxed from the rest of the system."

I'd worry that the userspace driver will also get less maintenance. I guess the sandboxing could be pretty restrictive if it literally only needs access to the one device and the fuse interface? On the other hand the ability to return arbitrary data and metadata, unexpected errors, etc., could offer a lot of potential attacks against any application accessing that filesystem.

I don't know, it seems like a hard problem to me.

Every time we have this discussion I have flashbacks to some 30 years ago--weren't removable media (floppies) a primary vector for malware?

Mounting an external drive

Posted Apr 13, 2018 8:03 UTC (Fri) by lacos (subscriber, #70616) [Link]

"For mounting the drive I would recommend running a file system driver in user space with fuse"

You can also use guestfish or guestmount from the libguestfs project. Those will use kernel drivers, but the kernel will be run in a virtual machine. (In fact, guestmount is a special case of fuse.)

http://libguestfs.org/guestfs-security.1.html
http://libguestfs.org/guestfs-faq.1.html#how-does-libgues...
http://libguestfs.org/guestmount.1.html

The first half of the 4.17 merge window

Posted Apr 7, 2018 0:53 UTC (Sat) by rgmoore (✭ supporter ✭, #75) [Link]

It's not like the user will ever answer "no" to the question "do you want to access the usb stick you just plugged in?"

That isn't necessarily the case. For example, I occasionally just want to format the device I've just inserted rather than mounting it. Sure, I can just unmount it first, but I might be paranoid enough not to want to mount it at all if it came from a questionable source.

Asking the user would also help in the case of a malicious device that the user doesn't know contains storage at all. Since USB devices are designed to be chained, it's possible for a single physical USB device to contain multiple logical devices. Some devices use this for good- ISTR there are some password storage sticks that contain a virtual keyboard device so they can type passwords into the correct space on forms- but it's easy to see how it could be used for evil. If I think I've just plugged in my USB powered keyboard vacuum and my system asks me if I want to mount the disk, I'm very unlikely to click "OK".

The first half of the 4.17 merge window

Posted Apr 7, 2018 2:12 UTC (Sat) by pizza (subscriber, #46) [Link]

Folks like you and I are several sigmas beyond the mean, and it is extremely foolish to generalize our relatively-clueful paranoia onto the general populace.

It is widely established that users will overwhelmingly say "yes" to any and every prompt confirming potentially risky behaviour. Usually without any conscious thought.

Especially given that the overwhelmingly common case is "Of course I want to access the files on the stick I plugged in! That's why I plugged it in!"

By all means, let's emulate Windows here, perform a quick read-only fsck upon insertion. and prompt if there's a potential problem, but no amount of filesystem-level paranoia will save the user if they click on the file labelled "hot avian pr0n.sh" and get their account (and more importantly, all the data they care about) completely compromised.

The first half of the 4.17 merge window

Posted Apr 7, 2018 12:25 UTC (Sat) by roc (subscriber, #30627) [Link]

One underlying cause of the slow pace of Linux desktop adoption, which you can see in the comments in this article, is that many Linux users and contributors don't much care whether desktop Linux is usable by "the general populace".

The first half of the 4.17 merge window

Posted Apr 8, 2018 9:33 UTC (Sun) by flussence (subscriber, #85566) [Link]

Linux hasn't exactly done better for lack of an equivalent to Windows XP's “what do you want to do with this device” autorun popup, or the infamous Have Disk prompt of despair for a hotplugged device. Sometimes I wish it did ask me before splatting a .ko into kernel memory in the background for those events; the USB authorization mechanism is already there and sits unused.

Or to put it more simply: I want a PoisonTap to present *at least* the same deliberately terror-inducing UI as I currently get for a self-signed SSL cert. Because one of these things is actually dangerous.

The first half of the 4.17 merge window

Posted Apr 10, 2018 3:47 UTC (Tue) by pabs (subscriber, #43278) [Link]

Install one of these:

https://usbguard.github.io/
https://github.com/kochstefan/usbauth-all/

The first half of the 4.17 merge window

Posted Apr 9, 2018 12:56 UTC (Mon) by cortana (subscriber, #24596) [Link]

Here's my suggestion for UI that will allow power users to confirm mounting without an annoying prompt that will turn off average users.

When you insert a disk, nautilus appears as normal (with the disk selected on the left-hand pane), but instead of the disk being mounted and the files displayed on the main pane, a prompt and button are displayed instead:

  +----------------------------------------------------+
  |            |                                       |
  | Recent     | Mount the device "USB disk"?          |
  | Home       |                                       |
  | Documents  | [x] Remember for this disk            |
  | Downloads  |                                       |
  | Music      | [Mount] [Eject]                       |
  | Pictures   |                                       |
  | Videos     |                                       |
  | Wastebasket|                                       |
  |            |                                       |
  +-USB-disk---+                                       |
  |            |                                       |
  |            |                                       |
  |            |                                       |
  |            |                                       |
  |            |                                       |
  |            |                                       |
  |            |                                       |
  +----------------------------------------------------+

The first half of the 4.17 merge window

Posted Apr 9, 2018 14:44 UTC (Mon) by jem (subscriber, #24231) [Link]

How do you define "this disk"? Is the disk the same after being reformatted? Is it regarded different after the file system has been doctored to exploit a vulnerability?

How do you know what the right answer (mount/eject) to the question is? If you have plugged it in, your intention was to use the disk. Why would you suddenly not want to use it after all?

The problem with removable storage is similar to running executables from an unknown source (except the storage problem is possibly worse, because the file system code is in the kernel). If the disk doesn't come from a trustworthy source, don't plug it in. Also, don't allow strangers to plug in disks at all on machines you are in charge of. I don't see the added value of asking "are you sure?"

The first half of the 4.17 merge window

Posted Apr 9, 2018 17:45 UTC (Mon) by droundy (subscriber, #4559) [Link]

I think the value of asking "are you sure?" is that the user might not have intended to plug in a disk. Perhaps I would like to allow someone to charge their phone using my computer, or I want to borrow a mouse, keyboard, or remote control for giving a presentation at a conference. In none of those cases would I be happy to have the device automatically mounted.

The first half of the 4.17 merge window

Posted Apr 10, 2018 10:25 UTC (Tue) by matthias (subscriber, #94967) [Link]

But you are happy to allow the kernel to automatically register a USB keyboard (or mouse) that can acknowledge the mount dialog? (or type rm -rf / from time to time to annoy terminal users)

Such a dialog would have to appear for every new device, not only mass storage. Furthermore, if you do not trust the device, you should not allow to register any component of the device. Which brings us back to square one: Why should you add a device that you do not trust? Ok, you could try to configure the keyboard (or mouse) that represents the presenter such that it can only forward inputs to the presentation software and not to some terminal or window manager. Is there anyone who really does this?

The bottom line is that you are usually doomed if you cannot trust the hardware that you are using, which is one of the reasons why Unix traditionally ultimately trusts any hardware.

The first half of the 4.17 merge window

Posted Apr 12, 2018 15:02 UTC (Thu) by bfields (subscriber, #19510) [Link]

"Folks like you and I are several sigmas beyond the mean, and it is extremely foolish to generalize our relatively-clueful paranoia onto the general populace.

It is widely established that users will overwhelmingly say "yes" to any and every prompt confirming potentially risky behaviour. Usually without any conscious thought."

Personally, I'm a little less sure that I'm any different from the "general populace".

After I've already clicked "yes" on a hundred of those dialogs, I'm not sure I could count on myself to click "no" on a dubious one.

The first half of the 4.17 merge window

Posted Apr 9, 2018 13:59 UTC (Mon) by bandrami (subscriber, #94229) [Link]

For 20 years my Linux systems did precisely nothing when I plugged in a USB drive other than register its presence with the kernel (and, later, with the in-memory device filesystem flavor of the week).

They did this because this is The Right Thing To Do when a new removable disk is added. This didn't need to change.

bug bounties are massively underrated

Posted Apr 19, 2018 18:46 UTC (Thu) by Garak (guest, #99377) [Link]

Ted has committed to fixing any known issues, and from what I have seen has been doing a good job with ext4. [...] I expect it will take years of work before there is confidence that ext4 has no exploitable holes if used on a malicious block device.

Sounds to me like another prime job for the oft-underrated combination of bug bounties and money. If there is a recurring and large enough bug bounty for successfully malicious filesystem images, the crowd will provide, and the resulting improvement timeframe will dramatically shrink (in direct proportion to the amount of money you throw into the bug bounty pot). Seriously, make it a sport/game with enough funding. If it matters that much.