User: Password:
Subscribe / Log in / New account

An x32 local exploit

Benefits for LWN subscribers

The primary benefit from subscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today!

By Jake Edge
February 5, 2014

So far, the x32 ABI—a 32-bit ABI for running on x86 processors in 64-bit mode—is not widely used. Only a few distributions have enabled support for it in their kernels (notably Ubuntu), which reduces the impact of a recently discovered local privilege escalation somewhat, but the bug has been in the kernel since 2012. It's a nasty hole, that required a quick fix for Ubuntu 13.10 (and two hardware enablement kernels for 12.04 LTS: linux-lts-raring and linux-lts-saucy).

It is the x32 version of recvmmsg() that has the bug. In the compat_sys_recvmmsg() function that is part of the compatibility shim for handling multiple ABIs in the kernel, a user-space pointer for the timeout value is treated as a kernel pointer (rather than copied using copy_from_user()) for the x32 ABI. The value of the timeout pointer is controlled by the user, but it gets passed as a kernel pointer that __sys_recvmmsg() (which implements the system call) will use. The kernel will dereference the pointer for both reading and writing, which allows a local, unprivileged user to get root privileges.

The problem was reported to the closed and linux-distros mailing lists on January 28 by Kees Cook, after "PaX Team" reported it to the Chrome OS bug tracker (in a still-restricted entry). It was embargoed for two days to give distributions time to get fixes out. After that, "Solar Designer" reported it publicly since Cook was traveling. It is a serious bug, but is somewhat mitigated by the fact that few distributions have actually enabled the ABI.

The x32 ABI came about largely to combat the amount of memory wasted on x86_64 processors for 64-bit pointers (and long integers) in programs that did not require the extra 32 bits for each value. It allows programs to use the extra registers and other advantages that come with x86_64 without paying the penalty of extra memory usage. In theory, that should lead to less memory usage and faster programs due to a smaller cache footprint. So far, though, those benefits are somewhat speculative—and controversial.

X32 does exist in the kernel, however, and can be enabled with the CONFIG_X86_X32 flag. If it is enabled, any user can build an x32 program using GCC with the -mx32 flag. The kernel will recognize such a binary and handle it appropriately.

The bug was introduced in a February 2012 commit that was adding support for 64-bit time_t values to x32. The problematic code is as follows (from compat_sys_recvmmsg()):

            return __sys_recvmmsg(fd, (struct mmsghdr __user *)mmsg, vlen,
                                  flags | MSG_CMSG_COMPAT,
                                  (struct timespec *) timeout);
The timeout value is passed to that function as:
    struct compat_timespec __user *timeout
It is clearly annotated as a user-space pointer, but just gets passed to __sys_recvmmsg(). The fix is to use compat_get_timespec() to copy the data from user space before the call to __sys_recvmmsg() and compat_put_timespec() to copy any changes back to user space afterward.

Exploits have started to appear (for example, one by rebel and another by saelo). The basic idea is to use the fact that recvmmsg() will write the amount of time left in the timeout to the location specified by the timeout pointer. Since the value of that pointer is controlled by the user, it can be arranged to write known values (another exploit-controlled address, say) to somewhere "interesting", for example to a function pointer that gets called when the /proc/sys/net/core/somaxconn file is opened (as rebel's exploit does). The program will already have arranged to have "interesting" code (to gain root privileges) located at that address. When the function is called by the kernel via that pointer, the exploit's code is run.

Users of Ubuntu 13.04 should note that it reached its end of life two days before the bug was found, so no update for that kernel has been issued. One possible solution for those who have not yet upgraded to 13.10 (or are running some other distribution kernel and do not want to patch and build their kernel) is a module that disables the x32 version of the recvmmsg() system call.

As PaX Team noted in the report (quoted by Solar Designer), the presence of this bug certainly calls into question how much testing (fuzz testing in particular) has been done on the x32 ABI. For a bug of that nature to exist in the kernel for two years would also seem to indicate that it isn't just testing that has fallen by the wayside—heavy use would also seem to be precluded. In any case, the problem was found, reported, and fixed, now it is up to users (and any distributions beyond Ubuntu since we have received no other security advisories beyond those mentioned above) to update their kernels.

(Log in to post comments)

The bug is detected with Sparse

Posted Feb 6, 2014 9:11 UTC (Thu) by error27 (subscriber, #8346) [Link]

Here are the Sparse warnings for that file:

net/compat.c:76:24: warning: incorrect type in assignment (different address spaces)
net/compat.c:76:24: expected void *msg_name
net/compat.c:76:24: got void [noderef] <asn:1>*
net/compat.c:77:23: warning: incorrect type in assignment (different address spaces)
net/compat.c:77:23: expected struct iovec *msg_iov
net/compat.c:77:23: got void [noderef] <asn:1>*
net/compat.c:78:27: warning: incorrect type in assignment (different address spaces)
net/compat.c:78:27: expected void *msg_control
net/compat.c:78:27: got void [noderef] <asn:1>*
net/compat.c:90:63: warning: incorrect type in argument 1 (different address spaces)
net/compat.c:90:63: expected void [noderef] <asn:1>*uaddr
net/compat.c:90:63: got void *msg_name
net/compat.c:162:22: warning: cast removes address space of expression
net/compat.c:190:22: warning: cast removes address space of expression
net/compat.c:369:54: warning: incorrect type in argument 4 (different address spaces)
net/compat.c:369:54: expected char [noderef] <asn:1>*optval
net/compat.c:369:54: got char *<noident>
net/compat.c:430:54: warning: incorrect type in argument 4 (different address spaces)
net/compat.c:430:54: expected char [noderef] <asn:1>*optval
net/compat.c:430:54: got char *<noident>
net/compat.c:430:71: warning: incorrect type in argument 5 (different address spaces)
net/compat.c:430:71: expected int [noderef] <asn:1>*optlen
net/compat.c:430:71: got int *<noident>
net/compat.c:573:57: warning: cast removes address space of expression
net/compat.c:573:57: warning: incorrect type in initializer (different address spaces)
net/compat.c:573:57: expected struct compat_group_req [noderef] <asn:1>*gr32
net/compat.c:573:57: got void *<noident>
net/compat.c:594:65: warning: cast removes address space of expression
net/compat.c:594:65: warning: incorrect type in initializer (different address spaces)
net/compat.c:594:65: expected struct compat_group_source_req [noderef] <asn:1>*gsr32
net/compat.c:594:65: got void *<noident>
net/compat.c:615:60: warning: cast removes address space of expression
net/compat.c:615:60: warning: incorrect type in initializer (different address spaces)
net/compat.c:615:60: expected struct compat_group_filter [noderef] <asn:1>*gf32
net/compat.c:615:60: got void *<noident>
net/compat.c:653:52: warning: cast removes address space of expression
net/compat.c:653:52: warning: incorrect type in initializer (different address spaces)
net/compat.c:653:52: expected struct compat_group_filter [noderef] <asn:1>*gf32
net/compat.c:653:52: got void *<noident>
net/compat.c:786:40: warning: cast removes address space of expression

The very last line in that text splatch is the security vulnerability. It is frustrating because it's hard to see when there are so many other bogus warnings. Some of the warnings seem tricky to fix but others are just a matter of adding the correct Sparse annotations.

Fengguang Wu created his automated build notification system in June 2012 but this patch was applied the previous February. These days the patch would have generated an automated response. I can't promise that it would have been fixed but hopefully.

The bug is detected with Sparse

Posted Feb 6, 2014 11:17 UTC (Thu) by NAR (subscriber, #1313) [Link]

My first thought was that this bug seems to be detectable by static analysis - indeed it is.

The bug is detected with Sparse

Posted Feb 6, 2014 13:23 UTC (Thu) by PaXTeam (guest, #24616) [Link]

and this is what it looks like from my checker plugin (after fixing up some usage of __user on non-ptr types, etc that make it incompatible with the address space concept):
linux-3.13.1/net/compat.c: In function 'compat_sys_recvmmsg':
linux-3.13.1/net/compat.c:786:35: warning: cast to generic address space pointer from disjoint __user address space pointer [enabled by default]
                                   (struct timespec *) timeout);
my advice: right tool for the right job ;)

The bug is detected with Sparse

Posted Feb 6, 2014 20:27 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

Possibly a dumb question, but why this isn't an error? Is there an actual use case for such a cast?

The bug is detected with Sparse

Posted Feb 6, 2014 22:49 UTC (Thu) by PaXTeam (guest, #24616) [Link]

i don't know all the address space rules off the top of my head but my best guess is that gcc is being generous here by providing at least a warning despite the explicit type cast (that converts the pointer to an incompatible address space), without a cast it gives the expected error:
linux-3.13.1/net/compat.c:786:35: error: passing argument 5 of '__sys_recvmmsg' from pointer to non-enclosed address space
frankly, given the hack i needed to get this to work at all (IIRC, address space support is in C99 while the kernel's compiled in gnu89 mode) i'm glad it works as well as it does ;). the only reason i can't enable it unconditionally is because the kernel isn't thoroughly annotated unfortunately.

An x32 local exploit

Posted Feb 6, 2014 16:30 UTC (Thu) by deater (subscriber, #11746) [Link]

Re-compiled my "perf_fuzzer" tool for x32 and ran it and it's turning up all kinds of bugs :( And my fuzzer is just exercising a handful of syscalls. Time to send some messages to lkml. My poor fuzzing machine never catches a break.

An x32 local exploit

Posted Feb 7, 2014 13:59 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

Do you have pointers on how to run your fuzzer? These kinds of bugs are down my alley of things I like to fix in projects I use.

An x32 local exploit

Posted Feb 7, 2014 15:36 UTC (Fri) by deater (subscriber, #11746) [Link]

> Do you have pointers on how to run your fuzzer? These kinds of bugs are
> down my alley of things I like to fix in projects I use.

On further testing on x32 I get one WARN message that looks scary, a reboot-without logging anything issue, and then I run into the 2 or 3 crash-the-machine-with-useless-messages bugs you also find on regular x86 systems.

Sadly none of the bugs are deterministic. So while the fuzzer quickly hits them within minutes, it is really hard to get kernel devs to look an them without at easy reproducer or a good kernel backtrace log.

If you want to try out the fuzzer, you can download the perf_event_test
suite from here:
enter the fuzzer dir, run "make" then "./perf_fuzzer" and away you go.
To get x32 support you have to hack the makefiles to add -mx32 and link statically (for some reason the dynamic x32 libraries provided by debian don't work for me).

My fuzzer mostly concentrates on fuzzing the perf_event interface though. If you want to do more generic kernel fuzzing you should try out trinity (and in fact my fuzzer shares some code with trinity).

An x32 local exploit

Posted Mar 5, 2014 16:31 UTC (Wed) by kpfleming (subscriber, #23250) [Link]

Minor point: while I haven't tested it myself, I doubt that GCC's compilers need to be run on an x32-enabled kernel in order to support the -mx32 architecture option and produce x32-compatible binaries. The sentence that talks about -mx32 shouldn't link those two actions together, as they are independent.

Copyright © 2014, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds