Hertz: Abusing privileged and unprivileged Linux containers

[Posted June 1, 2016 by corbet]

This white paper by Jesse Hertz [PDF] examines various ways to compromise and escape from containers on Linux systems. "A common configuration for companies offering PaaS solutions built on containers is to have multiple customers’ containers running on the same physical host. By default, both LXC and Docker setup container networking so that all containers share the same Linux virtual bridge. These containers will be able to communicate with each other. Even if this direct network access is disabled (using the –icc=false flag for Docker, or using iptables rules for LXC), containers aren’t restricted for link-layer traffic. In particular, it is possible (and in fact quite easy) to conduct an ARP spoofing attack on another container within the same host system, allowing full middle-person attacks of the targeted container’s traffic."

Hertz: Abusing privileged and unprivileged Linux containers

Posted Jun 2, 2016 7:28 UTC (Thu) by corsac (subscriber, #49696) [Link] (7 responses)

I'll say it before spender, then: why (again) no mention at of all grsecurity hardening, which is an effective way to confine userland and protect the kernel against attacks?

Hertz: Abusing privileged and unprivileged Linux containers

Posted Jun 2, 2016 8:44 UTC (Thu) by spender (guest, #23067) [Link] (2 responses)

Most of the things discussed in the paper are a result of bad defaults and not knowing how capabilities and certain syscalls are supposed to work (the TOCTOU race with seccomp for instance). It didn't really cover kernel exploitation at all. A number of the /proc issues would be handled automatically by grsec (due to removals, additional capability checks etc) but the majority of these issues are simply the container shooting itself in the foot. Since container usage is evolving so rapidly (as well as what actions one expects to be possible within a container) it perhaps makes more sense for the time being to simply use the configurable tools already present to limit the activity. Of course, running privileged containers on a host and depending on blacklists to prevent badness is fundamentally fraught with problems.

I do note a potential error/misstatement in the text, it mentions that ptrace is disabled in the containers simply by removing CAP_SYS_PTRACE, but the two are orthogonal. CAP_SYS_PTRACE is for permitting the ptracing of processes outside of what could be performed by an unprivileged user, it's obviously not a control over using ptrace at all otherwise ptrace would be a purely privileged operation. Presumably the real fix being used is a blacklist via seccomp.

-Brad

Hertz: Abusing privileged and unprivileged Linux containers

Posted Jun 2, 2016 17:10 UTC (Thu) by jejb (subscriber, #6654) [Link]

Agree here: one of the great powers of the linux container API is that it allows you to do many things, some of which are definitely not "secure". It's actually a feature of containers that we can do things (like pass file descriptors between containers) which other virtualization technology can't, but which deliberately requires a degree of trust between the two containers doing the sharing. The problem is that security isn't an absolute, its a tradeoff between the risks you're willing to run for the features you want to enable.

I suppose there's some vague value in a paper identifying common misconfigurations, but realistically in a complex interface like this, there's always going to be ways of misusing it. Perhaps we should start classifying APIs on the Rusty type scale starting with "impossible to use securely".

Hertz: Abusing privileged and unprivileged Linux containers

Posted Jun 4, 2016 5:22 UTC (Sat) by khz (guest, #109129) [Link]

thanks for the feedback spender. as aaron said below, we're big fans of grsec :)

will be incorporating the SYS_PTRACE correction (as well as another minor one someone else pointed out) next week

Hertz: Abusing privileged and unprivileged Linux containers

Posted Jun 2, 2016 14:15 UTC (Thu) by jwildebo (guest, #38479) [Link] (2 responses)

It also completely ignores the hard work done on using SELinux to help in container security and isolation. I think it is a bit unfair to claim "Linux" in teh title of this paper but looking only at some aspects that are not really universal. It mentions AppArmor, so I guess the author comes from the Debian/Ubuntu side of Linux?

Jan (disclaimer: Red Hat's EMEA Evangelist)

Hertz: Abusing privileged and unprivileged Linux containers

Posted Jun 3, 2016 6:07 UTC (Fri) by drag (guest, #31333) [Link]

Docker is essentially Ubuntu. Specifically Ubuntu 14.04. At least that is the perception. I don't like that personally (as going back to 14.04 feels like time travelling to the bad-old-days), but that is the standard setup.

Hertz: Abusing privileged and unprivileged Linux containers

Posted Jun 4, 2016 5:27 UTC (Sat) by khz (guest, #109129) [Link]

hey, author here. yes, I did *totally* ignore the hard-work being done on SELinux + non-ubuntu/debian based systems. this is nothing against those systems (or for them). in my time as a penetration tester working on container systems, almost every single one I have evaluated was either LXC or Docker, using a debian/ubuntu baseOS, with AppArmor used as the LSM. this paper was very much intended to be 'stories from the trenches', so it represents what I've encountered, and how they can often be insecure by default (or in somewhat subtle ways).

if you know of anyone using RHES / SELinux based containers, feel free to send em my way. I know me and aaron would love to audit them :)

Hertz: Abusing privileged and unprivileged Linux containers

Posted Jun 3, 2016 16:09 UTC (Fri) by dyn (guest, #103684) [Link]

Good point! Jesse Hertz (Author) is a co-worker of mine at NCC Group, and (I believe) the paper was focused on exploring several different attacks against LXC (and some indirectly related to Docker) and the related configuration weaknesses/defaults. Jesse and I have talked about (and are big fans of) grsecurity in general and to many clients for container hosts/Linux, but the paper doesn't explore this avenue of hardening. I however did discuss this in my Understanding and Hardening Linux Containers white paper, released a bit earlier this year* that is more comprehensive on recommendations and more holistic in nature for LXC, Docker and Rkt ( which Jesse reviewed). *Spender also had some constructive comments/notes I'm finally merging in next week, along with a few more minor tweaks/updates.