Unprivileged chroot()

By Jonathan Corbet
March 15, 2021

It is probably fair to say that most Linux developers never end up using chroot() in an application. This system call puts the calling process into a new view of the filesystem, with the passed-in directory as the root directory. It can be used to isolate a process from the bulk of the filesystem, though its security benefits are somewhat limited. Calling chroot() is a privileged operation but, if Mickaël Salaün has his way with this patch set, that will not be true for much longer, in some situations at least.

Typically, chroot() is used for tasks like "jailing" a network daemon process; should that process be compromised, its ability to access the filesystem will be limited to the directory tree below the new root. The resulting security boundary is not the strongest — there are a number of ways to break out of chroot() jails — but it can still present a barrier to attackers. chroot() can also be used to create a different view of the file system to, for example, run containers within.

This system call is not available to just anybody; the CAP_SYS_CHROOT capability is required to be able to call chroot(). This restriction is in place to thwart attackers who would otherwise try to confuse (and exploit) setuid programs by running them inside a specially crafted filesystem tree. As a simple example, consider the sort of mayhem that might be possible if setuid programs saw a version of /etc/passwd or /etc/sudoers that was created by an attacker.

The limitations of chroot() have long limited its applicability; in recent years it has fallen even further out of favor. Mount namespaces are a much more flexible mechanism for creating new views of the filesystem; they can also be harder to break out of. So relatively few developers see a reason to use chroot() for anything new.

Thus, some folks were a bit surprised when Salaün showed up with his chroot() patch. Once applied, unprivileged processes are able to call chroot(), but only if a few conditions apply:

The process in question must have done a prctl() call with the PR_SET_NO_NEW_PRIVS option. That prevents the process from gaining any new privileges; running setuid and setgid programs will no longer gain the privileges of the owner of the executable file, for example. Since privileged programs no longer exist in that mode, their privileges cannot be exploited.
The process cannot be sharing its filesystem context (struct fs_struct, which contains the root and current working directories) with any other processes; otherwise the chroot() call would affect both processes, and the other one may not be expecting its filesystem environment to change abruptly.
The new root must be underneath the current root in the filesystem hierarchy. This prevents trickery that could otherwise facilitate escape from an existing jail or mount namespace.

If these conditions are met, it is argued, it is safe to allow a process to call chroot().

There is still the question of why one might want to do that. Among other things, a functioning chroot() environment normally needs to have a minimally populated /dev directory; creating device nodes remains a privileged operation. And, as noted above, Linux has had better options than chroot() for some time now. But Salaün says that there are use cases where a process might want to sandbox itself after the things it needs from the wider environment (libraries, for example) have been loaded, and device files can often be done without.

The initial reception for this patch has been a bit chilly at best. Eric Biederman worried about the security implications of unprivileged chroot() when mixed with other mechanisms:

Still allowing chroot after the sandbox has been built, a seccomp filter has been installed and no_new_privs has been enabled seems like it is asking for trouble and may weaken existing sandboxes.

Casey Schaufler argued that chroot() is obsolete and also worried about interactions: "We're still finding edge cases (e.g. ptrace) where no_new_privs is imperfect". He also pointed out that access to chroot() is already finely controlled with the CAP_SYS_CHROOT capability:

CAP_SYS_CHROOT is specific to chroot. It doesn't give you privilege beyond what you expect, unlike CAP_CHOWN or CAP_SYS_ADMIN. Making chroot unprivileged is silly when it's possibly the best example of how the capability mechanism is supposed to work.

Salaün has not answered all of these points, but seems undeterred; he posted a second version of the patch set after that discussion had occurred. Without a stronger answer, though, upstreaming this change is likely to be difficult. Security-oriented developers will need some convincing that chroot() merits any improvements at all; the bar for changes that raise worries about unexpected interactions with other security mechanisms will be higher.

The discussion is likely to come down to use cases in the end; is there truly a need for unprivileged chroot() in 2021? If there are users out there who could benefit from this feature, now would probably be a good time for them to come forward and explain why they need it. In the absence of that information, unprivileged chroot() seems likely to be one of those ideas that didn't quite make it.

Index entries for this article
Kernel	System calls/chroot()
Security	chroot()

Unprivileged chroot() and Outrun

Posted Mar 15, 2021 19:04 UTC (Mon) by nickodell (subscriber, #125165) [Link] (12 responses)

Here's a context where unprivileged chroot would be useful: a tool called Outrun. (https://github.com/Overv/outrun)

Outrun lets you execute a local command using the processing power of another Linux machine. In order to do this, it runs the process on the remote system, and redirects all filesystem calls back to the local system. It does this through two systems: FUSE and chroot. FUSE can be done in userspace with no extra permissions. chroot, however, requires root. For that reason, Outrun requires root privileges, even if the application you're running doesn't.

There doesn't seem to be a great way to solve this problem under the current permission scheme. Sure, there's a chroot capability. But how do you give that chroot capability to processes running in Outrun? Outrun spawns processes from a normal login shell. If you give all login shells chroot capability, then that opens a security hole, due to setuid programs which can't be allowed to run inside chroots.

One solution which Outrun discussed was to write a setuid helper, which could run the chroot syscall on behalf of Outrun. However, those carry their own security risks. (See also: calibre's setuid helper.)

For these reasons, I think this patchset would be useful.

Unprivileged chroot() and Outrun

Posted Mar 15, 2021 19:20 UTC (Mon) by mcon147 (subscriber, #56569) [Link] (11 responses)

Can outrun use 'unshare' ?
It seems like you can do
$ unshare -mr chroot os-tree-dir bash

Unprivileged chroot() and Outrun

Posted Mar 15, 2021 20:59 UTC (Mon) by floppus (guest, #137245) [Link] (1 responses)

That will fail if unprivileged user namespaces are disallowed, which is still the default on Debian, for instance.

Unprivileged chroot() and Outrun

Posted Mar 15, 2021 22:30 UTC (Mon) by josh (subscriber, #17465) [Link]

Debian allows unprivileged user namespaces these days, as of version 5.10.4-1 of the Linux packaging.

Unprivileged chroot() and Outrun

Posted Mar 16, 2021 7:52 UTC (Tue) by smurf (subscriber, #17840) [Link] (8 responses)

My program might want to open a sqlite database and a log file, *then* chroot to /run/user/UID/my-jail. When I restart the thing it might want to do the same thing and read the session files which the first program dumped there.

There's no way to do that without chroot.

Unprivileged chroot() and Outrun

Posted Mar 16, 2021 19:07 UTC (Tue) by floppus (guest, #137245) [Link] (7 responses)

That doesn't require an *unprivileged* chroot; you could do something like:

logfile = fopen("foo.log", "a");
sqlite3_open("foo.db", &db);
sprintf(rootdir, "/run/user/%d/my-jail", getuid());
chdir(rootdir);
unshare(CLONE_NEWUSER);
chroot(".");
caps = cap_get_proc();
cap_clear(caps);
cap_set_proc(caps);

Unprivileged chroot() and Outrun

Posted Mar 17, 2021 10:09 UTC (Wed) by smurf (subscriber, #17840) [Link] (5 responses)

I know that there are several options to do this with a privileged process. My point is that this task should not require any. That way, if the user does get hacked, at least there's no setuid executable for them to play with.

Unprivileged chroot() and Outrun

Posted Mar 17, 2021 17:39 UTC (Wed) by floppus (guest, #137245) [Link] (4 responses)

Right, but the point is that you *don't* need a setuid executable. Creating a user namespace (calling "unshare") normally doesn't require any special privileges.

Unprivileged chroot() and Outrun

Posted Mar 18, 2021 1:03 UTC (Thu) by pabs (subscriber, #43278) [Link] (3 responses)

It does if your distro or sysadmin has disabled unprivileged namespaces by default.

Unprivileged chroot() and Outrun

Posted Mar 18, 2021 12:54 UTC (Thu) by domenpk (guest, #12382) [Link] (2 responses)

That same distro or sysadmin will almost surely also disable unprivileged chroot (being a newer and less tested feature), so you won't gain anything.

Unprivileged chroot() and Outrun

Posted Mar 27, 2021 18:54 UTC (Sat) by l0kod (subscriber, #111864) [Link] (1 responses)

chroot(2) is much more simple (and limited) than namespaces, which is why there is no valid reason to be able to disable it (i.e. this unprivileged chroot is not, by design, a security risk).

Unprivileged chroot() and Outrun

Posted Apr 6, 2021 19:28 UTC (Tue) by immibis (subscriber, #105511) [Link]

is less of* a security risk

Unprivileged chroot() and Outrun

Posted Mar 17, 2021 22:39 UTC (Wed) by NYKevin (subscriber, #129325) [Link]

Reading through user_namespaces(7), I can think of the following problems:

1. You have to set up uid_map and gid_map if you want to interact with the filesystem. Since you are using chroot(), you almost certainly do want to interact with the filesystem, so this is an obvious source of friction. Not impossible, just annoying.
2. Assuming you don't have CAP_SETUID/GID (in the parent user namespace), which is a safe assumption because otherwise you wouldn't be asking for "unprivileged chroot" in the first place, then the man page appears to say that you can only map your own UID/GID. That certainly makes logical sense (the whole point of this operation is to give you a "containerized" or unprivileged CAP_SETUID, so we need to constrain it somehow), but it also means that stat(2) will lie to you about the ownership of any file you don't own (the UID/GID is unmapped, so it gets converted to a generic "don't know" value in the child namespace).
3. SCM_CREDENTIALS will also produce unmapped garbage, as will plenty of other UID/GID-related interfaces. If you want to IPC with any process owned by a different user (e.g. a daemon running under a role account), you basically can't confirm its identity, although it can confirm yours (which may be sufficient in some cases).
4. Pervasively fixing all of the above, testing it, and maintaining everything, is likely harder than just granting CAP_SYS_CHROOT in the first place.

Unprivileged chroot()

Posted Mar 16, 2021 0:11 UTC (Tue) by roc (subscriber, #30627) [Link] (5 responses)

I ran into a use-case for this just recently.

Our application runs in a container. It needs access to subtrees of the host filesystem. We mount each subtree /a/b/c under /host/a/b/c. Unfortunately this breaks because absolute symbolic links in the host filesystem (e.g. /a/b/c -> /foo/bar) don't exist in the container's mount namespace (it would need to be interpreted as /host/foo/bar). I ended up writing an implementation of `realpath` that manually resolves symbolic links and knows how to rebase absolute symbolic links to the /host directory. It's probably not nearly as efficient as doing it in the kernel though. I would have thought a lot of people ran into a need for this.

Obviously unprivileged chroot() would provide a solution. Though, maybe unprivileged chroot alone wouldn't be that great for us; we'd have to fork a helper process to do the chroot and pass fds back to the main process, which would be fairly complicated and maybe not faster than manual symlink resolution.

Unprivileged chroot()

Posted Mar 16, 2021 6:30 UTC (Tue) by dbnichol (subscriber, #39622) [Link] (4 responses)

If you're already in a container with it's own mount and user namespace (presumably), then can't you just supply it with CAP_SYS_CHROOT?

Unprivileged chroot()

Posted Mar 16, 2021 10:05 UTC (Tue) by roc (subscriber, #30627) [Link] (3 responses)

That's a good point. I prefer the code in question to run unprivileged inside its container, though.

Unprivileged chroot()

Posted Mar 16, 2021 13:45 UTC (Tue) by gscrivano (subscriber, #74830) [Link] (1 responses)

Have you already considered openat2(RESOLVE_IN_ROOT)? Wouldn't that be enough to replace chroot()?

Unprivileged chroot()

Posted Mar 17, 2021 2:07 UTC (Wed) by roc (subscriber, #30627) [Link]

Oooh, I didn't know about RESOLVE_IN_ROOT. That solves my use-case perfectly!

Unfortunately I can't use it yet because I can't guarantee we're running on 5.6 or above, but this is the right API for me.

Unprivileged chroot()

Posted Mar 17, 2021 0:28 UTC (Wed) by dbnichol (subscriber, #39622) [Link]

Right. The way I've seen this done before is to start the new process with several capabilities, setup the environment, and then drop all but the required caps before starting the real work. In a sense it's better than what you could do with the unprivileged chroot that's being suggested here. Once you do the intended chroot, you can drop the capability and then the rest of the code can't use it anymore.

Unprivileged chroot()

Posted Mar 16, 2021 0:50 UTC (Tue) by geofft (subscriber, #59789) [Link] (4 responses)

Unprivileged chroot, I think, was one of the original motivations for PR_SET_NO_NEW_PRIVS back in the day.

Anyway, there's one advantage of direct unprivileged chroot over making an unprivileged user + mount namespace and calling chroot inside there: you retain a full UID map of the outside system. If you use "unshare -cm --keep-caps," you get to map a single UID, your own, and so things like "ls -l /bin" don't display the results you'd expect. Since unprivileged chroot wouldn't create a user namespace, things would still look normal.

Maybe this could be worked around by saying something like, if you're inside a user namespace, you have no capabilities, and you map to the same UID outside the namespace, and you call PR_SET_NO_NEW_PRIVS, you get the ability to write an identity map to uid_map and gid_map, even if they've already been written to. After all, in no new privs mode, you can't switch users or gain any capabilities, so it doesn't matter what UID mapping you see. But it seems extremely tricky to get the detail of that right and you'd probably introduce exploitable bugs the first few times you try.

(I don't believe CAP_SYS_CHROOT is a meaningful alternative here. How would you grant it? Would you give filesystem capabilities to the chroot command? It won't be enforcing the NO_NEW_PRIVS requirement, then, and will turn into an immediate local root escalation. It _can't_ enforce that requirement, in fact: if you run a setcap program under NO_NEW_PRIVS, those capabilities are ignored, specifically because you asked for no new privs! So it wouldn't work if run from a no-new-privs parent process. If you somehow could avoid that constraint and run a setcap chroot, you could call it twice, the second time with a modified /lib thanks to being able to modify your chroot, and you could use that to escape the first chroot and hold onto chrooting privileges. It is technically true that CAP_SYS_CHROOT is the best example of how the capability mechanism is supposed to work - it's a fantastic demonstration of how useless that mechanism is.)

Unprivileged chroot()

Posted Mar 18, 2021 9:14 UTC (Thu) by matthias (subscriber, #94967) [Link] (3 responses)

Would it be possible to create a chroot command that has CAP_SYS_CHROOT filesystem capabilities, does the chroot, drops CAP_SYS_CHROOT and sets NO_NEW_PRIVS before calling the supplied command?

Unprivileged chroot()

Posted Mar 18, 2021 12:26 UTC (Thu) by winstonx86 (subscriber, #138536) [Link]

Yes but I suppose you couldn’t perform a second chroot because of the NO_NEW_PRIVS

Unprivileged chroot()

Posted Mar 18, 2021 16:54 UTC (Thu) by floppus (guest, #137245) [Link] (1 responses)

It's dangerous to allow that if the process is already chrooted, since it lets you escape from the outer chroot.

For that reason (I think), unprivileged processes can't create user namespaces when they're already chrooted, and the proposed unprivileged chroot would likewise be forbidden.

Unprivileged chroot()

Posted Mar 18, 2021 17:05 UTC (Thu) by matthias (subscriber, #94967) [Link]

Yes, my suggestion was to use the modified chroot command as an alternative to unprivileged chroot() syscall. And it was not meant to be used repeatedly. Obviously, it cannot be used repeatedly as after one execution NO_NEW_PRIVS is set and the CAP_CHROOT filesystem capability will have no effect.

And of course, if someone chroots a process without NO_NEW_PRIVS in a classic way, there should be no enchanted chroot command that gets capabilities from the filesystem laying around inside the new root.

Unprivileged chroot()

Posted Mar 16, 2021 3:47 UTC (Tue) by rsidd (subscriber, #2582) [Link] (1 responses)

The last time I used chroot was to run a linux subsystem in an android tab. I had a full xfce-based desktop on android, and did actual work on it. It required a rooted tab and, even so, later android releases made it hard. I haven't tried it recently but it seems these days they use proot for this purpose, and root is not needed.

Unprivileged chroot()

Posted Mar 16, 2021 4:32 UTC (Tue) by pabs (subscriber, #43278) [Link]

Some of the other chroot-on-Android options are listed here:

https://wiki.debian.org/ChrootOnAndroid

Unprivileged chroot()

Posted Mar 16, 2021 8:21 UTC (Tue) by l0kod (subscriber, #111864) [Link] (3 responses)

> Salaün has not answered all of these points

They are answered, especially with the third version: https://lore.kernel.org/lkml/20210311105242.874506-2-mic@...

Unprivileged chroot()

Posted Mar 17, 2021 12:49 UTC (Wed) by walters (subscriber, #7396) [Link] (2 responses)

Just echoing my comments from 2012 in that thread with Andy: I still think running a process without at least the "API devices" e.g. `/dev/null` (and `/proc`) is just unnecessary painful for everyone authoring a shared library. There's a lot of software that opens `/dev/null` when spawning child processes. And things like wanting to read `/proc/cpuinfo` as part of a multiprocessing library to determine how many threads to spawn.

And yes, there's now a syscall instead of `/dev/urandom` but still.

Unprivileged chroot()

Posted Mar 17, 2021 13:15 UTC (Wed) by l0kod (subscriber, #111864) [Link]

These concerns have been taken into account in the commit message. ;)

Unprivileged mknod() or use FUSE?

Posted Mar 26, 2021 23:13 UTC (Fri) by jrincayc (guest, #29129) [Link]

Hm, I wonder if you could get around the lack of things like no dev by either doing a "dev" in FUSE or possibly by another patch to allow some unprivileged uses of mknod? I would think that it might be possible to allow making /dev/null, /dev/zero, /dev/random and /dev/urandom be unprivileged.

Unprivileged chroot()

Posted Mar 17, 2021 17:58 UTC (Wed) by metalheart (guest, #89328) [Link] (2 responses)

Kernel noob here. This change will not affect the /usr/sbin/chroot tool?

Unprivileged chroot()

Posted Mar 18, 2021 1:48 UTC (Thu) by NYKevin (subscriber, #129325) [Link]

It depends on how your system's chroot(8) was written.

* If it explicitly checks geteuid() == 0, then it will continue to fail for non-root. This is probably a bad design decision, but not impossible if the application writer was trying to be "helpful" and provide a more explicit error message. On non-Linux systems, it would not be wrong to insert such a check, and some of these tools are written for "any random Unix-like" rather than Linux specifically.
* Unless it calls prctl() with PR_SET_NO_NEW_PRIVS, it will continue to fail for non-root. I see nothing about this in the man page for the GNU version, but it's possible a vendor might ship a version of chroot which does this. If this patch does get implemented, future versions of the GNU tool might grow a command-line argument to enable this functionality (or they might not; I can't read the GNU people's collective mind).
* Because chroot(8) runs a separate executable after doing the chroot, shared libraries etc. need to be accessible from within the chroot environment. It is complicated (but not categorically impossible) for a non-privileged user to set this up.

TL;DR: You probably still need to be root to profitably use chroot(8), even with this patch.

Unprivileged chroot()

Posted Mar 18, 2021 11:01 UTC (Thu) by l0kod (subscriber, #111864) [Link]

This should work with setpriv --no-new-privs /usr/sbin/chroot /new/root /bin/sh

Unprivileged chroot()

Posted Mar 18, 2021 16:45 UTC (Thu) by jcpunk (subscriber, #95796) [Link]

I'm certain it is my ignorance showing, but is there a reason `chroot()` isn't a type of mount namespace these days?

Unprivileged chroot()

Posted Mar 18, 2021 22:11 UTC (Thu) by kentonv (subscriber, #92073) [Link] (5 responses)

Do chroots stack? Or does calling chroot simply update the root pointer to point at something new?

In the latter case, I think allowing unprivileged chroot() ironically makes it possible to escape a preexisting chroot jail by the following means:

1. chdir("/foo")
2. chroot("/bar")
3. open("../..")

Step 3 opens the parent of the previous root! Because ".." is no longer recognized as being the current root, the kernel doesn't prevent traversing up past it.

Verifying that the current directory is under the new root is not enough... Instead of chdir() in step 1 you could also open a file descriptor to "/foo" and then openat() in step 3.

Verifying that all open file descriptors are under the new root still isn't enough, because file descirptors could be transmitted via SCM_RIGHTS over a unix socket from an accomplice program that isn't inside the new chroot.

I think it only works if chroots stack, but my understanding is that they don't.

Unprivileged chroot()

Posted Mar 19, 2021 0:12 UTC (Fri) by flussence (guest, #85566) [Link] (4 responses)

This is a trap as old as time in using chroot securely: you're supposed to call chroot() immediately followed by - at a bare minimum - chdir("/"), to prevent escapes via old cwd handles, but the API lends itself well to forgetting.

Unprivileged chroot()

Posted Mar 19, 2021 0:22 UTC (Fri) by kentonv (subscriber, #92073) [Link] (3 responses)

Right, but, my point is that the proposed feature would let anyone break out of chroots even if they were set up "correctly".

Unprivileged chroot()

Posted Mar 19, 2021 18:32 UTC (Fri) by l0kod (subscriber, #111864) [Link] (2 responses)

This is the reason of the unprivileged chroot limitations. It is only allowed to chroot one time: https://lore.kernel.org/lkml/20210316203633.424794-2-mic@...

Unprivileged chroot()

Posted Mar 19, 2021 21:56 UTC (Fri) by kentonv (subscriber, #92073) [Link] (1 responses)

Ahhhhh I see.

That seems like a disappointing limitation though... any program that uses this feature will mysteriously break when run in a chroot.

Unprivileged chroot()

Posted Mar 21, 2021 10:50 UTC (Sun) by smurf (subscriber, #17840) [Link]

Running in a plain chroot isn't a good idea anyway; as soon as you do anything nontrivial things tend to break. The new unprivileged-chroot sycall is just one more example of many.

Much better to use systemd-nspawn or some other tool that sets up a "real" file system namespace. The unprivileged chroot(2) will work there.