LWN: Comments on "Local root vulnerability in snap-confine"

Local root vulnerability in snap-confine

Wol — Thu, 24 Feb 2022 11:07:31 +0000

What you need is something that OPTIONALLY enforces all this stuff. Like David Wheeler's comments on filenames for example - a *create*time flag for a filesystem that says "enforce these safety rules".

Sure, all hell could then break loose, as old apps assume they can write garbage, etc etc, but it gives *developers* the opportunity to have a flag day and clean up their own stuff. Then they recommend that users enable this flag, which can't be enabled on existing systems to avoid breaking them, and all this goodness is enforced going forwards.

Then you reverse the default state of the flag so all of a sudden, people with old apps have to ask for the old behaviour ... But the breakage is minimised by making it easy for those in the know to be early adopters and test their own systems. You only need something like SUSE's Open Build System flipping their default setting, as developer's build systems start breaking and a distro could then rapidly switch their own default knowing all the apps they've built are compliant ...

Cheers,
Wol

Local root vulnerability in snap-confine

mathstuf — Tue, 22 Feb 2022 02:29:34 +0000

Heh. The C string library functions routinely shows as "hot" when I benchmark certain programs because there's no way to use a view reliably (C++ helps out here, but if it were C…there wouldn't even be the ability to factor out that `strlen` from `if/else` trees at least using static strings). To use a substring, you have to allocate and copy it to new memory just to put a NUL at the end and that adds up over time. So I don't think performance is really the thing keeping C back from a new string library because it's frankly just not that great for string manipulation-heavy tasks to begin with IME.

As for convenience, sure it's handy to be able to pass a "string" around in a register because it's "just" the size of a pointer, but weighing that against the inconvenience of having to debug mishandling of these APIs, they are by *far* in the red in my book. I mean, we have how many flavors of "append a string"? `strcat`, `strncat`, and `strlcat`? Toss `snprintf` in there too if you want to be fancy. And there's not even *consensus* on this set. Reviewing code dense with use of the C string library requires notes because I need to track where lengths are coming from, are the right APIs being used, and "oh, that's just a poorly coded `strdup`" recognitions. I don't find it useful.

So given that it scores (IMO) a 2/5 on performance (sure, registers get you something I guess), 1/5 in (overall) convenience, and 0/5 on security because the APIs just don't help out half the time, why, specifically, would roc's "attitude" not be warranted?

Local root vulnerability in snap-confine

Smon — Mon, 21 Feb 2022 23:32:37 +0000

@smcv: Thank you very much for your detailed insight. It was really interesting to me!

Local root vulnerability in snap-confine

Cyberax — Mon, 21 Feb 2022 18:04:24 +0000

> David Wheeler's proposal would ban every ~1/(26+26+10+4) = 1/66-th video downloaded by youtube-dl. It is too late to ban anything.

Just add automatic escaping for such names.

Local root vulnerability in snap-confine

adobriyan — Mon, 21 Feb 2022 17:33:08 +0000

David Wheeler's proposal would ban every ~1/(26+26+10+4) = 1/66-th video downloaded by youtube-dl. It is too late to ban anything.

GNU ls(1) started to quote filenames with spaces at some point, it was huge step in right direction because copy paste suddenly started working "out of the box". More features lke this are necessary.

Local root vulnerability in snap-confine

nybble41 — Mon, 21 Feb 2022 16:37:51 +0000

> What is actually wanted is system calls, that return unambigous, well defined data..

We have the system calls (open, read, write). What is missing is the "unambiguous, well-defined data". Given an appropriate serialization format there is nothing wrong with communicating the data through /proc. I see no need to restrict access to languages with support for making special-purpose system calls.

Local root vulnerability in snap-confine

rgmoore — Mon, 21 Feb 2022 15:20:12 +0000

I think your comment gets very nicely at the root of our problems. We will never get security right as long as it is given lower priority than convenience and performance.

Local root vulnerability in snap-confine

zdzichu — Mon, 21 Feb 2022 14:15:03 +0000

Everything except NULL and / is allowed in filenames. But it doesn't mean it's a good idea to use it. Please take time to read the essay, it is eye-opening.

And you do not need a website. Just touch -- --foot-shooter (or touch -- -rf\ . if you like).

Local root vulnerability in snap-confine

adobriyan — Mon, 21 Feb 2022 13:37:24 +0000

> Even "-" character should not be allowed as the first character of a filename.

'-' is allowed in URLs. Any site issuing randomised names (like: YT) to user submitted content will create a file starting with minus easily.

Local root vulnerability in snap-confine

zdzichu — Mon, 21 Feb 2022 12:43:27 +0000

Even "-" character should not be allowed as the first character of a filename. There are many gotchas with various characters. David A. Wheeler summarized them neatly in https://dwheeler.com/essays/fixing-unix-linux-filenames.html .

"owning" a symlink?

kronat — Mon, 21 Feb 2022 10:14:14 +0000

> The legitimate uses of symlinks are mostly static [...]
Based on the above, isn't it possible to "block" any modification to the symlink until the application(s) that "locked" the symlink by opening it for reading is (are) done (doing a close on it)?

Local root vulnerability in snap-confine

epa — Mon, 21 Feb 2022 07:01:55 +0000

I meant mount points should be restricted to alphanumeric and -_. characters. For filenames in general I agree that humans expect to use ().

On some platforms space is not a legal character in filenames but you can use non-breaking space, which isn’t treated specially by the shell. I’m not sure whether that is better or worse.

Local root vulnerability in snap-confine

judas_iscariote — Sun, 20 Feb 2022 23:42:57 +0000

> proper serialization formats when communicating complex data structures to between kernel and userland, instead of this stringly typed complex to parse hodgepodge that > is "mountinfo".

What is actually wanted is system calls, that return unambigous, well defined data.. same on all archictectures or kernel config options.. no new proc files to parse or to write to...

Local root vulnerability in snap-confine

epa — Sun, 20 Feb 2022 16:59:03 +0000

I meant the names of mount points, not file names in general.

Local root vulnerability in snap-confine

smcv — Sun, 20 Feb 2022 13:11:19 +0000

Another reason why Snap needs root, but Flatpak usually does not, is that Snap uses an LSM (AppArmor) as part of its sandboxing but Flatpak does not.

As with the different scopes I described in my previous comment, this is a trade-off, and neither is trivially better than the other.

The fact that Snap uses AppArmor means it can rely on the LSM mechanism in the kernel, which is quite powerful, particularly if you are willing to patch the kernel (I believe Ubuntu's kernel patches to add SO_PEERSEC support to AppArmor still haven't gone upstream). If the kernel gives you information about a peer's LSM labels, then you can trust that information to be correct and securely obtained; and if a process is restricted by LSM policies, the kernel will forbid anything that those policies forbid, without needing to rely on other mechanisms like userns and seccomp as heavily as Flatpak does.

On the other hand, you need CAP_MAC_ADMIN (i.e. root) to manipulate AppArmor profiles, and if running on a distribution like Fedora that uses a different "big" LSM, or a distribution that doesn't compile any of the "big" LSMs into its kernel at all, then you're not going to be able to benefit from AppArmor at all.

Flatpak has gone for the approach that makes it work equally well on "most" distros, and give it an equal level of sandboxing on "most" distros even if installed as non-root. Snap has gone for the approach that is maximally powerful on e.g. Ubuntu, at the cost of working less well on non-Ubuntu and requiring root.

Local root vulnerability in snap-confine

smcv — Sun, 20 Feb 2022 12:54:33 +0000

The flatpak-system-helper is part of the distribution/installation layer of Flatpak rather than part of the sandboxing layer: it's the part that lets semi-privileged (but non-root-equivalent) users install an app or runtime system-wide, into /var/lib/flatpak. It's only about getting files into the right directories, and gets invoked when you install or upgrade an app, not when you run it.

The sandboxing layer of Flatpak *usually* (on modern distros) runs with the privileges of its caller, and can't do anything its caller couldn't already do on their own.

The exception to that is when the distro doesn't allow unprivileged users to create new user namespaces, for example Debian 10 and RHEL 7, in which case the bubblewrap executable (typically /usr/bin/bwrap or /usr/libexec/flatpak-bwrap) needs to be setuid root. I don't think it's coincidence that two out of four CVEs reported against bubblewrap have only been applicable when it's setuid root, and those two were the really serious ones. Setuid is dangerous and it's good to avoid it whenever possible (see also the recent pkexec vulnerability).

(CVE-2016-8659 and CVE-2020-5291 were root privilege escalation when setuid root, but were not applicable when not setuid root; CVE-2017-5226 wasn't really a vulnerability in bubblewrap at all, but more like a vulnerability in Flatpak and other bubblewrap users; CVE-2019-12439 only applied in unusual system configurations, and as far as I'm aware it was only a denial-of-service.)

Local root vulnerability in snap-confine

smcv — Sun, 20 Feb 2022 12:38:34 +0000

Flatpak is designed for a narrower use-case than Snap, so it's willing to accept more restrictions on what it can do, and doesn't need to be as "powerful" as Snap. On one hand, this makes Snap useful in situations where Flatpak isn't; on the other hand, this means Flatpak developers can focus on the situations that it supports, and ignore the situations it doesn't.

In particular, when Flatpak developers encounter something that would be hard to implement securely, we are often able to declare it to be out-of-scope and not implement it at all, resulting in fewer attack vectors.

Flatpak is designed to sandbox "apps", with a meaning that is not 100% well-defined, but "programs that behave like an Android/iOS app" or "GUI programs with a .desktop file" are reasonable approximations. Snap is designed to sandbox "apps", but also non-"app" things like system services: you can install lxd as a Snap, but it would not be possible to install lxd as a Flatpak app.

I personally think that's a strength of Flatpak rather than a weakness, but it would be reasonable for Snap users/developers to disagree on that: neither approach is trivially better than the other.

Local root vulnerability in snap-confine

foom — Sun, 20 Feb 2022 03:03:55 +0000

Eh? No use for spaces and parentheses? Both of those are incredibly useful in filenames for humans! Characters like newlines or tabs, sure -- those have no place in filenames. And you could arguably exclude double quotes as well, but a filename like "Aaron's Report: Cats (2010)" is completely reasonable.

If there's blame to be had, it'd be for the Unix shell language which treats strings in a variable unsafely by default!

Local root vulnerability in snap-confine

jra — Sat, 19 Feb 2022 23:21:20 +0000

If I can get this mount flag to be easily used I will certainly add it to the recommended settings for a Samba server.

I'm sick of symlink insanity.

Local root vulnerability in snap-confine

spigot — Sat, 19 Feb 2022 18:02:57 +0000

Amusingly, the Acknowledgments section of the write-up has this dedication:

This advisory is dedicated to 8lgm -- followers of symbolic links, overflowers of stack buffers, and dereferencers of NULL pointers: [...]

Local root vulnerability in snap-confine

intgr — Sat, 19 Feb 2022 10:26:26 +0000

> whoever came up with the mountinfo format, but that may be a little unfair as I'm sure it has simply evolved rather than being designed

I don't really blame whoever came up with it, but I blame kernel developers for not offering better alternatives for new use cases and migrating old kludges over.

Kernel developers should decide on a new unambiguous and extensible serialization format for complex nested data structures. In 1990s these lessons hadn't really been learned yet, but in 2022 we have protobuf, Avro, Arrow, ASN.1, JSON and plenty more. I'm sure we can find at least one that suits the kernel use case.

> blame whoever decided that ( ) and space should be legal characters in a mount point path name

This wouldn't even help that much. Restricting characters in path names would solve this particular issue for paths, but there are still plenty of other strings with arbitrary content that may end up being communicated to user space. So would you propose blocking a set of characters in *all* kernel inputs?

I feel like a broken record at this point, but again: if you're having trouble transmitting certain characters in your serialization format, it's the serialization format that sucks.

Local root vulnerability in snap-confine

adobriyan — Sat, 19 Feb 2022 09:40:38 +0000

The technical bug is that (deleted) is appended, not prepended so it is impossible to disambiguate just by looking at the output.

Local root vulnerability in snap-confine

roc — Sat, 19 Feb 2022 08:55:12 +0000

> Please use (create) a proper C library of high-level string manipulation functions, no shortcuts!

The problem is that C doesn't provide enough language features to make a library string type as convenient and performant to use as char*. (Language features such as destructors.) Therefore C programmers will be disinclined to use it. It's not an accident that after all these years no such library is in broad use.

Local root vulnerability in snap-confine

intelfx — Sat, 19 Feb 2022 06:23:17 +0000

> "This application is only known to be secure when used on a filesystem mounted with the MNT_NOSYMFOLLOW option. Use on filesystems allowing symlinks can lead to race conditions and security vulnerabilities."

I wouldn't use such software.

Local root vulnerability in snap-confine

walters — Fri, 18 Feb 2022 21:57:43 +0000

I only glanced at https://github.com/snapcore/snapd/tree/61b66aaa1beb861f30... and there's a lot going on there, but...I think the flatpak equivalent of part of this is https://github.com/flatpak/flatpak/blob/main/system-helpe... which runs as a DBus service, not a suid binary.

Local root vulnerability in snap-confine

ms-tg — Fri, 18 Feb 2022 21:11:09 +0000

> It's "sacred" because getting any traction for moving away from such permissiveness means an unknown amount of work for an unknown number of people who don't even know you're planning to impose that work on them.

I wonder if there are steps that, over some years, could help make the unknown number of people visible? For example, what if Linux, either in the file systems or more likely in distro-level utilities, simply detected filenames that violated certain rules, and let the user know that these existed and might need to be renamed to follow a proposed RFC naming convention in the future if it becomes accepted?

Perhaps users could opt-in to also reporting counts back to a central server?

I'm even picturing a nice web page, the way big security vulnerabilities have these days? Maybe 'linux-filenames-rfc.io'? Which could include the latest text of the proposal, links to Github where changes can be proposed, mailing lists where it is discussed, etc?

Using these sorts of community consensus approaches, might it be possible to get more of a defined sense of how often how big a real-world impact would be felt by any proposed reduction in Linux filename permissiveness?

Local root vulnerability in snap-confine

Smon — Fri, 18 Feb 2022 21:06:07 +0000

I enjoyed reading the report. Thank you!

Flatpaks bwrap is able to run as non root and use user-namespacing. So this cannot happen there.
Why does snap need a suid binary?

Local root vulnerability in snap-confine

developer122 — Fri, 18 Feb 2022 20:50:18 +0000

A surprisingly elegant solution, imo.

Local root vulnerability in snap-confine

developer122 — Fri, 18 Feb 2022 20:41:50 +0000

Side question: would it have done anything if it was originally required that the user have permissions for the destination if they wished to create a simlink? Is this feasible? (invalid symlinks might be a tricky edgecase, especially if they're not invalidated when their target is removed)

Local root vulnerability in snap-confine

mpr22 — Fri, 18 Feb 2022 18:33:08 +0000

It's "sacred" because getting any traction for moving away from such permissiveness means an unknown amount of work for an unknown number of people who don't even know you're planning to impose that work on them.

Local root vulnerability in snap-confine

jra — Fri, 18 Feb 2022 18:21:09 +0000

One of the things I'd like to see is wide knowledge and availability of MNT_NOSYMFOLLOW use, and making it available as a normal mount flag.

Then applications can state:

"This application is only known to be secure when used on a filesystem mounted with the MNT_NOSYMFOLLOW option. Use on filesystems allowing symlinks can lead to race conditions and security vulnerabilities."

Let admins know how to protect themselves from this misfeature.

Local root vulnerability in snap-confine

nix — Fri, 18 Feb 2022 17:43:23 +0000

> This would be fail-safe for older code

What? It would cause new failures for older code, i.e. it would introduce countless DoS vectors, only a microscopic proportion of which correspond to actual attacks, but all of which would annoy users (where "annoy users" spans the entire spectrum from "slightly annoying" through "bug we can't track down that goes away on restarting" through to "oops now the system is unbootable because of an unexpected and undiagnosed EREFRESH while installing something boot-critical").

Hell no.

Local root vulnerability in snap-confine

epa — Fri, 18 Feb 2022 16:20:48 +0000

I think you can also blame whoever decided that ( ) and space should be legal characters in a mount point path name. It serves no purpose to allow them. But this is a cultural hot button issue, since the original Unix decision to allow any byte except / and NUL in a filename has become somehow sacred. (I'm not even talking about filenames in general, just the name of mount points!)

You can also blame whoever came up with the mountinfo format, but that may be a little unfair as I'm sure it has simply evolved rather than being designed. With hindsight, if you are pasting together strings in a formatted text output you should think in advance of what to do if one of the formatting characters (like, I don't know, newline) appears in a string. But again, there's no cultural consensus on what to do, and any scheme you invented would probably be parsed wrongly by lots of userland code, since it's an uncommon case.

Local root vulnerability in snap-confine

intgr — Fri, 18 Feb 2022 15:35:58 +0000

> https://www.qualys.com/2022/02/17/cve-2021-44731/oh-snap-...

Thanks for the link, this report is pure gold and worth reading in full.

Truly, props to snap-confine developers for implementing mitigations to make exploitation so complicated.

As for the other vulnerabilities they found......... *sigh*

> CVE-2021-3995: Unauthorized unmount in util-linux's libmount

> only the first "sz" characters of the
> current user's uid are compared to the filesystem's "user_id" option (sz
> is user_id's length). This second vulnerability allows an unprivileged
> user to unmount the FUSE filesystems that belong to certain other users;
> for example, if our own uid is 1000, then we can unmount the FUSE
> filesystems of the users whose uid is 100, 10, or 1

This is hilarious and sad at the same time. It would be obvious to blame the C language. But I think it's time to start putting blame on developers as well, when instead of using proper primitives like "compare string A and B and tell me if they're equal", they insist on using low-level primitives with pointer arithmetic and buffer manipulation. This is the same mindset that led to the recent "pkexec" vulnerability.

This way of parsing and handling strings should be unacceptable in a culture that values security. Please use (create) a proper C library of high-level string manipulation functions, no shortcuts!

> CVE-2021-3996: Unauthorized unmount in util-linux's libmount

> when parsing /proc/self/mountinfo, the libmount blindly
> removes any " (deleted)" suffix from the mountpoint pathnames

> For example, on Fedora, /tmp is a tmpfs, so we can mount a basic FUSE
> filesystem named "/tmp/ (deleted)" (with FUSE's "hello world" program,
> ./hello) and unmount /tmp itself (a denial of service)

Hilarious. This, again, is a culture problem. We should be migrating to proper serialization formats when communicating complex data structures to between kernel and userland, instead of this stringly typed complex to parse hodgepodge that is "mountinfo".

The "deleted" flag should be a boolean or flags field, not part of the path name string.

> CVE-2021-3997: Uncontrolled recursion in systemd's systemd-tmpfiles

> if we create thousands
> of nested directories in /tmp, then "systemd-tmpfiles --remove" (when
> executed as root at boot time) will call its rm_rf_children() function
> recursively (on each nested directory) and will exhaust its stack and crash

> - for example, on Ubuntu 21.04, systemd-tmpfiles fails to create the
> directory /run/lock/subsys; but because /run/lock is world-writable,
> attackers can create their own /run/lock/subsys; and because various
> legacy packages and daemons write into /run/lock/subsys as root, the
> attackers may create arbitrary files via symlinks in /run/lock/subsys.

Mind=blown. One would expect the crash of a startup service to "fail safe" and possibly disable part of the system. But the auditors managed to turn it into a privilege escalation vulnerability.

Not sure where to put the blame, I could see myself making this mistake just as well.

Local root vulnerability in snap-confine

Wol — Fri, 18 Feb 2022 14:57:57 +0000

Yup. Don't snapshot your symlinks per process, but make it so the first access caches it - REMEMBERING THE INODE - and any further attempts to access the symlink get the same inode until the cache is actively flushed.

Cheers,
Wol

Local root vulnerability in snap-confine

epa — Fri, 18 Feb 2022 12:49:16 +0000

One of the vulnerabilities, CVE-2021-44731, is a symlink attack.

Local root vulnerability in snap-confine

epa — Fri, 18 Feb 2022 12:46:45 +0000

Maybe if a symbolic link has appeared or changed since you last called refresh(), then path lookup operations traversing that link would fail with EREFRESH. This would be fail-safe for older code, and newer code might be able to handle it sanely (it certainly sounds easier than meticulously rewriting all your code with the 'at' system calls). At worst, you just have to restart your daemon. Again, symlinks don't change often in practice so this doesn't seem too high a price.

It would have to be based on real world time, not just whatever mtime is in the file system. A fully snapshottable filesystem would be impossible to graft on to current POSIX semantics, but if we're just making a sticking plaster for race conditions with symlinks, some kind of per-process view of the world seems possible. You're not trying to snapshot exactly what symlinks existed at a point of time, but only to note whether one has changed in any way, and if so fail when it's used.

Local root vulnerability in snap-confine

mathstuf — Fri, 18 Feb 2022 11:51:00 +0000

That is a lot of state that needs to be tracked. It also raises the question of "how do I debug this path (name) not resolving in this daemon" with fun "oh, it hasn't updated its symlink view" kinds of answers.

Local root vulnerability in snap-confine

auerswal — Fri, 18 Feb 2022 11:48:36 +0000

> Oh look, *ANOTHER* symlink vulnerability.

Please read the report, there are no symlink vulnerabilities mentioned at all.

Local root vulnerability in snap-confine

epa — Fri, 18 Feb 2022 08:59:02 +0000

Would it be fair to say that most of these security holes are not caused by symlinks alone, but by race conditions with symlinks?

The legitimate uses of symlinks are mostly static -- a shared library linked to a different name, an old user who no longer exists. They don't change much. Can we somehow restrict the times at which symbolic links can be created or changed? Perhaps a process should only be able to see the symlinks that existed when that process started, unless it calls a refresh_symlinks() system call. That would avoid path traversal race conditions while still letting us have the uses of symlinks we've come to depend on.