LWN: Comments on "The long road to a fix for CVE-2021-20316"

The long road to a fix for CVE-2021-20316

nybble41 — Wed, 23 Feb 2022 19:37:56 +0000

> Given that ownership (and pretty much all other metadata) is stored with the inode, and a hard link is just another reference to the same inode, it's impossible to create a hard link with different ownership (or other metadata) than the original.

Yes, but you can create a hardlink to a file you don't own in a location which the owner of the hardlinked inode can't access. If they later lose their own path to the inode (by unlinking it, for example) then the inode continues to exist, because of the hardlink, and thus continues to count against their quota, but they have no way to remove it.

The long road to a fix for CVE-2021-20316

anton — Wed, 23 Feb 2022 12:11:27 +0000

* Symlinks interact better with filesystem quotas. If you can create a hard link to a file owned by another user in a location they can't access, and the original path is later deleted, then you can cause that part of their quota to be permanently inaccessible to them. (Alternatively, the system can disallow hard links to other user's files—in which case the advantage of symlinks is not having this restriction.)

Given that ownership (and pretty much all other metadata) is stored with the inode, and a hard link is just another reference to the same inode, it's impossible to create a hard link with different ownership (or other metadata) than the original. On some file systems you can create a reflink (new inode, shared data); I don't know how these file systems implement the quota in this case, but IMO the way to handle this is to blame the cost of the file in full on all users (i.e., as in a regular copy).

[/tmp:128248] ls -l foo
-rw-r--r-- 1 root root 4 Feb 23 12:56 foo
[/tmp:128249] ln foo bar
ln: failed to create hard link 'bar' => 'foo': Operation not permitted
[/tmp:128251] cp --reflink foo bar
[/tmp:128252] ls -l bar
-rw-r--r-- 1 anton users 4 Feb 23 12:57 bar

The long road to a fix for CVE-2021-20316

Jandar — Tue, 22 Feb 2022 16:56:07 +0000

> This is how many corporate managed devices, and all iOS and Android devices work.

And this is why an Android device even with a Linux kernel doesn't provide a very Unix like experience.

If my desktop, laptop or servers would restrict me in the same way as an Android device I would immediately switch the distribution or to something more different like one of the BSDs.

The long road to a fix for CVE-2021-20316

Fowl — Mon, 21 Feb 2022 00:38:44 +0000

Well, yeah. This is how many corporate managed devices, and all iOS and Android devices work.

The long road to a fix for CVE-2021-20316

foom — Sat, 19 Feb 2022 21:56:08 +0000

Classic macos did, kinda, have a shell, shell scripts, and command line apps, in MPW (https://en.m.wikipedia.org/wiki/Macintosh_Programmer's_Workshop). But that was just an app, not the fundamental underpinnings of the system.

(Of course modern macos does have both command line POSIX apps and aliases)

The long road to a fix for CVE-2021-20316

Jandar — Sat, 19 Feb 2022 17:10:19 +0000

> Windows has admin-only created symlinks.

Marvelous, a kind of symlink a user can't use. What comes next, only admin approved (*) programs can create executables? Doubtless it would be a security enhancement if no executable memory-mapping could contain bytes generated by an arbitrary program.

Most symlinks I use or encounter are made by users within their own directories pointing into other parts of their own space.

* The mainframe I used at university had such a concept. IIRC only an admin at the console could set the compiler-permission.

The long road to a fix for CVE-2021-20316

nybble41 — Fri, 18 Feb 2022 17:46:37 +0000

> [from the linked commit] The rationale is that if you can create symlinks owned by user X you *are* user X (or root), and if you also own directory Y you can do whatever with it already, including change permissions.

This is a false assumption. The owner of a file or directory can only change permissions or make other changes to it *if they can obtain a reference*. If you own /a/b but don't have search permissions (+x) on /a (and don't have /a/b as your current directory or some similar corner case) then you can't do anything with /a/b. However, under this rule you can still create a symlink to /a/b which would be followed by other users because you own /a/b, even though you can't access it yourself through /a. (The same issue impacts /proc/sys/fs/protected_symlinks=1.)

One option I haven't seen suggested yet would be to use the intersection of the permissions available to the original user and the owner(s) of any symlink(s) followed while resolving the path. Though in practice you might need more information than just the symlink's UID and GID to serve as the credentials, especially when relying on "negative ACLs", pluggable security modules like SELinux, or narrower group permissions overriding, rather than supplementing, broader "other" permissions.

The long road to a fix for CVE-2021-20316

nix — Fri, 18 Feb 2022 01:50:44 +0000

Aha! I see there are no new ideas, only ideas someone else already implemented, better :) The 'target directory owner' idea is a nice addition, which could probably be added in to what I thought up above to close even more cases where legitimate use might be broken without enabling any troublesome cases at all.

The long road to a fix for CVE-2021-20316

Karellen — Thu, 17 Feb 2022 14:43:09 +0000

I think the parent commenter was referring to the case where you have a symlink somewhere pointing to /foo/bar, and /foo/bar is replaced through its canonical path, rather than through the symlink. In that case, the symlink (wherever it is) remains pointing at the new /foo/bar

The long road to a fix for CVE-2021-20316

khim — Thu, 17 Feb 2022 12:52:18 +0000

> However, the downside is that they are not functional for command-line/cross platform programs, which don't use alias-aware file access apis.

But classic MacOS had no support for command-line apps which solved that problem.

The long road to a fix for CVE-2021-20316

ssmith32 — Wed, 16 Feb 2022 19:24:48 +0000

You'd still need to deal with giving multiple users different levels of access (r/w, at least), to the same file/directory. But it would help with the directory escape bugs.

The long road to a fix for CVE-2021-20316

jra — Wed, 16 Feb 2022 18:55:51 +0000

Bind mounts are an interesting case. They're much newer than symlinks, and the security implications of allowing any user to perform them are not yet well explored. But they are a separate user-initiated operation which is not normally exposed over network filesystems, making them really a local-only attack.

They actually change the view of the filesystem hierarchy in a way that symlinks don't. Symlinks are a point attack that can modify a specific path quickly and easily to anywhere on the filesystem if allowed or exposed over a network filesystem. IMHO they are much more dangerous.

The long road to a fix for CVE-2021-20316

foom — Wed, 16 Feb 2022 15:06:26 +0000

Classic macos didn't have symlinks either. It did have "aliases", but those were (and are -- they still exist in modern macos, alongside symlinks and hardlinks and copy-on-write copies..) more akin to windows lnk files, as they're resolved at a higher level than basic filesystem traversal.

This allows them to be less fragile -- they store multiple ways of finding the original file, so it can still be used even in case the target has been moved or renamed. They can also do things like display a gui prompt to ask the user to insert the appropriate floppy disk which contains the target file, or automatically mount the proper remote volume.

However, the downside is that they are not functional for command-line/cross platform programs, which don't use alias-aware file access apis.

The long road to a fix for CVE-2021-20316

mathstuf — Wed, 16 Feb 2022 14:22:20 +0000

> * If a file is deleted and replaced (i.e. the ubiquitous "atomic rename" pattern) this breaks hard links, but not symlinks.

Some programs still break symlinks because they operate on the pathname, not the result of following the symlink (basically, they replace the symlink instead of replacing the destination file).

The long road to a fix for CVE-2021-20316

pmatilai — Wed, 16 Feb 2022 08:59:53 +0000

Yup, that is exactly the approach rpm started taking internally since 2017: https://github.com/rpm-software-management/rpm/commit/f2d...
I've yet to see it break any legitimate case in the packaging world.

That commit is of course only a tiny fraction (and buggy at that) of the overall problem which we're only just now getting close to resolving it in a wider sense - much like Samba, rpm traditionally used absolute paths for everything.

The long road to a fix for CVE-2021-20316

Cyberax — Tue, 15 Feb 2022 20:13:03 +0000

You can transform the tree into a graph using bind mounts. So arguably it's the POSIX API that is not beautiful.

The long road to a fix for CVE-2021-20316

Wol — Tue, 15 Feb 2022 19:45:40 +0000

> Which OS do you use that doesn't have symlinks?

DID I use ... they pre-date linux, MacOS and Windows ...

A Multics-derivative called Pr1mos, various DOSes (I don't think I used CP/M ...) (and I don't mean MS-DOS and derivatives ...)

There used to be a world with many different OSes, not just the three behemoths we have now.

Cheers,
Wol

The long road to a fix for CVE-2021-20316

jra — Tue, 15 Feb 2022 19:24:06 +0000

Symlinks are seductive. They provide all the benefits you enumerate, but at the cost of completely destroying application security when working with pathnames. They took the beautiful POSIX API of a directory tree structure and turned it into a graph.

After enough years, I'm convinced they were a foundational mistake in POSIX. We need to move past them and stop this mistake from being repeated. More in my SambaXP talk :-).

The long road to a fix for CVE-2021-20316

nybble41 — Tue, 15 Feb 2022 17:44:04 +0000

> So what ARE the advantages?

* You can symlink across filesystems, but hard links only work within filesystems.

* If a file is deleted and replaced (i.e. the ubiquitous "atomic rename" pattern) this breaks hard links, but not symlinks.

* Symlinks can point to directories, while hard links to directories are either disallowed altogether or heavily restricted.

* Symlinks interact better with filesystem quotas. If you can create a hard link to a file owned by another user in a location they can't access, and the original path is later deleted, then you can cause that part of their quota to be permanently inaccessible to them. (Alternatively, the system can disallow hard links to other user's files—in which case the advantage of symlinks is not having this restriction.)

* You can read a symlink to determine where it goes, but with hard links you only have an inode number.

* Directory entries accessed through symlinks have canonical paths, but hard-linked inodes have multiple, equally authoritative, paths.

> Coming from OSs that didn't have symlinks, I don't use them.

Which OS do you use that doesn't have symlinks? Not Linux, or MacOS, or Windows… though symlinks in Windows are a bit different, it does have them. You create them with the "mklink" command. They aren't strictly limited to administrator accounts either; that's just a default setting. The biggest difference is that Windows distinguishes between symlinks to files and symlinks to directories. It also has "junction points" which work much like symlinks (but only for directories). Also let's not forget "shortcuts" (*.lnk files), which are basically symlinks implemented somewhat inconsistently at a layer above the kernel. These have been around since Windows ran under DOS.

The long road to a fix for CVE-2021-20316

Wol — Tue, 15 Feb 2022 16:07:59 +0000

> they are also one of the killer features

What's so "killer feature" about them?

Coming from OSs that didn't have symlinks, I don't use them. I do make use of hard links, mostly to provide multiple links to semi-archive material (and also to protect against accidental deletion, which symlinks don't). So what ARE the advantages?

Cheers,
Wol

The long road to a fix for CVE-2021-20316

rahulsundaram — Tue, 15 Feb 2022 14:09:40 +0000

> On the Windows command line, there was no equivalent to "ln -s". They might as well not exist, for all the good they do there. Maybe Windows uses them internally for something, but no end-user does.

Your info is heavily outdated. It is well supported and well documented these days.

https://docs.microsoft.com/en-us/windows-server/administr...

The long road to a fix for CVE-2021-20316

eru — Tue, 15 Feb 2022 12:18:09 +0000

The Windows symlink-like feature is hard to use, and last I tried it, it was nearly undocumented (admittedly that was several years ago when I needed it). On the Windows command line, there was no equivalent to "ln -s". They might as well not exist, for all the good they do there. Maybe Windows uses them internally for something, but no end-user does.

The long road to a fix for CVE-2021-20316

jra — Mon, 14 Feb 2022 17:32:27 +0000

No, they aren't anymore. Windows has admin-only created symlinks. The key difference is only root can create them.

The long road to a fix for CVE-2021-20316

eru — Mon, 14 Feb 2022 08:07:03 +0000

Symlinks may be problematic, but IMHO they are also one of the killer features Linux (and relatives) have over that other OS.

The long road to a fix for CVE-2021-20316

metalheart — Sun, 13 Feb 2022 19:40:32 +0000

What is the use-case to support symbolic links on Windows file sharing?
From my past experience it makes changes in organizational structure easier to implement on file server directory structure (you do not need to copy data from one share to another). What else?

The long road to a fix for CVE-2021-20316

mjg59 — Sun, 13 Feb 2022 19:02:30 +0000

Hm. I wonder how viable it would be to implement this using the BPF LSM?

The long road to a fix for CVE-2021-20316

khim — Sun, 13 Feb 2022 18:23:09 +0000

> Wouldn't this make analysis harder? That is, would root follow arbitrary symlinks?

Root would follow only root-created symlinks. Everyone else would also follow root-created symlinks. Everyone else would just follow their own symlinks.

It's actually interesting idea. Would need to see how many packages would break, but this wouldn't effect things like git (people rarely use unix permissions in the middle of git repos) while most distro-provided system symlinks would work.

I think there are some container solutions which use per-app UID (like on Android), would need to decide what to do about these.

The long road to a fix for CVE-2021-20316

marcH — Sun, 13 Feb 2022 16:53:41 +0000

Sounds like https://www.infoq.com/news/2014/06/everything-google-cont...

The long road to a fix for CVE-2021-20316

cozzyd — Sat, 12 Feb 2022 08:53:10 +0000

Did SELinux mitigate this issue? Not sure hownit handles Samba.

The long road to a fix for CVE-2021-20316

bartoc — Sat, 12 Feb 2022 02:55:06 +0000

Generally, I think linux is fine adding in (usually default to off) options that break userspace in the name of security (I could be wrong). I think something like this is worth trying out at least to see how much stuff breaks, given it could remove a whole class of toctou bugs.

My userspace was just broken by lack of `/dev/mem` on my fedora box (I was trying to fetch the EDID of my display)

The long road to a fix for CVE-2021-20316

bartoc — Sat, 12 Feb 2022 02:52:04 +0000

embracing the AT calls is still a good idea imo, there have been windows apps with toctou bugs that would have been fixed with a proper openat equivalent, and windows requires administrator ACLs to create symlinks (for exactly this reason)

The long road to a fix for CVE-2021-20316

gerdesj — Sat, 12 Feb 2022 00:20:34 +0000

Samba is an amazing piece of kit. Your user base is *cough* technically varied in its skill set. It is expected to dance on a shitty old NAS with wheezing discs to the latest bleeding edge SAN as a side trick and all things in between. The expectations of those users is broader than the smile on a croc that has discovered a zebra nursery ... must work on that analogy - a bit brutal.

I can remember testing out Ben Greer's smart new VLAN code in the kernel to get a set of smbd and nmbds running on a fairly large network to get a browse list together. This is me a few years back: https://lwn.net/Articles/75489/ whittering on about it.

Samba makes CIFS/SMB work in ways that MS has never even imagined. That's the thing - imagination. Samba is imaginative where the MS option is rather staid and boring.

Now if it was possible to get ACLs to work like NetWare nwfs/nss ie dynamically calculated on the fly, that would be quite handy.

Anyway, cheers Jez. That was a lot of work fixing things up. Thank you.

The long road to a fix for CVE-2021-20316

jra — Fri, 11 Feb 2022 18:43:22 +0000

We did explore a chroot solution. Problem is there are many restrictions on that which make it impossible to use with Samba without a complete rewrite. Rewriting the VFS was an easier task, believe me :-).

The long road to a fix for CVE-2021-20316

NYKevin — Fri, 11 Feb 2022 18:00:29 +0000

Can you at least stick the Samba process in a container (or maybe a chroot) so that it can't get to random other paths?

The long road to a fix for CVE-2021-20316

jra — Fri, 11 Feb 2022 17:47:27 +0000

Samba (smbd) is just an application running on Linux. By default it can access anywhere on the filesystem the logged on user has access to. The point of Samba is to designate a small area of the filesystem (e.g. /data/exported/for/group) and ensure that *no* access outside oe the path "/data/exported/for/group" or sub-directories below it is ever possible.

The long road to a fix for CVE-2021-20316

jra — Fri, 11 Feb 2022 17:45:01 +0000

It's not file systems. It's applications. Or even system libraries. Most of them are just not symlink-safe. Some of the ones I know about are system *security* libraries. It's a big mess.

The long road to a fix for CVE-2021-20316

ldearquer — Fri, 11 Feb 2022 16:39:10 +0000

>> the server makes sure that the requested directory actually lies within the exported SMB share rather than being at some arbitrary location elsewhere in the server's filesystem

But why do Samba servers have write access to any locations other than the SMB share in the first place? Or is this for cases where SMB share == the whole filesystem?

The long road to a fix for CVE-2021-20316

phlogistonjohn — Fri, 11 Feb 2022 15:59:17 +0000

For the curious, I believe the projects Jeremy is referring to are our efforts here:
https://github.com/samba-in-kubernetes/samba-container/
and the related projects in our org https://github.com/samba-in-kubernetes/

Note that despite the name "kubernetes" in the org, the container images are designed not to be k8s
specific. I'd love to see other uses of the container images for docker/docker-compose, podman, etc. The name was partly chosen because we do have other k8s specific integration plans... and we could abbreviate it as "SINK" ;-)
Thank you for the opportunity for a bit of free advertising.

The long road to a fix for CVE-2021-20316

Paf — Fri, 11 Feb 2022 15:23:18 +0000

Having read more carefully, I can see the history here that leads to this, but I doubt many other *file systems* share this vulnerability. But processing a *path* more than once during what should be an atomic operation makes the operation obviously and deeply non-atomic. It’s also going to be less efficient, possibly a lot less. Other projects probably have issues with symlinks, but my guess is not many file systems will.

The long road to a fix for CVE-2021-20316

Paf — Fri, 11 Feb 2022 15:17:42 +0000

“ Unfortunately, there is inevitably a window between when the server performs the check and when it actually creates the directory.”

There doesn’t have to be, or at least not this kind…. Having a separate *path navigation* for security check and for the operation is … wow. You do a security oriented lookup on the entity (in this case the parent), checking permissions as you go, then you have the entity. You then just use it. That the path is processed twice is … loopy and obviously a huge problem.