Race-free process creation in the GNU C Library

(copy-pasta, that should have been --mount-proc=/tmp/img - give us an edit button already!)

> what kind of process would you need to track in a chroot

Any process that wants to spawn a process and use pidfd, but also write the pid in a log file or debug trace? Ignoring portability for a second, it could even be something like make or cargo.

There's a practical problem that a Kubernetes container that is not marked "privileged" (which is a Kubernetes concept, rather different from the ordinary meaning of "privileged" as in "runs as root") gets certain things in /proc overmounted, e.g., /proc/sysrq-trigger and /proc/kcore, as a form of sandboxing. The goal is to reduce the impact of a malicious pid 0 inside a container. (User namespacing would also work, but most Kubernetes deployments don't use it yet - it's an alpha feature on k8s' end and only supports one container runtime.) This is, in isolation, an understandable / defensible feature, and I can see systems other than Kubernetes doing it (e.g., I can totally see it being a systemd Restrict option down the line).

Meanwhile, the kernel has a feature where, if your current /proc is in any way overmounted, you're not allowed to mount a new /proc - because that would give you access to the files that are supposed to be hidden to you. This is also, in isolation, an understandable / defensible feature.

The intersection of these features is that you can't correctly mount /proc inside a nested container or container-like thing inside a non-privileged Kubernetes container. If you make a new pidns (either because you're root or via a new userns, as in your example), all the paths in /proc are wrong because they refer to outer PIDs.

(The intersection of these features also ceases to be really defensible in the case where you don't allow your Kubernetes workloads to run as pid 0, which is a really good idea on its own.)

There have been some patches for a second procfs (whose exact name I'm forgetting) that provides /proc/$pid/ and the /proc/self/ symlink but not anything else in /proc, but I don't think they've been merged. If those could get merged and guaranteed mountable by anyone with CAP_SYS_MOUNT in their current namespace, regardless of what the existing /proc outside it looks like or even whether it exists, that would satisfactorily address the issue.

I suppose another option would be for /proc to always enumerate the calling process's PID namespace, but maybe that gets weird with open file descriptors passed between PID namespaces.

> The 1980s are calling and want their problems back ;-) In 2023 and on modern Linux, file descriptors are only limited by available memory. Open as many as you want.

Oh, the youthful banter of someone who hasn't spent a few hours this week debugging issues caused by file descriptor exhaustion!

(In this case, it was due to a hypervisor that booted a VM with trivial amounts of memory, the VM kernel adjusted system-wide file descriptor limits down accordingly, then the hypervisor would hotplug another 32GB of RAM later...)

For what it's worth, I agree this _should_ have been a problem relegated to history. I want to live in the future!

> The 1980s are calling and want their problems back ;-) In 2023 and on modern Linux, file descriptors are only limited by available memory. Open as many as you want.

Is that really common to have no ulimit for them? 1024 fds limit has been very typical what I've seen (since default FD_SET size is that, so most programs that use select() will break on high fds)

> Considering they've been available as-is for 4 years and nobody bothered to do anything about that,

People have been bothering to do something about that, and it has taken this long to get something on a potential path to acceptance.

It's the fault of libc that we cannot simply call clone3 directly. It's the responsibility of libc to *stop hiding the underlying useful functionality* just because it thinks it knows better.