Cost vs benefit ?

user32.dll and macOS libc, in contrast, are extremely stable. I can compile an app on Windows 11, and it will likely work on Windows XP, as long as it doesn't use any missing functions.

I understand that symbol versioning theoretically prevents subtle bugs with slightly different function behaviors between versions. But we're talking about freaking libc!

So, I don't know about Zig, but the Go runtime is among the most well-engineered. I'd generally trust them to handle any footguns around process spawning. I took a quick look at their code, and for some reason they use CLONE_VFORK only if not using CLONE_NEWUSER. Which is by far most of the time, but it is still odd that they do fork. At least in the Go world, it is not common to encounter pthread_atfork issues. So, rare use of fork is just a cost thing most of the time. But yes, occasionally they'll face trouble using it.

With that said, libc isn't always 100% right, either. Looks like I need to elaborate (and I welcome people pointing out any errors in the reasoning below).

So, first, let me note that I silently conflated classic vfork and CLONE_VFORK. Let's remind ourselves that a classic vfork returns to _the same stack frame_ in both the parent and the child. So we're permanently one compiler optimization away from complete disaster. One can easily imagine a tiny asm "trampoline" function that makes classic vforking safe. Just have it invoke the user-supplied function in the child. But this is not how people commonly do vfork. On the other hand, CLONE_VFORK forces you to give a dedicated stack to the child (which can be allocated on the parent's stack, since spawning doesn't need much stack). Making it nice and clean.

But not all libc-s use CLONE_VFORK. glibc started doing so around 2016. Before that, defaulting to fork. With the "use-vfork" flag available and defaulting to off. Notably, bionic still does the exact same behavior (fork by default, vfork (classic one!) if asked).

Now, what is the problem with fork. Apart from being much more expensive (and more so with larger and more heavily threaded processes, as pointed out above), it is effectively incompatible with multi-threading. To remind people. Fork snapshots the parent process's memory. So a bunch of locks will occasionally be in the "locked" state, and some global state protected by those locks will be messed up. Among those messed-up things, you're not allowed to invoke malloc/free (but see below). Of course, if all you do in the child is invoking few syscalls and exec-ing, it doesn't matter. That is what posix_spawn is built to do. All the "actions" are prepared in advance (which involves dynamic memory allocation) and only a carefully curated set of essentially bare syscalls are invoked in the child. The only global state ever touched is errno (which is already implemented to be async-signal-safe).

So, in theory posix_spawn-ing via fork versus CLONE_VFORK becomes just a matter of efficiency. But in practice, we have pthread_atfork complications.

pthread_atfork allows custom code to be invoked before and after fork (in both the parent and child). A somewhat typical use is to take lock(s) before fork, so that whatever global state is in the snapshot is safe and consistent, and then release the lock(s) after fork. The issue is making sure the ordering of taking the locks is safe. Which, in the most general case, is ~impossible to get right. A particularly notable use case for that is jemalloc. Being reasonably advanced malloc, they have a number of locks. And they have a very elaborate code scattered across every single module that carefully takes those locks in the right order. This, and the fact that jemalloc sets up its atfork handler early enough, makes malloc/free usable in practice in child processes after forking. I maintain gperftools, and we also do the locking/unlocking dance and early registration. But we don't have absolutely all the locks covered, and doing so would be quite challenging. Our "sibling" project, abseil tcmalloc, doesn't do any pthread_atforks. And doing so is explicitly forbidden by Google's policies.

To elaborate on the locking order troubles of pthread_atfork. Imagine there are modules foo and bar. Foo occasionally invokes bar while holding some lock. In that case, whatever lock(s) bar has cannot be taken before foo's lock. Violating that order triggers an obvious deadlock. pthread_atfork runs the handlers in the reverse order of registration. So we "simply" need to have bar's atfork handler registered before foo's. But with realistic programs having many thousands of modules, arranging this specific order of registration is basically impossible. That is the reason Google forbids atfork.

In general, it is reasonably well documented that mixing multithreading and fork is unsupported. You're supposed to either use threads (like e.g. mysql) or to use fork without threads (e.g. postgresql). There are some notable exceptions that have to be super-careful. E.g. redis. Both ruby and python do threads and expose fork. At least python does loud warning. Not their worst design mistake, but clearly a mistake.

There are other uses of atfork. For example, if your process mmaps some hardware MMIOs (e.g. for GPU access or NVMe or NIC), you really cannot let hardware conflate parent and child accesses. So something has to be done. Way back nvidia drivers actually had a bit of a problem there. I had bug reports against gperftools of processes stuck somewhere inside the system() call. Note that, in this case, there was no intention to actually use nvidia stuff from both processes. Turns out whatever nvidia drivers did invoked malloc, and it was before we carefully ensured early registration of gperftool's atfork handler. So our atfork handler ended up running before theirs, and everything deadlocked when their handler tried to malloc or free something. Their current drivers only reset some flag in their atfork handler, and avoid any trouble. It may cost them, presumably, in the slight overhead of having to check that flag in ~all their calls that touch shared state. So pthread_atfork cannot be entirely avoided, but at least sometimes it can be done in a way that is guaranteed safe.

So the issue with fork in posix_spawn (or, e.g., system()) is that your multi-threaded process is exposed to the fundamentally messy aspects of fork completely unintentionally.