Out-of-memory victim selection with BPF

Posted Aug 19, 2023 20:50 UTC (Sat) by izbyshev (guest, #107996)
In reply to: Out-of-memory victim selection with BPF by mb
Parent article: Out-of-memory victim selection with BPF

> Well, do programs actually still use fork()?

Last time I checked the situation across the languages was the following:

* musl and modern glibc implement posix_spawn() via vfork().
* OpenJDK has been using vfork() by default since forever.
* Modern .NET (nee .NET Core) switched to vfork() at some point.
* CPython uses vfork() in subprocess on Linux by default since 3.10 (and can use posix_spawn() in some non-default cases since 3.8).
* Go uses vfork()-equivalent clone(CLONE_VM|CLONE_VFORK).
* IIRC Rust uses posix_spawn() with a fork() fallback for cases whey they need something not supported by the former.

So it's feasible to avoid fork() nowadays in most cases.

Out-of-memory victim selection with BPF

Posted Aug 21, 2023 16:59 UTC (Mon) by ibukanov (subscriber, #3942) [Link] (1 responses)

Thanks for the info!

But then one should be able disable overcommit on Linux and things will work since most apps use vfork and that does not require for the kernel to reserve the memory as the child shares the memory with the parent until exec. Or have I missed something?

Out-of-memory victim selection with BPF

Posted Aug 21, 2023 22:37 UTC (Mon) by izbyshev (guest, #107996) [Link]

> But then one should be able disable overcommit on Linux and things will work since most apps use vfork

While vfork() is used behind the scenes in many languages, I'd be surprised if most C/C++ programs migrated from fork(). And yes, posix_spawn() is suitable in many cases, but it still lacks portable options for some trivial stuff like changing the current directory (glibc and musl have posix_spawn_file_actions_addfchdir_np() extension for that). One could use a simple wrapper executable to tweak the child attributes and then execve() to the real program, but all of this is annoying.

So I'd expect that whether disabling overcommit is fine or not still depends on your set of apps heavily.

And of course, even if fork() had been the main reason to default to overcommit, there is stuff like mmap'ing lots of memory (without MAP_NORESERVE) and then not touching most of it, like those sparse data structures that people mentioned in the comments. Hyrum's law means that after all these years somebody definitely relies on this.