User: Password:
Subscribe / Log in / New account

Toward reliable user-space OOM handling

Toward reliable user-space OOM handling

Posted Jun 9, 2013 0:13 UTC (Sun) by nix (subscriber, #2304)
In reply to: Toward reliable user-space OOM handling by tstover
Parent article: Toward reliable user-space OOM handling

The problem is not swap. The problem is fork(). On Solaris I used to routinely have problems running tiny git instances under my Emacs because the Emacs (VSZ 860Mb) wanted to fork() and exec() a 15Mb git instance, and Solaris stopped it, because there was only 700Mb swap free, and you never know but that Emacs might want to COW *all* its pages after that fork()! It never ever does (and almost no programs ever do), but the no-overcommit policy stopped me doing useful work nonetheless, just in case.

No-overcommit sucks.

(If you want proper no-overcommit you'd better also think about interactions with mmap(), and about what to do with stack allocations that run out of memory. You can't return NULL for *those*... so the program has to be ready to be OOM-killed on any function call or any new basic block in any case. So I don't see what overcommit gives you, other than the really annoying behaviour described above. It certainly doesn't buy you safety, just an insane proliferation of NULL checks and error paths that never get tested. Of course I write them anyway, but I know they'll never be executed so they nearly all just exit(), faking an OOM killer in case some idiot turned it off...)

(Log in to post comments)

Toward reliable user-space OOM handling

Posted Jun 9, 2013 12:05 UTC (Sun) by cortana (subscriber, #24596) [Link]

Forgive me if the answer is obvious, but why didn't Emacs (and other programs) use the vfork system call?

Toward reliable user-space OOM handling

Posted Jun 9, 2013 12:27 UTC (Sun) by mpr22 (subscriber, #60784) [Link]

Because generally when emacs forks, it's likely to want to capture the output and/or control the input of whatever program the child execs. This is nearly painless with fork(), and a hideous mess with vfork().

Toward reliable user-space OOM handling

Posted Jun 18, 2013 12:16 UTC (Tue) by nix (subscriber, #2304) [Link]

I suppose it could in theory have used posix_spawn*(), which were added to make no-MMU POSIX implementations happy (which must implement fork() via immediate copying) and are in themselves an indictment of the non-fork()/exec() model, with a huge proliferation of functions for every possible thing that you might want to do between fork() and exec(). Except of course that it's not every possible thing, it's just a few things, and if you want to add a new thing you need to change the libc implementation (at least; quite possibly the kernel implementation on some platforms) and jam it through the Austin Group and wait for years, while in the fork()/exec() model it's just a few lines of code and a few minutes' work.

fork()/exec() is so clearly the Right Thing in this respect that I can't imagine why anyone ever argues against it. But people still do! (Though not here. This has been your out-of-place rant for the day.)

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds