Zeuthen: Writing a C library, part 1

the problem with your suggestion (only allow overcommit for processes that just forked), is that I don't see that working. you have no way of knowing if the process is going to exec something (relatively) soon, or if it's apache that forked a child that is going to stay running for the next year.

And I don't think it helps anyway.

the problem scenario is

large process A forks (creating A`), almost all it's memory is COW

process B allocates some memory, but doesn't touch it yet

process A` changes some memory (breaking COW), requiring real memory to hold the result.

process B then tries to use the memory it had previously allocated and finds that it is not available.

if you could somehow define 'forked recently' in a way that could be cheap enough, then you could possibly do it.

All this said, I really don't see many cases in practice where disabling overcommit will really help.

yes, you avoid the OOM killer kicking in and instead the process that tried to allocate memory dies instead.

but the idea that (in the general case), this will make your system more predictable is not something I believe. you have no way of knowing _which_ process (including system daemons) will need to allocate more memory at the instant that you are out, so you really don't know which process will die anyway. (and no, in general processes and libraries don't do anything except die when they run out of memory).

in some ways, it would make it easier to DOS a system, just have your memory hog _not_ die if a malloc fails, instead sleep and try again. eventually something else in the system will need memory and die, then you can repeat the process. you won't even be able to ssh in to the box to fix it, as you won't be able to spawn/fork a new process (as that will require memory allocation)

there's also the problem that without overcommit you need to have significantly more swap enabled in the system (since you have to have enough ram+swap to handle the peak theoretical memory use from large processes doing a fork+exec), and with the increasing gap between memory speed and disk speed, your system will dive into swap to the point of being useless (including the inability to login to it) before you start getting memory failures. With overcommit you can have a small amount of swap (including none) and instead count on the OOM killer + watchdog timers to bring the box down (and possibly even reboot it to recover) rather than having the box 'up' but unable to provide service.