Requesting 'real' memory [LWN.net]

Requesting 'real' memory

Posted Feb 6, 2008 3:25 UTC (Wed) by giraffedata (subscriber, #1954)
In reply to: Requesting 'real' memory by dlang
Parent article: Avoiding the OOM killer with mem_notify

I wonder why we still have fork. As innovative as it was, fork was immediately recognized, 30 years ago, as impractical. vfork took most of the pain away, but there is still this memory resource allocation problem, and some others, and fork gives us hardly any value. A fork-and-exec system call would fix all that.

Meanwhile, if you have the kind of system that can't tolerate even an improbable crash, and it has processes with 200M of anonymous virtual memory, putting up an extra 200M of swap space which will probably never be used is a pretty low price for the reliability of guaranteed allocation.

(Log in to post comments)

Requesting 'real' memory

Posted Feb 6, 2008 5:26 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]

many people would disagree with your position that vfork is better then fork. (the issue came
up on the lkml within the last week and was dismissed with something along the lines of 'vfork
would avoid this, but the last thing we want to do is to push more people to use vfork')

I agree that a fexec (fork-exec) or similar call would be nice to have, but it wouldn't do
much good for many years (until a significant amount of software actually used it)

as for your comment of just add swap space to avoid problems with strict memory allocation.

overcommit will work in every case where strict allocation will work without giving
out-of-memory errors, and it will work in many cases where strict allocation would result in
errors. overcommit will also work in many (but not all) cases where strict allocation would
result in out of memory errors.

if it's trivial to add swap space to avoid the OOM errors in strict allocation, that same swap
space can be added along with overcommit and the system will continue to work in even more
cases.

the only time strict allocation will result in a more stable system is when your resources are
fixed and your applications are fairly well behaved (and properly handle OOM conditions), even
then the scenerio of one app allocating 99% of your ram, preventing you from running other
apps, is still a very possible situation. the only difference is that the timing of the OOM
error is more predictable (assuming that you can predict what software will be run when in the
first place)

Requesting 'real' memory

Posted Feb 7, 2008 0:35 UTC (Thu) by giraffedata (subscriber, #1954) [Link]

Many people would disagree with your position that vfork is better then fork

No, they wouldn't, because I was talking about the early history of fork and comparing the original fork with the original vfork. The original fork physically copied memory. The original vfork didn't, making it an unquestionable improvement for most forks. A third kind of fork, with copy-on-write, came later and obsoleted both. I didn't know until I looked it up just now that a distinct vfork still exists on modern systems.

the only time strict allocation will result in a more stable system is when your resources are fixed and your applications are fairly well behaved (and properly handle OOM conditions)

The most important characteristic of a system that benefits from strict allocation is that there be some meaningful distinction between a small failure and a catastrophic one. If all your memory allocations must succeed for your system to meet requirements, then it's not better to have a fork fail than to have some process randomly killed, and overallocation is better because it reduces the probability of failure.

But there are plenty of applications that do make that distinction. When a fork fails, such an application can reject one piece of work with a "try again later" and a hundred of those is more acceptable than one SIGKILL.