The "too small to fail" memory-allocation rule

Posted Dec 24, 2014 9:23 UTC (Wed) by epa (subscriber, #39769)
In reply to: The "too small to fail" memory-allocation rule by ibukanov
Parent article: The "too small to fail" memory-allocation rule

fork-exec is one of the aspects of classical Unix which looks really neat in a textbook but hasn't held up quite so well in the real world. Remembering that 'things should be as simple as possible, but no simpler', it is just a bit too simple. POSIX does define spawn functions which userspace code might call instead.

Essentially, when you fork() the kernel has no way of knowing that you are just about to call exec() immediately afterwards, so it has to either reserve space for a complete copy of all your process's memory, or end up overcommitting and resorting to an OOM killer if it guessed wrong. If userland can give the kernel a bit more information then the kernel can do its job better.

The "too small to fail" memory-allocation rule

Posted Dec 24, 2014 9:35 UTC (Wed) by roc (subscriber, #30627) [Link] (9 responses)

One great feature of the fork()/exec() model is that you can set up the child's environment by running code in the child before exec --- setting resource limits, manipulating file descriptors, dropping privileges, etc. It's easy to do this in a race-free way. With Windows' CreateProcess you can't do this, so it takes 10 parameters including a flags word with 16 flags and a STARTUPINFO struct with over a dozen members, and it's still less flexible than fork()/exec().

Also, copy-on-write fork()ing has other valuable uses. In rr we use it to create checkpoints very efficiently.

The "too small to fail" memory-allocation rule

Posted Dec 24, 2014 9:54 UTC (Wed) by epa (subscriber, #39769) [Link] (3 responses)

This is true. But the manipulation of file descriptors shouldn't require a complete copy of the parent process's memory. It would be handy if you could fork passing a buffer of memory and a function pointer. The child process will be given a copy of that buffer and will jump to the function given. Then you have some flexibility in what you can do before exec() but you can still be frugal in memory usage without relying on overcommit.

The "too small to fail" memory-allocation rule

Posted Dec 25, 2014 18:44 UTC (Thu) by quotemstr (subscriber, #45331) [Link]

> The child process will be given a copy of that buffer and will jump to the function given

vfork does what you want. The "child" shares memory with the parent until it calls exec, so you avoid not only the commit charge catastrophe of copy-on-write fork, but also gain a significant performance boost from not having to copy the page tables.

Re: shouldn't require a complete copy of the parent process's memory

Posted Dec 25, 2014 22:18 UTC (Thu) by ldo (guest, #40946) [Link] (1 responses)

That issue was solved a long time ago, which is why the vfork(2) hack is obsolete nowadays.

Re: shouldn't require a complete copy of the parent process's memory

Posted Dec 26, 2014 7:33 UTC (Fri) by epa (subscriber, #39769) [Link]

Sorry what issue are you referring to? The conjecture so far is that there is a problem, since a forked process gets a copy of its parent's address space, which requires either reserving enough memory (RAM or swap) to provide that, or else overcommitting and crossing your fingers (with OOM killer for when you get it wrong). It is true that copy-on-write means the kernel doesn't have to copy all the pages straight away, but that is just a time saving; it doesn't resolve the underlying issue of needing overcommit.

vfork() doesn't make a copy of the address space and so doesn't require either over-caution or over-committing. But it has other limitations.

The "too small to fail" memory-allocation rule

Posted Dec 24, 2014 10:18 UTC (Wed) by ibukanov (subscriber, #3942) [Link] (4 responses)

> One great feature of the fork()/exec() model is that you can set up the child's environment by running code in the child before exec

To customize the child before exec one does not need full fork, vfork is enough and does not require overcommit.

The "too small to fail" memory-allocation rule

Posted Dec 24, 2014 11:34 UTC (Wed) by pbonzini (subscriber, #60935) [Link] (3 responses)

Only in Linux. The POSIX description is "you can store the return value from vfork(), then you have to call _exit or exec".

The "too small to fail" memory-allocation rule

Posted Dec 24, 2014 12:31 UTC (Wed) by ibukanov (subscriber, #3942) [Link] (1 responses)

Linux semantic of vfork is one of the most usable as it blocks only the thread in parent that calls it. Thus one can get rather safe and efficient alternative to fork/exec by creating a new thread, calling vfork from it, preparing the child and calling the exec. Compare that with, say, FreeBSD or Solaris that, according to the manual pages, suspends the whole parent process during the vfork call.

The "too small to fail" memory-allocation rule

Posted Dec 30, 2014 0:04 UTC (Tue) by klossner (subscriber, #30046) [Link]

By "create a new thread" do you mean call pthread_create? That just ends up in do_fork(), which creates a new process with which to call vfork(), which ends up in do_fork() to create another new process. Safe, okay, but not really efficient.

The "too small to fail" memory-allocation rule

Posted Dec 24, 2014 14:27 UTC (Wed) by justincormack (subscriber, #70439) [Link]

Most versions allow you to do more than this though, not just the Linux one. Makes it all rather non portable though.