User: Password:
|
|
Subscribe / Log in / New account

Toward reliable user-space OOM handling

Toward reliable user-space OOM handling

Posted Jun 6, 2013 18:52 UTC (Thu) by tstover (guest, #56283)
In reply to: Toward reliable user-space OOM handling by alankila
Parent article: Toward reliable user-space OOM handling

Agreed. Setting /proc/sys/vm/overcommit_memory to 2 supposedly will go back to the traditional malloc() == NULL behavior. I have to ask the obvious questions. What is wrong with swap? Why is a user space OOM killer better than just not letting user space programs over commit, and actually do memory management? ...

Oh! I see .... java ... sad sad sad

How much do you think this is really about the android platform - ie no swap, java, etc?


(Log in to post comments)

Toward reliable user-space OOM handling

Posted Jun 7, 2013 4:22 UTC (Fri) by Ben_P (guest, #74247) [Link]

In the standard Java library, you need to explicitly request some mmap'd memory that isn't mlock()'d (MappedByteBuffer). Otherwise, the JVM will be moving around chunks of your heap during compactions and such. Perhaps it's difficult to maintain reasonable performance across all the jvm platforms if you allow parts of the heap to get paged out?

On a related note, until the API changes or the Java spec changes to 64 bit indexed arrays you'll be unable to use mmap64() from that class.

Toward reliable user-space OOM handling

Posted Jun 9, 2013 0:13 UTC (Sun) by nix (subscriber, #2304) [Link]

The problem is not swap. The problem is fork(). On Solaris I used to routinely have problems running tiny git instances under my Emacs because the Emacs (VSZ 860Mb) wanted to fork() and exec() a 15Mb git instance, and Solaris stopped it, because there was only 700Mb swap free, and you never know but that Emacs might want to COW *all* its pages after that fork()! It never ever does (and almost no programs ever do), but the no-overcommit policy stopped me doing useful work nonetheless, just in case.

No-overcommit sucks.

(If you want proper no-overcommit you'd better also think about interactions with mmap(), and about what to do with stack allocations that run out of memory. You can't return NULL for *those*... so the program has to be ready to be OOM-killed on any function call or any new basic block in any case. So I don't see what overcommit gives you, other than the really annoying behaviour described above. It certainly doesn't buy you safety, just an insane proliferation of NULL checks and error paths that never get tested. Of course I write them anyway, but I know they'll never be executed so they nearly all just exit(), faking an OOM killer in case some idiot turned it off...)

Toward reliable user-space OOM handling

Posted Jun 9, 2013 12:05 UTC (Sun) by cortana (subscriber, #24596) [Link]

Forgive me if the answer is obvious, but why didn't Emacs (and other programs) use the vfork system call?

Toward reliable user-space OOM handling

Posted Jun 9, 2013 12:27 UTC (Sun) by mpr22 (subscriber, #60784) [Link]

Because generally when emacs forks, it's likely to want to capture the output and/or control the input of whatever program the child execs. This is nearly painless with fork(), and a hideous mess with vfork().

Toward reliable user-space OOM handling

Posted Jun 18, 2013 12:16 UTC (Tue) by nix (subscriber, #2304) [Link]

I suppose it could in theory have used posix_spawn*(), which were added to make no-MMU POSIX implementations happy (which must implement fork() via immediate copying) and are in themselves an indictment of the non-fork()/exec() model, with a huge proliferation of functions for every possible thing that you might want to do between fork() and exec(). Except of course that it's not every possible thing, it's just a few things, and if you want to add a new thing you need to change the libc implementation (at least; quite possibly the kernel implementation on some platforms) and jam it through the Austin Group and wait for years, while in the fork()/exec() model it's just a few lines of code and a few minutes' work.

fork()/exec() is so clearly the Right Thing in this respect that I can't imagine why anyone ever argues against it. But people still do! (Though not here. This has been your out-of-place rant for the day.)

Toward reliable user-space OOM handling

Posted Jun 10, 2013 15:40 UTC (Mon) by alankila (guest, #47141) [Link]

I don't think you have any idea how java works.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds