User: Password:
Subscribe / Log in / New account

Toward reliable user-space OOM handling

Toward reliable user-space OOM handling

Posted Jun 10, 2013 16:57 UTC (Mon) by giraffedata (subscriber, #1954)
In reply to: Toward reliable user-space OOM handling by alankila
Parent article: Toward reliable user-space OOM handling

Yes, if people aren't even willing to earmark the virtual memory at malloc time, they certainly wouldn't be willing to earmark it at program startup time.

Of course, some (rare) people are willing to earmark at malloc time (they turn off overcommit), and some of them would probably appreciate being able to earmark it at startup time. I have seen that done with user space memory allocators: the program at startup time (or other convenient decision point) gets from the kernel a large enough allocation of virtual memory for its future needs, then suballocates from that.

This is orthogonal to eliminating the OOM killer. You do that by turning off overcommit. What this adds is that when your program fails randomly because of other processes' memory usage, it fails at startup in code designed to handle that eventuality instead of at some less convenient point where you're trying to allocate 80 bytes to build a string.

(Log in to post comments)

Toward reliable user-space OOM handling

Posted Jun 10, 2013 20:17 UTC (Mon) by alankila (guest, #47141) [Link]

Yeah, preallocating the heap ahead of time and then touching it (to make sure it is backed by kernel) seems like a way to escape a later OOM and could be coalesced into a startup routine, though with the downside that those pages could be mostly zero but nevertheless become anonymous memory that the kernel must track. Oh well.

Toward reliable user-space OOM handling

Posted Jun 10, 2013 21:43 UTC (Mon) by giraffedata (subscriber, #1954) [Link]

Touching pages of the preallocated heap ahead of time to prevent OOM later is probably a poor way to go about it, because you're just trading a later necessary OOM kill for a sooner possibly unnecessary OOM kill, and that doesn't seem better. Also, unless every process in the system does this preallocation, even the ones doing it could get killed later by the OOM killer. No one is safe from the OOM killer.

Better just to disable overcommit (/proc/sys/vm/...). Then there's no OOM kill ever, there are no unnecessary page faults, and when the swap space runs out, the programs that preallocate the heap just get an allocation failure as they start up and can terminate gracefully.

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds