Great!

Posted May 18, 2015 15:45 UTC (Mon) by epa (subscriber, #39769)
In reply to: Great! by Limdi
Parent article: Rust 1.0 released

I admit, if running on Linux there isn't much you can do to avoid being squashed by the OOM killer. There are however operating systems which take a more cautious approach.

Avoiding the OOM Killer by Quotas?

Posted May 20, 2015 17:42 UTC (Wed) by gmatht (guest, #58961) [Link] (1 responses)

It seems to me that (1) allocating memory and (2) reserving memory are two different things; where (1) adds memory to an address space, while (2) requests a guarantee that some amount of memory is reserved for your use. As I understand, Linux allows you to do (1), but doesn't allow you to do (2) without disabling overcommit (which comes with its own problems).

However (2), could be a syscall of its own. Roughly, the idea is that to avoid the OOM killer, a process is responsible for reserving memory prior to allocating it. This allows, a process to choose, for example, to:
1) not bother reserving memory, it probably won't be OOM killed anyway,
2) Permanently reserve 640K (which should be enough, right?),
3) Release all reservations when in a safe idle state, allowing a OOM kill,
4) Make sure it has 10MB extra in reserve before accepting a new connection, or
5) Wrap every call to malloc, fork, etc. to make sure it has enough in reserve for all children.

Doing (5) may mean the process is denied reservations long before the machine is low on memory, but that's already what happens with overcommit disabled. That may be what the application writer wants. (1) can be quite nice too; sometimes it is much easier to recover from an abort than to recover from low memory. Allowing some processes to choose 1 and 5 already seems like an advantage. For lots of user space code (4) seems cleaner to me than checking the return of every "malloc(sizeof int)". When a bug is found fixing the result of corruption from an abort may be easier than fixing the result of mishandled null pointer. How often do we really need (5) in userspace?

Avoiding the OOM Killer by Quotas?

Posted May 23, 2015 8:53 UTC (Sat) by epa (subscriber, #39769) [Link]

Reserving memory in advance is a useful operation - so you can fail earlier if memory is short. Even if your app does have full checking of all allocations with rollback when necessary, it may just be easier for all concerned to refuse the incoming connection if there probably won't be the resources to service it. I think I would still prefer that malloc() of ten megabytes would reserve that space - not just pretend to succeed and then dynamite my process at some undetermined later point. But for handling legacy or 'lazy' code which prefers to rely on overcommit, a mixed model such as those you suggest could work.