LWN: Comments on "Toward more predictable and reliable out-of-memory handling"

Toward more predictable and reliable out-of-memory handling

slack1256 — Fri, 12 Apr 2019 09:44:47 +0000

FYI you can trigger the OOM killer yourself via Alt-SysRq-f.

Toward more predictable and reliable out-of-memory handling

mikachu — Tue, 23 Aug 2016 06:09:38 +0000

These patches were merged in 4.7 and the result is that the OOM killer is now invoked way too easily, when there's plenty of reclaimable memory still around. Hopefully it will be fixed in the stable series, otherwise in 4.8.
http://lkml.iu.edu/hypermail/linux/kernel/1608.2/04321.html

Toward more predictable and reliable out-of-memory handling

Det — Mon, 16 May 2016 17:29:34 +0000

> By 4.6, I might have already ended up with a new motherboard and more ram, so this fix would be too late.

Sooo.. did you?

Toward more predictable and reliable out-of-memory handling

oak — Fri, 15 Apr 2016 23:27:20 +0000

> there are most certainly programs which allocate large amounts of address space without necessarily using that memory in the end.

For example programs that create threads. While the stack in main thread grows dynamically, rest of the threads have a fixed sized stack. While one can specify what sized stack threads get, most programs don't.

By default, thread stack sizes on Linux are currently 8MB. If you have a lot of programs that use a lot of threads, without overcommit that would eat quite a bit of RAM.

Toward more predictable and reliable out-of-memory handling

Shabbyx — Fri, 25 Dec 2015 05:40:02 +0000

I would be so happy to see this merged. I wish it were merged today and backported to all previous kernels.

It's so annoying when OOM happens, it's infinitely more annoying when the kernel decides to freeze the whole system up for minutes, until it says "oh, sorry, I was supposed to kill this guy". I wished many times it would just go ahead with the kill immediately.

By 4.6, I might have already ended up with a new motherboard and more ram, so this fix would be too late.

Toward more predictable and reliable out-of-memory handling

ploxiln — Wed, 23 Dec 2015 01:28:03 +0000

I can't help but chime in with a "me too", I'd much prefer processes be killed quickly when free-able memory is close to running out. If it's anywhere near close, say less than 5% of memory available for use as fs cache, something is already wrong.

The last few times I ran into what should have been an OOM situation, everything just froze, for longer than I cared to wait (at least a few minutes). I've seen this for the whole system, and also for a couple processes in a memory cgroup. I've seen it with no swap at all, and also with some swap space. Freezing for ~15 seconds is worse than death IMHO.

I "devops" some servers that run some processes that occasionally act up, and I've resorted to writing a script that checks memory conditions every few seconds and if there isn't a nice large buffer of free+cache memory, goes about sending SIGTERM and then SIGKILL to the biggest members of the suspected set of processes (and notifying me).

Toward more predictable and reliable out-of-memory handling

eternaleye — Wed, 23 Dec 2015 00:39:22 +0000

Well, the first issue is that while those two points are part of why, they're far from the whole story.

In particular, there are most certainly programs which allocate large amounts of address space without necessarily using that memory in the end.

In addition, changing every use of fork to posix_spawn is... a decidedly nontrivial undertaking.

Besides that, failing to check the return value of malloc() is a pervasive issue, which has been worsened by the default-overcommit behavior on Linux has trained people that "malloc() never fails."

And then, if the user program DOES check the return value of malloc(), what are its options?
- Drop some internal caches
- Loop forever
- Die

i.e., the exact same issue as the OOM killer, with the added downside of not having the ability to take advantage of global information - say, page cache pages being clean.

And while it may _seem_ as though there's a benefit there, where the allocating process takes the hit, it's a false fairness - after all, a 3.9 GB allocation may have just succeeded for another program, and now your shell wants one measly page and can't get it.

Toward more predictable and reliable out-of-memory handling

sam.thursfield — Mon, 21 Dec 2015 16:13:53 +0000

I agree! The out-of-memory killer is infinitely better than having my system freezing up due to swap-thrashing. I've never once thought "oh, I wish my system had hung indefinitely there instead of killing Firefox".

Toward more predictable and reliable out-of-memory handling

sorokin — Mon, 21 Dec 2015 13:07:45 +0000

Do I understand correctly that all problems with OOM handling in Linux are caused by memory overcommitting? Is it true that memory overcommitting is required for two reasons: (1) because fork is used to create child processes and (2) because shared objects are mapped as MAP_PRIVATE?

Can't the (1) be fixed by using something like posix_spawn and can't the (2) be fixed by mapping as MAP_PRIVATE only GOT and PLT and MAP_SHARED everything else in shared objects? As I understand in this case it will be possible to default to vm.overcommit_memory=2 and forget about oom-killer.

Toward more predictable and reliable out-of-memory handling

rgb — Thu, 17 Dec 2015 09:00:13 +0000

"..., but it seems unlikely that users will appreciate it any more than they do now."

Of course you don't really appreciate a crashing program, but I very much prefer that over a freezing up of the whole system. So I do appreciate a system responing to my input very much!