Avoiding the OOM killer with mem_notify

Posted Feb 1, 2008 4:57 UTC (Fri) by ikm (guest, #493)
Parent article: Avoiding the OOM killer with mem_notify

> When memory gets tight, it is quite possible that applications have memory allocatedoften
caches for better performancethat they could free.

Not many programs actually have any adjustable caches. More often it would be some useless
unreclaimed junk (and of course i'm talking java here). So this change should primarily
benefit them, I guess.

Avoiding the OOM killer with mem_notify

Posted Feb 1, 2008 13:27 UTC (Fri) by nix (subscriber, #2304) [Link]

I don't think I've ever written a substantial program that didn't have 
*some* sort of caching in it, to trade off space against time somewhere 
where `spend the time, every time' was undesirable. Often these caches are 
not expected to be terribly large, and have the exciting expiration policy 
`never', but it would be fairly trivial to respond to a mem_notify signal 
by just ditching the entire contents of all of those caches.

Avoiding the OOM killer with mem_notify

Posted Feb 1, 2008 21:36 UTC (Fri) by droundy (subscriber, #4559) [Link] (3 responses)

There are certain obscure programs like firefox and gimp that have very large caches which
could be dumped under pressure.

Avoiding swap IO with mem_notify

Posted Feb 2, 2008 17:49 UTC (Sat) by riel (subscriber, #3142) [Link] (2 responses)

The patch series is indeed designed primarily to increase system performance by avoiding the
IO penalty of swapping out (and back in) memory that contains data that is useless or can be
easily recalculated.

Decompressing (part of) a jpeg just has to be faster than swapping in something from disk,
simply because disk seek times are on the order of 10ms.

Avoiding the OOM killer is a secondary goal.  I am not sure why that is the headline of the
article...

Avoiding swap IO with mem_notify

Posted Feb 3, 2008 4:14 UTC (Sun) by njs (subscriber, #40338) [Link]

Oh!  This makes *much* more sense.  (Especially the otherwise unintelligible part of the
original article that talks about pages getting swapped out, which has nothing to do with
OOM.)

In fairness, though, the LKML patch announcement just talks about it being good to avoid the
OOM.

Avoiding swap IO with mem_notify

Posted Feb 3, 2008 21:02 UTC (Sun) by oak (guest, #2786) [Link]

The article talks also about embedded systems. Those use use flash which 
doesn't suffer from the seek problem like hard disks do.  On embedded 
memory usage is much more of a problem though and kernel gets pretty slow 
too on devices without swap when memory gets really tight (all kernel does 
is page read-only pages from disk to memory and then discard them again 
until it finally does an OOM-kill).

I thought the point of the patch is for user-space to be able to do the  
memory management in *manageable places* in code.   As mentioned earlier, 
a lot of user-space code[1] doesn't handle memory allocation failures. And 
even if it's supposed to be, it can be hard to verify (test) that the 
failures are handled in *all* cases properly.  If user-space can get a 
pre-notification of a low-memory situation, it can in suitable place in 
code free memory so that further allocations will succeed (with higher 
propability). 

That also allows doing somehing like what maemo does.  If system gets 
notified about kernel low memory shortage, it kills processes which have 
notified it that they are in "background-killable" state (saved their UI 
state, able to restore it and not currently visible to user). I think it 
also notifies applications (currently) through D-BUS about low memory 
condition. Applications visible to user or otherwise non-background 
killable are then supposed to free their caches and/or disable features 
that could take a lot of additional memory.  If the caches are from heap 
instead of memory mapped, it's less likely to help because of heap 
fragmentation and it requiring more work/time though.

[1] Glib and anything built on top of it, like Gtk, assume that if process 
is still running, it got the memory, otherwise it's aborted.