User: Password:
|
|
Subscribe / Log in / New account

Avoiding the OOM killer with mem_notify

Avoiding the OOM killer with mem_notify

Posted Feb 1, 2008 3:03 UTC (Fri) by vomlehn (subscriber, #45588)
In reply to: Avoiding the OOM killer with mem_notify by jzbiciak
Parent article: Avoiding the OOM killer with mem_notify

> I'm guessing mem_notify will try to wake apps sufficiently far ahead that it can avoid those
"need RAM to free RAM" situations in practice, but setting proper thresholds seems like it
ought to be rather tricky

I don't see how the kernel can possibly know enough for it to notify applications far enough
ahead about the need to free memory; the memory allocation behavior of applications is just
too unpredictable.

An approach that seems like it would be better would be to notify the kernel that certain
pages in your application are being used to cache data. The kernel is then free to simply grab
them if it needs them. If your application decides it needs the data later, it uses as sytem
call to notify the kernel that the pages are no longer being used as a cache. If the kernel
didn't need the pages, they would still have their old data and the application could use them
directly.

On the other hand, if the kernel did have to grab the pages in the interim, the system call
used to grab the pages back would return an error. Your application would then know it needs
to remap the pages and regenerate the data. Of course, it's possible the pages can't be
remapped because memory is too low. The application would handle that as though the data
wasn't cached and it couldn't get the memory to read it. It already has to be able to do this,
so this doesn't add to the application's complexity.

The advantages of this approach are that the pages are immediately available to the kernel
without having to wake the process up. No need to figure out complex threshholds, no need to
allocate enough memory for the process to run, no delay in making the needed memory available.
You could even allow for priorities when telling the kernel the pages are being used for cache
so that the kernel would grab lower priority pages first.

I wish I had the time to code this and submit it because I think that mem_notify is an awful
botch that will cause unending pain as people add patch on patch to try to make it work. But
that's just my personal opinion...


(Log in to post comments)

Avoiding the OOM killer with mem_notify

Posted Feb 1, 2008 19:47 UTC (Fri) by zlynx (subscriber, #2285) [Link]

Applications can already see if they're missing memory pages by using mincore().

You may not have been carefully reading the mem_notify patch descriptions.

What it does is trigger on memory pages going into the inactive list.  This is what happens to
prepare memory pages that are good candidates for swapping.

Here is the Changelog from version 5 of the mem_notify patch, see the v3 changes:
Changelog
-------------------------------------------------
  v4 -> v5 (by KOSAKI Motohiro)
    o rebase to 2.6.24-rc8-mm1
    o change display order of /proc/zoneinfo
    o ignore very small zone
    o support fcntl(F_SETFL, FASYNC)
    o fix some trivial bugs.

  v3 -> v4 (by KOSAKI Motohiro)
    o rebase to 2.6.24-rc6-mm1
    o avoid wake up all.
    o add judgement point to __free_one_page().
    o add zone awareness.

  v2 -> v3 (by Marcelo Tosatti)
    o changes the notification point to happen whenever
      the VM moves an anonymous page to the inactive list.
    o implement notification rate limit.

  v1(oom notify) -> v2 (by Marcelo Tosatti)
    o name change
    o notify timing change from just swap thrashing to
      just before thrashing.
    o also works with swapless device.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds