The out-of-memory (OOM) killer is a longstanding source of controversy in
Linux development circles. The killer comes into play if the kernel
encounters a memory shortage so severe that the ongoing functioning of the
system is endangered. Rather than panic or lock up, the kernel brings in
the OOM killer, which goes looking for processes to kill. The killer has a
complicated set of heuristics built into it in an attempt to have it target
the processes that are least likely to be missed. Anybody who has seen the
OOM killer in action, however, knows that it can still make unfortunate
choices. Choosing the process which (1) is among the least valuable
on the system, and (2) is a significant part of the memory problem is
a difficult task.
As a result of discomfort with this grim reaper lurking within the kernel,
and of recently merged VM improvements, the OOM killer has been removed
from the 2.4.23 prepatch series.
For 2.6, Rusty Lynch has just posted a different
answer that should, perhaps, have been obvious from the beginning.
Rather than trying to come up with a set of OOM killer heuristics that
works for everybody, Rusty's patch sets up a notifier-based mechanism that
allows for pluggable OOM killer modules. With this patch, anybody who
wants to set up a different response to memory shortages need only write a
module implementing that technique.
The patch includes the standard OOM killer, along with an example
alternative which simply panics the system. But there is already talk of
creating OOM killer modules implementing different policies. One, which
has been posted already, targets processes if they are seen to be forking
children which fall victim to the OOM killer; it works on the assumption that the
parent is the real source of the problem. A "blame Mozilla" module has
been suggested. And Alan Cox has suggested involving the security module
code so that a site's security policies can be part of the OOM reaction
It's unclear how far this process will go. But pluggable OOM killers is a
clear way of ending the long discussion over what the right policy should
be. Linux is, after all, about choice, even when the choices are
to post comments)