This could be a generalization of the OOM killer. When memory is tight, some unlucky process is chosen and any pages it hasn't used recently are marked as volatile. It keeps running but if it tries to access one of those pages then boom! This might be a bit better than just killing it immediately. I would guess that many user-space applications leak memory, in the sense that they allocate pages, use them for a bit and then never touch them again.
Posted Nov 5, 2012 20:23 UTC (Mon) by scottwood (subscriber, #74349)
[Link]
The downside to that is you could still have time bombs lurking after the out-of-memory situation is cleared up. And if you notify the task that it had better clean up and restart, that could cause the bad memory to be touched when it otherwise wouldn't have...
Volatile whether you like it or not
Posted Nov 5, 2012 20:25 UTC (Mon) by jimparis (subscriber, #38647)
[Link]
> This could be a generalization of the OOM killer. When memory is tight, some unlucky process is chosen and any pages it hasn't used recently are marked as volatile. It keeps running but if it tries to access one of those pages then boom! This might be a bit better than just killing it immediately.
That's a cool idea, but potentially a security risk via information leak -- if you use a lot of memory on a system and trigger the OOM killer, you can now determine how another process is behaving or treating its input by whether it goes boom or not. I can only think of contrived examples at the moment, but attackers are more clever.