Still waiting for swap prefetch [LWN.net]

Still waiting for swap prefetch

Posted Jul 26, 2007 2:21 UTC (Thu) by butlerm (subscriber, #13312)
Parent article: Still waiting for swap prefetch

Isn't this precisely the sort of thing that should be controlled from user space? i.e. let the kernel provide the mechanism and let a user space daemon implement the policy?

For example, couldn't one create a virtual file called /proc/<pid>/swapin that when written to, fetches all the swapped data for that process back into ram?

The advantage, of course, would be that no existing users would be affected, and the optimal swap prefetch policy could be adjusted according to the application.

(Log in to post comments)

Still waiting for swap prefetch

Posted Jul 26, 2007 6:32 UTC (Thu) by sitaram (subscriber, #5959) [Link]

well I already do something like that :-)

All my cron jobs finish by around 7am, so at about 8am (I usually come in to work around 8:15), cron runs "swapoff -a; swapon -a".

Seems to be OK. I guess one day I will leave FF running with a gazillion tabs and it will get OOM-killed on the swapoff command, but hasn't happened yet...

Still waiting for swap prefetch

Posted Jul 26, 2007 8:41 UTC (Thu) by tialaramex (subscriber, #21167) [Link]

What you've described isn't quite the Right Thing though.

Eliminating swap with swapoff will force the kernel to bring back anonymous pages that had been pushed out, but not pages which are associated with existing storage, such as (almost all) program text, and mmap'd data files.

It's obviously possible to arrange for the kernel to notice that there's a lot of free RAM and a lot of pages, anonymous or otherwise that aren't in RAM and fix that. Unfortunately merely having a good idea isn't enough in Free Software, and in the kernel it's not even enough to have a good idea and an implementation.

One worry which swap prefetch people might be able to reassure me on - does it understand that if I have 2GB of as-yet unused RAM and a fresh 2GB mmap() I don't necessarily want it to try to pull all those pages off disk? ie does it know the difference between pages that have been pushed out and those which were simply never needed yet?

Still waiting for swap prefetch

Posted Jul 29, 2007 23:27 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]

but if the system is pulling the pages off of disk while the system is idle, and puts those pages at the end of the LRU list so that they are the first thing thrown away if you need more memory, what does it hurt you to pull them in and never use them?

Still waiting for swap prefetch

Posted Jul 27, 2007 12:52 UTC (Fri) by IkeTo (subscriber, #2122) [Link]

> The advantage, of course, would be that no existing users would be
> affected, and the optimal swap prefetch policy could be adjusted according
> to the application.

The disadvantages are, of course, that (1) few programs will use that interface, since it is too system dependent, (2) few programs will use that interface at the time it matters, since developers usually cannot exactly know when their program will need a lot of memory (who knows that a crazy user create ten thousand bookmarks in his browser and call expand all...), and (3) large program with VM even larger than the available RAM will fail (or cause the whole system to run very slowly) the moment it calls that interface, and most unluckily, it happens only on some small systems that the developer doesn't have access to.