Still waiting for swap prefetch

Still waiting for swap prefetch

Posted Jul 27, 2007 12:32 UTC (Fri) by IkeTo (subscriber, #2122)
Parent article: Still waiting for swap prefetch

> A concern which has been raised a few times is that the morning swap-in
> problem may well be a sign of a larger issue within the virtual memory
> subsystem, and that prefetch mostly serves as a way of papering over that
> problem. And it fails to even paper things completely, since it brings back
> some pages from swap, but doesn't (and really can't) address file-backed
> pages which will also have been pushed out. The conclusion that this
> reasoning leads to is that it would be better to find and fix the real
> problem rather than hiding it behind prefetch.

My interpretation: there are 3 classes of systems:

1. Those that have loads of memory and very few programs requiring big memory that ever run, and as such never write anything to the swap. Swap prefetch is of course a no-op in such systems.
2. Those that have loads of memory and very few programs requiring big memory that ever run, but still the swap is being written. Swap prefetch improves the performance of such system, but developers would ask, "why the hell did it happen in the first place?"
3. Those that have not such much memory to run all the programs requiring big memory that ever run, and naturally swap prefetch does help, as expected.

So because of the unclear reason that (2) happens, and perhaps because those systems in class (1) might see regressions and need to manually turn off prefetching via a kernel option (never mind that none is currently known after 18 months of testing), those (typically) systems among (3) have to suffer? After all, prefetch is not something that is done only to the swap. A block I/O system without prefetch also have horrible performance, and we have prefetch there for ages. So why swap has to be treated differently? Should we perfectly expect that an application being swapped (for whatever reason) should perform much worse than an application being loaded the first time if prefetching happens when the application is first being loaded and does not happen when the application is pushed to the swap due to uncontrollable memory pressure? Is it such a surprise that swap prefetch is something needed anyway?

And what is the consequence to (2) if swap can be prefetched? It means there is no way to detect such a problem exists? Of course not, the kernel keeps page fault counters, if developers care to write a script and collect the stats of each of the running processes. The only thing that will happen is that people will no longer be so unhappy about the problem, because that doesn't hit their bottom line: enjoying hot coffee while working in the morning. And the end result is, unsurprisingly, less attention to the problem. But... is it really such a bad thing after all, that some hard problem can be put aside because the problem no longer cause serious user dismay?

Perhaps I'm understanding something really wrong.

---

(Log in to post comments)