User: Password:
|
|
Subscribe / Log in / New account

Can swap-in be improved?

Can swap-in be improved?

Posted May 24, 2007 7:44 UTC (Thu) by mgb (guest, #3226)
Parent article: On-demand readahead

The place where readahead is really needed is when swapping in a long-idle application.

Anyone with a modest desktop will have experienced the problem of switching back to an application that has sat idle for half an hour. The drive thrashes away for minutes as apparently individual pages are faulted in. This often takes much longer than starting the application from scratch.

I wonder if the kernel could detect this situation, perhaps by noticing an unusually high sustained rate of reads from the swap file for a given process, and try temporarily reading some larger number of contiguous pages instead of one.

There are undoubtedly corner cases where this would make things worse but I suspect that overall something like this would be a considerable improvement.


(Log in to post comments)

Can swap-in be improved?

Posted May 24, 2007 10:03 UTC (Thu) by MathFox (guest, #6104) [Link]

Swap-in is non-sequential reads, so it's hard to predict which pages to read ahead. OTOH, memory is pretty cheap nowadays (1 GB ~ $50); buying a DIMM may quickly give you a more responsive system. Professional system administrators don't want their systems to swap under normal load.

Can swap-in be improved?

Posted May 24, 2007 14:43 UTC (Thu) by i3839 (guest, #31386) [Link]

More ram isn't necessary to avoid swapping. If you have swap usage you don't have a moderate desktop, but instead a decadent corpulent bloated desktop. That, or you're using something that needs hogs of memory, like video encoding apps, but that's beyond moderate too. I'm doing fine with 256 MB for years, and although ram is dirt cheap, I simply see no reason to buy more of it.

mgb: You might be interested in Con's swap prefetch.
See http://members.optusnet.com.au/ckolivas/kernel/

Can swap-in be improved?

Posted Jul 12, 2007 18:18 UTC (Thu) by xorbe (guest, #3165) [Link]

Yeah, but if you are not using an app, Linux throws out your program to cache more of your hard drive. Unavoidable.

Can swap-in be improved?

Posted May 24, 2007 14:44 UTC (Thu) by mgb (guest, #3226) [Link]

DIMMs are cheap in this part of the world but laptops to holder larger DIMMs are not. In some circumstances it is economical to have sufficient RAM to hold one or two apps but to use swap when three or four are open and temporarily idle.

The thought was that one could overcome the slow random swap-in reads by reading ahead pages from swap which had been used earlier and would probably therefore be used again.

Can swap-in be improved?

Posted May 24, 2007 18:32 UTC (Thu) by NAR (subscriber, #1313) [Link]

When I leave from work, I don't turn off the laptop, just lock the screen. Next morning when I unlock the screen, it takes about 10-15 seconds to have the browser or OpenOffice swapped back. Probably a cronjob needs some memory, that's why they are swapped, but it's still annoying.

Bye,NAR

Can swap-in be improved?

Posted May 25, 2007 1:34 UTC (Fri) by interalia (subscriber, #26615) [Link]

A plausible culprit is the cronjob that runs updatedb so that your 'locate' database is up-to-date. That churns the cache quite a bit since it involves a lot of disk seeking.

Can swap-in be improved?

Posted May 26, 2007 2:10 UTC (Sat) by giraffedata (subscriber, #1954) [Link]

One thing that would improve this and various other things is actual swapping. What we call swapping in Linux is really just virtual memory paging. Swapping is a much older technology in which you remove an entire process from memory to let other processes run, then bring the entire process back when it's turn comes around again. It's also known as long term scheduling.

If Linux could notice that a process hasn't accessed any of its pages (hasn't run) in a long time, it could write all the resident pages out to contiguous swap space. The next time the process runs, it could read it all back.

Furthermore, if Linux is stealing a page that was accessed quite recently, it ought to steal them all and then wait a while before swapping them all back in again and swapping out somebody else.

By the way, it would be a mistake to believe that all that paging activity when you sit down to a long-idle session is Linux "swapping." A lot of it is reading in memory mapped program files. Speeding that up isn't as easy.

Can swap-in be improved?

Posted May 31, 2007 17:04 UTC (Thu) by anton (subscriber, #25547) [Link]

A lot of it is reading in memory mapped program files. Speeding that up isn't as easy.
I think it's not very hard. You just need some history-based prediction mechanism. A student of mine has done his Master's thesis (in German) on speeding up the startup time of programs by recording the blocks accessed when starting a program, and then prefetching them (Windows XP does something like that for the OS startup). Similarly, one could record the pages or blocks accessed soon after a major page fault, and then prefetch all of them the next time the page faults.

Can swap-in be improved?

Posted Jun 2, 2007 2:30 UTC (Sat) by giraffedata (subscriber, #1954) [Link]

That's actually different, and a little harder, than what we're talking about here. We're talking about resuming a process that has been idle for a while. So all the history you need is the list of pages that were resident the last time the process ran.

That by itself can't get you to the level of speed that we discussed with anonymous pages, where all those pages were contiguous on disk.

I know I've seen somewhere a system where you don't ordinarily start a program, but rather reload and resume an entire process. In that case, the two prefetching problems are probably the same.

Can swap-in be improved?

Posted Jun 7, 2007 13:29 UTC (Thu) by anton (subscriber, #25547) [Link]

Yes, fetching all the pages that were resident is certainly an approach for the problem that would work.

Using a history-based prefetcher might have the following advantages, though:

  • It could fetch fewer pages (e.g., not pages loaded on startup, but no longer needed, but that were still resident when the process was swapped out), which would be faster and less disruptive for other processes.
  • The prefetcher would also be useful in other situations, like on startup, or when a process enters a new phase (and therefore needs many new pages from executable or data files).
That being said, a well-balanced history-based prefetcher is certainly a much larger project than loading the once-resident pages.

Can swap-in be improved?

Posted Jul 25, 2007 23:00 UTC (Wed) by Blaisorblade (guest, #25465) [Link]

> A lot of it is reading in memory mapped program files. Speeding that up
> isn't as easy.

Surely swap prefetch won't help here, indeed. But good readahead would.

About swapping, the reason it's old is that, as Rik van Riel (and another
reader below) puts it, systems are slower than ages ago. Would you swap
out firefox, ever? Or X?

> Furthermore, if Linux is stealing a page that was accessed quite
> recently, it ought to steal them all and then wait a while before
> swapping them all back in again and swapping out somebody else.

Well, the swap token was introduced by Rik van Riel to implement something
similar, based on a research paper. Actually it's mostly a mean to prevent
thrashing (which can be what you describe, depending on the times
involved). It didn't work at first and was disabled, but now it should
have been fixed; a new variation of the algorithm was implemented in
7602bdf2fd14a40dd9b104e516fdc05e1bd17952 (in 2.6.20-rc1). LWN described it
well, see Kernel Page Index for more details.

Can swap-in be improved?

Posted May 25, 2007 8:58 UTC (Fri) by dlang (subscriber, #313) [Link]

there is another set of patches that was sent out in the last week or so that addresses readahead for swap-in

the reports on it were extremely favorable


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds