| |||||||||||||
|
| |||||||||||||
Compcache: in-memory compressed swappingCompcache: in-memory compressed swappingPosted May 27, 2009 7:23 UTC (Wed) by macc (subscriber, #510)In reply to: Compcache: in-memory compressed swapping by JoeF Parent article: Compcache: in-memory compressed swapping
Shoudn't that be layered?
mem -> compressed swap|blockdev --> to disk
snitching on disk IO should work for hibernation too
MACC
(Log in to post comments)
Compcache: in-memory compressed swapping Posted May 28, 2009 10:24 UTC (Thu) by rvfh (subscriber, #31018) [Link]
Maybe the swap partition should be compressed too, so we don't
mem -> compressed swap -> uncompressed swap on disk
Compcache: in-memory compressed swapping Posted May 29, 2009 4:15 UTC (Fri) by nitingupta (subscriber, #53817) [Link]
> Maybe the swap partition should be compressed too, so we don't
> mem -> compressed swap -> uncompressed swap on disk
This is the idea I'm working on. Swap-out entire xvmalloc pages -- each containing multiple compressed pages -- to swap disk. The aim here is not to save disk space but to improve performance.
Would xvmalloc and swap readahead play nice? Posted Jun 6, 2009 7:37 UTC (Sat) by gmatht (subscriber, #58961) [Link]
Wouldn't swapping out xvmalloc pages prevent swap readahead from being of any use, given that adjacent pages are unlikely to be allocated in adjacent positions by xvmalloc? On a conventional HDD, reading an uncompressed page should take only ~0.1ms while seeking to the page should take ~10ms. My concern is that optimizing the 0.1ms while forcing a 10ms seek for every page would be a big performance loss.
There seems to be a big difference between the optimal layout for a memory allocator where seek is not a problem and the optimal layout on a conventional hard disk were seek times dwarf virtually everything else.
If, OTOH, adjacent pages were written out in adjacent positions on disk this could *halve* the cost of swap readahead; both halving the time required to read in the extra pages and also halving the memory used by pages that where read from disk but not used.
(I can see just swapping out xvmalloc pages being a win for SSD, where seek is not a problem for random reads. Also clearly if you are writing out an xvmalloc page there should be very little overhead, and you know you will get 4k of real memory back for each page swapped out. Even so, wouldn't you still have to read in the entire 4K xvmalloc page just to access one of the compress pages stored on that page?)
Would xvmalloc and swap readahead play nice? Posted Jun 7, 2009 5:22 UTC (Sun) by nitingupta (subscriber, #53817) [Link]
> Wouldn't swapping out xvmalloc pages prevent swap readahead from being of any use, given that adjacent pages are unlikely to be allocated in adjacent positions by xvmalloc? On a conventional HDD, reading an uncompressed page should take only ~0.1ms while seeking to the page should take ~10ms. My concern is that optimizing the 0.1ms while forcing a 10ms seek for every page would be a big performance loss.
With compressed swapping to disk, the seek times will also reduce as pages will be spread over a smaller area on disk. Still, in general swapping out xvmalloc pages is expected to incur higher swap read overhead due to more no. of seeks involved - an xvmalloc page contains almost unrelated pages.
> There seems to be a big difference between the optimal layout for a memory allocator where seek is not a problem and the optimal layout on a conventional hard disk were seek times dwarf virtually everything else.
Yes, this the whole problem. Theoretically, this problem could be solved by first collecting together physically contiguous pages (w.r.t disk sectors) in a single memory page and then swap this page to disk. However, when pages are swapped out this way, we are not guaranteed that we will be able to free even a single page. Also, this will increase in-memory fragmentation as these pages will be taken out from random xvmalloc pages. So, after lots of such pages are swapped out, we have to do some in-memory defragmentation (not yet implemented) to bring down fragmentation and free pages.
> If, OTOH, adjacent pages were written out in adjacent positions on disk this could *halve* the cost of swap readahead; both halving the time required to read in the extra pages and also halving the memory used by pages that where read from disk but not used.
In general, swap readahead in its present state is almost meaningless in case most of the pages are in (compressed) memory. Decompressing pages is almost instant. Instead, more useful will be to implement some sort of prefetch ioctl for ramzswap so its prefetches pages from backing swap and keeps them compressed in memory. But which pages to prefectch? This will need more study and experimentation.
> (I can see just swapping out xvmalloc pages being a win for SSD, where seek is not a problem for random reads. Also clearly if you are writing out an xvmalloc page there should be very little overhead, and you know you will get 4k of real memory back for each page swapped out. Even so, wouldn't you still have to read in the entire 4K xvmalloc page just to access one of the compress pages stored on that page?)
Yes, reading in single xvmalloc page will bring in bunch of unrelated pages to memory. These additional pages may be kept/discarded based on configurable/hardcoded policy in ramzswap.
Compcache: in-memory compressed swapping Posted May 29, 2009 5:57 UTC (Fri) by zmi (subscriber, #4829) [Link]
> mem -> compressed swap -> uncompressed swap on disk
> [and from the article:] > allow swapping of xvmalloc memory to physical swap disk
That was my immediate idea when reading the article. I'd love it to be a
The disk swap should support compressed pages directly, and you can also
If this feature arrives, the vm.swappiness can be increased to more quickly
I wonder if ramzswap will help on my 8GB desktop, and want to test it.
BTW: shouldn't there be a compressed name also? Like zap or just zp ;-)
|
|
||||||||||||