Weekly Edition
Return to the Kernel page
| |||||||||||||
zcache: a compressed page cache
Last year, Nitin Gupta was pushing the compcache patch, which
implemented a sort of swap device which stored pages in main memory,
compressing them on the way. Over time, compcache became "ramzswap" and
found its way into the staging tree. It's not clear that ramzswap can ever
graduate to the mainline kernel, so Nitin is trying again with a
development called zcache.
But zcache, too, currently lacks a clear path into the mainline.
Like its predecessors, zcache lives to store compressed copies of pages in memory. It no longer looks like a swap device, though; instead, it is set up as a backing store provider for the Cleancache framework. Cleancache uses a set of hooks into the page cache and filesystem code; when a page is evicted from the cache, it is passed to Cleancache, which might (or might not) save a copy somewhere. When pages are needed again, Cleancache gets a chance to restore them before the kernel reads them from disk. If Cleancache (and its backing store) is able to quickly save and restore pages, the potential exists for a real improvement in system performance. Zcache uses LZO to compress pages passed to it by Cleancache; only pages which compress to less than half their original size are stored. There is also a special test for pages containing only zeros; those compress exceptionally well, requiring no storage space at all. There is not, at this point, any other attempt at the unification of pages with duplicated contents (as is done by KSM), though. There are a couple of obvious tradeoffs to using a mechanism like zcache: memory usage and CPU time. With regard to memory, Nitin says:
While compression reduces disk I/O, it also reduces the space
available for normal (uncompressed) page cache. This can result in
more frequent page cache reclaim and thus higher CPU
overhead. Thus, it's important to maintain good hit rate for
compressed cache or increased CPU overhead can nullify any other
benefits. This requires adaptive (compressed) cache resizing and
page replacement policies that can maintain optimal cache size and
quickly reclaim unused compressed chunks. This work is yet to be
done.
The current patch does allow the system administrator to manually adjust the size of the zcache area, which is a start. It will be a rare admin, though, who wants to watch cache hit rates and tweak low-level memory management parameters in an attempt to sustain optimal behavior over time. So zcache will almost certainly have to grow some sort of adaptive self-tweaking before it can make it into the mainline. The other tradeoff is CPU time: it takes processor time to compress and decompress pages of memory. The cost is made worse by any pages which fail to compress down to less than 50% of their original size - the time spent compressing them is a total waste. But, as Nitin points out: "with multi-cores becoming common, benefits of reduced disk I/O should easily outweigh the problem of increased CPU usage." People have often wondered what we are going to do with the increasing number of cores on contemporary processors; perhaps zcache is part of the answer. One other issue remains to be resolved, though: zcache depends on Cleancache, which is not currently in the mainline. There is some opposition to merging Cleancache, mostly because that patch, which makes changes to individual filesystems, is seen as being overly intrusive. It's also not clear that everybody is, yet, sold on the value of Cleancache, despite the fact that SUSE has been shipping it for a little while now. Until the fate of Cleancache is resolved, add-on patches like zcache will be stuck outside of the mainline. (Log in to post comments)
zcache: a compressed page cache Posted Jul 29, 2010 6:45 UTC (Thu) by Tara_Li (subscriber, #26706) [Link]
Hard drives have gotten *SO* slow, relative to CPU/memory speeds, that I have to wonder that there's any question about the value of compressing storage. And it's not like we need to use the absolute best compressor - anything reasonable should be more than good enough to get us a big boost. I know right now, my Ubuntu 10.04 system is using /dev/ramzswap0 (which seems to be some kind of compressing RAM swap system) to great effect - I can almost instantly tell when swap starts having to hit the hard drive.
zcache: a compressed page cache Posted Jul 29, 2010 10:01 UTC (Thu) by saffroy (subscriber, #43999) [Link]
Of course, there is question about the value of compressing stored data.
Without compression, I/O can occur with little participation of the CPU, which "only" has to setup data structures, do a bit of device I/O, and let the DMA process all data while the CPU can do other useful stuff for you. Of course, uncompressed data is larger, and take more time to read or write, but that's not the whole story.
With compression, the CPU has to process each and every byte of data that's read or written. Your compression algorithm has better be fast and efficient on your data. And you consume a LOT more CPU time than before, and some more memory bandwidth, and you also probably trash your memory caches heavily. And you may drain your laptop battery faster too (depends which of large disk I/O vs. compression+small I/O needs more power).
Now I definitely support the idea of compressing data before I/O, it's just that it has to be carefully thought out in order to be a definite improvement, it does not work equally well on every workload. For instance, I bet the bitmaps of the photos I'm editing in the Gimp are hard to compress fast; how will the kernel know that it should probably not try to compress them?
zcache: a compressed page cache Posted Jul 29, 2010 23:56 UTC (Thu) by jzbiciak (✭ supporter ✭, #5246) [Link]
The tradeoff skews very heavily in favor of compression these days, at least if you're using a traditional HD.
Hard drive seek times are measured in multiple milliseconds. 1ms is 1 million cycles on a 1GHz CPU. You can do a lot in that time. For every seek you eliminate, you buy back millions and millions of CPU cycles.
As for your GIMP working set? The main gain you'll get there is from the other data in your system compressing, making room for GIMP. And besides, GIMP does its own paging via its tile cache, doesn't it?
zcache: a compressed page cache Posted Jul 30, 2010 20:09 UTC (Fri) by joern (subscriber, #22392) [Link]
Actually, hard drives penalize compression as well. You have plenty of bandwidth to waste anyway, so compressing the data to roughly half size does not help much. But with compression, it becomes much nearly impossible to avoid fragmentation. Whenever content changes, the compressed size changes.
Wrt. swap, compression may well end up reducing seeks. But for file backed data, compression costs a lot more performance than the cpu cycles spent in zlib or whereever.
zcache: a compressed page cache Posted Jul 31, 2010 3:00 UTC (Sat) by jzbiciak (✭ supporter ✭, #5246) [Link]
The tradeoff with swap is that it can delay (and maybe completely avoid) hitting the disk at all. There are plenty of workloads that compression can make fit in RAM, whereas without compression, it can hit the disk a fair bit.
If you take hitting the disk as a given, as is the case with a compressed filesystem with compressed files, then the tradeoffs get much more complex. This is especially true whenyou consider that most files are small relative to disk block sizes.
But with VM, it's about dirty anonymous pages (heap and such). The only reason to push these to disk is that you're trying to use more RAM than is available. Disk acts as RAM overflow. If compression can push the "swap to disk" threshold out, then it is a clearer win.
zcache: a compressed page cache Posted Jul 31, 2010 6:42 UTC (Sat) by saffroy (subscriber, #43999) [Link]
"There are plenty of workloads that compression can make fit in RAM"
Interesting. Got more details or pointers about this?
zcache: a compressed page cache Posted Jul 31, 2010 14:34 UTC (Sat) by jzbiciak (✭ supporter ✭, #5246) [Link]
Here's an older one about compcache: https://code.google.com/p/compcache/wiki/Performance/LTSP...
That at least illustrates the principle that compression can keep you from hitting the disk and also increase the size of workload a machine can handles effectively.
zcache: a compressed page cache Posted Jul 31, 2010 3:04 UTC (Sat) by jzbiciak (✭ supporter ✭, #5246) [Link]
And, I should add that with file-backed pages, it sound like it just compresses the in-memory copies. This defers writeback, which can lead to more contiguous allocations (in the case of new files or files that are getting extended), or for transient files, fewer temp files pushing blocks to disk.
zcache: a compressed page cache Posted Jul 29, 2010 13:56 UTC (Thu) by edt (subscriber, #842) [Link]
compcache became ramzswap which then became zram. zram is a compressed block device. It can be used for swap or as a backing store for a filesystem. It does not depend on cleancache. zcache is a second subsystem that reuses some of the code zram uses.
zcache: a compressed page cache Posted Jul 30, 2010 9:19 UTC (Fri) by juanjux (guest, #11652) [Link]
What I know is that in my old, now retired, HTC G1, activating the compcache option on Cyanogen's ROM improved the performance and the number of apps loaded in memory *a lot*. I'm not activating it on my Nexus because, unlike the G1, it has a lot of memory to spare.
zcache: a compressed page cache Posted Aug 7, 2010 4:09 UTC (Sat) by nwmcsween (guest, #62367) [Link]
zlib was shown many years ago to be suboptimal w.r.t memory compression. There are a few new papers (2-4 years old I think) that pit zlib vs. lzo vs. a solution specific to 'normal' operating system memory usage patterns with the latter coming in around 1.3x better and 4x faster. The author should really look into these papers if he/she hasn't already.
|
|||||||||||||