Close but no cigar

Posted Apr 30, 2009 6:25 UTC (Thu) by khim (subscriber, #9252)
In reply to: KSM tries again by ncm
Parent article: KSM tries again

Of course all such pages come from common block somewhere on Debian's build server. But there are no easy way to track this relationship at all! If you are thinking about shared libraries and such - this is different kettle of fish and THESE pages are shared already. KSM is designed to work with KVM and Xen where identical pages come from different virtual filesystems and thus from different physical places. May be in the future btrfs will change this, but KSM works here and now...

Memory deduplication by common disk source

Posted May 3, 2009 2:47 UTC (Sun) by giraffedata (guest, #1954) [Link] (1 responses)

Your point about KSM working in the here and now is good, but as a question of long term strategy, ncm's is equally good. Maybe KSM should be thought of as a stop-gap.

In the systems where the memory duplication is a big problem, the common disk source of the memory pages is a lot closer than Debian's build server. If you have a hundred virtual machines, you probably did the Debian install once and copied the disk image a hundred times locally.

If that copy is a naive physical copy, then it's still hard for the hypervisor to know that two memory pages have the same ultimate source. But if you apply the same deduplication strategy to the duplicated disk (and there's plenty of work going on to make that a reality), then you have 100 virtual disk devices all sharing the same physical disk blocks and the hypervisor knows it. So it can easily back 100 virtual memory pages with a single real memory page.

Memory deduplication by common disk source

Posted Nov 22, 2009 21:26 UTC (Sun) by kabloom (guest, #59417) [Link]

There's another situation where this is useful: interpreted languages. Supposing you run a rails app on your server. To take advantage of concurrency you fork the rails app several times. All of the Rails code (parsed already, and byte-compiled if you're on Ruby 1.9) is on Copy-on-write pages. (Ruby Enterprise Edition is intended to keep the garbage collector from writing to those pages.) Now suppose you run several apps but they don't come about by forking. The Rails code (parsed or byte-compiled) can't be traced back to a single disk block (because it's been parsed) but it's probably got the same page layout despite being loaded in different interpreters. Here, KSM should help.