User: Password:
Subscribe / Log in / New account

Re: Frontswap [PATCH 0/4] (was Transcendent Memory): overview

From:  Avi Kivity <>
To:  Dan Magenheimer <>
Subject:  Re: Frontswap [PATCH 0/4] (was Transcendent Memory): overview
Date:  Sun, 02 May 2010 18:35:47 +0300
Cc:  Jeremy Fitzhardinge <>, Dave Hansen <>, Pavel Machek <>,,,,,,,,,,,
Archive-link:  Article, Thread

On 05/01/2010 08:10 PM, Dan Magenheimer wrote:
>> Eventually you'll have to swap frontswap pages, or kill uncooperative
>> guests.  At which point all of the simplicity is gone.
> OK, now I think I see the crux of the disagreement.

Alas, I think we're pretty far from that.

> NO!  Frontswap on Xen+tmem never *never* _never_ NEVER results
> in host swapping.

That's a bug.  You're giving the guest memory without the means to take it
back.  The result is that you have to _undercommit_ your memory resources.

Consider a machine running a guest, with most of its memory free.  You give
the memory via frontswap to the guest.  The guest happily swaps to
frontswap, and uses the freed memory for something unswappable, like
mlock()ed memory or hugetlbfs.

Now the second node dies and you need memory to migrate your guests into.
But you can't, and the hypervisor is at the mercy of the guest for getting
its memory back; and the guest can't do it (at least not quickly).

> Host swapping is evil.  Host swapping is
> the root of most of the bad reputation that memory overcommit
> has gotten from VMware customers.  Host swapping can't be
> avoided with some memory overcommit technologies (such as page
> sharing), but frontswap on Xen+tmem CAN and DOES avoid it.

In this case the guest expects that swapped out memory will be slow (since
was freed via the swap API; it will be slow if the host happened to run out
of tmem).  So by storing this memory on disk you aren't reducing
performance beyond what you promised to the guest.

Swapping guest RAM will indeed cause a performance hit, but sometimes you
need to do it.

> So, to summarize:
> 1) You agreed that a synchronous interface for frontswap makes
>     sense for swap-to-in-kernel-compressed-RAM because it is
>     truly swapping to RAM.

Because the interface is internal to the kernel.

> 2) You have pointed out that an asynchronous interface for
>     frontswap makes more sense for KVM than a synchronous
>     interface, because KVM does host swapping.

kvm's host swapping is unrelated.  Host swapping swaps guest-owned memory;
that's not what we want here.  We want to cache guest swap in RAM, and
that's easily done by having a virtual disk cached in main memory.  We're
simply presenting a disk with a large write-back cache to the guest.

You could just as easily cache a block device in free RAM with Xen.  Have a
tmem domain behave as the backend for your swap device.  Use ballooning to
force tmem to disk, or to allow more cache when memory is free.

Voila: you no longer depend on guests (you depend on the tmem domain, but
that's part of the host code), you don't need guest modifications, so it
works across a wider range of guests.

>    Then you said
>     if you have an asynchronous interface anyway, the existing
>     swap code works just fine with no changes so frontswap
>     is not needed at all... for KVM.

For any hypervisor which implements virtual disks with write-back cache in
host memory.

> 3) You have suggested that if Xen were more like KVM and required
>     host-swapping, then Xen doesn't need frontswap either.

Host swapping is not a requirement.

> BUT frontswap on Xen+tmem always truly swaps to RAM.

AND that's a problem because it puts the hypervisor at the mercy of the

> So there are two users of frontswap for which the synchronous
> interface makes sense.

I believe there is only one.  See below.

> I believe there may be more in the
> future and you disagree but, as Jeremy said, "a general Linux
> principle is not to overdesign interfaces for hypothetical users,
> only for real needs."  We have demonstrated there is a need
> with at least two users so the debate is only whether the
> number of users is two or more than two.
> Frontswap is a very non-invasive patch and is very cleanly
> layered so that if it is not in the presence of either of
> the intended "users", it can be turned off in many different
> ways with zero overhead (CONFIG'ed off) or extremely small overhead
> (frontswap_ops is never set; or frontswap_ops is set but the
> underlying hypervisor doesn't support it so frontswap_poolid
> never gets set).

The problem is not the complexity of the patch itself.  It's the fact that
it introduces a new external API.  If we refactor swapping, that stands in
the way.

How much, that's up to the mm maintainers to say.  If it isn't a problem
for them, fine (but I still think swap-to-RAM-without-hypervisor-decommit
is a bad idea).

> So... KVM doesn't need it and won't use it.  Do you, Avi, have
> any other objections as to why the frontswap patch shouldn't be
> accepted as is for the users that DO need it and WILL use it?

Even ignoring the problems above (which are really hypervisor problems and
the guest, which is what we're discussing here, shouldn't care if the
hypervisor paints itself into an oom), a synchronous single-page DMA API is
a bad idea.  Look at the Xen network and block code, while they eventually
do a memory copy for every page they see, they try to batch multiple pages
into an exit, and make the response asynchronous.

As an example, with a batched API you could save/restore the fpu context
and use sse for copying the memory, while with a single page API you'd
probably lost out.  Synchronous DMA, even for emulated hardware, is out of
place in 2010.

error compiling committee.c: too many arguments to function

To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to  For more info on Linux MM,
see: .
Don't email: <a href=mailto:""> </a>

(Log in to post comments)

Copyright © 2010, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds