Native Linux KVM tool v2

Posted Jun 16, 2011 1:30 UTC (Thu) by starlet (subscriber, #48949)
Parent article: Native Linux KVM tool v2

It seems that the comparison is done between Native Linux KVM and Qemu,
not between Native Linux KVM and Qemu-KVM. Right ?

Will the result be changed if the competitor is Qemu-KVM (w/ virtio) ?

Native Linux KVM tool v2

Posted Jun 16, 2011 5:39 UTC (Thu) by stefanha (subscriber, #55072) [Link] (3 responses)

The comparison was not between two identical virtual machines so it is not a useful comparison. See aliguori's comment above for details on configuration.

If qemu.git was used instead of qemu-kvm.git, then there are additional things that should be done to ensure the configurations are identical. QEMU needs to be configured with --enable-io-thread and invoked with -enable-kvm. Otherwise the guest will be launched in TCG (dynamic binary translation) mode instead of KVM mode. It's not clear to me whether qemu.git or qemu-kvm.git was used for the comparison yet.

Native Linux KVM tool v2

Posted Jun 16, 2011 5:41 UTC (Thu) by stefanha (subscriber, #55072) [Link]

For fairness sake I should mention that I contribute to QEMU.

Native Linux KVM tool v2

Posted Jun 16, 2011 6:30 UTC (Thu) by penberg (guest, #30234) [Link] (1 responses)

We tested kvm-qemu.git master. It turns out there are some command line options that will make Qemu perform better but we've yet to test them.

It's never going to be an apples to apples comparison but it goes to show how important _sane default values_ are. I'm willing to bet most people don't know about all the Qemu magic that makes things run faster (which is especially important for virtualized I/O).

And just to set the record straight: the only reason we did the comparison was to see how *badly* we'd compare against Qemu. Turns out we didn't do badly at all so there was no point in hiding our results - even if we have to eat our hats later on!

Disclaimer: I contribute to the tools/kvm project.

Native Linux KVM tool v2

Posted Jun 17, 2011 13:53 UTC (Fri) by aliguori (subscriber, #30636) [Link]

From Pekka:

It turns out we were benchmarking the wrong guest kernel version for
qemu-kvm which is why it performed so much worse. Here's a summary of
qemu-kvm beating tools/kvm:

https://raw.github.com/gist/1029359/9f9a714ecee64802c08a3455971e410d5029370b/gistfile1.txt

I'd ask for a brown paper bag if I wasn't so busy eating my hat at the moment.

Native Linux KVM tool v2

Posted Jun 16, 2011 6:34 UTC (Thu) by penberg (guest, #30234) [Link] (5 responses)

We tested with virtio too (we didn't realize Qemu didn't use it by default!) and while Qemu performance was little better, tools/kvm still beat it with fair margin. Unfortunately I posted the non-virtio results.

That said, don't read too much into the results. It's *one benchmark* on *one machine*. If you care about performance, you'd better benchmark things yourself. The point of publishing the results is just to show that there's some real potential in tools/kvm and that Qemu probably should improve their default configuration.

Native Linux KVM tool v2

Posted Jun 16, 2011 11:07 UTC (Thu) by stefanha (subscriber, #55072) [Link] (4 responses)

Although the default configuration doesn't provide optimal performance for your particular benchmark it is a conservative configuration that is safe and compatible with all sorts of guest operating systems. The scope of qemu-kvm is different from native kvm tool - it needs to run old and broken non-Linux guests in addition to supporting cutting edge guests.

qemu-kvm users normally use virt-install or virt-manager to create VMs that are configured pretty well by default. This benchmark used the raw qemu-kvm command-line without ensuring an equivalent VM configuration - not a fair benchmark.

Native Linux KVM tool v2

Posted Jun 16, 2011 15:09 UTC (Thu) by bronson (subscriber, #4806) [Link] (3 responses)

If it takes an hour of reading manpages and tweaking command line configs to make one version fast, and the other version is fast out of the box, then that's not a fair benchmark either.

(Not saying it does, just that your position seems oversimplified)

Native Linux KVM tool v2

Posted Jun 16, 2011 22:41 UTC (Thu) by aliguori (subscriber, #30636) [Link] (2 responses)

The mode the they are using is not the default in QEMU because it's faster than bare metal assuming an otherwise uncontended system with plenty of memory to burn. The results are inconsistent and the safety of doing this depends on how well behaved your guest is.

It's usually not what people want to use even if it makes for artificially high benchmarks.

Native Linux KVM tool v2

Posted Jun 17, 2011 11:41 UTC (Fri) by Darkmere (subscriber, #53695) [Link] (1 responses)

Frankly, even when using virt-install I'm never certain if it's the right speed or not. Mostly _io_ tends to send systems into 20+ load "wait until world peace happens" even with (supposedly?) virtio enabled.

the current state of qemu-kvm is at times, a headache to deal with.

Speeding up qemu-kvm

Posted Jun 17, 2011 22:04 UTC (Fri) by jrn (subscriber, #64214) [Link]

Based on

> Some block drivers perform badly with cache=writethrough, most notably, qcow2. If performance is more important than correctness, cache=writeback should be used with qcow2.

I switched to cache=unsafe, and life became way better. The kind of tests I do in VMs let me recreate the image if there's a sudden crash, so safety is not such a worry. Not sure if virt-install does something like that.