KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 6, 2007 15:23 UTC (Tue) by ajross (guest, #4563)
Parent article: KQEMU 1.3.0pre10 released - under the GPL

Can anyone knowledgable comment on how this differs from the kvm stuff? The kvm environment is already a kernel-side virtualization framework that uses QEMU as its userspace hardware implementation.

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 6, 2007 15:29 UTC (Tue) by glikely (subscriber, #39601) [Link]

IIRC, kvm requires hardware virtualization support from the CPU. kqemu does not.

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 6, 2007 15:32 UTC (Tue) by jdahlin (subscriber, #14990) [Link] (14 responses)

kvm depends on new hardware support (Intel VT/AMD-V), which has only been available a year or two.
kqemu is similar to vmware, it uses segmentation tricks to execute user mode code natively and translates the rest on the fly [which requires kernel support, thus requiring a kernel module].

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 6, 2007 16:56 UTC (Tue) by nix (subscriber, #2304) [Link] (12 responses)

Nope, it's smarter than that :) only code running with interrupts off needs complete interpreting. In user mode, it runs at near-native speed; in kernel mode, all segment loads must be intercepted so it's slower.

See the rather good technical docs. (Compare this to the way many companies in particular free code: it's the opposite of the usual 'tossed over the wall without documentation' rubbish.)

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 6, 2007 18:59 UTC (Tue) by mikov (guest, #33179) [Link] (11 responses)

To me the really impressive part of QEMU is not the virtualization using the kernel module, but rather the JIT-based emulation (which AFAIK has always been GPL).

When not using the kernel module, QEMU dynamically recompiles all guest instructions to the host instruction set, which makes it much faster than traditional emuilators like Bochs. It is actually possible to run Windows in QEMU in such emulated mode, and it is completely usable. Additionally, in this mode QEMU can emulate x86 with good speed running on any host (e.g. PowerPC). On top of that it is a 100% user node app, with no privileges (doesn't even have to run as root) so it is completely safe and secure.

AFAIK, QEMU uses GCC to generate binary code for the emulated instructions and then simply chains together these binary blobs. This allows it to be (almost) trivially ported to any architecture supported by GCC.

I think there is an opportunity for somebody to improve QEMU by replacing the code generation engine with a hand-tuned one (sacrificing the easy portability to any host). Perhaps LLVM could be used. I think this can bring the emulation speed from its current 10x slowdown to about 3-5x (without any kernel code!). This is a very exciting area.

QEMU slowdown and portability

Posted Feb 6, 2007 19:37 UTC (Tue) by jreiser (subscriber, #11027) [Link] (4 responses)

It is actually possible to run Windows in QEMU ... and it is completely usable.

QEMU and KQEMU are impressively good, but not that good. Installing TurboTax onto WindowsXP Pro running under Win4Lin Pro 3.0-0.7 [uses KQEMU] took around two hours, versus just a few minutes under native WindowsXP Pro running on the same hardware: a Pentium4 with 1GB RAM and "nothing" else running. Also, CPU utilization for (K)QEMU usually exceeds 90% even for simple things such as actually running TurboTax. WinXP graphics such as progress bars frequently do not get updated at all under Win4Lin. This shows that there is very little CPU power available for background tasks. TurboTax downloading updates thinks that something has gone wrong because the slowdown is excessive.

QEMU uses GCC ... (almost) trivially ported to any architecture supported by GCC

Except that QEMU's strategy requires a GCC version less than 4.0.

QEMU slowdown and portability

Posted Feb 6, 2007 23:54 UTC (Tue) by Richard_J_Neill (subscriber, #23093) [Link]

I think you've been unlucky. At any rate, I find Windows XP fairly usable(*) when running under Qemu/KQemu on a fast Pentium4. The slowdown is about 2-fold. This could be a Win4Lin-ism; alternatively, it could be a known bug (sorry, can't recall the reference) that is fixed by installing WinXP SP2.

(*)Disclaimer: I am not implying that Windows is ever truely "usable"!

QEMU slowdown and portability

Posted Feb 7, 2007 0:19 UTC (Wed) by mikov (guest, #33179) [Link]

I am not closely familiar Win4Lin, but I am pretty sure that you cannot draw conclusions about QEMU from Win4Lin. They are completely different products, so I don't really see the relevance. AFAIK, Win4Lin does not use QEMU's user mode component, which is quite important.

I suggest you test with QEMU and install Win2K instead of WinXP before you form an opinion. I have never experienced such terrible performance as you are describing even without the accelerator. When running with full kernel virtualization (which was added recently,AFAIK), I found the performance comparable to VMWare. Plus, it certainly did not take me many hours to install Windows itself, and it is much larger than TurboTax.

I fail to see your point about GCC 4: this is not a fundamental technological limitation. Plus, there are no significant host architectures supported only by GCC4 and not by GCC3 (are there any at all?).

BTW, I am not affiliated with QEMU in any way, except by being a happy user. It has literally enabled me to do things that I didn't think possible, like running Windows-only applications as software appliances on a headless Linux box, which I find to be incredibly cool.

I will grant you that QEMU is not perfect though - for example saving and loading the virtual machine state doesn't work well yet, as of version 0.9.

QEMU slowdown and portability

Posted Feb 7, 2007 1:18 UTC (Wed) by mgalgoci (guest, #24168) [Link] (1 responses)

I believe you are mistaken about Win4Lin. Win4Lin is a linux port of ancient SCO software called "Merge". It even goes so far as to replace windows DLLs with special versions that attach to hooks inside linux, so it's not really virtualization as much as it is running two OS at the same time.

http://www.sco.com/products/merge/

Of course that was last time I looked, which was like three years ago. Maybe Win4Lin swapped out their core technology for something more sane.

QEMU slowdown and portability

Posted Feb 7, 2007 5:59 UTC (Wed) by rloomans (subscriber, #759) [Link]

That, indeed, was the previous version. It only supported up to Win98/Me. The current version
appears to be based on QEMU and supports Win2k and up.

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 7, 2007 22:08 UTC (Wed) by danshearer (guest, #18686) [Link] (5 responses)

> AFAIK, QEMU uses GCC to generate binary code for the emulated instructions > and then simply chains together these binary blobs. This allows it to be
> (almost) trivially ported to any architecture supported by GCC.

That was the idea, but it is quite fragile and tied to specific GCC versions.

> I think there is an opportunity for somebody to improve QEMU by replacing
> the code generation engine with a hand-tuned one (sacrificing the easy
> portability to any host). Perhaps LLVM could be used. I think this can
> bring the emulation speed from its current 10x slowdown to about 3-5x
> (without any kernel code!). This is a very exciting area.

See https://nowt.dyndns.org/ where Paul Brook has done exactly this, and I think Fabrice intends to merge Paul's code.

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 8, 2007 0:16 UTC (Thu) by mikov (guest, #33179) [Link] (2 responses)

See https://nowt.dyndns.org/ where Paul Brook has done exactly this, and I think Fabrice intends to merge Paul's code.

Very interesting. Are there any details available on the implementation of the generator ? Intermediate representation, what optimizations it performs, etc ? Or any performance benchmarks ?

I couldn't find anything relevant with Google. All I was able to find was one sentence claiming 30% improvement, which is somewhat disappointing.

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 8, 2007 11:09 UTC (Thu) by danshearer (guest, #18686) [Link] (1 responses)

> Very interesting. Are there any details available on the implementation of
> the generator ?

Not that I know of, apart from the code.

If you have the skills, why not document what you see as you read the code? That would be helpful to people like me. I am playing with Paul's work because it addresses a problem I care about (portability, preventing me from creating internal test suites for QEMU) but so long as it works my interests lie elsewhere inside QEMU. But if there was a guided tour I'd have a look for sure.

Dan

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 8, 2007 20:12 UTC (Thu) by mikov (guest, #33179) [Link]

I think the original author is in the best position to write something like that. It shouldn't take long to describe the basic idea of what he has done - just an outline. Anyway, I think writing a guide is not so much an issue of skill, but of time ... :-(

HP's Dynamo, Intel's IA32-EL, Transmeta and others have shown that there is definite potential in re-optimizing binary code. It is possible that an advanced optimizing code generator for QEMU could even outperform native code! Of course it would have to go significantly beyond the current QEMU infrastructure and rely on dynamic recompilation, trace optimization, etc. Some more info on this subject is available here: http://www.cag.csail.mit.edu/rio/#pubs/

In any case I will definitely take a closer look at Paul Brook's work. BTW, how does it address portability ?

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 8, 2007 11:12 UTC (Thu) by jfj (guest, #37917) [Link] (1 responses)

> That was the idea, but it is quite fragile and tied to specific GCC versions.

Some people bring this up as if it's a bad thing.

Why exactly it is do difficult to download gcc-core-4.0 and install it to /opt/gcc4.0 and use it for qemu?

The installation of gcc is very clean and easy, with modern computers it won't take more than 10 minutes, there are lots of mirrors of GNU software and you will always be able to find gcc-4.0.

And we are talking about developers. Distributions can arrange binaries that won't need gcc 4.

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 8, 2007 11:29 UTC (Thu) by danshearer (guest, #18686) [Link]

> > That was the idea, but it is quite fragile and tied to specific GCC
> > versions.
> Some people bring this up as if it's a bad thing.
:
> And we are talking about developers. Distributions can arrange binaries
> that won't need gcc 4.

1. gcc 4 doesn't neccesarily work at all on a given architecture, and this will always be the case in some circumstances as versions shift under our feet. I was in this situation with QEMU on a pure AMD64 setup last year.

2. Most QEMU use is not production. For experimenting and testing people often want to compile QEMU, even if just to incorporate a patch someone posted. Such QEMU users usually have some other primary goal -- testing networking maybe, or some funny OS -- and building GCC isn't necessarily something they will think of or even be able to easily do.

KQEMU 1.3.0pre10 released - under the GPL

Posted Feb 6, 2007 17:00 UTC (Tue) by mces (subscriber, #27668) [Link]

Looking at the "Future Developments" section in the "Documentation" page, this may sooner or later change:

"Support for hardware virtualization. The performance gains, if any, will be small but there would be no limitations regarding what the guest OS can do."