A look at Xen

The Xen virtual machine monitor is starting to pop up all over the place, or at least in several Linux distributions. Fedora Core 4 comes with Xen packaged as part of the release. SUSE Professional 9.3 includes Xen, there's the Xenophilia Linux distribution that is based around Xen, and Xen is in Debian unstable as well. XenSource, a company founded by Xen project developers, has also been in the news, and is getting funding from Kleiner Perkins Caufield & Byers and Sevin Rosen Funds as well as technology contributions from Intel.

This seems like a good time to take a look at Xen, see what it's capable of, and where it's going. We decided to test out Xen in Fedora Core 4, and the latest release from the Xen project in the form of a live CD to see how mature Xen is. According to the Xen Quickstart guide, Xen in FC4 is based on the Xen unstable tree, so some features will be a bit rough. Users who want to test Xen without installing FC4 can download demo CDs based on Debian from the Xen website. We also spoke with XenSource's Simon Crosby, a founder of XenSource and former professor at the University of Cambridge where Xen got its start.

Xen is a "hypervisor," or virtual machine monitor, which can execute several virtual machines on a single piece of hardware. Xen isn't unique in being able to run virtual hosts -- Linux users can run virtual machines using User-Mode Linux (UML), bochs, VMware products, SWsoft's Virtuozzo and a number of other virtualization technologies.

Xen operates a bit differently, however, than UML or VMware Workstation. Xen requires that a OS be ported to run on Xen's hypervisor, rather than attempting to emulate an x86 virtual machine completely as VMware Workstation does. The Xen approach is supposed to offer superior performance -- a performance comparision between VMware Workstation, Xen, native Linux and UML is available on the Xen website -- but it means that unmodified operating systems will not run on top of Xen. Users who want to run a virtual instance of Microsoft Windows, for example, will have to look elsewhere, at least for now. Crosby said that work is being done that will allow unmodified guest OSes to run on top of Xen, but that won't be complete until some time after 3.0 is released.

Also, Xen runs only on x86 systems with 686 processors or better, though ports to x86_64 and other processors are in progress. Crosby said that IBM is working on Power5 support, HP is working on Xen on IA64 and that he believes Sun is working on a Sparc port as well. The current Xen release will run on SMP systems, but does not include SMP support for guests. However, Crosby said that work is being done in this area, and the 3.0 roadmap calls for SMP support within guest hosts as well.

In addition to allowing a system to run multiple instances of Linux, Xen also works with NetBSD and FreeBSD, so users aren't restricted to using a Linux host for running Xen. Using the Xen live CD, we ran instances of Debian with the 2.4 and 2.6 series kernels alongside instances of FreeBSD and NetBSD.

We installed the Fedora Core 4 with the default "Workstation" set of packages. Xen's packages are not installed by default so we used Yum to grab the Xen host kernel, the Xen guest kernel and support packages. Xen in FC4 still requires a great deal of manual setup. There's no point-and-click GUI interface included to allow easy creation of Xen virtual hosts, and some users might find the steps to setting up Xen to be somewhat daunting. We followed along with the Fedora+Xen Quickstart guide to install Xen and create virtual hosts, and the Xen users' manual to get started with the basic Xen utilities.

After installing the Xen0 kernel, we disabled SELinux support and restarted the host to boot into the Xen0 kernel. SELinux needs to be disabled in order to create the guest filesystems. After rebooting, we created a 2GB file to use for the filesystem and then installed the Fedora Core 4 base system using Yum. It is also possible to export block devices directly to guest domains, so users could choose to use entire partitions for Xen guest filesystems.

After creating the filesystem, and creating a configuration file for the guest system under /etc/xen, we started up the guest host. We gave the guest 128 MB of RAM on a system with 1 GB total. We then tested the system a bit by creating a network interface, installing Apache with Yum and so on. The guest and host performance seemed fine, even when we started up a second guest with the same configuration on the same machine.

Xen also includes a web-based control interface. This interface didn't work in FC4, but worked just fine with the Xen live CD. After firing up "xensv" we were able to connect to the localhost on port 8080 and perform most of the functions available via the command line using the web-based interface.

The control interface for Xen is adequate, but certainly won't be winning any awards for ease of use. Crosby acknowledged that "you have to be something of a guru to use it", but noted that Xen's is very polished in the area of stability. Indeed, we didn't run into any stability issues with Xen while testing, and it looks like it's already suitable for utility computing. Crosby noted that XenSource is running its website and other services within Xen hosts.

Another interesting feature in Xen is the ability to move Xen instances from one physical machine to another. Crosby said that it's possible to move a Xen virtual machine "so that the guest is only non-responsive to the outside world for tens of miliseconds".

Xen 3.0 is scheduled for sometime in the July time frame according to the Xen roadmap. Crosby said that 3.0 will fork "in a few weeks time", and that the Xen team was waiting on a few features from the community before forking. When 3.0 forks in July, Crosby said that the Xen team would be working with the community, partners and distributions to hammer out the bugs.

We also talked to Crosby about the direction of XenSource, and whether its future offerings would be released as open source. Crosby said that the company planned to ship some proprietary tools for use with Xen, though Xen would continue to be open source. He also said that XenSource is interested in a world where the hypervisor is "ubiquitous" and provides an ecosystem with "a whole load of opportunities for vendors to compete in, creating a big pie... and we aim to have a fair slice of that pie".

While Xen is still a little rough around the edges, it's well worth a look for users who want a free software solution for virtualization. Xen's performance seems very good, and it looks like a good solution for Linux testing and perhaps web hosting and so forth. Given the interest from investors, Intel, SUSE, Red Hat and others, it seems likely that Xen will continue to improve at a rapid pace.

Index entries for this article
GuestArticles	Brockmeier, Joe

---

to post comments

A look at Xen

Posted Jun 23, 2005 2:07 UTC (Thu) by yodermk (subscriber, #3803) [Link] (5 responses)

>>> Crosby said that it's possible to move a Xen virtual machine "so that the guest is only non-responsive to the outside world for tens of miliseconds."

That sounds great, but I'm not sure I believe it. Wouldn't it take at least a few seconds to transfer an image to another box over 100 megabit ethernet?

Also, I read very briefly in a comment somewhere else that next-gen chips from Intel and AMD would allow for full virtualization without a modified guest, just as mainframes have done for decades. Does anyone know more about this and when they will come out? Will Xen support them right off the bat?

A look at Xen

Posted Jun 23, 2005 3:26 UTC (Thu) by dcreemer (guest, #5103) [Link] (2 responses)

Xen guest VM migration between hosts completes very quickly. The actual transfer takes longer, but the VM stays running on the source machine as long as possible. It only (usually) takes less than a second to stop the VM on the source host, transfer any remaining differences, then bring up the VM on the target machine.

A look at Xen

Posted Jun 23, 2005 18:04 UTC (Thu) by patha (subscriber, #6986) [Link] (1 responses)

> It only (usually) takes less than a second to stop the VM on the source host, transfer any remaining differences, [...]

Does this also holds for a machine with 64GB RAM running GUPS (http://www.dgate.org/~brg/files/dis/gups/) all over the memory? ;)

Seriously, were can I read about how the step "transfer any remaining differences" is implemented?

A look at Xen

Posted Jun 30, 2005 14:32 UTC (Thu) by MarkWilliamson (guest, #30166) [Link]

For details of live migration see:
http://www.cl.cam.ac.uk/netos/papers/2005-migration-nsdi-...

Other papers under:
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/architectu...

If you run software that dirties large amounts of memory then the precopy
approach doesn't work very well: in this case, the live migration code
will figure it out and cuts its losses by doing "stop and copy" -
incurring a longer stoppage but more efficient use of network bandwidth.

For real world server workloads it works well: Running Quake 3 servers
incurred a downtime of about 60ms (I think), whilst an Apache webserver
running SpecWeb incurred a downtime of 300ms and didn't drop any clients.

A look at Xen

Posted Jun 24, 2005 22:24 UTC (Fri) by erich (guest, #7127) [Link]

Hi,
according to talk recently at BaLUG, they copy and note which pages have become "dirty" in the meantime, after a few cycles they then stop the vm and migrate the rest over, start it on the new one.
According to them, they've done that with a Quake3 server without the users noticing much. That sounds really impressive.
Apparently "downtimes" of ~200ms is typical when you have a 100Mbit link between the two hosting machines.

A look at Xen

Posted Jun 26, 2005 8:40 UTC (Sun) by csamuel (✭ supporter ✭, #2624) [Link]

Both Intel and AMD have made public declarations of support for Xen with respect to their virtualisation technology.

It's this which will make it possible to run unmodified OS's under Xen in the future (fingers crossed)..

Chris

A look at Xen

Posted Jun 23, 2005 15:32 UTC (Thu) by maceto (guest, #16498) [Link]

wow Redhat needs yast, that`s way! easier on suse to setup, it`s like vmware vs dos...

A look at Xen

Posted Jun 24, 2005 17:21 UTC (Fri) by tsinclai (guest, #11399) [Link] (2 responses)

I've been following this project for quite some time and am looking forward to trying it out.

As a technical instructor, this would seem to make setting up a virtual network in a classroom
much easier. Our Windows classes use Virtual PC for that purpose right now, but I prefer a
Free (as well as free) solution.

Am I right in thinking that Xen takes a different tack than VMWare?

A look at Xen

Posted Jun 24, 2005 23:34 UTC (Fri) by erich (guest, #7127) [Link] (1 responses)

Do you know the basics of OS programming? About the ring model?
They are mentioned in http://en.wikipedia.org/wiki/X86

Xen runs in ring-0, requiring the OS to run in the lower rings.
Usually linux itself is in ring 0; userspace apps run in ring 2 or 3.
Basically, inner rings have more privileges, and are protected from outer rings.
Xen manages the memory on a high level, and linux instead of just handling memory itself, requests memory from xen via special calls. Basically thats the same way, the linux kernel is protected from userspace applications.

If Linux would try to do it directly, it would cause a segmentation fault; but having this protection, it cannot trash xen or the other vms.

VMWare probably detects these calls in the binary and replaces that code. VMware is MUCH closer to emulating a separate PC; Xen just takes away some privileges from the OS in order to separate them, and requires the OS to "politely ask" for the resources. Thats why you need a different kernel.
But since everything accessing the hardware is handled by the linux kernel, all changes needed for Xen are within the kernel.

A look at Xen

Posted Jul 1, 2005 14:55 UTC (Fri) by tsinclai (guest, #11399) [Link]

I *am* familiar with OS programming and the ring model, but I've never used VMWare and I wasn't
clear on how the Xen memory model worked.

However, you gave a good, clear explanation and I appreciate that.

linux-vserver.org?

Posted Jun 30, 2005 8:34 UTC (Thu) by gvy (guest, #11981) [Link]

For "utility computing" everyone already uses VServer ;-) (which "only" allows for contexts within the same Linux kernel but that suffices for a wide range of applications, hosting/testing/resource provisioning being the most evident)

It's in a different weight but still the same category of virtualization software, and I wonder why mention UML (which is basically dead to me) and Virtuozzo (which is proprietary) and forget of VServer.

PS: shameless plug: yes, ALT Linux has the support out-of-box with 2.4 release. :-)

no nptl though.

Posted Jun 30, 2005 19:15 UTC (Thu) by astrophoenix (guest, #13528) [Link] (1 responses)

the lack of nptl support was a killer for me. I wanted to run a virtual redhat advanced server 3 using gentoo as a host system. but the whole reason I wanted AS3 running virtually was so I could test production multi-threaded code using redhat's 2.4 kernel with back-ported nptl. in production, the code would be using nptl but on my virtual machine, nptl would have to be disabled to get xen to work.

no nptl though.

Posted Aug 3, 2005 0:42 UTC (Wed) by rimuhosting (guest, #31061) [Link]

Xen runs NPTL code. There is a performance penalty (not sure how significant). There are glibc patches out to reduce that performance penalty.

--
Peter
RimuHosting - Xen VPS Hosting