Virgil 3D: A virtual GPU

Providing 3D acceleration for virtual machines, by way of a virtual GPU, is challenging. The existing solutions are proprietary, so Dave Airlie has undertaken a "research project" to create an open-source virtual GPU for QEMU. He described his Virgil 3D project in a session at LinuxCon North America in Chicago on August 22.

The idea behind Virgil 3D is to allow QEMU Linux guests on a Linux host to have access to the GPU and the 3D acceleration it provides. The virtual GPU will use Mesa drivers to begin with, though he plans to eventually have DirectX and Direct3D drivers as well.

Background

Airlie said that VMware has the "best solution currently available" for a virtual GPU. It is based on DirectX 9, though, so it has some limited capabilities and is all closed source, except for the part that runs in a Linux guest. VirtualBox also has a solution, but it is based on OpenGL, which is the "worst choice" because it is hard to secure. OpenGL has a lot of entry points and it is too easy for user space to cause buffer overflows and other problems. There was also the vGallium driver, but that was essentially a research effort that never reached a point where it could be deployed.

Virgil 3D is based on virtio, which is a standard for virtualized I/O created by Rusty Russell. The basic concept of virtio is to have a set of queues to communicate with the driver, along with a configuration space that can look something like PCI.

On the graphics side, Mesa implements OpenGL, while its Gallium3D component makes it easier to write 3D drivers. Below Mesa is the EGL API that sits between OpenGL and the hardware. EGL was originally for embedded devices, but it is "used more and more on everything", he said, and will eventually "take over". One more piece of the puzzle is SPICE, which is a protocol for "desktop remoting"—it provides remote access to audio, video, input devices, and more.

On real graphics hardware, there is some sort of ring buffer where the kernel puts commands. The user-space graphics drivers create a stream of these commands (which manipulate textures, surfaces, shaders, etc.) to hand to the kernel. Those user-space commands are considered insecure commands. They are copied into the kernel, then validated to turn them into "secured" commands. Those commands are then placed into the ring, followed by a "fence". Since the GPU is asynchronous, the kernel can only find out that a set of commands has been executed by using the sequence number associated with a completed fence operation.

2D support

So the first challenge for Virgil 3D, long before getting to the "3D bits that I was interested in" was to get a basic virtio GPU driver working, Airlie said. It supported multiple monitors, hardware cursors, and a basic set of 2D commands. The driver can provide memory-mapped I/O, PCI, or VGA devices, and should work on any architecture that supports virtio. The "basic component is a command ring", though the original design had two rings, with one for fences. A QEMU developer rewrote that to use a single command ring—the fence ring can be added back in later if its lack becomes a problem.

The basic object in the driver is a resource, he said, each of which has a unique ID. For 2D, each resource has a width, height, and bit depth associated with it. For 3D, there are a "bunch more" attributes; it is "a lot more complicated". The guest just refers to a resource by its ID, while the host side allocates the memory and creates the resource. The driver takes pages of guest memory to attach to the resource as its backing store.

The resources are mapped into OpenGL objects. For 2D, the driver simply copies data from the guest pages attached to the resource into the OpenGL object. Transferring 3D data to the host is more complicated, Airlie said. For 2D, the host is just passing data through, but for 3D there may be a need to be able to read back data from the hardware to the guest.

That simple driver provides "a resource and a way to put data into it", he said; the next step is to turn a resource into a "front object", that way "people can see it". That is done using set_scanout(), which is like setting a mode in a regular graphics device. There is also a need to do a flush_resource() to cause the OpenGL drawing to happen. All of that makes for the "basics of the simple GPU", most of which is in the process of being merged into QEMU.

3D

For 3D rendering, an insecure command stream is generated from Mesa in the guest and submitted to the driver using the SUBMIT_3D command. 3D contexts are created for each guest user-space program and resources are attached to those contexts. That means that a guest program cannot access a resource it did not create because it will not be attached to its context.

The 3D command stream has several different types of operations. There are operations on object state (create, bind, and destroy), various types of shaders (including vertex, fragment, and geometry), handling of surface and texture objects, and non-state operations (scissors and viewports). Beyond that, there are also rendering operations like draw, clear, and blit. There is a set of queries that round out the 3D command set. All of those need to be handled by Virgil 3D.

There are five separate components to making this all work on Linux, Airlie said. There are two pieces that live on the host: the virtio GPU driver in QEMU and a renderer library. The latter is the larger piece. The guest's kernel must have a DRM/KMS (direct rendering manager/kernel mode setting) driver that supports the virtio GPU. In user space, it needs a Gallium3D driver for Mesa to talk to the kernel and may require a device-dependent X (DDX) driver for the X server.

When Airlie started the Virgil 3D project, QEMU only supported Simple DirectMedia Layer (SDL) version 1.2 for its graphics, which didn't support multiple monitors. That meant he needed to port QEMU to use SDL 2.0. That work has been merged into the QEMU mainline. The virtio GPU code "is in someone's tree" and headed for a merge into QEMU.

The renderer library is what "contains the fun", he said. It takes a command stream and turns it into OpenGL API calls. It also converts shaders into GLSL (the shader language used by OpenGL).

For deployment, SPICE is used to talk to a QEMU virtual machine running in a libvirt sandbox. That means that only the user's remote viewer is directly talking to the graphics hardware, which is not what he wants. He would like to have QEMU also talk to the hardware.

In order to do that, his code uses the kernel dma_buf buffer-sharing feature and passes file descriptors back and forth between QEMU and the remote viewer. The SPICE socket uses TCP, however, and descriptors can only be passed over Unix-domain sockets. For now, he has hacked around some Unix-socket permission problems (by making the socket world-writable for experimental purposes only) to get that working.

There are some OpenGL issues that need to be resolved, including how to handle GL 3.1 features that are little-used and painful to implement. If something in the guest uses GL 3.1, Virgil 3D would need to handle it, but GL 3.1 is not implemented for Linux (nor by Apple, for that matter). Many of the features from 3.1 can be implemented using other OpenGL features and commands, so emulating them might be a possibility.

The "to do" list for Virgil 3D is short, Airlie said, but all of the items on it are "huge". They include OpenGL capability handling, using the TGSI (Tungsten Graphics Shader Infrastructure) text representation of shaders rather than the binary representation used today (which will be "messy"), handling the GL 3.1 issue, adding threads to the GL access part of QEMU, and, last but not least, upstreaming all of the code.

Airlie finished his presentation with a demo of the 3D game Xonotic running in a virtual machine on his laptop. It was, he said, rendering 60 frames per second, "at least". He answered an audience question by noting that using a remoting protocol across the network would likely have to use some kind of video encoding on the host and shipping the results to the viewer, rather than shipping the 3D objects and commands. He would love to be wrong about that, though, and plans to try it as one of his next projects.

[I would like to thank the Linux Foundation for travel assistance to Chicago for LinuxCon North America.]