LWN.net Weekly Edition for September 11, 2014

Virgil 3D: A virtual GPU

Providing 3D acceleration for virtual machines, by way of a virtual GPU, is challenging. The existing solutions are proprietary, so Dave Airlie has undertaken a "research project" to create an open-source virtual GPU for QEMU. He described his Virgil 3D project in a session at LinuxCon North America in Chicago on August 22.

The idea behind Virgil 3D is to allow QEMU Linux guests on a Linux host to have access to the GPU and the 3D acceleration it provides. The virtual GPU will use Mesa drivers to begin with, though he plans to eventually have DirectX and Direct3D drivers as well.

Background

Airlie said that VMware has the "best solution currently available" for a virtual GPU. It is based on DirectX 9, though, so it has some limited capabilities and is all closed source, except for the part that runs in a Linux guest. VirtualBox also has a solution, but it is based on OpenGL, which is the "worst choice" because it is hard to secure. OpenGL has a lot of entry points and it is too easy for user space to cause buffer overflows and other problems. There was also the vGallium driver, but that was essentially a research effort that never reached a point where it could be deployed.

Virgil 3D is based on virtio, which is a standard for virtualized I/O created by Rusty Russell. The basic concept of virtio is to have a set of queues to communicate with the driver, along with a configuration space that can look something like PCI.

On the graphics side, Mesa implements OpenGL, while its Gallium3D component makes it easier to write 3D drivers. Below Mesa is the EGL API that sits between OpenGL and the hardware. EGL was originally for embedded devices, but it is "used more and more on everything", he said, and will eventually "take over". One more piece of the puzzle is SPICE, which is a protocol for "desktop remoting"—it provides remote access to audio, video, input devices, and more.

On real graphics hardware, there is some sort of ring buffer where the kernel puts commands. The user-space graphics drivers create a stream of these commands (which manipulate textures, surfaces, shaders, etc.) to hand to the kernel. Those user-space commands are considered insecure commands. They are copied into the kernel, then validated to turn them into "secured" commands. Those commands are then placed into the ring, followed by a "fence". Since the GPU is asynchronous, the kernel can only find out that a set of commands has been executed by using the sequence number associated with a completed fence operation.

2D support

So the first challenge for Virgil 3D, long before getting to the "3D bits that I was interested in" was to get a basic virtio GPU driver working, Airlie said. It supported multiple monitors, hardware cursors, and a basic set of 2D commands. The driver can provide memory-mapped I/O, PCI, or VGA devices, and should work on any architecture that supports virtio. The "basic component is a command ring", though the original design had two rings, with one for fences. A QEMU developer rewrote that to use a single command ring—the fence ring can be added back in later if its lack becomes a problem.

The basic object in the driver is a resource, he said, each of which has a unique ID. For 2D, each resource has a width, height, and bit depth associated with it. For 3D, there are a "bunch more" attributes; it is "a lot more complicated". The guest just refers to a resource by its ID, while the host side allocates the memory and creates the resource. The driver takes pages of guest memory to attach to the resource as its backing store.

The resources are mapped into OpenGL objects. For 2D, the driver simply copies data from the guest pages attached to the resource into the OpenGL object. Transferring 3D data to the host is more complicated, Airlie said. For 2D, the host is just passing data through, but for 3D there may be a need to be able to read back data from the hardware to the guest.

That simple driver provides "a resource and a way to put data into it", he said; the next step is to turn a resource into a "front object", that way "people can see it". That is done using set_scanout(), which is like setting a mode in a regular graphics device. There is also a need to do a flush_resource() to cause the OpenGL drawing to happen. All of that makes for the "basics of the simple GPU", most of which is in the process of being merged into QEMU.

3D

For 3D rendering, an insecure command stream is generated from Mesa in the guest and submitted to the driver using the SUBMIT_3D command. 3D contexts are created for each guest user-space program and resources are attached to those contexts. That means that a guest program cannot access a resource it did not create because it will not be attached to its context.

The 3D command stream has several different types of operations. There are operations on object state (create, bind, and destroy), various types of shaders (including vertex, fragment, and geometry), handling of surface and texture objects, and non-state operations (scissors and viewports). Beyond that, there are also rendering operations like draw, clear, and blit. There is a set of queries that round out the 3D command set. All of those need to be handled by Virgil 3D.

There are five separate components to making this all work on Linux, Airlie said. There are two pieces that live on the host: the virtio GPU driver in QEMU and a renderer library. The latter is the larger piece. The guest's kernel must have a DRM/KMS (direct rendering manager/kernel mode setting) driver that supports the virtio GPU. In user space, it needs a Gallium3D driver for Mesa to talk to the kernel and may require a device-dependent X (DDX) driver for the X server.

When Airlie started the Virgil 3D project, QEMU only supported Simple DirectMedia Layer (SDL) version 1.2 for its graphics, which didn't support multiple monitors. That meant he needed to port QEMU to use SDL 2.0. That work has been merged into the QEMU mainline. The virtio GPU code "is in someone's tree" and headed for a merge into QEMU.

The renderer library is what "contains the fun", he said. It takes a command stream and turns it into OpenGL API calls. It also converts shaders into GLSL (the shader language used by OpenGL).

For deployment, SPICE is used to talk to a QEMU virtual machine running in a libvirt sandbox. That means that only the user's remote viewer is directly talking to the graphics hardware, which is not what he wants. He would like to have QEMU also talk to the hardware.

In order to do that, his code uses the kernel dma_buf buffer-sharing feature and passes file descriptors back and forth between QEMU and the remote viewer. The SPICE socket uses TCP, however, and descriptors can only be passed over Unix-domain sockets. For now, he has hacked around some Unix-socket permission problems (by making the socket world-writable for experimental purposes only) to get that working.

There are some OpenGL issues that need to be resolved, including how to handle GL 3.1 features that are little-used and painful to implement. If something in the guest uses GL 3.1, Virgil 3D would need to handle it, but GL 3.1 is not implemented for Linux (nor by Apple, for that matter). Many of the features from 3.1 can be implemented using other OpenGL features and commands, so emulating them might be a possibility.

The "to do" list for Virgil 3D is short, Airlie said, but all of the items on it are "huge". They include OpenGL capability handling, using the TGSI (Tungsten Graphics Shader Infrastructure) text representation of shaders rather than the binary representation used today (which will be "messy"), handling the GL 3.1 issue, adding threads to the GL access part of QEMU, and, last but not least, upstreaming all of the code.

Airlie finished his presentation with a demo of the 3D game Xonotic running in a virtual machine on his laptop. It was, he said, rendering 60 frames per second, "at least". He answered an audience question by noting that using a remoting protocol across the network would likely have to use some kind of video encoding on the host and shipping the results to the viewer, rather than shipping the 3D objects and commands. He would love to be wrong about that, though, and plans to try it as one of his next projects.

[I would like to thank the Linux Foundation for travel assistance to Chicago for LinuxCon North America.]

Comments (11 posted)

What's new in calibre 2.0?

When it comes to ebook-management systems, one project continues to stand out: calibre, an open-source, cross-platform solution. The project recently hit its 2.0 milestone. As is usual, weekly updates, in the form of versions 2.1 and 2.2, have also been released. The 2.x series has some interesting new features; in particular, a comprehensive built-in ebook editor. While 2.x brings some new things to the table, the outward changes are not huge since our calibre 1.0 review a year ago.

Longtime users hoping that the major changes implied by the move to a 2.x series included a user-interface overhaul will be disappointed; the program looks much the same as it has since its 0.x days. A plethora of options is listed in a horizontal bar at the top, some of which may be confusing. For example, "Add books" will open a prompt to select a local directory to import ebooks into the calibre library. "Get books", however, opens up a search window to find and download ebooks from a variety of free and commercial online services. "Convert books" allows changing the file format of one or more books, while "Edit book" opens up the new built-in ebook editor.

Somewhat irritatingly, the "Preferences" option, which allows configuration of calibre itself, is not easily found; a small button in the top right corner of the interface expands the options bar to reveal "Preferences". One gets the sense that a simplifying overhaul is overdue for the nearly eight-year-old project.

The biggest user-facing change in the 2.x series is calibre's new ebook editor. A video demo [WebM] by lead developer Kovid Goyal outlines many of its features. The editor splits up EPUBs and AZW3s (Amazon's proprietary ebook file format) into separate files, including HTML for the text itself, CSS, images, and fonts. The ebook editor presents a view with three panels: one on the left which lists the ebook's files, one in the center where source code edits can be made, and one on the right which shows a preview of how the ebook appears.

Readers, writers, and editors can make use of various built-in tools to enhance their reading/writing experience using the ebook editor, either by manipulating either individual files or the collection of files as a whole. For example, a typographically-inclined reader can replace the fonts in an ebook with others that are preferred. A writer and/or editor could add images or play around with the CSS to, for instance, improve the aesthetics of chapter headings.

Bulk actions are available to modify the ebook as a whole. A "Check Book" tool checks the entire ebook for typos and others errors; it can fix these mistakes automatically. A font manager tool can quickly replace and embed fonts throughout the ebook. The tool shows promise for picky ebook power users, but it can be difficult to see writers using the tool over comprehensive alternatives like LyX that abstract away the need for writers to fiddle with the underlying formatting.

Other major changes include automatic recognition of Android devices on OS X; gesture support on the ebook reader for Windows 8 tablets; and "emblems": icons that can be used to distinguish books in calibre's alternative cover view, which organizes books by cover images. These can be added by selecting "Look and Feel" from the "Preferences" menu, clicking on the "Emblems" tab on the side, and clicking "Show emblems next to covers". One can add emblems (locally stored images) by defining an "if this, then that" rule set, where emblems are automatically added next to books according to rules made by the user. It would be nice if calibre could add the ability to manually add emblems in the future, and came with a built-in set of icons to choose from, instead of expecting users to fish around in their filesystem for them. As it stands, the feature feels half-baked.

The project also decided to switch from Qt 4 to Qt 5, which has some interesting implications. The benefits include allowing "calibre command line tools to be run on Linux without needing a running X server. This has led to significant memory usage and performance improvements when running some command line tools, such as the standalone content server and ebook-convert."

The built-in ebook viewer also shows improvement as a result of the move to Qt 5: "Fallback fonts now work, embedded fonts whose names inside the font file do not match the names in the @font-face rule work, hyphenation is greatly improved and so on." There remain some nuisances for users: some plugins currently do not work with calibre, although most have been ported; the team and the community are continuing to port the rest.

Somewhat strangely, the project states on its Windows download page that the reliance on Qt 5 prevents calibre 2.x from running on Windows XP: Although it can be somewhat cumbersome, there are guides online to get Qt 5 working on Windows XP. A discussion on the calibre forums suggests that some users had been able to install 2.0 on Windows XP machines, but one user pointed out that from calibre 2.1 on, the program "will refuse to install or run on any computer running Windows XP."

Downloading and installing calibre for Linux remains less than pleasant. On the download page, Linux users are instructed not to use the version in their distribution, but to copy and paste a command to run as root (a wget of a Python script, then executing it). While there are instructions on how to "do an 'isolated' install that only touches files inside the installation directory and does not need to be run as root", those instructions are further down the page and may be overlooked.

The project continues its practice of phoning home to gather installation statistics:

It should be noted that calibre also reports the operating system type and version number of the program at startup time.

Development remains active, with nearly weekly updates on Fridays. The project has a Launchpad page for bug reports and triage, and an IRC channel on Freenode. A bug report I filed regarding my inability to download books from Project Gutenberg, a repository of public domain ebooks, was promptly replied to and fixed by Goyal.

After eight years, calibre remains a compelling tool for ebook readers. With its new ebook editor, the project has further appeal for picky, technical ebook readers. It will be interesting to see what calibre has in store for the future.

Comments (none posted)

The KDE Visual Design Group

Many large open-source projects organize a yearly conference to cover new technical ground, as well as provide an opportunity for contributors to mingle, put faces to the IRC nicknames they are used to working with, and build up inspiration and motivation. The KDE community organizes Akademy, which was held in Brno, Czech Republic this year, as its annual event. Not all of the talks on the program are strictly technical, though, and Andrew Lake used this opportunity to give a talk entitled "Community Design and the KDE Visual Design Group," which highlighted the efforts of a new group of KDE contributors focused specifically on design issues.

Introducing the Visual Design Group

Lake presented the newly formed KDE Visual Design Group (VDG), a virtual studio that was officially started earlier this year to serve all of the KDE community's visual design needs. He explained that the VDG tries to perform design in the open as much as possible, naming this process "community design." This openness, he said, is the "most important thing". Everyone is welcome to participate, regardless of skill level, and the community tries to maintain a positive environment.

The group was created to combat two myths about design, he said: first, that "only privileged people can participate" and second, the idea of the "lone genius" designer. Citing Steve Jobs as a negative example of the "lone genius" model, Lake said he rejects the notion that someone has to be a jerk in order to be a designer, saying it drove him "batshit crazy." Lake is a longtime KDE fan and had wanted to participate in design, he explained. But, in his view, KDE is a community, and it is difficult to talk sincerely about design without that discussion also being a community-driven exercise.

He praised the KDE project's earlier work on Oxygen, calling it "a great icon set and style in general". But, in his opinion, the problem is that if you take one brilliant designer and tell them to "go be brilliant," they will work furiously, then at some point burn out and be unable to continue. Therefore, he said, it is important to produce a sustainable community effort. This concept, which he proposed for the VDG, is even more interesting in the context of KDE's community.

There have been challenges, Lake said. One of the biggest was the fear that grouping together people who have no skills—but where everyone has a say—will lead to a design by committee scenario, where there is no unifying vision for the project. He disagrees, adding that, for him, "it must be possible to have multiple people of various skill levels produce something meaningful." It is possible for a group to create a high quality design, he feels, because more people means more ideas and more creativity. Individual brilliance is great, but a mix of ideas could be even better, "if we know how to do it."

He then reminded the audience that similar things have been said about the development of free and open-source software. The VDG has learned a lot from the success of open-source developers, he said, and is confident that it will be able to figure out a way to make design work as well. The process is not chaotic and is very similar to open-source software development: talented people usually emerge and earn influence. People come to the forums with few skills, and are actively encouraged; according to Lake, "several great designs emerged that way." Once a community forms around an idea, other people start commenting, and the designers with influence and experience can also participate, helping to develop the proposal. He added that it is really important to limit the time allotted to produce a result, even though that is not standard practice in design work.

The other challenges for the VDG have been building a sustainable community and learning how to best interact with developers—especially knowing how to approach a developer in order to make sure there is a productive exchange. Lake admitted that they have had both positive and some "not-so-great interactions." To build its community, the VDG needs to focus on producing high-quality results over the long term and on providing an encouraging learning environment.

It does this by explicitly encouraging constructive feedback, setting examples, and stopping destructive behavior, as the entire premise depends on the community being constructive. Lake feels that they have done "a reasonable job" at this, adding that 90% of the time it is enough to just ignore destructive remarks; banning people and yelling are techniques that have not been used. Another, related effort of the VDG is to provide the appropriate tools for the community, such as forums and support for the KDE Human interface guidelines.

Current status

Lake then moved on to more practical aspects of the VDG's work. He explained that it is important to focus on the low-hanging fruit, and that it is paramount to respond whenever a developer asks for help. In his words, "it's great to shoot for the moon, but in this early stage, it's important to focus on short-term successes." He defined effectiveness in design as the product of correctness and commitment; if a developer is committed to working on the implementation, the designer must be willing to "take the 50%" solution now, knowing they can always improve the design later. According to Lake, it is also important to agree on what the deliverable of the VDG is: "it's not software, it's just the design."

Good interaction with developers is another important aspect for the VDG. Lake said that the best cooperation has occurred when developers came to the VDG and asked for help. Communication in the other case, where ideas pop out and the designers approach the developers, is "trickier, and some don't go so well". Lake said they are learning that "it's not great for an enthusiastic designer to storm into a developers office". The VDG is also creating the idea of a "resident designer," the design equivalent of a project maintainer, and is working on putting it into practice. As was mentioned in a later design talk, there is a resident designer working with KDE's Telepathy project now.

The community design process the VDG has created is working well, in Lake's opinion. In short, a candidate design is developed and a target cycle length set, then the group iterates on the design: incorporating feedback into the candidate and posting updates. Some of their results so far include new cursor themes, redesigned tool icons for Krita and various Plasmoid designs (such as keyboard layouts, a system monitor, and a dictionary); work is currently progressing on the Plasma Media Center. Lake admitted that not all of these results have emerged from community design—they did have some "lone genius" designers—but he hopes to move more toward open design over time.

Lake concluded by discussing the next steps for the VDG: capturing more use cases and requirements is needed, the designers should try harder to interact with developers, and they need to think of ways to provide consistent communication of design specifications to developers. The road ahead, he said, is to build a foundation: first to work on core design patterns, then move onto applications. More specific information, he said, is available in a recent blog post about VDG. He emphasized, though, that the order of items discussed in the post represents priority levels, and not necessarily sequencing.

Design was a recurring topic at Akademy: Lake's talk was followed by a presentation by Jens Reuterberg and Thomas Pfeiffer about improving collaboration between developers and designers, suggesting that interest in the subject is growing. Large developer gatherings like Akademy provide an excellent opportunity to discuss such "softer" topics. As evidenced by the talks and by the ongoing work of the VDG, the KDE community has evidently recognized the need for a larger focus on design as a component of KDE's success.

Comments (2 posted)