Wayland in GNOME: two progress reports

The X11 replacement protocol Wayland has been in development since 2010. Compared to X11 itself, it is still a relatively new project, but the enthusiasm with which distributions and large software projects announced their intent to support Wayland makes it at least understandable that users would ask how much longer they need to wait before Wayland is made available to them. At GUADEC 2014 in Strasbourg, France, a pair of talks presented the latest status of Wayland support in various GNOME desktop components.

The desktop and GTK+

Jasper St. Pierre presented an overview of GNOME's Wayland support on July 28. St. Pierre's talk started off with an atypical question-and-answer session as he debugged some last-minute problems with his current Wayland session in GNOME's Mutter. One audience member asked when Wayland would be available by default in a Linux distribution; in reply St. Pierre joked "I'll tell you at the end of this talk when it has run without crashing the whole session." More seriously, another audience member asked whether Wayland would support multiple copy/paste buffers; St. Pierre responded that nothing explicitly forbids it, but that in some places the protocol assumes it. Adding multiple buffers would require changing the protocol in ways that the Wayland developers "really don't want to do."

Although many in attendance were already familiar with X11's security problems, St. Pierre illustrated them at the top of his talk by showing a quick-and-dirty GTK+ program he had written that would exploit X11's lack of application isolation—by, among other things, logging keystrokes and drawing into another application's window. These insecurities are at the heart of GNOME's interest in switching to Wayland, he said. His keylogger could record passwords typed even when the GNOME screen-lock was active, he noted. There have been attempts to improve X security through extensions, he said, but they were "so complex to configure that even the SELinux people said 'we're not going to try implementing that.'"

But in addition to improving security by isolating each process's connection to the display server, Wayland offers GNOME real feature benefits, he said, such as automatic scaling of content on high-DPI monitors and real synchronization of redraws. The most common demos of Wayland, he explained, show off oddities like tilting windows to an angle. That understandably means little to users, he said, but to display developers it shows instantaneously that there are important things possible in Wayland that were impossible in X. Wayland will enable smoother redrawing when dragging and resizing windows. The current tearing and lag that users see is really a shortcoming in X, he said, but users often just blame GNOME or Linux in a generic sense.

The current state of GNOME support for Wayland is a mixed affair. The demonstration apps run quite well under GNOME's Mutter window manager, he said, but support for the GTK+ toolkit comes from the Wayland backend for the lower-level GIMP Drawing Kit (GDK) library, which he called "very, very iffy." Most noticeably, he said, is that GDK's Wayland backend is missing clipboard and drag-and-drop support.

Mutter support is also incomplete, as things are still stabilizing. "I just fixed screen recording support last week," he said. The stability of the Wayland version of Mutter also depends on which GPU driver is used, he explained. The Intel drivers are the most complete; the open-source drivers for NVIDIA chips are in between, while the closed-source NVIDIA drivers do not work with Wayland at all. The reason is that NVIDIA does not offer all of the functions that the open drivers require for Wayland; the company's binary X.org drivers are tied directly into X, so only the company can implement Wayland support for them.

Outstanding questions

In response to an audience question, St. Pierre further explained that supporting a new GPU, generally speaking, is not any harder under Wayland than under X. Supporting a new GPU on Linux essentially boils down to supporting mode setting and supporting direct rendering. Wayland support hinges on the direct-rendering aspect, but most of the work required to support a new GPU is the same for Wayland as it is for X.

There is also ongoing work to figure out how best to support taking screenshots of the whole desktop. Users expect the feature, but it has major security implications. If an application can take a screenshot unattended, it can also use that capability to record username/password combinations and other sensitive data. The straightforward solution is to force screenshot capture to require a user to verify the action; that would at least allow the user to detect if an unauthorized screenshot attempt is made.

St. Pierre spent a large portion of the session slot responding to questions. One question dealt with whether Wayland will be faster or more efficient than X11. It will certainly not be faster in the first few releases, St. Pierre said, but the potential is there for a lot of optimization work. For example, he said, a lot of video content is encoded in YUV color and has to be converted to RGB color for X. But many modern graphics chips support YUV natively; skipping that color-space conversion would be a big time-saver. In addition, he said, "performance" is sort of a generic term. A user might not see a noticeable frame-rate increase in Wayland, but would benefit from other performance improvements like lower CPU and memory overhead.

Another audience member asked whether Wayland would require applications to use client-side window decorations (which GNOME 3 implements but other desktops do not). St. Pierre responded that it would not; the subject is a longstanding debate, and no position has been written into the Wayland protocol. Making both options work was mostly a social problem to solve, not an engineering one, he said. If KWin decides it needs specific changes for its server-side window decorations, St. Pierre said, it is far better for him to sit down with maintainer Martin Gräßlin and come up with a handshake protocol than to try and work around the issue in Mutter.

Two other questions raised more difficult application-compatibility concerns. One person asked how Wayland's isolation of a process's window contents would affect graphics applications like GIMP and Pitivi, which often have a color-picker tool that lets the user copy a pixel's RGB value. St. Pierre replied that Wayland would not currently allow that feature. Similarly, in response to another question, he noted that Wayland will cause a lot of trouble for supporting (and reassigning) global keybindings.

When another audience member asked "how do we sell users on this change, if the fact is that GIMP doesn't work anymore but you can drag windows without tearing?" St. Pierre joked that "I installed Wayland and all I got was new bugs" t-shirts would become popular. The truth, he said, was that a lot of bugs will appear because of the switch, and that if that fact is not handled properly, there is a real risk that the most noticeable outcome will be people getting unhappy about the change.

WebKit

St. Pierre covered GNOME's Wayland support at the desktop environment and toolkit level, but Žan Doberšek followed up with a session specifically about Wayland support in the GTK+ version of WebKit, which GNOME uses for its Web browser and as the HTML renderer in several other applications.

In large part, he said, the WebKitGTK+ support for Wayland relies on the same GDK backend support as other applications. But there are also several factors unique to the WebKit case. First, GDK does not support hardware-accelerated video compositing, which WebKit needs for video playback. The solution the project has taken is "inventive," he said. Only specific HTML and CSS features trigger the need for the feature, so they can be treated as a special case. Under X, these compositing operations can be handed off to OpenGL.

But, under Wayland, that same technique is not allowed because it involves handing graphics to a separate process—a major security risk. Instead, WebKitGTK+ has its own nested Wayland compositor that runs inside the main UI process. Making this work required writing an extension to Wayland, but only a small one. It currently works on Intel graphics, but is less well supported elsewhere, because it relies on several specific OpenGL extensions.

The other troublesome factor confronting WebKitGTK+ is how to handle plugins. Opinion is divided, he said. "Personally, I don't want to tackle it because I want plugins to die," he added. The only solution that looks plausible, he said, is if there is reusable work from Mozilla's JavaScript-based, open-source Flash renderer.

The hardware-accelerated compositing work, though, is on track for a future release already. It could land in the WebKitGTK+ 2.6 version, he added, although there has not been a stable candidate released yet. Performance issues still have to be addressed, as does WebGL support.

Overall, there may not be a clear answer to the question of "when will GNOME have Wayland support?" But it is still helpful to see an update on the project's process. The WebKitGTK+ engine is one of the few special cases—most applications will rely on Mutter and GTK+ for their Wayland support—but, on the other hand, accessing the web is one of the most important uses of a modern desktop.

[The author would like to thank the GNOME Foundation for travel assistance to attend GUADEC 2014.]