FOSDEM: The Wayland display server

In his talk at FOSDEM (Free and Open Source Software Developers' European Meeting) 2012 in Brussels, Kristian Høgsberg gave an overview of the Wayland display system, where it comes from and how the Linux graphics stack has evolved to make Wayland feasible. He also shared some information about the schedule of the 0.85 and 1.0 releases. If all goes according to plan, we'll see the first stable version of Wayland by the end of the year.

Replacing a tried-and-true technology such as X by a new one such as Wayland is no small task, and the idea alone has already stirred up the Linux community in the past. However, not every critic understands exactly what Wayland is. Apparently some even have various strange opinions about Kristian, for instance that he holds a grudge against X or that he even doesn't know anything about how X works. To dot the i's, Kristian started his talk by clarifying that he has been a core X developer since 2004. "I admire the design and implementation of X, it just happens that it's the wrong solution for what we do now." Kristian started developing Wayland in 2008 in his spare time while working for Red Hat, and now he is working on it at Intel in the graphics team of the Open Source Technology Center.

Where do we come from?

A while ago, X did lots of things related to its primary task as a display server. For instance, it did font management and font rendering, graphical mode setting and acceleration code was tied up in X specific drivers, the X server had input drivers to parse various input device serial formats, and so on. However, bit by bit many of these tasks were moved into separate components, many of them in the kernel. Wearing his Red Hat hat, Kristian worked on AIGLX (which allowed compositors to run with hardware acceleration on X), DRI2 (which provides OpenGL hardware acceleration), KMS (Kernel Mode Setting), and GEM (Graphics Execution Manager, doing memory management for graphics drivers). In principle, all hardware drivers needed by X are now in the Linux kernel (KMS) or in Mesa (an OpenGL implementation).

Much of this work was done as part of Kristian's job as a member of Red Hat's X team to make a composited desktop possible. Without a compositor, each application renders its output directly into the buffer of the X server. With a compositor (such as Compiz, KDE's KWin, GNOME 3's Mutter, and Xfce's Xfwm), applications render their output in their own private off-screen buffers. The compositor renders the final desktop output by painting those buffers onto the screen's frame buffer. On a modern Linux desktop, compositing has become a basic expectation and requirement, Kristian says: "Compositing is not only about those 2D and 3D visual effects, but it has other fundamental advantages, such as flicker-less movements of windows."

But now that we have reached a point where most of the hardware drivers implement KMS or Mesa and compositing is used for drawing application windows on the screen, most of the work in the window system is done by the compositor and the applications themselves; the X server is just used as a middle-man for input. Kristian wanted to create a display server that directly supports this new window system architecture from the ground up. This became Wayland, which integrates the display server, window manager, and compositor into one component, and is, according to Kristian, "essentially just consolidating existing practices and simplifying the X architecture to what we are using it for nowadays." The result is a window system architecture that is more responsive and has less graphical artifacts such as tearing or flickering. More details about Wayland's architecture and the difference with X can be found on the project's wiki.

Hardware support

So, with this new window system architecture, Wayland merges the compositor into the display server, and applications talk directly to the compositor. The compositor reads input from devices such as the keyboard and mouse using the kernel's evdev subsystem, and it distributes this input to the applications. The compositor uses KMS to bring up the display, and applications push their buffers to the compositor, which combines the buffers from the various applications and renders the result using direct rendering, through OpenGL ES, to the KMS buffers. "There is no hardware-specific code whatsoever in the compositor code," Kristian emphasizes, "and there is also no rendering API for drawing lines, text, and so on: it's all just direct rendering."

Kristian's short answer to the question which graphics hardware support Wayland is "If the Mesa driver for your graphics hardware supports DRI2 under X and has KMS support, you can run Wayland." Currently, this means that you'll have most luck with the open source drivers for Intel, AMD and NVIDIA graphics hardware, but when driver writers add new chipset support in Mesa, it will automatically support Wayland. It's still unclear what AMD and NVIDIA will do with their proprietary drivers.

Wayland support in graphical toolkits

But applications have to be modified to be able to use Wayland instead of X, of course. Fortunately, most applications don't talk to the X window system directly but are using graphical toolkits, which provide the common user interface elements such as text fields, scrollbars, buttons, and so on. So most of the work in porting an application from X to Wayland is in porting the toolkit it uses. According to Kristian, all major toolkits are being ported to Wayland as we speak, including GTK+ 3, Qt 5, EFL (Enlightenment Foundation Libraries), Clutter, and SDL (Simple DirectMedia Layer). The Qt team at Nokia is maintaining and driving the Qt port themselves, he said. Wayland's web site has instructions on how to use these toolkits with Wayland. Applications that are using their own toolkit, such as Blender and LibreOffice, will have to do the porting effort themselves.

Porting a graphical toolkit to Wayland poses some interesting challenges, for instance because Wayland doesn't support some things that are quite natural in X, such as pointer grabbing: in Wayland an application currently cannot claim the keyboard or mouse input. However, games regularly grab the mouse cursor to prevent the player from accidentally losing mouse control over their game, and pop-up windows rely on grabbing the keyboard and pointer too. Another challenge is the support for client-side window decorations, which allows individual applications to control their appearance. Both challenges are being worked on.

Applications that aren't using a Wayland-compatible graphical toolkit or that have some problems running on Wayland can still run on an X server that runs as a Wayland client. Of course this adds some overhead, but it should be minimal. Running a rootless X Server as a Wayland client could also be a temporary solution to keep using network transparency, a feature that Wayland lacks. In his talk, Kristian reassured the audience that the current lack of network transparency is not definitive: "While supporting remote displays is not a priority now for Wayland, nothing in Wayland's architecture makes this impossible."

Toward a stable API

The first real release of Wayland happened a few days after Kristian's talk at FOSDEM: Wayland 0.85. The code is divided in two parts: Wayland is the protocol and IPC mechanism, while Weston is the reference implementation of the compositor (and thus also the display server). "Weston has around 10,000 lines of code and it can be used as the base for a compositor for mobile or embedded systems," according to Kristian, who ran Weston on his laptop during his FOSDEM presentation. However, existing X compositors such as KWin and Mutter can also be modified to become a Wayland compositor.

Kristian promises that the 0.85 release is going to be protocol and API stable. The point of the release is that developers can begin experimenting with Wayland and porting their toolkits and applications. While it used to be that you had to compile a special KMS "pageflip" kernel, a patched Mesa branch, and Kristian's own EGL library to be able to run Wayland on your system, now everything should be upstream. Kristian hopes to put out a Wayland 1.0 release in late 2012 or early 2013. So while Wayland will not replace X overnight, we will likely see the first mainstream use of the new window system architecture in 2013.