Ubuntu unveils its next-generation shell and display server

Ubuntu publicly announced its plan for the future of its Unity graphical shell on March 4, a plan that includes a new compositing window manager designed to run on the distribution's device platforms as well as on desktop systems. The plan will reimplement the Unity shell in Qt and replace Compiz with a new display stack called Mir that will incorporate a compositor, input manager, and several other pieces. Mir is not designed to use the Wayland display protocol (although the Ubuntu specification suggests it could be added later), a decision that raised the ire of developers in several other projects.

Announcements, announcements, announcements

Oliver Ries made the announcement on the ubuntu-devel mailing list, saying it was timed to coincide with the start of the distribution's Ubuntu Developer Summit, where more detail and discussion would follow. Ries said the changes were necessary "in order to implement the vision of converged devices"—namely the Ubuntu Touch project to build an interface compatible with phones, tablets, and smart TVs. The plan involves porting Unity from its current OpenGL-based implementation to Qt and implementing the Mir server. There are descriptions available on the Ubuntu wiki for both Mir and "Unity Next."

In a blog post, Ries elaborated further on the "overhaul" and the reasons behind it. It was already clear that the current implementation of the Unity shell (which runs as a plugin to Compiz) would eventually need to go; handling multiple monitors is problematic, as is implementing the global menu bar. The Ubuntu Touch devices are expected to add to the complexity by relying on different pointing devices and text input methods. In addition, Compiz itself was put into maintenance mode by its lead developer in December 2012.

In evaluating the options, Canonical decided that the X server needed replacing, and that Wayland was not viable for running on handheld devices. The new solution, Mir, is designed to run both on the system-on-chip (SoC) hardware found in phones and on standard desktop graphics hardware. The port of Unity from the OpenGL-based Nux toolkit is a reversal of sorts, Ries noted, but when Ubuntu halted work on its previous Qt-based implementation of Unity (the "fallback" mode for systems without sufficient graphics power to run Nux-based Unity) it did so in large part because Qt's future was uncertain. The project was in the midst of a hand-off from corporate owner Nokia to a community-governed council, and it was not clear that the Qt-based Unity would offer a comparable feature set to the OpenGL version. Now that Qt is in a stable and healthy position, Ubuntu has resumed work on the Qt-based Unity.

Thomas Voß wrote a blog post of his own, which lists several other rationales for Mir, including the ability to leverage existing Android drivers, and the desire for an input system suitable for mobile device usage. In addition, Weston, the reference implementation of Wayland, suffered from a "lack of a clearly defined driver model as well as the lack of a rigorous development process in terms of testing driven by a set of well-defined requirements." For a short-term solution, the project borrowed Android's SurfaceFlinger, but will replace it with Mir in time for the release of Ubuntu 14.04. Builds of both Mir and Unity Next will be available starting in a few months, although Ubuntu 13.04 and 13.10 are not expected to use them.

Specifics and protocols

The Mir wiki page goes into considerably more detail about the architecture of the system. It consists of a server-side library called libmir-server, a matching client communication library called libmir-client, and the unity-system-compositor. Other re-written Unity components include the Unity shell, Unity Greeter, and bindings for GUI toolkits (initially Qt, with GTK+ and SDL to follow). The Unity Next page further details how the changes will affect applications, such as the environment's launchers and notification system.

But the majority of the public discussion about the announcement has centered around the Mir display server—and in particular why it will not simply be an implementation of the Wayland protocol. For now, the Mir page lists a few reasons why Ubuntu decided Wayland did not fit the bill, including input event handling, input methods (that is, flexible mechanisms for text input, which are a more complicated issue for logographic writing systems like Chinese), and the manner in which shells and sessions are treated distinctly from normal client applications. On the other hand, the page does emphasize that Mir is designed to be "protocol-agnostic" and that Wayland support could be added in the future.

Not everyone found the reasons listed compelling, of course, particularly developers working on Wayland and Weston. Kristian Høgsberg started a discussion thread about it on Google Plus, in which several took issue with the Mir wiki page's description of Wayland's input event system. Most notably, the wiki page had initially said that Wayland's input events duplicated insecure semantics from X11's input system. Canonical's Christopher James Halse Rogers ("RAOF") later visited the Wayland IRC channel, and was asked about the security issue, which Høgsberg said was incorrect. Halse Rogers said he was unaware that the wiki mentioned the security issue, and subsequently removed it from the page.

The IRC discussion log makes for interesting reading, once one wades through the less compelling flame-like comments. Høgsberg also took issue with the Mir wiki page's comments about Wayland's distinction between normal client applications and special processes like the shell and session manager. The wiki page said that the shell-integration parts of the Wayland protocol were privileged, a design that the Ubuntu team disagreed with because it would require additional security measures. Høgsberg argued that the APIs provided were unprivileged, and that Ubuntu could replace any of them that it did not like without altering the core interfaces. In particular, the interfaces in question (wl_shell and wl_shell_surface) are actually optional extensions. In an interesting wrinkle, Wayland developer Tiago Vignatti posted a blog entry on March 5 describing the special protocol available to user shells, although he, too, said that it was not a privileged protocol.

In the IRC discussion, Halse Rogers countered that removing and replacing multiple interfaces would result in a display server that was not really Wayland anyway (and would require Ubuntu to maintain separate integration code for the GUI toolkits in particular). He added that Mir also uses a different (server-side) buffer allocation scheme in order to support ARM devices. Høgsberg replied that Wayland could add support for that as well, noting "I realize that this all isn't really documented, but it's not like Wayland only works with client side allocated buffers."

Divergent or convergent projects

Two other people in the IRC discussion raised a non-technical complaint, commenting that Ubuntu should have brought its issues with Wayland's design to the Wayland development mailing list, rather than develop a separate project. That does sound ideal, but on the other hand it is not easy to demonstrate that such a discussion would have guaranteed that Wayland evolved into the protocol that Ubuntu wanted. After all, at one point in the past, Ubuntu was planning on adopting Wayland; Mark Shuttleworth announced that intention in November 2010.

Canonical's Chase Douglas subsequently did join the Wayland mailing list, and on at least one occasion he weighed in on a design issue. The topic was touch input support in particular, in February 2012, and Douglas did not seem pleased with Wayland's touch event handling, noting specifically that it would not work when the user was sending touch events to more than one application. Most mobile platforms do not support having multiple foreground applications, but the tablet builds of Ubuntu Touch do.

Touch event handling is an interesting case to consider. The Mir wiki page cites input event handling as one of the project's points of disagreement with Wayland. It goes into frustratingly little detail on the subject, but Ubuntu is clearly interested in multi-touch and gesture recognition support due to its push on tablets and handheld devices. It debuted a multi-touch and gesture input stack with the release of Ubuntu 10.04, which it still maintains. Wayland, meanwhile, has stayed focused primarily on the desktop. In August 2012, there was an effort to revive the dormant weston-tablet-shell, although based on the commit logs it has been receiving less attention subsequently.

Certainly multi-touch and gesture-recognition are critical for phone and tablet user interfaces. Perhaps if Ubuntu is dead-set on implementing a touch-and-gesture-aware input event system that it can ship within the year, then the argument could be made that Wayland is not ready. There are few alternatives to Ubuntu's gesture framework; GTK+ gained multi-touch support in 3.4, but GNOME has only recently started working on its own touch event implementation. One might also make the case that no distributions have moved to Wayland itself, either, and it is not clear when Mutter or other window managers will implement it in a form ready for end users. There are other potential incompatibilities, such as licensing—Wayland and Weston are MIT-licensed; Mir is GPLv3. So Ubuntu could merge in Weston code, but submitting Mir patches upstream would mean adopting the other project's non-copyleft license.

None of those arguments are likely to sway the opinions of any Wayland developers toward Mir, of course. The best hope for peace probably lies in getting the two projects together to discuss their differences. On that point, Halse Rogers offered a tantalizing possibility in the IRC discussion, noting on line 215 that someone (possibly Voß) is attempting to organize such a meeting. In the meantime, however, most end users will simply have to sit still and wait to see what happens next. Ubuntu has declared its intention to ship Mir and Unity Next with Ubuntu 14.04; at the moment there is not a large distribution with a public launch date for its Wayland implementation, so it will be interesting to see which arrives first.

But one thing the Mir announcement and subsequent debate has made crystal clear is that both projects have fallen far short on the documentation front. The Mir wiki page and its associated Launchpad blueprints are all that currently exists, and they are short on detail. Then again, the Mir page is only a few days old at this stage. Wayland has been in active development for years, and its documentation, too, is sparse, to put it mildly. Høgsberg admitted as much in the IRC discussion, but who knows how many points of disagreement about Mir and Wayland compatibility could have been avoided entirely with more thorough documentation of the core and extensions. Neither project does itself—much less users—any favors when the only way to learn implementation details is to track down the lead developer and ask specific questions over email or IRC.

Ubuntu says it will begin making builds of Mir and Unity Next available in May 2013. Where both projects head over the course of the following year remains to be seen—as is also true with Wayland. A year from now, perhaps the two teams will have found common ground. If not, a head-to-head comparison of the software will surely make for a more interesting debate than does this week's strictly hypothetical discussion.