Merging GTK+ and Clutter

At GUADEC 2014 in Strasbourg, France, Emmanuele Bassi presented an update on his progress in merging the scene-graph capabilities of the Clutter toolkit into GTK+. This project is, to many in the GNOME and GTK+ community, one worth watching carefully. It has been a perceived shortcoming of the platform that GTK+ and Clutter both provided applications with drawing APIs—but separate and incompatible ones. Bassi announced his intention to add Clutter-like functionality to GTK+ at GUADEC 2013; this year, he reported, he has made progress, but it is only now approaching a state where it is ready to be made public.

The delay in completing the project is only the result of other work projects requiring his time, Bassi said. He is still committed to simplifying GTK+'s offerings for application developers by adding a scene-graph library that shares the same basic concepts and makes the same assumptions about the window manager as standard GTK+.

In fact, he said, the truth is that GTK+ has been migrating steadily to a Cascading Style Sheet (CSS)-based model for how widgets are styled, rendered, and animated, and GTK+ by itself is hitting the limits of what it can do. Shadows, borders, padding, and style properties have well-defined semantics in CSS, and it is becoming painful to try and implement some of those semantics in the basic GTK+ drawing model. There were major changes to the drawing model between GTK+ 3.8 and 3.10,which was difficult for application developers.

The project is also hitting the limits of what CSS-style animations can do. For example, he said, CSS can animate the transition between one widget state and another (such as opening a menu), but it cannot animate other events—like the mouse pointer appearing in a widget. Anyone wishing to tackle these other animations would need to write their own layer-management code; thus, such code is a clearly needed addition to GTK+.

Reexamining the plan and taking detours

Over the past year, Bassi said, he has rewritten almost all of the demo code he had ready at the last GUADEC. Last year, he had started off by adding OpenGL support to the GIMP Drawing Kit (GDK), which is GTK+'s low-level drawing library. He got as far as writing a GLX context for GDK, then realized that his approach meant that he would have to duplicate the same effort for all of GDK's supported platforms. Some searching revealed that there was already a good, compact library called libepoxy that handles all of the needed OpenGL function dispatching on all of the target platforms—and is even in use by other mature projects like KDE. Consequently, he dropped his own code and switched to libepoxy.

But enabling OpenGL support is only the first step. Developers don't want to do drawing in OpenGL; they want to composite and blend. As an aside, Bassi noted that Broadway, the HTML5 backend for GDK,is likely to require special handling for OpenGL rendering with a scene graph. The problem is that a GTK+ scene graph composites and blends graphics internally; getting the composited image into an HTML5 app could be done either by dumping everything into an <canvas> element or by adding WebGL support to GDK in addition to OpenGL.

Either way, he said, the result will be different enough that GNOME needs to think about a long-term maintenance model for it. Without one, Broadway might have to removed, which Bassi said he really did not want to see. Although Broadway is not among the most-popular GDK backends, he noted that it is evidently in production usage.

Next, he looked at what would be needed to do the 3D transformation operations that blending and styling will require of the scene graph. Cairo might seem to be the obvious choice, he said, but it has the serious shortcoming of only supporting affine transformation. Cairo also does not support basic geometry primitives needed by OpenGL, such as points, rectangles, vectors, quaternions, and matrices. Here again, he speculated that the maintenance question presented a roadblock.

Adding geometry primitives and transformations to Cairo would be a massive undertaking, and one that would "likely sit unreviewed for a long time." The past few years, he said, have demonstrated that Cairo's release-management model is not a good fit for GTK+, and that the situation "needs to be addressed sooner rather than later."

Without the necessary support in Cairo, Bassi's next step was to write a small (about 7,000 line) geometry library called Graphene. Graphene provides an API for working with geometric objects as primitives, rather than manually operating on C arrays of floating-point numbers. The first release took place in May at the GNOME Developer Experience Hackfest, and he hopes the library will prove useful for projects outside of GTK+.

Introducing GSK

With libepoxy and Graphene in place, Bassi then turned to the actual scene-graph code. The original approach, he said, had been to take Clutter and graft it directly into GDK, attaching Clutter's top-level ClutterActor object to a GdkWindow. Several obstacles make this impossible, though. First, exposing all of Clutter's existing object types directly in GTK+ would cause symbol collisions. Any application that linked against both GTK+ and the (existing) Clutter would be unable to run if the same types are exposed in both shared libraries. But second, and more importantly, Clutter contains a lot of old assumptions and designs that need to be removed anyway.

As a result, he decided to start over from scratch, "or at least as close as I can without getting the frontal lobotomy it would take for me to forget having written Clutter." The new-from-scratch code is called GTK+ Scene-graph Kit, or GSK, and Bassi walked the audience through the header file.

GSK's base type is a transparent rectangle called GskLayer. The top-level GskLayer is attached to a windowing system surface, and the scene graph is a hierarchy of layers attached below it. Children of the top-level layer can be positioned, drawn in, and transformed. By default, child layers are not constrained to the bounding boxes of their parent (except for the top-level layer), although they can be clipped.

The "layer" terminology is a departure from Clutter's own "stage" and "actor" monikers—which Bassi called "an inside joke that was nice enough, but stupid." The layer hierarchy is similar to the model used by Apple, he said, which validates its overall approach. Every layer has parent, sibling, and child relationships, and the graph can be traversed in both depth-first or breadth-first fashion.

Each layer has its own coordinate system, based in the top left corner, and a "pivot point" that is normalized to the size the layer: the default is in the middle, at (0.5, 0.5). Each layer also contains a transformation matrix that is applied before the layer is drawn; multiple transformations are applied in the order that the developer lists them. Drawing the contents of the layer is done with the @GskLayerClass.draw() virtual function.

There are also virtual functions for queuing changes to the layer layout and to layer content. These functions pass along the origin of the redraw or relayout request, so that parent layers can intercept them and handle them without necessitating full recursion up to the top-level layer. Most of the rest of the API—for adding, inserting, and removing layers from the graph, is little changed from Clutter. One of the biggest points is the use of virtual functions like @GskLayerClass.draw(), rather than having each layer draw itself directly. This, he said, was done to enforce a delegation-first model, which he called one of the best decisions made in Clutter's API design.

GSK is not yet finished, Bassi said. Setting visibility, positioning, essential transformations, and clipping are all implemented. Other pieces, such as layout rules concerning grids and box models, will be left to GTK+ itself. He hopes to complete the core API by the time GTK+ 3.15 development begins, although it might be too big a crunch to plan for integrating GSK into 3.16.

That core API will cover embedding a scene graph in a GTK+ widget and embedding a GTK+ widget in a scene graph—a scope that covers the same functionality of the current Clutter and Clutter-GTK. With that in place, existing applications using both Clutter and GTK+ for rendering (of which there are only a handful of major examples) should be able to begin migrating off of the old toolkit, once and for all.

[The author would like to thank the GNOME Foundation for travel assistance to attend GUADEC 2014.]