Development

Key GNOME component updates

GNOME is a massive project with a lot of moving pieces that are directed by separate individuals so, at times, it can be hard to up to date with where things are moving. At GUADEC 2016 in Karlsruhe, Germany, developers provided updates on the Flatpak packaging format, the GTK+ Scenegraph Kit (GSK) API, and a new logging system for GNOME applications.

Flatpak

Alexander Larsson presented an update on the progress within the Flatpak project over the past year, followed by a look at what comes next. In addition to "the great renaming," he said, the past year saw the project move from Freedesktop.org to GitHub, add several tools, and add lots of polish.

The new features include flatpak-builder, a program that builds Flatpak packages on the command line. Previously, he said, users had to write their own scripts using low-level APIs, which certainly hindered adoption. Support was also added for AppStream metadata, so Flatpak packages now provide user-level information (such as descriptions of the program). This allowed the GNOME Software installer tool to add support for managing Flatpak packages.

Package installations are now system-wide by default (although per-user packages are still supported). Permissions are now handled by PolicyKit, which makes system-wide installs more reliable. The first iteration of portals was added, enabling users to mediate an application's requests for access to potentially risky services. The project also released a stripped-down version of the basic container used in Flatpak packages, called bubblewrap, that can be used to isolate any program.

The Flatpak package format was also extended to support portable "single-file bundles" that can be sent via email or exchanged on USB sticks, with no system infrastructure required. The first Flatpak runtimes (separate packages provide the system APIs a Flatpak requires to run) were also released; a GNOME 3.20 runtime and a generic Freedesktop.org runtime have been produced so far. New architectures are now supported as well, including 32-bit and 64-bit ARM. Finally, several external adopters have started using Flatpak bundles, notably LibreOffice and Endless Mobile; KDE has also done some initial work toward building a compatible runtime.

Moving forward, Larsson said, portals will be an area of major work. Several new portals are needed, including one for access to DConf, one for sharing media (such as over social networks), one for accessing contacts, and several new device classes (including optical drives, USB sticks, and game controllers). More infrastructure work is needed to support some of the portals, however. Wayland is still missing pieces like cross-window processing (which makes clipboard sharing problematic), PulseAudio "is completely unsafe" at present, and the Pinos camera-access API lacks a viable permissions model.

Larsson added that he hopes to see the Flatpak work dispersed more throughout the GNOME project in the future, since, at present, it is almost a solo effort. The GNOME Release Team could maintain runtimes, he suggested, and individual application teams could build their own Flatpak packages and maintain the necessary build information and metadata. Further out, Larsson speculated that the project might need to consider "long-term support" runtimes that provide a minimal set of services but would be suitable for applications that are rarely updated (such as proprietary programs). It would also be nice, he said, to offer a personal package archive (PPA)-like site for Flatpak distribution and to support paid programs.

GSK

Emannuele Bassi began his session by noting that "the GSK talk" had become a GUADEC staple, much like "the Clutter talk" was before it. GSK is Bassi's Clutter replacement, a drawing API that will allow application developers to create relatively free-form user interface elements by defining geometric primitives in hierarchical layers and transforming them using standard positioning, visibility, and animation operations.

But GSK is still in development, so, rather than cover the same ground as last year by discussing incremental progress, he chose instead to spend most of the allotted time recapping the history of how GTK+ has drawn pixels to the screen over its lifetime. The project is close to 20 years old, he said: the first release was in 1997, roughly when MP3s first hit the internet, Windows 95 dominated the OS marketplace, and the browser wars had not yet started.

At that time, "the apex of GUI toolkits was Motif," and GTK (with no "+") looked exactly like it. Each widget was backed by an X11 window, each had its own color map on the X server, and all drawing commands were sent over the wire. The 1.2 release in 1999 added the first theming support, but it was not until 2.0 in 2002 that GTK+ was "all grown up" and the project decided to "bring the light to other platforms" like Windows and Mac OS X. That meant reimplementing the X11 drawing API for those platforms, which is what led to GDK.

By the time GTK+ 2.6 came along, several new projects had been started, most notably Cairo and the XRender extension. GTK+ added support for them, replacing X11 calls with Cairo calls and moving to client-side window rendering—but support for the deprecated APIs was not removed. In 2011, GTK+ 3.0 was released, and OpenGL drawing was added—although it was not yet taking advantage of GPUs. GTK+ also added support for the CSS drawing model, which replaced the Cairo drawing model.

But the Cairo API has remained part of the 2D rendering stack, even though it is only optimal for a few specific pipelines, Bassi said. That is because it turns out that "OpenGL is bad at drawing GUI elements unless your elements look like Quake." Thus, GTK+ today continues to use Cairo "for what it's good at" while using OpenGL for what it was good at.And OpenGL has been getting better, he said; in the long run Cairo may eventually be dropped, but for the time being it will remain supported.

This approach is "the way out," he said; retaining both the Cairo and OpenGL APIs lets GTK+ make more efficient use of multiple cores, without forcing application developers to rewrite everything for a new drawing API. Moreover, maintaining support for an old API while adding a new one has always been GTK+'s approach: the same decision was made when adding GDK to X11, when adding Cairo to GDK, and when adding OpenGL to Cairo.

This history lesson, Bassi said, was meant to reassure developers that the current work on GSK would not disrupt their applications. He is still working hard on GSK itself, optimizing rendering on the GPU not the CPU. "So are we in 'the future' yet? We're pretty close," he concluded. GSK can do high-quality animations, 3D transformations, compositing and rendering CSS primitive, and more. But it is important to remember that there will be always be something new coming down the pipeline, he said.

GSK will not be available in the upcoming GTK+ 3.22 release, but it is likely to be merged for the subsequent version.

Structured logs

Philip Withnall discussed his recent work adding a structured logging facility to GLib. The library's existing logging interface is GLog, "which everyone has been using forever," he said. But GLog is rather limited, so he has been borrowing ideas about structured logging from journald and other projects in order to develop a superior replacement. The work is being done in upstream Glib, even though Withnall has been doing it on behalf of Collabora.

The upshot is that structured logging uses a set of "key: value" fields, rather than a simple string, for each log message. There are numerous benefits. In particular, structured logging allows each log message to have a unique message ID, enabling searches within the log and in the source code. There is a new writer function that replaces the old GLog log handlers, and the new Glib log API hooks into journald, so that it can tell if journald is running and whether or not log messages are being directed to the journald socket.

The new API supports context pointers in log messages, which allows logging complex state information without the use of global variables. And the API also supports color console output, he added, "which I think is probably the most important change." In light of the new features, he said, "we're encouraging people to log everything," then to do whatever filtering is necessary in the log viewer, rather than trying to fine tune what messages are logged in advance or requiring users to set an "esoteric" log level via a command-line switch.

Withnall then recapped the shortcomings of GLog. In that system, each library would install its own log handlers to do "something different" with log messages from applications. How various log domains and log levels interacted was, essentially, undefined, and there was little that a developer could do to customize behavior. Some features could be tweaked in a log handler, but others (like how fatal errors were handled or when to abort on an error) were hard coded in GLib.

There was also some global state that various libraries on a system would fight over (such as which errors were regarded as fatal), leading to unpredictability. And there was no way for unit-testing code to test for complex sequences of messages; the GTest facility could watch for a specific sequence of messages, but it cold not cope with optional messages or variation in the delivery order.

Next, he presented the new structured log format. The key-value pairs in a log message do not need to be formatted into a string immediately, so that task can be put off as long as is convenient. Keys can be namespaced to implement application-specific message features. Performance is potentially better than the old log API, he said, although it depends on how it is used. Applications that do string formatting on all of their log messages will, of course, pay the price for that. Without string formatting, there is a zero-copy path all the way to the socket output.

Porting existing programs to the new API is straightforward, he said. Developers merely need to include:

    #define G_LOG_USE_STRUCTURED
    #include <glib.h>

and add a g_message("foo") call in their code. This will pass through the message ("foo") to the structured logging facility, plus the file name, line number, and function name. Full control over the log message can be achieved with the g_log_structured() function, which lets the developer supply whatever key-value pairs are of interest.

There is also a g_log_structured_array() function ("the turbo option") that can be used to pass in an entire array of structured log fields. Among the other changes is a new G_DEBUG_HERE macro that is expanded into the standard GLib debug-message call.

Each application can also now define its own log-writer function; this is a change from the old API, in which log handlers were implemented in libraries and application code had little to no flexibility over log message content or logging policy. There are default log-writer functions provided (one that simply logs to journald and one that sends output to stdout) that application developers can extend as required.

There were a few questions from the audience at the end of the session. One person asked how message IDs are generated; Withnall replied that they can be any string that the developer wants—which prompted Christian Hergert to suggest that development tools like Builder could be configured to generate them automatically. Lennart Poettering asked if the facility could be extended to handle error codes; Withnall responded that it sounded like a feasible idea and promised to explore it.

[The author would like to thank the GNOME Foundation for travel assistance to attend GUADEC 2016.]

Comments (none posted)

Quotes of the week

— Chris Webber

— Hubert Figuière

Comments (none posted)

KDE Applications 16.08.0 is available

Version 16.08 of the KDE Applications collection has been released. Several additional applications have been ported to KDE Frameworks 5; the Kontact Suite has undergone a significant clean-up effort and now offers improved VCard 4 support, and Marble 2.0 has been added, featuring experimental support for locally rendering vector map tiles.

Full Story (comments: none)

Microsoft announces PowerShell for Linux and Open Source

Microsoft has announced the release of its PowerShell automation and scripting platform under the MIT license, complete with a GitHub repository. "Last year we started down this path by contributing to a number of open source projects (e.g. OpenSSH) and open sourcing a number of our own components including DSC resources. We learned that working closely with the community, in the code and with our backlog and issues list, allowed us prioritize and drive the development much more responsively. We’ve always worked with the community but shifting to a fine-grain, tight, feedback loop with the code, energized the team and allowed us to focus on the things that had the most impact for our customers and partners. Now we are going big by making PowerShell itself an open source project and making it available on Mac OS X, Ubuntu, CentOS/RedHat and others in the future."

Comments (78 posted)

kdenlive 16.08.0 released

The kdenlive video editor project has announced the 16.08.0 release. "Kdenlive 16.08.0 marks a milestone in the project’s history bringing it a step closer to becoming a full-fledged professional tool." Highlights include three-point editing, pre-rendering of timeline effects, Krita image support, and more.

Comments (2 posted)

KDevelop 5.0 released

Version 5.0.0 of the KDevelop integrated development environment (IDE) has been released, marking the end of a two-year development cycle. The highlight is a move to Clang for C and C++ support: "The most prominent change certainly is the move away from our own, custom C++ analysis engine. Instead, C and C++ code analysis is now performed by clang." The announcement goes on to describe other benefits of using Clang, such as more accurate diagnostics and suggested fixes for many syntax errors. KDevelop has also been ported to KDE Frameworks 5 and Qt 5, which opens up the possibility of Windows releases down the line.

Comments (4 posted)

Introducing OpenStreetView

The OpenStreetMap (OSM) project has unveiled a new project named OpenStreetView that provides "free and open street level imagery platform" compatible with OSM. A web interface and apps for Android and iOS are available; like OSM, the new project allows individual users to contribute imagery to the database. The mobile apps can also be tethered to the diagnostic ports of most cars (using a Bluetooth adapter), which enables the collection of finer-grained speed and position data for placing captured images on the map.

Comments (3 posted)

Development newsletters from the past week

• What's cooking in git.git (August 18)
• GNU Toolchain Update (August 24)
• This Week in GTK+ (August 22)
• OCaml Weekly News (August 23)
• Perl Weekly (August 22)
• PostgreSQL Weekly News (August 21)
• Python Weekly (August 18)
• Ruby Weekly (August 18)
• This Week in Rust (August 23)
• Wikimedia Tech News (August 22)

Comments (none posted)

Mozilla rebranding: Now for the fun part

Mozilla has launched an "open design" process to evaluate potential rebranding ideas with input and feedback from the public. Tim Murray has declared the process underway, noting "we’ve jumped off this cliff together, holding hands and bracing for the splash." Seven potential designs have been posted, and the project invites the public to "have a look at the seven options and tell us what you think." A variety of feedback mechanisms are provided.

Comments (2 posted)