Development

Working with xdg-app application bundles

One of the oft-recurring topics at GUADEC 2015 was the xdg-app application-packaging system currently being developed. Xdg-app's lead developer Alexander Larsson gave a presentation on its current status on the first day, and it featured prominently in Christian Hergert's keynote about reaching new developers as well as in Bastien Nocera's talk about hardware enablement. Perhaps the most practical discussion of the subject, however, came in Stephan Bergmann's talk about his recent attempts to bundle LibreOffice into an xdg-app package.

Bundles

By way of background, xdg-app is a scheme for bundling individual applications into isolated, standalone packages that can be safely installed on any Linux distribution. The packaged application is isolated from the running system by way of namespaces, control groups, and other standard Linux technologies; the plan is for xdg-app to provide a fully sandboxed environment as well—to secure the host system against misbehavior by the application and to secure applications against one another, although this aspect of the plan is still in development.

Key to xdg-app's success will be xdg "runtimes" that are, in a sense, special-purpose application bundles that provide a base level of services and libraries intended for other app bundles to utilize. The plan is for future GNOME releases to be published as runtimes and, since they will be isolated from one another, it will possible for the GNOME 3.20 and (say) GNOME 3.24 runtimes to coexist on a single system. Applications that need one or another specific runtime never even need be aware whether their target runtime is the latest version—nor what the underlying distribution is.

Larsson gave a recap of the latest developments in xdg-app. At the moment, one of the bigger concerns is defining and implementing the suite of system services that a bundled app can use to do meaningful tasks while inside of its sandbox. "Setting up a sandbox is easy," he said; "actually doing something in a sandbox is harder." The xdg-app approach is for the app to "request that a hole be poked in the sandbox" by the system. The host system will define a set of services (network access, file access, printer access, IPC access, and so forth). When a bundled app needs to, say, open a file, the host system will prompt the user to approve or cancel the access (in the file case, for example, by showing a file-chooser dialog).

The team refers to the various services that mediate these requests as "portals." Wayland and PulseAudio are the portals for graphics and audio, a filtered D-Bus service will be the IPC portal, and so on. Several of the portals have yet to be fleshed out and require some more development—such as the printing and geolocation portals. Some of them have not quite been defined yet—it seems clear that users will want a "sharing" portal akin to the ones on iOS and Android (to send files to mobile devices or post material to social media), but its exact make-up is still an open question.

An audience member asked how app bundles request the permissions they want. Larsson replied that they do not—"it's not like Android," he said, where permissions are requested at install time. The portals are designed to trap all requests in a firewall-like (i.e., "deny by default") manner and let the user make the decisions interactively. Larsson invited those interested in further exploring the idea to follow the xdg-app mailing list and to visit the SandboxedApps section of the GNOME wiki.

Office in a box

On August 9, Bergmann, a LibreOffice developer working at Red Hat, presented his results from attempting to bundle LibreOffice with xdg-app. LibreOffice is a complex beast: the project is several million lines of code, has hundreds of submodules and dependencies, and is complex enough that it can break almost any toolchain. "Most of my time with LibreOffice is spent breaking compiler plugins" by triggering corner cases with LibreOffice builds, he commented. Linux is a special challenge because distributions compile and package LibreOffice separately with different configurations—and yet users still want a button they can press on the LibreOffice web site to download the latest release.

The project's current approach is to provide a crippled, somewhat restricted Linux download in both Debian and RPM format. That version uses an old version of GTK+, an old virtual filesystem library, and is missing several optimizations.

Thus, the benefits of xdg-app for LibreOffice are enormous, he said. If the project could provide a cross-distribution xdg-app bundle, that would save LibreOffice developers considerable hassle. In addition, he said, opening strange office files is one of the most common malware vectors, so using xdg-app's sandboxing feature would offer LibreOffice users valuable protection.

There were a handful of up-front challenges to building LibreOffice into an xdg-app bundle, Bergmann said. For starters, the xdg-app SDK is based on Yocto, which is "somewhat obscure" and can be difficult for people not used to embedded development. But Yocto also lacks some necessary packages, like Mesa's GLU or Perl's Archive::Zip module, which is employed during the LibreOffice build process. He also discovered that the Yocto layer's xml-config was hard-coded to always return an exit status of 1, which meant all calls to it failed. Bergmann fixed most of these problems, in some cases by modifying the Yocto side and in others by modifying LibreOffice.

He then set out to find which combination of the over 300 configuration switches in LibreOffice were necessary to build for xdg-app. Some things had to be disabled, such as printing, which is not yet supported in xdg-app. He also removed Java (which is used by the database modules and several extensions), and had to configure the build for GStreamer 1.0 (as opposed to GStreamer 0.10, which is another LibreOffice compile-time option). All told, he said, he had to remove about 30 libraries.

The build he came up with is X11-only and includes only the English localization. The project has not decided whether bundling every localization into one package is a good idea or not; his English-only package weighed in at 350MB already, whereas the every-language-included versions built for Windows are "monster sized" by comparison.

The process took time, but Bergmann eventually did successfully build a LibreOffice xdg-app bundle, which he then demonstrated. The look and feel is identical to a standard LibreOffice window, as one would expect—until one tries to open a file. At that point, all the app bundle sees is an empty home directory, since the file-access portal is not implemented.

On that point, he noted that the permissions model seems clear: if the user opens a file through the file chooser, that should be interpreted by the host system as implicitly granting the bundled app permission to read and write the file. But it turns out that LibreOffice has a more complicated of a model of its own. For example, LibreOffice creates lock files to prevent multiple users from editing the same file simultaneously; this breaks when xdg-app has sandboxed the program: LibreOffice tries and fails to create a lock file, then opens the requested file in read-only mode.

There are some other open questions already, he added, such as how to handle opening files by URL—which LibreOffice supports, but which uses a different code path than local-file access. The program also takes pains not to have more than one instance running at a time for any given user; the first LibreOffice process opens a socket, and any subsequent instances launched communicate with it. How to implement this in xdg-app requires some exploration. So do printing and offline help content, he added—but, all things considered, the experiment was a successful one. Simpler packaging is good, he concluded.

Obviously LibreOffice is only capable of providing one data point, but it is a large and complex point (if that does not stretch the metaphor too far), and is certainly a critical application for a great many users. Xdg-app has the potential to change Linux packaging for several classes of project and software packager: distributions, independent vendors, large project families, and so on. The mostly successful LibreOffice experiments certainly seems to bode well for xdg-app's prospects for third-party application vendors.

[The author would like to thank the GNOME Foundation for travel assistance to attend GUADEC 2015.]

Comments (26 posted)

Quotes of the week

— Christian Hergert at GUADEC 2015, when asked if GNOME should teach new developers C or something else.

— Shaun McCance at GUADEC 2015, listing the benefits of volunteering to organize the next GUADEC.

Comments (none posted)

Firefox 39.0.3 is out

Firefox 39.0.3 has been released. This update contains exactly one change: a fix for the recently reported PDF vulnerability that is being actively exploited on the net.

Comments (1 posted)

Firefox 40 is available

Mozilla has released Firefox 40. There are several new features listed in the release notes such as; improved scrolling, graphics, and video playback performance with off main thread compositing, added protection against unwanted software downloads, a new style for add-on manager based on the in-content preferences style, and an improved graphic blocklist mechanism.

Comments (16 posted)

Ubuntu One file-syncronization code released

Ubuntu has announced the release of the file-synchronization code behind its "Ubuntu One" service. The release is about as "over-the-wall" as it gets, though: "Will you take patches? In general, no. We won’t have anybody assigned to reviewing and accepting code. We’d encourage interested maintainers to fork the code and build out a community around it."

Comments (3 posted)

OpenSSH 7.0

The OpenSSH 7.0 release is out. It fixes a number of problems and adds a few new configuration features, but the main focus of 7.0 is taking things out: "This focus of this release is primarily to deprecate weak, legacy and/or unsafe cryptography." More old crypto is slated for removal in 7.1; see the announcement for the list.

Full Story (comments: 20)

Thor: another free video codec

Cisco, it seems, is unhappy with the patent mess around video codecs, so it has launched a project called "Thor" to make one that can be freely distributed. "The effort is being staffed by some of the world’s most foremost codec experts, including the legendary Gisle Bjøntegaard and Arild Fuldseth, both of whom have been heavy contributors to prior video codecs. We also hired patent lawyers and consultants familiar with this technology area. We created a new codec development process which would allow us to work through the long list of patents in this space, and continually evolve our codec to work around or avoid those patents."

Comments (69 posted)

Docker 1.8 released

The 1.8 release of the Docker container system is out, with a number of new features. "Docker Content Trust is a new feature in Docker Engine 1.8 that makes it possible to verify the publisher of Docker images. When a publisher pushes an image to a remote registry, Docker signs the image with a private key. When you later pull this image, Docker uses the publisher’s public key to verify that the image you are about to run is exactly what the publisher created, has not been tampered with, and is up to date."

Comments (none posted)

ncurses 6.0 released

Version 6.0 of the ncurses library has been released. The banner feature of the new release is a new ABI; all existing ncurses-5.x-using programs should be source-compatible. The new ABI includes support for more than 16 colors and more than four mouse buttons—thus enabling mouse-wheel support.

Full Story (comments: none)

Development newsletters from the past week

• GNU Toolchain Update (August 5)
• LLVM Weekly (August 10)
• Perl Weekly (August 10)
• PostgreSQL Weekly News (August 9)
• Python Weekly (August 6)
• This Week In Rust (August 10)
• Ruby Weekly (August 6)
• Tor Weekly News (August 8)
• Wikimedia Tech News (August 10)

Comments (none posted)

Federated Cloud Sharing in ownCloud 8.1 (ownCloud blog)

The ownCloud blog has a post about federated file sharing between ownCloud instances in ownCloud 8.1, but it also looks at the wider view of federation between various kinds of cloud servers. ownCloud founder Frank Karlitschek has a series of posts (It is Time to Federate Our Clouds, The Next Generation File Sync and Share Technology, and The Federated Architecture of Next Generation File Sync and Share) on federation technology and has also proposed a cross-cloud-platform federation API: "In addition, today Frank proposed a draft of a Federated Cloud Sharing API to the Open Cloud Mesh working group with the goal of jump-starting a discussion about what is needed to enable federation between different file sharing implementations. Sharing among ownClouds is great, but the true power of a federated file cloud is available when you can share among different implementations seamlessly, because you all speak the same common language. This is the goal of the Open Cloud Mesh working group (of which ownCloud is a member as well), and outside of that, drafts have been shared with a number of well known standards organizations around web technologies and fellow open source file share and sync projects to get the work started."

Comments (1 posted)

Grasch: A Frank Look at Simon: Where To Go From Here

On his blog, Peter Grasch considers the future for the Simon speech-recognition system for KDE. He is passing the torch and will no longer be actively participating in the project, but he spent some time passing on his knowledge and some thoughts on where things might go from here. In addition, he built a working prototype of a speech-based command and control system for the Plasma desktop called Lera. "If anything, Lera is a starting point. The next steps would be to move Simon’s “eventsimulation” library into a separate framework, to be shared between Lera and Simon. Lera could then use this to type out the recognition results (see Simon’s Dictation plugin). Then, I would suggest porting a simplified notion of “Scenarios” to Lera, which should only really contain a set of commands, and maybe context information (vocabulary and “grammar” can be synthesized automatically from the command triggers). The implementation of training (acoustic model adaption) would then complete a very sensible, very usable version 1.0."

Comments (none posted)