Ubuntu's app store and package format

As Ubuntu continues its march toward Ubuntu-based mobile devices, it has undertaken a number of efforts designed to attract third-party mobile app developers. The latest development is the beta launch of the project's mobile app store, which is tied with an app-development contest slated to run through September 15. Perhaps more interesting in the long run, though, is the publication of the specifications for how the platform's mobile apps are to be packaged. The format is called Click, and it is related to the .deb package format used for standard Ubuntu packages—but with a few key changes.

Martin Albisetti announced the opening of the app store and the publication of the package format documentation in an August 30 blog post. The Ubuntu Touch store is only accessible by registered developers for the time being; they can upload packages to the Ubuntu "MyApps" service. Uploads are then subject to a manual review process before they are published.

The process is much the same as the one offered to third-party developers of desktop applications seeking publication in the Ubuntu Software Center service, except that mobile apps can only be written for the APIs supported by the Ubuntu SDK. At the moment, just two app frameworks are supported—QML and HTML5—with HTML5 support relying on the Apache Cordova library layer.

Support for compiled apps is apparently under discussion, although no clear timetable has been publicized. One open question on that front is how to encourage app developers to license their code under GPL-compatible terms; for the contest, this is important because the winning entries will be included in subsequent Ubuntu Touch builds. Since apps written in QML and HTML are (at least in theory) their own source code, not handling binary apps neatly sidesteps the question. For the time being, the app store does not support selling apps, although that feature is expected to arrive eventually as well.

Click, here

To put an app store in place, of course, one must have some idea of what format apps will take. The format defined for Ubuntu Touch is called "Click," using the .click file extension—although there are copious caveats that the actual name of the format could still change before the Ubuntu Touch app store opens to the public. Click is derived from the .deb Debian package format, and in most ways a .click file is a valid .deb file too. Consequently, the distinct file extension helps to make it less likely that a user will attempt to install a .click app using dpkg or Apt.

There is HTML documentation on the format available, but a newer revision of the specification can be found in the Click source repository on Launchpad. There was a lengthy debate on the ubuntu-devel mailing list about the suitability of existing package formats, such as Android's .apk and cross-platform efforts like Listaller. In the end, Colin Watson said that:

Similarly, by basing Click on the Debian package format already in use, many of the existing package tools can be reused. Like a .deb, a .click package is an ar archive, which itself contains "control" and "data" tar archives. The control archive includes package management information, while the data archive holds the app's filesystem.

Click apps are designed to be self-contained; the directory tree in the package's data archive is unpacked into a single directory and cannot access the filesystem above its own root. The app filesystem is uncompressed and stays on disk, though—as opposed to being stored as a compressed image and loop-mounted.

The control archive must contain both a control file and a manifest.json file. The manifest includes several required fields like app name, app version, and installed size, and may include several optional fields as well (such as maintainer information, a reference to the app's icon, or descriptions in various languages). The control file duplicates some of this manifest information for use by the package-management tools, and it also includes a field listing the version number of the Click format used.

Because Click apps are meant to be stand-alone entities, some of the manifest and control fields diverge somewhat from their equivalents in a standard Debian package. The app name is specified in reverse-network-domain name format like com.example.catpictures, which is the same scheme used in D-Bus, in Android app naming, and in plenty of other places. The "framework" field is a simplified version of the Debian package format's way to specify dependencies. Click apps cannot have dependencies on anything other than base system services. In other words, they cannot introduce dependencies on other apps or on components installed through other means, and they cannot depend on specific versions of system libraries (which in the worst-case scenario could make dependencies unresolvable). As such, for now the only valid value for "framework" is the current Ubuntu Touch SDK framework. Perhaps in the future, different versions of the framework will need to be designated, or non-phone form factors will require separate frameworks, but this has not been explored in the documentation.

But the fact that Click apps cannot have dependencies has benefits beyond simply avoiding dependency hell. Since Click apps are isolated from system packages, the app installer can install them without querying or searching the system's dpkg database—which can be quite slow relative to unpacking a tar archive to disk.

Another—perhaps major—distinction from the Debian package format is that maintainer scripts (such as preinst, postinst, or prerm) are forbidden. This is to isolate Click apps from the system; since they are intended to be self-contained, they should not generally trigger any system-level events which could potentially be exploited. Not allowing them makes the task of auditing app submissions considerably simpler. There is one exception to the general maintainer-script moratorium: apps can include a preinst script that prevents the package from being installed directly with dpkg.

Hook 'em

On the downside, however, one useful feature lost in disabling maintainer scripts is the ability to easily hook into system services at install time. Even for self-contained apps, there are a number of system packages that need to be informed of the installation, such as updating the icon cache to include the new app's icon, locating the .desktop launcher to add to the app menu, or registering a service with D-Bus.

But Ubuntu Touch still needs to alert system services to the app-installation event somehow. Currently the plan is to use a "hook" mechanism inspired by dpkg's triggers. Triggers are calls out to other programs that are generally placed in a Debian package's postinst script. A trigger mechanism could, for example, tell Fontconfig to update the font info cache and pick up a newly-installed font. The problem with this approach is that Click packages can be installed to any directory on the system. That is to say, there is no fixed location for them; the directory name for an app is created based on a hash. This approach enables the updating of apps by installing the new version in a separate directory, then garbage-collecting the old one. The unpleasant side-effect is that, in this example, Fontconfig needs some way to learn which directory it should scan to update its cache. Dpkg triggers cannot pass user data, so apps cannot simply report their location through the trigger mechanism.

The "hooks" solution works by enumerating a stable, well-known set of services that need to perform such post-app-installation updates. Each app is responsible for listing the hooks that it needs to call in its manifest file. The hooks field lists the hooks used and the relative pathname of each new resource (e.g., fonts, icons, or .desktop files) within the app's filesystem. The app installer is then responsible for creating a symlink from the newly-installed app resource to the canonical location for such resources. Based on the examples in the hook documentation, there will be separate directories for these symlinks, so that new .desktop entries might be placed in /opt/click.ubuntu.com/.click/desktop-files/ as opposed to /usr/share/applications/.

Still to come

The Click format is still in rapid development; although the current app store contest ends in mid-September, there is no indication when the mobile app store itself will go live to the public. But that probably matters little while there are no Ubuntu Touch devices in widespread distribution.

There are still several unanswered questions, such as how best to support "fat" packages built for multiple architectures, and how to implement security signatures. For now, the selected approach to signatures is to use debsigs. Developers will sign their uploads to the app store, and the store maintainers will verify those signatures. When the app is pushed out to users, it will carry the store's signature, which is all that end-user devices are expected to verify.

The format may still undergo significant changes before any Ubuntu Touch devices hit the market, as may the upload and review processes used for Click apps. However, it does seem likely that the format will retain its roots in the .deb package format; the Click installer reuses dpkg code in a number of places, even if third-party apps are destined to remain in total isolation from the rest of an Ubuntu Touch system.