Development

Ubuntu's app store and package format

As Ubuntu continues its march toward Ubuntu-based mobile devices, it has undertaken a number of efforts designed to attract third-party mobile app developers. The latest development is the beta launch of the project's mobile app store, which is tied with an app-development contest slated to run through September 15. Perhaps more interesting in the long run, though, is the publication of the specifications for how the platform's mobile apps are to be packaged. The format is called Click, and it is related to the .deb package format used for standard Ubuntu packages—but with a few key changes.

Martin Albisetti announced the opening of the app store and the publication of the package format documentation in an August 30 blog post. The Ubuntu Touch store is only accessible by registered developers for the time being; they can upload packages to the Ubuntu "MyApps" service. Uploads are then subject to a manual review process before they are published.

The process is much the same as the one offered to third-party developers of desktop applications seeking publication in the Ubuntu Software Center service, except that mobile apps can only be written for the APIs supported by the Ubuntu SDK. At the moment, just two app frameworks are supported—QML and HTML5—with HTML5 support relying on the Apache Cordova library layer.

Support for compiled apps is apparently under discussion, although no clear timetable has been publicized. One open question on that front is how to encourage app developers to license their code under GPL-compatible terms; for the contest, this is important because the winning entries will be included in subsequent Ubuntu Touch builds. Since apps written in QML and HTML are (at least in theory) their own source code, not handling binary apps neatly sidesteps the question. For the time being, the app store does not support selling apps, although that feature is expected to arrive eventually as well.

Click, here

To put an app store in place, of course, one must have some idea of what format apps will take. The format defined for Ubuntu Touch is called "Click," using the .click file extension—although there are copious caveats that the actual name of the format could still change before the Ubuntu Touch app store opens to the public. Click is derived from the .deb Debian package format, and in most ways a .click file is a valid .deb file too. Consequently, the distinct file extension helps to make it less likely that a user will attempt to install a .click app using dpkg or Apt.

There is HTML documentation on the format available, but a newer revision of the specification can be found in the Click source repository on Launchpad. There was a lengthy debate on the ubuntu-devel mailing list about the suitability of existing package formats, such as Android's .apk and cross-platform efforts like Listaller. In the end, Colin Watson said that:

Similarly, by basing Click on the Debian package format already in use, many of the existing package tools can be reused. Like a .deb, a .click package is an ar archive, which itself contains "control" and "data" tar archives. The control archive includes package management information, while the data archive holds the app's filesystem.

Click apps are designed to be self-contained; the directory tree in the package's data archive is unpacked into a single directory and cannot access the filesystem above its own root. The app filesystem is uncompressed and stays on disk, though—as opposed to being stored as a compressed image and loop-mounted.

The control archive must contain both a control file and a manifest.json file. The manifest includes several required fields like app name, app version, and installed size, and may include several optional fields as well (such as maintainer information, a reference to the app's icon, or descriptions in various languages). The control file duplicates some of this manifest information for use by the package-management tools, and it also includes a field listing the version number of the Click format used.

Because Click apps are meant to be stand-alone entities, some of the manifest and control fields diverge somewhat from their equivalents in a standard Debian package. The app name is specified in reverse-network-domain name format like com.example.catpictures, which is the same scheme used in D-Bus, in Android app naming, and in plenty of other places. The "framework" field is a simplified version of the Debian package format's way to specify dependencies. Click apps cannot have dependencies on anything other than base system services. In other words, they cannot introduce dependencies on other apps or on components installed through other means, and they cannot depend on specific versions of system libraries (which in the worst-case scenario could make dependencies unresolvable). As such, for now the only valid value for "framework" is the current Ubuntu Touch SDK framework. Perhaps in the future, different versions of the framework will need to be designated, or non-phone form factors will require separate frameworks, but this has not been explored in the documentation.

But the fact that Click apps cannot have dependencies has benefits beyond simply avoiding dependency hell. Since Click apps are isolated from system packages, the app installer can install them without querying or searching the system's dpkg database—which can be quite slow relative to unpacking a tar archive to disk.

Another—perhaps major—distinction from the Debian package format is that maintainer scripts (such as preinst, postinst, or prerm) are forbidden. This is to isolate Click apps from the system; since they are intended to be self-contained, they should not generally trigger any system-level events which could potentially be exploited. Not allowing them makes the task of auditing app submissions considerably simpler. There is one exception to the general maintainer-script moratorium: apps can include a preinst script that prevents the package from being installed directly with dpkg.

Hook 'em

On the downside, however, one useful feature lost in disabling maintainer scripts is the ability to easily hook into system services at install time. Even for self-contained apps, there are a number of system packages that need to be informed of the installation, such as updating the icon cache to include the new app's icon, locating the .desktop launcher to add to the app menu, or registering a service with D-Bus.

But Ubuntu Touch still needs to alert system services to the app-installation event somehow. Currently the plan is to use a "hook" mechanism inspired by dpkg's triggers. Triggers are calls out to other programs that are generally placed in a Debian package's postinst script. A trigger mechanism could, for example, tell Fontconfig to update the font info cache and pick up a newly-installed font. The problem with this approach is that Click packages can be installed to any directory on the system. That is to say, there is no fixed location for them; the directory name for an app is created based on a hash. This approach enables the updating of apps by installing the new version in a separate directory, then garbage-collecting the old one. The unpleasant side-effect is that, in this example, Fontconfig needs some way to learn which directory it should scan to update its cache. Dpkg triggers cannot pass user data, so apps cannot simply report their location through the trigger mechanism.

The "hooks" solution works by enumerating a stable, well-known set of services that need to perform such post-app-installation updates. Each app is responsible for listing the hooks that it needs to call in its manifest file. The hooks field lists the hooks used and the relative pathname of each new resource (e.g., fonts, icons, or .desktop files) within the app's filesystem. The app installer is then responsible for creating a symlink from the newly-installed app resource to the canonical location for such resources. Based on the examples in the hook documentation, there will be separate directories for these symlinks, so that new .desktop entries might be placed in /opt/click.ubuntu.com/.click/desktop-files/ as opposed to /usr/share/applications/.

Still to come

The Click format is still in rapid development; although the current app store contest ends in mid-September, there is no indication when the mobile app store itself will go live to the public. But that probably matters little while there are no Ubuntu Touch devices in widespread distribution.

There are still several unanswered questions, such as how best to support "fat" packages built for multiple architectures, and how to implement security signatures. For now, the selected approach to signatures is to use debsigs. Developers will sign their uploads to the app store, and the store maintainers will verify those signatures. When the app is pushed out to users, it will carry the store's signature, which is all that end-user devices are expected to verify.

The format may still undergo significant changes before any Ubuntu Touch devices hit the market, as may the upload and review processes used for Click apps. However, it does seem likely that the format will retain its roots in the .deb package format; the Click installer reuses dpkg code in a number of places, even if third-party apps are destined to remain in total isolation from the rest of an Ubuntu Touch system.

Comments (10 posted)

Quote of the week

— Bradley Kuhn

Comments (none posted)

SQLite 3.8.0 released

Version 3.8.0 of the SQLite database management library is out. "Do not fear the zero in this version number! The 3.8.0 release might easily have been called 3.7.18 instead. We decided to increase the minor version from 7 to 8 because of the rewrite of the query planner. But the software is quite stable and is ready for production use." Along with the new query planner, this release adds partial indexes and more; see the changelog for details.

Full Story (comments: none)

Apache Cassandra 2.0 released

The Apache Software Foundation has announced the release of version 2.0 of the Cassandra distributed database system. "New features in Apache Cassandra v2.0 include lightweight transactions, triggers, and CQL (Cassandra Query Language) enhancements that increase productivity in creating modern, data-driven applications."

Comments (none posted)

Gtk-gnutella 1.0 stable has been released

Version 1.0 of the peer-to-peer file sharing client gtk-gnutella has been released. The release notes list a handful of new features, such as the ability to define maximum lease times for UPnP and NAT-MPM mappings, persistent DHT keys, and the ability to prioritize the rarest chunks of a file for download first. Nevertheless, the announcement also highlights what comes next, saying "this new release is an important milestone because it is the last version that will be mono-threaded. Future releases will use a new runtime that will allow multiple threads to run concurrently, to be able to exploit common multi-core systems nowadays."

Full Story (comments: none)

Rhythmbox 3.0 available

Version 3.0 of the GTK+ audio player application Rhythmbox has been released. Changes include the migration of plugins to Python 3, support for "composer" tags in audio metadata, separation of constant- and variable-bitrate encoding options into separate presets, and numerous UI improvements.

Full Story (comments: none)

MediaGoblin 0.5 released

Version 0.5 of MediaGoblin is now available. This release takes initial steps toward true network federation by supporting the pump.io flavor of OAuth, and it adds support for logins using OpenID or Mozilla Persona. Also new are notifications of new comments, comment previews, and Unicode filename support. Finally, starting with this release, all media types and authentication schemes are supported via plugins, which should make the system easier to extend.

Comments (none posted)

Plasma Active 4 - ready when you are (KDE.News)

KDE.News announces the release of Plasma Active 4, KDE's interface for consumer electronics devices like tablets, smartphones, set-top boxes, and more. The release has improvements to the File application, a better on-screen keyboard, ownCloud integration, is based on the Mer core, and has additional new features and improvements described in the post. "Plasma Active 4 is a stabilization and performance release, the result of developers focusing on fit and finish throughout. This release is intended to complete the evolution of Plasma Active to a polished product relative to its proof of concept first release. The emphasis is now on providing a solid foundation for third party applications, additional content and device adaptations."

Comments (2 posted)

Development newsletters from the past week

• Caml Weekly News (September 3)
• What's cooking in git.git (August 28)
• What's cooking in git.git (September 3)
• Haskell Weekly News (August 28)
• OpenStack Community Weekly Newsletter (August 30)
• Perl Weekly (September 2)
• PostgreSQL Weekly News (September 2)
• Ruby Weekly (August 29)
• Ruby Weekly (September 5)
• Tor Weekly News (September 4)

Comments (none posted)

ALERT Project concludes successfully

KDE News presents a report on the successful conclusion of the ALERT Project. The project aimed to help open source developers to work more effectively and to produce better software by improving bug tracking, resolution and software quality tools. "KDE's involvement in the early stages was mostly in the form of contributing to documents describing the problems that should be solved. For example, Bugzilla is a popular tool to track bugs, but for large projects such as KDE there are problems with duplicate reports and with reports being filed to the wrong team (it is not always easy for a user to understand that an apparent problem with a web browser failing to show pages is actually due to a separate software handling wireless connections). With this in mind, the KDE experts decided to focus on Solid (the KDE software components dealing with hardware interaction) as a base for KDE's testing of the ALERT software."

Comments (5 posted)

KDE Release Structure Evolves

KDE is changing the way it does releases. "After the 4.x series, the KDE Release Team expects the releases of the Frameworks, Workspaces and Applications to diverge. Separate release cycles will benefit both users and developers. Individual components can skip releases if they require a longer development cycle. Separate cycles will encourage developers to have an always-releasable master while work goes on porting to KDE Frameworks 5. There is more emphasis on continuous integration and other automated testing to improve development work."

Comments (none posted)