Application metadata with AppData

GNOME is deploying GNOME Software, an "app store"–style software manager, with the GNOME 3.10 release. The goal is to facilitate finding an application to install by scouring a catalog of human-readable descriptions and illustrative screenshots—in contrast to the raw details exposed by a traditional Linux package manager, which can get in the user's way. One challenge that might not be immediately obvious is collecting and collating the descriptions for all of the applications in a standardized fashion. To that end, there is the newly-drafted AppData specification.

The AppData specification is the brainchild of GNOME Software developer Richard Hughes. GNOME Software is meant to complement a regular package manager, not to replace it. The idea is that GNOME Software will help users find an application when they know what they want to do, but they do not know what applications are available. So they may browse by category, look at popular applications, or search, but in all cases they rely on a well-written description of the application to guide them. For example, users might know they want a tool for scanning photographs, but might not know—or care—that such a tool involves SANE. These well-written descriptions are not easy to come by, however, nor are they readily available to import in bulk into GNOME Software.

In August of 2013, Hughes discussed several potential sources for the human-readable application descriptions, such as the .desktop files or Description of a Project (DOAP) files released by the application project, descriptions found in distribution packages, and "About" text from within the application itself. Each had drawbacks.

For instance, DOAP files are by definition one-per-project; thus LibreOffice has a single DOAP file, but GNOME Software would want separate descriptions for the office suite's Calc and Impress components—either one of which might be the application that the user is searching for. Package file descriptions and "About" text are inconsistent and frequently too terse to be helpful to an unfamiliar user in search of an application to install. Furthermore, most of the existing sources of human-readable descriptions fail to make room for multiple translations of the file to exist simultaneously, so it was almost a given that some form of new file would need to be created.

Hughes eventually drafted the AppData specification, with the intention that it would be usable for projects in addition to GNOME Software itself. Although the specification is hosted on the Freedesktop.org site, it is not currently a formal specification (to the extent that Freedesktop.org is ever "formal"). The file format is XML, and upstream projects can provide their own AppData files to be installed in /usr/share/appdata/.

The format includes an <id> element that should specify the name used in the application's .desktop file, a <licence> element that should specify the license on the AppData text itself (not the license of the application, which is found in the software package), a <description> element for the main application description text, plus <url> for the project's homepage, <updatecontact> for a contact email address, and <project_group> to associate an application with an umbrella project. The file can also link to image URLs with one or more <screenshot> elements. Subsequently, Hughes was asked by Debian's legal team to add a field for a copyright statement, which is necessary to comply with some software licenses.

The hard part: people

The format is simple enough, but the mere existence of a simple file format does not write project descriptions. For that, Hughes turned to the public to solicit contributions. The specification page includes a list of suggestions for writing a helpful description, such as ensuring that the description says what the application actually does, not what it is planned to do. Authors are encouraged to avoid lists where possible, avoid references to trademarked competitors by name, and told: "don't mention what language an application is written in, users don't care".

On August 29, Hughes started a campaign on his blog to get application developers, package maintainers, or knowledgeable volunteers to write descriptions and create AppData files. Over the course of the next month, that campaign evolved into a formal goal that all GNOME applications would provide an AppData file by the release of GNOME 3.14. An even bigger target soon followed: that all major end-user applications shipped in Fedora 20 would include AppData.

Gauging the progress of the campaign can be a little tricky, as the GNOME and Fedora goals overlap, but are (of course) tracked separately. In a series of posts on his blog, Hughes highlighted high-profile applications that were missing AppData or had incomplete AppData files (where "complete" means having both the human-readable description and at least one screenshot). As of September 27, there were still three dozen or so important applications with incomplete entries. There is a Google spreadsheet that tracks the current Fedora application list (which as of press time includes 17 entries, many of which were not listed in the blog post).

The list of incomplete or missing AppData files does appear to be shrinking, however. But the files do need some further care even when they are complete. There is a validation tool to check the syntax of the files and report formatting problems. GNOME also maintains a list a list of the 3.10 applications that have AppData files but have not passed validation.

Use your words

Aside from validation, one of the biggest challenges will be getting the application descriptions—and in many cases, the screenshots—translated into as many languages as possible. One of the benefits of the AppData specification is that (unlike DOAP), it allows multiple translations of the metadata file. Both ITS Tool and intltool are supported paths to translation; projects can ship their AppData as an .xml.in file for use with intltool or use the official appdata.its template for ITS Tool. GNOME maintains a to-do list on the AppData goal page of the GNOME wiki, which indicates not only which applications still lack AppData files, but which are still missing translations.

It may seem like a lot of work has gone into AppData for what is essentially metadata that a lot of experienced users already know. But that is part of the problem: someone who has been using desktop Linux for years likely knows what the evolution and aisleriot packages install. Someone who has just discovered them through Yum, Synaptic, or another package manager would have a hard time guessing.

GNOME and Fedora seem to be on track for adopting AppData as the preferred mechanism for delivering this sort of metadata, but Hughes is interested in making it a standard that upstream projects use, rather than an add-on supplied by packagers. It will be interesting to see how that process goes. There is certainly a need; the same problem is faced by other popular desktop distributions, but none of them have attempted to formalize the process. Ubuntu, for example, uses the Debian package description format implemented by the Debian Description Translation Project. These descriptions do not come with a formal file format (they exist primarily in the distributions' databases), although they are quite similar to what is asked for by the GNOME AppData effort.

Neither Debian nor Ubuntu pushes upstream projects to supply descriptions themselves (although the comments on Hughes's blog suggest that a number of AppData entries made use of the existing Debian/Ubuntu package descriptions). But both do offer web-based tools to facilitate making translations. Perhaps some cross-project cooperation is plausible; users would no doubt be happy as a result, with the prospect of a major reduction in the amount of head-scratching required to decipher the peculiar packages names that are so popular in free software.