Development [LWN.net]

Collections in the XMMS2 music player

June 13, 2007

This article was contributed by Sébastien Cevey

The number of music players on Linux has been steadily increasing lately, but while these projects have been getting more and more polished, we have yet to see revolutionary improvements in terms of user experience. Indeed, the trend has been to borrow as many features as possible from other projects, rather than questioning the reasons behind their design.

This article describes XMMS2's attempt to address long-standing limitations of music players, through its new support for Collections.

Design Rationale

I have been concerned with the state of music players for a long time. Two years ago, I wrote a Manifesto for a Better Music Player. Although my ideas have evolved since then, the general conclusions of that article still hold.

One important argument I made is that the design of a music player should focus on the users' needs, rather than on a list of well-known features. All the traditional features (playlist, media library, cover browsing, etc) and hacks (play queue, random mode, etc) stem from the needs users have for:

playing music non-linearly
searching for specific media
browsing their media library
organizing their music

Non-linear playback was first introduced in a crude form as the "random mode", directly inspired from legacy CD players. iTunes later popularized its "Party Shuffle" mode, which solved the unpredictability of playback by maintaining a queue of randomly selected songs. What we are still waiting for, though, is a smarter mode that would also take into account beat, artist similarity, or other semantic information.

Music players that are based on a media library typically provide a search feature. Unfortunately, the power of the search function is often hindered by annoyingly complex forms used to choose the fields to query. Few developers seem to have noticed the success of Google's search interface: minimalistic, but enriched by rating heuristics and a rich syntax for advanced users.

The other axis required by our ever-growing music libraries is browsing. Media library browsing is always present in some form, although mostly simplistic and uninspired. When they are not cloning iTunes genre/artist/album filters or the browsing of cover art, most music players simply present the users with the list of all their media in a plain multi-column layout. Easy to implement, but hard on the eyes for the users. Interestingly, Foobar2000 (freeware) is the only popular player to allow a rich customization of the layout, which greatly improves readability.

The lack of features that help users organize their media library contributes to the difficulty of addressing the two previous issues. In the physical world, users can arrange their CDs spatially in their own personal way (by artist, date of release, mood, etc), set a couple of albums aside for playing at a party, or highlight their latest acquisitions on a shelf. This lets them build a cognitive map of the location of items. On computer-based music players, however, they are barely provided with the possibility to create playlists, possibly dynamic, but seldom integrated well enough to be used powerfully. Even bare files have richer organizational possibilities, using directories!

The reason behind these limitations is not that they are inherently unsolvable. The truth is that a lot of effort is required to implement new approaches in any of these fields. Experimentation, either conceptual or in terms of interface, is expensive.

The Collections Concept

The goal of Collections is to address this problem by creating a common abstraction layer. Search, browsing and organization all share one property: they act on subsets of the media library. Computers are especially good at handling sets, but music players haven't really exploited that fact yet.

A collection is defined as a subset of the media library. This set of media (songs) can be dynamic, for instance "All media by Kraftwerk released prior to 1980" or "All media added to the media library last week, except those by Justin Timberlake". A static set, for instance hand-picked media selected for parties, is just a special case of dynamic sets.

Note that a collection is not merely what some players call a "Smart Playlist" (or "Dynamic Playlist"). A "Smart Playlist" is only used to play an arbitrary list of media, while a collection is a generic representation of a set of media. For instance, this includes the results of a search, a filtered view of the media library, the list of tracks from a given album, etc.

Because a collection is an abstract representation, it can be used ubiquitously throughout all the features of the music player: browsing, searching in the media library or the playlist, enqueuing, jumping, etc. A collection can also be saved on the server, thus allowing the users to organize their music and reuse their selection in homogeneous and flexible ways.

Collections for the XMMS2 player

The XMMS2 project turned out to be the perfect ground to implement collections. Unlike its popular predecessor XMMS, XMMS2 hasn't gathered much attention yet. However, it features all that you would expect from a recent music player: a media library, support for many audio formats and multiple platforms (Linux, *BSD, OS X, Windows, etc), bindings for many languages (C, C++, Ruby, Python, Perl, Java), and a friendly community open to innovation.

In addition, the player was designed according to a client-server architecture, so that the server is responsible for all the boring work (audio decoding, media library management, tag extraction, etc), while any flavor of user interface can be implemented as a client connected to the server, possibly across the network.

Collections have been implemented in XMMS2 as a student project during the Google Summer of Code 2006, and finally merged into the stable tree on May 20, 2007 as part of the DrJekyll release.

Support for collections was implemented on the server as a layer above the media library, and playlists are exposed to the clients through a collections API. This API allows clients to save collections on the server, query the media library, enqueue the content of a collection, etc. Thus, although the user interface depends on the client, the server and the clients all share the same abstract representation.

Clients are also freed from the need to generate complex SQL queries themselves; instead, they can easily build a (DBMS-agnostic) collection and the tedious query is performed by the server. In addition, a parser is provided to generate a collection from a string with an enriched search syntax.

Collections make it essentially trivial to browse and search the media library. Moreover, advanced features are either natively available or very easy to implement: iTunes-like Party Shuffle, recursive filtering (e.g. search inside the playlist), display Top 10 or never played songs, changing the equalizer settings if the playing song is in a particular collection (e.g. "Jazz Vinyl rips"), etc.

Implementation

Strictly speaking, collections are implemented as a directed acyclic graph (DAG), each node of which is a collection operator. In fact, because the structure is recursive, each node of the graph corresponds to a collection. This model was chosen to emphasize the aggregated nature of users' music collections.

Collection operators come in four different flavors:

set operators
filter operators
list operators
reference operator

The set operators take an arbitrary number of operands and returns the collection obtained by applying the corresponding set operation to them. For instance, "any music by The Beatles or any music by The Rolling Stones". Available set operators: union, intersection, complement.

The filter operators enforce conditions on properties of the media; the resulting collection only contains the media that match the filtering attributes. For instance, "all the songs with 'stairway' in their title". Available filter operators: equals, match (partial matching of strings using wildcards), larger/smaller (for numbers), has (checks whether a property is present).

The list operators are a bit special. The basic list operator (called "idlist") does not accept any operands; instead, it simply generates the collection corresponding to the custom list of media it contains. Because list operators store static, ordered lists of media, they are used as playlists in XMMS2. Available list operators: list, queue (pop songs once they have been played), Party Shuffle (takes an operand, used to randomly feed the list with new entries).

The reference operator is simply used to refer to the content of a saved collection or playlist. For instance, "all the songs released in 2007 in the Foo playlist". A reference operator is also used to refer to the whole media library (all media).

Now, let's illustrate all this with a sample collection structure: [Collection Diagram] The nodes represent collection operators, while edges simply connect operands to operators.

Here, "All Media" is a reference to the whole media library, and we use a Match operator to only keep media for which the artist has a name starting by "A" (1). We then take the union (3) of this and the content of the "Rock 90's" saved collection (2). The result is passed as an operand to a Party Shuffle operator (4), which we save under the name "Interesting" (5).

When the user plays the "Interesting" playlist, songs are popped from the list as soon as they are finished, and new songs matching the operand collection (3) are automatically enqueued, so that the list always contains at least 20 items. This is specified by the "size" attribute of the Party Shuffle. Of course, the user can also edit the playlist and add tracks to it manually.

This is only one example of collections among many. As you can see, the modular structure of collections allows virtually unlimited possibilities. As such, they have been tightly integrated both on the server and in the client API.

On the server, a dedicated module is responsible for handling collection features. When a collection is queried, it serializes the structure into an SQL query, runs it in the media library and returns the matching media, either as a list of media ids or hashes containing the requested media properties. When a collection is saved on the server, it is added to the collection DAG and kept in memory while the server is running. On shutdown, the whole DAG is serialized into the database. Note that playlists are nothing but collections, albeit restricted to list operators and saved into a dedicated namespace.

In the client API, collections introduced many important changes. First, executing raw SQL queries has been deprecated; all queries are now to be performed using collections. Collection data structures can be built either using a set of dedicated functions, or by calling the collection parser on a string given by the user. Finally, many XMMS2 methods have been extended to support collections (e.g. to enqueue media) and new methods allow clients to query, save and retrieve collections from the server.

If you want to learn more about the concept of collections, please have a look at the collections concept page on the XMMS2 wiki. For more details about the implementation, check the collections design page and the API documentation.

Adoption and future directions

Several XMMS2 clients have started offering features based on collections, including Abraca (GTK2 client) and gntxmms2 (console client). Other clients have ported search and browsing to the collections API: Esperanza (Qt4 client), gxmms2 (GTK2 client) and the official command-line interface.

Hopefully, client developers will start exploring new directions now that collections are in the main release. The XMMS2 CLI client has already been scheduled for a full rewrite.

Several improvements are also expected to address current limitations of the collections implementation. One limitation is that all collections are treated equally as media sets; if a filter is applied on a playlist, the order and duplicated items will be lost. A smarter internal distinction between lists and sets inside the DAG is in the works. An ordering collection operator could then be introduced to transform a set into an ordered list, as well as an operator to select subsequences of such lists, similarly to SQL LIMIT operation. They could be used to create a collection containing the "list of the 20 most recently added media". The SQL query generator could also be further optimized, unless we decide to replace the database backend completely.

Collections have just made it into the official XMMS2 distribution, but people already use them through features like search, Party Shuffle or groups of songs saved in the media library. They are a powerful toy for developing new features in the clients and hopefully helping users organize and use their music library.

It's an exciting time to come up with fresh ideas in the XMMS2 world, and I hope the rest of the developers in the music player community will take the time to reflect on and discuss all these questions earnestly!

Comments (19 posted)

System Applications

Database Software

PostgreSQL Weekly News

The June 10, 2007 edition of the PostgreSQL Weekly News is online with the latest PostgreSQL DBMS articles and resources.

Full Story (comments: none)

Agile Database Refactoring with Hibernate (O'Reilly)

Gilad Buzi, Kelley Glenn and Jonathan Novich discuss the process of changing data models on O'Reilly. "In this article, we will show readers how to upgrade their faulty schemas and data models without affecting existing applications or processes. By using the latest technology from Hibernate (version 3.0 and up)--along with a combination of database views, stored procedures, and standard design patterns--application developers and data architects can repair a faulty data model, one piece at a time."