Leading items

The N9 and the future of MeeGo at Nokia

The story has been told many times at this point: just as work on MeeGo was reaching some sort of stable point, Nokia, one of the principal partners in the MeeGo project, went through a change in management, dropped MeeGo, and committed the company to a close partnership with Microsoft. MeeGo remains under development, but, with the primary handset manufacturer gone, the mood is somewhat subdued. Nokia's plan was always to produce one MeeGo handset, though; that handset has now been announced in the form of the Nokia N9. Is the N9 a one-off device, or might there be more where it came from?

If your editor had an N9 in hand (hint), he could write a proper review of it; in the absence of such luck he'll have to take the word of others. By all accounts, the N9 is a nice device indeed. The hardware is nicely designed with entirely respectable specifications. The software is said to be the realization of much of the potential that many of us have seen in MeeGo. All told, it seems to be a clear statement that, despite indications to the contrary, Nokia can indeed make an interesting, competitive handset in 2011. It's the touchscreen-based smartphone that Nokia desperately needed to make.

There have been some questions as to whether the system running on this phone is actually MeeGo or not. By the terms of the MeeGo specification, it is not; it retains too much of its original Maemo heritage. That said, Nokia did obtain permission to use the MeeGo trademark with this system, which has been dubbed "MeeGo 1.2 Harmattan," so it is, in some sense, MeeGo. One thing this version of MeeGo does have is Qt as the graphics toolkit, so it does show Nokia's intended direction for the MeeGo user interface. Of course, the use of proprietary applications at the "user experience" level has also been part of the MeeGo plan from the beginning.

Given how well this phone has been received, one might well hope that Nokia might reconsider its plan to move away from the MeeGo platform. After all, it has demonstrated that it can produce an interesting handset based on some version of MeeGo; why not continue, especially if the N9 sells well? Unfortunately, it does seem like Nokia is not hugely interested in making this phone a success. Developers will be understandably reluctant to put time into creating applications for a one-off device. The "check availability" page lets potential customers request a notification when the phone becomes available - but not if they are in the United States or much of Europe. And then there is the matter of that deal with Microsoft; the folks in Redmond may think that the billions of dollars apparently sent to Nokia should be enough to keep competing operating systems off Nokia's smartphones.

So the situation looks grim. Still, the world can be a surprising place, especially where corporations are involved. An awful lot could happen over the course of the next year. The N770 and N800 tablets did not have clear successors either - they weren't even phones. The N900 was expected to be a one-off as well. So history suggests that Nokia might yet see an interest in creating more MeeGo-based phones in the near future, regardless of what has been said in the last few months.

For those who like to read tea leaves, there is this posting from Nokia's Quim Gil. Quim says that there are four important software components to the N9: Qt, the Linux kernel, WebKit, and the "swipe" user interface. Nokia, he notes, continues to invest in the development of all of these components. He says:

One could speculate endlessly on what "a form or another" means; it might not be handsets. Regardless, it seems possible that MeeGo is not entirely dead at Nokia. A breeze from the right direction might just get things moving in a more interesting direction again.

That breeze could come from the corporate direction; Stephen Elop, having presided over a nearly 50% drop in Nokia's stock price in just a few months, might struggle to retain his position. For a number of reasons, Windows has struggled to gain any serious success in the handset market; that track record may not change in the next year or so. For some wilder (and voluminous) speculations, see this article suggesting that Microsoft's acquisition of Skype has turned the carriers against Windows-based handsets and that Nokia is in the middle of a big about-face.

What will come of all this is unknowable; expecting rational, consistent, or predictable behavior from corporations is a good way to be surprised. But Nokia may just change its mind about MeeGo phones again; we may also see other manufacturers develop an interest in Linux-based alternatives to Android. All we really know is that the N9 has seemingly shown the world how good a MeeGo-based handset (for some value of "MeeGo") can be. Hopefully good things will come from that.

Comments (30 posted)

Echoprint: Open acoustic fingerprinting

Acoustic fingerprinting has been given a tremendous boost by the mobile smartphone business. You have probably seen the basic scenario in television commercials, if not in person: a user holds up a phone to capture a few seconds of audio playing nearby, and the application computes a "fingerprint" of the track, which is then used to query a remote database for the mystery artist and track name. The space has been dominated by proprietary software, but a new — and open source — project was unveiled last week, named Echoprint.

Fingerprints on the databases

Despite the name, acoustic fingerprinting has little in common with hash-based digital fingerprinting techniques used to detect alteration of a file. While a hash function is sensitive to changes in individual bits, an acoustic fingerprint function must robustly analyze the way the audio sounds, in a manner independent of the codec used, bitrate, or even static and ambient noise. Acoustic fingerprints focus on extracting perceptual data from the audio track, such as its tempo, average spectrum, and pattern of recurring tones. The canonical use is to discover the track information of an unknown audio clip, but other uses are possible as well, such as finding similar-sounding music (on any number of factors supported by the algorithm).

The proprietary software market is home to several acoustic fingerprinting services, most famously Shazam, SoundHound, and Gracenote. Gracenote is known to many in the free software community due to the controversy that erupted a decade ago when its corporate parent suddenly restricted the usage of CDDB, its user-built compact disc identifying database. Many felt betrayed by the policy change, because the CDDB data was submitted voluntarily by users when playing or ripping CDs, not entered or harvested by CDDB's owners themselves. The database was an early example of Internet crowd-sourcing, and many saw the sudden cut-off of access as outright exploitation of their effort.

Fast-forward to 2011, and most open source applications use competing, "open content" services instead, such as MusicBrainz, which is managed by the 501(c)(3) nonprofit MetaBrainz Foundation. For the past several years, MusicBrainz has supported acoustic fingerprinting through the closed-source MusicDNS service provided by MusicIP (later renamed AmpliFIND).

Although MusicBrainz had a perpetual contract with AmpliFIND for the service, it was never considered a good fit, since MusicBrainz's social contract requires it to remain "100% free". Recently, a handful of open source acoustic fingerprinting projects started picking up steam — such as Lukáš Lalinský's Acoustid — and MusicBrainz decided to start looking for open source and open content replacements for MusicDNS. Around the same time, the team at acoustic fingerprinting startup Echo Nest decided that its best strategy was to take its entire product open source and attempt to commoditize acoustic fingerprint services, rather than attempt to take on the entrenched players head-to-head.

Echo Nest and MusicBrainz had collaborated in the past on projects such as Rosetta Stone — a utility to match artist and track IDs between various music services' ID databases — so the mutual decision to launch Echoprint as an open project and begin integrating it with MusicBrainz was a good fit from both perspectives. But it did not hurt that AmpliFIND also sold off its intellectual property holdings — including MusicDNS and the portable unique identifier (PUID) database — to none other than Gracenote.

The Echoprint release

The Echoprint system consists of three components. The Codegen fingerprint generator takes an audio file (or audio sample) as input, and generates a fingerprint based on the Echo Nest Musical Fingerprint (ENMFP) algorithm. The Echoprint server maintains a database of fingerprints indexed to track information, and supports remote queries as well as inserting new fingerprints and tracks. The Echoprint database itself contains publicly-accessible track and fingerprint data. The database contains fingerprint codes for the entire duration of each track, but as in most acoustic fingerprinting techniques, only a shorter segment is usually sent for comparison. Echo Nest claims that Echoprint provides accurate matches for fingerprint blocks computed from samples of at least 20 seconds in length.

In practical usage, an application would sample audio (either captured or from a file), use the Codegen library to compute a fingerprint, and query a compatible Echoprint server. The server would return any matching track records in JSON format. Alternatively, if there are no decent matches, the application could submit its fingerprint information to the server's database along with track metadata acquired through some other means.

The code for Codegen, the server, and various utilities (including an example iPhone app) are hosted at GitHub. The Codegen application and shared library are available under the MIT license, while the server (which is based on Apache Solr and Tokyo Tyrant) are under the Apache License 2.0.

The public Echoprint database is provided under its own terms, dubbed the "Echoprint Database License". It allows for commercial and noncommercial usage, and requires that anyone who downloads the data and adds to it contributes the additional data back to Echo Nest. That clause is something less than a Creative Commons-style "Share Alike" requirement, because it requires sending the data to Echo Nest alone. The preamble to the license seems to indicate that all such contributions will be shared with the public, but Echo Nest assumes no obligation to share the data. The initial release is "seeded" with approximately 13 million fingerprints generated from online music vendor 7Digital's digital holdings, with metadata provided by MusicBrainz.

There are some other potentially worrisome terms in the agreement, including a requirement to use Echoprint "powered by" logos in any application that accesses the data. In addition, the agreement is not clear about how Echo Nest can modify or terminate the agreement down the road. For those who were burned by the CDDB debacle, this agreement should give them pause as it is not at all clear that the same couldn't happen with the Echoprint database.

At the moment, Echo Nest has not published the details of its algorithm in a form suitable for casual reading. The source code for Codegen is provided, of course, but a white paper is supposed to be released shortly that will explain the process at length. Unfortunately, the current legal documents do not explicitly address patent grants in relation to the software (the MIT license is very brief), which might concern some developers. Acoustic fingerprinting is a patent-laden field, and indeed a little searching reveals several relevant filings in the name of Echo Nest and its founders Brian Whitman and Tristan Jehan. On the plus side, all of the proprietary acoustic fingerprinting services are in roughly the same position.

Currently Echo Nest's own "song/identity" server is the only up-and-running Echoprint database, although obviously any application authors could set up their own servers for testing purposes. The Codegen command-line application will build on any reasonably modern Linux system; the only significant dependencies are TagLib, Boost, and FFmpeg. The application generates a fingerprint from a file argument (optionally followed by a start time and duration, both in seconds). The output is a JSON object including ID3 tag information from the file and a base64-encoded representation of the fingerprint. This output can be posted directly to the Echo nest server with cURL or a similar tool, as documented in the Codegen README file.

Play or Pause

MusicBrainz's Robert Kaye said that the project plans to retain support for PUIDs and MusicDNS in the MusicBrainz database for the foreseeable future (or until "people pester me to get rid of it."). The project is running a test server that uses Echoprint in lieu of MusicDNS, but there is no time frame to add tables to the main database to support Echoprint.

Kaye said that he expects more tuning to be done to the Echoprint product before it is ready for widespread adoption, but he observed that "critical mass" is the most important factor — meaning support in client applications and a sizable database of reliable fingerprints. The 13 million tracks pre-loaded with 7Digital's help may sound like a lot, but for comparison, Shazam claims more than one billion songs in its database to have identified more than one billion songs.

Given the number of open source audio projects that use MusicBrainz, it is safe to say that Echoprint has its foot in the door. It is the first entirely open source acoustic fingerprinting system to hit the market in "ready to use" form, so it may spawn considerable development of song recognition in open source mobile applications. Without the burden of licensing fees, the technology could spread beyond stand-alone song-recognition-apps, open or closed.

Nevertheless, Kaye emphasized that MusicBrainz post-MusicDNS move is meant to make the project agnostic to acoustic fingerprinting algorithms. Acoustid is still in active development, too, has documented the details of its algorithm, and does not require changing the MusicBrainz database format for support.

Whether the two fingerprinting techniques overlap, complement, or compete may ultimately be up to the users to determine. Echoprint is so new that it is difficult to predict where it will go from here. The MusicBrainz support is naturally a big boost, but better technical documentation and clarification of the fuzzy legal questions may be required before application authors can be expected to pick up the technology in large numbers. But without doubt, it is poised to fill a visible hole in open source mobile software. An open solution that works well with the crowd-sourcing techniques needed to build the fingerprint database will likely have staying power in a niche with so many similar proprietary offerings.

Comments (11 posted)

Who is that code for?

Free software development projects put a lot of thought and energy into what they are trying to create. Some projects also put effort into communicating their goals clearly to the world. Arguably fewer projects ponder the question of who their users are. But a project's vision of who it is developing for deeply influences the code it produces and the way it manages its releases. Some high-profile projects have been the target of extensive criticism storms recently; arguably the real problem in these incidents has been confusion over who the target users are. In particular, when groups find that they are not seen as users by a project that they had thought was keeping their interests in mind, they tend to get upset.

The recent examples are fairly obvious. Consider the firestorm which has resulted from the Firefox 5 release, the quick abandonment of Firefox 4, and the prospect of the same thing happening again in the near future with Firefox 6. For individual desktop users, this release scheme may work just fine, depending on how many extensions break (your editor reports with dread that this update broke Firemacs - and, thus, Emacs keybindings - in the browser). If, instead, you run a corporate network where software must go through an extended approval cycle prior to deployment, losing support for a major release constitutes a big setback. For "enterprises" with this type of policy, Firefox is increasingly unsuitable; it is thus not surprising that a lot of complaints have come from that direction.

The response from Firefox developers has been clear: the Firefox project is not targeting enterprise users. Firefox is successful because it was adopted by individuals, not their employers, and the plan is to continue to target those individuals. This view can be seen in the draft vision statement under consideration by the project now:

Making life easier for corporate information technology managers does not really figure into this plan. One can argue - as many have - that this approach can only serve to cement the role of Internet Explorer in corporate settings, but that is the choice the Firefox developers have made.

The other recent, high-profile misunderstanding with regard to target users is the GNOME project, and the GNOME 3 release in particular. There can be no doubt that some users are upset by the changes brought by GNOME 3 and GNOME Shell, even if there is disagreement over how large that group is. GNOME developer Bastien Nocera recently made it clear that at least some of those people are not in the GNOME project's target audience:

The project has made numerous other decisions which implement that same view of its user community. In this case, many of the affected users felt that, once upon a time, they were somebody that the GNOME project cared about. If GNOME has ever truly targeted users who care about their terminal emulator, it has since seen greener grass elsewhere and left those users behind. Those users have made their feelings known; at this point, the best thing to do is almost certainly to wish GNOME success with the users it is targeting and, if GNOME is no longer a suitable working environment, move on to something else that is.

The community is full of examples of how a view of the users affects the way the code is managed. Consider PostgreSQL; this is a project which does see "enterprise" users as interesting. PostgreSQL developers also assume that their users care a lot about their data. The results include a highly conservative, review-intensive patch merging policy and a five-year support policy. PostgreSQL 8.2, released in late 2006, will continue to be supported through the end of this year. The wait for new features may be longer than one might like, but rapid feature development is usually not at the top of the "must have" list for users of database management systems.

For a completely different view, see OpenBSD, which only grudgingly admits the existence of a user base outside of its development community at all. OpenBSD leader Theo de Raadt put it clearly a while back:

This position evidently works for this particular project, though it does lead to occasional misunderstandings when users forget their place.

Other examples abound. Git and Mercurial are similar tools with different views of their users - a difference especially felt, it seems, by Windows users. Fedora and Red Hat Enterprise Linux share a common ancestor and a lot of code, but they are developed for different people. Prior to the early 2.6.x days, the kernel community (mostly) saw itself developing directly for end users; the even/odd development process was a direct result of that perception. Current kernels are developed (mainly) for distributors, with an associated focus on frequent stable releases and rapid merging of features. Pulseaudio targets different users than JACK. And so on.

It is a rare project that can be all things to all people; projects which try often do not succeed. A clear idea of who the users are can do a lot to focus a project's activity, attract like-minded developers, and increase adoption. So projects need a good idea of who they are developing for; they also need to communicate that vision clearly. If a project does not prioritize the needs of specific users, it is best if those users understand that before they invest a lot of time into the software. Users, in turn, should pay attention: just as we would not expect a video editor to do source code highlighting, we should not fault a project which is clearly targeting a specific group of users for not meeting the needs of others. We are a large and rich community; if one project does not meet a specific set of needs, there are probably several others with a different and more suitable focus.

Comments (54 posted)

The next weekly edition will be published July 8

The US Independence Day holiday falls on Monday, July 4, so next week's weekly edition will be delayed until Friday, July 8 (in the early hours UTC for those who can't wait). Too much sun and food, along with some fireworks, are in our plans for Monday, but whether you celebrate the holiday or not, we wish all of our readers a happy 4th of July.

Comments (none posted)