LWN.net Weekly Edition for June 25, 2009

Amarok weathers its own storm of reactions

"It's our KDE 4.2," said Amarok developer Jeff Mitchell. He was referring to version 2.1 of the popular media player, including the recent 2.1.1 maintenance release. He was implying, too, that the Amarok 2.0's reception mirrored that of KDE 4, provoking hostile early reactions that were only quelled as later releases answered the worst of the complaints. It's an analogy that seems fully borne out by a hands-on look at 2.1.1's basic interface and customization features.

The analogy is to the hostile reception that KDE 4.0 received when released in January 2008. Despite warnings that the release was not intended for general use, distributions packaged KDE 4.0 as soon as possible. Almost immediately, users began complaining about the changes in the interface and the lack of features found in earlier releases. These complaints subsided only when the 4.1 and 4.2 releases re-introduced missing features and more customization, although you can still hear rumblings whenever KDE 4 is mentioned online.

"The development process of Amarok 2 has followed a similar path," said Mitchell. Like KDE, "We had visions for where we could go with Amarok, and a code base that had sprawled from Amarok's much more limited and simple beginnings and was in no shape to take us there. So, much like KDE 4, Amarok 2 was not a simple porting of Amarok to Qt4 and KDE4libs; it was an almost total rewrite."

A major part of this rewrite was a transition to the new technologies of KDE 4.0. The switch to Phonon, KDE's multimedia API, allowed Amarok to stop maintaining its own media engines and, in 2.1, to add a graphic equalizer to the settings dialogue. In much the same way, Amarok now includes an embedded version of Plasma, KDE's core desktop technologies. This change allows the context features in Amarok's middle pane — for instance, the lyrics and artist information drawn from the web — to be written as Plasma applets. The integration of such KDE 4 technologies improves performance and reduces the code base that needs to be maintained while adding new functionality to the KDE 4 technologies themselves.

In total, Amarok 2 added some 200,000 lines of code. Since neither the project, nor any distributions, packaged the pre-releases, Mitchell admitted that the additions were not thoroughly tested. However, the project decided to release in December 2008, mainly because "motivation really tends to falter when you develop with no end in sight. Releasing 2.0 invigorated us, since we could finally get out of our endless feature freeze and commit huge new chunks of functionality, [and] get more bug reports and patches sent in."

The downside of releasing 2.0 was a reception almost identical to that received by KDE 4.0 — a similarity that commenters picked up on almost immediately. If you do an Internet search, you will find no shortage of complaints about the change of interface from the previous 1.4 release. Complaints about a missing playlist editor, statistics, sound equalizer, and other features were almost as common.

This reception was not completely unexpected by the project, according to Mitchell. Released on 4 June 2009, version 2.1 was intended "to be our first user release," Mitchell explained. "So far, this prediction has come true. We, the developers, feel it is far more polished and that it is ready for general consumption; users have responded in kind, reversing the scads of negative 2.0 reviews with (as far as we've seen) generally very positive 2.1 reviews."

He admitted that the interface continues to receive complaints and might need work. On the whole, though, Mitchell described the Amarok team as already looking towards new developments, such as using Solid, KDE's device integration framework, for detecting portable media, and integrating with Strigi, KDE's desktop search daemon. Other upcoming features will include breadcrumb navigation and new context applets for YouTube and Flickr.

The new interface

For those who want to test Mitchell's assertions for themselves, Amarok 2.1.1 is already available for most major distributions and many minor ones. In many cases, though, you need to look outside the official repositories to find it. It is available, for example, from Launchpad for Kubuntu, and the Experimental repository for Debian.

Those used to the 1.4 Amarok window are likely to find the changes in the interface overwhelming at first. Rather than displaying sources and auxiliary information in a single pane on the left, Amarok 2.1.1 now spreads the same information over the first two panes on the left, leaving only the third for playlist tracks.

This layout is not ideal from one perspective. In the playlist, the artist and album are listed as a header, rather than repeated for each track, which can be inconvenient in a long list. Similarly, track details and collection sorting are available only from context menus that at first you might miss.

However, from another perspective, the layout removes unnecessary information while making it still available. I am assuming, of course, that most people are already familiar with the artist and album, and only occasionally want to look at cover art.

Moreover, the middle context pane, with its applets in tabs along the bottom, delivers far more information than 1.4, including lyrics and Wikipedia information. The applets are not always perfect — for example, if the Wikipedia applet hits a disambiguation page first, clicking the correct link opens it in your web browser rather than in Amarok. But the applets are welcome all the same, and with these additions, some information had to give way.

On the whole, I think that Amarok has prioritized the information correctly, even if you do risk occasionally overlooking a feature. Besides, if you are uninterested in the context information, you can always adjust the size of the panes so that the middle one is hidden and Amarok looks more as it did in version 1.4

Advanced and customizing features

One feature that you might overlook at first is the PopUp Dropper (PUD). If you drag a track from the collection pane to the playlist, PUD appears as the cursor moves across the middle pane. If you drop the track on Append to Playlist or Replace Playlist, that action is immediately carried out. However, you can also hover the cursor over More, and PUD's menu expands to reveal less common options. Although the feature may not be strictly necessary, it's a convenience that I wouldn't mind seeing in ordinary file managers — especially when using a small screen like those on a netbook, where it could eliminate the need to scroll.

Other features that are tidied away until you discover them expand Amarok's basic functionality. Depending on how your distribution ships Amarok, you might be able to enable more Internet Services from Settings -> Configure Amarok -> Internet Services. These services are more numerous and more varied than in version 1.4.

For still more variety, you can check Tools -> Script Manager to add extra features to the basic installation. The available scripts include one to to stream audio books from Librivox, as well as a service that lists Internet radio stations. Click the Get More Scripts button, and you can download additional scripts, such as local audio streams, as well as applets or extra functionality, such as a musical alarm and a random album player. Activating these scripts will generally require restarting Amarok.

A new feature in 2.1.1 is the ability to created biased playlists from a random collection. Using tracks' metatags, you can set the proportion of certain characteristics that you want. For example, you might want 25% of the playlist to include a certain artist, or 10% of a particular album. Alternatively, you might use the Fuzzy Bias feature so that the list includes only songs that are less three minutes long, or were released in a certain year. With such features, Amarok seems more customizable than ever.

In a nut shell

While some features from earlier releases are still missing in 2.1.1, whether you miss them is largely a matter of preference. Some might miss the setup wizard. Others might miss the ability to use a database other than MySQL, although, since the rest of KDE uses MySQL, the restriction seems natural enough when you consider the latest version's tighter integration into KDE. As for the ability to use a credit card to buy tracks from Magnatune, that is a service discontinued by Magnatune, not Amarok. Probably other features are missing as well, but, if there are any major ones, I failed to notice them in five days of living with the new release.

What I did notice was that the new interface contains more information, and makes controls more noticeable. An obvious example is the enlargement and movement of the basic playing controls to the upper left corner from the lower right.

I also appreciate the context information applets, and the way that advanced functionality and customization options are readily available, but not immediately obvious. With this arrangement, you can discover advanced features gradually, instead of being overwhelmed by complexity when you open the application for the first time.

Being asked to adjust to the unfamiliar is always difficult. Probably most users will have one or two complaints about the new Amarok. Mine are the notification window that pops up when a new track starts and Amarok does not have the focus, and the noticeable lag before the Stop button has an effect.

However, if you can put aside the preconceptions based on earlier versions, then you will probably conclude that Amarok's developers are right: With version 2.1.1, Amarok really has left most of its problems behind. Another release or two, and it should regain its former popularity, and relegate the often hostile reception of version 2.0 to a fading memory.

Comments (17 posted)

Valencia moves to free software

Over the last decade or so there have been multiple reports of governments or agencies making the switch to free software. Some have been relatively successful, like Munich—though not without some bumps along the way—others have been less so. A recent report [PDF], from a ministry of the Valencia Autonomous Community in Spain, provides a nice look inside the transition to free software. It will be a helpful guide for other organizations who are thinking about making the switch.

The Ministry of Infrastructure and Transport (CIT) started to look into free software in 2003: "This change in commercial software sales strategy, together with the CIT's policy of having legal licences for all our users, meant a considerable increase in licence costs which became unsustainable as the majority of our budget went on acquiring these licences." So, the Organization and Computing Department—headed by Martín García Hernández, who wrote the report's introduction—made a proposal to switch all of CIT's systems to free software. The proposal was accepted in September of 2003, and, after a feasibility study, the gvPONTIS project was launched in January 2004.

While it is not an enormous organization, CIT does employ around 1000 people, in a variety of tasks, with 600 administrative employees and 400 engineers and architects. This diversity meant that there were various kinds of software, from office applications to GIS and CAD systems, that required migration. In the process, gvPONTIS also created two new free software projects, gvSIG and MOSKitt, that clearly demonstrate its understanding of the advantages of free software.

Hernández noted that the biggest problem gvPONTIS faced was the "fear of change". This is a common problem encountered when switching users from a familiar environment to something new—not just for software. But, the project had a plan: "In our case, we have faced up to the challenge with well-laid plans, training and an alternative plan of action just in case."

The report is quite detailed in the steps gvPONTIS had to take, the applications and infrastructure that it needed to migrate, as well as the tools it used to get the job done. While the specifics of CIT's environment are unlikely to be replicated elsewhere, the decisions and thought processes that went into the migration will be applicable to other organizations considering a transition to free software.

Moving from existing Access and Oracle databases was eased by using UnixODBC to connect from applications, such as OpenOffice, to those existing databases. But, new development was targeted for a free software alternative. After comparing Interbase, MySQL, and PostgreSQL, it was determined that the latter best fit the needs of the project. The report has a detailed account of the switch to PostgreSQL and the difficulties encountered. There are, of course, still problem areas, which the report clearly indicates, for example:

The project took a pragmatic approach, by continuing to use the existing applications (many of which ran on Windows or other proprietary systems, like Oracle) as it worked towards alternative free software solutions—even if it had to develop its own. Virtualization with VMWare was used to provide access to some Windows applications, for example.

After looking at SUSE 9.0 for the office desktops, the project decided to use the LliureX distribution for which there was local support. LliureX is an education-oriented distribution that was created by the Valencia government and is based on Ubuntu. Standard free software tools replaced much of the proprietary applications for web browsing, email, and office tasks.

One of the biggest successes of the project is the gvSIG GIS/CAD package that was developed for the project. By having a free software solution—not licensed per seat—it allowed more users to access the GIS and CAD capabilities. Instead of 90 GIS/CAD users, there are 400 now. The project has also gained quite a following outside of Valencia, so several pages of the report outline the spread of gvSIG usage throughout the world.

There is great deal more to the report, and it is well worth a read for anyone interested in free software migration. There is information on the networking environment, servers, document management system, CASE modeling tools, and more, from the perspective of moving to free solutions. It clearly shows what a dedicated organization can accomplish if it is truly important to migrate to free software. It undoubtedly had its bumps as well, but Hernández sums it up nicely:

As long as proprietary software companies cling to per-seat licensing and proprietary data formats, there is likely to be more of this kind of migration over time. Organizations that are interested in doing so need to realize that it cannot happen overnight. It took decades of building up their in-house systems, so it will certainly take some time to migrate away from them as well. It would seem that Valencia has provided a nice road map for one way to get there.

[ Thanks to Ismael Olea for pointing us to this story. ]

Comments (4 posted)

Why OpenAerialMap failed where OpenStreetMap succeeded

OpenStreetMap (OSM) is an ambitious project: creating a crowd-sourced, user-editable, free vector map of the world's roads, footpaths, and trails. Despite the scale of the task it has been remarkably successful, with more than 100,000 registered users and more than 29,000,000 ways. In 2007, a group of interested OSM users set out to do the same thing for raster-based satellite and aerial photography that OSM did for road data — but that project, OpenAerialMap (OAM), never reached the same level of success, and closed its doors in December of 2008. Looking back, OAM's creator finds both technical and social causes for the project's fate, and lessons for the future.

Christopher Schmidt launched OAM in hopes of building a free replacement for the copyrighted satellite image layer used by Google Maps. The OSM project had struck a deal with Yahoo to permit usage of Yahoo's aerial image layer in the Potlatch map editor, but only for the purpose of editing GPS traces against a satellite background image: the data was to remain Yahoo's and it could not be displayed in the OSM web map. A sister project, OpenTopoMap, attempted to do the same thing for topographic map data, using a similar methodology, but Schmidt put the greater effort into OAM due to its presumably wider user-appeal.

OAM's initial map set was a Landsat layer donated by i-Cubed. Where aerial photography was available, OAM would support that instead thanks to its greatly increased resolution. The project chose to store its map tiles as uncompressed GeoTIFF images in the standardized EPSG:4326 map projection; that way, although more storage space was required than for a compressed format, the openaerialmap.org server could serve up map tiles without worrying about the CPU overhead of converting every tile served.

By the middle of 2008, however, things had started to slow down. Although there were scores of free, often public-domain aerial data sets, they were not organized or easy to work with: many in the U.S. belonged to individual state or county government projects, running on legacy systems — if online at all — in different projections, and in inconvenient file formats like the proprietary MrSID. On top of that, the data began to grow faster than the disk space on the donated server could keep up with.

The technical dimension

By the time Schmidt decided to pull the plug on the project, he had uploaded ten terabytes of map data, which was only a fraction of what was available — the U.S. Geological Survey (USGS) generously offered to donate its entire aerial data set of the United States, which would have taken up more than 100 terabytes of compressed storage.

It is possible that a benefactor could have been found to donate storage hardware, but that was not the only technical hurdle. Converting the data sets from their original form into the standard adopted by OAM was a time-consuming process, and one that varied from data set to data set, Schmdit said. "We wanted to have large mosaics, EPSG:4326 projected, as GeoTIFFs. Building that kind of tileset out of hundreds, or thousands, of small MrSID files requires significant resources. Just loading MrSID itself can be a beast since MrSID is closed source and not built into many tools."

MrSID (mutli-resolution seamless image database) is a wavelet-based geographic image file format, created and patented by map software company LizardTech. Each file can be extracted in multiple resolutions and quality levels, and sub-blocks of the image can be extracted without reading the entire file. The format is popular in commercial applications, but there is no open source implementation. Several free tools can decode MrSID to convert it to other formats.

In contrast to the process required to convert a MrSID archive to GeoTIFF, OpenStreetMap map-making requires the comparatively straightforward conversion of GPS traces to OSM ways. From the beginning of OAM, Schmidt undertook the lion's share of this "technical gruntwork" himself, rather than building software tools to automate the process for end users. In retrospect, he reflected, that decision severely hindered the development of a community around the project. "The system was simply not designed for users, because it was kind of a one man show. My lack of delegation skills is apparent in many of the things I do :)." A handful of users got up to speed on the conversion process, he added, but most simply gave pointers to available data sets and waited for someone else to take care of them.

The social dimension

Without easy-to-use conversion tools, the number of contributors to the project remained low, but, Schmidt said, the task of building a raster graphics map did not lend itself towards community participation either. In OSM, large data set imports are rare; the majority of the contributions come from individual users collecting GPS traces of their local surroundings, and editing their local map.

In spite of having similar goals, OAM found itself in almost the exact opposite situation. "Aerial imagery gathering doesn't lend itself to crowdsourcing the same way vector data gathering does," Schmidt said, "Almost all of your data is going to be big dumps from state agencies and the like." A few users collected their own aerial photos with kites or drones, he said, but their images amounted to just a tiny fraction of the total.

"I think the important thing is to let people take ownership of what they're doing," Schmidt remarked. Without the level of direct participation found in the OSM project, users did not feel like they "owned" the final product. "Looking back, I can see a lot of places to build community, but at the time, I wasn't really interested in building a community, and by the time that I realized I was burnt out, there was already an expectation that things would just 'magically happen'." Consequently, when Schmidt decided he was burnt out, there was no one else ready to take on the task of stewarding the project.

The next level

Although he switched off the openaerialmap.org tile server in December 2008, Schmidt has not stopped thinking about OAM, and how to grow a community around it. "The desire for easy to find an access aerial imagery is definitely there. The continued interest in OAM even after the web frontend essentially stopped working speaks to that, and the continued attempts to participate even with terrible documentation, poor setup, etc."

He is considering re-launching the project with a different set of goals and different ways of attracting user participation, placing less emphasis on serving as a single-source repository of image tiles. Instead, OAM could function like a directory for the existing aerial imagery map sets hosted elsewhere, like the approach taken by Web Map Service (WMS) directory wms-sites.com. Such a service would be almost as useful as serving the tiles themselves, he said, since so much of the difficulty now is in finding the data set for a particular area.

Building a wiki-like directory that points to data hosted elsewhere might also allow users to build "ownership" of their contributions, Schmidt added. "The imagery is only a first step towards the end result. Does the imagery have metadata about it? Is it aligned properly? Do people know how to find it?" Most often, he said, the only way to find a data set is search the web, and then the results have no user interface, "just an open FTP site linked from one page under 'GIS Data'."

The mapping agencies are usually helpful, he said, but do not have the manpower or IT resources to run a WMS or tile server. "This is why I'm thinking a catalog approach might work: if you let the people who maintain this imagery at places like MassGIS contribute to a site information *about* the imagery — concentrating on things like quality, capture dates, etc. — you can have them have a product they can feel like they're the owner of."

Although he has not taken any formal steps toward this goal, it is undeniable that there is latent interest in resurrecting the OAM project in some form. The map server is down, but the OAM wiki remains up, and other users have started a discussion page to brainstorm ideas for the future of the project. Its main archive is off-line, but people continue to find the mailing list via third-party sites and write in to ask questions.

Rarely does an open source software project that has failed to get off the ground take a serious look at why — although the benefits of doing so are clear. Schmidt's assessments of what went wrong with the first incarnation of OAM will help make the next effort more technically prepared, and better matched to the user community it needs in order to build up critical mass and thrive. Similarly, looking back at how the practical differences between raster and vector mapping led OAM down a completely different path than OSM (in spite of their seemingly similar goals) is an object lesson other projects would be wise to study as well.

Comments (1 posted)