My quest for a Linux audio player (Linux.com) [LWN.net]

My quest for a Linux audio player (Linux.com)

Posted Apr 4, 2006 18:41 UTC (Tue) by jwb (guest, #15467)
Parent article: My quest for a Linux audio player (Linux.com)

I've done something along the lines of this testing over the last two weeks, and I'd be happy to share my results.

Rhythmbox: A reasonably good player but sharing the library with iTunes still does not work. Keeps tracks in the library even if the volume where the files are stored is not mounted, which is a feature I appreciate. Can interact with all my network-stored music via GNOME VFS2. Uses GStreamer, which I consider a fundamental design flaw.

Amarok: Launches with a useless mini window and no obvious way to manage your library. Becomes confused by its own dotfiles when you upgrade versions. Stores album art out-of-band, which is not useful. Crashes.

Banshee: Written using Mono by GNOME hackers, but doesn't use GNOME VFS, so can't interact with my network library. Doesn't seem to be any way to get a reasonable sorted list: you can sort by album but then the songs are disordered by track. If a song is in your library more than once, the song is shown multiple times in the library view. Has automatic tagging features, but no feedback, so you can't tell if they are working. Consumes insane amounts of memory. Has O(N^2) library import time: by 1000 songs imported it's taking several seconds per song. Good iPod support. Working interaction with iTunes. Uses GStreamer.

One thing I really came to appreciate while comparing these programs (and lightly hacking them) is that GStreamer is a mistake. If your program uses GStreamer, there is basically no way to report useful error messages back to the user (or even back to the program) and no way to just get a damn array of audio samples. I was trying to add MusicIP fingerprinting to these programs but it's just not straightforward when you use GStreamer.

See also Video I/O on Linux: Lessons Learned from SGI, specifically the sections "Just Give Me the Data!" and "Plug-in Dataflow Graphs."

We also incorrectly assumed that most application writers would want "help" creating complex graphs of interconnecting components (video/file sources, processing plugins, video/file sinks) and pushing data through those graphs. Our design became complex: writing the API itself, and writing applications with that API, became difficult. Eventually we were spending most of our time focusing on this complexity problem we had created ourselves. We weren't helping developers with the high-level mechanisms: we were putting obstacles between them and their data.

That's exactly how I felt all last week.

(Log in to post comments)

My quest for a Linux audio player (Linux.com)

Posted Apr 5, 2006 6:23 UTC (Wed) by piman (subscriber, #8957) [Link]

From the perspective of any of the applications reviewed for this article, 0.8 and 0.10's design is basically the same. 0.10 fixes a lot of bugs, but the core design still makes it difficult to cope with shared resources, error propogation, debugging and profiling, etc.

My quest for a Linux audio player (Linux.com)

Posted Apr 5, 2006 12:16 UTC (Wed) by drag (subscriber, #31333) [Link]

Funny that.

I am using Amarok right now and it uses Gstreamer.

Problems with GStreamer

Posted Apr 5, 2006 11:03 UTC (Wed) by Uraeus (subscriber, #33755) [Link]

Heh, its always fun when people pull out that article by Chris Pirazzi about the lessons learned from SGI as it was something Chris wrote on my request as I was a writer/editor on linuxpower back in those days. I requested the article so that we could get some interesting viewpoint/input into the development of GStreamer at the time.

I think both he and you misses some important points. Part of the problem experienced by SGI was caused by developing a closed source library. Which meant that when application developers needed to add or change stuff at that level they where basically shut out and away from their data. With an open source project like GStreamer this isn't the case. If you need stuff at the GStreamer level you add it at the GStreamer level, not the application level.

So instead of trying to hack this into each of these 3 players you should have tried adding it to GStreamer. There already is a musicbrainz plugin and I am guessing a MusicIP plugin would be very similar. Once you had that added to GStreamer you could have easily gotten it to work in your application of choice, with the added bonus that the other two players (and many more) could also easily add the functionality too.

Your complaints about the error handling might be interesting and I am sure Thomas who have been working a lot on the error handling in GStreamer would be interested in your feedback.

My quest for a Linux audio player (Linux.com)

Posted Apr 5, 2006 11:53 UTC (Wed) by thaytan (guest, #25908) [Link]

I'm not quite sure how to respond to your comment. I'm a GStreamer developer, so obviously I don't agree with your assessment that it's a 'fundamental design flaw' for players to use it.

Your statement that "If your program uses GStreamer, there is basically no way to report useful error messages back to the user (or even back to the program) and no way to just get a damn array of audio samples." is wrong on both points, although the first is partly true.

1) Producing useful error messages in a generic streaming framework is hard. The big problem currently is that many GStreamer plugins report generic errors where they should report specific ones. We're improving that as we go, and they're getting better. In 0.10 there is a clear mechanism for producing error messages to the application, categorised into one of a bunch of defined error classes and providing a mechanism for supplying translations to different languages.

2) You mentioned trying to hack your MusicIP fingerprinting into Rhythmbox. I assume that means during the song import? You can do this in a couple of ways:
a) Write a GStreamer element based on the BaseTransform base class and drop it into the pipeline somewhere. You'll pretty quickly have a GStreamer element that can be reused in every existing GStreamer based player with the addition of maybe 10 lines of code to each.
b) Look at the part of the metadata importer that connects fakesink to the pipeline and either add a buffer-probe to the input of fakesink or connect to the handoff signal. Either one will provide you with a way to have GStreamer call a callback whenever there's a buffer of audio data to be processed.

a) has a longer lead time, but is more reusable. b) will get you there quicker.

http://gstreamer.freedesktop.org/data/doc/gstreamer/head/... shows you sample code of how to do b)

http://webcvs.freedesktop.org/gstreamer/gst-plugins-good/... is a good example of a filter element that analyses the audio passing through and sends custom messages to the application reporting the results.

In case you didn't already find it last week when you were having trouble, you are welcome to jump on the #gstreamer channel of the freenode.net IRC server and ask before deciding the mechanism isn't there.

My quest for a Linux audio player (Linux.com)

Posted Apr 5, 2006 18:55 UTC (Wed) by jwb (guest, #15467) [Link]

Thanks for your thoughtful reply. I'm not really planning to argue about it, because, really, what's the point? It's not like convincing you to stop developing GStreamer would be a positive accomplishment, and as a developer you have presumably already decided that the design of GStreamer is to your liking. And there is clearly a community of GStreamer users who think that GStreamer is also to their liking.

So I'll just reply to say that GStreamer programming requires a certain way of thinking that doesn't really agree with my brain. I find this code easy to understand and modify:

vf = ogg.vorbis.VorbisFile(name)
while 1:
(buff, bytes, bit) = vf.read(4096)

if bytes == 0:
break

musicip.consume(buff, bytes)

Whereas I find that when I need to work with GStreamer I quickly have lots and lots of code dedicated to factories, pads, signals, callbacks and lots of other things that are identified by strings. Even in python this adds up to many lines of code.

Here's an example of the error handling stuff I'm talking about. Suppose you build your pipe but then you get an error while settings the state to STATE_PLAYING. How can my program intelligently deal with any of the dozens of errors that could have caused the state change failure?

My quest for a Linux audio player (Linux.com)

Posted Apr 6, 2006 15:32 UTC (Thu) by thomasvs (guest, #36955) [Link]

Well, how would your example application deal with any of the errors that could happen when trying to read from a file ?

Most applications special-case a few of the errno values, and then print a generic string with strerror()

Are these generic strings very good ? No, because they are generic - they don't have enough contextual information about what the application was doing when it triggered these problems.

In your example, using Python, you would get a confusing traceback because of a raised exception - for example, IOError if it couldn't find the file.

You have to accept that handling errors correctly is hard intrinsically. The application needs to combine information about the lower-level error *and* its execution context to interpret what has gone wrong and show a useful error for it.

Also, remember that your simplified example is not an application on the scale of totem or rhythmbox - there is yet more code involved to show a decent dialog with this information.

My quest for a Linux audio player (Linux.com)

Posted Apr 6, 2006 15:42 UTC (Thu) by zaheer (guest, #36979) [Link]

Now change your code so that it handles other formats autodetecting on the way, say flac or mp3 or wav

Now also add a feature to take it from an icecast stream rather than a file, or from another machine over ssh/sftp.

Guess what, this code now becomes a lot harder and you start implementing a mini framework.

My quest for a Linux audio player (Linux.com)

Posted Apr 6, 2006 19:38 UTC (Thu) by lysse (guest, #3190) [Link]

Gstreamer has to do that, because it has to be coded for any eventuality and be usable by anyone. However, if you already know that you don't require anything other than the one format, or the one source of music, there's no need for that additional complexity.

Using off-the-shelf software is a trade-off; it states that your time is worth more than your machine's memory, and that writing code is Always Hard. Sometimes that's not the case - sometimes it's quicker to just get out the text editor and write exactly the code you need than it is to figure out the all-singing, all-dancing framework / library / whatever. For a proficient coder, "sometimes" might become "more often than not".

To tie it back to an earlier discussion, bloat is the price of not writing your own need-specific code.

My quest for a Linux audio player (Linux.com)

Posted Apr 5, 2006 13:15 UTC (Wed) by thomasvs (guest, #36955) [Link]

Hi,

two questions:

a) Could you elaborate on "there is basically no way to report useful error messages back to the user (or even back to the program)" ?

There are a number of programs that do this just fine with the given API. Take for example this Flumotion screenshot: http://www.flumotion.net/images/flu-error.png

I'm interested in knowing if there really is a problem here.

b) Did you know there are at least two different ways to just get at the raw samples ? There are buffer probe callbacks - triggered for every buffer passing through a pad - and there is a fakesink element with a handoff signal that can give you every buffer.

I'm also confused by what you said about adding a MusicIP feature. I don't know what MusicIP is, but basically you are saying that you would have preferred to add this particular feature *three times* for each player, over just implementing this once as an element and then simply using it from each player. I am assuming MusicIP is something that, like musicbrainz, does some fingerprinting of the audio data and then generates some ID from that.

GStreamer 0.8 has a trm element that works just fine, and could easily be used as a basis for your MusicIP element. To be honest, I don't think that whatever you are trying to do would take much work. But you seem to only want to hack on one layer - the player. I can understand that, but in reality these days if you're only ever going to hack on one layer of the cake, you're probably not going to end up solving any real user problems.

Lastly, communication is key. You could mail the GStreamer mailing list with your questions, or drop by on IRC and ask around. We're a friendly bunch and it wouldn't have been hard to help you out.

Hope that helps.

My quest for a Linux audio player (Linux.com)

Posted Apr 5, 2006 13:39 UTC (Wed) by bronson (subscriber, #4806) [Link]

That article is horribly oversimplified. He talked about "buffers of video data" all over the place. Just what is a buffer of video data? Is it an uncompressed RGB frame, ready to be blitted? Maybe scaled and filtered YUV 4:4:4? Maybe pre-scaled YUV 4:2:2? Maybe macroblocks, maybe pre-mc, maybe a muxed PES, h.264, ... All of these are different ways of representing the same basic video.

And all representations are necessary. Most video cards can do a YUV->RGB conversion while scaling, filtering and blitting. A number of video cards can do basic macroblock transforms. The Via C3 does motion compensation in hardware. The IVTV chips can decode a single MPEG stream. Sure, in an ideal world, we'd have a single video format that everyone could buffer. Until then, DAGs and processing pipelines are pretty much necessary. How else are you going to solve this problem?

That said, GStreamer could be a hell of a lot better with its error reporting. Most people here have probably seen Totem's: "Error! Video could not be played because: no error."

My quest for a Linux audio player (Linux.com)

Posted Apr 5, 2006 18:01 UTC (Wed) by thomasvs (guest, #36955) [Link]

wrt: That said, GStreamer could be a hell of a lot better with its error reporting. Most people here have probably seen Totem's: "Error! Video could not be played because: no error."

First of all, there has never been *any* dialog that has said this. You're probably over-simplifying for comic effect.

Second, the person who was maintaining the backend in the past repeatedly was told exactly what sort of things needed to be caught and displayed correctly. All the facilities were there in GStreamer, they were just being ignored by the implementer. This has since been fixed by other people. This is an application bug - without any changes to GStreamer Totem now shows decent error messages.