My quest for a Linux audio player (Linux.com)

I think both he and you misses some important points. Part of the problem experienced by SGI was caused by developing a closed source library. Which meant that when application developers needed to add or change stuff at that level they where basically shut out and away from their data. With an open source project like GStreamer this isn't the case. If you need stuff at the GStreamer level you add it at the GStreamer level, not the application level.

So instead of trying to hack this into each of these 3 players you should have tried adding it to GStreamer. There already is a musicbrainz plugin and I am guessing a MusicIP plugin would be very similar. Once you had that added to GStreamer you could have easily gotten it to work in your application of choice, with the added bonus that the other two players (and many more) could also easily add the functionality too.

Your complaints about the error handling might be interesting and I am sure Thomas who have been working a lot on the error handling in GStreamer would be interested in your feedback.

Your statement that "If your program uses GStreamer, there is basically no way to report useful error messages back to the user (or even back to the program) and no way to just get a damn array of audio samples." is wrong on both points, although the first is partly true.

1) Producing useful error messages in a generic streaming framework is hard. The big problem currently is that many GStreamer plugins report generic errors where they should report specific ones. We're improving that as we go, and they're getting better. In 0.10 there is a clear mechanism for producing error messages to the application, categorised into one of a bunch of defined error classes and providing a mechanism for supplying translations to different languages.

2) You mentioned trying to hack your MusicIP fingerprinting into Rhythmbox. I assume that means during the song import? You can do this in a couple of ways:
a) Write a GStreamer element based on the BaseTransform base class and drop it into the pipeline somewhere. You'll pretty quickly have a GStreamer element that can be reused in every existing GStreamer based player with the addition of maybe 10 lines of code to each.
b) Look at the part of the metadata importer that connects fakesink to the pipeline and either add a buffer-probe to the input of fakesink or connect to the handoff signal. Either one will provide you with a way to have GStreamer call a callback whenever there's a buffer of audio data to be processed.

a) has a longer lead time, but is more reusable. b) will get you there quicker.

http://gstreamer.freedesktop.org/data/doc/gstreamer/head/... shows you sample code of how to do b)

http://webcvs.freedesktop.org/gstreamer/gst-plugins-good/... is a good example of a filter element that analyses the audio passing through and sends custom messages to the application reporting the results.

In case you didn't already find it last week when you were having trouble, you are welcome to jump on the #gstreamer channel of the freenode.net IRC server and ask before deciding the mechanism isn't there.

two questions:

a) Could you elaborate on "there is basically no way to report useful error messages back to the user (or even back to the program)" ?

There are a number of programs that do this just fine with the given API. Take for example this Flumotion screenshot: http://www.flumotion.net/images/flu-error.png

I'm interested in knowing if there really is a problem here.

b) Did you know there are at least two different ways to just get at the raw samples ? There are buffer probe callbacks - triggered for every buffer passing through a pad - and there is a fakesink element with a handoff signal that can give you every buffer.

I'm also confused by what you said about adding a MusicIP feature. I don't know what MusicIP is, but basically you are saying that you would have preferred to add this particular feature *three times* for each player, over just implementing this once as an element and then simply using it from each player. I am assuming MusicIP is something that, like musicbrainz, does some fingerprinting of the audio data and then generates some ID from that.

GStreamer 0.8 has a trm element that works just fine, and could easily be used as a basis for your MusicIP element. To be honest, I don't think that whatever you are trying to do would take much work. But you seem to only want to hack on one layer - the player. I can understand that, but in reality these days if you're only ever going to hack on one layer of the cake, you're probably not going to end up solving any real user problems.

Lastly, communication is key. You could mail the GStreamer mailing list with your questions, or drop by on IRC and ask around. We're a friendly bunch and it wouldn't have been hard to help you out.

Hope that helps.

And all representations are necessary. Most video cards can do a YUV->RGB conversion while scaling, filtering and blitting. A number of video cards can do basic macroblock transforms. The Via C3 does motion compensation in hardware. The IVTV chips can decode a single MPEG stream. Sure, in an ideal world, we'd have a single video format that everyone could buffer. Until then, DAGs and processing pipelines are pretty much necessary. How else are you going to solve this problem?

That said, GStreamer could be a hell of a lot better with its error reporting. Most people here have probably seen Totem's: "Error! Video could not be played because: no error."