A Guide Through The Linux Sound API Jungle

Posted Sep 26, 2008 2:55 UTC (Fri) by obi (guest, #5784)
In reply to: A Guide Through The Linux Sound API Jungle by kev009
Parent article: A Guide Through The Linux Sound API Jungle

First of all, it's true that if you're developing a QT4 app and you should definitely consider Phonon when evaluating sound APIs. It exists, it's out there, and if you're linking to QT4 already, there's no reason to avoid it.

That being said, I've always had a bit of trouble understanding Phonon's raison d'être. From what I can tell - and feel free to correct me -, Phonon was made to:

1) have a clear and simple API that meshes well with QT,
2) make portability to Windows and MacOS a bit easier,
3) make sure that QT/KDE libs ensure ABI compatibility - i.e. avoid the risk of having to maintain an old branch of an unfamiliar multimedia framework (f.e. gstreamer 0.8) on their own for the duration of the QT/KDE 4.xx cycle,
4) avoid the politics of having to pick and bless a multimedia framework for use with QT or KDE.

However, I wonder whether in doing so Phonon users lost more than they gained. Not picking an existing framework and creating a abstraction layer means that:

1) Phonon apps can only use the "simplified" API: they can't f.e. manipulate the gstreamer pipeline the Phonon gst backend creates for them
1b) ... which means apps will be tempted to stick to the functionality Phonon provides; brave koolaid-drinking app developers might try to extend the Phonon API (and the backends), others will just pick a multimedia framework that allows them to express what they need for their slightly more advanced app. But of course, they will all choose a different framework ;-)
2) resorting to an abstraction for portability was unnecessary. IIRC, gstreamer can bridge to and reuse Directshow components on Windows, and Quicktime components on MacOSX.
3) they had to write decent Phonon backends which probably is quite a bit of work and deals with the framework they're not familiar with. To avoid _the possibility_ that the framework might move on and stop maintaining the branch QT or KDE committed to for the duration of the ABI cycle. Um...?
4) ... and as mentioned before, while they avoid the politics of having to bless one multimedia system, the discussion won't really go away. The developers of non-trivial apps will still have to make a choice.

End result: if you have some simple and some advanced multimedia apps running, you'll end up with Phonon + backend + one or more multimedia frameworks in memory. I believe that just picking one would've saved a lot of work and memory.

(Log in to post comments)

A Guide Through The Linux Sound API Jungle

Posted Sep 26, 2008 11:49 UTC (Fri) by quintesse (guest, #14569) [Link]

"I believe that just picking one would've saved a lot of work and memory"

They did once and it backfired completely saddling them up with an unmaintainable situation in what was most of the KDE3.x lifetime.

And they specifically say that Phonon is for those applications that have only modest audio/video needs. So it will do relatively simple playback and capture which will be enough for 99% of the applications (which includes full featured media players). Basically the only applications it will not be enough for are those that do audio/video processing/editing. But in that "application space" GStreamer isn't the only one either because a lot of those apps want to support things like JACK as well.

And let's not forget that Linux is about choice, I think in an an environment like KDE/Qt that needs to be binary compatible for MANY years betting on one single audio layer (that you don't control) is just stupid. GStreamer might never go away, true, but maybe in 2 years something even better comes along and we want to use that for our apps. Why not make that a possibility? (Only half a year ago I would always use the Xine back-end for most of my media apps because it just worked better on my system.)

A Guide Through The Linux Sound API Jungle

Posted Sep 26, 2008 15:26 UTC (Fri) by obi (guest, #5784) [Link]

"They did once and it backfired completely saddling them up with an unmaintainable situation in what was most of the KDE3.x lifetime."

Maybe they just chose the wrong framework? From what I can remember, not a lot of people were actually working on aRts, and it didn't actually already have the features needed for a multimedia framework.

I don't want to hold up Gstreamer as the only shining example of everything that's good and proper, but at least Gstreamer has been around a while and has actually been doing video and complicated pipelines for a while. Experience over a few release cycles make me trust it a lot more. Plus it has a lot of mindshare in general, so it would seriously surprise me that it would all of a sudden die off in the middle of a KDE cycle.

For the same reason, I don't think a new framework will pop up from nowhere that supplants Gstreamer - these things take time to mature, and even if one does appear, I doubt it would be ready before KDE 5.xx.

As the adage goes "make sure the simple things are kept simple, and the hard things possible". It's nice (and important) that Phonon makes the simple things easy for 90% of the applications - however the hard things are simply impossible with Phonon, without dragging in the frameworks themselves anyway.

A Guide Through The Linux Sound API Jungle

Posted Sep 26, 2008 17:01 UTC (Fri) by quintesse (guest, #14569) [Link]

Well with the advantage of hindsight it's obvious they made the wrong choice, but you just can't look into the future so it makes sense to "protect" yourself.

And that making a new audio framework takes time is undoubtedly true but if I see it correctly GStreamer was in development for 3 years before it got included in GNOME (I imagine because it was good-enough by then, but maybe even before that?) while the entire lifetime of KDE3 has been 6 years! And that's up to now, KDE3 isn't dead yet. A lot can happen in 6 years.

But it the end it just comes down to what you yourself are saying: make the simple things easy. Well that's what Phonon does, it makes it very easy for you to add media capabilities to your application and it will even let the user/distro decide which framework to use. That's a pretty big advantage! And it still makes the hard things possible. How? Well you are not forced to use Phonon! Just go ahead and use the any underlying framework that you want. And if in the end it's GStreamer that wins out over all the other frameworks (which at this point in time at least seems likely) you don't have to worry about having several frameworks installed on your system either, both your media studio app and Phonon will be using the same one!

A Guide Through The Linux Sound API Jungle

Posted Sep 27, 2008 23:32 UTC (Sat) by obi (guest, #5784) [Link]

As for hindsight - of course; when KDE adopted aRts the multimedia world was a very different place. I don't think they could have foreseen that. But we've come a long way in the mean time, and I think it should've been possible to have a better idea of where things were going in 2006. Yes, you ran the risk of getting burnt again, but the chances were smaller.

I'd also forgotten KDE 3.xx was out for over 6 years. But still, see my reply to pynm0001 - I think that maintaining gst 0.10 would not have been an impossible task; at least the odds you'd get outside help would be bigger - I imagine there's more people familiar with gstreamer's codebase than Phonon's.

A Guide Through The Linux Sound API Jungle

Posted Sep 28, 2008 13:00 UTC (Sun) by quintesse (guest, #14569) [Link]

I don't know Obi, why on earth do you care so much that KDE has made a small, thin wrapper layer around existing audio frameworks?

Because it's obvious in your reply to pynm0001 where you say "weighing the work required to create Phonon and its backend vs the potential work needed to maintain gst 0.10 for a while on your own" that you seem to think these are somehow related jobs in terms of time, effort and knowledge needed to be actually able to do it. They're not! Phonon is a pretty thin layer, no extensive knowledge of audio frameworks is needed neither to maintain it nor to make the backend "drivers", so the KDE devs are pretty confident that this time around, come hell or high water, they WILL be able to take KDE into the future.

Now if you can somehow come up with a reason why NOT using GStreamer directly is a huge mistake there would be something worth talking about, but right now we're just wasting our time.

A Guide Through The Linux Sound API Jungle

Posted Sep 28, 2008 14:25 UTC (Sun) by nix (subscriber, #2304) [Link]

obi's argument appears to be that maintaining phonon is wasted effort
because you have to adapt to changes in gstreamer's API anyway.

But this is erroneous, because if you have to maintain gstreamer yourself,
even if you don't add any features it's like maintaining a hundred
phonons: all those plugins then need to be maintained, and each of *those*
requires tracking changes in the downstream APIs... much more work.

A Guide Through The Linux Sound API Jungle

Posted Sep 28, 2008 23:50 UTC (Sun) by obi (guest, #5784) [Link]

Actually my argument was more along the line of: "I'd rather have a lot of people help maintain one framework (albeit an old branch), than have several libraries maintained by just a a few people each"; However if Phonon really is as thin as you guys say, and maintaining a stable gst 0.10 really would be like a hundred phonons - then I guess I'm just wrong. The point is moot especially considering Phonon is out there; sorry for wasting your time.

A Guide Through The Linux Sound API Jungle

Posted Sep 29, 2008 0:02 UTC (Mon) by nix (subscriber, #2304) [Link]

gstreamer basically consists of two pieces: the core, which provides
plumbing to connect together plugins, and the plugins. The majority of
those plugins are interfaces to other audio/video libraries (everything
from monsters like ffmpeg to little things like libogg or libmad). As
such, gstreamer has to track changes to the APIs of the libraries used by
all those plugins. (Thankfully, most are quite stable, but not all:
pulseaudio has bumped API in the last couple of years, and ffmpeg hardly
ever stops changing...)

A Guide Through The Linux Sound API Jungle

Posted Sep 26, 2008 21:33 UTC (Fri) by pynm0001 (guest, #18379) [Link]

"For the same reason, I don't think a new framework will pop up from
nowhere that supplants Gstreamer - these things take time to mature, and
even if one does appear, I doubt it would be ready before KDE 5.xx."

A new framework isn't the issue. All that is required is to break API
compatibility in gstreamer itself.

This happened no less than two times in my time with JuK in KDE 3. We had
C++ bindings to gst-0.6 which we used in JuK. These bindings were broken
by gst-0.8 and we eventually simply coded gst-0.8 support directly into
JuK. Then gst-0.10 came out and we had to re-do the support again.
gst-0.10 has been out for awhile but do you want to be the one to guarantee
that gst-0.12 won't be out while KDE 4 is still supported?

I've gone over all of these arguments more than 2 years ago:
http://lwn.net/Articles/183462/

A Guide Through The Linux Sound API Jungle

Posted Sep 27, 2008 23:14 UTC (Sat) by obi (guest, #5784) [Link]

I see, what you say makes sense - it's not unlikely there will be a gst 0.12 within the lifetime of KDE 4.x.

However - weighing the work required to create Phonon and its backends vs the potential work needed to maintain gst 0.10 for a while on your own (there will likely be other organisations interested in a stable 0.10 branch too however) if a gst were to happen during 4.xx, I'd probably avoid building something completely new.

However, this is a judgment call, and I trust you have a lot more experience with it than me. You address the problems with supporting 0.10 yourselves in your mail from 2 years ago:

"So if we just rely on gstreamer 0.10, now we're stuck with an abandoned code base, which KDE developers are unfamiliar with. Does this sound familiar to anyone? ;)"

I think this is the crux of the issue.

You've avoided this by creating Phonon. However, I believe 0.10 has quite a bit of mindshare, and I believe that the KDE people would not have found themselves all alone maintaining the abandoned 0.10 (while OTOH they are pretty much on their own maintaining Phonon). Well, I do see how this might seem very similar to the aRts situation, but aRts wasn't used a lot outside of the KDE world. Maintaining Phonon + backends (which have to interface with the codebases you're not familiar with) must take quite a lot of effort too.

A Guide Through The Linux Sound API Jungle

Posted Oct 3, 2008 19:43 UTC (Fri) by jospoortvliet (subscriber, #33164) [Link]

There won't be much work maintaining Phonon, as it's incredibly small
compared to gstreamer (or any other multimedia framework). After all, it's
only a wrapper. Besides, Qt Software (Nokia) is maintaining it now ;-)

Don't skip over the advantage of transparent support for Mac & Windows
either - and Phonon also allows users to use another multimedia layer like
Xine or VLC. So more choice for users, easier api for developers and
insurance against api changes. I'd say a bargain ;-)