|
|
Subscribe / Log in / New account

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Over on the Collabora blog, Julian Bouzas writes about PipeWire, which is a relatively new multimedia server for the Linux desktop and beyond.

PipeWire was originally created to only handle access to video resources and co-exist with PulseAudio. Earlier versions have already been shipping in Fedora for a while, allowing Flatpak applications to access video cameras and to implement screen sharing on Wayland. Eventually, PipeWire has ended up handling any kind of media, to the point of planning to completely replace PulseAudio in the future. The new 0.3 version is marked as a preview for audio support.

But why replace PulseAudio? Although PulseAudio already provides a working intermediate layer to access audio devices, PipeWire has to offer more features that PulseAudio was not designed to deliver, starting with a better security model, which allows isolation between applications and secure access from within containers.

Another interesting feature of PipeWire is that it unifies the two audio systems used on the desktop, JACK for low-latency professional audio and PulseAudio for normal desktop use-cases. PipeWire was designed to be able to accommodate both use cases, delivering very low latency, while at the same time not wasting CPU resources. This design also makes PipeWire a much more efficient solution than PulseAudio in general, making it a perfect fit for embedded use cases too.



to post comments

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 1:17 UTC (Fri) by gerdesj (subscriber, #5446) [Link] (5 responses)

Pipewire sounds like a VPN to this particular portion of the great unwashed. However if it can do both JACK and Pulse in one go then I'll put up with the name.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 13:50 UTC (Fri) by sandsmark (guest, #62172) [Link] (1 responses)

The dependency on gstreamer is a bit more annoying than the name, though still fairly minor. I'm not really a big fan of gstreamer (their architectural choices and whatnot) and it's a pretty huge project to drag in just to be able to play sound (assuming you don't use e. g. flatpak but just plain old distro packages).

But just skimming the pipewire code quickly it seems like it's fairly sane wrt. interdependencies in the code. And while I'm not sure if it's a module that's loaded at runtime at the moment it seems to be fairly easy to write a simpler replacement module (or one using libvlc or whatever people want).

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 15:18 UTC (Fri) by sdalley (subscriber, #18550) [Link]

As far as I can see from the architecture diagrams, Gstreamer isn't an actual dependency of pipewire. Rather, gstreamer has been kitted out with pipewire plugins that allow apps that already use gstreamer, to carry on undisturbed when the layer underneath gstreamer (namely, pulseaudio) changes to pipewire. Apps that don't use gstreamer at the moment will not need to use it in the future.
The document I am looking at is https://gstreamer.freedesktop.org/data/events/gstreamer-c...

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 16:21 UTC (Fri) by andrel (guest, #5166) [Link] (2 responses)

Something VPNish is what we need. I'm using a remote Raspberry Pi for audio processing (it's got an RTL-SDR hanging off it), and it would be nice if the VNC remote desktop sent over sound as well as video.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 7, 2020 12:15 UTC (Sat) by seneca6 (guest, #63916) [Link] (1 responses)

Did you try x2go?

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 9, 2020 22:43 UTC (Mon) by andrel (guest, #5166) [Link]

Didn't know about x2go, thanks for the suggestion!

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 3:43 UTC (Fri) by flussence (guest, #85566) [Link]

Sounds good. I don't particularly care about JACK support, but if it has enough security to keep apps the hell away from my hardware volume control at all times that'd be enough for me (and probably many others).

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 7:54 UTC (Fri) by mads (subscriber, #55377) [Link] (23 responses)

Why can't these features be added to PulseAudio? PulseAudio solves many difficult problems that's not trivial to just reimplement, throwing all that out seems ill-advised at best. Can't shake the feeling that this will do more harm than good for the linux desktop experience.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 8:58 UTC (Fri) by smcv (subscriber, #53363) [Link] (19 responses)

Adding features can require breaking protocol compatibility. In this case it almost certainly does, because Pulseaudio doesn't act as a security boundary between mutually distrusting apps, so Pulseaudio apps can safely assume they're fully privileged.

If Pulseaudio broke compatibility, old apps would stop working: it seems better to give the new thing a new name or major version number, and have them work in parallel (presumably one can be made to output via the other) until the last Pulseaudio app goes away, at which point you no longer need Pulseaudio.

In packages that document that they break compatibility with each major version (like GTK, Qt, and taking an example from outside Unix, Direct3D) maybe the incompatible version would have been called "Pulseaudio 2", but Pulseaudio doesn't use major versions like that: Pulseaudio 13 is compatible with Pulseaudio 1.

See also the renames between the X11 and Wayland protocols, the OpenGL and Vulkan APIs, and the Win32 and WinRT APIs. (And contrast with Direct3D, where Direct3D 12 has little in common with older versions.)

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 9:12 UTC (Fri) by mads (subscriber, #55377) [Link] (18 responses)

"until the last Pulseaudio app goes away, at which point you no longer need Pulseaudio." I don't see this happen. And saying "can require" sounds like hand-waving.

I get the feeling that the whole reason for making a new sound server is because people don't like PulseAudio for personal reasons.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 9:46 UTC (Fri) by intelfx (subscriber, #130118) [Link] (3 responses)

> I get the feeling that the whole reason for making a new sound server is because people don't like PulseAudio for personal reasons.

Haters gonna hate.

Those who disliked PA (and still do so when there are little to no rational reasons left for such dislike) can be well expected to contempt PipeWire even more.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 11:30 UTC (Fri) by LtWorf (subscriber, #124958) [Link] (2 responses)

> Haters gonna hate.

The fact that something works for you does not imply that it works for everyone else in the world.

For several years on my computers, whenever the sound stopped working it meant that some package had pulled in pulseaudio.

Of course I'm sure I could have put in the time to debug the issue and fix it. On the other hand, why bother if removing it was all it took to fix the issue?

Also remember that the default configuration of pulseaudio does a thing about relative volumes of streams, which means you'll end up blasting full volumes sounds and possibly getting hearing damage. Distributions are expected to workaround this.

Anyway, I still don't use it because despite now it works ok out of the box, it still solves an issue I've never had. And without it I've always had multiple processes producing sounds.

So please don't be a fanboy and do not discount other people's legitimate issues just because you don't have them. It's not very inclusive.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 12:04 UTC (Fri) by intelfx (subscriber, #130118) [Link] (1 responses)

> The fact that something works for you does not imply that it works for everyone else in the world.

I'm not joining you on this flame war.

What has been said is simple: if someone dislikes PulseAudio for personal reasons (i. e., the definition of "hater"), it's more than likely that they will dislike PipeWire even more.

Therefore, it's unlikely that this specific consideration was the driving force behind PipeWire.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 18:43 UTC (Fri) by joib (subscriber, #8541) [Link]

Well, people that hate pulseaudio because Lennart will have to come up with a new excuse.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 10:07 UTC (Fri) by Wol (subscriber, #4433) [Link] (12 responses)

> I get the feeling that the whole reason for making a new sound server is because people don't like PulseAudio for personal reasons.

Well, the reason PulseAudio was written was because Alsa was fundamentally flawed, and it was easier to start from scratch. PipeWire has been written because PulseAudio is fundamentally flawed, and it's easier to start from scratch. Fundamentally, that is, as in the design itself is wrong.

PulseAudio is a sound *playback* system. If you have problems, you increase the buffer and delay the output - bit like "live" tv is delayed a couple of seconds to allow the staff to take action if something happens that shouldn't. That's why Jack was written - because the option of delaying output is simply unacceptable in a feedback situation (ie, the lead is singing con belto into a mike on stage).

So just as PulseAudio sought to provide a single unified response to the problem of sound in early linux distros (where each application thought it had total control into the sound card), now PipeWire is fixing PulseAudio's delusion that it has total control of sound output.

Cheers,
Wol

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 11:42 UTC (Fri) by mads (subscriber, #55377) [Link] (11 responses)

Why couldn't you stitch on security functionality in PulseAudio - implement it as a newer protocol, but let the daemon be backwards compatible? Why insist on breaking as much userspace as possible, and throw all those years of getting PulseAudio working with god knows what weird hardware setups are out there out with the bathwater? Bluetooth, multi speaker setups, virtual sinks, resampling, DTS, network sinks - PulseAudio does _a lot_ of difficult stuff that it took years to implement.

From the outside this all looks like some cadt thing.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 19:13 UTC (Fri) by ocrete (subscriber, #107180) [Link] (10 responses)

PipeWire actually provides a replacement libpulse.so, so applications shouldn't see the difference.

Wim actually started by trying to add security enforcement to PulseAudio, and it quickly turned into a disaster. And that is before starting to talk about unfixable design issues that make the latency quite high. And then having latency means you need things like rollback: being able to undo any effect you apply and re-apply a new effect, so that changing the volume is instant. All of those things make the PulseAudio code quite complex in ways that PipeWire is a much more simple and elegant solution.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 7, 2020 15:53 UTC (Sat) by kleptog (subscriber, #1183) [Link] (5 responses)

So Pulse did one thing right, they defined an API that was sensible enough that the implementation could be swapped out and replaced with a better one.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 8, 2020 22:33 UTC (Sun) by Wol (subscriber, #4433) [Link] (4 responses)

Not really. As far as I can make out, part of PulseAudio was was a soundcard emulation you could stick aRts on top, so several applications could be fooled that they had sole control of the sound card.

So now you have an interface where you fool PulseAudio into thinking it has control of the soundcard ...

This is the way computing *should* advance - the new layer fools the old layer into thinking nothing's changed ... I always think of IBM mainframes, where an IBM 360 program runs on a virtual 360, which used to run on a 370 and now runs on a virtual 370 which runs on ... well I don't know the progression of mainframes, but old 360 applications sit on top of three or four emulators to run on the latest hardware.

Cheers,
Wol

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 17, 2020 21:49 UTC (Tue) by nix (subscriber, #2304) [Link] (3 responses)

That's almost entirely wrong. One thing PA did right -- and PipeWire is following suit -- was to mimic the userspace APIs of almost all previously-existing sound server systems, so it could support almost all existing userspace programs. This was very difficult for OSS and ALSA -- it had to use an LD_PRELOAD wrapper you had to turn on explicitly for programs that needed it -- but for everything else it was more or less trivial. At no point in this does PA do anything one could reasonably describe as "soundcard emulation". Equally, PipeWire's PA emulation does not work by kicking up a copy of PulseAudio and giving it a virtual sound card: it merely provides a shared library that makes PipeWire appear to be PulseAudio to PA consumers using that shared library (basically all of them). The thing on the other end of the shared library is PipeWire, not PA-talking-to-an-emulated-sound-card.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 18, 2020 11:06 UTC (Wed) by smurf (subscriber, #17840) [Link] (1 responses)

Well, it didn't emulate the soundcard hardware, that's the kernel's domain in any case, but it did (and still does, in case anybody needs it, which is rather unlikely these days) emulate the OSS driver.

ALSA is/was way easier, pulseaudio hooks itself into it reasonably seamlessly, though the extra layer is painfully visible at times.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 19, 2020 22:00 UTC (Thu) by nix (subscriber, #2304) [Link]

Yeah, it had to emulate whatever API was available, and for OSS that meant it had to intercept mmap and stuff like that. It's still not actually emulating a *sound card* though.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 18, 2020 14:55 UTC (Wed) by nybble41 (subscriber, #55106) [Link]

> At no point in this does PA do anything one could reasonably describe as "soundcard emulation".

For ALSA, PulseAudio provides an ALSA plugin which connects to PA. That is essentially PA emulating a sound card, if one considers other ALSA plugins like dmix or plughw which appear the same as sound cards to applications to be "soundcard emulation". Only OSS required the LD_PRELOAD, since OSS applications accessed the device nodes directly. With CUSE and osspd to provide literal "soundcard emulation" for OSS apps one doesn't even need that.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 7, 2020 21:07 UTC (Sat) by luto (guest, #39314) [Link] (3 responses)

How does PipeWire handle latency? Suppose I have an audio device with a big enough buffer for 5s of audio. If an audio player is running on an otherwise idle system, ISTM the most efficient strategy is to queue up several seconds of audio every few seconds. If the user hits pause or changes the track, that buffer needs to be zapped immediately.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 8, 2020 17:16 UTC (Sun) by Wol (subscriber, #4433) [Link]

Dunno, but you're missing the Jack compatibility.

As far as Jack is concerned you can NOT buffer the audio - it needs HARD realtime with a latency measured in fractions of a millisecond. (Miss that target, and the result will - literally - be painful for your listeners.)

If you've got sub-millisecond latency, sounds like your requirements would be fairly easy to meet.

Cheers,
Wol

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 14, 2020 13:03 UTC (Sat) by wtay (guest, #55923) [Link] (1 responses)

>How does PipeWire handle latency?

It keeps buffers as small as the lowest requested latency by all clients. The largest buffer it can fill is 8192 samples (171ms). Buffers are usually 1024 samples or 21ms.

It will wake up more often and never needs to zap large buffers. This makes things much less complex so that you can make those frequent wake-ups so much more efficient.

There is some performance data here:
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/...

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 19, 2020 3:19 UTC (Thu) by flussence (guest, #85566) [Link]

I'm surprised PA seems to not even bother with its “intentional bufferbloat” mode of operation until the very last test; that's supposed to be one of its main features.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 19:53 UTC (Fri) by flussence (guest, #85566) [Link]

At one time, people acted like esound/aRts would never go away too. They were unavoidable, as was the cursing people shot at them. PulseAudio isn't going to be an immutable foundation either.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 9:48 UTC (Fri) by smurf (subscriber, #17840) [Link]

PulseAudio does single-pipe audio. Teaching it to handle the complete multimedia landscape, including screen recording and whatnot, *and* adding a security model would be a lot of work rewriting code and ensuring that every old code path and every existing module fits into the new larger scheme.

Better to start with a design that takes all of this into account from the beginning and worry about PA compatibility later, it's not that difficult (in contrast to JACK, pulseaudio doesn't even require hard realtime – you can just increase the deadline and ask your source to send the audio earlier if things are tight) and frankly the least of your worries when writing a new multimedia subsystem.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 11:13 UTC (Fri) by kazer (subscriber, #134462) [Link]

One of the things involves lifting session and policies out of the audio "daemon" so that it won't decide things for you. I imagine very low latency (for professional audio) would be hard as an afterthought as well..

But it looks like PW has stable API that is backwards-compatible with PA already.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 13:10 UTC (Fri) by darwi (subscriber, #131202) [Link]

> Why can't these features be added to PulseAudio?

Unfortunately the security issues are unfixable due to core design decisions made by PA in its very early days. I've summarized these issues some long time ago here:

https://www.freedesktop.org/wiki/Software/PulseAudio/Docu...
https://lists.freedesktop.org/archives/pulseaudio-discuss...

This is not to blame PA by the way. As said in the article, it was created in a completely different time. Computer usage scenarios are much different today.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 8:31 UTC (Fri) by gfernandes (subscriber, #119910) [Link] (5 responses)

Why can't this be a standard package, available as part of the distribution? Why FlatPak?

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 9:01 UTC (Fri) by alexl (subscriber, #19068) [Link] (2 responses)

This is a system feature, and in no way a flatpak. What made you think that?

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 18:28 UTC (Fri) by gfernandes (subscriber, #119910) [Link] (1 responses)

Thanks! That's a relief.

I guess I misread this from the linked article:
"A huge effort is currently underway to bring the Linux desktop into the future with the help of containerization technologies such as Flatpak."

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 19:16 UTC (Fri) by ocrete (subscriber, #107180) [Link]

The mention to Flatpak is because PipeWire's security enforcement makes it possible to allow only limited audio access to Flatpak'ed applications. For example, you can allow an application to playback, but not to capture from a microphone, etc. While the PulseAudio model means that if you give access to an application, it has full access to the audio system.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 9:05 UTC (Fri) by smcv (subscriber, #53363) [Link] (1 responses)

Pipewire *is* part of the OS distribution layer, and can be used without Flatpak being involved at all. The relationship is the other way round: Pipewire doesn't need Flatpak, but some Flatpak features need Pipewire.

If you want a Flatpak app to be able to do *some* audio/video operations (like play sound through your speakers) but you don't want them to be able to do *all* audio/video operations (like record sound from your microphone), you need the app to talk to something in the OS distribution layer that acts as a security boundary (a "portal" in Flatpak terminology), and can apply policies like "OpenArena can play audio, but can't record". Pipewire provides that.

At the moment Flatpak apps that need audio playback talk to Pulseaudio, but that isn't a good long-term solution, because Pulseaudio trusts all of its clients equally, so giving an app access to the Pulseaudio socket gives it complete control over audio.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 18:29 UTC (Fri) by gfernandes (subscriber, #119910) [Link]

Thanks for the explanation! :-)

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 10:08 UTC (Fri) by roc (subscriber, #30627) [Link] (11 responses)

What can a malicious app do with Pulseaudio access?

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 10:23 UTC (Fri) by Baughn (subscriber, #124425) [Link]

Record audio, for one.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 10:40 UTC (Fri) by smurf (subscriber, #17840) [Link] (4 responses)

Record audio (both microphones and speakers). DDoS your network without network access. Play loud sound (hearing endangering levels) on your headphones. Find a resonant frequency in your living room and cause property damage.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 15:21 UTC (Fri) by dezgeg (subscriber, #92243) [Link] (3 responses)

Does anybody know if that is any different from what apps on OS X and Windows are allowed to do by default?

At least I do not remember Windows asking permission for any of those (have not used Windows Store apps, though) so quite frankly, these seem like a non-issue to me...

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 16:30 UTC (Fri) by andrel (guest, #5166) [Link]

Almost every laptop I see has something opaque taped over the camera. The concern about being surreptitiously recorded is very widespread. Which is why phones ask the user to authorize an app accessing the camera/mic.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 19:18 UTC (Fri) by ocrete (subscriber, #107180) [Link]

The goal is not to reproduce Windows/macOS, both of which has designs that date from the same era as X11 and PulseAudio. In that era, the idea was that all applications were trusted and could have almost full access to the computer. PipeWire is trying to bring the Linux OS to the era of Android and iOS, where you can run less trusted applications, because they only have access to the minimal persmissions.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 7, 2020 9:26 UTC (Sat) by comex (subscriber, #71521) [Link]

It’s relatively new (added in the 2018 release), but macOS does have a per-app permission prompt for microphone access. (And another one for camera access.)

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 11:46 UTC (Fri) by alexl (subscriber, #19068) [Link] (3 responses)

All sort of stuff, like:
* Record audio from a microphone.
* Listen to the output of other running apps
* Turn down the volume of other apps
* Load and configure modules into the pulseaudio daemon

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 17:10 UTC (Fri) by mcatanzaro (subscriber, #93033) [Link] (2 responses)

"Load and configure modules into the pulseaudio daemon"

Does this mean: "execute arbitrary code"? It sure sounds like that's what it means.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 17:45 UTC (Fri) by smurf (subscriber, #17840) [Link]

No, the module are loaded from /usr/lib/pulse-VERSION/modules. The problem is that nobody prevents you from loading 1000 modules, linking their streams in interesting ways, and sending them some CPU- or network-intense data.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 9, 2020 15:29 UTC (Mon) by alexl (subscriber, #19068) [Link]

Not by design, you can't just point it to some .so and load that. However, I wouldn't be shocked if there is an exploit somewhere in some module you can trigger.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 6, 2020 12:01 UTC (Fri) by kazer (subscriber, #134462) [Link]

I would imagine it would be annoying in a vehicle if malicious app hijacked your navigator audio and substituted it with something else..
It looks like the automotive-people have a lot more concerns to handle than what PA can (check the video in the linked article).

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 9, 2020 15:29 UTC (Mon) by intgr (subscriber, #39733) [Link] (14 responses)

How come all these people are suddenly coming out of the woodwork claiming that PulseAudio is inefficient, resource-intensive and high-latency?

But back in the day I read several blog posts about PulseAudio where it appeared to go out of its way to reduce context switches and offer best latency possible with minimal CPU wakeups, with some clever buffer trickery (compared to something like JACK that does it by brute force, by reducing audio buffers to a minimum and causing a high number of CPU wakeups).

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 9, 2020 16:46 UTC (Mon) by smurf (subscriber, #17840) [Link] (12 responses)

Well, it is. Open pavucontrol and the thing goes from 5% to 50% CPU if your setup is sufficiently interesting. I reported that years ago. Yes it's reproducible.

PA is not high-latency per se, but it doesn't have hard bounds on its latency either. You add a sink that delays for half a second? fine, that's your new channel latency, and your video player needs to notice the change and deal with it. That's fine if your source is in fact a video player, not so if it's a live phone call. Add a multi-microphone setup with a DSP that's supposed to isolate the driver's voice from all the other noises in and around the car (not to mention the other side of the phone call) and you're in trouble.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 9, 2020 19:37 UTC (Mon) by flussence (guest, #85566) [Link] (2 responses)

pavucontrol creates ~30Hz taps for all sources/sinks in order to show the fancy VU meters. It could capture the raw streams and pay the CPU cost in its own process, but then someone else would probably complain that PA stops working over their wifi when it's open.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 9, 2020 21:03 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

Could it change the strategy based on whether the networking support is loaded or not?

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 9, 2020 21:32 UTC (Mon) by smurf (subscriber, #17840) [Link]

Yeah, I know. But that's not a good-enough excuse for 50% load. That 5% is caused by one or two application sources going to one hardware sink. Everything else is dormant. Thus PA's pavucontrol would only have to enable my three hardware sources, preferably with suitably lowered bitrate. Thus, 15% at most. Everything above that is bad design (and would kill audio on a system that does actual multi-channel processing).

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 10, 2020 5:12 UTC (Tue) by foom (subscriber, #14868) [Link] (2 responses)

But if I _do_ have a sink that delays by half a second (eg Bluetooth sometimes), what's the audio system gonna do better than to report it?

Sure, that's an extremely obnoxious delay, but isn't it good if it can deal with that reality as best as possible, instead of falling over?

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 10, 2020 6:50 UTC (Tue) by smurf (subscriber, #17840) [Link]

There's a number of things you can do. Like blocking the connection of a new sink if it introduces a delay of more than .1 second or whatever, or keeping it out of the loop.

For instance, a phone call on Bluetooth with that kind of delay is annoying but survivable, while he same thing on speaker is not. Also, a phone call shouldn't suddenly goes from .05 to .5 seconds latency because Pop's hearing aid also wants to listen in.

Decisions like these (do you *want* Pop to listen? should his microphone be cross-connected? or should the call switch to him entirely?) should be under the purvey of a policy manager, not any single application. PA doesn't have that.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 10, 2020 9:55 UTC (Tue) by abo (subscriber, #77288) [Link]

If the audio system is also the video system it can delay the video to match.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 10, 2020 10:42 UTC (Tue) by intgr (subscriber, #39733) [Link] (5 responses)

> Open pavucontrol and the thing goes from 5% to 50% CPU
That may be true but it's a red herring; I'm interested in behavior under normal circumstances. The pavucontrol UI is deprecated and normal users don't keep it open for long periods of time. pavucontrol is not compatible with PipeWire so it's not like PipeWire can improve over that anyway.

> but it doesn't have hard bounds on its latency either. You add a sink that delays for half a second? fine, that's your new channel latency
In normal circumstances you output to a single sink at a time. If your sink has high latency, well, there's nothing that PipeWire can do to improve that either, right?

Please, let's stop the hand-waving and anecdotes and concentrate on what PipeWire actually TECHNICALLY does (or even theoretically could do) that's so much better.

My issue is that TFA and other official PipeWire sources make sweeping claims suggesting that PulseAudio is high-latency, resource-intensive etc and then do nothing to back those claims up, nor explain how they improve over it. And this comment thread is just another example of that.

For comparison, PulseAudio has technical explanation about how they implement a *low-latency* and low-wakeups (i.e. not resource intensive) sound server: http://0pointer.de/blog/projects/pulse-glitch-free.html and it seems to me it does everything it can within the constraints of the hardware.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 13, 2020 11:50 UTC (Fri) by jschrod (subscriber, #1646) [Link] (1 responses)

> The pavucontrol UI is deprecated

Could you please elaborate what should be used instead?

https://www.freedesktop.org/wiki/Software/PulseAudio/About/ doesn't list pavucontrol as deprecated; i.e., it's not in the section "obsolete or out-of-date UI tools".

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 13, 2020 21:15 UTC (Fri) by intgr (subscriber, #39733) [Link]

> doesn't list pavucontrol as deprecated

Sorry you are right, it's not deprecated. I may be confusing it with "paman".

But it's my understanding that pavucontrol was not intended to be the primary UI, in favor of tools integrated into the desktop system, such as "gnome-control-center sound". This sentiment is already present in a 2007 post by Poettering about integrating PulseAudio with Gnome. [1] So I don't consider CPU usage of a specialist UI to reflect badly on PulseAudio itself.

[1] https://mail.gnome.org/archives/desktop-devel-list/2007-O...

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 14, 2020 6:53 UTC (Sat) by wtay (guest, #55923) [Link]

> Please, let's stop the hand-waving and anecdotes and concentrate on what PipeWire actually >TECHNICALLY does (or even theoretically could do) that's so much better.
>
>My issue is that TFA and other official PipeWire sources make sweeping claims suggesting that >PulseAudio is high-latency, resource-intensive etc and then do nothing to back those claims up, >nor explain how they improve over it. And this comment thread is just another example of that.

I have posted some Performance measurements between PipeWire, PulseAudio and JACK2 on the Wiki now: https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/...

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 14, 2020 13:15 UTC (Sat) by wtay (guest, #55923) [Link] (1 responses)

>pavucontrol is not compatible with PipeWire so it's not like PipeWire can improve over that anyway.

You can run pavucontrol under pipewire just fine and I often do to check signals and change volumes etc.

Pipewire runs the peaks resamplers in the pavucontrol process. The effect is that the load is shifted from the server to the client. I don't have numbers to compare but I remember pavucontrol uses approximately the same CPU while the Daemon much much less in the case of pipewire.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 14, 2020 16:58 UTC (Sat) by smurf (subscriber, #17840) [Link]

The flip side is that under PipeWire pavucontrol needs a whole lot more network bandwidth. Not an issue locally but a potential problem when we're talking about a larger network of servers.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 10, 2020 9:57 UTC (Tue) by abo (subscriber, #77288) [Link]

PulseAudio was created back when single core Pentium 4:s was still a thing, but I remember having to temporarily kill it and configure VLC to go directly to ALSA to get smooth video playback.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 13, 2020 16:01 UTC (Fri) by motiejus (subscriber, #92837) [Link] (3 responses)

Can Pipewire transfer its payload over network? Like netjack*[1] or pulseaudio[2]?

I would love to send digital audio from desktop to my receiver without an hdmi cable with reasonable latency (may rule out pulseaudio) and with least moving parts (rules out pulseaudio+jack).

I tried looking around, to no avail.

[1]: https://jackaudio.org/faq/netjack.html
[2]: https://wiki.archlinux.org/index.php/PulseAudio/Examples#...

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 14, 2020 12:51 UTC (Sat) by wtay (guest, #55923) [Link]

>Can Pipewire transfer its payload over network? Like netjack*[1] or pulseaudio[2]?

It does not have anything nicely integrated yet but you can run zita-n2j for example

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 19, 2020 12:21 UTC (Thu) by mgedmin (subscriber, #34497) [Link] (1 responses)

When I last tried to transfer sound over the network with PulseAudio over wifi, I got regular skips every couple of minutes or so seconds. Annoying, unusable. (Today I suspect that what I experienced was my wifi driver doing periodic background scans so the network manager menu could show me all the avalable wifi networks. I think it can be turned off, but how many users will be able to figure it out?)

What worked surprisingly well was running mplayer over ssh so the sound played locally, and the video was displayed over X11 over wifi.

This was maybe in 2005, so I would love to hear how well things work nowadays.

Bouzas: PipeWire, the media service transforming the Linux multimedia landscape

Posted Mar 19, 2020 12:33 UTC (Thu) by smurf (subscriber, #17840) [Link]

Not much better, last time I tested that. Wifi is not a realtime transport … I have taught my Wifi receiver (just one thankfully) to get its stream via icecast, that tends to work OK.


Copyright © 2020, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds