TDC: Audio routing for vehicles

One of the oft-cited tensions in the Tizen project (as in its predecessor, MeeGo) is that the end goal is to offer a common application platform for some Linux systems that are strikingly different under the hood. That is, application developers are understandably interested in offering their social networking apps and games on phones, smart TVs, and in-vehicle entertainment units—but the hardware and environments defining these products can be radically different. Phones have battery life and tiny screens to consider; smart TVs have the "ten foot UI" and users' aversion to keyboards. But perhaps the clearest challenge in this space has always been in-vehicle infotainment (IVI) systems, where audio routing is the canonical illustration of how different the car is from a desktop Linux system. Media playback, hands-free phone calls, navigation, and alert messages all contend for the right to deliver audio to the driver and passengers, without distracting the driver.

Intel's Jaska Uimonen presented the solution in the works for Tizen IVI at the 2013 Tizen Developer Conference in San Francisco. The audio management problem, he explained, comes down to three core issues: routing, policy-controlled volume, and policy-controlled preemption. The routing issue involves multiplexing the multiple audio input and output points of a vehicle (such as speaker sets for front and rear seats, headphone jacks, hands-free microphones, traditional audio dash-units, and hot-pluggable Bluetooth devices—a category that can include headsets and mobile phones). The users' expectation is that audio will always be routed to the correct zone, but a wide array of combinations is possible, and configuration can change on the fly. For example, an incoming hands-free phone call may need to target the driver alone, while the radio continues to play to the other seats (perhaps including the front passenger seat). This differs from mobile device audio management, Uimonen said, which typically makes a "select and route" decision; IVI audio requires a full switching matrix.

But there is still more. In addition to simply connecting each audio stream to the appropriate port, Uimonen said, the audio manager must also mix the volumes of the audio streams in a useful and safe manner. Turn-by-turn navigation directions are more important to hear than the radio, and thus the radio volume should dip momentarily for each navigation instruction—and hopefully do so smoothly. Then again, there are also situations when one audio stream may need to preempt others entirely, such as a safety alarm. Exactly which preemption and volume scenarios are appropriate is a system policy decision, of course, which will likely vary from manufacturer to manufacturer. Thus Tizen IVI's audio management framework needs to be flexible enough to support configurable policies.

Sound off

The audio management solution developed for Tizen IVI uses PulseAudio for routing, which is not surprising, although it does introduce some new code to support the notion of audio "zones" (e.g., driver, passenger, rear seat entertainment unit, etc.) which is not found in PulseAudio on the desktop. It implements the necessary policy-based decision making with Murphy, a relatively new "resource routing" engine started in 2012.

Murphy maintains an internal database of system state for whichever resources it is responsible for routing (audio, video, and network streams are the example uses). Policies for resource control are described in Lua statements, and separate plugins can implement control for each resource Murphy monitors. The Murphy daemon will listen for events from different sources (such as udev or Websockets), and can execute scripts or C programs to re-route resources in response.

For example, in the IVI audio scenario, a Bluetooth headset appearing or disappearing would trigger a D-Bus event from BlueZ, while a speed-sensitive volume adjustment might read speedometer data from the vehicle's CAN bus or Automotive Message Broker. The Murphy-driven audio manager runs as a plugin to PulseAudio; thus when a Murphy policy rule dictates routing a particular input to a particular output, the plugin should connect the associated PulseAudio sources and sinks.

But the actual implementation is a bit more complicated, because PulseAudio does not natively support certain useful options, such as combining sinks and sources or combining multiple sinks. Thus, the Tizen IVI audio manager adds the concept of "nodes"—essentially an abstraction layer, nodes represent a "routing endpoint" which might have a source (e.g., microphone) and a sink (e.g., speaker). A telephony application needs both to function, so the routing policy treats them as a single object. Playing audio over all of the speaker sets in the vehicle might actually involve routing the same audio stream to multiple independent speaker clusters, so the routing policy offers a "combined" node that serves them all.

The audio policies are based around tagging each audio stream with a class; for example, "music," "navigation," or "ui event." The system integrator can then write routing, volume, and preemption rules based on the class of the stream and the various nodes in the car. Obviously not every possible node will be present when the car leaves the factory (Bluetooth headsets and portable music players being the most obvious examples), but there are generally a fixed number of speaker and microphone devices, and only a handful of well-understood hot-pluggable device types to worry about.

When an application creates a new audio stream, the system automatically creates a default route for it based on its class (which might specify that "music" streams are routed to the whole-car speaker system, but "navigation" streams are routed to the front-seat speakers only). Users can request an explicit route to override the default if desired, assuming there is application support, that is.

This explicit routing feature is designed to work with GENIVI's audio management system, although GENIVI's system is not currently built for Tizen images. However, the GENIVI system does define the client-side interface through which applications can request specific audio routing, and since GENIVI is an industry-wide effort, the Tizen system supports the interface. As for application support within the existing Tizen application ecosystem, Uimonen noted that Tizen's WebKit runtime has been patched to support tagging streams with classes; HTML5 applications can tag their <audio> and <video> elements. There is no word yet on progress of the feature in Tizen's native application APIs.

The answer is simple. Volume.

The volume and preemption rules for audio streams prioritize certain classes over others, Uimonen explained, but the audio manager's decision to lower the volume of a stream is hidden from the application itself. Permitting applications to know when their volume level was being lowered in the mix might have some unwanted effects, he said—misbehaving applications might fight it by raising the volume of the stream, which is dangerous on its own but could also potentially lead to race conditions between competing applications.

The audio manager also adds one more feature to PulseAudio: the ability to smoothly ramp up or ramp down the volume of a stream. The plan is to send that patch upstream, Uimonen said, although it has not been sent yet. The other code is available in the Tizen repositories under the "IVI" profile. All of the code to implement the audio manager will be open source, however, the actual policies and configurations are hardware-specific; as with other such system configuration points, it may be expected that many auto manufacturers will choose not to release their audio routing config files or scripts.

Uimonen commented that the volume ramping work and the other additions to PulseAudio are hopefully going to be merged with the project upstream, because there are use cases outside of the IVI context that would benefit from the ability to perform more complex audio routing. That list might include home theater systems or whole-house audio setups, but even the basic desktop configuration these days is likely to incorporate multiple sound devices (Bluetooth, USB audio adapters, and so on). For the moment, then, the unusual demands of the IVI industry may be pushing development, but ultimately everyone stands to gain.

[The author wishes to thank the Linux Foundation for travel assistance to Tizen Dev Con.]