Development

GStreamer Conf: The approach of GStreamer 1.0

The GStreamer framework handles media capture and delivery for many of desktop Linux's most popular applications, but more than ten years out, it still has not made a 1.0 release. To be sure, the actual version number does not matter nearly as much as the stability of the software and the reliability of the API, but GStreamer's 1.0 release has seemed tantalizingly close for the past several months. The third annual GStreamer Conference was held in San Diego the last week of August, and, while 1.0 did not make its debut at the event, the core developers did give a solid deadline, which should be welcome news to application authors and distributions alike.

Ironically, one of the key impediments to a 1.0 release has been the success of the current (0.10) stable release. With the framework working well, a 1.0 release might seem like a simple project governance decision. But because an N.0 release carries with it the implicit promise of API and ABI stability, the project decided it had to implement several key changes before it could put the final stamp on the release. GStreamer maintainer Wim Taymans opened the conference with a keynote talk covering the 0.11 development cycle, including what needed to change from the 0.10 series.

The two biggest challenges, he said, were the advent of embedded Linux (often system-on-chip (SoC) boards) systems using GStreamer and the move toward GPU-accelerated rendering on the desktop. SoC platforms posed a challenge because they include substantially different hardware, such as different memory architectures and hardware rendering pipelines (which could include discrete components such as a video scaler unit). GPU acceleration on the desktop can result in significant performance gains, but it comes at the cost of supporting multiple, incompatible GPU libraries from the various hardware vendors. Those library incompatibilities are not limited to the lowest levels, either. Some GPU chipsets support higher-level features like subtitle compositing, while others do not.

Along with the changing face of hardware, the project had several wishlist items it wanted to implement for a 1.0 release. One was an infrastructure that allowed a GStreamer application to attach useful non-media metadata to a buffer. Such data might include the coordinates of a "region of interest," for example. If the GStreamer pipeline processing a video knows that it will need to crop the video in order to display it, the optimal performance is usually obtained by marking the region, but let the video hardware do the cropping at render time. There are other uses for attaching metadata, of course, such as tagging parts of an image with facial recognition information. Along similar lines, GStreamer 1.0 will support passthrough of compressed audio to the playback hardware. This allows headsets and USB sound cards (an increasing number of which support MP3 or AAC decoding in hardware) to handle the audio stream, again saving CPU cycles.

Last but not least, the GStreamer team wanted to fix the outstanding issue of dynamic pipelines. GStreamer allows application authors to construct pipelines that connect media "source" elements, apply transformations and filters, then deliver the stream to a "sink" element. But up through the 0.10 series, changing the pipeline after its initial setup was painful. Pulling out and replacing an element caused downstream elements to lose context information like timing and formatting details, forcing the application to re-examine the pipeline. It was not esoteric use-cases hit by this limitation, either: plugging in a headset while audio was playing forced a pipeline change (by using a different sink element), as did applying a video filter in an application like Cheese (by inserting a filter element).

On top of the wishlist features were the usual improvements to memory management and cleanups that accumulate over the lifetime of a complex framework. Version 1.0 allows applications to pool buffers to reducing copying, query the supported memory types (such as video acceleration API (VA-API) surfaces), and split buffers up into separate memory blocks (which could allow faster handling of streaming media like VoIP packets). Also 1.0 strives to simplify application authors' work by providing a number of new audio and video "base classes" that represent the most commonly-needed media formats — in contrast to 0.10, which required the author to fully describe each media format.

Taymans reported that the 0.11 code had replaced GStreamer's master branch in March 2012, and was currently at version 0.11.93. The team is "kind of out of ideas of things to add", he said, and most of the common media players using GStreamer have already been ported over to the 0.11/1.0 API. Nevertheless, there are a few pieces that require more testing, including the dynamic pipeline support and the new base classes. However, the Linux distributions may have other challenges that hold up deployment. For example, he said, the Fluendo MP3 codec and DVD playback element shipped by Ubuntu are built to work on 0.10, and may not be updated for 1.0.

GStreamer 0.10 and 1.0 will be installable in parallel, which means Ubuntu could include the Fluendo elements in a 0.10 pipeline and still use 1.0 elsewhere. But this is not a good long-term solution, Taymans added. For one thing, 0.10 is deprecated and will eventually stop receiving updates. But the more pressing issue from GStreamer's perspective is that GNOME has decided to use 1.0 for its upcoming GNOME 3.6 release. GNOME cannot reasonably be expected to base its entire desktop environment on a deprecated version of GStreamer, but it cannot ship with a dependency on an unreleased branch either. As a result, GStreamer will formally release its version 1.0 before the release of GNOME 3.6. That does not give a precise date (although Taymans said the team hoped to settle on one over the course of the conference), but it does provide a more concrete anchor to the when-will-1.0-arrive question than GStreamer users have had up until now. GNOME 3.6 is scheduled for release on September 26, and while it might slip a few days, that still places GStreamer 1.0 within the month.

GStreamer also saw the first release of its software development kit (SDK) this summer, as Andoni Morales explained in another session. The GStreamer SDK is a bundle designed to help GStreamer application authors by providing stable, installable builds for Linux, Windows, Mac OS X, and soon for Android as well. The issue that led to the SDK's creation is that despite GStreamer's staunch commitment to being a cross-platform media framework, it is still predominantly a Linux-only tool. But a big part of the reason behind that situation is the lack of installers for Mac and Window developers. The SDK provides binary installers, plus tutorial-based documentation. In addition, it splits up the GStreamer package into more modular components — the "good,""bad," and "ugly" plugin packages found in Linux repositories are not particularly descriptive, and they are quite large.

The first version of the SDK is based on 0.10, and provides a tested set of playback codecs, plus Python bindings and integration with Apple's Xcode IDE and Windows' Visual Studio. Several of the other talks over the course of the event also dealt with GStreamer development tools, including Rene Stadler's debugging tool (which turns GStreamer's famously verbose log files into easier to parse visualizations) and Edward Hervey's survey of CPU and memory optimization utilities. Consequently, whenever GStreamer 1.0 does arrive, it looks as if the project will be prepared to make a bigger push to engage application developers.

Comments (1 posted)

Quotes of the week

— Miguel de Icaza

— Dan Williams

Comments (13 posted)

Diaspora to become a "community project"

The Diaspora project, aiming to create a more freedom-oriented social networking system, has announced that it is becoming a more community-oriented project. "This will not be an immediate shift over. Many details still need to be stepped through. It is going to be a gradual process to open up more and more to community governance over time. The goal is to make this an entirely community-driven and community-run project. Sean Tilley, our Open Source Community Manager will spearhead community efforts to see that this happens."

Comments (11 posted)

Sony opens up the Dynamic Android Sensor HAL (DASH)

Sony has announced that the Dynamic Android Sensor Hardware Abstraction Layer (HAL) is now open for collaboration on GitHub. Since DASH was opened up in February, there have already been contributions from the CyanogenMod team, and this move is meant to foster more of that. "As a next step, we are now making DASH available as an open source project on GitHub. Here, custom ROM developers can find the source code files and “make” files for the sensors in Xperia™ smartphones, which is used to enable and disable multiple sets of sensors for each device, including the accelerometer, proximity sensor, ambient light sensor, magnetometer, gyroscope, and pressure sensor. We plan to keep adding more sensor code as we release new phones. Anyone in the Android open community is free to contribute their own sensor implementations and other improvements to the DASH code."

Comments (12 posted)

"We the people" source released

The White House (the office of the US president) has made its "We the people" platform, the Drupal-based system that handles its petitions site, available on Github under the GPLv2+ license. "Releasing the source code for this application is meant to empower other governments and organizations to use this platform to engage their own citizens and constituencies. In addition, public review and contribution to the application’s code base will help strengthen and improve the platform."

Comments (10 posted)

Systemd v189 available

Lennart Poettering has announced release 189 of systemd, sporting a number of changes, including deprecation of reading kernel messages from /proc/kmsg in favor of /dev/kmsg, and cryptographig "sealing" of log files to prevent attackers from modifying logs undetected.

Full Story (comments: none)

MongoDB 2.2 Released

Version 2.2 of MongoDB is available. This release incorporates a number of new features, most notably a new aggregation framework and "data center awareness" to better manage geographically separate clusters.

Full Story (comments: none)

Development newsletters from the last week

• Caml Weekly News (August 28)
• What's cooking in git.git (August 27)
• Haskell Weekly News (August 22)
• Mozilla Hacks Weekly (August 23)
• OpenStack Community Weekly Newsletter (August 24)
• Perl Weekly (August 27)
• PostgreSQL Weekly News (August 26)
• Ruby Weekly (August 23)

Comments (none posted)

Oracle Broadens Support for Open-source R Analytics (PCWorld)

At PCWorld, Chris Kanaracus is reporting that Oracle has announced several new initiatives supporting the open source statistical analysis tool R. Ports to Solaris and AIX are among the new offerings, as is compatibility with some additional tools. "Oracle will deliver an open-source distribution of R that can work with the Sunperf, MKL and ACML math libraries, which are provided by Oracle, Intel and AMD, respectively. This will allow R 'to run extremely fast on all compatible hardware,' Oracle said in a statement".

Comments (16 posted)

Has cash corrupted open source? (The Register)

At The Register, Matt Asay discusses the effects that the increasing number of corporate-sponsored open source projects have on development. "There was a brief honeymoon when Google, Facebook and other tech giants were able to release open-source code without commercial involvement, but this didn't last long, with startups setting out to monetise projects such as Hadoop, Cassandra, and Storm."

Comments (30 posted)