The GStreamer framework handles media capture and delivery for many
of desktop Linux's most popular applications, but more than ten years out,
it still has not made a 1.0 release. To be sure, the actual version number
does not matter nearly as much as the stability of the software and the
reliability of the API, but GStreamer's 1.0 release has seemed tantalizingly
close for the past several months. The third annual GStreamer Conference was
held in San Diego the last week of August, and, while 1.0 did not make its debut at the event, the core
developers did give a solid deadline,
which should be welcome news to application authors and distributions alike.
Ironically, one of the key impediments
to a 1.0 release has been the success of the current (0.10) stable
release. With the framework working well, a 1.0 release
might seem like a simple project governance decision. But because an N.0 release
carries with it the implicit promise of API and ABI stability, the project decided
it had to implement several key changes before it could put the final stamp on
the release. GStreamer maintainer Wim Taymans opened the conference with a
keynote talk covering the 0.11 development cycle, including what needed to change
from the 0.10 series.
The two biggest challenges, he said, were the advent of embedded Linux (often system-on-chip
(SoC) boards) systems using GStreamer and the move toward GPU-accelerated rendering
on the desktop. SoC platforms posed a challenge because they include substantially
different hardware, such as different memory architectures and hardware rendering
pipelines (which could include discrete components such as a video scaler unit).
GPU acceleration on the desktop can result in significant performance
gains, but it comes at the cost of supporting multiple, incompatible GPU libraries
from the various hardware vendors. Those library incompatibilities are not
limited to the lowest levels, either. Some GPU chipsets support higher-level features
like subtitle compositing, while others do not.
Along with the changing face of hardware, the project had several wishlist items
it wanted to implement for a 1.0 release. One was an infrastructure that allowed a
GStreamer application to attach useful non-media metadata to a buffer. Such data
might include the coordinates of a "region of interest," for example. If the GStreamer
pipeline processing a video knows that it will need to crop the video in order to
display it, the optimal performance is usually obtained by marking the region, but let the video hardware do the cropping at render time. There are other
uses for attaching metadata, of course, such as tagging parts of an image with facial
recognition information. Along similar lines, GStreamer 1.0 will support passthrough
of compressed audio to the playback hardware. This allows headsets and USB sound cards
(an increasing number of which support MP3 or AAC decoding in hardware) to handle
the audio stream, again saving CPU cycles.
Last but not least, the GStreamer team wanted to fix the outstanding issue of
dynamic pipelines. GStreamer allows application authors to construct pipelines that
connect media "source" elements, apply transformations and filters, then deliver the
stream to a "sink" element. But up through the 0.10 series, changing the pipeline after
its initial setup was painful. Pulling out and replacing an element caused downstream
elements to lose context information like timing and formatting details, forcing the
application to re-examine the pipeline. It was not esoteric use-cases hit by this limitation,
either: plugging in a headset while audio was playing forced a pipeline change (by using
a different sink element), as did applying a video filter in an application like Cheese (by inserting a filter element).
On top of the wishlist features were the usual improvements to memory management
and cleanups that accumulate over the lifetime of a complex framework. Version 1.0 allows
applications to pool buffers to reducing copying, query the supported memory types (such as
video acceleration API (VA-API) surfaces), and split buffers up into separate memory blocks
(which could allow faster handling of streaming media like VoIP packets). Also 1.0 strives to simplify application authors' work by providing a number of new audio and video "base classes" that represent the most commonly-needed media formats — in contrast to 0.10, which required the author to fully describe each media format.
Taymans reported that the 0.11 code had replaced GStreamer's master branch in March
2012, and was currently at version 0.11.93. The team is "kind of out of ideas of
things to add," he said, and most of the common media players using GStreamer
have already been ported over to the 0.11/1.0 API. Nevertheless, there are a few pieces
that require more testing, including the dynamic pipeline support and the new base classes.
However, the Linux distributions may have other challenges that hold up deployment. For example, he said, the Fluendo MP3 codec and DVD playback element shipped by Ubuntu are built to work on 0.10, and may not be updated for 1.0.
GStreamer 0.10 and 1.0 will be installable in parallel, which means Ubuntu could include the Fluendo elements in a 0.10 pipeline and still use 1.0 elsewhere. But this is not a good long-term solution, Taymans added. For one thing, 0.10 is deprecated and will eventually stop receiving updates. But the more pressing issue from GStreamer's perspective is that
GNOME has decided to use 1.0 for its upcoming GNOME 3.6 release. GNOME cannot reasonably
be expected to base its entire desktop environment on a deprecated version of GStreamer, but it cannot ship with a dependency on an unreleased branch either. As a result, GStreamer will formally release its version 1.0 before the release of GNOME 3.6. That does not give a precise date (although Taymans said the team hoped to settle on one over the course of the conference), but it does provide a more concrete anchor to the when-will-1.0-arrive question
than GStreamer users have had up until now. GNOME 3.6 is scheduled for release on September 26, and while it might slip a few days, that still places GStreamer 1.0 within the month.
GStreamer also saw the first release of its software development kit (SDK) this summer, as Andoni Morales explained in another session.
The GStreamer SDK is a bundle designed to help
GStreamer application authors by providing stable, installable builds for
Linux, Windows, Mac OS X, and soon for Android as well. The issue
that led to the SDK's creation is that despite GStreamer's staunch
commitment to being a cross-platform media framework, it is still
predominantly a Linux-only tool. But a big part of the reason behind that
situation is the lack of installers for Mac and Window developers. The
SDK provides binary installers, plus tutorial-based documentation. In addition, it splits up the GStreamer package into more modular components
— the "good,""bad," and "ugly" plugin packages found in Linux
repositories are not particularly descriptive, and they are quite large.
The first version of the SDK is based on 0.10, and provides a tested set of playback codecs, plus Python bindings and integration with Apple's Xcode IDE and Windows' Visual Studio. Several of the other talks over the course of the event also dealt with GStreamer development tools, including Rene Stadler's debugging tool (which turns GStreamer's famously verbose log files into easier to parse visualizations) and Edward Hervey's survey of CPU and memory optimization utilities. Consequently, whenever GStreamer 1.0 does arrive, it looks as if the project will be prepared to make a bigger push to engage application developers.
to post comments)