Mozilla raised eyebrows in mid-March when a patch materialized that
would allow Gecko to fall back on operating system or hardware media
decoders for multimedia content — in particular for patent- and
royalty-encumbered codecs like H.264, which are not supported natively in
Gecko. The project had fought hard to promote the adoption of unencumbered
alternatives (such as Ogg Theora or Google's WebM), so many on the mozilla.dev.platform discussion group saw enabling any support for H.264 as a violation of principle. Mozilla's Chief Technology Officer Brendan Eich argued, however, that the decision is an improvement over the existing Flash-fallback method, that Mozilla has more important fights to focus on, and that the blame for WebM's snail-paced adoption lies squarely at the feet of Google.
History of the WebM, part 1
Eich posted his take on the situation in a round-up at his own blog (which was then syndicated at the official Mozilla Hacks blog), which started with a history lesson on H.264, WebM, and the HTML5 <video> element. As far back as 2007, Eich and Mozilla had argued for the standardization of the <video> and <audio> elements to include "unencumbered" baseline codecs — at the time, Ogg Vorbis for audio and Ogg Theora for video. Eich argued that the term "unencumbered" most accurately describes the state of the codecs needed to ensure that the open web remains open; the issue is not about open source (for there are open source implementations of encumbered codecs), nor is it about patents (for standards bodies can and will accepted a patented codec if the patent holders agree to license it under royalty-free terms).
In 2007's <video> element fight, the main opponent of
Theora was H.264, which was being pushed by a royalty-collecting consortium
of companies. The protracted battle ultimately resulted in a non-decision,
with the default codec language being removed from the draft standard. But
the situation appeared to take a sudden shift in favor of unencumbered
codecs in 2009, when Google purchased codec-maker On2 and released the WebM codec under unencumbered
WebM was far newer than Theora and offered quality roughly on par
with H.264; Theora's relative performance was a big reason Google did not
it as the default codec.. Google subsequently announced its intention to transcode YouTube videos to WebM, and in 2010 Adobe announced that it would support WebM in its Flash products (meaning not just the browser plug-ins, but the content-creation tools and media-delivery servers as well). In January 2011, Google went a step further, and publicly announced that it would drop support for H.264 from its Chrome browser.
But that change never happened. 14 months later, Chrome still supports
H.264, and Eich and other Mozilla employees report that Google has remained
silent about the decision when asked. Adobe didn't implement the WebM
support that it promised either.
Meanwhile, Eich said, H.264 adoption has continued to spread, which has hampered Firefox's growth. For starters, Google's oft-cited promise to trancode YouTube's content to WebM is not all it is cracked up to be: to date Google has only transcoded half of the site's videos, and more importantly, YouTube only delivers WebM content to desktops, and only for those videos that serve no ads (which Eich said makes up a shrinking portion of the total). No other major sites have rejected H.264 in favor of WebM delivery either, while the consumer electronics industry builds more and more H.264 encoding support into video cameras.
On the desktop, Firefox falls back on the Flash plug-in's H.264 support, but on mobile devices, there is no such option. Mozilla believes that mobile browsers are clearly the battlefield deserving the most attention, and on top of that, the project's Boot to Gecko (B2G) effort stands no chance of making it into device-makers' products without H.264 support, Eich said.
Patches and a new API
The combination of Google not pushing WebM and content creators adopting
H.264 puts Mozilla in an untenable position, Eich said. Yet the majority
of phone and tablet designs ship with a pre-authorized (meaning the royalty
fees have been paid) H.264 decoder in silicon, which is what led developer
Andreas Gal to propose letting Gecko hand H.264 decoding (and perhaps other
encumbered formats like MP3 and AAC) down to the OS or hardware level.
Although other option have been discussed, Eich endorses Gal's solution.
The upshot is that Firefox and B2G users will be able to see H.264
content on platforms that support it, Mozilla will not have to pay royalty fees (or pass them on to users), and the project can turn its attention to fighting for unencumbered codecs in the next round of standardization battles.
Those battles are not far off, Eich said, starting with the WebRTC
real-time chat standard — which is already in-progress and looks
poised to recommend unencumbered codecs. There will be other fights, he
said, and other generations of video streaming codecs. Continuing to
ignore H.264, particularly on mobile devices, would ultimately risk making
Mozilla irrelevant further down the line, if not nonexistent altogether:
Losing a battle is a bitter experience. I won’t sugar-coat this pill. But
we must swallow it if we are to succeed in our mobile initiatives. Failure
on mobile is too likely to consign Mozilla to decline and irrelevance.
Gal's patch is attached to bug 714408.
In essence, it creates a new API (which Gal dubbed the Media Player API, or
MPAPI) for use by OS or hardware decoders. On desktop OSes like Linux,
MPAPI would likely be tied in to a media framework like GStreamer (which
can use encumbered codecs). It is also possible that the Mozilla Flash
plug-in could continue to serve as the H.264 decoding chain on desktop
systems, which would require work to expose MPAPI to the Gecko's main
plug-in API, NPAPI. Of course, Adobe has also said that it will stop developing its NPAPI Flash
plug-in for Linux, which means Flash-fallback will not remain a solution
for free OSes in the long term.
As of now, the patch
itself is the main source of information on MPAPI, which is still very much
a work in progress. MPAPI would hand audio and video frames back to the
main Gecko rendering toolchain, however, meaning the content would be fully
<video> or <audio> element. The vast
majority of the energy expended about the move has not been with its technical points, but with whether or not Mozilla's decision itself is wise, foolish, too pessimistic, or long overdue.
The mozilla.dev.platform discussion thread about Gal's proposal is rife with critics arguing that Mozilla is making the wrong move, and with defenders inside and outside the project. The criticisms fall into three basic camps: those that think the H.264 situation is not as bad as described, those that feel Mozilla has not tried hard enough to advocate WebM (or that it should try Just One More Time), and those that object to enabling any form of H.264 playback on principle alone.
Critics who say that WebM has not lost to H.264 tend to point to Google's YouTube transcoding effort (which Eich countered in his blog post), or hold out hope that Google will indeed drop H.264 support from Chrome (often based on the number of WebM "supporters" listed). On the latter point, Eich argues that Chrome's H.264 support is moot, because the desktop browser would simply fall back on the Flash plug-in like Firefox does today. In fact, he said, Chrome's heavily optimized Flash plug-in amounts to a "practically custom" Flash offering "best-of-breed fallback."
Christopher Blizzard added
that ultimately, the content-delivery sites simply are not interested in
I keep talking to people building sites and there are only a couple of
organizations that are willing to embrace WebM because it's the right thing
to do. Transcoding & hosting costs are huge. Beyond that I've not really
run into anyone who wants to do WebM. It's just seen as a cost that
Firefox is incurring on web developers.
Justin Lebar typified the position that Mozilla is giving up too
it "going down without a fight," and saying that he would:
publicly call on Google to fulfill its promises of old. I'd communicate
through official channels why we don't want to support H.264, MP3, etc,
and why we think Google is harming the web. [...] And I might set a
public deadline — if Google doesn't un-support H.264 by date X, then
we'll start supporting system H.264 and MP3 codecs.
Side-stepping the fact that Lebar's public deadline idea paradoxically threatens to increase support for H.264 if it is not abandoned, Mozilla's Robert O'Callahan calls it "grossly unfair" to suggest that Mozilla has not fought hard enough or long enough against H.264 adoption:
We have fought. We, alone of all major browsers (sorry Opera desktop), have held out against supporting patent-encumbered codecs for a long time. I feel it's grossly unfair to our efforts to describe that as "not a real fight".
We held the line in the hope that the industry would follow, and that Google would do a lot to improve and support WebM, especially removing H.264 support from Chrome. So we've held the line, and watched, and waited, and personally I am extremely disappointed by the results.
Likewise, Asa Dotzler confirmed that Mozilla has spent months trying to get a response from Google on the H.264 support question, only to be met with silence. O'Callahan also observed that there is already mobile hardware capable of decoding WebM video, but that Google does not enable it on Android devices. In the absence of support from the format's owners, Mozilla says, it alone cannot move WebM forward.
The objection on principle is trickier. Some in the discussion thread
expressed personal hurt that Mozilla was not standing its ground against
H.264 playback support, but more were concerned that relenting would make
it harder for the organization to lobby against encumbered formats in the
future. Eich argued that anyone who ignores the fact that Firefox users watch H.264 video via the Flash plug-in is "hiding behind Mother Adobe's skirts" and is not taking a "realistic view of the entire fallback logic chain, and of Firefox's current acute dependency on Flash," which is not different in kind from falling back on an OS decoder. Gal concurred, noting that the MPAPI proposal is only "using existing accelerated decoders that already are licensed and available on the system."
Regarding Mozilla's ability to advocate for open and unencumbered formats in the future, Doztler said that the project has backed down on other lost battles in past, such as the document.all DOM feature, but has maintained it credibility. Mozilla can influence the web, and from time to time kill a bad idea, he said in another message, but "sometimes the Web decides to go where we don't want it;" not supporting it only costs the project developers and users.
Ultimately, though, Dotzler argues that even in light of H.264's popularity, Mozilla's WebM advocacy does not constitute a wasted effort:
WebM is in much better shape because of Mozilla's efforts. Not only is WebM in better shape, but I think it actually proves that open codecs can compete. It didn't win, but it demonstrated viability and it may yet go on to claim a critical role in WebRTC.
Finally, I think there's something important in our having taken that stance. We've demonstrated that we don't default to "what's easy". We may not be able to win every battle, but we don't shy away from fighting the good fight.
A continuing story
It is worth remembering that however one feels about the H.264 lobby
and its royalty-collecting schemes, the presence of dedicated video
decoding chips is hardly an isolated situation. There are currently a
great many components in our computers which are covered by patents, and
many chips for which we do
not have source code. Yet enabling software access to those components
is not perceived as a violation of principle. Furthermore, it is hard to
argue that the Flash plug-in that Firefox currently falls back on is turf
worth fighting for — it has a history of bugs and security holes
unrelated to the video decoders it ships.
The debate over enabling H.264 playback via MPAPI shows little sign of calming down. Eich, Dotzler, Gal, and the other project members continue to argue that enabling the format is a purely pragmatic move, and that it would be a better use of Mozilla's energy to combat encumbered codecs on the still-in-development format battles.
They got a boost on March 18 when Mozilla chief Mitchell Baker posted
her own blog
entry in support of the proposed change. Baker said that "giving
our users a great experience" is both one of the project's key
values, and a demanding goal that drives realistic product development.
It's possible to fall into the view that the only way to live up to Mozilla
values is to ship the product we think people should want. This aspect is
one element, but it's not the only one. Another critical element is
shipping products that work for people now so they can love them.
The comment thread on Baker's blog follows much the same pattern as the discussion group. There are supporters, commiserators, and vocal critics. But wherever H.264 itself ends up on future versions of Firefox and B2G, one thing is for sure: H.264-vs-WebM is not the last codec fight the software world will see. As several in the thread pointed out, progress on H.265 is already well underway, and the players involved are similar — there can be little doubt that the battle will be similar, too.
to post comments)