Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web [LWN.net]

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 19:17 UTC (Mon) by nhippi (subscriber, #34640)
Parent article: Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

There is a major difference to the GIF debacle. GIF was always implemented in software. OTOH H.264 decoding happens mostly in hardware[1].

Even on PC's, H.264 is supported in hardware by recent nvidia (purevideo HD) and ati cards (avivo HD)- and I presume intel follow. Lets see if intel will do it on a codec on their GMA graphics chips or with SSE264 extension featuring DECODE264 SIMD instruction will be interesting to see...

Why is that relevant? Because if implemented in hardware, the software doesn't need a patent license. When implemented in hardware, it will be no more relevant than the patented X86 instructions are to users. Sure, they eliminate competition, but it doesn't free software from using them.

I'm afraid mozilla has choosen an fight they can't win. IE, safari and chrome will support H.264, and someone will write a firefox plugin to do it as well. The only thing the fight will create is delay in html5 video adoption - and meanwhile the video still gets distributed in H.264 - in flash container. Which leads to the question:

Is it more important to eliminate flash or H.264? Mozilla has seemingly taken the latter one as more important.

[1] Which actually might be a DSP executing firmware.

to post comments

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 19:28 UTC (Mon) by coriordan (guest, #7544) [Link] (3 responses)

I don't keep up to date on low-level systems programming anymore, so I've two questions that might be obvious:

1. Are the codecs in unwritable memory on the cards? Or do they get put there by injecting binary blobs into the cards on bootup/upgrade? If the latter, then we should replace those binardy blobs with free software. So that "solution" only works if we accept binary blobs?

2. Say I want to encode a video into H.264, and say I want to modify a H.264 video and save it in another format. Wouldn't I need H.264 software on my harddisk for these tasks? The blob in my video card wouldn't do this for me, would it? (I know these tasks aren't what Firefox does, but if we accept it for online video, we'll be stuck with it for other things)

Right? Wrong?

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 19:46 UTC (Mon) by Oddscurity (guest, #46851) [Link] (2 responses)

The way I understand it: nVidia's VDPAU doesn't even do all of the decoding
on the GPU, it just accelerates some parts like colour conversion,
deblocking and other parts of the inner loop.

So it's still software, part of which runs on the CPU as implemented in the
driver, and part of which ends up running on the GPU.

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 19:54 UTC (Mon) by drag (guest, #31333) [Link]

Yeah.

Like stated above there are 22 different companies involved in the Mpeg
group. And if the GSM folks are any indication of how these sort of industry
'IP' groups operate they do everything they can to shovel as many patents
into the pool as possible.

So it's likely that there are dozens and dozens of patents covering all
sorts of different aspects of the codecs, and software and hardware related
to the codecs. So even if the video card folks build codecs-as-hardware they
still only likely take a some patents out of the equation and not all of
them. Meaning that even with hardware acceleration your still not going to
escape from the licensing requirements.

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 21:02 UTC (Mon) by nhippi (subscriber, #34640) [Link]

> The way I understand it: nVidia's VDPAU doesn't even do all of the decoding
> on the GPU, it just accelerates some parts like colour conversion,
> deblocking and other parts of the inner loop.

I believe that is what the early VDPAU versions did. Later versions offload full decoding to the video card. Then again, this is based on external observations of cpu load rather than knowing what the VDPAU library really does..

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 19:34 UTC (Mon) by Imroy (guest, #62286) [Link] (2 responses)

I don't know about all of those pieces of hardware, but I'm sure the nVidia and ATI implementations are simply running on the GPU - I seriously doubt they would devote so much effort and chip area to having a squillion little processing units, and then add separate MPEG-1/2/4 video decoders in hardware as well. So the decoders are still software. And they'd be licensed of course.

Oh, and I can't find anything about this "DECODE264" instruction.

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 19:43 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

"I seriously doubt they would devote so much effort and chip area to having a squillion little processing units, and then add separate MPEG-1/2/4 video decoders in hardware as well."

Yet they do exactly this. And the main reason is power consumption. Specialized ASICs are still faster even compared to GPGPU.

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 20:47 UTC (Mon) by nhippi (subscriber, #34640) [Link]

> Oh, and I can't find anything about this "DECODE264" instruction.

It was a joke. The latest intel processors already have AESDEC and AESENC so..

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 25, 2010 19:42 UTC (Mon) by Per_Bothner (subscriber, #7375) [Link] (9 responses)

Remember, broadcasters also have to pay non-trivial money to the patent pool, so you can't host an H.264 on your web-site without infringing. Also, those prices are scheduled to go up, so some big players (possibly including Google) have an incentive to find an alternative.

Is it more important to eliminate flash or H.264? While Flash is more annoying because of bugs etc, it is in principle the lesser evil, since (I believe) it can be legally re-implemented in Free Software. H.264 cannot, in many parts of the world.

how implementable Flash is

Posted Jan 25, 2010 21:12 UTC (Mon) by coriordan (guest, #7544) [Link] (3 responses)

I'd've thought that implementing Flash would raise lots of patent problems, but this doesn't seem to have happened. When I heard Rob Savoye of Gnash complaining about software patents last February, he was complaining about codec patents.

I can't find any info online right now, but I seem to remember him saying the the main legal difficulty with implementing Flash is that he can't hire any programmers in the USA because there's a draconian clause in Adobe's licence that's enforceable in the USA and nowhere else. Something about "if you use this software, you agree not to use the gained knowledge for the purpose of writing a competing program". Not sure where you'd find the details... anyone know what I'm talking about?

how implementable Flash is

Posted Jan 25, 2010 21:50 UTC (Mon) by MarkVandenBorre (subscriber, #26071) [Link]

I've talked to Rob in the past and heard him say exactly the same. I seem to vaguely remember him writing that this situation has changed somewhat for the better though. But don't quote me on that.

how implementable Flash is

Posted Jan 26, 2010 2:20 UTC (Tue) by butlerm (subscriber, #13312) [Link] (1 responses)

Something about "if you use this software, you agree not to use the gained knowledge for the purpose of writing a competing program"
It it is somewhat questionable whether a term like that in a retail end user license agreement would in fact be enforceable, especially if the user of the software was not the person who installed it. Courts in the United States have decided both ways. If the EFF wanted to do something useful, they would find people with standing to challenge the extremely dubious arguments behind retail shrinkwrap licenses and push the counterargument based on traditional legal reasoning and copyright law until the disingenuousness of the idea that consumers do not own "copies" of software they buy of the shelf (and the consequent nullity of most end user SLAs) is obvious to every judge in the country.

how implementable Flash is

Posted Jan 26, 2010 13:31 UTC (Tue) by dannyobrien (subscriber, #25583) [Link]

If the EFF wanted to do something useful, they would find people with standing to challenge the extremely dubious arguments behind retail shrinkwrap licenses and push the counterargument based on traditional legal reasoning

It's funny you should say that... actually EFF is heavily involved in this fight (it was counsel on two of the three cases listed on the page you linked to), and I do believe we're expecting to work on a number of key cases in this area in the next year. We generally describe this as the fight over first sale (partly because the fight also includes patent issues as well as shrinkwrap licenses), but it's the same counterargument.

Blizzard: HTML5 video and H.264 - what history tells us and wihy we're standing with the web

Posted Feb 2, 2010 9:37 UTC (Tue) by njwhite (guest, #51848) [Link] (1 responses)

While Flash is more annoying because of bugs etc, it is in principle the lesser evil, since (I believe) it can be legally re-implemented in Free Software. H.264 cannot, in many parts of the world.

I think it's still the case though that codecs you send over flash are still patentable (and largely patented). Which is why the gnash devs get so annoyed that people get annoyed when it doesn't work with youtube, because they won't distribute the codecs themselves, so have to hope that the distributions will (and often distributors get it wrong).

Even if flash is perfectly reimplemented, then, we still have all the problems with codec patent issues.

gnash codecs

Posted Feb 2, 2010 17:21 UTC (Tue) by DonDiego (guest, #24141) [Link]

gnash could just distribute the codecs themselves. We've been doing it at FFmpeg/MPlayer for 10 years without problems. If they are afraid to do it themselves, they should get a hoster that is not.

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Feb 2, 2010 15:19 UTC (Tue) by DonDiego (guest, #24141) [Link] (2 responses)

> Flash is in principle the lesser evil, since (I believe) it can be legally
> re-implemented in Free Software. H.264 cannot, in many parts of the world.

This is wrong, please get a clue about patents. It is not illegal to implement a patented technique, in free or proprietary software, anywhere in the world.

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Feb 2, 2010 16:41 UTC (Tue) by Trelane (guest, #56877) [Link] (1 responses)

This is wrong, please get a clue about patents. It is not illegal to implement a patented technique, in free or proprietary software, anywhere in the world.

http://www.uspto.gov/web/offices/pac/doc/general/index.html#patent

The right conferred by the patent grant is, in the language of the statute and of the grant itself, the right to exclude others from making, using, offering for sale, or selling the invention in the United States or importing the invention into the United States. What is granted is not the right to make, use, offer for sale, sell or import, but the right to exclude others from making, using, offering for sale, selling or importing the invention. Once a patent is issued, the patentee must enforce the patent without aid of the USPTO.

(emphasis mine)

patents and legality

Posted Feb 3, 2010 1:47 UTC (Wed) by DonDiego (guest, #24141) [Link]

Correct. The patent holder receives a *right*, not an obligation. That very much does not make the act of implementing a patented technique illegal. The patent holder may choose never to enforce rights or you may reach some sort of deal.

This is very much unlike stealing or murder, where the state has an obligation to go after you and where you can never be excused or make a deal.

Blizzard: HTML5 video and H.264 - what history tells us and why we're standing with the web

Posted Jan 26, 2010 2:05 UTC (Tue) by roc (subscriber, #30627) [Link]

> Why is that relevant? Because if implemented in hardware, the software
> doesn't need a patent license.

Even if it was true that all hardware supported H.264 so there were no free software issues on the client, there would still be huge problems for content providers and Web authors, as Blizzard's post explains.

> Is it more important to eliminate flash or H.264? Mozilla has seemingly
> taken the latter one as more important.

No, it's a strategic issue.

Flash is deeply entrenched on the Web. Reducing its usage is a long-term project, which first requires us to create standards-based alternatives for Web authors to use instead of Flash. We're doing tons of work on that, but we're still far away from being able to disable Flash and still have a browser a lot of people would use.

On the other hand, <video>+H.264 is still very new on the Web, and there is Flash fallback, so pushing back against it is still possible without wiping out our market share and destroying all our influence over the Web.