Chiariglione: A crisis, the causes and a solution

[Posted January 30, 2018 by corbet]

Worth a read: this blog posting from Leonardo Chiariglione, the founder and chair of MPEG, on how (in his view) the group is being destroyed by free codecs and patent trolls. "Good stories have an end, so the MPEG business model could not last forever. Over the years proprietary and 'royalty free' products have emerged but have not been able to dent the success of MPEG standards. More importantly IP holders – often companies not interested in exploiting MPEG standards, so called Non Practicing Entities (NPE) – have become more and more aggressive in extracting value from their IP." (Thanks to Paul Wise).

to post comments

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 17:16 UTC (Tue) by mageta (subscriber, #89696) [Link] (5 responses)

I wonder why anyone but MPEG LA would think MPEG LA dying is a bad thing. The only thing about that he writes is this (IMOH):

> My concerns are at a different level and have to do with the way industry at large will be able to access innovation. AOM will certainly give much needed stability to the video codec market but this will come at the cost of reduced if not entirely halted technical progress. There will simply be no incentive for companies to develop new video compression technologies, at very significant cost because of the sophistication of the field, knowing that their assets will be thankfully – and nothing more – accepted and used by AOM in their video codecs.

So his take is, once MPEG LA is dead, no new technology in the field will emerge because companies can't earn money with the codec directly - rather than say with the HW they sell. Well, this needs to be seen. I for now will not shed a tear if we finally get good royalty free video codecs - music seems to exist already - which seem to emerge.

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 18:07 UTC (Tue) by excors (subscriber, #95769) [Link]

I'd expect there is still a strong incentive for companies like Google and Netflix to develop new compression technologies: better compression ratios translate directly into savings on bandwidth, which translate directly into money, and at their scale it's a substantial amount of money. And Google can strongly encourage industry-wide adoption by applying pressure on both the provider side (via YouTube) and consumer side (via Android and Chrome), so they have the incentive and the ability to continue codec development.

That does mean all the world's video compression experts will have to go and get jobs at Google (or companies that Google chooses to work closely with), which doesn't sound great, but it's not the same as arguing there will be no technical progress at all.

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 18:46 UTC (Tue) by nix (subscriber, #2304) [Link]

It's really quite remarkable. He calls the demise of MPEG LA 'tragic' right after calling HEVC 'unusable', but everything else he says admits his blinkers.

He doesn't appear to understand that "Patent holders who allow use of their patents get hefty royalties" implies by the simple law of pricing reduced usage of his codec; he doesn't appear to realise that uses where payment is not possible might even *exist*, though they are obvious to anyone with eyes; he gives royalty-free codecs quotation marks, as if the term is so unusual his readers might not know what it meant; he considers it "obvious" that the patent-free codecs he was considering would be intentionally crippled but from his language appears to consider people who want to use such codecs the enemy ('the "proprietary" threat'); he repeatedly claims to have foreseen dangers coming and devised strategies to counter them but then describes the way they failed with the wide-eyed astonishment of a child, as if they were utterly unforeseeable (is it *hard* to predict that if there are entities who don't cooperate with you who claim on everything you do, that if you do more things that are intentionally worse, that those entities will claim on those as well, even if you'd like them not to?).

He knows what's gone wrong, but he doesn't seem to realise that the way he was thinking, of codecs as something that must be financialized and have rent extracted from them, was the root of the problem. If you financialize them, don't be surprised if ever more middlemen come along to extract ever more rent, and don't be surprised if some of them don't want to cooperate with you (and that they get greedy enough that they end up killing your work, like a parasitic vine killing the host: after all, *they* didn't put in any work other than writing some intentionally vague and meaningless crap and submitting it to the patent office).

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 20:37 UTC (Tue) by Anssi (subscriber, #52242) [Link]

MPEG LA is the patent pool administration company, MPEG is the ISO working group responsible for the standards. The author is talking about the latter, which he is the chairman of.

If MPEG dies, MPEG LA doesn't necessarily die with it - they seem to administrate quite a few non-MPEG patent pools, like Firewire and DisplayPort.

But your point is still valid even with s/MPEG LA/MPEG/ :)

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 21:29 UTC (Tue) by jensend (guest, #1385) [Link] (1 responses)

He's not just talking about MPEG LA dying, he's talking about MPEG dying. Despite the names, the two are surprisingly unrelated.

MPEG is a working group in the ISO, founded by Chiariglione and others in 1988. To a first approximation, their approach to IP has been "everybody contribute whatever tech might help; we'd rather let other people worry about the legal stuff, and we trust people will get together to offer a reasonable license." Involved organizations are supposed to disclose any IP claims, but as long as the patent holders say they'd provide "fair, reasonable, and non-discriminatory" licenses, the IP claims weren't supposed to affect decisions of what to include.

Honestly, it's remarkable that this kind of loose policy worked as well as it did as long as it did. Navigating five dozen different patent holders' ideas about what constitutes a fair and reasonable license fee is too prohibitive to allow a new format to be adopted. The patent holders are all together better off if they can offer a single simple license so the format will get adopted and they all get royalties. But without MPEG putting policies in place that required more legal clarity, only voluntary collusion held things together. Cartels and collusions are normally unstable; like the prisoner's dilemna, someone is going to defect.

MPEG LA is a patent pool company/cartel founded in 1997 by all the major IP holders for MPEG-2 video. (Plenty of MPEG standards both before and after have had licensing arrangements other than a MPEG LA patent pool; for instance, the MP3 patents were licensed through Thomson/Technicolor.) The MPEG LA managed to get enough of those with IP in h.264 etc to join that they were able to provide full licensing for those standards and split the royalties. But with h.265, two new cartels and a bunch of individual companies all chose to defect.

Here's a previous comment from Chiariglione:

Unfortunately some parties have decided to break the MPEG social contract. The HEVC standard, the latest MPEG standard targeting unrestricted video compression, was approved in January 2013. Close to 5 years later, there is no easy way to get a licence to practice that standard.
What I am going to say is not meant to have and should not be interpreted in a legal sense. Nevertheless I am terribly serious about it. Whatever the rights granted to patent holders by the laws, isn’t depriving billions of people, thousands of companies and hundreds of patent holders of the benefits of a standard like HEVC and, presumably, other future MPEG standards, a crime against humankind?

To ensure workable licensing arrangements, MPEG would have needed stronger IP policies requiring real contracts rather than just social contracts.

There are other reasons beyond the FOSS desire for royalty-free formats why this kind of arrangement spells trouble. For instance, the incentive for everyone to try to get some piece of their IP included in the standard - not just to get royalties but to be protected from others (mutual assured destruction) - is high, resulting in pressure towards 'patent soup' rather than clean solutions.

Chiariglione: A crisis, the causes and a solution

Posted Jan 31, 2018 16:42 UTC (Wed) by mageta (subscriber, #89696) [Link]

Thank you for clearing things up more. I was indeed not aware of all details here.

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 17:35 UTC (Tue) by mfuzzey (subscriber, #57966) [Link] (1 responses)

By his reasoning the Linux kernel should have stopped improving years ago since no one pays royalties for it.
Yet it's going strong and more and more people are being paid to work on it...

I think that as long as there are commercially viable *uses* for better video codecs investment will continue and people will be paid to improve them.

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 18:11 UTC (Tue) by rahvin (guest, #16953) [Link]

His assumption that the only economic benefit is tolling the IP with patents is deeply flawed as you note, it flies in the face of the entire open source community and even significant numbers of businesses outside opensource. Codecs will continue to be developed, it's just going to be done by companies that aren't interested in patenting because they want the widest possible deployment, like another poster said that covers companies like Google, and Facebook who have strong financial incentives to develop newer and better compression technology.

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 17:40 UTC (Tue) by antiphase (subscriber, #111993) [Link]

The total destruction of these scalpers would be a benefit to humanity. There are many organizations who have a net interest in the progress of video coding technology who see it as more than just a cash cow to be milked to death, and the idea that only commercial patent-driven research can provide innovation in that field is absolutely risible.

God blinds those He wants to lose.

Posted Jan 30, 2018 17:48 UTC (Tue) by moltonel (subscriber, #45207) [Link] (6 responses)

> There will simply be no incentive for companies to develop new video compression technologies, at very significant cost because of the sophistication of the field
> Companies will slash their video compression technology investments, thousands of jobs will go and millions of USD of funding to universities will be cut.

Really ? How does he think that AOM managed to develop its AV1 format, which is alledegdly better than MPEG's best ? How did this happen without an incentive ? Or is he implying that this was a once-off, and that the incentives that gave AV1 to the world will suddenly disappear ?

> A successful “access technology at no cost” model will spread to other fields.

The video compression field is actually late to the "access at no cost" game.

> So don’t expect that in the future you will see the progress in video compression technology that we have seen in the past 30 years.

This is mainly true because compression ratio can't improve forever and we'll eventually reach a plateau, but that'd apply to an MPEG-only world as well.

On the other hand, I'm looking forward to seeing quicker improvements to compression technology : thanks to lessening presure from patents, as more companies invest in compression technology because they want to use it, not because they want to license it.

God blinds those He wants to lose.

Posted Jan 30, 2018 18:13 UTC (Tue) by Wol (subscriber, #4433) [Link]

> On the other hand, I'm looking forward to seeing quicker improvements to compression technology : thanks to lessening presure from patents, as more companies invest in compression technology because they want to use it, not because they want to license it.

The next time a patent like this comes up to SCOTUS (or perhaps even the Federal Court), I strongly expect it to be gutted, eviscerated, and whacked with a pick-axe in the head.

So the whole model is going to fall apart anyway, because the patents will cease to exist. Hopefully also new patents will stop being born because the US Patent Office will get its fingers rapped and told "They're maths, stupid! Not patentable!"

Cheers,
Wol

God blinds those He wants to lose.

Posted Jan 30, 2018 21:35 UTC (Tue) by ballombe (subscriber, #9523) [Link]

What he do not realize is that most of the 'significant costs' are due to the patent mine field.

God blinds those He wants to lose.

Posted Jan 31, 2018 0:35 UTC (Wed) by intgr (subscriber, #39733) [Link] (1 responses)

> How does he think that AOM managed to develop its AV1 format, which is alledegdly better than MPEG's best ? How did this happen without an incentive ?

To set the record straight, pretty much the only incentive large enough to get so many companies together and form AOM was that MPEG LA was steeply increasing its HEVC licensing fees. See https://youtu.be/lzPaldsmJbk?t=17s

There were some open codecs showing promise before that, but with the fragmentation and without so many companies backing them, it's very well possible that they wouldn't have achieved significant success in the end.

God blinds those He wants to lose.

Posted Feb 3, 2018 0:26 UTC (Sat) by JanC_ (guest, #34940) [Link]

One of them would likely have had success eventually, but maybe not as fast or as much.

God blinds those He wants to lose.

Posted Feb 2, 2018 8:38 UTC (Fri) by ttelford (guest, #44176) [Link] (1 responses)

> On the other hand, I'm looking forward to seeing quicker improvements to compression technology

I’m not convinced the development pace will change significantly.

Signals can only be compressed so much.

Netflix and Youtube isn’t going to keep re-encoding content in the year’s new shiny format.

Cheap, efficient chips (or modules within a SOC) take just as long to design whether the codec uses patents or not.

Supporting more codecs (and by extension more legacy codecs) means more die space - and higher cost.

God blinds those He wants to lose.

Posted Feb 2, 2018 19:56 UTC (Fri) by gioele (subscriber, #61675) [Link]

> Cheap, efficient chips (or modules within a SOC) take just as long to design whether the codec uses patents or not.

Often sub-optimal (sw/hw) code needs to be used to workaround patents that encumber part of a codec.

Searching for this sub-optimal workarounds takes time (= money) and implementing them may require more chip area (= +money).

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 18:28 UTC (Tue) by tialaramex (subscriber, #21167) [Link] (5 responses)

Like "novelist", "codec developer" isn't necessarily something you get paid to do, let alone a primary source of income - it can be done by an interested lay person, and there's no particular reason that getting rich will make you any better at it, although it might free up more of your time if you manage not to spend it all in meetings with the people holding the purse strings.

This is quite different from, say, "brain surgeon", where there's just no practical way you could afford it as a hobby, with all the very necessary staffing and equipment needs or even "rocket scientist" (there are a rich people who own rocket companies, but all the rocket scientists so far as I can tell are _employees_ not owners of the company).

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 18:40 UTC (Tue) by smurf (subscriber, #17840) [Link]

… with the possible exception of Elon Musk.

codec development involves real research work

Posted Jan 30, 2018 22:47 UTC (Tue) by jensend (guest, #1385) [Link] (3 responses)

Just because something doesn't require expensive equipment doesn't mean a hobbyist is likely to make revolutionary contributions working by themselves in their spare time. You don't get a Fields Medal or a science Nobel by puttering around by yourself for an hour after your day job, even though any "interested lay person" without equipment can study differential topology or quantum field theory.

I'm pretty sure Mozilla and Google together - not to mention any other companies in AOM - have at least a couple dozen people who are primarily employed to work on AV1.

Some things I saw seemed to claim that royalties on patented formats provided research institutions on the order of a hundred million dollars a year. That's a lot of grad student stipends or postdoc salaries.

codec development involves real research work

Posted Jan 30, 2018 23:48 UTC (Tue) by smurf (subscriber, #17840) [Link]

Sure, the money rolled in when MP3 got licensed. What the patent also did, however, is to stifle innovaton – why would you improve on a proprietary codec if you couldn't use it, much less share the work? Also, a lot of video files, in those days, consisted of ~30% audio, uncompressed, because freely-available codecs weren't allowed to support MP3.

Only the LAME people seriously improved the state of the art (they implemented a new psychoacoustic model which improved the quality of the encoded data stream a LOT), and they basically disregarded the whole patent mess and, fortunately, managed to not be sued for it.

I have no idea how much work the Vorbis creators spent on circumventing the MP3 patents, but it's certainly not zero.

And so on.

codec development involves real research work

Posted Feb 1, 2018 1:31 UTC (Thu) by tialaramex (subscriber, #21167) [Link] (1 responses)

"You don't get a Fields Medal or a science Nobel by puttering around by yourself for an hour after your day job"

Mmm. So, the most famous example of someone who _didn't_ get the medal is Andrew Wiles. Now, in a sense Wiles looks like an example of the phenomenon you're talking about, working at prestigious universities such as Oxford and Princeton. So this was his day job, right?

Except, nobody was paying Wiles to do breakthrough work on Fermat's Theorem, it was a crank hobby, people with no serious mathematical background tinkered with it, you couldn't get serious research money to keep chipping away at this century's old puzzle, even if it seemed tantalisingly close. Wiles had to _pretend_ to be continuing his earlier work in related fields, lie to academics he worked alongside, parcel out small discoveries he'd actually made years ago, rewrite papers, and so on, looking like a _failure_, somebody who was destined to dead-end to cover up the fact that all he was really spending his time thinking about was Fermat and how to prove something called the modularity theorem which was in Wiles' area and if it's true (which it is) proves Fermat was right.

I'm very familiar with the "good cause" argument for why bad ideas are actually OK, there's a play called "Peter Pan" you've probably seen adapted in some form, and in my country (only) a famous children's hospital has been granted a perpetual royalty right in this play, transcending any ordinary "intellectual property" protection such as copyright licensing, on the argument that it's a good cause, so we should keep the money flowing forever. I do not need to think that treating sick children is a bad idea to recognise that this decision was inappropriate. Likewise then, patent licenses aren't free money, somebody is paying those license fees, if we need lots of graduate students let us instead be honest and use tax revenue to pay them, not try to somehow shackle the research work to codec patent licenses, as if these have anything in common.

codec development involves real research work

Posted Feb 1, 2018 17:23 UTC (Thu) by jensend (guest, #1385) [Link]

Wiles was working with experts in the field on elliptic curves and Iwasawa theory as a (paid) graduate research assistant by 1975. After he finished his PhD in 1980, he was at four different academic institutions (IAS, Princeton, IHÉS, ENS), working full time on elliptic curves with the world's best and being paid to do so, before he heard in 86 the epsilon conjecture had been proved and decided to focus on the modularity theorem (which would also prove FLT).

"Cranks" weren't working on the modularity theorem. This was not a crank thing to be doing. It was precisely the kind of mathematics Princeton (and later Oxford) were paying him to research, discuss with colleagues, and lecture on. The secrecy for seven years wasn't because this was a strange hobby. It's because 1) he didn't want to let hints slip to others and get beat to the punch, he wanted full credit and 2) he knew that the normal thing would be to focus on producing a steady stream of smaller results which would erode the seemingly insoluble problems over time rather than bang your head against something when there's not visible publishable progress. After he publicized his work, it took another two years of working with colleagues who helped him correct errors and finish the proof.

Yes, he wasn't going around telling people "I'm focused on Fermat's Last Theorem" for those seven years. But he was doing exactly what he was paid to do full time, which he'd been paid to do full time for twenty years by the time the corrected proof was finished, and he was able to do it because he'd been mentored by and worked with the best minds in the field.

Your Pan diatribe is a total irrelevancy.

Even though grants have to be part of the picture, that has its own tradeoffs. Saying "divorce the rewards of good research from the researchers and instead use tax money, compelling people via the threat of imprisonment to pay for lots of researchers, without regard to whether the researchers achieve results or whether those forced to pay actually use the results themselves" is not a flawless answer to the problem at hand.

And though I think patents and royalties aren't inherently evil and that their merits can exceed their ills in some fields, I agree that using them for the basic formats of information exchange creates far too many problems. (Even if MPEG had a saner IP policy and even if the licensing had been administered more rationally and fairly than MPEG LA &c have done it wouldn't have been enough.)

I've been following Xiph and their efforts closely for fifteen years. I'm personally very glad that with Opus and AV1 and with the massive HEVC defections it looks like open and RF formats will finally win out. I'm just saying we have to recognize there are real tradeoffs here rather than claim innovations will henceforth arise out of vacuum. Xiph folks would be the first to say their efforts have relied on others' academic work (e.g. Fisher, "A Pyramid Vector Quantizer," published in IEEE Inf Thy in 85, is vital to Opus), and Xiph's formats were a third place niche until they had plenty of corporate support.

Option 3

Posted Jan 30, 2018 18:51 UTC (Tue) by smurf (subscriber, #17840) [Link]

would be to admit that, while the "patent the hell out of the algorithms and license the crap" business model led to some short-term monetary gains, it reached a dead end as soon as the first NPE showed up and acquired a patent that blocked further progress.

So, Mr. MPEG, just admit that there is no more money to be gained (and progress to be achieved) that way, and deed the patents to the public before they expire on their own.

In any case, it's instructive to look at the audio world. MP3 and AAC are how old exactly? We now have Vorbis (better than MP3) and Opus (better than all the others), plus an MP3 encoder that semi-legally became the best of the bunch, precisely because it was open sourced and people could actually build upon the work of others instead of re-inventing wheels.

Enough said.

Chiariglione: A crisis, the causes and a solution

Posted Jan 30, 2018 22:38 UTC (Tue) by flussence (guest, #85566) [Link]

The sooner this protection racket dies and makes a few examples out of those complicit in it, the better.