User: Password:
|
|
Subscribe / Log in / New account

This is a badly researched article.

This is a badly researched article.

Posted May 26, 2010 1:23 UTC (Wed) by DonDiego (guest, #24141)
Parent article: Swift and predictable reactions to WebM

This article does not meet the quality standards I have come to expect from lwn.net.

False claims are made about Jason Garrett-Glaser's article on VP8, but not even a link to that article is given in order for the readers to draw conclusions on their own. Here is the missing link:

http://x264dev.multimedia.cx/?p=377

Contrary to what this article claims Jason *did* post comparison images, here are two out of many:

http://doom10.org/compare/vp8.png
http://doom10.org/compare/x264.png

The apparent confusion about MPEG vs. MPEG LA has already been mentioned.

I could go on, but I just find it sad to see that most recent codec-related articles on lwn.net are often tinted by wishful thinking instead of presenting hard facts.


(Log in to post comments)

This is a badly researched article.

Posted May 26, 2010 1:41 UTC (Wed) by jake (editor, #205) [Link]

> Here is the missing link

While your other complaints may have some merit, this one is not correct. That link is quite prominent in the article, second paragraph in fact.

I am sorry that you didn't find it to your liking, but I don't at all think it was 'badly researched', whatever the mistakes made.

jake

This is a badly researched article.

Posted May 26, 2010 2:05 UTC (Wed) by sfeam (subscriber, #2841) [Link]

I have to agree with this criticism of the article. It presents misleading or incorrect summaries of the primary sources. While the article serves a useful function in providing links to them, it does a disservice in summarizing them badly before moving on to a "follow the money" argument rather than a technical overview.

It would have been more useful, for example, to approach the issue of patent avoidance from a different angle. Jason's analysis points out several things notably missing from VP8, first among them B-frames, that one would expect could be the starting point for further improvements. But if they were deliberately omitted to avoid patent claims, then this avenue of improvement may be closed off. This issue was raised in several of the primary sources linked to in the article, but could have been brought out more clearly in the article itself.

B-frames.

Posted May 26, 2010 6:19 UTC (Wed) by eru (subscriber, #2753) [Link]

first among them B-frames,

Didn't B-frames already appear in MPEG-1? Any patent on the idea has expired or will expire soon (drafts of the MPEG-1 spec were published already in 1990 claims this).

B-frames.

Posted May 26, 2010 11:16 UTC (Wed) by cortana (subscriber, #24596) [Link]

B-frames may have been present in MPEG-1, but that does not mean the patents have expired, in every country, and that there are no other patents that continue the earlier patents, or that are worded differently enough to be considered separate patents, and hence filed later, but similarly enough to still cover the same subject matter.

This is not a badly researched article.

Posted May 26, 2010 10:08 UTC (Wed) by bawjaws (guest, #56952) [Link]

I'd read all the articles before I read this and didn't spot any that were poorly summarized, even though I personally would have added some different context to some of them.

This article has more info on what design decisions in VP8 are likely to have been impacted by patents:

http://carlodaffara.conecta.it/?p=420

I note that the original Dark Shikari article has been updated multiple times by the author (search for "update:"). The initial burst of updates where to add more damning evidence of On2/Google incompetence in not doing things the obviously better MPEG-approved way, but the later ones acknowledge that the decisions made more sense in light of patent issues he was unaware of, or of features of the VP8 encoder that he didn't fully understand.

Static v.s. moving images

Posted May 26, 2010 10:10 UTC (Wed) by AndyBurns (guest, #27521) [Link]

> Contrary to what this article claims Jason *did* post comparison images
> here are two out of many:
>
> http://doom10.org/compare/vp8.png
> http://doom10.org/compare/x264.png

Given several seconds to flick backwards and forwards between the images I can see differences, but not necessarily say which is better; with 24 or 30 images per second I suspect it would be even harder to decide which is best.

Static v.s. moving images

Posted May 26, 2010 11:41 UTC (Wed) by paulj (subscriber, #341) [Link]

You'd likely perceive one as "sharper" than the other, even without consciously noting the extra details. Our brains are quite good at such fast, parallel processing.

Static vs moving images

Posted May 29, 2010 17:21 UTC (Sat) by giraffedata (subscriber, #1954) [Link]

You'd likely perceive one as "sharper" than the other, even without consciously noting the extra details. Our brains are quite good at such fast, parallel processing.

Actually, fast is one thing the brain is not. That's the reason that 24 frames per second is ususally indistinguishable from continuous motion.

Parallel, yes.

In this case, the most important feature of the brain is it's ability to track a moving object. It sees the object, not a series of scenes with the object in different places. So watching a ball move across the screen for 2 seconds is as good as staring at a single frame for 2 seconds for noticing how sharp the ball is.

Static v.s. moving images

Posted May 26, 2010 15:25 UTC (Wed) by DonDiego (guest, #24141) [Link]

x264 clearly preserves more detail, VP8 is blurrier. Thus x264 is better.

You are free to choose VP8 (or any other codec) nonetheless, but the quality difference is not something that can be honestly disputed.

Static v.s. moving images

Posted May 26, 2010 17:14 UTC (Wed) by aleXXX (subscriber, #2742) [Link]

Are these images from the same bitrate and quality settings ?
The 264 one is indeed much sharper, it looks like twice the resolution of the vp8 one.

Alex

Static v.s. moving images

Posted May 28, 2010 20:23 UTC (Fri) by dododge (subscriber, #2870) [Link]

From his blog post:
all encoders are optimized for optimal visual quality wherever possible. [...] the bitrate is (as close as possible to) the same on all of these files.
He also provides the motion clips the frames came from, as well as comparisons with H.264 Baseline, Theora, Dirac, and others.

The test was unfair

Posted May 29, 2010 7:47 UTC (Sat) by bawjaws (guest, #56952) [Link]

The test *is* biased, just not in brutally obvious ways. Not that I think VP8 would have won in a fair fight with x264, but in many ways that just makes it less acceptable.

The test clip benefits greatly from an x264 feature not generally found in other encoders, even other H.264 ones. He actually calls this out earlier in the article as the greatest single improvement in x264 and links to before and after images:

before: http://doom10.org/compare/parkrun_psnr.png
after; http://doom10.org/compare/parkrun_ssim.png

Notice anything familiar? Yes, it's basically the same clip that he's been tuning for, and the problems you see in the competing encoders are very similar to the ones you see in x264 before this was added (blurring in high frequency areas).

On top of that he's done the old classic of choosing the testing bitrate so that the favoured codec looks 'bad' but the competitors look 'terrible'. If you compare with the original (conveniently not provided) you'll notice that even the best x264 encode totally wrecks the human figures, making them look like the pixelated sprites from the original Mortal Kombat. If the bitrate had been increased to make x264 look actually good then the competitors would have increased quality proportionally more, if it had been dropped lower than both would have looked different shades of terrible.

You can see this effect better in the comparisons at http://www.quavlive.com/video_codec_comparison where the bitrates where chosen to match standard Youtube sizes and bitrates. One clip, with the bee, is 'easy' so both encoders look effectively identical, while the parkjoy clip from the x264 test and another riverbed one, are difficult so both look kind of rubbish.

Maybe extreme clips and bitrates are necessary for codec quality testing when encoders start to converge in quality, and maybe comparing your known strong points against competitors' known weak points is standard practice in marketing material, but both seem out of place in an article billed as an in-depth technical analysis of a new codec. It features a single image of the codec output and it happens to be of a carefully engineered failure mode. Is it any wonder that the second most common summary of this analysis you see accompanying links to it (after "turns out it infringes H.264 patents") is "turns out it sucks".

This is a badly researched article.

Posted May 26, 2010 10:25 UTC (Wed) by KotH (subscriber, #4660) [Link]

Badly researched doesnt really cover it.

It's realy unnerving to see how much FUD is spread over LWN when it comes to video coding. Please, if you want to write about video coding, then talk to the guys who actually do work on it. Talk to the guys from FFmpeg, VLC, MPlayer, Xine, XviD, x264,...

Then, about the article. Yes, Jason might be biased. Yes, he is an x264 developer. But if you read his blog post you'll see that he tries to have a balanced, technical analysis of VP8 compared to h.264. He isn't attacking it, neither does he bash it. And he definitly does not claim that VP8 violates any patents (he writes that the claim that VP8 is patent free is dodgy at best). And unlike most of the people who do compare different video codecs, he actually knows how to do a fair comparison without bias towards one or the other, even accounting for suboptimal implentations.

I wonder what the "long line of multimedia projects and companies announcing support for VP8 and WebM" is. FFmpeg (for those who don't know, it's the one single project that matters most when it comes to video coding as _every_ OSS video software uses FFmpeg somewhere and a lot, if not most comercial software does too) did not announce anything, neither was it "in the know before the deal was made". Also, FFmpeg did not get any patches until after the public anouncement. Some of the simpler patches were quickly commited, the rest will at least take a few weeks until all of them are ready to be included.

I also think that google should be a lot more open on whos work they are building on. For example, the container of webm is matroska, just relabled as webm. But with no word are they thanked for the great work they have done to create this container. They are simple forgotten in all the hype creation machinery.

This is a badly researched article.

Posted May 26, 2010 12:23 UTC (Wed) by liljencrantz (guest, #28458) [Link]

For the "long line of multimedia projects and companies announcing support for VP8 and WebM" that you're wondering about, look no further than the WebM home page:

Adobe
AMD
ARM
Brightcove
Broadcom
Collabora
Digital Rapids
Encoding.com
Grab Networks
iLinc
INLET
Kaltura
Logitech
MIPS
Mozilla
Nvidia
Ooyala
Opera
Qualcomm
Skype
Sorenson
Telestream
Texas Instruments
Verisilicon
ViewCast
Wildform

This is a badly researched article.

Posted May 26, 2010 13:57 UTC (Wed) by foom (subscriber, #14868) [Link]

I also think that google should be a lot more open on whos work they are building on. For example, the container of webm is matroska, just relabled as webm. But with no word are they thanked for the great work they have done to create this container. They are simple forgotten in all the hype creation machinery.

Perhaps you'd like them to say something like this on their "about" page?

What is WebM?

WebM is an open, royalty-free, media file format designed for the web.

WebM defines the file container structure, video and audio formats. WebM files consist of video streams compressed with the VP8 video codec and audio streams compressed with the Vorbis audio codec. The WebM file structure is based on the Matroska container.

This is a badly researched article.

Posted May 26, 2010 17:36 UTC (Wed) by Sho (subscriber, #8956) [Link]

While I agree with you, it's worth mentioning that Google worked together with the Matroska folks on WebM, and that the Matroska website has posted a notice of full support, as have individual Matroska developers in their blogs.

This is a badly researched article.

Posted May 26, 2010 18:15 UTC (Wed) by ballombe (subscriber, #9523) [Link]

Yes, I would like to see that. All I can see now is

is WebM?

an open, royalty-free, media file format designed for the web.

fines the file container structure, video and audio formats. WebM files
video streams compressed with the VP8 video codec and audio streams 
ed with the Vorbis audio codec. The WebM file structure is based on the 
a container.
No mention of Matroska.

This is a badly researched article.

Posted May 26, 2010 18:37 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link]

Must be some problem with your browser or something. It clearly does mention Matroska and links to it even.

This is a badly researched article.

Posted May 27, 2010 8:54 UTC (Thu) by mpr22 (subscriber, #60784) [Link]

There is clearly a broken interaction between their stylesheet and your browser.

This is a badly researched article.

Posted May 27, 2010 3:39 UTC (Thu) by roc (subscriber, #30627) [Link]

> And he definitly does not claim that VP8 violates any patents (he writes
> that the claim that VP8 is patent free is dodgy at best)

Right, he doesn't make any concrete claims that could be refuted, he just speculates and suggests. That's called FUD.

This is a badly researched article.

Posted May 27, 2010 9:37 UTC (Thu) by nye (guest, #51576) [Link]

>Right, he doesn't make any concrete claims that could be refuted, he just speculates and suggests. That's called FUD.


Yes, because clearly anyone trying to point out any problem somebody might have, but without investing millions in a full solution and handing it over on a silver platter with dinner and a movie, is engaging in FUD.

Please, now you're just being offensively paranoid.

This is a badly researched article.

Posted May 27, 2010 10:12 UTC (Thu) by farnz (subscriber, #17727) [Link]

Cut the rhetoric, please. I didn't see anyone asking for a full solution on a silver platter, except you in this comment. And I'm afraid that I agree with roc; he doesn't point to patents which are infringed, and say "I think you infringe this patent". Instead, he says, "well, this is very similar to H.264 (which we know is patent encumbered), but slightly different. Ergo, the patents we know about might affect it, so you can't claim you don't need a patent licence to use it without being on dodgy legal ground".

It's no different from me saying "well, we know Microsoft have software patents on Windows and Office, and Linux with GNOME and OpenOffice.org is awfully similar to Windows and Office. Ergo, the patents we know about might affect it, so you can't claim you don't need a patent licence to use it without being on dodgy legal ground."

This is a badly researched article.

Posted May 29, 2010 10:20 UTC (Sat) by roc (subscriber, #30627) [Link]

He didn't point out any specific problems, that's why it's FUD.

If he said something like "VP8 infringes patent #1234567 on flux capacitor tuning", that would have identified a problem specific enough to be refuted, and would not have been FUD.

This is a badly researched article.

Posted Jun 1, 2010 11:27 UTC (Tue) by nye (guest, #51576) [Link]

Do you have any idea of the kind of work you are demanding? It's fairly obvious that you are strongly biased here, so there is probably no convincing you, but let's try to take this into another domain which may be more comfortable.

Let's imagine we're talking about, say, performance. One individual has undertaken to provide a review of another's work, though it is of no benefit to them. Looking over it, they point out several sections and say: 'I think this technique might not be too good. Perhaps you could research this bit and see if you might improve it. Right now it would cost me a lot of time to look into it for you myself, and I could well be wrong, but I hope this pointer is of some use'. Is that FUD? 'He pointed out an area which looks suspicious, but didn't fully analyse or benchmark it! That monster!'.

This is a badly researched article.

Posted May 31, 2010 22:26 UTC (Mon) by jschrod (subscriber, #1646) [Link]

Would you please care to disclose your own involvement in video codec development, or other multimedia projects, or explain that there is no involvement?

Thanks.

This is a badly researched article.

Posted May 26, 2010 10:31 UTC (Wed) by liljencrantz (guest, #28458) [Link]

I think you're misreading the part of the article about comparison images. I believe the article does not mean to imply that Garrett-Glaser did not post comparison images, I think it meant to imply that unlike the comparison images posted by Garrett-Glaser, those posted by StreamingMedia used comparable bit rates. Ironically, the article is still wrong, as all files are between 16 and 17 MB in size, the VP8 one being among the larger.

If you read what the actual comparisons compared, though, you'll find something interesting. Garret-Glaser compared VP8 to various codecs, including H264 with either of baseline and high profile, and concluded that VP8 is roughly comparable to the baseline profile, but noticeably worse than the high profile. StreamingMedia compared VP8 only to the H264 baseline profile and also found they where very much comparable, i.e. the exact same result with a rather different slant.

The question is what people _should_ be comparing VP8 with. It seems to me that the H264 baseline profile is what is actually used in the real world today. The video players on my system (mplayer and Totem under Jaunty) could not correctly play the high profile H264-clip from Garret-Glaser's site. My n900 phone claims to support h264 but only supports the baseline profile. My previous phone, the iPhone, also only supports the baseline profile. As near as I can tell from my own experiences, if we're considering VP8 as the challenger that needs to displace the currently entrenched H264 with it's vast base of installed players, we must compare VP8 with H264 baseline, because that seems to be the only thing that can be reliably played on systems advertising H264 support. On the other hand, if we're talking pure tech and wondering about which format has the coolest technical toys, there is no reason to restrict our comparison to the baseline profile any more.

This is a badly researched article.

Posted May 26, 2010 10:36 UTC (Wed) by liljencrantz (guest, #28458) [Link]

Just checked, content encoded with the main H264 profile works fine on my Ubuntu system, it is only high profile that doesn't work. Also, Android devices don't support anything but H264 baseline.

This is a badly researched article.

Posted May 26, 2010 21:04 UTC (Wed) by Tester (guest, #40675) [Link]

Almost all mobile DSP chips that claim to support H.264 only support the baseline profile.

This is one of the untold reasons why Flash video isn't supported on most mobile devices (because it uses the Main profile)

This is a badly researched article.

Posted May 31, 2010 22:28 UTC (Mon) by jschrod (subscriber, #1646) [Link]

Would you please disclose your own involvement in video codec development, or multimedia projects, or declare explicitly that you are not involved and just a user?

Thanks in advance.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds