Did you actually read the article? [LWN.net]

Did you actually read the article?

Posted Mar 15, 2012 22:01 UTC (Thu) by rqosa (subscriber, #24136) [Link]

> the only value I've seen you articulate around it i the ability to induce patent infringement and then try to claim harmlessness

No, I wasn't ever suggesting that "the ability to induce patent infringement" is a good reason to adopt GStreamer. I already said here, here, and here what the real values of adopting GStreamer would be; to summarize:

It allows for hardware-accelerated codec implementations (something that may be required to achieve good performance and battery life on mobile devices);
It allows for XVideo-based hardware-accelerated video frame scaling (something necessary for good performance on slightly-older desktop/notebook GPUs, like the Intel 945GM);
Generalizing the above reason: GStreamer's code is likely to be more mature and more featureful than the current equivalent in Gecko (by which I mean the "glue" between the codec libraries and X / ALSA);
With regards to the patent situation, it is not any worse than what Mozilla is currently doing — in either case, Mozilla would not be distributing patent-infringing code;
And finally, the network-effects of a hugely popular application like Firefox adopting GStreamer would bring about great benefits (e.g. faster security bugfixes) for the user/developer community of GStreamer (and by extension, for the FLOSS ecosystem as a whole).

> you would have to recreate a HTML HTML/CSS rendering layer (which couldn't work with xvideo regardless because of RGV vs YUV color model), unless you want to copy the frames and blow all possibility of acceleration.

I don't know how it does it, but I do know that QtWebKit uses GStreamer to play HTML native video, and on my system (which is about 5 years old and has a 945GM GPU) it can play native video (WebM from Youtube, among others) in fullscreen (1400 x 1050) without dropping frames, whereas Chromium drops frames like crazy, and Firefox either drops frames or (worse) else it "stutters" (that is, the playback is continuously "pausing" and "unpausing" itself). Or to put it another way: before QtWebKit supported video playback, HTML native video playback had worse performance than even Adobe Flash in every browser that I tried, whereas QtWebKit now gives better video playback performance than Flash.

(I'm not completely sure that QtWebKit with GStreamer is using XVideo rather than OpenGL for scaling. However, I do seem to remember trying out both the OpenGL scaler and the XVideo scaler methods in DOSBox, and the OpenGL one was a lot slower on my hardware.)

Did you actually read the article?

Posted Mar 15, 2012 22:29 UTC (Thu) by gmaxwell (subscriber, #30048) [Link]

Ah. I see where the disconnect is. You're incorrect about the advantages.

It allows for hardware-accelerated codec implementations (something that may be required to achieve good performance and battery life on mobile devices);

GStreamer is not especially useful for this, when the video needs to remain in the software video pipeline regardless (for effect and overlays) the hardware acceleration support is purely a function of the decoder library. (E.g. OMAP3 accelerated Theora works just fine without gstreamer)

It allows for XVideo-based hardware-accelerated video frame scaling (something necessary for good performance on slightly-older desktop/notebook GPUs, like the Intel 945GM);

It does not— The HTML5 rendering model is not compatible with XVideo acceleration.

HTML5 video tag is not just a dumb player embedded in a browser— we had that over ten years ago and no one wants to use that. HTML5, like flash, allows (and mandates) the ability to have dynamic/interactive overlays and to basically use the whole library of CSS propertys and effects on the videos.

(And if it were compatible, XVideo support is trivial and could easily be used in the browser without importing gstreamer)

Generalizing the above reason: GStreamer's code is likely to be more mature and more featureful than the current equivalent in Gecko

This is easy to demonstrate as not true. Though I'll grant you that (5) probably is true on account of this. But now you're just demanding that Firefox subsidize gstreamer development without any benefit to Firefox. I'm doubtful this subsidy would even be that valuable for GStreamer, since the motivations and directions are so different.

I'm not sure what QtWebKit does— but if it's rendering via xvideo it's either not HTML5 compliant or its only fast in a limited subset of cases where you can't tell the difference between compliance and not.

Firefox support is fast and accelerated without special casing— but only if you're using GL rendering, which may be disabled on your system because your GL support is slow or broken.

The exact thinking you suggest with "network-effects of a hugely popular application like Firefox" is part of why dumb video player like usage doesn't get special cased. By only accelerating the full imaging model people are encouraged to fix their software and hardware so developers can freely use the full featureset, it also avoids having more code and testcases to cope with an additional 'fallback fastpath'.

With regards to the patent situation, it is not any worse than what Mozilla is currently doing — in either case, Mozilla would not be distributing patent-infringing code;

Sure it is. Patent infringement doesn't just end at the boundary of what you distribute. If you distribute a kit, even one missing some key but easily obtained component, so that people can assemble a device which violates some patent you have not escaped legal liability. The law does not generally smile on evasive behavior. It's also worse from the perspective of bringing about a world where non-patented formats are widely offered by being incompatible with them.

Did you actually read the article?

Posted Mar 15, 2012 23:56 UTC (Thu) by rqosa (subscriber, #24136) [Link]

> the hardware acceleration support is purely a function of the decoder library. (E.g. OMAP3 accelerated Theora works just fine without gstreamer)

So, you're saying that libtheora itself supports OMAP3-accelerated decoding, so that any frontend to libtheora gets that feature "for free"?

> HTML5 video tag is not just a dumb player embedded in a browser

The YouTube HTML5 player does things that the old "embedded dumb players" never could, such as: JavaScript play / pause / seek controls, pop-up menus and "drop-up" menus (like drop-down but upside-down) overlayed on the video, text annotations overlayed on the video, picture-in-picture seek-point previews (when dragging the seek bead) overlayed on the video, etc. And it seems to works just fine in QtWebKit with GStreamer.

> This is easy to demonstrate as not true.

Well, it depends on what features are important to you. For the features I care about, QtWebKit supports them better.

> or its only fast in a limited subset of cases where you can't tell the difference between compliance and not

Maybe that's what it's doing — but that's just fine for the user as long as the the common cases that need to be fast are fast. (It's sufficient for everything I currently use HTML5 video for to support the case where there's just scaled-up video playing back with no overlays or with a few fully opaque overlays, and QtWebKit handles this just fine.)

> but only if you're using GL rendering, which may be disabled on your system because your GL support is slow or broken.

Well, it works fine for compositing in KWin (except that there are a few effect plugins that can't be enabled), and for all Linux-native OpenGL games that I've tried (Torus Trooper, No Gravity, Tux Racer and Extreme Tux Racer), but it didn't work for WebGL apps in Firefox. I don't know why…

Anyway, nothing you might say can change the fact that, when running on hardware like mine, QtWebKit is the only browser engine for Linux that gives an acceptable user experience for playing HTML native video. And I can hardly believe that it's a coincidence that the one that gives the best user experience is also the one that uses GStreamer.

> Patent infringement doesn't just end at the boundary of what you distribute.

I won't believe that unless you cite case law supporting it. For one thing, it would imply that any software that uses the POSIX virtual filesystem API (open(2), stat(2), etc.) infringes Microsoft's VFAT patent, because the Linux kernel (or at least older versions of it, which are still out there) violates the VFAT patent. For another thing, that would mean that all software written for the OpenSSL API (even when in source form) was infringing the RSA patent before it expired. And in general (as I said here), if you take that line of reasoning to its logical conclusion, then any software that's written against any external API must infringe all patents that might potentially be infringed by any possible implementation of that existing API. For example,

> It's also worse from the perspective of bringing about a world where non-patented formats are widely offered by being incompatible with them.

Well in that case, it's bad that the Mozilla software stack is FLOSS, because people might fork it to add support for the patented codecs.

And also, if Mozilla did adopt GStreamer, then they could whitelist a few unencumbered codecs and exclude all the rest — as I said here, they probably should do that anyway for security reasons (that is, to shrink the attack surface).

Did you actually read the article?

Posted Mar 22, 2012 17:16 UTC (Thu) by wookey (subscriber, #5501) [Link]

> Patent infringement doesn't just end at the boundary of what you distribute.
I won't believe that unless you cite case law supporting it.

It's called 'contributory infringement': http://www.invention-protection.com/ip/publications/docs/Contributory_Patent_Infringement.html It's designed to stop the case where you copy a device but miss out some vital part that would make it infringing and then get that made overseas or bought from some other supplier. That quite a lot like the case of making your browser with everythig except the codec, which you get from elsewhere. Sucks doesn't it? The whole patent system stinks, but you can't say 'they' don't have the legal angles well lined-up. After all, that is 'their' area of expertise. The MPEG-LA and the whole anti-society effects like this stupid inability to standardise an HTML5 codec simply falls out of the incentives that patents and the legal system provide. If it makes you angry then you are a reasonable person. And yes, I know I haven't provided any case law. I'm not sure that there is any directly-applicable (i.e in the software arena). There may well be, but I'm afraid I have more important work to do than try to search it out right now. My point is there there is potentially-relevant law. gmaxwell isn't just making it up. The law does not work the way a reasonable engineer/person thinks it should in this area.

Did you actually read the article?

Posted Mar 15, 2012 22:41 UTC (Thu) by rqosa (subscriber, #24136) [Link]

Oh, and one more reason:

6. GStreamer is supposed to support multiple platforms, including Windows and MacOS X; so, adopting GStreamer could reduce the amount of platform-specific special case code in Gecko

Also, about the redundant network stack: I believe that the same situation is present in KDE; it has its own HTTP client library in KIO, but (IIUC) Phonon-based media players using the GStreamer Phonon backend will use GStreamer's HTTP client implementation instead of KIO's one when playing an Icecast stream, and similarly Konqueror and other QtWebKit-based browsers for KDE will use GStreamer's HTTP implementation when playing HTML5 video. However, one could say that that redundancy still exists even in the case that Firefox / XULRunner / Gecko isn't using GStreamer, because most desktop/notebook Linux installations now (excluding Android) will have both Firefox and GStreamer present.