LWN.net Weekly Edition for November 21, 2012

Gnash, Lightspark, and Shumway

Adobe's proprietary and often annoying Flash format is dying, to be replaced by a bagful of open technologies like HTML5, CSS3, SVG, JavaScript, and royalty-free media codecs. Or so we are told. Of course, we have been told this story often enough over the years that it is difficult to muster genuine excitement at the news. Nevertheless, the most recent combatant to enter the ring is Mozilla's Shumway, which constitutes a distinctly different life form than existing free software projects like Lightspark and Gnash. Rather than implement Flash support in a browser plugin, Shumway translates .swf content into standard HTML and JavaScript, to be handled by the browser's main rendering engine.

The sparking and gnashing of teeth

Gnash and Lightspark are both reverse-engineered implementations of a Flash runtime (and, naturally, come with an accompanying Netscape Plugin API (NP-API) browser plugin), but they cover different parts of the specification. Gnash, the older of the two projects, implements versions 1 and 2 of Flash's ActionScript language, and the corresponding first generation of the ActionScript Virtual Machine (AVM1). This provides solid coverage for .swf files up through Flash 7, and partial support of Flash 8 and Flash 9 (including a significant chunk of .flv video found in the wild). Lightspark implements ActionScript 3 and the AVM2 virtual machine, which equates to support for Flash 9 and newer. Lightspark does have the ability to fall back on Gnash for AVM1 content, though, which enables users to install both and enjoy reasonably broad coverage without having to know the version information of Flash content in advance.

As is typical of reverse engineering efforts, however, neither project can claim full compatibility with the proprietary product. In practice, development tends to focus on specific use-cases and popular sites. Gnash, for example, was founded with the goal of supporting Flash-based educational games, and previous releases have been pinned to fixing support for popular video sites like YouTube. Lightspark maintains a wiki page detailing the status of support for common Flash-driven web sites. But the sheer variety of Flash content makes it virtually impossible to implement the full specification and offer any meaningful guarantee that the plugins will render the content without significant errors.

But an even bigger problem remains one of time and funding. Gnash in particular has struggled to raise the funds necessary for lead developer Rob Savoye to devote significant time to the code. Gnash has been a Free Software Foundation (FSF) high-priority project for years, and Savoye was the 2010 recipient of the FSF's Award for the Advancement of Free Software, but fundraising drives have nevertheless garnered low returns — low enough that as recently as March 2012, Savoye reported that the hosting bills for the site were barely covered. The last major release was version 0.8.10 in February 2012, which included OpenVG-based vector rendering along with touchscreen support. A student named Joshua Beck proposed a 2012 Google Summer of Code (GSoC) project to add OpenGL ES 2.0 support under Savoye's mentorship, but it was not accepted. Traffic on the mailing lists has slowed to a trickle, though there are still commits from Savoye and a devoted cadre of others.

Lightspark has made more frequent releases in recent years, including two milestone releases in 2012. In June, Version 0.6.0.1 introduced support for Adobe AIR applications and the BBC web site's video player. Version 0.7.0 in October added support for LZMA-compressed Flash content and experimental support for runtime bytecode optimization.

Both projects regularly make incremental additions to their suites of supported Flash opcodes and ActionScript functions, but neither has much in the way of headline-grabbing features in new releases. This is a bigger problem for Gnash, which does not have Adobe's newer enhancements to Flash to worry about (and is probably a key reason Gnash has had a hard time attracting donations). Lightspark can still tackle a host of new features with each update of Adobe Flash.

Of course, both projects' real competition has come from the easy availability of a freely-downloadable official browser plugin for Linux, but Adobe announced in February 2012 that Flash 11.2 would be the last release available as an NP-API plugin for Linux. Subsequent Linux releases would only be made as the built-in Flash plugin in Google's Chrome. The move has seemingly not motivated Flash-using Linux fans to cough up support for Gnash and Lightspark — but perhaps the next major update to Flash will.

I did it Shumway

Mozilla developer Jet Villegas wrote a blog post introducing Shumway on November 12, but the code has been available for several months. Shumway is described as an "experimental web-native runtime implementation" of the .swf format. Shumway essentially pushes handling of the formerly-Flash content to the browser's rendering engine and JavaScript interpreter. This protects against misbehaving plugins that eat up too many resources or simply crash. Shumway is available as a Firefox extension [XPI], though it is only expected to work on the most recent Firefox beta builds.

The recent Firefox build is required because Shumway parses Flash content and translates it into HTML5 elements, such as <canvas> and <video> elements, WebGL contexts, and good-old-fashioned embedded images. Shumway translates ActionScript into JavaScript to handle interactivity. Both AVM1 and AVM2 are supported, as are ActionScript versions 1, 2, and 3. The extension supports the use of <object> and <embed> tags to incorporate Flash into the page. As for multimedia codecs, Shumway can automatically take advantage of whatever codecs are available on the system.

At the moment there is not a definitive compatibility list, so Shumway's support for any particular Flash file is a gamble. Villegas did say in a comment that the project is targeting Flash 10 and below, which he said accounts for "the vast majority of existing content."

The idea of translating Flash content into HTML5 is not original to Shumway, but its predecessors have been focused on Flash-based advertising. Google offers a web service called Swiffy that translates uploaded Flash files into JSON objects, targeted at advertisers wanting to deploy animated ads. Smokescreen was a JavaScript player designed to render Flash ads on iOS devices.

Slaying the Flash gorgon

Mozilla's goal with Shumway is to remove Flash from the equation altogether, replacing it with "open web" technologies. By demonstrating that HTML5 content is capable of reproducing anything that can be done in Flash, the thinking goes, the browser-maker can encourage more content creators to drop Flash from their workflows. One might think it fair to ask whether supporting Flash in any sense genuinely "promotes" the use of Flash alternatives. After all, in December 2010, Mozilla's Chris Blizzard told Joe Brockmeier that the organization was not interested in funding Flash support, open source or otherwise:

Blizzard's comment was in response to a question about supporting Gnash and Lightspark development. Sobhan Mohammadpour asked the same thing on the Shumway blog post, to which Villegas replied:

Such a distinction might seem like splitting hairs to some. In particular, Villegas suggests that Gnash and Lightspark are a greater security risk than an .xpi browser extension. The Gnash team might take offense at that, especially considering the work the project has done to enforce a solid testing policy. But it is certainly true that massaging Flash content into generic web content has the potential to bring .swf and .flv support to a broader range of platforms. Both Gnash and Lightspark are developed primarily for Linux, with only intermittent working builds for Windows. On the other hand, Gnash and Lightspark also offer stand-alone, offline Flash players, which can be a resource-friendly way to work with Flash games and applications.

History also teaches us that it would be unwise to embrace Shumway too tightly, writing off Gnash and Lightspark as also-rans, for the simple reason that Shumway is still an experimental Mozilla project. Sure, some Mozilla experiments (such as Firefox Sync) move on to be fully integrated features in future browsers — but far more are put out to pasture and forgotten, with nary an explanation. Firefox Home, Chromatabs, Mozilla Raindrop — the list goes on and on.

It is also not clear exactly what to make of Villegas's statement about Flash 10 being the newest supported version. If that is long-term limitation, then Shumway may be of finite usefulness. True, Flash may die out completely before there is ever a Flash 12, and Flash 11 may never account for a significant percentage of the web's .swf files. In that case, users everywhere will enjoy a blissful HTML5-driven future with plugin-crashes a forgotten woe, and free unicorns as far as the eye can see. But where have we heard that one before?

Comments (47 posted)

Relicensing VLC from GPL to LGPL

Select a software license can be tricky, considering all of the effects that such a choice has for the future: library compatibility, distribution, even membership in larger projects. But agreeing on a license at the beginning is child's play compared to trying to alter that decision later on. Case in point: the VLC media player has recently been relicensed under LGPLv2.1+, an undertaking that required project lead Jean-Baptiste Kempf to track down more than 230 individual developers for their personal authorization for the move.

VLC had been licensed under GPLv2+ since 2001; the development team decided to undertake the relicensing task for a number of reasons, including making VLC compatible with various gadget-vendor application stores (e.g., Apple's). Making the core engine available under LGPL terms would make it a more attractive target for independent developers seeking to write non-GPL applications, the argument goes, which benefits the project through added exposure, and may even attract additional contributor talent.

The license migration was approved by team vote in September 2011. The first big milestone was cleared the following November, a relicensing of libVLC and libVLCcore (which implement the external API and internal plugin layer, respectively), plus the auxiliary libraries libdvbpsi, libaacs, and libbluray. Kempf described the process involved on his blog. Because VLC contributors retain the authors' rights to their contributions, no matter how small, Kempf needed to locate and obtain permission from all of the roughly 150 developers who had written even a minor patch during the project's long history.

To do so, he harvested names and email addresses from the project's git repository and logs, and undertook a lengthy process of sifting through the records (both to weed out false matches, and to identify contributors who were credited in unofficial spots like in-line comments). With the list in hand, Kempf set out to contact each of the contributors to approve the licensing change. He was ultimately successful, and the change was committed. The commit notes that more than 99% of the developers consented to the change, and those agreeing account for 99.99% of the code, which he said is sufficient from a legal standpoint.

The modular community

But, as Kempf described in a follow-up post, the same method was less successful when he set out in 2012 to relicense the next major chunk of VLC code: the major playback modules. Together, they constitute a much larger codebase, with considerably more contributors (including some who are not necessarily committed VLC team members). After emailing the module authors, he said, he received responses from only 25% of them. Two rounds of follow-up emails edged the number up closer to 50%, but for the remainder he resorted to "finding and stalking" the holdouts through other means. Those means included IRC, GitHub, social networks, mutual friends, employers, and even whois data on domain names.

In the end, he managed to get approval from the overwhelming majority of the contributors, but there were some "no"s as well, plus a handful of individuals who never replied at all. At that point, he had to examine the unresolved contributions themselves and decide whether to delete them, reimplement them, refactor them into separate files, or drop the offending modules altogether. He made the license-changing commit on November 6, and listed about 25 modules that were not included. They include the work of 13 developers who either declined give their approval or were unreachable, plus a set of modules that were ports from other projects (such as Xine or MPlayer) and thus not in the VLC team's purview.

By all accounts, the legwork required to hunt down and cajole more than 230 developers was arduous: in the second blog post, Kempf noted that it could get "really annoying" to contact people "over, over, over and over, and over" to ask for an answer. That is probably an understatement; in an email Kempf said at the outset that no one thought it would even be doable.

He also elaborated on what comes next. Not every VLC module was targeted for the relicensing work of the previous year, he said. Out of the roughly 400 modules being developed, about 100 remain non-LGPL. First, for those who rarely venture beyond VLC's desktop media player functionality, it can be easy to forget all of the other functions it provides; those modules will remain under their existing licenses. In particular, VLC's media server, converter, and proxy functionality will remain in GPL modules. Other modules, including scripting and visualization, will remain GPL-licensed at least for the time being, because they do not impede the ability of third-party developers to write non-GPL playback applications, which was the leading use-case motivating the change. VLC's user interface and control modules will also remain GPL-licensed, in order to discourage non-free forks.

Kempf also pointed out that the VideoLAN non-profit organization holds the trademarks to VLC, VideoLAN, and other names, and restricts their usage to open source code. That reflects the project's concern that the move away from the GPL will be misinterpreted by someone as a move away from free-ness (in multiple senses of the word); in addition to the trademark policy, both of the announcements about the relicensing project have emphasized that despite the change, VLC will remain free software.

Holdouts

But despite the consensus reached by the majority of core and module developers, there is still the problem of those twenty-odd playback modules that, for one reason or another, are not being relicensed. Kempf explained that the main VLC application will still be able to use all of the non-LGPL modules, and that only third-party libVLC applications will encounter any difficulties with license compatibility.

Authors of such applications may write their own modules for the missing functionality, or simply migrate to another module — given the modular nature of VLC, there are several modules out there that duplicate functionality implemented elsewhere. "The results might be slightly different, but I doubt many people will notice. There are a few exceptions, (probably 2 or 3) that will get rewritten, at some point, I think."

There are two modules Kempf predicted will never be reimplemented in LGPL code — DVD playback and Teletext support — because they rely on other GPL-licensed packages without viable non-GPL alternatives. He still holds out hope for tracking down a few of the still-unreached contributors, of course — only the authors of the iOS, Dolby, Headphone, and Mono modules outright declined to relicense their work.

It is not possible to predict exactly what effect the LGPL-relicensing work will have on third-party developers targeting iOS or other "app store" markets, thanks to the often opaque processes governing which content gets in and which gets rejected. But VLC was yanked from the iOS App Store in January 2011, a decision believed to be due to the GPL license. But because Apple does not provide details about its decisions, the situation remains nebulous.

Nevertheless, hunting down several hundred individual developers from more than a decade of development is an impressive feat of, shall we say, logistical engineering. Relicensing a community project is rarely a simple task; one is reminded of the multi-year process required to relicense the Heyu home automation engine, which involved tracking down the estates of developers no longer with us. Many large software projects have contemplated a license change at one time or another, and typically the scope of tracking down and persuading all of the former developers is cited as a reason that such a change is unworkable. For example, VLC's contributor pool is far smaller than the kernel's, to be sure. But the fact that Kempf was able to successfully chase down virtually the full set of both uncooperative and unintentionally-AWOL contributors in such a short time frame is an admirable achievement. Then again, the VLC team has long enjoyed a reputation for admirable achievements.

Comments (21 posted)

Android 4.2, tablets, and related thoughts

The kind folks at Google decided that your editor was in need of a present for the holidays; soon thereafter, a box containing a Nexus 7 tablet showed up on the doorstep. One might think that the resulting joy might be somewhat mitigated by the fact that your editor has been in possession of an N7 tablet since last July, and one might be right. But the truth of the matter is that the gift was well timed, and not just because it's nice to be able to install ill-advised software distributions on a tablet without depriving oneself of a useful device.

It was not that long ago that a leading-edge tablet device was a fairly big deal. Family members would ask where the tablet was; the house clearly wouldn't contain more than one of them. What followed, inevitably, was an argument over who got to use the household tablet. But tablets are quickly becoming both more powerful and less expensive — a pattern that a few of us have seen in this industry before. We are quickly heading toward a world where tablet devices litter the house like notepads, cheap pens, or the teenager's dirty socks. Tablets are not really special anymore.

They are, however, increasingly useful. Your editor recently purchased a stereo component that locates his music on the network (served by Samba), plays said music through the sound system with a fidelity far exceeding that available from portable music players, and relies on an application running on a handy Android (or iOS) device for its user interface. Every handset and tablet in the house, suddenly, is part of the music system; this has led to a rediscovery of your editor's music collection — a development not universally welcomed by your editor's offspring. Other household devices, thermostats for example, are following the same path. There is no need to attach big control surfaces to household gadgets; those surfaces already exist on kitchen counters and in the residents' pockets.

So the addition of a tablet into a household already containing a few of them is not an unwelcome event; it nicely replaces the one that will eventually be found underneath the couch.

What's new in Android 4.2

About the time this tablet showed up, the Android 4.2 release came out as an over-the-air update. Some of the features to be found there would seem to have been developed with the ubiquitous tablet very much in mind. At the top of the list, arguably, is the new multiuser support. A new "users" settings screen allows the addition of new users to the device; each user gets their own settings, apps, lock screen, etc. Switching between users is just a matter of selecting one from the lock screen.

Android users are still not as strongly isolated as on a classic Linux system. Apps are shared between them so that, for example, if one user accepts an app update that adds permissions, it takes effect for everybody. The initial user has a sort of mild superuser access; no other users can add or delete users, for example, and the "factory reset" option is only available to the initial account. There doesn't seem to be a way to parcel out privileges to other accounts. The feature works well enough for a common use case: a tablet that floats around the house and is used by multiple family members. Perhaps someday the face unlock feature will recognize the user of the tablet and automatically switch to the correct account.

A feature that is not yet present is the ability to clone one tablet onto another. As we head toward the day when new tablets will arrive as prizes in cereal boxes, we will lose our patience with the quaint process of configuring the new tablet to work like the others do. Google has made significant progress in this area; a lot of useful stuff just appears on a new tablet once the connection to the Google account has been made. But there is still work to do; the process of setting up the K9 mail client is particularly tiresome, for example. And, naturally, storing even more information on the Google mothership is not without its concerns. Wouldn't it be nice to just put the new tablet next to an existing one and say "be like that other one"? The transfer could be effected with no central data storage at all, and life would be much easier.

Much of the infrastructure for this kind of feature appears to already be in place. The near-field communications (NFC) mechanism can be used to "beam" photos, videos, and more between two devices just by touching them together. The "wireless display" feature can be used to transmit screen contents to a nearby television. It should not be hard to do a full backup/restore to another device. Someday. Meanwhile, the "beaming" feature is handy to move photos around without going through the tiresome process of sending them through email.

Another significant new feature is the "swipe" gesture typing facility, whereby one spells words by dragging a finger across the keyboard from one letter to the next. Gesture typing has been available via add-on apps for a while, but now it's a part of the Android core. Using it feels a little silly at the outset; it is like a return to finger painting in elementary-school art class. For added fun, it will attempt to guess which word is coming next, allowing the typing process to be skipped entirely — as long as the guesses turn out to be accurate. In your editor's experience, gesture typing is no faster than tap-typing; if anything, it is a little slower. But the resulting text does seem to be less error-prone; whoever wrote the code doing the gesture recognition did a good job.

One interesting change is that the notification bar at the top has been split into two. The downward-swipe gesture on the left side gives the usual list of notifications — though many of them have been enhanced with actions selectable directly from the notification. On the right side, instead, one gets various settings options. The new scheme takes a while to get used to; it also seems like it takes a more determined effort to get the selected screen to actually stay down rather than teasing the user and popping right back up.

Various other new features exist. The "photo sphere camera" is evidently an extension of the panorama feature found in earlier releases; alas, it refuses to work on the N7's (poor) front-facing camera, so your editor was unable to test it out. The camera also now evidently has high dynamic range (HDR) processing functionality. On the Nexus 10 tablet, the "Renderscript" mechanism can use the GPU for computational tasks; no other device has the requisite hardware support at the moment. There is a screen magnification feature that can be used to zoom any screen regardless of whether the running app was written with that in mind. And so on.

One other change in the 4.2 release is the replacement of the BlueZ-based Bluetooth stack with a totally new stack (called "Bluedroid") from Broadcom. This stack, according to the release notes, "provides improved compatibility and reliability." A message on the android-platform list gives some additional reasons for the change, including the ability to run Bluetooth processes in a separate process, elimination of the D-Bus dependency, and more. The licensing of the new "Bluedroid" stack has raised some questions of its own that have not been clarified as of this writing.

Bluetooth stack questions aside, the obvious conclusion is that the Android platform continues to advance quickly. Each release improves the experience, adds features, and generally cements Android's position as the Linux-based platform for mobile devices. Your editor would still like to see an alternative platform, preferably one that is closer to traditional Linux, but that seems increasingly unlikely as the spread of Android continues unabated and unchallenged. The good news is that Android continues to be (mostly) free software and it continues to improve. This stage of the evolution of the computing industry could easily have taken a highly proprietary turn; thanks to Android, the worst of that has been avoided.

(Thanks to Karim Yaghmour for pointers to the Bluedroid discussion).

Comments (34 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

• Security: A rootkit dissected; New vulnerabilities in Java, Mozilla products, MySQL, Xen, ...
• Kernel: Checkpoint/restore in user space; VFS hot-data tracking; Module signing.
• Distributions: Gentoo's udev fork; Mint, ...
• Development: Random numbers; GNOME 3; Color Management Hackfest; UEFI bugs; ...
• Announcements: FSFE and secure boot; Mozilla Foundation annual report; Bottomley on UEFI signing; Apple patents; open standards in Portugal; ...

Next page: Security>>