Mozilla Popcorn: Enabling interactive video elements

Prior to HTML5, web video lived exclusively inside browser plug-ins (like Flash), which put it outside the reach of JavaScript, CSS, and other techniques for interacting with the rest of the surrounding document. Mozilla's Popcorn project is an effort to reverse this, by building an event-driven API to hook <video> content in to other page elements and local browser settings. Popcorn makes videos fully-interactive HTML elements, so that the timeline can both trigger and listen for events. Mozilla declared that the core Popcorn.js library had reached version 1.0 on November 4, and rolled out a set of auxiliary libraries to simplify authorship and to tie videos in to popular web services.

The cornerstone of the Popcorn project is HTML5's HTMLMediaElement (which, despite the heavy emphasis on video, is applicable to audio content as well). In particular, Popcorn relies on the currentTime attribute, which reports the position in the element's timeline. By wrapping a <video> or <audio> element in a Popcorn instance, the page developer can add, remove, and alter the other elements of the Document Object Model (DOM) based on the playback position of the media.

The simplest example is a subtitle track, which is a series of text fragments to be displayed and removed at particular moments in the timeline. But Popcorn builds a more flexible framework on top of HTMLMediaElement, so that developers can trigger custom functions, respond to events, and manipulate the playback of the media itself (such as jumping to particular time codes, pausing/unpausing playback, or changing the playback rate or volume). The code is modular, with base functionality provided in Popcorn.js, and separate "plugins" (arguably a poor choice of terminology, since Mozilla uses the nearly identical term "plug-in" to refer to media-enabling browser add-ons as well) for common use-cases and to link in to particular third-party web services. There are plugins for accessing Twitter, Flickr, and OpenStreetMap, to name just a few.

The core

Popcorn.js defines a Popcorn() JavaScript object, whose only required parameter is the #element_id of an HTMLMediaElement on the page. Primitive playback is controllable with Popcorn.play(), Popcorn.pause(), and Popcorn.currentTime(seconds), with which you can jump to any point in the timeline. The simplest action-triggering functionality is Popcorn.exec(time,callback_function), which merely executes the provided callback_function at the specified time. The exec function is also aliased as Popcorn.cue(), in a nod towards the lingo used by videographers.

Apart from simple time-based triggers, you can use Popcorn.listen(event,callback_function) to bind the callback function to a specified event. Built-in events are provided to handle typical web playback scenarios, such as "play," "pause," "loadstart," "stalled," "volumechange," and so on, or you can define custom events. You can directly trigger an event with Popcorn.trigger(event[,data]), where the data parameter allows you to send a data object to the listener.

The core API also defines pass-through methods for querying and setting basic properties of the media player (the state of the playback controls, the percentage of the media that is buffered, etc.), plus parsers for ancillary formats(including JSON, XML, and external video subtitle tracks), and static methods for utility operations like fetching remote scripts or getting locale and language information from the browser. Popcorn is designed primarily to work with native HTML5 media elements, but the library supports add-in "player" modules to implement compatibility for participating third-party video services, too. As of now there are just two, YouTube and Vimeo, plus a "null player"-like object called Baseplayer, which allows you to bind listeners and trigger events with a timeline that has no media attached.

The rest of Popcorn's functionality is implemented with plugins. This allows you to use the Popcorn build tool to create the smallest possible Popcorn.js to serve up to site visitors. The lowest-level plugin is "code", which allows you to specify a start and end time (in seconds or frames) and run JavaScript code in response. While the core API deals directly with the HTMLMediaElement content, most of the other plugins are designed to act on specific HTML elements elsewhere in the document — such as displaying plain text or images inside of another "target" element, which in the examples is usually a <div>. So-called "special effects" are provided through the applyClass method, which applies a CSS class (or a space-separated list of classes) to a plugin — and thus, by extension, to an element on the page.

The plugins

Although a handful of the existing 1.0 plugins implement utility functions, most are designed to simplify typical interactive tasks needed by video publishers. For example, the subtitle plugin allows you to define subtitles as a straightforward

    myvideo.subtitle({ 
        start: 12,
        end: 24,
        text: "Share your pain with me... and gain strength from the sharing.",
    })

Interestingly enough, some of these plugins are extremely similar — for instance, the footnote plugin also permits you to render text onto an HTML element, with a specified start and end time; the only difference appears to be that subtitle does not require a target element, and will render the text over the video element if none is provided. Mozilla has been developing Popcorn for well over a year, so presumably the initial set of plugins reflects the input of at least part of the web video community. There are plugins designed to implement bottom-of-the-video "news crawls," to tag people (complete with an avatar and link to a profile page), to list citation and attribution information. For the latter case, imagine a political ad and the brief citations that pop on screen as the damaging quotes about the candidate's opponent are read.

There are also general-purpose plugins, to do things like open arbitrary URIs inside of <iframe>s or to call other JavaScript libraries like Mustache templates or Processing sketches. However, many of the plugins are built for the express purpose of integrating a particular web service into the page. The initial list includes Facebook, LinkedIn, Wikipedia, Lastfm, Flickr, Twitter, Google Maps, and the open source mapping library OpenLayers.

These plugins enable you to display and hide everything from "like" boxes and profiles to maps and Twitter searches. Whether or not any particular plugin seems useful is probably in the eye of the beholder and is likely to be dependent on the API of the service involved. There are examples elsewhere in the Popcorn site where automatically bringing up informational content at the appropriate time really does seem to enhance a video — showing a map of the location being discussed, or related images and real-time Twitter commentary on the subject. On the other hand, simply popping up a static web page does not sound particularly innovative.

Authorship

To the average web developer, Popcorn will probably feel similar enough to existing client-side JavaScript libraries like jQuery to make getting started easy (and indeed, Mozilla refers to the project on the overview page and blog as "the jQuery of open video"). But because Mozilla is keen on attracting non-developers to the project — particularly video professionals — it has also set out to build "user-friendly" add-ons that make building Popcorn-enabled pages less like programming and more like the editing room. The centerpiece of this effort is Popcorn Maker, an in-browser editor that support drag-and-drop rearranging of video clips and Popcorn primitives. The project describes it as a "creative tool" for "filmmakers, journalists, and creative people of all stripes." It is built on top of an editing abstraction toolkit with the cute-if-not-very-informative name Butter.js.

Popcorn Maker is still alpha quality, but it allows you to place subtitles, footnotes, and images into a project by dropping them onto a timeline, and stretching their start- and end-points out to the desired duration. At the moment, it does not seem to enable editing of the HTML source of the file, which limits where you can place elements, and if you read through the API reference, writing the JavaScript by hand is not much more complicated. One should expect that to change as more of the plugins are ported to Popcorn Maker. A blog post describing the history of the editor suggests that it is still undergoing high-level redesign. The post does emphasize that Popcorn Maker is page editor, not a non-linear video editor. That is, you do not use Popcorn Maker to cut videos as you would with PiTiVi or Kdenlive — the video has to be completed before uploading to the application. Popcorn Maker is just a tool for crafting the interactivity of the rest of the HTML that surrounds the video at the center.

That distinction may be too fine a detail to get across; I suspect that ultimately, if Popcorn takes off, non-developers will look for a tool that handles both video editing and designing the interactivity to accompany it in-page. After all, once you start thinking about tying interactive content into your final work, it will start affecting the way you edit. Of course, aside from the demo films linked to from the project's site, virtually no one has attempted something similar to Popcorn, but you can already see how designing a video with web interactivity in mind affects the final product. The flashiest example of Popcorn is without question One Millionth Tower, a documentary project that incorporates captioning, layered video, and even animation. The end result is far more than just a video that triggers the display of page events.

As a 1.0 release, Popcorn is complete enough for people to begin designing their own interactive content. The supported third-party services cover the basics, and the home page links to a few other still-in-development add-on projects. The project certainly offers a compelling picture of what DOM-interactive audio/video content can do. Now if we can just get the video codec disputes solved once and for all....