Development
Mozilla Popcorn: Enabling interactive video elements
Prior to HTML5, web video lived exclusively inside browser plug-ins (like Flash), which put it outside the reach of JavaScript, CSS, and other techniques for interacting with the rest of the surrounding document. Mozilla's Popcorn project is an effort to reverse this, by building an event-driven API to hook <video> content in to other page elements and local browser settings. Popcorn makes videos fully-interactive HTML elements, so that the timeline can both trigger and listen for events. Mozilla declared that the core Popcorn.js library had reached version 1.0 on November 4, and rolled out a set of auxiliary libraries to simplify authorship and to tie videos in to popular web services.
The cornerstone of the Popcorn project is HTML5's HTMLMediaElement (which, despite the heavy emphasis on video, is applicable to audio content as well). In particular, Popcorn relies on the currentTime attribute, which reports the position in the element's timeline. By wrapping a <video> or <audio> element in a Popcorn instance, the page developer can add, remove, and alter the other elements of the Document Object Model (DOM) based on the playback position of the media.
The simplest example is a subtitle track, which is a series of text fragments to be displayed and removed at particular moments in the timeline. But Popcorn builds a more flexible framework on top of HTMLMediaElement, so that developers can trigger custom functions, respond to events, and manipulate the playback of the media itself (such as jumping to particular time codes, pausing/unpausing playback, or changing the playback rate or volume). The code is modular, with base functionality provided in Popcorn.js, and separate "plugins" (arguably a poor choice of terminology, since Mozilla uses the nearly identical term "plug-in" to refer to media-enabling browser add-ons as well) for common use-cases and to link in to particular third-party web services. There are plugins for accessing Twitter, Flickr, and OpenStreetMap, to name just a few.
The core
Popcorn.js defines a Popcorn() JavaScript object, whose only required parameter is the #element_id of an HTMLMediaElement on the page. Primitive playback is controllable with Popcorn.play(), Popcorn.pause(), and Popcorn.currentTime(seconds), with which you can jump to any point in the timeline. The simplest action-triggering functionality is Popcorn.exec(time,callback_function), which merely executes the provided callback_function at the specified time. The exec function is also aliased as Popcorn.cue(), in a nod towards the lingo used by videographers.
Apart from simple time-based triggers, you can use Popcorn.listen(event,callback_function) to bind the callback function to a specified event. Built-in events are provided to handle typical web playback scenarios, such as "play," "pause," "loadstart," "stalled," "volumechange," and so on, or you can define custom events. You can directly trigger an event with Popcorn.trigger(event[,data]), where the data parameter allows you to send a data object to the listener.
The core API also defines pass-through methods for querying and setting basic properties of the media player (the state of the playback controls, the percentage of the media that is buffered, etc.), plus parsers for ancillary formats(including JSON, XML, and external video subtitle tracks), and static methods for utility operations like fetching remote scripts or getting locale and language information from the browser. Popcorn is designed primarily to work with native HTML5 media elements, but the library supports add-in "player" modules to implement compatibility for participating third-party video services, too. As of now there are just two, YouTube and Vimeo, plus a "null player"-like object called Baseplayer, which allows you to bind listeners and trigger events with a timeline that has no media attached.
The rest of Popcorn's functionality is implemented with plugins. This allows you to use the Popcorn build tool to create the smallest possible Popcorn.js to serve up to site visitors. The lowest-level plugin is "code", which allows you to specify a start and end time (in seconds or frames) and run JavaScript code in response. While the core API deals directly with the HTMLMediaElement content, most of the other plugins are designed to act on specific HTML elements elsewhere in the document — such as displaying plain text or images inside of another "target" element, which in the examples is usually a <div>. So-called "special effects" are provided through the applyClass method, which applies a CSS class (or a space-separated list of classes) to a plugin — and thus, by extension, to an element on the page.
The plugins
Although a handful of the existing 1.0 plugins implement utility functions, most are designed to simplify typical interactive tasks needed by video publishers. For example, the subtitle plugin allows you to define subtitles as a straightforward
myvideo.subtitle({
start: 12,
end: 24,
text: "Share your pain with me... and gain strength from the sharing.",
})
rather than writing text to a page element in separate .exec() actions.
Interestingly enough, some of these plugins are extremely similar — for instance, the footnote plugin also permits you to render text onto an HTML element, with a specified start and end time; the only difference appears to be that subtitle does not require a target element, and will render the text over the video element if none is provided. Mozilla has been developing Popcorn for well over a year, so presumably the initial set of plugins reflects the input of at least part of the web video community. There are plugins designed to implement bottom-of-the-video "news crawls," to tag people (complete with an avatar and link to a profile page), to list citation and attribution information. For the latter case, imagine a political ad and the brief citations that pop on screen as the damaging quotes about the candidate's opponent are read.
There are also general-purpose plugins, to do things like open arbitrary URIs inside of <iframe>s or to call other JavaScript libraries like Mustache templates or Processing sketches. However, many of the plugins are built for the express purpose of integrating a particular web service into the page. The initial list includes Facebook, LinkedIn, Wikipedia, Lastfm, Flickr, Twitter, Google Maps, and the open source mapping library OpenLayers.
These plugins enable you to display and hide everything from "like" boxes and profiles to maps and Twitter searches. Whether or not any particular plugin seems useful is probably in the eye of the beholder and is likely to be dependent on the API of the service involved. There are examples elsewhere in the Popcorn site where automatically bringing up informational content at the appropriate time really does seem to enhance a video — showing a map of the location being discussed, or related images and real-time Twitter commentary on the subject. On the other hand, simply popping up a static web page does not sound particularly innovative.
Authorship
To the average web developer, Popcorn will probably feel similar enough to existing client-side JavaScript libraries like jQuery to make getting started easy (and indeed, Mozilla refers to the project on the overview page and blog as "the jQuery of open video
"). But because Mozilla is keen on attracting non-developers to the project — particularly video professionals — it has also set out to build "user-friendly" add-ons that make building Popcorn-enabled pages less like programming and more like the editing room. The centerpiece of this effort is Popcorn Maker, an in-browser editor that support drag-and-drop rearranging of video clips and Popcorn primitives. The project describes it as a "creative tool
" for "filmmakers, journalists, and creative people of all stripes
". It is built on top of an editing abstraction toolkit with the cute-if-not-very-informative name Butter.js.
Popcorn Maker is still alpha quality, but it allows you to place subtitles, footnotes, and images into a project by dropping them onto a timeline, and stretching their start- and end-points out to the desired duration. At the moment, it does not seem to enable editing of the HTML source of the file, which limits where you can place elements, and if you read through the API reference, writing the JavaScript by hand is not much more complicated. One should expect that to change as more of the plugins are ported to Popcorn Maker. A blog post describing the history of the editor suggests that it is still undergoing high-level redesign. The post does emphasize that Popcorn Maker is page editor, not a non-linear video editor. That is, you do not use Popcorn Maker to cut videos as you would with PiTiVi or Kdenlive — the video has to be completed before uploading to the application. Popcorn Maker is just a tool for crafting the interactivity of the rest of the HTML that surrounds the video at the center.
That distinction may be too fine a detail to get across; I suspect that ultimately, if Popcorn takes off, non-developers will look for a tool that handles both video editing and designing the interactivity to accompany it in-page. After all, once you start thinking about tying interactive content into your final work, it will start affecting the way you edit. Of course, aside from the demo films linked to from the project's site, virtually no one has attempted something similar to Popcorn, but you can already see how designing a video with web interactivity in mind affects the final product. The flashiest example of Popcorn is without question One Millionth Tower, a documentary project that incorporates captioning, layered video, and even animation. The end result is far more than just a video that triggers the display of page events.
As a 1.0 release, Popcorn is complete enough for people to begin designing their own interactive content. The supported third-party services cover the basics, and the home page links to a few other still-in-development add-on projects. The project certainly offers a compelling picture of what DOM-interactive audio/video content can do. Now if we can just get the video codec disputes solved once and for all....
Brief items
Quotes of the week
Frankly, and GNOME is by far not the only one suffering from this, a project like this SHOULD have "open source contributor" as one of their prime audiences (note that I say "contributor" not "developer").
These contributors, like you and me, are the folks that will write bugreports in diff -u form. Heck we're the folks that conquer your bugzilla and FILE the bug in the first place. We're the folks that give you the translations you want. We're the folks who very vocally tell you what is wrong (and sometimes we even tell you what is right) with your software. We're the folks who advocate your software to our tech-noob friends. We're the folks that look at your code sometimes, to find security issues and tell you nicely about it before we go public. We're the folks who form the pool out of which you're going to be fishing your next developer set. We are the 99%^Wone percent, I will grant you that. But we are the one percent you as project NEED to get better in the long term, we are the one percent that your project needs to survive.
The reason I am so convinced of the importance of this change is so simple it took me nearly a year to identify it. While the ethos of Apache may have been "Community over Code" it required those in the community to understand and internalize that ethos for it to be fully realized. Social problems became political problems because the ethos had to be enforced by the institution.
The new era, the "GitHub Era", requires no such internalization of ethos. People and their contributions are as transparent as we can imagine and the direct connection of these people to each other turn social problems back in to social problems rather than political problems. The barrier to getting a contribution somewhere meaningful has become entirely social, in other words it is now the responsibility of the community, whether that community is 2 or 2000 people.
Cappucino 0.9.5
Cappucino is a framework for the creation of web-based applications. New features in the 0.9.5 release include vanishing scrollbars, improved documentation, a new "popover" widget, tooltips, a predicate editor, and more.The first GNOME Boxes release
GNOME Boxes is, according to this blog post by one of its authors, "designed to be the easiest way to use or connect to applications running on another Windows, Mac, or Linux system. Whether the system is virtual and local, a home computer you need to access from the road, or a centrally hosted corporate login - we'll get you there." The first release is now available; see this post for more information and screenshots. (Thanks to Paul Wise).
The Journal - a proposed syslog replacement
Lennart Poettering and Kay Sievers discussed their concept of the "journal" at the 2011 Kernel Summit; now they have posted a detailed document describing how they think their syslog replacement should work. "Break-ins on high-profile web sites have become very common, including the recent widely reported kernel.org break-in. After a successful break-in the attacker usually attempts to hide his traces by editing the log files. Such manipulations are hard to detect with classic syslog: since the files are plain text files no cryptographic authentication is done, and changes are not tracked. Inspired by git, in the journal all entries are cryptographically hashed along with the hash of the previous entry in the file. This results in a chain of entries, where each entry authenticates all previous ones. If the top-most hash is regularly saved to a secure write-only location, the full chain is authenticated by it. Manipulations by the attacker can hence easily be detected." The plan is to get an initial implementation into the Fedora 17 release.
PyPy 1.7 released
Version 1.7 of the PyPy Python interpreter is out. As is usual, much of the work in this release is focused on performance. "The main topic of this release is widening the range of code which PyPy can greatly speed up. On average on our benchmark suite, PyPy 1.7 is around **30%** faster than PyPy 1.6 and up to **20 times** faster on some benchmarks." There is also improved (but incomplete) stackless support and improved (but still incomplete) NumPy support.
Newsletters and articles
Development newsletters from the last week
- Caml Weekly News (November 22)
- Perl Weekly (November 21)
- PostgreSQL Weekly News (November 20)
- Tahoe-LAFS Weekly News (November 22)
News from the GNOME Outreach Program for Women
GNOME has put out a press release that highlights the accomplishments of seven women in the Google Summer of Code program for GNOME along with eight participants in GNOME Outreach Program for Women internships. "The accomplishments of the women who participated in Google Summer of Code this year are impressive. For example, Nohemi Fernandez implemented a full-featured on-screen keyboard for GNOME Shell, which makes it possible to use GNOME 3.2 on tablets. Raluca Elena Podiuc added the ability to create an avatar in Empathy with a webcam. Srishti Sethi created four activities for children to discover Braille for the GCompris educational software. [...] There were also eight women who participated in the GNOME Outreach Program for Women internships during the same time period as Google Summer of Code. Five of them worked on documentation, creating new topic-based help for the core desktop, as well as for the Accerciser accessibility tool, Vinagre remote desktop viewer, Brasero CD/DVD burner, Cheese webcam application, and GNOME System Monitor."
Page editor: Jonathan Corbet
Next page:
Announcements>>