Leading items

The LGPL and video codecs

Video for the web is generally a contentious topic—the HTML 5 specification does not, yet, have a required codec—but browser developers have begun making their own codec decisions. Google has included support for the patent-encumbered MPEG-4 codecs in its Chrome browser, which has caused some consternation among those who are working on HTML 5. The underlying concern is for web video interoperability, especially for browsers who cannot or will not license codecs, but it has also manifested itself as a question about free software licensing and its relationship to patents.

The Web Hypertext Application Technology Working Group has been working on HTML 5, and a posting from Josh Cogliati to its mailing list was the jumping-off point for a wide-ranging discussion. Cogliati suggested using an MPEG-1 subset as the required video codec for the HTML 5 <video> tag, but, since MPEG-1 is rather old and is considered technically inferior, the idea was not met with much support. Ian Fette noted that Google Chrome has chosen to "support H.264 + AAC as well as Ogg (Theora + Vorbis) for <video>", so MPEG-1 support is not likely to be of interest for Chrome. But the mention of H.264, which is covered by multiple patents and licensed by the MPEG Licensing Authority (MPEG-LA), caused a rather negative reaction among many of the other mailing list participants—some of whom are notably affiliated with competing browsers.

Chrome added support for H.264 by way of FFmpeg, which is a free software implementation of various multimedia formats, and is covered by the LGPL version 2.1 (though some optional parts are GPLv2). This led some, including Anne van Kesteren of Opera Software, to question the relationship of the patents to the LGPL. As he, and others saw things, Google couldn't license those patents without passing them on to anyone else who distributed FFmpeg. This is because of LGPL Section 11, which reads, in part, as follows:

Google's position, as outlined by Chris DiBona and Daniel Berlin, is that it has a license from MPEG-LA which licenses the patents for use in Chrome. That license does not say anything about FFmpeg, so there is nothing that "would not permit royalty-free redistribution" of FFmpeg. Thus, from Google's perspective, its patent license does not interact with Section 11 at all.

Others are less sure about that interpretation. Opera CTO Håkon Wium Lie asks:

Berlin makes the argument that the example in Section 11 must be interpreted with the rest of that section, not in isolation:

But others have interpreted the LGPL differently. Miguel de Icaza said that Moonlight would have preferred to use FFmpeg, along with a license from MPEG-LA, rather than the proprietary codecs it ended up using. He wonders if anyone has checked with the Free Software Foundation (FSF) for their interpretation. Berlin points out that the FSF's opinion is not particularly relevant in this case as only the FFmpeg project has any standing to complain about LGPL violations of its code. So far, FFmpeg has not, publicly at least, complained.

In the end, as DiBona states, Google—along with its lawyers, presumably—is comfortable that it is complying with the terms of the LGPL. Whether that might cause others to re-examine their decisions about using FFmpeg remains to be seen. But the LGPL compliance question is really a bit of a distraction from the real underlying issue regarding patented codecs for web video. Both Mozilla and Opera employees are quite obviously unhappy with Google's decision to support H.264 for the <video> tag.

Mozilla's Robert Sayre points out that the licensing issue, if any, can easily be dealt with by Google, but: "The incredibly sucky outcome is that Chrome ships patent-encumbered 'open web' features, just like Apple. That is reprehensible." DiBona takes exception to that label:

DiBona's argument is that H.264 is demanded by users because of quality issues. Lie is not so sure, noting: "YouTube is very popular despite the fact that its video clips resemble the transmission from the moon landing in 1969." He continues:

That is really the root cause of the dispute. To many, the "open web" demands open codecs, while Google doesn't necessarily disagree with the intent, they are not so idealistic as to ignore a de facto video standard for Chrome. In addition, YouTube, which is owned by Google, has an enormous collection of H.264-encoded video that Google would undoubtedly like to support directly in Chrome.

Slowly but surely, though, the Theora codec is building momentum. Chrome, Opera, and Mozilla will all be supporting it for the <video> tag, the Thusnelda encoder is becoming competitive with H.264, and more sites are starting to carry Theora video. The mailing list thread contained multiple mentions of Dailymotion and Wikipedia as supporters—and, more importantly, adopters—of Theora.

While Mozilla, Opera, and probably others are not forcing Theora as the only choice for their users, they are making it the only default video codec. Plugins or extensions will most certainly support H.264 (along with other codecs), but the idea is that web site owners can be sure that their visitors browser will support Theora. Currently, web owners are often forced to rely on Adobe Flash plugins as their video delivery system; an in-browser solution will clearly be a step up.

There are those—Nokia and Apple among them—that are unconvinced about the unencumbered nature of Theora, and consider it an open question whether there will be patent suits against deep-pocketed implementers. Google would seem to fit that bill, so, with luck, the Chrome implementation will flush out any submarine patents that some patent troll believes Theora infringes. Given that Google was willing to pay for a license from MPEG-LA, one might guess they would be willing to do the same for Theora, but at least that would bring the submarines to the surface. Perhaps that is not the perfect outcome, but is certainly better than the uncertainty that exists today.

Comments (33 posted)

OpenMoko: its present and future

OpenMoko has had a tough time. Since announcing in 2006 that it was creating an open, Linux-powered mobile phone, it has been plagued with hardware problems and immature, buggy software (the software stack has been rewritten at least twice). The problems with the product has resulted in poor sales which caused smart, motivated staff to leave (or be laid off), thus creating a vicious circle.

LWN has chronicled OpenMoko's progress and problems over the years, culminating in another recent round of lay-offs and OpenMoko (OM) refocusing on a mysterious "Project B" about which little is known except that it is not a mobile phone.

Ironically, just at the time OM is (temporarily?) refocusing away from phones, its phone: the Neo FreeRunner (aka GTA02) is just becoming a stable, usable phone suitable for use by the typical LWN reader.

There are a number of distributions available for the FreeRunner. The majority of those that have ongoing development are based on a telephony stack called freesmartphone.org (FSO) developed by (now-)former OM employees. This stack provides a number of D-BUS interfaces for controlling various phone components including GPRS, wifi, and GPS, as well as the GSM phone functionality. Different distributions can then build on top of the FSO stack.

Stable Hybrid Release (SHR) was one of the earliest FSO distributions; it originally aimed to be a hybrid (hence the name) of the best bits from the various official OM distributions. It has evolved into a general purpose, community-driven distribution that has regular testing releases (and a continuously updating unstable release) that incorporate new software and features frequently.

OM2009 is OM's officially blessed distribution and, like SHR, it is based on FSO and OpenEmbedded. OM2009 is minimal in the features and software it provides; OM wanted to concentrate on creating a working mobile phone before trying to create a "smart" phone. This is reflected in the choice of the Paroli application, which is a GUI for controlling basic phone functionality such as a dialer and SMS. Not all the FSO distributions use OpenEmbedded, for example Hackable:1 (which has created a number of popular applications) is based on Debian and there are also Gentoo and Slackware distributions. There are also a number of non-FSO based software stacks including a port of Android.

All the FSO based distributions are still relatively immature; Neither SHR or OM2009 has yet to make a formal stable release though the "testing" images are widely used. Looking at mailing list traffic and IRC, most users seem to have switched to using them (I've been using SHR for about a month as my daily phone).

Using SHR as a day-to-day phone

SHR contains all of the features you'd expect in a basic phone: a dialer, messaging, and contacts applications. They all work well, but are rudimentary. Call quality is noticeably worse than that of my Nokia in noisy environments, but is still acceptable. The phone is configured to suspend in order to save battery life, but reliably wakes up on incoming calls and texts. With a fairly typical usage pattern, the phone battery lasts a couple of days before needing to be recharged; though recharging it every night is not a bad idea.

Wifi, GPRS (the FreeRunner is a 2G phone), and GPS all work and can be configured via the GUI. The default browser (Midori) is more suited to a traditional desktop - it lacks finger scrolling and zooming which makes using it quite cumbersome. Other browsers are available, in particular, a Hackable:1 community member has created a WebKit-based browser named Woosh which is designed for the FreeRunner. Woosh is still in its infancy but if it, or another similar project mature it will be a big boon for the OpenMoko platform.

A variety of other software is available, most is available in the repositories for the various distributions but opkg.org exists as a useful showcase. Some popular apps that I regularly use include Navit (GPS car navigation system that requires some fiddling with text files to install and configure, Intone (music player) and MokoMaze (simple but addictive accelerometer based ball-in-maze game). You can also use applications like Dictator to transform your phone into a basic dictation machine or to record phone calls.

There are a large number of keyboards/software input methods available (e.g. Hackable:1's Xkbd and qwo) but partly because the screen is inset from the case (and partly just because it is a touch-screen) entering lots of text can be a laborious operation. Because it's an open phone based on a recent Linux kernel, though, it has support for Bluetooth keyboards which make text entry much more efficient.

In short, it has all the basic requirements for a simple phone. In addition, there is a community creating and refining software for it and you have the ability to do things like ssh in and use vi (or to make use of a simple D-BUS interface to determine your location). There are plenty of rough edges, though, for example, the speaker phone button doesn't seem to work in the dialer, there is currently no GUI for altering the volume during a phone call, and connecting to a wireless network requires two applications: one to turn the wifi power on, the other as a GUI for wpa_supplicant.

There are OM distributors around the world who will happily sell you a FreeRunner, however it is worth noting that there have been a number of hardware revisions. Those prior to the most recent (A7) revision were susceptible to "buzz" on the line during phone calls with certain combinations of GSM frequencies and carriers. OM has supported distributors (e.g. Golden Delicious and SDG) in providing hardware buzz fixes to earlier models, but confirming which revision of the FreeRunner you are purchasing could prevent a lot of hassle after purchase.

The future is uncertain for OM and its relationship with mobile phones. OM has opened the hardware schematics for the FreeRunner and a community project is making a number of updates. Whether we will ever see new phones and major new software revisions coming from OM itself is an open question, but, given the open, community-oriented nature of both the hardware and the software, that is not necessarily a death knell for the phone. OM has plenty of stock of FreeRunners and can continue to produce more if the demand is there.

Despite the rough edges, the Neo FreeRunner is a usable phone. What sets it apart from other phones is its openness; unlike others, the user rather than the phone vendor or the carrier are in control. All the software that runs on the phone's main CPU is open source (the separate GSM module is closed) and hardware schematics are available. However, given the perilous state of OM's finances, if you want open phone hardware with a growing community, now might be a good time to buy one.

Comments (2 posted)

PiTiVi video editor previews rewritten core

Video editing stands to get significantly easier for Linux users this year. The GStreamer-based nonlinear video editor (NLE) PiTiVi has released its first public build since undergoing a substantial rewrite. Version 0.13.1 was released May 27, 2009, and adds a handful of user-visible changes but makes substantial improvements under the hood, laying down groundwork for the next development cycle.

Video editing continues to be an unsettled topic among Linux users. Among the handful of applications currently available, each has its share of limitations, user interface quirks, and stability problems. Kino, for example, is very capable at capturing DV video from cameras, but uses an unfamiliar UI for an NLE, without the familiar editing timeline, and requiring a rendering step for every title, transition, or effect. Kdenlive and Cinelerra offer broader feature sets, but are plagued by widely-reported installation and stability problems. For its part, the chief criticism of PiTiVi has always been missing functionality. The rewrite of the 0.13 series, begun in 2008, promises to change that.

PiTiVi (pronounced "P-T-V") uses Python for its user interface and top-level logic, but all audio and video processing — from playback to filters — takes place in GStreamer. That gives the application better performance, the ability to take advantage of GStreamer's multi-threading, and independence of codec and file formats. Any format supported by GStreamer is supported by PiTiVi, which is convenient for users, and distributions can include the application without introducing a package dependency on potentially legally-risky media engines such as FFmpeg. GStreamer's plugin-based abstraction allows PiTiVi to function regardless of whether any particular codec pack — patented, commercial, proprietary, or free — is installed, and does not require maintaining a specially-tailored playback engine to avoid legal conflicts.

Users will notice that PiTiVi 0.13.1 supports multiple layers in the timeline, permitting simple overlay of multiple video and audio sources, support for still images as video tracks, and the ability to trim audio and video clips in the editor. As the before (at left) and after (below, at right) screenshots depict, audio waveforms and video thumbnails are now used to represent tracks in the timeline, making it easier to distinguish between clips while editing. In addition to the visible changes, however, the more substantial work of the 0.13.x series is the refactoring of the core, which will eventually make the editor more powerful.

Refactoring

Lead developer Edward Hervey said that the refactored application core reflects several years of feedback from users and discussions with professional video editors. "The previous design was more the evolution of a certain view I had of video editing and its workflow back in 2003." The old design included a lot of hard-coded features that held the application back in real-world workflows, such as limits on the number of audio and video tracks, a limit on the number of layers, and only being able to have one project open at a time.

One example Hervey cites is the internal representation of the editing timeline. The old PiTiVi core was tied directly to the timeline provided by the Gnonlin library (a dependency of PiTiVi that implements NLE features as GStreamer plugins), which limited it to one object per stream and track. The new core adds an intermediary layer, permitting multiple objects and tracks.

The rewrite is also considerably more modular, making it easier for the developers to add additional features. Features planned for the 0.13.x cycle include critical editing tasks like titling, transitions, and effects — operations which are already possible in GStreamer plugins. Hervey explained:

Most of the user interface work has been taken on by new team addition Brandon Lewis, with assistance from Jean-Francois Tam. As is often the case, the team is trying to maintain a clean separation between the application core and its user interface, but for PiTiVi this choice has implications for its customizability. Hervey explained that the goal is to keep the UI as simple as possible in order to make the program usable by novices, but to leave room for building additional functionality on top of it through the use of plugins.

GStreamer

PiTiVi development is intrinsically tied to GStreamer development. As Hervey explained in an interview with Gnomedesktop.org, he, Lewis, and core developer Alessandro Decina are all employed by Collabora Multimedia, which sponsors ongoing GStreamer work alongside other open source projects. Some of Collabora's revenue comes from consulting jobs that involve building custom GStreamer processing and editing solutions, so building a dependable NLE is an important task, which influences the direction in which GStreamer itself moves.

One such influence was the notion of "segments" that was added in GStreamer 0.10. Segments allow PiTiVi to track in- and out-markers for each video and audio buffer without altering the buffer itself, greatly speeding up the process of rearranging clips in the timeline. "Previously in order to do time-shifting we'd have to modify the timestamps of all buffers, whereas with segments we can give information as to when buffers that follow that event will be displayed, at what speed, etc..." In addition, Hervey added, PiTiVi development tests all GStreamer plugins to ensure that they fully conform to the API, such as making sure that all decoders can do sample-accurate decoding.

Furthermore, as with any GStreamer project, the modular nature of the media framework means that other projects can share in the improvements that originate in PiTiVi, and vice-versa. Because PiTiVi represents a different usage of GStreamer than the considerably more common "media player" paradigm, it stress-tests plugins in different ways.

Up next on PiTiVi

The next milestone in PiTiVi development is scheduled to be released in July, adding essential features like undo/redo, blending and compositing, and video capture. Because it is officially designated a development branch, the team has not pushed the 0.13.x series for inclusion in Linux distributions. Until then, users interested in testing the development releases can either download and compile source code packages, or if using Debian or Ubuntu, access development binaries through the project's Personal Package Archive (PPA). The PPA includes updated packages for GStreamer, GStreamer plugins, Gnonlin, and other key dependencies. Building from source also requires satisfying these dependencies, so instructions are provided on the PiTiVi wiki.

NLEs are notoriously complex beasts, and a stable, capable NLE remains a missing piece of the free software desktop for many users. Because it can leverage GStreamer for the heavy lifting of media processing, PiTiVi stands a better chance than most young projects of reaching dependable usability, and if the first release of the newly-rewritten core in 0.13.1 is any indication, the application is well on its way.

Comments (6 posted)

Sugar moves from the shadow of OLPC

Fourteen months ago, when One Laptop per Child (OLPC) announced that it was preparing to work with Windows, the free software community treated the news as a betrayal. However, nowhere was the reaction stronger than within OLPC itself. Within days, Walter Bender, who oversaw the development of Sugar, OLPC's graphical interface, had resigned and announced the creation of Sugar Labs, a non-profit organization for ensuring Sugar's continuation. Now, looking back over the year since then, Bender considers that Sugar Labs has progressed steadily towards its main goals: getting organized, taking advantage of the nature of free software to enhance education, and developing a community of teachers and students to help direct Sugar's development.

"It turns out that One Laptop per Child didn't really go down the Windows path," Bender admitted. "They're still shipping Linux and Sugar with every laptop, and they've just announced that their 1.5 machine is also going to be Sugar-based." However, he had no way of knowing that last year.

At any rate, the proposed change of operating systems was not the only reason for the creation of Sugar Labs. "At the same time, I was thinking that Sugar should be broader than just one particular hardware platform," Bender added. "We spent a lot of the first year undoing any specific dependencies on the One Laptop per Child hardware. What we've done is made Sugar run pretty much everywhere."

Sugar is now available in most major distributions, and Sugar Labs is currently in the middle of developing Sugar on a Stick, a USB installation. In addition, Sugar Labs has been discussing Sugar deployment with several netbook manufacturers.

"I think we've got a lot of ways to go in terms of raising awareness about Sugar," Bender said. "I don't think the world knows that Sugar's out there, and that it's a separate thing that can run outside of One Laptop per Child. That will take time. Our marketing budget is the same as all the rest of our budgets — zero — but we'll get there."

At the same time, Sugar Labs has established a structure more in keeping with the free software and interactive learning ideals of its members. Modeling itself after free software organizations like the GNOME Foundation, Sugar Labs is run by an elected Oversight Board, which is deliberately designed "to be pretty toothless. About the only power that the Oversight Board has is to appoint a committee to solve problems."

Instead, it is the project's various teams that make most of the decisions, with all discussions occurring on the same IRC channel. As with most free software projects, this organization is born out of necessity, since Sugar Labs does not actually have any office space, but Bender suggested that it is also an advantage in building the type of community that project members desire.

By contrast, he suggested that OLPC, which does have a physical location, "was struggling a bit to maintain communication with the global community. Conversations were not deliberately obfuscated, but, because they were happening in a room as opposed to online, a lot of people weren't hearing the conversation, and a lot of people felt they weren't part of the project because of that. We don't have one physical center of activity, and that plays out to our advantage, I think."

All in all, Bender stated, "Things have gone remarkably smoothly. We've been a pretty disciplined bunch. For instance, if you look at our release map, we're not letting features get ahead of our ability to deliver something that is robust and on time. For the most part, it's been a great year."

Free software ideals and education

However, for Bender, Sugar Lab's greatest accomplishments have not been in its organization, but in the advancement of its educational goals — goals that Bender views as meshing remarkably well with the ideals of free software

From the start, Sugar has been intended as more than an interface. Instead, Sugar Labs prefers to describe the software it develops as a learning platform. "We're not interested in anything except learning," Bender said. "as we make decisions about what we release and what we do, the first question we always ask is, 'How does this impact learning?' But 'platform' is important, too, because Sugar is not just a finished product that we give you to use. The implication is that Sugar is the platform on which you are going to build as well. We give you some scaffolding, but that scaffolding is there for you to build upon, as opposed to something you just use."

In this respect, Bender views Sugar as radically different from proprietary learning systems. "Often times, the tendency is to give children toy versions of a program, so that they don't hurt themselves. Also, the professional versions are expensive. Sugar takes a different approach. We want to give them tools that are real, and don't limit them in any way. But, at the same time, we're very cognizant that we have a path that doesn't require them to be an expert to get started. Really, that's an approach that is possible to achieve in free software, but quite difficult to achieve in any other way."

For example, Sugar's music activities begin with "tools that are literally accessible by a two year old — a sort of pound-on-the-keyboard activity. They can take that tool and use it over the network to make a band, and use it for sequencing and to compose music. then they can go into a synthesizer and start to play with wave forms, random number generators and envelope curves. Then they can go into a scripting language called cSounds and start to understand how music is scripted in the machine — and that's the same language that the pros use in Hollywood for scripting special effects. Then they can go into our View Source editor and hack the Python code that's underlying all these tools. They have the ability to go deeper into anything — and anything is determined by the child's interests, [or] by the teacher. It's not determined by the people writing the code."

In much the same way, Bender considers the collaboration framework that is part of the basic Sugar interface as a direct invitation for open-ended critical discussion. "This has a very direct connection to free software," he said. "One of the things about free software is that not only do you share, but you also engage in a critical dialogue about what you're sharing. The idea that ideas are there to be critiqued is one that a child has got to learn, and Sugar collaboration is as much about being engaged in criticism as it is about sharing." In other words, Sugar Labs considers the collaborative development found throughout free software projects as a model of exactly the sort of interaction that is ideal in education.

Community Building

Another area of success in the past year has been the increased participation of teachers and students in Sugar development.

"We've been really persistent about constantly going back to the teachers and saying, 'You've got to participate. You've got to give us feedback,'" Bender said. "'If you don't give us feedback, we're not going to learn, and you're not going to get the kinds of tools you need.'"

Bender acknowledged that finding the right channel for this feedback is difficult. He would personally prefer IRC, "because on IRC you've got this discussion among experts [who are] solving problems," as well as access at any time of the day. However, experience has taught him that IRC is not "a tool that teachers are going to be comfortable with, at least initially." Instead, teachers seem to prefer email, despite the fact it is not instantaneous and that communication is asynchronous.

But, regardless of the medium, teachers and students are starting to let their views be known. For instance, in Uruguay, teachers using the Turtle Art activity (Sugar's name for an application) requested a square root function they could use to teach the Pythagorean Theorem. At first, Bender told them to write the function themselves. But, eventually he realized that such a contribution was beyond most teacher's ability. He added an extension that gave them several different ways to add to the code, including some that did not require programming expertise, and the square root function got done.

What was more important than the specific function, though, was the lesson Bender learned about coding styles in general. "I wasn't thinking how I could make things extendable by teachers when I [wrote] it," Bender said. "Now, that's at the forefront of my mind."

The last year does not seem to have produced any example of student involvement comparable to one Bender remembered from 2007, when Igbo-speaking students in Nigeria produced their own spell-check dictionary for Sugar's Abiword-based Write activity. However, Bender did mention that he was scheduled to be interviewed soon by students in Boston about Sugar, and an upcoming project to have students write Sugar documentation, so Sugar Labs is trying to engage students in discussion as well.

Still another mechanism for receiving feedback is the relatively new practice of establishing Sugar Labs wherever the interest exists. The goal, Bender said, is to "have those local centers be responsible for local support and localization. Those centers can be pretty much structured in any way that's appropriate to that region. We aren't going to impose a structure on them. So long as they share our core values, they can be Sugar Labs. That's beginning to happen now."

A Sugar Labs now exists in Colombia, and others are being organized in Washington, D.C. and Lima, Peru. "Each of them has its own set of issues they're trying to focus on, but all of them at the same time are participating in the global dialog," Bender said.

Building for a pedagogical future

Fourteen months after Sugar Labs was established, OLPC machines remain the major deployment for Sugar. However, Bender is encouraged by signs that the educational ideas behind Sugar are starting to be adopted elsewhere in free software.

Sugar has a long relationship with Fedora, the basis for the original OLPC operating system and an active participant in everything from Sugar on a Stick to the Oversight Board. But, in the last year, Sugar Labs has started to work more closely with distributions that are specifically geared towards education. For example, Bender suggests that his discussions with the developers of Skolelinux, an educational distribution based on Debian, may have helped them move their planning beyond the mechanics of installation and system maintenance or of software selection.

"I think they were only just beginning to think deeply on how Linux can impact on learning," Bender said. "I think that's why there's a lot more interest in Sugar recently, because we've been thinking about that question right from the beginning. I'm convinced that there's not just technical merit in Linux for learning — there's cultural and pedagogical merit."

Increasingly, Bender hoped, other parts of the free software community will take over some of the technical aspects of Sugar, such as solving their own problems with adapting Sugar to specific distributions. If that happens, Bender said, "that means our attention can be elsewhere, focusing on the dialog with teachers and with students, and making sure that deployment is specific to the needs of schools is being done."

Concluding, Bender said, "This is important stuff. We've got to do something about giving every child an opportunity to learn and be a critical thinker. There's all these problems that the wold is facing right now, and the only way that these problems are going to be solved is by putting more minds around them. Whether it's Sugar or something else, I think that the community really needs to be working hard at engaging more people in problem-solving and engaging them in learning. And I think that's the strength of the Linux community."

At a time when discussions about free and open source software tend to be centered on business, such comments may sound outdated in their idealism. But, after listening to Bender, the only conclusion one can come to is that is why Sugar Labs was created in the first place: To provide an alternative perspective that is harder to maintain in a commercial venture like OLPC. By that criteria, Sugar Lab's first year can be counted as a solid success.

Comments (5 posted)

Page editor: Jake Edge
Next page: Security>>