Leading items

Book review: Open Advice

The recently released Open Advice has much to offer those who are new to free software and its communities, but there is plenty of interest to veterans as well. It is a collection of essays from an auspicious number of contributors (42) to free and open source software (FOSS) that centers around the idea of "what we wish we had known when we started". As might be guessed, the book encompasses more than that—it ranges all over the FOSS map—including recollections, war stories, philosophical musings, academic research, and good advice.

The book was the brainchild of KDE contributor Lydia Pintscher, who served as the editor as well as contributing one of the essays, "Being Allowed to Do Awesome". The book was released at the recent FOSDEM conference in Brussels and is available in a variety of formats (PDF, EPUB, Mobi). But the book is licensed under the CC-BY-SA license, which means that the LaTeX source is also available. In addition, printed versions of the book can be ordered from Lulu and, soon, Amazon.

The material is spread out over 16 chapters after a foreword by Free Software Foundation Europe founder Georg Greve. He sets the tone for the book by introducing the ideas of free software and communities, specifically connecting the rise of the internet and free software as "co-dependent" developments. Not only does free software run much of the internet, but many of the internet giants that are popular today (Google, Facebook, and Twitter are specifically mentioned) could not have gotten so far so fast without depending on free software.

What follows is a bit of a wander through the different facets of our communities, with an eye toward passing on some of the hard-won knowledge that these contributors have gained. Armijn Hemel notes that projects need to coalesce around code, so that there is something to work with and improve. All the design documents and ideas in the world will not actually build into a community. Evan Prodromou furthers that idea by noting that once there is code, the founder needs to step back and let others contribute. It is, he says, sometimes hard to do, but is essential:

Markus Krötzsch and Felipe Ortega talk about the connection between academia and FOSS, but come at it from different angles. Krötzsch looks at the challenges that researchers face when opening their code, many of which are applicable to anyone trying to form a community of users and contributors. He likens the effort to gardening. Ortega looks at how different FOSS communities evolve as project members come and go. He notes several studies of different kinds of projects and how they have overcome the "generational relay" problem—handing off the project to new leadership over time.

But in order to increase a project's contributors, some recruitment and mentoring are needed. That's the subject of one of the chapters. In it, Leslie Hawthorn points out that contrary to their belief, people new to the project have something unique to offer:

Hawthorn's point crosses into the realm of documentation, which is the subject of a later chapter, but that just highlights one of the strengths of the book. Few of the essays neatly fit into the categories of the chapter they appear in. They largely represent a personal view of the experiences of the author, which often range across various parts of the FOSS world. They are also uniformly encouraging to those who may know little or nothing about how to participate and why they might want to.

Pintscher's entry notes that the biggest enemy of free software is "not who most people on the Internet think it is", but is, instead, a lack of participation. She notes that it takes active effort from existing project members to get new contributors involved, but that it's worth the effort. It's also important to recruit from outside the existing contributor base:

Henri Bergius writes about cross-project collaboration and notes that creating libraries, rather than frameworks, better fosters that collaboration. In addition, he says, meeting in person can go a long way toward helping projects work more closely together:

That sentiment is echoed by others, including Nóirín Plunkett in the chapter on "Conferences and Sprints":

But, all is not "sweetness and light" in the free software world as Máirín Duffy Strode describes in her look at the interaction between designers and developers. Her essay, "Don't Be Shy", suggests that designers (and, by extension, all new contributors) make their needs known so that they can "help the project help you help them", which can't happen without making it clear what's needed to get the job done. She also notes a bit of cautionary tale about being chased away from a project and suggests that new contributors be persistent:

There are several essays on various aspects of documentation. From Atul Jha's reflection on how Eric S. Raymond's writings (in particular the jargon file and "How To Ask Questions The Smart Way") inspired him, to Shaun McCance's look at how to use the "crowd" to generate project documentation, there is a wealth of interesting information. Rich Bowen plays off Larry Wall's famous "virtues of a good programmer" (laziness, impatience, and hubris) and notes that those virtues unfortunately give some programmers a "license to be jerks". He goes on to describe virtues for another group:

Anne Gentle introduces an interesting idea about how a new contributor can "take the pulse" of a project to get a feeling for how hard or easy it will be to get involved:

In the realm of usability, Guillaume Paumier offers up some things he's learned working with the Wikimedia Foundation. He suggests that developers sit down and passively watch users interact with their application. It is "truly an eye-opening experience". Furthermore, he offers up an important point that sometimes gets lost in the development process: "Users are an unpredictable species. But they are on your side. Learn from them."

Federico Mena Quintero has a lengthy essay on "Software that Has the Quality Without A Name". In it he takes concepts from the architectural (i.e. buildings, not software) studies done by Christopher Alexander and others and applies them to software design. The "quality without a name" is embodied in buildings and spaces that are eminently livable. Alexander has come up with a number of properties that govern such spaces and Quintero (by way of Richard Gabriel's Patterns of Software) applies those ideas to software. It is one of the more philosophical essays in the book, and well worth reading in its entirety.

There is also some real "nuts and bolts" advice on things like conference planning (from Dave Neary) and a nice explanation of "How to Ask for Money" (for conferences) by Selena Deckelmann. Beyond that, there is advice on community management and the role of a community manager from Jono Bacon, thoughts on the intersection of law and FOSS from Till Jaeger, ideas about testing from Ara Pulido (and others), and on and on. Not mentioning one of the essays in this review is in no way an indictment of the essay or author, there is more there than one could hope to cover. In addition, each reader will undoubtedly have their own slant on the most interesting and useful essays in the book.

Open Advice is a book that will be helpful to those who are new to FOSS, but, because of the individual voices, styles, and tones, it doesn't read like a "how to". It could even be recommended to those who aren't necessarily interested in contributing, but are curious about what this "free software thing" is all about. It is, in short, a great book for a variety of audiences and the (mostly) two or three page essays make it easy to read, while the anecdotes and recollections personalize it. The authors, editor, and everyone else who helped should be very pleased with the result. Readers will be too.

Comments (16 posted)

FOSDEM: The Wayland display server

In his talk at FOSDEM (Free and Open Source Software Developers' European Meeting) 2012 in Brussels, Kristian Høgsberg gave an overview of the Wayland display system, where it comes from and how the Linux graphics stack has evolved to make Wayland feasible. He also shared some information about the schedule of the 0.85 and 1.0 releases. If all goes according to plan, we'll see the first stable version of Wayland by the end of the year.

Replacing a tried-and-true technology such as X by a new one such as Wayland is no small task, and the idea alone has already stirred up the Linux community in the past. However, not every critic understands exactly what Wayland is. Apparently some even have various strange opinions about Kristian, for instance that he holds a grudge against X or that he even doesn't know anything about how X works. To dot the i's, Kristian started his talk by clarifying that he has been a core X developer since 2004. "I admire the design and implementation of X, it just happens that it's the wrong solution for what we do now." Kristian started developing Wayland in 2008 in his spare time while working for Red Hat, and now he is working on it at Intel in the graphics team of the Open Source Technology Center.

Where do we come from?

A while ago, X did lots of things related to its primary task as a display server. For instance, it did font management and font rendering, graphical mode setting and acceleration code was tied up in X specific drivers, the X server had input drivers to parse various input device serial formats, and so on. However, bit by bit many of these tasks were moved into separate components, many of them in the kernel. Wearing his Red Hat hat, Kristian worked on AIGLX (which allowed compositors to run with hardware acceleration on X), DRI2 (which provides OpenGL hardware acceleration), KMS (Kernel Mode Setting), and GEM (Graphics Execution Manager, doing memory management for graphics drivers). In principle, all hardware drivers needed by X are now in the Linux kernel (KMS) or in Mesa (an OpenGL implementation).

Much of this work was done as part of Kristian's job as a member of Red Hat's X team to make a composited desktop possible. Without a compositor, each application renders its output directly into the buffer of the X server. With a compositor (such as Compiz, KDE's KWin, GNOME 3's Mutter, and Xfce's Xfwm), applications render their output in their own private off-screen buffers. The compositor renders the final desktop output by painting those buffers onto the screen's frame buffer. On a modern Linux desktop, compositing has become a basic expectation and requirement, Kristian says: "Compositing is not only about those 2D and 3D visual effects, but it has other fundamental advantages, such as flicker-less movements of windows."

But now that we have reached a point where most of the hardware drivers implement KMS or Mesa and compositing is used for drawing application windows on the screen, most of the work in the window system is done by the compositor and the applications themselves; the X server is just used as a middle-man for input. Kristian wanted to create a display server that directly supports this new window system architecture from the ground up. This became Wayland, which integrates the display server, window manager, and compositor into one component, and is, according to Kristian, "essentially just consolidating existing practices and simplifying the X architecture to what we are using it for nowadays." The result is a window system architecture that is more responsive and has less graphical artifacts such as tearing or flickering. More details about Wayland's architecture and the difference with X can be found on the project's wiki.

Hardware support

So, with this new window system architecture, Wayland merges the compositor into the display server, and applications talk directly to the compositor. The compositor reads input from devices such as the keyboard and mouse using the kernel's evdev subsystem, and it distributes this input to the applications. The compositor uses KMS to bring up the display, and applications push their buffers to the compositor, which combines the buffers from the various applications and renders the result using direct rendering, through OpenGL ES, to the KMS buffers. "There is no hardware-specific code whatsoever in the compositor code," Kristian emphasizes, "and there is also no rendering API for drawing lines, text, and so on: it's all just direct rendering."

Kristian's short answer to the question which graphics hardware support Wayland is "If the Mesa driver for your graphics hardware supports DRI2 under X and has KMS support, you can run Wayland." Currently, this means that you'll have most luck with the open source drivers for Intel, AMD and NVIDIA graphics hardware, but when driver writers add new chipset support in Mesa, it will automatically support Wayland. It's still unclear what AMD and NVIDIA will do with their proprietary drivers.

Wayland support in graphical toolkits

But applications have to be modified to be able to use Wayland instead of X, of course. Fortunately, most applications don't talk to the X window system directly but are using graphical toolkits, which provide the common user interface elements such as text fields, scrollbars, buttons, and so on. So most of the work in porting an application from X to Wayland is in porting the toolkit it uses. According to Kristian, all major toolkits are being ported to Wayland as we speak, including GTK+ 3, Qt 5, EFL (Enlightenment Foundation Libraries), Clutter, and SDL (Simple DirectMedia Layer). The Qt team at Nokia is maintaining and driving the Qt port themselves, he said. Wayland's web site has instructions on how to use these toolkits with Wayland. Applications that are using their own toolkit, such as Blender and LibreOffice, will have to do the porting effort themselves.

Porting a graphical toolkit to Wayland poses some interesting challenges, for instance because Wayland doesn't support some things that are quite natural in X, such as pointer grabbing: in Wayland an application currently cannot claim the keyboard or mouse input. However, games regularly grab the mouse cursor to prevent the player from accidentally losing mouse control over their game, and pop-up windows rely on grabbing the keyboard and pointer too. Another challenge is the support for client-side window decorations, which allows individual applications to control their appearance. Both challenges are being worked on.

Applications that aren't using a Wayland-compatible graphical toolkit or that have some problems running on Wayland can still run on an X server that runs as a Wayland client. Of course this adds some overhead, but it should be minimal. Running a rootless X Server as a Wayland client could also be a temporary solution to keep using network transparency, a feature that Wayland lacks. In his talk, Kristian reassured the audience that the current lack of network transparency is not definitive: "While supporting remote displays is not a priority now for Wayland, nothing in Wayland's architecture makes this impossible."

Toward a stable API

The first real release of Wayland happened a few days after Kristian's talk at FOSDEM: Wayland 0.85. The code is divided in two parts: Wayland is the protocol and IPC mechanism, while Weston is the reference implementation of the compositor (and thus also the display server). "Weston has around 10,000 lines of code and it can be used as the base for a compositor for mobile or embedded systems," according to Kristian, who ran Weston on his laptop during his FOSDEM presentation. However, existing X compositors such as KWin and Mutter can also be modified to become a Wayland compositor.

Kristian promises that the 0.85 release is going to be protocol and API stable. The point of the release is that developers can begin experimenting with Wayland and porting their toolkits and applications. While it used to be that you had to compile a special KMS "pageflip" kernel, a patched Mesa branch, and Kristian's own EGL library to be able to run Wayland on your system, now everything should be upstream. Kristian hopes to put out a Wayland 1.0 release in late 2012 or early 2013. So while Wayland will not replace X overnight, we will likely see the first mainstream use of the new window system architecture in 2013.

Comments (38 posted)

Gray areas in software licensing

This year saw a new addition to FOSDEM in Brussels - "legal issues" gained its own DevRoom and joined the long list of topics presented and discussed during the conference. I was privileged to be asked to speak at the track, and when Bradley Kuhn asked me what I'd like to speak about, my initial reaction was the theme: "how much I hate having to care about the law".

Developers hate the law. Or, to be more precise, I hate the law. At least, whenever I'm working on a software project. In my opinion, every time a software developer has to stop and think about the law, that's a bug. Either he wants to do something that he shouldn't be doing, or the law is getting in the way of him doing something which it would be reasonable for him to do.

In case it's not obvious yet, I am not a lawyer, so nothing in here should be taken as legal advice. But, over the years, while working as a GIMP developer, as community and product manager of the OpenWengo project, or as an independent consultant, I have had to stop and consider the potential impact of the law on what I was doing. And every time that I have sought legal advice on the issues, the result has been unsatisfactory. Here are a few examples from my personal experience.

GIMP plug-ins

In the late 1990s and early 2000s, the licensing of plug-ins was a hot topic in the GIMP community that came up repeatedly. Does the GPL allow people to develop proprietary GIMP plug-ins? It was certainly the intention of the developers active at the time to allow proprietary plug-ins. To explain why this question is even interesting, some technical details on GIMP plug-ins are useful.

GIMP plug-ins run in a separate process from the GIMP core. They link to libgimp, a library that was licensed under the LGPL in 1997, which is a wrapper around the PDB, the Procedural Database. The Procedural Database is how plug-ins communicate with the core GPL application: data is sent via shared memory, with the PDB defining arguments and return values from plug-in functions. Plug-ins can call core functions, again via the PDB, to get image data from a drawable (a GIMP object representing anything you can draw on), or for any of a range of core functions.

But some argued that since GIMP plug-ins could only work with the GIMP core, that they were really derivative works, and were thus covered by the GPL. When we asked for definitive legal advice on the issue, the answer we got back was not entirely satisfying.

Maybe.

Ugh. So to clarify the situation, the GIMP developers added a license clarification to the source code, alongside the GNU GPL COPYING file, stating that we considered plug-ins to be "mere aggregation", and not a "derivative work". This may not be legally rigorous, but it was enough to allow us to forget about the issue and get back to coding.

Distributing GIMP plug-ins

That is, until it was pointed out by Debian that GIMP was shipping some GPL-incompatible code in plug-ins that we were distributing in the gimp .tar.gz. This issue ended up blocking the release of GIMP 1.2.4 for over a year. So the question arose: can we include non-GPL plug-ins in a GIMP package? The answer was, you guessed it, "maybe". At least, that's as far as we could tell by discussing it in a Debian bug report and on gimp-developer.

While GIMP plug-ins might be a "mere aggregation" with the GIMP core, apparently, there was an expectation that all the plug-ins which were shipped with GIMP would be GPL compatible. Over the years, we had included a number of code snippets from other open source projects which were licensed under other licenses. One of the projects was SIOD, Scheme in one Defun, a Scheme implementation by George Carrette. SIOD was the back-end of Script-fu, GIMP's scripting plug-in, at the time. SIOD had a very liberal license which included an "advertising clause", requiring that "the permission notice appear in supporting documentation", a clause which was widely argued to be incompatible with the GPL at that time. It turned out that there were about a dozen occurrences of this advertising clause in the code for plug-ins at the time.

Since Debian wouldn't ship GIMP with these plug-ins unless we made the licensing consistent, and since some GIMP developers also expected us to ship only GPL-compatible plug-ins, this was a major problem. So I and others spent a month in 2003 chasing down authors at five-year-old email addresses, removing or rewriting code snippets when we could do so, and on occasion removing offending copyright headers when we decided that the amount of code covered by the problematic licenses was so small that it was not a problem. I am sure I would have preferred to do other things with my time - and I was certainly not as methodical as Gervase Markham when he was relicensing Mozilla code - a process that took over 4 years. Were we legally rigorous? Maybe.

OpenWengo codecs

My next example is audio codecs from the OpenWengo project (site now dead). Some codecs like G729 were patent encumbered. In addition, widely used reference implementations of codecs like iLBC were included as sample code in standards documents and were only available under GPL-incompatible and non-free software licenses. The way we tried to get around this problem was to encapsulate these codecs in a standard API, and install them in a common directory. At start-up, we scanned the directory, and used dlopen() to open the libraries present there. We could then call the standard function to register the codecs and pass data streams for decoding and encoding.

A question arose for some of the non-GPL codecs: could we do something in the start-up process to make it easier for the user to obtain these codecs? One possibility would be to present the user with a dialog, as part of the first application start-up, which fetched the proprietary codecs and installed them in the common directory. But, would that be against the spirit of the GPL, and have us running the risk of being called a GPL violator by one of our users?

The answer, apparently, is "maybe".

By divorcing the loading of the plug-in at run-time from the distribution of the plug-in, we were probably absolving ourselves of any GPL obligations. At least, that is the legal advice Wengo obtained at the time. However, whether making it easier to get the codecs at start-up was the legal equivalent to including the codecs in the package was, at best, ambiguous.

Hunspell dictionaries

Last year, I was asked an interesting question by a client: Can I include Hunspell and the Czech dictionary in a proprietary application?

The answer, of course, is "maybe".

Hunspell changed its license in 2006, to the MPL/GPL/LGPL tri-license to enable inclusion in Mozilla. It is used as the spell-checker for OpenOffice.org/LibreOffice as well as Mozilla Thunderbird and Firefox. Of course, the tri-license enables the embedding of Hunspell in proprietary applications. So where's the problem? It turns out that some Hunspell dictionaries, including Catalan, Danish and Czech, are licensed under GPL only. It's a bit curious why anyone would use the GPL, a license intended for use with source code, for a dictionary. Regardless, the question is: does including Hunspell in another application, with a GPL dictionary, constitute a derivative work, or an aggregation?

At one level, the question is whether it even makes sense for a dictionary to have a license. Thanks to Feist v Rural, copyright only applies to a "work" if there is significant creativity involved in it. If a Hunspell dictionary were just a list of words in alphabetical order, then at least in the US, copyright would not apply. However, in other jurisdictions, there are laws covering the constitution of databases, and some form of protection is afforded to the author. In addition, a Hunspell dictionary isn't really just a list of words - it also contains grammatical and hyphenation information, which involves some creativity. So there's a solid argument that the dictionary is copyrightable.

The second question is whether including the dictionary with Hunspell and other software makes a derivative work, or an aggregate work. One part of that is whether the component is useful without the Hunspell spellchecker. Since it's just a dictionary, it can be used with other spell-checkers, including aspell. Hunspell could easily use another Czech dictionary released under another license. So, since the two bits are easily dissociable, Hunspell with the dictionary probably makes an aggregate work, and the license of the dictionary does not impose any requirements on other software shipped with it. This is the advice that the OpenOffice.org project received from the FSF licensing list, and so the GPL dictionaries are included in the LibreOffice and OpenOffice.org distributions.

Mozilla, on the other hand, do not ship any GPL dictionaries. When I asked Gervase why, he answered:

In other words, the intent of the author trumps all in this case, for Mozilla.

So - maybe.

Conversations with lawyers

The problem for developers is that conversations with lawyers typically end up with "maybe". We're looking for what we may and may not do, and often, in the absence of jurisprudence, lawyers deal in shades of gray. All too often, the end result for developers is to avoid any risk.

A friend once told me a story of a meeting with her boss and "legal", where she came out of the meeting disappointed. When her boss asked her what was wrong, she said "the lawyers told us we can't do what we'd like to." To which he replied, "No, they just told us the risks involved. Now we have to decide whether we're prepared to accept those risks or not." Instead of being the end of the conversation, "maybe" could be the start of it. Lawyers (and , in the free software world, some non-lawyers) will help you identify risks. But then it is up to the project to decide whether to take the risks or not.

Comments (17 posted)

Page editor: Jonathan Corbet
Next page: Security>>