The recently released Open Advice has much to offer those who are
new to free software and its communities, but there is plenty of interest
to veterans as well. It is a collection of essays from an auspicious
number of contributors (42) to free and open source software (FOSS) that centers
around the idea of "what we
wish we had known when we started". As might be guessed, the book
encompasses more than that—it ranges all over the FOSS
map—including recollections, war stories, philosophical musings,
academic research, and good advice.
The book was the brainchild of KDE contributor Lydia Pintscher, who served
as the editor as well as contributing one of the essays, "Being Allowed to
Do Awesome". The book was released at the recent FOSDEM conference in
Brussels and is available in a variety of formats (PDF, EPUB, Mobi). But
the book is licensed under the CC-BY-SA license,
which means that the LaTeX
source is also available. In addition, printed versions of the book can be
ordered from Lulu and, soon, Amazon.
The material is spread out over 16 chapters after a foreword by Free
Software Foundation Europe founder Georg Greve. He sets the tone for the
book by introducing the ideas of free software and communities,
specifically connecting the rise of the internet and free software as
"co-dependent" developments. Not only does free software run much of the
internet, but many of the internet giants that are popular today (Google,
Facebook, and Twitter are specifically mentioned) could not have gotten so
far so fast without depending on free software.
What follows is a bit of a wander through the different facets of our
communities, with an eye toward passing on some of the hard-won knowledge
that these contributors have gained. Armijn Hemel notes that projects need
to coalesce around code, so that there is something to work with and
improve. All the design documents and ideas in the world will not actually
build into a community. Evan Prodromou furthers that idea by noting that
once there is code, the founder needs to step back and let others
contribute. It is, he says, sometimes hard to do, but is essential:
If your software is just for you, you can keep the codebase and
surrounding infrastructure as a personal playground. But if you
want your software to be used, to mean something to other people,
to (maybe) change the world, then you are going to need to build up
a thriving, organic community of users, core committers, admins and
add-on developers. People need to feel like they own the software, in
the same way that you do.
Markus Krötzsch and Felipe Ortega talk about the connection between
academia and FOSS, but come at it from different angles. Krötzsch looks at
the challenges that researchers face when opening their code, many of which
are applicable to anyone trying to form a community of users and
contributors. He likens the effort to gardening. Ortega looks at how
different FOSS communities evolve as project members come and go. He notes
several studies of different kinds of projects and how they have overcome
the "generational relay" problem—handing off the project to new
leadership over time.
But in order to increase a project's contributors, some recruitment and
mentoring are needed. That's the subject of one of the chapters. In it,
Leslie Hawthorn points out that contrary to their belief, people new to the
project have something unique to offer:
As a new contributor to a project, you are
an invaluable asset not for your knowledge, but for your ignorance.
When first starting work in FOSS, nothing seems (to you) so obvious
as to be unworthy of mention. Take notes on the problems you
have encountered and how they were fixed, then use those notes to
update the project documentation or work with the community to
prepare screen casts or other training materials for particularly tough
problems. When you encounter something truly frustrating, realize
you are in the spectacular position of helping make sure the next
person who comes along does not encounter the same difficulties.
Hawthorn's point crosses into the realm of documentation, which is the
subject of a later chapter, but that just highlights one of the strengths
book. Few of the essays neatly fit into the categories of the chapter they
appear in. They largely represent a personal view of the experiences of
the author, which often range across various parts of the FOSS world.
They are also uniformly encouraging to those who may know little or nothing
about how to participate and why they might want to.
notes that the biggest enemy of free software is "not who most people
on the Internet think it is", but is, instead, a lack of
participation. She notes that it takes active effort from existing project
members to get new contributors involved, but that it's worth the effort.
It's also important to recruit from outside the existing contributor
the people you already have are great, but think about all the other
great people you are missing out on, who could bring new ideas and
skills to your project.
Henri Bergius writes about cross-project collaboration and notes that
creating libraries, rather than frameworks, better fosters that
collaboration. In addition, he says, meeting in person can go a long way
toward helping projects work more closely together:
Meet the other guys. If you are from the GNOME project, go
to aKademy and give a talk, and if you are a KDE developer,
go and talk at GUADEC. After you have shared a beer or two
collaboration over IRC happens much more naturally.
That sentiment is echoed by others, including Nóirín Plunkett in the
chapter on "Conferences
It is the richness of our communities that makes Open Source what
it is, and the shared striving towards common goals. And of course,
the music sessions, the meals, the pints, and the parties! These are
the things that bring us together, and you will find that once you
have met people in person, even your email interactions will be much
richer, much more fulfilling, and much more fruitful, than they had
But, all is not "sweetness and light" in the free software world as Máirín
Duffy Strode describes in her look at the interaction between designers and
Her essay, "Don't Be Shy", suggests that designers (and, by extension, all
new contributors) make their needs known so that they can "help the
project help you help them", which can't happen without making it
clear what's needed to get the job done. She also notes a bit of
cautionary tale about being chased away from a project and suggests that
new contributors be persistent:
This set me back a few years in getting involved – just a few harsh
words on IRC made me afraid to try again for almost 5 years. I did
not discover until much later that the person who had essentially
chased me out of that project's IRC channel was on the fringes of
the project and had a long history of such behavior, and that I really
had not done anything wrong. If I had only kept trying and talked
to other people, I may have been able to get started back then.
There are several essays on various aspects of documentation. From Atul
Jha's reflection on how Eric S. Raymond's writings (in particular the jargon
file and "How To Ask Questions The Smart Way") inspired him, to Shaun
McCance's look at how to use the "crowd" to generate project documentation,
there is a wealth of interesting information. Rich Bowen plays off Larry
Wall's famous "virtues of a good programmer" (laziness, impatience, and
hubris) and notes that those virtues unfortunately give some programmers a
"license to be jerks". He goes on to describe virtues for
What I have learned over my years in Open Source documentation
is that the three primary virtues of a documentation specialist, and,
more generally, of customer support, are laziness, patience, and humility. And that the over-arching virtue that ties these all together
Anne Gentle introduces an interesting idea about how a new contributor can
"take the pulse" of a project to get a feeling for how hard or easy it will
be to get involved:
I wish when I started that I had some ability to gather the "social
weather" of an online community. When you walk into a restaurant
with white tablecloths and couples dining and a low-level volume
of conversations, the visual and auditory information you receive
sets the ambiance and gives you certain clues about what to expect
from your dining experience. You can translate this concept of social
weather to online communities as well. An open source community
gives certain clues in their mailing lists, for example. A list landing page prefixed with a lot of rules and policy around posting will
be heavy in governance. A mailing list that has multiple posts emphasizing that "there are no dumb questions" is more approachable
for new doc writers.
In the realm of usability, Guillaume Paumier offers up some things he's
learned working with the Wikimedia Foundation. He suggests that developers
sit down and passively watch users interact with their application. It is
"truly an eye-opening
experience". Furthermore, he offers up an important point that
sometimes gets lost in the development process: "Users are an unpredictable species. But they are on your side.
Learn from them."
Federico Mena Quintero has a lengthy essay on "Software that Has the Quality Without
A Name". In it he takes concepts from the architectural (i.e. buildings,
not software) studies done by
Christopher Alexander and others and applies them to software design. The
"quality without a name" is embodied in buildings and spaces that are
eminently livable. Alexander has come up with a number of properties that
govern such spaces and Quintero (by way of Richard Gabriel's Patterns of
Software) applies those ideas to software. It is one
of the more philosophical essays in the book, and well worth reading in its
There is also some real "nuts and bolts" advice on things like conference
planning (from Dave Neary) and a nice explanation of "How to Ask for Money"
(for conferences) by Selena Deckelmann. Beyond that, there is advice on
community management and the role of a community manager from Jono Bacon,
thoughts on the intersection of law and FOSS from Till Jaeger, ideas about
testing from Ara Pulido (and others), and on and on. Not mentioning one of
the essays in this review is in no way an indictment of the essay or
author, there is more there than one could hope to cover. In addition, each reader will
undoubtedly have their own slant on the most interesting and useful essays
in the book.
Open Advice is a book that will be helpful to those who are new to
FOSS, but, because of the individual voices, styles, and tones, it doesn't
read like a "how to". It could even be recommended to those who aren't
necessarily interested in contributing, but are curious about what this
"free software thing" is all about. It is, in short, a great book for a
variety of audiences and the (mostly) two or three page essays make it easy
to read, while the anecdotes and recollections personalize it. The
authors, editor, and everyone else who helped should be very pleased with
the result. Readers will be too.
Comments (16 posted)
In his talk at FOSDEM (Free and Open Source Software Developers' European Meeting) 2012 in Brussels, Kristian Høgsberg gave an overview of the Wayland display system, where it comes from and how the Linux graphics stack has evolved to make Wayland feasible. He also shared some information about the schedule of the 0.85 and 1.0 releases. If all goes according to plan, we'll see the first stable version of Wayland by the end of the year.
Replacing a tried-and-true technology such as X by a new one such as
Wayland is no small task, and the idea alone has already stirred up the
Linux community in the past. However, not every critic understands exactly what Wayland is. Apparently some even have various strange opinions about Kristian, for instance that he holds a grudge against X or that he even doesn't know anything about how X works. To dot the i's, Kristian started his talk by clarifying that he has been a core X developer since 2004. "I admire the design and implementation of X, it just happens that it's the wrong solution for what we do now." Kristian started developing Wayland in 2008 in his spare time while working for Red Hat, and now he is working on it at Intel in the graphics team of the Open Source Technology Center.
Where do we come from?
A while ago, X did lots of things related to its primary task as a display
server. For instance, it did font management and font rendering, graphical
mode setting and acceleration code was tied up in X specific drivers, the X
server had input drivers to parse various input device serial formats, and
so on. However, bit by bit many of these tasks were moved into separate
components, many of them in the kernel. Wearing his Red Hat hat, Kristian worked on AIGLX (which allowed compositors to run with hardware acceleration on X), DRI2 (which provides OpenGL hardware acceleration), KMS (Kernel Mode Setting), and GEM (Graphics Execution Manager, doing memory management for graphics drivers). In principle, all hardware drivers needed by X are now in the Linux kernel (KMS) or in Mesa (an OpenGL implementation).
Much of this work was done as part of Kristian's job as a member of Red
Hat's X team to make a composited desktop possible. Without a compositor, each application renders its output directly into the buffer of the X server. With a compositor (such as Compiz, KDE's KWin, GNOME 3's Mutter, and Xfce's Xfwm), applications render their output in their own private off-screen buffers. The compositor renders the final desktop output by painting those buffers onto the screen's frame buffer. On a modern Linux desktop, compositing has become a basic expectation and requirement, Kristian says: "Compositing is not only about those 2D and 3D visual effects, but it has other fundamental advantages, such as flicker-less movements of windows."
But now that we have reached a point where most of the hardware drivers
implement KMS or Mesa
and compositing is used for drawing application windows on the screen, most
of the work in the window system is done by the compositor and
the applications themselves; the X server is just used as a middle-man
for input. Kristian wanted to create a display server that directly
supports this new window system architecture from the ground up. This
became Wayland, which integrates the display server, window manager, and
compositor into one component, and is, according to Kristian,
"essentially just consolidating existing practices and simplifying
the X architecture to what we are using it for nowadays." The result
is a window system architecture that is more responsive and has less
graphical artifacts such as tearing or flickering. More details about Wayland's
architecture and the difference with X can be found on the project's
So, with this new window system architecture, Wayland merges the
compositor into the display server, and applications talk directly to the
compositor. The compositor reads input from devices such as the keyboard
and mouse using the kernel's evdev subsystem, and it distributes this input
to the applications. The compositor uses KMS to bring up the display, and
applications push their buffers to the compositor, which combines the
buffers from the various applications and renders the result using direct
rendering, through OpenGL
ES, to the KMS buffers. "There is no hardware-specific code
whatsoever in the compositor code," Kristian emphasizes, "and
there is also no rendering API for drawing lines, text, and so on: it's all
just direct rendering."
Kristian's short answer to the question which graphics hardware support Wayland is "If the Mesa driver for your graphics hardware supports DRI2 under X and has KMS support, you can run Wayland." Currently, this means that you'll have most luck with the open source drivers for Intel, AMD and NVIDIA graphics hardware, but when driver writers add new chipset support in Mesa, it will automatically support Wayland. It's still unclear what AMD and NVIDIA will do with their proprietary drivers.
Wayland support in graphical toolkits
But applications have to be modified to be able to use Wayland instead
of X, of course. Fortunately, most applications don't talk to the X window
system directly but are using graphical toolkits, which provide the common
user interface elements such as text fields, scrollbars, buttons, and so
on. So most of the work in porting an application from X to Wayland is in
porting the toolkit it uses. According to Kristian, all major toolkits are
being ported to Wayland as we speak, including GTK+ 3, Qt 5, EFL (Enlightenment
Foundation Libraries), Clutter, and SDL (Simple DirectMedia Layer). The Qt
team at Nokia is maintaining and driving the Qt port themselves, he
said. Wayland's web site has instructions on how to use these toolkits with Wayland. Applications that are using their own toolkit, such as Blender and LibreOffice, will have to do the porting effort themselves.
Porting a graphical toolkit to Wayland poses some interesting challenges, for instance because Wayland doesn't support some things that are quite natural in X, such as pointer grabbing: in Wayland an application currently cannot claim the keyboard or mouse input. However, games regularly grab the mouse cursor to prevent the player from accidentally losing mouse control over their game, and pop-up windows rely on grabbing the keyboard and pointer too. Another challenge is the support for client-side window decorations, which allows individual applications to control their appearance. Both challenges are being worked on.
Applications that aren't using a Wayland-compatible graphical toolkit or that have some problems running on Wayland can still run on an X server that runs as a Wayland client. Of course this adds some overhead, but it should be minimal. Running a rootless X Server as a Wayland client could also be a temporary solution to keep using network transparency, a feature that Wayland lacks. In his talk, Kristian reassured the audience that the current lack of network transparency is not definitive: "While supporting remote displays is not a priority now for Wayland, nothing in Wayland's architecture makes this impossible."
Toward a stable API
The first real release of Wayland happened a few days after Kristian's talk at FOSDEM: Wayland 0.85. The code is divided in two parts: Wayland is the protocol and IPC mechanism, while Weston is the reference implementation of the compositor (and thus also the display server). "Weston has around 10,000 lines of code and it can be used as the base for a compositor for mobile or embedded systems," according to Kristian, who ran Weston on his laptop during his FOSDEM presentation. However, existing X compositors such as KWin and Mutter can also be modified to become a Wayland compositor.
Kristian promises that the 0.85 release is going to be protocol and API
stable. The point of the release is that developers can begin experimenting
with Wayland and porting their toolkits and applications. While it used to
be that you had to compile a special KMS "pageflip" kernel, a patched Mesa
branch, and Kristian's own EGL library to be able to run Wayland on your
system, now everything should be upstream. Kristian hopes to put out a Wayland 1.0 release in late 2012 or early 2013. So while Wayland will not replace X overnight, we will likely see the first mainstream use of the new window system architecture in 2013.
Comments (38 posted)
This year saw a new addition to FOSDEM in Brussels - "legal issues"
gained its own
DevRoom and joined the long list of topics presented and discussed during the
conference. I was privileged to be asked to speak at the track, and when
Bradley Kuhn asked me what I'd like to speak about, my initial reaction was
the theme: "how much I hate having to care about the law".
Developers hate the law. Or, to be more precise, I hate the law. At least, whenever I'm working on a software project. In my opinion, every time a software developer has to stop and think about the law, that's a bug. Either he wants to do something that he shouldn't be doing, or the law is getting in the way of him doing something which it would be reasonable for him to do.
In case it's not obvious yet, I am not a lawyer, so nothing in here should be taken as legal advice.
But, over the years, while working as a GIMP developer, as community and product manager of the OpenWengo project, or as an independent consultant, I have had to stop and consider the potential impact of the law on what I was doing. And every time that I have sought legal advice on the issues, the result has been unsatisfactory. Here are a few examples from my personal experience.
In the late 1990s and early 2000s, the licensing of plug-ins was a hot topic in the GIMP community that came up repeatedly. Does the GPL allow people to develop proprietary GIMP plug-ins? It was certainly the intention of the developers active at the time to allow proprietary plug-ins. To explain why this question is even interesting, some technical details on GIMP plug-ins are useful.
GIMP plug-ins run in a separate process from the GIMP core. They link to libgimp, a library that was licensed under the LGPL in 1997, which is a wrapper around the PDB, the Procedural Database. The Procedural Database is how plug-ins communicate with the core GPL application: data is sent via shared memory, with the PDB defining arguments and return values from plug-in functions. Plug-ins can call core functions, again via the PDB, to get image data from a drawable (a GIMP object representing anything you can draw on), or for any of a range of core functions.
But some argued that since GIMP plug-ins could only work with the GIMP core, that they were really derivative works, and were thus covered by the GPL. When we asked for definitive legal advice on the issue, the answer we got back was not entirely satisfying.
Ugh. So to clarify the situation, the GIMP developers added a license clarification to the source code, alongside the GNU GPL COPYING file, stating that we considered plug-ins to be "mere aggregation", and not a "derivative work". This may not be legally rigorous, but it was enough to allow us to forget about the issue and get back to coding.
Distributing GIMP plug-ins
That is, until it was pointed out
by Debian that GIMP was shipping some GPL-incompatible code in plug-ins
that we were distributing in the gimp .tar.gz. This issue ended up
release of GIMP 1.2.4 for over a year. So the question arose: can we
include non-GPL plug-ins in a GIMP package? The answer was, you guessed it,
"maybe". At least, that's as far as we could tell by discussing it in a
Debian bug report and on gimp-developer.
While GIMP plug-ins might be a "mere aggregation" with the GIMP core,
apparently, there was an expectation that all the plug-ins which were
shipped with GIMP would be GPL compatible. Over the years, we had included
a number of code snippets from other open source projects which were
licensed under other licenses. One of the projects was SIOD, Scheme in one
Defun, a Scheme implementation by George Carrette. SIOD was the back-end of
Script-fu, GIMP's scripting plug-in, at the time.
SIOD had a
very liberal license which included an "advertising clause", requiring
that "the permission notice appear in supporting documentation", a clause
which was widely argued to be incompatible with the GPL at that time. It
turned out that there were about a dozen occurrences of this advertising
clause in the code for plug-ins at the time.
Since Debian wouldn't ship GIMP with these plug-ins unless we made the
licensing consistent, and since some GIMP developers also expected us to ship only GPL-compatible plug-ins, this was a major problem.
So I and others spent a month in 2003 chasing down authors at five-year-old email addresses, removing or rewriting code snippets when we could do so, and on occasion removing offending copyright headers when we decided that the amount of code covered by the problematic licenses was so small that it was not a problem. I am sure I would have preferred to do other things with my time - and I was certainly not as methodical as Gervase Markham when he was relicensing Mozilla code - a process that took over 4 years. Were we legally rigorous? Maybe.
My next example is audio codecs from the OpenWengo project (site now
dead). Some codecs like G729 were patent encumbered. In addition, widely
used reference implementations of codecs like iLBC were included as sample
code in standards documents and were only available under GPL-incompatible and non-free software licenses.
The way we tried to get around this problem was to encapsulate these
codecs in a standard API, and install them in a common directory. At
start-up, we scanned the directory, and used dlopen() to open the
libraries present there. We could then call the standard function to
register the codecs and pass data streams for decoding and encoding.
A question arose for some of the non-GPL codecs: could we do something in
the start-up process to make it easier for the user to obtain these codecs?
One possibility would be to present the user with a dialog, as part of the
first application start-up, which fetched the proprietary codecs and
installed them in the common directory. But, would that be against the spirit of the GPL, and have us running the risk of being called a GPL violator by one of our users?
The answer, apparently, is "maybe".
By divorcing the loading of the plug-in at run-time from the
distribution of the plug-in, we were probably absolving ourselves of any GPL obligations. At least, that is the legal advice Wengo obtained at the time. However, whether making it easier to get the codecs at start-up was the legal equivalent to including the codecs in the package was, at best, ambiguous.
Last year, I was asked an interesting question by a client: Can I include Hunspell and the Czech dictionary in a proprietary application?
The answer, of course, is "maybe".
Hunspell changed its license in 2006, to the MPL/GPL/LGPL tri-license to enable
inclusion in Mozilla. It is used as the spell-checker for
OpenOffice.org/LibreOffice as well as Mozilla Thunderbird and Firefox. Of
course, the tri-license enables the embedding of Hunspell in proprietary
applications. So where's the problem?
It turns out that some Hunspell dictionaries, including Catalan, Danish and
Czech, are licensed under
GPL only. It's a bit curious why anyone would use the GPL, a license intended for use with source code, for a dictionary. Regardless, the question is: does including Hunspell in another application, with a GPL dictionary, constitute a derivative work, or an aggregation?
At one level, the question is whether it even makes sense for a dictionary to have a license. Thanks to Feist v Rural, copyright only applies to a "work" if there is significant creativity involved in it. If a Hunspell dictionary were just a list of words in alphabetical order, then at least in the US, copyright would not apply. However, in other jurisdictions, there are laws covering the constitution of databases, and some form of protection is afforded to the author. In addition, a Hunspell dictionary isn't really just a list of words - it also contains grammatical and hyphenation information, which involves some creativity. So there's a solid argument that the dictionary is copyrightable.
The second question is whether including the dictionary with Hunspell
and other software makes a derivative work, or an aggregate work. One part
of that is whether the component is useful without the Hunspell
spellchecker. Since it's just a dictionary, it can be used with other
spell-checkers, including aspell. Hunspell could easily use another Czech
dictionary released under another license. So, since the two bits are
easily dissociable, Hunspell with the dictionary probably makes an
aggregate work, and the license of the dictionary does not impose any
requirements on other software shipped with it. This is the advice
that the OpenOffice.org project received from the FSF licensing list, and
so the GPL dictionaries are included in the LibreOffice and OpenOffice.org distributions.
Mozilla, on the other hand, do not ship any GPL dictionaries. When I asked Gervase why, he answered:
It is arguable whether the GPL would cross the
dictionary->application boundary. But again, we prefer to respect people's
intentions, and it seems to us that if someone licenses their dictionary
GPL-only, then they probably only want it used in GPL apps.
In other words, the intent of the author trumps all in this case, for Mozilla.
So - maybe.
Conversations with lawyers
The problem for developers is that conversations with lawyers typically end up with "maybe". We're looking for what we may and may not do, and often, in the absence of jurisprudence, lawyers deal in shades of gray. All too often, the end result for developers is to avoid any risk.
A friend once told me a story of a meeting with her boss and "legal",
where she came out of the meeting disappointed. When her boss asked her
what was wrong, she said "the lawyers told us we can't do what we'd like
to." To which he replied, "No, they just told us the risks
involved. Now we have to decide whether we're prepared to accept those
risks or not." Instead of being the end of the conversation, "maybe"
could be the start of it. Lawyers (and , in the free software world, some
non-lawyers) will help you identify risks. But then it is up to the project to decide whether to take the risks or not.
Comments (17 posted)
Page editor: Jonathan Corbet
Inside this week's LWN.net Weekly Edition
- Security: Tor's obfuscated bridges; New vulnerabilities in cvs, java, kernel, mysql, ...
- Kernel: Android and the kernel mainline; Trusting the hardware too much; ARM big.LITTLE.
- Distributions: A chat with new Fedora project leader Robyn Bergeron; #!, SUSE, Ubuntu, ...
- Development: Radio station management with Airtime; PyPy, LibreOffice, Wayland...
- Announcements: Booktype; patents; GPL enforcement; events.