LWN.net Weekly Edition for October 29, 2009

Community contributions and copyright assignment

Over the course of the last month or so, your editor has been to six conferences on three continents. When engaged in that kind of travel, it is, of course, obligatory to determine which country has the best beer; normally, substantial amounts of research are required. It's also normal to hear what's on one's co-researchers' minds while carrying out this task. This time around, your editor heard grumbles from a surprising number of people, all about the same topic: copyright assignment policies.

In particular, developers are concerned and unhappy about the copyright assignment policy that Canonical has chosen for all of its projects. This agreement [PDF] is a relatively simple read; it fits on a single page. It applies to a long list of projects, including Bazaar, Launchpad, Quickly, Upstart, and Notify-osd; contributions to any of those projects must be made under the terms of this agreement.

So what do contributors agree to? The core term is this:

So Canonical gets outright ownership of the code. In return, Canonical gives the original author rights to do almost anything with that code.

Assigning copyright to Canonical could well be an obstacle for potential contributors, but there are a couple of other terms which make things worse. One of them is this:

There are many free software developers who might balk at giving their code away to somebody who "ordinarily" will make it available under a free license. And the final sentence is even worse; "other license terms" is, of course, euphemistic language for "proprietary terms."

Finally, there is the patent pledge:

This language is likely to be just fine for many developers who have no intention of asserting patents against anybody anyway. But it's worth noting that (1) the patent grant is broad, including anything which might be added to the program (by others) in the future, and (2) there is no "self defense" exception allowing patents to be used to fight off litigation initiated by others. So, to a patent holder, this language is going to look like a unilateral disarmament pledge with unknown (and unknowable) scope. For many companies - even those which are opposed to software patents in general - that requirement may well be enough, on its own, to break the deal.

Contributor agreements abound, of course, though their terms vary widely. One might compare Canonical's agreement with the Free Software Foundation's language, which reads:

Not all developers are big fans of the FSF, but most of them trust it to live up to those particular terms. The FSF agreement makes no mention of patents at all (though GPLv3 is certainly not silent on the subject).

What about other projects? The Apache Software Foundation has an agreement by which the ASF is granted a license (which it promises not to use "in a way that is contrary to the public benefit or inconsistent with its nonprofit status") but the author retains ownership of all code. Sun's contributor agreement [PDF], which now covers MySQL too, gives Sun the right to do anything with the code, but shares joint ownership with the author. An extreme example is the SugarCRM agreement, which appears to transfer not just the author's copyrights, but his or her patents (the actual patents, not a license) as well.

Agreements like Sun's and SugarCRM's are common when dealing with corporate-owned projects; they clearly prioritize control and the ability to take things proprietary over the creation of an independent development community. More community-oriented projects, instead, tend to take a different approach to contributor agreements. Canonical is being criticized in a way that SugarCRM is not, despite the fact that Canonical's agreement appears to be the friendlier of the two. A plausible reason for that difference is that Canonical presents itself as a community-oriented organization, but it is pushing a more corporate-style contributor agreement.

Canonical's policy is especially likely to worry other Linux distributors. They are often happy to contribute to a project controlled by a different distributor, but they do not normally do so under terms which allow the recipient to take the code proprietary. Licenses like the GPL ensure fair dealing between companies; contributor agreements which allow "other license terms" remove any assurance of fair dealing. It is not surprising that some people are uninterested in contributing code under such terms.

The real sticking point, at the moment, appears to be Upstart. Other distributors either have adopted it or are considering doing so; it does appear to be a substantial improvement over the old SYSV init scheme. In the course of adopting Upstart, these distributors are certain to fix problems and make improvements to suit their needs. But they are rather less certain to contribute those changes back under Canonical's terms. In his wanderings, your editor has heard developers talk about possibly forking Upstart. Another developer claimed to be working on a completely new alternative system for system initialization which would take lessons from Upstart, but which would be an independent development. Neither of these outcomes seems optimal.

Your editor sent in a query asking what prevents Canonical from adopting more contributor-friendly terms, but got no answer over the course of a couple of days. Groups requiring copyright assignment often claim that it's necessary for them to be able to take action against copyright infringers. But the projects which have had the most success in that area - the Linux kernel and Busybox, for example - have no copyright assignment policy. The other thing that copyright assignment allows, of course, is a relicensing of the code. The FSF has made use of this privilege to move its projects to GPLv3; companies like MySQL have, instead, used it to ship code under proprietary terms. One might assume that Canonical has no such intent, but the fact that Canonical has explicitly reserved the right to do so is unlikely to make people comfortable.

When developers contribute code to a project, they tend to get intangible rewards in return. So asking them to hand over ownership of the code as well might seem to be pushing things a little too far. Even so, many developers are willing to contribute under such terms. But there are limits, and allowing a competitor to take code proprietary may well be beyond those limits - as are overly-broad patent grants for contributors who are concerned about such things. Companies which demand such rights may find that their community projects are not as successful as they would like.

Comments (45 posted)

Mozilla refactors messaging with Raindrop

Mozilla Labs recently pulled the covers off of Raindrop, a new project that attempts to rethink how messaging software presents information to users. In one sense, Raindrop is designed to function as a "grand unified inbox" aggregating email, instant messaging, and a wide range of site-specific message channels. These channels otherwise exist in complete isolation, which requires users to check multiple applications and web services to collect their incoming communication. But Raindrop also strives to better present the aggregated dispatches and notices, automatically sorting individual conversations from group discussions, and personal messages from automated announcements and updates.

Messages, messages everywhere....

Raindrop's web page says its mission is making it "enjoyable to participate in conversations from people you care about, whether the conversations are in email, on twitter, a friend's blog or as part of a social networking site." To that end, the application abstracts away from the user the work of retrieving messages, notifications, and replies from the various web and email accounts, and presents them together as a unified whole. On top of that, Raindrop attempts to figure out which messages and conversations are most likely to be important to the user, and filters them up to the top of the stack. One of the introduction videos gives the example of sorting personal email to the top of the stack, while putting automatically-generated alerts to the bottom.

Clearly, there are more than just two categories of message (personal and automatic); Raindrop filters list email as well, but pays more attention to threads in which the user is participating. On the microblogging front, Raindrop classifies direct messages and "@" replies above general status notices, and can thread back-and-forth exchanges just like email.

Raindrop's interface is undergoing constant study and redesign, but at present it features a "home" screen with a combined list of all messages, plus links to specialized views for content, including "Direct Messages." "Group Conversations," "Mail Folders," and "Mailing Lists." Raindrop's home screen sorts the newest conversations at the top, and all messages appear as conversation "bubbles" with a preview of their contents. Raindrop threads related messages together, and it flags each conversation with an icon to distinguish between what it believes are person-to-person conversations, group discussions, and announcements or other impersonal messages. At the bottom of the home screen is a summary block for content in which the user is not a direct participant — general Twitter updates, mailing list threads between other people, and so on.

The Raindrop team is making considerable efforts to solicit input and feedback from real-world users in order to adapt its design. The project's "guiding principles" emphasize its user-centric and participatory process. The project has not yet made a pre-packaged release, but has tagged a 0.1 milestone in its source code repository. In keeping with the participatory goal, the designers have issued two previews of interface changes, although neither is yet available to run.

Those interested can install and run the current code on Linux, Mac, and Windows machines, however. Raindrop is a web application, written primarily in Python and JavaScript and using CouchDB for storage, but it contains its own web server, so it can run on a normal desktop machine. Users can check out the Mercurial tree and install either the 0.1 milestone from October 22, or the less stable trunk. Installation instructions are provided on the Mozilla wiki; the notable dependencies include CouchDB version 0.10 or newer and Python 2.5 or 2.6.

After checking out the code, the included script check-raindrop.py will check for specific Python packages (including Twisted, Paisley, and several support packages for dealing with specific services such as Twitter and RSS feeds) and check that CouchDB is configured and running. Once the script reports that everything is satisfied, users must manually create a ~/.raindrop file and populate it with account information for the services they wish to monitor. The current release includes IMAP email accounts, generic RSS feeds, and the popular commercial services Gmail, Twitter, and Skype.

Once Raindrop is configured, users start the service with the included run-raindrop.py script. The first time through, according to the installation guide, this script should be executed as:

    run-raindrop.py sync-messages --max-age=Ndays

which will fetch the previous N days' worth of incoming messages from each account. run-raindrop.py can take several minutes to process each account, so it is best to choose a small value for N. When the import finishes, users can access the Raindrop application from a browser at:

    http://127.0.0.1:5984/raindrop/inflow/index.html

Functionality and ongoing development

At first glance, Raindrop's home screen shows what one would expect from any email client: message threads. More subtle than this automatic threading is Raindrop's attempt to combine all message sources into one "inflow" (as the project calls it). Each conversation category Raindrop presents combines threads from all of the configured accounts; "Group Conversations" contains email and @replies, "Sent" contains outgoing tweets and mail, and so on.

Lead designer Bryan Clark describes this automated sifting of content by message type as one of the key goals of the project. The first iteration of the user interface looks much like a webmail client, but the prototypes posted by the developers indicate that they plan to push the separation of different content types even further, perhaps clustering announcements and other non-personal messages into separate areas of the screen, giving more room to the "important" conversations with a "dashboard"-style layout.

As of today, it is difficult to get a solid feel for how this intelligent processing of messages will work in practice, because there are so few supported services. It is certainly handy to access all of the assorted account inboxes in a single location, but the actual value of merging message sources increases the more sources there are to consider. Users who interact with the same contacts via Twitter, Skype, and email can test the combined-message-threading more rigorously, but for many users, additional services may have to be added to make the user experience diverge significantly from a traditional, single-source web application — perhaps even a service from outside the web itself, such as an SMS gateway.

The Raindrop team solicits input from outside users and developers through user and developer mailing lists, and has posted documentation on the front- and back-end architecture. In addition, the design team working on the user interface and user experience maintains a blog chronicling its work, and posts its design ideas and mock-ups to Flickr. Users and developers are encouraged to send feedback and ideas to both groups.

One interesting feature of Raindrop that should help encourage its open development is that it has Mozilla Bespin's code-editing functionality built-in. At the bottom of each page is a link labeled "Extend" that opens a "Raindrop Extender" code editor window in a new tab. Raindrop is structured to permit easy addition of extensions written in JavaScript, HTML, and CSS.

Raindrop Extender includes two extensions that the user can activate and begin using on his or her local Raindrop installation immediately. One parses each message for URLs and appends a list of the URLs found in each message to the message's preview bubble for easy access. The second performs a similar task for Flickr URLs, but rather than providing a link, it fetches and displays a thumbnail image of the file in the preview.

Rethinking messaging

Several early blog reactions to Raindrop compared it to Google's Wave, but beyond the aggregation of multiple content sources, the two projects have little in common. Wave centers around real-time and collaborative content editing, while Raindrop focuses on filtering messages in a user-centric way. Other projects have attempted to "rethink the inbox" over the years — much of the Getting Things Done (GTD) craze took aim at message processing, for example, and, although it has never attracted critical mass, a big part of the Chandler project's goal was to merge calendar, email, and to-do into a unified stream. A horde of Firefox and Thunderbird extensions exist to try and combine the multitude of single-site message streams into a single application.

Raindrop has certainly found a problem in need of a solution; even with open standards and open protocols, online communication today has splintered into more and more messaging services that are blindly ignorant of each other — consider how many ostensibly "VoIP services" also provide instant messaging functionality as if users were in need of another IM account. But above all else, the Raindrop design team seems to understand that to a user, an incoming message is important or unimportant based on who sent it and what it says — it does not matter which from which site or protocol the message originated. Its "grand unified inbox" does not stop at un-splintering all of the incoming content, it actually tries to make useful sense out of it.

The current 0.1 milestone of Raindrop is clearly just the first drop in the bucket, but it deserves kudos for tackling this complicated issue — and for doing it in a completely open way. Clark and the other Raindrop developers at Mozilla Messaging are the team that developed Thunderbird; whether Raindrop's concepts remain limited to a web application, become integrated into Thunderbird and other stand-alone clients, or some combination of both remains to be seen. Wherever it goes, though, Raindrop will be an interesting experiment to watch.

Comments (3 posted)

A report from JLS

Like a number of Asian countries, Japan has, in the past, had a reputation for being a great consumer of Linux: Japanese companies have been happy to make use of it when it suited them, but contributions back to Linux have been relatively scarce. The situation has changed over the years, and Japanese developers are now a significant part of our community. We get a lot of code from Japan, and, increasingly, ideas and leadership as well. Japan is pulling its weight, and, possibly, more than that.

Given this context, it makes sense that the 2009 Kernel Summit went to Tokyo. Japan (and the Linux Foundation) did a great job of hosting this high-profile event; some developers were heard to suggest that the summit should be held there every year. But one also should not overlook the significance of the first Japan Linux Symposium which followed the Summit. JLS 2009 is the beginning of what is intended to be an annual, world-class Linux gathering. Your editor's impression is that this event has gotten off to a good start.

The JLS program featured a long list of developers from Asia and beyond. Your editor will summarize a few of the talks here; others will be covered separately.

Ruby

Arguably, one important prerequisite to the creation of a thriving development community is the existence of local rock-star programmers who can serve as an inspiration to others. Japan certainly has one of those in the form of Yukihiro Matsumoto, best known as the creator of the Ruby language. He is known in Japan as an inspirational speaker, though, your editor fears, some of that inspiration was lost as the simultaneous translators worked flat-out to keep up with his fast-paced talk. Certainly the audience was clearly thrilled to have an opportunity to hear him speak.

His talk, held during the first-day keynote block, was aimed at a non-technical audience; it thus offered relatively little that would be new to LWN readers. "Matz" talked about the Unix philosophy and how it suits his way of working - "simplicity," "extensibility," and "programmability" were the keywords here. Open source was a good thing for him as well; it allowed him to play with (and learn from) a wide variety of software and set the stage for the development of Ruby. The posting of Ruby itself was a big surprise - he had bug reports and patches within hours of the creation of the mailing list. Without the open source community, Ruby would never have reached its current level of functionality or adoption.

Amusingly, Matsumoto-san noted that his objective at the outset was to create an object-oriented Perl. He did not know about Python at the time; had he stumbled across that language earlier, things might have gone much differently.

TOMOYO Linux

Security modules are among the most difficult types of code to merge into the kernel. Pathname-based access control techniques are a hard sell even by the standards of security code in general; one need only look at the fate of AppArmor to see how difficult it can be. So a first-time contributor who merges a security module using pathname-based techniques has accomplished something notable. That contributor is Toshiharu Harada, who saw TOMOYO Linux merged into 2.6.30, two years after its initial posting. Harada-san talked about his experience in a session at JLS.

Getting started with kernel development is hard, despite the existence of a lot of good documents on how to go about it. We still make mistakes. The biggest problems are simple human nature and the fact that we don't like reading documentation; these, he said, are difficult issues to patch. There is too much stuff under the kernel's documentation directory, and we would much rather go and code something than read. But there are things we should look at; he suggested HOWTO, SubmitChecklist, and CodingStyle. He also liked Linus's ManagementStyle document, which contains such un-Japanese advice as:

Linux kernel documentation, Harada-san noted, is tremendously practical.

His advice - derived from the many mistakes made in the process of getting TOMOYO Linux merged - was equally practical. Send patches, not just URLs. Stick to the coding style. Keep your patch series bisectable. Use existing data structures and APIs in the kernel. Be sure to send copies to the right people. Don't ask others to make changes for you - just make them. Try not to waste reviewers' time. And so on.

There are, he noted, lots of kernel developers who are willing to help those trying to figure out the system. Arguably the real lesson from the talk - never explicitly stated - was related to that: Harada-san was able to overcome obstacles and get his code into the kernel because he listened to the people who were trying to help him. If more developers would adopt that approach, we would have fewer failed attempts to engage with the development process.

On Japanese participation

Satoru Ueda is one of the strongest proponents of the use of - and contributions to - Linux within Sony. His efforts once led to a Sony vice-CEO asking him whether he was actually working for Panasonic, which seemed to be the beneficiary of his efforts. Ueda-san used his JLS talk to examine why Japanese developers often hesitate to work with the development community.

Is Japanese non-participation, he asked, a cultural problem? In part it might be. In general, he says, Japanese people tend to respond to strangers with fear, worrying about what unknown people might do to them. Westerners, instead, tend to be much more aware that strangers, while potentially dangerous, can also bring good things. That makes them more open to things like working in development communities.

That said, Japanese attitudes in general - and toward the open source community in particular - are changing. Japanese hesitation in this area is not really a cultural issue, set in stone; instead, getting past it is just a matter of adaptation.

Economics is also an important issue. Japanese executives are starting to see the economic advantages of open source software, and that is making them fairly excited about being a part of it. Mid-level managers are decidedly less enthusiastic; they fear that community participation could erode their power and influence within the company. They also feel stronger than the community and feel a need to keep core development competence within the company. Developers, too, are hesitant. The high visibility afforded by community participation is relatively unhelpful in Japan, where labor mobility is quite low. They fear that managers may not understand what they do, they worry about working in an unfamiliar language, and they fear being flamed in public.

Again, things seem to be getting better. Labor mobility is on the rise in Japan, and some managers are beginning to figure things out. And there are a lot of open-source developers in Japan. So, in the end, Ueda-san is optimistic about the future of Japanese participation in the development community.

Looking at how the Japan Linux Symposium went, your editor would be inclined to agree with that optimism. The event was well attended by highly-engaged developers from Japan and beyond. Questions during the talks were subdued in the Japanese fashion, but the hallway discussions were lively. JLS mirrors a growing and enthusiastic development community. This event is off to a good start; if it can retain its success next year in the absence of the Kernel Summit, it may well become one of the definitive conferences worldwide.

Comments (8 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

• Security: "Evil Maid" attack against disk encryption; New vulnerabilities in firefox, kernel, poppler, sahana,...
• Kernel: Generic receive offload; A Btrfs update; Transparent hugepages.
• Distributions: What is Fedora?; Ubuntu 9.10 RC; Fedora's target audience; ATI R600/700 3D Support In Fedora 12; openSUSE Board Election; openSUSE 11.2 Maintenance; Ubuntu Technical Board meeting.
• Development: FatELF: universal binaries for Linux, Symbian releases microkernel, openSUSE Medical, new versions of MySQL, lighttpd, nginx, Tryton, GNOME, X11, SQL-Ledger, Wine, SeaMonkey, LLVM, Python, GIT.
• Announcements: EFF hall of shame, FSFE on Oracle/Sun, l2ork, White House uses Drupal, Red Hat/EnterpriseDB partnership, Sequoia to release voting software, DoD memo on open-source, TILE-Gx processor, Web 2.0 coverage, CeBIT cfp, Video hackfest, PGconf JP.

Next page: Security>>