User: Password:
Subscribe / Log in / New account Weekly Edition for March 29, 2012

ZeroMQ and Crossroads I/O: Forking over trademarks

By Jake Edge
March 28, 2012

Trademarks and free software projects have something of a troubled history. On one hand, a trademark can protect the project from others using the name but offering sub-standard—or malicious—versions of the software. On the other, whoever holds the trademark also holds much of the power over the project as a whole. That can lead to conflicts, which is what we are seeing in a recent fork of the ZeroMQ (aka 0MQ or ØMQ) project.

ZeroMQ is a high-performance asynchronous messaging system that is based on BSD sockets. Two of the developers working on the project, Martin Sustrik and Martin Lucina, wrote about ZeroMQ for LWN in January 2010. Sustrik is the chief architect of ZeroMQ and, up until recently, served as the de facto "benevolent dictator" for the project, while Lucina has been working on the project since late 2009. On March 15, the two of them announced a fork of the LGPLv3-licensed ZeroMQ code as Crossroads I/O version 1.0.0.

The ZeroMQ trademarks

There were two main reasons cited for the fork in that announcement, but based on an email exchange with Sustrik and Lucina, the trademark issue is clearly paramount, at least for Sustrik. The trademark is held by iMatix Corporation which funded the early development of ZeroMQ. That trademark is governed by policy that essentially gives iMatix full control over how the names (ZeroMQ, 0MQ, ØMQ, and zmq) can be used.

That worries Sustrik for a couple of reasons. He believes that the underlying protocols for ZeroMQ should be standardized via the Internet Engineering Task Force (IETF), but that is not something that is easily done using volunteers:

Consider just the trivial case of standardisation of underlying protocols in IETF. The protocol developers should meet at IETF meetings, with average costs being some $4000 per developer and year. It's definitely not a volunteer activity. Same need for funding applies to large-scale testing, hardware implementation etc.

His solution to the funding problem is to allow companies to make money by releasing modified versions, plugins, and extensions that can use the ZeroMQ name, which is something he says the current policy does not allow:

Here the trademark policy comes into the picture. People can technically do this stuff but cannot link its name back to 0MQ. You cannot create "Foobar Enterprise 0MQ" in the way that Red Hat have created "RH Enterprise *Linux*". You cannot write an extension and release it as "0MQ toaster control unit". Etc.

Given that trademarks are at the center of the disagreement, it is ironic that the trademark policy was created at Sustrik's request in May 2011. In a message on the zeromq-dev mailing list, iMatix CEO Pieter Hintjens noted that he created a policy and asked for feedback. There were no major complaints about the policy in that thread, but a suggestion from Sustrik that the trademarks be handed to a foundation (like Software in the Public Interest, which holds the Debian trademark) was not something that Hintjens was interested in:

To be honest, the trademarks represent non-trivial value to iMatix and it would be hard to literally give them away. This is not really an option, though it's one I've considered. There would have to be compelling reasons (e.g. real dysfunction that puts the community at risk), and it'd have to include the domain names.

But Sustrik is concerned that some day the trademarks could fall into "hostile" hands, which could essentially take control of various ZeroMQ-related projects (like the language bindings for roughly 30 languages) by asserting a different trademark policy. That also may make it difficult for ZeroMQ to attract companies to work on the project: "If I was a commercial entity I would be hesitant about investing in such a project". One of the early goals for Crossroads I/O is to find a "neutral entity" to control the trademark by the end of 2012. Until then, the policy states: "'Crossroads I/O' is trademark of Martin Sustrik. Martin Sustrik grants everyone the right to use the trademark with products related to Crossroads I/O (packages, plugins, extensions, tools and similar)."

Development process changes

Since a trademark owner can decide what gets to be called "ZeroMQ", it can also make fairly sweeping decisions about how the project is governed and released. According to Sustrik, after the 3.1.0 beta release, he was "explicitly prohibited to use [the] ZeroMQ trademark, which gives me no other chance than to fork". That prohibition came in a private message from Hintjens, he said. Around the same time, Hintjens posted a proposal for maintenance of libzmq (the heart of ZeroMQ) that makes no mention of a single maintainer, which is the role that Sustrik thought he had been filling.

In a Google+ posting (which Lucina called an "objective writeup"), Ian Barber pointed to a problem that may have been part of what led to the new development process: "Martin Sustrik was effectively a bottleneck for getting improvements into core, and was clearly being pulled multiple directions on features and functionality in the project".

As might be guessed, Hintjens has a somewhat different view from that of Sustrik and Lucina. In an email exchange, he noted that Sustrik was "in danger of burning out", which is part of what led to the changes. In addition, he doesn't see the trademark policy as a problem:

Read that "hostile trademark policy". What it really says is, "iMatix will use its trademarks to ensure that only work done by the 0MQ community can call itself 0MQ". Martin Sustrik asked me to draft it, I did, he liked it, and we used it.

According to Hintjens, Sustrik and Lucina "happily ignored about a year of consensus on process, and unilaterally decided to take back control over stable releases" when they made the 3.1 release. That didn't sit well with some of the other contributors to ZeroMQ, which is what led to codifying the process and, ultimately, the fork.

The new process (scroll down to "Contributing to libzmq") is more wide open than that of most free software development projects. Anyone can fork the Github repository, make changes, and request a pull into the mainline. The maintainers are only meant to enforce process, ensure that the test cases pass, and are not allowed to impose their opinions on the quality of the code or feature. If others in the community are not happy with a particular patch, they are supposed to make another patch that fixes or reverts it—the maintainers are just there to apply it. The underlying assumption is that the community members will find and fix problems quickly by getting those changes in front of them sooner.

Barber calls the process "incredibly open", in fact, more open than he is comfortable with, but it has worked well so far:

That said, there hasn't been a stable release rolled under the current process yet and whether it'll be a usable long-term model remains to be seen - but the plus side is that if it does work, it'll definitely scale. It is, in a large way, an experiment, to draw in more code, and more ideas and I respect the thinking behind that.

Crossroads I/O is going to use the more common "benevolent dictator" model, with Sustrik in that role. Barber says that makes sense, because it is a proven model for open source development, but he is concerned that Crossroads will hit the same problems that ZeroMQ ran into. He also notes that while Sustrik and Lucina have both contributed a great deal to ZeroMQ, they are far from the only contributors to the project. It's not clear, at least yet, if others from ZeroMQ will start working on Crossroads I/O, nor whether the language binding authors will support both projects. That leaves users in a bit of a quandary, he said.

In any case, the new process installed by Hintjens is something of an experiment: "We will try it for a while and if it proves broken, we will fix the breakage and continue to improve it." But that experiment, coupled with what Sustrik sees as a restrictive trademark policy, is enough to cause the fork. What remains to be seen is if Sustrik and Lucina can build enough of a community to continue with their plans.

It is, at least in some sense, a friendly fork. Hintjens greeted the Crossroads I/O announcement with: "Congratulations on this, guys. It's nice to see the LGPL in action." Based on the emails from Hintjens, Sustrik, and Lucina, there was certainly some initial hostility and hurt feelings because of the fork, but, in the end, both sides seem to wish the other well. The two code bases share the same license, so there is no reason that patches cannot flow between them.

Hintjens noted that some may criticize the fork because it will split the community, but he strongly defended Crossroads I/O for a number of reasons. For one, he sees it as a place to do more experimental work, while ZeroMQ focuses on stability. He is also interested in seeing which of the two development models works out better over the long run. It will also provide a choice of free solutions for users, he said.

This fork, like most, comes out of a difference in philosophies between two sub-groups of the project. Unlike other forks, though, it doesn't come about because of license disagreements. It is interesting for another reason, however, as it serves as a reminder that a trademark holder can exercise a great deal of control over a project. Since the owner of the trademark can effectively decide what gets to be called by that name, they can override the wishes of longtime developers.

From all appearances, much of the ZeroMQ community is not upset with the changes or the fork. But most also seem to be keeping an eye on what Crossroads I/O is up to. If Sustrik and Lucina can make the benevolent dictator model produce a better messaging library than the more open ZeroMQ model does, there may be a shift toward Crossroads. By taking the trademark question off the table with a more liberal policy, Crossroads may attract some of the companies that Sustrik worries are put off by the ZeroMQ policy. That remains to be seen, of course, but what we do see is yet another example of the problems inherent in the coexistence of trademarks and free software.

Comments (9 posted)

Fedora release naming "is a" bit contentious

March 28, 2012

This article was contributed by Nathan Willis

Fedora's advisory board is debating changing its long-followed tradition of selecting a code name for each new release according to a peculiar formula. Although the code names have never been an integral part of the distribution's marketing plan, the impending release of Fedora 17, with its crowd-selected moniker "Beefy Miracle", has stirred up several critics — some to take offense at the name itself, some who find it a burden to explain to outsiders, and some who simply want to do away with release code names altogether.

Fedora's release code names have a history pre-dating the project itself; the tradition started at Red Hat, which assigned a code name to each Red Hat desktop release according to a peculiar pattern wherein each pair of consecutive code names shared some relationship, but that relationship differed for every pair of releases. In those days, however, the connection between each new name and its predecessor remained a secret; the code name originated behind closed doors and when the new release landed, figuring out the link was part of the game.

That all changed when Fedora picked up the code name mantle, and instituted a public procedure for suggesting, vetting, and voting on each new release name. The project now takes suggestions on its wiki for each new development cycle, where submitters list the connection between the previous code name and their proposed follow-up. Red hat's legal department and the Fedora Advisory Board whittle down the options to a manageable number, and voting takes place on the Fedora project site.

The miraculous

Code name selection has not always been a smooth process, but no name was quite as divisive as "Beefy Miracle," which was first proposed as a code name for Fedora 16, and which lost out narrowly to the eventual winner "Verne" — despite a public marketing campaign run on the behalf of "Beefy Miracle". The name then resurfaced as a proposal for Fedora 17, and this time won the poll. Fedora 17 is slated for release on May 8.

Even during the Fedora 16 voting cycle, "Beefy Miracle" was a controversial choice, partly for its nonsensical nature, but also because its connection to the previous code name "Lovelock" stretched the rules. The official guidelines state that each pair of code names must share an "is a" relationship (e.g., Laughlin is a city in Nevada, and Lovelock is a city in Nevada). The submitted connection between "Lovelock" and "Beefy Miracle" was that both strings would eventually produce the number five when fed through an iterative formula. The code name lost out to "Verne," of course, but the submitted connection for the Fedora 17 release name was also tenuous: that both "Verne" and "Beefy Miracle" had been proposed as possible code names for Fedora 16.

In October 2011, after the voting, there were several critics. Michael Schwendt called it impossible to explain to users; Christoph Wickert said it had no connection to the actual product, and another user named Roger lamented that it did not signify anything "important." But the most extreme reaction came from Bob Jensen, who stated his desire to end the use of code names altogether.

The tedious

Jensen's 2011 call to end code names outright was terse and isolated to his sole email, but more users echoed the same sentiment as the planning cycle for Fedora 18 got underway in March. Stephen Smoogen called for an end to the naming cycle or changing to a different format in a message to the Fedora advisory-board list, saying "I am not sure what 'naming' does for us anymore. It is way too early in a release cycle to 'produce excitement'. [And if we can change out init systems, filesystem layout, we can surely examine whether naming does much or if there is something new we want to try.]"

In the resulting discussion, many of those who favor abolishing code names seemed to see no value in the actual names themselves. Seth Vidal, for example, said that selecting names consumes time and effort, but that no one remembers them. In a separate message, he said that release numbers were more useful because they enable one to quickly figure out how old a particular release is by counting backwards, which Fedora's unordered code names do not.

Fedora project leader Robyn Bergeron countered that the code names are incorporated into wallpapers, other design elements, and marketing materials, that participants in the naming process enjoy it, and further contended that release numbers are only "memorable" because people memorize how to count when they are children. Máirín Duffy (who leads the Fedora design team) said that the designers rarely find the code names inspirational, and although she was not in favor of abandoning them entirely (saying release numbers alone would be "cold"), she advocated selecting an consistent, ongoing theme like many other distributions use. "The current naming system usually results in awkward names that require lengthy explanation to those not involved."

Others pointed out additional value in having code names. Jason Brooks observed that when looking for help, code names make for more useful search terms than do numbers. Zoltan Hopper likened naming releases to naming children, and said each release is the creation of the community and thus ought to have a personality or identity.

Nevertheless, many of those who stood up for preserving release code names found room for improvement in the process itself. Clint Savage (who defended the use of code names as a part of Fedora's traditions and pointed to the value of non-artwork marketing such as "Beefy Miracle"-themed hot dog roasts), argued that the value of the code names outweighed the work involved in the process of selecting them, and that "maybe the problem is just a matter of streamlining."

Hopper, like Duffy, suggested picking a single theme. Choosing a theme wisely, such as scientists or inventors, he said, would also assist in marketing the distribution by lining up with Fedora's efforts to stay cutting-edge. Replacing the current process with a new one was also suggested by Matthias Clasen, who said "the naming thing started as a fun game, then it got 'standardized', and now it is just one more process that has stopped to be either fun or useful. [...] Time to reevaluate and come up with something fresh and fun that we can do for each release."

Selecting a code name that offends no one

A distinct (and arguably more serious) issue raised on the advisory-board list is the difficulty of choosing a code name that does not offend the beliefs or sensibilities of anyone. That tricky subject was raised first in a non-public ticket filed on the Advisory Board's issue tracker. The bug filer, Rajesh Ranjan, subsequently agreed to make the ticket's comment thread public by posting it to the advisory-board list.

The specific problem Ranjan raises is that the name "Beefy Miracle" is "is having a huge negative cultural, social, and political connotation with respect to India and several religions of India and [the] world." The cow is considered sacred in Hinduism, he said, so the term "beefy" is offensive because it refers to food made from beef, and it may be offensive to members of other religious groups as well as secular people from the Indian subcontinent. One commenter replied that as an international, non-religious and non-political project, Fedora should not seek to "appease" any specific belief systems — and that any code name chosen will inevitably offend someone.

That led others on the list to suggest picking a non-offensive code name "theme" — although upon further debate, finding a neutral theme is not so simple. Mario Juliano Grande Balletta suggested astronomy terms, but Josh Boyer pointed out that many astronomical object are named for potentially offensive things like the Roman god of war, and that any names of deities can be offensive to atheists. Duffy suggested coffee drinks, but Richard Fontana observed that some religions find caffeine offensive. Fontana also pointed out that the suggestion of famous scientists will undoubtedly skew towards men for historical reasons, while Ranjan noted that even numbers themselves can have negative cultural connotations.

Of course, picking names that have positive connotations everywhere on the globe is no simple task; consider the Mer project's debate in October 2011, which produced gems like "Meer" (which sounds diminutive in English), "MerDE" (which is unflattering in French), "Mermade" (which looks like a typo), and "Mermer" (which sounds difficult to understand on a phone). Given the impossibility of pleasing all of the people all of the time, talk turned to practical measures that the project could take to weed out offensive names during the code name selection process. Ranjan proposed asking the translation mailing list to look for offensive terms, since it encompasses a wide cultural circle. Nicu Buculei said that sounded like too much work, and user John Rose (aka inode0) speculated that it would be too hard to get volunteers to tackle the necessary code name vetting.

Practical steps

Bergeron acknowledged the importance of avoiding offensive terms, but expressed her faith in the Fedora community to catch and call attention to such offensive terms in code names and elsewhere:

I believe that 99.999% of the time, if or when an egregiously offensive *anything* comes up, people speak up. We've seen this in the past with a small handful of issues, and the community has always taken the steps to assess and sometimes correct those issues, case-by-case, through discussion and reasonable judgement.

Nevertheless, she advocated forwarding the code name suggestion list to the the translators for review, at the same time that it is sent to Red Hat's legal office for approval.

As to abandoning code names altogether, there has yet to be a real consensus on the board. Several people suggested adding an option to the Fedora 18 ballot to drop code names in future releases, while others felt that that question should be submitted to a separate vote. The window for suggesting code names for Fedora 18 is now closed, and the board has until March 30 to perform its own analysis of the suggestions. Voting will commence April 6; thus whichever route the board decides to take on the no-more-code-names question, the world will see it soon enough.

Fedora would not be the only distribution to make releases without a code name if it does choose that route; openSUSE, Slackware, and many others work without them, too. On the other hand, code names are an accepted practice — although most others do use a set theme. Debian uses Toy Story characters, Sugar uses fruit. A dimension not raised on the advisory board thread is that many projects stick to an alphabetic sequence for their code names, which makes them easier to sort into the correct order. That list includes Android's desserts, Ubuntu's adjective animals, and even Maemo's trade-winds. Plenty of other open source projects also use release names, whether they are chosen to communicate a message or are fully tongue-in-cheek. Perhaps Fedora's experience with "Beefy Miracle" will prompt a change of pace, but whichever direction the project heads from here, it will at least have made a conscious decision about what role — if any — its code names ought to play in its overall message.

Comments (66 posted)

A turning point for GNU libc

By Jonathan Corbet
March 28, 2012
The kernel may be the core of a Linux system, but neither users nor applications deal with the kernel directly. Instead, almost all interactions with the kernel are moderated through the C library, which is charged with providing a standards-compliant interface to the kernel's functionality. There are a number of C library implementations available, but, outside of the embedded sphere, most Linux systems use the GNU C library, often just called "glibc." The development project behind glibc has a long and interesting history which took a new turn with the dissolution of its steering committee on March 26.

In its early days, the GNU project was forced to focus on a small number of absolutely crucial projects; that is why the first program released under the GNU umbrella was Emacs. Once the core was in place, though, the developers realized they would need a few other components to build their new system; a C library featured prominently on that list. So, back in 1987, Roland McGrath started development on the GNU C library; by 1988, it was seen as being sufficiently far along that systems could be built on top of it.

By the time the Linux kernel came around in 1991, GNU libc was, like many other GNU components, the obvious choice for those working to build the first distributions. But, while GNU libc was a good starting point, it quickly became clear that (1) it still needed a lot of work, and (2) the GNU project's management style was, in typical fashion, making it hard to get that work done and merged. So, early on in the history of Linux, the GNU C library was forked and a Linux-specific version ("Linux libc") was maintained and shipped with Linux distributions.

The GNU project was rather slow to come around to the idea that Linux was the system that they had been trying to build all these years; they continued to develop their C library on their own, despite the fact that Linux was not using it. Early on most of the work came from Roland, but, slowly, other contributors appeared; the first contribution from Ulrich "Uli" Drepper appears to have been made in September, 1995. Early 1997 saw the first glibc 2.0 release; by then, most commits were made by Ulrich (though the code sometimes came from others).

While Linux libc had been adequately maintained, it had not seen vast amounts of new development work. There came a point where it became clear that, in fact, glibc 2 was a better C library in many ways. Distributors of Linux came to the conclusion that the time had come to switch back to the GNU library and, for all practical purposes, simply abandon the Linux libc project. Those of us who lived through the subsequent transition (as seen in the Red Hat 5.0 and Debian 2.0 releases) still tend to shudder; it was a time when programs would not build and bugs abounded. But things eventually settled down and glibc has been the standard C library for most Linux distributions since.

Glibc has continued to evolve up to the present at varying speeds; during most of this time, the bulk of the work has been done by Ulrich Drepper. Of the nearly 19,000 commits found in the project's git repository (which contains changes back to 1995), over 12,000 were made by Ulrich. Never known for a willingness to defer to others, Ulrich quickly became the decision maker for glibc. The project's steering committee was formed by the FSF in 2001 as a way of asserting some control over the project, but, to a great extent, that control remained in Ulrich's hands. In 2008, Roland described the situation this way:

drepper, roland have blanket discretion to commit. (This means that Uli gets to go upside my head after I commit something he doesn't like. It doesn't mean you get to lobby me to commit something that Uli has refused.)

Four other developers (Andreas Jaeger, Jakub Jelinek, Richard Henderson, and Andreas Schwab) had the ability to commit "after approval." But that approval, in the end, had to come from Ulrich.

One need not look far to find complaints about how Ulrich managed the glibc project. He was never one to suffer fools, and, evidently, he saw himself surrounded by a lot of fools. Attempts to get changes into glibc often resulted in serious flaming or simply being ignored. Developers often despaired of getting things done and simply gave up trying. It is easy to criticize Ulrich's attitude at times, but one should also not lose track of all the work he got done. This includes steady improvements to glibc, big jumps in functionality (as with the Native POSIX Thread Library work done with Ingo Molnar) and lots of fixes. Ulrich's work has made all of our systems better.

That work notwithstanding, Ulrich is widely seen as, at best, an unpleasant maintainer who has turned the glibc project into an unwelcoming place. One need not challenge that characterization to point out that glibc actually had a number of problems associated with highly cathedral-style projects: a well-ensconced priesthood, a lack of interest in or awareness of the wider world's needs, etc. At some point, it seemingly became clear to a number of developers associated with glibc that it needed to become a more open project with more than a single point of control. A number of changes have been made to prod it in that direction.

The switch to Git for source code management in May, 2009 was one such change. At that time, the repository was made slightly more open so that non-core maintainers could keep branches there. Ulrich opposed that decision, but Roland responded:

Please don't stand in the way of minor matters the rest of the community wants to improve collaboration, on vague grounds of conservatism or a default of mistrust.

The project started creating separate release branches and putting maintainers other than Ulrich in charge of them. The restricted-access "libc-hackers" mailing list fell into disuse and has been closed down. The use of Fedora as a sort of private testing ground has been halted after things broke one too many times. And a strong effort has been made to encourage more developers and to merge more code. The result is a release history that looks like this:

2.3 2002-10-03 3204
2.4 2006-03-06 5276
2.5 2006-09-29 348
2.6 2007-05-15 357
2.7 2007-10-18 446
2.8 2008-04-12 281
2.9 2008-11-13 293
2.10 2009-05-09 344
2.11 2009-10-30 408
2.12 2010-05-03 389
2.13 2011-01-17 261
2.14 2011-05-31 256
2.15 2011-12-23 582

(The 2.15 date is when the release was tagged; the announcement was, for whatever reason, not sent out for a few months). The 2.15 release suggests that development activity is on the increase after a sustained low point. Indeed, it is at its highest point since semi-regular releases began in 2006. Activity since 2.15 (483 commits, as of this writing) continues to be high by recent historical standards; perhaps more tellingly, only about 25% of those commits came from Ulrich. Given that he accounts for over 60% of the changes from 2.10 to 2.14, that suggests that a significant shift has taken place.

Also significant is the fact that Ulrich left Red Hat in September, 2010; according to his LinkedIn page, he is now "VP, Technology Division at Goldman Sachs." One can easily imagine that his new position might just have requirements that are unrelated to glibc maintenance. So a drop in participation is not entirely surprising; what is significant is that a growing community is more than taking up the slack.

The culmination of these trends came on March 26, when Roland announced that there was no longer any need for a glibc steering committee:

With the recent reinvigoration of volunteer efforts, the community of developers is now able to govern itself. Therefore, the Steering Committee has decided unanimously to dissolve. The direction and policies of the project will now be governed directly by the consensus of the people active in glibc development.

Glibc will remain a project under the GNU umbrella; Roland, along with Joseph Myers and Carlos O'Donell, will be the official GNU maintainers. But that role will give them no special decision-making powers; those powers are meant to be exercised by the project's development community as a whole. David Miller (an active glibc contributor) was quick to celebrate this change:

If you've tried to contribute to glibc in the past and were rudely treated and therefore gave up, I encourage you to come and try again, things are much different now. I promise.

David also hinted that one other possible obstacle to contribution - the requirement to assign copyrights to the Free Software Foundation - is also being worked on, though the form the solution might take is unclear. Other developers are already talking about working to merge at least some of the changes from the EGLIBC fork back into the glibc mainline and reopening old bugs and change requests.

So the GNU C library project has moved into a new phase, one where, for the first time in its long history, it will not be under the control of a single developer. It will be interesting to watch where things go from here. If the project succeeds in building robust community governance and shedding its hostile-to-contributors image, the pace of development could pick up considerably. Given the crucial position occupied by glibc on so many systems, these changes are almost certainly a good thing.

Comments (148 posted)

Page editor: Jonathan Corbet

Inside this week's Weekly Edition

  • Security: OpenOffice and document encryption portability; New vulnerabilities in chromium, freetype, openssl, php, ...
  • Kernel: 3.4 Merge window part 2; IMA appraisal extension; AutoNUMA.
  • Distributions: Debian Edu/Skolelinux: A distribution for education; Scientific Linux, Vector, ...
  • Development: Taskwarrior 2.0; Cairo, Django, Go, glibc, ...
  • Announcements: Collaborative editing in LibreOffice, Enforcing the GPL, Nonprofit open source organizations, software patents, ...
Next page: Security>>

Copyright © 2012, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds