User: Password:
|
|
Subscribe / Log in / New account

LWN.net Weekly Edition for August 28, 2014

Forking instead of fighting

By Jake Edge
August 27, 2014
LinuxCon North America

Bradley Kuhn is widely known for his GPL-enforcement efforts. He has spoken about them at many different conferences along the way, but his talk at LinuxCon North America in Chicago was on a different tack entirely. Instead of trying to enforce the GPL, he and others routed around a violation of the license by writing code—forking the project rather than fighting the violation.

[Bradley Kuhn]

"Sometimes it makes sense not to enforce the GPL", Kuhn said, which many may find a surprising thing for him to say. He normally talks about violations in the embedded Linux space, since they are the most "prevalent and insidious", but violations come in all shapes and sizes. This one was different than many others, which led to a different kind of resolution.

Kuhn then took a short detour to ensure that the audience was up to speed on the GPL. For the purposes of the talk, there were just a few things that the audience should be willing to agree to—at least for 45 minutes or so. The GPL requires that the source code for a covered work (the "whole work") to be released with the "complete corresponding source" (CCS) for that work. There are differences of opinion about what, exactly, goes into the whole work and he could give an entire talk on just that topic. There is limited guidance from either the laws or courts on what constitutes the whole work, but he believes that will change in his lifetime—we will see a court case that makes a judgment about the reach of "whole work".

There is a fundamental assumption in the software industry, he said, that proprietary software makes more money than free software. The veracity of the assumption is immaterial, as that is the perception which causes companies to try to keep as much of their code proprietary as they can. Developers, on the other hand, would share their code, "all things being equal".

Developers will share code when it is convenient to do so, but even sometimes when it is not so convenient. A Japanese developer once told him about early code-sharing in that country by way of floppies placed on a sushi counter when meeting for lunch. Developers are not all "freedom zealots" like he is, but they tend to err on the side of code-sharing. That is the backdrop for this tale, he said.

The birth of hg-app

In January 2010, a developer noticed that there were no free-software code-sharing sites supporting the Mercurial revision control system. There were a variety of proprietary solutions (e.g. GitHub), but most of those only supported Git. So, the developer scratched an itch and created something called "hg-app" that was released in June 2010 under the MIT license. That immediately led to a flame war about the license, since Mercurial is released under the GPL and hg-app is based on Mercurial (in a GPL sense). Some pointed out that the MIT license is GPL-compatible, thus it might not make any practical difference. But in the end, hg-app's developer switched to the GPLv2+.

Sometime later, the project was renamed to RhodeCode; Mercurial developers also started to contribute to it under the GPL. Some developers were also paid to work on improvements to RhodeCode. This is all pretty standard fare for a free software project, he said, but things were about to change.

The company

The primary author of RhodeCode formed a company, RhodeCode GmbH (which Kuhn said he would refer to as "the company" to avoid confusion with the software of the same name). The company announced a license change and added a 20-user maximum into the Python code for RhodeCode. That led to complaints, threats, and ultimately a patch to remove the 20-user restriction. The company then threatened the author of that patch.

Mercurial is a member of the Software Freedom Conservancy (SFC), which Kuhn is the president of, so SFC got involved in the dispute at that point as a mediator, more or less. Some Mercurial developers and other community members sought aggressive action against the company. But, Kuhn said, SFC's goal was to have a calm conversation about the issue with the company.

That conversation broke down quickly. The company claimed to have 100% of the copyright in RhodeCode, even though patches had been accepted from others under the GPL. There is also the question whether the whole work includes Mercurial itself, which would then also require a GPL release for RhodeCode.

The company cannot revoke the GPL on earlier releases of the code under that license (and, he stressed, it has not disputed that). It is a question of the future copyrights: can those be licensed under non-GPL terms. Kuhn believes the company does have an obligation to release under the GPL going forward, both because of the GPL patches accepted and due to the whole work question.

But the company did not agree. When friendly negotiations break down, he said, you have to look at the options. In most cases, the only option is litigation. For example, in the embedded space, there is typically no code at all, or it is so far from the CCS that it is not useful. So, a lawsuit has to be filed to force the release of the CCS.

But this case is a bit different. The company is not a completely bad actor; in fact, it spent a "long time as a good actor", he said. The reason that this situation exists at all is because the company did a good thing in the past (released its code). It won't be doing that going forward, which is lamentable, but we can take the code that was released and move on, he said.

The company will still be violating the GPL, he and others believe, but a lawsuit to pursue that will take far too long. The famous USL v. BSDi lawsuit essentially shut down development on free BSDs for 18 months, he said. That is actually a fairly short time frame; it could have been much longer.

Fork

Rather than do that, the developers decided to fork the code base and move forward. The original code is under the GPL, but some of the newer additions are as well. Done carefully, some of that newer code could be pulled into the fork.

The company's license is complex and self-contradictory, Kuhn said. The code is split into two parts, with Python and HTML code being licensed under GPLv3 and everything else, including CSS, images, and design, released under a separate proprietary license.

Under GPLv3, that second part could be considered "non-permissive additional terms" that could be removed based on section 7 of GPLv3. But the company would likely fight that interpretation, so to avoid conflict, SFC used a conservative reading of what the license said and followed it, Kuhn said.

SFC decided not to take any action on behalf of the Mercurial project. It had also gotten the ball rolling on the idea of a fork, but any fork would not automatically be a member of SFC. There was a big debate in the membership committee, Kuhn said, about whether to take on the fork as a member. As part of the decision to do so, the committee came up with a set of conditions that would clarify the provenance of the fork's code, so that the risk to SFC and its other member projects was low.

That led to a four-step process that would be followed before releasing the fork. First, find the last version of the code base without the new license. Second, extract useful patches of Python and HTML code from the post-license-change versions. Third, rebrand the project to a new name. And, finally, ensure "beyond reproach" compliance with the license.

The work was largely done by Kuhn, as part of his SFC work, and by Mads Kiilerich, who is a volunteer, with the assistance of a few others. The first step was fairly easy. Using the Mercurial repository for the project, identifying the changeset where the license change was made was straightforward.

Even with a "hyper-conservative reading" of the license, Python and HTML files are still clearly released under GPLv3. Pulling those kinds of changes out of the post-license-change versions was a bit tricky. In many cases, the changesets also touched other kinds of files. They came up with Mercurial commands to pull out what they wanted and Kuhn vetted all of the changes. Any edge cases were "discussed carefully with legal counsel", he said.

Rebranding was rather painful, overall. RhodeCode is the company's name and trademark, so the fork could not use that name except in the usual ways that anyone can (i.e. nominative use). They came up with the name "Kallithea", which is a location on the Greek island of Rhodes. But there was more to it than just renaming the project, as the string rhodecode_ was used throughout the code. While it was probably unnecessary to do so, he said, they wrote 300 lines of sed and Perl to replace all of the uses.

JavaScript and the GPL

In order to be "beyond reproach" in its license compliance, Kallithea needed to ensure that it was providing the CCS for its code. Even if the company was violating GPLv3, that doesn't give the project (which is using a large chunk of code that the company holds copyright to) the right to do so. The biggest problem for providing the CCS turned out to be JavaScript under the GPL.

It is a problem that other projects have, he said, but they may not know it. Typically, you publish a .js file at some URL and it gets downloaded as part of an HTTP request. Under the GPLv3, you have distributed the code at that point, so you must provide the CCS. For RhodeCode/Kallithea, though, there is bunch of JavaScript code from all over, some of it was written for RhodeCode, but lots of it was from elsewhere.

The first problem was tracking down what version of the external code is being used (and what license it is under) so that the license text accompanying Kallithea could be kept up to date. That part was fairly straightforward (if tedious), but the real problem came from "minified" JavaScript. Under the GPL, that is considered to be "object code", so the source JavaScript had to be tracked down to be added to the CCS of Kallithea.

For example, YUI 2.9 is a deprecated Yahoo user interface library written in JavaScript that can be found in many places in minified form. That's fine, since the library is BSD licensed, but it is not fine for a GPL-licensed package to release it that way. It would hypocritical for Kuhn to release code without the CCS, he said, given that he has spent many years fighting for the CCS to various other programs in court. They were eventually able to figure out how to get the source and to build it into the minified version, so the instructions to do so are now part of the license file for Kallithea.

There are some minutia to the GPL, he said, but they are normally easily met. The first release of Kallithea was done on the same day (August 22) as Kuhn's talk. A late-breaking problem that they ran into before the release was the license notification that appeared in the HTML of each page of the interface. RhodeCode has an incorrect one, he said, but getting that right is not necessarily easy. He would rather maintain a single page (like "About") rather than something on each page. It is, he said, the first time he has felt burdened by the GPL—it is something he may try to get Richard Stallman to change down the road.

Kuhn cited three lessons for developers that resulted from this episode. Don't just grab JavaScript from anywhere and incorporate it into a web application, he said. Part of the problem is that Python programmers (in this case) don't really take JavaScript very seriously, which can lead to problems as it did here.

When contributing to a new project, immediately check to see who holds the domain name and trademark—if it is only one person, start talking to an organization like SFC. Kuhn "pre-announced" a new "Conservancy-lite" program that SFC is offering to projects who are just looking for a place to park their domain name and trademark.

Lastly, he suggested that developers keep their own copyrights to their code. The code can be contributed to a project under the same license the project releases its code under. Developers should make it clear they expect the license that the contribution was made under to be upheld.

Meanwhile, Kallithea is an early success. It has released version 0.1, which supports both Mercurial and Git, and anyone can run their own instance. And all of it is developed in the open under GPLv3.

[I would like to thank the Linux Foundation for travel assistance to Chicago for LinuxCon North America.]

Comments (13 posted)

Discourse releases version 1.0

By Nathan Willis
August 27, 2014

Since early 2013, the Discourse project has been developing a modern commenting-and-discussion system intended to be plugged in to a wide variety of web-based content-management systems (CMSes). Although the code has been available for outsiders to experiment with or even use on production sites for quite some time, the project only officially released version 1.0 on August 26.

The project's goals are to develop a free-software web-based discussion system that implements features expected by readers, but which were not available in the commenting systems of existing open-source CMSes. For example, Discourse supports "infinite scrolling," in which additional comments are loaded on the page as the user scrolls downward. There are also features to condense long discussion threads, to allow users to rate individual comments (with "likes"), methods to hide text in a comment (for protecting "spoilers"), to enable comment moderation, and to send users real-time notifications about replies.

[Discourse 1.0]

A lot of attention has also been paid to the actual comment-editing process; the editor pops up as an overlay on top of the page, and it supports several text-markup languages (e.g., HTML, Markdown, BBCode). Drafts of users' comments are automatically saved as they edit, and there are handlers for image upload, emoji, and automatic URL embedding.

Discourse is also designed to integrate with popular outside services. Although it has a built-in login system, it also supports OAuth2 (which is used by Google) as well as logins through several prominent social-media services (e.g., Twitter and Facebook). Similarly, it supports both local user-account avatars and those from remote services. It can be hooked into various commercial content-delivery networks (CDNs) for increased access speed. In addition, Discourse can be deployed in a mailing-list emulation mode, in which all messages on a topic can be relayed to users via email, and email replies are also visible as posts.

By any account, that is a hefty set of features. Even looking only at the fundamentals, Discourse is an attempt to provide a single codebase that fills several distinctly different roles: a full-featured discussion forum (like one might see deployed with phpBB), a commenting system for use on blog posts or individual articles, and even a mailing list (and its accompanying web archive). "Do all the things and do them well," after all, is not the Unix philosophy—often with good reason.

Adoption and deployment

Nevertheless, thus far, Discourse appears to have amassed quite a sizable following. The project has made considerable headway in a relatively short time, in part by working with several high-traffic partner sites (such as Boing Boing and Ubuntu's discussion forum) that provided real-world testing as development continued. During that process, however, the main Discourse site actively warned users against installing Discourse, due to its pre-release nature. Those warnings have vanished with the arrival of the 1.0 release, of course.

But it is probably worth noting that the major "partner" sites that helped Discourse during its pre-release development phase seem to focus on the standalone web-forum deployment scenario. Similarly, the full-forum approach is used by Discourse's own discussion site, meta.discourse.org, and if the discussions on meta are any guide, it seems to be the preferred use case of most Discourse users.

The blog-commenting engine use case can most easily be explored via the official Discourse plugin for Wordpress. Interestingly enough, the WordPress plugin page points visitors to the site How-To Geek in order to see the plugin in action, although How-To Geek's site seems to run an essentially standalone web forum for its discussions. If there are any sites that have deployed Discourse in mailing-list mode, they are not easy to locate.

Hopefully this disparity will change, now that the 1.0 release is out the door and interested parties are being encouraged to take it for a spin. The only officially supported installation method is via a Docker container, although there are guides available for a number of other deployment scenarios. Alternatively, the Discourse project offers a hosting option, the revenue from which goes to support development.

Discourse is built with Ruby on Rails—which, according to the documentation, is the main justification for requiring Docker deployment: correctly configuring the necessary Rails stack otherwise is, evidently, difficult. It also uses PostgreSQL 9.1 (or later) and the Redis storage system. The hardware requirements include a dual-core CPU and 1GB of available RAM (though 2GB are recommended). The code itself is hosted at GitHub, and is released under GPLv2.

A nice touch for sites with existing communities is that Discourse includes several utilities for importing existing user and comment databases from several other forum packages (including vBulletin, phpBB, Drupal, and BBpress). Here again, though, the project's attention seems to be centered squarely on the discussion-forum use case; when a user asked recently about importing messages from Mailman, the answer was that it should not be attempted at all. There are also periodically questions about how to integrate Discourse with other web frameworks, such as those built with PHP; the current state of affairs seems to be that it is possible, but requires considerable work.

That is not to suggest that Discourse is falling short of its goals, of course. Rather, the salient point is that Discourse 1.0 can already deliver a demonstrably beautiful and feature-rich discussion site—but the broader goal of bringing the same level of features and aesthetics to other discussion systems is still an ongoing process. The Discourse interface is undeniably pretty to look at it, and it does an admirable job of presenting posts, activity statistics, and metadata in a clean and easy-to-find manner. But integration with other packages will be a trickier task—especially if Discourse wants to preserve its aesthetics and features when welded to other software.

But world domination is not necessary to declare Discourse a success. There will always be users and sites who prefer other frameworks for one reason or another. Infinite scrolling, for example, can be an obstacle to in-page text searching, while others may simply find Discourse's heavy reliance on JavaScript to be a negative. But, when viewed in conjunction with other recent advances in free-software discussion software—like Mailman's Hyperkitty interface, it is undeniable that the free software community is pushing the bar forward when it comes to conducting its conversations online.

Comments (11 posted)

Visual legerdemain abounds in G'MIC 1.6.0

By Nathan Willis
August 27, 2014

A new stable release of the G'MIC image-processing framework was recently released. Version 1.6.0 adds a number of new commands and filters useful for manipulating image data, as well as changes to the codebase that will hopefully make G'MIC easier to integrate into other applications. Because most users will employ G'MIC scripts written by others—rather than directly altering image files—the integration features are arguably the most important aspect of the program. Recently, the project has been putting extra emphasis on documentation and on working with G'MIC power users to deliver the best possible presentation of G'MIC's capabilities.

David Tschumperlé announced the release of version 1.6.0 in the G'MIC discussion group on Flickr. To the average free-software fan, Flickr might sound like a peculiar venue on which to post a release announcement, but it is home to a large community of G'MIC's most active constituency: photographers. The new release is available through several means. There are source bundles and binaries of the command-line G'MIC tool provided for download through SourceForge.net, the GIMP plugin can be downloaded separately, and a personal package archive (PPA) for Ubuntu is available. The code is distributed under the CeCILL license, which is GPL-compatible.

Learning G'MIC

Most users will likely want to choose the GIMP plugin, since it is easier to get started with. The precompiled plugin itself needs to be installed, which is done by uncompressing the downloaded archive into ~/.gimp-2.x/plug-ins/. The next time GIMP is run, there will be a "G'MIC" menu item, which launches the G'MIC filter-selection window. This window resembles GIMP's other filter-preview windows in the most basic sense: there is a small-scale thumbnail showing a live preview image of what the chosen filter's current settings will do to the image file (or selection), plus a box of sliders, drop-down selectors, and other widgets with which one can fiddle with the settings.

[G'MIC Mighty details filter]

But unlike standalone items in GIMP's "Filters" menu, the G'MIC selector is really a front end for G'MIC's own library of pre-built filters—so the window also includes a lengthy, scrollable list of all of the available choices. The 1.6.0 plugin advertises 395, although that number does not include a large set of effects marked "testing"—the G'MIC site says that there are currently 714 available in total. Plenty of them are variations on a similar theme, of course, but almost every effect one can think of is available: from image-enhancement tools (like local-contrast enhancement to bring out details) to artistic mutations akin to Instagram effects, plus a wide assortment of what one might call purely mathematical filters (Fourier transforms, discrete Laplacian operators, etc.).

The good news is that G'MIC's selector makes it simple to familiarize oneself with what could otherwise become a dizzying assortment of possibilities. The installed filters are grouped by category and the majority of them offer a near-instant preview of the final effect they will produce. Those that do not offer a preview tend to be the ones that require interactive editing, such as the famous "inpainting" filter used to remove objects from a scene—the user has to select the object to remove in the image, so no automatic preview is possible. Each filter also has its author and last update listed in the interface. Some have additional text metadata, such as instructions.

Under the hood, though, even the most complex G'MIC filter is nothing more than a series of commands passed to the G'MIC interpreter. All of the filters seen in the GIMP plugin can be duplicated by running G'MIC commands in the gmic command-line tool. It is quite similar to ImageMagick or GraphicsMagick in this respect; the GIMP plugin merely provides an easy-to-use interface for a suite of predefined "scripts." Although whether G'MIC commands are complex enough to qualify as true scripts is debatable; there are some control structures available, but a lot of limitations—including the fact that only image data can be operated upon.

The project has considerably detailed documentation about the available commands and how they are interpreted, for those interested in digging a bit further. In a nice touch, the documentation's command reference now links to tutorials from each command's definition, for those commands that have a tutorial available.

New tricks

Because the GIMP plugin is merely a front end for the command-line tool (which is the focus of development), G'MIC release notes tend to describe new features in their command-line terms. 1.6.0 is no exception. The release notes highlight a handful of new and changed commands, plus several new filters that have been added to the library. New image-operation commands include -oneminus, which computes (1 - pixelvalue) for each pixel in an image and -mul_channels, which can be used to multiply each channel of an image by a different value. Several existing commands have been improved: the -map command accepts a boundary parameter, the -median command accepts a threshold argument, and the -blend command now accepts an opacity argument between zero and one. Some non-functional cleanups include renaming some commands to better reflect their functionality and adding shortcuts for others.

[G'MIC Various film-emulation presets]

Considerably more commands have been added to those used to manipulate files and interact with the system. G'MIC filters can now create their own custom error messages in GIMP, name output files according to more flexible rules (such as renaming multiple input files to multiple distinct output filenames), and use color terminal output in Bash. The command-line gmic tool now automatically looks for a custom command file at startup (at ~/.gmic on Linux systems).

Although the changes to the command language are of most interest to users who write their own effects, some new pre-built filters have also been added to the default set included in the GIMP plugin. One of these is "Film emulation / Various," which alters the colors of an image to simulate the effects of different real-world film stocks. The main set of film emulations was created with the help of photographer Pat David (who presented some of the work at Libre Graphics Meeting 2014 in Leipzig).

There are about 20 new emulation presets available in this release. These are mostly useful for artistic effect, of course, but as older film stocks like Kodachrome (and the associated developing chemicals) are discontinued, computer simulation becomes the only option to match the old look. Many of the "Various" presets, rather than targeting a specific film stock, implement more generic effects to simulate vintage and aged photographs.

Another new filter is "Mighty details," one of several options that can be used to heighten the clarity of details in an image. Detail enhancement is a convoluted process, the effectiveness of which varies considerably from image to image (and overdoing it can heighten the visibility of unpleasant artifacts). So the name "Mighty details" may not communicate much more than "Freaky details" (another filter), other than suggesting that one will see a less extreme outcome. This is understandable, but can be a source of frustration. Some of G'MIC's filters are better documented on the web than others; there does not seem to be a detailed look at "Mighty details" available so far, but the G'MIC GIMP plugin does include links to other filters' documentation (including "Freaky details").

[G'MIC Montage filter]

On the other hand, David did thoroughly document the creation of another new filter, "Montage," in a blog post. This filter can be used to intelligently rescale a set of images to fill an arbitrary page size. Unlike manually arranging images, it is fast and easy to rearrange the component images in the preview window, so this filter may attract plenty of fans.

Last but not least, David also wrote a post exploring the new "patch-based inpainting" filter. Inpainting, as mentioned earlier, is an intelligent fill operation that removes an object from an image by blending in samples from the surrounding image. The patch-based technique is better for situations where larger or distinct objects are in the surrounding image. Blending with random shapes like clouds and grass is fairly simple; blending with samples taken from human faces or buildings tends not to produce natural-looking results.

From the perspective of a casual GIMP user, G'MIC's greatest strength—the vast array of effects it can produce—is a double-edged sword. Even the pre-built filters that ship with the G'MIC GIMP plugin have the potential to be overwhelming to sort through. But this is a problem that the project has put considerable effort into fixing, with improved documentation accessible from multiple sources and with well-organized tools and settings presented in the user interface. Die-hard image manipulators will likely be pleased with the new commands available for writing custom G'MIC scripts and filters, but for everyone else, the 1.6.0 release provides easy access to a wealth of effects that would otherwise demand considerable time and experimentation with another image-editing tool.

Comments (5 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

  • Security: SELinux on Android; New vulnerabilities in ansible, eglibc, php, python-django, ...
  • Kernel: Power-aware scheduling; Networking update; Kernel.org update.
  • Distributions: A look at elementary OS "Freya", Debian, ...
  • Development: OpenSSH 6.7; Kallithea 0.1; Calibre 2.0; New HIG for GNOME and GTK+; ...
  • Announcements: LF TAB election results, LibreMusicProduction.com, Cluetrain, ...
Next page: Security>>

Copyright © 2014, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds