Collaborative GPL enforcement

GPL compliance is probably a more important topic for the embedded Linux community than it is for any other free-software community, Bradley M. Kuhn said to start his Embedded Linux Conference (ELC) talk. After three years of trying, he was glad to be able to give his presentation at this year's ELC. In it, he covered a wide range of information about the GPL itself, compliance with it, how the GPL has been enforced, and where enforcement is heading next.

It turns out that he has spent the bulk of his career enforcing the GPL, first at the Free Software Foundation (FSF) starting in 1999 and now at the Software Freedom Conservancy (SFC). Those are two organizations that are doing what he called "community enforcement". GPL enforcement is, he said, his only claim to fame—something he has embraced over the last few years.

GPL operation

There is a difference between how the GPL operates in theory and how it works in practice, Kuhn said, and the latter only becomes clear when you try to enforce the license. In theory, the GPL is a copyright license and copyright is "more or less standardized" throughout the world. Like all copyright licenses, the GPL grants permission to do things that would not otherwise be allowed with the copyrighted work. But the GPL "hacks" copyright, into copyleft, by making those permissions dependent on granting the four freedoms to any downstream recipient of the code. Copyleft is one of those things that it is easy to understand once it has been explained, but is hard to come up with when no one else has done so—it was "a stroke of genius".

But in the real world, there are those who violate the GPL. If everyone "played by the rules", Kuhn said, he would use and advocate the Apache License. When there are violations, he believes that "social pressure" is always the first step to take.

When social pressure doesn't work, the copyright holder needs to use copyright law if they want to enforce their copyright (or left). He is not a fan of copyright law, in general, given the way that the movie studios and others have abused it, but copyleft enforcement depends on copyright law. He sees it as a case of "using the tools we have for the cause of good".

It is important to recognize that failing to follow the rules of the license means that the violator loses the right to distribute the GPL-covered coded. Further distribution is copyright infringement, he said, even if it is done in compliance with the license. The only way to get back the right to distribute is to "beg the copyright holder" for permission.

One of the complexities of modern GPL enforcement is that some current enforcement activities are not software-freedom motivated, Kuhn said. Oracle now holds the MySQL copyrights and enforces the GPL in a "corrupt use of copyleft". Oracle says that sending SQL statements to a MySQL server makes the client code a derivative work (thus subject to the GPL). In his mind, in any kind of enforcement, compliance must be the paramount goal. Oracle's goal is to convince people to buy licenses, not to get them to comply. That is not "community enforcement", which puts compliance above all other interests. Community enforcement is done for the public good, by or on behalf of the community.

It is the community (users) that report the violations that he ends up doing GPL enforcement on. Those violations are typically in some embedded device like a TV or a router, Kuhn said. Either the manual has an offer to provide the source, but no source is provided when someone tries to get it (he calls that "offer fail") or it is clearly running Linux but there is no offer to provide the source. The SFC gets a report of that sort weekly or even more often.

Standard procedure

Once a violation report is received, and enforcement is pursued, there is a standard procedure that gets followed. First, verify that there really is a violation, then send a "cease and desist" letter to the violator. "Cease and desist" is the proper technical term, but he doesn't like it because he would really rather see the violator keep using the software, but come into compliance.

At that point there is a loop. The violator is asked for the "complete, corresponding source" (CCS), as required by the GPL, and the SFC then builds that code and tries to make it work, which it almost never does. So SFC sends a report to the violator explaining why the code it sent is not the CCS and asks for it again. Sometimes a patch is sent with the report, he said, that will produce (or help produce) the CCS. That loop can happen many times. The record is 23 times through the loop, but five to seven is the median.

Once the CCS has been sorted out, the SFC asks the violator to inform its customers of that product that the CCS is available and to provide the CCS (as described by the GPL) going forward. It asks the violator to pay a reasonable hourly cost for the work it has done. After that, SFC restores the copyright permissions so that the now ex-violator can legally distribute the software again.

The money is "controversial" in the community, Kuhn said, but "no community enforcer is getting rich"—"maybe Oracle is" with its form of enforcement. In fact, since SFC is a 501(c)(3) US non-profit, you can see its tax filings online. That means you can see how much it got and how much it spent on enforcement, as well as the salaries of Kuhn and other decision-makers at SFC.

When he was at FSF, Kuhn was "the holdout" on the money question, as he didn't think FSF should ask for money. But, at one point, Dan Ravicher asked him who should pay for enforcement. Should it be those companies that donate to the FSF, most of which are in compliance? Or should it be the individual donors to the organization? The money has to come from somewhere, Ravicher told him.

In addition, if there is no deterrent to violating the license, no violators will ever voluntarily comply. If they know they can just "wait until you come knocking", without any financial penalty, they will do just that.

The financial settlements are confidential, but that is at the request of the violators. That upsets him, as he would rather see that information be public.

There are a number of things that SFC does not ask for as part of compliance. It has jumped through "amazing hoops" to make sure that products don't get junked because it is "bad for the environment". In one case, 80,000 units would have had to go to the landfill, but SFC found a way to avert that, Kuhn said. SFC also tries to avoid injunctions, though it has gotten them on occasion. When that happens, the violator has been a year or more out of compliance, had many warnings, and knew that an injunction had been filed for.

Another thing the SFC avoids is getting companies to switch away from using GPL-covered software. Instead, it tries to make it easy for those companies to continue using the software. Lastly, the organization tries to avoid lawsuits. Those are "always a last resort". By the time a lawsuit gets filed, it is only after "hundreds of hours" trying to get the violator to comply.

Building the code

The point of the GPL is not just to be able to examine the source code. The CCS includes "the scripts used to control compilation and installation", so users can actually build the code, not just look at it. That's part of the "freedom to modify", but it can be difficult to check that the scripts included in the CCS will actually build something that will work on an embedded device.

But ensuring that it will build something useful is important, as the WRT54G story shows. In 2003, there were "dozens" of reports about violations in the Linksys WRT54G wireless router. Discussions began between the FSF and Cisco (who had bought Linksys weeks before), but then someone posted the story to Slashdot. There is a mistaken belief that making a violation public will get it resolved more quickly, Kuhn said, but it actually makes it take much longer.

The FSF put together a group to enforce the GPL for that product, which included Erik Andersen of BusyBox and Harald Welte, who had copyrights in the Linux kernel. After many "rounds" of getting CCS candidates, the FSF eventually got everything working (except for two proprietary kernel modules). That CCS became the first check-in for the OpenWrt project, which is now a major replacement firmware option for wireless routers. OpenWrt credits the WRT54G enforcement action as the starting point for the project, Kuhn said.

The FSF was initially shy about lawsuits. Welte participated in the WRT54G enforcement, but tried to get the FSF to file more lawsuits, which it was loath to do. Kuhn said that there was a conference call every week for 30 or 40 weeks in which Welte asked "why haven't you sued them?". In retrospect, Welte was right, Kuhn said. When it became clear that FSF was not going to do so, Welte filed multiple lawsuits in Germany and was "quite successful" in enforcing the GPL in those suits. These days, though, Welte is working on other projects, so his gpl-violations.org project is mostly defunct now, except for hosting the mailing list.

By mid-2006, Andersen had become unhappy with the lack of GPL compliance for BusyBox, particularly in routers and network-attached storage (NAS) devices. He asked SFC to help with BusyBox license enforcement, so SFC became his agent for enforcement, while also receiving other BusyBox developers' copyright assignments for enforcement.

Since 2007, SFC has always had more than 100 violations queued up for enforcement. The list of violations currently stands at more than 300. The enforcement that it is has done on both BusyBox and Linux has made a real difference, Kuhn said.

Samsung is an example of a compliance success story, he said. SFC sued Samsung at one point over code in one of its TVs. That suit was settled and the CCS that came out of it was the basis for the SamyGO project, which creates replacement firmware that enables features like video recording on certain Samsung TV models. More recently, SFC worked with Samsung to fix a GPL-compliance problem in the company's exFAT filesystem. Normally violations take quarters or years to fix, but that one was resolved in weeks. It shows that, as Samsung now knows, compliance is not actually all that difficult, Kuhn said.

But why are there so many violations? He said he doesn't think downstreams (like device makers) are the problem here; the problem comes from upstream. He has tried to get violators to go on the record blaming their upstream suppliers, so that he can go after the supplier instead, but no one seems willing to do that. All of the violators ask that he not talk to their upstream about compliance, as the violators want to work with the upstream on compliance, which is a bit puzzling to him.

What developers can do

Developers, and embedded developers in particular, can help stop these violations. When you get code from a supplier, ensure that you can build it, he said, because someone will eventually ask. Consider using the Yocto project, as Beth Flanagan has been adding a number of features to Yocto to help with GPL compliance. Having reproducible builds is simply good engineering practice—if you can't reproduce your build, you have a problem.

He recommends putting the CCS online and noting the URL in the manual. Lots of people think that the GPL requires that, even though it doesn't. But doing so makes compliance much easier. He doesn't want to have to test offers from the manual to get the source code; "your fulfillment department will screw this up", he said. You can avoid all of that by having the source code online.

If possible, help select the suppliers and ask them about CCS before buying from them. Companies can also demand legal indemnity from their suppliers. Verizon got indemnity from a supplier who promised not to put any open-source software into a product, which saved Verizon from being the target of an enforcement action when the device was found to contain both Linux and BusyBox.

For many years, BusyBox enforcement was used to require device makers to comply with the licenses of other GPL code they were distributing. Typically that was the Linux kernel. But the community was split about using BusyBox as a lever to get kernel sources. Some kernel developers were unhappy about that, while others were supportive. Current BusyBox maintainer Denys Vlasenko convinced Kuhn that it should not only be BusyBox carrying the load for enforcement and that Linux kernel developers should get involved as well.

Matthew Garrett had been asking Kuhn to help him enforce GPL compliance on the kernel for some time. Garrett is a kernel copyright holder, but for years it was just easier to continue doing enforcement with BusyBox. Once Kuhn changed his mind on that, it turned out there were other kernel developers interested in enforcing the GPL for the kernel as well. That led to the GPL compliance project for Linux developers, which was created in May 2012.

That compliance project has now spread beyond just BusyBox and Linux, with Samba and Mercurial joining up. SFC is also doing "passive enforcement" for other projects where there are no known violations, but that SFC will enforce compliance if any are found. Garrett and David Woodhouse have publicly stated that they engaged SFC to enforce the GPL, but there are other kernel developers (roughly a dozen in all) who have joined the efforts anonymously. SFC is also in discussions with two "major free-software projects" to do enforcement for them.

Kernel modules

But there is an "elephant in the room" with respect to Linux and the GPL: kernel modules. Kuhn and the SFC lawyers believe that Linux kernel modules are almost always derived works of the kernel (thus subject to the GPL). Many corporate lawyers disagree. Since there is limited case law in this area, there is little guidance. That means there are no general rules, so it comes down to the specific facts of the case.

Since both sides believe they are right, this is the kind of dispute that turns into a big court battle. Kuhn's "political opponents" call that battle "the ground war of GPL", he said. He believes that it is time to have that ground war. He also believes that a GPL case will go before the US Supreme Court in the next 20 years or so, he said when answering a question from the audience.

Kuhn wrapped up his talk with an invitation to anyone who has code upstream in the kernel to "join the coalition". For some, their employer will hold the copyright. Others may not want to enforce the license, which is fine as it is not required, he said, and he doesn't blame those who don't want to do so. But for those who do, he asked that they see him after the talk as he had brought along forms for them to sign.