Emacs distributions are not GPL-compliant

[Posted July 29, 2011 by corbet]

From:		Richard Stallman <rms-AT-gnu.org>
To:		Paul Eggert <eggert-AT-cs.ucla.edu>
Subject:		Re: Compiled files without sources????
Date:		Thu, 28 Jul 2011 19:00:17 -0400
Message-ID:		<E1QmZYv-0000Np-MO@fencepost.gnu.org>
Cc:		dak-AT-gnu.org, emacs-devel-AT-gnu.org

    They are in Emacs releases 23.2 and 23.3.  They have been in all
    pretests since emacs-23.1.90, dated 2009-12-09.  They were merged
    into the Emacs trunk in bzr 97804, dated 2009-09-28; here's a
    URL for that:

We have made a very bad mistake.  Anyone redistributing those versions
is violating the GPL, through no fault of his own.

We need to fix those releases retroactively (or else delete them), and
we need to do it right away.

I see two quick ways to fix them: to delete the compiled files, or to
add the sources they are made from.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 13:07 UTC (Fri) by SEJeff (guest, #51588) [Link] (14 responses)

Props to Richard for owning up to and announcing he made a mistake. A lot of egos in the "FLOSS Community" would never consider such a thing.

Yes, props

Posted Jul 29, 2011 13:18 UTC (Fri) by david.a.wheeler (subscriber, #72896) [Link]

I agree. "I'm sorry, I made a mistake, and I will correct it" is something that we should all say occasionally, but it's rarely said. Congrats to RMS for owning up, and more importantly, for stating that he will correct the problem.

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 17:45 UTC (Fri) by VelvetElvis (guest, #69142) [Link]

It's most likely not even his mistake and he's still apologizing for it.

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 0:53 UTC (Sat) by sbergman27 (guest, #10767) [Link] (11 responses)

The terms "hoist" and "petard" come to mind. Of course, nobody really cares about this, so it must have been quite easy for RMS to admit to the mistake. He's managed to create liscensing of such complexity that even he doesn't completely understand the implications. If he fessed up and apologized for that, I'd be impressed. But apologizing for a violation that most FLOSS users don't care about is more about PR than anything else.

Don't get me wrong. GPL, especially v2, has a lot going for it. As a software user and community member, it is my preferred license. But it has its own set of significant disadvantages as well. The fact that the project leader (or whatever his official capacity is with emacs), author of the GPL Vx, and evangelist of Free Software, could have let this mistake go unnoticed for so long a time says something about the state of affairs.

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 1:26 UTC (Sat) by tialaramex (subscriber, #21167) [Link] (3 responses)

"such complexity that even he doesn't completely understand the implications"

So far as I've seen there's no evidence of that here. All that happened is someone imported a working piece of source code without understanding that this source code included parsers which were not hand rolled but generated by a parser generator. If it's not obvious (which I think it should be to any competent software developer) the GPL FAQ already explains that in this situation the grammar or other input for the parser generator are the source, and GNU include such grammars elsewhere, but in this case there was an oversight.

Still, as you've said it's very easy to admit to mistakes and they're "more about PR than anything else". So I'm sure you will promptly admit you made a mistake. Right?

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 14:17 UTC (Sat) by sbergman27 (guest, #10767) [Link] (2 responses)

It depends. If I did, would I get a string of posts in response going on about what a great guy I was for doing it?

Emacs distributions are not GPL-compliant

Posted Aug 1, 2011 8:44 UTC (Mon) by niner (subscriber, #26151) [Link] (1 responses)

RMS didn't know that either before he apologized. But since you ask that question, you missed your chance anyway, because now it's clear that your motive for apologizing would be to solicit ego boosting replies rather than admitting error.

Emacs distributions are not GPL-compliant

Posted Aug 2, 2011 16:43 UTC (Tue) by sbergman27 (guest, #10767) [Link]

Sure he did. You really can't observe the Linux serial without noticing that Stallman has a cult following. Of course he knew he would receive accolades.

Regarding my alleged solicitation of ego-boosting posts... do you really think I was looking for ego-boosting posts?

At any rate, one only needs to review the uncountable threads in many forums to see that GPLvX's "teeth" have a downside in the form of complex implications. Friendly fire is a real problem. And it's charitable to call the fire "friendly" since it is the aim of the GPL to use its teeth even against other Free Licenses.

I certainly have no problem admitting that this situation is not an example. Was that not implicit in my previous posts? It's really no problem at all. Despite the costs, I guess I'm still pro-copyleft. But that's a topic which merits periodic re-evaluation.

I will say one other thing. I've been away for a while. Linux is still my one and only OS for home and for work. But if you (that's an impersonal "you") stop visiting Linux news sites for a while, and then come back, everything looks different. There is a certain indoctrination which comes from having one's views reinforced by like-minded people.

You can safely consider the last paragraph to be soliloquy, It's not directed at you.

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 1:39 UTC (Sat) by Trelane (subscriber, #56877) [Link] (5 responses)

> apologizing for a violation that most FLOSS users don't care about is more about PR than anything else.

Or, alternately, he really does care about not violating the license.

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 15:34 UTC (Sat) by sbergman27 (guest, #10767) [Link] (4 responses)

I'm sure that he does. Promoting the GPL is his raison d'être. I simply find it amusing that people are patting him (of all people) on the back for doing what any decent person would do regarding a GPL violation.

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 17:51 UTC (Sat) by tialaramex (subscriber, #21167) [Link] (3 responses)

Is it your claim that you're _not_ a decent person?

In my experience, after dealing with more than a dozens minor copyright violations, "This is a mistake, we will fix it" is actually one of the least frequent responses, coming far behind such chestnuts as "If I download something from the web then it's public domain and mine to do with as I please". Perhaps decent people are just very rare...

As to the fact that people receive praise for doing what _should_ be just part of their job, that's surely no criticism of either the people being praised or those who perceive this as extraordinary enough to praise them. The "Miracle on the Hudson" was a relatively straight forward emergency landing, performed by two experienced professionals. You should expect no less of anyone qualified to do their job. But one of them (often mis-identified as "the pilot" when in fact of course all such commercial airliners have at least two pilots) received particular praise and honour for ensuring the safe outcome.

Emacs distributions are not GPL-compliant

Posted Jul 31, 2011 0:15 UTC (Sun) by sbergman27 (guest, #10767) [Link] (2 responses)

"Is it your claim that you're _not_ a decent person?"

Wow. As long as we've sunk to this level, I may as well go ahead and say "I'm rubber, you're glue. Whatever you say bounces off me and sticks to you."

What I find interesting about all this is that people are acting as though Stallman might well have just swept this under the rug, but didn't this time.

Sorry if I seemed to be disrespecting your god. I'd forgotten how easy it was to rile the Cult of Stallman, even if unintentionally. I say this in hopes of not finding a bag of cat entrails on my doorstep in the morning.

Emacs distributions are not GPL-compliant

Posted Jul 31, 2011 7:50 UTC (Sun) by tialaramex (subscriber, #21167) [Link] (1 responses)

What you're illustrating is that in fact when you make a mistake as you did above you just get defensive and lash out rather than admit to the mistake and do what you can to fix it. This is pretty common, most people do it once in a while. I know I do (I think I already hear my colleagues agreeing).

But it is in sharp contrast to what you've said a "decent" person would do.

We have veered far off the topic. Meanwhile the original Emacs thread has developed, with the revelation that these machine generated parsers were changed by hand before they were committed. The maintainer apparently responsible doesn't seem to grok that this is a horrible situation not just for third parties who might want to alter the parser but for continued maintenance of GNU Emacs by the GNU project.

Emacs distributions are not GPL-compliant

Posted Jul 31, 2011 15:26 UTC (Sun) by sbergman27 (guest, #10767) [Link]

"What you're illustrating is that in fact when you make a mistake as you did above you just get defensive and lash out rather than admit to the mistake and do what you can to fix it."

You're greatly overestimating the degree to which I care about these trivialities. Some of us don't obsess upon FOSS mintutia. I used to do a bit of that, but decided it was a waste of time.

"The maintainer apparently responsible doesn't seem to grok that this is a horrible situation..."

Yes, truly horrible. I'd put in on par with the Oklahoma City bombing of 1995. Perhaps its time for NATO to get involved.

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 22:50 UTC (Sat) by ewan (guest, #5533) [Link]

He's managed to create liscensing of such complexity that even he doesn't completely understand the implications.

I don't see any evidence that anyone failed to understand what they were supposed to do; just that they accidentally didn't do it. If I trip over and faceplant into the ground it's not because I don't understand that it's going to hurt, it's because I tripped up.

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 13:16 UTC (Fri) by mrshiny (guest, #4266) [Link] (1 responses)

Couldn't they just make the source available somewhere? Presumably they have it. Put it on an FTP server, and bingo?

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 20:45 UTC (Fri) by tialaramex (subscriber, #21167) [Link]

Not bingo. As they point out, people who in good faith redistribute the tarballs that already shipped will be in violation, now and forever.

That violation isn't the kind of thing that would ever see court (because it's someone else's mistake and it's trivially rectified anyway) but it technically will still be a violation.

It's a common misunderstanding of the GPL to assume that "there's an FTP site somewhere with the source code" meets your obligation to either redistribute or provide a written offer (and there's a variant which is even more pernicious, where people feel that since their changes to get the code to build on platform X were just a few hours work, it's OK to just post the platform X binaries with no source ...)

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 15:03 UTC (Fri) by groualland (guest, #77502) [Link]

Looks like the source of the missing files is available in the CEDET repository on sourceforge, so hopefully this will be fixed soon.

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 16:09 UTC (Fri) by iabervon (subscriber, #722) [Link] (4 responses)

Alternatively, they could just maintain the C version of the parsers in the future, instead of making changes to the missing .y files. Considering that the emacs source has C files with dynamically scoped variables (among other oddities), it wouldn't be the strangest choice of "preferred form for modification", although it probably is a bit more convoluted than average.

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 16:47 UTC (Fri) by bitshifter (guest, #70716) [Link] (3 responses)

I think I'm missing something here. How can one have dynamically scoped variables in C?

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 17:02 UTC (Fri) by quotemstr (subscriber, #45331) [Link] (1 responses)

Emacs C sources are tightly integrated into the Lisp engine, and C functions can bind variables just as Lisp can. In fact, we've even laid the groundwork for making these bindings thread-local when we have threads one day.

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 17:41 UTC (Fri) by bitshifter (guest, #70716) [Link]

Could you point me at a reference for this? I'm intrigued on the mechanics and my google-fu is proving to be a little weak.

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 17:22 UTC (Fri) by iabervon (subscriber, #722) [Link]

Above is a good description of why you'd do that; as for how you can do it: they're accessed with macros, and the C preprocessor can do interesting things.

Emacs distributions are not GPL-compliant

Posted Jul 29, 2011 17:08 UTC (Fri) by emk (subscriber, #1128) [Link] (8 responses)

Reading through the thread, it looks like the code in question is part of CEDET, which attempts to make Emacs more IDE-like. Specifically, CEDET includes Elisp files that were created using a parser generator, but it does not contain the original grammar files used as input to the parser generator.

Further complications: the output of bison was modified by hand

Posted Jul 31, 2011 6:25 UTC (Sun) by nicku (guest, #777) [Link] (7 responses)

The output of bison was further modified by hand. How do you package all this? Package a patch to the output of bison to produce the code actually used?

Further complications: the output of bison was modified by hand

Posted Jul 31, 2011 8:14 UTC (Sun) by tialaramex (subscriber, #21167) [Link] (6 responses)

Well, if you actually want to maintain your code, you take a good hard look at those modifications and you have to make a decision

Maybe all or most modifications are cosmetic. Since the output of a parser generator isn't meant to be read (on the whole) unlike other code it's OK that it doesn't obey your house style, so these changes can be reverted. This is in many ways the best case.

Maybe the modifications can equivalently be performed in the generator input, perhaps by a more experienced programmer or one familiar with the quirks of the generator. This is also a good outcome.

Maybe you can live without the modifications. Perhaps they simplify some code outside the parser, but aren't strictly necessary. The reduced maintenance cost of using unmodified generator output may justify doing the extra work in your scaffolding. This isn't ideal, but you can live with it.

Maybe you need to modify the generator, to add a feature or fix a bug so that it can generate the code you want. This is in many ways the worst case, since you take on a significant additional maintenance burden unless you can get your changes upstream promptly. This solution is of course impossible if the generator is not provided as source code, or if your project needs to bootstrap on platforms where the generator is provided but you cannot rebuild it. But it might still beat your final option:

Otherwise maintain the patch as you described, knowing that it is brittle and may require frequent manual intervention to keep it working in the face of tweaks to the generator input, the generator, or the surrounding environment.

The same situation applies for regular compiled code too -- it's just that GCC gives you a heck of a lot more rope to try MacGyver-esque escape plans before you have to resort to patching the intermediary assembler files.

Further complications: the output of bison was modified by hand

Posted Aug 1, 2011 10:00 UTC (Mon) by dwmw2 (subscriber, #2063) [Link] (5 responses)

Or you declare that the output of the generator is actually the preferred form of the work for making modifications to it.

That's not so far-fetched; sometimes your generator is a one-off thing which just provides a template that you then fill in with real code. I've seen GUI design tools which work that way, for example: it just generates code (and function prototype and stub callbacks etc.) to match your design, and you're supposed to fill in the missing parts to make it actually do something.

If they've gone so long without regenerating the parser from scratch and they've never noticed, and they've been modifying it by hand, it sounds like it could be plausible in this case too.

Further complications: the output of bison was modified by hand

Posted Aug 1, 2011 13:35 UTC (Mon) by tialaramex (subscriber, #21167) [Link] (2 responses)

Yes, that would be an acceptable way forward if it were true.

I have used generators of the sort you're describing and you're right that in these cases there was no "real" source, the output of the generator was the preferred form for future modifications as far as I (the main or only programmer) was concerned and the generator was of no further use to the project.

But in this Emacs case it appears more like False Laziness to me. There doesn't seem to be anyone claiming that the generator's output is now their preferred form for working on the actual parsers, just that it was easier to hack the output than make the generator work as they wished. Well, easier at the time.

Further complications: the output of bison was modified by hand

Posted Aug 1, 2011 13:52 UTC (Mon) by dwmw2 (subscriber, #2063) [Link] (1 responses)

Well there was obviously a point at which for someone, the preferred form for modification was the output of the generator. Even if it was just because they were being lazy and it seemed easier at the time.

And right now, if I had to make simple modifications to it, that would probably be my preferred form too. I'd have a choice: either I could make my simple change to the C code that has already been modified by hand... or I could try to look at the existing modifications to the C code, then fix the generator (and its input) so it can generate functionally equivalent code, and *then* make my changes.

Further complications: the output of bison was modified by hand

Posted Aug 1, 2011 16:32 UTC (Mon) by nix (subscriber, #2304) [Link]

Why is everyone talking about C code? These are Emacs Lisp files, generated by running a generator written in Emacs Lisp over a grammar. However, said generator (which *is* in upstream CEDET) has requirements which the packages in downstream Emacs cannot satisfy (mostly EIEIO features, but also Emacs renamed a huge pile of files for 8+3 compatibility which upstream has not yet done in its trunk). Thus, even if the parser generator was integrated, you couldn't *run* it without getting upstream CEDET anyway.

Further complications: the output of bison was modified by hand

Posted Aug 4, 2011 19:21 UTC (Thu) by JoeBuck (subscriber, #2330) [Link] (1 responses)

Abraham Lincoln used to tell a riddle to make a point: how many legs does a horse have if you call a tail a leg? The answer is four; calling a tail a leg doesn't make it one.

Likewise, declaring that the code generator output is the preferred form for modification is simply to lie; it is disproved by the fact that the developers don't work that way; they make changes to the input and re-run the code generator. The copyright holder can cross out that part of the GPL and write something else, but then it's no longer really the GPL.

Further complications: the output of bison was modified by hand

Posted Aug 4, 2011 20:52 UTC (Thu) by nix (subscriber, #2304) [Link]

Actually in this case he *did* modify it by hand, so things are more blurred.

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 10:43 UTC (Sat) by schily (guest, #60311) [Link] (1 responses)

If this was the only problem with software from the FSF, things would look different.

But why did it take 10 years (and help from Suse) to find a solution for an intentional(*) Copyright problem in vcdimager?

Why are there still other problems with code from projects distributed by the FSF that is announced to be GPLv2+ or even GPLv3 even though the original authors did not give permission?

*) The problem was not intentional but accidental before the maintainer has been informed about the problem in 2001, but denied to fix it.

Emacs distributions are not GPL-compliant

Posted Jul 30, 2011 15:34 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link]

Helpful if you provide references. For projects that people have assigned copyright to FSF, FSF doesn't require permission to relicense to GPLv3+. FSF copyright assignment would prevent them from relicensing under proprietary licenses however.