The trouble with firmware [LWN.net]

By Jake Edge
January 5, 2011

Within the various communities that make up the free software world, there are many different opinions on exactly what constitutes "freedom". Those opinions run the gamut from extremely pragmatic to exceedingly strict definitions of "free". Firmware blobs for drivers—almost always released without source code and often without any clear license—have been particularly troublesome for certain projects. Debian has struggled with the problem over the years, finally resolving it by moving the non-free firmware out of its main repository for the upcoming 6.0 ("Squeeze") release. But there are others who find even that insufficient and would like to see any mention of the non-free firmware files removed from the kernel.

The Linux-libre project aims to deliver a completely free (under its definition) Linux distribution. It does that by removing any programs that either don't come with source code, or that has source code that is licensed in a way that does not allow users to modify it. Many Linux distributions already follow those guidelines, but Linux-libre takes things one step further and removes anything that "induces or requires you to install additional pieces of non-Free Software".

As might be guessed, Linux-libre has a rather liberal interpretation of what "induces" means, which is where the kernel firmware loading mechanism runs afoul of the distribution's goals. Linux-libre has already eliminated the non-free firmware from its repositories, but there is still a problem: the kernel will make a request to user space for (potentially) non-free firmware when it loads a driver that requires it. So in the minds of the Linux-libre developers, the kernel is now inducing users to install non-free software.

If user space cannot find the firmware file, then the driver obviously cannot load it into the hardware. That makes the hardware non-functional, presumably, but far worse from an inducement perspective, the kernel puts a message into the log specifically noting the filename that couldn't be found. Evidently, that will induce users to go find a file by that name and install it, thus resulting in a non-free system.

In a discussion about Debian's Squeeze kernel on the linux-libre mailing list, Richard Stallman mentioned a plan to obfuscate the names of the firmware files within the Linux-libre kernel. Part of his message was picked up as a Kernel page "quote of the week", partly because it seems like something of a contradiction to promote freedom by hiding the proper names of firmware files. It's also a little puzzling because the GPL and the software freedom movement is so clearly focused on the rights of users.

But Stallman isn't trying to stop users from finding the firmware, he just wants to ensure that the distribution doesn't induce them to install it. That distinction may seem rather arbitrary to many, but not to him. In response to a concern expressed by Alexandre Oliva about web pages that would show the mapping between the Linux-libre obfuscated names and those of the actual filenames, Stallman explained:

Remember, the goal is not to prevent people from finding these files. [...] Our goal is to avoid steering users towards nonfree software. If they find these identifiers and look them up, I think it will be clear to them that we are not.

The plan seems to go far beyond just creating a kernel with obfuscated firmware names, though. It would be fairly straightforward to run a script on the kernel source that applied some kind of obfuscation filter to the firmware filenames before building the kernel, but that doesn't completely alleviate the problem for Linux-libre. It would also like to have a kernel Git repository with the mangled filenames, but there are problems with that approach as Oliva notes:

I'd love to have this in a git repository rather than as deblobbing scripts, but a solution for the problem of creating a git repository that can track Linux upstream without carrying the non-Free bits that the Linux git repository carries has so far eluded me.

From there, the conversation starts to take on some Orwellian tones. It is not enough to just have the new firmware filenames in the HEAD of the Git repository, Oliva and Stallman would like to eliminate references to the original filenames (and the firmware files themselves) from the repository history. But they also would like to be able to pull from Linus Torvalds's mainline tree periodically. It is a tricky set of conditions to try to fulfill.

As part of the discussion on how to handle it, Stallman seems to bristle over the term "rewriting history"—perhaps because of its Orwellian overtones. He sees that term as (somehow) pushing Torvalds's views:

In effect, it claims that "If Torvalds let someone put nonfree software into a present or even past version of Linux, everyone who wants to refer to a repository of Linux is obligated to refer to that nonfree code." That is his views. We believe it is wrong to tell anyone about this nonfree software, and we will not follow his views.

Stallman doesn't necessarily see being able to pull from the mainline as much more than a convenience. But Oliva, who clearly has more Git and kernel workflow experience than Stallman, is fairly adamant:

However, it is an absolute must for any of us to be able to trivially pull changes from upstream. Actually, make that upstreams, for any branch that tracks Linus' tree and has additional patches might be desirable for any of us to merge into our personal or published repositories. If the Linux-libre repository doesn't fit into people's regular workflow, there will be a strong incentive against using it, against developing improvements for Linux on it. It would be self-defeating. I don't want this kind of pain, not for myself, not for other contributors.

To that end, Oliva asked for ways to accomplish this goal on the Git mailing list. He comes up with some plausible scenarios where others might want this kind of functionality, beyond just the needs of Linux-libre:

Say, portions that are illegal, immoral or too risky in my jurisdiction: patented stuff that lawyers say I should not distribute in any way, unauthorized or otherwise copyright-infringing bits, text or pictures that are offensive or even illegal to publish, i.e., stuff that I must not be caught distributing and that, ideally, I could arrange to not even possess.

A few suggestions of possible workflows were made in that thread, but Oliva seems to be leaning toward using git-filter-branch and either some scripts or changes to Git to facilitate this effort. While his scenarios are a little bit of a stretch—why distribute a project that contains these problematic pieces?—they aren't completely beyond the pale. Others could benefit from whatever solution Linux-libre finally settles on.

It's certainly not the first time that Stallman has seemed to be tilting at windmills, and plenty of folks will find this effort to be pointless, even comical. There are some good arguments on both sides of the firmware debate, but it is a bit hard to see how making it more difficult to get hardware working on Linux-libre is going to lead to more adoption of that distribution. Far more likely, it would seem, is that folks just grab a different distribution (or operating system entirely) and move on.

(Log in to post comments)

The trouble with firmware

Posted Jan 6, 2011 17:19 UTC (Thu) by bronson (subscriber, #4806) [Link]

> changing one entry in the history will ripple sha1 hash changes throughout the whole tree.

This is absolutely true. VCS bridges like git-svn and git-hg store the meta information in each commit's comment. The git-filter-branch script could, as you say, add a "Original-Commit: CF22A1..." line.

Not sure it would be worth it though -- I can't imagine the number of people using the FSF's tree would ever be very high (witness Gnusense).

The FSF is has a decent point, but the way they're making it is simply bizarre. It seems like they're trying to appeal only to their most fanatical supporters and discouraging everybody else. If they took a more mellow, community-oriented, and sustainable approach, people might actually listen. As it is, I bet this effort will go the way of the GNU/Linux appendage: lots of discussion, fizzle, and forget.

I don't care about Linux-libre, but I *do* care about ability to "change the history"

Posted Jan 6, 2011 18:29 UTC (Thu) by khim (subscriber, #9252) [Link]

In theory it's nice to have immutable history. In practice it'll be needed to remove some files from your server... completely. For legal reason.

So while it find the hoopla about Linux-libre strange and unneeded I'd like to see the solution for the problem discussed. I'm not needed it till now and I hope to never see the need for it, but... better safe than sorry...

I don't care about Linux-libre, but I *do* care about ability to "change the history"

Posted Jan 6, 2011 22:07 UTC (Thu) by bronson (subscriber, #4806) [Link]

I erased some history a few months ago when a production password was accidentally committed to our repo. I totally agree that the need exists, but I'm not sure we need a new tool. git filter-branch worked perfectly.

Yes, master's SHA changed, but that's exactly what I would expect to happen! I'd hate for someone to be able to change my history without any hint whatsoever.

I do wish git handled rewritten remote branches better. When some people pulled, they ended up with a bizarre-looking merge, one branch of which I suppose had missing objects (I didn't look, just blew it away and checked out a fixed tracking branch). git should say, "hey dude, upstream has yanked the rug out. Maybe check your email for instructions?" and refuse to do anything until you've rebased things back to normal.

But, yes, the SHA should always change!

The trouble with firmware

Posted Jan 6, 2011 6:37 UTC (Thu) by josh (subscriber, #17465) [Link]

I applaud the Debian developers for working to make Debian's packaged version of the Linux kernel entirely Free Software with source, and I applaud the Linux kernel developers for working with Debian and others to reach this goal (rather than writing it off as a downstream issue).

However, I don't see how going further and attempting to obfuscate the firmware references will provide any useful improvement for Free Software. As far as I can tell, the Debian approach helps all classes of users: those who want to avoid non-free software entirely can install the kernel from main and know they have an entirely Free kernel, and those who have hardware that requires firmware and want that hardware to work can install a package of firmware files.

What class of users does this further effort attempt to help? I don't see how anyone will become "induced" to use proprietary firmware because the Linux kernel references. They become "induced" to use proprietary firmware because their hardware requires it. If anything, this issue needs *more* attention, not less. A list of hardware that requires proprietary firmware would help anyone who wanted to avoid that. Even better if that list includes alternatives that don't require proprietary firmware, with special attention to those pieces of hardware that have entirely Free firmware (rather than just having burned the proprietary firmware into a flash part). (Such hardware does exist; for instance, several Atheros wireless chipsets can use Free firmware.)

The trouble with firmware

Posted Jan 6, 2011 18:31 UTC (Thu) by cesarb (subscriber, #6266) [Link]

Well, git-filter-branch *is* rewriting history. It is what it was designed to do: create a new "history" by copying another one while changing parts of it.

What they seem to be wanting is something different, and not something which can be done with normal git tools: to do a partial clone, ignoring some parts of the history. Some thing like "clone me that tree except blobs abcdef1 and def2abc", and being able to checkout a commit which references these blobs even though they do not exist. This might need some deep changes in the git code; AFAIK it does assume that if it has a particular commit, it has all the trees and blobs referenced by it (but perhaps not the parent commits, due to shallow clones).

A clone made with that magic git would not have the bits for all the blobs they decided to ignore, while keeping the same commit, tree, blob, and even tag hashes (the trees would simply be referencing blobs which do not exist).

Of course, this does not work when downloading from a normal git server; the remote server will create (or reuse one it already has) a highly-compressed pack with all the history for the requested commits. But after that, they could unpack and clone from *that* with their magic git, and since both sides are now the same hacked git, it will omit the bits they are scared of.

The trouble with firmware

Posted Jan 6, 2011 23:05 UTC (Thu) by bronson (subscriber, #4806) [Link]

It sounds like the FSF would disagree with the "except blobs abcdef1 and def2abc" bit. The user is still "obligated to refer to that nonfree code" even if it's just by SHA1.

The trouble with firmware

Posted Jan 8, 2011 3:41 UTC (Sat) by gjmarter (subscriber, #5777) [Link]

It seems plausible to create a git that does not store certain blobs that it claims to have. What seems more difficult is to have a git which would alter the trees so that the offending filename is not listed. How would you do that and still be able to verify the integrity of the tree against the SHA1?

I think the filename in the tree is even a bigger concern than the blob references.

The trouble with firmware

Posted Jan 10, 2011 13:47 UTC (Mon) by jwarnica (subscriber, #27492) [Link]

I think, at one point, RMS was a wackjob, had interesting politics, and someone who produced useful technology. One looked first at his technology (Emacs significantly, the gnu toolset, etc), then at his politics, and didn't dig much deeper.

It is easy to like someone you don't really know.

On a personal basis, RMS stopped producing technology decades ago. And GNU/FSF is only a shadow of its ones productive self, at least WRT other groups and companies producing free software.

Politically, OSS has subverted much of the "freedom" goals, and OSS if not "free" software is fairly common these days. The war isn't won, but many battles have been. There is a big choir these days, and RMS talks, declarations, and decrees are generally targeted at the now overwhelmed choir.

So there really isn't anything else, nothing new, from RMS, except for the wackjob part.

The trouble with firmware

Posted Jan 10, 2011 14:48 UTC (Mon) by foom (subscriber, #14868) [Link]

I don't think the "war" is anywhere near won -- in fact I'd say just the opposite: it seems like it's in serious danger of being completely lost -- for users.

It's true that the tools *developers* depend on, for development, are overwhelmingly open source. However, the tools *users* depend on (that is: gmail, facebook, etc) are overwhelmingly closed source (even though they're often internally based heavily upon open source software).

And as more and more kinds of tools are moving to the web, that's true more and more, even for users of Linux, who previously had been using almost wholly open source software.

The trouble with firmware

Posted Jan 10, 2011 16:16 UTC (Mon) by jwarnica (subscriber, #27492) [Link]

Perhaps you are proving my unstated point. The number of fronts is expanding, and RMS is spending his time on a really very minor issue. He has really lost all perspective. Or rather, his perspective has remained the same, and once was mostly relevant.

He doesn't use the web (http://lwn.net/Articles/262570/) and I remember, I think from early Playstation 3 days, him admitting ignorances of the very concept of a gaming console. Not of Linux on the PS3, which might get a pass, but of being intentionally ignorant of the existence of non-general purpose electronic devices. He is simply out of touch with how regular people, and for that matter, regular geeks, use and interact with computers.

Basically all hardware uses microcode, and has for ever. To me, firmware is a minor, near irrelevant point. But even if I was to grant it was important, this strategy is the akin to plowing salt into your own fields and retreating from the land you have destroyed. Maybe we can shelf this point for a while, and worry about Facebook, and not OpenID, becoming the SSO/AuthN standard?

Political or spiritual advisor - maybe. Tactician, not at all.

The trouble with firmware

Posted Jan 11, 2011 3:15 UTC (Tue) by quotemstr (subscriber, #45331) [Link]

Simone de Beauvoir once said, "If you live long enough, you'll see that every victory turns into a defeat."

We're definitely experiencing a retrenchment of proprietary software, primarily in the form of "app stores" and Tivoized devices. This model has been lucrative, and is actually far more frightening than the boxed-software model that originally motivated RMS and inspires free software. Like any successful disruptive technology, the new proprietry software will grow to consume the market. It will be much more difficult for free software to gain a foothold when a few powerful vendors can not only forbid the distribution of software, but send existing programs on users' devices down the memory hole. It will be interesting to watch, if nothing else.