|
|
Subscribe / Log in / New account

The trouble with firmware

By Jake Edge
January 5, 2011

Within the various communities that make up the free software world, there are many different opinions on exactly what constitutes "freedom". Those opinions run the gamut from extremely pragmatic to exceedingly strict definitions of "free". Firmware blobs for drivers—almost always released without source code and often without any clear license—have been particularly troublesome for certain projects. Debian has struggled with the problem over the years, finally resolving it by moving the non-free firmware out of its main repository for the upcoming 6.0 ("Squeeze") release. But there are others who find even that insufficient and would like to see any mention of the non-free firmware files removed from the kernel.

The Linux-libre project aims to deliver a completely free (under its definition) Linux distribution. It does that by removing any programs that either don't come with source code, or that has source code that is licensed in a way that does not allow users to modify it. Many Linux distributions already follow those guidelines, but Linux-libre takes things one step further and removes anything that "induces or requires you to install additional pieces of non-Free Software".

As might be guessed, Linux-libre has a rather liberal interpretation of what "induces" means, which is where the kernel firmware loading mechanism runs afoul of the distribution's goals. Linux-libre has already eliminated the non-free firmware from its repositories, but there is still a problem: the kernel will make a request to user space for (potentially) non-free firmware when it loads a driver that requires it. So in the minds of the Linux-libre developers, the kernel is now inducing users to install non-free software.

If user space cannot find the firmware file, then the driver obviously cannot load it into the hardware. That makes the hardware non-functional, presumably, but far worse from an inducement perspective, the kernel puts a message into the log specifically noting the filename that couldn't be found. Evidently, that will induce users to go find a file by that name and install it, thus resulting in a non-free system.

In a discussion about Debian's Squeeze kernel on the linux-libre mailing list, Richard Stallman mentioned a plan to obfuscate the names of the firmware files within the Linux-libre kernel. Part of his message was picked up as a Kernel page "quote of the week", partly because it seems like something of a contradiction to promote freedom by hiding the proper names of firmware files. It's also a little puzzling because the GPL and the software freedom movement is so clearly focused on the rights of users.

But Stallman isn't trying to stop users from finding the firmware, he just wants to ensure that the distribution doesn't induce them to install it. That distinction may seem rather arbitrary to many, but not to him. In response to a concern expressed by Alexandre Oliva about web pages that would show the mapping between the Linux-libre obfuscated names and those of the actual filenames, Stallman explained:

Remember, the goal is not to prevent people from finding these files. [...] Our goal is to avoid steering users towards nonfree software. If they find these identifiers and look them up, I think it will be clear to them that we are not.

The plan seems to go far beyond just creating a kernel with obfuscated firmware names, though. It would be fairly straightforward to run a script on the kernel source that applied some kind of obfuscation filter to the firmware filenames before building the kernel, but that doesn't completely alleviate the problem for Linux-libre. It would also like to have a kernel Git repository with the mangled filenames, but there are problems with that approach as Oliva notes:

I'd love to have this in a git repository rather than as deblobbing scripts, but a solution for the problem of creating a git repository that can track Linux upstream without carrying the non-Free bits that the Linux git repository carries has so far eluded me.

From there, the conversation starts to take on some Orwellian tones. It is not enough to just have the new firmware filenames in the HEAD of the Git repository, Oliva and Stallman would like to eliminate references to the original filenames (and the firmware files themselves) from the repository history. But they also would like to be able to pull from Linus Torvalds's mainline tree periodically. It is a tricky set of conditions to try to fulfill.

As part of the discussion on how to handle it, Stallman seems to bristle over the term "rewriting history"—perhaps because of its Orwellian overtones. He sees that term as (somehow) pushing Torvalds's views:

In effect, it claims that "If Torvalds let someone put nonfree software into a present or even past version of Linux, everyone who wants to refer to a repository of Linux is obligated to refer to that nonfree code." That is his views. We believe it is wrong to tell anyone about this nonfree software, and we will not follow his views.

Stallman doesn't necessarily see being able to pull from the mainline as much more than a convenience. But Oliva, who clearly has more Git and kernel workflow experience than Stallman, is fairly adamant:

However, it is an absolute must for any of us to be able to trivially pull changes from upstream. Actually, make that upstreams, for any branch that tracks Linus' tree and has additional patches might be desirable for any of us to merge into our personal or published repositories. If the Linux-libre repository doesn't fit into people's regular workflow, there will be a strong incentive against using it, against developing improvements for Linux on it. It would be self-defeating. I don't want this kind of pain, not for myself, not for other contributors.

To that end, Oliva asked for ways to accomplish this goal on the Git mailing list. He comes up with some plausible scenarios where others might want this kind of functionality, beyond just the needs of Linux-libre:

Say, portions that are illegal, immoral or too risky in my jurisdiction: patented stuff that lawyers say I should not distribute in any way, unauthorized or otherwise copyright-infringing bits, text or pictures that are offensive or even illegal to publish, i.e., stuff that I must not be caught distributing and that, ideally, I could arrange to not even possess.

A few suggestions of possible workflows were made in that thread, but Oliva seems to be leaning toward using git-filter-branch and either some scripts or changes to Git to facilitate this effort. While his scenarios are a little bit of a stretch—why distribute a project that contains these problematic pieces?—they aren't completely beyond the pale. Others could benefit from whatever solution Linux-libre finally settles on.

It's certainly not the first time that Stallman has seemed to be tilting at windmills, and plenty of folks will find this effort to be pointless, even comical. There are some good arguments on both sides of the firmware debate, but it is a bit hard to see how making it more difficult to get hardware working on Linux-libre is going to lead to more adoption of that distribution. Far more likely, it would seem, is that folks just grab a different distribution (or operating system entirely) and move on.

Index entries for this article
KernelCopyright issues
KernelFirmware


to post comments

The trouble with firmware

Posted Jan 6, 2011 2:50 UTC (Thu) by Baylink (guest, #755) [Link]

RMS is important.

If he hadn't written the GPL, and been a zealot about it, and Linus hadn't applied it to the Linux kernel, no piece of the technology world would look the way it does today; I firmly believe this.

That said, this approach is a real pain in the ass for those of us who *do* have to live in the Real World, and I certainly won't be adopting it.

We need zealots. These "monomaniac with a mission", as Tom Peters described them 30 years ago, are the people who get things accomplished; who change the world.

But their right to swing their fist stops *just* short of my nose. :-)

The trouble with firmware

Posted Jan 6, 2011 3:40 UTC (Thu) by JohnLenz (guest, #42089) [Link] (3 responses)

It is not enough to just have the new firmware filenames in the HEAD of the Git repository, Oliva and Stallman would like to eliminate references to the original filenames (and the firmware files themselves) from the repository history. But they also would like to be able to pull from Linus Torvalds's mainline tree periodically.

If you do this, doesn't git recompute the sha1 hashes for the commit id for all future commits, since the sha1 of the parent is included in the current commit? So changing one entry in the history will ripple sha1 hash changes throughout the whole tree.

If so, this sounds like a bad idea. If you report a bug on linux-kernel and someone says "This is fixed in 4d5227500c8f ...", that sha1 hash won't be in your tree, the same commit will be referenced with some other hash. Also, if do a git bisect, you won't be able to report it because the hashes from your git bisect won't match with what you need to report on linux-kernel.

I suppose you could keep around a map between Linus's sha1 and your sha1 ids and remember to convert whenever you send email to linux-kernel.

The trouble with firmware

Posted Jan 6, 2011 17:19 UTC (Thu) by bronson (subscriber, #4806) [Link] (2 responses)

> changing one entry in the history will ripple sha1 hash changes throughout the whole tree.

This is absolutely true. VCS bridges like git-svn and git-hg store the meta information in each commit's comment. The git-filter-branch script could, as you say, add a "Original-Commit: CF22A1..." line.

Not sure it would be worth it though -- I can't imagine the number of people using the FSF's tree would ever be very high (witness Gnusense).

The FSF is has a decent point, but the way they're making it is simply bizarre. It seems like they're trying to appeal only to their most fanatical supporters and discouraging everybody else. If they took a more mellow, community-oriented, and sustainable approach, people might actually listen. As it is, I bet this effort will go the way of the GNU/Linux appendage: lots of discussion, fizzle, and forget.

I don't care about Linux-libre, but I *do* care about ability to "change the history"

Posted Jan 6, 2011 18:29 UTC (Thu) by khim (subscriber, #9252) [Link] (1 responses)

In theory it's nice to have immutable history. In practice it'll be needed to remove some files from your server... completely. For legal reason.

So while it find the hoopla about Linux-libre strange and unneeded I'd like to see the solution for the problem discussed. I'm not needed it till now and I hope to never see the need for it, but... better safe than sorry...

I don't care about Linux-libre, but I *do* care about ability to "change the history"

Posted Jan 6, 2011 22:07 UTC (Thu) by bronson (subscriber, #4806) [Link]

I erased some history a few months ago when a production password was accidentally committed to our repo. I totally agree that the need exists, but I'm not sure we need a new tool. git filter-branch worked perfectly.

Yes, master's SHA changed, but that's exactly what I would expect to happen! I'd hate for someone to be able to change my history without any hint whatsoever.

I do wish git handled rewritten remote branches better. When some people pulled, they ended up with a bizarre-looking merge, one branch of which I suppose had missing objects (I didn't look, just blew it away and checked out a fixed tracking branch). git should say, "hey dude, upstream has yanked the rug out. Maybe check your email for instructions?" and refuse to do anything until you've rebased things back to normal.

But, yes, the SHA should always change!

The trouble with firmware

Posted Jan 6, 2011 6:37 UTC (Thu) by josh (subscriber, #17465) [Link]

I applaud the Debian developers for working to make Debian's packaged version of the Linux kernel entirely Free Software with source, and I applaud the Linux kernel developers for working with Debian and others to reach this goal (rather than writing it off as a downstream issue).

However, I don't see how going further and attempting to obfuscate the firmware references will provide any useful improvement for Free Software. As far as I can tell, the Debian approach helps all classes of users: those who want to avoid non-free software entirely can install the kernel from main and know they have an entirely Free kernel, and those who have hardware that requires firmware and want that hardware to work can install a package of firmware files.

What class of users does this further effort attempt to help? I don't see how anyone will become "induced" to use proprietary firmware because the Linux kernel references. They become "induced" to use proprietary firmware because their hardware requires it. If anything, this issue needs *more* attention, not less. A list of hardware that requires proprietary firmware would help anyone who wanted to avoid that. Even better if that list includes alternatives that don't require proprietary firmware, with special attention to those pieces of hardware that have entirely Free firmware (rather than just having burned the proprietary firmware into a flash part). (Such hardware does exist; for instance, several Atheros wireless chipsets can use Free firmware.)

The trouble with firmware

Posted Jan 6, 2011 7:32 UTC (Thu) by james (subscriber, #1325) [Link] (1 responses)

While his scenarios are a little bit of a stretch—why distribute a project that contains these problematic pieces?
Fedora insists that its source RPMs not contain any "problematic pieces" (e.g. patent-afflicted code): not compiling them into the binary or patching them out of the source isn't good enough.

For example, Damien Miller of OpenSSH was rather annoyed six years ago to find that Fedora was not building OpenSSH from standard OpenSSH tarballs.

It turned out that Red Hat had chosen not to ship the ACSS code in OpenSSH, since it was "believed to be compatible with" the CSS DRM in DVDs, and both possession and distribution of CSS-compatible code is illegal under many laws, including the American DMCA.

So if Fedora were to switch to downloading sources from Git at build-time, rather than storing them in the SRPM, they would need something like this.

James

The trouble with firmware

Posted Jan 6, 2011 16:20 UTC (Thu) by joey (guest, #328) [Link]

So does Debian. And both Debian and Fedora have solid legal reasons to require this. For example, CDs of source need to be sellable, which is difficult to do if the source contains non-free stuff. Debian is forced to modify roughly 1 out of every 50 source packages due to such issues.

Moreover I have developed the "3.0 (git)" source package format for Debian which uses git bundles. Of course non-free bits in git history makes using that more problematic, and the best workaround available at present is to bundle only a shallow clone of the repository that omits the non-free historical bits.

So any technical progress in this area could really be valuable, no matter what your opinion of the firmware file obfuscation part.

No software is an island

Posted Jan 6, 2011 14:50 UTC (Thu) by felixfix (subscriber, #242) [Link]

I appreciate all the work RMS has done and continues to do, but sometimes purity has to reach a limit. Just as hippies who returned to the earth still depended on the society they left for basic tools, basic medicine, and even money at times, so too must free source software at some point depend on proprietary tools, such as chip foundries and the microcode inside chips.

More than anything else, even before the article got around to mentioning rewriting the git history, my mind flashed to the old Soviet practice of erasing people from pictures when they fell from grace. I always wondered not only how they got rid of all past copies of books and archival newspapers, but how far back they went in erasing pictures? Did they work up new pictures from boy scout (equivalent) days? School pictures? Army pictures?

RMS has to reach a limit. You can't be pure turtles all the way down.

The trouble with firmware

Posted Jan 6, 2011 15:13 UTC (Thu) by ibukanov (subscriber, #3942) [Link] (3 responses)

It is unfortunate that the term "rewriting history" was used. The issue is about been able *not to* download things one do not want that live in a history of some remote git repository. So it is more in line of not receiving unwanted mail together with some wanted one.

The trouble with firmware

Posted Jan 6, 2011 18:31 UTC (Thu) by cesarb (subscriber, #6266) [Link] (2 responses)

Well, git-filter-branch *is* rewriting history. It is what it was designed to do: create a new "history" by copying another one while changing parts of it.

What they seem to be wanting is something different, and not something which can be done with normal git tools: to do a partial clone, ignoring some parts of the history. Some thing like "clone me that tree except blobs abcdef1 and def2abc", and being able to checkout a commit which references these blobs even though they do not exist. This might need some deep changes in the git code; AFAIK it does assume that if it has a particular commit, it has all the trees and blobs referenced by it (but perhaps not the parent commits, due to shallow clones).

A clone made with that magic git would not have the bits for all the blobs they decided to ignore, while keeping the same commit, tree, blob, and even tag hashes (the trees would simply be referencing blobs which do not exist).

Of course, this does not work when downloading from a normal git server; the remote server will create (or reuse one it already has) a highly-compressed pack with all the history for the requested commits. But after that, they could unpack and clone from *that* with their magic git, and since both sides are now the same hacked git, it will omit the bits they are scared of.

The trouble with firmware

Posted Jan 6, 2011 23:05 UTC (Thu) by bronson (subscriber, #4806) [Link] (1 responses)

It sounds like the FSF would disagree with the "except blobs abcdef1 and def2abc" bit. The user is still "obligated to refer to that nonfree code" even if it's just by SHA1.

The trouble with firmware

Posted Jan 8, 2011 3:41 UTC (Sat) by gjmarter (guest, #5777) [Link]

It seems plausible to create a git that does not store certain blobs that it claims to have. What seems more difficult is to have a git which would alter the trees so that the offending filename is not listed. How would you do that and still be able to verify the integrity of the tree against the SHA1?

I think the filename in the tree is even a bigger concern than the blob references.

The trouble with firmware

Posted Jan 6, 2011 17:14 UTC (Thu) by nye (subscriber, #51576) [Link] (4 responses)

Being slow is not the only problem. What you're suggesting is what the git documentation calls rewriting history , and it explains why doing this would make it impossible to perform merges, including updating our tree from Linus'.
The term "rewriting history" is a metaphor which expresses Torvalds' views. In effect, it claims that "If Torvalds let someone put nonfree software into a present or even past version of Linux, everyone who wants to refer to a repository of Linux is obligated to refer to that nonfree code." That is his views. We believe it is wrong to tell anyone about this nonfree software, and we will not follow his views.
I'm usually pretty sympathetic to RMS's views, but in the words of Jim Raynor, this is bullshit. This is a disgustingly cynical piece of doublespeak that would be sickening even from a career politician. He knows full well that what he's saying is a lie.

It's like some demon took over RMS a couple of years ago hellbent on driving his reputation into the dirt.

The trouble with firmware

Posted Jan 10, 2011 13:47 UTC (Mon) by jwarnica (subscriber, #27492) [Link] (3 responses)

I think, at one point, RMS was a wackjob, had interesting politics, and someone who produced useful technology. One looked first at his technology (Emacs significantly, the gnu toolset, etc), then at his politics, and didn't dig much deeper.

It is easy to like someone you don't really know.

On a personal basis, RMS stopped producing technology decades ago. And GNU/FSF is only a shadow of its ones productive self, at least WRT other groups and companies producing free software.

Politically, OSS has subverted much of the "freedom" goals, and OSS if not "free" software is fairly common these days. The war isn't won, but many battles have been. There is a big choir these days, and RMS talks, declarations, and decrees are generally targeted at the now overwhelmed choir.

So there really isn't anything else, nothing new, from RMS, except for the wackjob part.

The trouble with firmware

Posted Jan 10, 2011 14:48 UTC (Mon) by foom (subscriber, #14868) [Link] (2 responses)

I don't think the "war" is anywhere near won -- in fact I'd say just the opposite: it seems like it's in serious danger of being completely lost -- for users.

It's true that the tools *developers* depend on, for development, are overwhelmingly open source. However, the tools *users* depend on (that is: gmail, facebook, etc) are overwhelmingly closed source (even though they're often internally based heavily upon open source software).

And as more and more kinds of tools are moving to the web, that's true more and more, even for users of Linux, who previously had been using almost wholly open source software.

The trouble with firmware

Posted Jan 10, 2011 16:16 UTC (Mon) by jwarnica (subscriber, #27492) [Link]

Perhaps you are proving my unstated point. The number of fronts is expanding, and RMS is spending his time on a really very minor issue. He has really lost all perspective. Or rather, his perspective has remained the same, and once was mostly relevant.

He doesn't use the web (http://lwn.net/Articles/262570/) and I remember, I think from early Playstation 3 days, him admitting ignorances of the very concept of a gaming console. Not of Linux on the PS3, which might get a pass, but of being intentionally ignorant of the existence of non-general purpose electronic devices. He is simply out of touch with how regular people, and for that matter, regular geeks, use and interact with computers.

Basically all hardware uses microcode, and has for ever. To me, firmware is a minor, near irrelevant point. But even if I was to grant it was important, this strategy is the akin to plowing salt into your own fields and retreating from the land you have destroyed. Maybe we can shelf this point for a while, and worry about Facebook, and not OpenID, becoming the SSO/AuthN standard?

Political or spiritual advisor - maybe. Tactician, not at all.

The trouble with firmware

Posted Jan 11, 2011 3:15 UTC (Tue) by quotemstr (subscriber, #45331) [Link]

Simone de Beauvoir once said, "If you live long enough, you'll see that every victory turns into a defeat."

We're definitely experiencing a retrenchment of proprietary software, primarily in the form of "app stores" and Tivoized devices. This model has been lucrative, and is actually far more frightening than the boxed-software model that originally motivated RMS and inspires free software. Like any successful disruptive technology, the new proprietry software will grow to consume the market. It will be much more difficult for free software to gain a foothold when a few powerful vendors can not only forbid the distribution of software, but send existing programs on users' devices down the memory hole. It will be interesting to watch, if nothing else.

Why bother with Linux?

Posted Jan 6, 2011 20:42 UTC (Thu) by NAR (subscriber, #1313) [Link] (2 responses)

They have that Hurd thingy developed in the last two decades that I presume doesn't contain any evil firmware. Why don't they use it?

Why bother with Linux?

Posted Jan 8, 2011 11:47 UTC (Sat) by drago01 (subscriber, #50715) [Link] (1 responses)

Again the "evil firmware" is located somewhere else too, the system does not become "100% free" only because the non free firmware is not stored on a particular medium (hard disk).

So removing it from the kernel or changing the kernel would not result into a "100% free system" but it will create a false impression. "As long as it isn't on my hard disk it does not exist".

Why bother with Linux?

Posted Jan 8, 2011 23:19 UTC (Sat) by vonbrand (subscriber, #4458) [Link]

OK, so the FSF will have to delete said oh-so-evil firmware files everywhere, and make sure they aren't referenced anywhere. Good luck with that!

The trouble with firmware

Posted Jan 8, 2011 20:08 UTC (Sat) by marineam (guest, #28387) [Link]

What I don't understand is why making the firmware hard to find improves their "inducing the user" issue. Either way the kernel is asking for the non-free firmware. If the existence of non-free software is so offensive that the very mention of it by a piece of free software is unacceptable they why haven't they just removed all of the drivers that depend on the firmware from their kernel tree?

If a user of Linux-libre has the crazy notion that wifi is nifty then they can compile/install a "non-free" mainline kernel and Linux-libre need not even acknowledge that a website called "kernel.org" exists. This dancing around fully supporting the use of non-free firmware while pretending not to by only acknowledging it as "the-file-that-must-not-be-named" baffles me.

The trouble with firmware

Posted Jan 13, 2011 11:56 UTC (Thu) by massimiliano (subscriber, #3048) [Link] (1 responses)

Bah... a driver that cannot work without without a "non free blob" is useless in a "free kernel".

So, either they entirely remove those drivers from the source, or they simply add a sentence to the log message that warns the user, like "The kernel tried to open this file, which does not exist on this system. And BTW, the file is non-free, so if you want a free system do not install it".

If they really think that users should be free to find the "non free blobs" and install them, why make their lives intentionally harder? Just warning them should be enough!

My 2c,

Massimiliano

The trouble with firmware

Posted Jan 13, 2011 23:21 UTC (Thu) by dlang (guest, #313) [Link]

then every system is useless for you.

every CPU that you use includes a 'non free blob' of firmware (otherwise known as microcode), in many modern chips, it's even updated, and uploaded into the CPU from the kernel.

not to mention your system BIOS (unless you are running coreboot), your hard drive, your optical drive, your keyboard. just to name a few.


Copyright © 2011, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds