On binary drivers and stable interfaces [LWN.net]

There has recently been a surge of discussion, once again, on whether the Linux kernel should support closed-source drivers. The debate was driven, perhaps, by the suspicion (later put to rest) that OSDL was supporting the creation of a stable binary driver ABI for Linux. So perhaps the time has come to review the reasons why the kernel developers are opposed to closed-source drivers. Our apologies to all of you who have seen this before.

Support for binary-only drivers seems, on the surface, like it could be a good idea. Companies could provide Linux drivers for their hardware without exposing their "valuable intellectual property" to the world. Users would have a higher degree of assurance that their hardware would simply work. All of the current hardware hassles would go away, and everybody would be happy. What could be wrong with that?

One obvious problem is that, with a proprietary driver, a Linux system loses one of its best characteristics: independence from vendors. A user of a proprietary driver depends on the vendor for fixes and updates, but the vendor is under no obligation to provide them. Computing hardware has a notoriously short product life; if the vendor drops driver support when a product hits the end of its life, there is little that a user can do. If the vendor goes out of business, there will be no further support for the driver. If the vendor decides to start charging for driver updates, the user has little option but to pull out the wallet. If the driver has a bug which affects the stability of the system, only the vendor can fix it.

And history shows that proprietary drivers tend to have plenty of bugs. They are often written by developers with little time and even less expertise with the Linux kernel. The code does not go through any sort of peer review, so obvious problems will persist into the final product. And, since only the vendor can fix the driver, bugs can last for a long time.

Binary drivers are brittle. The kernel API can and does change; that aspect of the kernel is not going away. Freezing an API would limit the developers' ability to fix poor interfaces, improve how the kernel works, and remove cruft. So binary drivers will always be likely to break between kernel releases, and users will have to wait for the vendor to get around to catching up with the current API.

Linux kernel developers will not help users who have proprietary drivers loaded into their systems. That is not because the developers want to be petty and vengeful (well, perhaps one or two of them do); it is simply that the developers have no way to track down problems when closed-source code is running.

Even if a vendor offers top-quality drivers and support, it is unlikely that said vendor supports all of the architectures that run Linux. Freedom to run on something other than i386 is one of the great advantages of Linux; proprietary code takes that freedom away.

Finally, proprietary drivers may constitute copyright infringement. Certainly some developers feel that kernel modules are derived products of the kernel itself, and thus required to carry the kernel's (GPL) license. Whether the module interface constitutes a boundary which the GPL cannot cross can only, in the end, be determined by the courts. Until then, every proprietary driver carries with it a degree of legal uncertainty.

None of this is new; here's what Linus Torvalds said back in 1999:

Basically, I want people to know that when they use binary-only modules, it's THEIR problem. I want people to know that in their bones, and I want it shouted out from the rooftops. I want people to wake up in a cold sweat every once in a while if they use binary-only modules.

The alternative to cold sweats is to stick with hardware which comes with free drivers. In most areas, finding such hardware is not a challenge. In the cases where it can be a problem (video adapters, some wireless network cards), the solution is not to weigh down the kernel with some sort of set-in-stone ABI. As Linux continues to grow in popularity - and proprietary drivers get harder to write and maintain - recalcitrant vendors should eventually come around. That's exactly what has tended to happen thus far.

Index entries for this article
Kernel	Copyright issues

On binary drivers and stable interfaces

Posted Nov 10, 2005 4:08 UTC (Thu) by bignose (subscriber, #40) [Link]

Top quality presentation of the issues. Thanks, Jon; this is what I pay my subscription for.

On binary drivers and stable interfaces

Posted Nov 10, 2005 4:31 UTC (Thu) by subhasroy (guest, #325) [Link] (6 responses)

The current policy of Linux kernel maintainers is kernel developer friendly perhaps but neither user-friendly nor device driver developer friendly. I think it is impractical and nonscalable to forever carry around numerous historical driver code for all the old and obsolete devices in the kernel. I am not a kernel dev but all the usual arguments against stable ABI/API seem unconvincing to me.

I wish the kernel devs force themselves to think hard on designing a stable driver model and an API instead of going their merry ways. A driver model and API designed cleanly and thoughtfully can be published as stable. OS theory and practice is already mature enough to be able to do that. A new stable version can be published every 3 years for example. Such documented stable interface would help both open-source driver authors as well as proprietary driver authors. It should not impede innovation. Of course Linux kernel devs write code for fun and so they don't feel responsible for user's plights as much.

On binary drivers and stable interfaces

Posted Nov 10, 2005 5:52 UTC (Thu) by drag (guest, #31333) [Link] (4 responses)

Well it depends what you want.

Do you only want new kernel releases every 3 years? Because that's pretty much exactly what would happen.

For instance I like to use Ingor's 'realtime' latency patch. I wouldn't be able to use that if the last 'stable kernel abi' was released around mid-2.4.x. Stuff like that requires modifications to drivers and other related items. I need support for softirqs, changes in DMA access, make everything preemptive, etc etc.

What about power management or other things like that? It seems very obvious that nowadays that you'd want to have a mechanism for almost all kernel drivers, server-related, network-related, desktop-related, laptop-related, so that they would support more advanced power management. But how practical or plausable was that 3 years ago? And even right now there are lots of problems that need to be worked out, do you realy want no improvement in suspend-to-ram, suspend-to-sleep, and other items for the next 3 years? Thats what would happen because any improvements or changes in the code related to power management is going to have to have the support of driver-level code.

And there are other things. Like the more advanced filesystem VFS stuff and generic 802.11x protocol stack. Those things make driver developer's lives much easier and improve performance and stability for everybody, but they wouldn't be avaible to be used in new drivers with a stable ABI. Also not to mention that if you pick a stable ABI you're going to be stuck with the same bad ideas, potentionally obsolete interfaces, and such for the next 3 years.

But we'll see. OpenSolaris people love stable ABI. They like stable kernel interfaces and have no problem using closed source drivers. Once they figure out the legal hurdles of GPL software on CDDL base then the only real difference between 'Linux' and 'Solaris' will be the kernel and some other low-level stuff.

Personally I feel that the only real solution for a 'stable abi' is a Microkernel, or at least a very microkernel-like setup were you run modules in userspace seperate from the kernel itself.

So in the future hopefully it will be easy to select (just picking on debian for right now) Debian/GNU/Solaris or Debian/GNU/Linux or Debian/GNU/Hurd. Then it'll be easy for us to pick 'stable kernel ABI' vs 'rapid developement kernel' vs 'microkernel' and get the 'best tool for the job' irregardless of the actual software we want to run on it. Then the best kernel developement method will win.

I like Linux. It's fast, generally stable, and improves rapidly. It gives me all sorts of neat stuff to play with, but I have no very special attatchment to it. I do have strong attatchments to the Free software I use everyday, however. If something better comes along and is Free software like Linux, Mozilla, or BSD is then I'll use it.

(Although actually, I would like to see a specialized Free Software Single-User OS just for simple desktop and embedded applications. Something along the lines of BeOS or Windows 95 (obviously stable of course). Not every system needs to be able to run a entire Unix enterprise-ready enviroment.)

On binary drivers and stable interfaces

Posted Nov 10, 2005 6:47 UTC (Thu) by cventers (guest, #31465) [Link] (1 responses)

Just addressing your last point about a single-user OS.

Convex predicted many years ago that the market for specialty computer
hardware would diminish as technology moved forward.

I see the same thing with software, and I think it's really evident. Why
have a special wireless-router OS when it can run Linux? Wireless routers
certainly don't need a full "enterprise UNIX" - VFS, etc...

The stock 2.6 kernel already runs on everything from embedded devices
like wireless routers, TiVOs, cell phones to huge supercomputers, and
save for some places it hasn't fully innovated yet, it shines on every
platform. I'd rather have all the interesting work going into one or a
few open source operating systems than many specialized ones. By having
one Linux kernel, for example, you have lots and lots of developers
working on one product. Compare that to (and please don't let this turn
into a flame war) BSD, where you have OpenBSD, NetBSD and FreeBSD all
working towards somewhat different goals and quite obviously *not* moving
at the pace of Linux.

On binary drivers and stable interfaces

Posted Nov 11, 2005 2:03 UTC (Fri) by bk (guest, #25617) [Link]

I agree. The single-user OS idea is valid and a good niche to explore, however there's no reason why the Linux kernel can't be part of it. Linux is a *general purpose* kernel, not a Unix operating system in and of itself.

It's a problem of userspace engineering. Although I'm going to be booed for saying it, Lindows (or whatever they're calling it now) tried to fulfill the ideal of a simple consumer desktop OS and got nothing but skepticism from the majority of people. It was single user (root), simple, largely non-configurable, point-and-click everything and so on. You could argue the *implementation* of the idea was sub-optimal (I would agree), but that says nothing about the merit of the core ideal.

Perhaps Lindows didn't go far enough; it still exposed enough of its Linux underpinnings that people reviewed it as if it were just another Linux distribution. Linux 2.6 has the facilities to completely replace the classic Unix user/group security scheme, if one had a lot of ambition and venture capital another attempt at a mass market single-user OS might be worthwhile.

On binary drivers and stable interfaces

Posted Nov 10, 2005 13:59 UTC (Thu) by wookey (guest, #5501) [Link] (1 responses)

That was a generally excellent post (thank you) spoilt by one little thing which I feel compelled to nitpick: There is no such word as 'irregardless'. You mean regardless, or irrespective.

OT: non-standard, and American, but still a word

Posted Nov 11, 2005 18:08 UTC (Fri) by illtyd (guest, #2124) [Link]

irre'gardless, a. and adv.
Chiefly N. Amer.

In non-standard or humorous use: regardless.

[Prob. blend of irrespective and regardless.]

912 in WENTWORTH Amer. Dial. Dict. 1923 Lit. Digest 17 Feb. 76 Is there such a word as irregardless in the English language? 1934 in WEBSTER (labelled Erron. or Humorous, U.S.). 1938 I. KUHN Assigned to Adventure xxx. 310, I made a grand entrance and suffered immediate and complete obliteration, except on the pay-roll, which functioned automatically to present me with a three-figure cheque every week, irregardless, as Hollywood says. 1939 C. MORLEY Kitty Foyle xxvii. 267 But she can take things in her stride, irregardless what's happened. 1955 Publ. Amer. Dial. Soc. XXIV. 19, I don't think like other people do and irregardless of how much or how little dope would cost me [etc.]. 1970 Current Trends in Linguistics X. 590 She tells the pastor that he should please quit using the word irregardless in his sermons as there is no such word. 1971 M. MCSHANE Man who left Well Enough iv. 96 The sun poured down on Purity irregardless of the fact that it received no welcome.

from the OED online
(note that quite a few of the textual references are quotes suggesting it isn't a word, but then I'm not a linguistic prescriptivist)

On binary drivers and stable interfaces

Posted Nov 10, 2005 22:25 UTC (Thu) by hazelsct (guest, #3659) [Link]

There's a lot to pick on in your post, but I'll focus on "historical" hardware. Tons (thousands of tons?) of perfectly good electronics hardware goes to waste because of dropped software support. The Linux model prevents that from happening; from PPC Macs which will all be drop-kicked by Apple in a couple of years, to older PCI cards which vendors haven't supported in Windows since NT.

Where have you seen drivers for old hardware break, or slow down development, under Linux? Did you report the bug? I've never seen it, and I have some hardware as old as a 1990 Amiga 2000 and a 1997 Motorola StarMax (603e Power Mac clone) which still run Debian just fine -- long after the vendors dropped support. (Modern GNOME/KDE desktop performance is even quite good on the StarMax.) I really don't see the problem for users here, and in fact, freedom from planned obsolescence is a huge benefit *for* Linux users.

Put somewhat differently, there's far too much electronics scrap in the world as things stand. Do you really want to accelerate the disposal of lead-tainted hardware -- and prevent schools, developing nations, and poor entrepreneurs from using perfectly good computers -- just because the vendors of new equipment want it to be so?

Truly Free drivers could use a stable API too

Posted Nov 10, 2005 5:00 UTC (Thu) by elanthis (guest, #6227) [Link] (8 responses)

Let's take a really simple situation:

1) User has Distro 4.0 installed on their machine, which has kernel 2.6.14.
2) Distro 4.0 will never provide an update to any other kernel version, because kernels need to be tested and patched and stabilized and all that effort is going into Distro 4.1 to be released in six months.
3) New hardware comes out, and it has a driver which has been included into the kernel... versions 2.6.16 and onward, anyways.
4) User buys said new hardware but has no way to get it to work without manually downloading, possibly patching, and installing the kernel, and also thereby giving up automatic security updates from the vendor.
5) User is screwed.

A stable ABI isn't really necessary, I'll note. A stable API is. With a stable API, at the very least, a tool like DKMS can do its magic and users can pop in that new GPLd driver for that new hardware, wait a couple seconds while a nice GUI install tool shows a little progress bar while GCC compiles the driver, and then have working hardware.

There are cases for stable ABIs even with purely GPLd drivers, mostly for installer needing to write to disks that lack drivers for the controller in the install image, but I suppose installers could start offering minimalistic in-memory compiler toolchains to work around that.

Today's Linux userbase can probably deal with shopping around for hardware that's several months/years old to ensure their bleeding-edge development distribution will support it, but the mainstream market just won't accept that. I'm not personally sold on the idea that we have to get the mainstream market to accept it, but there are vendors who do think so (Canonical, Linspire, etc.), so for their sake I hope the kernel developers can find some kind of solution.

I guess another way of putting it is that the kernel needs better support for out-of-tree (even 100% GPL) drivers, because in-tree drivers aren't always available to users running older systems.

Stable interface

Posted Nov 10, 2005 6:43 UTC (Thu) by ncm (guest, #165) [Link]

There's already a stable ABI (for ia32, anyhow): it's called NDIS. It's not very satisfying. There's little reason to think a stable API would be much more satisfying, however much it also succeeded in inconveniencing proprietary-driver maintainers.

Truly Free drivers could use a stable API too

Posted Nov 10, 2005 9:06 UTC (Thu) by khim (subscriber, #9252) [Link] (5 responses)

You can shorten your tirade to this:

1) New hardware comes out, and it has a driver which has been included into the kernel... versions 2.6.16 and onward, anyways.
2) User is screwed.

Why ? Easy: it's common practice to only support "certified hardware" - and I fail to see why it's so much easier for vendor to certify some alien driver then it is to certify 2.6.16 (or to backport driver to vendor-patched 2.4.14).

This is not a hoax, you know, this is from server on our work:
$ uname -a
Linux mccme.ru 2.6.14 #1 PREEMPT Sat Nov 5 00:00:31 MSK 2005 i686 unknown
$ cat /etc/redhat-release
Red Hat Linux release 6.2 (Zoot)

I fail to see the need to separate drivers and kernel. Supported drivers - are included in kernel, the way to get new drivers is to install new version of kernel. All other drivers are unsupported and experimental - use on your own risk, but YMMV. 3 years is clearly not enough (system discussed above was installed back in 1997 and while moved two times from one hardware set to another is still in active use). Interfaces between kernel and userspace are stable enough, in-kernel interfaces are in constant flux. This is how it always was, why should we change this ?

Truly Free drivers could use a stable API too

Posted Nov 10, 2005 10:09 UTC (Thu) by simlo (guest, #10866) [Link] (4 responses)

To install a new kernel you have to reboot - very Windoze like :-(

I totally agree with the long comment above: What is needed is a "micro-kernel" approach. Drivers should be able to run in user-space. The stable driver API should be in user-space. Hardware vendors can use that to write their close source drivers. That way experiments like Ingo's realtime-patch would still work since no closed source module have access to any kernel internals - only a few system calls. If hardware vendors complain over performance: Release the driver under GPL and move it into the kernel.
If a company do not want to do that, someone could trace what is going on between the driver and the kernel and thus more easily write a free driver.

I have dropped the smb/cifs drivers in the kernel and use smbnetfs which is build on top of fuse. Now a bug in the smb implementation can't crash my machine. I can even use Kerberos as authentification. Moving stuff into userspace _works_ - at least sometimes. Please, don't go into religious war between microkernel vs. monolitic kernels. As I see it Linux should be a _mixture_ using both techniques where they fit best weighing performance, stability, ease of development, flexibility and licensing issues.

Truly Free drivers could use a stable API too

Posted Nov 10, 2005 15:19 UTC (Thu) by kfiles (subscriber, #11628) [Link] (3 responses)

> To install a new kernel you have to reboot - very Windoze like :-(

Well, to install a new card you have to reboot too, so it doesn't seem so strange to reboot to a new kernel at the same time. :)

Now for USB stuff and network protocols, DKMS hotpluggable drivers certainly seem more appropriate. I have installed 3 different versions of the PPTP module via DKMS without rebooting.

Thanks,
--kirby

kexec()

Posted Nov 10, 2005 18:26 UTC (Thu) by AnswerGuy (guest, #1256) [Link]

Depending on your precise definition of "reboot" it's also possible with newer kernels to run the kexec wrapper around the kexec() system call
and thus load a new kernel without a firmware reboot. (Also available
with the old 2-kernel monte patches and utility).

Of course all user processes are killed through a kexec() ... and all
devices are re-initialized by the new kernel. Thus this is a "reboot"
in that sense of the term.

Combine that with some Mosix and Xen like process checkpointing and suspension and you could very well devise a Linux system that would not
have to be "rebooted" in order to switch in a new kernel.

But it's quibbling to compare the necessity of rebooting into a new kernel
with the infamous reboots required for trivial system settings in earlier versions of MS Windows. I used to have a poster with a genuine appearing MS Windows GUI dialog that read:

You have moved your mouse.
A system reboot is required for this change to take effect.
[Ok]

JimD

Truly Free drivers could use a stable API too

Posted Nov 10, 2005 22:52 UTC (Thu) by Duncan (guest, #6647) [Link] (1 responses)

>> To install a new kernel you have
>> to reboot - very Windoze like :-(
>
> Well, to install a new card you
> have to reboot too[.]
>
> Now for USB stuff and network protocols[...]

Actually, for hotpluggable stuff like USB, there are kernel modules.
Kernel modules can be compiled separately, against the codebase used to
compile the running kernel, and can then be modprobed, without rebooting.

As for closer to the core hotpluggability, memory and CPUs and the like,
the kernel isn't quite there yet, but support is actively being worked up
and is already there to some degree. For this sort of support, modules
aren't generally needed, because support for a base level will be compiled
in and additional CPUs or memory use the existing support.

Of course, one still has to reboot (or kexec) to a new kernel if switching
to hardware of a different architecture. However, I don't know if
anything supports that, and the call for such support isn't that great,
anyway, tho it could be useful in certain high availability situations.

BTW, the user mode drivers comments someone made are a good idea, as well.
There's nothing stopping that (tho more integrated/direct kernel support
in some cases would be very useful, think fuse implementation before and
after the kernel support for it), and indeed, the kernel's GPL preamble
specifically states that the restrictions of the GPL do NOT apply to
userspace, so binary drivers are with no controversy completely legal
there (altho there remains disagreement of whether they are /moral/, but
it's the usual controversy of whether /any/ binary-only code is /moral/).
The problem of course is the performance bite. It's fine in some cases,
particularly where there's direct kernel support for the userspace drivers
as with fuse, but would suck for something like NVidia's video drivers.
Very good point made, none-the-less.

Duncan

Truly Free drivers could use a stable API too

Posted Nov 11, 2005 1:48 UTC (Fri) by giraffedata (guest, #1954) [Link]

Actually, for hotpluggable stuff like USB, there are kernel modules. Kernel modules can be compiled separately, against the codebase used to compile the running kernel, and can then be modprobed, without rebooting.

In this thread, we're talking about unstable APIs. You probably cannot just separately compile the loadable kernel module against the currently running kernel because the base kernel / device driver LKM interface has changed.

The original poster makes the point that with a stable API, your method would work. But without one, you need to upgrade your entire kernel in order to use a new loadable kernel module. And a reboot is just one of the pains of doing that.

User space drivers solves the problem only if we assume that the user space interface is stable like user space interfaces we know today.

Truly Free drivers could use a stable API too

Posted Nov 17, 2005 19:11 UTC (Thu) by oak (guest, #2786) [Link]

I would say this to be an exellent reason to backup /home/, change to a
better distro and restore the backup + do a bit of reconfiguring.

Or maybe start using Knoppix and /home/ on a USB-stick...

On binary drivers and stable interfaces

Posted Nov 10, 2005 7:50 UTC (Thu) by bangert (subscriber, #28342) [Link]

this issue coming up again and again shows one thing in particular:
vendors are looking at writing drivers for linux (and at an easy way out).

good stuff - keep them coming

On binary drivers and stable interfaces

Posted Nov 10, 2005 10:50 UTC (Thu) by Quazatron (guest, #4368) [Link] (10 responses)

I think that the OSS community does not need the vendors to provide GPL drivers.

If the vendors want to protect their "valuable intellectual property", all they have to do is to publish the specifications of the product, how to interface with it. The OSS community can then write the GPL driver.

That way, both sides win. Users get solid drivers, vendors get happy costumers and don't need to spend a cent to suport OSS OSes.

Of there's that other problem of the drivers that have to load firmware into the devices before working, but I think that could be solved with a combination of userspace firmware loading tools and GLP drivers. That would also require the vendors to allow distribution of the (closed) firmware binaries.

On binary drivers and stable interfaces

Posted Nov 10, 2005 11:33 UTC (Thu) by drag (guest, #31333) [Link] (6 responses)

The thing about firmware (also known as microcode, which is more familar to some people) is that it's not 'software' so much. It's code alright, but it's not designed to actually do anything except run on hardware's proccessor/dsp/whatever.

It's completely independant from the OS and any hardware platform. For all practical purposes it is part of the actual hardware on whatever card it's used for. That's why Linux is able to have firmware images in it's source code tree and not violate the GPL, even though if you look at it a certain way the firmware is 'closed source'.

This is why firmware is different from a 'binary blob' that is used by stuff like madwifi and nvidia drivers. This 'binary blob' is system specific, written in a regular programming language, runs on the regular cpu and is platform specific. For instance I can't run nvidia drivers or madwifi drivers on my powerpc computer, but with firmware it will work fine.

Realy though... manufacturers should have the firmware on the actual device on non-volitile ram or rom or whatever instead of having the OS load it via the drivers. It's a bit of a hack to save a bit of money, but it makes things more inconvienent for their customers.

On binary drivers and stable interfaces

Posted Nov 10, 2005 18:33 UTC (Thu) by arafel (subscriber, #18557) [Link]

Leaving aside the rest of it, I think you'll find it's not usually to save memory so much as it is to allow firmware updates to work around bugs, add new features etc. Harder to do that in ROM; they could use flash, I guess...

On binary drivers and stable interfaces

Posted Nov 11, 2005 4:30 UTC (Fri) by Ross (guest, #4065) [Link] (1 responses)

Microcode is actually something different -- it is the glue on top of the lowest levels of hardware to create the "native" machine language instruction set. Some firmware is microcode and some is not. In many cases, microcode can't be "uploaded" at all (though that isn't the case with Intel CPUs). Also, nothing actually limits firmware to "just" driving the hardware. Entire Linux distributions are loaded as firmware in some cases.

what is microcode?

Posted Nov 11, 2005 19:33 UTC (Fri) by giraffedata (guest, #1954) [Link]

Thanks for making a point that needed to be made.

We should clarify though, that people (IBM engineers are the only ones I've heard do it) do often use the term "microcode" to refer to any software built into a device. They're all wrong.

Also for the sake of precision, microcode isn't just for CPUs. Any machine can be driven by microcode -- it's really just a style of software; a class of instruction set. In days when CPUs were expensive, many machines were implemented with microcode. I don't know how much microcode is in use today, but I imagine communication processors (e.g. inside a network switch or TCP Offload Engine) get some use out of it go faster than any conventional CPU could go.

I found a pretty good definition at http://www.webopedia.com/TERM/M/microcode.html .

"firmware," unfortunately, isn't the right word either. That just means instructions that are somewhat permanent, as in a ROM chip. The permanence isn't really what we're talking about here; in fact, I think we're specifically talking about software that has to be loaded every time the device powers up. I'd just call it "the device control program."

On binary drivers and stable interfaces

Posted Nov 11, 2005 14:22 UTC (Fri) by N0NB (guest, #3407) [Link] (2 responses)

As I understand it, not all firmware or binary blobs exist solely because a hardware manufacturer is unwilling to release specs for the interface. As I understand it, in the USA at least, the FCC mandates that wireless devices be tamper proof. A user is prohibited by law from tweaking such things as RF power, carrier frequency, or the coding protocol. To get a device certified for sale by the FCC the control interface must be restricted to only those the FCC allows.

So here we have the Free Software philosophy and US law at odds with each other (not for the first or only time). How can this be resolved?

I have two laptops with Atheros based 802.11b/g adapters. Madwifi works very well even though I must build the package manually on my Debian systems. The kernel complains of "tainting" when loading the modules and otherwise works flawlessly.

Do I understand correctly that the binary blob of Madwifi is code the CPU executes? Yet it still is controlling the Atheros chipset. I guess the only difference is that it occupies the main memory of my computer rather than being loaded into RAM on the device, correct? Still, I don't see how the blob affects the kernel directly as it doesn't seem to be a part of it. The wrapper is definitely connected to the kernel and it talks to the blob.

It seems the real issue is the location of the firmware code. Why is it bad if the proprietary blob happens to occupy main memory and is executed by the CPU, but things are just fine if that same blob were encoded onto the firmware and the kernel is controlling the hardware in the same way?

I'm not trolling. I genuinely want to know the difference. With Madwifi it appears we have a manufacturer working as well as they can with the kernel while adhering to USA law. I don't think they are necessarily "tainting" the kernel.

On binary drivers and stable interfaces

Posted Nov 11, 2005 21:07 UTC (Fri) by dlang (guest, #313) [Link] (1 responses)

no, the firmware blob is what in older devices would be in the ROM soldered to the device (which nobody object to useing)

it's not just a simple matter of getting specs of the device to recreate this, and it's definantly not something that is likly to run on any other device (even a nre revision of a card will frequently require a different version of the firmware)

vendors do this for a couple reasons.

1. it's cheaper to leave the ROM/Flash off of the card.

2. it's more flexible as it allows for updates and bugfixes

as long as the API to talk to the device once the firmware is loaded is documented people should not have much grounds to fuss about it.

On binary drivers and stable interfaces

Posted Nov 16, 2005 13:33 UTC (Wed) by zblaxell (subscriber, #26385) [Link]

"no, the firmware blob is what in older devices would be in the ROM soldered to the device (which nobody object to useing)"

Actually, I object (although in practical terms my objection is mostly meaningless since I use them anyway ;-).

Let's assume for example that I wanted to conduct covert surveillance of someone else's machine, and I don't want to get caught. I'd want to be able to bypass any software they might have on the machine that might detect such surveillance, or worse, put a stop to it. So viruses, worms, and trojans targeted at the main OS won't work--they'll all be caught by the usual rootkit detection methods.

Solution? Attack some software that is running in the machine, but not in the OS. Even better if that software can transparently access some sort of communication device, so I can receive data from it. Best of all if I don't have to physically modify the target machine to do it.

What better place to find such software than in a network device? A wireless card would be even better--with a wired network card, my victim could plug their machine into a LAN where I don't control all of the network devices, and see a bunch of data unexpectedly flowing somewhere with traceroute. A wireless card could quietly radiate data that I could pick up with a suitable long-range antenna.

A bus-mastering card has routine access to the entire system RAM through DMA, since Intel CPU's don't have an IOMMU--although in practice I don't see why even having an IOMMU would necessarily be a barrier unless the user also uses a fully virtualized OS. From a bus-mastering card under my control, I can remotely browse through my victim's RAM and retrieve encryption keys, cleartext, and other goodies.

Some other devices, like video outputs connected to a CRT, can do a similar function--modulate the hsync signal slightly, and a CRT will broadcast a fairly powerful signal (although it may be so powerful that it resonates audibly in nearby bits of metal). Disk device drivers can squirrel away data in the HPA or other parts of the disk, then pretend that the disk has failed so the drive can be captured on its way to warranty replacement (even better if the disk itself can participate in the pretense--that way all of the diagnostic utilities--free or proprietary--will present a consistent view of disk failure, until I get the disk and send it my covert-data-retrieval key). ACPI BIOSes can do some *very* interesting things with proprietary PCI bridge interfaces and various OS callbacks. The threat is not just from network device firmware.

On binary drivers and stable interfaces

Posted Nov 11, 2005 1:57 UTC (Fri) by giraffedata (guest, #1954) [Link] (2 responses)

If the vendors want to protect their "valuable intellectual property", all they have to do is to publish the specifications of the product, how to interface with it. The OSS community can then write the GPL driver.

So you're assuming the valuable intellectual property is the driver software (copyright)?

It's not. The company doesn't sell drivers. The valuable intellectual property is the trade secrets about how the device works. Publishing specs would be worse than shipping source code in that respect.

On binary drivers and stable interfaces

Posted Nov 12, 2005 19:38 UTC (Sat) by Ross (guest, #4065) [Link] (1 responses)

Revealing the programming interface for their device is not the same as publicly documenting how their device works. The more compliated the device the less similar that information is.

The only plausible excuse I've heard is that it allows other companies to make work-alikes without having to write their own drivers or have them certified by Microsoft.

On binary drivers and stable interfaces

Posted Nov 12, 2005 20:26 UTC (Sat) by giraffedata (guest, #1954) [Link]

Good point. I was vague when I said the trade secret in question is "how the device works." The trade secret is the programming interface, and your explanation is the best I've heard for why the company perceives that secret as valuable.

On binary drivers and stable interfaces

Posted Nov 10, 2005 11:58 UTC (Thu) by hanwen (guest, #4329) [Link] (2 responses)

I think the issue with binary drivers revolves around the wrong point. The real problem is

proprietary drivers tend to have plenty of bugs. They are often written by developers with little time and even less expertise with the Linux kernel

No amount of standards, conventions, ABIs, APIs, etc. is going to solve this. Having a driver ABI just makes it easier for vendors to put crappy drivers up, and add a cute Penguin logo to their packaging.

On binary drivers and stable interfaces

Posted Nov 10, 2005 13:12 UTC (Thu) by fenrus (guest, #31654) [Link] (1 responses)

this wasn't so much about proprietary drivers.. those are illegal anyway in many of the existing cases.

the point was about drivers shipped in binary form (even though the source may well be available). Some vendors only want to ship "certified" binaries.. even when they are older and more buggy than newer, not yet certified drivers

On binary drivers and stable interfaces

Posted Nov 10, 2005 15:04 UTC (Thu) by coopj (subscriber, #1139) [Link]

I often have to deal with drivers which while written under GPL, are
very poorly written. Sometimes they may perform well enough, and
not be very buggy, but they use all kinds of hacks to work with
only one or a few kernel releases, and are almost impossible to
port forward to even minor changes in kernel API and headers.

Overall I think this is a good thing -- by releasing under GPL
they at least give kernel developers a start and an opportunity
to push a better driver upstream. However, I wish rather than
producing these hacks they'd put a little more effort into
respecting kernel practices and standards -- in many cases it
would be less work, not more.

The "you can do anything you want as long as it is GPL" attitude
is very widespread in the corporate culture, and while it is
better than promotion of binary drivers, further education is needed.

It's the Freedom stupid!

Posted Nov 10, 2005 12:20 UTC (Thu) by gallir (guest, #5735) [Link]

OK, nice explaination of why FOSS drivers are desired and better than
close one. Good legal argument, the kernel is GPL and binary drivers
could violate the license. It's all nice.

But there is something missing in the article and also obviously ignored
or misunderstood by some comments' authors.

Why the kernel ios free? why it is GPLed? There are two explicit reasons
among the community of kernel developers:

1. Those that think, like Linus Torvalds, that FOSS is the way to get
better OS and software in general. They thinks that developing free
software in a community is fun.

2. Those that think, like Alan Cox, that proprietary software lead to a
moral dilemmas and that the free software is THE way to avoid those
dilemmas and the exclusionary relationships established around
proprietary software. They tend to think that developing free software in
general and a OS kernel in particular is also fun.

Why should kernel developers spend efforts to establish an API that limit
thenselves just for making easy the coexistence of Linux and propietary
drivers if the latter are a clearly againsts the [two] reasons of the
very existence of Linux?

Perhaps it's better explained by
http://geekz.co.uk/lovesraymond/archive/gpl-3-democracy

On binary drivers and stable interfaces

Posted Nov 10, 2005 17:14 UTC (Thu) by iabervon (subscriber, #722) [Link]

The kernel's always had one stable ABI, which has gotten only minor if any incompatible changes ever: the interface to userspace. There's also been a push towards moving more and more capabilities into userspace. The USB scanner on the computer I'm sitting at only needs userspace drivers, for example. Jeff Garzik was just talking about block device implementations in userspace, there's discussion of filesystems in userspace, TTYs should be pretty easy, since there aren't so many deadlock possibilities, and most other stuff isn't done with a kernel API, so it's looking good on the regular-userpsace-API side. USB devices can already be handled from userspace, and I think that SCSI devices can mostly be.

It seems to me that it would be good to have a stable ABI for drivers, where that ABI doesn't let you mess with things you shouldn't, doesn't let you arbitrarily mess with the kernel, and isn't used by in-kernel drivers (although it may be used by open-source projects that want to, like hpoj currently). This obviously won't work well for all types of device, because interrupt latency won't be great, but it should suffice for letting binary drivers exist much like binary executables do. It also means that all of the interaction between the driver and the hardware is going through kernel code, so you don't need to probe the bus to find out exactly what the driver does to make the hardware work.

Some quibbles

Posted Nov 14, 2005 23:29 UTC (Mon) by sepreece (guest, #19270) [Link] (1 responses)

I don't have any trouble with agreeing that open-source drivers are preferable. However, I think you overstate some of the issues you raise:

The statement, "The code does not go through any sort of peer review, so obvious problems will persist into the final product." is generally untrue. Most mature software organizations do peer reviews. I suspect you mean they don't do open, external peer reviews, but the phrase "peer review" has never implied that.

You say "it is simply that the developers have no way to track down problems when closed-source code is running." However, many OS vendors do, in fact, provide support for customers running code they are unwilling to share with the vendor. It's harder and demands more of the customer, but it's part of the job.

"Even if a vendor offers top-quality drivers and support, it is unlikely that said vendor supports all of the architectures that run Linux." But, of course, the kernel maintainers don't support all the architectures that run Linux, either. And if you're far enough off the beaten track, you're pretty much on your own anyway. A binary driver at least MAY have a vendor behind it that can offer support. This is particularly the case for those of us in the embedded space.

In fact, a lot of your comments are general-purpose, desktop-centric. For those of us who build products with Linux inside, hardware changes are largely irrelevant - the hardware will continue to be whatever it ships on. And the API/ABI won't change, either, because we won't let it. This doesn't reduce the validity of your points for desktop and enterprise users, but it does restrict the scope of your claims. We mostly come from using OSs that change infrequently and in big steps and are generally OK with that model, because product cycles can start fresh with whatever is the current release of the OS and, if necessary, do the upmerging needed to make any reused components work on it.

Again, though, I agree that open-source drivers are preferable...

Some quibbles

Posted Nov 18, 2005 16:02 UTC (Fri) by beoba (guest, #16942) [Link]

Minor nit to pick:

I suspect you mean they don't do open, external peer reviews, but the phrase "peer review" has never implied that.

I dunno about the realm of CS, but in the scientific community, that's exactly what it implies.