Weekly Edition
Return to the Kernel page
| |||||||||||||
On multi-platform drivers
LWN recently looked at the discussion around
moving the Broadcom wireless driver into the mainline from the staging
tree. This driver raises a number of issues on how the kernel community
interacts with hardware manufacturers. One important aspect of the
discussion, though, did not come up until after that article was written.
Linux drivers are expected to be drivers for Linux, only. Attempts to
maintain a Linux driver as a multi-platform driver will lead to
unhappiness, for a number of reasons. What follows is an unabashedly
partisan article on why multi-platform drivers do not fit well with the
Linux kernel.
Broadcom developer Henry Ptasinski brought the issue to the fore while talking about why the company was not interested in supporting the in-mainline b43 driver:
The brcmsmac driver has architectural alignment with our drivers
for other operating systems, and we intend to to enhance and
maintain this driver in parallel with drivers for other operating
systems. Maintaining alignment between our Linux driver and
drivers for other operating systems allows us to leverage feature
and chip support across all platforms.
To developers who have worked with the kernel for a while, these words look like a fundamental mistake. To others, though, they seem reasonable: if Broadcom wants to support its hardware, why not let the company do things its way? One clear problem with trying to maintain "architectural alignment" with drivers for other operating systems is that only the original company can maintain that alignment. The other associated drivers are almost certainly not open source; nobody else in the community has any way to know which changes are consistent with those other drivers and which are not. Not even the relevant subsystem maintainer can make that kind of call. One also must consider that most other kernel developers have no motivation for - or interest in - maintaining the correspondence between the drivers, even if they did know how to do it. The obvious conclusion here is that allowing a vendor to maintain a multi-platform driver in the kernel tree will only work if that vendor is given absolute control over the code. If others can make arbitrary changes, there is no way for the vendor to keep the drivers consistent. But, in the kernel, nobody has that kind of absolute control with the possible exception of Linus Torvalds. If something needs to be fixed or changed, anybody with the relevant technical skills can do it. If a piece of the kernel tree were to be fenced off and made off-limits for kernel developers, the kernel as a whole becomes a little less free. And that freedom matters. Consider the problem of internal API changes. As anybody who watches kernel development knows, internal interfaces are changed all the time in response to problems and changing needs. Those changes can, at times, force significant changes in users of the affected interfaces. Contemporary rules call for a developer who makes an interface change to fix any code broken by that change. Code that has been designated as off limits will be hard to fix in this way, slowing down the evolution of the kernel as a whole. As one example, consider the removal of the big kernel lock; that job required significant locking changes in many places. Literally hundreds of drivers were modified in the process. Impeding those changes would have made the BKL removal task even slower - and maybe impossible. Manufacturers are not known for long-term support of their products; they have no real reason to support five-year-old chipsets that they no longer sell. Indeed, they have every reason to end such support and encourage the replacement of older hardware with shiny new stuff. Linux, instead, tends to support hardware for as long as it is in use. Giving a vendor absolute control over a driver is certain to create conflict when that vendor moves to drop support for older chipsets. A vendor's agenda can differ from the community's needs in other ways as well. Vendors may not appreciate patches to enable undocumented features or make low-end offerings behave more like their more expensive alternatives. Or consider Hans Reiser's opposition to the addition of extended attribute and access control list support to reiserfs. His argument was that users should wait for the shiny new Reiser4 filesystem to obtain such features; had he been listened to, reiserfs users never would have had support for those basic filesystem capabilities. The kernel works well because it is maintained as the best kernel for all users over the long term, even if that occasionally causes conflicts with short-term vendor desires. Multi-platform drivers from vendors tend to be written around the minimal set of support functions that are available on all platforms. The result is a lot of code duplicating functionality already found in the Linux kernel; consider, for example, just how many wireless drivers initially came with their own 802.11 stacks built in. Developing and maintaining just one rock-solid 802.11 implementation is hard; having several of them in the kernel would bloat the kernel and endow it with a number of second-rate implementations - all of which must be maintained into the future. Other kernel support code - from simple linked lists through complicated memory management - is also often avoided by multi-platform drivers. Those drivers will be bigger, buggier, and harder for kernel developers to read and support. They are also much less likely to behave consistently with other Linux drivers for the same type of hardware. Beyond all of the above, it is also far from clear that maintaining a multi-platform driver actually saves any work. Drivers written for Linux can make full use of the available support infrastructure. Multi-platform drivers must, instead, duplicate much of that functionality and maintain an operating system abstraction layer as well. Maintaining a multi-platform driver means maintaining a larger body of code without help from the community. In summary: trying to maintain a single driver for multiple operating systems may look like a good idea on the surface. But it is only sustainable in a world where the vendor keeps complete control over the code. Even then, it leads to worse code, duplicated effort, long-term maintenance issues, and more work overall. Linux works best when its drivers are written for Linux and can be fully integrated with the rest of the kernel. The community's developers understand this well; that is why multi-platform drivers have a hard time getting into the mainline. (Log in to post comments)
Being upstream to multiple projects Posted Sep 9, 2011 7:56 UTC (Fri) by geofft (subscriber, #59789) [Link]
Even in the ideal world where the other OSes were free software (maybe they're working on FreeBSD and OpenSolaris drivers, too) and the entire driver development were in the open with no worries about undocumented features, this situation is not handled particularly well. If they have a fix to the driver, they want it to go into the largely-equivalent code that exists on all OSes. Meanwhile Linux refactorings should be Linux-only.
How do you structure your repositories to do that? I don't think git will handle this situation well, today.
On multi-platform drivers Posted Sep 9, 2011 15:23 UTC (Fri) by josh (subscriber, #17465) [Link]
Some successful multi-platform drivers do exist. For a large example, consider ACPICA, a portable implementation of ACPI. While it runs on many different platforms, it also goes to a great deal of effort to follow the native conventions of those platforms.
On multi-platform drivers Posted Sep 9, 2011 20:12 UTC (Fri) by utoddl (subscriber, #1232) [Link] This is an overreaction and misinterpretation. Why should "architectural alignment" necessarily mean iron-fisted control of the code? Rather, it offers a hope that by the time new hardware makes its way to retail shelves people running modern kernels will find that it Just Works out of the box. It also could mean that issues found and fixed by those krazy Linux kids may improve the vendor's overall product line, thus rewarding financially a company who's willing to make the effort to play by our rules."Architectural alignment" does not suggest to me that Broadcom would be less than accepting of changes that make make their Linux driver or support better. They've obviously been graciously taking suggestions from established kernel devs for the last year in the way we would hope. I see no reason to expect less from them in the future. They've spent the effort to learn how to work "with" vs. "in spite of" the Linux Way. Good on 'em. Turn the argument around: Why would a vendor and hardware manufacturer be in any worse position to maintain -- in coordination with a highly competent kernel development community -- drivers for current and future hardware that they understand fully than a handful of developers working in their spare time with incomplete specs and spurious bug reports? This looks to me like a company trying hard to provide exactly the kind of support that others have been criticized for not providing for years. Kudos to Broadcom. This may be the ideal model for companies to work with the community; time will tell. I'm willing to give it a chance before declaring it an untenable approach.
On multi-platform drivers Posted Sep 10, 2011 20:46 UTC (Sat) by alecs1 (guest, #46699) [Link]
These are mostly my thoughts too, but the article is useful in explaining the logic behind kernel development way of thinking, not necessarily an overreaction.
If the Broadcom driver gets into the kernel now based on merit, it doesn't necessarily mean the vendor will be able to do whatever it wants in the future. Broadcom is not the Linux kernel gatekeeper. If things don't clean up between the main actors in this, I trust Linus will make the correct compromise.
On multi-platform drivers Posted Sep 12, 2011 21:03 UTC (Mon) by michaeljt (subscriber, #39183) [Link]
> If the Broadcom driver gets into the kernel now based on merit, it doesn't necessarily mean the vendor will be able to do whatever it wants in the future. Broadcom is not the Linux kernel gatekeeper. If things don't clean up between the main actors in this, I trust Linus will make the correct compromise.
Right. It seems to me that the main problem raised in the article (that kernel people will want to be able to make changes without consulting Broadcom first) is something Broadcom will have to deal with, not something that should be much of a problem for other kernel developers. Broadcom have made their code GPL, and there is always the - pretty easy - option of forking maintenance of it if they don't play the way other kernel developers would like, so it seems reasonable to give them a chance. If their multi-platform driver maintenance works out it the experience could benefit everyone.
On multi-platform drivers Posted Sep 12, 2011 20:28 UTC (Mon) by kleptog (subscriber, #1183) [Link]
I agree there's a lot of noise going around. The phrase "architectural alignment" seems to have been interpreted to mean the worst possible things, but the driver was being proposed for merging to mainline so evidently people thought it would be acceptable code-wise. I certainly haven't seen anyone post a single line of code showing anything the kernel devs won't stand for.
On multi-platform drivers Posted Sep 18, 2011 2:00 UTC (Sun) by kabloom (guest, #59417) [Link]
I haven't looked at any of the code, and I understand people's concern about what might happen in future refactorings of the kernel wireless system, but the kernel's code review process is pretty demanding in terms of how much work (and modification) a module developer has to do to conform to the kernel's way of doing things before they get accepted into the mainline.
If Broadcom has managed to satisfy the kernel developers' demands for conformance, and they still feel that there's sufficient architectural alignment between the Linux driver and their other drivers, then Broadcom's definition of "sufficient architectural alignment" is probably loose enough that they'll be able to maintain sufficient architectural alignment in the face of future kernel development work.
On multi-platform drivers Posted Sep 12, 2011 4:26 UTC (Mon) by niklam (subscriber, #64765) [Link] This issue has some similarities with the firefox on linux distro issue (bundling their own versions of libraries etc), although there Mozilla's motivations align much better with the interests of its product's customers (e.g. faster response to security issues, in general). Here, the benefit to the customer is less clear (although utoddl's comment above does provide one possible example of the potential for earlier "Just Works out of the box" availability). This tension can be eliminated entirely if hardware vendors support driver development by publishing thorough and open developer-oriented documentation for their products. That way they can leave it to the operating system communities to handle driver development. This would serve their customers best.
On multi-platform drivers Posted Sep 16, 2011 8:31 UTC (Fri) by hpro (subscriber, #74751) [Link]
My guess, however, is that a lot of the time the driver development is done in parallel with documentation. I.e., Broadcom knows 'kinda' how the driver should interact with the hardware, but the actual details are ironed out when the driver is built.
This means that the documentation will, at best, be done at the same time as a driver would be.
(This is not an opinion on how the Broadcom issue should be handled, but rather a speculation based on observations in the wild)
Owner of bcm58xx here, totally agree with author. Posted Sep 16, 2011 20:42 UTC (Fri) by lowlymortal (guest, #78175) [Link]
Disclaimer: I am the author of bcm58xx, a derivative of bcm5820, Broadcom's abandoned driver for their BCM58xx hardware security co-processors. The project is at http://sourceforge.net/projects/bcm58xx/.
It was my itch to scratch, i.e. learn device driver development under Linux. And I had some bcm5820s to play with, and knowledge of OpenSSL, so I went for it. I have been trying to clean up that driver for the last 3(!) years, but haven't completely succeeded (the goal is to move it into mainline, no free time etc.).
The old driver, bcm5820, was for 2.4.x line and was heavily coded for multi-platform development. Everything that the author mentions here is spot-on. I would even go further and add to the list the quality of the code as well. Let's just say that if I had known the shear masochism required to cleanup that driver, I would have chosen a different piece of hardware to play with. I have not looked at the drivers in question, but the code that I saw that came out of Broadcom, I would really run away screaming if I had to maintain it.
The worst part? I had FreeBSD's ubsec driver to look at. And weep. :(
On multi-platform drivers Posted Sep 20, 2011 4:42 UTC (Tue) by smowton (guest, #57076) [Link]
So remind me again: why do Broadcom actually care about merging this driver? That's not their goal, is it? Their goal is surely that Linux users should have an easy time running BC hardware, which is not the same thing.
How about this for a solution: Let's suppose for a moment that Broadcom can't just open their entire driver because there's all manner of copyright owners and the GPL doesn't sit well with that. They do however have a driver core and a Linux adaptation layer that works.
Now the Linux crowd say hey, this adaptation layer is silly, it duplicates functionality we've already written into the kernel just because it's not available on Windows 95. For shame!
At this point, surely the logical thing to say to the Linux crowd is: to hell with you. Broadcom haven't made a Linux driver that gets into the mainline, but what they have made is an open-source Linux compatible reference driver. From here it's the kernel mob's business to go about creating a Linux driver, either by straightforwardly merging it if they can stomach the code duplication, or else by hacking it a bit. At the very least they're likely to keep the core logic that's to do with how the device works, not how the kernel works.
Then Broadcom push changes to their reference driver, not the Linux tree, and the kernel team modify their own copy. Both sides watch the other's commit list, and kind-hearted souls on either team go a little further and port their patch for the other.
This seems to me like the appropriate division of labour -- the vendor provides a very implementation-friendly spec in the form of a reference driver, whilst the kernel maintainers attend to their own concerns. Where's the problem again?
On multi-platform drivers Posted Nov 10, 2011 2:05 UTC (Thu) by vomlehn (subscriber, #45588) [Link]
In addition to all of the reasons mentioned, consider issues like endianess and register size. Most drivers are target towards x86-based systems, but the company that developed it may not care about other platforms. Rigid control of development locks developers out the being able to port them to non-x86-based platforms.
|
|||||||||||||