On multi-platform drivers
Broadcom developer Henry Ptasinski brought the issue to the fore while talking about why the company was not interested in supporting the in-mainline b43 driver:
To developers who have worked with the kernel for a while, these words look like a fundamental mistake. To others, though, they seem reasonable: if Broadcom wants to support its hardware, why not let the company do things its way?
One clear problem with trying to maintain "architectural alignment" with drivers for other operating systems is that only the original company can maintain that alignment. The other associated drivers are almost certainly not open source; nobody else in the community has any way to know which changes are consistent with those other drivers and which are not. Not even the relevant subsystem maintainer can make that kind of call.
One also must consider that most other kernel developers have no motivation for - or interest in - maintaining the correspondence between the drivers, even if they did know how to do it.
The obvious conclusion here is that allowing a vendor to maintain a multi-platform driver in the kernel tree will only work if that vendor is given absolute control over the code. If others can make arbitrary changes, there is no way for the vendor to keep the drivers consistent. But, in the kernel, nobody has that kind of absolute control with the possible exception of Linus Torvalds. If something needs to be fixed or changed, anybody with the relevant technical skills can do it. If a piece of the kernel tree were to be fenced off and made off-limits for kernel developers, the kernel as a whole becomes a little less free.
And that freedom matters. Consider the problem of internal API changes. As anybody who watches kernel development knows, internal interfaces are changed all the time in response to problems and changing needs. Those changes can, at times, force significant changes in users of the affected interfaces. Contemporary rules call for a developer who makes an interface change to fix any code broken by that change. Code that has been designated as off limits will be hard to fix in this way, slowing down the evolution of the kernel as a whole. As one example, consider the removal of the big kernel lock; that job required significant locking changes in many places. Literally hundreds of drivers were modified in the process. Impeding those changes would have made the BKL removal task even slower - and maybe impossible.
Manufacturers are not known for long-term support of their products; they have no real reason to support five-year-old chipsets that they no longer sell. Indeed, they have every reason to end such support and encourage the replacement of older hardware with shiny new stuff. Linux, instead, tends to support hardware for as long as it is in use. Giving a vendor absolute control over a driver is certain to create conflict when that vendor moves to drop support for older chipsets.
A vendor's agenda can differ from the community's needs in other ways as well. Vendors may not appreciate patches to enable undocumented features or make low-end offerings behave more like their more expensive alternatives. Or consider Hans Reiser's opposition to the addition of extended attribute and access control list support to reiserfs. His argument was that users should wait for the shiny new Reiser4 filesystem to obtain such features; had he been listened to, reiserfs users never would have had support for those basic filesystem capabilities. The kernel works well because it is maintained as the best kernel for all users over the long term, even if that occasionally causes conflicts with short-term vendor desires.
Multi-platform drivers from vendors tend to be written around the minimal set of support functions that are available on all platforms. The result is a lot of code duplicating functionality already found in the Linux kernel; consider, for example, just how many wireless drivers initially came with their own 802.11 stacks built in. Developing and maintaining just one rock-solid 802.11 implementation is hard; having several of them in the kernel would bloat the kernel and endow it with a number of second-rate implementations - all of which must be maintained into the future. Other kernel support code - from simple linked lists through complicated memory management - is also often avoided by multi-platform drivers. Those drivers will be bigger, buggier, and harder for kernel developers to read and support. They are also much less likely to behave consistently with other Linux drivers for the same type of hardware.
Beyond all of the above, it is also far from clear that maintaining a multi-platform driver actually saves any work. Drivers written for Linux can make full use of the available support infrastructure. Multi-platform drivers must, instead, duplicate much of that functionality and maintain an operating system abstraction layer as well. Maintaining a multi-platform driver means maintaining a larger body of code without help from the community.
In summary: trying to maintain a single driver for multiple operating
systems may look like a good idea on the surface. But it is only
sustainable in a world where the vendor keeps complete control over the
code. Even then, it leads to worse code, duplicated effort, long-term
maintenance issues, and more work overall. Linux works best when its
drivers are written for Linux and can be fully integrated with the rest of
the kernel. The community's developers understand this well; that is why
multi-platform drivers have a hard time getting into the mainline.
| Index entries for this article | |
|---|---|
| Kernel | Development model |
Posted Sep 9, 2011 7:56 UTC (Fri)
by geofft (subscriber, #59789)
[Link]
How do you structure your repositories to do that? I don't think git will handle this situation well, today.
Posted Sep 9, 2011 15:23 UTC (Fri)
by josh (subscriber, #17465)
[Link]
Posted Sep 9, 2011 20:12 UTC (Fri)
by utoddl (guest, #1232)
[Link] (4 responses)
"Architectural alignment" does not suggest to me that Broadcom would be less than accepting of changes that make make their Linux driver or support better. They've obviously been graciously taking suggestions from established kernel devs for the last year in the way we would hope. I see no reason to expect less from them in the future. They've spent the effort to learn how to work "with" vs. "in spite of" the Linux Way. Good on 'em.
Turn the argument around: Why would a vendor and hardware manufacturer be in any worse position to maintain -- in coordination with a highly competent kernel development community -- drivers for current and future hardware that they understand fully than a handful of developers working in their spare time with incomplete specs and spurious bug reports?
This looks to me like a company trying hard to provide exactly the kind of support that others have been criticized for not providing for years. Kudos to Broadcom. This may be the ideal model for companies to work with the community; time will tell. I'm willing to give it a chance before declaring it an untenable approach.
Posted Sep 10, 2011 20:46 UTC (Sat)
by alecs1 (guest, #46699)
[Link] (1 responses)
If the Broadcom driver gets into the kernel now based on merit, it doesn't necessarily mean the vendor will be able to do whatever it wants in the future. Broadcom is not the Linux kernel gatekeeper. If things don't clean up between the main actors in this, I trust Linus will make the correct compromise.
Posted Sep 12, 2011 21:03 UTC (Mon)
by mjthayer (guest, #39183)
[Link]
Right. It seems to me that the main problem raised in the article (that kernel people will want to be able to make changes without consulting Broadcom first) is something Broadcom will have to deal with, not something that should be much of a problem for other kernel developers. Broadcom have made their code GPL, and there is always the - pretty easy - option of forking maintenance of it if they don't play the way other kernel developers would like, so it seems reasonable to give them a chance. If their multi-platform driver maintenance works out it the experience could benefit everyone.
Posted Sep 12, 2011 20:28 UTC (Mon)
by kleptog (subscriber, #1183)
[Link] (1 responses)
Posted Sep 18, 2011 2:00 UTC (Sun)
by kabloom (guest, #59417)
[Link]
If Broadcom has managed to satisfy the kernel developers' demands for conformance, and they still feel that there's sufficient architectural alignment between the Linux driver and their other drivers, then Broadcom's definition of "sufficient architectural alignment" is probably loose enough that they'll be able to maintain sufficient architectural alignment in the face of future kernel development work.
Posted Sep 12, 2011 4:26 UTC (Mon)
by niklam (guest, #64765)
[Link] (1 responses)
This issue has some similarities with the firefox on linux distro issue (bundling their own versions of libraries etc), although there Mozilla's motivations align much better with the interests of its product's customers (e.g. faster response to security issues, in general). Here, the benefit to the customer is less clear (although utoddl's comment above does provide one possible example of the potential for earlier "Just Works out of the box" availability). This tension can be eliminated entirely if hardware vendors support driver development by publishing thorough and open developer-oriented documentation for their products. That way they can leave it to the operating system communities to handle driver development. This would serve their customers best.
Posted Sep 16, 2011 8:31 UTC (Fri)
by hpro (subscriber, #74751)
[Link]
This means that the documentation will, at best, be done at the same time as a driver would be.
(This is not an opinion on how the Broadcom issue should be handled, but rather a speculation based on observations in the wild)
Posted Sep 16, 2011 20:42 UTC (Fri)
by lowlymortal (guest, #78175)
[Link]
It was my itch to scratch, i.e. learn device driver development under Linux. And I had some bcm5820s to play with, and knowledge of OpenSSL, so I went for it. I have been trying to clean up that driver for the last 3(!) years, but haven't completely succeeded (the goal is to move it into mainline, no free time etc.).
The old driver, bcm5820, was for 2.4.x line and was heavily coded for multi-platform development. Everything that the author mentions here is spot-on. I would even go further and add to the list the quality of the code as well. Let's just say that if I had known the shear masochism required to cleanup that driver, I would have chosen a different piece of hardware to play with. I have not looked at the drivers in question, but the code that I saw that came out of Broadcom, I would really run away screaming if I had to maintain it.
The worst part? I had FreeBSD's ubsec driver to look at. And weep. :(
Posted Sep 20, 2011 4:42 UTC (Tue)
by smowton (guest, #57076)
[Link]
How about this for a solution: Let's suppose for a moment that Broadcom can't just open their entire driver because there's all manner of copyright owners and the GPL doesn't sit well with that. They do however have a driver core and a Linux adaptation layer that works.
Now the Linux crowd say hey, this adaptation layer is silly, it duplicates functionality we've already written into the kernel just because it's not available on Windows 95. For shame!
At this point, surely the logical thing to say to the Linux crowd is: to hell with you. Broadcom haven't made a Linux driver that gets into the mainline, but what they have made is an open-source Linux compatible reference driver. From here it's the kernel mob's business to go about creating a Linux driver, either by straightforwardly merging it if they can stomach the code duplication, or else by hacking it a bit. At the very least they're likely to keep the core logic that's to do with how the device works, not how the kernel works.
Then Broadcom push changes to their reference driver, not the Linux tree, and the kernel team modify their own copy. Both sides watch the other's commit list, and kind-hearted souls on either team go a little further and port their patch for the other.
This seems to me like the appropriate division of labour -- the vendor provides a very implementation-friendly spec in the form of a reference driver, whilst the kernel maintainers attend to their own concerns. Where's the problem again?
Posted Nov 10, 2011 2:05 UTC (Thu)
by vomlehn (guest, #45588)
[Link]
Being upstream to multiple projects
On multi-platform drivers
This is an overreaction and misinterpretation. Why should "architectural alignment" necessarily mean iron-fisted control of the code? Rather, it offers a hope that by the time new hardware makes its way to retail shelves people running modern kernels will find that it Just Works out of the box. It also could mean that issues found and fixed by those krazy Linux kids may improve the vendor's overall product line, thus rewarding financially a company who's willing to make the effort to play by our rules.
On multi-platform drivers
On multi-platform drivers
On multi-platform drivers
On multi-platform drivers
On multi-platform drivers
On multi-platform drivers
On multi-platform drivers
Owner of bcm58xx here, totally agree with author.
On multi-platform drivers
On multi-platform drivers