The Linux Driver Project takes off [LWN.net]

To Greg Kroah-Hartman

Posted Sep 28, 2007 14:53 UTC (Fri) by JoeBuck (subscriber, #2330) [Link]

Provided that the NDAs are only temporary, and that in the end a fully documented driver (without any obfuscation) is released, I'm cool with it, and I think that you should be too.

To Greg Kroah-Hartman

Posted Sep 28, 2007 15:13 UTC (Fri) by Zoborov (guest, #29327) [Link]

Provided that the NDAs are only temporary

What conceivable incentive would manufacturers have for eventually releasing their APIs, given that Kroah-Hartman's at pains to assure them they've no cause to:

All that is needed is some kind of specification that describes how your device works, or the email address of an engineer that is willing to answer questions every once in a while.

[...]

If your company is worried about NDA issues surrounding your device's specifications, we have arranged a program with OSDL/TLF's Tech Board to provide the legal framework where a company can interact with a member of the kernel community in order to properly assure that all needed NDA requirements are fulfilled.

To Greg Kroah-Hartman

Posted Sep 28, 2007 16:21 UTC (Fri) by sobdk (subscriber, #38278) [Link]

>What conceivable incentive would manufacturers have for eventually releasing their APIs

APIs? OK I'm not exactly sure what it is you are referring to, but I think you have a misunderstanding of why a NDA might be needed. In order to write a device driver you need to know how to communicate with that device. Ideally the manufacturer would give you a register level programming manual for the device which documents all of the registers and any other special information you may need to know. Of course the problem is that despite popular belief these manuals don't write themselves. Writing this documentation is tedious, error prone and expensive, and for a lot of companies there is no gain in writing such a manual when internally they have access to hardware developers and their "write once" Windows driver.

So why require a NDA? Because this allows the companies to simply give access to same information they would have internally without having to worry about making it safe for external eyes only. This does not prevent anyone from creating a well documented open source driver that can be further improved by others who did not sign a NDA. In fact many of these same companies would probably be just fine with you using your NDA to write the RLP manual if that is how you would rather spend your time.

To Greg Kroah-Hartman

Posted Sep 28, 2007 17:24 UTC (Fri) by Zoborov (guest, #29327) [Link]

This does not prevent anyone from creating a well documented open source driver that can be further improved by others who did not sign a NDA.

If register level programming documentation's necessary to writing a fully functional driver, then how can "others who did not sign a NDA", i.e., those not privy to that documentation, able to substantially improve on it?

In fact many of these same companies would probably be just fine with you using your NDA to write the RLP manual if that is how you would rather spend your time.

A manual that, as a NDA signatory, I presumably wouldn't be able to show to fellow Linux developers (let alone to developers working on FOSS operating systems other than Linux)?

To Greg Kroah-Hartman

Posted Sep 28, 2007 18:59 UTC (Fri) by sobdk (subscriber, #38278) [Link]

>If register level programming documentation's necessary to writing a fully functional driver, then how can "others who did not sign a NDA", i.e., those not privy to that documentation, able to substantially improve on it?

Well, really that depends on the NDA (Non Disclosure Agreement). Specifically it depends on what the agreement says you are not allowed to disclose. Often they simply prevent you from disclosing information about the device before it is released to the public. Or they may prevent you from disclosing information you learn about the design process or other internal hardware/software (Maybe you'll get to see the VHDL code).

As long as the NDA allows you to write a non obfuscated GPL driver then the source code should provide all of the information others will need to work on the driver in the future. All registers and "magic" numbers will be defined as clearly named constants, functions and variables will have useful names, and of course there will be comments.

Improving the driver

Posted Sep 30, 2007 12:07 UTC (Sun) by filker0 (subscriber, #31278) [Link]

It also depends on how the driver is being improved. If the improvement is in how it interacts with
the hardware, having access to the additional information may be very useful. If the improvement
is, however, in any other area of the driver, that can be done without reference to the hardware docs
(if any exist).

To Greg Kroah-Hartman

Posted Sep 28, 2007 18:49 UTC (Fri) by k8to (subscriber, #15413) [Link]

Requirements of NDAs vary. Do not presume they all have the issues you are railing against.

Perhaps you should investigate (I have not) what types of NDAs would be considered acceptable by this project, and if you believe they are overlooking a freedom issue, urge caution, or engage folks in discussion of the matter. Polemics are sometimes useful, but in this situation I think you have a real opportunity for dialogue.

To Zoborov

Posted Sep 28, 2007 19:14 UTC (Fri) by filker0 (subscriber, #31278) [Link]

I've been party to NDAs that allowed me to write support code for an existing product, but the
documentation that I was given access to under the NDA contained more than just descriptions
of the registers for the device in question, it contained design and interface information for the
next generation (all speculative). The code was released in source form, the docs I used were
not.

I have also signed NDAs for documentation, the terms of which were amazingly restrictive. I
wasn't allowed to discuss anything from their docs, I wasn't allowed to distribute the code I wrote
based on their docs, even in binary form. I was only allowed to use it "in house", otherwise, the
company in question considered it "irreparable harm". The funny thing is that this was the docs
for the font format that Xerox had on the engine that DEC got from them to put in the LN01 laser
printer. I worked for DEC at the time in a related department. The LN01 was a DEC product. I
wrote a font editor to create loadable fonts for it, thus making it more useful, thus increasing
sales. Xerox was having none of it, though, back in 1983 or thereabouts.

I am currently party to at least 3 NDAs. Two through my employer, another through another
agency. When I was a contractor, I had to sign NDAs wherever I went.

NDAs vary. No open source driver can be obfusticated to a point that will hide the interface that
it is being written to from knowlegable observers, and Greg Kroah-Hartman is not one to
produce poor or obfusticated code. Signing NDAs is not the same as a patent agreement. It is a
fact of life in the industry. Would you rather have no open source drivers for these devices at all?

Part of the idea is that once these companies that are somewhat leery of the Open Source model
see the quality of the drivers written and maintained by that community, they will be more willing
to open up the process. Having shrill voices trying to shout them down as they finally start to
warm up to us is not the way to foster good will.

To Zoborov

Posted Sep 30, 2007 7:25 UTC (Sun) by Zoborov (guest, #29327) [Link]

No open source driver can be obfusticated to a point that will hide the interface that it is being written to from knowlegable observers

If only that were true. As the OpenBSD developer Stephan A. Rickauer wrote in reply to Kroah-Hartman:

Signing NDA's ensures that Linux gets a working driver, sure, but the internals are indistinguishable from magic. It is a source code version of a blob.

Part of the idea is that once these companies that are somewhat leery of the Open Source model see the quality of the drivers written and maintained by that community, they will be more willing to open up the process.

No it isn't. As Kroah-Hartman cheerfully admitted in his Linux Driver Project FAQ:

Q: This is a lame publicity stunt, Linux development has always been done this way.

A: Well, the NDA program that we have set up with The Linux Foundation is new. But yes, other than that, this is exactly how Linux kernel development has been done. But it is good to point out exactly how it all works for those who are not familiar with how it works.

Having shrill voices trying to shout them down as they finally start to warm up to us is not the way to foster good will.

Finally? As with binary blobs in the official kernel, it's just business as usual.

To Zoborov

Posted Sep 30, 2007 7:40 UTC (Sun) by Zoborov (guest, #29327) [Link]

Signing NDAs is not the same as a patent agreement. It is a fact of life in the industry.

No it isn't. OpenBSD, and (to the best of my knowledge) DragonFlyBSD won't even touch them.

Would you rather have no open source drivers for these devices at all?

If the actual information allowing someone else to improve or rewrite them remains under lock and key, releasing them under a free license is disingenuous at best.

To Zoborov

Posted Sep 30, 2007 9:54 UTC (Sun) by Zoborov (guest, #29327) [Link]

The funny thing is that this was the docs for the font format that Xerox had on the engine that DEC got from them to put in the LN01 laser printer. I worked for DEC at the time in a related department. The LN01 was a DEC product. I wrote a font editor to create loadable fonts for it, thus making it more useful, thus increasing sales. Xerox was having none of it, though, back in 1983 or thereabouts.

Ironic, indeed. Xerox's refusal to provide adequate documentation spawned the entire GNU project.

GNU Project

Posted Sep 30, 2007 10:23 UTC (Sun) by filker0 (subscriber, #31278) [Link]

I believe, actually, that it was when Unipress Emacs (based on James Gosling's C version of EMACS,
based on RMS's original TECO EMACS) was in the market and RMS wasn't able to get the source
(based on his original work) from Unipress so that he could modify it to fit his needs.

All of this was over 20 years ago. I remember RMS being more than a little put out at the time. I
was sort of on the perpiphery of his social circle at the time, and for few years around that time I
saw him fairly frequently at social and technical gatherings.

I don't recall Xerox having much to do with the formation of GNU. But I wasn't a close confidant, so
there's much I probably don't know about first hand.

Have you ever written a device driver?

Posted Sep 30, 2007 10:59 UTC (Sun) by filker0 (subscriber, #31278) [Link]

Zoborov: Have you ever actually written a device driver? It's not magic, you know. The driver will use the registers to take advantage of the device. Since you can tell what the driver is trying to do when it writes the registers, having the source for a working driver is sometimes better than having the register level docs.

What particular open source projects are unwilling to touch does not change the face of the industry, unless you define the industry as only open source projects. NDAs are a fact of life when dealing with the companies that design hardware, especially when dealing with advance information.

I've been writing device drivers on and off for over 25 years, and I can assure you, most of them have been done under NDA or for hardware designed by the company that I was working for where the work agreement was pretty much the same as having signed an NDA.

If Nvida were to have one of their programmers do exactly what Greg would do and release the driver as open source but not publish the hardware register level and theory of operation information, would you consider it equally evil?

Have you ever written a device driver?

Posted Sep 30, 2007 13:49 UTC (Sun) by Zoborov (guest, #29327) [Link]

I'd much rather have all interested parties request NDA-unencumbered access to the device documentation. Such is FOSS political principle and community solidarity.

OpenBSD development works on the basis of this principle, why not Linux? Is it because "world domination" can't wait?

In fact, it's obvious that Kroah-Hartman couldn't give a damn about the *BSDs, as we can see from his FAQ:

Q: What about the BSDs?

A: What about them? They are free to do whatever they wish, I have no input into their development at all, sorry.

Not much point

Posted Sep 30, 2007 18:07 UTC (Sun) by filker0 (subscriber, #31278) [Link]

I can see that there's no exchange of views going on here. Nothing that is being done is in any
way hostile to BSD. The driver model in BSD is different. No doubt, information from the Linux
driver could be used to write a BSD driver. The license conflicts will have to be worked out, but
that's not the issue. You're just unwilling to accept that this might not be a selling out of FOSS.

You wish for a morally pure world, where "right" and "good" are as defined by Richard M.
Stallman. If it deviates one breath from that, it's not worth having. I think you'll find that a lot of
BSD people think that RMS is a bit of a lunatic, though. I know at least two who did 10 years ago.
You don't see that freedom is no less pure when it is achieved by the sweat of the brow as it is
when given on a silver platter. Un-encumbered access to the information required to write a
driver is almost impossible to get on anything that's not a commodity component. Video cards
tend to be full of custom ASICs. They are very complicated, interdependent, and sometimes
contain parts that are not used, but may be in future products. The speed with which the video
hardware development moves makes writing generally useful docs very costly and doomed to
failure. Also, they don't want to let the competition know what they're planning.

You will only be happy when the device manufacturers all design their hardware in the public eye
with full community participation. You want open source hardware. Either that, or you want a
significant delay in getting drivers for new devices.

You never answered the question, though -- have you ever written a device driver? For any OS, it
doesn't matter.

Not much point

Posted Oct 1, 2007 5:08 UTC (Mon) by Zoborov (guest, #29327) [Link]

You're talking as if projects that eschew NDAs were somehow vaporware or the dead-end pursuit of naïve idealists; I've cited OpenBSD, a burgeoning project if ever there was one.

You never answered the question, though -- have you ever written a device driver? For any OS, it doesn't matter.

Perhaps I know better than to endorse ad hominem reasoning?

Not much point

Posted Oct 1, 2007 12:37 UTC (Mon) by filker0 (subscriber, #31278) [Link]

You're talking as if projects that eschew NDAs were somehow vaporware or the dead-end pursuit of naïve idealists; I've cited OpenBSD, a burgeoning project if ever there was one.

That is not at all my intent. What I said was "the industry". It doesn't matter what OS. Right now with the way things are, NDAs are (and have been for longer than Linux has been around; BSD's roots are much more ancient) almost always required for advance information beyond marketing and basic specifications. You may wish that were not true, but it has been for at least 20 years; probably longer. FOSS projects that don't accept NDAs are just fine, but they're likely to lag behind on new device support.

As far as ad hominem reasoning, I'm not sure what you're getting at. Your position seems to be that anything short of the ideal world (where the HW manufacturers all provide all the technical information for all the devices they make free of any NDA or other restrictions) is unacceptable, and that any project that submits to an NDA is neither useful nor valid in the FOSS world. You criticise those people who are willing to compromise and sign an NDA with the understanding that the resulting driver will be FOSS, without NDA restrictions. In the current environment, you won't see that. Maybe in the future, as open standards and open software become the normal way that these companies do business, but they're not there yet.

The reason that I asked about whether you've ever written a driver is that anyone who has written a non-trivial device driver, and gotten it to work well, has the skill set to determine what each register access (read/write) is doing within a driver based on the desired effect and how the device reacts. I've worked with quite a few parts where the docs from the supplier told me where the registers were and what the bits in the registers were called, but almost nothing else (in some cases, the published register information was just plain wrong). Having a working driver as a reference was the best documentation that I could get. If you've written a driver and still feel that having a working driver is just as bad as having no driver at all (or a binary blob), then you've got a completely different philosophy than I do.

I suppose what would be best for all of FOSS is an Open Documentation Project, where a team of literate engineers sign NDAs, work with the Open Source driver folks, and produce unencumbered device programming level (registers and theory of operation) documentation that is released under the Creative Commons license or some such. A nice idea, but there are fewer people who can do this than who can write the drivers, and it's harder than writing the drivers because testing it is almost impossible. I have been a technical writer (and editor) in the past, but I'm not sure how many other experienced engineers have that background.

My biggest gripe with what you've been posting in this thread is that you seem to be completely unwilling to contemplate that GKH is anything but a sell-out who doesn't care about FOSS, and that the results of the Linux Driver Project are going to be anything but bad for FOSS. The fact that he's got nothing to do with BSD (and says so) does not mean he's hostile to it, just that he's not involved with it. I don't know his personal feelings about BSD, but I saw nothing in the FAQ from which you quoted as being hostile. If there were a more congenial relationship between the BSD and Linux communities, he'd probably have more to say. I'm not involved in that conflict, but I've seen signs of it for many years.

You criticise. You quote alot. You don't seem to draw from personal experience. I've been involved FOSS since before GNU. Things were much more open back before chips more complicated than, say, a M68000 CPU. Today, ancillary chips (such as 3D graphics chips) are far more involved than anything that was available in the early days. For a while, a lot of communications chips were being built to work with Windows and nothing else. I don't know if Microsoft actively made deals with the chip makers, but it sure seemed like it. I think that this is still the case with the graphics chip sets. Microsoft gives the chip makers advance DirectX internals information and the chips get designed to work best with the interfaces expected by DirectX. Non-windows drivers, then, become more difficult, as X doesn't have the Windows GDI (which is a good thing). The generic programming interfaces may be a complete mess. No cogent programming information exists beyond the VHDL, which is very proprietary. Hence the NDAs. I've even had to sign NDAs for high capacity NOR flash programming info in advance of release of the parts even though the only significant differences between it (2+GiBit) and the previous (<=1GiBit) were the ID codes and erase timings.

In any case, you seem convinced that the project cannot do the FOSS community any good, and are intent on criticising GKH and those who work with him for undertaking it. I am not trying to stop you so much as I'm defending an endevour that I see as being good for FOSS. My "politics" are those of moderation; I don't see this NDAs as a moral evil, and I believe that trade Secrets are better than Software Patents. Your position, and world view, are very different.

I'm a reformer, you appear to be a crusaider. As such, I'll be one of the first up against the wall when the revolution comes. I've had a long and productive life, so I suppose that's not too big a loss.

Not much point

Posted Oct 1, 2007 18:36 UTC (Mon) by jejb (subscriber, #6654) [Link]

Arguing without the facts is just as bad as ad hominem attacks.

Perhaps if you looked at the structure of the actual set of NDA's that this is based on you'd have some better arguments? They're published by the Linux Foundation, who found a set of lawyers to help craft them. The general programme is here:

http://www.linux-foundation.org/en/NDA_program

The actual legal docs are at the bottom of the page. The relevant sections to this current thread are:

Section 2A "no Open Source Product shall contain any of the Contributor's confidential information except to the extent required in order to effect the purposes". This somewhat opaque legalese means that any information in the confidential doc that's needed to actually program the device may be incorporated into the resulting driver or otherwise published. i.e. register definitions, APIs, the lot can move directly into the public domain via documentation within the actual written driver.

Section 2B is a patent licence for any and all patents the contributing company happens to hold in the technology needed to drive the device.

Section 7 limits the absolute term of confidentiality to five years.

This seems to be about as transparent as it can be and true to the principles of Open Source: all information needed actually to program the device can become public domain. Any other embellishments in the docs that usually cause lawyers to insist on redacting them before they're released (like slanderous aspersions on the competition or misguided marketing plans) can remain confidential.

Not much point

Posted Oct 2, 2007 7:24 UTC (Tue) by Zoborov (guest, #29327) [Link]

Notwithstanding anything to the contrary above, except as expressly provided below, LF acknowledges and agrees that the Contributor’s Confidential Information is being provided for reference only, and that no Open Source Software Product shall contain any of the Contributor’s Confidential Information except to the extent required in order to effect the Purposes, for example, where components of the Contributor’s Confidential Information must be incorporated into the source code of an Open Source Software Product in order for the Open Source Software Product to function properly.

except to the extent required in order ... for the ... Product to function properly: nothing in there, certainly nothing explicit, permitting rigorous device documentation within the actual source code.

Not much point

Posted Oct 2, 2007 18:26 UTC (Tue) by jejb (subscriber, #6654) [Link]

That's almost exactly what "except to the extent required in order to effect the Purposes" means in legal terms.

Not much point

Posted Oct 3, 2007 5:40 UTC (Wed) by Zoborov (guest, #29327) [Link]

If the "Confidential Information" may be disclosed only to the extent that it effects the "Purposes", and if the exemplary instance of these "Purposes" is a functional driver, then no, since there's no technical need on the developer's part to provide explicit device documentation. After all, perfectly functional device drivers can have hopelessly obfuscated source code.

I'm not a Veterinarian

Posted Oct 3, 2007 20:05 UTC (Wed) by filker0 (subscriber, #31278) [Link]

But I do believe that my wife (who is) would concur - This horse is dead. Please don't make us call
the SPCDA (Society for the Prevention of Cruelty to Dead Animals).

The Linux Driver Project will go on whether you're happy about it or not. If I had the time, I'd
volunteer to write a driver or two.

I'm not a Veterinarian

Posted Oct 5, 2007 9:15 UTC (Fri) by Zoborov (guest, #29327) [Link]

The Linux Driver Project will go on whether you're happy about it or not.

The fact/value distinction in a nutshell, there.

Drivers reveal their interfaces

Posted Sep 30, 2007 10:37 UTC (Sun) by filker0 (subscriber, #31278) [Link]

The driver will serve, for those who can read source code, as a key to deciphering the hardware
interface to which the source code is written. If the driver source is available outside of NDA,
someone with the aptitude and skill set to do reverse engineering should have very little trouble
figuring out what every hardware interaction is doing even if GKH chooses to obfusticate the code
(which I seriously doubt he would). The provinence of an open source driver reverse engineered
from another open source driver is much clearer than that produced by examining the disassembly
of proprietary binary blob.

I disagree strongly with Stephan A. Rickauer on that. I also disagree with your position that this is
the same as having a binary blob. A driver that is in the kernel distribution will be maintained and
updated as the kernel APIs change and can be ported to other architectures. We are not left waiting
for the manufacturer to come out with a new blob or meta-module.

Drivers reveal their interfaces

Posted Sep 30, 2007 10:57 UTC (Sun) by Zoborov (guest, #29327) [Link]

A driver that is in the kernel distribution will be maintained and updated as the kernel APIs change and can be ported to other architectures.

Be that as it may, in the words of Theo de Raadt:

I can assure you that other people without the docs cannot fix the bugs in those drivers

Fixing bugs

Posted Sep 30, 2007 11:15 UTC (Sun) by filker0 (subscriber, #31278) [Link]

Someone with the docs is always in a better position to fix the bugs, but not having them does not preclude fixing bugs, Theo de Raadt's comments notwithstanding.

It's not always fun, but it is possible. I've done it more than once, at least once where all I had was a binary driver, a scope, and a knowlege of what the driver was supposed to be doing. Having non- obfusticated source code is very much like having register level docs, plus a little theory of operation thrown in.

What you're really doing is questioning GKH's integrity, his intelligence, and his committment to Free and Open Source. Until he's produced a driver, you are welcome to worry that he might produce the source eqivilent to a binary blob, but you conclude that you will.

Drivers reveal their interfaces

Posted Oct 2, 2007 13:27 UTC (Tue) by krishna (subscriber, #24080) [Link]

> I can assure you that other people without the docs cannot fix the bugs
in those drivers

And while I can't assure you, I'll give you 2-to-1 odds that given a
working driver and the docs, you'll never finish getting the bugs out of
the *documentation*.

Various types of NDA exist

Posted Sep 29, 2007 5:11 UTC (Sat) by farnz (subscriber, #17727) [Link]

NDAs vary quite considerably; the last NDA I was under for a chip was simply because the "documentation" consisted of the VHDL source for the various bits of chip, an e-mail address for the hardware team responsible, and a phone number to get hold of the team leader.

I was permitted to use this information to document the chip on a "pins" basis; basically, I couldn't say how the chip did something, but I could say things like "the value in the register at offset 0x2c controls the width of the output pulse on pin 23. It's a count of the number of pulses of XTAL1 (wired between pins 5 and 6) for which pin 23 stays high". An NDA of this form would let Greg release both a driver and register-level documentation for the chip he's working on.