| |||||||||||||
KS2010: Big out-of-tree projects
For almost as long as the kernel has existed, there have been large kernel
projects which live outside of the mainline. At the 2010 Kernel Summit,
Roland Dreier and Chris Mason led a session to discuss these projects and
what, if anything, should be done about them.
Roland started off by talking about the OpenFabrics Enterprise Distribution (OFED), which is described as:
...a downloadable, networking open-source software package that
implements Remote Direct Memory Access (RDMA) and kernel bypass
mechanisms to deliver high-efficiency computing, wire-speed
messaging, ultra-low microsecond latencies and fast I/O for
servers, block storage and file systems.
In reality, it's a large chunk of out-of-tree code distributed by the OpenFabrics alliance. There are a number of enhancements to the InfiniBand infrastructure, backports to older kernels, and so on. Hardware companies have been telling their customers to use the OFED stack instead of mainline; the enterprise distributions have been shipping it. So why does OFED exist? Roland says it's partly his fault; he has not been merging stuff into the mainline quickly enough. He has had trouble getting people to help review all of that code. But, he says, there is also an incentive for OFED: it gives the OpenFabrics Alliance an ongoing reason to exist. Perhaps OFED should go into the staging tree? The OFED tree consists mainly of patches to code which is already in the mainline; it's not a set of standalone drivers which can sensibly be put into the staging tree. It was suggested (nobody was able to confirm it) that Red Hat will not be shipping updated OFED stacks in the future. Chris said that the distributors need to push back more against this kind of out-of-tree code. Oracle, he noted, is doing that. Ted Ts'o asked about the scope of the problem - what other large, out-of-tree projects exist? The list started with Xen's Dom0 code. Jeremy Fitzhardinge took a moment to update the crowd on the status of Dom0; most of the trickiest bits have been merged for 2.6.37, so it's actually possible to boot a Dom0 kernel. There are no backend drivers, though, so that kernel will have no devices available; Jeremy allowed that this limitation reduces the usefulness of that kernel somewhat. Some of those drivers will be merged in forthcoming development cycles; others are so ugly that they may never get in. Other significant out-of-tree projects include the Lustre filesystem, the realtime preemption tree, OpenVZ, the LTTng trace kit, the grsecurity patches, and the always controversial SCSI target code. In general, it was agreed, we are doing much better than before. The process needs to continue, though. According to Chris, Oracle has found every vendor driver it has ever encountered to be horribly broken and unsupportable. So Oracle has taken a strong position that only upstream drivers can be supported. Ted said that, for enterprise systems, the problem has mostly been solved; the same is, alas, not true in the embedded area. So we will have to go through the same education process with embedded vendors. It may be the same problem, but out-of-tree code may be harder to eradicate in the embedded world. Vendors of embedded systems are incredibly price-sensitive; upstream drivers fall by the wayside when compared to a savings of a few cents on a component. What's needed is to make "upstream drivers" a stronger part of the procurement process; it must be a technical requirement for any candidate components. Tim Bird said that this problem had been discussed at a recently concluded embedded Linux summit; the result is a new effort to encourage a "preference" for open-source drivers. Many vendors apparently claim that they have never even heard that free drivers are preferable; the first step is to cure them of that impression. The session concluded with a statement that, while out-of-tree projects still exist, things have improved and are continuing to do so. (Log in to post comments)
KS2010: Big out-of-tree projects Posted Nov 2, 2010 8:06 UTC (Tue) by Lumag (subscriber, #22579) [Link]
Hmm. What about Google/Android kernel. It's also a "big" oot project, isn't it?
KS2010: Big out-of-tree projects Posted Nov 3, 2010 14:24 UTC (Wed) by neilbrown (subscriber, #359) [Link]
Google/Android is not really one big oot project - it lots of distinct bits - drivers for various hardware, the suspend-blocker thing, and some IPC subsystem. So maybe it is lots of little projects that can be merged one by one, and to some extent are. I haven't heard any suggestion of the IPC thing being merged in any form though - I wonder if anyone else has heard anything.
KS2010: Big out-of-tree projects Posted Nov 2, 2010 14:19 UTC (Tue) by ccurtis (guest, #49713) [Link] To echo Tim's sentiments, a place I worked would actually _prefer_ vendor code to open code. We had one vendor supply a library that we had to use to access their hardware (the kernel driver was just a shim). This library was a non-stop source of bugs and instability in the product. Eventually the company decided we needed a new revision of the device and went to a different outside vendor to design one. They came back with something designed around a Broadcom chip. I immediately began asking about source code; the response was "we will own everything - the design, the software - everything". Only after asking for a third time about the Broadcom driver specifically, the response was, "Oh, well, nobody has access to THAT". The design went ahead as planned...
KS2010: Big out-of-tree projects Posted Nov 2, 2010 14:23 UTC (Tue) by ccurtis (guest, #49713) [Link] I should clarify that last comment ... the project went ahead as planned, but the design was changed considerably. The more questions I asked about the software, the more problems I found. Eventually it was admitted that no consideration was made for software design at all in this project. Fortunately, I'm not there now to see how the product turned out.
KS2010: Big out-of-tree projects Posted Nov 3, 2010 12:54 UTC (Wed) by NAR (subscriber, #1313) [Link]
It's interesting to compare this quote "Chris said that the distributors need to push back more against this kind of out-of-tree code" with a quote from an earlier page: "In general, it can be better if new interfaces stay out of the kernel for a while." It might compare apples with oranges (ABIs vs out-of-tree projects), but still interesting.
OpenVZ Posted Nov 20, 2010 21:00 UTC (Sat) by Lennie (subscriber, #49641) [Link]
When you say OpenVZ is a big out-of-tree project.
One of the reasons has always been that it was unclear how to get the changes of projects like Linux-VServer and OpenVZ into the kernel and they are actively working on getting those changes into the kernel in the form of Linux Containers. And many things already are in the kernel. So it isn't so much out-of-tree anymore.
I keep wondering what lxc is still missing that not more people are moving to lxc instead of OpenVZ and Linux-VServer.
Is it just some kind of migration-tooling ? Or some host-migration-tooling, I think OpenVZ has the ability to stop a container and move it over to an other machine, but the mainline kernel does not yet have that.
Or is it just that not all userspace tooling has been created/needs more documentation ?
|
|||||||||||||