The Linux Kernel's Fuzzy Future (InformationWeek)

The good folks who are doing this marvelous job developing the linux kernel are not participating in the age-old practice of "Don't buy my competitors product today, because the product I am going to introduce in the future will blow it away" hype and garbage that essentially constitutes public vapour ware sales campaign that goes by the name of the Microsoft roadmap.

And this is a bad thing?

That approach stands in contrast to the one taken by Bill Gates' team at Microsoft, where Windows releases aren't only planned, but publicly disclosed, for the next three years.

So, what happens in practice: There are dozens of partial roadmaps (wanted features, structural redesign plans). In various discussions, things are decided. Through contributions of dozens of parties, the kernel grows in the general direction that the users want.
Sorry, this is the process. And you can influence the result; try that with Microsoft's roadmap.

What he misses is that it is all very well to have a three year plan, but you have also got to deliver on it. How many features have been dropped from Longhorn so that it can be delivered when it is scheduled? How much QA will have been skipped to ensure that it is delivered on time? How does a plan you don't keep to help your customers? In many respects the reason for a three year plan is to keep end customers locked in, expecting jam tomorrow.

By the way, does anyone else wonder about his name?

Don

An example of this that shows both sides of the same problem is that since Linux didn't have a concrete roadmap, large components of the community were able to get behind Xen fairly quickly over UML. The inertia coming out of OLS for Xen was quite impressive. Not that UML is dead, or Xen is totally ready, but there was a strategic shift there than I have a hard time seeing in a large corporate software development shop. The engineer that had suggested to management, that in turn promised customers 'UML-virtualization' would have a hard time shifting as quickly as an unstructured market with shared responsibility. Case in point, SuSE is likely to have to spend substantial effort supporting and improving UML while other distros have the option of supporting Xen (or any better options coming along).

LOL.

"Without stable, reference milestones it will artificially increase the barrier to moving beyond dependence on a single vendor," one observer wrote in July in an online forum on LWN.net, a Web site focused on Linux happenings. "Innovation outside the corporate sponsored environment dries up ... Goodbye, Linux."

The real issue is that it doesn't generally take Linux development three years to get from an idea being sufficiently thought-out that it could go on a roadmap to having it done, at least for features with any publicity. A lot of features come from people realizing that something useful is now really easy, due to other work.

On the other hand, it might be useful to have a running list of things that people have announced that they are working on.

As for the fears about the changes in the development process, they run entirely contrary to the actual evidence. Assuming 2.6.10 comes out in about a week, there will have been 11 major releases of 2.6 (counting 2.6.8.1) in 12 months. For 2.4, under the old process, in the first year, there were 17 releases. 2.6 has only recently completed the period, common to all stable series, of having monthly releases, so it is too soon to say anything about the regular rate of releases, although it looks like it is probably not going to be more than twice as fast as 2.4 was once 2.4 settled down (2.4 had a long fast period, then a couple of slow periods, and recently has gotten more consistent).

There has been a concern that the 2.6 series wouldn't include kernels that end users could actually use, and that they would have to use distribution-provided kernels instead. However, in 2.4, no major distribution provided an unmodified kernel, or even close, because they were all backporting changes they felt necessary from the 2.5/2.6 tree, which meant that it was often impossible to go from a distribution kernel to a kernel.org kernel and have the system continue to work correctly. For example, one 2.4 kernel I've used is linux-2.4.19-rmk7-pxa1-cerf1; this is at least 9 versions added onto the official 2.4 kernel to get it to work, and it doesn't include bugfixes in the past 9 2.4 kernels. The 2.6 kernel for the board is 2.6.7-cerfb1, with one set of patches and tracking the official kernel pretty well (2.6.8.1 was the newest kernel at the time).

The main points of the new process are really to minimize the differences between the kernels that distributions were shipping and the kernels that the developers were releasing, and avoid the periods at the start of a stable series where versions come out monthly and don't work very well. A better way of thinking about the new process is that, in the old way, there was a lot of stuff that got backported by distros in divergant ways; in the new process, the kernel team does the backports so they stay consistant. The development series is kept similar, such that bugfixes can be forward-ported to the development kernels as well, and things don't regress going into the next major version due to fixes not propagating. From the standpoint of terminology, it makes sense to keep the version numbers comparable, and the version numbers matter most to the stable kernels, so those numbers are used.

The only issue I see, currently, is that the fixes to the version numbering made in the 2.4 series, where -pre versions were known to not be ready, -rc versions could really become final at any point, and final versions were really quite reliable. I still think it's very odd that Andrew Morton is more in charge of the development series and Linus is more in charge of the stable series.

He was a salesman (and later president) of an Allied Van Lines agent here in Florida. When he was just a young turk salesman, one of the older guys gave him a great selling idea: when talking to the potential customer, work into the conversation, just off hand, that "a good salesman would never smoke in your house" (this was the 60's :-).

That is, put the other salesmen at a disadvantage they don't even know they possess, by inventing some cock-a-mamie criterion that they don't know they're being judged on.

That's what's happening here, too.

"We have no Five Year Plan. We must be Bad!"

Nuh uh.

Cause, I mean, c'mon: go look at the companies in the Unix market that *do* have a Five Year Plan. They're in, by and large, about as good shape as *another* organization notable for Five Year Plans.

Oh, wait: it's pretty much only SCO (and they're a fucked company walking), and Sun (and they're "open sourcing"). Oh, and AIX, and IBM's given us most of the good parts.

Got it.

Yeah; let's put on the Jack Welch hat, and do some strategic planning.

Cause that's workin' real well.

(This message brought to you by the "So many things are just me" radio network.)

Dear Mr. Foley:

I'm a bit puzzled by your article. When Microsoft supplies a ``roadmap'', it sounds like you're getting a chart of an area with all the possible paths laid out for you to choose. In actuality, so far as I can see, such a roadmap is more a travelogue describing the only way you can go. Yes, it's nice to plan for what MS will offer next, but there's not a lot of choice involved, especially if what you need is missing.

Linux, on the other hand, has no roadmap because its advances are unplanned---they occur as people improve on extant capabilities, and others add what they need. Who could have planned IBM's addition of JFS, or NUMA, or RCU three years before each was released? That early, it was unclear whether IBM even planned to play nicely.

Under Linux, on the other hand, useful additions are rapid. If the current kernel meets your specifications, why care about a roadmap? If it doesn't, check whether anyone's working on the required function. If so, send resources. If not, hire a kernel hacker or two and add it now, not when a roadmap shows it's planned for. If the mod is accepted into the mainline kernel, you're in clover. Even if it isn't, you've still got the kernel you need, and if it's writeable as a module, it won't be too hard to keep it current with newer kernel releases.

So, what would a roadmap (assuming anyone could define and enforce one) do for you? It's another unnecessary evil that has been eliminated from Linux.

Best wishes,

Max Hyre

If CIOs are focused on the Linux kernel release and development schedule then those companies probably have other issues. It would be far better to list of some kernel technologies that are missing and mission critical to their businesses.

• Having a plan does not help your customers in any way unless they can actually to some degree rely on the future reality bearing some sort of resemblance to the plan 3 years earlier.
• Linux is developed independently by a large group of people from different companies with different motivations. Each of them individually may have plans, but this doesn't mean that they all agree on one master plan.
• There are noone with the authority to say "This is what you should work on for the next 3 years", or anything similar. So whomever made the plan would only speak for himself, or atmost for the small subset of kernel-developers he employs.
• There are in fact fairly clear directions. It's been clear from Linux 2.0 until today, for example, that one area where the kernel will be developed is SMP and the issues associated with it, such as preemption, the death of the big-kernel-lock and so on.
• 90% or more of the things people say they want from "Linux" are actually user-space things. It's LITERALLY been years since I had *any* customer select something other than Linux for reasons to do with the kernel, with one important exception: hardware-support. And the plan there is clear: continue to improve it, continue to work with manufacturers to get open specs and drivers for as much hardware as possible.
• IT is a fast-changing field. Making a 3-year-plan migth not be such a good idea afterall, even if you could.

In short, he is complaining about a non-problem. There *are* real problems that needs to be solved to improve on Linux and free software in general, but a 3 year plan clearly isn't it.

Sure a Proprietary OS vendor can trumpet 5 year plans and can discuss
the details of the next Great Leap Forward. The issue is that these tend
to list OS features, some of which required kernel enhancements.

Similarly eatures that get contributed to the Linux kernel tend to be end
up there because they enable user land features that will keep the user
happy - be that user Aunt Tilly or a Google PhD.

So the same applies to a Linux distro, Sun and Microsoft. Of course
the GNU/Linux development process is very different to the other two
examples. The point is that for a fair comparison you must compare the
development planning process for Microsoft Windows with that for the
GNU/Linux system as a whole - not just the kernel.

For an attempt at a proper comparison the writer should have talked to
people like leading distributions, to major application developers in a
position to influence kernel and system development (Oracle etc.), to the
average hacker with itchy fingers.

The planning committee for Microsoft consults market research and makes
great plans that the marketing department love. The path that Linux takes
is demand driven - not marketing driven. It is determined by a
heterogeneous bunch, but in the end it provides what the people who
really use the system really need. And with a lot fewer broken promises.

I would love an article every 6 months or so where LWN presents Linux's 1 to 3 year plan. Of course, to be comparable to the Windows roadmap, you'd want to include all the related software that regular people think of as part of the OS. So, you'd report not just on improving pre-emption and Infiniband support in the kernel, but summarize the roadmaps for X.org, Gnome, KDE, Firefox, yum, OpenOffice, etc. And, you'd want to spend special time talking about projects like Xen and Reiserfs that will bring powerful new features when and if they're incorporated.

And, what about the fact that the roadmap would certainly be wrong? So what? It would likely be just as accurate as Microsoft's, and LWN's opinions of where things are going are just as valid and relevant as Linus's, Stallman's, or anyone elses.

In fact, I think this kind of roadmap could be great PR for LWN, where you could issue a press release every 6 months detailing the major advances and changes in the 3 year plan. This would likely be picked up by the rest of the tech press (and sometimes even the NYT and WSJ), and bring LWN a lot of new subscribers. (Note that the dumb Information Week article prompting the idea already references lwn.)

The roadmap itself could largely consist of links to old articles on each area and to the external roadmaps that the individual projects publish. So, there probably would not even be that much work involved. But, the lwn roadmap could be a central repository of links to cool stuff coming down the line.