LWN.net Logo

KS2007: The distributor panel

KS2007: The distributor panel

Posted Sep 6, 2007 17:48 UTC (Thu) by mgb (guest, #3226)
In reply to: KS2007: The distributor panel by mgross
Parent article: KS2007: The distributor panel

Kernel hackers want a permanent playground rather than producing stable releases as in the past. That's their decision to make but blaming the distros for the consequences of that decision is not appropriate.


(Log in to post comments)

KS2007: The distributor panel

Posted Sep 7, 2007 12:50 UTC (Fri) by hmh (subscriber, #3838) [Link]

When you factor in bad design bugs that made it into mainline and we need a way to be able to fix them, you will see it is not just a matter of a "playground" for developers. The playground is probably where people are when they send to mainline badly designed APIs and ABIs.

Now, regressions and ABI breakage to *userspace* are a major pain, but one we can't just do without completely. Userspace ABI changes need to happen sometimes, or we will end up as another Microsoft Windows, loaded with past mistakes and design errors.

There are ways to lessen, or at least share that pain better, though. Such ABI breaks should at least be strongly documented, and tested for. Yes, break it. But know that you are breaking it, exactly how you are breaking it, and document it explicitly in a easy-to-find, standard place.

KS2007: The distributor panel

Posted Sep 7, 2007 18:23 UTC (Fri) by mrshiny (subscriber, #4266) [Link]

The thing is, ABI changes don't need to be forbidden, they should just happen less frequently. With the modern 2.6 kernel, EVERY SINGLE RELEASE can potentially contain ABI changes. At least in the 2.0/2.2/2.4 days there were more guarantees of stability for the lifetime of the kernel. This could be enough to get drivers out the door and into customer hands and simultaneously into the tree.

As another article mentioned, even if a vendor gets their driver into the tree it might be months or years before this driver gets into customer hands. In the meantime, for anyone currently running Linux, the driver may as well not exist because basically nobody runs kernel.org kernels. The only option is if the vendor backports the driver to popular kernels, of which there are dozens.

People keep comparing ABI stability to Windows, but they forget some things: 1. Windows isn't necessarily ABI-compatible from release to release, but it is supposed to be for the lifetime of one release including service packs.
2. Even if Windows maintains driver ABI forever, Linux doesn't need to do the same thing. But right now there is no ABI and not even a stable API (i.e. even recompiling the code may not work). Despite GregKH's arguments to the contrary I still think the kernel devs have made a mistake in abandoning API/ABI stability. We don't need to keep bad APIs or bugs around forever, but _some_ stability would go a long way.

KS2007: The distributor panel

Posted Sep 7, 2007 19:54 UTC (Fri) by vonbrand (subscriber, #4458) [Link]

Despite GregKH's arguments to the contrary I still think the kernel devs have made a mistake in abandoning API/ABI stability. We don't need to keep bad APIs or bugs around forever, but _some_ stability would go a long way.

Pray tell, how much is "some"? Whatever you decide, some parties will be unhappy...

KS2007: The distributor panel

Posted Sep 7, 2007 20:42 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]

quote:
The thing is, ABI changes don't need to be forbidden, they should just happen less frequently. With the modern 2.6 kernel, EVERY SINGLE RELEASE can potentially contain ABI changes. At least in the 2.0/2.2/2.4 days there were more guarantees of stability for the lifetime of the kernel. This could be enough to get drivers out the door and into customer hands and simultaneously into the tree.

did you actually use the 2.0/2.2/2.4 kernels that you are talking about? I did (I've been using Linux in production environments since around 2.0.30 and on my own systems prior to 1.0) and there was no more ABI stability between different releases then there is with the 2.6 kernel, every release changed something.

now the 2.6 development is much faster, so the amount of changes that took place in 2.0 in a year are taking place in a few months in 2.6, but they are spread across a larger code base as well.

and you are ignoring the fact that in the 1.2/2.0/2.2/2.4 days the kernels from RedHat, SuSE, and Linus were frequently not compatible with each other. you had to change userspace noticably when moving from one kernel series to another (the NPTL stuff is RedHat is the biggest example, but far from the only one).

the other big problem from the 1.2/2.0/2.2/2.4 days was the fact that you frequently couldn't run a stable kernel on your hardware, the driver (or a _working_ driver) was commonly only available in the development series (and developers, including leading ones like Alan Cox would tell you to run the development kernel rather then the stable kernel). I ended up deploying a 2.1.166 kernel on a production box that I mailed across the country to the data center because I needed the 3com NIC driver to work.

I'll take the current rate of change over these sorts of problems any day.

KS2007: The distributor panel

Posted Sep 8, 2007 20:52 UTC (Sat) by malor (subscriber, #2973) [Link]

This is precisely correct. This is the real problem. The Linux devs no longer have reliability as the central, core goal. It's A goal, but third- or fourth-tier... the primary goal is writing new code, making it a fun thing for developers to play with. When it went to the fast-release cycle, they declared their permanent derision for end users, the people who have to actually use this stuff in real life. In real life, stuff has to work, and the only way to make sure it does is with testing and slow development. A two- or three-month release cycle is a gigantic middle finger to the people that need to trust the code.

When the 2.6 kernels started to fall to shit, a lot of us complained. We were told that we weren't supposed to run kernel.org kernels, and and that the distributions would take care of testing and bugfixing. Instead of actually DEALING with the problem and shipping reliable code, they waved their hands in the air and expected others to handle it for them.

Well, guess what? Others are indeed handling it, but not how the poor kernel devs want. Awwww. The way they've found to deal with it is by not running kernel.org code; it's by using older releases and very, very carefully thinking about what to backport. The kernel people are mad because they got exactly what they asked for. The distributions are bugfixing and providing their customers a product that can actually be used.

Nobody in their RIGHT MINDS runs code straight from Linus anymore, because it's a seething, bug-infested mass of crap. The release cycle is ridiculously fast, the code quality has fallen to shit, and nobody can even be bothered to fix bugs.

And they DARE to complain that people aren't using their most recent product? This is insane. Ship code that WORKS and people will use it. Stuffing crap in a crate and expecting it to turn into gold by lots of user testing just infuriates your users, whose jobs depend on this garbage working. Every time.

Reliability is what made Linux successful in the first place; you could trust the 2.2 kernels with your life. Early 2.4 was terrible, and never got as good as 2.2, but by late in that cycle it was quite robust. 2.6 has been a steaming bag of crap.

Speed of change is not a useful metric of progress. Speed of RELIABLE change might be, but that's going to take a total shakeup of the broken dev process in place now.

KS2007: The distributor panel

Posted Sep 24, 2007 7:16 UTC (Mon) by turpie (guest, #5219) [Link]

I think part of the problem is Andrew Mortons -mm kernels have gotten out of control. At some stage they were a testing stage for almost ready code before it was sent to mainline. People didn't mind running it on their system as it was only slightly less stable than mainline, but now there is a much greater chance of problems so people can't be bothered using it. What is needed is a "stuff that will be sent to Linus for the next -rc1 release" branch to go between the current anything goes -mm and Linus trees.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds