LWN: Comments on "Kernel prepatch 2.6.27-rc8"

e1000e bricking?

vonbrand — Wed, 01 Oct 2008 17:50:19 +0000

AFAIU, nobody really knows if this issue is in the kernel (in the driver or elsewhere) or if it is some weird interaction with userland.

Kernel prepatch 2.6.27-rc8

iabervon — Tue, 30 Sep 2008 17:11:04 +0000

I recently hit a regression going from 2.6.24 to 2.6.25, because of an overly-general quirk that would break old Lenovo 3000 N100s with old BIOS; 2.6.26 had already been released when anybody actually tried 2.6.25 on such a machine. At some point, you have to estimate that the known regressions aren't a big deal compared to the unknown regressions anyway; those will have to be handled as user support instead of QA, and so user support might as well take over the remaining known regressions, too, if they're sufficiently minor. Once the remaining regressions can be taken care of by -stable when they get fixed, it's time to do the next merge window.

Why would anybody even think about it?

JoeBuck — Tue, 30 Sep 2008 16:30:24 +0000

There's no danger of that; it's clearly a release blocker.

Kernel prepatch 2.6.27-rc8

bronson — Tue, 30 Sep 2008 16:11:14 +0000

You'll care about regressions the moment you hit one, I promise.

Kernel prepatch 2.6.27-rc8

pr1268 — Tue, 30 Sep 2008 15:41:33 +0000

Either a fix is found, or the driver needs to be reverted to its 2.6.26 form. Somehow.

That, of course, assumes that 2.6.26 isn't affected. I haven't read the 2.6.27 changelog (yet) so I'm unsure whether the e1000e driver has changed between these releases.

Kernel prepatch 2.6.27-rc8

felixfix — Tue, 30 Sep 2008 15:34:15 +0000

-- What has Ubuntu got to do with this?

I believe it was Mark Shuttleworth of Ubuntu who wanted releases done by the calendar instead of the software state. But I could be wrong; I don't pay much attention to Ubuntu.

Kernel prepatch 2.6.27-rc8

SEJeff — Tue, 30 Sep 2008 12:42:44 +0000

Yeah, one that (sadly) doesn't work all that well. They have went over it again and again, but the current form of kernel development works as well as it gets right now.

The biggest change recently was the -next tree. The only point of it is to make sure all sorts of git trees can actually build and play nicely together. Without it, Andrew Morton was starting to have trouble keeping up.

Kernel prepatch 2.6.27-rc8

dlang — Tue, 30 Sep 2008 12:04:44 +0000

in an ideal world every regression report would get tracked down and confirmed/eliminated before a release.

in practice this doesn't happen for several reasons.

sometimes the reporters don't have the time to respond

sometimes it a really odd race condition that's been around for years and just now triggered

some regression reports are hardware bugs/failures

and sometimes it's just not worth holding _everything_ up just to wait for one regression.

would you be willing to wait a year for the next kernel becouse of a bug in a ISA network card driver?

someone mentioned the debian release cycle, they don't really hit zero regressions before a release either, they will remove entire packages from a release if that package doesn't fix all the 'release critical' bugs. going from a working package to no package at all is definantly a regression.

as for the argument that you should back-out any changes that cause regressions, the problem is that sometimes you can't easily tell what change caused the problem. If someone who can duplicate the problem can do a bisect to find what change caused the problem it usually gets fixed very quickly (by backing out the change if nothing else works), but in cases where this isn't clear it's hard to do.

examples of some of the items on the regression list

Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11569
Subject : Don't complain about disabled irqs when the system has paniced
Submitter : Andi Kleen <andi@firstfloor.org>
Date : 2008-09-02 13:49 (26 days old)
References : http://marc.info/?l=linux-kernel&m=122036356127282&...

Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11512
Subject : sort-of regression due to "kconfig: speed up all*config + randconfig"
Submitter : Alexey Dobriyan <adobriyan@gmail.com>
Date : 2008-09-05 22:50 (23 days old)
References : http://marc.info/?l=linux-kernel&m=122065498013858&...

for a full report look at http://groups.google.com/group/fa.linux.kernel/browse_thr...

some of these look like they really should be fixed, some are questionable.

the biggest reason for Linus' comment is that experiance has shown that there is a point where waiting longer doesn't end up getting these things fixed any faster. (remember the year long 'freezes' of the kernel in prior development models?)

Why would anybody even think about it?

hmh — Tue, 30 Sep 2008 12:04:36 +0000

Yes.

And a regression that bricks *very* common hardware, and one that is usually embedded in mainboards (be them of servers, desktops or laptops) at that, is well into the "critical, release-blocker" category.

Yes, the kernel *always* releases with one or two regressions. But it will really be a rotten day for us Linux developers (PR-wise) if Linus decides to release with the current e1000e bricking issue.

Kernel prepatch 2.6.27-rc8

rvfh — Tue, 30 Sep 2008 12:02:53 +0000

What has Ubuntu got to do with this? Mandriva is in the same boat, and I guess many others (Fedora?). In fact we all are in the same boat!

Let's wait a bit, calm down, breath... Either a fix is found, or the driver needs to be reverted to its 2.6.26 form. Somehow.

Why would anybody even think about it?

eru — Tue, 30 Sep 2008 10:19:55 +0000

A regression is a special case of a bug, because you already have both a test case for it and a version of software that does not have the bug (the previous version). So it should be easier to fix. A regression also annoys people more than a new bug in a new feature or a previously unknown bug that also affects the previous version, because people expect the new version to be better or at least as good than the old one, but with regressions it clearly isn't.

Kernel prepatch 2.6.27-rc8

kragil — Tue, 30 Sep 2008 10:01:58 +0000

I think they really should postpone the release...

.. then again Mark is proud of releasing on time.

But not every Ubuntu user is always updating his/her system .. so I am not really happy ..

Why would anybody even think about it?

tialaramex — Tue, 30 Sep 2008 09:40:19 +0000

"Why would anybody even think about it?"

Because all non-trivial software contains bugs, and all subsequent releases will contain regressions. So the question is, at what point do the new features and bug-fixes of a new version out-weigh the regressions. If your answer is "never" then you don't need to argue with the LKML, the only conceivable solution for you is to run the same release forever.

Kernel prepatch 2.6.27-rc8

kasperd — Tue, 30 Sep 2008 09:01:20 +0000

I think the kernel should not be released with known regressions. Why would anybody even think about it? If it is just because it delays their possibility to start merging new features, maybe there should be a development branch that is always open for merges. That would lead to the following model:

There are three branches, which would currently be 2.6.26.x releases, 2.6.27-rc release candidates, and 2.6.28 development. Only fixes for crashes, security problems, and regressions are accepted for the first two, only the development branch gets new features.

Once the release candidate has been free from known regressions for a week or two, it gets promoted to an actual release. That would mean at once 2.6.26.x is discontinued. The latest 2.6.27-rc is pronounced 2.6.27. And the latest 2.6.28 development version is branched into 2.6.28-rc1 and 2.6.29 development.

And now that I have thought that whole thing through, it sounds a lot like the debian model. Did I just reinvent an existing development model?

Kernel prepatch 2.6.27-rc8

flewellyn — Tue, 30 Sep 2008 08:13:13 +0000

Thank you, Microsoft Quality Assurance Department.

Kernel prepatch 2.6.27-rc8

nix — Tue, 30 Sep 2008 07:34:46 +0000

Some of us would rather the kernel not melt our hardware without warning
nor allow other software to melt it, thanks. Or even someone else's.

It obeys the rules. Really?

khim — Tue, 30 Sep 2008 06:13:28 +0000

I've dounf that software which DO "obeys the rules" rarely have a regressions. Software which bends them is in trouble...

Kernel prepatch 2.6.27-rc8

jwb — Tue, 30 Sep 2008 02:39:10 +0000

One wonders if the e1000e problems will also hold up the release of Ubuntu 8.10. It's hard to imagine them shipping without 2.6.27, and it's also hard to imagine them shipping without e1000e.

Regarding the e1000e, it's very interesting to note how many bugs can be fixed if people just pay attention. I count patches for at least 3 separate bugs, so far, none of which would have been fixed except people are scouring the driver to find the source of the NVRAM corruption.

Perhaps the kernel would be improved by some kind of "driver emergency of the week"?

Kernel prepatch 2.6.27-rc8

jreiser — Tue, 30 Sep 2008 02:06:28 +0000

Who cares [about regressions]? I care. I want my software to run properly. It runs today; it obeys the rules. I expect it to run properly tomorrow.

Kernel prepatch 2.6.27-rc8

miguelzinho — Tue, 30 Sep 2008 01:02:11 +0000

Who cares if there is regressions!? Lets release the damn thing anyway.