A set of stable kernel updates [LWN.net]

A set of stable kernel updates

Posted Nov 22, 2010 20:07 UTC (Mon) by linuxjacques (subscriber, #45768) [Link] (1 responses)

:-\

I was hoping 2.6.27.56 would be patched so powerpc kernel would build with make 3.82 (as found in F14 and I'm sure several other recent distros).

A set of stable kernel updates

Posted Nov 22, 2010 20:30 UTC (Mon) by daney (guest, #24551) [Link]

All is not lost, Greg seems to do these releases fairly often. Just send him an e-mail saying which upstream patch you think should be pulled, and it is likely that it will be done.

A set of stable kernel updates

Posted Nov 22, 2010 20:48 UTC (Mon) by nix (subscriber, #2304) [Link] (8 responses)

My e1000e still doesn't work in 2.6.36 (spontaneous permanent loss of packet flow with register corruption in hours to days, no longer). I suppose I'd better ping the mailing list again.

A set of stable kernel updates

Posted Nov 22, 2010 21:35 UTC (Mon) by pr1268 (guest, #24648) [Link] (7 responses)

> I'd better ping the mailing list again.

That assumes you can get your e1000e working again, of course.

</dorky humor>

A set of stable kernel updates

Posted Nov 22, 2010 23:06 UTC (Mon) by nix (subscriber, #2304) [Link] (6 responses)

Pretty much. Wonderful, really: server-class headless hardware, more dependent on the network card than anything else I've ever owned -- and the network card has failed in *half* the releases since 2.6.30. It's more unreliable even than the old NE2k I used to own back in the day. Whether the drivers suck or the hardware is so irregular that even the manufacturer can't make it work for more than one kernel release at a time I do not know. Obviously I'll never be buying them again.

A set of stable kernel updates

Posted Nov 23, 2010 2:37 UTC (Tue) by mfedyk (guest, #55303) [Link] (5 responses)

what models of system do you have?

A set of stable kernel updates

Posted Nov 23, 2010 8:29 UTC (Tue) by nix (subscriber, #2304) [Link] (4 responses)

Only one e1000e, an 82574L, in a DNUK Netserver. So far, I've had jumbo frames cause a card hang (though to be fair that happened on other network cards as well at the same time), the card fail to come up at all, and now this instance of spontaneous card hangs at random times.

A set of stable kernel updates

Posted Nov 23, 2010 12:47 UTC (Tue) by nix (subscriber, #2304) [Link]

Sorry, that was a lie. There are two identical such cards in that machine, but one is only working at 100Mbps and has been trouble-free so I forgot it was there :)

A set of stable kernel updates

Posted Nov 23, 2010 14:52 UTC (Tue) by hmh (subscriber, #3838) [Link] (2 responses)

Well, the 82574L is a LOM chip for small designs, at most it should be in a desktop or cheaper workstation board. It shouldn't be found in a server board in the first place, IMHO.

Due to chip errata, try disabling MSI-X on the 82574L devices, use plain MSI instead. Also, make sure PCIe link power management (L0/L1) is disabled. lspci -vv can tell you about it. I believe you can disable MSI-X through a module parameter (I didn't check). Check whether that fixes the lock-ups, and report in the kernel bugzilla.

Also, make sure anything remotely related to ASF, and/or IPMI BMC that could be trying to use a side-channel on these devices is disabled in the BIOS.

Other things to test: jumbo frames (disable, limit to 3KiB...). Disable hardware offloading functions that would get disabled at 100Mbit/s...

It is a nice chip for a desktop or small workstation... *if* the driver AND the motherboard hardware works around the errata. It is not a server chip, at all.

BTW: here the 2.6.32.y driver IS using MSI-X for this device on the SuperMicro C7X58 motherboard. No lock-ups so far, but the MSI-X errata for this ship seems to depend on several external factors to cause problems, and I have very little network load in the first place so it might never have a chance to come into play.

A set of stable kernel updates

Posted Nov 23, 2010 17:35 UTC (Tue) by nix (subscriber, #2304) [Link] (1 responses)

L0/L1s is automatically disabled by the e1000e upstream driver. The e1000e devs suggested I try that. I did. No change.

MSI-X, well, hasn't that been on forever? Wasn't it on in 2.6.35? 'cos that works perfectly well. Jumbo frames have always worked perfectly well in earlier kernel versions and if they don't work the card may as well be useless so I hope they're not the problem! IPMI is a horror for many reasons and is already turned off (though it never interfered with the driver: my objection to it was more the BMC's habit of going into infinite loops at the drop of a hat and locking up, requiring *physical power removal* to reset it. Stupid closed-source crud.)

I'll try flipping off MSI-X and offloading, but, again, 2.6.35 works, and none of these things are new in 2.6.36 that I can see. PCIe-ASPM support *is* new but appears to be already turned off, and that doesn't help. The lockup has happened under high load and under no load at all.

(Regarding the 'not a server chip', maybe someone should tell DNUK about that. Even their great big server systems have e1000e's in them, or did last time I bought one from them, and they are widely known as 'good Linux vendors' in the UK.)

This discussion should surely continue on the e1000e list: lwn is the wrong place.

A set of stable kernel updates

Posted Nov 23, 2010 19:33 UTC (Tue) by hmh (subscriber, #3838) [Link]

Well, according to the errata documentation, MSI-X on the 82574L is not safe under load unless specific conditions are met. You might want to ask the Intel e1000e people about it.

Jumbo frames not working could easily be a driver bug instead of a hardware bug (which might have a work-around anyway).

As for DNUK also using e1000e on bigger boxes, well, that doesn't say much. The 82574L is just one of several chips that the e1000e driver can handle. We have some 82756-based NICs here, also handled by e1000e, and they are seriously nice and powerful server hardware. I just noticed Intel does claim 82574L is "for servers" in some of its docs (but not on all)... well, an ASF/IPMI side port a server NIC doesn't make IMO :-)

You're right about moving this to the e1000e list.

A set of stable kernel updates

Posted Nov 22, 2010 22:23 UTC (Mon) by eds (guest, #69511) [Link] (5 responses)

So much for 2.6.35 as the "Embedded Linux Flag Version," declared a mere three weeks ago.

A set of stable kernel updates

Posted Nov 22, 2010 22:30 UTC (Mon) by jengelh (guest, #33263) [Link]

My word exactly - http://lwn.net/Articles/413341/ , comment #1

Embedded flag version

Posted Nov 22, 2010 22:33 UTC (Mon) by corbet (editor, #1) [Link] (3 responses)

But Greg never said that he was going to maintain this version. Indeed, he said just the opposite. That maintenance will be done by the embedded companies involved; one assumes they'll be releasing updates on occasion. They are probably still putting their mechanisms in place.

Embedded flag version

Posted Nov 22, 2010 23:07 UTC (Mon) by eds (guest, #69511) [Link] (2 responses)

If the designated maintainer stops maintaining without having a designated successor, doesn't that leave the mess of having many slightly-different vendor kernels?

I don't see a lot of benefit to that situation. I thought that was the kind of thing the "flag version" scheme was supposed to end.

Embedded flag version

Posted Nov 23, 2010 10:26 UTC (Tue) by tialaramex (subscriber, #21167) [Link] (1 responses)

The "embedded flag version" is not Greg's baby. Thus it is also not Greg's responsibility. This isn't Microsoft, department A don't get to make promises and then expect department B to fulfil under the guise of being "team players". If the embedded vendors want a stable release maintainer, it's they who need to hire somebody, buy some build boxes, etc.

It may be that they have someone, and they're busy (after all right now there is a perfectly good stable tree of this version) or they're running silent to avoid stepping on Greg's toes.

It may be that they're still trying to get themselves sorted out, in which case they'd better hurry up if they don't want to get left looking stupid.

Or it might be that each individual representative went back to their vendor and got told "That's nice, but we'll sit this one out" and so the actual will to implement the plan does not exist and there will be no flag version.

You can lead a horse to water but you can't make it drink. If the embedded vendors say nice words but can't bring themselves to co-operate in order to save money that probably says more about embedded hardware vendors than it does about Linux.

Embedded flag version

Posted Nov 23, 2010 10:29 UTC (Tue) by tialaramex (subscriber, #21167) [Link]

... and apparently if I could read, I'd have noticed http://lwn.net/Articles/416700/ which makes it clear that it was the middle option. Good luck Tim, and I mean it.

A set of stable kernel updates

Posted Nov 22, 2010 23:49 UTC (Mon) by tbird20d (subscriber, #1901) [Link] (1 responses)

I plan to maintain the CE flag version for Linux 2.6.35. I'm waiting for an account on kernel.org right now. Please bear with me. I'm expecting a rather painful learning experience in the short term.
-- Tim Bird, Sony Network Entertainment

A set of stable kernel updates

Posted Nov 23, 2010 0:35 UTC (Tue) by vonbrand (subscriber, #4458) [Link]

A heartfelt thanks! I just hope you get help.