User: Password:
Subscribe / Log in / New account

Kernel competition in the enterprise space

Kernel competition in the enterprise space

Posted Mar 15, 2012 9:44 UTC (Thu) by dag- (subscriber, #30207)
Parent article: Kernel competition in the enterprise space

What I think this article lacks to mention is the effect of newer kernels on userspace. While in theory kernels do not break binary compatibility, in practice means that if you switch kernels, a bunch of user-space tools need to get updated. Even up to udev and Gnome integration.

We, at the ELRepo project, offer backported drivers for RHEL and RHEL-clones (like CentOS, Scientific Linux and Oracle Linux) and one way of getting people to test newer drivers for us to backport is by providing mainline and stable kernels for RHEL. Our experience from testing these mainline kernels on RHEL5 is that the removal of certain infrastructure (in a specific case /proc/acpi entries or /dev/rtc nodes) would lead to various problems in userspace. For more information read Akemi Yagi's well written article "A kernel too far" at:

We did report those issues to the kernel developers, but there was little interested to get this fixed.

And while it may be easier for SuSE and Oracle to contain such problems in userspace than it is in kernelspace, in the long run even this is unsustainable and forward-porting older infrastructure may be necessary to avoid having to upgrade larger software parts. I look forward to the next 5 years and see how this plays out, but at least Red Hat's way of working, while expensive and tedious, proved to have worked out well. Oracle will have to give up its RHEL compatibility at some point though in its 10 year life-span.

Either you are leading the way or you are following, trying to manage to do both will tear you apart ;-)

(Log in to post comments)

Kernel competition in the enterprise space

Posted Mar 15, 2012 19:49 UTC (Thu) by iabervon (subscriber, #722) [Link]

I suspect that SuSE and Oracle may have better luck getting newer kernels to work with older userspace, in part because they have kernel developers on staff to prepare patches to newer kernels that fix the incompatibilities they find, and in part because they have more influence on the kernel development community. If, for example, SuSE wanted to fix sysfs in 3.0 to accommodate older udev, they'd have to find someone to write a change to sysfs stuff, and they'd have to get the sysfs maintainer to sign off on it, and they'd have to get the stable maintainer to include it. At the time, this would have been Greg KH getting himself to write a patch, and convincing himself and himself.

Also, SuSE intended from the release of SLES 11 with 2.6.32 to be able to upgrade the major release of the kernel, and exerted pressure to keep things in mainline from breaking SLES 11 userspace. It's a lot easier to NACK patches and revert commits that haven't seen an official release yet than revert behavior changes where different programs may have come to rely on each behavior.

Kernel competition in the enterprise space

Posted Mar 16, 2012 4:34 UTC (Fri) by dlang (subscriber, #313) [Link]

as someone who has been deploying custom kernels in production (in what decidedly qualifies as an "enterprise" datacenter) for 15 years, I can say that as long as I avoided some of the most convoluted RedHat systems (back in the kernel 2.1 and 2.3 days when the 'stable' kernel was so out of date that nobody could use it). There are really very few cases where a new kernel requires a userspace upgrade.

There are some cases where you need some new userspace tools to take advantage of some of the new kernel features, but Linus is pretty good at preventing changes that require userspace updates (and at the same time RedHat has gotten better at avoiding userspace incompatible kernel modifications)

I expect there to be very few problems with these updates

Kernel competition in the enterprise space

Posted Mar 21, 2012 20:09 UTC (Wed) by dag- (subscriber, #30207) [Link]

Well, for my RHEL6.2 system kernel 3.2 is already one kernel too far. The Intel chipset on my Thinkpad X201 starts flickering once Xorg/Gnome is started. Adding intel_iommu=igfx_off surpresses some kernel messages, but does not fix the screen flickering. Kernel 3.3 makes no difference.

And we're not even 2 years into RHEL6, 8+ years to go :-)

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds