|
|
Subscribe / Log in / New account

White paper: Vendor Kernels, Bugs and Stability

White paper: Vendor Kernels, Bugs and Stability

Posted May 19, 2024 17:32 UTC (Sun) by bluca (subscriber, #118303)
In reply to: White paper: Vendor Kernels, Bugs and Stability by mb
Parent article: White paper: Vendor Kernels, Bugs and Stability

> I didn't say that it is. I did not talk about this specific thing, because I don't know anything about it. I was talking about "do not break userspace" in the general form, not in this specific case.

This is very much about "do not break userspace" in the general form. It's the perfect example of why that mantra needs to be put to bed, once and for all, as it's completely disconnected from reality.

> Well, they pretty much do exactly that.

No, they very much do not. Look at all the enthusiastic comments from kernel people pointing to the paper in the article and saying "See? Vendor kernels are BAD, just upgrade to upstream kernels, it's fine really", and when told that new kernel version break applications and that's the real reason why vendor kernels are used, they shrug it away as "impossible, we do not break userspace"

> No. See my example of trace points.

Yes, it is exactly what you did, and there was no mention anywhere of trace points:

> Yes, it is annoying, if systemd/udev are affected by an interface change. Especially, if this interface change could have been avoided. But it's not the end of the world.
> Every other decades old application will continue to work. That is what counts.

You have made up a new rule according to which it's fine to break systemd or udev (if it's not made up, then just point to where on https://kernel.org/doc/ it is defined), but *unspecified other applications* must continue to work. That is very convenient of course, it's always unspecified other applications that are supported, and the ones that break are never actually supported. That's a very easy way of guaranteeing compatibility - every time something goes wrong just say that case was never actually supposed to continue working and move on.


to post comments

White paper: Vendor Kernels, Bugs and Stability

Posted May 19, 2024 18:10 UTC (Sun) by mb (subscriber, #50428) [Link] (11 responses)

>You have made up a new rule

There have always been exceptions and I didn't make that up. That's just silly. I even gave you an example (tracepoints).
It's up to you to ignore that. But please stop saying that I made it up.

I respect you for what you do for Linux, Systemd and so on. But you're acting like a child right now.

>it's always unspecified other applications that are supported

Yes. That is exactly like it is.

I understand that you are upset that the kernel apparently frequently breaks systemd/udev. But keep in mind that these applications are tightly coupled to the kernel. It's natural that these see more breakage than other average applications.
Yes, that is unfortunate and could certainly be improved.
But please don't generalize to other applications.

>every time something goes wrong just say that case was never actually supposed to continue working and move on.

That's not how things are done, though.
There have been reverts of ABI changes due to application breakages in the past.
It's done on a case by case basis.

Now you will reply: You have made up yet another rule!

White paper: Vendor Kernels, Bugs and Stability

Posted May 19, 2024 18:32 UTC (Sun) by bluca (subscriber, #118303) [Link] (10 responses)

> There have always been exceptions and I didn't make that up. That's just silly. I even gave you an example (tracepoints).
It's up to you to ignore that. But please stop saying that I made it up.

Literally nobody has mentioned tracepoints. I mean I'm not even sure that really qualifies as a userspace interface - maybe it does, it would seem strange, but I am not a tracing experts. But it is completely unrelated to mount options being removed.

> I understand that you are upset that the kernel apparently frequently breaks systemd/udev. But keep in mind that these applications are tightly coupled to the kernel. It's natural that these see more breakage than other average applications.

Says who? That is very much not true. Every interface that I can think of is used by multiple unrelated applications. I have no idea where you get this from. Cgroups and namespaces? Throw a rock in the general direction of a container runtime and you'll hit either or both. Netlink? There are as many network and interface managers as there are Linux vendors. Process management? That's been around since literally forever, and see the point about container management again. Mounting filesystems? fstab is older than me, I am quite sure.

'We do not break userspace, as long as userspace is a statically linked printf("hello world\n") /sbin/init' doesn't sound as catchy, does it now?

> It's done on a case by case basis.

I am well aware. And the triaging of that case by case goes like this: did it affect the machine that Linus happened to boot on that week? If so, it gets reverted and unpleasant emails are shot left and right. Else, nothing to see, move along.

White paper: Vendor Kernels, Bugs and Stability

Posted May 19, 2024 18:48 UTC (Sun) by mb (subscriber, #50428) [Link] (9 responses)

> Literally nobody has mentioned tracepoints.

What the? I did. I mentioned them as an example for a non-stable interface. After you have asked.

> I mean I'm not even sure that really qualifies as a userspace interface

Oh. I get it. *You* want to define what a userspace interface is and what not.
And everybody who disagrees is "making it up" or talking "nonsense".

That is silly.

> Says who?

Me. But I'm not sure why that matters.

> We do not break userspace, as long as userspace is a statically linked printf("hello world\n") /sbin/init

Well. I have never experienced a breakage due to a kernel interface change.
I run a two decades old binary and it still works fine.

That is my experience.

White paper: Vendor Kernels, Bugs and Stability

Posted May 19, 2024 19:31 UTC (Sun) by bluca (subscriber, #118303) [Link] (8 responses)

> I mentioned them as an example for a non-stable interface. After you have asked.

Again, I do not know the first thing about tracepoints and have zero interest in that. Maybe it's a supported interface, maybe it's not, I really cannot say, nor care, and can't see what it has to do with mount options.

> *You* want to define what a userspace interface is and what not.

No, userspace defines what is a userspace interface, as per Hyrum's Law.

> Me. But I'm not sure why that matters.

Because it's just wrong, as explained, there are no "special custom interfaces" being used anywhere, just bog standard stuff used by most components of an operating system.

> I run a two decades old binary and it still works fine.

'We do not break userspace, as long as userspace is mb's statically linked printf("hello world\n") /sbin/init' still not quite as catchy I'm afraid

White paper: Vendor Kernels, Bugs and Stability

Posted May 19, 2024 20:23 UTC (Sun) by mb (subscriber, #50428) [Link] (7 responses)

Ok. Everybody is wrong, except you. I got it.
I'll stop here.

White paper: Vendor Kernels, Bugs and Stability

Posted May 20, 2024 9:30 UTC (Mon) by LtWorf (subscriber, #124958) [Link] (2 responses)

I mean. The not breaking userspace only applies to a subset of things userspace can do.

I've had to fix software because of a kernel update, because some files in /sys were moved. But for some reason that doesn't count.

White paper: Vendor Kernels, Bugs and Stability

Posted May 20, 2024 9:41 UTC (Mon) by mb (subscriber, #50428) [Link]

> I mean. The not breaking userspace only applies to a subset of things userspace can do.

That is exactly what I was saying. Yet, I'm apparently wrong.

White paper: Vendor Kernels, Bugs and Stability

Posted May 20, 2024 9:45 UTC (Mon) by bluca (subscriber, #118303) [Link]

> I mean. The not breaking userspace only applies to a subset of things userspace can do.

Where is that subset defined?

White paper: Vendor Kernels, Bugs and Stability

Posted May 20, 2024 11:09 UTC (Mon) by wtarreau (subscriber, #51152) [Link]

> Ok. Everybody is wrong, except you. I got it.
> I'll stop here.

Welcome to discussions with bluca. Agressivity, half-reading of arguments, and accusations often arrive in the second or third message when he disagrees with you. There are such people who constantly criticize Linux and who would probably do good to the community by switching to another OS of choice :-/

White paper: Vendor Kernels, Bugs and Stability

Posted May 20, 2024 11:26 UTC (Mon) by bluca (subscriber, #118303) [Link] (2 responses)

You are literally trying to gaslight me into believing that dropping a mount option (not just making a noop, literally deleting it so that a hard error is returned where it wasn't previously), that is currently in use by userspace, is not a compatibility breakage. Make of that what you will.

White paper: Vendor Kernels, Bugs and Stability

Posted May 20, 2024 11:54 UTC (Mon) by mb (subscriber, #50428) [Link] (1 responses)

>You are literally trying to gaslight me

Wow. This is a new level.

Stop here please

Posted May 20, 2024 12:57 UTC (Mon) by corbet (editor, #1) [Link]

This clearly is not going anywhere useful, can we all let it go at this point, please?

White paper: Vendor Kernels, Bugs and Stability

Posted May 23, 2024 15:48 UTC (Thu) by anton (subscriber, #25547) [Link]

This is very much about "do not break userspace" in the general form. It's the perfect example of why that mantra needs to be put to bed, once and for all, as it's completely disconnected from reality.
Is it? When the breakage of existing code is reported as a bug, do the kernel developers declare the bug report as invalid, or do they fix the bug? If it's the latter, they live up to the principle. Sure, one might wish that such bugs would never happen, but apparently they feel that that going for that would be too constricting for kernel development.

Whether that means that vendor kernels are needed, or that one can use upstream kernels if one is selective about them is up to the vendors and their customers to decide.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds