LWN: Comments on "How to unbreak LTTng"

How to unbreak LTTng

mpr22 — Tue, 28 Apr 2020 22:06:37 +0000

Modules include things like "device driver to operate an FPGA payload that has less than one monkeysphere of users, most of whom know the driver author personally".

How to unbreak LTTng

ecree — Tue, 28 Apr 2020 21:35:23 +0000

> It’s (in many cases) a huge amount of ongoing work

Maintaining an in-tree module is *way* less work than maintaining an out-of-tree one. Heck, most of the time you just have to say "Ack" on the patches that the person changing some kernel infrastructure _writes for you_ to keep your module working. Whereas out-of-tree you end up with elaborate compatibility scripts, that you have to keep updated without any outside help, unless you're happy for your module to either only build on the latest kernel, or be tied to 2.6.32 forever 'cos that's what was ubiquitous when you started the project.

If it's tight enough with the kernel to be a module (rather than a userspace executable, using the APIs that are actually designed to be stable), then it belongs in the upstream tree; and if it's not good enough quality to be upstream (even in staging or behind a 'default n' kconfig) then it doesn't belong anywhere.

How to unbreak LTTng

ballombe — Fri, 24 Apr 2020 16:00:28 +0000

> You can still freely use AOSP and ignore those requirements as long as you don't call it Android, and some large companies that compete with Google already do that, which makes it harder to argue that Google is being anti-competitive here.)

True, however if you do not call it Android, you cannot ship the Google apps with it. So there are some real consequences.

How to unbreak LTTng

pizza — Wed, 22 Apr 2020 21:15:33 +0000

> which makes it harder to argue that Google is being anti-competitive here.

You forget that Google is obviously the only reason why $DomesticBusinessSector is not making lots of money, so any restrictions they impose upon folks using their stuff is clearly anticompetitive behavior that must be harshly punished.

(I'm not saying I agree with this, but many others do)

How to unbreak LTTng

excors — Wed, 22 Apr 2020 20:26:19 +0000

Google already has a big list of "if you do that, you can't call it Android" requirements, including requirements on specific Linux kernel features: https://source.android.com/compatibility/android-cdd

(You can still freely use AOSP and ignore those requirements as long as you don't call it Android, and some large companies that compete with Google already do that, which makes it harder to argue that Google is being anti-competitive here.)

How to unbreak LTTng

Wol — Wed, 22 Apr 2020 18:41:16 +0000

Well, Google is just using Trademark Law in exactly the manner it was meant to be used. All those companies complaining won't get very far in any sane jurisdiction (yes, I know, since when can any country claim to have a sane jurisdiction ...)

Cheers,
Wol

How to unbreak LTTng

corbet — Wed, 22 Apr 2020 16:38:42 +0000

That is already a part of the plan as I understand it. The kernel will become part of the generic system image, so it's provided by Google, not the device vendors, who are limited to providing kernel modules. They they will indeed not be able to set that config flag.

How to unbreak LTTng

pizza — Wed, 22 Apr 2020 16:33:14 +0000

> Google can easily stop that. There's nothing to prevent vendors doing it, but Google can just say "if you do that, you can't call it Android".

...And then those companies start complaining to their various national anti-trust bodies, and Google gets threatened with billion-dollar fines.

How to unbreak LTTng

Wol — Wed, 22 Apr 2020 14:42:56 +0000

> Why? Sure some decisions may make there life harder but "punish" implies an "intention to hurt". What makes you think this is the case rather than just trying to do what is best for the kernel as a whole (even if that makes things harder for out of tree modules)?

Because there's evidence that some kernel maintainers, at least, do not make these decisions based on technical evidence, but on how much grief it's going to cause people that these maintainers perceive as "bad actors".

The fact that it causes a lot of grief to innocent bystanders does not get taken into account.

Cheers,
Wol

How to unbreak LTTng

Wol — Wed, 22 Apr 2020 14:38:31 +0000

> Is that really adequate? Wouldn't phone vendors just set the config flag?

Google can easily stop that. There's nothing to prevent vendors doing it, but Google can just say "if you do that, you can't call it Android".

Cheers,
Wol

How to unbreak LTTng

pbonzini — Wed, 22 Apr 2020 11:52:37 +0000

Is they can set the flag they can also patch the kernel to export whatever symbol they need.

How to unbreak LTTng

neilbrown — Wed, 22 Apr 2020 00:25:30 +0000

Quoting from https://lwn.net/Articles/813350/

> But that only holds if modules are restricted to the exported-symbol interface; if they start to reach into arbitrary parts of the kernel, all bets are off. Deacon doesn't say so, but it seems clear that some vendors are, at a minimum, thinking about doing exactly that.

So while I agree with what you say, I don't think it is relevant to my comment.
Thanks.

How to unbreak LTTng

nilsmeyer — Tue, 21 Apr 2020 16:10:21 +0000

> Generally excluding duplicate functionality seems to be a good idea for long term maintenance and code size surely.

What about SELinux and AppArmor (and Smack and TOMOYO)? There are probably a lot of other examples. It often feels arbitrary to the outside observer,

How to unbreak LTTng

Paf — Tue, 21 Apr 2020 14:09:02 +0000

“ Why? For non-free ones sure but you explicitly said free modules.”

It’s (in many cases) a huge amount of ongoing work, requiring not only commitment and effort but persuasion of others that you have commitment and effort.

The idea is that not everything that exists is up to the standards for mainline inclusion, and not everything that exists is going to avoid duplication to the level that’s appropriate for the mainline.

This is “let a thousand flowers bloom” stuff - it’s about being open to a world of stuff beyond the domain of what’s good enough to be a permanent part of mainline. Hobby projects (that still have outside users), small stuff, differing approaches to existing functionality that are not clearly better in general (see LTTTNG for an example of that) but are preferred by some users, etc.

I recognize this is a sticky issue (well, sort of), but this is still a choice to be more closed.

How to unbreak LTTng

mfuzzey — Tue, 21 Apr 2020 13:28:11 +0000

>LTTng is a good example: there is already a tracing subsystem in-tree, so let's exclude any duplicate functionality.

Generally excluding duplicate functionality seems to be a good idea for long term maintenance and code size surely.

When competing solutions exist the best bits of each can be, and often are it seems to me, merged to make a better in tree solution.

And in this particular case Steve Rostedt was quoted in the article as being open to including LTTng

> In-tree status is unrealistic for many free kernel modules

Why? For non-free ones sure but you explicitly said free modules.

> to punish out-of-tree developers.

Why? Sure some decisions may make there life harder but "punish" implies an "intention to hurt". What makes you think this is the case rather than just trying to do what is best for the kernel as a whole (even if that makes things harder for out of tree modules)?

> with the lack of a stable kernel API

The reasons for this are well documented and make sense to many. This is really part of the Linux philosophy today I'd say. No one forces anyone to use Linux.

How to unbreak LTTng

smurf — Tue, 21 Apr 2020 12:11:52 +0000

The "all Android vendors hack their kernel" idea is on the way out. The idea is that vendor specific kernel modules (required to access vendor specific hardware) live in a /vendor partition. The device should otherwise use a generic Android kernel.

How to unbreak LTTng

ncultra — Tue, 21 Apr 2020 12:08:14 +0000

All of this sounds reasonable at first glance, but there are a couple points that are troublesome to me. Hellwig's objection declares the requirement for any exported symbol to have an in-tree user. Yet, in-tree status is arbitrary and exclusive. LTTng is a good example: there is already a tracing subsystem in-tree, so let's exclude any duplicate functionality. Any one of a clique of long-time maintainers can essentially veto a patch series submitted for upstream consideration. In the end, the decision (or lack of decision) is up to a single individual. There is a universe of free, out-of-tree modules that are useful but disadvantaged by this reality. In-tree status is unrealistic for many free kernel modules. Secondly, this is about in-tree symbols that were removed, not simply rejected. It is therefore punitive in addition to exclusionary. This is the latest in a series of technically dubious policies driven by certain maintainers, notably Kroah-Hartman, to punish out-of-tree developers. Maintaining a driver or module for Linux is a unique experience, in a bad way, with the lack of a stable kernel API and, not for the first time, removal of existing symbols.

It gets worse when considering the purpose of a symbol such as kallsyms_on_each_symbol() which is to be a support to developers, and is critical for maintaining a driver or module that can be compiled and linked on multiple kernel versions. This is an accepted practice that is now made much more difficult. In my opinion, this philosophy that punishes out-of-tree developers (regardless of license choice) is counter-productive to the continued existence of Linux as we know it.

How to unbreak LTTng

dyfrgi — Tue, 21 Apr 2020 11:50:02 +0000

> My preference would be that kallsyms_lookup_name(), could be exported or not depending on a CONFIG option. Android could set that to hide the function and LTTng wouldn't work on Android.

Is that really adequate? Wouldn't phone vendors just set the config flag? Though tbh I also don't understand why they wouldn't just patch the kernel. My understanding is that they all do it for hardware support anyway.

How to unbreak LTTng

neilbrown — Tue, 21 Apr 2020 01:05:41 +0000

> things like that would be distributed as a patch set against the upstream kernel, and users would be expected to recompile their kernel with it.

I suspect that today a more realistic approach is to ask the various distributors to apply that patch to their kernels.

I think it is worth reflecting for a moment on the motivation behind these changes. They seem to be coming from Android. The Android kernel has good reason to lock down the exported inferfaces so that phone vendors cannot 'abuse' them. I fully support that work, but don't think that it should necessarily impose what I do on my device or what a distro does with their supported kernel.
So if distros want to patch out these restrictions, that might make perfect sense.
Of course it could work the other way around - Android could patch in these restrictions. But we have a long history of trying to bring Android back to mainline - and requiring them to patch in the restrictions would hurt the momentum we have.

My preference would be that kallsyms_lookup_name(), could be exported or not depending on a CONFIG option. Android could set that to hide the function and LTTng wouldn't work on Android. Probably no loss there.
Other distros could set the config option the other way and LTTng would work fine on them.
When I build my own kernels, I can (of course) do whatever I want.
Problem solved.

How to unbreak LTTng

cesarb — Mon, 20 Apr 2020 23:05:07 +0000

> unable to function outside of the kernel, and unable to be merged.

Back in the day, things like that would be distributed as a patch set against the upstream kernel, and users would be expected to recompile their kernel with it. Is that no longer an option?