Another push for sched

Another push for sched_ext

Posted May 12, 2024 15:15 UTC (Sun) by marcH (subscriber, #57642)
Parent article: Another push for sched_ext

> There is not a single doubt in my mind that if I were to merge this, there will be Enterprise software out there that will mandate its own BPF sched thing, or else it won't work.

And? What's new?

I bet the vast majority of CPU cycles "scheduled" by Linux already involve large amounts of closed-source software. Even when most web browsers (the OS on top of the OS) are now "open-core", (1) they still include a fair amount of closed-source code (2) The javascript they run is complex and minified (3) WASM is probably closed-source most of the time.

In other words, it's never been possible to reproduce and test "Enterprise" or any other real workload out of the box. If you want help from the maintainer and community, then you've always had to simplify, open and share your workload first. If no one can understand what you do then you're on your own, good bye. That's always been the deal. Simple.

Same logic with source code. Products almost always ship with custom stable branches with various backports and out of tree code. Even Linux distributions have always done this. So to engage from the community and get "free" support, you always had to switch to the latest commit on the main branch and to the maintainer's .config first (minor exaggeration to get the point across).

So I really don't see why a different logic would suddenly apply to custom BPF schedulers. If it's private then you're on your own as usual. Same thing if you share your BPF scheduler but maintainers think it's cr*p, as in "doctor, it hurts when I do this..." The answer to that question has never changed.

BTW what are the open-source test suites and workloads available for the scheduler? I'm surprised none was mentioned, it seems like a key element in that discussion.

to post comments

Another push for sched_ext

Posted May 12, 2024 20:17 UTC (Sun) by Wol (subscriber, #4433) [Link] (9 responses)

> maintainers think it's cr*p, as in "doctor, it hurts when I do this..." The answer to that question has never changed.

The problem comes when the patient says "doctor it hurts when I do ..." something that's necessary for normal life, like go for a walk. That regularly lands my grand-daughter in considerable pain. It could easily land my daughter temporarily paralysed on the floor.

And this is the problem. I get the impression there are quite a few important people in the Linux community who's attitude seems to be "the computer is there to run the OS. Who cares about the users" ... this discussion seems to have added another one to the list ...

"BOFH, my computer crashes every time I run our production software" - "Well, don't run your production software, then ..."

Cheers,
Wol

Another push for sched_ext

Posted May 12, 2024 22:51 UTC (Sun) by marcH (subscriber, #57642) [Link] (8 responses)

More like:

- ... crashes when I run my production software.
- Interesting! What is that software, what does it do?
- Can't tell you.
- .....

Or like:
- Here is the list of all the stupid things the documentation told me not to do and that I'm doing anyway with my custom BPF scheduler...
- Good bye!

Another push for sched_ext

Posted May 13, 2024 9:43 UTC (Mon) by Wol (subscriber, #4433) [Link] (7 responses)

> Or like:

"We're running this commercial software on stock <distro of choice>".

Like my daughter / grand-daughter going for a walk.

What then?

Cheers,
Wol

Another push for sched_ext

Posted May 13, 2024 14:02 UTC (Mon) by pizza (subscriber, #46) [Link] (6 responses)

> What then?

"Contact <commercial software vendor> for support."

(And, more often than not, the <vendor> will say "you're not using the supported OS/hardware we specified, goodbye.")

Another push for sched_ext

Posted May 13, 2024 15:15 UTC (Mon) by Wol (subscriber, #4433) [Link] (5 responses)

Are you sure?

I'm running stock Excel on stock Windows. Response times are absolute shite - to the point that we are in danger of it taking longer to run than the time available.

(Crap choice of software, I know. Not my choice.)

The most likely response from the vendors I know of is "ask the community".

But the point is, you're all coming up with POSSIBLE excuses for the vendor. What happens when the vendor's recommended, supported environment is "not fit for the purpose intended" to quote UK legalese? ESPECIALLY if said environment contains obvious flaws (not necessarily the vendor's fault) that would help you massively if they were fixed.

That was the complaint with ext4 years ago. That's the complaint with ext_sched now. That's the complaint with Rust. There are people who are actively obstructing attempts to improve Linux, because all they can see is their personal downside.

Are they Luddites? I don't know, I sincerely hope not. After all, the true Luddites could see the benefits of the technology they destroyed - that's why they destroyed it, because they could see the benefits would go to others, not them.

Cheers,
Wol

Another push for sched_ext

Posted May 13, 2024 16:49 UTC (Mon) by pizza (subscriber, #46) [Link] (4 responses)

> The most likely response from the vendors I know of is "ask the community".

The fact that the vendors you know are universally crappy doesn't change whose obligation it is to provide support.

Again, $user has a *commercial* relationship with $vendor. $user has no such relationship with "the community", which made $user zero promises, received zero compensation, and thus has precisely zero legal or moral obligation to give $user the time of day, much less provide support for a product they had nothing to do with.

Another push for sched_ext

Posted May 13, 2024 17:08 UTC (Mon) by mb (subscriber, #50428) [Link] (3 responses)

Yes. All correct.
Yet, the users will spam the community mailing lists, because that's the next logical step for them, if the vendor refuses support.

Why should we merge a feature that doesn't benefit the community?
If sched_ext benefits the community, I'm all for it. If not, why can't the exceptional use cases just ship a patched kernel?

Another push for sched_ext

Posted May 13, 2024 17:30 UTC (Mon) by pizza (subscriber, #46) [Link]

> Yet, the users will spam the community mailing lists, because that's the next logical step for them, if the vendor refuses support.

Yep. And that's what $vendors are counting on as it lets them save money on knowledgeable support staff.

(Mind you, user support request to vendors are not necessarily reasonable. After all, you wouldn't expect a company that makes hammers to "support" a customer complaining that the structure they are building keeps falling apart)

> Why should we merge a feature that doesn't benefit the community?

Every feature benefits someone. Unfortunately every feature also brings along costs that rarely fall onto the same someones that are reaping the benefits.

Those costs can be short term ("performance regression under every other workload") or longer term (technical debt, combinatorial complexity, security vulnerabilities with cutsey names, etc)

Another push for sched_ext

Posted May 13, 2024 22:02 UTC (Mon) by marcH (subscriber, #57642) [Link] (1 responses)

> Yet, the users will spam the community mailing lists, because that's the next logical step for them, if the vendor refuses support.

And? What's new?

> If sched_ext benefits the community, I'm all for it. If not, why can't the exceptional use cases just ship a patched kernel?

Agreed: if sched_ext turns out to be used ONLY by closed-source vendors then it shouldn't be merged. But that seems rather unlikely, doesn't it?

Another push for sched_ext

Posted May 25, 2024 0:16 UTC (Sat) by mrugiero (guest, #153040) [Link]

Maybe the same policy as with DRM drivers should be applied here. Give me some useful BPF programs (not just toy schedulers) that I may be interested in running, and then I may merge the code needed to run them. That's the safe bet, if you require that you already vanished the possibility of those magical cool schedulers never coming to the wider public.