Extensible scheduler class rejected

Posted Jul 27, 2023 19:53 UTC (Thu) by flussence (guest, #85566)
In reply to: Extensible scheduler class rejected by mb
Parent article: Extensible scheduler class rejected

BFS/MuQSS solved the situation a decade and a half ago, as does PDS/BMQ today. Asking for a BPF plugin system is a *concession*, and until it happens people are just going to keep applying experimental out of tree patches, because there continues to be a very real dollar cost to using CFS.

These things are fixing a 25%+ performance hit in CPU-bound multithreaded applications. It sounds insane, and it is insane that the kernel has fought to maintain the status quo for this long. DOI:10.1145/2901318.2901326

Extensible scheduler class rejected

Posted Jul 27, 2023 20:18 UTC (Thu) by mb (subscriber, #50428) [Link] (3 responses)

>there continues to be a very real dollar cost to using CFS.

Fix CFS.
Or keep your patches.
Where's the problem, really?

>and until it happens people are just going to keep applying experimental out of tree patches

That is a good thing.
These out of tree patches can evolve into actual CFS improvements.
The custom BFS hacks cannot.

Extensible scheduler class rejected

Posted Jul 27, 2023 20:20 UTC (Thu) by mb (subscriber, #50428) [Link]

>BFS
Typo: BPF

Extensible scheduler class rejected

Posted Jul 27, 2023 20:44 UTC (Thu) by djk121 (subscriber, #152710) [Link] (1 responses)

> The custom BPF hacks cannot.

They can, and already are. The SHARED_RUNQ patches for CFS were born out of sched_ext experiments: https://lore.kernel.org/all/20230710200342.358255-1-void@...

Extensible scheduler class rejected

Posted Jul 27, 2023 20:49 UTC (Thu) by mb (subscriber, #50428) [Link]

So? If anything, then this shows that this doesn't have to be merged mainline.
It works as-is as a hacking playground.

Extensible scheduler class rejected

Posted Jul 27, 2023 21:43 UTC (Thu) by pizza (subscriber, #46) [Link] (7 responses)

> These things are fixing a 25%+ performance hit in CPU-bound multithreaded applications. It sounds insane, and it is insane that the kernel has fought to maintain the status quo for this long.

Perhaps the reason those patches remain out of tree are because they cause serious regressions on other workloads?

Extensible scheduler class rejected

Posted Jul 28, 2023 4:19 UTC (Fri) by quotemstr (subscriber, #45331) [Link] (6 responses)

Thus the idea that a single scheduler might not be appropriate for all workloads.

Extensible scheduler class rejected

Posted Jul 28, 2023 6:28 UTC (Fri) by mb (subscriber, #50428) [Link] (5 responses)

Or the idea that these schedulers are just bad hacks that paper over a problem elsewhere.

The problem with a user programmable scheduler is, that this will hurt the overall ecosystem in the long term by creating thousands of "special" flowers.

Extensible scheduler class rejected

Posted Jul 28, 2023 9:51 UTC (Fri) by kleptog (subscriber, #1183) [Link] (4 responses)

> The problem with a user programmable scheduler is, that this will hurt the overall ecosystem in the long term by creating thousands of "special" flowers.

This is a terrible argument.

The problem with a user-programmable computer is, that this will hurt the overall ecosystem in the long term by creating thousands of "special" flowers.

The problem with a user-modifiable kernel is, that this will hurt the overall ecosystem in the long term by creating thousands of "special" flowers.

The problem with a user-configurable desktop is, that this will hurt the overall ecosystem in the long term by creating thousands of "special" flowers.

Unless there are really good reason, the trend is always to *more* configurability, not less. The problem is now that the people with the broken workloads can't fix the scheduler, and the people who can fix the scheduler don't know the workloads. At least with a generally accepted user-modifiable scheduler people could start communicating and collaborating about what works for them and we'd get progress, because the people with the workloads can actually try things out easily. As long as it's out of tree that will never happen, because you have no way to testing whether a change will affect normal users negatively.

Also, the idea that it's possible to build a single scheduler that works well for everyone is (from what I can tell) an assumption without basis.

Extensible scheduler class rejected

Posted Jul 28, 2023 15:54 UTC (Fri) by mb (subscriber, #50428) [Link] (3 responses)

>The problem is now that the people with the broken workloads can't fix the scheduler

You just identified the real problem.
Why can't they patch the scheduler?

Extensible scheduler class rejected

Posted Jul 28, 2023 21:56 UTC (Fri) by kleptog (subscriber, #1183) [Link] (2 responses)

> You just identified the real problem.
> Why can't they patch the scheduler?

I said 'fix', not 'patch'. Anyone can patch the scheduler, the question is if it still works afterwards.

Since the goal of the scheduler is to be the 'one scheduler to rule them all', the term 'fix' means to makes it better for all workloads, even those you've never seen. Most people with an unusual workloads aren't qualified for that, and probably don't care either. They just want it to work for their workload. If a BPF controlled scheduler allows them to solve their problem without having to learn any kernel coding, that's a win.

Ofcourse, my thought is: is anyone qualified to 'fix' the scheduler on these terms? Possibly not. It would have to evolve by slow evolution, and for that you need data. Which can get best gotten by running experiments. You can do that best if you have a mechanism that allows people to quickly and easily try out different things. Like, say, by being able to load little programs into the scheduler to change the behaviour on the fly.

Extensible scheduler class rejected

Posted Jul 28, 2023 22:39 UTC (Fri) by mb (subscriber, #50428) [Link] (1 responses)

>the term 'fix' means to makes it better for all workloads

So developing a BPF scheduler is automatically better for all workloads?

> If a BPF controlled scheduler allows them to solve their problem without having
> to learn any kernel coding, that's a win.

No. It's a hack.

> Like, say, by being able to load little programs into the scheduler to change the behaviour on the fly.

or by patching the freely available C source.

Anybody who can't code C, shall not attempt to develop a new core component of the kernel.

Extensible scheduler class rejected

Posted Jul 29, 2023 14:10 UTC (Sat) by kleptog (subscriber, #1183) [Link]

> >the term 'fix' means to makes it better for all workloads

> So developing a BPF scheduler is automatically better for all workloads?

Why would it be? It doesn't need to be, because it's not claiming to be the one scheduler to rule them all. I see it as tool that allows people to solve their problems and experiment with alternatives without having to rebuild their kernel each time. It's not trying to replace the existing scheduler. It rather makes it simpler, because then the main scheduler only has to be best for *almost* all workloads, rather than all.

Peter obviously disagrees with this assessment, that's his prerogative. If enterprise software was going to depend on certain scheduler configurations they would have done that already. "Thou shalt only run with the kernel we give you" is not exactly unheard of. Instead we now have the kernel developers requiring people to use out of tree patches, which hardly seems better.