LWN: Comments on "The end of tasklets"

Upside-down halves

ssmith32 — Sun, 11 Feb 2024 06:32:19 +0000

The top is the highest level?
Like taking an elevator to the top.

The end of tasklets

Wol — Wed, 07 Feb 2024 11:04:49 +0000

> which triggered investigation into alternatives to CFS, and then inspired Zijlstra to implement EEVDF in a way that matched or beat CFS while also providing latency-nice.

And this is why Torvalds is such a good manager (and steward). He's not attached to his version, and he actively encourages these short-lived forks precisely to find the best way to do things. Which he then shamelessly appropriates :-)

Cheers,
Wol

The end of tasklets

farnz — Wed, 07 Feb 2024 10:54:28 +0000

To some extent, the old thing of multiple competing forks of Linux provided the competition; if you have a choice between Torvalds, Cox, Kolivas and other kernels, where the Torvalds fork is the blessed version, and the Cox, Kolivas and other forks make different compromises to Torvalds. And then, the fact that for some workloads, Cox or Kolivas is "better" than Torvalds (but not for others) provides the impetus to work out whether there's a way to do better than all the forks include Torvalds' fork.

To a limited degree, we've seen this with EEVDF; there were the latency-nice patches floating around (and efforts made to get latency-nice to fit in with the design of the CFS scheduler), which triggered investigation into alternatives to CFS, and then inspired Zijlstra to implement EEVDF in a way that matched or beat CFS while also providing latency-nice.

Upside-down halves

amarao — Tue, 06 Feb 2024 21:51:10 +0000

It's very easy to understand where up and down is. Up is the same as the left-most, or the same as the front. Everyone can look on the byte and day where it begins and where it ends.

... (Missing xkcd about using intuitive 4d navigation abstractions here)

Upside-down halves

intelfx — Tue, 06 Feb 2024 21:41:25 +0000

That seems correct, but I was rather making a different point. I learned these words in different contexts, and so it simply doesn't "click" in my brain that top/bottom is "almost like higher/lower". These words are "in different parts of my brain", that's why I never experienced the confusion described in the sibling thread.

Upside-down halves

willy — Tue, 06 Feb 2024 18:10:09 +0000

Hm, I thought in Linux the "top half" was the part that submitted work, and the "bottom half" was the deferred interrupt processing. I've been around Linux over 25 years, and I've never seen anyone call the interrupt handler the "top half". I've seen plenty of code call it the "DPC" (Windows terminology, I believe)

The end of tasklets

willy — Tue, 06 Feb 2024 18:07:10 +0000

Oh, muffin. It was 4x100Mbit. If you used a more natural 1Gbit card, Linux came out on top.

But this is, was and always has been the game. I was part of it when I worked for Intel on Linux in the mid-2000s. Team A would work on Benchmark B and produce a result that beat Linux. So we'd take a look at what bottlenecks Benchmark B had on Linux, eliminate one, rerun the benchmark. Repeat until we beat Team A. Send patches upstream. Team A would typically come back to us a month or two later with an improved result and we'd repeat until either we or Team A lost interest.

Competition is healthy, and as long as Benchmark B represents a real customer workload (and you're actually eliminating bottlenecks, not putting in special hacks for Benchmark B), this is a win for everybody. The downside of Linux basically making every other kernel irrelevant is that we've lost that impetus.

See also LLVM vs GCC, Firefox vs Chrome, etc, etc.

Upside-down halves

antiphase — Tue, 06 Feb 2024 18:03:26 +0000

top is synonymous with highest or uppermost, as in they are superlative-like; similarly (but reversed) for bottom/lowest/lowermost.

higher/lower are comparative and don't indicate an absolute position, although you can argue that top and bottom are relative to one another as well.

Ah, English.

The end of tasklets

Wol — Tue, 06 Feb 2024 15:40:15 +0000

> Back in 1999, there was a Microsoft sponsored web benchmark from Mindcraft that showed that Windows NT and IIS were faster than Linux and Apache. This was fixed by DavidM by tuning and fixing the networking stack.

MS had noticed that the linux stack bottle-necked on a single CPU, even on a multi-CPU machine. So they improved their stack to use all four CPUs on some monster machine, and successfully flooded 4 Gigabit network cards.

They rapidly learnt not to try that sort of stunt. I can't remember how quick the community responded but, as with any perceived bugs in linux, there were fixes for the bottleneck within a day or so, and a proper solution went live within about a week. MS was left trailing in the dust ...

Cheers,
Wol

Upside-down halves

intelfx — Tue, 06 Feb 2024 15:24:38 +0000

In fact, I believe this is actually the original intention behind this naming. (Also, see my reply to @mathstuf regarding higher/lower level.)

Upside-down halves

intelfx — Tue, 06 Feb 2024 15:24:17 +0000

Hm, that's curious. I wonder if this is due to me not being a native speaker — for me, top/bottom and higher/lower (level) are completely disparate pairs of words, with little to no semantic overlap.

Upside-down halves

bertschingert — Tue, 06 Feb 2024 14:12:59 +0000

That actually does make sense, and I hadn't thought about it that way before. But my brain landed on and stuck with the "closer to the hardware = lower level" approach. After all, in OS diagrams, the hardware is usually at the bottom of the diagram, not the top.

The end of tasklets

syang — Tue, 06 Feb 2024 13:24:02 +0000

I guess we're, uh... deferring their execution from the kernel.

Upside-down halves

mathstuf — Tue, 06 Feb 2024 10:05:01 +0000

Another way to think about it is that interrupts are "closer to the hardware" which is "lower level". Once you come back "up" from there, you can do that deferred work.

Not saying that this is right, but I've never found mnemonics like these easier to remember than the raw fact itself. For example, the I vs O in the power symbol is actually better remembered as 1 for "on" and 0 for "off" but I kept getting it mixed up with | is an open wire ("off") and a O is a closed circuit ("on").

Upside-down halves

intelfx — Tue, 06 Feb 2024 08:36:43 +0000

> I always thought the Linux usage of top/bottom half was confusing and counterintuitive

How so, though? If you imagine the work that needs to be done in response to an interrupt as a function written in an imperative language, with control flow from top to down, then the "top half" is the part that is executed first — in the actual interrupt handler — and the bottom half is executed second (using some sort of a deferred execution mechanism).

The end of tasklets

epa — Tue, 06 Feb 2024 07:36:46 +0000

So you mean the work keeps getting deferred?

The end of tasklets

alison — Tue, 06 Feb 2024 05:38:05 +0000

The threaded sysfs attributes take NET_RX handling mostly away from softirqs. The rcu_nocbs kernel cmdline parameter takes RCU handling out of softirqs. In the last year, as previously noted by LWN, Frederic Weisbecker tried and failed (again) to pull out timer softirqs. If tasklets (and presumably also HI) softirqs could come out, that would leave ksoftirqd truly diminished. Getting rid of softirqs altogether is still a dream, but not as implausible as it once seemed.

The end of tasklets

iabervon — Tue, 06 Feb 2024 04:59:51 +0000

It looks like they're not changing how it runs, just how you write the code and how it's tracked in memory. If anything, context switches have gotten more expensive relative to everything else since 1999, but that doesn't mean that the method for avoiding them can't benefit from more recent API design lessons.

Also, it's now possible to have other user space tasks that are more important and urgent than your web server performance, so it's worth being technically able to preempt it if necessary even if you normally wouldn't.

The end of tasklets

shemminger — Tue, 06 Feb 2024 03:38:01 +0000

The rational for tasklets (and softirq) goes all the way back to the networking overhaul that took place in 2.4 development. Back in 1999, there was a Microsoft sponsored web benchmark from Mindcraft that showed that Windows NT and IIS were faster than Linux and Apache. This was fixed by DavidM by tuning and fixing the networking stack.

It was before my involvement in Linux, so don't know the exact details but heard that was the motivation to add softirq so that network SYN packets would get processed without expensive context switch to application.

CPU vs network speeds have changed in 25 years, so tradeoffs made then maybe different.

The end of tasklets

chexo4 — Tue, 06 Feb 2024 02:33:06 +0000

Wait, so how exactly would that have worked?

Upside-down halves

bertschingert — Tue, 06 Feb 2024 00:14:39 +0000

I always thought the Linux usage of top/bottom half was confusing and counterintuitive. It's nice to know I'm maybe not the only one who thought the other way around makes more sense.

Upside-down halves

klossner — Mon, 05 Feb 2024 22:04:56 +0000

Linus inverted the bottle. In classic BSD Unix and its successors, the bottom half is the interrupt handler and the top half is the thread to which work is deferred. I have to look up the terms to avoid confusion. See e.g. UNIX Programmer's Supplementary Documents Volume 2 (PS2) 4.3 Berkeley Software Distribution

The end of tasklets

rgb — Mon, 05 Feb 2024 20:33:23 +0000

Why did that idea not get anywhere?

The end of tasklets

willy — Mon, 05 Feb 2024 17:58:35 +0000

You missed timers! IIRC one of the proposals for killing tasklets was to make them a timer that expired immediately.

The end of tasklets

kees — Mon, 05 Feb 2024 17:56:42 +0000

Actually way earlier than that...
https://lwn.net/Articles/239633/

The end of tasklets

kees — Mon, 05 Feb 2024 15:43:29 +0000

We've been trying to get rid of tasklets for more than 5 years...
https://github.com/KSPP/linux/issues/94
I hope it gets completed this time! :)