bufferbloat in the fediverse [LWN.net]

bufferbloat in the fediverse

Posted Dec 1, 2022 10:04 UTC (Thu) by farnz (subscriber, #17727)
In reply to: bufferbloat in the fediverse by mtaht
Parent article: Microblogging with ActivityPub

It's hard because it's a change from defaults, and changes are always scary. If those changes are always a win, then they need to be the defaults, so that people use them automatically - I note that with defaults on Fedora Linux KDE spin, I get one of your three preferred changes by default:

$ sysctl net.core.default_qdisc net.ipv4.tcp_congestion_control net.ipv4.tcp_ecn
net.core.default_qdisc = fq_codel
net.ipv4.tcp_congestion_control = cubic
net.ipv4.tcp_ecn = 2

The trouble with asking people to make changes from the defaults is that it's never quite clear why something doesn't work as expected, unless you fully understand the changes. So why make a change when not changing works just fine from your perspective?

to post comments

bufferbloat in the fediverse

Posted Dec 1, 2022 16:51 UTC (Thu) by mtaht (guest, #11087) [Link] (3 responses)

Well, there was an effort to enable cubic + ecn (vs bbr - ecn) also.

More recently I'm happy to have heard that "backpressure" for microservices, within a machine, has arrived in the kernel as of 6.0, which renders part of the issues debated above moot. ( cilium is bragging about this. ) On the other hand, all the developers of microservices I've met so far, seem to think backpressure exists for other things outside the box, or other protocols outside of tcp, and it *doesn't.

I've tried to engage with the cilium folk a couple times now.

bufferbloat in the fediverse

Posted Dec 2, 2022 14:16 UTC (Fri) by farnz (subscriber, #17727) [Link] (2 responses)

I think, personally, that a core part of the issue is that most of the fixes for bufferbloat involve making metrics worse for a few edge cases (packet loss counters when at saturation, for example), in return for a big improvement that's not hugely visible in metrics for the vast majority of traffic.

And so, where you're asking people to change from the defaults, you're asking them to regress one or two metrics they've "always" paid attention to, without improving other metrics they pay attention to. This is obviously a bad things to do - why make the numbers worse? - and thus you struggle unless your changes are the defaults (at which point, when they look into their tweaks, they find they make the metric better at the expense of something they actually care about).

Basically the traditional confusion between a metric and an outcome :-(

bufferbloat in the fediverse

Posted Dec 2, 2022 14:53 UTC (Fri) by gioele (subscriber, #61675) [Link] (1 responses)

> I think, personally, that a core part of the issue is that most of the fixes for bufferbloat involve making metrics worse for a few edge cases (packet loss counters when at saturation, for example), in return for a big improvement that's not hugely visible in metrics for the vast majority of traffic.

Isn't that addressed by publicizing another contrasting metric?

"Yes packet loss counters went 2.7% up, but 95%-ile latency is down by 84.3%!"

bufferbloat in the fediverse

Posted Dec 2, 2022 15:37 UTC (Fri) by farnz (subscriber, #17727) [Link]

My experience of debloating a network that's not a bottleneck is that that's not actually what you get; packet loss goes up 4%, but 99-%ile throughput and latency are unchanged. And it's hard to explain that this is actually a win - that what I've actually done is get you to a point where instead of needing expensive network upgrades when traffic doubles, you can hold off until traffic triples - because that's something in the far future.

Fortunately, the network I debloated was my home network, so while I can see that no metrics have improved significantly, I'm happy that I'll be able to run on the existing hardware even as demand increases.