bufferbloat in the fediverse [LWN.net]

bufferbloat in the fediverse

Posted Dec 1, 2022 16:02 UTC (Thu) by flussence (guest, #85566)
In reply to: bufferbloat in the fediverse by mtaht
Parent article: Microblogging with ActivityPub

In my experience (not in any way an active server, just a bedroom one) it's like IPFS traffic but in unpredictable, coordinated bursts. “Going viral” has a perverse disincentive because you're liable to get slammed by hundreds if not thousands of remote https connections over the course of a minute in response to any type of remote interaction; almost all of this server software uses grossly misconfigured HTTP client libraries - HTTP/1.1, no compression, no persistent connections. If you've posted a twitter-style long thread that receives attention it's even worse, there's no request batching at any layer of the stack. Mastodon in particular likes to tack on a dozen or two “just-in-case” metadata lookups on users it hasn't seen before (and then proceeds to discard the data only to do it again next time).

My network is as close to an ideal software setup as I can get — to cope with a negligent rural monopoly ISP — but this stuff still manages to knock it down at times. Cake works miracles with other types of traffic but it can't fix lazily-written garbage.

to post comments

bufferbloat in the fediverse

Posted Dec 1, 2022 17:57 UTC (Thu) by mtaht (guest, #11087) [Link] (1 responses)

Dear flussence:

As a long term cake fan, perhaps your ISP would be interested in a beta of libreqos.io?

It's always been a saner idea to have the ISP shape the down, and the CPE shape the up, but until recently it was very difficult to shape the down at ISP rates, but with XDP, ebpf and a few other tricks, it's now pushing 11Gbit/sec, 10k mixed fiber/FWA users, at 24% of 16 cores, aiming for 40Gbit/sec and 40k users in the next release I'm deleriously happy with where it's been going, you can find the devs at #libreqos:matrix.org.

I would hope that running cake at the isp would mediate far further depredations of the network as you describe, with ipfs and the fediverse as inbound shaping at the CPE simply can't cope, and the per host/per flow fq in cake would really moderate the impact, or so I hope.

I'd love to have a semi-repeatable test case for some of the behaviors everyone is describing, and to see what libreqos can do to help.

bufferbloat in the fediverse

Posted Dec 8, 2022 0:44 UTC (Thu) by flussence (guest, #85566) [Link]

Haha, my ISP is the type from hell that's never heard of IPv6, let alone QoS. There's no way to get hold of a technical contact as an end user, I've looked — because they don't have any. It's all outsourced to Cisco consultants, even actual line faults take weeks/months to get resolved. You could say their chain of command has a severe bufferbloat problem :)

I have a 16Mbit/830kbit DSL connection which hasn't seen a physical upgrade since the late 2000s. They've *started* building out fibre here where they charge obscene rates for less than a 3× increase *each way* (asymmetric FTTP! what a load of BS!), but in any case it doesn't seem like they have many choke points except the last hop they scalp their customers for. My latency's been consistently low even with this fediverse stuff. At this point I just wish they'd stop starving us of bandwidth…