bufferbloat in the fediverse [LWN.net]

bufferbloat in the fediverse

Posted Dec 1, 2022 9:30 UTC (Thu) by mbunkus (subscriber, #87248)
In reply to: bufferbloat in the fediverse by mtaht
Parent article: Microblogging with ActivityPub

It's hard because it's really not common knowledge. No one talks about it, unlike fq_codel, which was very much a topic picked up by a lot of blog posts, by media outlets, and most importantly, it was made the default by the either the kernel itself or by distros.

Defaults matter so much.

For example, I've never heard of the recommendation of disabling ECN in favor of enabling BBR. I tried a quick Google search, and there wasn't really a lot of good information out there that proposed exactly this together with a easy-to-digest rationale. Sure, there are somewhat scientific explanations of how BBR works, going into details about packet loss & stuff.

Nor are there articles discussing in which type of situation these settings matter (only on machines connected to the internet? all machines? why?).

I'd love to read more about this. Do you have any pointers?

Bufferbloat is a rather complex topic. I'm not surprised that most sysadmins don't know much if anything about it, even less how to fix it properly.

In short: it really, really isn't obvious that this is something that Jane Sysadmin should do.

bufferbloat in the fediverse

Posted Dec 1, 2022 16:19 UTC (Thu) by mtaht (guest, #11087) [Link] (2 responses)

The BBR folk at google use ECN differently than RFC3168 and disable negotation of it entirely... while apple clients tend to request it, and it's enabled by default across the rest of the internet. There was a patch rejected long ago that did the right thing here, and other patches to just make BBR obey RFC3168, also rejected... I am seeing a lot of BBR uptake (say, 11% of websites), without also disabling ECN negotiation, which means that fq_codel, cake, etc go around marking packets madly instead of dropping them, to no observable effect when BBR is in play.

Mess. My rightest answer would be to make BBRv1 do RFC3168-style ecn, perhaps not dropping the rate by half as per that spec, but by initiating a probe phase. Others differ.

As for recommendations as to good defaults, a lot of the non-controversial fixes have landed in the kernel and become defaults, tsq, bql, pacing, fq_codel, etc.

and, BBR has been shown to be a good step forward for many applications, and I do think it's probably a better transport than cubic for the kinds of long running, periodically bursty, autonomous applications like those in the fediverse, and especially over lte. It's also been shown to be problematic in multiple respects, but BBRv2 is hung up in testing, partially hinged on the non-backward compatible treatment of ECN in it.

It would be nice if more people were aware of these issues, taking packet captures, and worrying about the impacts on the network.

I'm sorry that the amount of "publicity" and discussions these problems have got is not easily visible on today's internet. Perhaps talking about it here will help, and a few more beleaguered sysadmins and users lean in. Google does not index mailing lists like ietf's tsvwg or bufferbloat.net's bloat list all that well, anymore, with over a decades worth of discussion on each.

bufferbloat in the fediverse

Posted Dec 1, 2022 18:09 UTC (Thu) by mtaht (guest, #11087) [Link]

The BBR mailing list.

I am not sure if this BBRv2 status is up to date

bufferbloat in the fediverse

Posted Dec 2, 2022 18:15 UTC (Fri) by intelfx (subscriber, #130118) [Link]

This should be in all the wikis and in form of clearly digestible HOWTOs with rationales for administrators of all skill levels.

With all the immense respect owed to the great work you're doing to tackle the bufferbloat problem — it's a really, really, really obscure problem and, the solutions are not really discoverable unless you make it your explicit objective.