LWN: Comments on "Tridge returns to rsync"

discord, bitkeeper and choices

farnz — Tue, 16 Apr 2024 15:46:05 +0000

And there's a social aspect that gets lost, too. People who prefer using Linux to other OSes correlate well with people who find ways round stupid IT policies that don't work for the business (as you did), and also are more likely to expect a decent explanation when they ask why an IT policy is set the way it is.

That, in turn, puts you in conflict with an IT department that's used to being able to control the business rather than serve it; you're going around their restrictions when they get in the way instead of following whatever arcane and unmanageable process they've built to get the restriction changed (e.g. requiring legal review and sign-off at director level), and you're more likely to ask hard questions about why those restrictions exist when you do go through the arcane process - which, in turn, puts them in the hot seat where they have to actually justify what they do.

In companies where the IT department isn't incompetent (I've worked in some), you don't have these problems, because they consider themselves as supporting the business needs, and you wouldn't be asking for Linux if there wasn't a good business need for it.

discord, bitkeeper and choices

pizza — Tue, 16 Apr 2024 15:06:09 +0000

More than knowledge/experience, it's also a matter of _willingness_ -- $dayjob-1, you had to get director-level signoff (After a complete legal review) to use Linux in _anything_, and the entire (Windows-only) IT department was (doubly) outsourced and run with an attitude that made the "computer says no" trope downright friendly in comparison. Small wonder my research group built our own shadow IT organization, complete with our own internet connection (faster, more reliable, _and_ more secure as it turned out), just so we could do the jobs we were hired for.

At $dayjob-2, despite most of the couple-thousand-strong engineering staff requiring desktop Linux applications to do their jobs, you had to get special Group-manager-level approval to get a native Linux desktop versus remoting into the (massive) compute farm. And there was no officially-sanctioned portable Linux solution (in part because the VPN was Windows-only) Which is a problem when you need direct-attached peripherals and there's a >1s ping at the customer site in Western Elbonia..

Tridge returns to rsync

marcH — Tue, 16 Apr 2024 14:47:32 +0000

> We ask that comments be polite and respectful, and nationalism is not something that aligns with those values.

Of course the thread later descended into off-topic politics and proved you right (*sigh*) but luna's initial comment was polite, respectful and not "nationalist". I have no idea whether it was factually correct or not but it was also on-topic considering the news was about rsync development. I use Discord and I like it but whether some Chinese company has some level of control on it is something I find relevant and interesting considering China is currently a strong dictatorship (among many others) exerting a very tight control on speech (among other control). Just as interesting as what the NSA and other western agencies do with tech companies - legally and illegally.

More generally speaking, technology and politics interact a LOT and banning the latter entirely in the comments would make some discussions pretty pointless.

discord, bitkeeper and choices

marcH — Tue, 16 Apr 2024 14:24:34 +0000

I think theses "capabilities" are more about knowledge and experience than about company size. Sysadmins across the world and industry have been managing Windows for decades. I wouldn't say they're "good" at it considering how incredibly complex hence unmanageable Windows is but for sure they feel "comfortable" doing it.

discord, bitkeeper and choices

mjg59 — Fri, 12 Apr 2024 23:03:45 +0000

We've got pretty much the same set of capabilities at my current gig, which is at the low end of medium sized.

discord, bitkeeper and choices

pizza — Fri, 12 Apr 2024 22:21:51 +0000

> [Google] had no difficulty enforcing the same level of management [of bare-metal Linux] as we did on Windows or Mac.

...I don't think Google should be used as an example of the capabilities of a typical medium-to-large corporate IT department.

discord, bitkeeper and choices

mjg59 — Fri, 12 Apr 2024 21:37:14 +0000

I was involved in helping manage Google's bare-metal corp Linux deployment, which was somewhere on the order of 100,000 systems. We had no difficulty enforcing the same level of management as we did on Windows or Mac. WSL2 is an extremely convenient opaque object that has access to host files and can't meaningfully be monitored by any local agents. I agree that it has all sorts of conveniences, and some thought has been put into how it interacts with complicated corporate environments, but without the ability to enforce specific images the level of control you have is limited to what Windows enforces on it.

discord, bitkeeper and choices

pizza — Fri, 12 Apr 2024 20:08:52 +0000

> WSL2 is much less manageable than an actual Linux distribution - it's not possible to enforce a specific distro image (you can ship your own, but you can't prevent users from booting other ones), so you're not guaranteed any management within the Linux image.

It's a _lot_ more manageable than a bare-metal (or dual-boot) Linux installation would be.

All the usual corporate checkboxes can be maintained (Bitlocker/managed FDE, enforced firewall/intrusion detection/data exfiltration suites, bla bla bla). Users get their own wholly-contained sandbox to be root in, but are prevented from mucking around with the super important stuff like the mandated backgrounds on the lock screen.

discord, bitkeeper and choices

mjg59 — Fri, 12 Apr 2024 03:51:37 +0000

WSL2 is much less manageable than an actual Linux distribution - it's not possible to enforce a specific distro image (you can ship your own, but you can't prevent users from booting other ones), so you're not guaranteed any management within the Linux image, and there's no way to pass through access to the host certificate store so you can't tie a given WSL instance to its host. To be fair, Crostini in ChromeOS has the same issues (hence the somewhat ironic situation of Google banning the use of Crostini in corp environments).

discord, bitkeeper and choices

tridge — Fri, 12 Apr 2024 03:01:09 +0000

Larry was quite aggressive in enforcing what he thought his license meant, regardless of the legality of it. At one point he threatened to "have me arrested" if I ever visited the USA. He had no basis to do so as I had never agreed to the bkl and what I did was completely legal, but it could still have caused me a lot of inconvenience trying to fight him, so I cancelled one trip to the US. My wife was attending a conference in Hawaii and I had planned to go along to do some tourism. I cancelled my trip as I just didn't want to deal with Larry's craziness. I still haven't seen Hawaii :-)
It seems like Larry had mellowed somewhat since those times.

discord, bitkeeper and choices

Cyberax — Fri, 12 Apr 2024 01:52:57 +0000

People are switching to WSL from the regular desktop Linux, that has never been anything but a footnote in the overall market (thanks, GNOME and KDE folks for that!). This market is so secondary right now, that Microsoft is literally switching it to free-to-play model with microtransactions^W ads for monetization.

The main Linux domain are servers and the growing AI/ML market, and Microsoft has nothing to offer there. The WSL itself is designed to stave off Windows' obsolescence in that area, with DirectX passthrough primarily designed for the AI/ML workloads (and not the graphical UI).

discord, bitkeeper and choices

pizza — Fri, 12 Apr 2024 01:25:56 +0000

> How would it look like in the case of WSL? I just don't see it. What can they do that would make users switch to Microsoft Linux?

Because (1) it's convenient; (2) it's managed by group policies covering all corporate buzzword salad security/management BS; (3) hardware compatibility no longer matters -- including native DirectX from within Linux[*] (4) nearly entirely painless setup (no dual booting) and data passthrough, (5) You don't have to pay extra for Virtualbox or VMWare or other stuff, etc etc.

It's a breathtakingly brilliant play, especially pushing APIs that only work when running under WSL.

[*] https://devblogs.microsoft.com/directx/directx-heart-linux/

discord, bitkeeper and choices

Cyberax — Fri, 12 Apr 2024 00:49:29 +0000

> (And then there's the minor detail of effectively making Linux "just another feature" of Windows; the classic EEE play.

The "EEE" has been overhyped for at least 2 decades. It almost never works, unless you have a lot of market force and you keep the changes proprietary.

Basically, you need to convince everyone to use your changed version (which needs to be genuinely better than the status quo), while preventing others from easily duplicating its features. So EEE could have worked with Java, and it almost succeeded with IE6. But it has zero successes afterwards.

How would it look like in the case of WSL? I just don't see it. What can they do that would make users switch to Microsoft Linux?

discord, bitkeeper and choices

pizza — Thu, 11 Apr 2024 18:59:52 +0000

> WSL2 isn't competing with Linux desktops, it's competing with Macbooks. They're going for the market of yuppies that spend 200 hours decorating their terminal prompt and terminal editor while blogging about the process all on a disposable $2k laptop.

No, WSL2 is going for the corporate market, to ensure IT departments have no reason to buy or deploy anything other than Windows for non-C-suite folks. (And of course that includes Macs)

Tridge returns to rsync

excors — Thu, 11 Apr 2024 16:12:18 +0000

Discord server admins can choose different levels of verification, with the highest requiring a verified phone number (plus a verified email address and being a server member for at least 10 minutes), which is useful for large public servers that have a problem with spam bots, scammers, trolls, etc, as it lets moderators ban people more effectively. (Source: https://support.discord.com/hc/en-us/articles/216679607--...)

Discord might also require verification if it thinks you're behaving in a suspiciously bot-like manner (e.g. rapidly joining many servers and sending many messages, or using "third party clients or modifications to Discord"). (https://support.discord.com/hc/en-us/articles/61817268882...)

If you haven't joined large servers and haven't triggered the suspicion heuristics then you probably won't have encountered the phone verification.

discord, bitkeeper and choices

flussence — Thu, 11 Apr 2024 15:55:18 +0000

WSL2 isn't competing with Linux desktops, it's competing with Macbooks. They're going for the market of yuppies that spend 200 hours decorating their terminal prompt and terminal editor while blogging about the process all on a disposable $2k laptop. Those people are never going to change that workflow to use cmd.exe or powershell, and MS knows it.

Tridge returns to rsync

mathstuf — Thu, 11 Apr 2024 14:10:20 +0000

I've never encountered that? AFAIK, Discord has no knowledge of my phone number (I've never given it).

Tridge returns to rsync

viiru — Thu, 11 Apr 2024 11:17:20 +0000

> Heh. I keep separate Discord accounts for different groups of related servers. I *prefer* separate accounts for
> separate uses instead of getting inundated by traffic/notifications from server X, Y, and Z when I'm really only there
> for purpose P.

How do you handle Discord's requirement for a working SMS-accepting phone number per account?

discord, bitkeeper and choices

adamg — Thu, 11 Apr 2024 09:03:07 +0000

Interesting email, thanks for sharing.

discord, bitkeeper and choices

adamg — Thu, 11 Apr 2024 09:01:37 +0000

I thought it was more restrictive, that by using bitkeeper one learned bitmover's IP, thus the restriction was meant to be valid even after license is revoked. But this would probably be illegal construct, so likely it was just restriction for as long as one was using bitkeeper.

Nationality as your "mark of evil"

roc — Wed, 10 Apr 2024 16:16:53 +0000

Yes, I think you're right. Thanks for pointing that out.

Nationality as your "mark of evil"

jzb — Wed, 10 Apr 2024 15:27:57 +0000

First of all, I'd like to apologize for making a mess of things. I've been part of the LWN community for a very long time, and hadn't quite internalized how other folks would respond to a comment from me as an editor vs. all the other times I've joined in discussions. Lesson learned there.

To the point about nationalism, yes - that was a poorly chosen word. What I should've said, aside from "nothing", was that we have a global audience. Calling out one investor's nationality ("Chinese investors") as a negative seemed like something that might alienate some of our readers. I had hoped to stave off the discussion that I instead sparked. My apologies again, I hope we can end this thread / topic of discussion here.

Tridge returns to rsync

farnz — Wed, 10 Apr 2024 14:01:05 +0000

Unfortunately, I don't have the time and energy to push through the bikeshedding, nor do I currently have a use case (I'm back in the embedded world, in a part where 10M is "incredibly fast", and 100M exists only because 10M Ethernet is basically dead). But I've put my idea out there, and if someone reads this and gets inspired, then I'm happy for my idea to be stolen :-)

Tridge returns to rsync

Sesse — Wed, 10 Apr 2024 13:54:03 +0000

I don't think this is about just one vendor, but I've certainly seen it in Cisco 2960(-S/X), 3560-X, ProCurve 2948, some Cisco 6509 line card (I don't know which, perhaps 6148?), and I have no reason to believe that newer gear in the same grade is different. When I searched around and/or asked networking people, the general gist was just “yeah, this is the cheap stuff, it's only suitable for quiet office LANs, duh”. (On some of them, I've been able to reduce the effect by changing QoS settings to move to a larger queue, e.g. some later 2948 firmware got a simple “single-queue” setting and I've done similar thing on the Ciscos through manual config.) I've seen it happen both on 10gig-to-1gig conversion and plain out 1gig (just happened to collide with other traffic coming out on the same uplink port, I guess—the drop counters on that port certainly went up and SACKs started coming back). At some point, we put a Cisco 4948 in the middle because it happened to have more buffer, and let it do downconversion from 10gig to 4x1gig to get better pacing for some TCP video streams :-)

I generally stopped worrying about this when I adopted packet pacing. First through HTB tricks (put each flow in its own bucket, pace each bucket—of course only works for video streaming or similar), later through fq and BBR. I believe some of the fq work came about partially because I whined enough at the time that people at work started listening, even though all of this was in a couple hobby projects.

Tridge returns to rsync

james — Wed, 10 Apr 2024 13:42:53 +0000

Just note: There are only two reasons why using multiple TCP connections would be faster...

Also, link aggregation¹ will normally only use one physical link for a single TCP connection. What the switches use for load balancing depends on the switches, but it's sometimes possible to include source and destination TCP/UDP port numbers in the hash (e.g. Cisco documentation, PDF, page 5).

If that's the bottleneck, multiple TCP connections would be more likely to be able to make use of all the available links.

¹ Multiple network links bonded together to act as one, probably using LACP: port channels in Cisco parlance.

Tridge returns to rsync

paulj — Wed, 10 Apr 2024 13:38:01 +0000

I should clarify "deliberately unfair" versus then saying "unwittingly" or "probably unintentionally". What I mean is someone is working on some new transport protocol and they see opportunities to send more packets, more quickly, so they do that - deliberate engineering in the first order - and those changes result in nice speed-ups immediately in testing. However, in large part because they do not really fully understand the higher-order fairness effects (which may be hard to understand, even somewhat hard to quantify).

So, deliberate in the first-order changes - "wow, networking is so easy, we just unlocked big performance improvements!". Unwitting in the knock-on effects on unfairness in large-scale deployment.

Nationality as your "mark of evil"

farnz — Wed, 10 Apr 2024 13:35:41 +0000

As an aside, the substitution rule is a good way to tell if what you're talking about is bias, or actual threat. Take an entity you trust (e.g. the Canadian government), and substitute them for the entity you distrust (e.g. the Chinese government); does your suggestion still sound bad to you?

So, applying that to WeChat censoring some topics outside China at the request of the Chinese government; if WeChat were to censor some topics outside of Canada at the request of the Canadian government, this is still bad. Substituting Canada for China hasn't flipped it from "boogeyman" to "good action", so it's probably not bias.

On the other hand "some proportion of Discord shares are owned by Canadians, and you can't trust the Canadian government to not lean on Canadians to get them to do bad things" doesn't sound sensible if you think the Canadian government is well-run, and makes it clear that the original statement around Chinese instead of Canadians is probably just coming from a place of bias.

Similarly, "Discord ToS has this clause that permits bad things" doesn't have any nationalities in it at all, and is a good reason to distrust Discord. As is "Discord's client is proprietary, and I can't audit it for bad behaviour as a result".

Tridge returns to rsync

paulj — Wed, 10 Apr 2024 13:30:41 +0000

Interesting. Do you remember which vendor or which chip? Was there some asymmetry? (Hot fast input port, sending out to a number of slower ports?).

Tridge returns to rsync

paulj — Wed, 10 Apr 2024 13:24:28 +0000

Ok, I probably misunderstood your point a little then.

Yes, you could initiate some kind of "more connections" arms race, trying to claim a larger share of some shared bottleneck, in the aggregate of your multiple connections. And someone else could then do the same, etc. In the long run, you and your competitors will end up with greater overheads from connection setup, relative to payload bearing traffic. However, you would indeed crowd out anyone not using this technique.

To some extent, the web went in that direction, with web apps creating ever greater numbers of connections as web pages specified ever greater numbers of resources spread over more and more end-points. Until they started hitting NAT limits, and had to back off a bit. E.g., IVR Google Maps started having issues with parallel downloading of map tiles in certain parts of the world with very dense NATing somewhere in the 00s, and had to re-engineer.

On the other hand, arguably, there are few "shared" bottlenecks anymore. Least, for competing users. DCs may have shared bottlenecks, but these days that's often one operator, who has an interested in ensuring the end-hosts try to remain fair to each other. End-users hosts may wish to engage in competitive, unfair behaviour, however those are typically located on access links where the bottleneck is close to the user and probably actively managed (AQM) by the network operator to enforce some kind of fairness and QoS. One could even argue that perhaps congestion doesn't really matter that much anymore for Internet flow control (an argument made in, e.g., https://dl.acm.org/doi/10.1145/3626111.3628204 ) - that speaks to your point about relying on AQM, more than end-host congestion control algos.

There are also transport protocols out there that get performance by being deliberately unfair. It's very easy for someone new to transport protocols to make network code faster by - unwittingly perhaps - ignoring fairness. I know of media protocols that are - probably unintentionally - engaging in network abuse to try get more bandwidth for themselves. To a certain extent, the "benefits" of BBR were achieved by being "unfair" to CUBIC. Though, on the other hand, is it right to stymie the adoption of better techniques for congestion control, simply because older techniques don't work well in their presence?

And yes, the end-host buffer defaults need revising. But... the bike-shedding potential around any such proposed changes is fast. If you think you could get it changed though, that'd would really help higher-end-network Linux users.

Nationality as your "mark of evil"

excors — Wed, 10 Apr 2024 12:56:33 +0000

> I see reports that Tencent owns 38% of Discord, e.g. [redacted to avoid helping its SEO]

That looks like a completely unreliable source. Eightify is an "AI YouTube Summary with ChatGPT" tool, and it has published about 5000 articles in /media in the past few weeks, so I bet they're also AI-generated and therefore garbage. The "resources" it cites are largely irrelevant - perhaps it picked them up because they mention e.g. "growing discord between China and the United States" and it's too dumb to realise that's not about Discord.

They probably got the number from https://youtu.be/uvNkdAggUGU?t=555 (linked further down the page in what's presented as a separate article), but that video cites no source, and its fearmongering tone doesn't engender much trust in its accuracy. Maybe the video's author read something like https://seekingalpha.com/article/4371548-tencents-dreams-... which says Tencent owns 38% of DouYu (a Chinese video streaming service) and "~2%" of Discord, and got the numbers mixed up because they both start with D, or something. (That article also cites no source, though at least I think it was written by a human). I can't find any more reliable sources for the 38%.

Since Discord is a private company, it doesn't have to disclose who its investors are, and it has chosen not to, so I wouldn't believe any number you read on the internet. (And especially not any from AI-generated articles.)

discord, bitkeeper and choices

Wol — Wed, 10 Apr 2024 12:07:36 +0000

> Perhaps it's time for the open source community to do the same.
> (For starters stop spreading "proprietary software is immoral" FUD/nonsense).

In case you hadn't noticed, the Open Source community has NEVER spread that FUD.

The Open Source community is pragmatic, Free and "proprietary" software will always co-exist, because some rather important software wouldn't exist if it wasn't paid for ...

Cheers,
Wol

Nationality as your "mark of evil"

pizza — Wed, 10 Apr 2024 11:54:07 +0000

> A certain party's screaming of "what about X, or Y, or Z!" is a strawman/whataboutism hybrid.

It's not "whataboutism" to point out the hystrionics are nearly entirely one-sided, elevating _potential_ problems of one convenient boogeyman with the _demonstrated actions_ of others (notably including their own).

China doesn't have to exercise subtle super secret control over Discord (or Tiktok) to get access to non-Chinese persons' data; they can do what everyone else does and just buy it on the nearly completely unregulated market of data brokers.

> Discord is a proprietary protocol. It is solely under the control of Discord inc.

This, not the ownership structure, is the _real_ problem of Discord.

Nationality as your "mark of evil"

farnz — Wed, 10 Apr 2024 11:53:52 +0000

Specifically what I see as problematic is the assertion that the presence of Chinese investors (regardless of their influence on the company) is bad. Taken as a guiding principle, this asserts that the Linux kernel cannot be trusted, because Huawei has influence over the kernel, and they're a Chinese company, too.

Given that you don't trust the Chinese government (personal decision), it's important to point out why the presence of a Chinese company with some influence is problematic with more than just "I don't like Chinese people having influence" - point to something that Discord (or whatever company you're talking about directly - not its owners, not other Chinese companies, but the company that we're discussing) has done that's problematic and that it's done for the Chinese government.

Otherwise, this all comes down to "my biases say that Chinese people are untrustworthy because they're Chinese, but that similar behaviour by the US and its allies is OK because I'm OK with Americans", and not a genuine cause for concern.

discord, bitkeeper and choices

pizza — Wed, 10 Apr 2024 11:45:35 +0000

> Eventually they learned how to cooperate, and there's a Linux kernel (wsl2) in every Windows installation now.

They "learned to cooperate" because it makes them a _lot_ of money.

(And then there's the minor detail of effectively making Linux "just another feature" of Windows; the classic EEE play.
I'm willing to bet there are far more _active_ WSL installations than there are native Linux desktops, complete with "Linux" software that can only function under WSL due to needing APIs that are passed through to Windows)

> Perhaps it's time for the open source community to do the same.

Perhaps when the open source community starts getting paid commeasurate with the value they provide?

Nationality as your "mark of evil"

dullfire — Wed, 10 Apr 2024 11:36:15 +0000

I think this has wander far off-topic. A certain party's screaming of "what about X, or Y, or Z!" is a strawman/whataboutism hybrid. The claims "refute" a point not said by the original poster, and accuses parties nobody talked about of doing things not originally said.

luna originally compared matrix to discord.

Matrix is a federated, open protocol, with multiple server implementations. It also has multiple client implementations. There are multiple opensource &| libre implementations of both client and server.

Discord is a proprietary protocol. It is solely under the control of Discord inc.

Matrix therefore *can not* have any single entity (regardless of who you are concerned about) make unilateral changes that would negatively impact the whole. Meanwhile, as the entity solely in control of discord, Discord Inc COULD unilaterally take any negative action.

luna also pointed out that a company beholden to the Chinese government has a notable about of shares in discord, and thus likely had some influence.

Then we had an LWN editor point out that this was not "polite and respectful", and that it was "nationalism" (At this point I STRONGLY encourage everyone to go lookup that word. It doesn't mean that). There has been no further elaborations on what the editor meant. However at this point I must point out I find the editors' message extremely concerning. Assuming that the editor meant the comment as they said it (if they did not, it would be wonderful if they added clarification), one of the following would have to be taken as not "polite and respectful"

- Requesting someone make a change to the communications platforms a project uses.
- Accusing an communications platform of adding ads.
- Suggesting influence from China(specifically China) is not desirable.
- Suggesting influence from any particular nation is not desirable.
- Suggesting influence from any group is not desirable.
- Suggesting influence from any person(s) is not desirable.

(The last four cover the said part of what luna said, but in progressively more less of a double-standard and more generic/universal)

I must say that ANY of these being true would greatly cripple any free/libre project governance talk on LWN. This is because it effectly says "there is a class of entities commenters are not allowed to cast in anything but the best light". Which means any conversation about, say, who should be debian project leader when multiple people are running... risks not being able to mention why you think person X shouldn't be DPL in a non malicious/hostile way(e.g. if someone wanted to say "X's skill set is best used elsewhere, and a poor fit for DPL", that might contravene the rules).

So I am going to hope/assume that the editor meant something other than what was written.

Finally, as I started with. This thread has veered far off the topic of "what community member would like for their interactions with the rsync project", and should likely be taken elsewhere.

Nationality as your "mark of evil"

roc — Wed, 10 Apr 2024 10:33:14 +0000

I see reports that Tencent owns 38% of Discord, e.g. https://eightify.app/media/fact-checking-discord-s-ties-t.... I wouldn't call that a "small amount". 38% in the hands of one entity is enough to have significant influence. I would be surprised if Chinese entities own a comparable amount of Alphabet. (That would be a $740B investment.)

It's possible that those reports are wrong, or that Discord is set up in such a way that Tencent has no influence they could abuse so there's nothing to worry about. Nevertheless it is reasonable to worry about it.

Tridge returns to rsync

Sesse — Wed, 10 Apr 2024 10:26:45 +0000

Traditionally, we expected people to have buffers for about one BDP. However, I've seen plenty of cases where things went through a switch that had something like 64 packets and that this caused loss, severely limiting the TCP speed for high-latency connections. (And these were not rinky-dink Netgear stuff from a store, they were expensive “enterprise” L3 switches with BGP and whatever. But they were not marketed as “routers” and thus did not have super-deep buffers or any reasonable sort of AQM.) And at some point, TCP pacing made a comeback and now you don't really need that much of a buffer, and this kind of random packet loss also doesn't matter as much as anymore (which is good, since Wi-Fi also tends to drop packets for similar not-really-congestion reasons).

…assuming you switch your congestion algorithm to BBR, that is. Which isn't default in any mainstream OS that I know of. :-/

Tridge returns to rsync

farnz — Wed, 10 Apr 2024 10:16:21 +0000

Network settings don't need to be changed if you're happy with TCP's congestion control, but where people want to go beyond that for some less visible idea of "fair", they'll do so on the assumption that you're using multiple connections. If you're using fewer than they expect, you lose out, so you'll start using more connections. The result is a loss for everyone, because in this world, the winning move is to have more connections than anyone else so that TCP's fairness gives you an edge (if there are 10 of us accessing a server, and 9 people use 10 connections each, while I use 90 connections, TCP fairness says I should get 50% throughput at the bottleneck).

Or, of course, TCP congestion control changes so that instead of being fair between connections, it becomes fair between IP addresses, or even entire netmasks (it's a lot harder to cheat if there's fairness between /32s, then within each /32, it's fair between /48s, and within each /48, it's fair between /64s, and within each /64, it's fair between /128s, and within each IP, it's fair between connections).

If I had the time and energy to push this, I'd change the default cap on TCP autotuning to be 1 second at the highest speed the fastest running network interface in the machine is capable of, with virtual adapters assumed to be capable of 1 Mbit/s by default. That way, if you have a 100G network card in there, the TCP autotuning algorithm is allowed to raise the window to enough to let one TCP stream saturate the card; I'd then be relying on proper AQM (with associated tiny amount of in-use buffering) in the rest of the network to keep latency and window sizes under control.

Tridge returns to rsync

paulj — Wed, 10 Apr 2024 10:12:32 +0000

Finally, to preempt Dave Taht, when it comes to the network you actually want the /fewest/ buffers possible. ;) You need /some/ buffering, but any additional buffering past that (hard to define) point is bad.

Tridge returns to rsync

paulj — Wed, 10 Apr 2024 10:10:44 +0000

+1 to this comment.

With the caveat that - as per my other comments - outside of local DC traffic, it's unlikely it's network buffer capacity throttling your TCP, but simply your max socket buffer sizes on your Linux server. But the multiple connection technique works around that too.