LWN: Comments on "Delay-gradient congestion control"

Delay-gradient congestion control

cov — Mon, 14 Sep 2015 15:47:53 +0000

How does this look on wireless links?

Delay-gradient congestion control

Lennie — Mon, 31 Aug 2015 15:44:01 +0000

I'm so glad people keep working on these kinds of things. These kinds of issues need a lot of work.

It does seem to me Apple has a pulse:

https://lists.bufferbloat.net/pipermail/bloat/2015-June/0...

Delay-gradient congestion control

kennetkl — Sat, 13 Jun 2015 23:14:52 +0000

We are pleased to announce that CDG is now part of net-next, the network development branch:
https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=2b0a8c9.

Thanks to Yuchung Cheng and others that provided feedback to help improve the implementation.

Delay-gradient congestion control

mtaht — Fri, 05 Jun 2015 21:14:35 +0000

"Very cool, these people invented a congestion control algorithm that the bufferbloat crowd claimed was impossible. Really looking forward to a packet loss-less Internet."

I don't think anyone has ever claimed that. Rather (I at least) have always pointed at every improvement in host tcps and driver infrastructure in every talk - BQL, TSQ, sch_fq's pacing, deprecating hystart, TSO/GSO/GRO sizing fixes, Quic, etc - as both needed and useful - and arguably sch_fq (which builds on all that) has had a far greater benefit thus far than all the work on fq and AQM combined.

Certainly there is a huge bias (based on the experimental evidence) that classic delay based tcps lost out to loss based in an undeployable fashion for most connections - but, for example, I would not mind a delay based tcp on 3g mobile handsets as a specialized use case. I know of some new CC algorithms that look *really* promising in other cases... and...

I welcome new attempts at making tcps better (or building, as I have thought, new protocols like quic that had a touch more awareness of the underlying link layer's properties, wireless or wired)...

and plan to test this new tcp as soon as I have time to do a recompile. (well, it looks like there have been quite a bit of feedback on it on the list)

As for packet-loss-free internet, well, I deployed ECN fully at multiple sites over the past few years, and am very close to that, already, and all people have to do to get there is turn ECN on in their tcp's and aqms, which is down to a couple sysctls now.

There are still some details regarding ECN to sort out, notably how to handle overload better, see cake and the latest fq_codel code for that.

http://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN

And lastly, still, I remain dubious about any form of e2e congestion control as the be-all-end-all solution. TCP-ers tend to crow with excitement when they can get results with 100-150ms of induced delay. With AQMs on the routers, we got in the 20-100ms range, and with fq, down to near zero for most traffic types we cared about. This was exciting enough to make the big push needed to try and fix the routers - knowing full well how hard it would be - worldwide... for 4.5 years now.

The combination of a world with fq+aqm+ecn + better e2e congestion control remains to be more fully explored. I have pointed out that in particular fq gives coupled flows (e.g. bittorrent, some videoconferencing stuff) a much better signal (clock) to measure against, but there has been very little (public) work in this area.

My focus these days is on applying all that we have learned about e2e, aqm, fq, ecn, etc, as best as we can, to dramatically improve wifi behaviors, with "make-wifi-fast".

"The bufferbloat community has been surprisingly successful",

I am shocked, myself. When we started I had no hope. I thought we were doomed to watching tv over the internet, only. It was a windmill I wanted to tilt at, and I had some spare time to do it....

In fact, when the first ideas for fixing wifi didn't work out back in late 2011, I had *quit*, and was looking for a "real" job, then about 2 weeks later - I somewhat randomly picked up on BQL (Which was a set of patches that had died on the vine 5 or so months prior)... and it worked! It had fundamental insights into everything we were doing wrong... and things have been building ever since. I think over the next year we are going to see a huge change in internet latency under load across hundreds of millions of connections, especially now that speedtest sites like dslreports are adding bufferbloat measurements.

Sometimes I am grateful to tom herbert for BQL, other times I kind of resent the invention of it, because I could have just gone back to a beach and surfed a few more years and ignored the internet a while longer.

"but it seems so much easier to change congestion control in a few major (endpoint) operating systems, rather than to fix buffer accounting and in tens/hundreds of middlebox vendors."

Hah. Finding anyone with a pulse at microsoft, BSD, or apple has been very difficult.

Delay-gradient congestion control

dlang — Fri, 05 Jun 2015 14:51:31 +0000

The question is if this does actually solve the problem in the absence of any improvement to the queue management on routers.

Especially when combined with traffic from systems that don't use the same algorithm. If systems that use this back off before the congestion is bad, but other systems don't, these systems will not get their fair share of bandwidth (as noted in the article), and if it stops backing off because it detects this situation, is it really solving the problem?

There are a lot of things besides congestion that can alter the RTT of a connection (including server performance at the other end) so it's not clear how reliable this signal will be in the real world.

It's worth testing, and getting it into the kernel as an option makes it far easier for people to test it. But it's far too early to say that it's the solution and will lead to a packet loss-less Internet.

Delay-gradient congestion control

intgr — Fri, 05 Jun 2015 09:52:57 +0000

(Oops, preview fail)
> to fix buffer accounting and

and queue management

Delay-gradient congestion control

intgr — Fri, 05 Jun 2015 09:51:49 +0000

Very cool, these people invented a congestion control algorithm that the bufferbloat crowd claimed was impossible. Really looking forward to a packet loss-less Internet.

The bufferbloat community has been surprisingly successful, but it seems so much easier to change congestion control in a few major (endpoint) operating systems, rather than to fix buffer accounting and in tens/hundreds of middlebox vendors.

Packet losses due to congestion vs. errors

vomlehn — Tue, 02 Jun 2015 21:53:57 +0000

This might help solve a long-term problem in applications with a high value of the product of the data rate and the transmission time between end nodes. These include high speed connections to satellites in high or geosynchronous orbits and ultra high band width over continental scale distances. The satellite issue is especially interesting because the error rate is also considerably higher than most Earth-bound applications. When errors occur, the current algorithim treats it as congestion detection and backs off its send rate. This would be correct for congestion but does nothing to change the error rate; it only reduces throughput. So, if this algorithm can distinguish between errors and congestion, it will be very welcome from the high-speed/long-haul community.

--cdg > --bwlimit

mtaht — Wed, 27 May 2015 18:57:58 +0000

I long ago posted patches to the rsync folk that would allow it to use alternate congestion control algorithms and diffserv. The patches sat in their patchwork for a while and got improved a bit, but I don't think they ever made their mainline tree.

The initial patches were here:

https://lists.samba.org/archive/rsync/2011-November/02711...

As rsync is often tunneled over ssh, I have long thought we could add stuff to ssh to better chose it's underlying cc, also.

Delay-gradient congestion control

ncm — Sat, 23 May 2015 14:57:41 +0000

Most of the dependence on packet-loss signaling is a consequence of not being able to trust the peer. Packet loss is a reliable signal, for a very specialized definition of "reliable". In cases where you control both ends of a connection, rate of change of transit time is a much better measure of congestion.

The problem with transit time as a measure is that the useful lifetime of a single measurement is less than an RTT, and the end that has the measurement is not the end that needs it. Control theory directly addresses that problem at its root: the sender needs a prediction of the transit time according to a model controlled by frequent updates from the receiver. Given a trusted remote endpoint, you can do much, much better by entirely ignoring packet loss.

Delay-gradient congestion control

Celest — Fri, 22 May 2015 12:18:20 +0000

How well would this algorithm work in presence if path change? Or when multiple paths are used for a single connexion?

--cdg > --bwlimit

flussence — Fri, 22 May 2015 01:42:32 +0000

It'd be *really* nice to have it in ffmpeg. Constantly tweaking -b parameters for an RTMP stream gets to be a headache....

--cdg > --bwlimit

gmatht — Thu, 21 May 2015 11:32:39 +0000

If CDG allows itself to be muscled out of a contended link in that manner, one can predict with high confidence that it will not find many users.

Perhaps, but its worth noting that in some niches that would be a feature and not a bug. It would be nice to be able to just do `rsync --cdg` inplace of `rsync --bwlimit 5000` and go as fast as possible without slowing down other users.