Increasing the TCP initial congestion window
That window has been capped by RFC 3390 at four segments (just over 4KB) for the better part of a decade. In the meantime, connection speeds have increased and the amount of data sent over a given connection has grown despite the fact that connections live for shorter periods of time. As a result, many connections never ramp up to their full speed before they are closed, so the four-segment limit is now seen as a bottleneck which increases the latency of a typical connection considerably. That is one reason why contemporary browsers use many connections in parallel, despite the fact that the HTTP specification says that a maximum of two connections should be used.
Some developers at Google have been agitating for an increase in the initial congestion window for a while; in July 2010 they posted an IETF draft pushing for this change and describing the motivation behind it. Evidently Google has run some large-scale tests and found that, by increasing the initial congestion window, user-visible latencies can be reduced by 10% without creating congestion problems on the net. They thus recommend that the window be increased to 10 segments; the draft suggests that 16 might actually be a better value, but more testing is required.
David Miller has posted a patch increasing the window to 10; that patch has not been merged into the mainline, so one assumes it's meant for 2.6.39.
Interestingly, Google's tests worked with a number of operating systems,
but not with Linux, which uses a relatively small initial receive
window of 6KB. Most other systems, it seems, use 64KB instead. Without a
receive window at least as large as the congestion window, a larger initial
congestion window will have little effect. That problem will be fixed in
2.6.38, thanks to a
patch from Nandita Dukkipati raising the initial receive window to 10
segments.
| Index entries for this article | |
|---|---|
| Kernel | Networking/Congestion control |
Posted Feb 10, 2011 3:59 UTC (Thu)
by butlerm (subscriber, #13312)
[Link] (1 responses)
First, the minimum retry time out should be reduced to approximately 1 ms, or whatever the timer accuracy allows. On local area networks a MIN_RTO of 1 second is ridiculous. Second, the syn retry timeout should be reduced from 1 second to 200 ms or so. The only reason to have it so high is if the bottleneck link is a 56 kbps modem or something.
It would be great if there were route attributes that allow these numbers (or something that allows them to be derived) to be further customized on a route specific basis. In particular to increase them on the default route in the case of a unusually slow link, and to decrease them for the route to the local LAN.
Posted Feb 14, 2011 3:23 UTC (Mon)
by ivannovick (guest, #72897)
[Link]
It would be great if this was configurable or lowered ... as the 200 miliseconds seems to cause a real bottleneck in some cases.
Does anyone know if this has been proposed or is being made configurable?
Cheers,
Posted Feb 10, 2011 6:49 UTC (Thu)
by mtaht (subscriber, #11087)
[Link] (6 responses)
The useful components in a google search result would fit into a single packet.
I would like it if google's search was more like DNS in structure and brevity.
Posted Feb 10, 2011 11:22 UTC (Thu)
by cortana (subscriber, #24596)
[Link] (3 responses)
Posted Feb 10, 2011 15:13 UTC (Thu)
by jengelh (guest, #33263)
[Link]
Posted Feb 10, 2011 18:29 UTC (Thu)
by mtaht (subscriber, #11087)
[Link] (1 responses)
One DNS provider (name.com), in addition to returning ptrs to ad sites on missing domain names, is ALSO returning completely invalid pointers to IPv6 addresses for the same:
d@cruithne:~$ host no.such.subdomain.bufferbloat.net
I've been meaning to discuss this with them once I can summon the needed diplomacy and tact. So far, no luck.
Be careful what you wish for.
Posted Feb 10, 2011 22:14 UTC (Thu)
by cortana (subscriber, #24596)
[Link]
Posted Feb 10, 2011 16:19 UTC (Thu)
by tstover (guest, #56283)
[Link] (1 responses)
I'm still waiting for gopher 2.0 with, say, links in utf-8 text.
Posted Feb 10, 2011 17:15 UTC (Thu)
by mtaht (subscriber, #11087)
[Link]
It works the command line and in emacs.
The latency improvement is remarkable. The S/N ratio is nice too.
Originally, it was a client/server protocol that ran over udp, I've been thinking of resurrecting the server piece, possibly using sctp, as I think latency can be further improved with a persistent connection for search.
Google's search IS so fast that in many cases the RTT for initial TCP setup dominates the entire session - I can get a 50ms response for a query located 2ms from google, and 200ms (at best) for one from Colorado.
I would certainly like more people to give the idea and tool a shot.
Gnugol also supports bing and a few other search engines - I thought what I wrote here about stackoverflow's interface, vs gnugol's to be pretty damning.
http://nex-6.taht.net/posts/Screen_Space/
Posted Feb 10, 2011 22:28 UTC (Thu)
by Lennie (subscriber, #49641)
[Link] (10 responses)
Posted Feb 10, 2011 23:34 UTC (Thu)
by dlang (guest, #313)
[Link] (9 responses)
If you are on high-speed connections, then the the congestion window issue is most significant for you.
if you have an overloaded link at some point in your path, then bufferbloat is your biggest problem.
Posted Feb 11, 2011 0:28 UTC (Fri)
by Lennie (subscriber, #49641)
[Link] (8 responses)
This could induce or increase the effect of bufferbloat
Posted Feb 11, 2011 6:31 UTC (Fri)
by butlerm (subscriber, #13312)
[Link] (7 responses)
Google has a proposal to fix this with a mid layer multi-stream protocol between HTTP and TCP. Even better would be a proper multi-streaming transport protocol like SCTP, but not so intelligent NAT devices appear to have killed that option for the time being. Meanwhile, Google Chrome appears to restrict the number of connections it uses pretty severely, which makes it noticeably slower than Internet Explorer on some applications which are not at all bandwidth constrained.
Posted Feb 11, 2011 10:08 UTC (Fri)
by Lennie (subscriber, #49641)
[Link] (5 responses)
Also it is a model which might not yet fit wel with some current webservers. It won't be an easy module to add to Apache for example. A proxy could work though.
Ironicly of all the currently deployed browsers, IE6 and IE7 from Microsoft are actually open the least amount of connections and the one with the most connections: ie8 (and I presume ie9)
Posted Feb 11, 2011 13:41 UTC (Fri)
by mtaht (subscriber, #11087)
[Link]
https://lists.bufferbloat.net/pipermail/bloat/2011-Februa...
Posted Feb 12, 2011 6:53 UTC (Sat)
by dtlin (subscriber, #36537)
[Link] (3 responses)
Posted Feb 12, 2011 12:04 UTC (Sat)
by Lennie (subscriber, #49641)
[Link] (2 responses)
http://tools.ietf.org/html/draft-agl-tls-nextprotoneg-01
I'm talking about this one, it's just sitting there (atleast that is what it looks like to me):
http://www.chromium.org/spdy/spdy-protocol
Without something at IETF or something which looks mostly final why would other browser vendors and webserver people implement it ? Without atleast that, it is useless.
This is the last status:
http://groups.google.com/group/spdy-dev/browse_thread/thr...
Who knows, maybe this is the reason why:
http://groups.google.com/group/spdy-dev/browse_thread/thr...
He is just to busy and there isn't anyone else to help him ?
;-)
Posted Feb 16, 2011 17:22 UTC (Wed)
by mbelshe (guest, #72948)
[Link] (1 responses)
Well, I'm glad you guys are interested. I assure you that we're still hard at work on it. It just takes time.
This year, we'll be much more active and look to get IETF support. Unfortunately, some things just can't be rushed.
Mike
Posted Feb 17, 2011 9:28 UTC (Thu)
by Lennie (subscriber, #49641)
[Link]
SPDY sounds to me like the right way, instead of adding more TCP-connections all with a larger initial windows size.
Bufferbloat is a different problem, but adding more TCP-connections with larger initial window sizes wouldn't help either.
SPDY seems to also be about delivering fast TLS. That would help a lot, we've seen to many problems with cookies getting picked up by others.
Maybe we don't want to use it for certain static files, atleast that is what people think when they see what is happening with Google SSL/Google Images. Maybe it is just a loadbalancer/TLS-sessions problem, it breaks TLS Session Resumption ?
If browsers that use SPDY are shown to be faster and people flock to it, it will also help spread the support for SNI. Although I think people who use IE on XP are probably not speedfreaks anyway.
But the more reasons people have not use IE on XP the better. :-)
Posted Feb 16, 2011 17:18 UTC (Wed)
by mbelshe (guest, #72948)
[Link]
SCTP does centralize cwnd across all open streams, but it uses the same core algorithms as TCP and fails to address the initial burst. Unless the initial burst is increased, SCTP is slower than HTTP over lots of TCP connections out of the gate. This is measurable and demonstrable.
Oh, and SCTP doesn't work over NAT. So still waiting for IPv6.
Posted Feb 14, 2011 1:32 UTC (Mon)
by showell (guest, #2929)
[Link] (2 responses)
Some optimisation of this protocol was started a while ago by Nortel prior to its breakup but I'm unaware of any further work.
It would benefit a large number of people if some smart TCP hacker could find a way to meet the congestion control required to keep our internetwork links running and at the same time optimise the wireless experience.
Posted Feb 19, 2011 21:56 UTC (Sat)
by mtaht (subscriber, #11087)
[Link] (1 responses)
See thread at: https://lists.bufferbloat.net/pipermail/bloat/2011-Februa...
old fashioned web proxies and split tcp.
Posted Feb 24, 2011 14:15 UTC (Thu)
by amh (guest, #1902)
[Link]
There are a couple of other things that should be done:
Increasing the TCP initial congestion window
Increasing the TCP initial congestion window
Ivan Novick
Increasing the TCP initial congestion window
Increasing the TCP initial congestion window
Increasing the TCP initial congestion window
Increasing the TCP initial congestion window
no.such.subdomain.bufferbloat.net has address 149.20.54.81
no.such.subdomain.bufferbloat.net has IPv6 address 1400:0:bce5:95ab:1e00:0:2800:0
Increasing the TCP initial congestion window
Increasing the TCP initial congestion window
Increasing the TCP initial congestion window
bufferbloat
bufferbloat
bufferbloat
The reason why multiple connections are useful is not so much to give you an unfair advantage (over yourself?) in the congestion control wars - it is rather because HTTP over TCP is a single stream protocol, which causes the serialization of all requests and responses to/from the same server. That is not remotely ideal.
bufferbloat
bufferbloat
Proxies, multi-streaming, wireless, bufferbloat
Look for bufferbloatnpn
EnableSpdy in chromium/src/chrome/browser/browser_main.cc for 90% of Google Chrome users, the browser will switch protocols to SPDY whenever the server advertises the capability.
All secure Google services advertise SPDY as an alternate protocol (using the next_protocol_negotiation extension of TLS). That's a lot of real-world users.
bufferbloatnpn
bufferbloatnpn
bufferbloatnpn
bufferbloat
Increasing the TCP initial congestion window
Increasing the TCP initial congestion window
Increasing the TCP initial congestion window