LWN.net Logo

Bufferbloat: Dark Buffers in the Internet (ACM Queue)

Bufferbloat: Dark Buffers in the Internet (ACM Queue)

Posted Dec 7, 2011 8:55 UTC (Wed) by marcH (subscriber, #57642)
In reply to: Bufferbloat: Dark Buffers in the Internet (ACM Queue) by dlang
Parent article: Bufferbloat: Dark Buffers in the Internet (ACM Queue)

> however, if your 1G network is connected to a 1M network, then your server buffer size should be 1/1000 the size to maintain the correct latency, but your desktop has no way of knowing that there is a 1m link somewhere in the middle.

No: in this case the queue(s) in your server do not matter because they will be empty most of the time. Packets will only stack up at the bottleneck (as the name implies).

> in this situation (1G - 1M - 1G links) you need the routers connecting to the 1M link to have small buffers and drop packets

Yes.

If every link makes sure not to buffer more than 100ms or so, then bufferbloat goes away in 90% of the traffic cases (10% are left for the researchers to have fun with). Let's fix the most obvious problems first.


(Log in to post comments)

Bufferbloat: Dark Buffers in the Internet (ACM Queue)

Posted Dec 7, 2011 10:45 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]

you are right that the buffers on the 1G machine will always be empty, you need to (eventually) detect packet loss and then throttle the sending speed

the reality is that there is a lot of equipment out there that you are not going to be able to get replaced for several years, and part of that reason is that the vendors are still building equipment with buffers that are way too large because they still aren't doing testing that shows them the problem. this sort of publication and formal paper is what's required to get them to notice the problem and start the process of fixing it.

Bufferbloat: Dark Buffers in the Internet (ACM Queue)

Posted Dec 7, 2011 12:05 UTC (Wed) by marcH (subscriber, #57642) [Link]

> you are right that the buffers on the 1G machine will always be empty, you need to (eventually) detect packet loss and then throttle the sending speed

TCP does that at the source. TCP is ACK-clocked on whatever is the current bottleneck thanks to the congestion or receiver window (whichever is smaller). No need to throttle TCP anywhere else.

Bufferbloat: Dark Buffers in the Internet (ACM Queue)

Posted Dec 7, 2011 12:17 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]

as I understand the effects of bufferbloat, the fact that these over-large buffers are queuing the packets instead of dropping them is breaking the TCP ack clocking mechansim

Bufferbloat: Dark Buffers in the Internet (ACM Queue)

Posted Dec 7, 2011 13:26 UTC (Wed) by marcH (subscriber, #57642) [Link]

> ... is breaking the TCP ack clocking mechansim

... which does not mean TCP will go mad and send even more packets and clog whatever queues even more. I would bet it is actually the opposite.

Anyway, I only had the "reasonable queue size @ bottleneck" case in mind in my previous post. In this "normal", non-bufferbloated case ACK-clocking works fine and there is no need for externally throttling TCP anywhere else than at the bottleneck's queue when it fills up.

*In theory* not just TCP but every other protocol should be a good citizen and follow TCP's lead in respect to congestion/throttling: http://en.wikipedia.org/wiki/Datagram_Congestion_Control_...
In practice no one is using DCCP but it's not too bad either: Skype for instance actively throttles itself down in case of congestion/high latencies. Unsurprisingly, no application tries to damage the network it is using itself.

I would not be surprised if TCP is actually the worst guy for falling in and filling bufferbloat traps. There is some irony in that considering it is usually considered the best congestion citizen *in the lack of bufferbloat*.

Bufferbloat: Dark Buffers in the Internet (ACM Queue)

Posted Dec 7, 2011 16:39 UTC (Wed) by martinfick (subscriber, #4455) [Link]

> I would not be surprised if TCP is actually the worst guy for falling in and filling bufferbloat traps. There is some irony in that considering it is usually considered the best congestion citizen *in the lack of bufferbloat*

I had the same thought myself and was wondering if TCP needs to be fixed to take latency into account? From my lame understanding it seems to only care about throughput, isn't that why the bittorrent folks came up with their own UDP based solution? So, while I am all for fixing buffers wherever possible, shouldn't there be more discussion about fixing TCP?

Bufferbloat: Dark Buffers in the Internet (ACM Queue)

Posted Dec 7, 2011 22:58 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]

actually, what is happening is that TCP (eventually) gets the acks for the packets that it sends, so it speeds up to try and send more traffic.

eventually the delay gets so large that it times out before the acks arrive and the speed collapses.

If you look at the graphs in Getty's paper, you will see exactly this sort of picket-fence for what he is calling 'goodput', which is the amount of traffic that is actually getting to the destination (with the rest of the available bandwidth being taken up by 'badput', which is packets that are going to be dropped by the time they get to the destination because they are either too old, or they are retransmissions of packets that are still in flight, and so will be duplicates by the time they get there)

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds