Bufferbloat: Dark Buffers in the Internet (ACM Queue)

in the past, entities have tested their systems for latency (usually with tiny packets using a small amount of bandwith) and separately for bandwidth (usually with huge packets)

if you bandwidth isn't saturated, the small packets never queue up and so the device shows good latency on the latency test. buffer size doesn't matter.

in the bandwidth test, larger buffers prevent packets from being dropped (and therefor retransmit times), so larger buffers help bandwith measurements (even if only marginally).

so this has lead to thinking that larger buffers are always better. Add in the fact that memory is getting cheaper (OLPCv1.5 went from 256M of ram to 1G of ram because it was cheaper to buy the 1G memory modules). If you are building a router and have more memory, what are you going to use it for besides larger buffers?

the bufferbloat problem is that when the bandwidth is saturated, latency becomes horrible. This is not a combination that vendors have been testing.