The CHOKe packet scheduler

Posted Jan 13, 2011 12:06 UTC (Thu) by Velmont (guest, #46433)
Parent article: The CHOKe packet scheduler

Sooo... Is this something for my local server-and-router for our home LAN? I guess it is way too small, this is more for ISPs, and more central routers? Or is it really part of our problem?

Or do I only have to think about bufferbloat? Are there any guides telling me how to fix my Ubuntu Linux-based server/router to not be a part of the bufferbloat problem.

The CHOKe packet scheduler

Posted Jan 13, 2011 21:07 UTC (Thu) by jhhaller (guest, #56103) [Link]

The only thing you can control within your network are transmitted packets. If you have much upstream traffic, or significant interference in your wireless network, those are places which could benefit from this scheduler. But, if you have asymmetrical traffic, with most traffic received from your ISP, and have no long transmit queues, your ISP has to do this scheduling on your flows.

One could do something similar by emulating a slightly slower link after receipt from your ISP. For example, if you have a 8 Mbps link from your ISP, you could put an artificial link of 7 Mbps out of your router (method for this left as an exercise for the reader). Then, if the queues start growing on that 7 Mbps link, packets could be discarded there. This would tend to prevent your 8 Mbps link from being congested, at the expense of not getting your full 8 Mbps. But, it would be much better for the ISP to install the CHOKe scheduler, so it would only start dropping packets when the average was over 8 Mbps.

The CHOKe packet scheduler

Posted Jan 13, 2011 22:26 UTC (Thu) by paulj (subscriber, #341) [Link] (4 responses)

There are 2 things you might wish to tweak:

- the size of the ring buffer used to transfer packets from the kernel to the NIC. See 'ethtool -g ethX' and 'ethtool -g <dev> tx X'

- the number of packets the kernel's network layer will queue, it's 1000 by default on ethernet interfaces, which is way too large if your ethernet has a wifi segment in the middle of it. Set this to something lower with 'ip link set <dev> txqueuelen Y'

The CHOKe packet scheduler

Posted Jan 14, 2011 9:28 UTC (Fri) by marcH (subscriber, #57642) [Link] (3 responses)

I think "in the middle of it" is misleading.

A queue size matters only when it becomes a bottleneck. When a queue is empty its maximum size obviously does not matter.

If your traffic goes first to a gigabit wire, and then through wifi, the queue size of your gigabit wire will never matter. Because the wifi queue will bottleneck first, fill up and drop the packets first.

It does not hurt to fine tune the queue size of every link just in case. But if you only want a quick fix you just need to look at your usual bottlenecks.

By the way, is the TX ring size in Linux finally adjusted according to the link speed? This should be a very basic thing to implement...

The CHOKe packet scheduler

Posted Jan 14, 2011 9:54 UTC (Fri) by paulj (subscriber, #341) [Link] (1 responses)

Over-sized buffers matter greatly, *especially* when they back on to a much slower link. Your GigE NIC will *NOT* respond to congestion at the wifi link, it'll keep blasting out packets (at least, it will seem so from the POV of a much slower wifi link). Which will just make the congestion on your wifi link even worse, particularly so given 802.11's malfeature of trying to implement reliability in a low-level link-layer - which just amplifies congestion problems.

There's no way for Linux to know just from your local link-speed what the correct size is. Ethernets in the past tended to be relatively homogeneous wrt segments, but these days they can be extraordinarily mismatched - with GigE segments often bridged together with 802.11 links whose reliability, latency and bandwidth make you yearn fondly for the thinnet of 20+ years ago.

The CHOKe packet scheduler

Posted Jan 14, 2011 10:50 UTC (Fri) by marcH (subscriber, #57642) [Link]

> Your GigE NIC will *NOT* respond to congestion at the wifi link, it'll keep blasting out packets

For sure the NIC *alone* would, but TCP (or DCCP) do NOT.

TCP is ACK-clocked on the bottleneck. Just try it! The name of this feature is "congestion avoidance", google it.

> There's no way for Linux to know just from your local link-speed what the correct size is.

ethtool eth0

> Ethernets in the past tended to be relatively homogeneous wrt segments, but these days they can be extraordinarily mismatched

It does not matter: every link needs to adjust according to its *own* speed.

The CHOKe packet scheduler

Posted Jan 14, 2011 15:31 UTC (Fri) by njs (subscriber, #40338) [Link]

> By the way, is the TX ring size in Linux finally adjusted according to the link speed? This should be a very basic thing to implement...

Nope. (And it's "ring sizes", not "ring size"; every driver has its own TX queue handling code, plus there's the TX queue in the network layer itself.) I do have some patches to auto-tune the iwlwifi driver's queues, but haven't had a chance to benchmark them properly...