A queue size matters only when it becomes a bottleneck. When a queue is empty its maximum size obviously does not matter.
If your traffic goes first to a gigabit wire, and then through wifi, the queue size of your gigabit wire will never matter. Because the wifi queue will bottleneck first, fill up and drop the packets first.
It does not hurt to fine tune the queue size of every link just in case. But if you only want a quick fix you just need to look at your usual bottlenecks.
By the way, is the TX ring size in Linux finally adjusted according to the link speed? This should be a very basic thing to implement...