Over-sized buffers matter greatly, *especially* when they back on to a much slower link. Your GigE NIC will *NOT* respond to congestion at the wifi link, it'll keep blasting out packets (at least, it will seem so from the POV of a much slower wifi link). Which will just make the congestion on your wifi link even worse, particularly so given 802.11's malfeature of trying to implement reliability in a low-level link-layer - which just amplifies congestion problems.
There's no way for Linux to know just from your local link-speed what the correct size is. Ethernets in the past tended to be relatively homogeneous wrt segments, but these days they can be extraordinarily mismatched - with GigE segments often bridged together with 802.11 links whose reliability, latency and bandwidth make you yearn fondly for the thinnet of 20+ years ago.