This code just kills efficiency by a thousand cuts. There is no single culprit, it is just that all that twisting and turning, calling lots of little helpers and layering everything through an skb editing API that successfully confuses the optimizer adds up to an embarrassing amount of overhead. First rule to remember? Function calls are not free. Not at the speeds networks operate these days.
Posted Jul 9, 2012 8:18 UTC (Mon) by nix (subscriber, #2304)
[Link]
Actually, predicted function calls *are* nearly free on modern CPUs. Of course, function calls stuck deep inside conditionals are less likely to be successfully predicted as taken -- and unpredicted/mispredicted function calls (like all other mispredicted, non-speculated branches) are expensive as hell. However, these days I don't believe there is much more reason to be concerned about function calls than there is to be concerned about any other conditional. (Specialists in deep x86 lore, which I am very much not and who I am merely reiterating from dim and vague memory, are welcome to contradict me, and probably will!)
Missing the AF_BUS
Posted Jul 9, 2012 23:06 UTC (Mon) by daglwn (subscriber, #65432)
[Link]
The call is cheap. The saving/restoring of registers and lost optimization opportunities are not.
Missing the AF_BUS
Posted Jul 9, 2012 18:40 UTC (Mon) by butlerm (subscriber, #13312)
[Link]
>This code just kills efficiency by a thousand cuts. There is no single culprit, it is just that all that twisting and turning, calling lots of little helpers...
Much of the complexity of that function has to do with kernel support for fragmented skbs, which is required for packets that are larger than the page size. That is the sort of thing that would go away if the kernel adopted a kernel page size larger than the hardware page size in cases where the latter is ridiculously small.
I am not sure what the real benefits are of managing everything in terms of 4K pages is on a system with modern memory sizes. Perhaps the idea of managing everything in terms of 64K pages (i.e. in groups of 16 hardware pages) could be revisited. That would dramatically simplify much of the networking code, because support for fragmented skbs could be dropped. No doubt it would have other benefits as well.