A reworked TCP zero-copy receive API

Posted May 18, 2018 16:01 UTC (Fri) by zuzzurro (subscriber, #61118)
Parent article: A reworked TCP zero-copy receive API

What seems frankly amazing for me is that such a change (even the latest one) could be implemented and written in stone in such a short period of time without a larger audience looking at it. I understand that too large an audience could just an exercise in bike shedding, but still. If we had a mechanism to implement test changes not subject to the ABI stability requirement would make experimenting a bit easier, but as this mechanism does not exist I would suggest to proceed more carefully.
In particular what seems strange is the fact that (and I may be wrong on this) mmap has been knows to be quite slow (isn't this the reason why using mmap for cp never worked better than read/write?), and therefore the initial mechanism should have been spotted as suboptimal immediately....

A reworked TCP zero-copy receive API

Posted May 18, 2018 22:21 UTC (Fri) by jhoblitt (subscriber, #77733) [Link] (5 responses)

Since `mmap()` is used to open shared libraries, I certainly would hope that it is fast to setup the page mapping.

```
strace cp foo bar |& grep mmap | wc -l
23
```

A reworked TCP zero-copy receive API

Posted May 19, 2018 11:48 UTC (Sat) by grawity (subscriber, #80596) [Link] (2 responses)

I'd say the performance requirements for mmapping a few dozen libraries once per exec don't come anywhere close to mmapping a million packets per second...

A reworked TCP zero-copy receive API

Posted May 19, 2018 13:50 UTC (Sat) by jhoblitt (subscriber, #77733) [Link] (1 responses)

The reworked API only has one mmap() call to setup the window. I'm not really grasping what the concern on this thread is.

A reworked TCP zero-copy receive API

Posted May 19, 2018 21:43 UTC (Sat) by eternaleye (guest, #67051) [Link]

You're missing that this thread isn't about a _design_ concern with the _new_ proposal, it's a _process_ concern regarding the _old_ proposal (which did call mmap() once per receive). Namely, "How did that proposal get so far along before someone noticed that issue?"

A reworked TCP zero-copy receive API

Posted May 20, 2018 15:00 UTC (Sun) by willy (subscriber, #9762) [Link] (1 responses)

mmap is not slow. munmap (specifically tearing down the TLBs on every CPU) is slow. You must not leave a TLB entry available to userspace when the kernel has reused the page for something else. There are various optimisations we've implemented and a few we haven't yet to speed this up, but fundamentally it's slow because you need to tell every CPU that this process has ever executed on that the mapping is no longer valid. So it gets slower on larger machines.

A reworked TCP zero-copy receive API

Posted May 20, 2018 18:03 UTC (Sun) by zuzzurro (subscriber, #61118) [Link]

This is what I'm referring to when I takj about mmap being slow:

http://lkml.iu.edu/hypermail/linux/kernel/0004.0/0728.html

But my question is really about the process, not about this particular instance.