Zero-copy network transmission with io_uring

Posted Jan 1, 2022 20:27 UTC (Sat) by jafd (subscriber, #129642)
In reply to: Zero-copy network transmission with io_uring by epa
Parent article: Zero-copy network transmission with io_uring

There is a small problem with this "synchronous send" idea. How do you define "sent"? Does it get sent as soon as it reaches the kernel's buffer? Or does it get sent as soon as it reaches the NIC? Or does it get sent as soon as the NIC has put it onto the wire? Note that the NIC has its own buffers, too. Or maybe when it reaches the nearest router? Or as soon as the remote side acknowledges receipt? But what about raw IP, UDP, and who knows what other stateless protocols?

This is a small problem, but it sure opens a huge can of worms.

Years ago I wanted this too. But then I read more about this problem (in the context of a rather good article about buffer bloat and how it actually harms both disk and network performance, who knows if it wasn't here at LWN, years ago), and I don't want it anymore. We can only sensibly agree that the packet is sent as soon as the postman (the kernel) found no immediate problems with it and took it for processing.

Zero-copy network transmission with io_uring

Posted Jan 1, 2022 23:38 UTC (Sat) by james (subscriber, #1325) [Link] (1 responses)

Or as soon as the remote side acknowledges receipt?

And "acknowledgement" can happen at different levels. For example, a mail server might indicate that TCP packets containing an email have been received, but you shouldn't consider that email "sent" until the receiving side sends a "250 OK" message (which might be after SpamAssassin has run, so potentially many milliseconds later).

Zero-copy network transmission with io_uring

Posted Jan 3, 2022 8:25 UTC (Mon) by taladar (subscriber, #68407) [Link]

While that is true the kernel can not really recover from a high level rejection of the message using just the buffer content anyway.

Zero-copy network transmission with io_uring

Posted Jan 2, 2022 9:56 UTC (Sun) by smurf (subscriber, #17840) [Link]

That's easy: as soon as the kernel is sure it doesn't need the buffer any more, i.e. when the TCP ACK has arrived for its last byte, or when the network interface tells the kernel that it has transmitted the data (UDP).

There is no kernel buffer. The whole point of this is to enable Zero Copy.