Mazzoli: How fast are Linux pipes anyway?

In the original article, he achieved the speed by changing the problem from "transferring bytes via Linux pipes" to "transferring pages via Linux pipes". You can't move bytes at 35GiB/s purely locally within the same program, so claims of doing it over pipes makes you wonder what he's smoking. Turns the title involved a little poetic licence. Perhaps the article could have been better called "zero copy via pipes".

In case it tickles someone's interest: when you transfer pages, you are transferring ownership (permission to write) of the page. The HN discussion centred on how that transfer of ownership was managed. In the article it was dealt with by not handling it at all. This is just a benchmark program, and so he could get away with writing to the pages once before he started and then never writing them them again during the test. But that means he was transferring a constant over the pipes over and over again by vmsplice()'ing a page that never changed. The HN discussion revolved around whether it was possible to write to (ie, later reuse) the transferred pages in a way that didn't create a race between the reader and writer and didn't need to set up some second channel (eg, a futex) to avoid the race condition. It centred around transferring ownership of the page to the kernel. It hadn't reached a conclusion when I read it.

The HN discussion was more interesting that the article in some ways, because it explored whether what was really just a clever sleight of hand could ever be useful in practice.