Mazzoli: How fast are Linux pipes anyway? [LWN.net]

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 5, 2022 17:10 UTC (Sun) by Wol (subscriber, #4433)
In reply to: Mazzoli: How fast are Linux pipes anyway? by atnot
Parent article: Mazzoli: How fast are Linux pipes anyway?

> write() takes the address of a buffer that the user owns. Because it can not make any assumptions about how long that data is going to remain valid after the syscall, it has no choice but to copy.

What about COW? (Might be tricky I know, but just COW the page containing the buffer.)

Cheers,
Wol

to post comments

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 5, 2022 18:58 UTC (Sun) by atnot (guest, #124910) [Link] (8 responses)

I don't think that would help. CoW is only faster if the memory isn't actually written to again. But applications don't really keep buffers around unmodified for posterity, nor would there be any real way for them to know when it's okay to reuse them. So in practice, you're likely to just end up with the page that contains the buffer being immediately written to again, at which point you're at the status quo again except with additional page faults and fragmentation. There's not really any way to solve this without deviating significantly from the write() API, which is probably one reason people keep inventing new ways of doing it.

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 8, 2022 1:08 UTC (Wed) by willy (subscriber, #9762) [Link]

Worse than that, write() would have to invalidate the TLB entry for the page in question (in order to make COW work). TLB invalidation are slower than copies.

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 9, 2022 5:09 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (6 responses)

Why can't we just mmap the pipe? It's ultimately "just" a buffer in kernelspace, is it really that hard to add a userspace mapping for it?

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 9, 2022 5:12 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (1 responses)

(Clarifying: I am aware that you cannot mmap pipes. I'm asking why this restriction shouldn't/can't be lifted.)

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 9, 2022 21:07 UTC (Thu) by anton (subscriber, #25547) [Link]

Peter Syrowatka added mmap() for pipes in his Diplomarbeit (master's thesis, in German).

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 9, 2022 15:01 UTC (Thu) by willy (subscriber, #9762) [Link] (3 responses)

What does it mean to mmap() a pipe?

Let's suppose I have a pipefd and addr = mmap(offset=0, length=1M). Then I call read(4kB) on pipefd. Does the pipe shuffle down so that *addr now refers to what was at 4kB, or does it still have a reference to what was at 0 when I called mmap()?

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 10, 2022 17:32 UTC (Fri) by NYKevin (subscriber, #129325) [Link] (2 responses)

Whichever is easier. As long as it's consistent and well-documented, userspace can figure out the rest.

However, I should point out that, if the pipe does not shuffle down, then you need to add an API for telling userspace the current read/write offsets, so that userspace knows where to begin reading or writing. Regardless, you also want an API for setting (or at least advancing) those offsets, so that userspace can emulate read/write calls. Therefore, you might as well not bother with the shuffling-down logic and just provide full get/set support for the offsets.

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 10, 2022 17:36 UTC (Fri) by willy (subscriber, #9762) [Link] (1 responses)

Haven't you just reinvented shared memory?

Mazzoli: How fast are Linux pipes anyway?

Posted Jun 10, 2022 17:55 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

Well gee, I thought that's what vmsplice(2) was trying to do in the first place.

But I think the big difference is this: If you e.g. create a file in /dev/shm (or any tmpfs) and just keep writing more and more data to it, it'll get bigger and bigger indefinitely, so you have to periodically seek to zero at both ends (and/or truncate it). Pipes are effectively ring buffers, so they don't have this problem. Your consumer can just call read, and doesn't have to know anything about your fancy mmap nonsense.