Rethinking splice()

Posted Feb 18, 2023 8:40 UTC (Sat) by NYKevin (subscriber, #129325)
In reply to: Rethinking splice() by Sesse
Parent article: Rethinking splice()

The argument isn't that buffers are illogical. The argument is that a more useful primitive would be one where I can just do

int ret = readthenwrite(infd, outfd, maybe_some_other_args);

and never even deal with buffers at all. Maybe under the hood the kernel deals with buffers, but as an application programmer, I would rather not have to think about them if I can avoid it.

This is a superset of splice, because splice is just a special case of it where one of the fds happens to be a pipe.

Rethinking splice()

Posted Feb 18, 2023 11:55 UTC (Sat) by Sesse (subscriber, #53779) [Link] (2 responses)

It's not really a strict superset, since splice() follows tee() (where you can read once and send many) or vmsplice()+tee() (write once from memory, send many; think standardized HTTP headers).

Most of this sort of feels obsolete with io_uring dominating, though.

Rethinking splice()

Posted Feb 19, 2023 8:50 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (1 responses)

Those syscalls are not splice. I never claimed that readthenwrite (or whatever less-terrible name you want to call it, since readthenwrite is obviously just a placeholder) would be a superset of "splice, and also a bunch of random other syscalls that have a vaguely similar implementation to splice." My specific beef is not with the splice family of syscalls, it's with the family of "syscalls that are semantically equivalent to read and then write, but also have arbitrary restrictions that make no sense to the average application developer and have to be looked up in the man page every time you use them." See my other comment regarding copy_file_range(2) and sendfile(2).

Rethinking splice()

Posted Mar 3, 2023 3:37 UTC (Fri) by njs (subscriber, #40338) [Link]

I think the logic would be:

- splice composes with tee/vmsplice/etc. to allow more complex zero-copy IO flows

- readthenwrite could not replace splice for this purpose, because splice has *super bizarre* semantics that are different from readthenwrite, as this article notes. (In particular, readthenwrite from a file into a pipe has to make a copy -- it skips bouncing through userspace, but it has to copy data from the page buffer into the pipe buffer, instead of doing wacky things with page pointers like splice does.) But those bizarre semantics are what let compositions of splice/tee/vmsplice/etc. be zero-copy.

I don't think it really works out; this stuff really needs to be replaced with something better. But I can at least see why splice seemed like a good idea at the time.