LWN: Comments on "splice() and the ghost of set_fs()"

splice() and the ghost of set_fs()

stem — Thu, 09 Jun 2022 17:06:19 +0000

> One is that while set_fs() is active, various security measures (like SMEP and SMAP) are defeated.
Are you sure?
afaik, set_fs() has nothing to do with SM*P, it was abused wrt access_ok() - copy_*_user().

splice() and the ghost of set_fs()

willy — Sat, 28 May 2022 03:56:23 +0000

That's not the problem.

The problem is twofold. One is that while set_fs() is active, various security measures (like SMEP and SMAP) are defeated. The other is that (on some architectures and eg on a 4GB/4GB split x86-32), you may not actually be able to access userspace because accessing userspace actually accesses kernel space. On x86-64, you can tell from the high bits of the pointer whether it's userspace or kernel space, but that's not true eg on SPARC or PARISC.

splice() and the ghost of set_fs()

wahern — Sat, 28 May 2022 02:37:26 +0000

This was exactly Linus' original reasoning wrt vmsplice:

On Thu, 20 Apr 2006, Piet Delaney wrote:
> 
> What about marking the pages Read-Only while it's being used by the
> kernel

NO!

That's a huge mistake, and anybody that does it that way (FreeBSD) is totally incompetent.

[...]

That cost is _bigger_ than the cost of just copying the page in the first place.

Source: https://lkml.org/lkml/2006/4/20/310

See also Linus' justifications of splice and tee earlier in that thread. vmsplice is also briefly mentioned, and it's implicit from the context that much of the net value of vmsplice comes from combining with tee, as you mentioned earlier.

splice() and the ghost of set_fs()

SLi — Fri, 27 May 2022 18:02:27 +0000

Is this something that is a problem only because RAII is not used (and couldn't you _really_ get something like that working for the kernel), or do I misunderstand the original problem?

It seems like a silly limitation to me; as far as I can tell, it's 100% knowable at the time of the initial set_fs() call that you want to reset it exactly when the current function (or block) exits. It's not even something that would require producing code that is slower than the correct C code.

splice() and the ghost of set_fs()

Sesse — Fri, 27 May 2022 17:44:53 +0000

I believe the gift flag is seen as a mistake in retrospect.

FreeBSD has CoW on send(), I believe, but of course that means you need to go through a rather expensive page fault when/if the data changes.

splice() and the ghost of set_fs()

NYKevin — Fri, 27 May 2022 17:42:32 +0000

That still doesn't explain why you need the silly GIFT flag. Why can't the kernel just mark the offending pages as COW, like fork(2) does? You could do that without a special flag, because it should be transparent to userspace. Indeed, you can do that even for write(2), if you really want to.

splice() and the ghost of set_fs()

Sesse — Fri, 27 May 2022 13:09:25 +0000

vmsplice() is for sending the same data multiple times, I believe? E.g., pre-canned HTTP headers or small responses. vmsplice() once to get it from userspace into the kernel, then you can splice multiple times with copy.

splice() and the ghost of set_fs()

gerdesj — Fri, 27 May 2022 01:03:21 +0000

"But it is true that this type of episode makes the kernel's "no regressions" rule look a bit more like just a guideline."

A bit like the Constitution for the UKoGBnNI - https://en.wikipedia.org/wiki/Constitution_of_the_United_... - "Unlike in most countries, no attempt has been made to codify such arrangements into a single document."

... and yet somehow we struggle along.

It might look a bit daft to conflate the Linux kernel's governance with the UK's legal system but I think it is quite instructional. One of our cherished principles is the idea that you should be able to "quietly enjoy" your property - I can't remember the exact term. I think a coal mine making a racket caused the formative judgment.

So, the no regressions thing can be seen in similar terms: Don't break people's code.

However we have to be practical and sometimes stuff has to change.

splice() and the ghost of set_fs()

NYKevin — Thu, 26 May 2022 23:55:06 +0000

I think that depends on the use case. When fd_in is a pipe, splice should be quite fast, because the man page says it just copies pointers to individual pages. If you use a naive sendfile-like implementation, suddenly you're making real copies. Or at least, that's what I was able to figure out from the man pages, anyway.

Side note: Why are there so many syscalls that do almost, but not quite, entirely the same thing in this space? We've also got copy_file_range(2), which seems to be the same as splice(2) but both fds must be normal files. And then there's vmsplice(2), which appears to be exactly the same as read(2)/write(2), but with an overly-complicated API, unless you pass SPLICE_F_GIFT, which looks to be the "I'm doing something ridiculous, don't judge me" flag. And I imagine there's also some io_uring equivalent to this madness, too. Why is there not a simple, all-purpose "move data from here to here and don't bother me about the details, just do whatever's fastest or most reasonable" syscall?

* splice isn't it, because splice requires one of the fds to be a pipe.
* copy_file_range isn't it, because it requires *both* of the fds to be normal files.
* sendfile isn't it, because it's missing an offset argument for the output file, and the input file must not be a socket.
* io_uring isn't it, because it's like five syscalls and a userspace buffer, not one fire-and-forget syscall.

splice() and the ghost of set_fs()

josh — Thu, 26 May 2022 22:23:24 +0000

splice has to keep working; does it have to keep working *fast*? Could it become a wrapper around sendfile-like semantics, and then just have specific cases where it can go faster?

splice() and the ghost of set_fs()

zx2c4 — Thu, 26 May 2022 21:09:35 +0000

The ensuing performance discussion is somewhat interesting. Currently it's taking place across a few threads. My tracking of it is:

- I noticed the slow down here:
https://lore.kernel.org/lkml/Yoey+FOYO69lS5qP@zx2c4.com/
- Jens confirmed it's around 3%:
https://lore.kernel.org/lkml/0a6ed6b9-0917-0d83-5c45-70ff...
- Relatedly, I had proposed doing the same thing to /dev/zero: https://lore.kernel.org/lkml/20220520135030.166831-1-Jaso...
- Jens liked the idea, but Al pointed out the
performance issues, and later started figuring out why:
https://lore.kernel.org/lkml/Yokmu7bQpg70Bp8R@zeniv-ca.li...
- Al references resurrecting a particularly relevent older thread on
fsdevel:
https://lore.kernel.org/linux-fsdevel/Yokl+uHTVWFxoQGn@ze...
- This thread is now a many messages deep. That's where things are at now.
- Looks like Al's got some patches he's playing with in
https://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs....

Hopefully those close the performance gap and then all drivers get faster.