How useful should copy_file_range() be?

Posted Feb 19, 2021 7:32 UTC (Fri) by Deewiant (subscriber, #97394)
In reply to: How useful should copy_file_range() be? by fw
Parent article: How useful should copy_file_range() be?

The current code in Rust does seem to handle arbitrary fds too. It's all here, but there's a lot of code to read through (which supports your point that it's difficult): https://github.com/rust-lang/rust/blob/7647d03c33339bd85a...

Looks like it starts by checking the underlying file type (with a stat() if necessary) and only tries copy_file_range on regular files of nonzero size (lines 122 and 168; and 284 and 469 for the logic leading to the stat() itself), while still falling back to other methods if copy_file_range only copied zero bytes (lines 175 and 563). Overall there seems to be a lot of logic around keeping track of what was actually written vs. what was reported by the syscalls.

How useful should copy_file_range() be?

Posted Feb 19, 2021 18:55 UTC (Fri) by fw (subscriber, #26023) [Link]

I see, there is a different code path that reaches the copy_file_range system call.

On the other hand, it is still deeply nested within the library, and it is not immediately obvious whether the callers of the universal copy routines expect consistent file offsets on errors. For a function that is directly modeled on the system call (which seems to be the case for Go), predictable file offset behavior seems quite important.

How useful should copy_file_range() be?

Posted Feb 19, 2021 20:29 UTC (Fri) by the8472 (guest, #144969) [Link] (4 responses)

You're looking at the linux implementation for io::copy which can copy arbitrary readers to writers but specializes to various syscalls when types wrapping file descriptors of unknown type are passed, hence the complexity.

There also is fs::copy which shares some code but has fewer cases to cover and does fewer checks before invoking copy_file_range since it's only meant to copy entire regular files.

The relevant parts are here and here

. Note that it doesn't use stat information to decide what to do in the fs::copy case, it just tries copy_file_range and then falls back in various cases, including 0 byte reads.

How useful should copy_file_range() be?

Posted Jul 10, 2021 12:57 UTC (Sat) by jaykrell (guest, #153190) [Link] (3 responses)

The bug is that /proc exists at all.
This data should be retrieved through strongly typed special purpose function calls.
Or at least, perhaps, allow open, for a hierarchical namespace, but not read, and reveal all information via ioctl.

Not everything is a file!

In fact, most things are not a file.
Is the screen a file?
Is the keyboard a file?
Are sockets files? They have read/write, but how about seek and mmap?

How useful should copy_file_range() be?

Posted Jul 10, 2021 13:54 UTC (Sat) by mpr22 (subscriber, #60784) [Link]

Those things are not files, but allowing userspace to treat them as file-like for the simplest common use case (sequential I/O) is one of the things that contributed to Unix eating most of the rest of the server operating system industry's breakfast, lunch, dinner, and face.

How useful should copy_file_range() be?

Posted Jul 10, 2021 21:48 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

> Or at least, perhaps, allow open, for a hierarchical namespace, but not read, and reveal all information via ioctl.
Windows NT tried that. It failed miserably.

How useful should copy_file_range() be?

Posted Jul 10, 2021 23:16 UTC (Sat) by flussence (guest, #85566) [Link]

>Is the screen a file?
>Is the keyboard a file?

You're being facetious, but it's occasionally very useful to be able to do things like check which port a monitor is plugged in on or framedump the console on a server using only ssh.