Sharing buffers between devices

A more efficient dd between storage devices could be implemented if the data did not need to be shunted in and out of the user space.

The shunting in and out of user space is easy to eliminate: mmap.

That leaves you with the copying from one device's buffer to the other. For that, Linus invented the 'splice' system call about ten years ago and, as justincormack points out in another comment on this article, actually implemented in 2005. Splice takes two file descriptors and a length as arguments and reads that many bytes from one of the devices and writes it to the other, by DMAing into, then out of, the same memory. https://lwn.net/Articles/119682/ . I don't know what the current state of deployment is, though.

The MCA thing would presumably be the next step, where the data doesn't have to stop over in system memory.

In big systems, where the storage devices are rather separate from the CPUs, this exists in the form that you can tell a device to send some of its contents directly to another device, e.g. through a fibre channel network. I guess the same thing over a PCI-class network can't be far behind.

In fact, my guess is that the bus protocol itself allows this in PCI Express and Infiniband; I don't think the CPU/main memory is particularly special in those protocols. Does somebody know?