| |||||||||||||
Two new system calls: splice() and sync_file_range()Two new system calls: splice() and sync_file_range()Posted Apr 4, 2006 14:13 UTC (Tue) by zlynx (subscriber, #2285)In reply to: Two new system calls: splice() and sync_file_range() by andersg Parent article: Two new system calls: splice() and sync_file_range()
I gathered from LKML that sendfile is going to become a call to splice. But sendfile will probably stay in the kernel for many versions to come for legacy support. Linus does try to keep ABI compatibility for user-space.
(Log in to post comments)
Two new system calls: splice() and sync_file_range() Posted Apr 4, 2006 15:02 UTC (Tue) by axboe (subscriber, #904) [Link] sys_sendfile() can never go, as it's a part of the user space ABI. However, the internal implementation can be replaced with a call to splice() instead. The splice git branch has support for fd -> fd splicing now (by using a virtual pipe), so sys_sendfile can basically just use that.
And what becomes of zero-copy? Posted Apr 6, 2006 1:13 UTC (Thu) by xoddam (subscriber, #2322) [Link] > The splice git branch has support for fd -> fd splicing now> (by using a virtual pipe), so sys_sendfile can basically > just use that. But the internal pipe buffer uses dedicated pages, so there is a minimum of one copy involved. Doesn't sendfile() do zero-copy from a file's pages to a socket, if the socket's driver supports it? Also why isn't splice() just called cat()?
It is zero-copy Posted Apr 6, 2006 5:59 UTC (Thu) by axboe (subscriber, #904) [Link] It is zero-copy, why do you think the pipe pages are copied before being transmitted? Splice can even do zero-copy file copies, by moving pages from one file to another.
It is zero-copy Posted Apr 6, 2006 6:42 UTC (Thu) by xoddam (subscriber, #2322) [Link] > It is zero-copySorry -- I misremembered Linus' original pipe-buffer discussion. It has been a while: http://lwn.net/Articles/118760/ > Splice can even do zero-copy file copies Wow! Copying without copying! splice() must truly be a great philosopher!
It is zero-copy Posted Apr 6, 2006 7:16 UTC (Thu) by axboe (subscriber, #904) [Link] > Wow! Copying without copying! splice() must truly be a great philosopher!
Yeah it sounds crazy, but it's really true :-). You bring in the pages from the source file, then migrate them to the address space of the target file. Bingo, zero copy copies!
It is zero-copy Posted Apr 7, 2006 22:46 UTC (Fri) by giraffedata (subscriber, #1954) [Link] It sounds funny only because you're using "copy" two different ways in the same sentence. When we talk about copying a file, we're talking about copying from disk to disk. When we say zero-copy, we're talking about copying data from memory to memory.A splice-based file copy does one disk-disk copy, and no memory-memory copy, as contrasted with the traditional file copy that does one disk-disk copy plus two memory-memory copies. Since the naive observer wouldn't even expect there to be memory-memory copies involved in copying files, "zero copy file copy" shouldn't sound odd at all.
It is zero-copy Posted Apr 8, 2006 11:14 UTC (Sat) by axboe (subscriber, #904) [Link] A splice based copy does no copies. The source data is DMA'ed from hardware to the source file page(s), then those pages are moved to the destination and dirted to so will eventually go out to disk again with another DMA operations.
A normal copy will DMA those pages in, allocate pages in the target file address space, copy that data over, and then it'll be DMA'ed to disk again. So two DMA operations, and one full copy of all the data.
So a splice based copy will be zero memory copies, and two DMA "copies" (the dma operation above may be a series of dma transactions, depending on how large the file is). A normal copy contains the same number of DMA operations, but includes a memory copy.
It is zero-copy Posted Apr 8, 2006 17:31 UTC (Sat) by giraffedata (subscriber, #1954) [Link] So a splice based copy will be zero memory copies, and two DMA "copies" I presume "copies" is in quotes because you agree with me that the DMA operations are not copies in the sense we're talking about. (If they were, a "zero-copy" file read wouldn't be zero-copy, and moving data from kernel file data cache to user space would be two copies (kernel memory to register, register to user memory)). However, the combination of the two DMAs constitutes one disk copy. A disk copy is an instance of replicating data from one place on a disk to another, just as a memory copy is an instance of replicating data from one place in memory to another. And it's worth talking about because it takes time. If you could truly do a zero-copy copy, that would be remarkable. As it stands, "zero-copy copy" is just a trick of words in which you change the definition of copy in the middle of the sentence. A normal copy contains the same number of DMA operations, but includes a memory copy. Actually, the most normal file copy includes two memory copies -- from kernel file data cache of the source file to user space buffer, and from that buffer to the other file's kernel file data cache. With mmap, you can get it down to one, and with direct I/O you can get to zero.
It is zero-copy Posted Apr 8, 2006 18:26 UTC (Sat) by axboe (subscriber, #904) [Link] Copy is in quotes, because it's not the CPU doing the copy. Which is what is interesting, and why zero-copy just means zero CPU copies. That is where you pay the cost, at least in CPU cycles and potentially also in cache. So zero-copy definitely isn't just a play on words. It may sometimes be used in silly marketing ways, but if you are CPU bound it makes all the difference in the world that the CPU doesn't have to touch the data.
And yes, the normal copy is indeed two copies, to and from kernel/user space.
You can continue talking if you want, but don't expect a response from me.
heavy-duty philosophy Posted Apr 10, 2006 1:03 UTC (Mon) by xoddam (subscriber, #2322) [Link] > splice() must truly be a great philosopher!This is an in-joke, from the text adventure game version of Douglas Adams' Hitch-Hiker's Guide to the Galaxy: >open door The door explains, in a haughty tone, that the room is occupied by a super-intelligent robot and that lesser beings (by which it means you) are not to be admitted. "Show me some tiny example of your intelligence," it says, "and maybe, just maybe, I might reconsider." >get tea no tea: Dropped. >get no tea no tea: Taken. >inventory You have: no tea tea a small key a flowerpot a thing your aunt gave you which you don't know what it is a babel fish (in your ear) >open door The door is almost speechless with admiration. "Wow. Simultaneous tea and no tea. My apologies. You are clearly a heavy-duty philosopher." It opens respectfully.
|
|||||||||||||