| |||||||||||||
A look at rsync performanceA look at rsync performancePosted Aug 19, 2010 22:53 UTC (Thu) by Comet (subscriber, #11646)In reply to: A look at rsync performance by ewen Parent article: A look at rsync performance
-W, --whole-file copy files whole (without rsync algorithm)
(Log in to post comments)
A look at rsync performance Posted Aug 20, 2010 4:29 UTC (Fri) by jcvw (subscriber, #50475) [Link]
I tried that. It doesn't help. The amount of user and system time is still incredibly high (compared to a simple cp). rsync doesn't use a tight read/write loop, like cp does, but (even in local cases) uses two processes, a socket and lots of select system calls. The -W doesn't change anything there (unfortunately).
A look at rsync performance Posted Aug 20, 2010 4:46 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
my first reaction on reading this is that the processes are stalling, and the behavior you describe to improve performance sounds like it's on the same tack.
did you try tweaking the buffer sizes that rsync uses to see if larger buffers may smooth things out a bit?
I would still expect it to take significantly more CPU time than a plain cp, I'm not bothered by that (it's really designed for a different job that it excels at, this is a degenerate corner case for it), but the throughput should be higher.
while the approach is less efficient than the read()/write() in a tight loop, if it can keep the buffers between the processes full it should be able to do fairly well (as your testing shows when you tweak things so that the threads aren't waiting for each other much), so perhaps larger buffers can avoid the stalls.
|
|||||||||||||