LWN.net Logo

A look at rsync performance

A look at rsync performance

Posted Aug 20, 2010 22:56 UTC (Fri) by jmorris42 (subscriber, #2203)
Parent article: A look at rsync performance

This isn't the only performance problem with rsync. I have a filesystem full of home directories. A 600G filesystem with about 500G filled. Doing an rsync between it and a backup server takes over 24 hours even though less than a GB typically changes between backups. Worse it consumes 1.6GB of ram and unless reniced makes the performance of the file server stink as it becomes entirely disk bound with a fair amount of CPU as well.


(Log in to post comments)

A look at rsync performance

Posted Aug 21, 2010 9:32 UTC (Sat) by shlomif (guest, #11299) [Link]

In this module-authors@perl.org thread, the CPAN admins have complained that rsync does an equivalent of "find . -type f" over the network every time, and, as a result, reducing the number of files in the CPAN will yield a good benefit.

A look at rsync performance

Posted Aug 23, 2010 19:39 UTC (Mon) by knobunc (subscriber, #4678) [Link]

What rsync flags are you using? I use -avP --delete to sync a multi-terabyte tree and it performs admirably.

A look at rsync performance

Posted Aug 26, 2010 11:31 UTC (Thu) by chojrak11 (guest, #52056) [Link]

-H for example can be a problem here. Again quote from the manpage:

Note that -a does not preserve hardlinks, because finding multi-
ply-linked files is expensive. You must separately specify -H.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds