LWN.net Logo

A look at rsync performance

A look at rsync performance

Posted Aug 19, 2010 10:35 UTC (Thu) by dafid_b (guest, #67424)
In reply to: A look at rsync performance by spaetz
Parent article: A look at rsync performance

when done as root, the tar pipe preserves ownership details etc..

I dont think that is part of cp.


(Log in to post comments)

A look at rsync performance

Posted Aug 19, 2010 10:48 UTC (Thu) by spaetz (subscriber, #32870) [Link]

I commonly use "cp -a" which includes --preserve=all: preserve the specified attributes: mode, ownership,time-stamps, context, links, xattr, all

but perhaps that is missing out things that tar manages to preserve. Not sure. Just curious in any case.

A look at rsync performance

Posted Aug 19, 2010 15:08 UTC (Thu) by bronson (subscriber, #4806) [Link]

It's a graybeard thing. 20 years ago, cp would screw up permissions, dates, ownership, symlinks, device files, etc. Different platforms would require different command-line options and then screw up different things. It was insane. Tar, on the other hand, pretty much got it right on every platform.

Nowadays cp -a works well everywhere (in my experience) so there's no need to resort to tar. It's just damage from the Unix wars.

A look at rsync performance

Posted Aug 19, 2010 19:31 UTC (Thu) by pj (subscriber, #4506) [Link]

One advantage is that it's easily modified to work over ssh:

tar cf - . | ssh user@remote "cd /dest/dir; tar xf -)

or

(ssh user@remote "cd /src/dir ; tar cf - . ") | (cd /dest/dir; tar xf -)

A look at rsync performance

Posted Aug 20, 2010 11:53 UTC (Fri) by NAR (subscriber, #1313) [Link]

A 'cp' command can also be easily modified to work over the network, just add an 's' to the front of 'cp' :-)

A look at rsync performance

Posted Aug 20, 2010 12:28 UTC (Fri) by dsommers (subscriber, #55274) [Link]

True ... but if you have a lot of files, especially smaller files, the tar path with ssh is way faster than scp. Try copying a git repository (~2-3MB) from one site to another site. My experiences is that tar+ssh beats scp significantly.

A look at rsync performance

Posted Aug 20, 2010 14:52 UTC (Fri) by spaetz (subscriber, #32870) [Link]

> but if you have a lot of files, especially smaller files, the tar path with ssh is way faster than scp. Try copying a git repository (~2-3MB) from one site to another site. My experiences is that tar+ssh beats scp significantly.

Only because you open a new ssh connection per file by default and tar+ssh opens only one. Which causes lots of overhead. If you reuse your ssh connection scp will be fast as well:
http://www.debian-administration.org/articles/290

A look at rsync performance

Posted Aug 21, 2010 2:05 UTC (Sat) by dmag (subscriber, #17775) [Link]

No, scp can't copy symlinks.

A look at rsync performance

Posted Aug 24, 2010 20:01 UTC (Tue) by BackSeat (subscriber, #1886) [Link]

No need for all the "cd" commands:
tar -C /src/dir -cf - . | tar -C /dest/dir -xf -

A look at rsync performance

Posted Aug 19, 2010 21:08 UTC (Thu) by evgeny (guest, #774) [Link]

There is one thing tar and cp -a do differently, which, depending on what you do could be either a feature or a misfeature. tar tries to restore files according to _literal_ names of the owner (if they exist; and they do by default). This can be overridden with the --numeric-owner flag. E.g. if you forget to specify this flag and untar a backup of a virtual container from the host, you'll end up with a mess of file ownerships. Was bitten by this once...

A look at rsync performance

Posted Aug 25, 2010 3:09 UTC (Wed) by roelofs (guest, #2599) [Link]

Nowadays cp -a works well everywhere (in my experience) so there's no need to resort to tar.

BSDs included? In my experience they've been mighty picky about the GNUisms (or "things that would have been GNUisms if someone else hadn't done them first") they're willing to implement. I remember being surprised by something along those lines just a couple of months ago, though I've forgotten the details already.

But perhaps cp -a came from BSD in the first place...

Greg

+1 Informative

Posted Aug 25, 2010 13:25 UTC (Wed) by dmarti (subscriber, #11625) [Link]

Just ssh-ed in to a FreeBSD 7.2 system -- `cp -a` works, and `-a` is in the man page.

A look at rsync performance

Posted Aug 19, 2010 10:50 UTC (Thu) by valhalla (subscriber, #56634) [Link]

cp -p preserves mode, ownership and timestamps, and the --preserve option can be used to do a finer selection of what should be preserved.

A look at rsync performance

Posted Aug 19, 2010 10:56 UTC (Thu) by dafid_b (guest, #67424) [Link]

ok - i read the fine cp manual page and now think that preserving ownership details etc are part of cp. However i am still confused as many of the notes in the manual page refer to topics I know nothing about.

I would use the tar pipe of old as I expect it to build a proper copy in the new location.

Having read the cp manual page again (and again) I fear my confidence in tar might be misplaced :(.

Anyone know of a tutorial for each of the cp options?

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds