LWN: Comments on "Deprecating scp"

Deprecating scp

wtarreau — Wed, 25 Aug 2021 13:30:04 +0000

Bah, it means it's time to reinstall rsh and rcp to get something durable. Reliable old tools never completely disappear, only their temporary replacents.

I like sshfs

RoyBellingan — Sat, 17 Apr 2021 20:46:21 +0000

Thank you, I felt alone using this marvellous tool.

Deprecating scp

flussence — Fri, 05 Feb 2021 19:39:56 +0000

One could cobble something together with rsync and sshfs (on the sender or both endpoints), which might work out faster if it's a partial copy. It's a bit of a shame rsync doesn't support this mode of operation directly.

Deprecating scp

mbunkus — Mon, 25 Jan 2021 12:17:04 +0000

You can approximate "scp -3" with ssh & tar, without having to use tunnels or anything else, as in:

ssh host1 "tar czf - /path /other/path" | ssh host2 "tar xzfC - /"

Deprecating scp

tconnors — Mon, 25 Jan 2021 09:44:47 +0000

> At this point, it is claimed to work for most basic usage scenarios; some options (such as -3, which copies files between two remote hosts by way of the local machine) are not supported

That would be unfortunate. The only time I've used scp in the past... 15 years(?) is for scp -3.

The alternative is setting up horrible reverse tunnels with ssh, from memory. Yes, I do believe I use this in production at work (because I needed sync of directory trees), but am not about to check because we're about to enter into a public holiday.

Deprecating scp

nix — Fri, 20 Nov 2020 22:46:16 +0000

That it was "out of scope", that rsync's job was to copy files, and nonportable stuff doesn't count. (I guess they define what is "nonportable".)

Deprecating scp

rodgerd — Thu, 19 Nov 2020 22:27:08 +0000

That's very disappointing. What was the rationale?

Deprecating scp

nye — Thu, 19 Nov 2020 17:46:27 +0000

> rsync does have an option that says "only write stuff that's changed"

--inplace --no-whole-file

By default rsync will only use --whole-file if both source and destination are local, so the second option is redundant in that case.

I like sshfs

Luyseyal — Sun, 15 Nov 2020 14:19:26 +0000

I just came here to say that I love using sshfs. Does it meet every scp use case? Nah. But it is super handy.

Deprecating scp

nim-nim — Fri, 13 Nov 2020 09:39:26 +0000

Well that probably would mean rewriting scp around HTTP/3, not sftp.

Browsers have become the elephant in the room everyone optimizes for.

Deprecating scp

jcpunk — Thu, 12 Nov 2020 14:45:00 +0000

I love the idea of taking what we've learned about high performance, secure, network protocols over the last 30 years and improving the overall landscape!

I don't love the idea of updating a billion scripts and having to explain why the documentation everyone finds all over the internet doesn't work anymore.

All in all, this is an optimistic mixed bag. But it sure is the kind of thing the community should be looking at.

Deprecating scp

Creideiki — Mon, 09 Nov 2020 22:09:33 +0000

Having had the misfortune of working in an environment that mixed modern Linux, ancient Solaris, and weird real-time UNIXoids, I laugh at the notion of "a tar stream", as if that is somehow well-defined. You don't even have to go that far out in the weeds to find incompatibilities; read https://mgorny.pl/articles/portability-of-tar-features.html and despair.

Deprecating scp

nix — Mon, 09 Nov 2020 17:10:56 +0000

After it's deprecated it will be a thing that used to be a binary. Perhaps related to SCP-2602 (which used to be a library).

Deprecating scp

rahulsundaram — Mon, 09 Nov 2020 16:18:30 +0000

Looks like they would need to clarify this

https://discussion.fedoraproject.org/t/improving-license-...

The intent is for all the content to be under the same free content license

Deprecating scp

ptman — Sun, 08 Nov 2020 11:09:14 +0000

But there are shells that restrict you to only scp. Like rssh or scponly

rsync -a is not the same as cp -R

Wol — Sun, 08 Nov 2020 00:03:28 +0000

Whoops :-)

I meant, of course, hard links.

Oh - and if you don't need to preserve hard links (which I do), it's not a good idea to try. I think my rsync or cp crawled, I had so many it needed to keep track of ...

Cheers,
Wol

rsync -a is not the same as cp -R

Sesse — Sat, 07 Nov 2020 23:22:46 +0000

FWIW, links in UNIX are either hard (hardlinks) or symbolic (symlinks). There's no such thing as “hard symlinks”.

Deprecating scp

ncm — Sat, 07 Nov 2020 06:45:36 +0000

The UNIX tradition is to read all the man pages from front to back and memorize it all.

That is harder nowadays, because there are a lot more with more pages each. But a reasonable selection of them deserve the old-school treatment. Rsync it probably one that does. Anyway, a curated selection of its options. (Eyes rolling? That's our world now.)

Instead of sitting and memorizing the whole collection, of an evening, why not bring home rsync(1), and memorize just that? What else would you do, look at your phone?

Deprecating scp

Comet — Fri, 06 Nov 2020 22:24:25 +0000

If memory serves, then the original SSH.com implementation, when they released protocol v2, made scp over v2 use the SFTP protocol.

At the time that OpenSSH was launched, the developers involved didn't yet see the value in dropping the simplicity of the rcp-over-ssh protocol or messing with interop.

Time passes, domain expertise is acquired. The Finns laugh last.

rsync options exist for a reason

marcH — Fri, 06 Nov 2020 21:02:47 +0000

> I disagree, the many other rsync options exist for a reason. I have a situation where I use three lines of options for rsync (!).

Of course other rsync options exist for a reason and I do look them up and use them sometimes. Above I was only referring to basic, [s]cp-like usage.

Answering other comments too: if you must preserve hardlinks, sparse files, ACLs or some other unusual stuff then you better check the man page of _any_ tool you use.

> The world needs a "simple replacement for cp that copies remotely" and a "sophisticated tool for remote synchronization". They don't need to be the same tool, and historically have been different.

They don't need to be different tools either. I've been using a unique tool for "simple replacement for recursive, _local_ cp" all the way to "sophisticated tool for remote synchronization" and it has made simple things easy and complex things possible. So if you want a solution that already exists now then give it a try. If scp muscle memory is just too strong then a sincere "best of luck!"

Deprecating scp

flewellyn — Fri, 06 Nov 2020 18:38:17 +0000

Maybe as a -J entry.

Deprecating scp

rahulsundaram — Fri, 06 Nov 2020 17:46:40 +0000

Don't know why you want that but there is openrsync

Deprecating scp

mirabilos — Fri, 06 Nov 2020 17:27:25 +0000

It would also be great if rsync could be relicenced to a BSD-style licence :/

rsync options exist for a reason

david.a.wheeler — Fri, 06 Nov 2020 17:02:06 +0000

I disagree, the many other rsync options exist for a reason. I have a situation where I use three lines of options for rsync (!).

The world needs a "simple replacement for cp that copies remotely" and a "sophisticated tool for remote synchronization". They don't need to be the same tool, and historically have been different. I'd like to see a more secure version of the first category. In the first category most people don't really need the scp *protocol*, they just need a simple replacement for "cp" that works remotely. That is, it's the CLI interface, not the protocol, that matters.

Deprecating scp

lwn@pck.email — Fri, 06 Nov 2020 16:00:44 +0000

HN comments are a dumpster fire on this one but this sort of stuff is why I pay for LWN, nice job.

rsync remote-to-remote

bkw1a — Fri, 06 Nov 2020 14:23:53 +0000

I love rsync and use it many times a day, but my fingers still sometimes type scp for single file transfers. One way in which rsync let me down recently was this: One of my users was asking about how to efficiently transfer files between two remote servers. I confidently told him he could do this with rsync, like "rsync -rv remote1:somewhere/ remote2:somewhere/". I was surprised and embarrassed to find that this doesn't actually work. I must have been thinking of scp, where such a thing really does work.

Deprecating scp

IanKelling — Fri, 06 Nov 2020 13:51:09 +0000

> Not sure why you consider that nonfree?

The page has no license. Looking more closely, there is a link to a terms and conditions, which states "Red Hat original content" is CC-BY-SA, but not other content. This page has no indication that it is "Red Hat original content" so, still no license which means nonfree.

Deprecating scp

nix — Fri, 06 Nov 2020 12:50:34 +0000

No, and apparently it never will use reflinks because it is out of scope. It also can't copy the filesystem-specific attributes (like the immutable flag) set via chattr/lsattr. Patches to do this have been proposed and rejected :(

Deprecating scp

ptman — Fri, 06 Nov 2020 12:49:14 +0000

Here's a good explanation: https://news.ycombinator.com/item?id=25006723

Deprecating scp

Wol — Fri, 06 Nov 2020 12:48:55 +0000

Dunno what option it is (and I don't think it's -a), but rsync does have an option that says "only write stuff that's changed". Great for making backups on a journalled file system as each snapshot is a full backup, but apart from the first only takes up the space of an incremental... (and yes for large files eg databases it only writes that part of the file)

Cheers,
Wol

Deprecating scp

nix — Fri, 06 Nov 2020 12:47:54 +0000

I'm now tempted to write an SCP entry to get the scp protocol into the Foundation as an SCP object itself. With the backtick-execution semantics, it would hardly need any tweaking at all to make it horrible and creepy enough... :)

Deprecating scp

dezgeg — Fri, 06 Nov 2020 12:19:32 +0000

SFTP server is not available everywhere either (in embedded systems especially).

Deprecating scp

gerdesj — Fri, 06 Nov 2020 11:47:40 +0000

"I've thought of a valid use for this kind of behaviour that someone might actually be relying on. :-)
...
(i.e. ensure that the destination directory exists before writing the file to it)"

Sounds like a job for smkdir

Deprecating scp

rahulsundaram — Fri, 06 Nov 2020 11:16:42 +0000

Fedora Magazine content is http://creativecommons.org/licenses/by-sa/3.0/ Not sure why you consider that nonfree?

Deprecating scp

CChittleborough — Fri, 06 Nov 2020 10:58:00 +0000

I see what you did there.

Warning: the SCP wiki can entertain^Wdistract you for hours.

Deprecating scp

grawity — Fri, 06 Nov 2020 10:52:30 +0000

My trick is to use "cp -a source/. dest/" to always get just the contents.

Deprecating scp

grawity — Fri, 06 Nov 2020 10:51:44 +0000

PuTTY has already done so; its "pscp" command uses SFTP when it can. The SSH.COM "scp2" is an SFTP client as well. OpenSSH is the only one who hasn't.

Deprecating scp

ptman — Fri, 06 Nov 2020 10:44:11 +0000

IIRC SFTP is more like NFS than FTP and latency limits its bandwidth quite a lot.

Deprecating scp

gray_-_wolf — Fri, 06 Nov 2020 10:32:52 +0000

> tl;dr: cp -R is insane, use rsync -a. Using it more than once is not just safer it's also much faster of course: it can be used as an actual backup tool.

Does rsync -a also use CoW the way cp does as long as the filesystem supports it?

rsync -a is not the same as cp -R

taneli — Fri, 06 Nov 2020 10:01:02 +0000

Indeed, you will need -H to preserve hard links.