curl | bash

Posted Apr 6, 2022 6:16 UTC (Wed) by NYKevin (subscriber, #129325)
In reply to: curl | bash by milesrout
Parent article: Debian still having trouble with merged /usr

If malicious software is running as you, with write access to your files, then it's already most of the way to the other side of the airtight hatchway, and you're effectively already lost (e.g. it could ptrace your shell, and next time you run sudo, it can rewrite the command that's passed to execve). The whole point of curl | bash is to run software that isn't already on your system. Just curling it to a local file will not, by itself, cause anything to run (yet).

curl | bash

Posted Apr 6, 2022 8:16 UTC (Wed) by wtarreau (subscriber, #51152) [Link] (26 responses)

It's not just about malicious stuff it's also about blindly trusting any assumption the script writer made about your system without having the opportunity to review them, and more importantly running the risk of executing an incomplete script if the transfer gets interrupted. For example if your install script ends with "rm -rf $HOME/.foobar" but the connection is cut after the "$HOME", "$HOME/" or "$HOME/." and curl stops after having printed what it received, you'll have a funny afternoon. And please don't tell me "bash will not do it since the line is not complete", it *will* do exactly what you wouldn't like it to do. Note that it also works with truncated variable names like "rm -rf $HOMEDIR", "rm -rf /tmp/foobar" and plenty of variants.

Nobody should ever use "curl|bash", that's irresponsible, there's no valid reason except laziness for using it. At least do "curl>foo && less foo" and only then "bash foo".
It's not difficult at all and offers more guarantees of correctness.

curl | bash

Posted Apr 6, 2022 10:08 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (6 responses)

> For example if your install script ends with "rm -rf $HOME/.foobar" but the connection is cut after the "$HOME", "$HOME/" or "$HOME/." and curl stops after having printed what it received, you'll have a funny afternoon.

If curl stops at this location, bash won't run that line because it won't receive a newline and will instead be SIGPIPE-d.

curl | bash

Posted Apr 6, 2022 14:22 UTC (Wed) by wtarreau (subscriber, #51152) [Link] (5 responses)

> If curl stops at this location, bash won't run that line because it won't receive a newline and will instead be SIGPIPE-d.

No, SIGPIPE is for the writer, not the reader, so it would be for curl that would receive it if bash had quit. Here bash receives a regular EOF, and as you can trivially test below, it *does* execute a truncated command that doesn't contain an LF:

$ echo -n 'echo blah' | bash
blah

curl | bash

Posted Apr 7, 2022 13:20 UTC (Thu) by MrWim (subscriber, #47432) [Link] (4 responses)

As an aside: I think it would be really neat if pipes (and sockets) had a separate way to signal EOF from closing the pipe, and then reading from a pipe that was closed, but not EOFed would cause SIGPIPE - much like writing to a closed pipe does. It feels like it would lead to much more robust software as a failure upstream in a pipeline would not be mistaken for successful complete transfer. It would remove most motivations for `set -o pipefail` too.

It's a suggestion about 40 years too late though :(.

curl | bash

Posted Apr 7, 2022 15:59 UTC (Thu) by wtarreau (subscriber, #51152) [Link] (3 responses)

> It's a suggestion about 40 years too late though :(.

50 years ;-) It would make pipes work a bit more like sockets, but with this come new classes of trouble such as who is responsible for closing in a process tree, etc. The success of pipes comes from their simplicity and we'd definitely lose quite a bit of it.

Also it would not necessarily address everything because the truncated script could for example start some processes and create temp files or mount temporary file systems, and being aborted in the middle by the transfer while it was expected to perform all the cleanup at the end.

Another example, imagine an install script that does something like this to convert ancient config files to the new format:

cleanup() { rm -rf /etc/svc-tmp; exit }
trap cleanup EXIT ERR
mkdir /etc/svc-tmp
migrate-cfg -i /etc/svc -o /etc/svc-tmp
find /etc/svc/ -name '*.cfg' | xargs rm
cp -a /etc/svc-tmp/* /etc/svc/

Of course it's completely made up you could end up with something bad if the script is truncated between "find" and "cp", you'd end up without your old files and with the new ones removed. And here catching the error doesn't change anything.

curl | bash

Posted Apr 8, 2022 12:54 UTC (Fri) by MrWim (subscriber, #47432) [Link] (1 responses)

> 50 years ;-) It would make pipes work a bit more like sockets, but with this come new classes of trouble such as who is responsible for closing in a process tree, etc.

Yeah, I guess for MPMC usage it would have to work on a file-descriptor level like `close` and some `fcntl` commands. Related to this I think it would be nice if child processes didn't inherit file-descriptors by default, except for stderr. Inheriting anything beyond that should be explicit, so child processes would start up with (0: /dev/null, 1: stderr, 2: stderr) by default.

On the subject of process trees and other changes to unix to dream about, I'd quite like it if all the child processes of a process were killed when the parent dies. You'd never have reparenting, or have to worry about killed processes leaving their children still running. Under this model if you wanted something to outlive parent (like with nohup) you'd have to ask a longer lived process (systemd?) to start it as a child.

Writing UNIX software is easy, as long as you're willing to tolerate weird behaviour when things fail. Writing robust UNIX software that does the right thing when you hit these edge cases is a massive PITA.

One can dream.

> The success of pipes comes from their simplicity and we'd definitely lose quite a bit of it.

True, it would definitely not be applicable to all pipes, and wouldn't solve all problems with them.

curl | bash

Posted Apr 8, 2022 14:03 UTC (Fri) by wtarreau (subscriber, #51152) [Link]

> I'd quite like it if all the child processes of a process were killed when the parent dies

You have prctl(PR_SET_PDEATHSIG) for this, and PR_SET_CHILD_SUBREAPER can also be useful to some extents.

> Writing UNIX software is easy, as long as you're willing to tolerate weird behaviour when things fail. Writing robust UNIX software that does the right thing when you hit these edge cases is a massive PITA.

Totally agreed.

curl | bash

Posted Apr 8, 2022 15:33 UTC (Fri) by nybble41 (subscriber, #55106) [Link]

For install script authors, an option that addresses most of the problem is to wrap your entire install script in a function, and then run that function at the end. This still has a potential issue if the script is truncated mid-word outside the function and the truncation happens to be a valid single-word command, but that's much more manageable than dealing with potential truncation anywhere in the script.

curl | bash

Posted Apr 6, 2022 13:07 UTC (Wed) by mgedmin (subscriber, #34497) [Link] (12 responses)

This is fixable by the author of the shell script structuring it so all the work is done in functions, so the overall script is just a bunch of function definitions followed by a call to main on the last line.

Of course you have to trust that the author of the script you're piping to bash is competent and not malicious.

But then again a Debian package's postinst script is a shell script that runs as root, so you also have to trust the author to be competent and not malicious.

curl | bash

Posted Apr 6, 2022 13:34 UTC (Wed) by LtWorf (subscriber, #124958) [Link]

But stuff on debian is signed, and perhaps people are less likely to upload malware under their real name.

curl | bash

Posted Apr 6, 2022 14:26 UTC (Wed) by wtarreau (subscriber, #51152) [Link] (10 responses)

> But then again a Debian package's postinst script is a shell script that runs as root, so you also have to trust the author to be competent and not malicious.

That's precisely my point. Poeple seem to think about "curl|bash" being secure because "it comes from a trusted site" or whatever, but it has nothing to do with trusting the origin for not being malicious, but for the shell doing the right thing. And we all do bugs all the time without being malicious, and in addition the way this is handled cannot protect against transport issues.

The only way to make this reliable is to first download, then execute if the download completed successfully (and you've done the necessary extra checks that depend on the origin and your level of trust).

A program to securely replace `curl | bash`

Posted Apr 6, 2022 15:31 UTC (Wed) by KJ7RRV (subscriber, #153595) [Link] (9 responses)

Would it be useful to have a program that would download a script, show it with a pager, and execute it with a shell? It could stay in memory the whole time, so unless an attacker can modify the process's memory they shouldn't be able to hijack the process (and if they can modify the memory of other processes, trying to protect against them likely won't do much good[?]). netrun https://example.com/a-shell-script.sh netrun --interpreter=python3 https://example.com/a-python-script.py

A program to securely replace `curl | bash`

Posted Apr 6, 2022 15:33 UTC (Wed) by KJ7RRV (subscriber, #153595) [Link]

Oops, the commands didn't format correctly.

netrun https://example.com/a-shell-script.sh

netrun --interpreter=python3 https://example.com/a-python-script.py

A program to securely replace `curl | bash`

Posted Apr 6, 2022 16:55 UTC (Wed) by wtarreau (subscriber, #51152) [Link] (7 responses)

Maybe. For sure most users wouldn't care, but it could at least excite their curiosity and encourage the authors to be even more rigorous. This could also highlight the insane UTF-8 homoglyphs and RTL hacks.

A program to securely replace `curl | bash`

Posted Apr 6, 2022 19:33 UTC (Wed) by KJ7RRV (subscriber, #153595) [Link] (6 responses)

Yes, that would be a good idea! It could possibly even run a simple malware scan to check for things like fork bombs, although it wouldn't be perfect.

A program to securely replace `curl | bash`

Posted Apr 7, 2022 19:42 UTC (Thu) by wtarreau (subscriber, #51152) [Link] (5 responses)

Now you know what to work on during your next holidays, you even found the name for it :-)
I wish that one day we can say that curl|bash is an error of the past waiting for netrun!

A program to securely replace `curl | bash`

Posted Apr 8, 2022 2:10 UTC (Fri) by KJ7RRV (subscriber, #153595) [Link] (4 responses)

I put a test version at https://git.kj7rrv.com/kj7rrv/netrun . It doesn't have all those features, but it does download a script, display it with less, ask if you want to continue, and if so, run it with bash.

A program to securely replace `curl | bash`

Posted Apr 14, 2022 16:47 UTC (Thu) by zamubafoo (subscriber, #154779) [Link] (3 responses)

If you really want to go fullbore, then have the process group running the script blocked from certain syscalls (eg. unlink(2), connect(2), etc). You could even have an interactive mode where the user can allow for the syscall to go through (or more likely a higher level abstraction than just direct syscalls).

You could do something like overlayFS rollback function, but I'm not sure if you can just apply the top layer to those below.

A program to securely replace `curl | bash`

Posted Apr 16, 2022 16:06 UTC (Sat) by KJ7RRV (subscriber, #153595) [Link] (1 responses)

Wouldn't that make most scripts you might want to run not work properly?

A program to securely replace `curl | bash`

Posted Apr 16, 2022 16:51 UTC (Sat) by KJ7RRV (subscriber, #153595) [Link]

The rollback would be good, if it's possible, but blocking syscalls seems like it would interfere with the functioning of a lot of scripts? (for example, blocking unlink() might result in a script creating temporary files and not being able to delete them)

A program to securely replace `curl | bash`

Posted Apr 17, 2022 15:32 UTC (Sun) by smurf (subscriber, #17840) [Link]

Rather than overlay, think copy-on-write. You can easily create a writeable snapshot of current /usr with btrfs and mount that on top of /usr. You want to roll back to the previous version? mount *that*. Problem solved, conceptually at least.

Cleaning up the resulting two (or three) subtrees from underneath the current one, in order to free the copyied-on-write bits, might require a couple of kernel enhancements, but that seems solvable.

curl | bash

Posted Apr 6, 2022 19:50 UTC (Wed) by gnu_lorien (subscriber, #44036) [Link] (5 responses)

"Nobody should ever use "curl|bash", that's irresponsible, there's no valid reason except laziness for using it. At least do "curl>foo && less foo" and only then "bash foo".
It's not difficult at all and offers more guarantees of correctness."

The valid reason is that people don't know how to make this safer and there's often no way to install the software they need other than doing this.

This is one of those areas where I think distros and package management software have really failed their users. If "It's not difficult at all and offers more guarantees of correctness" then why don't we have things like:

dpkg -i --web-bash-url=http://site-you-can-trust.org/install.bash
dnf -i --web-bash-url=http://site-you-can-trust.org/install.bash

One can envision how these systems could do something to keep a history of what those scripts did and what they changed that would be extremely difficult for users to put together on their own in a one-liner. Why don't they?

I could make a similar argument for package systems and their disdain for vendoring. Like it or not this is how a lot of packages are shipped to consumers and users are not choosing the distros over the software that they want. After all the hemming and hawing about how "npm should do better" all users really come away with is, "My distribution can't install the software I want, so I'll use whatever the website suggests."

curl | bash

Posted Apr 7, 2022 15:45 UTC (Thu) by wtarreau (subscriber, #51152) [Link]

> The valid reason is that people don't know how to make this safer and there's often no way to install the software they need other than doing this.

Yes there is another way, at least the one I mentioned to do it in two steps. 1) download, 2) execute. *Even* doing just that without reading the script is much safer because they only need to trust the origin and not the network. Really the only reason they proceed like this (and more importantly to provide such methods) is because it look cool and it impresses friends.

> Like it or not this is how a lot of packages are shipped to consumers

That's the first point I'm complaining about.

> and users are not choosing the distros over the software that they want

Yes they do choose to blindly copy-paste that dangerous line without checking. Downloading the script *before* executing it is not difficult, and if they cannot do that because they really have no idea what they're doing in this funny terminal, they'd rather not copy-paste random strings from the web because even copying extra delimiters or misplaced quotes can have devastating effect for them.

curl | bash

Posted Apr 7, 2022 21:37 UTC (Thu) by calumapplepie (guest, #143655) [Link] (3 responses)

> dpkg -i --web-bash-url=http://site-you-cant-trust.org/install.bash

Never going to happen. Dpkg isn't a tool for installing arbitrary software on your system: it is a tool for installing software intended for Debian systems, and formatted appropriately. Adding that option implies to the user that dpkg will somehow keep track of what the script does, and allow you to apt remove it later. Which you can't, because that script can do whatever it wants.

Perhaps someone will put together a separate shell script that does that for you in the future, but I doubt it. The trend of "just run this magic script" is a security nightmare, and simply logging what is run isn't enough: any decent malicious script will simply edit the log.

> Like it or not this is how a lot of packages are shipped to consumers and users are not choosing the distros over the software that they want.

I avoid any software that is not in my distribution like the plague. The literal plague. Just because you have a dozen outdated copies of Electron powering a dozen different apps doesn't mean the rest of us want that. The first place many users go for software, in my opinion, is a quick apt search. You know whatever gets added through there is trustworthy, can be easily removed, and won't cause duplication across your system. The same cannot be said for software that is self-published.

If a software is being shipped to consumers via a magic shell script, it is two things. One, it isn't a package: it contains many unrelated things that you probably already have, and it isn't being installed and managed by a centralized package manager. Two, it isn't trustworthy: anyone can publish a website, and anyone can but binaries up for download there. There is an independent review process for Debian packages: no such process exists for random websites. There is a signing mechanism for Debian packages, with a root of trust in keys that are placed there on installation: no such root of trust exists for a third party package. Sure, it might verify what it downloaded: but since it also just downloaded the key, that's no guarantee that an attacker didn't corrupt it.

If you want me to run your code, and it isn't absolutey essential I run your specific product, I will look for an alternative in my distribution.

curl | bash

Posted Apr 7, 2022 22:28 UTC (Thu) by mpr22 (subscriber, #60784) [Link] (1 responses)

> If a software is being shipped to consumers via a magic shell script

If software is being shipped to people via a magic shell script, it probably isn't being shipped to "consumers", but rather to techies with a dismally low standard of security.

I'm reasonably confident that the main route for software to be installed on Linux systems by "consumers" is probably the Steam client, with some favouring the GOG or itch.io clients instead.

curl | bash

Posted Apr 8, 2022 6:36 UTC (Fri) by kleptog (subscriber, #1183) [Link]

> If software is being shipped to people via a magic shell script, it probably isn't being shipped to "consumers", but rather to techies with a dismally low standard of security.

Sometimes you're just not given a choice. The recommended procedure for installing Rust is a curl | sh command. Sure, Debian ships with a version of Rust, but it's sometimes too old to build whatever software you're using which decided to start using Rust and require a super recent version. So you hold your nose and think "it's building inside a container anyway".

The situation got a lot better with Bullseye, but in a few places I wish there was a debian-backports with recent versions of Rust.

curl | bash

Posted Apr 8, 2022 14:44 UTC (Fri) by floppus (guest, #137245) [Link]

>Perhaps someone will put together a separate shell script that [keeps track of what the script does, and allows you to apt remove it later], but I doubt it.

At least one such script exists (checkinstall). I haven't used it in many years and it's certainly not perfect, but it worked acceptably well back in the day.