curl | bash

Posted Apr 7, 2022 15:59 UTC (Thu) by wtarreau (subscriber, #51152)
In reply to: curl | bash by MrWim
Parent article: Debian still having trouble with merged /usr

> It's a suggestion about 40 years too late though :(.

50 years ;-) It would make pipes work a bit more like sockets, but with this come new classes of trouble such as who is responsible for closing in a process tree, etc. The success of pipes comes from their simplicity and we'd definitely lose quite a bit of it.

Also it would not necessarily address everything because the truncated script could for example start some processes and create temp files or mount temporary file systems, and being aborted in the middle by the transfer while it was expected to perform all the cleanup at the end.

Another example, imagine an install script that does something like this to convert ancient config files to the new format:

cleanup() { rm -rf /etc/svc-tmp; exit }
trap cleanup EXIT ERR
mkdir /etc/svc-tmp
migrate-cfg -i /etc/svc -o /etc/svc-tmp
find /etc/svc/ -name '*.cfg' | xargs rm
cp -a /etc/svc-tmp/* /etc/svc/

Of course it's completely made up you could end up with something bad if the script is truncated between "find" and "cp", you'd end up without your old files and with the new ones removed. And here catching the error doesn't change anything.

curl | bash

Posted Apr 8, 2022 12:54 UTC (Fri) by MrWim (subscriber, #47432) [Link] (1 responses)

> 50 years ;-) It would make pipes work a bit more like sockets, but with this come new classes of trouble such as who is responsible for closing in a process tree, etc.

Yeah, I guess for MPMC usage it would have to work on a file-descriptor level like `close` and some `fcntl` commands. Related to this I think it would be nice if child processes didn't inherit file-descriptors by default, except for stderr. Inheriting anything beyond that should be explicit, so child processes would start up with (0: /dev/null, 1: stderr, 2: stderr) by default.

On the subject of process trees and other changes to unix to dream about, I'd quite like it if all the child processes of a process were killed when the parent dies. You'd never have reparenting, or have to worry about killed processes leaving their children still running. Under this model if you wanted something to outlive parent (like with nohup) you'd have to ask a longer lived process (systemd?) to start it as a child.

Writing UNIX software is easy, as long as you're willing to tolerate weird behaviour when things fail. Writing robust UNIX software that does the right thing when you hit these edge cases is a massive PITA.

One can dream.

> The success of pipes comes from their simplicity and we'd definitely lose quite a bit of it.

True, it would definitely not be applicable to all pipes, and wouldn't solve all problems with them.

curl | bash

Posted Apr 8, 2022 14:03 UTC (Fri) by wtarreau (subscriber, #51152) [Link]

> I'd quite like it if all the child processes of a process were killed when the parent dies

You have prctl(PR_SET_PDEATHSIG) for this, and PR_SET_CHILD_SUBREAPER can also be useful to some extents.

> Writing UNIX software is easy, as long as you're willing to tolerate weird behaviour when things fail. Writing robust UNIX software that does the right thing when you hit these edge cases is a massive PITA.

Totally agreed.

curl | bash

Posted Apr 8, 2022 15:33 UTC (Fri) by nybble41 (subscriber, #55106) [Link]

For install script authors, an option that addresses most of the problem is to wrap your entire install script in a function, and then run that function at the end. This still has a potential issue if the script is truncated mid-word outside the function and the truncation happens to be a valid single-word command, but that's much more manageable than dealing with potential truncation anywhere in the script.