Wheeler: Fixing Unix/Linux/POSIX Filenames [LWN.net]

Wheeler: Fixing Unix/Linux/POSIX Filenames

Posted Mar 26, 2009 1:16 UTC (Thu) by mrshiny (guest, #4266)
In reply to: Wheeler: Fixing Unix/Linux/POSIX Filenames by epa
Parent article: Wheeler: Fixing Unix/Linux/POSIX Filenames

I was actually thinking more along the lines of:

1. Prevent files that start with dash (technically not a shell problem)
2. Prevent files that contain control characters (newline included)
3. Make the shells easy to use in the face of filenames with spaces, semi-colons, colons, quotes, punctuation, etc.

The first item is more of an interaction between programs and the shell and not specifically a shell problem. If a program doesn't support -- then it can never be used securely.

The second item seems like an obvious step to take with no downside.

The third item is what I meant by fixing the shells: shells should make it braindead-easy to manipulate filenames without them turning into commands or other nonsense. Once a filename is loaded into a variable you shouldn't have to worry about characters in the name turning into shell commands. Once that's in place we can start fixing scripts. Maybe an environment variable can determine how that instance of the shell works: in secure mode or legacy mode.

to post comments

Wheeler: Fixing Unix/Linux/POSIX Filenames

Posted Mar 26, 2009 14:45 UTC (Thu) by mjthayer (guest, #39183) [Link] (6 responses)

One thing that would help make the shell more solid would be treating -* as hidden files and skip over them when expanding wildcards.

Wheeler: Fixing Unix/Linux/POSIX Filenames

Posted Mar 26, 2009 15:08 UTC (Thu) by mjthayer (guest, #39183) [Link] (5 responses)

It could also recognise the null character as an argument separator as in 'find -print0'. It could even set some environment variable to tell tools like find that this is supported so that they can use it by default when not outputting to the console. And when substituting environment variables and backticked commands to the arguments for other commands, it could sanitise out anything starting with a hyphen. While this would break a few things, it would probably fix many more. While on that subject, the shell could enforce that substitutions that resolve to the arguments for other commands are not allowed to spill over (e.g. VAR='myfile; rm -rf /'; ls $VAR).

Wheeler: Fixing Unix/Linux/POSIX Filenames

Posted Mar 26, 2009 19:49 UTC (Thu) by dwheeler (guest, #1216) [Link] (3 responses)

[The shell] could also recognise the null character as an argument separator as in 'find -print0'. It could even set some environment variable to tell tools like find that this is supported so that they can use it by default when not outputting to the console.

Yes, I already added the "shell could recognize null as separator". And you're right, adding an environment variable could help (though it could also backfire on older scripts!).

While on that subject, the shell could enforce that substitutions that resolve to the arguments for other commands are not allowed to spill over (e.g. VAR='myfile; rm -rf /'; ls $VAR).

This particular example doesn't do quite what you think; it just passes to ls several values: "myfile;", "rm", "-rf", and "/", and you end up with some error messages and a listing of "/". But with more tweaking, you can definitely get some exploits out of this approach. Which is why removing the space character from IFS is a big help - then VAR would become a single parameter again.

Wheeler: Fixing Unix/Linux/POSIX Filenames

Posted Mar 28, 2009 1:11 UTC (Sat) by nix (subscriber, #2304) [Link] (2 responses)

bash implemented an environment variable to tell subprocesses where
arguments began and ended at one point.

It was removed, but I can't remember why: some sort of compatibility
problem?

Wheeler: Fixing Unix/Linux/POSIX Filenames

Posted Mar 31, 2009 7:47 UTC (Tue) by mjthayer (guest, #39183) [Link] (1 responses)

I was wondering now whether to ask about this on the Bash mailing lists. Just out of interest, are you involved with the development of Bash/the GNU tools in any way? You seem well informed about them.

Wheeler: Fixing Unix/Linux/POSIX Filenames

Posted Mar 31, 2009 19:28 UTC (Tue) by nix (subscriber, #2304) [Link]

I've contributed fixes now and then, but I just read a lot. :) The
projects are public, after all.

Wheeler: Fixing Unix/Linux/POSIX Filenames

Posted Apr 3, 2009 18:49 UTC (Fri) by anton (subscriber, #25547) [Link]

It could also recognise the null character as an argument separator as in 'find -print0'.

A few weeks ago I wanted to process my .ogg files which contain all kinds of characters that are treated as meta-characters by the shell or other programs I use in sheel scripts. I eventually ended up writing a new shell dumbsh that uses NUL as argument separator, and feeding it from find, with some intermediate processing in awk (which is quite flexible about meta-characters).