Evolution of shells in Linux (developerWorks)

Posted Dec 8, 2011 10:43 UTC (Thu) by mpr22 (subscriber, #60784)
In reply to: Evolution of shells in Linux (developerWorks) by HelloWorld
Parent article: Evolution of shells in Linux (developerWorks)

Um, you're not citing a deficiency in the shell. You're citing deficiencies in the programs made available to it.

Evolution of shells in Linux (developerWorks)

Posted Dec 8, 2011 10:51 UTC (Thu) by HelloWorld (guest, #56129) [Link] (11 responses)

Yes in the sense that this problem can't be overcome by modifying the shell itself. No in the sense that the lack of some structured data format beyond byte streams is clearly a flaw in the original design of UNIX and its shell.

Evolution of shells in Linux (developerWorks)

Posted Dec 8, 2011 18:09 UTC (Thu) by niner (subscriber, #26151) [Link] (10 responses)

You could also argue in the other direction. aptitude simply does not adhere to the convention set by the shell. The shell and scripting tools want line based data while aptitude prints multi line records. In such a situation, PS would not help. A correct example would be PS mandating structured output while a certain program would stuff all it's output into a single string field.

Having other conventions does not help magically. It's adherence to such conventions that makes a system work.

Evolution of shells in Linux (developerWorks)

Posted Dec 8, 2011 20:29 UTC (Thu) by HelloWorld (guest, #56129) [Link] (9 responses)

> You could also argue in the other direction. aptitude simply does not adhere to the convention set by the shell. The shell and scripting tools want line based data while aptitude prints multi line records.
Even if aptitude would print one line per package (which, given the amount of information to be printed, would result in unreadable garbage), it still wouldn't work. There's no convention about which character encoding to use (which is why pipes really aren't text streams but merely byte streams), there's no convention about the separator character and there are no escaping rules, it's all just a huge clusterfuck.

> Having other conventions does not help magically. It's adherence to such conventions that makes a system work.
It's not about other conventions, it's about having conventions *at all*, and UNIX doesn't. Tools like cut(1) and awk(1) allow you to set the input record separator precisely because of that: there is no convention.

By the way, there's another, related problem in Unix shell design: the shell does the globbing, but it doesn't interpret the arguments, as that is left to the program being executed. But the program has no way to know whether an argument was produced by globbing or not, which gives you trouble when you have files named --foo around. It's all so broken...

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 2:30 UTC (Fri) by vonbrand (subscriber, #4458) [Link] (8 responses)

It's not about other conventions, it's about having conventions *at all*, and UNIX doesn't. Tools like cut(1) and awk(1) allow you to set the input record separator precisely because of that: there is no convention.

Right. Because you never have to sort stuff separated by anything but TAB. BTW, you can sort version-wise by sorting numerically and separating on '.'

By the way, there's another, related problem in Unix shell design: the shell does the globbing, but it doesn't interpret the arguments, as that is left to the program being executed. But the program has no way to know whether an argument was produced by globbing or not, which gives you trouble when you have files named --foo around. It's all so broken...

That is exactly the reason why many programs interpret "--" as "end of switches, rest is arguments." Besides, I have had the dubious pleasure of working with systems where each program was in charge of interpreting globs, and not too surprisingly each one did it their own way (or not at all), for all around confusion.

This is a command interpreter for interactive use we are talking about here, for crying out loud. That you can do a surpising amount of programming in it is a welcome bonus, so you don't have to go grab a full-fledged programming language all the time (in many cases you wouldn't bother and do it by hand or just give up). And most of the "problems" mentioned are just that the programs involved aren't built for easy chaining (can't ask for one-line records, hard to parse output, ...), not some "Unix failure" per se.

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 8:56 UTC (Fri) by HelloWorld (guest, #56129) [Link] (7 responses)

> That is exactly the reason why many programs interpret "--" as "end of switches, rest is arguments."
I know, but nobody ever thinks of using that in practice, making it useless in practice. Not to mention that there are also many tools that don't support this.

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 10:24 UTC (Fri) by nicku (guest, #777) [Link] (6 responses)

> That is exactly the reason why many programs interpret "--" as "end of switches, rest is arguments."
I know, but nobody ever thinks of using that in practice, making it useless in practice.

I don't understand why you say this. I and my workmates use this whenever required.

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 10:29 UTC (Fri) by HelloWorld (guest, #56129) [Link] (5 responses)

> I don't understand why you say this.
Because that's the way that it is.

> I and my workmates use this whenever required.
Then you're obviously more attentive than the average shell programmer.

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 10:33 UTC (Fri) by dlang (guest, #313) [Link]

how large a study did you make to define that the 'average shell programmer' doesn't know this?

and how do you define 'shell programmer'

I would say that people who are making their living on a team working with the shell (sysadmin types) have a pretty good chance of knowing this, but people who are tinkering and learning alone are less likely to have learned this, but such people are far more likely to have gaps of all kinds in their knowledge.

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 12:10 UTC (Fri) by mpr22 (subscriber, #60784) [Link] (3 responses)

I think it's safe to say that for any widely-used programming language, there is a sizeable group you can more or less reasonably describe as being "average $LANGUAGE programmers" who are not very good at adhering to best practice. So, saying "the average $LANGUAGE programmer doesn't do $BEST_PRACTICE_ITEM" seems likely to be true, obvious, and unenlightening.

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 12:32 UTC (Fri) by HelloWorld (guest, #56129) [Link] (2 responses)

The question is what consequences to draw from it. And the answer is obvious: solve the problem some other way so that the best practice isn't needed any longer. What this boils down to is that shell programming should be avoided wherever possible.

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 13:06 UTC (Fri) by dlang (guest, #313) [Link] (1 responses)

by that logic, no languange should ever be used for programming

people keep trying to produce things that can be extended to do things that weren't initially programmed in via config files of various kinds, but evenutally every one of these config files grows into (or adopts) some sort of programming language.

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 13:23 UTC (Fri) by HelloWorld (guest, #56129) [Link]

> by that logic, no languange should ever be used for programming
Well, this is not a binary thing. The thing is that in order to write reliable shell scripts, you need to jump through hoops *all the time* (i. e. every time you use a glob, every time you use sed in a locale other than the one you have tested your script with etc.) in order to stop bad things from happening.