Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Posted Dec 8, 2011 9:10 UTC (Thu) by tuxmania (guest, #70024)In reply to: Evolution of shells in Linux (developerWorks) by Cyberax
Parent article: Evolution of shells in Linux (developerWorks)
I have used PS a fair bit and the thing that stands out is how much work has been put into being different for the sake of being different, not better.
If Microsoft had instead ported bash onto Windows they would have an army of good scripters. Now, MS fanboys i know just snear at PS and discard it as "that crap from Linux" and the people who would know how to handle it is put off by its stupitity. Its a fail-fail situation sadly.
Posted Dec 8, 2011 9:36 UTC (Thu)
by HelloWorld (guest, #56129)
[Link] (67 responses)
Posted Dec 8, 2011 9:54 UTC (Thu)
by tuxmania (guest, #70024)
[Link] (66 responses)
Many things is much harder to do in Powershell despite this "structured data" holy grail you talk about. Why? Because third party support for Powershell is abysmal, and even Microsoft tends to avoid it like the plague in many places, especially where there is a GPO for roughly the same thing somewhere, making it next to impossible to do what you really need to do.
Posted Dec 8, 2011 10:13 UTC (Thu)
by HelloWorld (guest, #56129)
[Link] (65 responses)
Posted Dec 8, 2011 10:43 UTC (Thu)
by mpr22 (subscriber, #60784)
[Link] (12 responses)
Posted Dec 8, 2011 10:51 UTC (Thu)
by HelloWorld (guest, #56129)
[Link] (11 responses)
Posted Dec 8, 2011 18:09 UTC (Thu)
by niner (subscriber, #26151)
[Link] (10 responses)
Having other conventions does not help magically. It's adherence to such conventions that makes a system work.
Posted Dec 8, 2011 20:29 UTC (Thu)
by HelloWorld (guest, #56129)
[Link] (9 responses)
> Having other conventions does not help magically. It's adherence to such conventions that makes a system work.
By the way, there's another, related problem in Unix shell design: the shell does the globbing, but it doesn't interpret the arguments, as that is left to the program being executed. But the program has no way to know whether an argument was produced by globbing or not, which gives you trouble when you have files named --foo around. It's all so broken...
Posted Dec 9, 2011 2:30 UTC (Fri)
by vonbrand (subscriber, #4458)
[Link] (8 responses)
Right. Because you never have to sort stuff separated by anything but TAB. BTW, you can sort version-wise by sorting numerically and separating on '.' That is exactly the reason why many programs interpret "--" as "end of switches, rest is arguments." Besides, I have had the dubious pleasure of working with systems where each program was in charge of interpreting globs, and not too surprisingly each one did it their own way (or not at all), for all around confusion.
This is a command interpreter for interactive use we are talking about here, for crying out loud. That you can do a surpising amount of programming in it is a welcome bonus, so you don't have to go grab a full-fledged programming language all the time (in many cases you wouldn't bother and do it by hand or just give up). And most of the "problems" mentioned are just that the programs involved aren't built for easy chaining (can't ask for one-line records, hard to parse output, ...), not some "Unix failure" per se.
Posted Dec 9, 2011 8:56 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (7 responses)
Posted Dec 9, 2011 10:24 UTC (Fri)
by nicku (guest, #777)
[Link] (6 responses)
Posted Dec 9, 2011 10:29 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (5 responses)
> I and my workmates use this whenever required.
Posted Dec 9, 2011 10:33 UTC (Fri)
by dlang (guest, #313)
[Link]
and how do you define 'shell programmer'
I would say that people who are making their living on a team working with the shell (sysadmin types) have a pretty good chance of knowing this, but people who are tinkering and learning alone are less likely to have learned this, but such people are far more likely to have gaps of all kinds in their knowledge.
Posted Dec 9, 2011 12:10 UTC (Fri)
by mpr22 (subscriber, #60784)
[Link] (3 responses)
Posted Dec 9, 2011 12:32 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (2 responses)
Posted Dec 9, 2011 13:06 UTC (Fri)
by dlang (guest, #313)
[Link] (1 responses)
people keep trying to produce things that can be extended to do things that weren't initially programmed in via config files of various kinds, but evenutally every one of these config files grows into (or adopts) some sort of programming language.
Posted Dec 9, 2011 13:23 UTC (Fri)
by HelloWorld (guest, #56129)
[Link]
Posted Dec 8, 2011 10:53 UTC (Thu)
by lkundrak (subscriber, #43452)
[Link] (1 responses)
Thank you.
Posted Dec 8, 2011 11:03 UTC (Thu)
by HelloWorld (guest, #56129)
[Link]
Posted Dec 8, 2011 13:45 UTC (Thu)
by cortana (subscriber, #24596)
[Link] (30 responses)
...
I wrote that paragraph before I noticed that I already have a sort-dctrl program on my system; it's part of dctrl-tools. Unfortunately it doesn't actually work with aptitude's output; aptitude doesn't output deb822 data... it doesn't use a . to represent blank lines in its fields. :)
Anyway, I actually agree that PowerShell is pretty cool. I find it utterly unusable, however, because it's still welded to the utterly dreadful excuse for a terminal emulator that is conhost.exe. In 2011, essential features such as multiple tabs, run-based selection and the ability to click URLs are entirely absent, and the features that do exist, such as resizing windows, block based selection and changing the font/colours are buried behind a UI that seems to be actively designed to get in my way, making me want to nuke the computer and install Debian rather than be forced to use them for even one minute longer.
Posted Dec 8, 2011 14:54 UTC (Thu)
by HelloWorld (guest, #56129)
[Link] (27 responses)
> I wrote that paragraph before I noticed that I already have a sort-dctrl program on my system; it's part of dctrl-tools. Unfortunately it doesn't actually work with aptitude's output; aptitude doesn't output deb822 data... it doesn't use a . to represent blank lines in its fields. :)
> Anyway, I actually agree that PowerShell is pretty cool. I find it utterly unusable, however, because it's still welded to the utterly dreadful excuse for a terminal emulator that is conhost.exe.
Posted Dec 8, 2011 15:35 UTC (Thu)
by cortana (subscriber, #24596)
[Link] (10 responses)
I'm not saying that PowerShell isn't terribly clever and all that, but I don't see every tool I use on a daily basis being rewritten to output its data in some standard structured format, and consume data in that format. As a result, actually using Powershell is annoying because it comes with some nice built-in stuff, but not enough third-party programs bother to make use of it. At best you end up with a bunch of scripts that parse the structured output of other programs and turn them into whatever it is that PowerShell uses internally so that Sort-Object & co. will deal with them. At that point you may as well just use a programming language that provides a decent selection of data structures in the first place (e.g., python, perl), with a module that gives you the data you want in such a structure.
Finally, and to digress, thanks for the link to Console2. Everyone who uses Windows should have it installed. Although it makes using the command-line on Windows a much less frustrating experience, it doesn't do all of what I want and sadly introduces its own quirks into the mix. It's funny you say that the lack of a standard structured data format is a fundamental problem with Unix; after many years of being trying to various conhost replacements, I've come to the conclusion that there is in fact something terribly wrong within some fundamental layer of Windows itself that makes it impossible to implement a decent terminal emulator. If that were not the case then surely someone would have done so by now; the awfulness of the command prompt on Windows is not so much an itch that needs scratching as a foot-long needle through the eye that inflicts a constant searing pain upon all users of the platform. Whereas I think the problem with a standard for structured data is that there are already plenty of standards to choose from; line-based character-separated or deb822 if you merely want a flat list of data with attributes; JSON or XML if you want something that can represent a tree of data, to name but a few. The power of Unix's use of unstructured data is that you are able to pick and choose with of these formats, plus many others, you can use when glueing together other bits of software.
Posted Dec 8, 2011 16:07 UTC (Thu)
by halla (subscriber, #14185)
[Link]
Posted Dec 8, 2011 16:47 UTC (Thu)
by nix (subscriber, #2304)
[Link]
Posted Dec 8, 2011 19:16 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (4 responses)
Standard _console_ _emulator_ on Windows definitely sucks, that's true. But the console layer itself is quite robust and easy to use.
For example, there is no freaking way in Unix consoles to detect if a modifier key is pressed. Then there's the fact that Unix consoles are in reality a result of adding features to something that initially had been a line printer output.
As a result, making applications with good text UI in console is close to impossible. That's why Midnight Commander is such a buggy POS. In fact, Unix consoles are so bad that tmux/screen had to implement console emulators working on top of console emulators because it's impossible to reliably track the state of a console.
I really really love one good Windows console application - FAR Manager. It's far ahead of everything else on the planet for console-style work.
Posted Dec 8, 2011 21:19 UTC (Thu)
by quotemstr (subscriber, #45331)
[Link] (3 responses)
Are you on the payroll, or are you just completely unacquainted history and actual programming?
The Windows console layer has no pseudoterminals. That's why there are a dozen remote-command-invocation facilities in Windows, and it's why none of them works right. It's why Console2 fails badly when talking to some programs (e.g., cmd) --- Windows programs are forced to use pipes instead of consoles, so programs run under a console other than conhost itself don't even know they're being used interactively, resulting in various horrible issues the Unix world fixed 20 years ago.
Posted Dec 8, 2011 21:33 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (2 responses)
>The Windows console layer has no pseudoterminals.
It would have been OK, but terminal emulators suffer from terminal schizophrenia. They try to act as if they are line printers while at the same time trying to act as if they are driving a VT100 console. Which fails badly.
In Windows consoles are just consoles. I.e. a screen buffer on which you can draw text - and that's basically all. You can do whatever you want with it.
Instead of pseudoterms you can create child consoles (including invisible ones) and use them to start other programs. I'm not sure why console2 doesn't do it.
Posted Dec 8, 2011 21:44 UTC (Thu)
by quotemstr (subscriber, #45331)
[Link] (1 responses)
Consider using the VTE widget from GNOME: http://developer.gnome.org/vte/0.30/
> It's essentially a channel to transmit bytecode which is executed by terminal emulator to draw text.
Yes, the unix terminal system uses in-band signaling. In-band signaling has advantages and disadvantages with respect to out-of-band signaling, but in the case of the tty layer, the advantages outweigh the disadvantages, which have all been overcome. The chief advantage is being to use a terminal-agnostic channel (like a TCP connection) over which a terminal-program and a terminal-emulator can communicate. The Windows world has no equivalent because its console uses opaque out-of-band signaling.
> Which fails badly.
No, the terminal system actually works very, very well in practice.
> Instead of pseudoterms you can create child consoles (including invisible ones) and use them to start other programs. I'm not sure why console2 doesn't do it.
The reason console2 doesn't use the technique you describe is that it doesn't work. Yes, you can create "child consoles" and associate processes with them, but there's no "master side" interface to these child consoles: there is no way to extract text from them, to see what your child programs are attempting to output and output it yourself in a controlled manner. Yes, you can scrape the console, but the scraping process has inherent race conditions. (And scraping is messy besides: you're complaining about terminal control codes?)
Posted Dec 8, 2011 21:54 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
I want it to work in plaing text consoles (over SSH). There are more than enough GUI file managers. Right now I'm trying to layer it over tmux, and I'm slowly making progress.
>Yes, the unix terminal system uses in-band signaling. In-band signaling has advantages and disadvantages with respect to out-of-band signaling, but in the case of the tty layer, the advantages outweigh the disadvantages, which have all been overcome.
I don't mind the idea of transmitting bytecode for drawing. I just don't like the way it works right now. It's messy, ugly, buggy and so on.
Have you ever tried to read the code of terminal emulators? Nightmares of escape sequences still plague my dreams :)
>The reason console2 doesn't use the technique you describe is that it doesn't work. Yes, you can create "child consoles" and associate processes with them, but there's no "master side" interface to these child consoles: there is no way to extract text from them, to see what your child programs are attempting to output and output it yourself in a controlled manner. Yes, you can scrape the console, but the scraping process has inherent race conditions. (And scraping is messy besides: you're complaining about terminal control codes?)
You can associate a child console with parent which would make it visible. But yes, I guess it's one week spot. As far as I remember, conman (console manager) worked around it by intercepting console layer calls by injecting a DLL into address space of child processes.
If I were to design a console layer right now, then I'd have have chosen Windows model with explicitly defined marshalling protocol.
Posted Dec 8, 2011 20:50 UTC (Thu)
by HelloWorld (guest, #56129)
[Link] (2 responses)
Posted Dec 8, 2011 22:15 UTC (Thu)
by ebiederm (subscriber, #35028)
[Link] (1 responses)
Posted Dec 9, 2011 12:23 UTC (Fri)
by nix (subscriber, #2304)
[Link]
Posted Dec 8, 2011 23:13 UTC (Thu)
by GhePeU (subscriber, #56133)
[Link] (15 responses)
I don't find 'ps -Ao user,pid,ppid,time,command | sort -k3n' particularly messy.
Posted Dec 9, 2011 9:06 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (14 responses)
Posted Dec 9, 2011 9:28 UTC (Fri)
by khim (subscriber, #9252)
[Link] (13 responses)
Not if your programs continue to produce "untyped byte streams". PowerShell is classic example of well-known problem. With bash you need to know how to handle mess which existing programs produce and consume. With PowerShell you still need to know that (see lots of threads above about how to handle it - it's as ugly as in bash if not uglier), but you also have thus nice, user-friendly way which does not work all that often, but presented as the solution "for the future". Just like all other "grand unification schemes" before and after it'll fail so to say that PowerShell is better then bash is crazytalk: it's strictly worse and this is very easy to prove... PowerShell must die - but I'm content because it'll do just that anyway. It'll take a long time, but now, when madness around JVM and CLR is slowly dying because people understand that they were duped it's only matter of time. It'll be interesting to see if Microsoft will survive destruction of it's .NET dream or if it'll die off with too. I'd be relighted to see that, but don't hold my breath: Microsoft still looks way too resilient for that. Albeit lately it does so many stupid things die-off is actually likely. We'll see.
Posted Dec 10, 2011 2:52 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link] (12 responses)
PowerShell _is_ a unified language which interoperates with all other .NET languages. It does allow to transfer structured data in a unified way. UNIX people should sometimes leave their self-righteousness at the door and actually check what other people are doing.
PowerShell is definitely more powerful than bash or zsh because IT IS BUILT ON A GOOD FOUNDATION. It's that simple.
Bash knows nothing but byte streams. That's nice - for 1973. You can construct pipes in bash and then try use them to do something. But:
Posted Dec 10, 2011 6:18 UTC (Sat)
by khim (subscriber, #9252)
[Link] (4 responses)
In other words: it's LISP Machines all over again. In a hypothetical world where everyone is using .NET it may even be a good thing. In real world where .NET rage is on decline and it's now clear that .NET will not ever take over the world... not so much. UNIX people have looked from sidelines when first coming of LISP machines have come and gone. They'll do that a second time (and third time - if that'll ever happen), they are patient guys. They'll pick some good ideas from the whole sad story, as usual, but the whole "let's redo everything from scratch" thing is just not what UNIX people plan to ever do. It's more powerful but it's more complex, too. You may as well say that emacs prompt is more powerful then bash - and you'll be right. But it only helps you if you want to want in "Emacs OS". When you need to interact with a real world it's more complex then bash or zsh because it needs to do what bash/zsh are doing and it needs to do it in a way which makes it possible to easily do things in it's own world. Sure, it's possible to use emacs as as shell replacement - but few people do. PowerShell is the same way. But unlike Emacs it's just as shell wannabe, nothing else, so I'm not sure it'll survive when all the rage about this generation of LISP Machines will die off. Long list of arguments skipped. Just note how everything you've cited was solved in "the first coming" (on LISP Machines), too. The difference is in scale: first time tens of millions were burned thus only thousands of machines were produced and the whole saga took about ten years on sidelines. Second time architecture astronauts had billions so the whole story took longer and rage was raised higher. The result is the same (one company was burned to a crisp, second one is stagnating, but is more-or-less ready to throw all these baubles out), it just took longer. When the hype will die off PowerShell will either die with .NET or it'll be used in some limited settings, while bash will live as a mainstream tool (albeit may be in split personality if Apple will continue to fear GPLv3). P.S. Note that I'm not saying .NET will die off totally: "first coming" left Emacs behind, I'm pretty sure "second coming" will leave few IDEs behind, too. Apparently that's the niche where all your vaunted advances are really helpful.
Posted Dec 10, 2011 7:18 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
Nothing had ever came close to PowerShell in functionality and usability for system shells. And no, old elisp shells or Rexx scripts ARE NOT the answer. They don't offer the two most powerful features of PowerShell: introspection and structured pipelines.
>It's more powerful but it's more complex, too. You may as well say that emacs prompt is more powerful then bash - and you'll be right. But it only helps you if you want to want in "Emacs OS". When you need to interact with a real world it's more complex then bash or zsh because it needs to do what bash/zsh are doing and it needs to do it in a way which makes it possible to easily do things in it's own world.
Sorry, but PowerShell can do ANYTHING bash can do. Without exceptions. Some things might be more clumsy than in bash - because PowerShell is not bash and things are done differently there.
And PowerShell even beats bash at its own game! PowerShell integration with git is much nicer than in bash: https://github.com/dahlbyk/posh-git - and achieves the similar functionality in a fraction of lines of code. A few screenshots can be seen here: http://markembling.info/2010/03/git-powershell-revisited
Posted Dec 10, 2011 8:41 UTC (Sat)
by khim (subscriber, #9252)
[Link] (2 responses)
They do offer introspection and as for "structured pipelines"... this is only important if you want to pretend you are replacing "shell proper". So what? Any turing-complete language with support for fork/exec (or CheateProcess on Windows) can do the same. And that's the point: PowerShell can not replace bash so bash will survive while PowerShell will die (or at least will be relegated for niche work). If you don't like LISP Machines analogue then I have another for you: Space Shuttle. This extremely expensive program was invented to replace old rockets: Atlas, Delta, Saturn, etc. Just like PowerShell it promised the world and delivered very little¹. After all the hype and all the expenses it was only able to replace few of them - the ones which were forcibly killed to "open the road for the future". Today Space Shuttle is history and in it's place is huge glaring void - while people who were sensible enough to continue to use "old school" tools (Delta, Proton, Soyuz, etc) are still around and doing what Space Shuttle was supposed to do. It does not matter how many lines of code it requires. The same story as with Space Shuttle: sunk costs. When people try to justify Space Shuttle craze they compare development costs of Saturn V² and Space Shuttle - but this is completely wrong: it was not possible to get money spent on Saturn V back, thus you need to compare ongoing costs of Saturn V with new development costs of Space Shuttle - and there are no comparison. CMD/Bash is here, it's not goes away so program will need to support it anyhow. This means all additional developments (things like cmdlets) should be counted extra. ¹) Well, one nice property of this stupidity was the fact that they managed to convince USSR to waste a lot of resources for the mirror project - and if this was the goal then the project can be considered success. ²) Why Saturn? Well, alternative for Space Shuttle idiocy was continuation of Saturn program so it's only natural to compare Space Shuttle to Saturn.
Posted Dec 10, 2011 13:15 UTC (Sat)
by dlang (guest, #313)
[Link]
Posted Dec 11, 2011 1:03 UTC (Sun)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Please show me how I can tab-complete SQL queries in any elisp shell. I'd like to be able to at least complete all table names in all contexts.
PowerShell is freakingly powerful in that regard - I already use it instead of psql shell to work with Postgres databases. Mostly because autocompletion in posh based on simple introspection is miles ahead of what's present in psql.
And that says something.
>If you don't like LISP Machines analogue then I have another for you: Space Shuttle
Yup. It's funny because bash is very much like Space Shuttle: it's old, it's expensive to use (bash hell scripts are not easy to write), it's prone to crashes, etc. The only reason it's used is because of a sunken costs of tons of hell scripts.
Posted Dec 11, 2011 0:11 UTC (Sun)
by nix (subscriber, #2304)
[Link] (6 responses)
Posted Dec 11, 2011 5:36 UTC (Sun)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
PowerShell allows to embed documentation directly into objects and also structure it by parameters, methods, etc.
And there's nice "-online" switch that leads you directly into the TechNet article associated with tool, with Q&A and other additional functionality.
Can you make your scripts in your home directory have their documentation automatically be included and made searchable into the central help system?
>I'd rather use man than HTML Help with its abysmal searching, insulting baby talk, and horrendous security holes.
You're in luck because PowerShell doesn't use HTML Help :)
>Which would be why POSIX, uh, standardized them more than ten years ago, and imposed rules which virtually all standard tools follow.
Yeah, sure.
> dd if=... of=...
Then there are --argument=blah and "--argument blah" forms which don't always work. And then there are short forms which sometimes require argument to be written immediately without intervening spaces.
In PowerShell _everything_ is standardized _and_ autocompleteable. I can do things like "my-command -p<tab>" and get the list of parameters (with description and default values!) starting with 'p'. This is sort of possible in bash/zsh with cooperating tools, but in PowerShell it's all completely automatic.
And since PowerShell is a static language with type inference, it'll warn me if I write stuff like this "destroy-hard-drive -in hello -seconds" (because 'hello' is not an integer).
>You carefully named the two shells which have extremely extensive autocompletion, in zsh's case shipped with the package
I know perfectly well how bash/zsh autocompletion work.
>Yes, it's not 100% automatable without a bit of per-tool scripting. The workload is minimal compared to writing the tools. Perhaps in an ideal world it could be completely automated, but that just shifts the burden from writing the autocompletion code to writing some sort of reflective description of the system. Big deal.
And in PowerShell I get autocompletion basically for free. And it's good. It's VERY good. I'm actually using PowerShell instead of Postgres's psql because PowerShell is much more powerful.
I can autocomplete table names while writing SQL queries. In command line.
This is not really possible in bash/zsh because text matching games only can lead you so far. Even autocompletion for scp in bash is already straining things.
I wish people would sometimes go out and see what's happening outside of the Linux/Unix world. It's not a wonder that the most popular Linux distribution is actually barely a Unix system.
Posted Dec 11, 2011 11:07 UTC (Sun)
by nix (subscriber, #2304)
[Link] (1 responses)
Most of the rest of what you say is a combination of ignorance of what the Unix tools you discuss can actually do, and complaints that things are not acceptable because they're not just like PowerShell does them. We get that you like it, but we've all been through this parochial 'the newest system I just saw is the answer to everyone's prayers' phase, and, y'know? It's always wrong. There are limitations there: you're just not seeing them.
Posted Dec 11, 2011 11:50 UTC (Sun)
by Cyberax (✭ supporter ✭, #52523)
[Link]
I'm claiming that it's not a GOOD system.
>Ah. Like POD, doxygen, and similar systems, all of which can generate manpage output. i.e., dead heat here.
So... How do I make documentation for my bash script (with all its options), make a man page, link it into the central system and all of it without doing anything more than simply declaring options?
And no, doxygen won't help you - it doesn't support bash (I'd actually tried to find an automatic documentation system for bash some time last year - there was none).
>That's great -- if and only if Microsoft wrote the tool. Only useful in a software monoculture.
That's easily adapted if tools' authors provide their own URLs (which they can do in PowerShell - VMWare has its own doc system, for example).
>Yes. Learn about apropos databases and MANPATH. Manpages can be stored absolutely anywhere (as can info pages).
I know perfectly well how most of Linux tools work. I know that one can maintain their own local man databases. And I also know perfectly well that almost nobody does, mostly because it's complicated and error-prone.
>Most of the rest of what you say is a combination of ignorance of what the Unix tools you discuss can actually do, and complaints that things are not acceptable because they're not just like PowerShell does them.
I know most of the standard Unix tools. I've built my own custom distributions from scratch (first time without the benefit of the LFS book) and support a network of embedded devices. I've been using Linux on my desktops since 90-s and can recollect all the steps that have been taken to make Linux to be at least possible to use on desktop.
A lot of these steps involved bashing at least some old Unix-heads with spiked hammers: udev, HAL, KMS, dbus to name a few. Oh, and the whole 'Android' thingie. Now the same thing repeats with pulseaudio, systemd and journald.
>We get that you like it, but we've all been through this parochial 'the newest system I just saw is the answer to everyone's prayers' phase, and, y'know? It's always wrong. There are limitations there: you're just not seeing them.
Some things do solve all the (existing) problems. Because they are designed to solve them.
PowerShell is one such example. It's designed to be a better shell than text-based shells and it excels at it. It's not yet as polished as bash/zsh but it's getting better with each new release.
Of course, PowerShell has limitations and a set of new problems, but so does bash/zsh. And limitations of bash/zsh are MUCH more constricting.
Posted Dec 16, 2011 5:11 UTC (Fri)
by tom.prince (guest, #70680)
[Link]
>And in PowerShell I get autocompletion basically for free. And it's good. It's VERY good. I'm actually using PowerShell instead of Postgres's psql because PowerShell is much more powerful.
You only get this for free, if your software is designed to work with PowerShell, otherwise you are in the same boat as with zsh/bash.
There is at least one framework (twisted-python) that automatically generates zsh completions from the description of options.
Posted Dec 11, 2011 16:21 UTC (Sun)
by anselm (subscriber, #2796)
[Link] (1 responses)
Some of the most basic aspects were indeed standardised by POSIX, but that doesn't detract from the fact that, say, the option to specify a field delimiter is »-d« for cut(1), »-t« for sort(1), »-F« for awk(1), and so on. POSIX basically codified the wild hodgepodge that existed at the time.
Now the meaning of common options (as opposed to basic option syntax) would have been something actually worth standardising, but of course it would have rendered 20 years' worth of shell scripts virtually useless, so it didn't happen.
Posted Dec 11, 2011 17:03 UTC (Sun)
by raven667 (subscriber, #5198)
[Link]
Posted Dec 9, 2011 0:06 UTC (Fri)
by grg (guest, #76756)
[Link] (1 responses)
Posted Dec 9, 2011 2:44 UTC (Fri)
by cortana (subscriber, #24596)
[Link]
Posted Dec 9, 2011 4:26 UTC (Fri)
by jthill (subscriber, #56558)
[Link] (18 responses)
It's a one-liner.
Posted Dec 9, 2011 5:09 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (17 responses)
Posted Dec 9, 2011 5:26 UTC (Fri)
by jthill (subscriber, #56558)
[Link] (16 responses)
It's a perfectly reasonable one-liner.
Posted Dec 9, 2011 8:53 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (15 responses)
Posted Dec 9, 2011 20:29 UTC (Fri)
by jthill (subscriber, #56558)
[Link] (14 responses)
> OK, I admit I didn't know about the -k switch
But before you take it there, you might want to spend more than the few seconds you've apparently spent on the attempt.
Posted Dec 9, 2011 20:39 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (13 responses)
Posted Dec 9, 2011 21:27 UTC (Fri)
by jthill (subscriber, #56558)
[Link] (12 responses)
Posted Dec 9, 2011 21:40 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (1 responses)
Well, I probably shouldn't be wasting my time with a bloody stupid troll like you.
Posted Dec 9, 2011 22:26 UTC (Fri)
by corbet (editor, #1)
[Link]
Posted Dec 9, 2011 22:10 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (9 responses)
Posted Dec 9, 2011 23:10 UTC (Fri)
by jthill (subscriber, #56558)
[Link] (8 responses)
Posted Dec 9, 2011 23:33 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (7 responses)
Posted Dec 10, 2011 2:19 UTC (Sat)
by nybble41 (subscriber, #55106)
[Link] (6 responses)
Posted Dec 11, 2011 10:39 UTC (Sun)
by HelloWorld (guest, #56129)
[Link] (5 responses)
Posted Dec 11, 2011 22:48 UTC (Sun)
by nybble41 (subscriber, #55106)
[Link] (4 responses)
An aptitude-equivalent under PowerShell would need to export the ability to compare package objects by version for any third-party sort utility to work with the data anyway, which is 90% of the way toward just implementing the -O option internally.
"Structured data" is nearly always the first thing anyone tries when implementing a new system. A number of older systems don't support anything *but* structured data for disk files, for example--I've worked on operating systems where even plain text files were stored, one row per line, as tables in a database. The fact that all modern systems actually treat disk files and most inter-process communications as plain byte streams should tell you something. Standardized data *formats*, encoded as plain byte streams, are more flexible and resilient than standardized APIs, as used by PowerShell.
Posted Dec 12, 2011 5:58 UTC (Mon)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
But it does. You can easily sort any data on any field even using your own custom comparator (try that with 'sort'!), using only one utility. Which had to be written, just as 'sort' utility in Unix.
>You could get the same effect by standardizing the labeled-groups-of-lines format output by 'aptitude show' and writing a sort utility specialized on that format (a 10-minute job in any major scripting language). Or you could just use one of the many portable interactive scripting language shells, like python or irb or scsh, which provide the same services for their respective bindings that PowerShell provides for .NET.
Well, of course you can achieve the same functionality as in PowerShell by replicating what PowerShell does (namely, by introducing structured data). The point is - no one is really doing it in the Unix world.
Posted Dec 12, 2011 6:25 UTC (Mon)
by nybble41 (subscriber, #55106)
[Link] (2 responses)
> But it does. You can easily sort any data on any field even using your own custom comparator (try that with 'sort'!), using only one utility. Which had to be written, just as 'sort' utility in Unix.
The point was that the 'sort' utility used in PowerShell only really adds anything over the standard Unix sort command if the source application includes bindings for .NET.[1] A similar utility could be written for Unix with the same features, if anyone was actually interested. It's not exactly a difficult problem, even including fancy features like custom comparators. The critical part is a runtime-specific binding for the source application. If aptitude had bindings for Python, Ruby, or Scheme you could do exactly the same thing from these shells under Unix today. It wouldn't require any more work than implementing the .NET bindings necessary to make aptitude work with PowerShell (assuming they worked on the same platform to begin with). In fact, if aptitude had a DBUS interface, as many other application s do, it would automatically with with all these Unix shells and more.
> Well, of course you can achieve the same functionality as in PowerShell by replicating what PowerShell does (namely, by introducing structured data). The point is - no one is really doing it in the Unix world.
For good reason, as I said--such systems have never won out in the past, why should .NET and PowerShell be any different? Anyway, my suggestion wasn't to replicate what PowerShell does, it was to implement a record-oriented sort utility within the existing Unix byte-stream framework, using a standard data encoding rather than opaque "structured data" in a particular runtime. Historically, forcing everything to work within a single complex runtime or RPC model has not been a winning strategy. Simple, portable, reliable interfaces as best.
----
[1] Purely out of curiosity, how does this kind of sort command work if the data is coming from a file, without the .NET APIs, rather than directly from an application? Is it even possible to get the same effect as redirecting the output of a command to a file before sorting it, or must the source and sort command be directly coupled? It seems like the extra features must get lost in translation, since only the textual representation will be saved on disk.
Posted Dec 12, 2011 8:59 UTC (Mon)
by Cyberax (✭ supporter ✭, #52523)
[Link]
The problem is, you have to write a special-case utility. While PowerShell provides you a generic mechanism for this. It provides a robust platform that other systems can use.
And some features of sort are quite difficult to replicate in bash, for example, custom comparators (I believe, the only way to do this is to create string ordinals suitable for the 'sort' utility).
You've mentioned D-BUS - it's actually quite similar to PowerShell in a lot of regards. It provided a platform that other applications could easily use and adapt (instead of inventing their own ad-hoc remoting). And so application quickly adopted it. The same is happening with PowerShell in Windows.
>For good reason, as I said--such systems have never won out in the past, why should .NET and PowerShell be any different?
Well, it is different because it's already adopted in the Windows world and it's clear that PowerShell is here to stay.
And I think the main reason for adoption is that the time has come for it. All the ingredients for PowerShell were finally in place - even twelve years ago it would have been impossible without the common environment that .NET provides (PowerShell based on COM? Shudder).
The same story with DBUS - there quite a few failed attempts to build a common desktop RPC system in Unix: DCE RPC, DCOP, CORBA in GNOME and I probably forgot quite a few others. Yet DBUS has clearly won and is now ubiquitous. We even have projects for kernel-accelerated DBUS.
>[1] Purely out of curiosity, how does this kind of sort command work if the data is coming from a file, without the .NET APIs, rather than directly from an application?
That depends on what you're trying to do. You surely can dump objects' content in XML/JSON/binary but it's rarely useful in practice. More often you'd use FT command (Format-Table) to create textual representation that fits your purposes. For objects like files or list of files you can dump them as simple filenames. PowerShell is quite flexible in that regard.
Posted Dec 12, 2011 18:59 UTC (Mon)
by raven667 (subscriber, #5198)
[Link]
What is this "winning" you are concerned about? PowerShell is already the preferred standard command shell and scripting environment on MS Windows servers and is not really relevant outside that space. UNIX shells like bash are really only relevant on UNIX systems, they have only marginal presence in the Windows space, so there is really no context where "winning" makes sense as they are not directly comparable.
The idea that the shell can be both built out of and expose a full featured programming language and library is not new as you point out but is well demonstrated by PowerShell and not a bad idea.
Evolution of shells in Linux (developerWorks)
Yeah right, except that it's unlike all bourne-style shells out there. Its pipes exchange structured data, and not just an unstructured byte stream. This is about as revolutionary as it gets as far as shells are concerned.
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
You must be a pretty unheeding person then. For example, the sort(1) utility is almost useless because pipes are just byte streams. Try sorting the output of, say, "aptitude show '~i'" by version. You can't, because "aptitude show" doesn't output one package per line (as that would be too long) and even if it did, you can't tell sort how to compare (i. e. by version instead of lexicographically).
Sorry, but UNIX shells are simply broken by design. The fact that they can be used to do useful things occasionally doesn't change that.
Um, you're not citing a deficiency in the shell. You're citing deficiencies in the programs made available to it.
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Even if aptitude would print one line per package (which, given the amount of information to be printed, would result in unreadable garbage), it still wouldn't work. There's no convention about which character encoding to use (which is why pipes really aren't text streams but merely byte streams), there's no convention about the separator character and there are no escaping rules, it's all just a huge clusterfuck.
It's not about other conventions, it's about having conventions *at all*, and UNIX doesn't. Tools like cut(1) and awk(1) allow you to set the input record separator precisely because of that: there is no convention.
Evolution of shells in Linux (developerWorks)
It's not about other conventions, it's about having conventions *at all*, and UNIX doesn't. Tools like cut(1) and awk(1) allow you to set the input record separator precisely because of that: there is no convention.
By the way, there's another, related problem in Unix shell design: the shell does the globbing, but it doesn't interpret the arguments, as that is left to the program being executed. But the program has no way to know whether an argument was produced by globbing or not, which gives you trouble when you have files named --foo around. It's all so broken...
Evolution of shells in Linux (developerWorks)
I know, but nobody ever thinks of using that in practice, making it useless in practice. Not to mention that there are also many tools that don't support this.
Evolution of shells in Linux (developerWorks)
I don't understand why you say this. I and my workmates use this whenever required.
> That is exactly the reason why many programs interpret "--" as "end of switches, rest is arguments."
I know, but nobody ever thinks of using that in practice, making it useless in practice.Evolution of shells in Linux (developerWorks)
Because that's the way that it is.
Then you're obviously more attentive than the average shell programmer.
Evolution of shells in Linux (developerWorks)
I think it's safe to say that for any widely-used programming language, there is a sizeable group you can more or less reasonably describe as being "average $LANGUAGE programmers" who are not very good at adhering to best practice. So, saying "the average $LANGUAGE programmer doesn't do $BEST_PRACTICE_ITEM" seems likely to be true, obvious, and unenlightening.
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Well, this is not a binary thing. The thing is that in order to write reliable shell scripts, you need to jump through hoops *all the time* (i. e. every time you use a glob, every time you use sed in a locale other than the one you have tested your script with etc.) in order to stop bad things from happening.
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Yes, and that is precisely what makes it useless. And actually, it can't even sort lines properly; try sorting, say, the output of ps by ppid or something. Yeah, it's doable, but it's messy, and it needn't be.
That pretty much proves the point. Now we already have two sorting utilities that are useless as they can't sort what I want them two. Contrast with PowerShell's Sort-Object, which makes it trivial to sort processes listed by Get-Process.
The terminal emulator can easily be replaced, for example with this one:
http://sourceforge.net/projects/console/
On the other hand, a fundamental problem like the lack of some kind of structured data exchange format can't be solved that easily.
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
the awfulness of the command prompt on Windows is not so much an itch that needs scratching as a foot-long needle through the eye that inflicts a constant searing pain upon all users of the platform
Now that's a QoTW candidate if ever I've seen one.
Evolution of shells in Linux (developerWorks)
Heh. I have COMPLETELY opposite experience. Console on Linux sucks and can't be fixed by any means short of going back in time and shooting K&R before they start writing the first line of Unix code.
</flamebait mode>
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Which is good. Because pseudoterminals are a braindead idea. It's essentially a channel to transmit bytecode which is executed by terminal emulator to draw text.
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Yes, and the only reason it's so limited is that the Unix shell paradigm doesn't allow anything better. Which was the point all along. Is that so hard to understand?
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Sorry, but this is wrong. Dead wrong.
That isn't necessary if there's a standard mechanism for structured data.
Sorry, but this is wrong. Dead wrong.
1) There is no static typing or checks. Everything is dynamic which combines nicely with...
2) No real way to unit-test your scripts.
3) No central documentation or help system. And no, 'man' isn't it. Scripts and utilities have all kinds of argument formats without any real convention. Which leads to...
4) No discoverability. Autocompletion is not sanely possible in bash or zsh.
5) Abysmal tooling. The only real tool is just a text editor. Continuous integration? Unit testing? Static analysis? Code coverage analysis? Who needs them, not us!
Keep dreaming...
PowerShell _is_ a unified language which interoperates with all other .NET languages.
UNIX people should sometimes leave their self-righteousness at the door and actually check what other people are doing.
PowerShell is definitely more powerful than bash or zsh because IT IS BUILT ON A GOOD FOUNDATION.
Bash knows nothing but byte streams. That's nice - for 1973.
Keep dreaming...
It does not matter...
Nothing had ever came close to PowerShell in functionality and usability for system shells. And no, old elisp shells or Rexx scripts ARE NOT the answer. They don't offer the two most powerful features of PowerShell: introspection and structured pipelines.
Sorry, but PowerShell can do ANYTHING bash can do. Without exceptions.
Some things might be more clumsy than in bash - because PowerShell is not bash and things are done differently there.
PowerShell integration with git is much nicer than in bash: https://github.com/dahlbyk/posh-git - and achieves the similar functionality in a fraction of lines of code.
It does not matter...
It does not matter...
Sorry, but this is wrong. Dead wrong.
3) No central documentation or help system. And no, 'man' isn't it.
So... the central Unix documentation system is not acceptable because... you say so? I'd rather use man than HTML Help with its abysmal searching, insulting baby talk, and horrendous security holes.
Scripts and utilities have all kinds of argument formats without any real convention.
Which would be why POSIX, uh, standardized them more than ten years ago, and imposed rules which virtually all standard tools follow. A couple of holes exist, mostly for backward compatibility or ease-of-use's sake (e.g. tail and head's - and +-based arguments) and a couple are just deeply unusual (e.g. find and dd), but most are pretty consistent, and the ones with weird user interfaces get comprehended in the end by frequent use. I'd prefer Lisp everywhere, but, face it, it's not gonna happen.
4) No discoverability. Autocompletion is not sanely possible in bash or zsh.
You carefully named the two shells which have extremely extensive autocompletion, in zsh's case shipped with the package. I don't know what you're trying to do, but you're doing such a good job arguing against yourself that I don't see why I need bother. (Yes, it's not 100% automatable without a bit of per-tool scripting. The workload is minimal compared to writing the tools. Perhaps in an ideal world it could be completely automated, but that just shifts the burden from writing the autocompletion code to writing some sort of reflective description of the system. Big deal.)
Sorry, but this is wrong. Dead wrong.
Nope. Man's shortcomings are well known. It's basically a simple indexed document storage, without built-in searching and poorly structured.
> tar czf myfile myfile.tgz
Damn, I just deleted 'myfile' with doubly-archived myfile.tgz
Very standard.
I'll agree that man is not a very nice documentation system, but it *is* one, so your claim that it does not exist is vacuous on its face.
Sorry, but this is wrong. Dead wrong.
PowerShell allows to embed documentation directly into objects and also structure it by parameters, methods, etc.
Ah. Like POD, doxygen, and similar systems, all of which can generate manpage output. i.e., dead heat here.
And there's nice "-online" switch that leads you directly into the TechNet article associated with tool, with Q&A and other additional functionality.
That's great -- if and only if Microsoft wrote the tool. Only useful in a software monoculture.
Can you make your scripts in your home directory have their documentation automatically be included and made searchable into the central help system?
Yes. Learn about apropos databases and MANPATH. Manpages can be stored absolutely anywhere (as can info pages).
Sorry, but this is wrong. Dead wrong.
Sorry, but this is wrong. Dead wrong.
Sorry, but this is wrong. Dead wrong.
Which would be why POSIX, uh, standardized them more than ten years ago, and imposed rules which virtually all standard tools follow.
Sorry, but this is wrong. Dead wrong.
essential features such as multiple tabs, run-based selection and the ability to click URLs are entirely absent
Evolution of shells in Linux (developerWorks)
I use xterm and reject all other X based terminal emulators. Its small, fast, and has enough features for me. I don't need multiple tabs or the ability to click urls and I don't know what the hell run-based selection is supposed to be - these don't seem particularly essential.
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Take it to Linux - Newbie. This isn't the forum for spoon-feeding.
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
OK, folks (plural), that's enough. I'm sure there's a nice site for name-calling over --------> there somewhere, but please try not to do it here.
Enough
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
sed -nr '/^Package:/!{H;/^Version:/{G;h};$!d};x;s/\n/\x0/gp'|sort -V|sed 's/[^\x0]*\x0//;s/\x0/\n/g'
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
Evolution of shells in Linux (developerWorks)
For good reason, as I said--such systems have never won out in the past, why should .NET and PowerShell be any different?