Evolution of shells in Linux (developerWorks) [LWN.net]

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 23:10 UTC (Fri) by jthill (guest, #56558) [Link]


sed -nr '/^Package:/!{H;/^Version:/{G;h};$!d};x;s/\n/\x0/gp'|sort -V|sed 's/[^\x0]*\x0//;s/\x0/\n/g'

Evolution of shells in Linux (developerWorks)

Posted Dec 9, 2011 23:33 UTC (Fri) by HelloWorld (guest, #56129) [Link]

We clearly have a different idea about what a reasonable one-liner is. And it doesn't work when another locale is set.

Evolution of shells in Linux (developerWorks)

Posted Dec 10, 2011 2:19 UTC (Sat) by nybble41 (subscriber, #55106) [Link]

aptitude search -F %p -O version '~i' | xargs aptitude show

Evolution of shells in Linux (developerWorks)

Posted Dec 11, 2011 10:39 UTC (Sun) by HelloWorld (guest, #56129) [Link]

You're missing the point. The point is that it's not possible to use the sort(1) tool to actually sort stuff. The fact that aptitude offers the -O switch for sorting is a symptom of that problem, as it wouldn't be necessary to implement sorting in aptitude if Unix shells sucked less.

Evolution of shells in Linux (developerWorks)

Posted Dec 11, 2011 22:48 UTC (Sun) by nybble41 (subscriber, #55106) [Link]

It's not like PowerShell automatically adds support for sorting arbitrary data to an existing sort utility, either. Microsoft also had to write a new sort utility to handle structured data, which in turn relies on API support from the applications supplying that data. You could get the same effect by standardizing the labeled-groups-of-lines format output by 'aptitude show' and writing a sort utility specialized on that format (a 10-minute job in any major scripting language). Or you could just use one of the many portable interactive scripting language shells, like python or irb or scsh, which provide the same services for their respective bindings that PowerShell provides for .NET.

An aptitude-equivalent under PowerShell would need to export the ability to compare package objects by version for any third-party sort utility to work with the data anyway, which is 90% of the way toward just implementing the -O option internally.

"Structured data" is nearly always the first thing anyone tries when implementing a new system. A number of older systems don't support anything *but* structured data for disk files, for example--I've worked on operating systems where even plain text files were stored, one row per line, as tables in a database. The fact that all modern systems actually treat disk files and most inter-process communications as plain byte streams should tell you something. Standardized data *formats*, encoded as plain byte streams, are more flexible and resilient than standardized APIs, as used by PowerShell.

Evolution of shells in Linux (developerWorks)

Posted Dec 12, 2011 5:58 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

>It's not like PowerShell automatically adds support for sorting arbitrary data to an existing sort utility, either.

But it does. You can easily sort any data on any field even using your own custom comparator (try that with 'sort'!), using only one utility. Which had to be written, just as 'sort' utility in Unix.

>You could get the same effect by standardizing the labeled-groups-of-lines format output by 'aptitude show' and writing a sort utility specialized on that format (a 10-minute job in any major scripting language). Or you could just use one of the many portable interactive scripting language shells, like python or irb or scsh, which provide the same services for their respective bindings that PowerShell provides for .NET.

Well, of course you can achieve the same functionality as in PowerShell by replicating what PowerShell does (namely, by introducing structured data). The point is - no one is really doing it in the Unix world.

Evolution of shells in Linux (developerWorks)

Posted Dec 12, 2011 6:25 UTC (Mon) by nybble41 (subscriber, #55106) [Link]

>> It's not like PowerShell automatically adds support for sorting arbitrary data to an existing sort utility, either.

> But it does. You can easily sort any data on any field even using your own custom comparator (try that with 'sort'!), using only one utility. Which had to be written, just as 'sort' utility in Unix.

The point was that the 'sort' utility used in PowerShell only really adds anything over the standard Unix sort command if the source application includes bindings for .NET.[1] A similar utility could be written for Unix with the same features, if anyone was actually interested. It's not exactly a difficult problem, even including fancy features like custom comparators. The critical part is a runtime-specific binding for the source application. If aptitude had bindings for Python, Ruby, or Scheme you could do exactly the same thing from these shells under Unix today. It wouldn't require any more work than implementing the .NET bindings necessary to make aptitude work with PowerShell (assuming they worked on the same platform to begin with). In fact, if aptitude had a DBUS interface, as many other application s do, it would automatically with with all these Unix shells and more.

> Well, of course you can achieve the same functionality as in PowerShell by replicating what PowerShell does (namely, by introducing structured data). The point is - no one is really doing it in the Unix world.

For good reason, as I said--such systems have never won out in the past, why should .NET and PowerShell be any different? Anyway, my suggestion wasn't to replicate what PowerShell does, it was to implement a record-oriented sort utility within the existing Unix byte-stream framework, using a standard data encoding rather than opaque "structured data" in a particular runtime. Historically, forcing everything to work within a single complex runtime or RPC model has not been a winning strategy. Simple, portable, reliable interfaces as best.

----

[1] Purely out of curiosity, how does this kind of sort command work if the data is coming from a file, without the .NET APIs, rather than directly from an application? Is it even possible to get the same effect as redirecting the output of a command to a file before sorting it, or must the source and sort command be directly coupled? It seems like the extra features must get lost in translation, since only the textual representation will be saved on disk.

Evolution of shells in Linux (developerWorks)

Posted Dec 12, 2011 8:59 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

>The point was that the 'sort' utility used in PowerShell only really adds anything over the standard Unix sort command if the source application includes bindings for .NET.[1] A similar utility could be written for Unix with the same features, if anyone was actually interested.

The problem is, you have to write a special-case utility. While PowerShell provides you a generic mechanism for this. It provides a robust platform that other systems can use.

And some features of sort are quite difficult to replicate in bash, for example, custom comparators (I believe, the only way to do this is to create string ordinals suitable for the 'sort' utility).

You've mentioned D-BUS - it's actually quite similar to PowerShell in a lot of regards. It provided a platform that other applications could easily use and adapt (instead of inventing their own ad-hoc remoting). And so application quickly adopted it. The same is happening with PowerShell in Windows.

>For good reason, as I said--such systems have never won out in the past, why should .NET and PowerShell be any different?

Well, it is different because it's already adopted in the Windows world and it's clear that PowerShell is here to stay.

And I think the main reason for adoption is that the time has come for it. All the ingredients for PowerShell were finally in place - even twelve years ago it would have been impossible without the common environment that .NET provides (PowerShell based on COM? Shudder).

The same story with DBUS - there quite a few failed attempts to build a common desktop RPC system in Unix: DCE RPC, DCOP, CORBA in GNOME and I probably forgot quite a few others. Yet DBUS has clearly won and is now ubiquitous. We even have projects for kernel-accelerated DBUS.

>[1] Purely out of curiosity, how does this kind of sort command work if the data is coming from a file, without the .NET APIs, rather than directly from an application?

That depends on what you're trying to do. You surely can dump objects' content in XML/JSON/binary but it's rarely useful in practice. More often you'd use FT command (Format-Table) to create textual representation that fits your purposes. For objects like files or list of files you can dump them as simple filenames. PowerShell is quite flexible in that regard.

Evolution of shells in Linux (developerWorks)

Posted Dec 12, 2011 18:59 UTC (Mon) by raven667 (subscriber, #5198) [Link]

For good reason, as I said--such systems have never won out in the past, why should .NET and PowerShell be any different?

What is this "winning" you are concerned about? PowerShell is already the preferred standard command shell and scripting environment on MS Windows servers and is not really relevant outside that space. UNIX shells like bash are really only relevant on UNIX systems, they have only marginal presence in the Windows space, so there is really no context where "winning" makes sense as they are not directly comparable.

The idea that the shell can be both built out of and expose a full featured programming language and library is not new as you point out but is well demonstrated by PowerShell and not a bad idea.