LWN.net Logo

Evolution of shells in Linux (developerWorks)

Evolution of shells in Linux (developerWorks)

Posted Dec 8, 2011 21:59 UTC (Thu) by quotemstr (subscriber, #45331)
In reply to: Evolution of shells in Linux (developerWorks) by Cyberax
Parent article: Evolution of shells in Linux (developerWorks)

> Uhm. PowerShell supports standard text redirection.

What do you think this PowerShell pipeline puts into foo.txt?

echo hello > foo.txt

If you guessed ASCII "hello" or "hello\n" or "hello\r\n", you'd be wrong. Horribly wrong.

PowerShell converts everything to UTF-16-LE, prepends a BOM, and puts the result in foo.txt. This behavior is almost too horrible for works. PowerShell even mangles the output of *native* programs, e.g.

ipconfig > ipconfig-out.txt

In cmd, ipconfig-out.txt contains ASCII. When the same pipeline is run in PowerShell, it's UTF-16-LE-with-BOM. This encoding is more verbose, less commonly supported, and (because of the BOM) far less friendly to splicing and concatenation. This behavior is just awful.

As a lemma, PoewrShell will also mangle the output of programs that produce non-text output: consider

ps2pdf < foo.ps > foo.pdf

foo.pdf will take each "line" of ps2pdf output, convert it to UTF-16 as best it can, then dump the result in foo.pdf. The result looks like a PDF sent through a meat grinder.

(Don't tell me you're supposed to use some cmdlet for doing the conversion: the point of a shell is to work with existing programs. If I wanted to use a library, I'd use a different programming language.)


(Log in to post comments)

Evolution of shells in Linux (developerWorks)

Posted Dec 8, 2011 22:14 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

That's the way Windows works. Sorry.

You can change default encoding to UTF-8 with "$OutputEncoding = New-Object -typename System.Text.UTF8Encoding". Binary mode is more tricky, you have to add "-encoding byte" everywhere.

Evolution of shells in Linux (developerWorks)

Posted Dec 8, 2011 22:27 UTC (Thu) by quotemstr (subscriber, #45331) [Link]

> That's the way Windows works. Sorry.

Thus my original point: I'll consider using PowerShell only after the critical issues I've identified are fixed. Until then, Unix shells (even running under Cygwin) are superior by far.

Evolution of shells in Linux (developerWorks)

Posted Dec 8, 2011 22:51 UTC (Thu) by tialaramex (subscriber, #21167) [Link]

Windows exists in a parallel universe where at some moment soon a great wizard will appear and banish everything outside the Basic Multilingual Plane so that they can go back to UCS-2 where things were simpler (yet conveniently incompatible with stodgy old Unix).

People who are never going to be happy about LLP64, UTF-16, or various other arbitrary yet defensible differences from Unix will never be comfortable on Windows. This isn't even a religious war, it's some fundamental cultural difference just like the relationship between directions and time.

For me "bringing forward" a meeting will always make it sooner, and UTF-8 will always be the more intuitive and sensible encoding. So no Windows for me.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds