Fish - The friendly interactive shell

A User-Friendly Shell

Introduction

A great deal of effort has been spent in the last decade trying to make computers more user friendly. While much progress has been made on making graphical user interfaces more user friendly, much less has happened with non-graphical programs such as shells. This is unfortunate, since there are still many things that are inherently easier to do using a shell. The concept of commands, pipelines and environment variables are somewhat complex, but I believe modern shells are harder to use than they have to be, both for the beginner and for the seasoned shell hacker. I have written a new shell called fish, or the friendly interactive shell, that tries to solve several issues that I have found with other shells.

fish features syntax highlighting, advanced tab completion features, discoverable help, a revised shell syntax and many other features. In this article, I will describe some of the features found in fish, and explain why I think they are useful.

Syntax Highlighting

Beginners may like syntax highlighting because the pretty colors make them feel comfortable. But syntax highlighting also makes it easier for humans to parse commands and find errors. fish features an advanced error detector that highlights a large number of errors in red. Errors that are flagged include misspelled commands, misspelled options, reading from non-existing files, mismatched parenthesis and quotes and many other common errors. fish also features highlighting of matching quotes and parenthesis.

Powerful History Mechanism

Modern shells save previous commands in a command history. You can view earlier commands by using the up and down arrows. Fish extends this concept by integrating the history search functionality. To search the history, simply type in the search string, and press the up arrow. By using the up and down arrow, you can search for older and newer matches. The fish history automatically removes duplicate matches and the matching substring is highlighted. These features make searching and reusing previous commands much faster.

Advanced Tab Completion

Tab completion is a feature that saves a lot of time for beginner and seasoned shell professional alike. The fish tab completion engine is powerful, but easy to use.

• fish has a large number of command-specific tab completions, including tab completions for manual pages when using the man command and completion of host names using the known hosts file for commands such as ssh. Both bash and zsh support command specific completions, but neither enables them by default.
• Completions feature a description. When tab-completing a command or a man page, the description is the whatis info. When completing a variable, it is the value of the variable. When completing a file, it is the file type description from the mime database, etc. zsh has limited support for this feature.
• It is possible to use tab completion on strings with wild cards like '*' or '?' as well as in strings with brace expansion like 'input{a,b,c}.txt'. zsh can be configured to expand wild cards, but it does this by replacing the wild cards with one instance of the text they match, making it impossible to tab complete strings that are meant to match more than one file.
• fish tries to squeeze in all completions on one page by truncating long completions and descriptions, but if this fails, a built-in pager is used that supports scrolling up and down using the arrow keys, page up/page down and the space bar. If the user presses any other key, the pager exits and the corresponding character is inserted into the command buffer.

Default Settings

zsh provides command-specific tab completions, a history file, tab completion of strings with wild cards and many, many other advanced functions. But none of them are turned on by default. In fact, a user who starts zsh for the first time would think it was a small improvement over the original Bourne shell. bash does better here, but features like command specific tab completions are turned off by default in bash as well, and the default history settings are not very useful.

It is quite possible that a few of my complaints against bash and zsh could be configured away, this is not on purpose from my side, but even though I have been an avid shell user for nearly a decade, I keep discovering useful new features that are poorly documented, turned off by default and implemented in a less useful way.

Fish does not hide it's functionality. The design philosophy of fish is to focus more on making things work right and less on making things configurable. As a result of this, there are very few settings in fish.

Context Sensitive, User Friendly Help Pages

While the man pages give you a decent amount of information on how to use specific commands, the documentation for the shell and it's built-in commands is often hard to use. The bash man page is nearly legendary for how hard it is to get the information you want. fish tries to provide context sensitive documentation in an easy to use form.

To access the fish help, one should use the 'help' command. Simply writing 'help' and pressing return will start the user's preferred web browser with the local copy of the fish manual. There are a large number of topics that can be specified with the help command, like 'help syntax', 'help editor', etc. These open up a chapter of the documentation on the specified topic. In order to make the help system easy to find, a message describing how to access it is printed every time fish starts up. Finding a specific help section is easy since the section names can be tab completed.

Built-in commands in fish support the -h and --help options, which result in a detailed explanation of how that command works. The only exception to this are the commands the start a new block of code, such as 'for', 'if', 'while', 'function', etc. To get help on one of these commands, type the command without any options.

Error reporting is an often-overlooked form of help. On syntax errors, fish tries to give a detailed report on what went wrong, and if possible, also prints a help message.

Desktop Integration

Since most users access the shell from inside a virtual terminal in a graphical desktop, the shell should attempt to integrate with the desktop. fish uses the X clipboard for copy and paste, so you can use Control-Y to paste from the clipboard to fish, and Control-K to move the rest of the line to the clipboard.

Opening Files

In a graphical file manager it is usually easy to open a document or an image. You simply double-click it and it is launched with a default application. From the shell, this is much more difficult. You need to know which program can handle a file of the given type, and also how to launch it. Launching an HTML file from the command line is no easy task, since most browsers expect a URL, possibly with an absolute path, not a filename. fish features a program called open that uses the same mime-type database and .desktop files used by Gnome and KDE to find the default application for a file, and opens it using the syntax specified by the .desktop file.

A Better Shell Syntax

While shells have gained some features since the seventies, the shell syntax of moderns POSIX shells like bash and zsh is very similar to the original Bourne shell, which is about 30 years old. There are a large number of problems with this syntax which I feel should be changed. Unfortunately, this means that the fish syntax is incompatible with other shells. While it should not be difficult to adapt to the new syntax, old scripts would have to be converted to run in fish.

Blocks

There are many cases in shell programming where you specify a list of multiple commands. This includes conditional blocks, loop blocks and function definitions. In regular shells there is very little logic in how these different types of blocks are ended. Conditional statements end with the reverse command, like: 'if true; echo yes; fi', but loops end with the 'done' command like: 'while true; do echo hello; done', individual case conditions end with ';;' and functions end with '}'. Arbitrary reserved words like 'then' and 'do' are also sprinkled over the code. fish uses a single, consistent method of closing blocks: the 'end' command. For a few examples of block syntax in POSIX shell and in fish, see the table below.

POSIX command	fish command
if true; then echo hello; fi	if true; echo hello; end
for i in a b c; do echo $i; done	for i in a b c; echo $i; end
case $you in *) echo hi;; esac	switch $you; case '*'; echo hi; end
hi () { echo hello; }	function hi; echo hello; end

Quoting

The original Bourne shell was a macro language. It performed variable substitution, tokenization and other operations on one line at a time without understanding the underlying syntax. This results in many unexpected side effects: Consider the following block of code:

smurf=blue;
smurf=evil; echo Smurfs are $smurf

On the Bourne shell, it will result in the output 'Smurfs are blue'. Macro languages like M4 and Bourne are not intuitive, but once you understand how they function, they are at least predictable and somewhat logical. bash is implemented as a standard language using a bison grammar, but still chooses to emulate some of the quirks from the original Bourne shell.

The above example would result in bash printing 'Smurfs are evil'. On the other hand variable values are still tokenized on spaces, meaning you can't write 'rm $file', since if the variable file contains spaces, rm will try to remove the wrong files. To fix this, the user has to make sure every use of a variable in enclosed in quotes, like 'rm "$file"'. This is a very common source of bugs in shell scripts since it is simply a case of the default behavior being unexpected and very rarely what is wanted.

In summary, by making bash a non-macro language that sometimes behaves like one, it becomes unpredictable and very hard to learn.

fish is not a macro language and does not pretend to be one. Variables with spaces are still just one token. Because of this, there is no need for the double quotes to mean something different from single quotes, so both types of quotes mean the same thing, and quotes can be nested.

Variable Assignment

Variable assignments in Bourne shell are whitespace sensitive. 'foo=bar' is an assignment, but 'foo = bar' is not. This is just a bad idea. fish does something somewhat unexpected while fixing this. It borrows syntax from csh, and uses a command called 'set' to assign variable values. The reason for doing this is that in fish everything is a command. Loops, conditionals and every other kind of higher level language construct is implemented as yet another built-in command, following the same syntax rules. This makes the language easier to learn and understand, as well as easier to implement.

To set the variable smurf to the value blue, use the command:

set smurf blue

By default, variables are local to the current block and disappear when the block goes out of scope. To make a variable global, you need to use the -g switch.

Two Methods of Creating Functions, and Both are Bad

bash, zsh and other regular shells allow you to create stored functions in two ways, either as aliases or as functions.

Aliases are defined using commands like 'alias ll="ls -l"'. Aliases are simply string substitutions in the command-line. Because of this, aliases have the following limitations:

• You can only redirect input/output to the last command in the alias.
• You can only specify arguments to the last command in the alias.
• Alias definitions are a single text string, this means complex functions are nearly impossible to create.

Because of these limitations, bash uses a second method to specify functions, using a syntax like:

ll() { ls $*;}

While this solves the issues with aliases, I think this is just as bad a syntax. It looks like C code, but anyone expecting it to work anything like C will discover it is really not. You can not specify argument names in the parenthesis, they are just there to make it look like C code. The curly brackets are some sort of pseudo-commands, so skipping the semicolon in the example above results in a syntax error. And perhaps the most strange quirk of all is that removing the whitespace between the opening bracket and 'ls' will also result in a syntax error. Clearly this is not a very well though out syntax. fish uses a single syntax for defining functions, and the definition is just another regular command:

function ll; ls $argv; end

This is slightly wordier than the above examples, but it solves all the issues with either of the above syntaxes in a way that is consistent with the rest of the fish syntax.

Impossible to Validate the Syntax

Since the use of variables as commands in regular shells is allowed, it is impossible to reliably check the syntax of a script.

For example, this snippet of bash/zsh code may or may not be legal, depending on your luck:

if true; then if [ $RANDOM -lt 1024 ]; then END=fi; else END=true; fi; $END

Both bash and zsh try to determine if the command in the current buffer is finished when the user presses the return key, but because of issues like this, they will sometimes fail.

fish solves this by disallowing variables as commands. Anything you can do with variables as commands can be done in a much cleaner way using either the eval command or by using functions.

Minor Problems

The strings '$foo', "$foo" and `$foo` all look rather similar, while doing three completely different things. fish solves this by making '$foo' and "$foo" mean the same thing, as described above, but also by making the syntax for sub-shells use parenthesis instead of back-ticks.

A large number of standard UNIX commands, like printf, echo, kill and test are implemented as shell built-ins in bash and zsh. As near as I can tell, there is only one advantage to this, a minimal performance increase. But the drawbacks are many:

• Bugs in one of these built-ins risk crashing or corrupting the entire shell.
• Commands will change their behavior based on what shell you are using.
• Users of other shells will not benefit from the commands you have written.
• It breaks the UNIX philosophy of doing only one thing but doing it well.
• Memory usage is increased.

For those reasons, fish implements as few built-in commands as possible. Including block commands such as 'for' and 'end', fish implements 24 built-ins, whereas bash implements between 60 and 70 of them.

Summing Up

None of the problems with the regular shell syntax are big enough to make the shells unusable. For thirty years, the shell has been a valuable tool for many computer users. But even well designed, powerful programs sometimes need to be updated to remove some of the old mistakes. The changes introduced in fish makes the shell language easier to discover and remember for beginners, and once learned, it should introduce fewer bugs for experienced users. Regular computer languages have evolved and been replaced multiple times in the last thirty years, why shouldn't that be the case for shell languages?

Future Plans

Mostly, the fish codebase is in good shape. It is well documented and reasonably small. But since the code is new and untested, it contains more bugs then one would like. It definitely needs an audit for security and stability problems, and a much more comprehensive test suite. There are also a number of new features that I wish to add, but none of these features require any major rewrites.

There is one important piece of syntax that is not yet implemented, namely full IO redirection for blocks and functions. It is currently possible to use functions in pipelines and to redirect the output of functions, but functions cannot output binary data, and input redirection is not supported. It is not possible to pipe or redirect the input/output of blocks of code.

A future version of fish will allow code like this:

for i in (find . -name "*.c"); echo $i; grep "mbstowcs" $i; end | less

to view the output of multiple commands in less, or:

cat foo.txt | while test -z quit
    read type
    switch $type
        case quit
            set quit 1
        ...
     end
end

to read multiple lines from a file. I am working on implementing these features and hope to release a new version with these features in the coming weeks.

Fish currently has command specific completions for about 60 commands. There are a great number of additional commands that would benefit from completions. I hope that it will be possible to use Doclifter to automatically convert manual pages into tab completion specifications.

Another area that still needs more attention is the documentation. While the project already has a great deal of documentation, fish would benefit from shell scripting tutorials, an introduction to UNIX concepts, and various other forms of documentation.

The syntax validation and error reporting of fish also needs more work. Fish already does some basic syntax checks, but a future implementation will be able to detect most syntax errors before running a file.

There are also a large number of minor features that I plan to try out:

• Undo/redo support.
• Interactive directory history, use Alt-up and Alt-down to insert next previous directory into the command buffer.
• Multi-line editing.
• Mouse support, including moving the cursor by clicking in the window, as well as selecting a completion by clicking on it.
• Suggest completions by writing them out in an understated color.
• Store the output of previous commands. Calculators often use the variable 'ans' to refer to the result of the previous calculation. It would be nice if $ans could be the output of the previous shell command.

If you think fish sounds interesting, give it a spin. You can download it at http://roo.no-ip.org/fish/. fish has been released under the Gnu General Public License (GPL), it is available as an srpm, an i386 rpm or an i386 deb package.

---

(Log in to post comments)

Why not just escape the variables instead?

set a Hello
set b '\$a is $a'
set a Goodbye
echo $b

should print "$a is Hello"

It is just a hassle to keep track of what will be expanded in bash scripts and I would really
like the distinction between ' and " to go away.

It might be incompatibl with other shells, but since I am creating a 
distribution for beginners, fish is the default login shell, whilst bash 
is the default /bin/sh.

int hash(char a[], int size, int prime) {
  char * p = a;
  int sum = 0;
  while (*p) {
    sum = sum * prime;
    sum = (sum + *p++) % size;
    //printf("%d ", sum);
  }
  return sum;
}

char str[1000];

int main(void) {
  int size, prime;
  scanf("%d %d", &size, & prime);
  while (scanf("%s", str) != EOF) {
    printf("%d\n", hash(str, size, prime));
  }
}

  for i in *.c; do
    [ "$i" = "*.c" -a ! -e "$i" ] && break
    do_stuff "$i"
  done

  for i in *.c; do_stuff $i; end

About your re-implementation of NeXT's "open" using mimecap and custome 
files:

see does exactly this, it's part of the mime handling utilities.