Bash 3.0 released [LWN.net]

August 4, 2004

This article was contributed by Joe 'Zonker' Brockmeier.

GNU Bash has been in the low 2.0x series for some time, so the version jump to 3.0 last week was something of a surprise, at least to those who haven't been following Bash development closely. Since Bash is a core piece of infrastructure for most of the Linux community, we decided to take a look at the 3.0 release and find out what changed, and what users could expect from the new release.

To that end, we touched base with Bash maintainer Chet Ramey, who was kind enough to reply to our questions about the latest release. The first question on our mind, of course, was "why the version bump?"

You have to look at the changes from 2.05 to 3.0, not any of the intermediate releases. The idea was to introduce major changes in intermediate releases following bash-2.05, let them stabilize, and then increment the major version.

The changes in 3.0 include support for the bash debugger and internationalization support, as well as a number of smaller features that had been requested for some time (time-stamped history entries, better brace expansion) and better POSIX compliance. To that you add the multibyte character support introduced in bash-2.05b and the code cleanups and programming improvements in bash-2.05a.

The whole set of changes deserves a major version bump.

Indeed, there are quite a few changes in this release. A look at the CHANGES file or the NEWS with the release source shows a slew of bugfixes and changes to Bash and Readline.

One interesting new addition to Bash 3.0 is new type of brace expansion. The syntax for the new brace expansion is {x..y} where x and y can be an integer or a single character in ascending or descending order. For example, the set {z..a} would match all of the letters from z to a in descending order. (z,y,x, etc.) The set {1..1000} would match the each of the integers from 1 to 1000.

Another new feature of interest is the addition of history timestamps. This allows users to see when commands were run, which can provide some useful and interesting information.

There are several new options with Bash 3.0. The "failglob" option will probably be of interest to many users. When set, this option will cause an error when a glob expression fails to match any files -- as opposed to running the command anyway. The new "pipefail" option tells Bash to return a failure status if any command in a "pipeline" fails, as opposed to the default behavior of returning the status of the last job in a pipeline.

Of course, one might wonder if all of the improvements and changes in the release will affect existing shell scripts. Many Bash users have a number of shell scripts that are vital to our day to day work, this writer included, and aren't eager to see them break on a new version of Bash. According to the release notes, there are some incompatibilities between 3.0 and Bash version 1.14, but no mention of 2.0x versions. According to Ramey:

Any major incompatibilities are the result of changes for POSIX compliance. There have not been comparable major additions to the shell's syntax as there were between 1.14 and 2.0.

This writer has tried a number of shell scripts against the Bash 3.0 release, and didn't find any incompatibilities or issues. In fact, a (permanent) switch to a Bash 3.0 login shell may be in my very near future.

We also asked Ramey what, if any, features were planned for future versions of Bash. Ramey said that there were already plans for future releases

Programming support: associative arrays, better integration with the bash debugger (a separate project), small improvements for programming convenience (e.g., a += operator to append to a variable value), and some object-oriented features like ksh's discipline functions for variables.

I'm intrigued by zsh's loadable module system as well.

As for interactive use, I think there's room for improvement in the programmable completion system.

Readline needs better support for threaded use (multiple threads in a single process all using separate instances of readline). This is very hard to do today.

Interested users who don't want to wait for Bash to ship with their favorite distribution can find the source on Ramey's Bash page, or the GNU mirrors.

(Log in to post comments)

globs vs regexps

Posted Aug 5, 2004 5:31 UTC (Thu) by rfunk (subscriber, #4054) [Link]

The "failglob" option will probably be of interest to many users. When set, this option will cause an error when a regular expression fails to match any files

Filename globs are not regular expressions.

Filename globs are those expressions where ? matches a single character, * matches any number of characters, and the beginning and end are always anchored (e.g. a*d matches abcd but not qabcdz).

Regular expressions have a precise meaning I don't remember at the moment, but commonly refer to expressions where a period matches a single character, a * modifies a previous bit of the expression to match any number of that bit, and beginning and end achors must be explicitly specified if they are desired. (a.*d matches both abcd and qabcdz.)

The single-star glob "*" is equivalent to the four-character regular expression "^.*$".

Confusion between these two systems trips up a lot of people, so it's important not to refer to one when you mean the other.

RE: globs vs regexps

Posted Aug 5, 2004 6:42 UTC (Thu) by stuart_hc (guest, #9737) [Link]

Yes, this is an important point to stress. Filename pattern matching was historically related to regular expressions but the two have always been distinct. Filename patterns are described under the Pattern Matching section of the Bash man page, or better still in the POSIX standard on Pattern Matching Notation. The Wikipedia page on Regular Expressions gives both a gentle introduction and a formal description.

regexps vs regexps

Posted Aug 5, 2004 8:22 UTC (Thu) by pkolloch (guest, #21709) [Link]

Regular expressions are first of all a theoretical concept. In that sense, regular expressions specify a (often infinite) language of accepted words. Fileglobs are usually not quite as powerful, but have a simplified syntax. There are some very frequently used regular expressions syntaxes as nicely described in the referenced WikiPedia page.

If we start nit picking, we should not mistake one of those syntaxes as the one and only real regular expression syntax.

regexps vs regexps

Posted Aug 5, 2004 14:26 UTC (Thu) by rfunk (subscriber, #4054) [Link]

Regular expressions are first of all a theoretical concept

Yes, I tried to account for that without getting into the theoretical details that I don't remember.

If we start nit picking, we should not mistake one of those syntaxes as the one and only real regular expression syntax.

Except that globs do not have the power required by a true (theoretical) regular expression. They can't express everything that a regular expression can.

globs vs regexps

Posted Aug 5, 2004 11:34 UTC (Thu) by Lasse99 (guest, #1899) [Link]

Actually, the single-star glob '*' is equivalent to the rather complicated
regular expression '^[^.].*$' or even '^[^./][^/]$'.
On the other hand, the simple regular expression '.*' must be expressed by
a set of two filename globs, namely ( '.*' '*' ).
So, it is debatable which syntax is more complicated...

globs vs regexps

Posted Aug 5, 2004 11:35 UTC (Thu) by Lasse99 (guest, #1899) [Link]

Ooops! That should have been '^[^./][^/]*$'

Some distribs (Gentoo) have it out already, at least in unstable.

Posted Aug 5, 2004 17:49 UTC (Thu) by Duncan (guest, #6647) [Link]

> Interested users who don't want to wait for Bash
> to ship with their favorite distribution [...]

Of course, other interested users already HAVE it shipping in their
favorite distribution, and have been using it for several days, if they
are running unstable, anyway.

Gentoo, anyone? A FreeBSD user on my ISP reported it was out for them as
well, altho that's obviously not a /Linux/ "distribution".

One bug confirmed both on Gentoo and FBSD, so it's apparently upstream, so
far: Where history scrollback (using the arrow keys, for instance) used
to let you go back in your command list, and then return to the normally
blank initial command, with 3.0, if one scrolls back at least two
commands, one can only scroll back forward to the last /history/ command,
the one BEFORE the previously blank command. To get back to the blank
command, one must use ^C. (If one scrolls only one command up, one can
still scroll that single command back down. It only breaks if one scrolls
at least two commands up.)

Or maybe that's a POSIX compliance thing? Do they standardize even
history scrolling to /that/ extent?

Duncan

Associative Arrays Still in the Future? Yuck!

Posted Aug 6, 2004 0:04 UTC (Fri) by AnswerGuy (guest, #1256) [Link]

Korn shell has had support for associative arrays (which python calls "dictionaries," and perl "hashes") for eons (well over a decade I think).

I'll also be interested in whether bash 3.x fixes one shortcoming of bash that's been around forever. Consider the following:

unset foo; echo bar | read foo; echo $foo

What should the output be?

In zsh and ksh you'll see that foo is set; they spawn the subshells to the left of the pipe. In bash you'll find it unset because the read occurs in a subshell which exits at the semicolon. (Thus necessitating braces or parents to group the part after the pipe; or force them all into yet another, explicit subshell).

To most shell scripters this is obscure; nobody cares. However, for serious shell scripting this sort of thing is important (not so much for reading values from subprocesses into single variables, but for reading multiple variables parsed on $IFS, for example).

(Yes, I know, most UNIX/Linux admins and programmers would counter that serious scripting isn't done in shell --- that "real programmers" (and scripters) use Perl. To which I can only respond: "Foo!"

JimD

Bash programming enhancements - worthwhile?

Posted Aug 7, 2004 14:50 UTC (Sat) by nix (subscriber, #2304) [Link]

Some of us like bash as a programming language too.

(comments below apply to python as well, I think).

You can't use perl in many environments, either because it isn't available, or because you're not allowed to replace an ancient and buggy module; perl has a hugely higher startup overhead than bash does; you can't sanely use perl scripts at system startup time; one-liners at the shell prompt tend to grow into scripts, and (at least for me) those one-liners are, well, bash scripting...

Yes, there is a reason to improve bash as a programming language. :)

Bash programming enhancements - worthwhile!

Posted Aug 10, 2004 14:44 UTC (Tue) by mwilck (guest, #1966) [Link]

The shell is superior to all of these if a program

doesn't handle complex data structures
carries out many file system activities like searching and moving files or manipulating text files
makes much use of command line programs e.g. from the Unix tool box

Maybe some time in the future all text files will be XML and the file system will be replaced by a CORBA interface to some data base. I hope that happens after I retire.

P.S. did you really mean Java ? Ever seen a shell that needs a 30MB VM to be loaded before it can print "hello world!" ?

Martin, not looking forward to object-oriented bash programming but still seeing room for enhancements.