Leading items

Fish shell 2.1

Version 2.1 of the fish shell has been released, complete with a string of new features and fixes sure to hook users.

We first looked at fish all the way back in 2005. In keeping with longstanding tradition for shell-naming, fish is an abbreviation, for "friendly interactive shell." In the early days, fish's friendliness was visible through features like automatic syntax highlighting, feature-rich tab-completion behavior, and context-aware help. In subsequent years, it is clear that fish has had an influence on other shells, as older projects like Bash have added and enhanced their own takes on some of those features.

Naturally, fish itself has continued to evolve over the same time period, too. The project still advertises its friendlier implementations of well-known shell features, however—among them syntax highlighting and tab completion, plus command history, prompts, and even common built-in commands like cd. In addition, there are unique features like a web-based configuration tool (which runs on a local Python-powered HTTP server).

The fish source code is hosted at GitHub, while the project's web site offers binary packages for a wide range of Linux distributions, Windows, and Mac OS X. Version 2.1.0, the latest release, was released on October 28. The release notes page provides an overview of the changes in 2.1 and a handful of previous releases, though not for the entire history of the project.

Less is more

Many shells offer tab-completion for commands and filenames, so that users only need to type enough letters to uniquely identify what follows—and, when there are multiple possibilities, the shell usually writes them to the terminal window to help. Fish takes this concept considerably further. First, its tab-completion feature expands not only commands and filenames, but usernames, environment variables, job IDs, and process names. The filename-completion also supports strings with wildcard characters; so typing "b??t" would correctly bring up "bait" as a tab-completion match.

Second, fish includes several command-specific completions, allowing manpage completion for man and whatis, Makefile target completion for make, mount-point completion for mount, package-name completion for apt-get, rpm, and yum, hostname completion for ssh, and username completion for su. For other programs, fish offers tab-completion for each program's command-line switches and options. Each of the program-specific completions is tailor-made for the specific command; for example, ssh hostname completion draws on the list of hosts in the user's known_hosts file. It is also possible to write custom completion options for any command.

The 2.1 release adds another factor to fish's tab-completion effort: fuzzy matching of filenames and commands. The shell first attempts to find a prefix match (that is, "fo" matches "foo"), but if it does not find one it will look for substring matches in the middle and at the end of filenames, and, as a last resort, it will look for true fuzzy matches. Thus, typing "far" could match "foobar," if there are no better matches.

On the other hand, tab completion requires the user to actually hit the tab key; those who find even that effort to be too strenuous will be relieved that fish also makes a best-guess attempt at predicting what the user will type next, in the form of autosuggestions that appear after the text cursor while the user is typing. To accept an autosuggestion, one only needs to hit the right-arrow key on the keyboard. The upshot is predictive typing not unlike the text-prediction on mobile phones or in web browser location bars, and like those autosuggestions, fish provides suggestions that are weighted to the most-recently-typed input, rather than simple string matching. Furthermore, while tab completion works one token at a time, fish's autosuggestions can match entire commands, and are thus—potentially—faster.

Fish also features a number of other expansions, such as numeric ranges (e.g., [1..9]), sets (e.g., {a,b,d}), and process IDs (using the percent sign %). The PID expansion will match process names and expand them to the appropriate PID, so typing kill %jav will match a running java process, allowing the user to kill it without looking up the PID first. A change in fish 2.1 is that % by itself expands to the last command backgrounded, so fg % will put the last backgrounded process back into the foreground.

Going off script

Fish is scriptable, and in most cases uses familiar syntax, but here too it adds its own twist on the feature. There are some special environment variables, the most notable of which is status, which stores the exit status of the last process. In addition to 0 for success and 1 for an error, fish will attempt to provide more information with some specific values that begin with 12. An status value of 126, for example, means that the last command exited with an error because the filename provided was not executable; 127 means that no matching command or function was found. A process that exits with a signal will set status to 128N, where N is the signal number. In these cases, of course, the value of the status variable is treated like a string, which could trip up shell scripts if not handled properly.

Fish also treats the shell history differently than many of its competitors. First, there is a history variable that is an array of the previous commands; among other possibilities, that makes the history available to scripts. But the built-in history command is different, too. Simply hitting the up arrow will step back through previous commands (as it does in other shells), but typing a few letters and then hitting the up arrow performs a history search for the typed characters.

For example, typing make followed by the up arrow will let the user step back through all the previous matches against make. Holding down the Alt key while using the up and down arrows allows the user to search only for the word beneath the text cursor; that could be useful for narrowing down a search with a lot of matches or particularly long commands. Users can also delete items from the history, and can prevent any command from being recorded in the history (by preceding the command with a space character).

The fish project also prides itself on its extensive built-in help, which certainly could come in handy for those who find all of the differences with Bash and other shells daunting. The documentation is provided as both manpages and HTML, and all of fish's built-in commands support the -h switch.

Looking good

Fish's syntax highlighting is another area where the project attempts to provide more functionality than other shells. Although many standard Unix commands are available with their own color-coding (ls perhaps being the most popular), fish color-codes all commands as the user types. Commands, filenames, autosuggestions, tab-completion matches, quoted strings, parameters, comments, and error messages are all colored differently. Color overload is perhaps an understandable fear, but the color values are user-selectable.

The best way to modify the color values is to use fish's web-based configuration tool. The tool is launched with fish_config, and starts up a web server running on localhost:8000, which it opens using the user's default browser. The configuration page lets users graphically select colors (foreground and background) for all of the syntax options, as well as whether each option is underlined. Most of the syntax options are used in the command line and output, but some (such as the current working directory) are also used in the shell prompt.

The shell prompt itself is also configurable within the web configuration tool; version 2.1 includes 14 presets, some of which are tailored toward specific use cases (there are two Git-specific prompts, for example). A right-side prompt (i.e., a prompt-like informational area that sits in the right-hand side of the command line), which is a feature also found in Zsh, is also supported. The configuration tool also lets the user browse through any functions and aliases defined in the current session, view the history array, and look at the environment variables.

Fish also supports many of the same configuration options expected by shell users, such as setting the terminal window title or setting a greeting message. There are a few differences, of course; for example, the fish equivalent to .bashrc is ~/.config/fish/config.fish, so it may take a few minutes of looking through the documentation to get up to speed. But on the whole, fish does a good job of adding its layer of friendliness onto the standard shell experience without introducing major disruptions to how users work. True, people with a significant standing investment in Bash scripts may not find migrating them to fish (or any other shell) an appealing prospect, but for more casual shell usage, it could be quite a good catch.

Comments (20 posted)

Creative Commons licenses reach 4.0

Creative Commons (CC) has released the long-awaited 4.0 version of its suite of licenses for artistic and creative works. The revisions are intended to fix specific issues that users had encountered over time with the current crop of licenses. While, as with the previous efforts, the process for drafting the 4.0 set was conducted in the open, this time there was a more formal structure in place that established a clear set of goals for the new licenses. The result is, hopefully, a set of licenses that better address releasing works internationally, are easier to understand and comply with, and that mesh better with legal requirements outside of copyright itself.

There are currently six CC licenses: Attribution (aka CC BY), Attribution-ShareAlike (CC BY-SA), Attribution-NoDerivs (CC BY-ND), Attribution-NonCommercial (CC BY-NC), Attribution-NonCommercial-ShareAlike (CC BY-NC-SA), and Attribution-NonCommercial-NoDerivs (CC BY-NC-ND). At the moment, the CC web site's licenses page still points to the old versions of each, but its license chooser includes the updated links. The wiki page for the 4.0 licenses outlines the changes and the rationale behind them.

The previous revision of the CC license set was the 3.0 suite released in 2007. Among other changes, the 3.0 set separated "US" and "generic" versions of the licenses, altered some language to more uniformly address the concepts of "moral rights" and "collecting societies," and introduced clarifications specifically to improve compatibility with the Debian Free Software Guidelines (DFSG) and compatibility with MIT's OpenCourseWare project.

According to the CC blog, the process to update the licenses began at the organization's 2011 Global Summit. As with the 3.0 revision, the key factors included addressing concerns over interoperability with other licenses and with legal rights that are distinct from copyright but that CC license users were encountering in the real world. A public process for crafting the revisions was rolled out in December 2011. A requirements-gathering period was open through February 2012, during which stakeholders could voice their concerns on a public discussion list.

Subsequently, the requirements proposed were organized, vetted, and discussed by CC and collected on a wiki page. The first draft of the revisions was posted in April 2012, with subsequent drafts following in August, February 2013, and September. Public comments were taken on each draft, leading to the eventual publication of the final versions on November 25.

Data, morals, and other rights

Of the changes in the 4.0 set, perhaps the biggest is the treatment of non-copyright rights (sometimes referred to as "neighboring rights") that the law in various countries grants to creators and publishers. Chief among these neighboring rights is the notion of sui generis database rights, a legal concept which is important in the EU and several other jurisdictions. Sui generis database rights are intellectual property rights granted to those who compile and edit a database or a data set. The investment made to create the data set is given legal protection, so that others cannot harvest or reuse the data without the compiler's consent. That means that extracting a "substantial" subset of the facts from the data set requires the creator's consent—even if the form or expression (which would be covered by copyright) is changed.

The CC organization itself is not in favor of sui generis database rights, because the concept is at odds with CC's free culture principles, but the earlier CC licenses did not properly address the issue. The EU "ports" of the 3.0 licenses specified that when derivative works were used in such a way that only database rights were triggered, the copyright requirements and prohibitions of the license (e.g., requiring attribution or disallowing derivatives) were waived. The non-EU versions of the licenses did not address database rights at all.

That approach reflected CC's stance against sui generis database rights, but it had the effect of restricting CC license adoption in many big data and scientific projects. Furthermore, it opened the door to alternative licenses like the Open Database License, which CC views as insufficiently free, difficult to comply with, and burdened with extraneous clauses (such as contractual obligations).

The 4.0 CC licenses have been rewritten to specifically cover database rights, performance rights, broadcast rights, and recording rights under the same terms as copyrights. Thus, a creator can place a work (including a data set) under one of the 4.0 licenses and provide a uniform set of conditions about how derivatives must be attributed, shared, or used for commercial purposes.

On the other hand, the 4.0 licenses specifically limit the licensor's "moral rights." Moral rights are another non-copyright right that some jurisdictions grant to the creators of works; they allow the creator to restrict a use of the work that the creator considers to have a harmful or objectionable affect on the creator's relationship to the work. The revised licenses do not allow a creator to offer a work under a CC license and simultaneously restrict licensees on moral rights grounds. This is a policy decision on CC's part, to maximize the rights granted to licensees. Similarly, the 4.0 licenses do not permit licensors to impose restrictions based on privacy rights, publicity rights, or personality rights.

In addition, all of the neighboring rights language has been unified across the licenses and edited to be uniform worldwide, so that there are no longer different "ports" for different jurisdictions. CC is working hard on providing translations in many languages, but the separate "EU" versus "non-EU" versions of the 3.0 series are gone.

Compatibility and compliance

The 4.0 licenses also make changes to improve compatibility with outside licenses. The first change specifically addresses the CC licenses which have a ShareAlike clause. Because it might not be clear otherwise whether or not a work under a ShareAlike license could be combined with another work under other terms (say, the GNU Free Documentation License, to pick one at random), in the 3.0 set, the CC BY-SA license had a compatibility clause pointing to an official list of other licenses that could be used to create a combined derivative work (meaning, specifically, that the combined work could be licensed under either license; as-is, CC BY-SA must be the license for the derivative). 4.0 extends this feature to the CC BY-NC-SA license as well. The fact that so far CC has not put any outside licenses on the official compatibility list might make the issue seem moot, but at least the mechanism is in place.

On a more practical front, the 4.0 set also adds "or later" language to the ShareAlike licenses, so it is clear that, for example, a CC BY-SA 4.0 work can be combined with a CC BY-SA 5.0 work without triggering any impossible legal puzzles.

There are several changes that are designed to simplify the job of determining whether a particular usage of a CC-licensed work complies with the license or not. First, if a licensee's rights under a CC license are terminated by not complying with the terms, the 4.0 licenses state that those rights will be automatically reinstated if the violation is corrected within 30 days of the licensee being made aware of the violation. Previous versions of the licenses required violators to expressly seek reinstatement from the licensor.

Second, the new licenses explicitly allow licensees to make "private adaptations" of a work—meaning that users can do whatever they want to the work if they are not republishing it or redistributing it. That is a common-sense interpretation in most cases, but in earlier versions of the licenses it was a big problem for the NoDerivs licenses. Without the clarification, some lawyers might decide that a CC BY-ND data set could not be used internally by a company at all if there was any adaptation of the data involved (as there most certainly would be). Now such uses are expressly permitted, and the permission includes copyrights as well as all of the neighboring rights.

Finally, the 4.0 licenses take an explicit stance on the question of DRM restrictions. Now, if a work is released under a CC license and someone creates a derivative work with it that is wrapped in a DRM scheme, downstream users are expressly given permission to circumvent the DRM. As the CC wiki page puts it:

This permission to circumvent only applies to works derived from the CC-licensed original, though. Transmission services are still able to wrap their distribution channel in its own DRM scheme.

License away

To be sure, many of the changes in the CC 4.0 license suite are details that only apply in certain legal jurisdictions or in specific sets of circumstances. But then again, that specificity is what legalese is for. On the whole the changes strengthen the rights and permissions that are granted to users of CC-licensed works, which is whole point of the Creative Commons organization. No longer can someone release a database under a CC license and still restrict others from using it; no longer can someone take a CC BY movie created elsewhere and wrap it in DRM to prevent users from repurposing it.

Some of the effects of the 4.0 changes may not be felt for quite some time; there is no telling when or if "CC compatible licenses" will actually be approved, and the "or later" clauses in the 4.0 suite do not retroactively affect 3.0-and-earlier licenses. Furthermore, the NoDerivs and NonCommercial licenses remain incompatible with most open source and free software licenses (after all, they do impose restrictions on the licensee); there were proposals put forward to deprecate NonCommercial or significantly revise it for the 4.0 suite, but CC decided none of the proposals justified the significant disruptions to existing CC usage that might result. On the whole, however, this latest revision makes it clearer what is allowed and what is not, which ought to make CC licensing a more appealing choice for a great many content creators.

Comments (9 posted)

2013 Linux and free software timeline - Q1

Here is LWN's sixteenth annual timeline of significant events in the Linux and free software world for the year. As per tradition, we will divide up the timeline up into quarters; this is our account of January–March 2013. Timelines for the remaining quarters of the year will appear in the coming weeks.

There are almost certainly some errors or omissions; if you find any, please send them to timeline@lwn.net.

LWN subscribers have paid for the development of this timeline, along with previous timelines and the weekly editions. If you like what you see here, or elsewhere on the site, please consider subscribing to LWN.

For those readers in a truly reflective mood, our timeline index page includes links to the previous timelines and other retrospective articles that date all the way back to 1998.

Canonical announces its plan to deliver Ubuntu for phones and tablets (announcement; LWN article).

FreeBSD 9.1 is released (announcement).

BlueZ 5.0 is released (LWN article).

Debian's m68k port is revived (LWN blurb).

LLVM 3.2 released (announcement).

Red Hat Enterprise Linux (RHEL) 5.9 is released (announcement).

Firefox 18 is released, introducing WebRTC support (announcement).

Fedora 18 is released (announcement; LWN article). The ARM release follows a few weeks later (announcement).

Kolab 3.0 is released after 7 years of development (announcement).

Well-known free software developer and co-author of the RSS specification Aaron Swartz takes his own life on January 11, while he was the target of a protracted federal court case about the public release of articles from the JSTOR digital archive. Among many others, Cory Doctorow reflects on the events themselves (LWN blurb); shortly afterward, the government formally drops its case (LWN blurb).

Long Term Support Initiative kernel 3.4 is released (LWN blurb).

LWN turns 15 years old; there is much rejoicing (LWN article).

Mozilla announces that developer phones will be released for its upcoming Firefox OS (LWN blurb).

Version 1.1 of the Trinity fuzz tester is released (announcement; LWN article).

linux.conf.au (LCA) is held in Canberra, Jan 28 to Feb 2 (LWN coverage).

FOSDEM 2013 is held in Brussels, February 2 to 3 (LWN coverage).

KDE 4.10 is released (announcement).

digiKam 3.0 is released (announcement; LWN article).

Krita 2.6 is released (announcement).

The Open Invention Network (OIN) passes 500 licensees, setting a new milestone (LWN blurb).

Linaro forms the Linaro Networking Group (LWN blurb).

GNOME selects JavaScript as its official development platform language (LWN blurb).

The WebRTC audio/video standard is demonstrated in a call between Google Chrome and Mozilla Firefox (LWN blurb; LWN article).

LibreOffice 4.0 is released (announcement).

The Python trademark is put at risk in the European Union by a competing application (LWN blurb); the situation is later resolved by the Python Software Foundation (LWN blurb).

The 2013 Android Builders' Summit is held in San Francisco, February 18 to 19 (LWN coverage).

Embedded Linux Conference (ELC) is held in San Francisco, February 20 to 22 (LWN coverage).

Linux 3.8 is released (announcement; merge window summaries 1, 2; statistics; KernelNewbies summary).

Tizen 2.0 is released (LWN article).

The Opera browser drops its own web rendering engine to adopt WebKit instead (LWN blurb).

Firefox 19 is released, marking the debut of the browser's built-in PDF renderer (announcement).

Southern California Linux Expo (SCALE) 11x is held in Los Angeles, February 22 to 24 (LWN coverage).

RHEL 6.4 is released (LWN blurb).

The first preview images of Ubuntu for phones and tablets are released (LWN blurb; LWN article).

Ruby 2.0 is released (LWN blurb).

Subsurface 3.0 is released (announcement), thus hitting the 3.0 mark approximately 14 times faster than the co-authors' previous project to reach the milestone.

BIND 10 is released (announcement).

LG acquires webOS from Hewlett Packard (LWN blurb).

Debian releases its first arm64 image (LWN blurb); openSUSE follows with its own Aarch64 preview a week later (LWN blurb).

Linaro Connect Asia is held in Hong Kong, March 4 to 8 (LWN coverage).

Ubuntu announces the Mir display server, a move which is met with controversy over the decision to not adopt Wayland (LWN blurb; LWN article).

Google and the MPEG-LA announce an agreement that prevents the formation of a patent pool around WebM and VP8 (LWN blurb).

Google announces the Zopfli compression algorithm (LWN blurb).

0install 2.0 is released (announcement).

openSUSE 12.3 is released (announcement; LWN article).

Google announces the shutdown of its Google Reader feed-reading product to considerable weeping and gnashing of teeth (announcement).

The candidates for the Debian Project Leader (DPL) election are announced (announcement; LWN article).

Following the acquisition of the embedded Linux news site LinuxDevices, founder Rick Lehrbaum launches LinuxGizmos (LWN blurb).

Plasma Media Center 1.0 is released (LWN blurb).

Ardour 3.0 is released (LWN blurb).

Emacs 24.3 is released (LWN blurb).

PyCon 2013 is held in Santa Clara, March 17 to 21 (LWN coverage).

The Django community mourns the passing of longtime contributor Malcolm Tredinnick (LWN blurb).

MongoDB 2.4 is released (announcement).

The microblogging service Identi.ca, popular with the free-software community, closes its doors to be replaced by a new, decentralized platform called pump.io (LWN article).

GCC 4.8.0 is released (announcement; LWN article). The release is the first to be written in C++ (LWN article).

OpenSSH 6.2 is released (announcement).

GNOME 3.8 is released (announcement).

Comments (1 posted)