Development

Vim's 25th anniversary and the release of Vim 8

2016 was a big year for project anniversaries. The Linux kernel, of course, turned 25. And Vim, that other iconic text editor, also celebrated its 25th anniversary. Bram Moolenaar, Vim's benevolent dictator for life, marked the occasion at the end of last year with a short note on the project's web site:

Vim's ancestry goes back even further. Vim, or "Vi IMproved", was started as a clone of the venerable vi editor, which was created by Bill Joy in 1976. Vi has its own complicated lineage: vi was the visual mode of Joy's own ex editor, which in turn can be traced back to Ken Thompson's ed editor from Bell Labs. O'Reilly's Learning the vi and Vim Editors has a complete family tree of the vi editors. Vim's 25th anniversary also coincides with the official release of version 8 on September 12, 2016. Vim 8 is the first major release since the appearance of version 7 in May 2006. Even the last point release, 7.4, was released back in August 2013.

Moolenaar happened to mark the occasion with a presentation [YouTube] at Google's Zurich office in December. He talked about the project's history as well as some of the decisions that went into the latest release. Moolenaar acknowledged he's been slow making new releases, but he believes the deliberate pace has been good for the project: "In the beginning it was easier to add new features," he said. "The further you get the harder it is to add a feature that's actually adding something... I always hesitate to add new features unless it's clear that it's good."

At last, Vim 8 has arrived. The release contains several new features, including some functionality that users have been waiting to see appear in Vim.

Packages

Managing Vim extensions used to require a separate plugin (such as pathogen) to be able to load other plugins automatically when the editor starts. Vim can now do this on its own. Users just need to move their plugins into a directory with their other configuration files. Vim 8 defines a new variable packpath, which points to $HOME/.vim/pack/*/start. This is now the default plugin directory for Vim. Plugins that use this directory structure will automatically be loaded at startup.

The introduction of packpath also includes new commands to activate plugins manually after Vim starts:

• :packadd will add a plugin from packpath
• :packloadall will load all packages under packpath

Vim 8 mirrors the directory structure many Vim users are already accustomed to. Vim 8 does not offer a package manager, but only provides the built-in framework for users to drop in their own plugins. Extensions added to the packpath folder can be managed with version control alongside the rest of a user's configuration files.

Asynchronous plugins

Vim has always been a single-threaded application, which meant that calling an external command from within Vim would block the editor for the duration of the command. While the core editor remains single-threaded, some changes have been made to allow the editor to proceed while external commands are running. Vim 8 introduces changes to its scripting language that enable communication between the core editor and other processes along with changes to start and stop those processes. Although these features are primarily of interest to plugin developers, the addition of interprocess communication and job control will have direct benefits for Vim's end users.

For example, Vim already has a built-in make command. And it's also possible to call the operating system's external make command. But both of these uses block the editor until the command finishes.

Vim 8 allows plugin developers to asynchronously execute commands in a separate process and pass the results back to Vim upon completion. Plugins that make use of the new API will no longer block Vim during long computations. For example, this means that end users can now run make or grep commands from within Vim and still be able to continue using the editor while the background job runs.

Two new additions to Vimscript make this possible. Channels allow the editor to exchange messages with other processes running in the background. Messages passed over the channels are encoded in JSON. And the job control addition can be used to start and stop background processes and communicate with running processes through channels.

Existing plugins will need to be patched to take advantage of the new features. One way to try out the new API in Vim 8 is with the new AsyncRun plugin. Just drop it into your packpath directory and you can run any shell command as a background job and still capture the output within Vim.

With Vim 8, there are now three ways to run make as an external command:

• :make

  Run Vim's built-in make. This will block the editor.

• :!make

  Run the operating system's make command. Prepending ! to a command tells Vim to run a shell command. Even though make is running in a separate process, Vim is still blocked while it waits for make to return.

• :AsyncRun make

  Use the AsyncRun external plugin to run the operating system's make command. This is new in Vim 8 and requires a plugin that takes advantage of the job control API. This is just like :!make, but now Vim doesn't have to wait for the result of make to return.

The asynchronous features are only available to plugins and do not change the core editor. Adding thread support to Vim is challenging for historical reasons, Moolenaar said in his presentation at Google: "Vim is really single threaded. Nearly all the code depends on that. Making it multi-threaded would be a really big, difficult thing with a lot of bugs. It's a lot easier if you run something in a different process and then add interprocess communication."

User demand for a multi-threaded Vim (e.g. a patch for supporting multiple threads) and its rejection is what prompted the Neovim fork in January 2014. In his presentation, Moolenaar concedes that Neovim did create some pressure to add a way to handle asynchronous jobs. Ultimately, the new feature was a compromise to give plugin authors the features they wanted without having to change Vim's fundamental design.

"Did they [Neovim] influence Vim development? A little bit. They found some bugs and fixed them and then gave them back to me," Moolenaar said in the presentation. "Of course jobs and channels are one thing where plugin authors told me 'hey this thing is something Neovim is working on and we should really have it'. I didn't take the Neovim implementation because I didn't like it that much."

A few sundry additions

Users who are new to Vim will have a better out of box experience with Vim 8. If no configuration file is present, Vim now loads a default configuration with some friendlier settings. Vim 8 now starts with vi compatibility disabled. This ensures new users will have access to Vim's extended feature set and can start using plugins right out of the box.

The default settings also include:

• Command history and auto-completion
• Enabling syntax highlighting
• Automatic file-type detection
• Saving the file position on exit

In addition to asynchronous support, Vim 8 also introduces a few other additions to the Vimscript language. Vimscript already has references to functions. Vim extends this with partials, which allow a calling function to bind the arguments for a referenced function. This can be useful for callbacks on channels, for example. Vim also introduces lambdas to allow defining anonymous functions to pass as arguments. In addition, closures are now available so that lambda functions can use the variables in the scope in which they are defined.

Users who work inside Vim with split windows can now rely on window identifiers remaining consistent when new splits are added. In Vim 8, split windows now have a unique ID separate from their window number. The graphical version of Vim now supports GTK+ 3. Users in older desktop environments should not be impacted: Vim 8 will still fallback to GTK+ 2. And Windows users have gained support for DirectX.

Anyone who has waited too long to update their computer should be aware that Vim 8 removes support for several old platforms, including MS-DOS, Windows 95, and OS/2.

Don't forget to smile

Finally, users who are burned out after a long day of hacking should try out Vim's new :smile command. (Technically, this feature was added to Vim 7.4 on the last day of 2015.) The changelog for this feature simply states that "Vim users are not always happy.", so this patch was added as a solution to that problem.

More details can be found in the full list of changes for Vim 8. Vim's built in documentation includes more detailed instructions on how to use the new features, as well as lots of information for plugin authors on changes to the scripting language. You can get Vim directly from the project web site, although the latest version has already started appearing in the package managers of many distributions.

Comments (2 posted)

Development quotes of the week

— Ben Simon

— Peter Hutterer

Comments (4 posted)

KDE Plasma 5.9 released

The KDE project has announced the release of the Plasma 5.9 desktop environment with a number of new features. "Global Menus have returned. KDE's pioneering feature to separate the menu bar from the application window allows for new user interface paradigm with either a Plasma Widget showing the menu or neatly tucked away in the window bar."

Comments (22 posted)

KDE and Slimbook Release a Laptop for KDE Fans (KDE.News)

KDE has announced a partnership with Slimbook, a Spanish laptop retailer, to create the KDE Slimbook. "The KDE Slimbook allows KDE to offer our users a laptop which has been tested directly by KDE developers, on the exact same hardware and software configuration that the users get, and where any potential hardware-related issues have already been ironed out before a new version of our software is shipped to them. This gives our users the best possible way to experience our software, as well as increasing our reach: The easier it is to get our software into users' hands, the more it will be used." The laptop is available for pre-order with systems shipping mid-March.

Comments (18 posted)

Krita 3.1.2 released

Version 3.1.2 of the Krita painting application has been released. This version features audio support for animations along with other improvements and bug fixes. "Audio is not yet available in the Linux appimages. It is an experimental feature, with no guarantee that it works correctly yet — we need your feedback!"

Comments (none posted)

LibreOffice 5.3 released

Version 5.3 of the LibreOffice office suite is out. "LibreOffice 5.3 represents a significant step forward in the evolution of the software: it offers an introduction to new features such as online with collaborative editing, which increase the competitive positioning of the application, and at the same time provides incremental improvements, to make the program more reliable, interoperable and user friendly."

Full Story (comments: 5)

Time To Upgrade Your Python: TLS v1.2 Will Soon Be Mandatory

The Python Software Foundation has announced that python.org and related sites will begin disabling the old TLS versions 1.0 and 1.1. "This change was imposed on us by our content delivery network, Fastly, in response to a change imposed on them by the Payment Card Industry Security Standards Council. In order to continue serving websites that take credit card payments, Fastly is required to disable the old, insecure versions of TLS. Since the PSF's servers, including PyPI, use Fastly, the old versions of TLS will be disabled as well."

Comments (12 posted)

Xpra 1.0

Xpra has reached its 1.0 milestone release. "Xpra used to be "just" an X11 proxy, mostly used for (re)connecting to seamless X11 sessions running on remote hosts with good compression. Since then, the project has grown a lot and it supports new modes of operations like shadowing and full desktop (VNC like). Xpra is now used by many projects for securing / segregating applications on the same host using containers or VMs." The 1.0.x branch will be supported for a minimum of 2 years.

Full Story (comments: none)

Development newsletters

• Emacs News (January 30)
• These Weeks in Firefox (January 31)
• What's cooking in git.git (January 26)
• What's cooking in git.git (January 31)
• Koha Community Newsletter (January)
• OCaml Weekly News (January 31)
• OpenStack Developer Mailing List Digest (January 27)
• Perl Weekly (January 30)
• PostgreSQL Weekly News (January 29)
• Python Weekly (January 26)
• Ruby Weekly (January 26)
• This Week in Rust (January 31)
• Wikimedia Tech News (January 30)

Comments (none posted)

How to get up and running with sweet Orange Pi (Opensource.com)

David Egts reviews the Orange Pi at Opensource.com. "Compared to a $5 Raspberry Pi Zero, the Orange Pi Zero is only a few dollars more expensive, but it is much more useful out of the box because it has onboard Internet connectivity and four CPU cores instead of one. This onboard networking capability also makes the Orange Pi Zero a better gift than a Raspberry Pi Zero because the Raspberry Pi Zero needs Micro-USB-to-USB adapters and a Wi-Fi USB adapter to connect to the Internet. When giving IoT devices as gifts, you want the recipient to enjoy the product as quickly and easily as possible, instead of giving something incomplete that will just end up on a shelf."

Comments (11 posted)

The state of Jupyter (O'Reilly)

Here's an O'Reilly article describing the Jupyter project and what it has accomplished. "Project Jupyter aims to create an ecosystem of open source tools for interactive computation and data analysis, where the direct participation of humans in the computational loop—executing code to understand a problem and iteratively refine their approach—is the primary consideration."

Comments (37 posted)

An Interview with Krita Maintainer Boudewijn Rempt (Renderosity)

Renderosity Magazine talks with Boudewijn Rempt about the Krita painting application. "Well, we make Krita for artists who want to create images. It's not an image editor with a brush engine, it's really meant for sketching, painting, illustrating. So that is what we optimize the workflow for. And people tell us that that works very well for them!"

Comments (7 posted)