JupyterLab: ready for users

In the recent article about Jupyter and its notebooks, we mentioned that a new interface, called JupyterLab, existed in what its developers described as an "early preview" stage. About two weeks after that article appeared, Project Jupyter made a significant announcement: JupyterLab is "ready for users". Users will find a more integrated environment for scientific computation that is also more easily extended. JupyterLab takes the Jupyter Notebook to a level of functionality that will propel it well into the next decade—and beyond.

While JupyterLab is still in beta, it is stable and functional enough to be used in daily work, and steadily approaching a 1.0 release. From the point of view of developers working on extensions or other projects that use the JupyterLab API, however, the beta status serves as a caution that its developer interfaces are still in flux; they should plan for the possibility of breaking changes.

JupyterLab arose in 2015 from the desire to incorporate the "classic" (as it is known now) Jupyter Notebook into something more like an integrated development environment running in the browser. In addition, the user was to have the ability to extend the environment by creating new components that could interact with each other and with the existing ones. The 2011 web technology that the Jupyter Notebook was built upon was not quite up to this task. Although existing JavaScript libraries, such as React, suggested a way forward, none of them had the power and flexibility, particularly in the area of interprocess communication, that was required. The JupyterLab team addressed this problem by developing a new JavaScript framework called PhosphorJS. JupyterLab and PhosphorJS are co-developed, with capabilities added to the JavaScript framework as they are needed for JupyterLab.

One drawback to the new, rewritten Notebook framework is that, while it is mostly backward compatible with existing Jupyter Notebooks, most JavaScript extensions will not work. This is because the Notebook component of JupyterLab has a new and incompatible extension mechanism, based on TypeScript, which is a superset of JavaScript that features optional type annotations and compiles to JavaScript. However, some JavaScript extensions, such as the interactive widgets that were demonstrated in the previous article, already have JupyterLab versions; note that Node.js is required to install them.

For those interested in more backstory, Karlijn Willems provides a good rundown of the history of the Jupyter Notebook and its relationship with IPython and similar notebook projects, from both the commercial and free-software worlds.

Using JupyterLab

Let's go through a brief rundown of JupyterLab's main features, to provide an idea of what it's like to use the new environment. Start JupyterLab by typing jupyter lab at the command line on a system with it installed (more about that below). If you do this from within a directory that contains an existing Jupyter (or JupyterLab) Notebook, JupyterLab will make it convenient to open the Notebook using the file browser. A new tab or window should open in the default browser (recent versions of Firefox, Chrome, and Safari are supported) that contains the JupyterLab environment.

If this is the first time starting JupyterLab, the window will appear as in the figure above, with a file browser on the left and the "launcher" on the right. The launcher's contents depend on what kernels are installed. Kernels provide the interface between Jupyter (or JupyterLab) and various programming languages, such as Julia, R, Go, and, of course, Python.

Besides the Python 3 and Python 2 kernels, I have a gnuplot kernel installed. I had been using this kernel with the Jupyter Notebook, and it works flawlessly with JupyterLab.

As can be seen, the launcher allows opening notebooks or consoles for any of the installed kernels. The consoles are REPLs (read-eval-print loops) attached to the given kernels, but gain more power, as we shall see, from their incorporation into the JupyterLab environment, because they can communicate with its other components. You can also launch a terminal, which behaves just like a normal xterm but is embedded in the browser. The "Text Editor" button opens an editor that can have various key-bindings, including Vim and Emacs, applied using the Settings menu. These are not actual instances of Vim, Emacs, etc., but can be useful for simple editing tasks; as is the case with the other components, the editor gains additional usefulness due to its integration with the JupyterLab system.

New notebook features

Opening a notebook presents a familiar vista if you've previously used Jupyter. The JupyterLab Notebook looks and behaves just as the classic notebook, but is more convenient to use. Now cells can be collapsed and expanded individually with a mouse click, or in groups with a selection from the View menu. This menu also has items to collapse or expand all code or all output cells. You can rearrange the order of cells by dragging them with a mouse, or drag cells between notebooks (dragging an input cell takes its associated output cell along with it). The classic notebook required selection, copy, and paste to get the same results.

Copying cells between notebooks is made practical by the ability to arrange components in almost any pattern within the browser window by dragging the component tabs. Notebooks, editors, consoles, and all other components can be arranged horizontally or vertically in any way that is convenient, and resized as well. This allows you, for example, to use one notebook freely as a scratchpad while dragging selected cells to another notebook beside it to construct an orderly narrative. I found the graphical interface for arranging components to be fluid and responsive, with no glitches or delays in redrawing the browser window.

The figure above shows an example of tiling the workspace with a notebook, a terminal session running the process monitor htop, and a console running IPython. The console is running an instance of IPython separate from the IPython kernel backing the open notebook. This is made clear in the area on the left, where the Running tab is selected to show all the kernel and terminal sessions backing the components in the workspace. The components can be shut down from the Running tab as well. (You can also quit the browser while keeping kernels running, and reattach to them later, which can be a great convenience.) The left-hand area can be collapsed to get more working space, and any component can be temporarily set to a quasi-full-screen mode by hitting ctrl-shift-return while it has the focus (the same key combination restores the tab arrangement).

You can also create a second, synchronized view of a notebook using the context menu (traditionally accessed with a mouse right-click). By placing the two views side-by-side, you can have access to different parts of a single notebook simultaneously. A related trick is using a cell's context menu to create a new view for the output just of that cell. For certain types of output, this allows you to interact with its representation in a separate panel, while preserving the embedded notebook version unchanged; for example, you can pan and zoom a map, or rotate a 3D model.

A quirk of both the old and new versions of the Notebook is that its cells can be executed in any order, leaving no explicit record of what that order happened to be. Also, there is normally no way to interact with the notebook's kernel, say to check the values of variables, without creating or changing notebook cells. JupyterLab allows you to address these issues by attaching a console to the same kernel backing a notebook, using another context menu, "New Console for Notebook". Now any time a cell is executed, the calculation is recorded in the console, and you can turn to the console to check variables and perform calculations that will not be recorded in the notebook.

File viewers

The first figure above shows that JupyterLab uses special icons in the file viewer for the file types that it knows about, which it identifies from the filename extensions. Aside from the notebook files, JupyterLab has special viewers for a handful of file types, with more likely to be added in short order. In this article we talk about two of these, Markdown and comma separated value (CSV) files (which can actually be tab or semicolon separated, but still need to have the .csv extension).

CSV files provide a good introduction to a concept central to understanding JupyterLab: different views into the same file. The figure above shows the result of opening two views onto the same million-line by six-column CSV file; on the left, an editor, and, on the right, a table viewer. If you change values using the editor view, they are reflected in the table view (after a short delay for this particular file). The table view is high-performance, able to scroll through the file with no hesitation, jumping from the top to the bottom as fast as you can move the mouse. The columns can be resized manually, which is also a smooth and instantaneous operation.

However, some glitches appeared on my machine: the editor view became corrupted when I attempted to scroll to the bottom. While some JupyterLab presentations have shown table views of files with billions and even trillions of rows, I found that attempting to view files with more than about 10 million rows crashed my browser. This may be because I was conducting these experiments on a laptop with "only" 4GB of RAM; it probably indicates that there are still problems in the CSV viewers that need to be addressed, as well.

Another example of multiple views into the same file is JupyterLab's handling of Markdown files. The figure below shows two views of the same Markdown file, an editor on the left and the formatted output on the right. Editing the file produces a fairly rapid result on the right-hand side, which is called the "Markdown preview" in JupyterLab terminology. Observe the equation, which is typeset using MathJax.

Components can be combined in various other ways. For example, you can attach a kernel to an editor, with another quick selection in a context menu. This opens a console with a REPL talking to the selected kernel. You can then select blocks of code in the editor and hit shift-enter, just as when executing a cell in a notebook. The code is copied to the console and executed in the REPL. If you happen to be editing a Markdown file, code blocks are recognized, so you merely need to have the cursor within a block of code, with no need to select it.

JupyterLab is designed to be extended; new components and file viewers are already appearing. Some examples are a graphical diagram editor and a LaTeX editor with a live preview.

JupyterLab may remind some of the Apple technology from the 1990s called OpenDoc, and similar efforts, which also allowed the user to arrange sundry disparate components within a single frame, each with its own editor and viewer. To the chagrin of many, this was one of several promising technologies killed off by Steve Jobs upon his return to Apple.

Installation

It is unlikely that your distribution's package-management system provides a recent enough version of JupyterLab; it may also not be sufficiently up to date with the Jupyter Notebook prerequisite. The two main ways of installing this software are to either use the pip command or to download the Anaconda distribution. In keeping with the latest fashion, the latter has its own package-management system; install JupyterLab using the conda installer. You will need some of the components installed with the Jupyter Notebook from version 5.3 or newer. Of course, you also need Python, NumPy (required for all numerical or scientific work with Python), and you may want to install SciPy if you don't already have it.

A 1.0 release of JupyterLab is planned for later this year; shortly after that, development work on the classic Jupyter Notebook will dwindle; it will be deprecated in favor of JupyterLab. Hence if you plan to work with the Notebook for some time, you may be compelled, as a practical matter, to switch to JupyterLab eventually.

The Jupyter Notebook has already won over many scientists and educators because of the ease with which it allows one to explore, experiment, and share. JupyterLab makes the Notebook part of a more complete, powerful, and extensible environment for pursuing computational science and disseminating the results, leaving little doubt that this free-software project will win over an even larger portion of the scientific community. I've tried to give some idea of the power and convenience of the JupyterLab interface, but to really appreciate this technology, you need to try it out yourself. Fortunately, this is easy to do, as it's simple to install and intuitive enough to get started without reading documentation—and it happens to be a great deal of fun.