NixOS: purely functional system configuration management

System configuration management is a notoriously difficult task. Upgrading packages, editing configuration files, and so on; there will always come a time that it goes wrong. To mitigate this problem, Eelco Dolstra of Delft University of Technology invented another approach. He implemented a purely functional package manager called Nix, which means "nothing" in Dutch. Dolstra began his work on the Nix package manager as a part of his PhD research [PDF] at Utrecht University:

This operating system NixOS was the work of Armijn Hemel, who wrote a prototype for his Master's thesis. After this success, other developers joined. There are no official statistics of the number of users, but according to Dolstra the latest release of the Nix Packages collection had 19 contributors.

Imperative versus functional systems

To be able to grasp the basics of NixOS, we first have to distinguish between imperative systems and functional systems. Traditionally, software packages and configuration data (/bin and /etc, respectively) are imperative data structures. System administrators update them in-place with various administration commands, e.g. a RPM or APT package manager or a configuration tool such as Cfengine. This is analogous to how imperative programming languages such as C work. Each configuration action is stateful: it depends on the current state of the system and transforms this state. This has some fundamental consequences, including:

• No traceability: a specific configuration can generally not be recreated from scratch on a pristine system. That is, there may not be a record of the sequence of configuration actions over time. So it's not easy to reproduce a configuration.
• No predictability: if a configuration action acts upon an ill-defined state, the end result may be equally ill-defined and thus unpredictable.
• No rollbacks: if the user upgrades his system configuration (e.g. by upgrading a set of packages), this is a destructive process and undoing this is hard. Possible solutions are reverting to a backup or installing previous versions of the packages, but both solutions are error-prone.

In contrast, NixOS uses a functional approach, analogous to functional programming languages like Haskell. As Dolstra and Hemel state in their paper Purely Functional System Configuration Management [PDF]:

The functional approach has several advantages:

• Traceability: a configuration can be realised deterministically from a formal description and reproduced easily on another machine.
• Predictability: realisations of a configuration are not stateful, and hence upgrading a configuration is as safe as installing from scratch.
• Rollbacks: configuration changes are not destructive. As a result, the user is always able to roll back to a previous configuration.

How does NixOS work?

All this sounds exciting, but is it more than an academic exercise? How does NixOS work in practice? To put this to the test, your author downloaded the latest ISO of the installation CD. This CD contains a basic NixOS installation and doesn't do any installation preparation. So the user has to partition and format the drives himself and mount it on the target file system. The installation procedure is explained in an online manual.

The installation itself is uncommon, already showing signs of the functional nature of the operating system. The user has to write a description of the configuration to a file on the target file system. This file contains a 'Nix expression' that defines the root file system, kernel modules and services. Fortunately, the user can generate an initial configuration with the command nixos-hardware-scan. The nixos-install command reads this file and installs the system.

The result is a bare bones Linux distribution with the Nix package management system and the Upstart init system. The Nix package collection contains about 2200 software packages. That makes it a rather small distribution, but it is usable, as it contains server software such as Apache and SSH, and desktop software such as X.org 7.4, KDE 4.2, parts of Gnome, Firefox and more. However, the curious user will soon find other signs of the strangeness of this distribution. In the filesystem layout, for example: there's no /sbin, /usr/ or /lib in the filesystem. There's only one symlink in /bin, /bin/sh, because Glibc's system() function hard codes the location of the shell, as many other programs do. Even most files in /etc are symlinks.

All this is by design: to be able to work in a purely functional way, all static parts of the NixOS operating system are stored as immutable files in directories under /nix/store. Each package has a 'Nix expression', which is a function that builds and installs this package from source. The build scripts store the built packages in the Nix store. Each package is stored in a directory with a name that begins with a 160-bit cryptographic hash of all inputs involved in building the package, for example 22bharrqlcisnwa11a5qr0xazgvv64hk-firefox-3.5b4. This means that any change to an input value causes the package to be rebuilt in a different path, which has as a side effect that previous versions of the package are left untouched. Input values include the sources of the package, the build script, any arguments or environment variables passed to the build script, and build time dependencies. Each package directory contains bin, lib, man, and other sub-directories for the package and is read-only.

The same scheme works for system configuration files and control scripts. So the system has Nix expressions for sshd_config, to build the Linux kernel, to build initrd, for boot scripts, etc. There's also a top-level Nix expression, system.nix, that builds the entire system configuration by calling all expressions. The output is an activation script that can be executed to make this configuration the current configuration of the system. For example, it modifies the GRUB boot menu to boot the system with the new configuration. Previous configurations are retained in the boot menu to roll back. The nixos-rebuild switch command builds system.nix, makes it the default configuration in the GRUB boot menu and calls the activation script.

By not storing components such as libraries, header files or programs in global locations, all packages are forced at build time to use a specific version of their dependencies, located in the Nix store. To make this happen, the developers have patched Glibc, GCC and the dynamic linker ld to not search files in any default locations. So if a dynamic library is not explicitly declared with its full path in an executable, the dynamic linker will not find it. This also means that if the developer fails to specify a dependency explicitly in the Nix expression language, the package will fail deterministically, even if the dependency already happens to be available in the Nix store.

Advantages and disadvantages

Fortunately the user doesn't have to know about the Nix store. The user doesn't have to type /nix/store/22bharrqlcisnwa11a5qr0xazgvv64hk-firefox-3.5b4/bin/firefox to start their favorite browser. Nix creates directory trees of symlinks to all activated components and calls them user environments, which also reside in the Nix store. After each Nix package action, such as an install or a rollback, a new generation is made, which is a symlink outside the store that points to a user environment in the store. All generations of a user are grouped together in a profile. The user's current profile is pointed to by the symlink ~/.nix-profile. Putting the directory ~/.nix-profile/bin in the user's PATH environment variable completes the picture. As a consequence, non-privileged users can also securely install software. If a user installs a package that another user has already installed previously, the package won't be built or downloaded a second time.

Apart from the obvious advantages of predictability and rollbacks, it's even possible to copy a package from one machine to another in NixOS. The command nix-copy-closure copies a Nix store path along with all its dependencies to or from another machine via the ssh protocol. It does this efficiently, because it doesn't copy store paths that are already present on the target machine. This makes Linux packages literally "portable".

One major drawback to the functional model is that it requires significant disk space. This is understandable as each time the user makes a change to his configuration a new package will be added without overwriting the older one. In the worst case disk space doubles, for example when the C library or compiler is changed, propagating to all other packages. Even if the user removes a component, Nix doesn't actually remove the component from the Nix store, because it might still be in use by another user's environment or be a dependency of another component. Moreover, if the component were removed, it would no longer be possible to perform a rollback. However, the user can always remove any old generations of his profile by nix-env --remove-generations old and then he can execute a garbage collection of all unneeded components with nix-store --gc. If your hardware is not supported out-of-the box, it can be a challenge to write the correct Nix expression to load the firmware. There's also no distinction between a stable and an unstable branch, so things tend to break now and then. Fortunately, a broken system can be rolled back easily.

One final apparent drawback is that the NixOS directory structure doesn't comply with the Linux Standard Base. However, according to Dolstra this is not such a big problem as it seems:

The future: declarative specifications of networks

Of course the work on NixOS is ongoing. The developers are currently working on a branch named modular-nixos to make it easier to extend NixOS with hardware support or system services. There's also a research project about distributed deployment. Dolstra explains this:

Conclusion

The purely functional model of Nix and the cryptographic hashing scheme of the Nix store give the user some important features that are lacking in most Linux distributions. It makes one wonder why enterprise Linux distributions haven't picked up this approach (or a more LSB-compliant version of it). A drawback is the amount of disk space and bandwidth used when upgrading a fundamental dependency. Perhaps the most pressing issue is that it requires a radically different mindset from the user.