Distributions

News and Editorials

NixOS: purely functional system configuration management

System configuration management is a notoriously difficult task. Upgrading packages, editing configuration files, and so on; there will always come a time that it goes wrong. To mitigate this problem, Eelco Dolstra of Delft University of Technology invented another approach. He implemented a purely functional package manager called Nix, which means "nothing" in Dutch. Dolstra began his work on the Nix package manager as a part of his PhD research [PDF] at Utrecht University:

This operating system NixOS was the work of Armijn Hemel, who wrote a prototype for his Master's thesis. After this success, other developers joined. There are no official statistics of the number of users, but according to Dolstra the latest release of the Nix Packages collection had 19 contributors.

Imperative versus functional systems

To be able to grasp the basics of NixOS, we first have to distinguish between imperative systems and functional systems. Traditionally, software packages and configuration data (/bin and /etc, respectively) are imperative data structures. System administrators update them in-place with various administration commands, e.g. a RPM or APT package manager or a configuration tool such as Cfengine. This is analogous to how imperative programming languages such as C work. Each configuration action is stateful: it depends on the current state of the system and transforms this state. This has some fundamental consequences, including:

• No traceability: a specific configuration can generally not be recreated from scratch on a pristine system. That is, there may not be a record of the sequence of configuration actions over time. So it's not easy to reproduce a configuration.
• No predictability: if a configuration action acts upon an ill-defined state, the end result may be equally ill-defined and thus unpredictable.
• No rollbacks: if the user upgrades his system configuration (e.g. by upgrading a set of packages), this is a destructive process and undoing this is hard. Possible solutions are reverting to a backup or installing previous versions of the packages, but both solutions are error-prone.

In contrast, NixOS uses a functional approach, analogous to functional programming languages like Haskell. As Dolstra and Hemel state in their paper Purely Functional System Configuration Management [PDF]:

The functional approach has several advantages:

• Traceability: a configuration can be realised deterministically from a formal description and reproduced easily on another machine.
• Predictability: realisations of a configuration are not stateful, and hence upgrading a configuration is as safe as installing from scratch.
• Rollbacks: configuration changes are not destructive. As a result, the user is always able to roll back to a previous configuration.

How does NixOS work?

All this sounds exciting, but is it more than an academic exercise? How does NixOS work in practice? To put this to the test, your author downloaded the latest ISO of the installation CD. This CD contains a basic NixOS installation and doesn't do any installation preparation. So the user has to partition and format the drives himself and mount it on the target file system. The installation procedure is explained in an online manual.

The installation itself is uncommon, already showing signs of the functional nature of the operating system. The user has to write a description of the configuration to a file on the target file system. This file contains a 'Nix expression' that defines the root file system, kernel modules and services. Fortunately, the user can generate an initial configuration with the command nixos-hardware-scan. The nixos-install command reads this file and installs the system.

The result is a bare bones Linux distribution with the Nix package management system and the Upstart init system. The Nix package collection contains about 2200 software packages. That makes it a rather small distribution, but it is usable, as it contains server software such as Apache and SSH, and desktop software such as X.org 7.4, KDE 4.2, parts of Gnome, Firefox and more. However, the curious user will soon find other signs of the strangeness of this distribution. In the filesystem layout, for example: there's no /sbin, /usr/ or /lib in the filesystem. There's only one symlink in /bin, /bin/sh, because Glibc's system() function hard codes the location of the shell, as many other programs do. Even most files in /etc are symlinks.

All this is by design: to be able to work in a purely functional way, all static parts of the NixOS operating system are stored as immutable files in directories under /nix/store. Each package has a 'Nix expression', which is a function that builds and installs this package from source. The build scripts store the built packages in the Nix store. Each package is stored in a directory with a name that begins with a 160-bit cryptographic hash of all inputs involved in building the package, for example 22bharrqlcisnwa11a5qr0xazgvv64hk-firefox-3.5b4. This means that any change to an input value causes the package to be rebuilt in a different path, which has as a side effect that previous versions of the package are left untouched. Input values include the sources of the package, the build script, any arguments or environment variables passed to the build script, and build time dependencies. Each package directory contains bin, lib, man, and other sub-directories for the package and is read-only.

The same scheme works for system configuration files and control scripts. So the system has Nix expressions for sshd_config, to build the Linux kernel, to build initrd, for boot scripts, etc. There's also a top-level Nix expression, system.nix, that builds the entire system configuration by calling all expressions. The output is an activation script that can be executed to make this configuration the current configuration of the system. For example, it modifies the GRUB boot menu to boot the system with the new configuration. Previous configurations are retained in the boot menu to roll back. The nixos-rebuild switch command builds system.nix, makes it the default configuration in the GRUB boot menu and calls the activation script.

By not storing components such as libraries, header files or programs in global locations, all packages are forced at build time to use a specific version of their dependencies, located in the Nix store. To make this happen, the developers have patched Glibc, GCC and the dynamic linker ld to not search files in any default locations. So if a dynamic library is not explicitly declared with its full path in an executable, the dynamic linker will not find it. This also means that if the developer fails to specify a dependency explicitly in the Nix expression language, the package will fail deterministically, even if the dependency already happens to be available in the Nix store.

Advantages and disadvantages

Fortunately the user doesn't have to know about the Nix store. The user doesn't have to type /nix/store/22bharrqlcisnwa11a5qr0xazgvv64hk-firefox-3.5b4/bin/firefox to start their favorite browser. Nix creates directory trees of symlinks to all activated components and calls them user environments, which also reside in the Nix store. After each Nix package action, such as an install or a rollback, a new generation is made, which is a symlink outside the store that points to a user environment in the store. All generations of a user are grouped together in a profile. The user's current profile is pointed to by the symlink ~/.nix-profile. Putting the directory ~/.nix-profile/bin in the user's PATH environment variable completes the picture. As a consequence, non-privileged users can also securely install software. If a user installs a package that another user has already installed previously, the package won't be built or downloaded a second time.

Apart from the obvious advantages of predictability and rollbacks, it's even possible to copy a package from one machine to another in NixOS. The command nix-copy-closure copies a Nix store path along with all its dependencies to or from another machine via the ssh protocol. It does this efficiently, because it doesn't copy store paths that are already present on the target machine. This makes Linux packages literally "portable".

One major drawback to the functional model is that it requires significant disk space. This is understandable as each time the user makes a change to his configuration a new package will be added without overwriting the older one. In the worst case disk space doubles, for example when the C library or compiler is changed, propagating to all other packages. Even if the user removes a component, Nix doesn't actually remove the component from the Nix store, because it might still be in use by another user's environment or be a dependency of another component. Moreover, if the component were removed, it would no longer be possible to perform a rollback. However, the user can always remove any old generations of his profile by nix-env --remove-generations old and then he can execute a garbage collection of all unneeded components with nix-store --gc. If your hardware is not supported out-of-the box, it can be a challenge to write the correct Nix expression to load the firmware. There's also no distinction between a stable and an unstable branch, so things tend to break now and then. Fortunately, a broken system can be rolled back easily.

One final apparent drawback is that the NixOS directory structure doesn't comply with the Linux Standard Base. However, according to Dolstra this is not such a big problem as it seems:

The future: declarative specifications of networks

Of course the work on NixOS is ongoing. The developers are currently working on a branch named modular-nixos to make it easier to extend NixOS with hardware support or system services. There's also a research project about distributed deployment. Dolstra explains this:

Conclusion

The purely functional model of Nix and the cryptographic hashing scheme of the Nix store give the user some important features that are lacking in most Linux distributions. It makes one wonder why enterprise Linux distributions haven't picked up this approach (or a more LSB-compliant version of it). A drawback is the amount of disk space and bandwidth used when upgrading a fundamental dependency. Perhaps the most pressing issue is that it requires a radically different mindset from the user.

Comments (15 posted)

New Releases

Karmic Alpha 2 released

The second alpha of Ubuntu Karmic Koala (9.10) is out. Kubuntu and Xubuntu variants are also available. "Alpha 2 includes a number of software updates that are ready for large-scale testing. Please refer to http://www.ubuntu.com/testing/karmic/alpha2 for information on changes in Ubuntu."

Full Story (comments: 1)

Fedora Electronic Lab 11 Leonidas Released

The Fedora Electronic Lab (FEL) is an official Fedora spin. The fourth consecutive release of FEL (Fedora 11 version) is now available. "Fedora Electronic Lab 11 Leonidas provides a vibrant environment for designing and simulating ASIC design and embedded design. The opensource EDA solutions are composed to satify high-end mixed-signal hardware design flows from design specification to final project handoff. This release comprises Perl modules to facilitate both design, HDL code generation and brings additional support for Engineering Change Order (ECO). After post chip fabrication, evaluation boards of those chips can also be designed."

Full Story (comments: none)

Announcing LXDE Fedora Remix 11

A community remix of Fedora 11 featuring the LDXE desktop is available. "It is a Live CD with LXDE as the default desktop environment. LXDE is a new light weight desktop environment. Refer to http://lxde.org/ for more details. This Live CD is only available for Intel compatible 32-bit systems currently."

Full Story (comments: none)

Distribution News

Fedora

Fedora 9 End Of Life (EOL)

Now that Fedora 11 is out the end-of-life for Fedora 9 is in sight. Specifically support for F9 ends July 10, 2009. "With the release of Fedora 11 now past us, it's come time to remind folks that per the release policy, maintenance for the N-2 Fedora release ends one month after the Fedora N comes out."

Full Story (comments: 3)

Fedora Board Recap 2009-06-11

Click below for a brief recap of the June 11, 2009 meeting of the Fedora Advisory Board. Topics include What is Fedora, Status of sponsorship work, Fedora 11 Release Follow-up, Security notification plan, PPC as a secondary arch, Toxicity--follow-ups, and Security notification plan.

Full Story (comments: none)

Slackware Linux

Slackware X.org overhaul

Slackware current has gotten a major overhaul of X.Org, according to the June 10 changelog entry. "This batch of updates includes a major overhaul of X.Org -- thanks to Robby Workman for doing a huge amount of work updating the build scripts and testing everything. A large number of packages were recompiled or upgraded to drop references to the now-obsolete libxcb-xlib and libXaw8 libraries. We have workarounds in place for old binaries so it wasn't strictly required, but recompiling anyway gives a cleaner system. Enjoy!"

Full Story (comments: none)

SUSE Linux and openSUSE

Stephen Shaw has been appointed as openSUSE Board Member

The openSUSE Board has announced that Stephen Shaw is its newest member. "Federico Mena-Quintero has ceded his seat on the openSUSE Board. We are all sad to see him go, but know we are grateful for his continued and ongoing contributions to the openSUSE Project."

Comments (none posted)

Ubuntu family

Ubuntu archive reorganisation: summary of pending work

Ubuntu is planning on a major reorganization of its repositories. "We divided the archive into components before there was even such a thing as Kubuntu, let alone the rich landscape of Ubuntu derivatives that exists today. The modern Ubuntu community has teams of developers working on everything from mobile devices through video editing to cloud computing. It is no longer clear to users which packages are recommended for and supported on which flavours of Ubuntu, and the simple division of the development team into ubuntu-core-dev and MOTU [Masters of the Universe] is looking increasingly artificial and does not do a good job of modelling how our development community really works. To fix these problems, we need to evolve our original design into a more fine-grained system."

Full Story (comments: none)

New Distributions

TITAN LEV

TITAN LEV is a Linux desktop OS for users with Windows experience, from the Israeli company Affordy. It comes bundled with 150 applications and supports English, Russian, Polish, Spanish, Portuguese, Hebrew and Arabic. (Found on blogs.zdnet)

Comments (none posted)

Distribution Newsletters

CentOS Pulse #0902

The CentOS Pulse for June 16, 2009 is out. "This issue of CentOS Pulse looks at the Artwork SIG, desktop multimedia capabilities and interesting threads in the CentOS community. An overview of the important security updates and upcoming events are included as well."

Comments (none posted)

DistroWatch Weekly, Issue 307

The DistroWatch Weekly for June 15, 2009 is out. "The delayed Fedora 11 was finally released last week. Does the new version of the popular distribution live up its standards? Did the delay help to squash all the bugs? And how does it fare in comparison with other desktop Linux products? Read our first-look review to find out. This week also sees the release of a new project to create more up-to-date installation media for FreeBSD. Currently shipping a 32-bit Xfce desktop, the project hopes to expand to many other areas, as needed. Meanwhile Fedora's Leonidas release is in full swing, but some users are encountering an issue when installing via the live CD as the system cannot yet boot from the default ext4 file system. Read on to discover the simple fix! Also, Debian derivative distribution sidux has copped some heat over its decision to remove non-free firmware from its 2.6.30 kernel, while Novell gets its users to help advertise their products with an online "Custom Geeko" creation tool. Finally, don't miss the freshly posted development roadmaps for Mandriva Linux 2010 and Fedora 12. Happy reading!"

Comments (none posted)

Fedora Weekly News 180

The Fedora Weekly News for June 15, 2009 is out. "In this week's issue, we open with useful links announcing the 'hot-off-the-bitpress' Fedora 11 (Leonidas) release, and also reminders about voting for the code name for Fedora 12 and other Fedora elections. There are many FUDCons, FADs and other Fedora events, helpfully listed as well. From Planet Fedora, two interesting samples: news from Fedora blogs and contributors including an interview with Eric Sandeen about ext4, linux filesystems and Fedora 11, and rave reviews on Presto, Fedora 11's enhanced DeltaRPM service that can be configured. In the Quality Assurance beat, review of the Bugzappers weekly meeting and changes this will have to triage work, as well as availability of a set of Fedora 11 delta ISO images." ...and much more.

Full Story (comments: none)

The Mint Newsletter - issue 86

This issue of the Mint Newsletter covers the release of Linux Mint 7 x64 RC1, a call for promotional materials, and more.

Comments (none posted)

OpenSUSE Weekly News/76

This issue of the OpenSUSE Weekly News covers OpenSUSE at the Southeast Linuxfest, Status Reports from Google Summer of Code, susegeek : openSUSE 11.2 Milestone 2 step by step install procedure, OpenSUSE-EDU, and more.

Comments (none posted)

Ubuntu Weekly News #146

The Ubuntu Weekly News for June 14, 2009 is out. "In this issue we cover SanDisk collaborates to improve Ubuntu netbook SSD performance, MOTU Council Results, Ubuntu Stats, Calling all LoCo Teams!, In the Press & Blogosphere, Upcoming Meetings & Events, Updates & Security, and much, much more!"

Full Story (comments: none)

Newsletters and articles of interest

Ubuntu aims for ten-second boot time with 10.04 (ars Technica)

ars Technica reports on efforts being made to reduce the boot time of Ubuntu. "In a presentation at the Ubuntu Developer Summit in Barcelona, developer Scott James Remnant noted that boot time decreased from 65 seconds in version 8.10 to only 25 seconds in 9.04. This is already a substantial improvement, but he believes that there is still room for more aggressive optimization. Canonical, the company behind Ubuntu, will continue pushing the limits of boot performance during the upcoming development cycle for Ubuntu 9.10, which is codenamed Karmic Koala. According to Remnant, the company aims to achieve a ten-second boot time next year for Ubuntu 10.04, the release that will follow after Karmic."

Comments (68 posted)

Distribution reviews

What's new in Fedora 11 (The H Open)

Thorsten Leemhuis reviews Fedora 11. "It's not just the new design and updated software that brings a sparkle to the eleventh version of Fedora (Leonidas), there are also a whole raft of technical enhancements. Fedora once again finds itself in the vanguard – expect to see many of these changes coming to other Linux distributions in the near future."

Comments (none posted)

Hands-on: new Fedora release goes up to 11 but doesn't rock (Ars Technica)

Ryan Paul reviews Fedora 11. "Fedora tends to ride the cutting edge of desktop Linux technology and is often the first distribution to include new upstream features. This makes Fedora a compelling choice for developers and Linux enthusiasts who like to stay ahead of the curve. The downside of dogfooding the latest stuff is that users occasionally have to contend with a release that is slightly undercooked. Fedora 11 isn't raw but it falls short of well done."

Comments (none posted)

Page editor: Rebecca Sobol
Next page: Development>>