Nexenta Core Platform 2: OpenSolaris for human beings

Nexenta Core Platform is an operating system that ties the OpenSolaris kernel to a Ubuntu user space. The project emerged just a couple of months after Sun Microsystems released the OpenSolaris code in June 2005, and it was first called Gnusolaris, a direct port of Desktop Ubuntu to the OpenSolaris kernel. The developers soon realized that supporting a desktop distribution was too difficult due to the large number of support requests and the huge list of packages to maintain. So the Nexenta developers refocused their effort onto a "core" distribution, leaving the desktop integration to others.

In February 2008, these developers released Nexenta Core Platform 1.0, a server system based on OpenSolaris build 85 and Ubuntu Dapper Drake (6.06 LTS). Now, the next version is available: Nexenta Core Platform 2.0, based on OpenSolaris build 104 and Ubuntu Hardy Heron (8.04 LTS). Over 13,000 packages are in the repository, which is a lot more than in OpenSolaris and around half of Ubuntu's packages. Nexenta Core is also the base of NexentaStor, a NAS storage solution developed by Nexenta Systems.

Installing Nexenta

Nexenta's installer is rather basic. The disk is partitioned automatically. If the user selects two or more equal-sized disks, the installer configures them as mirrored ZFS disks. The automatic partitioning uses the whole disk for a root and a swap partition. At the moment, it's not possible to choose another partitioning layout, e.g. for a dual boot configuration, but some users have reportedly been able to do it by hacking the installer script. As a more flexible partitioning support is requested frequently by users, Nexenta community leader Anil Gulecha promises this will be one of the enhancements in the next release.

After the system is installed and the user has logged in, it looks like a normal Ubuntu Server system, using around 800 MB disk space. After an "apt-get update", the "apt-cache show" command shows over 13,000 available packages to install. Although Nexenta positions itself as a file server and NAS platform, much of these packages are graphical programs. X.Org can be installed, as well as Firefox, Xfce 4 and most of Gnome. All packages are the versions in Ubuntu 8.04. The Nexenta project has set up its own repository on apt.nexenta.org and there are some mirrors available.

OpenSolaris goodies

The fact that Nexenta is using the OpenSolaris kernel means that hardware support depends on the drivers available in OpenSolaris. This is a domain where Linux distributions clearly have a big advantage over Nexenta. The best way to find out whether specific hardware components are supported by Nexenta is by searching the Solaris Hardware Compatibility List, however there are a few drivers in Nexenta that are not part of OpenSolaris. Nexenta's stable device driver interface does arguably provide an advantage over Linux in that Nexenta users can still run ten-year-old Solaris drivers.

Nexenta inherits other interesting features of Solaris, such as the operating system-level virtualization zones, a feature resembling Linux-VServer but with the advantage that it's not a separate patch set but supported in the official kernel. Applications within a zone appear to be running on a standalone system. Processes in different zones are completely isolated from each other. But although each zone appears as a standalone operating system, in reality a single instance of the OpenSolaris kernel is running, which means that zones are lightweight. Nexenta has full support for zones.

Transactional upgrades

Another powerful feature that Nexenta inherits from the OpenSolaris kernel is the ZFS file system; the Nexenta developers have built some innovative tools around it. With the apt-clone tool, the user can upgrade or install packages with the capability of an easy rollback if the install goes wrong. For example, with a simple "apt-clone dist-upgrade" command, the system will be upgraded, but not before a checkpoint is made. If the upgrade returns an error, the user has the option to rollback the changes to the checkpoint. So, in a few seconds, the system is returned exactly to its previous state, without a reboot. These upgrades are called transactional ZFS upgrades.

Even if an upgrade is successful, the user can decide at any time to revert to the previous state. After a successful upgrade, a new checkpoint is made. The user then has the option to reboot and activate the checkpoint in GRUB or first activate the newly created checkpoint and then reboot. If the user decides later that he doesn't like or need the upgrade, he reboots and selects the previous checkpoint in GRUB, after which he can revert to the previous state. The new checkpoint is then deleted.

This checkpoint system is not only usable for upgrades, but also for installing applications. Think about an application with rather intrusive effects on the system, such as the Apache web server. A simple command creates a checkpoint and installs Apache 2. If the user later decides that he doesn't want Apache or if he has made a mess of the configuration, he can activate the previous checkpoint and then reboot to revert the system to the state before Apache's install.

GNU and not GNU

The default behavior of Nexenta is to prefer GNU utilities, which are installed in /usr/bin and /usr/sbin and so on. The Sun versions of these utilities are installed in /usr/sun/bin and /usr/sun/sbin. Nexenta uses a trick to be able to switch between a GNU and a SUN personality: if the environment variable SUN_PERSONALITY is set to one, the search paths /usr/sun/bin and /usr/sun/sbin take preference, even if the user executes the commands explicitly by their absolute path, e.g. /usr/bin/sed. This ensures that Solaris-based scripts work in Nexenta without modifications. Nexenta also uses this functionality in its SVR4 package commands. They can be used to install native Solaris packages in SVR4 format, calling alien to convert the package on-the-fly to a Debian package.

Although most utilities are GNU ones, a couple of basic commands like ps and top are the OpenSolaris versions, which is confusing but understandable as they are tied closely to the OpenSolaris kernel. This confusing mix between GNU and Sun comes back in other domains. For example, the user space is almost completely built using GCC. In contrast, the OpenSolaris side, which consists of a couple of hundred packages, is built using Sun Studio. Gulecha explains this decision:

A difference in approach between Debian GNU/kFreeBSD and Nexenta is that the former is using a port of GNU libc, while the latter is using Sun libc. However, there's also a Debian GNU/kOpenSolaris port in the pipeline, which uses a port of GNU libc to the OpenSolaris kernel. The Nexenta developers seem to welcome this development and are even thinking about two architectures: NexentaCore Sun/OpenSolaris and NexentaCore GNU/kOpenSolaris. The Nexenta project actually started as an attempt to port glibc to OpenSolaris, but when the developers realized the complexity of the task, they decided to use Sun libc.

Porting packages to Nexenta

The Nexenta developers would like to make Nexenta the official port of Debian to the OpenSolaris kernel, like Debian GNU/kFreeBSD is the official port of Debian to FreeBSD's kernel. This would surely reduce the number of patches needed for Debian packages to cleanly build on Nexenta, because currently many patches don't make it upstream. To help make collaboration with Debian happen, the Nexenta developers have provided public root access to Nexenta zones for Debian developers who want to test their packages on the gnusolaris.org build machine. Any Debian package maintainer can request login details.

The progress on becoming the official Debian or Ubuntu port to the OpenSolaris kernel has been slow, however, as Gulecha admits:

The decision to base Nexenta on Ubuntu LTS instead of Debian was rather practical: "Upstream itself would be supporting packages for 3 years, which would make updates easier for us."

One of the big projects developed for NCP2 was the Autobuilder project by Tim Spriggs, one of the lead developers of Nexenta Core Platform. This is a package building toolkit, hosted at builder.tajinc.org. The autobuilder keeps track of packages in various repositories, including upstream Ubuntu. Build nodes can talk to the autobuilder and request jobs. This is how the developers ported over 13,000 packages automatically in a short time, making Nexenta the OpenSolaris distribution with the largest number of packages. The developers are planning an RSS feed with patches for each package on builder.tajinc.org, for upstream maintainers to be able to keep track of their package's patch feed.

The Nexenta developers guide new developers through the process of building packages. The process is simple: add the right deb-src lines from Nexenta and Ubuntu to the /etc/apt/sources.list, create GPG keys and install some development packages. Then get the sources of the not-yet-ported package and build the dependencies with "apt-get build-dep" and "apt-get source". Then modify the package version string to reflect that it is a Nexenta version and build the source with "dpkg-buildpackage". If all goes well, the package can be installed with "dpkg -i" and be tested. If it works, the porter can upload the generated deb files with "dput". If the Nexenta developers accept the package, it's added to the repository.

Of course a lot of Ubuntu packages will not build on Nexenta out-of-the-box. To help developers with porting Linux code to Nexenta, the project's website has a page that lists common porting issues and their solution. This varies from changing dependencies in the Dependency field of a Debian package or adding the right includes in C source to using special compiler flags.

Nexenta distributions

Not only is Nexenta very porter-friendly, the developers also encourage users to build custom distributions on Nexenta Core. The Nexenta Builder tool is a developer's kit that simplifies the process of creating a new distribution ISO, and it's the same tool as the Nexenta Core team is using to produce Nexenta Core images.

Recently, Andrew Stormont announced his project StormOS, an Xfce-based derivative of Nexenta Core Platform 2. This is a desktop operating system for people who like the ZFS/apt combo, but don't want to install the desktop packages themselves. Currently StormOS is a one-man project by Stormont, who is one of the Wine-Doors developers. The StormOS story started when Stormont was asked to create OpenSolaris packages for Wine-Doors, but he didn't like the package system IPS. So he decided to "fix" OpenSolaris and tried Nexenta, which he wanted to turn into a desktop operating system, hereby reviving the Nexenta project's original goal.

Conclusion

Nexenta Core Platform 2 is a solid operating system that will surely appeal to Linux users who like ZFS but don't want to learn OpenSolaris. The combination also comes with some clear advantages: Debian's APT is better and faster than IPS, and the apt-clone command gives users powerful transactional upgrades. This makes it a perfect system for a home server. Moreover, the community gives a lot of support to people trying to port packages to Nexenta. It will be interesting to see if and how the Nexenta community will work closer with their upstream distribution and with the GNU/kOpenSolaris project.