NetBSD 5.0

After more than two years of development, NetBSD 5.0 was released at the end of April. The thirteenth release of this derivative of UC Berkeley's 386BSD boasts improved performance and scalability on modern multiprocessor and multicore systems. This is the result of a lot of rewritten code: a new threading subsystem based on a 1:1 threading model, new kernel synchronization primitives, kernel preemption, a rewritten scheduler and real-time scheduler extensions.

The improvements in the areas of scheduling and threading are reviewed in a short document [PDF] written by NetBSD developer Mindaugas Rasiukevicius. NetBSD 5.0 provides some soft real-time extensions, which means that it doesn't meet latency requirements unconditionally, but tries to minimize the latencies. Rasiukevicius performed a latency test with two threads with a POSIX SCHED_FIFO (First in, first out) real-time scheduling policy when the system was under heavy load. This showed that NetBSD 5.0 with kernel preemption tends to respond within 5 microseconds on a modern pc, which is similar to other real-time operating systems. The details of the test and the different scheduling policies can be consulted in the document written by Rasiukevicius.

NetBSD 5.0 also has processor sets: the possibility to exclusively dedicate specific (groups of) processors to processes or threads. The API is expected to be nearly compatible with Solaris and HP-UX processor set interfaces. According to the NetBSD developers, almost all core kernel subsystems, including virtual memory, memory allocators and file system frameworks, were audited and overhauled to shift to a fine-grained locking model, which allows multiple processors to execute kernel code simultaneously.

Much of the development in the areas of SMP (symmetrical multi-processing) performance and scalability has been sponsored by the 2007 fundraising campaign that pretty much exactly met its targeted USD 50,000. NetBSD developer Andrew Doran was sponsored for parts of 2007 and 2008 to work full time on improving SMP in NetBSD, and later to help to get the NetBSD 5.0 beta in shape.

But improved performance is not the only difference with the previous major release: a number of NetBSD ports (platforms) now finally use X.Org instead of XFree86, a step that virtually all Linux distributions made years ago. The file system FFS (Fast File System) has a preview of metadata journaling, known as WAPBL (Write Ahead Physical Block Logging), which makes file system consistency checking after an unclean shutdown much faster. And one thing many laptop users will love to hear: ACPI suspend and resume now should be working on many computers.

What didn't change so much is NetBSD's installer, sysinst. This is still rather basic, but it does its job. One especially good thing is that sysinst doesn't let the user create an insecure password. There is no shortage of password enforcement tools on Linux, but it's nice to see this enforced by default in NetBSD.

Unit tests for an operating system

Also new is the Automated Testing Framework (ATF), a collection of libraries and utilities designed to ease unattended testing of applications. ATF provides the means to create test suites composed of multiple test programs, which in turn are a collection of test cases. When a test case detects an error, ATF provides as much information as possible about the failure.

ATF started as a Google Summer of Code 2007 project mentored by The NetBSD Foundation. Its original goal was to provide a testing framework for NetBSD, but it grew into an independent project because the framework could be made platform-independent. At the moment, only a few NetBSD-specific tests are available as ATF-based tests, but these are a good preview of what NetBSD 6.0 will have: many more test cases.

ATF is not only for developers. In fact, the NetBSD developers encourage all users to run the test suites themselves. They do not need to have development tools installed or source trees available to certify that a certain application works as advertised. This is the main difference from other test frameworks, which ship as part of a source package and are only run by developers after the program is built. Then the end user, who uses binary packages, never sees these tests or the results.

During the installation of NetBSD 5.0, the user sees ATF as a new tests.gz distribution set. If he chooses to install it, sysinst will populate /usr/tests with a collection of ATF test programs. Once installed, the user edits /etc/atf/NetBSD.conf to suit his system preferences and then runs atf-run | atf-report in the directory /usr/tests. At the end, the program gives a summary with the number of passed and failed test cases. If all test cases ran successfully, the user is sure the tested software behaves as it should on his hardware.

Of course it runs NetBSD.

Although NetBSD is one of the most portable operating systems (the NetBSD motto is "Of course it runs NetBSD.", your author had a fair amount of problems installing it. A physical install didn't work out because NetBSD doesn't recognize the keyboard of his Dell laptop, and resorting to virtualization proved problematic too. Only Xen seems to like NetBSD, and fortunately this process is explained well in the NetBSD Xen Guide [PDF].

What did impress your author is the cross-compiling framework: this allows a user to build a complete NetBSD distribution from another system, which could be on a different architecture or even running a different operating system. The only requirement is that the host operating system has a POSIX environment and C/C++ compiler. Because your author is generally suspicious of such claims, he put it to the test in Ubuntu 9.04 and OpenSolaris build 111 (2009.06), and the process worked flawlessly in both cases. After downloading and extracting the NetBSD sources, a simple

./build.sh -U -m i386 release iso-image

Towards NetBSD 6.0

The NetBSD Foundation is already looking forward to the future and developers have begun working on NetBSD 6.0. The next version will improve the performance and concurrency of the network stack, it will improve existing file systems and develop new ones. The operating system will also add features for embedded use, including high-resolution timers, and it will improve the quality assurance and automatic testing. A desktop improvement we will see is dhcpcd-gtk, an application that shows a systray icon with the state of dhcpcd and provides a graphical user interface to configure the network. While NetworkManager requires libnl and hal, which at the moment depend too much on Linux, dhcpcd-gtk only requires dhcpcd-dbus, which is much simpler to port.

In this year's Google Summer of Code program, 11 out of the 34 submitted applications by the NetBSD project have been chosen. The list shows some interesting projects, such as a port of Sun's ZFS file system, a minimalistic X server for embedded environments and a NetBSD router. The NetBSD Foundation has also announced the 2009 Fund Drive and expects to collect USD 60,000 by the end of the year.

Conclusion

Recently a couple of projects have emerged to create a better NetBSD desktop experience, such as the graphical user interface for dhcpcd and the NetBSD Desktop Project that was announced in February. The primary goal for this project, started by Andrew Doran and Jared D. McNeill, is to simplify the installer to be able to install a useful NetBSD system with Gnome desktop environment in 15 minutes. This will give new users a better chance to evaluate NetBSD. But this mature Unix system has always focused on the server and embedded applications. The improvements in the scheduler give NetBSD 5 an even stronger position as an embedded operating system, and some Google Summer of Code projects are paving the way for more embedded features in NetBSD 6.