Comparing Linux and Minix
[Posted February 5, 2007 by corbet]
Toward the end of
his
linux.conf.au talk, Andrew Tannenbaum put up a few slides on the
runtime cost of the microkernel approach. He had quite a few benchmarks,
but the bottom line was that the microkernel architecture
used in Minix imposed a roughly 5-10% performance penalty, depending on
what one is trying to do. While operating systems hackers would normally
cringe at the prospect of paying a 5% penalty, to many people this could
seem like a good deal: give up 5-10% of a processor which is mostly idle
anyway in exchange for a more reliable system.
In truth, neither the claim of a 5-10% penalty nor that of higher
reliability has been proved in any definitive way. At the conference,
a number of attendees questioned the way in which the benchmarks had been
done, suspecting that Minix had been benchmarked against a monolithic
version of itself. If that is the case, the benchmarks will capture the
context switching costs but will have nothing to say about the costs of the
message-passing architecture. To get a true measure of the penalty of
the microkernel architecture, it was suggested, one should benchmark Minix
against Linux.
As it turns out, the linux.conf.au swag bag contained a CD with Minix 3.1.2a
on it; one might almost think the organizers had this sort of test in mind.
So your editor came home with the intention of installing that version of
Minix and doing a bit of benchmarking. That job has now been done, and we
can talk about how Minix and Linux compare.
Time for a brief digression:
once, some years ago, your editor actually had a spare moment in which to
see how nethack was coming along. One must stay on top of all the
important development projects, after all. The graphics have improved, the
game contained more monsters than ever, etc. But there is an especially
amusing moment when one drops into a level and is informed of a sense of
having entered a more primitive place. The graphics on that level are
straight from
VAX-era rogue, and the whole thing feels rough and, well, primitive.
A similar feeling will come over a Linux user who tries to get things done
on a Minix system. It is a POSIX-like environment, and it has a working version
of the X Window system (but don't go in expecting GNOME or KDE), but that's
as far as it goes. The
shell is painful to use, many commands are
missing, and one runs into obstacles on every path. Since Minix
does not really do paging, memory quickly runs out if too many processes
are run; your editor had not seen the old "not enough core" message in
quite some time. One of the harder things to do on Minix, it turns out, is
to build any sort of non-trivial software package - even after figuring out
that the default C compiler is crippled but gcc can be found under
/usr/gnu. As a result, your editor had to give up on most of his
attempts to build current benchmarks; they just would not compile on Minix.
In the end, your editor succeeded in building and running two benchmark
programs: IOtest and UnixBench. Neither seems to be recent enough to have
a currently-maintained web page. IOtest is a disk exerciser, evidently
intended originally as a tool for driver developers. It's
useful for exercising drives in a serious way;
it also produces performance numbers on the side. UnixBench was developed
by Byte in the 1990's, and hasn't seen a whole lot of work since. It
remains, however, a useful way to get a snapshot of the relative speeds of
many operating system functions.
The benchmarks were run on an AMD Athlon 1700 system using an unremarkable
ATA disk. There are three partitions on the disk: one for the operating
system, one for swap (Linux only, since Minix does not support it), and one
for destructive disk tests. The partitioning was not changed between the
installations. Minix does not support partitions larger than 4GB (who
could ever need more than that?) so the disk tests were restricted to 4GB
on both systems. The Minix tests were done on a full installation of Minix
3.1.2a; the Linux side was represented by a late-September
Debian Etch snapshot running a 2.6.17 kernel.
The IOtest read test simply performs random reads of varying sizes,
starting with one process and going up from there. IOtest can run a large
number of competing processes, but your editor limited it to four so as to
avoid running into Minix's memory limitations. For the curious, the full Minix results and Linux results are available. The bottom line
is that the results are nearly comparable: for all practical purposes, the
two systems performed about the same. Similar things can be said about the
results (Minix, Linux) of the read/write test, which are
summarized in the plot to the right (the dashed line represents Minix).
Comparable results would be expected with a benchmark like this, since it
will be dominated by the drive's seek performance. The portion of the disk
being exercised (only 4GB, remember) was not enough to demonstrate a
difference in I/O scheduler implementations. The disk never comes near its
peak I/O rate. So the main conclusion to draw from these results is that
Minix does not get terribly in the way.
The UnixBench results (raw results: Minix, Linux) paint a rather different
picture. These results are summarized in the plot to the left; the upper
bar for each test represents Linux.
The measured system call overhead for Minix is a full ten times
higher than the value for Linux. The file copy tests ran between two and
ten times faster on Linux. Pipe throughput differed by a factor of seven;
Minix was 140 times slower at process creation. The difference in shell
script execution performance, however, was 1.4 - in Minix's favor. One
assumes that the rather simple shell provided by Minix is, at least, faster
than bash.
One can argue that Minix is a new and unfinished system which has not, yet,
had the benefit of a great deal of performance tuning. There is doubtless
some merit to that claim; the Minix folks will probably find a number of
ways to make things faster. On the other hand, it would not be
unreasonable to argue that Linux, by supporting much greater functionality
on a far wider range of hardware, has every right to be slower - but it's
not. Linux is quite a bit faster; the Minix folks certainly ran benchmarks
which showed a 5-10% difference, but they were not benchmarking against
Linux.
Dr. Tanenbaum made the claim that only a computer geek would accept better
performance if that trade brought with it lower reliability. By that
reasoning, it doesn't matter that Minix is much slower than Linux on the
same hardware; Minix is aiming for a different goal. But people do care
about performance; the fact that Dr. Tanenbaum felt the need to put up
benchmark results suggests that he cares too. Trading some performance for
reliability could well be a good deal. When one compares Minix (in its
current state) to Linux, however, the performance difference is large, and
the increased reliability is unproven.
(
Log in to post comments)