One of the best references for Linux and UNIX system administrators over
the years has been the "Handbooks" (either Linux Administration
Handbook (LAH) or UNIX System Administration Handbook (USAH) at various
points). But the last edition was published in 2000 (as USAH), and included
information on then-current Red Hat Linux 6.2 and FreeBSD 3.4. A new, updated version,
and Linux System Administration Handbook, Fourth Edition (ULSAH), is
due out any day now, and the principal authors,
Evi Nemeth, Garth Snyder, Trent R. Hein, and Ben Whaley, agreed to answer
some questions for LWN readers. Below are their answers on the book, the
impact of Linux,
the future for UNIX and Linux, and more.
Could you all please introduce yourselves? What are you working on when
you're not writing system administration books?
Ben: I wear a bunch of different hats as an engineer at AppliedTrust in
Boulder. In addition to consulting on architecture and operations in
UNIX and Linux environments, I think a lot about next generation
technologies. Virtualization (in its myriad forms), application
security, and the adoption of open source are all of interest. I also
have a interest in the history of computing generally, and of UNIX and
Linux in particular. Outside of computing, I enjoy the republic of
Boulder as much as possible.
Garth: I was one of the original authors of the UNIX book lo these many
years ago, but I've spent time on a variety of projects since then. Over
time, I've actually done more software development than administration.
Evi: I'm Evi Nemeth, former Computer Science faculty at the University
of Colorado. I'm now out of the UNIX/Linux/CS world altogether; I
retired, bought a sailboat, took off and am currently in French
Polynesia, half way across the Pacific en route to New Zealand. I'm
working on making my boat single-hander-friendly for when I run out of
crew. I do have email on the boat via my ham radio; it's like uucp at
about 100 characters per minute.
Trent: I'm a scientist at heart, and I love understanding all the
layers of a system. When I'm not writing, I'm deep in the trenches
working on IT infrastructure issues that range from low-level technical
issues to policy and management.
The new version of the Linux Administration Handbook is on its way. When
can we expect it, and what are you most excited about in this edition?
Ben: We anticipate a mid to late July shelf date. There is loads of new
material in this combined UNIX and Linux edition, including new chapters
on virtualization, scripting, and green IT practices. I'm very pleased
with the new set of cartoons and cover art. Also, I'm thrilled to be
included as a new author after contributing to the 2nd edition of the
Garth: Many of the existing chapters have had near-complete rewrites as
well. You might think that long-standing, basic technologies like disk
storage and email would be relatively stable, but that's not true at
all. The way that most sites manage these resources has changed
completely just in the last five years or so.
Evi: LAH and USAH have merged back together. It's due out sometime soon
(July 2010). I'm most excited that it's finally done; I left the boat in
Panama and came back to Colorado for a year to work full time on the
book — it's a huge amount of work.
Trent: It better be in bookstores in the next 2 weeks!! I'm super
excited about the new Green IT section — it presents opportunities for
sysadmins to make a huge difference to our planet.
What led to the decision to combine the UNIX and Linux versions of the book?
Ben: Linux is recognized as an enterprise-grade operating system that
has proven its capabilities throughout the explosion of computing in the
last twenty years. It has significant momentum behind it, much more so
than other leading UNIX variants. One could argue that UNIX lives on but
looks to Linux as its leader. It has enough in common with traditional
UNIX that it make sense to cover it all in one place.
Garth: One factor that's helped make it possible to reintegrate is the
dramatic shakeout in the UNIX market. There simply aren't as many major
versions of Unix around as there were ten years ago. We've seen a
similar consolidation in the Linux arena as well, with most activity
consolidating around a few major distributions.
Evi: Personally, I'm not sure we should ever have separated UNIX and
Linux into 2 books. We thought that conditional text (if Linux, blah,
blah, else ...) would make it easy to manage both books with minimal
effort. Not true.
We also feel that this is probably the last edition that will be on
paper and maybe the last edition period, so if we are doing one last
book, let's cover all the systems we can. I'm sorry we didn't manage to
include MacOS too.
Trent: Combining the books makes sense because system administrators
manage both UNIX and Linux systems ... organizations shouldn't be
separating duties for these platforms since they're so similar.
A lot of old-time Unix folks were taken by surprise when Linux hit the
scene. You've seen a lot of Unix variants come and go; what, do you
think, accounts for the way that Linux has been able to displace Unix in
so many settings?
Evi: I think the fact that Linux ran on PCs instead of the special
vendor specific hardware of most UNIX systems gave Linux a leg up.
University students ran Linux on their PCs, graduated, and went to work
in corporate America (Europe, etc.). Linux also embraced (or tolerated)
Windows more than the UNIX vendors and found ways for the systems to
Trent: The community. It's a lot easier to support Linux these days
because if there's any issue, there are hundreds or thousands of people
to turn to on the 'net for help. It's a lot easier than sitting on hold
with a vendor's call center for hours.
What do you think is the future of "true" Unix?
Ben: In the near term I believe that UNIX environments will continue as
they have, serving as venerable, tenured systems with proven stability
and some powerful capabilities. Most of the UNIX systems I work with run
heavy databases or specific enterprise applications. OpenSolaris is an
impressive system with advanced features that don't exist elsewhere. In
the longer term, it seems to me that Linux will displace UNIX. All the
major vendors contribute to Linux's development (as LWN's own data
suggests) and even promote standardization.
Garth: Traditional Unix vendors don't have the resources to compete
against Linux on every front, so they've had to pick their battles and
concentrate on distinguishing themselves through enterprise features
such as database or filesystem performance. However, the number of
domains in which it's possible to demonstrate a proprietary advantage
over Linux is continuously shrinking. I don't see any reason for the
current trends to change, so I'm pessimistic about the future of
Evi: UNIXes that run on PC hardware will survive in niches — FreeBSD in
embedded systems, OpenBSD in security conscious spots, Open Solaris if
Oracle leaves it alone, etc., but systems on proprietary hardware (AIX,
HP-UX, Solaris) will continue to decline in market share. Note that each
of these three vendors is hedging their bets with a Linux product or
with a Linux that runs on their hardware.
Trent: There's such a large installed base that I think Unix is going to
be around for a very long, long time. But, I can tell you that almost
all the new infrastructure I build is on Linux.
Is the success of Linux a good thing? Or would we be better off now if
some version of Unix had established itself more strongly in the
Ben: I was introduced to Linux before UNIX, and I suspect that I
wouldn't be on the same path that I'm on today without it. In fact, I
grew up from adolescence using Linux and today I look at closed systems
as a broken, legacy model of development. I love the creativity and
community that open systems offer. To me it's a great thing that has
kept UNIX alive.
Garth: In some ways, this is like asking, "The rise of humanity: good or
bad?" Linux has warts, but so do the alternatives. It's not the solution
so much as the context in which other things happen.
Outside of a thousand minor differences, there really isn't a dime's
worth of difference between the various Unix-like operating systems.
They are all basically the same. (Well, except perhaps for AIX. Whatever
else one might say about it, it's definitely not the same.) Fundamental,
game-changing advances — such as Solaris's ZFS filesystem — are few and
Evi: Yes, the success of Linux is a very good thing. Each open source
"Unix" has strengths. Linux is quick to see a cool idea in another
system, re-engineer it, and voila, it's part of Linux too.
Trent: Linux is great!! In a way, Linux is really UNIX for the people.
We needed that. It's forced the entire industry to be more open, which
I'm not sure a more established open-source UNIX would have achieved
because it's inspired a shift in mindset.
If you could command the Linux development community to do one thing
that would make life easier for system administrators, what would it be?
Ben: Settle on a few more standards, such as log formats, command line
arguments, configuration file syntax, and extensibility. As always, the
more documentation, the better. Active communities where developers and
users interact are extremely valuable.
Garth: Unix and Linux developers have traditionally designed software to
be as flexible and configurable as possible. A good example of this
approach is the original sendmail, which knew practically nothing about
email addressing or transport until you manually programmed that
knowledge into its configuration file. Many common systems still take
that general approach: here's the tool kit, build whatever you like.
This approach has its advantages, but it also imposes a cognitive burden
on everyone downstream of the developers. I'd like to see more focus on
simplicity and predictability in Linux distributions and less concern
about hypothetical scenarios and edge cases. A Linux server may be a lot
more complicated than an iPhone, but it surely doesn't need to be 1000
times more complicated, as it currently is.
Unfortunately, it's especially hard to promote simplicity and clarity in
the open source world. It's easy to accept patches that add incremental
features but hard to remove anything or break compatibility. Reaching
design consensus on future developments is hard enough without
revisiting the messes of the past.
Evi: Standardize! Make command names, arguments, and behavior the same.
Get rid of the not-invented-here attitude.
Trent: Keep it simple. The early success of both UNIX and Linux can be
attributed to their simple, modular approach. Too often these days folks
are developing packages that are like a giant corn maze (but I won't
name any names!). We'll all get farther if the development community is
focused on simplicity and modularity.
The world is now full of Linux users and administrators who have never
touched a traditional Unix machine. What lesson from Unix do these folks
risk missing in a Linux-only world?
Garth: The Linux community has put a lot of effort into strip-mining the
UNIX systems of the past and digging out the nuggets of value that
weren't nailed down. So no, I don't think administrators unfamiliar with
traditional UNIX systems are missing too much. On the other hand, we do
seem to be stuck in about 1990 with respect to our basic idea of what an
operating system should be. Look at all the incredible developments
we've seen over the last 20 years in application and web development;
software is completely different now. I hope the triumph of UNIX and
Linux don't lock us into the POSIX API indefinitely.
Evi: This generation of Linuxites have used UNIX, Linux is a UNIX
system. It has commands that you type to a shell instead of driving thru
a zillion menus, it has all the important concepts of UNIX, like pipes
and input output redirection, it has man pages, ...
Trent: Process adherence. In the UNIX world of yesterday, we had giant
machines that filled an entire room, and dozens or hundreds of users
shared them. In order to achieve a reasonable service level, system
administrators had to be very process-focused, else they would impact
many users if a mistake occurred. Today, a single user may have dozens
of systems, physical and virtual. System administrators still need to
have a well-developed, well-followed set of processes to maintain them
to provide an even higher level of service.
The world is also full of energetic new developers for whom open source
has always just been part of the environment. These developers will be
creating our next generation of systems. What kind of operating systems
do you hope they will build, and what advice would you offer them on
Ben: The cloud is clearly the direction that is currently leading the
pack, and I'm anxious to see what happens next in the space. I hope that
open source development continues to thrive. It will also be interesting
to see what happens with security. People are paying a lot of attention
to it these days, and we're well positioned to fix the less secure,
trust-based models of earlier systems.
Evi: A fundamental principle of UNIX-ish operating systems is that they
provide commands that do one thing well and then the plumbing to hook
them together to get a final result. Windows-ish operating systems try
to think of everything a user might want to do and make a command or
option for it. If the Windows developer didn't think of the thing you
need to do, you are out of luck. Next generation operating systems need
the UNIXy approach. Keep things simple, take small steps, but do it well.
Trent: I hope they build operating systems that perform well. Back in
the day, we had to optimize for every last cycle, because cycles were
few. Some new OS developers have had the luxury of hardware being
abstracted from them, and it's easy to forget what really happens at
that layer. It's easy to fall into a pattern of slapping layer upon
layer, resulting in a lot of kernel and application bloat. Keep it
simple, and think about how to optimize for performance.
Anything else you would like to say to LWN's readers?
Ben: I'm a lurker on LWN and I've learned a lot from both the articles
and the comments. I appreciate the active open source community on the
site. Thanks for reading.
Garth: I'm excited about Btrfs and really looking forward to its debut
as a production system. Check it out!
Evi: I need crew. Is there anyone out there (between jobs maybe) who
isn't busy between September and November 2010? We share expenses on
consumables; I pay for boat maintenance.
Trent: I hope in UNIX and Linux System Administration Handbook (4th
Edition) we teach folks how to be good system administrators and find
answers on their own, rather than providing every possible answer for
them. System administrators need knowledge acquisition and problem
solving skills more than anything.
We would like to thank Evi, Garth, Trent, and Ben for taking the time to answer
to post comments)