Review: Linux Kernel Development [LWN.net]

Writing books about the Linux kernel is hard. The subject matter is vast, complex, and highly technical. It also is very much a moving target;

today's kernel book becomes obsolete in a short period of time. So kernel authors have to pick their subject matter carefully, time things well, and enjoy their fifteen minutes of fame before somebody merges a patch and their words begin the inevitable slide into obsolescence.

The latest kernel book to hit the shelves is Linux Kernel Development by Robert Love. The goal of this book is perhaps best expressed by Andrew Morton in the foreword. He notes that kernel development has become increasingly complex as the kernel grows and scales to larger systems. That complexity increases the functionality and performance of the kernel, but it comes with a cost: the kernel is harder to understand than it once was.

I believe that this declining accessibility of the Linux source base is already a problem for the quality of the kernel, and it will become more serious over time. Those who care for Linux clearly have an interest in increasing the number of developers who can contribute to the kernel.

The purpose of this book is to help developers get to where they can make a contribution.

Linux Kernel Development covers a lot of ground. There are chapters on process management and scheduling, how system calls work, interrupt handling (but not device drivers in general), delayed work mechanisms, locking and mutual exclusion, timers, memory management, filesystems and the block layer, the page cache, kernel debugging, and more. This book, however, is just over 300 pages long, so it necessarily does not cover any of these topics in any great depth. Thus, for example, you will see what each of the inode_operations methods is, but there is little information on how to write one. Linux Kernel Development is a starting point which will prove useful to any developer trying to get up to speed with one or more kernel subsystems. Completing that process, however, will still require setting down the book and diving into the source.

That said, this book is truly a good starting point. After having perused the text on a particular subsystem, the reader will have enough background to be able to make sense of the source much more quickly than before. The presentation is clear, the writing is enjoyable to read^*, and the information is accurate and useful. Linux Kernel Development belongs on the shelf of any developer who is interested in kernel work.

One small complaint that one might make is that it is hard to figure out just which version of the kernel this book covers. The fine print on the back notes that it covers 2.6. Of course, the book was published in September, and now, at the end of November, the 2.6 kernel has not yet been released, so that statement is not entirely accurate. A suitably attentive reader can find places where the current kernel diverges from the text of this book - the listing of struct inode lacks the i_cdev field, for example. Kernel books will always tend to be like that, however; they are only completely accurate when they are out of date. Mr. Love appears to have timed things pretty well with this one; the differences between the text and the current development kernel are minimal - so far. For the time being, Linux Kernel Development is the best description of the structure and API of the 2.6 kernel available.

[Disclaimer: your reviewer is the co-author of a Linux Kernel book which could be seen as competing with Linux Kernel Development, though he sees the two as being entirely complementary. Buy Both.]

^*Though sprinkled with rather more footnotes that your reviewer might have preferred...

(Log in to post comments)

Kernel Documentation Fantasy

Posted Nov 26, 2003 15:01 UTC (Wed) by utoddl (subscriber, #1232) [Link]

Sometimes in my total geek daydreaming fantasy while I'm waiting for the next article to be posted to lwn.net, I imagine the whole Linux kernel converted to a literate programming resource in the spirit of Donald Knuth's The TeX Book, a complete CWEB conversion with coherent prose describing every line of the kernel in a way that can be read front to back like a book, or preprocessed and compiled into a working kernel. Then I imagine the bank of disk drives it would take to hold it all, and the power requirements, and how much cooling, etc., and I snap out of it again, and there's another lwn.net article posted just in time...

You probably didn't want to know all this. :-)

Kernel Documentation Fantasy

Posted Nov 26, 2003 22:05 UTC (Wed) by iabervon (subscriber, #722) [Link]

Two problems with literate programming: It's generally relatively easy to understand the bit of the code you're looking at, but hard to understand the overall implications of it in the context of the system as a whole. Where would the information be that tells you, for example, the lock ordering rules? It also tends towards digressions. I've read some of TeX, and it reads more like a book about the difficulty of writing working and portable Pascal than anything to do with TeX. E.g.: The bane of portability is the fact that different operating systems treat input and output quite differently, perhaps because computer scientists have not given sufficient attention to this problem.

Absolutely true, but why am I reading about this in this particular place? Because Knuth had to deal with it there. While interesting, this doesn't actually advance my knowledge of TeX, however.

Kernel Documentation Fantasy

Posted Nov 27, 2003 2:49 UTC (Thu) by utoddl (subscriber, #1232) [Link]

Caveat: I've done very little literate programming -- a couple of toy projects that were one-offs just to see what it looked like. It was fun. It was also a lot of work up front. I'll therefore take your comments at face value and consider that there is a core truth there.

But to your specific points: "Where would the information be that tells you, for example, the lock ordering rules?" Probably in a section or sub-section entitled Lock Ordering Rules, and indexed under "locks" and "rules" and probably a few other ways. Where is the discussion on lock ordering rules now? In some terse comments here and there in the kernel source? No, it's actually scattered about in several years of kerneldev mailing list archives. The hard-fought, well thrashed out issues are important to understand, but their subtle influence on some of the code will be easily lost on the next generation of kernel hackers who try to learn how Linux works by reading only the code and embedded comments, but without those discussions. Old mistakes will be made again and lost lessons relearned the hard way.

About your example digression: Yes, it is wordy. But I think it also advances my knowledge of TeX for the very reasons you cite. It gets across the context that we're about to deal with portability issues that may not be obvious to someone only looking at the code and who's familiar with only one or a few OSs, and only sees a "Danger, Will Robinson" comment with lots of exclamation marks.

:-) Besides, in my Documentation Fantasy, the prose are always perfect, succinct yet engaging, with just the right touch of humor, and they always correct themselves whenever patches are applied. And corbet always checks 'em over in his spare time just to be sure. Hey, as long as I'm dreaming...

Kernel Documentation Fantasy

Posted Dec 2, 2003 4:59 UTC (Tue) by iabervon (subscriber, #722) [Link]

Ah, but a section entitled Lock Ordering Rules, indexed under "locks" and
"rules", doesn't need to be part of the kernel source. In fact, it
probably would do better not to be interspersed with any code at all.
It's certainly something that's worth documenting (although the Linux
principal of lock ordering is, "if you have to document it, it's too
complicated"). For that matter, the ideal kernel document would contain
explanations of everything that isn't in the code, and why.

What the bit of TeX actually told me was why people use C instead of
Pascal in general (the Pascal standard omits most of the things necessary
for making a large project manageable).

Personally, I'd love to have a complete explanation of the kernel. But I
don't think that literate programming actually forms a clear presentation
of an explanation (nor does it help to clarify the code itself).