The real BKL end game

By Jonathan Corbet
January 26, 2011

The removal of the big kernel lock (BKL) has been on the kernel community's "to do" list almost since that lock was first added to make the kernel work on multiprocessor systems. Over time, the significance of the lock has diminished as finer-grained locking was added to various kernel subsystems, but the BKL itself has endured. Getting rid of it for good remained desirable because the BKL can still cause unwanted latencies at times. There's also a certain amount of pride involved in completing the job. That completion has been long in coming, though; once the worst performance issues associated with the BKL were resolved, interest in doing the low-level work needed to finish the job declined.

Two years or so ago, though, developers started working on BKL removal again. Some of this work was motivated by the realtime tree, where patience with latency sources is rather more limited. Still, it seemed like completion remained a distant goal; hundreds of BKL call sites remained in the kernel.

Then Arnd Bergmann took on the task of eliminating the BKL entirely. His cleanup work has been going on for some time; if he has his way, this patch set (or something derived from it) will remove the BKL entirely in 2.6.39. To get there, about a dozen modules need to be addressed. Some of them (i830, autofs3, and smbfs) are simply to be removed. Others (appletalk and hpfs) are to be moved to the staging tree for near-term removal, though there is some resistance to that idea. The remaining modules are to be fixed in some way. Once that's taken care of, the final patch in the series removes the lock itself. It will not be missed.

Index entries for this article
Kernel	Big kernel lock

to post comments

The real BKL end game

Posted Jan 27, 2011 7:42 UTC (Thu) by dambacher (subscriber, #1710) [Link] (2 responses)

If nothing else qualified until now - this would be worth a 3.0.0 release .-)

The real BKL end game

Posted Jan 27, 2011 8:32 UTC (Thu) by zxcvbnm (guest, #71201) [Link] (1 responses)

The only thing in my opinion that would justify a 3.0 branch would be an ABI change; just like the 2.0 branch.

The real BKL end game

Posted Jan 27, 2011 21:14 UTC (Thu) by nevets (subscriber, #11875) [Link]

It's already been decided to go to 3.0 once we hit 2.6.42, then we can leave that number there till the end of the Universe.

Non-free drivers

Posted Jan 27, 2011 8:35 UTC (Thu) by zxcvbnm (guest, #71201) [Link] (19 responses)

It'll be interesting to see what kind of havoc this brings down upon the developers of closed-source/non-free drivers. Not that I care mind you; though, it should be fun to watch.

Non-free drivers

Posted Jan 27, 2011 8:41 UTC (Thu) by bronson (subscriber, #4806) [Link] (17 responses)

Yep. And it's yet more proof that Linux's no-stable-abi policy is by far the best way to go.

Non-free drivers

Posted Jan 27, 2011 9:11 UTC (Thu) by zxcvbnm (guest, #71201) [Link] (16 responses)

I'm not sure how this proves the API policy is a bad idea. If the API was stable (read: written in stone) we'd be saddled with the BKL for years to come.

Non-free drivers

Posted Jan 27, 2011 9:18 UTC (Thu) by smitty_one_each (subscriber, #28989) [Link]

It seems like you may have inverted the logic of GP.

Non-free drivers

Posted Jan 27, 2011 9:41 UTC (Thu) by trasz (guest, #45786) [Link] (14 responses)

Actually, not at all. Systems maintaining stable ABI don't have any bigger problems with their BKL counterparts (assuming they still have something like that) than Linux has.

Non-free drivers

Posted Jan 27, 2011 10:49 UTC (Thu) by xav (guest, #18536) [Link] (13 responses)

Citation needed.

Non-free drivers

Posted Jan 27, 2011 11:05 UTC (Thu) by trasz (guest, #45786) [Link] (12 responses)

Same could be said for the opposite argument - the "stable kernel API nonsense" is just a nonsense, since all other operating systems do just fine with reasonably stable kernel API and ABI.

Non-free drivers

Posted Jan 27, 2011 16:15 UTC (Thu) by BrucePerens (guest, #2510) [Link] (11 responses)

For some arbitrary level of "fine", lower than our standard. Actually, those folks go through a tremendous lot of pain to improve their systems while preserving the old ABIs. I remember that HP had planned to put a Billion dollars into a rewrite of HP-UX around 2000.

Non-free drivers

Posted Jan 27, 2011 16:46 UTC (Thu) by trasz (guest, #45786) [Link] (10 responses)

Are you closely following the development of systems such as FreeBSD or Solaris, which keep reasonably stable API and ABI without any "tremendous lot of pain" or "Billion dollars"? Actually, the reverse is possible - if you think about interfaces before implementing them, you don't need to go through all the code base fixing it later.

Non-free drivers

Posted Jan 27, 2011 17:35 UTC (Thu) by BrucePerens (guest, #2510) [Link] (3 responses)

if you think about interfaces before implementing them, you don't need to go through all the code base fixing it later.

Can you now see the unconscious (at least I hope it was unconscious) disrespect for the kernel developers in that comment? You seem to believe that if the developers just thought about it, they'd get the ABIs right the first time and they'd never have to do it again.

Locking ABIs down means you just won't innovate that part of the system any longer, and you'll put up with what eventually turn out to be insufficient designs, as new requirements evolve, for the sake of stability.

Linux already has enough of that in the 40 year old Unix interface. One I/O context permanently glued to the process context. Asynchronous signals used where an OS event queue would make more sense. Whole-process fork. And then all of the later stuff made to work with the existing paradigm much less cleanly than would be necessary if we had the opportunity to redefine the process context.

Now, take those problems and multiply them, by spreading them across the entire kernel. That is what other OS maintainers have to cope with. It has been expensive for Sun, and it has been one of the factors that makes BSD the second fiddle.

Non-free drivers

Posted Jan 27, 2011 18:11 UTC (Thu) by trasz (guest, #45786) [Link] (2 responses)

It's not about "never", as there is no point in keeping stable kernel API and ABI forever. It's about keeping them stable for a reasonable amount of time, in a way that administrators and 3rd party developers can actually understand and adapt to. Red Hat does this, btw.

Keeping API stable does _not_ mean one can't innovate - in particular, it doesn't prevent anyone from adding new interfaces, or new functionality. In the worst case, one needs to reimplement old API as a wrapper. It sometimes happens in userland, sometimes in horrible ways (errno symbol vs. threads, for example); but for kernel interfaces is seldom needed, and does not cause any "horrible problems", contrary to what "stable API nonsense" claims.

Regarding the UNIX heritage - yes, it's there, and nothing can be done about it, unfortunately. However, the whole "stable API nonsense" thing is about interfaces for kernel code, not for userland aplications.

Non-free drivers

Posted Jan 27, 2011 18:30 UTC (Thu) by BrucePerens (guest, #2510) [Link] (1 responses)

It's about keeping them stable for a reasonable amount of time, in a way that administrators and 3rd party developers can actually understand and adapt to.

The only reason to do this is to provide an interface for outside-of-tree kernel facilities. Which are their own mistake. Even if they are only device drivers.

Semantics like those of the BKL, to the extent that they leak into the semantics of your ABI, are an excellent example of semantics that are difficult to just handwave in a wrapper. You'd have to make a real lock that at a minimum would lock out all of its callers, which would be other ABI clients. Consider the Vint Cerf piece today. They made mistakes because they simply did not consider that IP devices would be mobile. We make the same mistakes every day. It's really nice when they aren't cast in concrete. Remember the story of the syntactic white space in makefiles and why we still have it today.

I'm also thinking of the ext4 data loss argument, and the discussion of what the implied semantics of rename should be and whether programmers should have to explicitly call fsync. Some of this stuff is really hard to keep from spreading its dirty fingers far beyond your wrapper.

Non-free drivers

Posted Jan 28, 2011 22:24 UTC (Fri) by giraffedata (guest, #1954) [Link]

The only reason to do this [keep a stable interface protocol] is to provide an interface for outside-of-tree kernel facilities. Which are their own mistake.

What besides the need for the stable interface protocol makes outside-of-tree kernel facilities a mistake?

Non-free drivers

Posted Jan 27, 2011 20:38 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

FreeBSD still has a giant lock, which they are slowly splitting into smaller locks right now. And it has already caused several ABI breaks.

I don't know much about Solaris, however I did quite a lot of development on WinNT. It's almost possible to run WinNT4 drivers on Windows Vista there. However, it comes at a great price. For example, filesystem stack in Windows is routinely orders of magnitude slower than in Linux.

Non-free drivers

Posted Jan 28, 2011 9:48 UTC (Fri) by trasz (guest, #45786) [Link] (2 responses)

FreeBSD is roughly where Linux is in that regard - the Giant lock is gone from all but a few, not performance-critical, places. Last API or ABI-related change related to the Giant was removing support for non-MPSAFE (i.e. Giant-locked) network interfaces few years ago. Nothing that would cause problems with maintaining stable API and ABI in an expected way.

Non-free drivers

Posted Jan 28, 2011 13:01 UTC (Fri) by nix (subscriber, #2304) [Link] (1 responses)

Except that 'you cannot rely on the Giant Lock in your out-of-tree network interface driver' *is* an interface change which drivers must adapt to. I'd rather signal that by an actual API change, forcing people to update their drivers or fail to compile, than silently change it and open users up to problems due to absent locking.

Non-free drivers

Posted Jan 28, 2011 13:19 UTC (Fri) by trasz (guest, #45786) [Link]

Again - nobody is talking about not changing API and ABI _ever_. It's ok to keep it stable for a defined period of time, say, across a major version. The change you're describing happened in 2003. Since then, there were other changes - but it didn't change the fact that network drivers compiled for FreeBSD 7.0 will work fine with FreeBSD 7.4 once it's released.

Btw - of course there was a migration period, during which kernel warned about drivers that didn't declare themselves as MPSAFE ("Giant-free").

Non-free drivers

Posted Jan 28, 2011 4:14 UTC (Fri) by BenHutchings (subscriber, #37955) [Link] (1 responses)

FreeBSD doesn't even have a stable ABI between kernel and userland! Solaris, on the other hand, has obsessive API and ABI compatibility, but strangely enough it doesn't make life easy for driver developers.

Non-free drivers

Posted Jan 28, 2011 9:43 UTC (Fri) by trasz (guest, #45786) [Link]

Of course FreeBSD does have a stable syscall ABI. It's not uncommon to run FreeBSD 6 jails on FreeBSD 8 kernel, for example. The only exception here is KVM, which means several administrative tools, such as ipcs(1) or netstat(1), need to match the kernel version.

Non-free drivers

Posted Jan 27, 2011 15:28 UTC (Thu) by jengelh (subscriber, #33263) [Link]

That, together with init_MUTEX gone. Now all we need is more GPL exports..

The real BKL end game

Posted Jan 27, 2011 11:36 UTC (Thu) by JamieBennett (guest, #25846) [Link]

Kudos to Arnd for championing this seemingly uninteresting endeavor. Well done and even though the BKL isn't such a show-stopper anymore its great to see it gone.