The ongoing realtime story

Ingo Molnar continues to crank out patches at a high rate. The latest, -RT-2.6.9-mm1-V0 -RT-2.6.9-mm1-V0.3, is advertised as being rather more experimental than its predecessors - which is saying something. This patch set brings preemptible mutexes to (almost) the last, most difficult parts of the kernel, including the low-level memory allocators, wait queue code, kernel timers, and more. Says Ingo:

Some small pieces of this work have been put forward as independent patches; these include the enhancements to the completion interface mentioned last week. Linus has also made a couple of changes to the big kernel lock code in support of this sort of work: the BKL functions are now entirely out-of-line, and some of the code for managing the BKL itself has been made preemptible.

Ingo's patch also changes a number of semaphores in the kernel over to completions. For situations where one kernel thread needs to notify another that some task has been finished, completions are a better interface: they make the intent of the code clear, and they are better optimized for that use. Some of those patches have been posted separately as well, leading to some pushback from kernel developers who believe that their use of semaphores for that purpose is entirely legitimate. Bill Huey, the developer behind the mmlinux realtime project, is the person who has been pushing some of those patches; he responded to the resistance in this way:

This, of course, is just the sort of talk which will put many kernel developers off the realtime patches entirely; some of Mr. Huey's subsequent postings, being rather more inflammatory, did not help the situation either. Ingo went into damage control mode and smoothed things over, for now. If and when the realtime preemption patch is put forward for inclusion, however, chances are that the discussion could get heated indeed.

Paul McKenney, meanwhile, expressed a discomfort with the realtime work which must certainly be felt by many:

His message included a patch intended to point toward such a path. This patch, which assumes an SMP system, works by setting aside one CPU as a purely realtime processor; it is not part of the regular scheduling mechanism. Realtime processes may be assigned to that CPU by the system administrator. If they mostly work in user mode, all is well; they have a dedicated processor and need not worry about latency. As soon as a realtime process invokes a system call, however, it goes into non-deterministic mode and is booted out to one of the system's other processors. In this way, the dedicated, real-time processor never gets stuck waiting for a lock.

The downside, of course, is that, every now and then, it is actually nice to be able to use system calls. Paul's idea was that each Linux system call could be examined individually, and, if warranted, modified to be entirely preemptible. When any particular system call reaches a state where it is considered to be deterministic, it could be added to a list of such calls, and realtime processes using it need not be shifted away from the realtime processor. Over time, this list would grow to the point that realtime tasks which do actual, interesting work could be run on the mainline Linux kernel. In the mean time, there would be no need for a major flag day where the entire kernel locking scheme is changed at once.

The real challenge with this approach would be to make I/O deterministic, since realtime processes usually must act in response to external events. That cannot be done until it is clear that all filesystems and device drivers have been made entirely preemptible - and, at that point, much of the system has been affected. Meanwhile, it turns out that the 2.6.9 kernel already has part of this mechanism: the new isolcpus= boot parameter excludes one or more processors from regular scheduling. The scheme for migrating realtime processes when they invoke a non-deterministic system call is not present, however.