Hand-waving about a phased sleep [LWN.net]

Hand-waving about a phased sleep

Posted Jan 7, 2009 17:27 UTC (Wed) by BrucePerens (subscriber, #2510)
Parent article: Btrfs aims for the mainline

Pardon me for engaging in some hand-waving, and not really knowing what the kernel developers have tried to do about this issue over time.

Isn't the problem here that sleep and the following process wakeup are expensive? In the case that sleep might be necessary only for a short time, a phased sleep makes more sense in the context of multiprocessing or hyperthreads. Halt the processor, but leave the process context in place in the processor. Don't move the process context out of the processor until some time has passed. Handle wakeups during this period by restarting the processor.

First, you stop banging on the lock in a tight loop, and that's important because atomic operations make expensive use of inter-processor cache coordination, potentially keeping other processors from doing useful work for some time. Also, you get to put the CPU core to useful work if the stopped processor is actually a hyperthread.

Bruce

(Log in to post comments)

Hand-waving about a phased sleep

Posted Jan 7, 2009 17:55 UTC (Wed) by johntb86 (subscriber, #53897) [Link]

That's an interesting idea, but as far as I know to wake up the processor you'd need an IPI, so it's not exactly cheap. You'd also need to create a timer interrupt to wake up the processor if it's not woken up by some other method, so this could become a pretty expensive way to have the processor do no work for a while.

Hand-waving about a phased sleep

Posted Jan 7, 2009 18:26 UTC (Wed) by BrucePerens (subscriber, #2510) [Link]

Perhaps we need hardware support to make this work properly. A timed halt instruction, or a low-level atomic primitive that can wake a processor waiting for an address.

Hand-waving about a phased sleep

Posted Jan 7, 2009 21:29 UTC (Wed) by jlokier (subscriber, #52227) [Link]

x86 does have a primitive to wait on an address: monitor/mwait.
It waits until the monitored location is modified by another processor.

I read that it's a bit slow for this sort of thing.

In principle, if it were fast (and it could be with MESI caches),
it would be very well suited to replace all spinning waits.

Hand-waving about a phased sleep

Posted Jan 8, 2009 3:03 UTC (Thu) by nevets (subscriber, #11875) [Link]

Actually, what Peter Zijlstra has done (and we do in the -rt patch) is not to spin on any atomic operations. We simply spin and check a variable. For instance, when we fail to get the lock, we go into a loop, checking the lock owner, and if that owner is still running. No atomic operations involved. There are a few races in different implementations, but none of those races are detrimental. A missed race just means we may sleep when we could have gotten the lock (current behavior). When the lock owner changes, or the current owner goes to sleep, we try to retake the lock. If the owner is asleep, we then schedule.

In the -rt patch, we are a bit more conservative and avoid these races, but we still do not spin on any atomics. Thus, we do not slow down the bus traffic of other CPUS.