LWN: Comments on "(Nearly) full tickless operation in 3.10"

kernel thread/process and full tickless mode?

henryc — Tue, 09 Oct 2018 01:44:29 +0000

The article states, "the administrator should make extensive use of CPU affinities to keep unwanted processes (including kernel threads) off the relevant processors".

I thought it wasn't possible to affine migration and kworker processes to other cores as they are per-core processes that bind to the specific cores. RCU related processes can be moved as there is a kernel parameter for that.

The article suggests that in order for a core to be in full nohz mode, "a running CPU will only disable the timer tick if there is a single runnable process on that CPU". It sounds like it is necessary to move kernel processes such as migration and kworker out of the core. Or kernel processes don't count? I am a bit confused.

If kernel processes count, then how to move them to other cores? Can they be moved to another physical CPU?

Thanks.

(Nearly) full tickless operation in 3.10

YAK — Fri, 02 Aug 2013 06:37:52 +0000

In file kernel/time/Kconfig, option "NO_HZ_FULL" depends on 64BIT, So is it only supported for 64 bit architecture ? i wanted to try it out on 32-bit ARM cortex A9 but i guess its not possible ?

(Nearly) full tickless operation in 3.10

hummassa — Wed, 17 Jul 2013 02:56:56 +0000

Your link says "maintenance mode".

(Nearly) full tickless operation in 3.10

ParadoxUncreated — Tue, 16 Jul 2013 21:31:36 +0000

It´s way more than 1% more cpu. Atleast in terms of visual performance. I did a low-jitter config on linux some time ago, that I have now perfected:

http://ovekarlsen.com/Blog/turning-ubuntu-12-04-into-a-pr...

Choosing components for less jitter, reducing timer interrupts to 90hz, enabling low-latency behaviour etc, + renicing X etc, made doom 3 run perfectly on a core 2 duo, with a GTX 280. On Quakecon 2012 Carmack says that game is still taxing on some configurations. It actually requires an Intel E5 workstation with low jitter hardware, to run as good in a tweaked windows XP.

The game does 3 passes to OpenGL pr frame. That seems to make it very jitter-sensitive. But with a low-jitter configured kernel, it runs perfectly. That seems to apply to things like Wine also. Also Wine-games such as Half-life 2 still had some jitter, but much less.

So low-jitter matters, if you want smooth gameplay on advanced game-engines. And a lot of data is pushed there. Generally the system seems very nice and responsive also. Much more like an optimal desktop system indeed.

Peace Be With You.

(Nearly) full tickless operation in 3.10

methanol — Mon, 01 Jul 2013 12:28:38 +0000

Is there a way to show what the current "tick-mode" is? And can I force only one process to be on a particular cpu core?

Does Tickless supports for Octeon-2 and ARM as well ?

Ralf — Wed, 29 May 2013 09:46:36 +0000

MIPS support (tested on a 12 core Octeon+) is now available in http://git.linux-mips.org/?p=ralf/linux.git;a=commit;h=07...

Please send test reports to linux-mips@linux-mips.org. Thanks!

Does Tickless supports for Octeon-2 and ARM as well ?

Ralf — Tue, 28 May 2013 08:29:03 +0000

Octeon 2 support CONFIG_NO_HZ_IDLE but not yet CONFIG_NO_HZ_FULL. Working on that.

Does Tickless supports for Octeon-2 and ARM as well ?

ajaycavium — Fri, 24 May 2013 07:56:51 +0000

Can anybody please tell this feature is supported for Octeon-2 and ARM as well or it is for x86 only.

(Nearly) full tickless operation in 3.10

chenlb206 — Wed, 22 May 2013 07:45:35 +0000

Thank open source guys. You are great~

ticks vs. polling

cladisch — Tue, 21 May 2013 07:49:08 +0000

> ticks to look some day as outdated as polling?

Polling is regularly checking the status, just because something that needs handling might have happened.

Ticks are polling.

Hyperthreading

marcH — Mon, 20 May 2013 23:13:11 +0000

So, ticks to look some day as outdated as polling?

(Nearly) full tickless operation in 3.10

Tobu — Tue, 14 May 2013 12:18:52 +0000

Part of the work will be to adjust the various accumulators, statistics, governors and feedback mechanisms to work when the timeslice they measure isn't constant any more.

Hyperthreading

chloe_zen — Mon, 13 May 2013 21:46:04 +0000

I think you haven't been keeping up with current events (no pun intended); event-driven loops in a single OS thread are the New Way of getting the most I/O through a CPU. This is a realistically useful optimization.

(Nearly) full tickless operation in 3.10

dps — Mon, 13 May 2013 11:57:25 +0000

I believe that real tickless operation with pre-emptition is possible. Instead of recalculating that the same process keeps the CPU every n microseconds, until it does not, one could compute when another process would win the CPU and only schedule an interrupt and that time. Any periods in which the process would keep the CPU does not need interruption.

I know actually implementing this would not be trivial and I am not volunteering to do this myself.

Hyperthreading

PaulMcKenney — Fri, 10 May 2013 18:59:28 +0000

Cute, but inaccurate. ;-)

If there is only one CPU-bound task runnable on a given CPU, there is no point in any scheduling decisions.

If there are multiple tasks runnable on a given CPU, and if the currently running task is CPU-bound, then there is no point in any scheduling decisions until the next timeslice.

Of course, things might change in the meantime, but in that case, this CPU will receive an interrupt and can therefore adjust as appropriate at that point in time.

Hyperthreading

akeane — Fri, 10 May 2013 08:43:33 +0000

The config option should be called: CONFIG_WINDOWS_311

Really, "cooperative" multi-tasking?

Really?

Really...

(Nearly) full tickless operation in 3.10

bagder — Thu, 09 May 2013 20:58:33 +0000

There's also code for (32bit) ARM as well that works. I'm not sure exactly in which tree/state it is though.

Hyperthreading

drago01 — Thu, 09 May 2013 15:59:33 +0000

I doubt this will get you any noticeable power savings if at all. The most interesting part for power saving is the idle part which is already handled by CONFIG_NO_HZ

Hyperthreading

sheepdestroyer — Thu, 09 May 2013 13:14:21 +0000

Also interested by that particular issue. Got a SandyBridge ultra-portable so 2 "real" cores + 2 HT and would like to know how much if any power save should i expect?

(Nearly) full tickless operation in 3.10

nevets — Thu, 09 May 2013 02:16:40 +0000

It's confusing because there's two things at play here. There's the hrtick and the scheduler_tick.

The hrtick is used to denote exact time slices for the CFS scheduler to create more fairness. It really doesn't do much more than that. But this does not replace the scheduler_tick, which does among other things, keeps track of the SCHED_RR time slices, manages load balancing, and updates task timings.

But I'm sure in the future the hrtick may be used more to get rid of the periodic tick.

Hyperthreading

bgmarete — Thu, 09 May 2013 01:15:55 +0000

Thanks for the reply Corbet. I have just finished reading a long series of articles, including some at Intel, which make your point clear.

I would still like to know if each hyperthread can be independently put into full tickless mode (independent, that is, from its sibling), with the attendant power savings (if any). (Assume that I am interested only in saving battery power).

One-second tick

simlo — Thu, 09 May 2013 00:18:07 +0000

Hmm, wouldn't it be more appropiate just to make sure the timer is set at a maximum of 1 second in the future after last time the scheduler ran instead of being run every second?

Hyperthreading

corbet — Wed, 08 May 2013 23:05:32 +0000

If you want tickless for latency reasons, the last thing you're going to do is turn on hyperthreading.

(Nearly) full tickless operation in 3.10

bgmarete — Wed, 08 May 2013 22:56:21 +0000

With regard to the necessity of setting CPU affinities so that processors marked for tickless operation do not acquire more than one process, wouldn't something like systemd ease the related administrative work? After all, I don't want to be messing around with taskset(1) on my laptop.

Also, how does this relate to CPU hyperthreads? Can a hyperthread be in tickless mode while a sibling hyperthread is not? Or must the entire core be woken up (or not) by the timer ticks?

(Nearly) full tickless operation in 3.10

intgr — Wed, 08 May 2013 20:23:50 +0000

Sorry, it's much more likely that I am missing something.

AFAICT the scheduler doesn't switch tasks at every timer tick, even when there is contention for a CPU -- it has its own concept of timeslice length that changes with load. So why does a contended CPU need to run the timer tick if it's not going to switch tasks?

And grandparent wrote something that seemed to match that line of thinking:

> After a little RTFC, I found that a HR-timer was used to calculate the next preemption point. I.e. instead of preempting on 100 Hz clock, it preempts exactly when the timeslot of the current process ends.

(Nearly) full tickless operation in 3.10

corbet — Wed, 08 May 2013 19:52:13 +0000

Confused. If two processes are running, the period tick will run just like it does now. What am I missing? What is misleading?

(Nearly) full tickless operation in 3.10

intgr — Wed, 08 May 2013 19:48:29 +0000

I got the same impression as you. I think the wording in the article is unfortunate.

> Even then, a running CPU will only disable the timer tick if there is a single runnable process on that CPU. As soon as a second process appears, the tick is needed so that the scheduler can make the necessary time-slice decisions

It's misleading to call it "the tick" if it's not fixed to the HZ any more, seems more like a preemption timer.

(Nearly) full tickless operation in 3.10

blitzkrieg3 — Wed, 08 May 2013 18:50:28 +0000

Okay, this makes absolutely perfect sense, but because of the way operating systems have been designed for the past 40 years, that solution completely eluded me.

One-second tick

corbet — Wed, 08 May 2013 18:29:12 +0000

See this commit. One tick per second is still needed.

(Nearly) full tickless operation in 3.10

gby — Wed, 08 May 2013 18:02:23 +0000

There is no inherent one second timer tick except on on a single CPU on the system. I think you got this wrong.

(Nearly) full tickless operation in 3.10

linuxjacques — Wed, 08 May 2013 16:30:25 +0000

The 64-bit only constraint makes me wonder how much this has been tested on non-x86 archs.

I'm interested in it for 32-bit ppc.

(Nearly) full tickless operation in 3.10

simlo — Wed, 08 May 2013 16:24:34 +0000

After a little RTFC, I found that a HR-timer was used to calculate the next preemption point. I.e. instead of preempting on 100 Hz clock, it preempts exactly when the timeslot of the current process ends.

(Nearly) full tickless operation in 3.10

busterb — Wed, 08 May 2013 16:22:24 +0000

On a related note, it seems like quite a few multi-core SoC vendors have implemented their own patches for this behavior as well, e.g. Tilera's 'zero overhead' Linux. Glad to see if it becomes mainstream, maybe they'll all adopt just one way to do it.

http://www.6windblog.com/linux-based-fast-path/

(Nearly) full tickless operation in 3.10

busterb — Wed, 08 May 2013 16:12:46 +0000

So, what's the solution for running >1 process per CPU with NO_HZ_FULL? Use syscalls as the hook for an implicit schedule 'pump'?

Requiring the user to explicitly schedule with 'sched_yield' to produce a cooperative multitasking scenario might work well. I've worked on enough realtime systems to know that this can yield good performance if done correctly (since your performance-critical sections never get interrupted without you saying so), though it occasionally bites you if you forget to yield somewhere, e.g. waiting for a lock. I know I would have liked to have had something like this in past system designs Instead, 'yielding' was done with coroutines or tasklets/threads.

What about having the boot cpu do all the scheduling for all the other CPUs? Basically treat the other CPUs like a thread pool and distribute processes. Is the cost of IPIs too much for this to be practical?