| From: |
| Brian Gerst <brgerst-AT-gmail.com> |
| To: |
| linux-kernel-AT-vger.kernel.org, x86-AT-kernel.org |
| Subject: |
| [PATCH v3 00/11] Add a percpu subsection for cache hot data |
| Date: |
| Mon, 03 Mar 2025 11:52:35 -0500 |
| Message-ID: |
| <20250303165246.2175811-1-brgerst@gmail.com> |
| Cc: |
| Ingo Molnar <mingo-AT-kernel.org>, "H . Peter Anvin" <hpa-AT-zytor.com>, Thomas Gleixner <tglx-AT-linutronix.de>, Borislav Petkov <bp-AT-alien8.de>, Ard Biesheuvel <ardb-AT-kernel.org>, Uros Bizjak <ubizjak-AT-gmail.com>, Linus Torvalds <torvalds-AT-linuxfoundation.org>, Andy Lutomirski <luto-AT-kernel.org>, Peter Zijlstra <peterz-AT-infradead.org>, Andrew Morton <akpm-AT-linux-foundation.org>, Brian Gerst <brgerst-AT-gmail.com> |
| Archive-link: |
| Article |
Add a new percpu subsection for data that is frequently accessed and
exclusive to each processor. This replaces the pcpu_hot struct on x86,
and is available to all architectures and the core kernel.
ffffffff834f5000 D __per_cpu_hot_start
ffffffff834f5000 D hardirq_stack_ptr
ffffffff834f5008 D __ref_stack_chk_guard
ffffffff834f5008 D __stack_chk_guard
ffffffff834f5010 D const_cpu_current_top_of_stack
ffffffff834f5010 D cpu_current_top_of_stack
ffffffff834f5018 D const_current_task
ffffffff834f5018 D current_task
ffffffff834f5020 D __x86_call_depth
ffffffff834f5028 D this_cpu_off
ffffffff834f5030 D __preempt_count
ffffffff834f5034 D cpu_number
ffffffff834f5038 D __softirq_pending
ffffffff834f503a D hardirq_stack_inuse
ffffffff834f503b D __per_cpu_hot_pad
ffffffff834f5040 D __per_cpu_hot_end
This applies to the tip/x86/asm branch.
Changes in v3:
- Fix typo of CACHE_HOT_DATA()
- Move hardirq_stack_inuse to irq_64.c
- Add __per_cpu_hot_pad to show the end of the actual data
Brian Gerst (11):
percpu: Introduce percpu hot section
x86/percpu: Move pcpu_hot to percpu hot section
x86/preempt: Move preempt count to percpu hot section
x86/smp: Move cpu number to percpu hot section
x86/retbleed: Move call depth to percpu hot section
x86/softirq: Move softirq_pending to percpu hot section
x86/irq: Move irq stacks to percpu hot section
x86/percpu: Move top_of_stack to percpu hot section
x86/percpu: Move current_task to percpu hot section
x86/stackprotector: Move __stack_chk_guard to percpu hot section
x86/smp: Move this_cpu_off to percpu hot section
arch/x86/entry/entry_32.S | 4 +--
arch/x86/entry/entry_64.S | 6 ++---
arch/x86/entry/entry_64_compat.S | 4 +--
arch/x86/include/asm/current.h | 36 +++++----------------------
arch/x86/include/asm/hardirq.h | 4 +--
arch/x86/include/asm/irq_stack.h | 12 ++++-----
arch/x86/include/asm/nospec-branch.h | 11 ++++----
arch/x86/include/asm/percpu.h | 4 +--
arch/x86/include/asm/preempt.h | 25 ++++++++++---------
arch/x86/include/asm/processor.h | 16 ++++++++++--
arch/x86/include/asm/smp.h | 7 +++---
arch/x86/include/asm/stackprotector.h | 2 +-
arch/x86/kernel/asm-offsets.c | 5 ----
arch/x86/kernel/cpu/common.c | 25 +++++++++++++------
arch/x86/kernel/dumpstack_32.c | 4 +--
arch/x86/kernel/dumpstack_64.c | 2 +-
arch/x86/kernel/head_64.S | 4 +--
arch/x86/kernel/irq.c | 5 ++++
arch/x86/kernel/irq_32.c | 12 +++++----
arch/x86/kernel/irq_64.c | 7 +++---
arch/x86/kernel/process_32.c | 6 ++---
arch/x86/kernel/process_64.c | 6 ++---
arch/x86/kernel/setup_percpu.c | 7 ++++--
arch/x86/kernel/smpboot.c | 4 +--
arch/x86/kernel/vmlinux.lds.S | 6 ++++-
arch/x86/lib/retpoline.S | 2 +-
include/asm-generic/vmlinux.lds.h | 11 ++++++++
include/linux/percpu-defs.h | 13 ++++++++++
include/linux/preempt.h | 1 +
kernel/bpf/verifier.c | 4 +--
scripts/gdb/linux/cpus.py | 2 +-
31 files changed, 146 insertions(+), 111 deletions(-)
base-commit: 693c8502970a533363e9ece482c80bb6db0c12a5
--
2.48.1
