| From: |
| Vadim Fedorenko <vadfed-AT-meta.com> |
| To: |
| Borislav Petkov <bp-AT-alien8.de>, Alexei Starovoitov <ast-AT-kernel.org>, Daniel Borkmann <daniel-AT-iogearbox.net>, Andrii Nakryiko <andrii-AT-kernel.org>, Eduard Zingerman <eddyz87-AT-gmail.com>, Thomas Gleixner <tglx-AT-linutronix.de>, Yonghong Song <yonghong.song-AT-linux.dev>, Vadim Fedorenko <vadim.fedorenko-AT-linux.dev>, Mykola Lysenko <mykolal-AT-fb.com> |
| Subject: |
| [PATCH bpf-next v8 0/4] bpf: add cpu cycles kfuncss |
| Date: |
| Wed, 20 Nov 2024 16:08:10 -0800 |
| Message-ID: |
| <20241121000814.3821326-1-vadfed@meta.com> |
| Cc: |
| <x86-AT-kernel.org>, <bpf-AT-vger.kernel.org>, Peter Zijlstra <peterz-AT-infradead.org>, Vadim Fedorenko <vadfed-AT-meta.com>, Martin KaFai Lau <martin.lau-AT-linux.dev> |
| Archive-link: |
| Article |
This patchset adds 2 kfuncs to provide a way to precisely measure the
time spent running some code. The first patch provides a way to get cpu
cycles counter which is used to feed CLOCK_MONOTONIC_RAW. On x86
architecture it is effectively rdtsc_ordered() function while on other
architectures it falls back to __arch_get_hw_counter(). The second patch
adds a kfunc to convert cpu cycles to nanoseconds using shift/mult
constants discovered by kernel. The main use-case for this kfunc is to
convert deltas of timestamp counter values into nanoseconds. It is not
supposed to get CLOCK_MONOTONIC_RAW values as offset part is skipped.
JIT version is done for x86 for now, on other architectures it falls
back to slightly simplified version of vdso_calc_ns.
Selftests are also added to check whether the JIT implementation is
correct and to show the simplest usage example.
Change log:
v7 -> v8:
* rename kfuncs again to bpf_get_cpu_time_counter() and
bpf_cpu_time_counter_to_ns()
* use cyc2ns_read_begin()/cyc2ns_read_end() to get mult and shift
constants in bpf_cpu_time_counter_to_ns()
v6 -> v7:
* change boot_cpu_has() to cpu_feature_enabled() (Borislav)
* return constant clock_mode in __arch_get_hw_counter() call
v5 -> v6:
* added cover letter
* add comment about dropping S64_MAX manipulation in jitted
implementation of rdtsc_oredered (Alexey)
* add comment about using 'lfence;rdtsc' variant (Alexey)
* change the check in fixup_kfunc_call() (Eduard)
* make __arch_get_hw_counter() call more aligned with vDSO
implementation (Yonghong)
v4 -> v5:
* use #if instead of #ifdef with IS_ENABLED
v3 -> v4:
* change name of the helper to bpf_get_cpu_cycles (Andrii)
* Hide the helper behind CONFIG_GENERIC_GETTIMEOFDAY to avoid exposing
it on architectures which do not have vDSO functions and data
* reduce the scope of check of inlined functions in verifier to only 2,
which are actually inlined.
* change helper name to bpf_cpu_cycles_to_ns.
* hide it behind CONFIG_GENERIC_GETTIMEOFDAY to avoid exposing on
unsupported architectures.
v2 -> v3:
* change name of the helper to bpf_get_cpu_cycles_counter to
explicitly mention what counter it provides (Andrii)
* move kfunc definition to bpf.h to use it in JIT.
* introduce another kfunc to convert cycles into nanoseconds as
more meaningful time units for generic tracing use case (Andrii)
v1 -> v2:
* Fix incorrect function return value type to u64
* Introduce bpf_jit_inlines_kfunc_call() and use it in
mark_fastcall_pattern_for_call() to avoid clobbering in case
of running programs with no JIT (Eduard)
* Avoid rewriting instruction and check function pointer directly
in JIT (Alexei)
* Change includes to fix compile issues on non x86 architectures
Vadim Fedorenko (4):
bpf: add bpf_get_cpu_time_counter kfunc
bpf: add bpf_cpu_time_counter_to_ns helper
selftests/bpf: add selftest to check rdtsc jit
selftests/bpf: add usage example for cpu cycles kfuncs
arch/x86/net/bpf_jit_comp.c | 66 +++++++++++
arch/x86/net/bpf_jit_comp32.c | 41 +++++++
include/linux/bpf.h | 6 +
include/linux/filter.h | 1 +
kernel/bpf/core.c | 11 ++
kernel/bpf/helpers.c | 39 +++++++
kernel/bpf/verifier.c | 41 ++++++-
.../bpf/prog_tests/test_cpu_cycles.c | 35 ++++++
.../selftests/bpf/prog_tests/verifier.c | 2 +
.../selftests/bpf/progs/test_cpu_cycles.c | 25 +++++
.../selftests/bpf/progs/verifier_cpu_cycles.c | 104 ++++++++++++++++++
11 files changed, 365 insertions(+), 6 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/test_cpu_cycles.c
create mode 100644 tools/testing/selftests/bpf/progs/test_cpu_cycles.c
create mode 100644 tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c
--
2.43.5