io_uring: add IORING_OP_BPF for extending io_uring

Hello,

Add IORING_OP_BPF for extending io_uring operations, follows typical cases:

- buffer registered zero copy [1]

Also there are some RAID like ublk servers which needs to generate data
parity in case of ublk zero copy

- extend io_uring operations from application

Easy to add one new syscall with IORING_OP_BPF

- extend 64 byte SQE

bpf map can store IO data conveniently

- communicate in IO chain

IORING_OP_BPF can be used for communicate among IOs seamlessly without requiring
extra syscall

- pretty handy to inject error for test purpose

Any comments & feedback are welcome!


[1] lpc2024: ublk based zero copy I/O - use case in Android

https://lpc.events/event/18/contributions/1710/attachment...

V3:
	- add per-buffer iterator kfuncs for addressing Caleb's concern about
	  complicated kfunc interface, kernel mapped buffer produced by the
	  iterator can be read/written from bpf prog in zero copy style

	- rename to bpf_ext

	- change tests to copy between uring_buf and arena, which may match
	  with raid's use case

	- misc cleanup


V2:
	- per-ring struct ops (Stefan Metzmacher, Caleb Sander Mateos)
	- refactor io_import_fixed()/io_prep_reg_iovec()/io_import_reg_vec()
	  for allowing to handle multiple buffers for single request
	- kernel selftests
	- all kinds of comments from Caleb Sander Mateos
	- support vectored and registered vector buffer



Ming Lei (12):
  io_uring: make io_import_fixed() global
  io_uring: refactor io_prep_reg_iovec() for BPF kfunc use
  io_uring: refactor io_import_reg_vec() for BPF kfunc use
  io_uring: prepare for extending io_uring with bpf
  io_uring: bpf: extend io_uring with bpf struct_ops
  io_uring: bpf: implement struct_ops registration
  io_uring: bpf: add BPF buffer descriptor for IORING_OP_BPF
  io_uring: bpf: add per-buffer iterator kfuncs
  bpf: add bpf_uring_buf_dynptr to special_kfunc_list
  selftests/io_uring: add io_uring_unregister_buffers()
  selftests/io_uring: add BPF struct_ops and kfunc tests
  selftests/io_uring: add buffer iterator selftest with BPF arena

 include/linux/io_uring_types.h                |  14 +-
 include/uapi/linux/io_uring.h                 |  40 +
 init/Kconfig                                  |   7 +
 io_uring/Makefile                             |   1 +
 io_uring/bpf-ops.c                            |   7 +-
 io_uring/bpf_ext.c                            | 755 ++++++++++++++++++
 io_uring/bpf_ext.h                            |  71 ++
 io_uring/io_uring.c                           |   9 +-
 io_uring/io_uring.h                           |   6 +-
 io_uring/opdef.c                              |  16 +
 io_uring/rsrc.c                               |  46 +-
 io_uring/rsrc.h                               |  68 +-
 kernel/bpf/verifier.c                         |  12 +
 tools/include/io_uring/mini_liburing.h        |  10 +
 tools/testing/selftests/Makefile              |   3 +-
 tools/testing/selftests/io_uring/.gitignore   |   2 +
 tools/testing/selftests/io_uring/Makefile     | 173 ++++
 .../selftests/io_uring/bpf_ext_basic.bpf.c    |  94 +++
 .../selftests/io_uring/bpf_ext_basic.c        | 215 +++++
 .../selftests/io_uring/bpf_ext_memcpy.bpf.c   | 305 +++++++
 .../selftests/io_uring/bpf_ext_memcpy.c       | 517 ++++++++++++
 .../io_uring/include/bpf_ext_memcpy_defs.h    |  18 +
 .../selftests/io_uring/include/iou_test.h     |  98 +++
 tools/testing/selftests/io_uring/runner.c     | 206 +++++
 24 files changed, 2646 insertions(+), 47 deletions(-)
 create mode 100644 io_uring/bpf_ext.c
 create mode 100644 io_uring/bpf_ext.h
 create mode 100644 tools/testing/selftests/io_uring/.gitignore
 create mode 100644 tools/testing/selftests/io_uring/Makefile
 create mode 100644 tools/testing/selftests/io_uring/bpf_ext_basic.bpf.c
 create mode 100644 tools/testing/selftests/io_uring/bpf_ext_basic.c
 create mode 100644 tools/testing/selftests/io_uring/bpf_ext_memcpy.bpf.c
 create mode 100644 tools/testing/selftests/io_uring/bpf_ext_memcpy.c
 create mode 100644 tools/testing/selftests/io_uring/include/bpf_ext_memcpy_defs.h
 create mode 100644 tools/testing/selftests/io_uring/include/iou_test.h
 create mode 100644 tools/testing/selftests/io_uring/runner.c

-- 
2.53.0