VDUSE: Support registering userspace memory as bounce buffer
From: | Xie Yongji <xieyongji-AT-bytedance.com> | |
To: | mst-AT-redhat.com, jasowang-AT-redhat.com, xiaodong.liu-AT-intel.com, maxime.coquelin-AT-redhat.com, stefanha-AT-redhat.com | |
Subject: | [PATCH v2 0/5] VDUSE: Support registering userspace memory as bounce buffer | |
Date: | Wed, 06 Jul 2022 13:04:58 +0800 | |
Message-ID: | <20220706050503.171-1-xieyongji@bytedance.com> | |
Cc: | songmuchun-AT-bytedance.com, virtualization-AT-lists.linux-foundation.org, linux-kernel-AT-vger.kernel.org | |
Archive-link: | Article |
Hi all, This series introduces some new ioctls: VDUSE_IOTLB_GET_INFO, VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM to support registering and de-registering userspace memory for IOTLB as bounce buffer in virtio-vdpa case. The VDUSE_IOTLB_GET_INFO ioctl can help user to query IOLTB information such as bounce buffer size. Then user can use those information on VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM ioctls to register and de-register userspace memory for IOTLB. During registering and de-registering, the DMA data in use would be copied from kernel bounce pages to userspace bounce pages and back. With this feature, some existing application such as SPDK and DPDK can leverage the datapath of VDUSE directly and efficiently as discussed before [1][2]. They can register some preallocated hugepages to VDUSE to avoid an extra memcpy from bounce-buffer to hugepages. The kernel and userspace codes could be found in github: https://github.com/bytedance/linux/tree/vduse-umem https://github.com/bytedance/qemu/tree/vduse-umem To test it with qemu-storage-daemon: $ qemu-storage-daemon \ --chardev socket,id=charmonitor,path=/tmp/qmp.sock,server=on,wait=off \ --monitor chardev=charmonitor \ --blockdev driver=host_device,cache.direct=on,aio=native,filename=/dev/nullb0,node-name=disk0 \ --export type=vduse-blk,id=vduse-test,name=vduse-test,node-name=disk0,writable=on [1] https://lkml.org/lkml/2021/6/27/318 [2] https://lkml.org/lkml/2022/7/4/246 Please review, thanks! V1 to V2: - Drop the patch that updating API version [MST] - Replace unpin_user_pages() with unpin_user_pages_dirty_lock() [MST] - Use __vmalloc(__GFP_ACCOUNT) for memory accounting [MST] Xie Yongji (5): vduse: Remove unnecessary spin lock protection vduse: Use memcpy_{to,from}_page() in do_bounce() vduse: Support using userspace pages as bounce buffer vduse: Support querying IOLTB information vduse: Support registering userspace memory for IOTLB drivers/vdpa/vdpa_user/iova_domain.c | 134 ++++++++++++++++++++--- drivers/vdpa/vdpa_user/iova_domain.h | 9 ++ drivers/vdpa/vdpa_user/vduse_dev.c | 152 +++++++++++++++++++++++++++ include/uapi/linux/vduse.h | 45 ++++++++ 4 files changed, 327 insertions(+), 13 deletions(-) -- 2.20.1