| From: |
| ming.qian-AT-oss.nxp.com |
| To: |
| linux-media-AT-vger.kernel.org |
| Subject: |
| [RFC PATCH 0/7] media: amphion: Add DMA memory tracking support |
| Date: |
| Tue, 31 Mar 2026 15:23:10 +0800 |
| Message-ID: |
| <20260331072347.253-1-ming.qian@oss.nxp.com> |
| Cc: |
| mchehab-AT-kernel.org, hverkuil-cisco-AT-xs4all.nl, nicolas-AT-ndufresne.ca, sebastian.fricke-AT-collabora.com, shawnguo-AT-kernel.org, s.hauer-AT-pengutronix.de, kernel-AT-pengutronix.de, festevam-AT-gmail.com, linux-imx-AT-nxp.com, Frank.li-AT-nxp.com, xiahong.bao-AT-nxp.com, eagle.zhou-AT-nxp.com, imx-AT-lists.linux.dev, linux-kernel-AT-vger.kernel.org, linux-arm-kernel-AT-lists.infradead.org |
| Archive-link: |
| Article |
From: Ming Qian <ming.qian@oss.nxp.com>
This series adds DMA memory tracking support to the Amphion VPU driver,
allowing userspace to monitor memory consumption of codec instances.
To implement this feature in a reusable way, a new V4L2 memory tracking
infrastructure (v4l2-memtrack) is introduced, which can be adopted by
other V4L2 drivers in the future.
Background
==========
The Amphion VPU codec driver allocates various DMA buffers internally:
- Firmware boot region and RPC buffers
- Stream ring buffers
- Codec-specific buffers (MBI, DCP, reference frames)
- Encoder/decoder working buffers
These allocations are not visible to userspace, making it difficult to
debug memory issues or monitor resource usage in multi-instance scenarios
(e.g., transcoding servers with multiple simultaneous streams).
Solution
========
This series introduces:
1. V4L2_CID_MEMORY_USAGE control
- New read-only control reporting memory usage in bytes
- Standard V4L2 interface, queryable via VIDIOC_G_CTRL
2. v4l2-memtrack infrastructure
- Hierarchical memory tracking (device -> instance -> queue)
- debugfs interface at /sys/kernel/debug/v4l2-memtrack/
- Reusable by other V4L2 drivers
3. videobuf2 integration
- Automatic tracking of vb2 buffer allocations
4. Amphion VPU integration
- Tracks all internal DMA allocations
- Organized as: device -> instance -> buffers
Ming Qian (7):
media: v4l2-ctrls: Add V4L2_CID_MEMORY_USAGE control
docs: media: v4l2-ctrls: Add V4L2_CID_MEMORY_USAGE control
media: v4l2-memtrack: Add V4L2 memory tracking infrastructure
docs: media: v4l2-memtrack: Add driver API documentation
MAINTAINERS: Add entry for V4L2 memory usage tracker
media: videobuf2: Add memory tracking support
media: amphion: Add V4L2 memory tracking support
Documentation/driver-api/media/v4l2-core.rst | 1 +
.../driver-api/media/v4l2-memtrack.rst | 140 +++
.../userspace-api/media/v4l/control.rst | 22 +-
MAINTAINERS | 8 +
drivers/media/common/Kconfig | 1 +
drivers/media/common/Makefile | 2 +-
drivers/media/common/v4l2-memtrack/Kconfig | 19 +
drivers/media/common/v4l2-memtrack/Makefile | 3 +
.../common/v4l2-memtrack/v4l2-memtrack.c | 825 ++++++++++++++++++
.../media/common/videobuf2/videobuf2-core.c | 13 +
drivers/media/platform/amphion/Kconfig | 1 +
drivers/media/platform/amphion/vdec.c | 9 +
drivers/media/platform/amphion/venc.c | 9 +
drivers/media/platform/amphion/vpu.h | 7 +
drivers/media/platform/amphion/vpu_core.c | 6 +
drivers/media/platform/amphion/vpu_dbg.c | 5 +
drivers/media/platform/amphion/vpu_drv.c | 2 +
drivers/media/platform/amphion/vpu_v4l2.c | 35 +-
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
include/media/v4l2-memtrack.h | 220 +++++
include/media/videobuf2-core.h | 4 +
include/uapi/linux/v4l2-controls.h | 4 +-
22 files changed, 1340 insertions(+), 4 deletions(-)
create mode 100644 Documentation/driver-api/media/v4l2-memtrack.rst
create mode 100644 drivers/media/common/v4l2-memtrack/Kconfig
create mode 100644 drivers/media/common/v4l2-memtrack/Makefile
create mode 100644 drivers/media/common/v4l2-memtrack/v4l2-memtrack.c
create mode 100644 include/media/v4l2-memtrack.h
base-commit: 4fbeef21f5387234111b5d52924e77757626faa5
prerequisite-patch-id: 0000000000000000000000000000000000000000
--
2.53.0