| From: |
| Jason Gunthorpe <jgg-AT-nvidia.com> |
| To: |
| Leon Romanovsky <leon-AT-kernel.org>, linux-rdma-AT-vger.kernel.org |
| Subject: |
| [PATCH 00/10] Fix races around IB_MR_REREG_PD and mr->pd |
| Date: |
| Wed, 03 Jun 2026 22:27:39 -0300 |
| Message-ID: |
| <0-v1-29ebd2c229b5+fd5-ib_mr_pd_jgg@nvidia.com> |
| Cc: |
| Doug Ledford <dledford-AT-redhat.com>, Edward Srouji <edwards-AT-nvidia.com>, Leon Romanovsky <leonro-AT-mellanox.com>, Leon Romanovsky <leonro-AT-nvidia.com>, Matan Barak <matanb-AT-mellanox.com>, Michael Guralnik <michaelgur-AT-nvidia.com>, Noa Osherovich <noaos-AT-mellanox.com>, patches-AT-lists.linux.dev, Steve Wise <swise-AT-opengridcomputing.com> |
| Archive-link: |
| Article |
Sashiko pointed out an existing bug related to mr->pd: when IB_MR_REREG_PD
is used the mr->pd is changed while only holding the write side of the
MR's uobject lock.
Effectively, because IB_MR_REREG_PD is usually implemented by changing the
MR in-place, the mr->pd becomes unreadable outside an MR-specific system
call that holds the uobject lock. All the readers in this series could
race with an IB_MR_REREG_PD and potentially UAF the mr->pd.
https://sashiko.dev/#/patchset/20260427-security-bug-fixe...
This was presented as a simple 'oh look it can race with nldev' which is
correct. However, asking AI to fully audit mr->pd touches also revealed a
much more convoluted issue inside mlx5 ODP that is also using mr->pd from
the page fault work queue, advise mr work queue and advise mr system call
without any locking.
It turns out this mlx5 problem is entirely unnecessary since outside
implicit mr there are only three cases where the UMR actually flags the
PDN to be read by HW, umr_rereg_pas(), mlx5_ib_init_odp_mr() and
mlx5_ib_init_dmabuf_mr(). umr_rereg_pas() is particularly tricky because
it illegaly updates mr->pd inside the driver. Reorganize the giant call
chain from mlx5r_umr_set_update_xlt_mkey_seg() upward so that pdn is
passed down from those three functions instead of unconditionally picked
out at the bottom.
nldev however is trickier to fix. To avoid disurbing the happy paths build
a synchronize barrier by removing the mr from the xarray and then putting
it right back. The kref completion acts as a positive signal that the
mr->pd is no longer being used.
Jason Gunthorpe (10):
IB/mlx5: Don't take the rereg_mr fallback without a new translation
RDMA/mlx5: Create ODP EQ for non-pinned dmabuf MRs
IB/mlx5: Properly support implicit ODP rereg_mr
RDMA/nldev: Fix locking when accessing mr->pd
IB/mlx5: Remove unused mkc bits in mlx5r_umr_update_mr_page_shift()
IB/mlx5: Pull the pdn out of the depths of the umr machinery
IB/mlx5: Don't mangle the mr->pd inside the rereg callback
IB/mlx5: Push pdn above mlx5r_umr_update_xlt()
IB/mlx5: Push pdn above pagfault_real_mr()
IB/mlx5: Push pdn above pagefault_dmabuf_mr()
drivers/infiniband/core/nldev.c | 15 +++--
drivers/infiniband/core/restrack.c | 49 +++++++++++++++
drivers/infiniband/core/restrack.h | 1 +
drivers/infiniband/core/uverbs_cmd.c | 10 ++-
drivers/infiniband/hw/mlx5/mlx5_ib.h | 12 ++--
drivers/infiniband/hw/mlx5/mr.c | 37 ++++++++---
drivers/infiniband/hw/mlx5/odp.c | 82 +++++++++++++++----------
drivers/infiniband/hw/mlx5/umr.c | 92 +++++++++++++---------------
drivers/infiniband/hw/mlx5/umr.h | 11 ++--
include/rdma/ib_verbs.h | 5 ++
10 files changed, 203 insertions(+), 111 deletions(-)
base-commit: e43ffb69e0438cddd72aaa30898b4dc446f664f8
--
2.43.0
