Per-VMA lock support for swap and userfaults
From: | Suren Baghdasaryan <surenb-AT-google.com> | |
To: | akpm-AT-linux-foundation.org | |
Subject: | [PATCH v3 0/8] Per-VMA lock support for swap and userfaults | |
Date: | Mon, 26 Jun 2023 21:23:13 -0700 | |
Message-ID: | <20230627042321.1763765-1-surenb@google.com> | |
Cc: | willy-AT-infradead.org, hannes-AT-cmpxchg.org, mhocko-AT-suse.com, josef-AT-toxicpanda.com, jack-AT-suse.cz, ldufour-AT-linux.ibm.com, laurent.dufour-AT-fr.ibm.com, michel-AT-lespinasse.org, liam.howlett-AT-oracle.com, jglisse-AT-google.com, vbabka-AT-suse.cz, minchan-AT-google.com, dave-AT-stgolabs.net, punit.agrawal-AT-bytedance.com, lstoakes-AT-gmail.com, hdanton-AT-sina.com, apopple-AT-nvidia.com, peterx-AT-redhat.com, ying.huang-AT-intel.com, david-AT-redhat.com, yuzhao-AT-google.com, dhowells-AT-redhat.com, hughd-AT-google.com, viro-AT-zeniv.linux.org.uk, brauner-AT-kernel.org, pasha.tatashin-AT-soleen.com, surenb-AT-google.com, linux-mm-AT-kvack.org, linux-fsdevel-AT-vger.kernel.org, linux-kernel-AT-vger.kernel.org, kernel-team-AT-android.com | |
Archive-link: | Article |
When per-VMA locks were introduced in [1] several types of page faults would still fall back to mmap_lock to keep the patchset simple. Among them are swap and userfault pages. The main reason for skipping those cases was the fact that mmap_lock could be dropped while handling these faults and that required additional logic to be implemented. Implement the mechanism to allow per-VMA locks to be dropped for these cases. First, change handle_mm_fault to drop per-VMA locks when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED to be consistent with the way mmap_lock is handled. Then change folio_lock_or_retry (and rename it to folio_lock_fault) to accept vm_fault, which will be used to indicate mmap_lock/per-VMA lock's state upon exit. Finally allow swap and uffd page faults to be handled under per-VMA locks by dropping per-VMA locks when waiting for a folio, the same way it's done under mmap_lock. Naturally, once VMA lock is dropped that VMA should be assumed unstable and can't be used. Changes since v2 posted at [2] - Moved prerequisite patches to the beginning (first 2 patches) - Added a new patch 3/8 to make per-VMA locks consistent with mmap_locks by dropping it on VM_FAULT_RETRY or VM_FAULT_COMPLETED. - Implemented folio_lock_fault in 4/8, per Matthew Wilcox - Replaced VM_FAULT_VMA_UNLOCKED with FAULT_FLAG_LOCK_DROPPED vmf_flag in 5/8. - Merged swap page fault handling patch with the one implementing wait for a folio into 6/8, per Peter Xu Note: patch 3/8 will cause a trivial merge conflict in arch/arm64/mm/fault.c when applied over mm-unstable branch due to a patch from ARM64 tree [3] which is missing in mm-unstable. [1] https://lore.kernel.org/all/20230227173632.3292573-1-sure... [2] https://lore.kernel.org/all/20230609005158.2421285-1-sure... [3] https://lore.kernel.org/all/20230524131305.2808-1-jszhang... Suren Baghdasaryan (8): swap: remove remnants of polling from read_swap_cache_async mm: add missing VM_FAULT_RESULT_TRACE name for VM_FAULT_COMPLETED mm: drop per-VMA lock in handle_mm_fault if retrying or when finished mm: replace folio_lock_or_retry with folio_lock_fault mm: make folio_lock_fault indicate the state of mmap_lock upon return mm: handle swap page faults under per-VMA lock mm: drop VMA lock before waiting for migration mm: handle userfaults under VMA lock arch/arm64/mm/fault.c | 3 +- arch/powerpc/mm/fault.c | 3 +- arch/s390/mm/fault.c | 3 +- arch/x86/mm/fault.c | 3 +- fs/userfaultfd.c | 42 +++++++++++++------------ include/linux/mm_types.h | 4 ++- include/linux/pagemap.h | 13 ++++---- mm/filemap.c | 55 +++++++++++++++++++-------------- mm/madvise.c | 4 +-- mm/memory.c | 66 +++++++++++++++++++++++++--------------- mm/swap.h | 1 - mm/swap_state.c | 12 +++----- 12 files changed, 120 insertions(+), 89 deletions(-) -- 2.41.0.178.g377b9f9a00-goog