locking, sched: The PREEMPT-RT locking infrastructure

Folks,

the following series is an update to V2 which can be found here:

  https://lore.kernel.org/r/20210713151054.700719949@linutr...

It contains the bulk of the PREEMPT-RT locking infrastructure. In
PREEMPT-RT enabled kernels the following locking primitives are substituted
by RT-Mutex based variants:

  mutex, ww_mutex, rw_semaphore, spinlock, rwlock

semaphores are not substituted because they do not provide strict owner
semantics.

Of course raw_spinlocks are not touched either as they protect low level
operations in the scheduler, timers and hardware access.

The most interesting parts of the series which need a lot of eyeballs
are:

  - the scheduler bits which provide the infrastructure for spinlock and
    rwlock substitution to ensure that the task state is preserved when
    blocking on such a lock and a regular wakeup is handled correctly and
    not lost

  - the rtmutex core implementation to handle lock contention on spinlocks
    and rwlocks correctly vs. the task state

  - the rw_semaphore/rwlock substitutions which utilize the same
    implementation vs. the reader/writer handling

  - The new rtmutex based ww_mutex implementation.

  - the PI futex related bits to handle the interaction between blocking
    on the underlying rtmutex and contention on the hash bucket lock which
    is converted to a 'sleeping spinlock'.

The rest surely needs a thorough review as well, but those parts are pretty
straight forward: quite some code restructuring and the actual wrapper
functions to replace the existing !RT implementations.

The series survived internal testing in RT kernels and is part of the
upcoming v5.14-rc4-rt5 release.

For !RT kernels there is no functional change.

The series is also available from git:

  git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git rtmutex

and fully integrated into the upcoming v5.14-rc4-rt5 release.

Changes vs. V2:

  - Rework the task state change helpers (Waiman)

  - Add lockdep asserts for paranoia sake (Peter)

  - Revert the double wake_q implementation as it is broken
    and go back to the V1 implementation of rt_wake_q_head (Mike)

  - Fix the UNinterruptible copy & pasta fail in ww_mutex (Mike)

  - Pick up the futex changes from Peter

  - Remove duplicate defines (Waiman)

  - Fold the spin wait changes into one patch and rewrite changelog (Peter)

  - Fix typos all over the place (Daniel)

Thanks,

	tglx
---
 b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c |    4 
 b/include/linux/debug_locks.h                       |    3 
 b/include/linux/mutex.h                             |   93 +
 b/include/linux/preempt.h                           |    4 
 b/include/linux/rbtree.h                            |   30 
 b/include/linux/rbtree_types.h                      |   34 
 b/include/linux/rtmutex.h                           |   55 -
 b/include/linux/rwbase_rt.h                         |   38 
 b/include/linux/rwlock_rt.h                         |  140 ++
 b/include/linux/rwlock_types.h                      |   39 
 b/include/linux/rwsem.h                             |   58 +
 b/include/linux/sched.h                             |  119 +-
 b/include/linux/sched/wake_q.h                      |    8 
 b/include/linux/spinlock.h                          |   15 
 b/include/linux/spinlock_api_smp.h                  |    3 
 b/include/linux/spinlock_rt.h                       |  151 ++
 b/include/linux/spinlock_types.h                    |   83 -
 b/include/linux/spinlock_types_raw.h                |   65 +
 b/include/linux/ww_mutex.h                          |   50 
 b/kernel/Kconfig.locks                              |    2 
 b/kernel/futex.c                                    |  488 ++++++--
 b/kernel/locking/Makefile                           |    3 
 b/kernel/locking/mutex-debug.c                      |    5 
 b/kernel/locking/mutex.c                            |  431 -------
 b/kernel/locking/mutex.h                            |   33 
 b/kernel/locking/rtmutex.c                          | 1095 ++++++++------------
 b/kernel/locking/rtmutex_api.c                      |  590 ++++++++++
 b/kernel/locking/rtmutex_common.h                   |  122 +-
 b/kernel/locking/rwbase_rt.c                        |  263 ++++
 b/kernel/locking/rwsem.c                            |  109 +
 b/kernel/locking/spinlock.c                         |    7 
 b/kernel/locking/spinlock_debug.c                   |    5 
 b/kernel/locking/spinlock_rt.c                      |  257 ++++
 b/kernel/locking/ww_mutex.h                         |  569 ++++++++++
 b/kernel/locking/ww_rt_mutex.c                      |   76 +
 b/kernel/rcu/tree_plugin.h                          |    6 
 b/kernel/sched/core.c                               |  109 +
 b/lib/Kconfig.debug                                 |   11 
 b/lib/test_lockup.c                                 |    8 
 kernel/locking/mutex-debug.h                        |   29 
 40 files changed, 3759 insertions(+), 1451 deletions(-)