Remove b_end_io from struct buffer_head

There are four benefits to this patchset.  First, it removes an
indirect function call from the completion path.  Instead of setting
bio->bi_end_io to end_bio_bh_io_sync() which then calls bh->b_end_io(),
we set bio->bi_end_io to the appropriate completion handler, replacing
two indirect function calls with one.

Second, there is a slight security advantage to this.  It is one fewer
function pointer in the middle of a writable data structure that can
be corrupted.  Third, it shrinks struct buffer_head from 104 bytes to 96
bytes, allowing for appropriximately 7% reduction in the amount of memory
used by buffer_heads (or, alternatively, allows 7% more buffer_heads to
be cached in the same amount of memory).  Fourth, it removes some
atomic operations as the buffer refcount is no longer incremented before
calling the end_io handler.

I've run ext4 through its paces, and everything seems OK.  I've only
compiled ocfs2/gfs2/nilfs/md-bitmap.  Hopefully the maintainers can give
this series a try.  I'm sending the entire series to linux-fsdevel
and cc'ing the fs-specific mailing lists for the fs-specific patches.

v2:
 - Redo bio_endio_bh() to return a bool and the buffer_head instead of
   requiring callers to retried bi_status themselves (Jan)
 - Remove buffer refcount inc/dec (Jan)
 - Add kernel-doc for end_buffer_read_sync()
 - Collect tags from Jan, Ted, Joseph, Viacheslav and Ryusuke

Matthew Wilcox (Oracle) (34):
  buffer: Remove forward declaration of submit_bh_wbc()
  buffer: Add bh_submit()
  buffer: Remove mark_buffer_async_write_endio()
  buffer: Add bh_end_read(), bh_end_write() and bh_end_async_write()
  buffer: Convert write_dirty_buffer to bh_submit()
  buffer: Convert __bread_slow to bh_submit()
  buffer: Convert __sync_dirty_buffer to bh_submit()
  buffer: Convert __bh_read to bh_submit()
  buffer: Convert __bh_read_batch to bh_submit()
  buffer: Convert block_read_full_folio to bh_submit()
  buffer: Convert __block_write_full_folio to __bh_submit()
  ext4; Convert __ext4_read_bh() to bh_submit()
  ext4: Convert ext4_fc_submit_bh() to bh_submit()
  ext4: Convert write_mmp_block_thawed() to bh_submit()
  ext4: Convert ext4_commit_super() to bh_submit()
  jbd2: Convert journal commit to bh_submit()
  jbd2: Convert jbd2_write_superblock() to bh_submit()
  ocfs2: Convert ocfs2_write_block to bh_submit()
  ocfs2: Convert ocfs2_read_block to bh_submit()
  ocfs2: Convert ocfs2_read_blocks to bh_submit()
  ocfs2: Convert ocfs2_write_super_or_backup to bh_submit()
  gfs2: Convert gfs2_metapath_ra to bh_submit()
  gfs2: Convert gfs2_dir_readahead to bh_submit()
  gfs2: Remove use of b_end_io in gfs2_meta_read_endio()
  gfs2: Convert gfs2_aspace_write_folio to bh_submit()
  buffer: Remove mark_buffer_async_write()
  nilfs2: Convert nilfs_btnode_submit_block to bh_submit()
  nilfs2: Convert nilfs_gccache_submit_read_data to bh_submit()
  nilfs2: Convert nilfs_mdt_submit_block to bh_submit()
  md-bitmap: Convert read_file_page and write_file_page to bh_submit()
  buffer: Remove submit_bh()
  buffer: Remove b_end_io
  buffer: Change calling convention for end_buffer_read_sync()
  buffer: Remove end_buffer_write_sync()

 Documentation/filesystems/locking.rst |  14 -
 Documentation/trace/ftrace.rst        |   4 +-
 drivers/md/md-bitmap.c                |  27 +-
 drivers/md/raid5.h                    |   6 +-
 fs/buffer.c                           | 385 ++++++++++++++------------
 fs/ext4/ext4.h                        |  10 +-
 fs/ext4/fast_commit.c                 |   8 +-
 fs/ext4/ialloc.c                      |   6 +-
 fs/ext4/mmp.c                         |   5 +-
 fs/ext4/super.c                       |  18 +-
 fs/gfs2/bmap.c                        |  13 +-
 fs/gfs2/dir.c                         |  12 +-
 fs/gfs2/meta_io.c                     |  13 +-
 fs/jbd2/commit.c                      |  13 +-
 fs/jbd2/journal.c                     |   4 +-
 fs/nilfs2/btnode.c                    |   4 +-
 fs/nilfs2/gcinode.c                   |   4 +-
 fs/nilfs2/mdt.c                       |   4 +-
 fs/ocfs2/buffer_head_io.c             |  16 +-
 include/linux/buffer_head.h           |  16 +-
 mm/vmscan.c                           |   2 +-
 21 files changed, 288 insertions(+), 296 deletions(-)

-- 
2.47.3