get_user_pages(), pinned pages, and DAX

Layout leases in the kernel are different from a layout lease being taken by user space. While it is easy to force user space to take a Lease prior to a GUP (be it through RDMA, DirectIO, etc) it is more difficult to ensure they react to revocation of that lease. Basically there were a couple of simple scenarios I thought of.

1) User takes lease, GUP's pages, removes lease
2) User takes lease, GUP's pages, ignores SIGIO
3) User doesn't take a lease at all.

Number 3 is a nicety to have so I will accept ignoring it. But 1 and 2 are more serious because now the lease means nothing. The application is free to "run away" with this memory. Or can we allow the truncate to hang/fail?

Therefore, I came up with taking the lease in the GUP code (Which also supported 3 above). The use of SIGBUS was the way to ensure truncate would not fail. I'm not sure I agree that failing truncate blocks forward progress as some filesystems already do this.

The SIGBUS solution is all shown in the prototype[1]. However, during the session Jason brought up the fact that with RDMA it would be difficult, if not impossible, to properly ID the process which holds the GUP pin. At the time I was thinking of the FD of the file being mmap'ed rather than the FD representing the RDMA context. After the session, Jason and I spoke and he clued me into what he was talking about. This does put a wrinkle in the ability to send a SIGBUS to the proper application. I've spent some time looking into how difficult it would be to get this right and it would be difficult.

So if we don't (or can't) send a SIGBUS to the application holding the pin. We have a few options. (Potentially implemented as a "library" as Dan mentioned.)

1) Allow each file system to determine what their truncate behavior is. For example, ext4 does return EBUSY on truncate. So it may be easiest there to detect the GUP pin and just return EBUSY.

2) For other file systems which want to pursue the lease path we can...
2a) force a lease to be taken, and don't allow the lease to be removed if pages have been pinned
2b) and allow a user to ignore SIGIO
2c) Hang and/or fail the truncate

After a couple of days thinking about it I don't see a way that the truncate behavior for FS DAX "longterm" pinned pages is not going to be different from a page cache backed FS.

So one possible path forward would be to force the user to take, and maintain, a lease and let truncate hang (or return EBUSY) under these conditions. This ensures that only applications which specifically "opt in" to this behavior are allowed to do this.

Would that be acceptable?

[1] https://lwn.net/ml/linux-fsdevel/20190429045359.8923-1-ir...