> There's not much overlap, but I wonder how you can implement a transaction containing one or more open() if these operations can block, call userspace and eventually abort.
Transactions have been invented in databases, and in that context it's obvious that part of a transaction may fail; and even in btrfs transactions allow for failures. So, what's the problem here?
A bigger problem is instead that during the transaction the filesystem is locked, so userspace needs to avoid modifying the fs during the check, if btrfs is used. It's possible I guess, the atime change problem needs to be solved to perform reads, but that's doable. But if developers don't test this scenario, they won't notice.