Locking [LWN.net]

Locking

Posted Sep 23, 2024 19:51 UTC (Mon) by NYKevin (subscriber, #129325)
In reply to: Locking by atnot
Parent article: Resources for learning Rust for kernel development

> There's just a whole lot of things that could be a lot nicer with some sort of "here's a thing, please give it back to me when you're done with it" mechanism.

You can steal the API from std::thread::scope(), but I'm not sure how applicable that is in all cases. The basic idea is as follows:

* The caller passes in a closure, which takes some opaque scope object by reference.
* The scope object provides methods for the closure to do whatever thing(s) the caller wants to do (in the case of thread::scope(), spawn threads).
* The scope object internally keeps track of whatever thing(s) the closure does, and whatever cleanup operation(s) may be required in response (in the case of thread::scope(), join those threads to ensure they do not outlive the scope).
* Because the closure is called from the callee, the callee always gets control back after the closure returns (or panics, if unwinding is enabled and we use catch_unwind). The callee can ensure that all cleanup code is executed, by referencing the scope object's internal data. Since the closure does not receive ownership of the scope object, it cannot forget it.
* In the case of thread::scope(), the scope object also gives the closure "handles" which it can use to reference the operations (threads) it has performed (spawned). But if the closure forgets these handles, it has no effect on the scope object, because they're just handles - they do not need to be dropped for the scope object to do the appropriate cleanup.
* Lifetime parameters are used to prohibit the closure from smuggling these objects and references out to the caller and trying to carry out further mischief with them.

This is obviously not as flexible as true linear types would be, but it is better than nothing.

to post comments

Locking

Posted Oct 2, 2024 10:45 UTC (Wed) by taladar (subscriber, #68407) [Link] (2 responses)

Couldn't you just force (assuming code doesn't panic or otherwise never return) the "give it back" semantics by making the type of the closure return the opaque type you pass in that the closure has no way to construct and so it is forced to return the one it got in its own parameter?

Locking

Posted Oct 2, 2024 12:56 UTC (Wed) by excors (subscriber, #95769) [Link] (1 responses)

I think that if you have a closure, there's no need for anything that tricky - you can just pass an exclusive reference to some object into the closure. The closure can't drop the object or leak the reference outside of its scope. The closure can't fail to return except by infinitely looping or blocking (which is relatively easy to debug by looking at the offending thread's call stack). When the closure returns, you know you've got the object back, and you can do any relevant cleanup.

For example, mutexes could be designed like:

let mut m = ScopedMutex::new(42);
m.with_lock(|i: &mut i32| { println!("{}", i); });

where with_lock guarantees it will always lock and unlock correctly around the accesses (unless the closure never returns).

The issue is, it would be nice to get similar guarantees *without* the closure, because closures create a potentially-inconvenient nested code structure and can introduce some extra lifetime challenges. Usually that's fine, but sometimes it's rather annoying. Better to have an API like:

let mut m = Mutex::new(42);
let i: MutexGuard<_> = m.lock().unwrap();
println!("{}", i);
std::mem::forget(i); // this should be an error - it should be required to properly drop i (by letting it go out of scope, or calling i.unlock() which transfers ownership of i, etc)

Currently the forget() is allowed and the mutex will never get unlocked, and some unrelated code may deadlock later (which is much harder to debug). It also means the Mutex might be dropped while it is still locked, so Mutex has to know how to clean up from that unusual state. Prohibiting the forget() (and any other code with similarly forgetful behaviour) will require new Rust language features.

(I think that's an easy trap to fall into when designing APIs: you design a MutexGuard whose lifetime is less than Mutex's lifetime, so you know the MutexGuard cannot be dropped after the Mutex is dropped (which is true, and a very helpful guarantee), and intuitively that means the MutexGuard will be dropped before the Mutex is dropped. And that's almost always true, but it's not guaranteed - the MutexGuard might have been forgotten instead of dropped - so you have to either handle that correctly (which can be difficult), or switch to a closure-based API.)

Abuses of mutable references

Posted Oct 2, 2024 13:44 UTC (Wed) by farnz (subscriber, #17727) [Link]

Note when doing that sort of analysis that you also have to take into account std::mem::swap and friends. An exclusive reference to a type I cannot construct works fine for what you describe, but once you allow a user-chosen type into the closure parameters, you have to consider what happens if I swap it out for another one I have lying around. I think that for the case of a single layer of locking, it's fine, but it becomes more exciting once you consider the case of nested locks, and my brain isn't up to looking for obscure corner cases in that situation.