|
|
Subscribe / Log in / New account

Note similarity between Rust references and reader-writer locks

Note similarity between Rust references and reader-writer locks

Posted Mar 13, 2025 18:03 UTC (Thu) by NYKevin (subscriber, #129325)
In reply to: Note similarity between Rust references and reader-writer locks by farnz
Parent article: Capability analysis for the kernel

That is true, but unfortunately there is no easy way to write this generically. Or rather, there are too many ways of writing it generically:

* You could just use &Foo and &mut Foo (probably with lifetimes, at least if they're going into a struct). The main advantage is that these are the "lowest common denominator" - nearly everything else can be converted into one of these, except that pinning pointers cannot be converted into &mut. But none of the other generic options are compatible with Pin either (in this use case), so if you need to mutate through a Pin, you pretty much have to write it out separately (and hope the type supplies appropriate Pin<&mut Self> methods for actually doing the mutation). Unfortunately, the use of &Foo or &mut Foo means users will sometimes have to write &*foo or the like to convert some other type into a reference, which is not terrible but also not ideal.
* You could instead take Deref<Foo> and DerefMut<Foo>. This covers most reasonable smart pointers, in addition to &Foo and &mut Foo, which means the user can give us ownership (or shared ownership, a lock guard, etc.) if that is more convenient for them. Technically, there is nothing prohibiting a Deref implementation from doing something expensive like taking a lock, but the semantic meaning of Deref and DerefMut is "this conversion is so cheap and so obvious that it should happen implicitly," so taking a lock in Deref is evil (on top of that, the signature of Deref::deref() is not compatible with taking a lock unless you hold it for the entire lifespan of Self, so it makes more sense to implement Deref on the already-taken lock guard instead). For example, a Box<Foo> has all the methods of a &mut Foo, because even though it's technically not the same thing, we prefer to think of it as "a Foo that happens to live on the heap" rather than "a Box containing a Foo." Similarly, Vec<T> and [T; N] both deref coerce into &[T], because they're both conceptually "a contiguous sequence of T objects somewhere in memory," and so it would make no sense to require an explicit conversion to use slice methods on them. However, this can also be a problem, if you don't actually care about that property and just want "a thing that can cheaply and infallibly give me a &Foo, regardless of what it is conceptually supposed to be."
* Then there are AsRef<Foo> and AsMut<Foo>. These are the "cheap reference-to-reference conversion" traits. Just like Deref, they should not take locks or do other expensive operations (and just like Deref::deref(), their method signatures are not really compatible with locking anyway). Unlike Deref, they don't get invoked implicitly, which means that unlike Deref, you can implement them more than once on the same type, as well as on types that are conceptually distinct from one another. For example, the various string-like types can all be converted into &[u8] via AsRef, but they do not Deref as &[u8], because it would be very confusing if we mixed bytewise methods into the same namespace as Unicode-oriented methods. Unfortunately, &T does not blanket implement AsRef<T> due to a trait coherence constraint, which means these are not a good candidate for accepting arguments generically.
* There are also Borrow<Foo> and BorrowMut<Foo>. These are extremely similar to AsRef<Foo> and AsRefMut<Foo>, with two differences: Firstly, they semantically require that Self implements Eq, Ord, and Hash identically to the underlying (owned) type (i.e. if lhs == rhs, then lhs.borrow() == rhs.borrow()). This makes them suitable for use in hash tables, trees, heaps, etc. Secondly, there is a blanket implementation for &T, so they are more suitable for generic use than AsRef<T>. But you don't always need the Eq/Ord/Hash property, so this may be too restrictive in some use cases.
* In principle, there's also Into<&T>, but this doesn't exist in practice because the signature doesn't make sense (it consumes self and returns &T, but what is the lifetime of the &T it returns, given that self has been consumed and no longer exists?).

IMHO the least problematic option (at least in a case like this) is to take Deref<T> or DerefMut<T>, because:

* Conversion into &T or &mut T is rarely all that difficult (except for the Pin case noted above) and those always implement Deref/DerefMut.
* Most smart pointers are Deref<T>, so we still support the "pass an owning smart pointer or lock guard instead of a reference" use case. That's one less lifetime for the user to think about, but they don't have to do this if they don't want to (they can pass &T instead).
* If the end user wants to do some conversion that isn't "obvious" in the way that deref coercion is meant to be obvious, then probably they should do that explicitly at the callsite.

I'm actually not too upset about AsRef lacking that reflexive blanket implementation - if it had one, then it would be much easier to write generic code that does implicit conversions, and once that starts to become the standard way of writing generic code, everything starts feeling a little too much like C++ (before they invented the explicit keyword). In fact, part of me is tempted to say that they should not fix this, even if trait coherence gets more permissive in the future, because I'm not sure the resulting generality would be worth it. But OTOH that doesn't seem to have happened with From and Into, so maybe it's fine.


to post comments

Note similarity between Rust references and reader-writer locks

Posted Mar 13, 2025 19:12 UTC (Thu) by farnz (subscriber, #17727) [Link]

To be clear, this was more meant in terms of reading the code, not writing it - if you see an &mut, you know that you already have exclusive access to the thing you're looking at, and thus that you don't need to look for locking.

Thus, if your typestate had a way to get an Option<&mut P> from a ParentHandle, I'd be reduced to confirming that the only way for that option to be None is if the weak reference to parent was already dangling, and thus there is no parent to lock.

Note similarity between Rust references and reader-writer locks

Posted Mar 14, 2025 12:36 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (3 responses)

> * Then there are AsRef<Foo> and AsMut<Foo>. These are the "cheap reference-to-reference conversion" traits. Just like Deref, they should not take locks or do other expensive operations (and just like Deref::deref(), their method signatures are not really compatible with locking anyway). Unlike Deref, they don't get invoked implicitly, which means that unlike Deref, you can implement them more than once on the same type, as well as on types that are conceptually distinct from one another. For example, the various string-like types can all be converted into &[u8] via AsRef, but they do not Deref as &[u8], because it would be very confusing if we mixed bytewise methods into the same namespace as Unicode-oriented methods. Unfortunately, &T does not blanket implement AsRef<T> due to a trait coherence constraint, which means these are not a good candidate for accepting arguments generically.

Hmm. That's odd. I see `AsRef<Path>` generic parameters fairly often. I feel like `AsRef<OsStr>` is also around, but I've seen that less often. Is &Path and &OsStr better here perhaps (though I feel like String and str usages might need caller-side help then)?

Note similarity between Rust references and reader-writer locks

Posted Mar 14, 2025 13:51 UTC (Fri) by farnz (subscriber, #17727) [Link] (2 responses)

Path, OsStr and str "cheat" by manually implementing AsRef for the reflexive case. The problem is that trait coherence prevents you having impl<T: ?Sized> AsRef<T> for T, but instead requires the crate that defines T to add the trivial implementation manually.

If that crate fails to do so, then the orphan rules mean that you can't add it yourself. This means that, unless the type has "cheated" with a trivial manual implementation, you can't use an &T where a function asks for an AsRef<T>. Fortunately, there is impl<T> AsRef<[T]> for [T], so this restriction only applies to converting &T to AsRef<T>, not &[T] to AsRef<[T]>.

Note similarity between Rust references and reader-writer locks

Posted Mar 14, 2025 14:09 UTC (Fri) by intelfx (subscriber, #130118) [Link] (1 responses)

Is there any desire to relax the trait coherence rules enough for such a blanket impl to become possible?

Note similarity between Rust references and reader-writer locks

Posted Mar 14, 2025 14:30 UTC (Fri) by farnz (subscriber, #17727) [Link]

That's a question better asked on internals.rust-lang.org. My understanding, as an outsider, is that permitting this blanket implementation in parallel to impl<T: AsRef<U> + ?Sized, U: ?Size> AsRef<U> for &T runs into the same set of soundness problems as other forms of specialization.

If the issues around specialization can be resolved, then that would provide a path to permitting such a blanket imp for AsRef. But that's a big if, since there's a requirement to avoid permitting specialization based on lifetime annotations (mostly because lifetime annotations are supposed to be erased after type checking is complete, but also because there's no way in Rust to represent the statement "lifetimes 'a and 'b are exactly the same"; the closest you can get is "'a is at least as long as 'b and 'b is at least as long as 'a", which is not quite the same in terms of the logic used for type and borrow checking).


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds