Rust specialization problems with lifetimes

Posted Nov 8, 2024 11:27 UTC (Fri) by farnz (subscriber, #17727)
In reply to: Too complex for my small brain by NYKevin
Parent article: Safety in an unsafe world

Specialization is currently blocked (in all forms) because of lifetimes; currently, Rust requires that lifetimes do not influence runtime behaviour, but it's turning out to be really hard to implement specialization in a way that complies with this.

There's two reasons Rust wants to avoid having lifetimes influence runtime behaviour:

Right now, given a function of the form fn foo<'a, 'b, 'c: 'b + 'a>(ref1: &'a T, ref2: &'b T) -> &'c T, the compiler can assume that there will only be one monomorphization of the function for all possible concrete lifetimes substituting for the type parameters 'a, 'b and 'c. If lifetimes affect runtime behaviour, this becomes untrue, and a lot of work has to go into making sure that the compiler generates a minimal set of implementations (since chances are high that every call to foo will have different lifetime parameters).
There's no current need to determine the correct concrete value for a lifetime parameter; you can treat it as a constraints problem, and confirm that the return value of foo(&one, &two) lives for less time than one and two, without caring how long it actually lives for. If lifetimes affect runtime code, then this stops being true, and a lot more work has to be done to make sure that you've chosen the correct lifetime for the return value of foo(&one, &two).

And as a corollary of the above, if you "simply" ignore the problem, you're able to come up with routes to write a safe function with signature fn foo<'a, T>(foo: &'a T) -> &'static T that simply casts away the lifetime parameter, resulting in unsoundness problems. The standard library handles this by promising not to have specializations that let you write this sort of function.

Unfortunately, this makes safe specialization a hard problem; we need the chosen specialization to be the same regardless of lifetimes, which implies that any lifetime parameters must be unconstrained (since if the lifetime parameters are constrained, we can use a specialization to change runtime behaviour based on lifetimes), but lifetimes can be hidden in generics. For example, impl<T> Foo for (T, T) has a hidden constrained lifetime in it; if T is in fact &'a str, then both elements of the pair have lifetimes that are constrained to be the same modulo variance, but we've just said that to avoid runtime behaviour depending on lifetimes, we need the lifetimes to be unconstrained.

As a consequence, I would not expect specialization to stabilize any time soon; it's not clear how to address this such that you get both clear code (e.g. you should be able to have both impl<T> Foo for T and a specialization impl<T> Foo for (T, T), without needing special syntax to say "T can't hide a lifetime constraint") while also avoiding the compiler's reasoning for choosing a specialization from being obscure - you don't want the specialization on pairs to be ignored for ("foo", "bar"), for example. There is a trick with autoref and macros you can use in some cases to get specialization, but it's not usable in all cases, not least because it doesn't act as a trait bound.