No longer experimental
No longer experimental
Posted Oct 10, 2025 19:15 UTC (Fri) by PaulMcKenney (✭ supporter ✭, #9624)In reply to: No longer experimental by tialaramex
Parent article: Progress on defeating lifetime-end pointer zapping
Here is the canonical code we are concerned with. This is in C++, but it assumes the proposed changes in our three pointer-zap-related working papers:
https://godbolt.org/z/35GocdG6s
My guess is that a Rust variant could use expose_provenance() to expose the provenance of the pointer just prior to the .compare_exchange_weak() in the push() function. If the push() function had direct access to the ->next pointer, it could then apply with_exposed_provenance() to that pointer after a successful .compare_exchange_weak(), at least assuming that doing so did not modify the actual in-memory bits of that pointer.
However, the reason why the example does not use direct access to the ->next pointer is that people like to apply debugging to pointer accesses. I do not see a way of applying with_exposed_provenance() to the ->next pointer in this case, but perhaps that is due to a failure of imagination on my part. Maybe some sort of hook or tracepoint for accesses to a given pointer?
But is there a better way?
Posted Oct 13, 2025 0:33 UTC (Mon)
by tialaramex (subscriber, #21167)
[Link] (1 responses)
I am not an expert on Rust's semantics here, and I am definitely nowhere close to expert on the C++ semantics though I have read the "pointer zap" proposal papers and many other documents in that space and I should certainly like to believe I understood them.
As an avowed non-expert, I might be completely stupid here, but the thing about the exposure API is that it's about addresses (integer values) which *aren't* pointers, and in the C++ code these are all pointers. So, it doesn't seem like we even want the exposed provenance APIs anyway?
AIUI (of course tell where I'm wrong) the provenance issue in the C++ code is suppose there's a Jigglypuff as the only thing on the stack, at step 1 we get a pointer to the Jigglypuff, then while part way through step 2 our thread goes to sleep for whatever reason, while we're asleep some other thread popped the stack, getting the Jigglypuff, destroyed the Jigglypuff, then made a Pikachu but by chance with an identical address to the recently destroyed Jigglypuff, during step 3 it seems unavoidable that we succeed - yet we thought we've used a pointer to the Jigglypuff which was destroyed. How and why is this fine, are there negative consequences ?
The "it could then apply" seems like it's a race in your description. Another thread can see the pointer (which we believed pointed to the Jigglypuff but presumably when they dereference it they'll see a Pikachu) before we can apply such a change. If that's fine, why are we even applying such a change ?
Maybe I need to see the lowering and understand why it's a problem to do this with the strict APIs in Rust, or if maybe it isn't a problem.
Posted Oct 13, 2025 13:44 UTC (Mon)
by PaulMcKenney (✭ supporter ✭, #9624)
[Link]
So we have a concurrent LIFO stack, where the "concurrent" means (among other things) that races must be handled gracefully. Whatever is push()ed, anyone doing a pop_all() must be prepared to accept, and in pretty much any order. So if this is a stack of Pokémon, then the caller of pop_all() must be willing to accept a Pikachu (or a Blastoise or a Squirtle or...) as well as a Jigglypuff. But the caller need not worry about popping an ewok, gremlin, or whatever else, again assuming that only Pokémon are ever push()ed. And, given that this is in fact a stack of Pokémon, nothing but Pokémon should ever be push()ed. The question before us is how to make the language implementation deal gracefully with this.
And one concern is that the language implementation might notice at push() time that the top of stack is in fact a Jigglypuff, and somehow keep Jigglypuff-specific state associated with that element all the way through to the time that this same block of memory (now a Pikachu) is popped. As you say, this would be bad.
One hope is that something similar to the exposure API would allow the LIFO stack to mark the ->next pointer as having unknown provenance, keeping in mind that the actual provenance is subject to change until a successful compare_exchange_weak() operation. We need this reprovenancing to be automatic in C and C++ (hence the proposals involving angelic provenance), but Rust has the advantage that almost all authors of any Rust code are not only still alive but also have an excellent chance of remembering whether or not they ever coded a concurrent LIFO stack. So in theory, one possibility in Rust is to change all existing LIFO stack code to (say) pass a pointer stripped of provenance to set_next(). In practice, I have no idea whether this works for the Rust language or its modeling tools (such as Miri, for which this sadly seems unlikely). Another possibility is a special compare_exchange_weak() operation that resets the provenance of the "old" pointer, though this only works for stacks that have the ->next pointer exposed to the LIFO stack, not for ones using set_next() and get_next().
And to your last point, I too hope that this is somehow not a problem in Rust. But hope springs eternal, and mere hope is insufficient to consistently produce correct concurrent code. ;-)
No longer experimental
No longer experimental