Too complex for my small brain
Too complex for my small brain
Posted Nov 11, 2024 12:14 UTC (Mon) by daroc (editor, #160859)In reply to: Too complex for my small brain by khim
Parent article: Safety in an unsafe world
Posted Nov 11, 2024 15:49 UTC (Mon)
by khim (subscriber, #9252)
[Link] (4 responses)
You mean the code in production? That's not “my” code, just code I helped to develop. And I would need to spend a lot of time what dependencies we are tracking, how and why (it wasn't about lock tracking, rules there were different, that's why simple pair of I think it would be better to concentrate on that simplified example that started the whole discussion. I kinda assumed that it would be trivial to follow yet, apparently, it's not trivial… thus I would try to explain the ingredients that are used there and show how can they be combined together to detect violations of locking ordering. And that's… essentially it. We are done. Just need some syntax sugar to make it more pretty. I guess you would want to add something like Would also need some macro because you would be doing things like P.S. The while thing relies on the fact that correctness checking for the
Posted Nov 11, 2024 16:18 UTC (Mon)
by daroc (editor, #160859)
[Link] (3 responses)
It seems like the big difference between the two schemes is really just about the use of a partial order [1] versus a total order. Fuchsia's scheme also involves carrying around a zero-sized context that gets compared to see whether one element is less than another element at compile time — the difference is that the comparison has to be encoded into traits, because Rust doesn't support using custom types as const generics (yet).
I personally think that there are reasons to prefer a partial order, but that probably a scheme that used const generics to pass around a type with a custom PartialOrd instance would be the best of both worlds.
[1]: https://en.wikipedia.org/wiki/Partially_ordered_set#Parti...
Posted Nov 11, 2024 17:24 UTC (Mon)
by khim (subscriber, #9252)
[Link] (2 responses)
That's actually pretty minor issue. As I have said: I developed this scheme to validate more complicated situation, where nether total nor partial order work. I just used simple number for illustration purposes. And yes, we had function that accepted two enum numbers and then answered whether they are compatible. Pretty obvious modification. The main difference is when and how it's checked, not how it's encoded. With But how would it know? Think about function that stores something… if it would send it to USB storage, if it would save it on SSD then there would be different locks, if that file on the NFS server then you would hit yet another situation. And you need to specify one, single markup that would cover everything. Sure, instead of simple callback you may accept another trait which embeds information about what locks that callback would need… but what if you have two callbacks? What if they could be called in a different order? Instead of one, simple, rule you start develop elaborated schemes that may describe what you are really trying to achieve… but you are doing in a pretty limited world of traits and generics… which is total: if generic function accepts some type of callback – then it always accepts that type of callback! And if the calling function could't prove at the declaration time that it have the right to call that callback… oh, well… now we need another trait… and if we use two callbacks… we need to decide (and declare) order that we plan to use in the function declaration, too! And if order can be different… well that's another indirection! Traits are multiplying and fill your code with something that it doesn't really need, that's only there to help with lock order checking! As I have said: this leads to insane mess… and insane compilation times. Now, constants are validated after instantiation. That means that you simply just don't even need to care about combinations that are not happening in real code. And if you use function with a callback… callback would determine validity of your function automatically. You don't need to create an additional trait, put callback and lock in there and pass that around. Any generic function or generic callback already carries the required data, you don't need to duplicate it in separate, manually written, traits! Sadly that's not possible in Rust because traits couldn't carry You may create helper function, though. If your locks are defined in one place then it's trivial and if they are not defined in one place then P.S. One mode that's trivial with
Posted Nov 11, 2024 18:18 UTC (Mon)
by daroc (editor, #160859)
[Link] (1 responses)
As for why the Fuchsia team went with a trait-based approach — your comment about when panics in const blocks stabilized raises another theory: netstack3 was started in 2019. I'm not sure exactly when the LockBefore solution was added, but it could have been before the approach you described became possible.
Posted Nov 11, 2024 19:21 UTC (Mon)
by khim (subscriber, #9252)
[Link]
I think that was possible even in 2019, but it sure looked awfully ugly back then. Panic in I don't remember where I discovered the use of Now suspect that without But for me, since I knew about that approach essentially from my first attempt to help some of my friends to write their first large Rust program… it just feels much more natural than traits-based checks. Closer to how one would do that in C++17 and thus more… “traditional”, I would say. I guess is never occurred to me that it may be very obscure technique that is not known to everyone.
Too complex for my small brain
before
and after
traits were not enough).&'a mut T
and convert into PhantomData<&'a mut T>
then this zero-sized type acts precisely like a real reference… except you couldn't use it to derefence anything (but we don't have anything interesting in the context itself, the only interesting things are in the context type, thus we don't need to dereference anything) – in particular as long as PhantomData<&'a mut T>
exists the original context is not accesible (by using 1.2, obviously)enum
and then simply use LockPriority::IPV6LOCK us usize
(by using 1.3, obvious)const { assert!(…) }
or const { if … { panic!("…") }
and that's it. That's a bit strange because it looks as if that's a runtime check, but no, that's compiler-time check. So… you have two constants, compare and you would know if lock is Ok to take or not (ask me if you need to backport it to older versions of Rust, there you needed somewhat more complicated dance, but it's also doable).
Note that you have access to all types in the chain here, even if indirect, and could implement arbitrarily complex verification scheme… it's only hindered by the fact that const
functions couldn't be part of the trait, so you would probably want to put your machinery in separate module to not fill the namespace with helper functions.LockPriority
trait:
pub trait LockPriority : sealed::LockPriority {
const PRIORITY: usize;
}
impl<'a, T: ?Sized, const P: usize> LockPriority
for TakenLockPriority<'a, T, P> {
const PRIORITY: usize = P;
}
mod sealed {
pub trait LockPriority {}
impl<'a, T: ?Sized, const P: usize> LockPriority
for super::TakenLockPriority<'a, T, P> {}
}
let (mut ctx, guard) = mutex.lock(UsePriority(& mut ctx));
pretty often. Or maybe not, maybe it's better to keep that explicit. Attempts to use wrong context would be detected and stopped by the compiler, anyway.const
expression and detection of errors happens not when you define some generic function, but when it's instantiated. And it's obvious why it have to happen like this: correctness checking of something likes const { assert!(…); }
couldn't happen during definition time because what goes into assert!(…);
may define on constants that ass passed as arguments. And doing that in runtime wouldn't make any sense: this would mean that your constant value would, suddenly, depend on something in runtime…Too complex for my small brain
> It seems like the big difference between the two schemes is really just about the use of a partial order versus a total order.
Too complex for my small brain
const
functions can do arbitrary calculations, if you are worried that locks would be accepted in some random arbitrary order while you want the compile-time error (and explicit decision by developer who would declare whether storage or network have higher priority, instead) then this can be done via simple proc-macro which would generate an appropriate table. netstack3
is defining their locks in once place with pile of macros, anyway.netstack3
function declares what it accepts in it's signature… and that's it.const
functions and PartialOrd
couldn't be used in const
context, too.RED
/BLACK
scheme can be used (it's more-or-less inveitable with netstack3
approach, too, as was already discussed).const
-checking and which is, essentially, impossible to implement (without additional const
-coding, heh) with netstack3
approach: “just make me happy”. I have 2 (3, 4, 5, N) locks… how can I lock them all without causing the deadlock. How would you implement that? That's powerful capability, but it's not compatible with “just leave this machinery out” compile-speedup enhancement (where you just map everything to ()
and compiler accepts everything till you would compiler with checking-enabled
flag)…Too complex for my small brain
> I'm not sure exactly when the LockBefore solution was added, but it could have been before the approach you described became possible.
Too complex for my small brain
const
were always possible (just divide by zero and you get panic at compile time) and, indeed, in Rust 1.51 everything already works. But without const
generics all these checks would have to be based on One
and Zero
types and associated consts in traits and that looks pretty awful. When Rust 1.57 stabilized panics it became easier and with Rust 1.79 the need for strange dance with extra struct and const
implementation for it is no longer needed.const
for invariants checking (I think it was on URLO, but could be reddit, too), but I remember that current version of Rust was 1.5x, which meant that I don't need to do crazy dance with types to encode number, can just simply use const
generics, but still needed to use “divide by zero” trick. Discussion about the fact that panic in const
would be stabilized “soon”, that I vaguely recall, means that was middle of 2021.const
generics with the need to encode numbers in One
and Zero
types and then use pile associated constants for calculations it was just too ugly to really contemplate and one needed a leap of faith to believe that eventually it all would become much simpler and more usable to bet an implementation on that.