COW and volume management?

I've always believed in "just write the best code you can - the rest will sort itself out later". Not being upstream hasn't hurt the project at all - on the contrary, if anything adoption has accelerated over the past six months. We're no longer tied to a project with engineering practices that haven't changed much over the past 20 years, and right now I'm wondering how the kernel is going to manage the Rust transition over the next 10-20 years.

At some point, the broader engineering community is going to realize that all the code we depend on - essentially, all of it - is going to have to be rewritten in Rust. The level of complexity we can manage in C has been hitting limits; we cannot keep doing the same old forever. Systems engineering is going to get so much easier if we can manage the Rust transition, and there are so many techniques in debugability, introspection, static analysis and formal verification that that opens up. There is a mountain of work that needs to happen to get there - but it needs to happen, or we'll be stuck with a mountain of legacy code in a language new people aren't learning with the people that actually understand it dying off.

To understand how this impacts myself personally and the bcachefs project: my rule of thumb is that when a bug in upstream, non-bcachefs kernel code comes across my desk (and they do), it's going to take at least 10 times as long to debug as any bcachefs bug. And I'm going to be stuck with it personally; there are people who have been tearing through the bcachefs code and can debug anything (literally; I'm half-following the IRC channel right now) and will tell me exactly what's broken and what needs to be fixed - and I'd be more than comfortable just giving commit access to the repo but they seem to like using me as the coding bot - but when it's an mm-involved bug or anything else I'll hear back a week later - "uhh, maybe you better take this one on".

There's ways to write complex code and make it testable and debugable, but it has to be an object of serious study, you have to prioritize it. Fortunately, Rust is going to make this a hell of a lot easier, with things like the Display trait, and a type system that's comprehensive enough that you don't have to stare at code for a couple hours in order to safely change one little thing - but "how to write complex systems that we can maintain and debug in the wild" is something that needs more thought and study in the kernel community.

So I'll be taking a wait-and-see approach. I have seen some positive developments over the past six months - people who weren't finally seem to be talking somewhat more seriously about automated testing, and that's been a huge gap that was really painful for bcachefs (my test dashboard would blow up every rc1 rebase with bugs in other subsystems, and I can't do work or merge code without good test data, so all work would stop - often for weeks - while dealing with that). But I think things have a ways to go before I could seriously consider it - and that's if they even wanted bcachefs in the kernel, and I think everyone knows the rough outline of that part of the story :)