Thoughts and clarifications
Thoughts and clarifications
Posted Sep 4, 2024 17:13 UTC (Wed) by asahilina (subscriber, #166071)In reply to: Thoughts and clarifications by wtarreau
Parent article: Whither the Apple AGX graphics driver?
Now we're back to the same strawman arguments... the Rust team aren't asking the C maintainers to "change everything" or practically anything. If anything I've been finding bugs in drm_sched and its users in C code as part of my work. Me not doing this work would have led to the C folks having to debug those things. And Wedson wasn't asking FS developers to change anything in C, he was asking FS developers to help understand the existing C API so it could be mapped to Rust properly.
Of all the Rust abstractions I've written drm_sched was the exception, the one case where the existing design was so bad I really needed to improve it. Those changes were completely non-intrusive, the patch wasn't even 50 lines of code and did not change the API for existing drivers in any way.
You're making it sound like us Rust folks are pushing through rushed, questionable changes to C code just because we "need" it and that just isn't true, at all.
> Changing something that works is extremely difficult. Changing it without breaking it is even more difficult. Changing it in a way that guarantees that it will still be possible to change it later is the most difficult. This requires experience and honest cooperation, not bold accusations nor criticisms of everything.
None of the changes I asked for negatively impacted existing correct code, that was trivial to see since they only affected conditions which supposedly were previously not allowed to happen. They definitely didn't break anything. They definitely didn't introduce some horrible maintenance burden or touch things deep in the architecture. It was just some cleanup code and cloning some debug strings so they weren't dangling pointers. That's it.
Every time people excuse the C maintainers' reaction to Rust it's always strawmen. Things that aren't true. Things they wish were true so they could discount and attack Rust, but which aren't, so they just pretend they are.
And I and the other Rust people are getting really, really, really tired of this nonsense.
> To be honest, when I read your long message, I was a bit scared by the extreme confidence you have in the quality of your own work and your ability to spot many other "older" areas that are wrong.
Well, I'm pretty confident that drm_sched is buggy because I got oops reports from users and had to debug it, multiple times. And so far I haven't had any oops reports in my own driver code from users. So there's that...
Posted Sep 4, 2024 17:37 UTC (Wed)
by Wol (subscriber, #4433)
[Link] (8 responses)
Welcome to the club.
I'm known for being very anti-Relational. Well, why wouldn't I be, when I seem to spend most of my time waiting for Snoracle and the like to find data.
Relational is O(log(n)) which means searching 100K rows is O(5). Yes it's a guaranteed O(5), but I can provide a 5 nines guarantee my database will be faster (it's O(1) nearly all the time - IRREGARDLESS of the size of the data set).
It's a tradeoff. Do you want an "almost always 80% faster response", or a "guaranteed to get a result but all of them will be slow".
People can't understand why you want "it's impossible to specify a faulty memory operation", people can't understand why I want a database that returns data almost faster than I can hit "return" ...
Cheers,
Posted Sep 4, 2024 17:54 UTC (Wed)
by corbet (editor, #1)
[Link] (1 responses)
Posted Sep 4, 2024 19:59 UTC (Wed)
by Wol (subscriber, #4433)
[Link]
I do feel for Asahi, though, because I do my best to explain things and feel that people just don't want to understand.
I'll do my best to back off. Part of the trouble is that (especially as I'm in the wrong time zone for most people) that seriously gets in the way of a good discussion. Maybe that's a good thing. Stops me getting really out of hand :-)
Cheers,
Posted Sep 4, 2024 17:57 UTC (Wed)
by wtarreau (subscriber, #51152)
[Link] (1 responses)
I love this example because it's often a difficult choice that changes over time. For example those who pay for the CPU time will most of the time prefer "almost always 80% faster responses". And those who pay strong penalties on SLA will often prefer "guaranteed time but slower". And it happens that these choices are taken in a product at a moment, the product's area of popularity evolves over time, it's good, it moves to other areas and then the initial better choice is suddenly considered as an absurd one. Just because where it acts, the default preference is different.
Posted Sep 22, 2024 17:18 UTC (Sun)
by Rudd-O (guest, #61155)
[Link]
I am not sure entirely, however, how this connects to the conversation of Rust in the curve. Rust is focused on providing what they call cost-free abstractions. That is, all the typesafety and niceties that you see that some people like, some people hate, I myself hated them before... They don't cost anything. You don't pay for them in compute time. You don't pay for them in electricity. At least not doing execution. They're all handled at compile time.
The cost you pay for this is not measured in compute or electricity. The cost you pay for this is that you develop somewhat (a lot!) slower at the beginning. The compiler is constantly fighting you. Because you're doing things that you find natural and normal with your experience level at whenever language you were using before, you started using Rust, where you naturally have less experience at the beginning. Things that Rust defines as impossible using a compiler in a borrowed checker in the compiler.
It's an important cost; you wouldn't want to prototype something rapidly in Rust, especially if you don't have a lot of experience within it, because it's just gonna take you much longer to finish the prototype. But it is a cost that certainly worth paying when what you're doing is going to be used by potentially millions or hundreds of millions of people and some of those people are going to be very interested in seeing how they can exploit that so that they can harm others.
There are definitely trade-offs to be made, but I don't think the Linux kernel as a whole should be making the trade-off of "just keep doing what we were doing, mang, it's gonna be fine", because it really is not.
To be perfectly honest, one can write subpar, suboptimal, slow, and even crashy Rust code. Cough .clone().unwrap() cough. But that is an argument that applies equally to every language. In fact, more to other languages where it's actually easier to write all of those things. Because the language will help you do it.
Posted Sep 22, 2024 17:08 UTC (Sun)
by Rudd-O (guest, #61155)
[Link] (3 responses)
Not everyone can't understand that. But some can...
Rust (its "safe" part, at least) as a language is not a magical, can't crash your car, safety system. But it does provide certain niceties like, for example, seat belts. Or, you can't shift into park or reverse while you're rolling forward in drive. Or, it won't start until you've depressed the brake pedal. Or, backup cameras. Or, LATCH baby seat anchors. Or, airbags.
C's the ratrod.
Analogies aside, I am actually quite surprised, after only two years of programming in Rust, that it is almost impossible to deliberately cause a invalid memory access bug in Rust, unless you are using "unsafe" or you are using bindings to a C library which itself has a bug. Before that, I didn't even know that such a thing was possible. I thought the only way to get a memory-safe system is to have a garbage-collected system that just tracks pointers for you and you never have to think about that. And of course, never call into a C library.
I strongly suspect that the entire world of systems development, especially embedded in low-level programming, is going to experience a shift that in 20 years, or maybe less, will have most people wondering, why would anyone use C? With roughly the same amount of puzzlement as you would find yourself in, if you heard someone asking for a car with a steering wheel. Obviously every car has a steering wheel. That's just not something you ask.
Posted Sep 23, 2024 9:03 UTC (Mon)
by farnz (subscriber, #17727)
[Link] (2 responses)
Arguably, the biggest advance Rust brings is the borrow checker; it's hugely intrusive, but it allows Rust to formally verify that you're paying attention to whether or not a given pointer (in Rust's case, a class of pointers called "references") points to a valid place at time of use, without runtime assistance (e.g. a garbage collector).
Combine that with "either shared, or mutable, but not both", and you get a powerful tool for making it easier to reason about your code's behaviour.
Posted Sep 23, 2024 11:24 UTC (Mon)
by intelfx (subscriber, #130118)
[Link]
Posted Sep 25, 2024 9:54 UTC (Wed)
by Rudd-O (guest, #61155)
[Link]
Posted Sep 4, 2024 17:53 UTC (Wed)
by wtarreau (subscriber, #51152)
[Link] (21 responses)
I'm starting to sense how it can be difficult to have a technical-only discussion with you...
> the Rust team aren't asking the C maintainers to "change everything" or practically anything. If anything I've been finding bugs in drm_sched and its users in C code as part of my work. Me not doing this work would have led to the C folks having to debug those things.
I simply don't know because it's not my code. What I'm trying to explain is that it's very common when coming with a 2-line patch to reveal a deeper problem that needs more work, and *that* work can have serious long-term consequences, and as a result requires more thinking. Then the problem is that the patch author can feel like "they're rejecting my trivial patch, it's just because they hate me" while that trivial patch is in fact a brown paper bag over a deeper problem. This has happened to be many times and I have caused myself frustration many times to contributors who revealed such problems in my code. And believe me, a 2-line patch can end up with 6 months of work to redo lots of things differently. And in my project I don't have to deal with ABI compatibility issues nor stuff like this. That's why I'm saying that I understand to some extents when this can happen. I'm not saying this is the case but the way you seem to instantly map a patch rejection to "C vs Rust", I really don't like it because such form of victimization has already hurt the Rust community a lot in my opinion. I mean, this is a perfect example that will make me more careful later about discussions around this langauge, by fear of entering a maelstorm of false accusations.
> And Wedson wasn't asking FS developers to change anything in C, he was asking FS developers to help understand the existing C API so it could be mapped to Rust properly.
Maybe, I honestly don't know. But similar things happened to me in the past where some people asked me levels of details about my code that I simply didn't have, because the code is the doc when it comes to APIs, so I couldn't do more than reading it again and it takes a lot of time, plus I feel like I'm just wearing my eyes so that the requester can spend his time playing candy crush, which is not really cool.
> Of all the Rust abstractions I've written drm_sched was the exception, the one case where the existing design was so bad I really needed to improve it. Those changes were completely non-intrusive, the patch wasn't even 50 lines of code and did not change the API for existing drivers in any way.
Possibly, I don't know. But it's not the size of the change that matters when you're getting close to the core, it's the impacts and what the change reveals. You said yourself that you've found deep problems with the current API and that it's totally unsafe even for C. Did you ever think that the maintainer himself probably doesn't trust this code at all anymore and is not willing to permit more code to get more intimate with it ?
> You're making it sound like us Rust folks are pushing through rushed, questionable changes to C code just because we "need" it and that just isn't true, at all.
No, I'm not saying that it's what happens, I'm saying that you need to understand that it may be perceived like this sometimes by the person whom you're asking to accept the change, regardless of any language. It would be the same from C to C as well. The language has nothing to do there, yet you're bringing it on the table all the time as the supposed reason for your work not being accepted.
> None of the changes I asked for negatively impacted existing correct code, that was trivial to see since they only affected conditions which supposedly were previously not allowed to happen. They definitely didn't break anything. They definitely didn't introduce some horrible maintenance burden or touch things deep in the architecture. It was just some cleanup code and cloning some debug strings so they weren't dangling pointers. That's it.
Possibly. At least that's your analysis and I totally trust you that it's the intent. Sometimes for a maintainer, opening the possibility that some unsafe code is easier to use means more problems in the future, especially when trying to replace it. I don't know the details, but sometimes that can be an explanation as well.
> Every time people excuse the C maintainers' reaction to Rust it's always strawmen. Things that aren't true. Things they wish were true so they could discount and attack Rust, but which aren't, so they just pretend they are.
Languages again again again... "it's not us who started first it's them!" Please! Maybe if you tried to reach out to people to fix generic API bugs without presenting yourself as representing a language team and putting them by default in the supposed other language one they would be more confident in your motivations ? What do you have against C that you want to put everything non-rust in it and present it as a perpetual ennemy. It looks like pure politics where there's no place for facts nor technical excellence. We could go on with all the other languages in the kernel at this game.
> Well, I'm pretty confident that drm_sched is buggy because I got oops reports from users and had to debug it, multiple times.
Oh I've never questioned that, I totally trust you on that one. What I'm saying is that some maintainers might prefer to keep (for some time) bugs that are understood by them and impossible to trigger under some conditions rather than risk more complicated ones. Many of us had to run through such choices over time. They're unpleasant, they make you like the dirty coder in that corner overthere, and you know that at some point you need to address them. Sometimes you just can't have enough time on the table to deal with them, and you figure it's even harder to bring someone up to speed on them, so you're waiting for finding the best person for the task and it can take years. But such rare persons definitely don't start by placing people in boxes with a language name written on the side :-/
Posted Sep 4, 2024 18:00 UTC (Wed)
by daroc (editor, #160859)
[Link]
[And several other parts on similar lines.]
Please do avoid personal attacks in the comments. I think you may be talking past each other a bit; it's easy for all the attention to end up focused on the times when things go wrong, instead of the times when things go right, and that can make it hard to be charitable. Consider leaving this thread of comments here.
Posted Sep 4, 2024 19:43 UTC (Wed)
by rywang014 (guest, #167182)
[Link] (12 responses)
Posted Sep 4, 2024 20:11 UTC (Wed)
by wtarreau (subscriber, #51152)
[Link]
Posted Sep 4, 2024 20:26 UTC (Wed)
by pbonzini (subscriber, #60935)
[Link] (10 responses)
[1] the patches of the guy who NAKs
Posted Sep 5, 2024 0:59 UTC (Thu)
by Ashton (guest, #158330)
[Link] (6 responses)
Posted Sep 5, 2024 2:29 UTC (Thu)
by viro (subscriber, #7872)
[Link] (5 responses)
Posted Sep 5, 2024 2:44 UTC (Thu)
by viro (subscriber, #7872)
[Link]
Posted Sep 5, 2024 7:23 UTC (Thu)
by pbonzini (subscriber, #60935)
[Link]
I've gotten my share of stern NAKs from you, but you've always been extremely constructive and explained what I was doing wrong. But if there's no explanation whatsoever, it is not a win for anyone.
Posted Sep 5, 2024 12:08 UTC (Thu)
by Ashton (guest, #158330)
[Link]
Code rejections should come with clear actionable feedback, and it shouldn’t be the requesters job to extract that feedback from the reviewer. Even “I see this and need more time to review” is far more useful than an unexplained “no”.
This kinda behavior is totally fine if it’s an occasional thing, nobody is perfect. But if it’s common then it’s a sign something is up and someone with authority needs to step in and figure out why it’s happening.
Posted Sep 5, 2024 12:41 UTC (Thu)
by Wol (subscriber, #4433)
[Link]
I'm bitching slightly, I know, but this is the perfect example of people not stepping back and thinking. The GP said "something is going wrong". If your boss comes to you and asks - politely - "what's happening", and you respond "look at the size of my intray!", wouldn't you appreciate your boss going through it and saying "this is important, that's not, what do you think of the other?"
If that's what the GP meant - and I'm sure it is - he's exactly the sort of boss I would like! Yes, you're going to have to piss some people off, but at least you know your boss has got your back.
Cheers,
Posted Sep 5, 2024 16:50 UTC (Thu)
by rsidd (subscriber, #2582)
[Link]
Sorry if this comment violates lwn standards. But I remember Linus stepping back for a bit to examine his behaviour and learning to be better. I don't think he or was by any means the worst offender and if by publicly reassessing his interactions he was hoping to set an example that others would follow, well, nice try.
Posted Sep 5, 2024 3:11 UTC (Thu)
by viro (subscriber, #7872)
[Link]
Posted Sep 12, 2024 3:39 UTC (Thu)
by milesrout (subscriber, #126894)
[Link] (1 responses)
Posted Sep 13, 2024 14:32 UTC (Fri)
by MrWim (subscriber, #47432)
[Link]
https://lore.kernel.org/lkml/CAPM=9txcC9+ZePA5onJxtQr+nBe...
Quote Dave Airlie:
> The next NAK I see on the list will mean I block all patches from the
So it's not about documenting the reasons for a given NAK, it's about adding documentation to the drm_sched code actually describing how it works and how it can be used safely.
Posted Sep 5, 2024 0:22 UTC (Thu)
by Ashton (guest, #158330)
[Link] (3 responses)
No definition of politeness has ever required that someone not point out falsehoods. If having someone point out that what you’re saying is contrary to the written record is “difficult” then that’s your issue.
Posted Sep 5, 2024 13:44 UTC (Thu)
by Wol (subscriber, #4433)
[Link] (2 responses)
Case in point - journalists are ever eager to demand "scientific proof positive". THERE IS NO SUCH THING. And when two people are talking past each other, it's pretty obvious they have two different (and they could both be right!) definitions of truth.
When two logical, rational people disagree, it's a pretty safe bet they don't share the same set of facts. That appears especially true in this case.
Cheers,
Posted Sep 5, 2024 13:57 UTC (Thu)
by pizza (subscriber, #46)
[Link]
I disagree (heh).
Two logical, rational people can easily share the same set of underlying facts, but disagree about how how much weight each individual fact should carry in any given decision.
(This is epitomized by the expression "Fast, cheap, good -- pick two")
Posted Sep 22, 2024 17:40 UTC (Sun)
by Rudd-O (guest, #61155)
[Link]
However, the two controversies that I am seeing clearly here do not (at least to me) seem to be matter of opinion.
The technical controversy arising from that patch that fixed the circularity issue in the ownership of objects under DRM seems quite clear cut to me. Circularity is almost always bad design; and having some object hold a reference to another which holds a reference to the first (for printk()'s sake), which then prevents the first object from freeing the other or the other from freeing the first safely? Bad. Asahi's fix should have gone in without question, rather than invoke curt NAK.
The more social or human controversy about the animosity that Rust developers are getting from some kernel developers that do not want or do not appreciate some strictures that (they perceive) Rust is imposing on them? Well, you could argue it could go either way, but we have video. We have video of a rust developer explaining how the type system prevents certain classes of errors to an audience of C file system kernel developers. And one of the kernel developers interrupts him, doesn't let him finish and accuses him of trying to spread the Rust religion or impose it on the kernel space. This is comically easy to judge.
No one in the Rust for Linux project is trying to rewrite the kernel into Rust or trying to make everybody ditch C and learn Rust. (I would if I had a magic wand, but I don't). What I have seen indicates to me that Rust for Linux developers are discovering deficiencies in existing subsystems, and structures, and algorithms, and drivers, and APIs, and have been trying to fix those deficiencies with the best of intentions, and also have been trying to help the C developers see that there is a way that they can prevent having those deficiencies. And not everybody, not the majority, not many, But sadly a few kernel developers have reacted in a negative and destructive way to this effort.
A computer language that is more rigid than another has its drawbacks. And sometimes these drawbacks are very serious. But one thing that is not a drawback is with that language and its strictures help you find issues that were previously unseen before. Folks who are exposed to these previously unseen issues should be thankful that these issues have been brought to light because now that they can be seen and addressed, they can be fixed. Can you make bad APIs with Rust? Absolutely. Is it easier to make bad APIs with Rust than with C? struct void* struct {}.
Posted Sep 6, 2024 5:36 UTC (Fri)
by marcH (subscriber, #57642)
[Link] (1 responses)
> Possibly, I don't know. But it's not the size of the change that matters when you're getting close to the core, it's the impacts and what the change reveals. You said yourself that you've found deep problems with the current API and that it's totally unsafe even for C. Did you ever think that the maintainer himself probably doesn't trust this code at all anymore and is not willing to permit more code to get more intimate with it ?
I see this as the core issue and key contradiction here. On one hand the design is "very bad and incredibly complicated and brittle", but on the other hand you submitted a "small, non-intrusive patch that barely changes anything". Mmmmm... anyone with a little bit of experience now how this song _usually_ ends; I won't repeat wtarreau.
You look like one of those exceptional developers who are indeed capable of playing that sort of Jenga successfully. But it's not enough to be correct: building up the corresponding _trust_ unfortunately requires a massive amount of explanations, demonstrations, demo / test code and generally: time that you said you don't have. While the Rust versus C "racisms" do not help, I also suspect it would not be so different if you removed Rust from the picture. We've seen that story many times before - in a single language.
If Linux gets a new, generic scheduler entirely in safer Rust code then we all win? A complete rewrite is also a typical next chapter after "it's very bad and too complicated" - again even with a single language. Cause it's... faster.
Posted Sep 6, 2024 12:37 UTC (Fri)
by daenzer (subscriber, #7050)
[Link]
Posted Sep 22, 2024 17:24 UTC (Sun)
by Rudd-O (guest, #61155)
[Link]
Quite ironically, this is exactly what the rust for Linux people are trying to do. And it is not exaggeration to say that exactly that, wanting the code to document the behaviors of the API is what got Wedson interrupted by Ted.
Put the behavior of the API in the types. That is all what the Rust people are asking for. Nothing more. Actually, they're asking for something that's even less than that. They're asking for cooperation from the C part of the kernel so that they can do that work of putting the meaning of the API and the behaviors of the API into the type system so that the system can be safe and sound.
To me this controversy does sound like a few kernel developers, not everybody, not the majority, but a few, are seriously looking at a gift horse in the mouth. Maybe they just don't want the horse? And as a result, nobody else gets the pony we want.
Now I'm not a kernel developer, I'm simply a roast programmer, a fairly new one, but I do know, having used Linux since 1996, that I want superior kernel code quality and stability and fewer security issues. We are never getting there, unless we move on from C, at least for the critical parts of the system. And I frankly do not care if it is in Rust or in another language, But if we are going to jump out of a puddle and not fall into another, it has to be a type safe and sound language that protects against the sort of problems that see causes. There is this precious opportunity right now to take that chance by using Rust. I hope we don't squander the opportunity and have to wait 10 more years until either everybody is using a different operating system, or someone invents a superhuman AI that can type, check, see, and its runtime properties properly, or Linux has finally decided that it's going to incorporate sound and type safe programming tooling beyond C + glorified linters.
Posted Sep 4, 2024 18:50 UTC (Wed)
by mb (subscriber, #50428)
[Link]
Be assured that there are many people on your side of the discussion.
Thoughts and clarifications
Wol
Wol, slow down please? Perhaps we don't need to drag this stuff into every unrelated discussion? Please?
Slow down a little
Slow down a little
Wol
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
The borrow checker as a significant advance
The borrow checker as a significant advance
The borrow checker as a significant advance
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Wol
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
> sender until they write a documentation patch, because seriously this
> stuff is too hard for someone to just keep it in their head and expect
> everyone else to understand from reading the code.
Thoughts and clarifications
Thoughts and clarifications
Wol
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Thoughts and clarifications
Even if they are more silent than the verbose people who don't admit that they are wrong.