Deeper problems in the Linux kernel
Deeper problems in the Linux kernel
Posted Jan 31, 2025 17:43 UTC (Fri) by wtarreau (subscriber, #51152)In reply to: Deeper problems in the Linux kernel by npws
Parent article: Resistance to Rust abstractions for DMA mapping
What I'm reading in all this thread is a huge confusion between "maintaining" and "developing". It's basically said "the work's already done blablabla". No. That's the development part. The maintenance part *starts* once the code gets merged and *ends* for developers. Maintaining means:
- reading unqualified bug reports that possibly look like they might be related to what you're working on, and scratch one's head to try to guess if that could be related or not.
- helping reporters get more traces, debug info, run tests, patches etc to try to better figure the problem
- write fixes and test them or get them tested
- deal with backports (a nightmare in some API changes, especially when patches apply and build fine while you know the API differs)
- deal with security reports, in particular the mighty ones where you're supposed to be discrete and not request everyone's help since you're a maintainer and you know.
- and all that most often alone before others have a look, or even against those who initially say "it's probably something over there".
It's simply not a matter of sending a write-only code piece to someone and considering the job done over the week-end, it's a permanent, hard work. And the worst of this is that the maintainer's job is one of those that are the hardest to scale, because it's not just based on time and skills, it's based on experience. You can decide to have as many co-maintainers as you want, if you have 30 years of experience, it will take 30 years for a new one to be able to replace you with the same level of expertise, judgement and autonomy.
*That* is the problem.
I know Christoph can easily have harsh words, and I don't know the level of complexity of the proposed changes, but I know that as a maintainer of another software, it often happens to me to have to say "no" to some changes because I figure they will get us into hell, without the person in front of me being able to understand why, by lack of experience on that matter. So in this case, I have to sit down with the person and work for a few days to try to redesign their change with them in order to reach a point where nobody is able to envision a future major problem anymore (that doesn't mean there will be none, just that the maximum chances will have been put on our side). And it even happens with other highly skilled developers and maintainers for whom I have a lot of respect, just like some of them sometimes reject my changes for the same reasons. Keep in mind that every time an issue pops up, the first ones that are questioned are those who know the bits and bytes at the lowest levels and know how to read a backtrace, an hex dump or a network trace, and the vast majority of the time in the end it's not for them, it only allows to know whom to route it to.
The issue we're often reading here is "these bastards are rejecting our stuff, let's replace them". But it *exactly* illustrates the lack of maturity and experience needed to gauge the impacts of certain changes for existing maintainers. And there are areas that are trickier than others, just like there are maintainers who have more elasticity than other ones to adopt new technologies, and that's fine. But as a general rule-of-thumb, if you have not been forced to maintain an API you've designed for 10+ years regretting some early mistakes every single day you touch it, you just have no idea what a maintainers' difficulties are and you cannot qualify to judge their job nor their judgement, essentially based on intuition and recognized patterns.
It's certain that the form could be better. Christoph is known for not taking gloves. Others might just simply feel overwhelmed and decide to silently resign because they don't see themselves with a extra level of complexity. Others might just think "let's see, and I might give up later if/when it becomes too hard". Others might feel more at ease with estimating how to address shortcomings or might even try to themselves take part of the change. Nobody's equal and it takes of everyone to get a collective expertise as there is on Linux. Let's just not trivially dismiss the difficulties some may face in front of a task they did not decide to receive.
Posted Feb 1, 2025 0:07 UTC (Sat)
by NYKevin (subscriber, #129325)
[Link]
That is not what Christoph did. He said[1] that he would not accept any Rust in "core systems" of the kernel, that he would not work with them on iterating the proposed solution, and that he would not allow anyone else to do that work in his stead. He also said[2] that he is opposed to further Rust in the kernel entirely.
> The issue we're often reading here is "these bastards are rejecting our stuff, let's replace them". But it *exactly* illustrates the lack of maturity and experience needed to gauge the impacts of certain changes for existing maintainers.
That is not what the RfL developers did. They started with the premise that Christoph was suggesting an alternative approach, and asked for specifics.[3] They escalated to Linus and Greg *only* in response to [1], at which point it was obvious that there was no alternative other than dropping the whole patch (which would seriously jeopardize the future of any Rust drivers that want to use DMA).
I can absolutely believe that there was room for further discussion of the specific approach to Rust-DMA integration. Sure, the patch was on revision 8, but 8 is not a magic number, and it's entirely plausible that they could've done further revisions if Christoph had asked for them. Christoph did not ask for revisions, because he does not want Rust-DMA integration to happen. He also strongly implied that he wants the RfL project as a whole to fail (or at least, to stop merging patches that add Rust to Linux, which is the whole premise of RfL). These are not inferences on my part. He said all of these things just in that one thread alone.
Seeing as Linus has already decided that RfL will be allowed to move forward for the time being, escalating to him and Greg was the right call. Christoph is directly challenging that decision, and it is not the place of the RfL developers (or anyone else) to determine whether or not Linus is going to change his mind based on Christoph's argument. Linus is the only person who can resolve this dispute, so he needs to be involved.
[1]: https://lwn.net/ml/all/20250128092334.GA28548@lst.de/
Posted Feb 16, 2025 0:03 UTC (Sun)
by lacos (guest, #70616)
[Link]
One of the best comments I've ever read on the internet (not just on LWN).
If you are a maintainer, then one way or another, you *will* be held responsible for any symptom that emanates even a *whiff* of your subsystem. You'll indeed be asked to triage it, and oftentimes, confidently saying that the bug is in some other (particular) subsystem or layer, requires reproducing and *completely* tracking down the bug. Sometimes you know *where* it is only when you learn *what* it is.
> Others might just simply feel overwhelmed and decide to silently resign
Right; that's what I've done once in the past, when I found myself in a somewhat similar situation as a co-maintainer in another project (I disagreed with both the feature and how exactly it was being developed). This is ultimately a fight for power (= who dictates what direction the project follows). Not everybody has the oomph that Christoph does, to take on the fight (I'm saying this *without* praising his particular style) -- I did quit (not silently, but civilly for sure).
I think Christoph has every right to resist a direction that he perceives as disruptive to his ability to maintain a project to which he has dedicated decades of his life. If he doesn't budge -- some differences may be irreconcilable --, the proponents of Rust can fork Linux, or can try to eject or marginalize Christoph.
We should be aware that "forking" is older and broader than "forking a software project". Here's an example:
https://en.wikipedia.org/wiki/Vienna_Secession
(And the funniest part of it: "the [seceded] group *itself* split, when some of the most prominent members [...] resigned in a dispute over priorities" (emphasis and light editing mine).)
Deeper problems in the Linux kernel
[2]: https://lwn.net/ml/all/20250131075751.GA16720@lst.de/
[3]: https://lwn.net/ml/all/CANiq72=R2N+QiCiHZQu-6YJ9aYpn_bik3...
Deeper problems in the Linux kernel
