Rust-for-Linux developer Wedson Almeida Filho drops out
After almost 4 years, I find myself lacking the energy and enthusiasm I once had to respond to some of the nontechnical nonsense, so it's best to leave it up to those who still have it in them.
As an example of the sort of "nonsense" he referred to, he provided a link to the video from the
Rust for filesystems discussion at the 2024
Linux Storage, Filesystem, Memory-Management, and BPF Summit. His work was
fundamental to getting the project as far as it has come; he will be missed.
Posted Aug 29, 2024 14:54 UTC (Thu)
by b7j0c (guest, #27559)
[Link] (4 responses)
Posted Aug 30, 2024 12:22 UTC (Fri)
by arsen (subscriber, #161285)
[Link]
Posted Sep 1, 2024 13:18 UTC (Sun)
by SLi (subscriber, #53131)
[Link] (2 responses)
Posted Sep 3, 2024 4:08 UTC (Tue)
by marcH (subscriber, #57642)
[Link] (1 responses)
Management is finally waking up to the cost of vulnerabilities - and of quality/sloppiness more generally speaking. Better late than never.
Posted Sep 7, 2024 17:02 UTC (Sat)
by SLi (subscriber, #53131)
[Link]
Posted Aug 29, 2024 15:31 UTC (Thu)
by marduk (subscriber, #3831)
[Link] (21 responses)
Posted Aug 29, 2024 15:45 UTC (Thu)
by shironeko (subscriber, #159952)
[Link]
Posted Aug 29, 2024 16:44 UTC (Thu)
by e-rk (subscriber, #166547)
[Link] (3 responses)
Posted Aug 29, 2024 16:57 UTC (Thu)
by pizza (subscriber, #46)
[Link] (2 responses)
You forgot 'and (nominally) copyleft" with "permissive" '
Meanwhile, I'm not exactly holding my breath, because these large companies have universally shown that they're utterly incapable of cooperatively acting in their own longer-term interests. Heck -- even individually, they have a tendency to keep building (and using) footguns at every opportuntiy.
Posted Aug 30, 2024 7:23 UTC (Fri)
by marcH (subscriber, #57642)
[Link]
Joking aside, there's a big and obvious difference here between public and private companies.
Posted Aug 30, 2024 11:03 UTC (Fri)
by etrusco (guest, #4227)
[Link]
Posted Aug 29, 2024 17:56 UTC (Thu)
by lonely_bear (subscriber, #2726)
[Link]
Posted Sep 2, 2024 16:30 UTC (Mon)
by anthm (guest, #173221)
[Link] (14 responses)
Google/MS/Amazon/Meta could announce a new Rust kernel foundation tomorrow and have it staffed up with experienced kernel devs quite quickly. That's the real nightmare scenario for Linux. A well supported competitor that from day one has immense resources and technical expertise available, plus buy in from companies that currently push a *ton* of Linux code.
Posted Sep 2, 2024 17:08 UTC (Mon)
by pizza (subscriber, #46)
[Link] (12 responses)
This reminds me of a quote: "Do you know what Vegas says to folks that 'have a system'? Welcome!"
They'll find that recreating the sheer breadth and scope of what Linux provides *today* will take many, many, many man-years of effort, along with being forced to re-discover (and re-invent workarounds for) any number of scaling pain points.
> That's the real nightmare scenario for Linux.
How so? In the worst case Linux reverts back to a "just for fun" hobby project.
> A well supported competitor that from day one has immense resources and technical expertise available,
On "day one" there won't be anything other than a blank repository.
Posted Sep 2, 2024 19:38 UTC (Mon)
by NYKevin (subscriber, #129325)
[Link] (11 responses)
Then they will start carrying Rust patches against Linux, because it is less work than starting over from scratch, and much Rust-in-Linux code will already have been written by that point.
Then they start rewriting anything with complicated or "weird" lifetime semantics in Rust, so that their patches apply more cleanly and they don't have to constantly adapt to whatever upstream is doing.
Then they will stop contributing to upstream, because it just doubles the amount of work they have to do, and upstream is notoriously hostile to out-of-tree patches anyway.
Then their fork becomes the main fork, because the contributors who work for big tech companies outnumber the contributors who do not. The contributors who both work for big tech companies and are Rust-hostile are told to either learn Rust and start writing it, or find a new line of work.
At some point, I give good odds that they convince Linus to declare their fork the official Linux, similar to the EGCS/GCC split. By the time that happens, large portions of the kernel may have been rewritten in Rust, instead of the smaller interfaces we are now discussing.
This is a possible future. It is (obviously) far from inevitable. But the more hostile the current development process is to Rust, the more the big tech companies' incentives will align with this future, and the more likely it is that they will attempt to make the above, or something like the above, happen.
(Disclaimer: I work for Google, but the above is pure speculation and conjecture. I do not have any inside knowledge of Google's kernel development practices, or under what circumstances they might or might not choose to start publicly developing a Linux fork. I have even less knowledge of what any of the other tech companies are likely to do.)
Posted Sep 5, 2024 11:44 UTC (Thu)
by taladar (subscriber, #68407)
[Link] (10 responses)
Posted Sep 5, 2024 12:31 UTC (Thu)
by johill (subscriber, #25196)
[Link] (8 responses)
Posted Sep 10, 2024 9:55 UTC (Tue)
by taladar (subscriber, #68407)
[Link] (7 responses)
In a world of finite amounts of total effort available every bit of effort put into a project needs justification and so does every choice among mutually incompatible goals. The ratio of work generated to people who care to put in the work for every single decision is a big factor here.
Posted Sep 10, 2024 14:58 UTC (Tue)
by johill (subscriber, #25196)
[Link] (6 responses)
Your previous comment is implying that e.g. me saying I need to care about platforms that Linux supports today etc. is actually just being done in support of my pre-conceived notion that rust is bad and not going anywhere etc.
Do you actually think that helps rust's cause? Because if so I think this thread simply ends here, there's nothing I can say to that.
On the other hand, maybe you're just confused? I don't have a _choice_ to care about platforms. I'm not going to be in a position to say that from this day on WiFi simply doesn't work on any platform without rust. That would be a pretty major regression, don't you agree?
Doesn't even matter if I like to use rust or if I think it's a bad idea, but like I said, implying that I say that just because I don't even want rust ... I feel your attitude probably isn't representative, and that can only be a good thing.
Posted Sep 10, 2024 16:14 UTC (Tue)
by mb (subscriber, #50428)
[Link] (5 responses)
With wifi support I mean you must actually be able to put a wifi device in there and use it.
Then we have a base for judging whether we should care about those users or not.
Posted Sep 10, 2024 16:49 UTC (Tue)
by johill (subscriber, #25196)
[Link] (4 responses)
SH enables wifi in the defconfig too.
USB device support is probably pretty universal across architectures.
Anyway, I'm just pointing out that saying "you were always against rust, so you just pretend you care about architectures" is the wrong way to go about this.
Posted Sep 10, 2024 17:19 UTC (Tue)
by mb (subscriber, #50428)
[Link] (3 responses)
Rust compilers work pretty damn well, even if they don't have the official tier 1 blessing.
And it's not that all stable gcc versions support building the kernel.
Based on that, I think arm32 should not be a blocker.
We need to apply the same basic rules for gcc, clang and Rust.
>SH enables wifi in the defconfig too.
Ok. Is that really a big thing? Should that block us? These are the questions I'd like to see.
>USB device support is probably pretty universal across architectures.
Yes. But IMO that doesn't mean we should support users who stick an USB host controller into an Alpha machine to put an USB wifi into it.
>saying "you were always against rust, so you just pretend you care about architectures"
Yes, correct. This is wrong to say.
"you do" and "you just" sprinkled with "religion" and "you force me to..." is the biggest problem in this whole discussion. And it's also not limited to the kernel. I saw this exact same behavior in completely different contexts, too.
Posted Sep 10, 2024 18:01 UTC (Tue)
by johill (subscriber, #25196)
[Link] (2 responses)
The kernel doesn't support it, according to the rust documentation. Maybe that's outdated, or maybe you're saying that just about the compiler?
> Ok. Is that really a big thing? Should that block us? These are the questions I'd like to see.
I have no idea. WiFi is pretty universal these days.
But even the rust people don't yet seem to advocate making rust a hard dependency for much if anything, so I think the point is moot. They also don't have a story for calling rust from C rather than the other way around, which would actually be the use case I'd first be interested in here to make some critical parsing code better.
Posted Sep 10, 2024 18:35 UTC (Tue)
by mb (subscriber, #50428)
[Link]
Yes, I was only talking about the compiler. Adding kernel support looks like a minor issue to me.
>They also don't have a story for calling rust from C
The Rust code just has to provide an unsafe extern-C function to be called. That's it. There's nothing special about that. It's a Rust function that has normal C linkage.
>But even the rust people don't yet seem to advocate making rust a hard dependency for much if anything,
People need to understand why that is on a case by case basis.
Posted Sep 10, 2024 19:06 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
Calling Rust from C is fairly well established outside the kernel, with cbindgen helping to reduce the amount of C-side boilerplate you have to keep in-sync with the Rust #[no_mangle] code and #[repr(C)] data structures.
I would hope that the Rust-for-Linux people would be willing and able to help you do a good job if you were trying to write in-kernel Rust that's called from C - adding in things like cbindgen to the build system to make your life as simple as possible.
Posted Sep 7, 2024 1:00 UTC (Sat)
by DianaNites (subscriber, #160945)
[Link]
Posted Sep 2, 2024 17:09 UTC (Mon)
by mikebenden (guest, #74702)
[Link]
Oh, Lordy, *please* let them go and do that!
Worst case scenario, it would provide an outlet for all the "rustacean" kernel pushers to go be with their own kind. If they can actually come up with a useful result, that'd be a nice bonus... :)
Posted Aug 29, 2024 15:40 UTC (Thu)
by bredelings (subscriber, #53082)
[Link] (1 responses)
Posted Aug 29, 2024 15:45 UTC (Thu)
by nye (subscriber, #51576)
[Link]
Posted Aug 29, 2024 17:26 UTC (Thu)
by shironeko (subscriber, #159952)
[Link] (218 responses)
As a result it is a system that is resistant to change, because change always generate angry emails. The system self-select for people that share that mentality and as a result change is even harder. It is only when the angry emails that would result from not accepting the change out number the ones from it's acceptance do any change get merged.
IMO the mistake Rust-for-Linux repeatedly make is trying to upstream piecemeal and when they don't have enough users to send all the angry emails. Angry emails and the threat of angry emails is what drives the kernel development, if you want to participate then you gotta play this game.
Posted Aug 29, 2024 18:31 UTC (Thu)
by rc00 (guest, #164740)
[Link] (58 responses)
> IMO the mistake Rust-for-Linux repeatedly make is trying to upstream piecemeal and when they don't have enough users to send all the angry emails. Angry emails and the threat of angry emails is what drives the kernel development, if you want to participate then you gotta play this game.
This is brigading (something the Rust community is actually known for already) and it is not productive, nor is it something that should be encouraged or condoned. There are technical challenges to the adoption of Rust that Linus himself pointed out recently. There was an article on LWN recently about unsafe Rust in the kernel and the problems posed there as well. The feedback from key kernel developers might not be great but the proper community response is not to be worse. Focus on solving problems and the rest will follow.
Posted Aug 29, 2024 19:26 UTC (Thu)
by hunger (subscriber, #36242)
[Link] (54 responses)
Its not rust specific: Any community with young and enthusiastic members does that.
We definitely annoyed the heck out of everybody in the 1990s asking people to rewrite their code in C++ -- for the safety that enables.
Funny how annoyed those people are when 30+ yesrs later the same happens to them.
Posted Aug 29, 2024 19:33 UTC (Thu)
by rc00 (guest, #164740)
[Link] (53 responses)
Well to be fair, Linus had quite the response for the C++ devs. 😉
Though I do wonder why Linus didn't respond the same way to what is effectively this generation's C++.
Is it a generational rite of passage to annoy Linux kernel developers with the programming language du jour? 😅
Posted Aug 29, 2024 19:59 UTC (Thu)
by hunger (subscriber, #36242)
[Link] (11 responses)
IMHO because C++ significantly improved C, but Rust changed the game by proofing that you can have memory-safety without runtime overhead.
Whether or not you like rust, it has a huge impact due to demonstrating that -- inside and outside of our industry. E.g. the white house policies on cyber security would probably look different without that example.
Posted Aug 29, 2024 22:28 UTC (Thu)
by LtWorf (subscriber, #124958)
[Link] (10 responses)
Posted Sep 2, 2024 20:22 UTC (Mon)
by anthm (guest, #173221)
[Link] (8 responses)
Calling it "vague recommendations" is, IMO, deliberately misconstruing the situation. It's not banned becasue it's not practicable (yet) to ban it.
Posted Sep 2, 2024 21:46 UTC (Mon)
by LtWorf (subscriber, #124958)
[Link]
Posted Sep 3, 2024 1:45 UTC (Tue)
by viro (subscriber, #7872)
[Link] (6 responses)
You'll get a bunch of preexisting bugs obfuscated and made harder to find.
Posted Sep 5, 2024 7:15 UTC (Thu)
by NYKevin (subscriber, #129325)
[Link] (5 responses)
You cannot ever have multiple mutable references to the same array element (or any other object, for that matter), because the borrow checker forbids it (and if you cheat and do it anyway with unsafe code, it is immediate UB because of noalias). Without mutable aliasing, it's not possible for two different parts of the program to both think they own the Nth slot in the array at the same time.
The lack of mutable aliasing must be provable at compile time just from the types of the arguments. It immediately follows that you cannot conjure a mutable reference out of thin air in safe code - it must come from something you already own or another mutable reference. It does not matter whether you're indexing into an array or doing some more complicated dance. Mutability is transitive, and all of the relevant loopholes are either protected by unsafe (UnsafeCell<T>) or by runtime checking (RefCell<T> and most of std::sync).
In the specific case of the static fake heap, this fails on several different grounds:
0. All interactions with mutable statics are unsafe, because of thread safety. Unsafe code can, of course, do a UAF in Rust. That's not news, so in order for this argument to be interesting, the whole thing must work in safe code. But we can just pretend you said "a long-lived boxed slice" instead of a static array, so let's ignore this and continue.
I will admit that DARPA's goals seem (at best) extremely ambitious to me. But if they do succeed, their generated code will not look anything like what you describe (or else they won't have accomplished what they now claim to be trying to do).
Posted Sep 7, 2024 17:20 UTC (Sat)
by SLi (subscriber, #53131)
[Link] (1 responses)
Posted Sep 8, 2024 11:57 UTC (Sun)
by farnz (subscriber, #17727)
[Link]
That pretty much falls out from DARPA's mission statement. They exist to fund revolutionary technology that would otherwise fail to get funded, aiming for transformational change from every success, and eating a lot of failures in the process. Hence things like Silent Talk and TRACTOR - both high-risk projects, that would transform the world if they succeed.
And going back in time a bit, the ARPANET was a high-risk project when it started; at the time, communications was circuit-switched, and telecoms companies of every stripe were uninterested in taking the gamble to find out if wide-area packet switching could be made useful, when they were quite happy with circuits. Had DARPA not funded it, ARPANET would not have existed, and we'd probably not have any global packet switched networks, since there was a belief that the size of router buffers in packet switched networks would grow with the size of the network, disproven by ARPANET not needing large packet buffers.
Posted Sep 8, 2024 9:13 UTC (Sun)
by Wol (subscriber, #4433)
[Link] (2 responses)
The only reason I can think of that is a simple locking algorithm - write your pid into a variable, wait a timeout, if your pid is still there you have a lock. That obviously requires everyone to have mutable access to that variable.
Am I right in thinking that would be okay in unsafe code, provided the mutable reference never escaped the unsafe block? I can think of a couple of race conditions, so the unsafe block could lie to the compiler by accident, but the approach is sound if the implementation is correct?
Cheers,
Posted Sep 8, 2024 9:22 UTC (Sun)
by mb (subscriber, #50428)
[Link]
The unsafe block as such doesn't give you rights to violate rules.
Posted Sep 8, 2024 10:16 UTC (Sun)
by LtWorf (subscriber, #124958)
[Link]
Posted Sep 7, 2024 1:38 UTC (Sat)
by montj2 (guest, #111739)
[Link]
Posted Aug 29, 2024 20:02 UTC (Thu)
by kbusch (guest, #171715)
[Link] (35 responses)
I kinda thought it was as a social issue: the kids these days are not learning C like they used to, and kernel developers/maintainers are not eternal.
Posted Aug 29, 2024 20:31 UTC (Thu)
by rc00 (guest, #164740)
[Link] (34 responses)
That is false, at least in the United States. Colleges (and high schools with AP courses) are still teaching C, Python, and even Java predominantly. In fact, the professors that I have communicated with have avoided Rust specifically for many reasons. The overall complexity (especially for younger coders) and the lack of adequate opportunities in the labor market are the most frequently cited among the many reasons.
> and kernel developers/maintainers are not eternal.
I feel like the issue isn't whether or not kernel contributors are eternal but whether they are replaceable. Handing the baton to the next generation and making sure that they are good and capable hands isn't easy for any project or even company. Nothing last forever no matter how much we want it to.
Posted Aug 30, 2024 22:46 UTC (Fri)
by roc (subscriber, #30627)
[Link] (33 responses)
The problem is that once people are familiar with a language they tend to use it for everything, especially people who don't see programming as their primary job. So we have scientists writing code for their papers in Python and making egregious mistakes that a stricter language would have caught :-(. It's not clear how to fix that but the Julia folks are trying, at least.
In that light, C being replaced by Python in education is good. No-one sane will try to use Python to build a kernel or other infrastructure, so they'll be forced to learn another language, and hopefully it will be Rust and not C or C++.
Posted Aug 31, 2024 0:00 UTC (Sat)
by rc00 (guest, #164740)
[Link] (31 responses)
Agreed. I would probably take it a few steps further than that though. 😉
> Neither is C, though.
Strong disagree. C is a simple language, just not an easy one. All the fundamentals around how a computer works, syscalls, filesystems, etc. are lessons worth learning and that the language does a good job teaching. And on top of that, learning C means it is generally easier to learn other languages after the fact. C's lack of rules and restrictions are a double-edged sword but it is typically much more direct to code. Factually speaking, C helps to learn the fundamentals of programming.
> Java is much better, and Python better still.
Again, I communicate with professors and I would advise the same for you before making these kinds of comments. Some of the professors like Python because it is a much more readable language for newcomers and far easier to learn. The issue as I understand it from professors is that it sets a poor foundation. Too much is abstracted too early on in the development of these young programmers. Because there was a generation that only got Python given to them, they struggled to understand that sorting is not an operation that takes O(1) time. That is why some professors have returned to C. Explaining algorithms to someone who at least has an understanding of C data structures is far easier than someone with just Python.
> So we have scientists writing code for their papers in Python and making egregious mistakes that a stricter language would have caught
This hand-waves by too many distinct points so I'll try to break them out separately:
1. Python being a more approachable language for non-programmers meant that they were the perfect target audience. The mistakes are due to inexperience. But Rust is far from being the solution. Non-programmers have a task they want to accomplish, not spend the next two days fighting with the borrow checker and getting nowhere. The average person wants a car that they can enter and then just go. No car that requires gymnastics with multiple keys just to start the car is going to be adopted over that, no matter how shiny the features might be.* People enter cars to get from point A to point B. Back to Python, libraries like NumPy, SciPy, Pandas, PyTorch, TensorFlow, etc. etc. etc. combined with the simplicity of Python are the reason Python took a strong foothold in the data science and machine learning realm, along with the beginner-friendly nature. Performance is tolerable, it's a memory safe language, and it's easy for non-programmers to read *AND* write. Saying that a stricter language is the solution is like saying they should have twenty keys to start a car instead of one and acting like an accident is somehow inevitable. You can have memory leaks in Rust, you can have poor logic in Rust, and you can have egregious mistakes in Rust.
2. Rust proponents really don't understand history. The fact is that simple things see adoption and more complicated ones fall away. "Keep it simple" as it were. Rust's biggest impediment to real adoption is that it is too complex and prides itself on that. Traits, lifetimes, the borrow checker, async, unsafe, procedural macros, etc. etc. etc. There was a time that Go and Rust were considered to be on equal footing but that is not the case if you look at job openings. Today, there are almost 10,000 job openings related to Go. There are less than three hundred for Rust. (In my circles, Rust has been the "future" since 2017 or so.) The fact of the matter is that simpler languages will ultimately win out. Otherwise, Haskell and Elixir would have caught on by now instead of Python and Go. Rust's first-mover "advantage" over "new-langs" like C3, Odin, and Zig will be erased once those languages mature and find a footing. Why would a new person learn a complex language that takes longer to learn and then to write over a language that is easier to learn and profoundly more productive to code in? (I'm intentionally avoiding the topic of the productivity curse that is the compile speeds of Rust.) This same philosophy applies to things like the Linux desktop. The list of most-popular distributions has a huge overlap with the simplest ones. Linux From Scratch isn't a threat to Ubuntu's marketshare on the desktop space. Complexity is only a target for a niche audience though I also like how Terry Davis put it. 😉
*To expand on the example: If I offered you or anyone a flying car today but starting it required twenty keys and each key needed to be inserted/activated in the correct order just to get the car to start, would you or anyone adopt it? Some keys need to be inserted, some need to be twisted, and some are like buttons. Getting one key in the sequence out of order or used improperly (i.e. twisting a twist key the wrong direction) means you have to start the sequence over entirely. A very small percentage of people will embrace that and just because they can start the flying car doesn't mean they'll avoid all kinds of accidents like birds or other vehicles. Yet, the car would probably still be marketed as "safer." 🙃
Posted Aug 31, 2024 1:04 UTC (Sat)
by roc (subscriber, #30627)
[Link] (20 responses)
Filesystems have nothing to do with C. If you really want to learn about syscalls and how a computer works, you need to write or at least read assembly.
> And on top of that, learning C means it is generally easier to learn other languages after the fact. C's lack of rules and restrictions are a double-edged sword but it is typically much more direct to code.
C is full of rules and restrictions, they're just not enforced by the compiler. Try explaining undefined behavior to a new programmer! Of course, no-one teaches that, so students aren't really learning C, they're learning just enough to get the right output from one run of some toy code. And that's dangerous because they think they know how to write reliable code in C when they don't.
> Again, I communicate with professors and I would advise the same for you before making these kinds of comments
No need to be condescending. I have a PhD in computer science and a lot of my friends are professors at various universities in the USA and New Zealand. While we can have different opinions on what makes a good first language, what professors choose to teach is a matter of facts. Here's some actual data from 2019: https://research.leune.org/publications/ICCSE21-CS1CS2.pdf
> 1. Python being a more approachable language for non-programmers meant that they were the perfect target audience. The mistakes are due to inexperience. But Rust is far from being the solution.
I agree! But Rust isn't the only option that's stricter than Python. That's why I highlighted Julia for scientific users, not Rust.
> You can have memory leaks in Rust, you can have poor logic in Rust, and you can have egregious mistakes in Rust.
I wouldn't push Rust for scientific users but this kind of argument is a common fallacy that's language-independent. You can make mistakes in any language, but there are large classes of mistakes that are prevented in some languages but not others, and that matters a lot in practice.
When you're publishing scientific papers, programming errors can have huge impact --- not just on the results, but on whoever depends on those results --- and using a language that catches more errors has huge value that in many cases would justify the extra effort required.
> Rust's first-mover "advantage" over "new-langs" like C3, Odin, and Zig will be erased
None of those languages can replace Rust because none of them provide the combination of performance and memory safety that Rust does (let alone the other kinds of safety Rust provides, like data-race freedom).
I actually think a simpler language than Rust that hits the same performance+safety target could be built and might be great --- try taking Rust and replacing its generics and macros with Zig-style comptime, but keeping the aliasing and borrowing model. But AFAIK no-one has even started trying to do that, and in the 10-20 years it would take to reach maturity if started now, Rust will likely be too entrenched.
Posted Aug 31, 2024 1:49 UTC (Sat)
by rc00 (guest, #164740)
[Link] (18 responses)
Would you consider Unsafe Rust and Safe Rust to be two languages or two parts of the same language? I'm skipping over the fact that the argument with Rust proponents went from "no one writes unsafe Rust" to "unsafe Rust is required for the Linux kernel" (paraphrasing
Specifically regarding the content of the article on this page, Unsafe Rust is inevitable in the Linux kernel and yet Rust was designed to make Unsafe Rust undesirable and difficult. Why write Unsafe Rust (which also means opting out of the borrow checker) instead of something like Zig that has around 80% of the memory safety that you get from Rust's borrow checker feature set but with 0% of the downside of the loss of productivity in fighting the borrow checker? (https://zackoverflow.dev/writing/unsafe-rust-vs-zig/)
Arguing for Safe Rust while dismissing that Unsafe Rust is almost inevitable is a different kind of logical fallacy. It sidesteps the reality that a language that isn't inherently memory-safe but instead makes memory-unsafe operations as safe as possible is the better solution for the assignment at hand. I believe both Zig and Odin also do not have undefined behavior where Unsafe Rust and C do. Why not pick the best tool for the job?
> I actually think a simpler language than Rust that hits the same performance+safety target could be built and might be great
This strikes me as Zig 1.0 if the current trajectory holds true.
Posted Aug 31, 2024 2:25 UTC (Sat)
by roc (subscriber, #30627)
[Link] (9 responses)
That's not the right way to think about it. "unsafe" lets you deference raw pointers, which means you can bypass the borrow checker by using raw pointers instead of references, but you can use references in unsafe Rust and they still get borrow-checked. This matters because it means that entering an unsafe block because e.g. you want to call a C function does NOT mean that you are suddenly also susceptible to lifetime bugs.
> Unsafe Rust is inevitable in the Linux kernel
It is inevitable that unsafe Rust is present in the kernel, but it matters a great deal *how much* is present and where it is present. If you have unsafe Rust in the Rust wrappers around kernel APIs but you can write a lot of drivers with zero unsafe Rust (which is actually the goal), then that is likely to be a big win. You write the wrappers, you invest effort in reviewing that code and testing it, and then all your driver code (which is much more code than the wrappers, in the long run if not the short run) benefits from the safe Rust guarantees.
OTOH if every driver has to use copious amounts of "unsafe" then Rust in the kernel will have failed.
> instead of something like Zig that has around 80% of the memory safety that you get from Rust's borrow checker feature set but with 0% of the downside of the loss of productivity in fighting the borrow checker?
In release builds (i.e. builds with acceptable performance) you get no use-after-free checking with Zig, and those are most of the exploited memory safety issues these days. Plus of course Rust is giving you more safety than just memory safety.
> I believe both Zig and Odin also do not have undefined behavior where Unsafe Rust and C do
UAF is for sure undefined behaviour in Zig. There is no way to define it!
> This strikes me as Zig 1.0 if the current trajectory holds true.
Nope, no UAF protection in production code with Zig.
Posted Aug 31, 2024 3:10 UTC (Sat)
by rc00 (guest, #164740)
[Link] (8 responses)
I've already placed my wager. 😉
> In release builds (i.e. builds with acceptable performance) you get no use-after-free checking with Zig, and those are most of the exploited memory safety issues these days. Plus of course Rust is giving you more safety than just memory safety.
Let's debug more of your programming language knowledge:
* Zig's standard library offers a general-purpose allocator that can be used to prevent double-free, use-after-free, and can also detect memory leaks.
* Zig has a build mode named ReleaseSafe that is not only fit for production code but considered the main mode for releases. With ReleaseSafe, memory and safety checks are still enabled.
Let's try to avoid spreading misinformation.
Additional links and resources:
Posted Aug 31, 2024 3:17 UTC (Sat)
by intelfx (subscriber, #130118)
[Link] (2 responses)
> Let’s try <…>
Let’s try to get our facts and logic straight before directing condescension at others, shall we?
Posted Aug 31, 2024 3:37 UTC (Sat)
by rc00 (guest, #164740)
[Link] (1 responses)
It's a general-purpose heap allocator. For all intents and purposes, it can be thought of as the default allocator. You can certainly use others for specialized cases but during development and debugging, why not use the general-purpose one out of good habit? How is this any different from kmalloc? (These are rhetorical questions. See the last sentence of this reply.)
> Let’s try to get our facts and logic straight before directing condescension at others, shall we?
Can you approach this thread with correct information? It's quite the take that you want to apply a negative connotation to "condescension" but yet you state that toxic comments that suggest brigading are "funny."
No, I'm not going to interact with you beyond this dribble.
Posted Aug 31, 2024 3:41 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link]
JFYI, Linux has a variety of ways pointers are allocated, kmalloc is only one of them. One of the problems with Rust adoption was that Rust's standard library was not ready for custom allocators and fallible allocations.
Posted Aug 31, 2024 4:05 UTC (Sat)
by roc (subscriber, #30627)
[Link] (3 responses)
Which leaks virtual memory (to catch heap UAF) and is not suitable for production use.
> Zig has a build mode named ReleaseSafe that is not only fit for production code but considered the main mode for releases. With ReleaseSafe, memory and safety checks are still enabled.
ReleaseSafe does not affect the choice of allocator and therefore does not detect heap UAF unless you opt into the general-purpose allocator, which is not suitable for production use.
And FWIW Zig has *nothing at all* to detect stack UAF.
Posted Aug 31, 2024 5:05 UTC (Sat)
by rc00 (guest, #164740)
[Link] (2 responses)
> Which leaks virtual memory (to catch heap UAF) and is not suitable for production use.
The general-purpose allocator is configured with never_unmap set to false by default. Only when set to true is there a possibility that every allocation can be leaked. Again, this allocator can be used for production release modes.
https://www.openmymind.net/learning_zig/heap_memory/
> ReleaseSafe does not affect the choice of allocator and therefore does not detect heap UAF unless you opt into the general-purpose allocator, which is not suitable for production use.
Please read the previous links. This was directly addressed.
> And FWIW Zig has *nothing at all* to detect stack UAF.
The language is still not 1.0 yet. Here is the issue where use-after-free stack allocations will be addressed:
I am sharing these links in good faith. Please take the appropriate time to process them.
Posted Aug 31, 2024 9:04 UTC (Sat)
by roc (subscriber, #30627)
[Link] (1 responses)
Yes, I checked all your links (and more) before I responded, and the information in those links is consistent with what I said. Your new link is also consistent with what I said.
> The general-zurpose allocator is configured with never_unmap set to false by default. Only when set to true is there a possibility that every allocation can be leaked. Again, this allocator can be used for production release modes.
This doesn't support your claims. It doesn't mention never_unmap or describe exactly what GeneralPurposeAllocator guarantees or say what the performance impact is. In fact those things are not documented anywhere outside the source code, as far as I can tell; certainly nowhere you have linked to. So I find your attitude irksome.
But in the spirit of https://xkcd.com/386, I went ahead and looked at the source code: https://github.com/ziglang/zig/blob/master/lib/std/heap/g.... The situation is pretty complicated, but as far as I can tell, the following things are true in the default configuration with ReleaseSafe build mode:
In more detail: the source code comments say:
It's not entirely clear from the comment what "memory slots" means here, but looking at the code it's more clear: https://github.com/ziglang/zig/blob/37df6ba86e3f4e0f5d6a2...
However, when freeing a sub-page-size object, if that means the bucket is *completely empty*, the entire bucket is freed: https://github.com/ziglang/zig/blob/37df6ba86e3f4e0f5d6a2...
Note that the comment "for larger ones it is handled in the backing allocator" is false for the default PageAllocator. There is no logic to avoid reusing virtual memory in that code. So virtual memory isn't leaked, but in exchange you get UAF bugs.
For allocations >= the page size, you enable UAF bugs immediately because PageAllocator can reuse addresses.
This Reddit thread agrees with me that "UAF will not be reliably detected": https://www.reddit.com/r/Zig/comments/1eysv2k/general_pur...
andrewrk did write: https://github.com/ziglang/zig/issues/3180#issuecomment-6...
> I am sharing these links in good faith. Please take the appropriate time to process them.
Oh, I've spent way too much time processing this.
> Here is the issue where use-after-free stack allocations will be addressed: https://github.com/ziglang/zig/issues/3180
Will it really, though? It is not easy at all to detect and prevent stack UAF bugs. For example, if you take the address of a stack value and store it in a heap object, is that allowed? It's often useful to be able to do that, so I guess Zig would want to allow it, but if you allow it, it can be very difficult to prove that the pointer is not used after the function has returned. If they do solve this, there will be some overhead and/or some existing useful Zig code that is no longer legal.
For now, safe Rust prevents UAF and all Zig has is partial, inefficient solutions and promises.
Posted Aug 31, 2024 13:34 UTC (Sat)
by corbet (editor, #1)
[Link]
Posted Aug 31, 2024 5:32 UTC (Sat)
by pbonzini (subscriber, #60935)
[Link]
https://lwn.net/Articles/863459/ has two unsafe blocks, both at initialization time, and that was three years ago. It's a simple driver, sure, and some sources of unsafely are only apparent with e.g. devices that do DMA, but rest assured that the developers have done their homework.
Posted Aug 31, 2024 14:32 UTC (Sat)
by kleptog (subscriber, #1183)
[Link] (7 responses)
I've never understood this argument. Saying "Rust is a bad choice for the kernel because you need unsafe sometimes" is the same as "C is a bad choice for the kernel because you need ASM blocks sometimes". Except nobody complains about the number of ASM blocks in the kernel.
It would be even worse if the option wasn't there. Then it would definitely have been a non-starter.
Posted Aug 31, 2024 15:08 UTC (Sat)
by rc00 (guest, #164740)
[Link] (4 responses)
Do you remember the earlier days of this years-long hype cycle for Rust? "No one should write Unsafe Rust, that defeats the entire purpose."
No responsible person writing C ever said that Assembly should be avoided at all costs. There was a time that C was considered a higher-level programming language, especially when viewed through the lens of something like Assembly. In the same way that Python is today's mainstream higher-level language, there are still plenty of internal components for it written in C. Pairing higher-level languages with lower-level languages is far from novel or frowned upon.
Rust is explicitly not the right tool for the job and this is based on what some of the largest Rust proponents have said in their own words. However, the argument against Unsafe Rust by Rust proponents was discarded to prioritize Rust being thrust into the Linux kernel project. "Rust by any means," so to speak. This strategic and constant shifting of the goalposts is the very reason there is a groundswell of intelligent people ready to move on from Rust, its related hype, and many of the related individuals. We went through this with Haskell, and Scala, and so on.
Posted Sep 1, 2024 1:57 UTC (Sun)
by roc (subscriber, #30627)
[Link]
An important part of Rust has always been building safe abstractions around unsafe code. See e.g. https://smallcultfollowing.com/babysteps/blog/2016/05/23/...
If you heard "no-one should write unsafe Rust" then someone was very confused, and I suspect it's you.
Posted Sep 1, 2024 19:05 UTC (Sun)
by kleptog (subscriber, #1183)
[Link] (2 responses)
The irony is, if you type this phrase in Google with quotes, it returns exactly one match: your comment. So it's not surprising I've never heard of it before.
> this is based on what some of the largest Rust proponents have said in their own words.
to which I can only say [citation needed]. Because I spent on time looking but could find any explicit instances, just more claims that these people did exist.
Posted Sep 1, 2024 20:18 UTC (Sun)
by rc00 (guest, #164740)
[Link] (1 responses)
You're surprised that what has become an inferior search engine can't find the hordes of toxic examples from an even more toxic community known to suppress and try to rewrite reality and then history, and then gaslighting whenever adverse topics are brought up, insinuating that they're fabricated. No, this is not a thread I'm going to continue with you or anyone else.
Posted Sep 1, 2024 21:17 UTC (Sun)
by corbet (editor, #1)
[Link]
Posted Sep 2, 2024 21:55 UTC (Mon)
by anthm (guest, #173221)
[Link] (1 responses)
Posted Sep 3, 2024 10:08 UTC (Tue)
by mbunkus (subscriber, #87248)
[Link]
> Rust is no more secure than C/C++. Memory ownership is an important part of security BUT not the only one. That is simple problem with most folks who talk about rust being the magic bullet; it is not. Memory safety might feel like a hot issue right now but in 5 years there will be another one; even with moving to rust.
I find this to be completely ludicrous, given examples such as Microsoft's observation that 70% of their observed security vulnerabilities are, in fact, memory safety issues. I wonder if these types of comments come from malice or ignorance. I'm inclined to believe it's the latter.
Posted Aug 31, 2024 9:36 UTC (Sat)
by Wol (subscriber, #4433)
[Link]
Two cases in point - the entire diet industry is founded on a paper that concluded "fat is bad for you". That has now been thoroughly debunked. The alternatives pushed by the diet industry are far worse (they replaced fat with sugar, which has pretty much *caused* the current obesity, cancer and diabetes epidemics. Then there's the margarine and trans-fats which is probably behind all the heart disease).
And read Feynman, where he spent ages trying to work out why his sub-atomic-particle charges didn't add up. Then he thought "if the charge on this particle is wrong ... AHA - I remember seeing the proof and thinking it was screwy!!!". So he looked at it, and if he deleted just one outlier result, the proof flipped the charge! So we have 20 years of atomic research possibly derailed by just one paper where the author didn't understand statistics!
That's also why I hate seeing newspaper headlines "A new study has proved ..." - anybody who knows science knows that ALL new studies PROVE NOTHING. It's the REPEAT studies that prove the first one was correct! Unfortunately, it's very difficult to get funding to repeat a study ... and getting back to topic - it's getting better but how many studies cannot be repeated because the software required cannot be found or reliably recreated?
Cheers,
Posted Sep 1, 2024 11:58 UTC (Sun)
by ralfj (subscriber, #172874)
[Link] (7 responses)
Gotta love an argument by authority. Please bring actual data or logical arguments, not "some people with a title said so".
Also, speaking of professors, as a professor I happen to communicate with other professors regularly. Many of us think that Rust is a much better choice for a first language than C. C is a terrible choice because it's basically a big collection of foot-guns, and there's just not enough time in a semester to list them all.
But change is slow, so it'll take a while until CS departments pick up teaching Rust. Some already did, and more will follow I am sure.
Posted Sep 1, 2024 12:52 UTC (Sun)
by rc00 (guest, #164740)
[Link] (6 responses)
This is problematic. How do you define your mission statement? There are literally hundreds of job postings for Rust while there are thousands for Go, or C, or Java, etc. (Indeed claims 400 currently. LinkedIn lists 300. Actual data.) Also, these Rust jobs are mostly in Blockchain or Crypto. (I'm hard-pressed to find anyone worth their salt who doesn't believe these industries to be junk but would love to hear your thoughts.) And this is approaching nearly a decade of Rust being "the future." Is your goal to teach the undergrads Rust so they have no useful skills before sending them into the labor market?
Call it anecdotal data, but I can barely recall meeting any professor who thought that it was more important to focus on a specific greenfield language rather than to adequately structure the curriculum to best prepare the student body with useful skills and knowledge to enter the labor force and find immediate success. This is why programs partner with companies.
You could present some of the challenges with teaching C and say that other employable languages make more sense or offer more approachable teaching opportunities. How do you justify using an unemployable language for students without calling it a disservice?
P.S. Some small background information about myself. The reason I engage with professors from various local universities is not only for my own curiosity but because I regularly conduct speaking engagements with both undergraduate and graduate students. I've also taken on mentorship roles with quite a few over the years and to this day, these are some of my most cherished relationships. The only scope of bias is that these are all private universities and colleges. While I am curious about public universities and colleges, I don't have much in the way of experience or connections at that level.
Posted Sep 1, 2024 15:12 UTC (Sun)
by excors (subscriber, #95769)
[Link]
I'd hope their mission is to teach students the fundamentals that will make them effective at learning all the other technologies they'll need to use over their career, many of which haven't been invented yet, rather than training them in one specific language.
(The first language taught at my university was ML, specifically because it demonstrated the fundamentals of computation without all the distractions that C/Java/etc introduce, and because (being a practically useless language) it was unlikely any students had prior experience with it and everyone would start from a similar level. Then they taught maybe five other languages to varying extents, some more practical and some less.)
> There are literally hundreds of job postings for Rust while there are thousands for Go, or C, or Java, etc. (Indeed claims 400 currently. LinkedIn lists 300. Actual data.)
LinkedIn tells me there are 3000+ jobs for "Rust software engineer" in the US, so I'm not sure where you're getting 300 from. Certainly it's more niche than many other languages, but from an employability perspective it's not an extreme niche like e.g. Zig (4 jobs on LinkedIn).
For serious (non-blockchain) tech companies, Amazon alone has 173 job adverts that mention Rust - sometimes explicitly requiring Rust experience, though mostly asking for e.g. "Experience programming with at least one modern language such as Python, Ruby, Golang, Java, C++, C#, Rust". Knowing any of those will put you in a good position to learn the specific languages you need for the job. Microsoft has 97 mentioning Rust, e.g. "4+ years of experience with coding in one of C#, Python, Go, Rust, Java, C or C++", or "Knowledge/experience programming in RUST or other memory safe languages is highly desirable".
Google rarely mentions Rust in job adverts, but they've said over 1000 Google developers committed Rust code in 2022 and over 500 took a Google-developed training course. So I think that doesn't indicate they don't believe Rust experience is valuable, just that it's rare enough they don't bother calling it out in job ads, and they believe it's worthwhile to invest in training developers for their Rust-based projects, which could be seen as even stronger support than asking for prior experience.
Posted Sep 1, 2024 16:08 UTC (Sun)
by Cyberax (✭ supporter ✭, #52523)
[Link]
And yet there are even more jobs that require JavaScript. Let's rewrite the kernel as a NodeJS module.
Rust will never be as popular as Go or Java, simply because it's not the best language for regular server-side applications.
Posted Sep 1, 2024 16:18 UTC (Sun)
by Wol (subscriber, #4433)
[Link]
And this comment embodies everything I think is WRONG with the current education system. That attitude belongs with the UK polytechnics of old, not what Universities are SUPPOSED to be for. Not blaming Universities, it's the politicians who are desperate to see everybody educated, when maybe the majority of people are incapable of benefiting from education!
What the parent has just described is what I would describe as TRAINING, not Education, and that does not belong in a traditional University. That's what Polytechnics did - they prepared people for the world of work, they trained them, and that's fine.
Going back to a study I came across, apparently the ability to think *logically* is acquired at about age 14, and maybe half of all people never acquire the ability. Yet education - higher education certainly - is all about logical thought! So why on earth are the politicians (I think this is the UK Conservatives pre Tony Blair, so that dates it) desperate to send half the population to Uni? It's coming home to bite them now, as a lot of people see Uni as loading them up with debt to acquire worthless qualifications. And for so many, that actually is the case.
Uni should be for education. Uni should be for teaching people how to use their brains. To quote a lady from years back on Radio 4 - "I didn't send my son to school to do Domestic Science so he could cook. I sent him to school to learn Latin so he knew how to read a recipe book and follow instructions". Too many people think education is doing Domestic Sci. IT ISN'T. THAT'S TRAINING. Education is learning Latin and Maths, so you can get a career as a programmer (like I did).
Cheers,
Posted Sep 2, 2024 11:41 UTC (Mon)
by taladar (subscriber, #68407)
[Link]
Posted Sep 2, 2024 18:35 UTC (Mon)
by ralfj (subscriber, #172874)
[Link] (1 responses)
Thinking in terms of "(un)employable languages" is entirely the wrong way of thinking. Any CS graduate worth their salt is able to quickly pick up a new language and transfer the concepts they are familiar with -- that is what truly makes them employable.
Posted Sep 2, 2024 19:02 UTC (Mon)
by pbonzini (subscriber, #60935)
[Link]
If the language _forces_ the teachers to include the concepts of shared/exclusive reference or passing ownership, all the better, even if the students end up writing C and these three things all become a T*.
Posted Sep 2, 2024 9:47 UTC (Mon)
by nim-nim (subscriber, #34454)
[Link] (1 responses)
Go’s trump card is Kubernetes, a lot of software gravitates around containers, which means there are a lot of building blocks already written in Go people can reuse (there is a strong batteries included effect like for Python or Java, and understanding Go is necessary to fix or adapt existing deployed code).
Rust people do not seem to understand there is a point where you need to stabilise the API/ABI and commit to it so people start writing long-term apps and publish reusable components. You can only go so far in Gentoo mode.
Posted Sep 2, 2024 13:55 UTC (Mon)
by intelfx (subscriber, #130118)
[Link]
As if Go has a stable ABI or really anything besides "yolo static link the entire world"?
Posted Sep 2, 2024 21:45 UTC (Mon)
by anthm (guest, #173221)
[Link]
Even 15 years ago when I was in school it was all Java. C was kinda used in the OS class out of necessity, but that was the weird legacy case and it wasn't really taught as a language.
Posted Aug 29, 2024 21:43 UTC (Thu)
by SLi (subscriber, #53131)
[Link] (4 responses)
I had to look up brigading. I'm still not sure what it means in this context. The OPs message is certainly harsh in tone and a bit over the top, but it's hard to say it doesn't contain a level of truth to it. Being someone more of the old guard, I don't think it makes the message worthless or worse that I wouldn't have expressed it that strongly. On the contrary, it's good that grievances get aired, whether they are justified or not. Then we can discuss them and either decide that they are unfounded or that there's some truth to them. Both results are fundamentally positive.
Posted Aug 30, 2024 3:01 UTC (Fri)
by NYKevin (subscriber, #129325)
[Link] (1 responses)
TL;DR: It's sockpuppetry, except you get a bunch of real people to help out instead of doing it all yourself. Unfortunately, when you remove the defining property of sockpuppetry from sockpuppetry, it makes the whole thing really damn hard to pin down and quantify objectively.
Posted Aug 30, 2024 3:48 UTC (Fri)
by viro (subscriber, #7872)
[Link]
The thing is, most of the active developers have not been subscribed to l-k for decades now. I'm not sure what sustains the myth about LKML-as-a-place-where-development-happens, but it really, really had not been a function of that list for a very long time. _Some_ of us are still subscribed, but the volume is much too high for anything other than skimming the traffic. Reading every sodding message? Not realistic. It's easily over 1e3 posting/day. Often - twice as much.
Threads on other lists Cc'd over there for posterity? Sure. Pull requests Cc'd - ditto. Active development that is not just Cc'd over there (with most of the participants _not_ subscribed to that list) - not really.
25 years ago it was still used for development; 20 years ago - not so much; 15 years ago - not at all. Complaining there about anything may or may not be satisfying, but those complaints are not going to be read by the people not on the list. Which includes Linus, et cetera. The same goes for any kind of advocacy, pro or contra anything whatsoever - it's not going to be read by people who are not subscribed to the list it is posted on...
Posted Aug 30, 2024 18:37 UTC (Fri)
by jmalcolm (subscriber, #8876)
[Link] (1 responses)
Posted Aug 30, 2024 19:58 UTC (Fri)
by SLi (subscriber, #53131)
[Link]
But then I think you underestimate the current amount of discipline. Some of it is at the language level. A lot of it is at a more semantic level (when yo take locks, free resources, etc), which tend to be concerns where modern languages help a lot.
I mean, C clearly already goes some amount in this direction compared to previous languages and previous versions of C, for example in requiring casts instead of silently allowing you to mix types and get crashes. I don't think many serious C programmers would argue that that part of the language is not useful. It's just that some people have the idea that nothing useful for systems programming has happened in languages since then, which is clearly already disproven by the desire to use the GCC cleanup things (which don't work).
Posted Aug 29, 2024 19:54 UTC (Thu)
by shironeko (subscriber, #159952)
[Link]
Posted Aug 31, 2024 3:13 UTC (Sat)
by intelfx (subscriber, #130118)
[Link]
Posted Sep 2, 2024 17:16 UTC (Mon)
by anthm (guest, #173221)
[Link]
No one on the LKML is going to give any credence to things written by people who aren't doing any work.
But if you are doing the work and it's not getting in because other people are dragging their feet or making up excuses that don't pass technical muster then it's 100% right to start sending emails.
Posted Aug 29, 2024 20:49 UTC (Thu)
by atnot (subscriber, #124910)
[Link] (158 responses)
I think this really gets to the core of it. But it's not just about angry emails and it's not really about being resistant to change.
Refactors like Folios are a significantly more invasive and far reaching than anything Rust has proposed so far. It's certainly more invasive than the 500 line (including docs) patch wrapping struct device that took years to merge. Yet folios have not faced the same degree of reaction. Yes, sure there's been a bunch of grumbling about whether this is really necessary. But I've never heard a kernel developer complaining about the "folio religion wanting to make us all learn folios" or grandstanding about how they will valiantly refuse to use them, or even implying it might be some sort of shadowy woke agenda of big bookbinding.
The difference is not really the amount of scope here. It's that while folios may be a big change, Rust is a change that, if successful, would prove some of those world views that as you point out the kernel self-selects for wrong or at least put them into question.
I think there's a number of these like:
I'm sure you can think of more. Of course, the degree to which these beliefs are held in any one person will vary, and there are equally things that go the other way. But the general point is that views that extend far outside of the kernel and programming lead people to *really want* the project to either fail or succeed to prove or disprove those beliefs. Which makes things much harder than they need to be.
However I'm ultimately more positive about the outlook there. Between AGX and Nova, we're about to be in a situation where 2/4 desktop graphic card drivers will run on Rust. I think that, even with the resistance to Rust, the kernel will not be able to afford to keep these things out of tree forever if people keep maintaining them, just like PREEMT_RT. And just as Linux has selected for these things in the past, these things will change who it selects for in the future.
Posted Aug 29, 2024 21:28 UTC (Thu)
by corbet (editor, #1)
[Link] (150 responses)
The Rust work has gone slowly — as did folios, which took over a year of work and endless discussions before anything was merged. Rust is in many ways a harder problem; how do you integrate a new language and new ways of doing things into a 30-year-old project that you cannot afford to break, and which needs to plan for the next 30 years as well? Many of the questions around Rust come down to that. See, for example, this article on filesystem APIs.
There is not much opposition to Rust in the kernel community — far less than I had expected, honestly. There is opposition to merging Rust code just because it's Rust, without ensuring that it fits well into the kernel and that the result will be maintainable over the long term. And so the Rust developers have run into the kinds of difficulties that many developers trying to make big changes have hit. It's hard, often harder than it really should be, but I do not believe it is driven by the factors you have described.
Posted Aug 29, 2024 21:47 UTC (Thu)
by SLi (subscriber, #53131)
[Link] (20 responses)
Posted Aug 30, 2024 7:31 UTC (Fri)
by vasvir (subscriber, #92389)
[Link] (19 responses)
However in the long run I believe it was the right decision.
C++ brings a lot in the table but it still requires a huge amount of discipline. In every C++ project I have seen a C++ subset is deemed as blessed but that also requires discipline to enforce. Tests can help but in reality are also a discipline enforcement mechanism beyond the compiler.
If Linux had gone the C++ way it would face now the possibility of adding a 3rd language or a double migration to Rust. The premise is that with Rust discipline is not required, the compiler will flag out incorrect usage of APIs etc. There are big ifs here like:
* Are the rules you want to be disciplined about possible to be encoded in Rust?
I believe this exercise of integrating Rust to the Linux will answer these questions.
Disclaimer: I do not know Rust
Posted Aug 30, 2024 10:53 UTC (Fri)
by khim (subscriber, #9252)
[Link] (2 responses)
No, it wouldn't. Because these specific questions can be answered easily and in negative fashion and that is what Ted's possibly-sabotage is using as base. But you, presumably, know some C and some assember, right? And you know that Unix was written in assembler and then rewritten in C, right? Well… assembler makes it possible to do amazing tricks, not really replicable in C. E.g. GEOS used very peculiar yet efficient way of passing arguments: just put them right after It's very efficient: it's very easy to traverse the call stack with such approach and you don't even need to move arguments around! You just know that if function X have arguments If your drivers ABI is designed around such “extra-efficient” calling conventions then rewrite in C is, essentially, not possible. Or at least it would be quite inefficient. You may say that your assembler is just fine and these new guys can go to hell with their C… or you may actually cooperate and design some solution. Develop some secondary set of functions “for these C guys”. Or ask them to develop them… but then you need to cooperate with them when you change you API. Transition to Rust is very similar. You may ask Rust guys to develop API, sure, and they may keep it in such with C API, but… Doing what Ted is doing… to put it extra-polite as one redditer put it: that's behavior externally undistinguishable from purposeful sabotage. As in: while different explanations are possible but any guy hell-bent on sabotaging Rust-on-Linux effort and who is maintainer of some subsystem would find it logical to do precisely what Ted did.
Posted Aug 30, 2024 11:37 UTC (Fri)
by vasvir (subscriber, #92389)
[Link] (1 responses)
I am sure there are more ifs than the one I listed here. I mean it's a very big C project that tries to incorporate Rust. Lots of lessons will be drawn for all people that are unbiased. I agree though that the C side will have to make changes to accommodate Rust API. The question is: Will these changes make for a better C API? or more enforceable even with an external tool? I would suggest to grab some pop corn...
> But you, presumably, know some C and some assember, right? And you know that Unix was written in assembler and then rewritten in C, right?
Yes, yes and yes.
> It's very efficient: it's very easy to traverse the call stack with such approach and you don't even need to move arguments around!
Crazy stuff!!! and *no* I have never done that or even thought to do something like that. Didn't know about GEOS. Thanks for the pointer.
I saw the video and I would agree that Ted looked to me very rude. However, LWN coverage was more mild and his arguments came across as logical points which require some thought, work andcollaboration to address them.
Posted Aug 30, 2024 12:02 UTC (Fri)
by khim (subscriber, #9252)
[Link]
90% of time the answer is “yes”: C API have grown to be incompatible with Rust approach simply because no one cared about Rust and when you fix it you make it easier to use from C, too. 90% of the remaining 10% you can approach some compromise where Rust API doesn't look too awful and C API is not too crazy, either. But there are always 1-2% of these crazy corner cases where sane Rust API just pushes you to rewrite large pieces of already working code and then you either have to rewrite C code that is very fragile and rewrite-hostile, or, more often, add another implementation of the same API just for Rust. It's always a good idea to try to avoid that, but sometimes that's really the best approach. And if that happens then maintainership of that functionality becomes tangled between many different parties. And you spend way more than 1-2% of your time dealing with these corner cases.
Posted Sep 3, 2024 3:36 UTC (Tue)
by anthm (guest, #173221)
[Link] (15 responses)
If the cognitive load and discipline required for you to write C is lower than Rust I would question how correct your C code is. Rust requires you to work with a more difficult language but it gives you guarantees, C requires managing all those guarantees manually. Correct C code is extremely hard to write.
This is why I *hate* the "C is simple so it's good" argument. C is not simple. C just hides the complexity behind a language that's under-engineered.
Posted Sep 3, 2024 4:27 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link]
C++ has a huge number of features. So if you're using a subset of features, there's always pressure to use just one more feature just outside of the subset. U
Posted Sep 3, 2024 7:34 UTC (Tue)
by viro (subscriber, #7872)
[Link] (10 responses)
Don't get me wrong - memory safety issues do happen and automated tools are useful; what's more, C type system is genuinely not strong. But that's only a part of the picture. Far more often is the case when you need to change existing code (as opposed to "write a replacement from scratch and pray that CI will catch any logical problems") and _that_ is where the things get painful. Again, being able to reason is an absolute must-have; you _can't_ just treat the compiler as an oracle that needs to be appeased and be done with that. In any language.
And frankly, I don't understand the idio^Wpeople who treat language as identity - _any_ language (natural ones included) is a set of tools. It does not determine how you think, for fuck sake! You really can write FORTRAN in any language - certainly so in Rust. Familiarity with more languages is a good thing - it widens your toolkit, and if you really understand a language you will be able to come up with equivalent idioms in another - provided that you *do* understand them in the original, and not just on the level of "it's a magical incantation". Whorf-Sapir hypothesis is garbage; mixing it with Herderian nonsense is completely insane. "Rust Culture", "C Culture"... get a life, already.
Posted Sep 3, 2024 12:39 UTC (Tue)
by Wol (subscriber, #4433)
[Link]
rofl ...
Most of my C code was probably just FORTRAN and DataBASIC in disguise ... for the most part I just didn't see any advantage in using unfamiliar C idioms when Fortran did just as well - eg malloc a chunk of memory and access it with [] rather than *. If the C idiom was better than Fortran then I learnt it and used it, if I couldn't see an advantage I stuck with what I knew.
Cheers,
Posted Sep 3, 2024 12:46 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (7 responses)
Well, how do you think about tense, if your language does not have tenses? Given that you NEED language to enable you to think, the two are completely intertwined.
Your comment about "learning multiple languages" underlines this point! By learning another language, you learn that other people think different to you, and it broadens your mind.
But it's pretty obvious from - let's say khim's posts - that his language determines the way he thinks. He can't even express himself in English some times because his language does not have the context to enable him to grasp ours! I get the impression that many of my spats with him are down to the fact that the GRAMMAR gets in the way and obstructs mutual understanding.
Sorry khim - no disrespect - I'm sure you'd say pretty much the same about me from the other side of the fence :-)
But sorry, yes, language DOES determine how you think. If you don't have the words/grammar to say it, then you don't have the words/grammar to think it.
Cheers,
Posted Sep 3, 2024 15:49 UTC (Tue)
by viro (subscriber, #7872)
[Link] (6 responses)
As for the words... that's why we have mechanisms to introduce new words and expressions. There's a particularly debased form of Whorf-Sapir bollocks (to be fair, neither had been responsible for that bit of BS - that came from popularizers) that seriously claims that inhabitants of Caribbean in 1490s must've been unable to process what they were seeing and somehow developed a blind spot in place of caravels. I don't know how one could believe that, because we all know what really happens when there's no terms for what we are seeing. "Oi, what the fuck is that thing over there? Hadn't been there this morning... Looks like there's a bunch of wankers on it; are they wrapped in some strange shit? Seems rigid; some kinda shields strapped on, maybe? Don't like how that one is pointing our way whatever the hell he's holding, let's get out of the open and warn the blokes back in the village" is far more likely than ignoring the sight just because there's no proper term for caravel, cuirass, etc. More likely to get passed to offspring, anyway... Mind does not stop when running into the lack of proper terms; it cobbles something up (that's what the obscenity is for, often enough) and keeps going.
Posted Sep 3, 2024 16:44 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (5 responses)
Then why is it so hard to understand what khim is actually saying? I learnt the rules, I (think I) have a basic grasp of Russian grammar, I try to see it through his comprehension, and he STILL succeeds in saying something that is clearly not what he meant.
It's like double negatives. I know we Brits abuse it a lot, but a lot of other languages use an emphasised negative. Translate that naively into English, and you've just COMPLETELY changed the meaning. And it's hard to unlearn something. I can hear you perfectly, and still walk away with a completely wrong understanding of what you said.
Surface representation matters a LOT. It's behind many misunderstandings and wars.
Roosevelt: Let's table that idea.
And that's only the difference between English and American ...
Cheers,
Posted Sep 3, 2024 18:23 UTC (Tue)
by viro (subscriber, #7872)
[Link]
Posted Sep 3, 2024 20:57 UTC (Tue)
by rschroev (subscriber, #4164)
[Link]
> Then why is it so hard to understand what khim is actually saying?
Because English is a foreign language for khim? Of course it's more difficult to express yourself clearly (and/or succinctly) in a language that's not your native language. I can express my thoughts pretty fluently in Dutch; English is significantly harder, even though both languages are pretty closely related. French is almost hopeless even though our education system tried pretty hard making us learn that language. German is closer related still, but while I can read it a bit I can't write or speak it at all (other than trivial things like "Zwei Bier bitte"). If I try to express something in French, chances are French-speaking people will think it's difficult to understand too. And it's more difficult for things that are expressed differently in native versus foreign language.
How much study of Russian would you or I need before we are as proficient in it as khim is in English? Will khim wonder why he can't always understand our reasoning expressed in Russian?
I think you're thinking of the linguistic relativity, also known as the Sapir–Whorf hypothesis (and other terms), and more specifically it seems you're thinking of the strong form which says that language determines thought and that linguistic categories limit and restrict cognitive categories. Modern linguists don't accept that hypothesis anymore (or at least that stronger form).
Posted Sep 3, 2024 21:04 UTC (Tue)
by rschroev (subscriber, #4164)
[Link] (2 responses)
You can express repetition in different programming languages in different ways.
What you're saying is: The representation matters, because different compilers/interpreters/programmers will understand different ways of representation.
What viro was saying (or at the very least, how I interpreted it): The representation doesn't matter, in the sense that any programming language can express repetition, and the choice is a simple consequence of the choice of programming language. It doesn't influence what concepts can be expressed in the different languages.
Posted Sep 5, 2024 12:01 UTC (Thu)
by taladar (subscriber, #68407)
[Link] (1 responses)
Posted Nov 28, 2024 18:32 UTC (Thu)
by deepfire (guest, #26138)
[Link]
Posted Sep 3, 2024 18:53 UTC (Tue)
by SLi (subscriber, #53131)
[Link]
A serious question. Do you think you could reason equally well about high level behavior in C and assembly? Even if we ignore the memory safety aspect.
I find this idea surprising. In my experience, the language you work in significantly shapes your thinking.
Posted Sep 3, 2024 8:25 UTC (Tue)
by SLi (subscriber, #53131)
[Link] (2 responses)
And because you are unlikely to be tempted to write complex template machinery that nobody will ever quite understand, for the sinple reason that the language does not support anything that complex. That certainly also means you won't get the major benefits of templates. You can write macro hacks that are much more horrible, but there's no way to make them as insanely complex as modern C++ is (and I say this as someone who both loves and hates the language for that).
Posted Sep 5, 2024 12:04 UTC (Thu)
by taladar (subscriber, #68407)
[Link] (1 responses)
Posted Sep 5, 2024 16:04 UTC (Thu)
by SLi (subscriber, #53131)
[Link]
Posted Aug 29, 2024 23:05 UTC (Thu)
by gerdesj (subscriber, #5446)
[Link] (1 responses)
Now, that was a project and a half. Conducted when Mr Shouty was at peak shouty. Mind you, LWN discussions were a lot more civil or perhaps my specs have gone a bit rose tinted.
Posted Sep 11, 2024 1:05 UTC (Wed)
by sammythesnake (guest, #17693)
[Link]
- Mark Twain*
* Not Mark Twain - I just made it up
Posted Aug 30, 2024 5:10 UTC (Fri)
by pbonzini (subscriber, #60935)
[Link] (50 responses)
Looking at the linked article and discussion, I see objections of this kind and "C-side" developers participating into constructive discussions; but "because I don't know Rust, I am not going to fix the Rust bindings" is not one of them.
The moment rust/ was added to the Linux tree, anyone would have reasonably guessed that sooner or later all core kernel developers would have to either learn Rust, or collaborate with Rust for Linux developers whenever a tree-wide refactoring would happen.
The process of incorporating Rust bindings has been slow and sometimes tricky for several (mostly good) reasons, and when two teams with slightly different habits cross some attrition is expected (https://lwn.net/Articles/949270/). However, in the long term there is no sensible outcome other than people learning Rust, and in the long term people should realize that the ship has already sailed.
Posted Aug 30, 2024 11:50 UTC (Fri)
by jgg (subscriber, #55211)
[Link] (49 responses)
I think there is a sizeable contingent that does not believe that to be the case, and Ted's remarks of effectively wanting nothing to do with rust are not unique or unreasonable given the cloudy situation. I know enough people betting on the failure of this experiment.
IMHO the current situation of Rust does not look like success. It is basically unusable except for unmerged toy projects and it is still not obvious when that will change. Can we rely on RH10 kernel having full baseline rust support? If yes then in 4 years maybe the server/enterprise industry could actually we can take rust seriously.. Can rust even support the OOT backporting stuff everyone does? Is there a strong enough preprocessor? What is the situation with Android? Can an Android device vendor write a driver in rust today?
I've pondered if I should consider rust for some of the greefield projects I've done recently. iommufd in rust? Generic PT? But at the end of the day that work is being "paid" for by people who intend to backport it to old kernels. I can't write it in rust and meet that need. I bet a lot of people are in the same boat.
Posted Aug 30, 2024 13:29 UTC (Fri)
by pbonzini (subscriber, #60935)
[Link]
With a killer use case there's no doubt to me that RHEL 10 would be able to support Rust code. Right now as you point out there's none (the only non-toy one in the air is binder), but RHEL 10 will grow new functionality until 2028 so I wouldn't exclude that, especially for drm.
Posted Aug 30, 2024 15:21 UTC (Fri)
by willy (subscriber, #9762)
[Link] (32 responses)
Obviously, this is not a failure of the RustForLinux project. They have important work to do which is independent of code generation.
My biggest gripe is inline functions. It seems crazy to me that we have functions which are so performance critical that they must be inlined in C, and yet in Rust, we make function calls to an exported symbol.
And I haven't got very far through my Rust book. I did write a little userspace program (calculating Pascal's triangle) which went well enough, but most of what I was doing there was figuring out how to use the libraries rather than the kinds of things I'll need to do in the kernel.
And, yeah, I think what Ted did there was reprehensible. I'd've said so had I been in the room, but I was off in the MM track at the time.
Posted Aug 30, 2024 15:47 UTC (Fri)
by jgg (subscriber, #55211)
[Link] (22 responses)
As above, I think the Linux project needs to make a clear unambigous decision, or at least set a timeline to make it.
Either we are doing Rust and what Ted said is out of line, or we are not, and Ted's position is reasonable - do not burden busy people with a Rust experiment that is nobody is going to use.
I thought the point of this inbetween state was supposed to be doing experiments to decide if Rust is feasible. Is there a result yet? Your concern about inlines is news to me, that sounds kind of fatal honestly. What was the result of the nvme experiment? I saw some graphs that it was worse performance, that's not encouraging?? How much of the toolchain situation is sorted out? Last LPC they were saying constant upgrades required? GCC was working on a front end?
Posted Aug 30, 2024 17:55 UTC (Fri)
by Wol (subscriber, #4433)
[Link] (5 responses)
I was under the impression that - compiling huge monolithic blobs - Rust was quite capable of spotting and inlining functions like that all by itself. What's the point of an "inline" keyword if you can rely on the compiler to spot it?
(Dunno whether I like the downsides of huge blobs - it makes libraries more complicated - but it's horses for courses, or pick you poison, whichever suits ...)
Cheers,
Posted Aug 30, 2024 20:12 UTC (Fri)
by pbonzini (subscriber, #60935)
[Link] (4 responses)
Posted Aug 30, 2024 22:55 UTC (Fri)
by roc (subscriber, #30627)
[Link] (2 responses)
Posted Aug 31, 2024 5:38 UTC (Sat)
by pbonzini (subscriber, #60935)
[Link] (1 responses)
Posted Aug 31, 2024 6:16 UTC (Sat)
by roc (subscriber, #30627)
[Link]
Posted Aug 31, 2024 18:02 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Aug 30, 2024 22:46 UTC (Fri)
by kees (subscriber, #27264)
[Link] (15 responses)
Yes, there continues to be technical issues to be worked out. This is no different from C. It's the nature of technology. We're always moving to new compiler versions, refactoring to get rid of bad APIs, etc.
And Rust has shown its strengths very well, IMO. The M1 GPU driver is excellent and in constant use by a large set of distro users. The Binder driver is a drop-in replacement for the C version and will be shipped in Android soon.
I think the characterization of these Rust drivers being "toy projects" is pretty wildly inaccurate, if not outright insulting.
It's just a big change that is taking time to get everything resolved. I don't understand the resistance to learning new languages, especially given the large developer community associated with Rust. Us kernel devs are always hoping to get more people involved in Linux... Why push people away like this, especially given how many problems Rust permanently solves?
But to answer your questions:
Yes, Rust is feasible.
The NVMe driver works fine and the interfaces needed for it are going through review, though it's an uphill battle politically: as you said there are a few very loud voices in FS/Block that seem to be overwhelmed by their work loads.
The NVMe performance graphs looked very encouraging to me! A proof-of-concept driver with no explicit optimizations is identical to the C performance, except in 2 cases: one where it's within roughly 2%, and in another was _faster_.
What do you consider "the toolchain situation"? It works fine for me. :)
Yes, the upgrade cycle is faster than for GCC and Clang currently, but this is already slowing as the language features needed for Linux are stabilizing, so now there is a minimum version needed, not an exact version.
Yes, GCC has, I think, 2 front-end projects for Rust. I haven't paid too much attention to this myself, though.
Posted Aug 30, 2024 22:53 UTC (Fri)
by corbet (editor, #1)
[Link] (12 responses)
Posted Aug 31, 2024 18:43 UTC (Sat)
by josh (subscriber, #17465)
[Link]
rustc_codegen_gcc is making steady progress.
(That's leaving aside the question of how much value those targets provide.)
Posted Sep 1, 2024 11:38 UTC (Sun)
by ralfj (subscriber, #172874)
[Link] (10 responses)
gccrs is attempting an entirely independent second Rust implementation -- that's orders of magnitude more work than rustc_codegen_gcc, and I don't think there are significant benefits that would justify the cost. (This is not to say the gccrs devs should stop, if they're having fun doing what they do then by all means continue, but in terms of where to allocate resources and where to watch for medium-term results, I think rustc_codegen_gcc is clearly the better choice.)
rustc_codegen_gcc is unfortunately held back by GCC's reluctance to provide a nice library API for accessing its backends, but it seems using the libgccjit library works reasonably well.
Posted Sep 2, 2024 15:12 UTC (Mon)
by jjs (guest, #10315)
[Link] (9 responses)
Law dictionaries exist to help ensure legal language is precise & unambiguous. There's a reason IETF requires two, independent implementations before declaring something a Internet Standard - https://www.ietf.org/participate/runningcode/implementati.... If two implementations don't produce the same product, it's time to go back and fine tune the specification to clarify the ambiguities that arise. And the only way to check for ambiguities is via an independent implementation.
Posted Sep 2, 2024 18:43 UTC (Mon)
by ralfj (subscriber, #172874)
[Link] (8 responses)
It may do that. Or it may cause endless issues due to differences in behavior between implementations, as is the case in C. One reason why the standard leaves so many things as "Undefined Behavior" is that implementations happened to implement different behavior, and none of them wanted to change. It's easy for them to agree to make things UB, the consequences are beard by programmers... just look at the entire debacle with realloc-of-size-0 now being UB: https://queue.acm.org/detail.cfm?id=3588242
I don't deny that multiple independent implementations have advantages. But they also have serious disadvantages. And given the resources required to build and maintain them, I am not convinced that it's worth it overall. The fact that language implementations are typically open-source these days has removed one of the biggest arguments in favor of multiple implementations.
Posted Sep 2, 2024 22:09 UTC (Mon)
by jjs (guest, #10315)
[Link] (2 responses)
In either case, the spec is changed. I suppose a 3rd way is to ignore the problem, but, IMO that's worse.
"The fact that language implementations are typically open-source these days has removed one of the biggest arguments in favor of multiple implementations."
I'll argue the opposite - it's the language implementations being open source is one of the biggest arguments in favor of multiple implementations. Look at what went on with Linux and GCC/LLVM as LLVM began to work to compile the kernel. More defined behavior, from what I can tell. And a huge advantage of open source is everyone can contribute.
Posted Sep 5, 2024 11:40 UTC (Thu)
by taladar (subscriber, #68407)
[Link]
Posted Sep 5, 2024 14:28 UTC (Thu)
by ralfj (subscriber, #172874)
[Link]
That's not what happened here. In this case, the C standard was unambiguous since at least C89: "If size is zero and ptr is not a null pointer, the object it points to is freed". Some implementations violated the standard, and somehow it was deemed better to introduce UB into tons of existing code than to fix the buggy implementations.
Such a hypothetical case could of course happen, though. IMO in that case you have a buggy (unintentionally underdefined) standard -- which happens and which needs to be dealt with reasonably well. If you have multiple different implementations of the standard, they are very hard to fix (other than by making the standard so weak that it encompasses all implementations), and that explains some (but not all) of the oddities in C. If you only have a single implementation, it is a lot easier to fix such bugs in the standard/specification by adjusting either the spec (to still have a *defined* behavior! just maybe not the one that we'd ideally have liked to see) or the implementation. These kinds of things happen in Rust fairly regularly. A big part of what makes this possible is that we have the ability to add "future compatibility" lints to Rust so that there's many months or even years of advance notice to all code that might be affected by a compiler change. I worry that with multiple implementations, this kind of language evolution will become even harder than it already is due to the added friction of having to coordinate this across implementations.
Posted Sep 2, 2024 22:40 UTC (Mon)
by viro (subscriber, #7872)
[Link] (4 responses)
Posted Sep 5, 2024 11:41 UTC (Thu)
by taladar (subscriber, #68407)
[Link] (3 responses)
Posted Sep 7, 2024 17:10 UTC (Sat)
by jjs (guest, #10315)
[Link] (2 responses)
Posted Sep 7, 2024 17:40 UTC (Sat)
by intelfx (subscriber, #130118)
[Link]
Posted Sep 8, 2024 12:02 UTC (Sun)
by farnz (subscriber, #17727)
[Link]
That's where two implementations of the test suite comes in handy, since you now have two separate groups of people who've read the specification and agree on what it means; where one test suite fails and the other passes, you need to resolve that by either fixing the specification, or getting the passing test suite to agree that they had a gap in test coverage.
Posted Aug 31, 2024 2:03 UTC (Sat)
by sam_c (subscriber, #139836)
[Link] (1 responses)
I think faster is being generous. Nightly crates are still being used and the minimum Rust version is still extremely recent and being cranked up regularly. I think it's inevitable that Rust will be used in the kernel, but I find it hard to accept it's mature enough to merit being there when so many unstable features are needed.
Rust also doesn't, AFAIK, have any LTS versions for its compiler.
Posted Sep 2, 2024 11:55 UTC (Mon)
by taladar (subscriber, #68407)
[Link]
Posted Aug 30, 2024 23:05 UTC (Fri)
by roc (subscriber, #30627)
[Link] (7 responses)
Museum architectures should use museum kernels. It would be madness to let a few hobbyists veto kernel improvements that would benefit all other users.
But also, Rust does support m68k: https://doc.rust-lang.org/rustc/platform-support/m68k-unk...
Posted Aug 31, 2024 12:01 UTC (Sat)
by pizza (subscriber, #46)
[Link] (5 responses)
Except for the little detail that "museum architectures" (and the long tail of old drivers/filesystems/etc) are part of the mainline kernel.
Where do you draw the popularity line? Currently it's at "someone is actively maintaining it."
Posted Aug 31, 2024 21:25 UTC (Sat)
by roc (subscriber, #30627)
[Link] (4 responses)
If new hardware hasn't been sold for 20 years then I think that's probably a good enough line.
Posted Sep 1, 2024 18:01 UTC (Sun)
by willy (subscriber, #9762)
[Link] (3 responses)
https://www.nxp.com/products/processors-and-microcontroll...
I'm disappointed, mostly because I worked on a PowerQUICC board back in 2000 and the fact that they are still selling the 68360 24 years later makes me very sad.
Posted Sep 2, 2024 7:25 UTC (Mon)
by roc (subscriber, #30627)
[Link] (2 responses)
Posted Sep 2, 2024 16:03 UTC (Mon)
by Wol (subscriber, #4433)
[Link] (1 responses)
Cheers,
Posted Sep 2, 2024 19:12 UTC (Mon)
by pbonzini (subscriber, #60935)
[Link]
Posted Sep 1, 2024 11:40 UTC (Sun)
by ralfj (subscriber, #172874)
[Link]
Or, (in my view) more likely, rustc_codegen_gcc. :)
Posted Aug 31, 2024 18:40 UTC (Sat)
by josh (subscriber, #17465)
[Link]
https://doc.rust-lang.org/nightly/rustc/platform-support/...
It's still tier 3, but it exists.
As for alpha, if people still want to keep it alive, rustc_codegen_gcc will handle that eventually.
Posted Aug 30, 2024 21:09 UTC (Fri)
by asahilina (subscriber, #166071)
[Link] (14 responses)
I guess my Apple AGX GPU driver, which is the kernel side to the world's first and only OpenGL and Vulkan certified conformant driver for Apple Silicon GPUs, and also the FOSS community's first fully reverse engineered driver to achieve OpenGL 4.6 conformance, and which is used by thousands of Asahi Linux users in production, and that literally has never had a reported oops bug in production systems not caused by shared C code (unlike basically every other Linux GPU driver), is "an unmerged toy project".
Since you work for Nvidia, I'm sure you've heard of Nova, the up-and-coming Nouveau replacement driver that is also written in Rust using my Rust DRM abstractions. Is that also going to be "an unmeged toy project"?
This kind of demeaning of our work is why us Rust developers are getting very, very tired of the kernel community.
Posted Aug 30, 2024 22:41 UTC (Fri)
by jgg (subscriber, #55211)
[Link] (13 responses)
However, that doesn't change today's facts - AGX is currently unmerged and serves a tiny and niche user base with no commercial relavance. That is an unmerged toy by my definition.
There is nothing wrong at all with working on toy software. Linux itself started out as a toy, this is not an attempt to be demeaning.
The point, as pbonzini elaborated on, is a lack of "killer use case" to motivate RH to seriously turn on kernel Rust in RHEL10. AGX will not alter RH's plans.
Nova is barely started, let's wait a few years to see what impact it has. I'm optimistic that a completed Nova would convince several distros to turn on kernel Rust support. I was actually thinking primarily about the Rust NVMe driver.
Posted Aug 30, 2024 22:48 UTC (Fri)
by corbet (editor, #1)
[Link] (3 responses)
The story of why it is unmerged is something I've never quite managed to dig into, but would like to.
Posted Aug 31, 2024 20:32 UTC (Sat)
by jgg (subscriber, #55211)
[Link] (2 responses)
Posted Sep 2, 2024 7:34 UTC (Mon)
by roc (subscriber, #30627)
[Link] (1 responses)
Posted Sep 6, 2024 7:36 UTC (Fri)
by da4089 (subscriber, #1195)
[Link]
"Heritage" is the usual respectful euphemism, I think?
Posted Aug 30, 2024 22:49 UTC (Fri)
by Ashton (guest, #158330)
[Link]
“Toy project”? Can you please try and not be so petty and rude?
Posted Aug 30, 2024 23:56 UTC (Fri)
by airlied (subscriber, #9104)
[Link] (2 responses)
Like do we consider the open-gpu-kernel driver from NVIDIA a toy because it isn't upstream?
Asahi is not a driver on the enterprise radar, it won't make RHEL sit up and notice but in does that make it a toy.
I count asahi as a very successful fact finding mission, that in the end is very hard to upstream in a reasonable manner. It's why nova is approaching this from the other end, and building a driver upstream, where we fix the interactions with other subsystems in order as we go, building the ecosystem upstream rather than having it done in a private fork.
The main current focus is driver model and Greg at the moment, next after that will probbaly be getting pci, platform and drm device bindings into shape, KMS modesetting (which Asahi didn't have to tackle), and then the actual nova project.
I've already written a rust implementation that talks to NVIDIA's GSP firmwares and encapsulate the unstable ABI in a similiar form to the Asahi work, and the advantages of this over a C project to do the same are immense. Like night and day difference in how much code had to be written.
I think Linus has said at last year maintainers summit that he was supportive of this, and he thinks it will happen, I think if people start acting as active roadblocks to work, rather than sideline commentators who we can ignore, then I will ask Linus to step in and remove roadblocks, but so far we haven't faced actual problems that education and patience can't solve.
Posted Aug 31, 2024 9:21 UTC (Sat)
by Wol (subscriber, #4433)
[Link] (1 responses)
Given that is at least one (Christoff Hellwig) big developer (you make it sound like two - Greg KH) is throwing a lot of effort into cleaning up the kernel, sounds like they need to be brought on-side if they aren't already.
I'm picking up a lot about Rust-kernel bindings, and how the end result is much cleaner for both the C and Rust sides. So even if the resulting Rust work isn't merged, actually the effort spent creating the Rust interface would be a great help to both of them.
So you now have clean Rust interfaces for anybody who is interested ... and any "I'm not learning Rust" developers who tamper with those interfaces will get shouted at "don't you dare create any (C) bugs that this would have automatically caught!"
Cheers,
Posted Sep 6, 2024 11:46 UTC (Fri)
by jjs (guest, #10315)
[Link]
Not familiar with Rust, but it sounds like the efforts are having good benefits even without Rust being fully integrated.
Posted Aug 31, 2024 3:30 UTC (Sat)
by asahilina (subscriber, #166071)
[Link] (4 responses)
You are aware that the AGX driver is in fact the reason why Fedora is turning on Rust support in upstream kernels, right? I'm pretty sure that is doing more to push RHEL to eventually do the same than anything else, today.
https://gitlab.com/cki-project/kernel-ark/-/merge_request...
Neal is part of the Fedora Asahi SIG.
(Won't comment on your insistence on the "toy" designation since other replies have already done so.)
Posted Aug 31, 2024 7:38 UTC (Sat)
by airlied (subscriber, #9104)
[Link] (3 responses)
Don't confuse Fedora, CentOS or ARK with Red Hat here. RHEL is the boss level here, but I don't really care about that, I only care about getting things upstream lined up.
I think a lot of the complaints about rust will evolve away once there is an interesting in-tree consumer, toolchain versions will stabilise, better toolchain support for things upstream needs etc
Posted Aug 31, 2024 11:11 UTC (Sat)
by Conan_Kudo (subscriber, #103240)
[Link] (2 responses)
No. Rust is getting turned on because drm_panic is written in Rust. I started working on this for enablement because of AGX, but we need it turned on ASAP because drm_panic is approved for Fedora Linux 42. Nova is on literally nobody's radar right now because it doesn't exist beyond scaffold. There is no code that does anything yet, to the best of my knowledge, and there will not be any for a long while.
Posted Aug 31, 2024 12:10 UTC (Sat)
by airlied (subscriber, #9104)
[Link] (1 responses)
Posted Aug 31, 2024 12:31 UTC (Sat)
by Conan_Kudo (subscriber, #103240)
[Link]
Posted Aug 30, 2024 10:55 UTC (Fri)
by Deleted user 129183 (guest, #129183)
[Link] (6 responses)
Which is completely opposite to the predominating attitude of Rustbros, who think that their code is already superior just because it is In Rust. But no, it still has to be up to the standards, regardless of the programming language used.
Posted Aug 30, 2024 13:37 UTC (Fri)
by corbet (editor, #1)
[Link] (4 responses)
Posted Aug 30, 2024 19:58 UTC (Fri)
by Deleted user 129183 (guest, #129183)
[Link] (3 responses)
So those people with the expectation of ‘merging Rust code just because it's Rust, without ensuring that it fits well into the kernel’ are outside of the ‘kernel community’? Sorry, I’m confused now. But anyway, even if there are no such people now, they will definitely come later and make a mess. Because the Rust culture is broken, for example few days ago, we’ve had a guy in the LWN comments calling another person a ‘bigot’ and ‘Nazi’ just because the latter person wanted to see less Rust content here. And this is not a sign of a healthy attitude inside the Rust community.
Posted Aug 30, 2024 20:38 UTC (Fri)
by viro (subscriber, #7872)
[Link]
Posted Aug 30, 2024 21:18 UTC (Fri)
by pbonzini (subscriber, #60935)
[Link]
I am not sure why you'd think that he's a member of the Rust community as opposed to a bystander. And the attitude of Rust-for-Linux developers has always been all but professional and respectful. They absolutely don't want their code merged just because it's Rust, though of course they believe the language to be superior in certain ways, especially in how it encodes safe semantics as part of the types.
Posted Aug 30, 2024 22:36 UTC (Fri)
by intelfx (subscriber, #130118)
[Link]
No, it was not not because "he wanted to see less Rust content here", but because of his shitty and entitled attitude towards LWN editors and audience. And I'm fairly sure you do understand that.
Posted Aug 30, 2024 16:53 UTC (Fri)
by viro (subscriber, #7872)
[Link]
Posted Aug 30, 2024 12:21 UTC (Fri)
by b7j0c (guest, #27559)
[Link] (63 responses)
cavemen make the world go round
nothing Ted said in the related video snippet is remotely problematic; he doesn't raise his voice, he doesn't insult anyone...he just states his position directly.
Posted Aug 30, 2024 13:22 UTC (Fri)
by mb (subscriber, #50428)
[Link] (62 responses)
And that does not surprise me at all.
He doesn't think it's a big problem that crafted filesystem images can cause Undefined Behavior via out of bounds accesses in the kernel.
Well. We have tried that for *decades*. And we all failed miserably at it.
The biggest step with accepting a transition to Rust is accepting the failure of C first.
It's natural that people defend things they had been doing for decades.
And besides that, stating that "you will not force me to learn Rust" in a talk about Rust *is* insulting. Nobody is forcing anybody.
Refactoring of such cross language interfaces is no different than refactoring cross subsystem interfaces.
There will always be situations where a big refactoring hits areas unknown to the developer doing it.
I'm sure the Rust people would not refuse to work together with Ted to find a good solution.
Posted Aug 30, 2024 20:57 UTC (Fri)
by jgg (subscriber, #55211)
[Link] (60 responses)
The logical end game here is in 10 years the kernel will have a mixture of Rust and C. If you want to be a senior kernel contributor you will need to be a master of both. The idea you can work on the core kernel and be unable to touch large swaths of it is not realistic. People need to be honest about this! The idea that there will always be rust helping hands willing to do whatever the historical C maintainer wants is not reasonable, that might exist for a few years but not over the long haul.
Posted Aug 30, 2024 21:12 UTC (Fri)
by johill (subscriber, #25196)
[Link] (59 responses)
I for one would really like to learn and use rust e.g. for cfg80211's scan handling. That code is awfully complex, with all the lists for hidden, multi-bssid, etc. cases, and I'd think it might be possible to express certain restrictions etc. in rust to make working on it safer.
That also means rust currently remains restricted to something that need not work on all platforms, like certain (sometimes duplicate?) drivers, or drivers for special hardware that's only available on some platforms (like the graphics driver people keep talking about).
So I'm probably not going to learn it any time soon unless I change jobs, because realistically there isn't much _else_ I (want to) work on that would warrant learning rust, and I don't have time to RIIR something.
So while I don't necessarily think Ted is right about just breaking rust stuff left and right, there currently isn't even a large opening where as a kernel developer concerned with pretty much _all_ architectures (ok s390 doesn't have wifi, I think) can really even start thinking about it.
Posted Aug 30, 2024 22:10 UTC (Fri)
by mb (subscriber, #50428)
[Link] (35 responses)
Let's suppose mac80211 had a Rust driver interface.
It would mean that mac80211 maintainers would either have to know Rust (good enough) to maintain that interface themselves or talk to somebody who knows.
I don't see a problem either way.
To me this is a tempest in a teacup.
Learning Rust is not a multi-month approach. Not even multi-week if done seriously.
Posted Aug 31, 2024 0:21 UTC (Sat)
by rc00 (guest, #164740)
[Link] (32 responses)
This is a bold-faced lie. An experienced engineer won't learn Rust in under a year. Someone with the talent level of the Primeagen took two years to learn Rust and he is arguably better than the average. The language and their proponents literally pride themselves on the complexity. Do you already forget the "skill issues" toxic posts that sprung up? It was quite similar to the crypto bros and their "ngmi" nonsense.
At best, a senior programmer could reach an *intermediate* level for Rust. Ownership, borrowing, immutable variables by default (the name is an oxymoron), lifetimes (and their annotation syntax), the complex type system, traits, error handling, concurrency/async (`Arc`, `Mutex`, `RwLock`), procedural macros, and unsafe Rust. How do you navigate that list and come away with "not even multi-week"? Anything beyond helloworld.rs is at best a multi-month affair and the inexperience and/or below average programmers are easily multi-year range.
Posted Aug 31, 2024 0:38 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link] (8 responses)
This particular engineer learned Rust in about 2 weeks back around 2018 (IIRC). If you know C++, then it's a fairly straightforward process.
Posted Aug 31, 2024 0:44 UTC (Sat)
by rc00 (guest, #164740)
[Link] (7 responses)
Are you suggesting that a sample size of one that can't be corroborated is statistically or otherwise relevant?
Professors I engage with don't even think the language could be taught in a full academic year to upperclassmen.
Which sample size is more relevant when extrapolating to the general programmer?
Posted Aug 31, 2024 0:59 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
Posted Aug 31, 2024 11:49 UTC (Sat)
by pizza (subscriber, #46)
[Link] (2 responses)
Sure, except for the minor detail that "good enough level" _currently_ means "expert level in bleeding-edge Rust" on top of already being an expert in kernel-C and one or more subsystems. That isn't something you just pick up in a few weeks, even if you had no other demands of your time.
Posted Aug 31, 2024 18:07 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link] (1 responses)
You don't need to be a bleeding-edge expert in Rust to program for Linux. Just as you don't need to be Paul McKenney level of parallel-savvy to use the RCU mechanism.
Posted Sep 2, 2024 12:10 UTC (Mon)
by taladar (subscriber, #68407)
[Link]
Posted Aug 31, 2024 1:32 UTC (Sat)
by roc (subscriber, #30627)
[Link] (1 responses)
When you "teach" C and C++ to undergraduates they don't learn the full language and they don't learn to write reliable code. They just learn enough to get code that runs and which is likely chock-full of undefined behavior unless you run sanitizers as part of your grading process.
Teaching Rust would be harder, I agree, but the resulting code will be a lot more robust and I think the students will have actually learned a lot more and will probably be better programmers as a result, even if they end up using C++. Just like learning Hoare logic makes you a better programmer even if you never use it.
Posted Aug 31, 2024 2:30 UTC (Sat)
by roc (subscriber, #30627)
[Link]
I would bet good money that it is much easier to get the Rust group over this bar than the C group.
Posted Aug 31, 2024 1:40 UTC (Sat)
by mussell (subscriber, #170320)
[Link]
Besides professors are probably the worst people to ask as, from my personal experience, they only tend to teach what that have been taught themselves. The C++ course at my university doesn't teach smart pointers, or really anything after C++98 since the prof never had to use them herself. Professors also don't tend to help out students struggling to debug their C code, nor do they mark the final code, that is of course well below their pay grade and is the job of humble student TAs.
And if you want to know just how well professors are teaching C today, go run `git log -i --grep='null pointer deref' --no-merges v6.10..HEAD` in your kernel source tree.
Posted Aug 31, 2024 1:23 UTC (Sat)
by roc (subscriber, #30627)
[Link] (9 responses)
The situation should be similar now; you're more likely to encounter 'async' early, but OTOH across the board the tools are better (rust-analyzer!) and LLMs can be helpful.
Posted Aug 31, 2024 2:24 UTC (Sat)
by rc00 (guest, #164740)
[Link] (2 responses)
I want to dive into this notion of "productive" and other philosophical tangents another time and another place but I want to leave this anchor out there:
How can you be productive in a language while at the same time, avoiding most of the features? Is that "productive"? Or is that, "I learned conditionals, assignment, and control flow" which is fundamental to operating in any language? It wouldn't take long to be "productive" in any sane language using those criteria and remaining within those confines. At what point can you say you've actually used a language? Can we measure this by percentage of the language that has actually been used? (This would clearly bias smaller languages like Lua but still.)
Posted Aug 31, 2024 2:40 UTC (Sat)
by roc (subscriber, #30627)
[Link]
I didn't avoid "most" of the features. I didn't deliberately avoid any features. The reality is that for most programming tasks you don't need to write "unsafe" or macros. Of course you will use libraries that use those features, and in particular you will use #[derive] to apply proc-macros to your code, but that doesn't require you to understand them.
I do think there is value in a small language that you can thoroughly learn --- that's why C++ is such a disaster and getting steadily worse. The main reason is that when you dig into an existing project you want to be able to understand their code and they might be using any features of the language. But this is a different kind of problem --- e.g. you generally don't need to understand much about how type inference or borrow checking works to understand someone else's code. And for code understanding, the strong isolation that Rust generics give you compared to Zig's comptime is actually in Rust's favour.
Posted Sep 1, 2024 11:53 UTC (Sun)
by ralfj (subscriber, #172874)
[Link]
This is the result of an actual study they did, not just some anecdotes about professors you met. (And FWIW, as a professor, I think it is *much* easier to teach people correct concurrent programming with Rust than with C.)
Posted Aug 31, 2024 3:30 UTC (Sat)
by mikebenden (guest, #74702)
[Link] (5 responses)
You're the second data point (along with Cyberax elsewhere in the comments) who brings up "strong C++ background" as a precondition for "easily picking up Rust". That does make a lot of sense, actually.
As a C programmer with a good working familiarity on kernel and low level topics, I see both C++ and Rust as something that would take a LOT of getting used to, they are both heavily resistant to casual lecture of sources one didn't spend a lot of time and thought writing themselves :)
I don't see "know c++ well" as an obvious required skill for a kernel hacker, so the point that kernel hackers in general would all have to take lengthy detours in order to pick up Rust still stands (unless they're already C++ experts, which would help, but presuming they are is unreasonable, because they're *kernel* hackers, not high-level code monkeys :)
Posted Aug 31, 2024 4:17 UTC (Sat)
by roc (subscriber, #30627)
[Link]
Then there are other concepts like ownership and lifetimes that are central to Rust that actually exist in C++ (and C) but the language doesn't let you write them down.
So it's not so much that I'm a strong C++ programmer but that I've spent a lot of time working with large C and C++ programs, which means I already had to internalize many of the concepts of Rust and also appreciate its benefits.
Posted Aug 31, 2024 4:30 UTC (Sat)
by roc (subscriber, #30627)
[Link] (1 responses)
The thing is, most of what people fear about Rust is already stuff you have to think about in C in the kernel, you just can't write it down. You have to think about ownership (who is responsible for freeing each thing in memory) and how that is transferred. You have to think about borrowed references (i.e. pointers) and lifetimes. You have to think about whether the data you're referencing is immutable or could change behind your back. Rust traits are very much like the manual vtables of file_operations etc. Rust's Result just codifies the distinction between normal results and error result. Rust's Option just lets you write down "this could be null".
I think probably the biggest issue is going to be Rust's generic type system. It's easy to understand how Rust types like Vec<T> are similar to void*-based generic data structures in C (e.g. radix-tree), but in practice there's a lot more value and it takes a bit of experience to use well, especially when you're designing APIs.
Posted Aug 31, 2024 14:28 UTC (Sat)
by kleptog (subscriber, #1183)
[Link]
The end result is that after this experience with Rust, they can actually do C ok because they know what they need to look out for.
Why not C++? Well, because they needed to write things that worked with various C libraries, and C++ doesn't help you at all there, unless someone has written a C++ wrapper for those libraries. Fortunately, someone had written Rust wrappers for these libraries and cargo managed all the dependencies and installation. Managing C++ library dependencies is disaster in itself.
Posted Sep 1, 2024 1:49 UTC (Sun)
by roc (subscriber, #30627)
[Link] (1 responses)
Posted Sep 1, 2024 9:29 UTC (Sun)
by farnz (subscriber, #17727)
[Link]
Ancedotally, one of the biggest things I see beginner programmers get wrong is the difference between "this code is correct" and "this code works for the test cases I give it on my machine". In this regard, C is a really painful language for beginners, because you can write code that is semantically meaningless C (IFNDR, UB etc), but that happens to work for the test cases the beginner gives it on their machine.
A language that enforces more rules is, paradoxically, easier for a beginner, because the enforcement means that the code does not work for the test cases they give it on their machine - it does not compile and run on their machine, so they have to fix it.
Posted Aug 31, 2024 5:29 UTC (Sat)
by mb (subscriber, #50428)
[Link] (6 responses)
Welcome to my killfile.
Posted Sep 9, 2024 23:22 UTC (Mon)
by rc00 (guest, #164740)
[Link] (5 responses)
Combine that with idiotic echoes like the one above and you have the exit survey for why I am likely not to renew.
This isn't an indictment of you, your team, or your choices in coverage but it is clear that I am not your target audience. Best of luck.
Posted Sep 10, 2024 17:36 UTC (Tue)
by rbtree (guest, #129790)
[Link] (2 responses)
> Go ... and Clojure
Neither one is a good choice for system-level programming, nor a potential replacement for C in the most significant project this website is about ⇒ there's not much reason for LWN (formerly Linux Weekly News) to cover them.
Although they did cover PHP for a few months a couple of years ago, it seems there was not enough interest among readers to continue.
> idiotic echoes like the one above
You may have misunderstood mb. He blacklisted your future comments because the tone of that phrase is not appreciated; that is all. I see nothing "idiotic" in his comment.
(And yes, I did support LWN while I could, and would still be doing that if not for mindless carpet-bombing performed by certain individuals completely unrelated to this site. No hypocrisy on my end.)
Posted Sep 10, 2024 18:57 UTC (Tue)
by rc00 (guest, #164740)
[Link] (1 responses)
> there's not much reason for LWN (formerly Linux Weekly News) to cover them.
There doesn't appear to be a clearly defined charter here and that's part of the issue. You've chosen to scope it to just systems-level programming languages despite the fact that the Linux ecosystem makes use of many higher-level and lower-level languages. Worse yet, there is plenty of coverage on this site for a programming language like Python so even the scope you've opted for is wildly off-base.
> it seems there was not enough interest among readers to continue.
And this is the modern challenge that content creation/distribution channels have to face. You either pander to a niche but terminally online vocal minority or you stick to some predefined ethos. The former strategy is a means for survival. The latter strategy means possibly ending up like AnandTech. I don't have the answer or any recommendations here. On my end, my attention and support are finite resources that I would rather focus where I deem they would have the most value.
> I see nothing "idiotic" in his comment.
Inane? Unsubstantial? Witless? We're just going to have to agree to disagree on this one. Maybe it's a problem on my end? I find myself having a shorter and shorter fuse with the pro-Rust crowd. Somehow, they managed to take the worst parts of the Apple fanboys and the Pythonista/Scala/Haskell hype phases and then exponentiated them into something far more toxic and all around worse. I cast aside any benefit of the doubt years ago. At this point, you can call it bias if you want. I would call it a defense mechanism, not unlike one I've developed for the crypto space. When something is off and off-putting, cutting through the chaff is a more than rational response to develop over time.
This ended up being much more than I intended to write so apologies are in order. I also can't see myself continuing this thread too much longer so apologies in advance for not carrying on with this as well.
Posted Sep 10, 2024 20:37 UTC (Tue)
by Wol (subscriber, #4433)
[Link]
It's Jon's site. He's part of the core linux kernel team (inasmuch as there is such a team). It's down to him what he cares to put in.
And that's why it's such a good site. I go on about PJ, but the charter for Groklaw was similar - if she wasn't happy with it, it got deleted. That's why it was such a damn good site.
Cheers,
Posted Sep 10, 2024 19:09 UTC (Tue)
by intelfx (subscriber, #130118)
[Link] (1 responses)
Is this the point where we are supposed to bow our heads in shame?
Recalling some of your comments on the recent Ladybird article... I'd say good riddance.
Posted Sep 10, 2024 19:21 UTC (Tue)
by corbet (editor, #1)
[Link]
Posted Aug 31, 2024 10:06 UTC (Sat)
by hunger (subscriber, #36242)
[Link] (1 responses)
They claimed that 1/3 of the new Rust devs feel as productive as in Rust as in their previous language (whatever that was) in less than 3 month. The biggest advantage their devs reported was that code review became significant easier.
The biggest claim is that Rust devs are twice as productive as C++ devs.
https://youtu.be/QrrH2lcl9ew has the entire presentation. It has some details on howmthey collected their numbers.
Posted Aug 31, 2024 13:41 UTC (Sat)
by rc00 (guest, #164740)
[Link]
Posted Sep 3, 2024 10:41 UTC (Tue)
by tialaramex (subscriber, #21167)
[Link] (3 responses)
Some top CS graduates will have a similar experience because both Cambridge and Oxford still begin with an ML as First Language and I expect at least a few of the decent CS courses in the US do as well -- beginning with a language that doesn't even have a proper type system makes the initial work of a non-vocational CS course very difficult to motivate.
But even if you've got a more vocational background where you learned like C++ and Python, or Java and Go or something, Rust's adherence to its "Empowering Everyone" slogan makes this a much softer landing that it might be. When you screw up (and you will screw up in any language) Rust will say that you screwed up, why and possibly even explain how to fix that.
In Java or Go you get inscrutable error messages. In Python the program blows up at runtime. In C++ your program - despite being complete nonsense - still compiles, and executes but doesn't do what you expected because it's wrong. What Rust is doing here maybe shouldn't stand out, but it does, and for some programmers this makes a huge difference.
Posted Sep 3, 2024 14:48 UTC (Tue)
by mb (subscriber, #50428)
[Link] (2 responses)
And that process can even be turbocharged with tools like this:
Complicated and hard to understand compiler messages do exist in Rust, too. But they are very rare.
Yes, it takes a very long time to learn each and every detail of Rust.
But getting productive does not take weeks or months.
And learning Rust makes people better at programming in general.
I think it's much harder to come from the OOP-language side than from the C side when learning Rust, because OOP-thinking (inheritance) does not really work in Rust.
Posted Sep 5, 2024 12:09 UTC (Thu)
by taladar (subscriber, #68407)
[Link] (1 responses)
Which is why, when someone claims that Rust is hard because you constantly have to "fight the compiler" you know they are doing something fundamentally wrong in their approach to the language, most likely not paying attention to what the compiler is trying to tell them because the transfer experience with other compilers to Rust where understanding most error messages requires significant effort.
Posted Sep 7, 2024 12:05 UTC (Sat)
by khim (subscriber, #9252)
[Link]
No, it's not that. Many (most?) developers these days act like “human ChatGPT”: they combine various pieces of code that look somewhat relevant to what they are trying to achieve (often with help of IDE or even, these days, LLMs), then they run the generated mess and when it, inevitably, explodes – they fix the most egregious bugs. Rust blows that approach to smithereens: sure, compiler says that what you wrote is nonsense… and even shows some possible solutions… but you don't need all that, you need to make your program to run! Disconnect can be pretty severe and both people who find Rust conceptually easy and people who find it hard often talk “past” each other. P.S. Note: the fact that Rust is “conceptually” easy doesn't mean it's easy to learn to write programs in it. It's similar to snowboard, in some sense: incredibly easy and simple tool… yet one that requires quite a non-trivial amount of skill to use it. But linux kernel people are, actually, unique positioned to learn it, because many concepts that Rust enforces are, in reality, already in use in Linux kernel… only is come places there code that “violates it a tiny bit”… and Rust wants 100% compliance, not 99% compliance.
Posted Aug 31, 2024 15:37 UTC (Sat)
by johill (subscriber, #25196)
[Link] (1 responses)
Sorry, the example of using it there wasn't really my point.
But like you said, using the wifi example again, if someone were to add rust bindings to mac80211 (which in your view would be a good thing to have), the point is that I'm still going to have to maintain them once they're in the tree, but I get nothing out of it because I can't even use them. So I have to maintain something I can't even _use_ myself.
And I think that's similar for the filesystems: as long as you can't actually rely on rust in general, most stuff will have to be maintained in C.
And I think for me at least, the point is that for those of us working on sufficiently core things rust therefore just adds work, no value. And I can't even really learn rust in the context of the code I'm maintaining, because I can't actually use it there myself! I'd totally be at the mercy of whoever adds the rust bindings, and then try to figure out what that does, but I don't think there'd be much of a chance if I'm not going to use it.
Do I think it's possible to learn rust? Sure. But I don't think it'll happen overnight (nor within a couple of weeks), but more importantly I don't think I'm going to learn rust and how to apply it to mac80211 without even being able to use it there.
(Given enough time I could of course write a toy driver for it or something, but in reality I don't see where to find the time for that. Most maintainers are already overworked. There's probably some promise in rust actually making that better, but that only applies when you can actually use rust for the work you need to do, which is nowhere near true yet for most of us.)
Posted Aug 31, 2024 18:03 UTC (Sat)
by mb (subscriber, #50428)
[Link]
What does "use" mean?
If one Rust Wifi driver were added, it would be used. Of course, the interface would not be added without a driver that uses it.
>Most maintainers are already overworked.
Yes. That is true.
But scaring away people with a hostile environment doesn't help that.
It has been a hostile environment 20 years ago and apparently (according to the article) it's still hostile today.
I'm quite happy that the code I'm still officially a maintainer of has basically become obsolete.
(Just to avoid misunderstandings: I don't mean you personally. I like you and it was a lot of fun to work together with you. Most of what I am uncomfortable with is outside of the Wifi world)
Posted Aug 30, 2024 22:45 UTC (Fri)
by jgg (subscriber, #55211)
[Link] (17 responses)
Posted Aug 31, 2024 0:10 UTC (Sat)
by rc00 (guest, #164740)
[Link] (16 responses)
Rustlings? But seriously, how would you learn any other programming language? Making demands of it to be supported in the Linux kernel? Or on your own time with your own resources and learning material. There are plenty of Rust projects that only want to do Rust, why not use one of those for a proving ground? Make the same argument with Nim or Odin or C3. These logical blind spots for a programming language are severe and very problematic.
Posted Aug 31, 2024 21:57 UTC (Sat)
by jgg (subscriber, #55211)
[Link] (15 responses)
If Linus were to declare that Janurary has no releases - a "learn rust month long holiday" and we could all get off the treadmill and get a head start on learning then sure maybe?
I understand the chicken and egg situation, but understanding it doesn't mean it is solved.
Posted Sep 1, 2024 0:58 UTC (Sun)
by rc00 (guest, #164740)
[Link] (13 responses)
Either way, best of luck with all the work! We're all thankful for what you do! IOMMUFD is neat!
Posted Sep 1, 2024 11:43 UTC (Sun)
by jgg (subscriber, #55211)
[Link] (12 responses)
FWIW, after sleeping on it, I really like the idea of a "Rust Holiday" declared by Linus. Not only does it help give people space to do something productive it is also a clear, but still soft, endorsement and message of the project's intent and direction. Might be a great rallying point.
Yes, I agree with your assesment. I think there is alot of energy in the "opposition party", and I'm not optimistic this can be overcome with the current approach. I'm am quite surprised by the forceful statements from Dave and Kees that this is done and decided. Definately not going to get involved in that fight when it comes. Let me know when it is over and we have a clear decision and workable plan please!
I'm glad you enjoy the projects! Thanks!
Posted Sep 1, 2024 13:08 UTC (Sun)
by rc00 (guest, #164740)
[Link] (1 responses)
I do think there is value in the overall concept though! If some other language(s) were to prove viable and compatible with the Linux kernel project and effective (read: stable) infrastructure could be established, a development hiatus for upskilling and on-boarding would be tremendous for progress. To me, this as a takeaway could be the most valuable part of the Rust experimentation process.
Posted Nov 28, 2024 20:06 UTC (Thu)
by deepfire (guest, #26138)
[Link]
This makes me think that you still don't "get it" -- as mentioned by many, "C's simplicity" doesn't exist.
Tongue in cheek -- according to your definition of simplicity -- is assembly simpler or more complex?
If you agree, then why is your definition of simplicity a desirable goal?
If you disagree, then you must also disagree that C is simple.
Posted Sep 1, 2024 20:35 UTC (Sun)
by airlied (subscriber, #9104)
[Link] (9 responses)
You've been in the industry long enough to know how these things work, I find it unusal that anyone with so much industry experience can't see the writing on the wall.
You've described so many chicken/egg situations that just go away once the ball starts rolling down the hill, and often won't be solved in advance of getting over the top of the hill.
Objections like, rust in the kernel needs unstable toolchain features, will go away as people start to rely on rust in the kernel, not before. Of course as long as it is out of tree and has no hard deps in the tree, devs will push for newer toolchain features etc. Once the tree has a reason to lock down and not experiment that will all become harder.
Same for the we need support for ancient architectures, if it matters, someone will get to it, if it doesn't nobody will. I don't see the nvidia open driver supporting m68k for instance.
I would say the ball is 90% rolled up a hill, the big problem is the whole no code in the kernel without users, and we are solving it, but until that big first user lands in the kernel and people want to enable it in distros I don't think momentum will be achieved. Once all the basics for writing a driver and bindings to all the common things are in, then I expect people will have a lot happier time. I also only expect to have to drag Linus in when we get Ted style pushback from someone who matters.
I realised a year ago the value here for bringing on new developers, easier driver development and memory safety, before I'd even learned the language, I've only just seriously started writing code in rust in the past few weeks and I've brought up a bunch of nova in userspace across a lot of unstable firmware interfaces and it's much nicer than any C code I could have written.
Posted Sep 2, 2024 0:50 UTC (Mon)
by jgg (subscriber, #55211)
[Link] (2 responses)
So, I think, the tehnical side will reach 90% at some point and that will be the starting gun for the real fight. This is going to be chaotic and wild. Just look at the people who are already on-record being pretty significantly anti-rust. I think you see the same thing "Drag Linus in" is not exactly something you say when you anticipate a cordial walk in the park discussion. Seriously, count me out, call me when it is done.
We have rarely (ever?) seen the kernel community competently handle the choice of breaking someone's stuff vs allowing someone else's work, and it has very frequently leaned on the theme of "no regressions" for these kinds of arguments. Causally saying someone else will fix m68k, or not, is dangerously similar to what Ted suggested in that video, I don't see there will be alot of support for that idea.
So, a workable plan, is figure out some way to avoid or minimize the fighting! Come up with rules and agreement, in advance with timelines on when things will be allowed. There is a huge practical difference between merging something like drm_panic, or binder, and, say, me changing the iommu subsystem to require Rust. Figure out some kind of agreed framework for making these decisions.
For example, if we are at 90% today then that is enough to let things that can be disabled with minimal user impact be in Rust. It kind of feels like that is what we are doing informally today anyhow.
The next gate may be that Android has completed a full release cycle at 100% Rust support, then anything that is principally for Android/embedded can be in Rust.
Then maybe the next gate is RHEL/etc/etc have also completed a full cycle with 100% Rust support. Then Rust is fair game to use for general drivers.
And finally when we have full and complete support in all toolchains, architectures and distros for a year, then Rust is fair game to use anywhere.
Include some plan for training, etc. I like my Linus Rust Holiday idea quite a bit.
Have guidelines for rewriting/replacing existing code.
It needs to be written down, like the COC situation, where all kinds of people sign up to agree to it, and it needs to be on a *slow* timeline because we have a huge ecosystem that will take ages to swallow something this big.. Then, maybe, the fights would be small enough.. Communication is so important here.
I'm just making stuff up, to illustrate what I think a plan may sort of look like - I don't want to debate any of this (seriously, count me out). I really just want to read about the outcome in LWN :P
(I don't know anything about it, but I heard a story that someone sent obscure arch enablement patches to the NVIDIA driver's github. Since they are not part of the kernel community they have no obligation to act in any particular way, but if someone sent me m68k enablement patches for RDMA I would be expected to accept them as the duty of a maintainer)
Posted Sep 3, 2024 19:36 UTC (Tue)
by airlied (subscriber, #9104)
[Link] (1 responses)
I'd rather stick with, "people who do the work, decide the plan" and let the plan evolve. I've never seen any plans for the Linux kernel either internally at companies or upstream work out in any fashion.
We do evolution really well, we should stick to our strengths.
I don't think we'll be taking "just because" rust rewrites, but again each maintainer needs to determine their own comfort level here. If we had a component with a history of memory safety issues then a rust rewrite might make it worth the investment. But from a point of view of upstreaming, I think we need novel drivers and features to start bringing in rust, not just virtual and ext2 rewrites, which are useful for finding a path, but not for driving upstream change.
Posted Sep 4, 2024 18:32 UTC (Wed)
by jgg (subscriber, #55211)
[Link]
This isn't evolution. It is a step change, and it has built into it ongoing demands to other peoples time, not just "people who do the work pay the cost".
I don't think you will get novel drivers like this. People with a commercial interest will not get involved with this much uncertainty and backporting problems standing in the way. It is already hard to get C stuff merged, 'be the first rust in X' is 10x harder and more unpredictable.
Posted Sep 2, 2024 10:13 UTC (Mon)
by nim-nim (subscriber, #34454)
[Link] (5 responses)
For Rust to be successful in the kernel Rust code writers need to interact well with C code writers because the C core is not going anywhere in the short term. Your legacy language sucks look how much better mine is let’s rewrite all your unsafe code is not a good template for (human) interaction. Some better customs needs to emerge.
Unfortunately all this is more easily said than done.
Posted Sep 2, 2024 16:08 UTC (Mon)
by Wol (subscriber, #4433)
[Link] (4 responses)
> Unfortunately all this is more easily said than done.
And the majority of the problem seems to be that the C code writers don't want to interact :-(
Cheers,
Posted Sep 2, 2024 16:59 UTC (Mon)
by pizza (subscriber, #46)
[Link] (3 responses)
No, the majority of the problem is that the "C camp" and the "Rust camp" have (fundamentally?) different priorities, and there's no "higher authority" to disabuse one or both parties of their sense of self-importance.
Posted Sep 3, 2024 4:53 UTC (Tue)
by marcH (subscriber, #57642)
[Link] (2 responses)
Don't forget loud voices on the Internet are NOT representative.
It's really hard not to forget. I forget it every day. Everyone looking at TV news and social media does too. Every participant in this thread does.
Posted Sep 3, 2024 11:20 UTC (Tue)
by ms-tg (subscriber, #89231)
[Link] (1 responses)
While undoubtedly a good point and useful reminder, I have a follow-up question.
Are Ted T’so’s comments not representative of a set of kernel *maintainers*?
Wondering if, to address the question of representativeness directly, Linux might eventually need a list of (subsystem, maintainer, open to working with Rust for Linux Team to define interface semantics Y/N) tuples?
Posted Sep 3, 2024 15:12 UTC (Tue)
by marcH (subscriber, #57642)
[Link]
Like any other volunteer work, being a maintainer has an element of "trade": trading the work against dictating how it's done to some extent. This is summarized in the "BDFL" acronym. Again, all such "dictatorship" can be done very quietly and politely.
Of course this works only up to a point: no one wants a totally disconnected volunteer. But then:
To measure 1. you could organize... a shouting match? Could not resist sorry :-)
Posted Sep 1, 2024 1:35 UTC (Sun)
by Cyberax (✭ supporter ✭, #52523)
[Link]
That actually might be a great idea. He had one of this pauses during the introduction of git, after the BitKeeper fiasco.
Posted Aug 30, 2024 23:39 UTC (Fri)
by roc (subscriber, #30627)
[Link] (4 responses)
Posted Aug 31, 2024 15:39 UTC (Sat)
by johill (subscriber, #25196)
[Link] (3 responses)
And no, the reason I can't use it in the kernel is that it can't be in sufficiently core parts of the kernel, given the tools and architecture support story.
Posted Sep 1, 2024 15:10 UTC (Sun)
by gmatht (subscriber, #58961)
[Link] (2 responses)
Posted Sep 1, 2024 15:24 UTC (Sun)
by johill (subscriber, #25196)
[Link] (1 responses)
https://docs.kernel.org/kbuild/llvm.html#supported-archit...
And rust says:
https://docs.kernel.org/rust/arch-support.html
whereas the list of all architectures is currently
alpha, arc, arm, arm64, csky, hexagon, loongarch, m68k, microblaze, mips, nios2, openrisc, parisc, powerpc, riscv, s390, sh, sparc, um, x86, xtensa
So for something sufficiently core, you can't use rust.
I'm sure this will become better over time, but (IMHO) until it does there's a lot of stuff that's simply out of scope as I described.
Perhaps with the graphics driver(s?), NVMe driver and other things that have happened we should declare the rust experiment a success, and start focusing on the architecture story etc. (gccrs, code gen, whatever) so there can be hard dependencies. The extra dependency will probably still be hated by some (including some distributors perhaps), but at least it won't be a dependency that's literally impossible to satisfy.
I guess I just don't think it's fair to say it's all the fault of kernel maintainers, who are generally already overworked, have to support stuff on all architectures, have to (willing or not) learn a whole new thing, and then everyone says "oh but it works well for me so they're just being idiots." I mean, I guess we're used to being the scapegoats but still ... Both sides would do well with some empathy for the other side.
Posted Sep 3, 2024 13:14 UTC (Tue)
by gmatht (subscriber, #58961)
[Link]
BTW, rustc_codegen_gcc announced just over a year ago that they can now compile Rust-for-Linux. Informal benchmarks suggest it is 3% slower than regular rust.
I also had a look at mrustc. I was impressed that it could compile my rust code without issues, but it doesn't seem to support some essential things like
Posted Sep 4, 2024 6:31 UTC (Wed)
by ebiederm (subscriber, #35028)
[Link]
He has previously promised to fix all problems that are clearly identified. Finding all of them is the issue. Last I looked it was an open problem on how to do that.
I looked because it was a roadblock to my work.
Of course this was before the recent crop of fuzzers.
Posted Sep 2, 2024 23:38 UTC (Mon)
by anthm (guest, #173221)
[Link] (4 responses)
But there is opposition, and it's loud and aggressive as evidenced by the video.
Loud aggressive and seemingly not particularly caring about the technical merit of their arguments.
Posted Sep 3, 2024 0:32 UTC (Tue)
by pizza (subscriber, #46)
[Link] (3 responses)
There is "opposition" to nearly everything that is submitted for inclusion in Linux, for any number of reasons, technical or otherwise. Historically this opinionated maintainer gatekeeping activity has been lauded as a GoodThing(tm), and has been one of the primary contributors to steadily raising the overall quality bar for the past 30-odd years.
Meanwhile, on what basis do you assume that video is representative? Or is it just an example of confirmation bias?
Posted Sep 3, 2024 7:24 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (2 responses)
The problem, and history will also tell you this, is that "opinionated gatekeeping" from the "young turks" can be a damn good thing. The problem is when they become "old turks", they are a damn liability!!!
Robert Hooke (or was it Boyle?) is, iirc, a particularly good example, but it seems to be the norm. "Science advances one funeral at a time" or whatever the quote is. Why should Computer Science (and linux) be any exception?
Unfortunately, all the serious pushback from opinionated maintainers seems to be of the form "I don't like it so it's not going in". That's a VERY strong indicator that the maintainer in question should step down and retire! If the reaction was "I don't like this - I don't understand it but it's ringing alarm bells" then fair enough. That at least opens the door for dialog.
I'm afraid (hoping?) things could easily start going the way someone else suggested. All it will take is for Asahi to do what she's threatened - rewrite drmbuf or whatever it was. Linus then says "okay, it looks good, it's going in" - as he should if it lives up to its promises, and next minute all new graphic drivers will be in Rust.
At which point, all those opinionated gatekeeper maintainers will find their fiefdoms deprecated to "legacy" as new people come in with Rust versions and shove them aside. That, imho, would be an extremely successful manoeuvre for linux, even if tragic for the likes of Ted. (And there's plenty of alternative disk formats. No reason why it should always be ext2/3/4. The momentum for change already seems to have been building for a while ...)
Cheers,
Posted Sep 3, 2024 13:17 UTC (Tue)
by pizza (subscriber, #46)
[Link] (1 responses)
Linus has a lot of "soft power" but at the end of the day, he can't actually _force_ anyone to do anything. Other than enforcing the "Linux" trademark, anyway.
> Unfortunately, all the serious pushback from opinionated maintainers seems to be of the form "I don't like it so it's not going in".
Their objections have considerably more meat than that. Even Ted Tso's.
(And I recall numerous Rust folks having strong opinions about what should or shouldn't go in to stuff they maintain. So let's not pretend that opinionated maintainers are okay for me, but not for thee!)
> as new people come in with Rust versions and shove them aside.
(1) there isn't a legion of "Rust developers" just waiting at the gates, and (2) nearly everyone seems to vastly, vastly underestimate just how much _work_ these core maintainers actually do, day in and day out. Rust doesn't magically make most (if any) of that ongoing work become unnecessary.
Posted Sep 5, 2024 1:16 UTC (Thu)
by raven667 (subscriber, #5198)
[Link]
this seems to be a major sticking point, for someone who learned a Rust, understands the code and is past that experience it can be hard to understand the fear and trepidation from a maintainer who may feel like they are burnt out and just hanging on at the edge of their competence and time, who are told that they need to stretch and grow and spend even more mental effort learning when they don't feel like their current workload will take a holiday to let their brain relax and absorb new thoughts. It just looks like pain. Like being at mile 20 in a marathon and having someone start jogging alongside saying "what's wrong with you,why are you breathing so hard, can't you go faster".
The Rust design seems to do what it was designed to do well, which can clear up whole classes of hard to find bugs, but being right isn't the same as winning hearts and minds and its supremely frustrating to be right and still lose. It's one thing to be confident you are technically correct, it's another to make changes which affect millions of people when you are nagged by doubt that the house of cards will topple of you touch it wrong, from somewhere outside your awareness
Posted Aug 29, 2024 21:41 UTC (Thu)
by pizza (subscriber, #46)
[Link] (3 responses)
It was also _incremental_ and didn't require fundamentally changing your tooling, workflows, and cognitive load.
> - Believing good decisions can only be made by fiat of strong leaders and not collaboratively by community consensus.
Both of those have some pathological failure conditions -- eg the first makes "wrong" decisions, and the latter never actually decides anything at all. Debian is a great example of this. (Meanwhile, there are other models, incluidng "those that do the _work_ make the decisions" and "you've earned your place as a leader by showing up to do the work consistently for a decade")
> - It is not worth it to design systems holistically to eliminate certain problems from the beginning.
Meanwhile, you can't make an omelette without breaking some eggs. Your original design is _always_ wrong. But you won't know how/what is wrong until it encounters the real world.
- So-called "soft-skills" like writing documentation should not be treated as essential parts of programming. Someone else will come along and do those. That works for the dishwasher after all.
You can "should" all you want, but at the end of the day, this won't change as long as folks paid to produce code, not documentation. (And if we're not paid at all, then what gives you any say what we should or shouldn't do?)
- The linux kernel is not just a piece of software like any other. Linux kernel developers are not just normal, average software developers. Learnings from outside the linux kernel, even from other kernels, are thus not applicable to us.
Yes, Yes, and very-much Yes. Objectively so, in both scope and scale.
And why do you presume that learnings can only go one way? Hubris, indeed.
- Tools should be built primarily around purely technical concerns, not around the humans that use them. Acknowledging their fallability and flaws is an insult and leads to worse software.
How about "different tools have different purposes"? (And a tool that cannot solve the technical problem doesn't even merit consideration!)
- And let's not ignore the spicy one: Having sufficient visibly female and queer people that you can't just dismiss them as quota fillers will doom a project, as they are by their nature less meritable.
WTF is this bullshit?
> However I'm ultimately more positive about the outlook there. Between AGX and Nova, we're about to be in a situation where 2/4 desktop graphic card drivers will run on Rust. I think that, even with the resistance to Rust, the kernel will not be able to afford to keep these things out of tree forever if people keep maintaining them.
...Just like Linux was unable to afford to keep the ginormous vendor Android trees (and their GPU drivers) out? How well did that work out?
...Again, Rust is only optional if nothing you need depends on it. And once you need it, that's a flag day, and you might as well make other stuff depend on it too because you're already paying the price, so you might as well maximize your benefits.
Posted Aug 30, 2024 0:52 UTC (Fri)
by gerdesj (subscriber, #5446)
[Link]
You are aware that big Jon is the kernel documenter-in-chief? ... and he runs LWN.
Describing LWN and #1 as "soft-skills" and hence non essential is not going to endear you to anyone. The Linux kernel is not a dishwasher and a lot of people spend a lot of time and effort documenting it.
I'll leave it there.
Posted Aug 30, 2024 9:26 UTC (Fri)
by amacater (subscriber, #790)
[Link] (1 responses)
Even when things get really protracted and discussions go on for years (systemd), it ends up that the people that do the work get to decide the technical and practical implementations of any ridiculously long debate. The number of huge discussions is decreasing and flamewars are largely out of fashion.
On visible diversity in any project - that's a problem in any significant project which doesn't reflect that talent is found in the widest community and actively seek it out.
Posted Aug 30, 2024 11:51 UTC (Fri)
by pizza (subscriber, #46)
[Link]
That my exact (if poorly-expressed) point -- It's the "strong leaders" that actually make the decisions by virtue of actually doing the damn work while "the community" endlessly bikesheds. This is pretty much true in all "communities".
> On visible diversity in any project - that's a problem in any significant project which doesn't reflect that talent is found in the widest community and actively seek it out.
At some point, one has to accept that "Individual talent" and "Individual interest" are not synonymous, especially in volunteer settings where a paycheck isn't involved.
Posted Aug 29, 2024 22:11 UTC (Thu)
by rcampos (subscriber, #59737)
[Link]
(Not quoting, something along those lines is what I recall).
And not so much longer after that, they were merged.
Posted Aug 29, 2024 22:25 UTC (Thu)
by Wol (subscriber, #4433)
[Link] (1 responses)
Focusing on the wrong metric?
My recent exchange has shown me that database designers focus far too heavily on worst case access. Because MultiValue cannot (and makes no attempt to) guarantee fast worst case response, it's dismissed. But in order to guarantee a worst case of O(log(n)), relational pretty much also guarantees a best case of O(log(n)).
So let's take a database with 100,000 rows. Relational pretty much guarantees *all* accesses are O(5). Compare that to MultiValue which guarantees (absent a pathological mess) 95,000 rows will be accessed O(1). It also guarantees to five 9s that all accesses will be O(5) or better! (which is why I picked 100K :-)
And then, assuming the relational database is First Normal Form it gets even worse - because MultiValue is 4th Normal Form, 100K records in MV are probably the same as half a mill FNF rows ...
Which would you prefer - a database that is slow and leaves you waiting forever (unless it's running on the latest and greatest electricity-guzzling monster), or a database that's maybe ten times faster, but has a slight risk of the odd query taking for ever? I would have thought a five 9s guarantee that MV is faster than relational would at least earn SOME interest...
Plus MV doesn't protect database designers from crap decisions - at least relational is an equal opportunities slowcoach :-) It peanalises novice and expert designers alike. What MV does have is plenty of tools to say "your worst case will be order X". And I've personally had experience of accidentally pathological tables - they're still damn fast.
Cheers,
Posted Aug 30, 2024 15:19 UTC (Fri)
by ma4ris8 (subscriber, #170509)
[Link]
Languages differ quite a bit. There is a common set of erroneous code that
Some programming models are shared with C and Rust.
So both tools improve productivity, and prevent various incorrect programs.
To take advantage of Rust's ability to detect more classes of errors at compile and lint time,
Posted Aug 30, 2024 23:58 UTC (Fri)
by roc (subscriber, #30627)
[Link] (6 responses)
Interesting times ahead!
I don't really get the "cannot be expected to learn Rust" thing though. It's just not that hard compared to all the other things kernel developers are expected to know, especially given you get a lot more handholding from the tools.
Posted Aug 31, 2024 1:04 UTC (Sat)
by airlied (subscriber, #9104)
[Link] (5 responses)
Like even just for future job prospects you'd think you'd refrain from publically stating your own inabilities.
Posted Aug 31, 2024 1:13 UTC (Sat)
by roc (subscriber, #30627)
[Link]
Posted Aug 31, 2024 23:20 UTC (Sat)
by sashal (✭ supporter ✭, #81842)
[Link] (3 responses)
Posted Sep 1, 2024 20:25 UTC (Sun)
by airlied (subscriber, #9104)
[Link] (2 responses)
The ability to actually consider and embrace new ideas is usually consider valuable in engineers.
Posted Sep 2, 2024 3:04 UTC (Mon)
by pizza (subscriber, #46)
[Link]
Eh, not really.
it's clearly better to hire someone that claims to have five years experience with a technology only two years old than it is to expend the time/money to train up an existing engineer.
Posted Sep 2, 2024 7:20 UTC (Mon)
by LtWorf (subscriber, #124958)
[Link]
I'm sure he won't be homeless anytime soon -_-'
Posted Aug 31, 2024 11:51 UTC (Sat)
by mohkale (guest, #159931)
[Link] (2 responses)
Posted Sep 2, 2024 7:24 UTC (Mon)
by LtWorf (subscriber, #124958)
[Link] (1 responses)
So I guess in a professional setting it happens all the time.
Posted Sep 2, 2024 12:04 UTC (Mon)
by Wol (subscriber, #4433)
[Link]
A! O! L!
Or something's obviously broke, so they carry on blindly following procedure and make the resulting mess ten times worse ...
Cheers,
Posted Sep 9, 2024 19:26 UTC (Mon)
by 0x3333 (subscriber, #158599)
[Link]
Outlook seems bleak
Outlook seems bleak
Outlook seems bleak
Outlook seems bleak
Outlook seems bleak
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
If you can't join 'em...
I only hit a compiler bug once, so far. In a tier 3 architecture (esp32) a couple of years ago.
The kernel history is *full* of gcc workarounds and banning some gcc versions from building the kernel.
That can easily be done with Rust as well. If there's a buggy version, just ban it. Or the other way around pin a known-good set of Rust compilers. (That's how it currently works, as far as I understand it). That has been done with gcc, too.
And yes, the answer *can* be yes, if there are facts that show this.
>is the wrong way to go about this.
It's unprofessional.
We all need to calm down a bit and get back to talking about facts.
If you can't join 'em...
> And yes, the answer *can* be yes, if there are facts that show this.
If you can't join 'em...
Of course it has to be done and as long as it hasn't been done it's a blocker of course.
I completely agree with you that it's not prime time for Rust-Wifi, yet. But that doesn't say anything about how much work is left to that goal. And *that* actually is the interesting question that can be used to make progress.
Calling Rust from C
By that point the GCC Backend will likely be complete, and (unlikely) GCC's frontend might even exist by then. So long as the obscure platform supports GCC, it'll be fine. Lots of people do care about platform support and are working towards expanding it.
If you can't join 'em...
If you can't join 'em...
Who is talking in that video segment?
Who is talking in that video segment?
Linux-for-Rust or Rust-for-Linux
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
1. To get a mutable reference to the Nth element of an array (or slice/Vec/whatever-it-is), you must borrow the entire array mutably. It is sound to take out multiple non-overlapping mutable borrows against the same array, but you are limited by the API exposed by slice (see e.g. slice::split_at_mut()) unless you write your own unsafe code to do it manually. Regardless, you still cause the whole array to be borrowed mutably until all element borrows are returned.
2. Your array borrow must outlive the borrow of the individual element. This means that whatever part of the program thinks it owns the Nth slot in the array must not actually hold a reference to the element (or else it would prevent any other part of the program from using at least that slot in the array, if not the whole array). It may only hold an index into the array, it can only materialize that into a reference temporarily when needed, and it must receive a mutable reference to the array in order to do that. The latter reference must be constantly passed around pervasively throughout your entire program, or else the array must be protected by RefCell<T> (single-threaded) or a lock of some kind (multi-threaded). RefCell<T> will panic if you try to mutably alias the array at runtime (the locks will block, as you might expect, and if it deadlocks, oh well, that's your problem). This is still technically possible, but it looks very ugly and not at all like idiomatic code (which is what DARPA says they want as output).
3. It is not legal to drop anything in-place in safe code, because the dropped object would still be accessible, and someone might try to use it. So when we say that the object is "freed," we mean that the bitmap no longer considers it to belong to anyone. The object still exists and has a valid value. Some other part of the code might mutate it (through this careful dance of never holding a long-lived reference to anything as described above) or even replace it with an entirely new object, but it cannot replace it with random garbage. It must uphold all invariants of its type, which may be arbitrarily complicated, and which are enforced through the use of visibility restrictions. This can cause logic errors, sure, but it is a far cry from a "real" UAF in which random bytes are interpreted as some entirely inappropriate type.
Proper Community Engagement
DARPA's mission statement
Proper Community Engagement
Wol
Proper Community Engagement
It gives you access to deref of raw pointers, but you still have to adhere to the rules.
It does also give you access to some unsafe intrinsics, which might be what you need.
But you can't simply put an unsafe block around your data race and then hope it's fine. It's still a forbidden data race.
So you can for example use an unsafe atomic intrinsic to avoid the data race.
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
See Tables III and IV. C has actually been minuscule for a long time; Python was rising at the expense of Java and C++. As far as I can tell from anecdotal data, Python has only accelerated since then due to the soaring interest in AI.
Proper Community Engagement
https://lwn.net/Articles/982868/). I'm also skipping over the recent report of the percentage of crates found to be using Unsafe Rust.
Proper Community Engagement
Proper Community Engagement
> UAF is for sure undefined behaviour in Zig. There is no way to define it!
> Nope, no UAF protection in production code with Zig.
* https://zig.news/kristoff/how-to-release-your-zig-applica...
* https://medium.com/@shyamsundarb/memory-safety-in-c-vs-ru...
* https://ziglang.org/documentation/master/#Build-Mode
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
https://github.com/ziglang/zig/issues/3180
Proper Community Engagement
> https://www.openmymind.net/learning_zig/heap_memory/
-- GeneralPurposeAllocator reuses addresses of allocated objects in many cases and does allow UAF bugs in those cases
-- GeneralPurposeAllocator avoids reusing addresses of allocated objects in other cases, and therefore must suffer from serious fragmentation problems leading to performance degradation
-- GeneralPurposeAllocator lacks a lot of the optimizations essential to modern high-performance allocators and cannot be expected to be competitive with those allocators
> //! ### `OptimizationMode.debug` and `OptimizationMode.release_safe`:
...
> //! * Do not re-use memory slots, so that memory safety is upheld. For small
> //! allocations, this is handled here; for larger ones it is handled in the
> //! backing allocator (by default `std.heap.page_allocator`).
GPA passes through objects >= the page size through to the underlying page-level allocator (which defaults to td.heap.page_allocator). For smaller objects, it has a list of page-sized "buckets" per size class, with each bucket carved up into "slots" of that size class. To allocate an object, it finds a bucket for the right size class with empty space, and uses the *next slot in the bucket that has never been used*. I.e., even if some objects in the bucket have been freed, those slots won't get used. This is what inevitably leads to fragmentation and poor cache usage.
With the default PageAllocator and GPA configuration, this memory is returned to the operating system. The next time GPA needs to allocate a page, PageAllocator will likely return memory at the same address. GPA will hand out references to that memory and so classic UAF bugs can then be triggered --- references through a pointer to the freed object will work but access memory in the freshly allocated object, perhaps of a different type.
> Use after free is now solved as far as safety is concerned with heap allocations, if you use the std lib page_allocator or GeneralPurposeAllocator.
And maybe you believed him, but unfortunately that's not true, or at least it's not true now.
This language-advocacy discussion has gone fairly far from the original topic and seems unlikely to resolve anything. This seems like a good time to let it go.
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
That's from 2016. The Rust-for-Linux project has been following that exact playbook. Nothing has changed.
Proper Community Engagement
Proper Community Engagement
Indeed — please do not continue this thread any further. We do not need this kind of personal attack here.
Good thinking
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Wol
Proper Community Engagement
Proper Community Engagement
> But change is slow, so it'll take a while until CS departments pick up teaching Rust. Some already did, and more will follow I am sure.
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Wol
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Proper Community Engagement
Linux-for-Rust or Rust-for-Linux
- Believing good decisions can only be made by fiat of strong leaders and not collaboratively by community consensus.
- It is not worth it to design systems holistically to eliminate certain problems from the beginning.
- So-called "soft-skills" like writing documentation should not be treated as essential parts of programming. Someone else will come along and do those. That works for the dishwasher after all.
- The linux kernel is not just a piece of software like any other. Linux kernel developers are not just normal, average software developers. Learnings from outside the linux kernel, even from other kernels, are thus not applicable to us.
- Tools should be built primarily around purely technical concerns, not around the humans that use them. Acknowledging their fallability and flaws is an insult and leads to worse software.
- And let's not ignore the spicy one: Having sufficient visibly female and queer people that you can't just dismiss them as quota fillers will doom a project, as they are by their nature less meritable.
So I know it is fun to insult kernel developers and make them all out to be cavemen, but perhaps some of the respectful dealings we are asking for should be directed that way too?
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
* What is the cost (pain) of the maintenance of such rules? Does it worth it?
> I believe this exercise of integrating Rust to the Linux will answer these questions.
Linux-for-Rust or Rust-for-Linux
jsr instruction. Yes, without any stack or registers, just there, in the middle of code.x1, y1 and it calls Y, then Y calls Z, Z calls some DrawLine function… you can shift three stack frames up and find all these parameters. No need to explicitly pass them around and you save both code and execution time. Perfect… except it's entirely incompatible with how C deals with stack!
Linux-for-Rust or Rust-for-Linux
> The question is: Will these changes make for a better C API? or more enforceable even with an external tool?
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Wol
Linux-for-Rust or Rust-for-Linux
Wol
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Churchill: Oh no we can't!
Wol
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Wol
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
I'm hoping to learn more about the state of gccrs at Cauldron soon. That is one of my biggest concerns with this whole thing... GCC support is needed to reach all of the targets supported by the kernel, but the gccrs project seems to be languishing with little effort going into it. Somehow, I think, we have to find a way to bring some resources to bear on that problem.
GCC and Rust
GCC and Rust
GCC and Rust
Standardization - two independent implementations are good.
Standardization - two independent implementations are good.
Standardization - two independent implementations are good.
1. Follow what appears to be the C way - declare it UB in the spec. Also, what I understand from this article & other things I've read about Rust that the Rust community is trying to avoid.
2. Clarify the spec. Choose which behavior is correct (or a third way), and rewrite the spec to clarify it.
Standardization - two independent implementations are good.
Standardization - two independent implementations are good.
Standardization - two independent implementations are good.
Standardization - two independent implementations are good.
2nd Implementation tests the meaning of the specification
2nd Implementation tests the meaning of the specification
2nd Implementation tests the meaning of the specification
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
And hopefully gccrs will make these complaints go away for good.
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
> At some point, people have them as museum pieces. They might as well run museum kernels.
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Wol
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Honestly, "unmerged toy" seems like an unnecessarily dismissive term for something like this. How about "out-of-tree useful driver" - a term that we could apply to things like fwctl as well, perhaps :)
Unmerged toy
Unmerged toy
Unmerged toy
Unmerged toy
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Wol
Two implementations - another benefit
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Based on the conversations I've had with the RHEL kernel team so far, the main blocker for RHEL is the lack of modversions support for Rust, which is being worked on. I do think it'll get enabled before Nova is in a useful state, because there are other little drivers in-tree where there are C and Rust versions and the Rust versions are better than the C versions.
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
This attitude actually doesn't help either. I don't think we have a problem with "Rustbros" in the kernel community. The Rust-for-Linux developers are doing their best to work within the kernel process and have been making progress, even if slower than they would like. Tossing insults around doesn't help, maybe we can stop that?
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
So I'm not surprised that he refuses to learn Rust.
That's really the same thinking. Let's focus on making the existing C code better instead.
That is a social problem.
If you want to do that, get help from the developers of the other side!
Yes, you need to lean about the other side *OR* ask the people on the other side for help.
That is completely unrelated to the language.
It's not a technical problem. It's a social problem. Ask for help.
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
(Actually we know it has bugs, we just don't know why, and where they are.)
But I can't do that, because I can't well break wifi for everyone on platforms rust doesn't support.
Linux-for-Rust or Rust-for-Linux
But I think Ted's argument came from a slightly different direction.
That would be a totally valid and good thing to have today.
This is not rocket science.
And most of the learnt things can even be applied to C and C++ programs.
And the best thing is that people get their invested time back from the debugger by being able to write less buggy code.
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
> How can you be productive in a language while at the same time, avoiding most of the features?
WHAT?
WHAT?
WHAT?
WHAT?
Learning Rust vs C
WHAT?
Learning Rust versus learning C
WHAT?
WHAT?
WHAT?
WHAT?
WHAT?
Wol
WHAT?
Regardless of the goodness of the riddance, this seems like a good place to stop this subthread, please.
WHAT?
WHAT?
WHAT?
Not that hard
Not that hard
Contrary to legacy languages the Rust compiler is your friend and guide when writing code and when learning to write code. It tells the programmer what is wrong, provides a hint and a link to detailed information about the problem and often also the actual fix for the problem.
https://crates.io/crates/bacon
But the same is true for C++ and even C itself.
Many concepts from Rust can be used in other languages, too. In fact, the kernel uses many of them already and many uses do predate Rust.
That reduces the step height for kernel developers even further.
Not that hard
> most likely not paying attention to what the compiler is trying to tell them because the transfer experience with other compilers to Rust where understanding most error messages requires significant effort.
Not that hard
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Does it mean "what the Intel driver uses"?
That doesn't improve things and doesn't relieve maintainers.
I won't jump in again to new areas as long as there are people potentially shouting at me for no reason.
No thanks.
I guess I'm not the only one.
Linux-for-Rust or Rust-for-Linux
What?
What?
What?
What?
What?
What?
What?
What?
What?
What?
What?
What?
Wol
What?
What?
What?
What?
1. How do you measure the disconnection?
2. Do you even have someone else to take over?
What?
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
How could Rust improve?
How could Rust improve?
I wasn't trying to criticize kernel developers. I was hoping that in addition to Linux better supporting Rust, Rust could better support Linux.rustc_codegen_gcc announced it could compile Rust For Linux.
#![no_main]. The resulting C can be 200x the size of the input rust, but it seems to shrink during compiling, and the compiled executable was smaller than the rust executable. I guess that is rust's static linking for you. I am not sure if this means mrustc-generated C would introduce excessive overhead.
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Wol
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Linux-for-Rust or Rust-for-Linux
Wol
Linux-for-Rust or Rust-for-Linux
Rust compiler is a tool, to convert source code into executable program.
are detected at compiler time. Other set of erroneous code is detected with linters.
Some programming models are C specific, some are Rust specific.
Finding more error cases, might consume a bit more CPU time.
perhaps it could be that C code doesn't have to be changed,
or then to make life easier to both C and Rust developers, some
engineered solution might be needed to keep the implementation simple as a whole.
Interesting times
Interesting times
Interesting times
Interesting times
Interesting times
Interesting times
Interesting times
Disappointing but absolutely understandable
Disappointing but absolutely understandable
Disappointing but absolutely understandable
Wol
Youth
