|
|
Subscribe / Log in / New account

Rust-for-Linux developer Wedson Almeida Filho drops out

Wedson Almeida Filho, one of the key developers driving the Rust-for-Linux project, has retired from the project.

After almost 4 years, I find myself lacking the energy and enthusiasm I once had to respond to some of the nontechnical nonsense, so it's best to leave it up to those who still have it in them.

As an example of the sort of "nonsense" he referred to, he provided a link to the video from the Rust for filesystems discussion at the 2024 Linux Storage, Filesystem, Memory-Management, and BPF Summit. His work was fundamental to getting the project as far as it has come; he will be missed.


to post comments

Outlook seems bleak

Posted Aug 29, 2024 14:54 UTC (Thu) by b7j0c (guest, #27559) [Link] (4 responses)

Language rewrites can happen in industrial codebases because management can force people to do things if they want a paycheck. This power doesn't exist in volunteer projects.

Outlook seems bleak

Posted Aug 30, 2024 12:22 UTC (Fri) by arsen (subscriber, #161285) [Link]

good thing nobody was trying to rewrite anything

Outlook seems bleak

Posted Sep 1, 2024 13:18 UTC (Sun) by SLi (subscriber, #53131) [Link] (2 responses)

In my experience it's not the management that will ask for expensive rewrites without new features.

Outlook seems bleak

Posted Sep 3, 2024 4:08 UTC (Tue) by marcH (subscriber, #57642) [Link] (1 responses)

In this particular case, the "new feature" a lot of management (and engineers) are asking is called "memory safety".

Management is finally waking up to the cost of vulnerabilities - and of quality/sloppiness more generally speaking. Better late than never.

Outlook seems bleak

Posted Sep 7, 2024 17:02 UTC (Sat) by SLi (subscriber, #53131) [Link]

I didn't mean to imply Rust has no benefit. I like the language; the "no new features" tried to capture the core of "rewrite" (or also refactoring). I just mean to say that in my experience management wants to see new features, not rewrites, and you will get pushback if you say something needs to be rewritten or refactored. Often that's good—sometimes (especially slightly novice) devs tend to imagine a rewrite as a quick silver bullet. But usually it's the engineers who push for a rewrite, not management.

If you can't join 'em...

Posted Aug 29, 2024 15:31 UTC (Thu) by marduk (subscriber, #3831) [Link] (21 responses)

Maybe it's not something that the Linux community necessarily need to be concerned about, but there's always a possibility that someday, possibly even right now, there is some unknown college student doing a (free) operating system in Rust (just a hobby, won't be big and professional like Linux) because Linux didn't seem to welcome them. And the next thing you know...

If you can't join 'em...

Posted Aug 29, 2024 15:45 UTC (Thu) by shironeko (subscriber, #159952) [Link]

more free kernels is always a good thing

If you can't join 'em...

Posted Aug 29, 2024 16:44 UTC (Thu) by e-rk (subscriber, #166547) [Link] (3 responses)

Replace "unknown college student" with "alliance of major tech companies," remove "(just a hobby...)" and it will sound somewhat plausible.

If you can't join 'em...

Posted Aug 29, 2024 16:57 UTC (Thu) by pizza (subscriber, #46) [Link] (2 responses)

> Replace "unknown college student" with "alliance of major tech companies," remove "(just a hobby...)" and it will sound somewhat plausible.

You forgot 'and (nominally) copyleft" with "permissive" '

Meanwhile, I'm not exactly holding my breath, because these large companies have universally shown that they're utterly incapable of cooperatively acting in their own longer-term interests. Heck -- even individually, they have a tendency to keep building (and using) footguns at every opportuntiy.

If you can't join 'em...

Posted Aug 30, 2024 7:23 UTC (Fri) by marcH (subscriber, #57642) [Link]

When you write "long-term", do you mean 2 or 4 quarters?

Joking aside, there's a big and obvious difference here between public and private companies.

If you can't join 'em...

Posted Aug 30, 2024 11:03 UTC (Fri) by etrusco (guest, #4227) [Link]

Search "just a hobby, won't be big and professional like UNIX" on you preferred search engine.

If you can't join 'em...

Posted Aug 29, 2024 17:56 UTC (Thu) by lonely_bear (subscriber, #2726) [Link]

There already exist an OS development based on rust: https://www.redox-os.org/

If you can't join 'em...

Posted Sep 2, 2024 16:30 UTC (Mon) by anthm (guest, #173221) [Link] (14 responses)

I don't think it's the college students that are the issue, it's the massive trillion dollar companies that are getting increasingly wary of relying on code written in C.

Google/MS/Amazon/Meta could announce a new Rust kernel foundation tomorrow and have it staffed up with experienced kernel devs quite quickly. That's the real nightmare scenario for Linux. A well supported competitor that from day one has immense resources and technical expertise available, plus buy in from companies that currently push a *ton* of Linux code.

If you can't join 'em...

Posted Sep 2, 2024 17:08 UTC (Mon) by pizza (subscriber, #46) [Link] (12 responses)

> Google/MS/Amazon/Meta could announce a new Rust kernel foundation tomorrow and have it staffed up with experienced kernel devs quite quickly.

This reminds me of a quote: "Do you know what Vegas says to folks that 'have a system'? Welcome!"

They'll find that recreating the sheer breadth and scope of what Linux provides *today* will take many, many, many man-years of effort, along with being forced to re-discover (and re-invent workarounds for) any number of scaling pain points.

> That's the real nightmare scenario for Linux.

How so? In the worst case Linux reverts back to a "just for fun" hobby project.

> A well supported competitor that from day one has immense resources and technical expertise available,

On "day one" there won't be anything other than a blank repository.

If you can't join 'em...

Posted Sep 2, 2024 19:38 UTC (Mon) by NYKevin (subscriber, #129325) [Link] (11 responses)

They are not going to do this until they have become thoroughly convinced that Rust-in-Linux cannot be made to succeed through the regular Linux development process. At a minimum, that's probably several years from now.

Then they will start carrying Rust patches against Linux, because it is less work than starting over from scratch, and much Rust-in-Linux code will already have been written by that point.

Then they start rewriting anything with complicated or "weird" lifetime semantics in Rust, so that their patches apply more cleanly and they don't have to constantly adapt to whatever upstream is doing.

Then they will stop contributing to upstream, because it just doubles the amount of work they have to do, and upstream is notoriously hostile to out-of-tree patches anyway.

Then their fork becomes the main fork, because the contributors who work for big tech companies outnumber the contributors who do not. The contributors who both work for big tech companies and are Rust-hostile are told to either learn Rust and start writing it, or find a new line of work.

At some point, I give good odds that they convince Linus to declare their fork the official Linux, similar to the EGCS/GCC split. By the time that happens, large portions of the kernel may have been rewritten in Rust, instead of the smaller interfaces we are now discussing.

This is a possible future. It is (obviously) far from inevitable. But the more hostile the current development process is to Rust, the more the big tech companies' incentives will align with this future, and the more likely it is that they will attempt to make the above, or something like the above, happen.

(Disclaimer: I work for Google, but the above is pure speculation and conjecture. I do not have any inside knowledge of Google's kernel development practices, or under what circumstances they might or might not choose to start publicly developing a Linux fork. I have even less knowledge of what any of the other tech companies are likely to do.)

If you can't join 'em...

Posted Sep 5, 2024 11:44 UTC (Thu) by taladar (subscriber, #68407) [Link] (10 responses)

One thing is for certain though, in that potential future all those obscure platforms that aren't supported by LLVM that seem to be such a big concern for many of the existing Linux maintainers hostile to Rust will lose out anyway because none of the big companies funding development care about those either.

If you can't join 'em...

Posted Sep 5, 2024 12:31 UTC (Thu) by johill (subscriber, #25196) [Link] (8 responses)

I will just say - this is how you lose supporters, by declaring concerns to be invalid and only the result of pre-conceived hostility.

If you can't join 'em...

Posted Sep 10, 2024 9:55 UTC (Tue) by taladar (subscriber, #68407) [Link] (7 responses)

On the other hand you lose supporters much more quickly by insisting that your personal niche (e.g. keeping a platform alive that hasn't been in production for 20 years as a hobby) use case should trump every other priority in the project. In fact this kind of attitude might very well break the entire project if supporters wander off to form their own project without barriers like that instead of joining your project.

In a world of finite amounts of total effort available every bit of effort put into a project needs justification and so does every choice among mutually incompatible goals. The ratio of work generated to people who care to put in the work for every single decision is a big factor here.

If you can't join 'em...

Posted Sep 10, 2024 14:58 UTC (Tue) by johill (subscriber, #25196) [Link] (6 responses)

I'll ... try to actually take this on good faith, I guess?

Your previous comment is implying that e.g. me saying I need to care about platforms that Linux supports today etc. is actually just being done in support of my pre-conceived notion that rust is bad and not going anywhere etc.

Do you actually think that helps rust's cause? Because if so I think this thread simply ends here, there's nothing I can say to that.

On the other hand, maybe you're just confused? I don't have a _choice_ to care about platforms. I'm not going to be in a position to say that from this day on WiFi simply doesn't work on any platform without rust. That would be a pretty major regression, don't you agree?

Doesn't even matter if I like to use rust or if I think it's a bad idea, but like I said, implying that I say that just because I don't even want rust ... I feel your attitude probably isn't representative, and that can only be a good thing.

If you can't join 'em...

Posted Sep 10, 2024 16:14 UTC (Tue) by mb (subscriber, #50428) [Link] (5 responses)

Well, what actually are the platforms that support wifi today and at the same time don't have Rust compiler support today? That data is what I am missing for a judgement whether it would be a big regression.

With wifi support I mean you must actually be able to put a wifi device in there and use it.

Then we have a base for judging whether we should care about those users or not.

If you can't join 'em...

Posted Sep 10, 2024 16:49 UTC (Tue) by johill (subscriber, #25196) [Link] (4 responses)

I'd think ARM (32-bit) and MIPS are still relevant in routers, and MIPS is just getting an initial rust support as we speak.

SH enables wifi in the defconfig too.

USB device support is probably pretty universal across architectures.

Anyway, I'm just pointing out that saying "you were always against rust, so you just pretend you care about architectures" is the wrong way to go about this.

If you can't join 'em...

Posted Sep 10, 2024 17:19 UTC (Tue) by mb (subscriber, #50428) [Link] (3 responses)

Arm32 has tier 2 support. It works pretty well (I use it).

Rust compilers work pretty damn well, even if they don't have the official tier 1 blessing.
I only hit a compiler bug once, so far. In a tier 3 architecture (esp32) a couple of years ago.

And it's not that all stable gcc versions support building the kernel.
The kernel history is *full* of gcc workarounds and banning some gcc versions from building the kernel.
That can easily be done with Rust as well. If there's a buggy version, just ban it. Or the other way around pin a known-good set of Rust compilers. (That's how it currently works, as far as I understand it). That has been done with gcc, too.

Based on that, I think arm32 should not be a blocker.

We need to apply the same basic rules for gcc, clang and Rust.

>SH enables wifi in the defconfig too.

Ok. Is that really a big thing? Should that block us? These are the questions I'd like to see.
And yes, the answer *can* be yes, if there are facts that show this.

>USB device support is probably pretty universal across architectures.

Yes. But IMO that doesn't mean we should support users who stick an USB host controller into an Alpha machine to put an USB wifi into it.

>saying "you were always against rust, so you just pretend you care about architectures"
>is the wrong way to go about this.

Yes, correct. This is wrong to say.

"you do" and "you just" sprinkled with "religion" and "you force me to..." is the biggest problem in this whole discussion. And it's also not limited to the kernel. I saw this exact same behavior in completely different contexts, too.
It's unprofessional.
We all need to calm down a bit and get back to talking about facts.

If you can't join 'em...

Posted Sep 10, 2024 18:01 UTC (Tue) by johill (subscriber, #25196) [Link] (2 responses)

> Arm32 has tier 2 support. It works pretty well (I use it).

The kernel doesn't support it, according to the rust documentation. Maybe that's outdated, or maybe you're saying that just about the compiler?

> Ok. Is that really a big thing? Should that block us? These are the questions I'd like to see.
> And yes, the answer *can* be yes, if there are facts that show this.

I have no idea. WiFi is pretty universal these days.

But even the rust people don't yet seem to advocate making rust a hard dependency for much if anything, so I think the point is moot. They also don't have a story for calling rust from C rather than the other way around, which would actually be the use case I'd first be interested in here to make some critical parsing code better.

If you can't join 'em...

Posted Sep 10, 2024 18:35 UTC (Tue) by mb (subscriber, #50428) [Link]

>or maybe you're saying that just about the compiler?

Yes, I was only talking about the compiler. Adding kernel support looks like a minor issue to me.
Of course it has to be done and as long as it hasn't been done it's a blocker of course.

>They also don't have a story for calling rust from C

The Rust code just has to provide an unsafe extern-C function to be called. That's it. There's nothing special about that. It's a Rust function that has normal C linkage.

>But even the rust people don't yet seem to advocate making rust a hard dependency for much if anything,

People need to understand why that is on a case by case basis.
I completely agree with you that it's not prime time for Rust-Wifi, yet. But that doesn't say anything about how much work is left to that goal. And *that* actually is the interesting question that can be used to make progress.

Calling Rust from C

Posted Sep 10, 2024 19:06 UTC (Tue) by farnz (subscriber, #17727) [Link]

Calling Rust from C is fairly well established outside the kernel, with cbindgen helping to reduce the amount of C-side boilerplate you have to keep in-sync with the Rust #[no_mangle] code and #[repr(C)] data structures.

I would hope that the Rust-for-Linux people would be willing and able to help you do a good job if you were trying to write in-kernel Rust that's called from C - adding in things like cbindgen to the build system to make your life as simple as possible.

If you can't join 'em...

Posted Sep 7, 2024 1:00 UTC (Sat) by DianaNites (subscriber, #160945) [Link]

By that point the GCC Backend will likely be complete, and (unlikely) GCC's frontend might even exist by then. So long as the obscure platform supports GCC, it'll be fine. Lots of people do care about platform support and are working towards expanding it.

If you can't join 'em...

Posted Sep 2, 2024 17:09 UTC (Mon) by mikebenden (guest, #74702) [Link]

> Google/MS/Amazon/Meta could announce a new Rust kernel foundation tomorrow and have it staffed up with experienced kernel devs quite quickly. That's the real nightmare scenario for Linux.

Oh, Lordy, *please* let them go and do that!

Worst case scenario, it would provide an outlet for all the "rustacean" kernel pushers to go be with their own kind. If they can actually come up with a useful result, that'd be a nice bonus... :)

Who is talking in that video segment?

Posted Aug 29, 2024 15:40 UTC (Thu) by bredelings (subscriber, #53082) [Link] (1 responses)

In the 3m30s video segment that was highlighted, someone is speaking from off-camera, saying "you are not going to force all of us to learn rust" and that it is a question of pain allocation. Who is that?

Who is talking in that video segment?

Posted Aug 29, 2024 15:45 UTC (Thu) by nye (subscriber, #51576) [Link]

According to https://lwn.net/Articles/978738/, that's Ted Ts'o.

Linux-for-Rust or Rust-for-Linux

Posted Aug 29, 2024 17:26 UTC (Thu) by shironeko (subscriber, #159952) [Link] (218 responses)

Linux is a giant Rube Goldberg machine made of stick and mud and hacks, giant pieces from it are grafted from other Rube Goldberg machines because the function trumps complexity (hey amdgpu). People get paid handsomely to put in new functionalities but are generally paid in angry emails for keeping things comprehensible and coherent.

As a result it is a system that is resistant to change, because change always generate angry emails. The system self-select for people that share that mentality and as a result change is even harder. It is only when the angry emails that would result from not accepting the change out number the ones from it's acceptance do any change get merged.

IMO the mistake Rust-for-Linux repeatedly make is trying to upstream piecemeal and when they don't have enough users to send all the angry emails. Angry emails and the threat of angry emails is what drives the kernel development, if you want to participate then you gotta play this game.

Proper Community Engagement

Posted Aug 29, 2024 18:31 UTC (Thu) by rc00 (guest, #164740) [Link] (58 responses)

> It is only when the angry emails that would result from not accepting the change out number the ones from it's acceptance do any change get merged.

> IMO the mistake Rust-for-Linux repeatedly make is trying to upstream piecemeal and when they don't have enough users to send all the angry emails. Angry emails and the threat of angry emails is what drives the kernel development, if you want to participate then you gotta play this game.

This is brigading (something the Rust community is actually known for already) and it is not productive, nor is it something that should be encouraged or condoned. There are technical challenges to the adoption of Rust that Linus himself pointed out recently. There was an article on LWN recently about unsafe Rust in the kernel and the problems posed there as well. The feedback from key kernel developers might not be great but the proper community response is not to be worse. Focus on solving problems and the rest will follow.

Proper Community Engagement

Posted Aug 29, 2024 19:26 UTC (Thu) by hunger (subscriber, #36242) [Link] (54 responses)

> This is brigading (something the Rust community is actually known for already) and it is not productive, nor is it something that should be encouraged or condoned.

Its not rust specific: Any community with young and enthusiastic members does that.

We definitely annoyed the heck out of everybody in the 1990s asking people to rewrite their code in C++ -- for the safety that enables.

Funny how annoyed those people are when 30+ yesrs later the same happens to them.

Proper Community Engagement

Posted Aug 29, 2024 19:33 UTC (Thu) by rc00 (guest, #164740) [Link] (53 responses)

> Funny how annoyed those people are when 30+ yesrs later the same happens to them.

Well to be fair, Linus had quite the response for the C++ devs. 😉

Though I do wonder why Linus didn't respond the same way to what is effectively this generation's C++.

Is it a generational rite of passage to annoy Linux kernel developers with the programming language du jour? 😅

Proper Community Engagement

Posted Aug 29, 2024 19:59 UTC (Thu) by hunger (subscriber, #36242) [Link] (11 responses)

> Though I do wonder why Linus didn't respond the same way to what is effectively this generation's C++.

IMHO because C++ significantly improved C, but Rust changed the game by proofing that you can have memory-safety without runtime overhead.

Whether or not you like rust, it has a huge impact due to demonstrating that -- inside and outside of our industry. E.g. the white house policies on cyber security would probably look different without that example.

Proper Community Engagement

Posted Aug 29, 2024 22:28 UTC (Thu) by LtWorf (subscriber, #124958) [Link] (10 responses)

What policies? Did they completely ban C and C++ code? Not that I'm aware of. There's only a vague suggestion.

Proper Community Engagement

Posted Sep 2, 2024 20:22 UTC (Mon) by anthm (guest, #173221) [Link] (8 responses)

DARPA is pushing automated translation. The NSA has recommended against using memory unsafe languages (note they didn't push Rust, they gave a list).

Calling it "vague recommendations" is, IMO, deliberately misconstruing the situation. It's not banned becasue it's not practicable (yet) to ban it.

Proper Community Engagement

Posted Sep 2, 2024 21:46 UTC (Mon) by LtWorf (subscriber, #124958) [Link]

I'm not aware of any ban of new C in procurement. Until that happens it's vague.

Proper Community Engagement

Posted Sep 3, 2024 1:45 UTC (Tue) by viro (subscriber, #7872) [Link] (6 responses)

DARPA might or might not be pushing anything, but result of that automated translation will be unmaintainable *AND* certain to be unsafe. Consider a program that uses a static array as a poor man's heap, with indices used instead of pointers. A separate (also static) bitmap is used to maintain the set of free elements. All perfectly expressible in Rust, _including_ an equivalent of UAF (function gets an integer argument, clears the corresponding bit in bitmap and returns; caller continues to access the corresponding element of array and then proceeds to find a clear bit in bitmap, sets it and starts using the corresponding element of array in assumption that it does not coincide with the first one). Rust will accept that as memory safe. Not pretty and (possibly) with suspicious names of primitives, but if conversion is also a mindless style corrector, results are _guaranteed_ to be broken. They could argue that the sucker is memory safe - the underlying array is not freed, after all. However, for all intents and purposes it is a use after free.

You'll get a bunch of preexisting bugs obfuscated and made harder to find.

Proper Community Engagement

Posted Sep 5, 2024 7:15 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (5 responses)

Rust will not let you do that, unless you add so many restrictions that the whole thing is both completely unergonomic and also not much of a UAF.

You cannot ever have multiple mutable references to the same array element (or any other object, for that matter), because the borrow checker forbids it (and if you cheat and do it anyway with unsafe code, it is immediate UB because of noalias). Without mutable aliasing, it's not possible for two different parts of the program to both think they own the Nth slot in the array at the same time.

The lack of mutable aliasing must be provable at compile time just from the types of the arguments. It immediately follows that you cannot conjure a mutable reference out of thin air in safe code - it must come from something you already own or another mutable reference. It does not matter whether you're indexing into an array or doing some more complicated dance. Mutability is transitive, and all of the relevant loopholes are either protected by unsafe (UnsafeCell<T>) or by runtime checking (RefCell<T> and most of std::sync).

In the specific case of the static fake heap, this fails on several different grounds:

0. All interactions with mutable statics are unsafe, because of thread safety. Unsafe code can, of course, do a UAF in Rust. That's not news, so in order for this argument to be interesting, the whole thing must work in safe code. But we can just pretend you said "a long-lived boxed slice" instead of a static array, so let's ignore this and continue.
1. To get a mutable reference to the Nth element of an array (or slice/Vec/whatever-it-is), you must borrow the entire array mutably. It is sound to take out multiple non-overlapping mutable borrows against the same array, but you are limited by the API exposed by slice (see e.g. slice::split_at_mut()) unless you write your own unsafe code to do it manually. Regardless, you still cause the whole array to be borrowed mutably until all element borrows are returned.
2. Your array borrow must outlive the borrow of the individual element. This means that whatever part of the program thinks it owns the Nth slot in the array must not actually hold a reference to the element (or else it would prevent any other part of the program from using at least that slot in the array, if not the whole array). It may only hold an index into the array, it can only materialize that into a reference temporarily when needed, and it must receive a mutable reference to the array in order to do that. The latter reference must be constantly passed around pervasively throughout your entire program, or else the array must be protected by RefCell<T> (single-threaded) or a lock of some kind (multi-threaded). RefCell<T> will panic if you try to mutably alias the array at runtime (the locks will block, as you might expect, and if it deadlocks, oh well, that's your problem). This is still technically possible, but it looks very ugly and not at all like idiomatic code (which is what DARPA says they want as output).
3. It is not legal to drop anything in-place in safe code, because the dropped object would still be accessible, and someone might try to use it. So when we say that the object is "freed," we mean that the bitmap no longer considers it to belong to anyone. The object still exists and has a valid value. Some other part of the code might mutate it (through this careful dance of never holding a long-lived reference to anything as described above) or even replace it with an entirely new object, but it cannot replace it with random garbage. It must uphold all invariants of its type, which may be arbitrarily complicated, and which are enforced through the use of visibility restrictions. This can cause logic errors, sure, but it is a far cry from a "real" UAF in which random bytes are interpreted as some entirely inappropriate type.

I will admit that DARPA's goals seem (at best) extremely ambitious to me. But if they do succeed, their generated code will not look anything like what you describe (or else they won't have accomplished what they now claim to be trying to do).

Proper Community Engagement

Posted Sep 7, 2024 17:20 UTC (Sat) by SLi (subscriber, #53131) [Link] (1 responses)

It's my impression of DARPA that if it's not a bit crazy, it's not a project for them. I don't know the numbers, but it sounds to me like they only take up things that are much less likely to succeed or not, and then one in 20 does.

DARPA's mission statement

Posted Sep 8, 2024 11:57 UTC (Sun) by farnz (subscriber, #17727) [Link]

That pretty much falls out from DARPA's mission statement. They exist to fund revolutionary technology that would otherwise fail to get funded, aiming for transformational change from every success, and eating a lot of failures in the process. Hence things like Silent Talk and TRACTOR - both high-risk projects, that would transform the world if they succeed.

And going back in time a bit, the ARPANET was a high-risk project when it started; at the time, communications was circuit-switched, and telecoms companies of every stripe were uninterested in taking the gamble to find out if wide-area packet switching could be made useful, when they were quite happy with circuits. Had DARPA not funded it, ARPANET would not have existed, and we'd probably not have any global packet switched networks, since there was a belief that the size of router buffers in packet switched networks would grow with the size of the network, disproven by ARPANET not needing large packet buffers.

Proper Community Engagement

Posted Sep 8, 2024 9:13 UTC (Sun) by Wol (subscriber, #4433) [Link] (2 responses)

> You cannot ever have multiple mutable references to the same array element (or any other object, for that matter), because the borrow checker forbids it (and if you cheat and do it anyway with unsafe code, it is immediate UB because of noalias). Without mutable aliasing, it's not possible for two different parts of the program to both think they own the Nth slot in the array at the same time.

The only reason I can think of that is a simple locking algorithm - write your pid into a variable, wait a timeout, if your pid is still there you have a lock. That obviously requires everyone to have mutable access to that variable.

Am I right in thinking that would be okay in unsafe code, provided the mutable reference never escaped the unsafe block? I can think of a couple of race conditions, so the unsafe block could lie to the compiler by accident, but the approach is sound if the implementation is correct?

Cheers,
Wol

Proper Community Engagement

Posted Sep 8, 2024 9:22 UTC (Sun) by mb (subscriber, #50428) [Link]

>Am I right in thinking that would be okay in unsafe code, provided the mutable reference never escaped the unsafe block?

The unsafe block as such doesn't give you rights to violate rules.
It gives you access to deref of raw pointers, but you still have to adhere to the rules.
It does also give you access to some unsafe intrinsics, which might be what you need.
But you can't simply put an unsafe block around your data race and then hope it's fine. It's still a forbidden data race.
So you can for example use an unsafe atomic intrinsic to avoid the data race.

Proper Community Engagement

Posted Sep 8, 2024 10:16 UTC (Sun) by LtWorf (subscriber, #124958) [Link]

That algorithm doesn't work.

Proper Community Engagement

Posted Sep 7, 2024 1:38 UTC (Sat) by montj2 (guest, #111739) [Link]

To be fair, as one who spent years in the defense industry, "vague suggestion" is a signal. Nary a requisition shall be written in 2025 that doesn't respond to the rust signaling. It's going to be a good few years for underemployed rust enthusiasts to snag federal/defense contract gigs!

Proper Community Engagement

Posted Aug 29, 2024 20:02 UTC (Thu) by kbusch (guest, #171715) [Link] (35 responses)

> Though I do wonder why Linus didn't respond the same way to what is effectively this generation's C++.

I kinda thought it was as a social issue: the kids these days are not learning C like they used to, and kernel developers/maintainers are not eternal.

Proper Community Engagement

Posted Aug 29, 2024 20:31 UTC (Thu) by rc00 (guest, #164740) [Link] (34 responses)

> the kids these days are not learning C like they used to

That is false, at least in the United States. Colleges (and high schools with AP courses) are still teaching C, Python, and even Java predominantly. In fact, the professors that I have communicated with have avoided Rust specifically for many reasons. The overall complexity (especially for younger coders) and the lack of adequate opportunities in the labor market are the most frequently cited among the many reasons.

> and kernel developers/maintainers are not eternal.

I feel like the issue isn't whether or not kernel contributors are eternal but whether they are replaceable. Handing the baton to the next generation and making sure that they are good and capable hands isn't easy for any project or even company. Nothing last forever no matter how much we want it to.

Proper Community Engagement

Posted Aug 30, 2024 22:46 UTC (Fri) by roc (subscriber, #30627) [Link] (33 responses)

Rust is just not a good first language. Neither is C, though. Java is much better, and Python better still. In practice I think Python is taking over in programming education, partly at the expense of C. This is fine and does not mean Rust is bad. We should not expect the best first language to be the best language for everything including e.g. writing kernels or huge applications.

The problem is that once people are familiar with a language they tend to use it for everything, especially people who don't see programming as their primary job. So we have scientists writing code for their papers in Python and making egregious mistakes that a stricter language would have caught :-(. It's not clear how to fix that but the Julia folks are trying, at least.

In that light, C being replaced by Python in education is good. No-one sane will try to use Python to build a kernel or other infrastructure, so they'll be forced to learn another language, and hopefully it will be Rust and not C or C++.

Proper Community Engagement

Posted Aug 31, 2024 0:00 UTC (Sat) by rc00 (guest, #164740) [Link] (31 responses)

> Rust is just not a good first language.

Agreed. I would probably take it a few steps further than that though. 😉

> Neither is C, though.

Strong disagree. C is a simple language, just not an easy one. All the fundamentals around how a computer works, syscalls, filesystems, etc. are lessons worth learning and that the language does a good job teaching. And on top of that, learning C means it is generally easier to learn other languages after the fact. C's lack of rules and restrictions are a double-edged sword but it is typically much more direct to code. Factually speaking, C helps to learn the fundamentals of programming.

> Java is much better, and Python better still.

Again, I communicate with professors and I would advise the same for you before making these kinds of comments. Some of the professors like Python because it is a much more readable language for newcomers and far easier to learn. The issue as I understand it from professors is that it sets a poor foundation. Too much is abstracted too early on in the development of these young programmers. Because there was a generation that only got Python given to them, they struggled to understand that sorting is not an operation that takes O(1) time. That is why some professors have returned to C. Explaining algorithms to someone who at least has an understanding of C data structures is far easier than someone with just Python.

> So we have scientists writing code for their papers in Python and making egregious mistakes that a stricter language would have caught

This hand-waves by too many distinct points so I'll try to break them out separately:

1. Python being a more approachable language for non-programmers meant that they were the perfect target audience. The mistakes are due to inexperience. But Rust is far from being the solution. Non-programmers have a task they want to accomplish, not spend the next two days fighting with the borrow checker and getting nowhere. The average person wants a car that they can enter and then just go. No car that requires gymnastics with multiple keys just to start the car is going to be adopted over that, no matter how shiny the features might be.* People enter cars to get from point A to point B. Back to Python, libraries like NumPy, SciPy, Pandas, PyTorch, TensorFlow, etc. etc. etc. combined with the simplicity of Python are the reason Python took a strong foothold in the data science and machine learning realm, along with the beginner-friendly nature. Performance is tolerable, it's a memory safe language, and it's easy for non-programmers to read *AND* write. Saying that a stricter language is the solution is like saying they should have twenty keys to start a car instead of one and acting like an accident is somehow inevitable. You can have memory leaks in Rust, you can have poor logic in Rust, and you can have egregious mistakes in Rust.

2. Rust proponents really don't understand history. The fact is that simple things see adoption and more complicated ones fall away. "Keep it simple" as it were. Rust's biggest impediment to real adoption is that it is too complex and prides itself on that. Traits, lifetimes, the borrow checker, async, unsafe, procedural macros, etc. etc. etc. There was a time that Go and Rust were considered to be on equal footing but that is not the case if you look at job openings. Today, there are almost 10,000 job openings related to Go. There are less than three hundred for Rust. (In my circles, Rust has been the "future" since 2017 or so.) The fact of the matter is that simpler languages will ultimately win out. Otherwise, Haskell and Elixir would have caught on by now instead of Python and Go. Rust's first-mover "advantage" over "new-langs" like C3, Odin, and Zig will be erased once those languages mature and find a footing. Why would a new person learn a complex language that takes longer to learn and then to write over a language that is easier to learn and profoundly more productive to code in? (I'm intentionally avoiding the topic of the productivity curse that is the compile speeds of Rust.) This same philosophy applies to things like the Linux desktop. The list of most-popular distributions has a huge overlap with the simplest ones. Linux From Scratch isn't a threat to Ubuntu's marketshare on the desktop space. Complexity is only a target for a niche audience though I also like how Terry Davis put it. 😉

*To expand on the example: If I offered you or anyone a flying car today but starting it required twenty keys and each key needed to be inserted/activated in the correct order just to get the car to start, would you or anyone adopt it? Some keys need to be inserted, some need to be twisted, and some are like buttons. Getting one key in the sequence out of order or used improperly (i.e. twisting a twist key the wrong direction) means you have to start the sequence over entirely. A very small percentage of people will embrace that and just because they can start the flying car doesn't mean they'll avoid all kinds of accidents like birds or other vehicles. Yet, the car would probably still be marketed as "safer." 🙃

Proper Community Engagement

Posted Aug 31, 2024 1:04 UTC (Sat) by roc (subscriber, #30627) [Link] (20 responses)

> how a computer works, syscalls, filesystems

Filesystems have nothing to do with C. If you really want to learn about syscalls and how a computer works, you need to write or at least read assembly.

> And on top of that, learning C means it is generally easier to learn other languages after the fact. C's lack of rules and restrictions are a double-edged sword but it is typically much more direct to code.

C is full of rules and restrictions, they're just not enforced by the compiler. Try explaining undefined behavior to a new programmer! Of course, no-one teaches that, so students aren't really learning C, they're learning just enough to get the right output from one run of some toy code. And that's dangerous because they think they know how to write reliable code in C when they don't.

> Again, I communicate with professors and I would advise the same for you before making these kinds of comments

No need to be condescending. I have a PhD in computer science and a lot of my friends are professors at various universities in the USA and New Zealand. While we can have different opinions on what makes a good first language, what professors choose to teach is a matter of facts. Here's some actual data from 2019: https://research.leune.org/publications/ICCSE21-CS1CS2.pdf
See Tables III and IV. C has actually been minuscule for a long time; Python was rising at the expense of Java and C++. As far as I can tell from anecdotal data, Python has only accelerated since then due to the soaring interest in AI.

> 1. Python being a more approachable language for non-programmers meant that they were the perfect target audience. The mistakes are due to inexperience. But Rust is far from being the solution.

I agree! But Rust isn't the only option that's stricter than Python. That's why I highlighted Julia for scientific users, not Rust.

> You can have memory leaks in Rust, you can have poor logic in Rust, and you can have egregious mistakes in Rust.

I wouldn't push Rust for scientific users but this kind of argument is a common fallacy that's language-independent. You can make mistakes in any language, but there are large classes of mistakes that are prevented in some languages but not others, and that matters a lot in practice.

When you're publishing scientific papers, programming errors can have huge impact --- not just on the results, but on whoever depends on those results --- and using a language that catches more errors has huge value that in many cases would justify the extra effort required.

> Rust's first-mover "advantage" over "new-langs" like C3, Odin, and Zig will be erased

None of those languages can replace Rust because none of them provide the combination of performance and memory safety that Rust does (let alone the other kinds of safety Rust provides, like data-race freedom).

I actually think a simpler language than Rust that hits the same performance+safety target could be built and might be great --- try taking Rust and replacing its generics and macros with Zig-style comptime, but keeping the aliasing and borrowing model. But AFAIK no-one has even started trying to do that, and in the 10-20 years it would take to reach maturity if started now, Rust will likely be too entrenched.

Proper Community Engagement

Posted Aug 31, 2024 1:49 UTC (Sat) by rc00 (guest, #164740) [Link] (18 responses)

> None of those languages can replace Rust because none of them provide the combination of performance and memory safety that Rust does (let alone the other kinds of safety Rust provides, like data-race freedom).

Would you consider Unsafe Rust and Safe Rust to be two languages or two parts of the same language? I'm skipping over the fact that the argument with Rust proponents went from "no one writes unsafe Rust" to "unsafe Rust is required for the Linux kernel" (paraphrasing
https://lwn.net/Articles/982868/). I'm also skipping over the recent report of the percentage of crates found to be using Unsafe Rust.

Specifically regarding the content of the article on this page, Unsafe Rust is inevitable in the Linux kernel and yet Rust was designed to make Unsafe Rust undesirable and difficult. Why write Unsafe Rust (which also means opting out of the borrow checker) instead of something like Zig that has around 80% of the memory safety that you get from Rust's borrow checker feature set but with 0% of the downside of the loss of productivity in fighting the borrow checker? (https://zackoverflow.dev/writing/unsafe-rust-vs-zig/)

Arguing for Safe Rust while dismissing that Unsafe Rust is almost inevitable is a different kind of logical fallacy. It sidesteps the reality that a language that isn't inherently memory-safe but instead makes memory-unsafe operations as safe as possible is the better solution for the assignment at hand. I believe both Zig and Odin also do not have undefined behavior where Unsafe Rust and C do. Why not pick the best tool for the job?

> I actually think a simpler language than Rust that hits the same performance+safety target could be built and might be great

This strikes me as Zig 1.0 if the current trajectory holds true.

Proper Community Engagement

Posted Aug 31, 2024 2:25 UTC (Sat) by roc (subscriber, #30627) [Link] (9 responses)

> Why write Unsafe Rust (which also means opting out of the borrow checker)

That's not the right way to think about it. "unsafe" lets you deference raw pointers, which means you can bypass the borrow checker by using raw pointers instead of references, but you can use references in unsafe Rust and they still get borrow-checked. This matters because it means that entering an unsafe block because e.g. you want to call a C function does NOT mean that you are suddenly also susceptible to lifetime bugs.

> Unsafe Rust is inevitable in the Linux kernel

It is inevitable that unsafe Rust is present in the kernel, but it matters a great deal *how much* is present and where it is present. If you have unsafe Rust in the Rust wrappers around kernel APIs but you can write a lot of drivers with zero unsafe Rust (which is actually the goal), then that is likely to be a big win. You write the wrappers, you invest effort in reviewing that code and testing it, and then all your driver code (which is much more code than the wrappers, in the long run if not the short run) benefits from the safe Rust guarantees.

OTOH if every driver has to use copious amounts of "unsafe" then Rust in the kernel will have failed.

> instead of something like Zig that has around 80% of the memory safety that you get from Rust's borrow checker feature set but with 0% of the downside of the loss of productivity in fighting the borrow checker?

In release builds (i.e. builds with acceptable performance) you get no use-after-free checking with Zig, and those are most of the exploited memory safety issues these days. Plus of course Rust is giving you more safety than just memory safety.

> I believe both Zig and Odin also do not have undefined behavior where Unsafe Rust and C do

UAF is for sure undefined behaviour in Zig. There is no way to define it!

> This strikes me as Zig 1.0 if the current trajectory holds true.

Nope, no UAF protection in production code with Zig.

Proper Community Engagement

Posted Aug 31, 2024 3:10 UTC (Sat) by rc00 (guest, #164740) [Link] (8 responses)

> OTOH if every driver has to use copious amounts of "unsafe" then Rust in the kernel will have failed.

I've already placed my wager. 😉

> In release builds (i.e. builds with acceptable performance) you get no use-after-free checking with Zig, and those are most of the exploited memory safety issues these days. Plus of course Rust is giving you more safety than just memory safety.
> UAF is for sure undefined behaviour in Zig. There is no way to define it!
> Nope, no UAF protection in production code with Zig.

Let's debug more of your programming language knowledge:

* Zig's standard library offers a general-purpose allocator that can be used to prevent double-free, use-after-free, and can also detect memory leaks.

* Zig has a build mode named ReleaseSafe that is not only fit for production code but considered the main mode for releases. With ReleaseSafe, memory and safety checks are still enabled.

Let's try to avoid spreading misinformation.

Additional links and resources:
* https://zig.news/kristoff/how-to-release-your-zig-applica...
* https://medium.com/@shyamsundarb/memory-safety-in-c-vs-ru...
* https://ziglang.org/documentation/master/#Build-Mode

Proper Community Engagement

Posted Aug 31, 2024 3:17 UTC (Sat) by intelfx (subscriber, #130118) [Link] (2 responses)

And that allocator is going to be used if (for some inexplicable reason) Linux kernel gets to contain Zig code?

> Let’s try <…>

Let’s try to get our facts and logic straight before directing condescension at others, shall we?

Proper Community Engagement

Posted Aug 31, 2024 3:37 UTC (Sat) by rc00 (guest, #164740) [Link] (1 responses)

> And that allocator is going to be used if (for some inexplicable reason) Linux kernel gets to contain Zig code?

It's a general-purpose heap allocator. For all intents and purposes, it can be thought of as the default allocator. You can certainly use others for specialized cases but during development and debugging, why not use the general-purpose one out of good habit? How is this any different from kmalloc? (These are rhetorical questions. See the last sentence of this reply.)

> Let’s try to get our facts and logic straight before directing condescension at others, shall we?

Can you approach this thread with correct information? It's quite the take that you want to apply a negative connotation to "condescension" but yet you state that toxic comments that suggest brigading are "funny."

No, I'm not going to interact with you beyond this dribble.

Proper Community Engagement

Posted Aug 31, 2024 3:41 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

> It's a general-purpose heap allocator. For all intents and purposes, it can be thought of as the default allocator.

JFYI, Linux has a variety of ways pointers are allocated, kmalloc is only one of them. One of the problems with Rust adoption was that Rust's standard library was not ready for custom allocators and fallible allocations.

Proper Community Engagement

Posted Aug 31, 2024 4:05 UTC (Sat) by roc (subscriber, #30627) [Link] (3 responses)

> * Zig's standard library offers a general-purpose allocator that can be used to prevent double-free, use-after-free, and can also detect memory leaks.

Which leaks virtual memory (to catch heap UAF) and is not suitable for production use.

> Zig has a build mode named ReleaseSafe that is not only fit for production code but considered the main mode for releases. With ReleaseSafe, memory and safety checks are still enabled.

ReleaseSafe does not affect the choice of allocator and therefore does not detect heap UAF unless you opt into the general-purpose allocator, which is not suitable for production use.

And FWIW Zig has *nothing at all* to detect stack UAF.

Proper Community Engagement

Posted Aug 31, 2024 5:05 UTC (Sat) by rc00 (guest, #164740) [Link] (2 responses)

Your knowledge is either dated or you insist on spreading misinformation. I have provided numerous links and references. Please make use of them.

> Which leaks virtual memory (to catch heap UAF) and is not suitable for production use.

The general-purpose allocator is configured with never_unmap set to false by default. Only when set to true is there a possibility that every allocation can be leaked. Again, this allocator can be used for production release modes.

https://www.openmymind.net/learning_zig/heap_memory/

> ReleaseSafe does not affect the choice of allocator and therefore does not detect heap UAF unless you opt into the general-purpose allocator, which is not suitable for production use.

Please read the previous links. This was directly addressed.

> And FWIW Zig has *nothing at all* to detect stack UAF.

The language is still not 1.0 yet. Here is the issue where use-after-free stack allocations will be addressed:
https://github.com/ziglang/zig/issues/3180

I am sharing these links in good faith. Please take the appropriate time to process them.

Proper Community Engagement

Posted Aug 31, 2024 9:04 UTC (Sat) by roc (subscriber, #30627) [Link] (1 responses)

> I have provided numerous links and references. Please make use of them.

Yes, I checked all your links (and more) before I responded, and the information in those links is consistent with what I said. Your new link is also consistent with what I said.

> The general-zurpose allocator is configured with never_unmap set to false by default. Only when set to true is there a possibility that every allocation can be leaked. Again, this allocator can be used for production release modes.
> https://www.openmymind.net/learning_zig/heap_memory/

This doesn't support your claims. It doesn't mention never_unmap or describe exactly what GeneralPurposeAllocator guarantees or say what the performance impact is. In fact those things are not documented anywhere outside the source code, as far as I can tell; certainly nowhere you have linked to. So I find your attitude irksome.

But in the spirit of https://xkcd.com/386, I went ahead and looked at the source code: https://github.com/ziglang/zig/blob/master/lib/std/heap/g.... The situation is pretty complicated, but as far as I can tell, the following things are true in the default configuration with ReleaseSafe build mode:
-- GeneralPurposeAllocator reuses addresses of allocated objects in many cases and does allow UAF bugs in those cases
-- GeneralPurposeAllocator avoids reusing addresses of allocated objects in other cases, and therefore must suffer from serious fragmentation problems leading to performance degradation
-- GeneralPurposeAllocator lacks a lot of the optimizations essential to modern high-performance allocators and cannot be expected to be competitive with those allocators

In more detail: the source code comments say:
> //! ### `OptimizationMode.debug` and `OptimizationMode.release_safe`:
...
> //! * Do not re-use memory slots, so that memory safety is upheld. For small
> //! allocations, this is handled here; for larger ones it is handled in the
> //! backing allocator (by default `std.heap.page_allocator`).

It's not entirely clear from the comment what "memory slots" means here, but looking at the code it's more clear: https://github.com/ziglang/zig/blob/37df6ba86e3f4e0f5d6a2...
GPA passes through objects >= the page size through to the underlying page-level allocator (which defaults to td.heap.page_allocator). For smaller objects, it has a list of page-sized "buckets" per size class, with each bucket carved up into "slots" of that size class. To allocate an object, it finds a bucket for the right size class with empty space, and uses the *next slot in the bucket that has never been used*. I.e., even if some objects in the bucket have been freed, those slots won't get used. This is what inevitably leads to fragmentation and poor cache usage.

However, when freeing a sub-page-size object, if that means the bucket is *completely empty*, the entire bucket is freed: https://github.com/ziglang/zig/blob/37df6ba86e3f4e0f5d6a2...
With the default PageAllocator and GPA configuration, this memory is returned to the operating system. The next time GPA needs to allocate a page, PageAllocator will likely return memory at the same address. GPA will hand out references to that memory and so classic UAF bugs can then be triggered --- references through a pointer to the freed object will work but access memory in the freshly allocated object, perhaps of a different type.

Note that the comment "for larger ones it is handled in the backing allocator" is false for the default PageAllocator. There is no logic to avoid reusing virtual memory in that code. So virtual memory isn't leaked, but in exchange you get UAF bugs.

For allocations >= the page size, you enable UAF bugs immediately because PageAllocator can reuse addresses.

This Reddit thread agrees with me that "UAF will not be reliably detected": https://www.reddit.com/r/Zig/comments/1eysv2k/general_pur...

andrewrk did write: https://github.com/ziglang/zig/issues/3180#issuecomment-6...
> Use after free is now solved as far as safety is concerned with heap allocations, if you use the std lib page_allocator or GeneralPurposeAllocator.
And maybe you believed him, but unfortunately that's not true, or at least it's not true now.

> I am sharing these links in good faith. Please take the appropriate time to process them.

Oh, I've spent way too much time processing this.

> Here is the issue where use-after-free stack allocations will be addressed: https://github.com/ziglang/zig/issues/3180

Will it really, though? It is not easy at all to detect and prevent stack UAF bugs. For example, if you take the address of a stack value and store it in a heap object, is that allowed? It's often useful to be able to do that, so I guess Zig would want to allow it, but if you allow it, it can be very difficult to prove that the pointer is not used after the function has returned. If they do solve this, there will be some overhead and/or some existing useful Zig code that is no longer legal.

For now, safe Rust prevents UAF and all Zig has is partial, inefficient solutions and promises.

Proper Community Engagement

Posted Aug 31, 2024 13:34 UTC (Sat) by corbet (editor, #1) [Link]

This language-advocacy discussion has gone fairly far from the original topic and seems unlikely to resolve anything. This seems like a good time to let it go.

Proper Community Engagement

Posted Aug 31, 2024 5:32 UTC (Sat) by pbonzini (subscriber, #60935) [Link]

> I've already placed my wager

https://lwn.net/Articles/863459/ has two unsafe blocks, both at initialization time, and that was three years ago. It's a simple driver, sure, and some sources of unsafely are only apparent with e.g. devices that do DMA, but rest assured that the developers have done their homework.

Proper Community Engagement

Posted Aug 31, 2024 14:32 UTC (Sat) by kleptog (subscriber, #1183) [Link] (7 responses)

> Specifically regarding the content of the article on this page, Unsafe Rust is inevitable in the Linux kernel and yet Rust was designed to make Unsafe Rust undesirable and difficult.

I've never understood this argument. Saying "Rust is a bad choice for the kernel because you need unsafe sometimes" is the same as "C is a bad choice for the kernel because you need ASM blocks sometimes". Except nobody complains about the number of ASM blocks in the kernel.

It would be even worse if the option wasn't there. Then it would definitely have been a non-starter.

Proper Community Engagement

Posted Aug 31, 2024 15:08 UTC (Sat) by rc00 (guest, #164740) [Link] (4 responses)

You don't understand how using a language for its intended purpose makes sense?

Do you remember the earlier days of this years-long hype cycle for Rust? "No one should write Unsafe Rust, that defeats the entire purpose."

No responsible person writing C ever said that Assembly should be avoided at all costs. There was a time that C was considered a higher-level programming language, especially when viewed through the lens of something like Assembly. In the same way that Python is today's mainstream higher-level language, there are still plenty of internal components for it written in C. Pairing higher-level languages with lower-level languages is far from novel or frowned upon.

Rust is explicitly not the right tool for the job and this is based on what some of the largest Rust proponents have said in their own words. However, the argument against Unsafe Rust by Rust proponents was discarded to prioritize Rust being thrust into the Linux kernel project. "Rust by any means," so to speak. This strategic and constant shifting of the goalposts is the very reason there is a groundswell of intelligent people ready to move on from Rust, its related hype, and many of the related individuals. We went through this with Haskell, and Scala, and so on.

Proper Community Engagement

Posted Sep 1, 2024 1:57 UTC (Sun) by roc (subscriber, #30627) [Link]

> Do you remember the earlier days of this years-long hype cycle for Rust? "No one should write Unsafe Rust, that defeats the entire purpose."

An important part of Rust has always been building safe abstractions around unsafe code. See e.g. https://smallcultfollowing.com/babysteps/blog/2016/05/23/...
That's from 2016. The Rust-for-Linux project has been following that exact playbook. Nothing has changed.

If you heard "no-one should write unsafe Rust" then someone was very confused, and I suspect it's you.

Proper Community Engagement

Posted Sep 1, 2024 19:05 UTC (Sun) by kleptog (subscriber, #1183) [Link] (2 responses)

> "No one should write Unsafe Rust"

The irony is, if you type this phrase in Google with quotes, it returns exactly one match: your comment. So it's not surprising I've never heard of it before.

> this is based on what some of the largest Rust proponents have said in their own words.

to which I can only say [citation needed]. Because I spent on time looking but could find any explicit instances, just more claims that these people did exist.

Proper Community Engagement

Posted Sep 1, 2024 20:18 UTC (Sun) by rc00 (guest, #164740) [Link] (1 responses)

The irony is that if you were alive during the late 2010s, you would already know. You spent time looking but couldn't find blogs or Reddit posts that were either scrubbed at the time or scrubbed since? Have you ever heard of Actix(-web)? Read what little is left online of the toxic community's discourse during that time.

You're surprised that what has become an inferior search engine can't find the hordes of toxic examples from an even more toxic community known to suppress and try to rewrite reality and then history, and then gaslighting whenever adverse topics are brought up, insinuating that they're fabricated. No, this is not a thread I'm going to continue with you or anyone else.

Good thinking

Posted Sep 1, 2024 21:17 UTC (Sun) by corbet (editor, #1) [Link]

Indeed — please do not continue this thread any further. We do not need this kind of personal attack here.

Proper Community Engagement

Posted Sep 2, 2024 21:55 UTC (Mon) by anthm (guest, #173221) [Link] (1 responses)

Rust needs unsafe blocks therefore it's no different than a language that is just unsafe everywhere comes from the same place as not all bugs are memory bugs so memory safety is pointless.

Proper Community Engagement

Posted Sep 3, 2024 10:08 UTC (Tue) by mbunkus (subscriber, #87248) [Link]

Yeah, this is an argument I hear quite often. Here's a comment someone made to me a couple of days ago:

> Rust is no more secure than C/C++. Memory ownership is an important part of security BUT not the only one. That is simple problem with most folks who talk about rust being the magic bullet; it is not. Memory safety might feel like a hot issue right now but in 5 years there will be another one; even with moving to rust.

I find this to be completely ludicrous, given examples such as Microsoft's observation that 70% of their observed security vulnerabilities are, in fact, memory safety issues. I wonder if these types of comments come from malice or ignorance. I'm inclined to believe it's the latter.

Proper Community Engagement

Posted Aug 31, 2024 9:36 UTC (Sat) by Wol (subscriber, #4433) [Link]

> When you're publishing scientific papers, programming errors can have huge impact --- not just on the results, but on whoever depends on those results --- and using a language that catches more errors has huge value that in many cases would justify the extra effort required.

Two cases in point - the entire diet industry is founded on a paper that concluded "fat is bad for you". That has now been thoroughly debunked. The alternatives pushed by the diet industry are far worse (they replaced fat with sugar, which has pretty much *caused* the current obesity, cancer and diabetes epidemics. Then there's the margarine and trans-fats which is probably behind all the heart disease).

And read Feynman, where he spent ages trying to work out why his sub-atomic-particle charges didn't add up. Then he thought "if the charge on this particle is wrong ... AHA - I remember seeing the proof and thinking it was screwy!!!". So he looked at it, and if he deleted just one outlier result, the proof flipped the charge! So we have 20 years of atomic research possibly derailed by just one paper where the author didn't understand statistics!

That's also why I hate seeing newspaper headlines "A new study has proved ..." - anybody who knows science knows that ALL new studies PROVE NOTHING. It's the REPEAT studies that prove the first one was correct! Unfortunately, it's very difficult to get funding to repeat a study ... and getting back to topic - it's getting better but how many studies cannot be repeated because the software required cannot be found or reliably recreated?

Cheers,
Wol

Proper Community Engagement

Posted Sep 1, 2024 11:58 UTC (Sun) by ralfj (subscriber, #172874) [Link] (7 responses)

> I communicate with professors and I would advise the same for you

Gotta love an argument by authority. Please bring actual data or logical arguments, not "some people with a title said so".

Also, speaking of professors, as a professor I happen to communicate with other professors regularly. Many of us think that Rust is a much better choice for a first language than C. C is a terrible choice because it's basically a big collection of foot-guns, and there's just not enough time in a semester to list them all.

But change is slow, so it'll take a while until CS departments pick up teaching Rust. Some already did, and more will follow I am sure.

Proper Community Engagement

Posted Sep 1, 2024 12:52 UTC (Sun) by rc00 (guest, #164740) [Link] (6 responses)

> Many of us think that Rust is a much better choice for a first language than C.
> But change is slow, so it'll take a while until CS departments pick up teaching Rust. Some already did, and more will follow I am sure.

This is problematic. How do you define your mission statement? There are literally hundreds of job postings for Rust while there are thousands for Go, or C, or Java, etc. (Indeed claims 400 currently. LinkedIn lists 300. Actual data.) Also, these Rust jobs are mostly in Blockchain or Crypto. (I'm hard-pressed to find anyone worth their salt who doesn't believe these industries to be junk but would love to hear your thoughts.) And this is approaching nearly a decade of Rust being "the future." Is your goal to teach the undergrads Rust so they have no useful skills before sending them into the labor market?

Call it anecdotal data, but I can barely recall meeting any professor who thought that it was more important to focus on a specific greenfield language rather than to adequately structure the curriculum to best prepare the student body with useful skills and knowledge to enter the labor force and find immediate success. This is why programs partner with companies.

You could present some of the challenges with teaching C and say that other employable languages make more sense or offer more approachable teaching opportunities. How do you justify using an unemployable language for students without calling it a disservice?

P.S. Some small background information about myself. The reason I engage with professors from various local universities is not only for my own curiosity but because I regularly conduct speaking engagements with both undergraduate and graduate students. I've also taken on mentorship roles with quite a few over the years and to this day, these are some of my most cherished relationships. The only scope of bias is that these are all private universities and colleges. While I am curious about public universities and colleges, I don't have much in the way of experience or connections at that level.

Proper Community Engagement

Posted Sep 1, 2024 15:12 UTC (Sun) by excors (subscriber, #95769) [Link]

> How do you define your mission statement?

I'd hope their mission is to teach students the fundamentals that will make them effective at learning all the other technologies they'll need to use over their career, many of which haven't been invented yet, rather than training them in one specific language.

(The first language taught at my university was ML, specifically because it demonstrated the fundamentals of computation without all the distractions that C/Java/etc introduce, and because (being a practically useless language) it was unlikely any students had prior experience with it and everyone would start from a similar level. Then they taught maybe five other languages to varying extents, some more practical and some less.)

> There are literally hundreds of job postings for Rust while there are thousands for Go, or C, or Java, etc. (Indeed claims 400 currently. LinkedIn lists 300. Actual data.)

LinkedIn tells me there are 3000+ jobs for "Rust software engineer" in the US, so I'm not sure where you're getting 300 from. Certainly it's more niche than many other languages, but from an employability perspective it's not an extreme niche like e.g. Zig (4 jobs on LinkedIn).

For serious (non-blockchain) tech companies, Amazon alone has 173 job adverts that mention Rust - sometimes explicitly requiring Rust experience, though mostly asking for e.g. "Experience programming with at least one modern language such as Python, Ruby, Golang, Java, C++, C#, Rust". Knowing any of those will put you in a good position to learn the specific languages you need for the job. Microsoft has 97 mentioning Rust, e.g. "4+ years of experience with coding in one of C#, Python, Go, Rust, Java, C or C++", or "Knowledge/experience programming in RUST or other memory safe languages is highly desirable".

Google rarely mentions Rust in job adverts, but they've said over 1000 Google developers committed Rust code in 2022 and over 500 took a Google-developed training course. So I think that doesn't indicate they don't believe Rust experience is valuable, just that it's rare enough they don't bother calling it out in job ads, and they believe it's worthwhile to invest in training developers for their Rust-based projects, which could be seen as even stronger support than asking for prior experience.

Proper Community Engagement

Posted Sep 1, 2024 16:08 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

> This is problematic. How do you define your mission statement? There are literally hundreds of job postings for Rust while there are thousands for Go, or C, or Java

And yet there are even more jobs that require JavaScript. Let's rewrite the kernel as a NodeJS module.

Rust will never be as popular as Go or Java, simply because it's not the best language for regular server-side applications.

Proper Community Engagement

Posted Sep 1, 2024 16:18 UTC (Sun) by Wol (subscriber, #4433) [Link]

> Call it anecdotal data, but I can barely recall meeting any professor who thought that it was more important to focus on a specific greenfield language rather than to adequately structure the curriculum to best prepare the student body with useful skills and knowledge to enter the labor force and find immediate success. This is why programs partner with companies.

And this comment embodies everything I think is WRONG with the current education system. That attitude belongs with the UK polytechnics of old, not what Universities are SUPPOSED to be for. Not blaming Universities, it's the politicians who are desperate to see everybody educated, when maybe the majority of people are incapable of benefiting from education!

What the parent has just described is what I would describe as TRAINING, not Education, and that does not belong in a traditional University. That's what Polytechnics did - they prepared people for the world of work, they trained them, and that's fine.

Going back to a study I came across, apparently the ability to think *logically* is acquired at about age 14, and maybe half of all people never acquire the ability. Yet education - higher education certainly - is all about logical thought! So why on earth are the politicians (I think this is the UK Conservatives pre Tony Blair, so that dates it) desperate to send half the population to Uni? It's coming home to bite them now, as a lot of people see Uni as loading them up with debt to acquire worthless qualifications. And for so many, that actually is the case.

Uni should be for education. Uni should be for teaching people how to use their brains. To quote a lady from years back on Radio 4 - "I didn't send my son to school to do Domestic Science so he could cook. I sent him to school to learn Latin so he knew how to read a recipe book and follow instructions". Too many people think education is doing Domestic Sci. IT ISN'T. THAT'S TRAINING. Education is learning Latin and Maths, so you can get a career as a programmer (like I did).

Cheers,
Wol

Proper Community Engagement

Posted Sep 2, 2024 11:41 UTC (Mon) by taladar (subscriber, #68407) [Link]

As someone who had to (20 years ago) sit through lectures of trying to explain Kernel memory management in Java (a language without pointers or references as a language construct) I would consider the "employers want this" method of choosing a teaching language as tried and failed at this point.

Proper Community Engagement

Posted Sep 2, 2024 18:35 UTC (Mon) by ralfj (subscriber, #172874) [Link] (1 responses)

We're not teaching students the language/framework of the day, we're teaching them the underlying concepts. Even if they have to write C or Java later, having learned Rust will teach students how to structure large, concurrent programs in a way that scales. It's a transferable skill, but to learn that skill it is extremely valuable to have the compiler automatically check your work. "Learning Rust made me a better C/C++ programmer" is a common phenomenon.

Thinking in terms of "(un)employable languages" is entirely the wrong way of thinking. Any CS graduate worth their salt is able to quickly pick up a new language and transfer the concepts they are familiar with -- that is what truly makes them employable.

Proper Community Engagement

Posted Sep 2, 2024 19:02 UTC (Mon) by pbonzini (subscriber, #60935) [Link]

In 2008-2009 I taught a C/C++ course at a university and I was already explaining how callees could "take ownership" of an argument or return a "shared pointer". Universities rarely teach this stuff because traditionally it's something that you learn by doing, and it's very rare to find it explicitly mentioned in C textbooks. In my case I had to substitute for someone else and decided that screw everything, I was going to explain C the way *I* used it since there was no textbook anyway.

If the language _forces_ the teachers to include the concepts of shared/exclusive reference or passing ownership, all the better, even if the students end up writing C and these three things all become a T*.

Proper Community Engagement

Posted Sep 2, 2024 9:47 UTC (Mon) by nim-nim (subscriber, #34454) [Link] (1 responses)

> There was a time that Go and Rust were considered to be on equal footing but that is not the case if you look at job openings. Today, there are almost 10,000 job openings related to Go. There are less than three hundred for Rust.

Go’s trump card is Kubernetes, a lot of software gravitates around containers, which means there are a lot of building blocks already written in Go people can reuse (there is a strong batteries included effect like for Python or Java, and understanding Go is necessary to fix or adapt existing deployed code).

Rust people do not seem to understand there is a point where you need to stabilise the API/ABI and commit to it so people start writing long-term apps and publish reusable components. You can only go so far in Gentoo mode.

Proper Community Engagement

Posted Sep 2, 2024 13:55 UTC (Mon) by intelfx (subscriber, #130118) [Link]

> Rust people do not seem to understand there is a point where you need to stabilise the API/ABI and commit to it so people start writing long-term apps and publish reusable components. You can only go so far in Gentoo mode.

As if Go has a stable ABI or really anything besides "yolo static link the entire world"?

Proper Community Engagement

Posted Sep 2, 2024 21:45 UTC (Mon) by anthm (guest, #173221) [Link]

Python is taking over at the expense of Java.

Even 15 years ago when I was in school it was all Java. C was kinda used in the OS class out of necessity, but that was the weird legacy case and it wasn't really taught as a language.

Proper Community Engagement

Posted Aug 29, 2024 21:43 UTC (Thu) by SLi (subscriber, #53131) [Link] (4 responses)

Perhaps because you can afford to ignore improved languages for a few decades, but you cannot be stuck in an early days programming language for ages when the world has come up with something better. (Yeah, put ne in the camp who thinks Linus was wrong about C++, even if as a language it's a holy mess and you would have had to force everyone to be disciplined. It's not that many days since we had an article about kernel devs trying to do RAII in C—it was about file descriptors—and it not working.)

I had to look up brigading. I'm still not sure what it means in this context. The OPs message is certainly harsh in tone and a bit over the top, but it's hard to say it doesn't contain a level of truth to it. Being someone more of the old guard, I don't think it makes the message worthless or worse that I wouldn't have expressed it that strongly. On the contrary, it's good that grievances get aired, whether they are justified or not. Then we can discuss them and either decide that they are unfounded or that there's some truth to them. Both results are fundamentally positive.

Proper Community Engagement

Posted Aug 30, 2024 3:01 UTC (Fri) by NYKevin (subscriber, #129325) [Link] (1 responses)

"Brigading" in this context is a term I've seen used on Reddit (it probably predates that platform) to refer to a group of people from internet community A going over to community B, all at the same time, and aggressively participating in one or more active discussions there, with the result that those discussions are (in aggregate) more reflective of A's values and preferences than B's. If A and B are just internet forums, this is (IMHO) not terribly harmful because who cares, it's just people on the internet talking to each other (or more commonly past each other), and eventually the A folks will get bored and go back to A (having said that, the A people often end up misbehaving in various ways, which tends to really annoy B's moderators if it has any). But if B is a vehicle for making real decisions about real problems in the real world (e.g. B is LKML), then this can derail the decision-making process, or result in decisions that B would not have otherwise made if left to its own devices. Since most internet communities have very limited concepts of formal "membership" (the only major exception I can think of is Debian), it is not straightforward to write rules that prohibit this behavior, nor to identify specific instances of it.

TL;DR: It's sockpuppetry, except you get a bunch of real people to help out instead of doing it all yourself. Unfortunately, when you remove the defining property of sockpuppetry from sockpuppetry, it makes the whole thing really damn hard to pin down and quantify objectively.

Proper Community Engagement

Posted Aug 30, 2024 3:48 UTC (Fri) by viro (subscriber, #7872) [Link]

I'm not sure if I ought to mention that, but the idea of anyone coming to linux-kernel@vger.kernel.org with intent of affecting kernel development by applying the amazing power of whinge-a-cappella is honestly quite amusing.

The thing is, most of the active developers have not been subscribed to l-k for decades now. I'm not sure what sustains the myth about LKML-as-a-place-where-development-happens, but it really, really had not been a function of that list for a very long time. _Some_ of us are still subscribed, but the volume is much too high for anything other than skimming the traffic. Reading every sodding message? Not realistic. It's easily over 1e3 posting/day. Often - twice as much.

Threads on other lists Cc'd over there for posterity? Sure. Pull requests Cc'd - ditto. Active development that is not just Cc'd over there (with most of the participants _not_ subscribed to that list) - not really.

25 years ago it was still used for development; 20 years ago - not so much; 15 years ago - not at all. Complaining there about anything may or may not be satisfying, but those complaints are not going to be read by the people not on the list. Which includes Linus, et cetera. The same goes for any kind of advocacy, pro or contra anything whatsoever - it's not going to be read by people who are not subscribed to the list it is posted on...

Proper Community Engagement

Posted Aug 30, 2024 18:37 UTC (Fri) by jmalcolm (subscriber, #8876) [Link] (1 responses)

Having to force discipline in how the language is used is probably not how Linus wants to spend his time. If that was going to be a requirement of C++ usage, it makes sense that he instead chose to take a stand against C++ entirely.

Proper Community Engagement

Posted Aug 30, 2024 19:58 UTC (Fri) by SLi (subscriber, #53131) [Link]

That's not something I disagree with. And in the end he's probably the most central person who has to bear the consequences of the choice, so I think it's fair even if he chooses BASIC.

But then I think you underestimate the current amount of discipline. Some of it is at the language level. A lot of it is at a more semantic level (when yo take locks, free resources, etc), which tend to be concerns where modern languages help a lot.

I mean, C clearly already goes some amount in this direction compared to previous languages and previous versions of C, for example in requiring casts instead of silently allowing you to mix types and get crashes. I don't think many serious C programmers would argue that that part of the language is not useful. It's just that some people have the idea that nothing useful for systems programming has happened in languages since then, which is clearly already disproven by the desire to use the GCC cleanup things (which don't work).

Proper Community Engagement

Posted Aug 29, 2024 19:54 UTC (Thu) by shironeko (subscriber, #159952) [Link]

All I'm saying is they should follow Linux development patterns, rather than imagining the polite/collaborative way of doing things somehow works.

Proper Community Engagement

Posted Aug 31, 2024 3:13 UTC (Sat) by intelfx (subscriber, #130118) [Link]

I… really don’t think GP was actually advocating brigading. The entire comment comes across as faily tongue-in-cheek (and a pretty well made one, IMO).

Proper Community Engagement

Posted Sep 2, 2024 17:16 UTC (Mon) by anthm (guest, #173221) [Link]

It's not brigading if you are actually contributing.

No one on the LKML is going to give any credence to things written by people who aren't doing any work.

But if you are doing the work and it's not getting in because other people are dragging their feet or making up excuses that don't pass technical muster then it's 100% right to start sending emails.

Linux-for-Rust or Rust-for-Linux

Posted Aug 29, 2024 20:49 UTC (Thu) by atnot (subscriber, #124910) [Link] (158 responses)

> The system self-select for people that share that mentality and as a result change is even harder.

I think this really gets to the core of it. But it's not just about angry emails and it's not really about being resistant to change.

Refactors like Folios are a significantly more invasive and far reaching than anything Rust has proposed so far. It's certainly more invasive than the 500 line (including docs) patch wrapping struct device that took years to merge. Yet folios have not faced the same degree of reaction. Yes, sure there's been a bunch of grumbling about whether this is really necessary. But I've never heard a kernel developer complaining about the "folio religion wanting to make us all learn folios" or grandstanding about how they will valiantly refuse to use them, or even implying it might be some sort of shadowy woke agenda of big bookbinding.

The difference is not really the amount of scope here. It's that while folios may be a big change, Rust is a change that, if successful, would prove some of those world views that as you point out the kernel self-selects for wrong or at least put them into question.

I think there's a number of these like:
- Believing good decisions can only be made by fiat of strong leaders and not collaboratively by community consensus.
- It is not worth it to design systems holistically to eliminate certain problems from the beginning.
- So-called "soft-skills" like writing documentation should not be treated as essential parts of programming. Someone else will come along and do those. That works for the dishwasher after all.
- The linux kernel is not just a piece of software like any other. Linux kernel developers are not just normal, average software developers. Learnings from outside the linux kernel, even from other kernels, are thus not applicable to us.
- Tools should be built primarily around purely technical concerns, not around the humans that use them. Acknowledging their fallability and flaws is an insult and leads to worse software.
- And let's not ignore the spicy one: Having sufficient visibly female and queer people that you can't just dismiss them as quota fillers will doom a project, as they are by their nature less meritable.

I'm sure you can think of more. Of course, the degree to which these beliefs are held in any one person will vary, and there are equally things that go the other way. But the general point is that views that extend far outside of the kernel and programming lead people to *really want* the project to either fail or succeed to prove or disprove those beliefs. Which makes things much harder than they need to be.

However I'm ultimately more positive about the outlook there. Between AGX and Nova, we're about to be in a situation where 2/4 desktop graphic card drivers will run on Rust. I think that, even with the resistance to Rust, the kernel will not be able to afford to keep these things out of tree forever if people keep maintaining them, just like PREEMT_RT. And just as Linux has selected for these things in the past, these things will change who it selects for in the future.

Linux-for-Rust or Rust-for-Linux

Posted Aug 29, 2024 21:28 UTC (Thu) by corbet (editor, #1) [Link] (150 responses)

So I know it is fun to insult kernel developers and make them all out to be cavemen, but perhaps some of the respectful dealings we are asking for should be directed that way too?

The Rust work has gone slowly — as did folios, which took over a year of work and endless discussions before anything was merged. Rust is in many ways a harder problem; how do you integrate a new language and new ways of doing things into a 30-year-old project that you cannot afford to break, and which needs to plan for the next 30 years as well? Many of the questions around Rust come down to that. See, for example, this article on filesystem APIs.

There is not much opposition to Rust in the kernel community — far less than I had expected, honestly. There is opposition to merging Rust code just because it's Rust, without ensuring that it fits well into the kernel and that the result will be maintainable over the long term. And so the Rust developers have run into the kinds of difficulties that many developers trying to make big changes have hit. It's hard, often harder than it really should be, but I do not believe it is driven by the factors you have described.

Linux-for-Rust or Rust-for-Linux

Posted Aug 29, 2024 21:47 UTC (Thu) by SLi (subscriber, #53131) [Link] (20 responses)

Agreed, this is a good take on it. I'm not a fan of either being overly critical or uncritical about the Linux way of doing things, even when "already decided". But certainly the people involved seem to be doing their best in good faith; and just the success of Linux implies that they're not doing it that wrong. (While I think it was wrong to stick to C that far, I think it's important for me to recognize that I also might exceptionally be wrong about something ;))

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 7:31 UTC (Fri) by vasvir (subscriber, #92389) [Link] (19 responses)

When Linus took its public stance against C++ I didn't like it either.

However in the long run I believe it was the right decision.

C++ brings a lot in the table but it still requires a huge amount of discipline. In every C++ project I have seen a C++ subset is deemed as blessed but that also requires discipline to enforce. Tests can help but in reality are also a discipline enforcement mechanism beyond the compiler.

If Linux had gone the C++ way it would face now the possibility of adding a 3rd language or a double migration to Rust. The premise is that with Rust discipline is not required, the compiler will flag out incorrect usage of APIs etc. There are big ifs here like:

* Are the rules you want to be disciplined about possible to be encoded in Rust?
* What is the cost (pain) of the maintenance of such rules? Does it worth it?

I believe this exercise of integrating Rust to the Linux will answer these questions.

Disclaimer: I do not know Rust

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 10:53 UTC (Fri) by khim (subscriber, #9252) [Link] (2 responses)

> I believe this exercise of integrating Rust to the Linux will answer these questions.

No, it wouldn't. Because these specific questions can be answered easily and in negative fashion and that is what Ted's possibly-sabotage is using as base.

> Disclaimer: I do not know Rust

But you, presumably, know some C and some assember, right? And you know that Unix was written in assembler and then rewritten in C, right?

Well… assembler makes it possible to do amazing tricks, not really replicable in C. E.g. GEOS used very peculiar yet efficient way of passing arguments: just put them right after jsr instruction. Yes, without any stack or registers, just there, in the middle of code.

It's very efficient: it's very easy to traverse the call stack with such approach and you don't even need to move arguments around! You just know that if function X have arguments x1, y1 and it calls Y, then Y calls Z, Z calls some DrawLine function… you can shift three stack frames up and find all these parameters. No need to explicitly pass them around and you save both code and execution time. Perfect… except it's entirely incompatible with how C deals with stack!

If your drivers ABI is designed around such “extra-efficient” calling conventions then rewrite in C is, essentially, not possible. Or at least it would be quite inefficient.

You may say that your assembler is just fine and these new guys can go to hell with their C… or you may actually cooperate and design some solution. Develop some secondary set of functions “for these C guys”. Or ask them to develop them… but then you need to cooperate with them when you change you API.

Transition to Rust is very similar. You may ask Rust guys to develop API, sure, and they may keep it in such with C API, but…

  1. You my need to alter certain aspects of C API to better fit Rust, or, when that's unacceptable…
  2. You would have to cooperate with them to keep their alternate API working when you refector C API

Doing what Ted is doing… to put it extra-polite as one redditer put it: that's behavior externally undistinguishable from purposeful sabotage.

As in: while different explanations are possible but any guy hell-bent on sabotaging Rust-on-Linux effort and who is maintainer of some subsystem would find it logical to do precisely what Ted did.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 11:37 UTC (Fri) by vasvir (subscriber, #92389) [Link] (1 responses)

> No, it wouldn't. Because these specific questions can be answered easily and in negative fashion

I am sure there are more ifs than the one I listed here. I mean it's a very big C project that tries to incorporate Rust. Lots of lessons will be drawn for all people that are unbiased. I agree though that the C side will have to make changes to accommodate Rust API. The question is: Will these changes make for a better C API? or more enforceable even with an external tool? I would suggest to grab some pop corn...

> But you, presumably, know some C and some assember, right? And you know that Unix was written in assembler and then rewritten in C, right?

Yes, yes and yes.

> It's very efficient: it's very easy to traverse the call stack with such approach and you don't even need to move arguments around!

Crazy stuff!!! and *no* I have never done that or even thought to do something like that. Didn't know about GEOS. Thanks for the pointer.

I saw the video and I would agree that Ted looked to me very rude. However, LWN coverage was more mild and his arguments came across as logical points which require some thought, work andcollaboration to address them.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 12:02 UTC (Fri) by khim (subscriber, #9252) [Link]

> The question is: Will these changes make for a better C API? or more enforceable even with an external tool?

90% of time the answer is “yes”: C API have grown to be incompatible with Rust approach simply because no one cared about Rust and when you fix it you make it easier to use from C, too.

90% of the remaining 10% you can approach some compromise where Rust API doesn't look too awful and C API is not too crazy, either.

But there are always 1-2% of these crazy corner cases where sane Rust API just pushes you to rewrite large pieces of already working code and then you either have to rewrite C code that is very fragile and rewrite-hostile, or, more often, add another implementation of the same API just for Rust.

It's always a good idea to try to avoid that, but sometimes that's really the best approach.

And if that happens then maintainership of that functionality becomes tangled between many different parties. And you spend way more than 1-2% of your time dealing with these corner cases.

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 3:36 UTC (Tue) by anthm (guest, #173221) [Link] (15 responses)

how does C not require a huge amount of discipline?

If the cognitive load and discipline required for you to write C is lower than Rust I would question how correct your C code is. Rust requires you to work with a more difficult language but it gives you guarantees, C requires managing all those guarantees manually. Correct C code is extremely hard to write.

This is why I *hate* the "C is simple so it's good" argument. C is not simple. C just hides the complexity behind a language that's under-engineered.

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 4:27 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

> how does C not require a huge amount of discipline?

C++ has a huge number of features. So if you're using a subset of features, there's always pressure to use just one more feature just outside of the subset. U

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 7:34 UTC (Tue) by viro (subscriber, #7872) [Link] (10 responses)

Funny... I've caught quite a few bugs in C; memory safety problems were nowhere near the majority. And one thing that is really required is being able to reason about the program behaviour, which is where the simpler model helps a _lot_. What's your experience, if that's not a too tactless question?

Don't get me wrong - memory safety issues do happen and automated tools are useful; what's more, C type system is genuinely not strong. But that's only a part of the picture. Far more often is the case when you need to change existing code (as opposed to "write a replacement from scratch and pray that CI will catch any logical problems") and _that_ is where the things get painful. Again, being able to reason is an absolute must-have; you _can't_ just treat the compiler as an oracle that needs to be appeased and be done with that. In any language.

And frankly, I don't understand the idio^Wpeople who treat language as identity - _any_ language (natural ones included) is a set of tools. It does not determine how you think, for fuck sake! You really can write FORTRAN in any language - certainly so in Rust. Familiarity with more languages is a good thing - it widens your toolkit, and if you really understand a language you will be able to come up with equivalent idioms in another - provided that you *do* understand them in the original, and not just on the level of "it's a magical incantation". Whorf-Sapir hypothesis is garbage; mixing it with Herderian nonsense is completely insane. "Rust Culture", "C Culture"... get a life, already.

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 12:39 UTC (Tue) by Wol (subscriber, #4433) [Link]

> You really can write FORTRAN in any language

rofl ...

Most of my C code was probably just FORTRAN and DataBASIC in disguise ... for the most part I just didn't see any advantage in using unfamiliar C idioms when Fortran did just as well - eg malloc a chunk of memory and access it with [] rather than *. If the C idiom was better than Fortran then I learnt it and used it, if I couldn't see an advantage I stuck with what I knew.

Cheers,
Wol

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 12:46 UTC (Tue) by Wol (subscriber, #4433) [Link] (7 responses)

> It does not determine how you think, for fuck sake!

Well, how do you think about tense, if your language does not have tenses? Given that you NEED language to enable you to think, the two are completely intertwined.

Your comment about "learning multiple languages" underlines this point! By learning another language, you learn that other people think different to you, and it broadens your mind.

But it's pretty obvious from - let's say khim's posts - that his language determines the way he thinks. He can't even express himself in English some times because his language does not have the context to enable him to grasp ours! I get the impression that many of my spats with him are down to the fact that the GRAMMAR gets in the way and obstructs mutual understanding.

Sorry khim - no disrespect - I'm sure you'd say pretty much the same about me from the other side of the fence :-)

But sorry, yes, language DOES determine how you think. If you don't have the words/grammar to say it, then you don't have the words/grammar to think it.

Cheers,
Wol

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 15:49 UTC (Tue) by viro (subscriber, #7872) [Link] (6 responses)

Are you serious? "event X happened before event Y" can be expressed in a lot of ways; surface representation matters very little. The same goes for aspect and mood. Do you really think that learning Latin changes the thought process when you see several buggers who are already visibly plastered and keep switching between port and beer without bothering to eat anything? Just because Latin expresses the notion of "that bunch is going to be throwing up" with a single word while English uses several? The underlying notion will be the same; it can be approximated with words (some of them possibly even printable), but that's not how we are thinking.

As for the words... that's why we have mechanisms to introduce new words and expressions. There's a particularly debased form of Whorf-Sapir bollocks (to be fair, neither had been responsible for that bit of BS - that came from popularizers) that seriously claims that inhabitants of Caribbean in 1490s must've been unable to process what they were seeing and somehow developed a blind spot in place of caravels. I don't know how one could believe that, because we all know what really happens when there's no terms for what we are seeing. "Oi, what the fuck is that thing over there? Hadn't been there this morning... Looks like there's a bunch of wankers on it; are they wrapped in some strange shit? Seems rigid; some kinda shields strapped on, maybe? Don't like how that one is pointing our way whatever the hell he's holding, let's get out of the open and warn the blokes back in the village" is far more likely than ignoring the sight just because there's no proper term for caravel, cuirass, etc. More likely to get passed to offspring, anyway... Mind does not stop when running into the lack of proper terms; it cobbles something up (that's what the obscenity is for, often enough) and keeps going.

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 16:44 UTC (Tue) by Wol (subscriber, #4433) [Link] (5 responses)

> Are you serious? "event X happened before event Y" can be expressed in a lot of ways; surface representation matters very little.

Then why is it so hard to understand what khim is actually saying? I learnt the rules, I (think I) have a basic grasp of Russian grammar, I try to see it through his comprehension, and he STILL succeeds in saying something that is clearly not what he meant.

It's like double negatives. I know we Brits abuse it a lot, but a lot of other languages use an emphasised negative. Translate that naively into English, and you've just COMPLETELY changed the meaning. And it's hard to unlearn something. I can hear you perfectly, and still walk away with a completely wrong understanding of what you said.

Surface representation matters a LOT. It's behind many misunderstandings and wars.

Roosevelt: Let's table that idea.
Churchill: Oh no we can't!

And that's only the difference between English and American ...

Cheers,
Wol

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 18:23 UTC (Tue) by viro (subscriber, #7872) [Link]

Umm... Thinking as internal monologue is already somewhat questionable, but thinking as internal _dialogue_ with participants using different dialects... Ouch. I mean, DID exists, but it's not exactly common. IOW, examples you are giving are irrelevant to the way one thinks - grammatical differences between dialects would matter only if you switched the dialects halfway through thinking of something *and* forgot which one have you been using at earlier step. Not impossible, I guess, but that's hard to run into without recreational chemistry, which is likely to cause other problems...

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 20:57 UTC (Tue) by rschroev (subscriber, #4164) [Link]

> > Are you serious? "event X happened before event Y" can be expressed in a lot of ways; surface representation matters very little.

> Then why is it so hard to understand what khim is actually saying?

Because English is a foreign language for khim? Of course it's more difficult to express yourself clearly (and/or succinctly) in a language that's not your native language. I can express my thoughts pretty fluently in Dutch; English is significantly harder, even though both languages are pretty closely related. French is almost hopeless even though our education system tried pretty hard making us learn that language. German is closer related still, but while I can read it a bit I can't write or speak it at all (other than trivial things like "Zwei Bier bitte"). If I try to express something in French, chances are French-speaking people will think it's difficult to understand too. And it's more difficult for things that are expressed differently in native versus foreign language.

How much study of Russian would you or I need before we are as proficient in it as khim is in English? Will khim wonder why he can't always understand our reasoning expressed in Russian?

I think you're thinking of the linguistic relativity, also known as the Sapir–Whorf hypothesis (and other terms), and more specifically it seems you're thinking of the strong form which says that language determines thought and that linguistic categories limit and restrict cognitive categories. Modern linguists don't accept that hypothesis anymore (or at least that stronger form).

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 21:04 UTC (Tue) by rschroev (subscriber, #4164) [Link] (2 responses)

> Surface representation matters a LOT. It's behind many misunderstandings and wars.

You can express repetition in different programming languages in different ways.

What you're saying is: The representation matters, because different compilers/interpreters/programmers will understand different ways of representation.

What viro was saying (or at the very least, how I interpreted it): The representation doesn't matter, in the sense that any programming language can express repetition, and the choice is a simple consequence of the choice of programming language. It doesn't influence what concepts can be expressed in the different languages.

Linux-for-Rust or Rust-for-Linux

Posted Sep 5, 2024 12:01 UTC (Thu) by taladar (subscriber, #68407) [Link] (1 responses)

The flaw in that line of reasoning is that how easily something is expressed matters a whole lot more in programming, maintainability, readability,... than how possible it is to express it at all.

Linux-for-Rust or Rust-for-Linux

Posted Nov 28, 2024 18:32 UTC (Thu) by deepfire (guest, #26138) [Link]

Thank you for saying the obvious!

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 18:53 UTC (Tue) by SLi (subscriber, #53131) [Link]

> Funny... I've caught quite a few bugs in C; memory safety problems were nowhere near the majority. And one thing that is really required is being able to reason about the program behaviour, which is where the simpler model helps a _lot_.

A serious question. Do you think you could reason equally well about high level behavior in C and assembly? Even if we ignore the memory safety aspect.

I find this idea surprising. In my experience, the language you work in significantly shapes your thinking.

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 8:25 UTC (Tue) by SLi (subscriber, #53131) [Link] (2 responses)

Because it's a simple language. It makes it easy to make mistakes in ways that can be only somewhat be mitigated by discipline. The set of generally useful rules is much smaller, and less controversial, than in C++.

And because you are unlikely to be tempted to write complex template machinery that nobody will ever quite understand, for the sinple reason that the language does not support anything that complex. That certainly also means you won't get the major benefits of templates. You can write macro hacks that are much more horrible, but there's no way to make them as insanely complex as modern C++ is (and I say this as someone who both loves and hates the language for that).

Linux-for-Rust or Rust-for-Linux

Posted Sep 5, 2024 12:04 UTC (Thu) by taladar (subscriber, #68407) [Link] (1 responses)

The "simplicity" also makes it incredibly hard to write abstractions that actually fully take care of a problem so you don't need to waste mental capacity on it every time it comes up.

Linux-for-Rust or Rust-for-Linux

Posted Sep 5, 2024 16:04 UTC (Thu) by SLi (subscriber, #53131) [Link]

Yes, this is a good way to put it. A lot of the template machinery in e.g. C++ is there precisely to support you in doing things safely. Sometimes it's a delicate balance between the maintainability of the machinery and usability of it.

Linux-for-Rust or Rust-for-Linux

Posted Aug 29, 2024 23:05 UTC (Thu) by gerdesj (subscriber, #5446) [Link] (1 responses)

Folios ... pah! What about the big kernel lock - https://kernelnewbies.org/BigKernelLock ?

Now, that was a project and a half. Conducted when Mr Shouty was at peak shouty. Mind you, LWN discussions were a lot more civil or perhaps my specs have gone a bit rose tinted.

Linux-for-Rust or Rust-for-Linux

Posted Sep 11, 2024 1:05 UTC (Wed) by sammythesnake (guest, #17693) [Link]

"Rose Tinted Spectacles are often that colour from blood spilled in those earlier times"

- Mark Twain*

* Not Mark Twain - I just made it up

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 5:10 UTC (Fri) by pbonzini (subscriber, #60935) [Link] (50 responses)

> There is opposition to merging Rust code just because it's Rust, without ensuring that it fits well into the kernel and that the result will be maintainable over the long term.

Looking at the linked article and discussion, I see objections of this kind and "C-side" developers participating into constructive discussions; but "because I don't know Rust, I am not going to fix the Rust bindings" is not one of them.

The moment rust/ was added to the Linux tree, anyone would have reasonably guessed that sooner or later all core kernel developers would have to either learn Rust, or collaborate with Rust for Linux developers whenever a tree-wide refactoring would happen.

The process of incorporating Rust bindings has been slow and sometimes tricky for several (mostly good) reasons, and when two teams with slightly different habits cross some attrition is expected (https://lwn.net/Articles/949270/). However, in the long term there is no sensible outcome other than people learning Rust, and in the long term people should realize that the ship has already sailed.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 11:50 UTC (Fri) by jgg (subscriber, #55211) [Link] (49 responses)

If the ship has already sailed then really we need someone like Linus & co to stand up and clearly say those words to everyone, and repeat it a few times.

I think there is a sizeable contingent that does not believe that to be the case, and Ted's remarks of effectively wanting nothing to do with rust are not unique or unreasonable given the cloudy situation. I know enough people betting on the failure of this experiment.

IMHO the current situation of Rust does not look like success. It is basically unusable except for unmerged toy projects and it is still not obvious when that will change. Can we rely on RH10 kernel having full baseline rust support? If yes then in 4 years maybe the server/enterprise industry could actually we can take rust seriously.. Can rust even support the OOT backporting stuff everyone does? Is there a strong enough preprocessor? What is the situation with Android? Can an Android device vendor write a driver in rust today?

I've pondered if I should consider rust for some of the greefield projects I've done recently. iommufd in rust? Generic PT? But at the end of the day that work is being "paid" for by people who intend to backport it to old kernels. I can't write it in rust and meet that need. I bet a lot of people are in the same boat.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 13:29 UTC (Fri) by pbonzini (subscriber, #60935) [Link]

I am not going to be at maintainers summit but that would be a very good occasion for that to happen.

With a killer use case there's no doubt to me that RHEL 10 would be able to support Rust code. Right now as you point out there's none (the only non-toy one in the air is binder), but RHEL 10 will grow new functionality until 2028 so I wouldn't exclude that, especially for drm.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 15:21 UTC (Fri) by willy (subscriber, #9762) [Link] (32 responses)

I don't think we're at the point of "complete success" until the compilers catch up. I don't think, for example, that you can compile Rust for m68k or alpha at this point. I'd be happy to be told I'm mistaken.

Obviously, this is not a failure of the RustForLinux project. They have important work to do which is independent of code generation.

My biggest gripe is inline functions. It seems crazy to me that we have functions which are so performance critical that they must be inlined in C, and yet in Rust, we make function calls to an exported symbol.

And I haven't got very far through my Rust book. I did write a little userspace program (calculating Pascal's triangle) which went well enough, but most of what I was doing there was figuring out how to use the libraries rather than the kinds of things I'll need to do in the kernel.

And, yeah, I think what Ted did there was reprehensible. I'd've said so had I been in the room, but I was off in the MM track at the time.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 15:47 UTC (Fri) by jgg (subscriber, #55211) [Link] (22 responses)

I was in the room and sort of thought Ted was "capturing the mood" - lets be blunt - there are some loud voices in FS/Block that are very anti-rust.

As above, I think the Linux project needs to make a clear unambigous decision, or at least set a timeline to make it.

Either we are doing Rust and what Ted said is out of line, or we are not, and Ted's position is reasonable - do not burden busy people with a Rust experiment that is nobody is going to use.

I thought the point of this inbetween state was supposed to be doing experiments to decide if Rust is feasible. Is there a result yet? Your concern about inlines is news to me, that sounds kind of fatal honestly. What was the result of the nvme experiment? I saw some graphs that it was worse performance, that's not encouraging?? How much of the toolchain situation is sorted out? Last LPC they were saying constant upgrades required? GCC was working on a front end?

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 17:55 UTC (Fri) by Wol (subscriber, #4433) [Link] (5 responses)

> Your concern about inlines is news to me, that sounds kind of fatal honestly.

I was under the impression that - compiling huge monolithic blobs - Rust was quite capable of spotting and inlining functions like that all by itself. What's the point of an "inline" keyword if you can rely on the compiler to spot it?

(Dunno whether I like the downsides of huge blobs - it makes libraries more complicated - but it's horses for courses, or pick you poison, whichever suits ...)

Cheers,
Wol

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 20:12 UTC (Fri) by pbonzini (subscriber, #60935) [Link] (4 responses)

These are cross-language calls, so the call from Rust to C (say, to spin_lock()) will not be inlined.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 22:55 UTC (Fri) by roc (subscriber, #30627) [Link] (2 responses)

Mozilla has been inlining across the Rust-C++ boundary, using LTO, for five years: https://www.reddit.com/r/cpp/comments/ch7g6n/mozilla_just...

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 5:38 UTC (Sat) by pbonzini (subscriber, #60935) [Link] (1 responses)

Linux however it's typically compiled with GCC. I remember people working on LTO for Linux a few years ago but I think that's not particularly common.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 6:16 UTC (Sat) by roc (subscriber, #30627) [Link]

OK, but the important thing is that there's an obvious fix for cross-language inlining if that becomes important.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 18:02 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

Even without LTO, a simple practical solution might be to duplicate some of the hottest code (e.g. spin_lock) in Rust. It's not great, but most of such code has been stable for quite a while.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 22:46 UTC (Fri) by kees (subscriber, #27264) [Link] (15 responses)

We are doing Rust and what Ted said is out of line. :)

Yes, there continues to be technical issues to be worked out. This is no different from C. It's the nature of technology. We're always moving to new compiler versions, refactoring to get rid of bad APIs, etc.

And Rust has shown its strengths very well, IMO. The M1 GPU driver is excellent and in constant use by a large set of distro users. The Binder driver is a drop-in replacement for the C version and will be shipped in Android soon.

I think the characterization of these Rust drivers being "toy projects" is pretty wildly inaccurate, if not outright insulting.

It's just a big change that is taking time to get everything resolved. I don't understand the resistance to learning new languages, especially given the large developer community associated with Rust. Us kernel devs are always hoping to get more people involved in Linux... Why push people away like this, especially given how many problems Rust permanently solves?

But to answer your questions:

Yes, Rust is feasible.

The NVMe driver works fine and the interfaces needed for it are going through review, though it's an uphill battle politically: as you said there are a few very loud voices in FS/Block that seem to be overwhelmed by their work loads.

The NVMe performance graphs looked very encouraging to me! A proof-of-concept driver with no explicit optimizations is identical to the C performance, except in 2 cases: one where it's within roughly 2%, and in another was _faster_.

What do you consider "the toolchain situation"? It works fine for me. :)

Yes, the upgrade cycle is faster than for GCC and Clang currently, but this is already slowing as the language features needed for Linux are stabilizing, so now there is a minimum version needed, not an exact version.

Yes, GCC has, I think, 2 front-end projects for Rust. I haven't paid too much attention to this myself, though.

GCC and Rust

Posted Aug 30, 2024 22:53 UTC (Fri) by corbet (editor, #1) [Link] (12 responses)

I'm hoping to learn more about the state of gccrs at Cauldron soon. That is one of my biggest concerns with this whole thing... GCC support is needed to reach all of the targets supported by the kernel, but the gccrs project seems to be languishing with little effort going into it. Somehow, I think, we have to find a way to bring some resources to bear on that problem.

GCC and Rust

Posted Aug 31, 2024 18:43 UTC (Sat) by josh (subscriber, #17465) [Link]

> GCC support is needed to reach all of the targets supported by the kernel

rustc_codegen_gcc is making steady progress.

(That's leaving aside the question of how much value those targets provide.)

GCC and Rust

Posted Sep 1, 2024 11:38 UTC (Sun) by ralfj (subscriber, #172874) [Link] (10 responses)

I think the focus should be on rustc_codegen_gcc: reusing the entire Rust frontend and middle-end, and just replacing the codegen backend. That has a much higher chance of delivering a compiler that is able to keep up with Rust's development and access GCC's backends.

gccrs is attempting an entirely independent second Rust implementation -- that's orders of magnitude more work than rustc_codegen_gcc, and I don't think there are significant benefits that would justify the cost. (This is not to say the gccrs devs should stop, if they're having fun doing what they do then by all means continue, but in terms of where to allocate resources and where to watch for medium-term results, I think rustc_codegen_gcc is clearly the better choice.)

rustc_codegen_gcc is unfortunately held back by GCC's reluctance to provide a nice library API for accessing its backends, but it seems using the libgccjit library works reasonably well.

Standardization - two independent implementations are good.

Posted Sep 2, 2024 15:12 UTC (Mon) by jjs (guest, #10315) [Link] (9 responses)

gccrs being a second, independent implementation is good. It ensures that the specification is, in fact, clear. I've seen many projects (HW & SW) where the specifications seemed clear to the writers who did an implementation, but someone else followed the specifications, made something that matched the specifications, yet it was not interoperable with the original version. This is the nature of language - there's lots of places where the meaning of a word is not a singular, universally agreed meaning (check any dictionary).

Law dictionaries exist to help ensure legal language is precise & unambiguous. There's a reason IETF requires two, independent implementations before declaring something a Internet Standard - https://www.ietf.org/participate/runningcode/implementati.... If two implementations don't produce the same product, it's time to go back and fine tune the specification to clarify the ambiguities that arise. And the only way to check for ambiguities is via an independent implementation.

Standardization - two independent implementations are good.

Posted Sep 2, 2024 18:43 UTC (Mon) by ralfj (subscriber, #172874) [Link] (8 responses)

> gccrs being a second, independent implementation is good. It ensures that the specification is, in fact, clear.

It may do that. Or it may cause endless issues due to differences in behavior between implementations, as is the case in C. One reason why the standard leaves so many things as "Undefined Behavior" is that implementations happened to implement different behavior, and none of them wanted to change. It's easy for them to agree to make things UB, the consequences are beard by programmers... just look at the entire debacle with realloc-of-size-0 now being UB: https://queue.acm.org/detail.cfm?id=3588242

I don't deny that multiple independent implementations have advantages. But they also have serious disadvantages. And given the resources required to build and maintain them, I am not convinced that it's worth it overall. The fact that language implementations are typically open-source these days has removed one of the biggest arguments in favor of multiple implementations.

Standardization - two independent implementations are good.

Posted Sep 2, 2024 22:09 UTC (Mon) by jjs (guest, #10315) [Link] (2 responses)

Yes, they can define the behavior as UB -which means they've changed the spec. If you have a spec with defined behavior, and two implementations have different behavior, but meet the spec, you really have two choices, IMO -
1. Follow what appears to be the C way - declare it UB in the spec. Also, what I understand from this article & other things I've read about Rust that the Rust community is trying to avoid.
2. Clarify the spec. Choose which behavior is correct (or a third way), and rewrite the spec to clarify it.

In either case, the spec is changed. I suppose a 3rd way is to ignore the problem, but, IMO that's worse.

"The fact that language implementations are typically open-source these days has removed one of the biggest arguments in favor of multiple implementations."

I'll argue the opposite - it's the language implementations being open source is one of the biggest arguments in favor of multiple implementations. Look at what went on with Linux and GCC/LLVM as LLVM began to work to compile the kernel. More defined behavior, from what I can tell. And a huge advantage of open source is everyone can contribute.

Standardization - two independent implementations are good.

Posted Sep 5, 2024 11:40 UTC (Thu) by taladar (subscriber, #68407) [Link]

The C way was to declare it undefined behavior in the spec because the committee of representatives from multiple implementations that already implemented things differently failed to find a consensus whose code should change, not because there is ever any advantage at all in having undefined parts in a spec.

Standardization - two independent implementations are good.

Posted Sep 5, 2024 14:28 UTC (Thu) by ralfj (subscriber, #172874) [Link]

> If you have a spec with defined behavior, and two implementations have different behavior, but meet the spec, you really have two choices, IMO

That's not what happened here. In this case, the C standard was unambiguous since at least C89: "If size is zero and ptr is not a null pointer, the object it points to is freed". Some implementations violated the standard, and somehow it was deemed better to introduce UB into tons of existing code than to fix the buggy implementations.

Such a hypothetical case could of course happen, though. IMO in that case you have a buggy (unintentionally underdefined) standard -- which happens and which needs to be dealt with reasonably well. If you have multiple different implementations of the standard, they are very hard to fix (other than by making the standard so weak that it encompasses all implementations), and that explains some (but not all) of the oddities in C. If you only have a single implementation, it is a lot easier to fix such bugs in the standard/specification by adjusting either the spec (to still have a *defined* behavior! just maybe not the one that we'd ideally have liked to see) or the implementation. These kinds of things happen in Rust fairly regularly. A big part of what makes this possible is that we have the ability to add "future compatibility" lints to Rust so that there's many months or even years of advance notice to all code that might be affected by a compiler change. I worry that with multiple implementations, this kind of language evolution will become even harder than it already is due to the added friction of having to coordinate this across implementations.

Standardization - two independent implementations are good.

Posted Sep 2, 2024 22:40 UTC (Mon) by viro (subscriber, #7872) [Link] (4 responses)

When specification is "whatever the interpreter actually does", you get wonders like sh(1). Which is _not_ a good language to write in...

Standardization - two independent implementations are good.

Posted Sep 5, 2024 11:41 UTC (Thu) by taladar (subscriber, #68407) [Link] (3 responses)

Which is why there is an effort to develop a Rust spec but that still doesn't require a second implementation, just a test suite that checks if the one implementation conforms to the spec.

2nd Implementation tests the meaning of the specification

Posted Sep 7, 2024 17:10 UTC (Sat) by jjs (guest, #10315) [Link] (2 responses)

That test suite can determine if one implementation meets what the spec writers interpret the spec to mean. It can't detect if the spec always means what the spec writers think it means (the wonders of human language). The purpose of a second implementation is to check that the wording of the spec actually only means what the spec writers think it means. I.e. catch unseen errors in the spec. Again, there's a reason IETF requires two independent implementations of an RFC before they declare it a standard.

2nd Implementation tests the meaning of the specification

Posted Sep 7, 2024 17:40 UTC (Sat) by intelfx (subscriber, #130118) [Link]

Or you can simply have one team write the compiler (and maybe the spec) and some other team to write the tests using the spec.

2nd Implementation tests the meaning of the specification

Posted Sep 8, 2024 12:02 UTC (Sun) by farnz (subscriber, #17727) [Link]

That's where two implementations of the test suite comes in handy, since you now have two separate groups of people who've read the specification and agree on what it means; where one test suite fails and the other passes, you need to resolve that by either fixing the specification, or getting the passing test suite to agree that they had a gap in test coverage.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 2:03 UTC (Sat) by sam_c (subscriber, #139836) [Link] (1 responses)

> Yes, the upgrade cycle is faster than for GCC and Clang currently, but this is already slowing as the language features needed for Linux are stabilizing, so now there is a minimum version needed, not an exact version.

I think faster is being generous. Nightly crates are still being used and the minimum Rust version is still extremely recent and being cranked up regularly. I think it's inevitable that Rust will be used in the kernel, but I find it hard to accept it's mature enough to merit being there when so many unstable features are needed.

Rust also doesn't, AFAIK, have any LTS versions for its compiler.

Linux-for-Rust or Rust-for-Linux

Posted Sep 2, 2024 11:55 UTC (Mon) by taladar (subscriber, #68407) [Link]

What benefits do you expect from an LTS version for a compiler where the most recent version is supposed to compile all code previous versions compiled? Sure, there might be the occasional bug compromising that goal but that can (and does) happen with backports to LTS versions too.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 23:05 UTC (Fri) by roc (subscriber, #30627) [Link] (7 responses)

> I don't think, for example, that you can compile Rust for m68k or alpha at this point.

Museum architectures should use museum kernels. It would be madness to let a few hobbyists veto kernel improvements that would benefit all other users.

But also, Rust does support m68k: https://doc.rust-lang.org/rustc/platform-support/m68k-unk...
And hopefully gccrs will make these complaints go away for good.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 12:01 UTC (Sat) by pizza (subscriber, #46) [Link] (5 responses)

> Museum architectures should use museum kernels. It would be madness to let a few hobbyists veto kernel improvements that would benefit all other users.

Except for the little detail that "museum architectures" (and the long tail of old drivers/filesystems/etc) are part of the mainline kernel.

Where do you draw the popularity line? Currently it's at "someone is actively maintaining it."

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 21:25 UTC (Sat) by roc (subscriber, #30627) [Link] (4 responses)

I'm actually paraphrasing Linus: https://lkml.iu.edu/hypermail/linux/kernel/2210.2/08845.html
> At some point, people have them as museum pieces. They might as well run museum kernels.

If new hardware hasn't been sold for 20 years then I think that's probably a good enough line.

Linux-for-Rust or Rust-for-Linux

Posted Sep 1, 2024 18:01 UTC (Sun) by willy (subscriber, #9762) [Link] (3 responses)

The problem is that you can still @#$&%^= buy them!

https://www.nxp.com/products/processors-and-microcontroll...

I'm disappointed, mostly because I worked on a PowerQUICC board back in 2000 and the fact that they are still selling the 68360 24 years later makes me very sad.

Linux-for-Rust or Rust-for-Linux

Posted Sep 2, 2024 7:25 UTC (Mon) by roc (subscriber, #30627) [Link] (2 responses)

Rust actually supports M68K so the real problem is if you could buy new Alpha chips. I don't think you've been able to do that for a long time.

Linux-for-Rust or Rust-for-Linux

Posted Sep 2, 2024 16:03 UTC (Mon) by Wol (subscriber, #4433) [Link] (1 responses)

As I understood it (I never used them) the M68K chips had a sane design, unlike the x86 ones. Maybe the reason you can still buy them is people value them for their simplicity and "easy to understand"ness - that can be worth a lot.

Cheers,
Wol

Linux-for-Rust or Rust-for-Linux

Posted Sep 2, 2024 19:12 UTC (Mon) by pbonzini (subscriber, #60935) [Link]

The current m68k chips (ColdFire) are a reduced and simplified version of the original instruction set. I doubt that a backwards compatible 680x0 with the addition of SIMD, 64-bit support, CFI, virtualization and whatnot (for example system-wide innovation such as multiprocessor and a fast superscalar microarchitecture) would be overall any more manageable than x86.

Linux-for-Rust or Rust-for-Linux

Posted Sep 1, 2024 11:40 UTC (Sun) by ralfj (subscriber, #172874) [Link]

> And hopefully gccrs will make these complaints go away for good.

Or, (in my view) more likely, rustc_codegen_gcc. :)

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 18:40 UTC (Sat) by josh (subscriber, #17465) [Link]

> I don't think, for example, that you can compile Rust for m68k or alpha at this point. I'd be happy to be told I'm mistaken.

https://doc.rust-lang.org/nightly/rustc/platform-support/...

It's still tier 3, but it exists.

As for alpha, if people still want to keep it alive, rustc_codegen_gcc will handle that eventually.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 21:09 UTC (Fri) by asahilina (subscriber, #166071) [Link] (14 responses)

> It is basically unusable except for unmerged toy projects and it is still not obvious when that will change.

I guess my Apple AGX GPU driver, which is the kernel side to the world's first and only OpenGL and Vulkan certified conformant driver for Apple Silicon GPUs, and also the FOSS community's first fully reverse engineered driver to achieve OpenGL 4.6 conformance, and which is used by thousands of Asahi Linux users in production, and that literally has never had a reported oops bug in production systems not caused by shared C code (unlike basically every other Linux GPU driver), is "an unmerged toy project".

Since you work for Nvidia, I'm sure you've heard of Nova, the up-and-coming Nouveau replacement driver that is also written in Rust using my Rust DRM abstractions. Is that also going to be "an unmeged toy project"?

This kind of demeaning of our work is why us Rust developers are getting very, very tired of the kernel community.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 22:41 UTC (Fri) by jgg (subscriber, #55211) [Link] (13 responses)

You should be very proud of what AGX has accomplished, it is amazing software, and an incredible piece of work. In fact everyone I've talked to about it has shared that view.

However, that doesn't change today's facts - AGX is currently unmerged and serves a tiny and niche user base with no commercial relavance. That is an unmerged toy by my definition.

There is nothing wrong at all with working on toy software. Linux itself started out as a toy, this is not an attempt to be demeaning.

The point, as pbonzini elaborated on, is a lack of "killer use case" to motivate RH to seriously turn on kernel Rust in RHEL10. AGX will not alter RH's plans.

Nova is barely started, let's wait a few years to see what impact it has. I'm optimistic that a completed Nova would convince several distros to turn on kernel Rust support. I was actually thinking primarily about the Rust NVMe driver.

Unmerged toy

Posted Aug 30, 2024 22:48 UTC (Fri) by corbet (editor, #1) [Link] (3 responses)

Honestly, "unmerged toy" seems like an unnecessarily dismissive term for something like this. How about "out-of-tree useful driver" - a term that we could apply to things like fwctl as well, perhaps :)

The story of why it is unmerged is something I've never quite managed to dig into, but would like to.

Unmerged toy

Posted Aug 31, 2024 20:32 UTC (Sat) by jgg (subscriber, #55211) [Link] (2 responses)

That's for the feedback John. It is admittedly hard to know where colourful language stops being entertaining and people are offended. For instance down below someone was calling m68k/etc a "museum architecture" which is a phrase I've seen many times before. It seems like a popular and accurate term to describe it, yet I would also think that is dismissive to the consistent work Geert and others put in.

Unmerged toy

Posted Sep 2, 2024 7:34 UTC (Mon) by roc (subscriber, #30627) [Link] (1 responses)

Feel free to popularize a different term to describe such architectures. We do need a term for the situation where the effort of supporting an architecture (across the entire project, so including the costs of vetoing changes that would benefit other architectures) exceeds the practical benefits of being able to run the latest kernels on machines of that architecture.

Unmerged toy

Posted Sep 6, 2024 7:36 UTC (Fri) by da4089 (subscriber, #1195) [Link]

> Feel free to popularize a different term to describe such architectures.

"Heritage" is the usual respectful euphemism, I think?

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 22:49 UTC (Fri) by Ashton (guest, #158330) [Link]

Boy, the existing contributor base of Linux is not covering itself in glory today.

“Toy project”? Can you please try and not be so petty and rude?

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 23:56 UTC (Fri) by airlied (subscriber, #9104) [Link] (2 responses)

Throwing out toy without defining what you mean(even with a definition, it's still not a great word choice), is not exactly a great look here.

Like do we consider the open-gpu-kernel driver from NVIDIA a toy because it isn't upstream?

Asahi is not a driver on the enterprise radar, it won't make RHEL sit up and notice but in does that make it a toy.

I count asahi as a very successful fact finding mission, that in the end is very hard to upstream in a reasonable manner. It's why nova is approaching this from the other end, and building a driver upstream, where we fix the interactions with other subsystems in order as we go, building the ecosystem upstream rather than having it done in a private fork.

The main current focus is driver model and Greg at the moment, next after that will probbaly be getting pci, platform and drm device bindings into shape, KMS modesetting (which Asahi didn't have to tackle), and then the actual nova project.

I've already written a rust implementation that talks to NVIDIA's GSP firmwares and encapsulate the unstable ABI in a similiar form to the Asahi work, and the advantages of this over a C project to do the same are immense. Like night and day difference in how much code had to be written.

I think Linus has said at last year maintainers summit that he was supportive of this, and he thinks it will happen, I think if people start acting as active roadblocks to work, rather than sideline commentators who we can ignore, then I will ask Linus to step in and remove roadblocks, but so far we haven't faced actual problems that education and patience can't solve.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 9:21 UTC (Sat) by Wol (subscriber, #4433) [Link] (1 responses)

> The main current focus is driver model and Greg at the moment, next after that will probbaly be getting pci, platform and drm device bindings into shape, KMS modesetting (which Asahi didn't have to tackle), and th

Given that is at least one (Christoff Hellwig) big developer (you make it sound like two - Greg KH) is throwing a lot of effort into cleaning up the kernel, sounds like they need to be brought on-side if they aren't already.

I'm picking up a lot about Rust-kernel bindings, and how the end result is much cleaner for both the C and Rust sides. So even if the resulting Rust work isn't merged, actually the effort spent creating the Rust interface would be a great help to both of them.

So you now have clean Rust interfaces for anybody who is interested ... and any "I'm not learning Rust" developers who tamper with those interfaces will get shouted at "don't you dare create any (C) bugs that this would have automatically caught!"

Cheers,
Wol

Two implementations - another benefit

Posted Sep 6, 2024 11:46 UTC (Fri) by jjs (guest, #10315) [Link]

Seen it in other projects. You get something that works. Someone else builds a clean implementation - and you start seeing the cleanup in the design/specs as you make the two compatible. Which, in the long term, helps both teams.

Not familiar with Rust, but it sounds like the efforts are having good benefits even without Rust being fully integrated.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 3:30 UTC (Sat) by asahilina (subscriber, #166071) [Link] (4 responses)

> The point, as pbonzini elaborated on, is a lack of "killer use case" to motivate RH to seriously turn on kernel Rust in RHEL10. AGX will not alter RH's plans.

You are aware that the AGX driver is in fact the reason why Fedora is turning on Rust support in upstream kernels, right? I'm pretty sure that is doing more to push RHEL to eventually do the same than anything else, today.

https://gitlab.com/cki-project/kernel-ark/-/merge_request...

Neal is part of the Fedora Asahi SIG.

(Won't comment on your insistence on the "toy" designation since other replies have already done so.)

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 7:38 UTC (Sat) by airlied (subscriber, #9104) [Link] (3 responses)

Red Hat will not turn on rust prior to nova if I had to guess, I'm not seeing any other motivator, I don't see any other motivator.

Don't confuse Fedora, CentOS or ARK with Red Hat here. RHEL is the boss level here, but I don't really care about that, I only care about getting things upstream lined up.

I think a lot of the complaints about rust will evolve away once there is an interesting in-tree consumer, toolchain versions will stabilise, better toolchain support for things upstream needs etc

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 11:11 UTC (Sat) by Conan_Kudo (subscriber, #103240) [Link] (2 responses)

No. Rust is getting turned on because drm_panic is written in Rust. I started working on this for enablement because of AGX, but we need it turned on ASAP because drm_panic is approved for Fedora Linux 42.

Nova is on literally nobody's radar right now because it doesn't exist beyond scaffold. There is no code that does anything yet, to the best of my knowledge, and there will not be any for a long while.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 12:10 UTC (Sat) by airlied (subscriber, #9104) [Link] (1 responses)

DRM panic in Fedora is not a Red Hat commitment to rust in RHEL. I think directly said not to confuse Fedora and Red Hat here.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 12:31 UTC (Sat) by Conan_Kudo (subscriber, #103240) [Link]

Based on the conversations I've had with the RHEL kernel team so far, the main blocker for RHEL is the lack of modversions support for Rust, which is being worked on. I do think it'll get enabled before Nova is in a useful state, because there are other little drivers in-tree where there are C and Rust versions and the Rust versions are better than the C versions.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 10:55 UTC (Fri) by Deleted user 129183 (guest, #129183) [Link] (6 responses)

> There is opposition to merging Rust code just because it's Rust

Which is completely opposite to the predominating attitude of Rustbros, who think that their code is already superior just because it is In Rust. But no, it still has to be up to the standards, regardless of the programming language used.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 13:37 UTC (Fri) by corbet (editor, #1) [Link] (4 responses)

This attitude actually doesn't help either. I don't think we have a problem with "Rustbros" in the kernel community. The Rust-for-Linux developers are doing their best to work within the kernel process and have been making progress, even if slower than they would like. Tossing insults around doesn't help, maybe we can stop that?

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 19:58 UTC (Fri) by Deleted user 129183 (guest, #129183) [Link] (3 responses)

> I don't think we have a problem with "Rustbros" in the kernel community.

So those people with the expectation of ‘merging Rust code just because it's Rust, without ensuring that it fits well into the kernel’ are outside of the ‘kernel community’? Sorry, I’m confused now. But anyway, even if there are no such people now, they will definitely come later and make a mess. Because the Rust culture is broken, for example few days ago, we’ve had a guy in the LWN comments calling another person a ‘bigot’ and ‘Nazi’ just because the latter person wanted to see less Rust content here. And this is not a sign of a healthy attitude inside the Rust community.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 20:38 UTC (Fri) by viro (subscriber, #7872) [Link]

Advocates != developers. I'm not particularly fond of Rust (and IMO the notion of "$LANGUAGE Culture" is among the more ridiculous variations of Whorf-Sapir bollocks, anyway), but confusing Rust advocates with Rust developers is fundamentally unfair.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 21:18 UTC (Fri) by pbonzini (subscriber, #60935) [Link]

> Because the Rust culture is broken, for example few days ago, we’ve had a guy in the LWN comments calling another person a ‘bigot’ and ‘Nazi’ just because the latter person wanted to see less Rust content here. And this is not a sign of a healthy attitude inside the Rust community.

I am not sure why you'd think that he's a member of the Rust community as opposed to a bystander. And the attitude of Rust-for-Linux developers has always been all but professional and respectful. They absolutely don't want their code merged just because it's Rust, though of course they believe the language to be superior in certain ways, especially in how it encodes safe semantics as part of the types.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 22:36 UTC (Fri) by intelfx (subscriber, #130118) [Link]

> we’ve had a guy in the LWN comments calling another person a ‘bigot’ and ‘Nazi’ just because the latter person wanted to see less Rust content here

No, it was not not because "he wanted to see less Rust content here", but because of his shitty and entitled attitude towards LWN editors and audience. And I'm fairly sure you do understand that.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 16:53 UTC (Fri) by viro (subscriber, #7872) [Link]

Objection: you are using the wrong pronoun in there. In that kind of relative clause 'their' refers to the antecedent. IOW, your syntax conflates the authors of (hypothetical) code with the advocates claiming that code to be superior for no better reason than the language it might use. IMO insults of that magnitude ought to be spelled out explicitly, _especially_ considering the technics used by the specimen (atnot) who'd started that subthread.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 12:21 UTC (Fri) by b7j0c (guest, #27559) [Link] (63 responses)

> So I know it is fun to insult kernel developers and make them all out to be cavemen

cavemen make the world go round

nothing Ted said in the related video snippet is remotely problematic; he doesn't raise his voice, he doesn't insult anyone...he just states his position directly.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 13:22 UTC (Fri) by mb (subscriber, #50428) [Link] (62 responses)

Yes. His *position* is what is problematic.

And that does not surprise me at all.

He doesn't think it's a big problem that crafted filesystem images can cause Undefined Behavior via out of bounds accesses in the kernel.
So I'm not surprised that he refuses to learn Rust.
That's really the same thinking. Let's focus on making the existing C code better instead.

Well. We have tried that for *decades*. And we all failed miserably at it.

The biggest step with accepting a transition to Rust is accepting the failure of C first.
That is a social problem.

It's natural that people defend things they had been doing for decades.

And besides that, stating that "you will not force me to learn Rust" in a talk about Rust *is* insulting. Nobody is forcing anybody.

Refactoring of such cross language interfaces is no different than refactoring cross subsystem interfaces.
If you want to do that, get help from the developers of the other side!
Yes, you need to lean about the other side *OR* ask the people on the other side for help.

There will always be situations where a big refactoring hits areas unknown to the developer doing it.
That is completely unrelated to the language.
It's not a technical problem. It's a social problem. Ask for help.

I'm sure the Rust people would not refuse to work together with Ted to find a good solution.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 20:57 UTC (Fri) by jgg (subscriber, #55211) [Link] (60 responses)

> And besides that, stating that "you will not force me to learn Rust" in a talk about Rust *is* insulting. Nobody is forcing anybody.

The logical end game here is in 10 years the kernel will have a mixture of Rust and C. If you want to be a senior kernel contributor you will need to be a master of both. The idea you can work on the core kernel and be unable to touch large swaths of it is not realistic. People need to be honest about this! The idea that there will always be rust helping hands willing to do whatever the historical C maintainer wants is not reasonable, that might exist for a few years but not over the long haul.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 21:12 UTC (Fri) by johill (subscriber, #25196) [Link] (59 responses)

Maybe it's all just trying to go too fast?

I for one would really like to learn and use rust e.g. for cfg80211's scan handling. That code is awfully complex, with all the lists for hidden, multi-bssid, etc. cases, and I'd think it might be possible to express certain restrictions etc. in rust to make working on it safer.
(Actually we know it has bugs, we just don't know why, and where they are.)
But I can't do that, because I can't well break wifi for everyone on platforms rust doesn't support.

That also means rust currently remains restricted to something that need not work on all platforms, like certain (sometimes duplicate?) drivers, or drivers for special hardware that's only available on some platforms (like the graphics driver people keep talking about).

So I'm probably not going to learn it any time soon unless I change jobs, because realistically there isn't much _else_ I (want to) work on that would warrant learning rust, and I don't have time to RIIR something.

So while I don't necessarily think Ted is right about just breaking rust stuff left and right, there currently isn't even a large opening where as a kernel developer concerned with pretty much _all_ architectures (ok s390 doesn't have wifi, I think) can really even start thinking about it.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 22:10 UTC (Fri) by mb (subscriber, #50428) [Link] (35 responses)

Yes, it doesn't make sense to integrate Rust into core kernel code that other C code depends on, yet.
But I think Ted's argument came from a slightly different direction.

Let's suppose mac80211 had a Rust driver interface.
That would be a totally valid and good thing to have today.

It would mean that mac80211 maintainers would either have to know Rust (good enough) to maintain that interface themselves or talk to somebody who knows.

I don't see a problem either way.
This is not rocket science.

To me this is a tempest in a teacup.

Learning Rust is not a multi-month approach. Not even multi-week if done seriously.
And most of the learnt things can even be applied to C and C++ programs.
And the best thing is that people get their invested time back from the debugger by being able to write less buggy code.

WHAT?

Posted Aug 31, 2024 0:21 UTC (Sat) by rc00 (guest, #164740) [Link] (32 responses)

> Learning Rust is not a multi-month approach. Not even multi-week if done seriously.

This is a bold-faced lie. An experienced engineer won't learn Rust in under a year. Someone with the talent level of the Primeagen took two years to learn Rust and he is arguably better than the average. The language and their proponents literally pride themselves on the complexity. Do you already forget the "skill issues" toxic posts that sprung up? It was quite similar to the crypto bros and their "ngmi" nonsense.

At best, a senior programmer could reach an *intermediate* level for Rust. Ownership, borrowing, immutable variables by default (the name is an oxymoron), lifetimes (and their annotation syntax), the complex type system, traits, error handling, concurrency/async (`Arc`, `Mutex`, `RwLock`), procedural macros, and unsafe Rust. How do you navigate that list and come away with "not even multi-week"? Anything beyond helloworld.rs is at best a multi-month affair and the inexperience and/or below average programmers are easily multi-year range.

WHAT?

Posted Aug 31, 2024 0:38 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (8 responses)

> An experienced engineer won't learn Rust in under a year.

This particular engineer learned Rust in about 2 weeks back around 2018 (IIRC). If you know C++, then it's a fairly straightforward process.

WHAT?

Posted Aug 31, 2024 0:44 UTC (Sat) by rc00 (guest, #164740) [Link] (7 responses)

> This particular engineer learned Rust in about 2 weeks back around 2018 (IIRC). If you know C++, then it's a fairly straightforward process.

Are you suggesting that a sample size of one that can't be corroborated is statistically or otherwise relevant?

Professors I engage with don't even think the language could be taught in a full academic year to upperclassmen.

Which sample size is more relevant when extrapolating to the general programmer?

WHAT?

Posted Aug 31, 2024 0:59 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

We're not talking about general programmers, but about kernel developers, who have to intimately understand all the low-level complexities of programming. I just don't buy that it's going to take them significant time to learn Rust on a good enough level.

WHAT?

Posted Aug 31, 2024 11:49 UTC (Sat) by pizza (subscriber, #46) [Link] (2 responses)

> We're not talking about general programmers, but about kernel developers, who have to intimately understand all the low-level complexities of programming. I just don't buy that it's going to take them significant time to learn Rust on a good enough level.

Sure, except for the minor detail that "good enough level" _currently_ means "expert level in bleeding-edge Rust" on top of already being an expert in kernel-C and one or more subsystems. That isn't something you just pick up in a few weeks, even if you had no other demands of your time.

WHAT?

Posted Aug 31, 2024 18:07 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

> Sure, except for the minor detail that "good enough level" _currently_ means "expert level in bleeding-edge Rust"

You don't need to be a bleeding-edge expert in Rust to program for Linux. Just as you don't need to be Paul McKenney level of parallel-savvy to use the RCU mechanism.

WHAT?

Posted Sep 2, 2024 12:10 UTC (Mon) by taladar (subscriber, #68407) [Link]

You also don't need to be an expert in a language to fix some bindings you already changed and semantically understand in another language. Most of those changes are just things like an added field of a type that is already in use elsewhere or some extra parameter to some function, not a complete re-evaluation of every invariant in the system and how it is affected by advanced optimizations on obscure platforms.

WHAT?

Posted Aug 31, 2024 1:32 UTC (Sat) by roc (subscriber, #30627) [Link] (1 responses)

What it means to "teach a language" is quite ambiguous.

When you "teach" C and C++ to undergraduates they don't learn the full language and they don't learn to write reliable code. They just learn enough to get code that runs and which is likely chock-full of undefined behavior unless you run sanitizers as part of your grading process.

Teaching Rust would be harder, I agree, but the resulting code will be a lot more robust and I think the students will have actually learned a lot more and will probably be better programmers as a result, even if they end up using C++. Just like learning Hoare logic makes you a better programmer even if you never use it.

WHAT?

Posted Aug 31, 2024 2:30 UTC (Sat) by roc (subscriber, #30627) [Link]

Actually an interesting exercise would be to try to teach undergraduates to write a significant chunk of code for the kernel, some in C and some in Rust, up to the standards the kernel aspires to --- something you can throw fuzzers and sanitizers at. If the fuzzing and sanitizing finds a bug, don't tell them what the bug was, just tell them to try again.

I would bet good money that it is much easier to get the Rust group over this bar than the C group.

WHAT?

Posted Aug 31, 2024 1:40 UTC (Sat) by mussell (subscriber, #170320) [Link]

And how many of those professors have tried teaching Rust? From the way you're phrasing things, it sounds like 0.

Besides professors are probably the worst people to ask as, from my personal experience, they only tend to teach what that have been taught themselves. The C++ course at my university doesn't teach smart pointers, or really anything after C++98 since the prof never had to use them herself. Professors also don't tend to help out students struggling to debug their C code, nor do they mark the final code, that is of course well below their pay grade and is the job of humble student TAs.

And if you want to know just how well professors are teaching C today, go run `git log -i --grep='null pointer deref' --no-merges v6.10..HEAD` in your kernel source tree.

WHAT?

Posted Aug 31, 2024 1:23 UTC (Sat) by roc (subscriber, #30627) [Link] (9 responses)

In 2016, starting with a strong C++ background and some knowledge of the ideas of Rust, but never having written any Rust code, I was productive with Rust in under a week. Looking back at the code I wrote then, it's still fine. I didn't need to know concurrency, async, macros or unsafe right away (async didn't even exist). A few weeks in I started messing with threads, but it was months before I had to use unsafe and years before I wrote a proc-macro or used async.

The situation should be similar now; you're more likely to encounter 'async' early, but OTOH across the board the tools are better (rust-analyzer!) and LLMs can be helpful.

WHAT?

Posted Aug 31, 2024 2:24 UTC (Sat) by rc00 (guest, #164740) [Link] (2 responses)

And yet, in the same timeframe of a few months, I can fit the majority, if not the entirety, of Go, Odin, or Zig in my head. As can many more programmers than those that could do the same with Rust or even C++. I would rather spend my programming time debugging my application and not debugging my knowledge of the language. By your own admission, years into writing Rust, you were still debugging your knowledge of the language. I would consider that counterproductive.

I want to dive into this notion of "productive" and other philosophical tangents another time and another place but I want to leave this anchor out there:

How can you be productive in a language while at the same time, avoiding most of the features? Is that "productive"? Or is that, "I learned conditionals, assignment, and control flow" which is fundamental to operating in any language? It wouldn't take long to be "productive" in any sane language using those criteria and remaining within those confines. At what point can you say you've actually used a language? Can we measure this by percentage of the language that has actually been used? (This would clearly bias smaller languages like Lua but still.)

WHAT?

Posted Aug 31, 2024 2:40 UTC (Sat) by roc (subscriber, #30627) [Link]

> By your own admission, years into writing Rust, you were still debugging your knowledge of the language. I would consider that counterproductive.
> How can you be productive in a language while at the same time, avoiding most of the features?

I didn't avoid "most" of the features. I didn't deliberately avoid any features. The reality is that for most programming tasks you don't need to write "unsafe" or macros. Of course you will use libraries that use those features, and in particular you will use #[derive] to apply proc-macros to your code, but that doesn't require you to understand them.

I do think there is value in a small language that you can thoroughly learn --- that's why C++ is such a disaster and getting steadily worse. The main reason is that when you dig into an existing project you want to be able to understand their code and they might be using any features of the language. But this is a different kind of problem --- e.g. you generally don't need to understand much about how type inference or borrow checking works to understand someone else's code. And for code understanding, the strong isolation that Rust generics give you compared to Zig's comptime is actually in Rust's favour.

WHAT?

Posted Sep 1, 2024 11:53 UTC (Sun) by ralfj (subscriber, #172874) [Link]

You might enjoy this talk https://www.youtube.com/watch?v=QrrH2lcl9ew by a Google employee reporting that they found in-boarding with Rust to be no harder than with Kotlin (around the 10 minute mark).

This is the result of an actual study they did, not just some anecdotes about professors you met. (And FWIW, as a professor, I think it is *much* easier to teach people correct concurrent programming with Rust than with C.)

WHAT?

Posted Aug 31, 2024 3:30 UTC (Sat) by mikebenden (guest, #74702) [Link] (5 responses)

> starting with a strong C++ background and some knowledge of the ideas of Rust, but never having written any Rust code, I was productive with Rust in under a week

You're the second data point (along with Cyberax elsewhere in the comments) who brings up "strong C++ background" as a precondition for "easily picking up Rust". That does make a lot of sense, actually.

As a C programmer with a good working familiarity on kernel and low level topics, I see both C++ and Rust as something that would take a LOT of getting used to, they are both heavily resistant to casual lecture of sources one didn't spend a lot of time and thought writing themselves :)

I don't see "know c++ well" as an obvious required skill for a kernel hacker, so the point that kernel hackers in general would all have to take lengthy detours in order to pick up Rust still stands (unless they're already C++ experts, which would help, but presuming they are is unreasonable, because they're *kernel* hackers, not high-level code monkeys :)

WHAT?

Posted Aug 31, 2024 4:17 UTC (Sat) by roc (subscriber, #30627) [Link]

You don't need to know C++ well to understand Rust. There is a huge amount of C++ machinery that is irrelevant to Rust (e.g. inheritance, overloading, advanced templates (SFINAE), exceptions). OTOH some things do carry over like move semantics (much simpler in Rust though), smart pointers, RAII, abstract base classes (similar to Rust traits), and template member types (Rust associated types).

Then there are other concepts like ownership and lifetimes that are central to Rust that actually exist in C++ (and C) but the language doesn't let you write them down.

So it's not so much that I'm a strong C++ programmer but that I've spent a lot of time working with large C and C++ programs, which means I already had to internalize many of the concepts of Rust and also appreciate its benefits.

WHAT?

Posted Aug 31, 2024 4:30 UTC (Sat) by roc (subscriber, #30627) [Link] (1 responses)

> so the point that kernel hackers in general would all have to take lengthy detours in order to pick up Rust still stands

The thing is, most of what people fear about Rust is already stuff you have to think about in C in the kernel, you just can't write it down. You have to think about ownership (who is responsible for freeing each thing in memory) and how that is transferred. You have to think about borrowed references (i.e. pointers) and lifetimes. You have to think about whether the data you're referencing is immutable or could change behind your back. Rust traits are very much like the manual vtables of file_operations etc. Rust's Result just codifies the distinction between normal results and error result. Rust's Option just lets you write down "this could be null".

I think probably the biggest issue is going to be Rust's generic type system. It's easy to understand how Rust types like Vec<T> are similar to void*-based generic data structures in C (e.g. radix-tree), but in practice there's a lot more value and it takes a bit of experience to use well, especially when you're designing APIs.

Learning Rust vs C

Posted Aug 31, 2024 14:28 UTC (Sat) by kleptog (subscriber, #1183) [Link]

Anecdotally, for the junior developers I've had to help, Rust was way easier than C. C was just giving them segfaults all the time, because lifetimes and mutability are things you have to care about but C doesn't help you. When they were writing Rust however, the compiler basically trained them to think about these things, and how to structure programs while keeping object lifetimes in mind.

The end result is that after this experience with Rust, they can actually do C ok because they know what they need to look out for.

Why not C++? Well, because they needed to write things that worked with various C libraries, and C++ doesn't help you at all there, unless someone has written a C++ wrapper for those libraries. Fortunately, someone had written Rust wrappers for these libraries and cargo managed all the dependencies and installation. Managing C++ library dependencies is disaster in itself.

WHAT?

Posted Sep 1, 2024 1:49 UTC (Sun) by roc (subscriber, #30627) [Link] (1 responses)

Another datapoint, my son (second year university student) has been learning Rust and was able to produce useful code within a couple of weeks of work. He's smart but has very little C or C++ background. I answered a couple of his questions but didn't hold his hand. https://github.com/scrying-circle/nofi/blob/main/rust/src... Not the greatest Rust code, but not too bad. He felt the initial learning curve was steep but he feels like he's already surmounted it.

Learning Rust versus learning C

Posted Sep 1, 2024 9:29 UTC (Sun) by farnz (subscriber, #17727) [Link]

Ancedotally, one of the biggest things I see beginner programmers get wrong is the difference between "this code is correct" and "this code works for the test cases I give it on my machine". In this regard, C is a really painful language for beginners, because you can write code that is semantically meaningless C (IFNDR, UB etc), but that happens to work for the test cases the beginner gives it on their machine.

A language that enforces more rules is, paradoxically, easier for a beginner, because the enforcement means that the code does not work for the test cases they give it on their machine - it does not compile and run on their machine, so they have to fix it.

WHAT?

Posted Aug 31, 2024 5:29 UTC (Sat) by mb (subscriber, #50428) [Link] (6 responses)

>This is a bold-faced lie.

Welcome to my killfile.

WHAT?

Posted Sep 9, 2024 23:22 UTC (Mon) by rc00 (guest, #164740) [Link] (5 responses)

Editor, last Thursday, Go released two security updates and Clojure also released a major update. The only programming language release covered by LWN that day was Rust.

Combine that with idiotic echoes like the one above and you have the exit survey for why I am likely not to renew.

This isn't an indictment of you, your team, or your choices in coverage but it is clear that I am not your target audience. Best of luck.

WHAT?

Posted Sep 10, 2024 17:36 UTC (Tue) by rbtree (guest, #129790) [Link] (2 responses)

OK, I'm not the editor, but...

> Go ... and Clojure

Neither one is a good choice for system-level programming, nor a potential replacement for C in the most significant project this website is about ⇒ there's not much reason for LWN (formerly Linux Weekly News) to cover them.

Although they did cover PHP for a few months a couple of years ago, it seems there was not enough interest among readers to continue.

> idiotic echoes like the one above

You may have misunderstood mb. He blacklisted your future comments because the tone of that phrase is not appreciated; that is all. I see nothing "idiotic" in his comment.

(And yes, I did support LWN while I could, and would still be doing that if not for mindless carpet-bombing performed by certain individuals completely unrelated to this site. No hypocrisy on my end.)

WHAT?

Posted Sep 10, 2024 18:57 UTC (Tue) by rc00 (guest, #164740) [Link] (1 responses)

I didn't intend for this to become a long thread so I don't know how much more I'll reply but here goes.

> there's not much reason for LWN (formerly Linux Weekly News) to cover them.

There doesn't appear to be a clearly defined charter here and that's part of the issue. You've chosen to scope it to just systems-level programming languages despite the fact that the Linux ecosystem makes use of many higher-level and lower-level languages. Worse yet, there is plenty of coverage on this site for a programming language like Python so even the scope you've opted for is wildly off-base.

> it seems there was not enough interest among readers to continue.

And this is the modern challenge that content creation/distribution channels have to face. You either pander to a niche but terminally online vocal minority or you stick to some predefined ethos. The former strategy is a means for survival. The latter strategy means possibly ending up like AnandTech. I don't have the answer or any recommendations here. On my end, my attention and support are finite resources that I would rather focus where I deem they would have the most value.

> I see nothing "idiotic" in his comment.

Inane? Unsubstantial? Witless? We're just going to have to agree to disagree on this one. Maybe it's a problem on my end? I find myself having a shorter and shorter fuse with the pro-Rust crowd. Somehow, they managed to take the worst parts of the Apple fanboys and the Pythonista/Scala/Haskell hype phases and then exponentiated them into something far more toxic and all around worse. I cast aside any benefit of the doubt years ago. At this point, you can call it bias if you want. I would call it a defense mechanism, not unlike one I've developed for the crypto space. When something is off and off-putting, cutting through the chaff is a more than rational response to develop over time.

This ended up being much more than I intended to write so apologies are in order. I also can't see myself continuing this thread too much longer so apologies in advance for not carrying on with this as well.

WHAT?

Posted Sep 10, 2024 20:37 UTC (Tue) by Wol (subscriber, #4433) [Link]

> There doesn't appear to be a clearly defined charter here and that's part of the issue. You've chosen to scope it to just systems-level programming languages despite the fact that the Linux ecosystem makes use of many higher-level and lower-level languages. Worse yet, there is plenty of coverage on this site for a programming language like Python so even the scope you've opted for is wildly off-base.

It's Jon's site. He's part of the core linux kernel team (inasmuch as there is such a team). It's down to him what he cares to put in.

And that's why it's such a good site. I go on about PJ, but the charter for Groklaw was similar - if she wasn't happy with it, it got deleted. That's why it was such a damn good site.

Cheers,
Wol

WHAT?

Posted Sep 10, 2024 19:09 UTC (Tue) by intelfx (subscriber, #130118) [Link] (1 responses)

> why I am likely not to renew

Is this the point where we are supposed to bow our heads in shame?

Recalling some of your comments on the recent Ladybird article... I'd say good riddance.

WHAT?

Posted Sep 10, 2024 19:21 UTC (Tue) by corbet (editor, #1) [Link]

Regardless of the goodness of the riddance, this seems like a good place to stop this subthread, please.

WHAT?

Posted Aug 31, 2024 10:06 UTC (Sat) by hunger (subscriber, #36242) [Link] (1 responses)

Google collected some numbers converting dev teams from to Rust.

They claimed that 1/3 of the new Rust devs feel as productive as in Rust as in their previous language (whatever that was) in less than 3 month. The biggest advantage their devs reported was that code review became significant easier.

The biggest claim is that Rust devs are twice as productive as C++ devs.

https://youtu.be/QrrH2lcl9ew has the entire presentation. It has some details on howmthey collected their numbers.

WHAT?

Posted Aug 31, 2024 13:41 UTC (Sat) by rc00 (guest, #164740) [Link]

I saw this and it was pretty soundly rejected by more intelligent individuals. A common reaction to the absurdity of the asymmetric tests:

https://youtu.be/vB3ACXSesGo

Not that hard

Posted Sep 3, 2024 10:41 UTC (Tue) by tialaramex (subscriber, #21167) [Link] (3 responses)

Nah, it's not that hard. It happened to be especially easy for me because I have decades of experience in semi-colon languages, especially C, and then also the First Language for my CS degree was the Standard ML of New Jersey, so all of Rust makes sense as it's essentially an ML disguised as a semi-colon language. So I'm the easy case, I barely remember "learning" Rust, much of it was like I'd always known this language.

Some top CS graduates will have a similar experience because both Cambridge and Oxford still begin with an ML as First Language and I expect at least a few of the decent CS courses in the US do as well -- beginning with a language that doesn't even have a proper type system makes the initial work of a non-vocational CS course very difficult to motivate.

But even if you've got a more vocational background where you learned like C++ and Python, or Java and Go or something, Rust's adherence to its "Empowering Everyone" slogan makes this a much softer landing that it might be. When you screw up (and you will screw up in any language) Rust will say that you screwed up, why and possibly even explain how to fix that.

In Java or Go you get inscrutable error messages. In Python the program blows up at runtime. In C++ your program - despite being complete nonsense - still compiles, and executes but doesn't do what you expected because it's wrong. What Rust is doing here maybe shouldn't stand out, but it does, and for some programmers this makes a huge difference.

Not that hard

Posted Sep 3, 2024 14:48 UTC (Tue) by mb (subscriber, #50428) [Link] (2 responses)

That is correct.
Contrary to legacy languages the Rust compiler is your friend and guide when writing code and when learning to write code. It tells the programmer what is wrong, provides a hint and a link to detailed information about the problem and often also the actual fix for the problem.

And that process can even be turbocharged with tools like this:
https://crates.io/crates/bacon

Complicated and hard to understand compiler messages do exist in Rust, too. But they are very rare.

Yes, it takes a very long time to learn each and every detail of Rust.
But the same is true for C++ and even C itself.

But getting productive does not take weeks or months.

And learning Rust makes people better at programming in general.
Many concepts from Rust can be used in other languages, too. In fact, the kernel uses many of them already and many uses do predate Rust.
That reduces the step height for kernel developers even further.

I think it's much harder to come from the OOP-language side than from the C side when learning Rust, because OOP-thinking (inheritance) does not really work in Rust.

Not that hard

Posted Sep 5, 2024 12:09 UTC (Thu) by taladar (subscriber, #68407) [Link] (1 responses)

> Contrary to legacy languages the Rust compiler is your friend and guide when writing code and when learning to write code. It tells the programmer what is wrong, provides a hint and a link to detailed information about the problem and often also the actual fix for the problem.

Which is why, when someone claims that Rust is hard because you constantly have to "fight the compiler" you know they are doing something fundamentally wrong in their approach to the language, most likely not paying attention to what the compiler is trying to tell them because the transfer experience with other compilers to Rust where understanding most error messages requires significant effort.

Not that hard

Posted Sep 7, 2024 12:05 UTC (Sat) by khim (subscriber, #9252) [Link]

> most likely not paying attention to what the compiler is trying to tell them because the transfer experience with other compilers to Rust where understanding most error messages requires significant effort.

No, it's not that. Many (most?) developers these days act like “human ChatGPT”: they combine various pieces of code that look somewhat relevant to what they are trying to achieve (often with help of IDE or even, these days, LLMs), then they run the generated mess and when it, inevitably, explodes – they fix the most egregious bugs.

Rust blows that approach to smithereens: sure, compiler says that what you wrote is nonsense… and even shows some possible solutions… but you don't need all that, you need to make your program to run!

Disconnect can be pretty severe and both people who find Rust conceptually easy and people who find it hard often talk “past” each other.

P.S. Note: the fact that Rust is “conceptually” easy doesn't mean it's easy to learn to write programs in it. It's similar to snowboard, in some sense: incredibly easy and simple tool… yet one that requires quite a non-trivial amount of skill to use it. But linux kernel people are, actually, unique positioned to learn it, because many concepts that Rust enforces are, in reality, already in use in Linux kernel… only is come places there code that “violates it a tiny bit”… and Rust wants 100% compliance, not 99% compliance.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 15:37 UTC (Sat) by johill (subscriber, #25196) [Link] (1 responses)

Aside from the "learn it quickly" subthread here ...

Sorry, the example of using it there wasn't really my point.

But like you said, using the wifi example again, if someone were to add rust bindings to mac80211 (which in your view would be a good thing to have), the point is that I'm still going to have to maintain them once they're in the tree, but I get nothing out of it because I can't even use them. So I have to maintain something I can't even _use_ myself.

And I think that's similar for the filesystems: as long as you can't actually rely on rust in general, most stuff will have to be maintained in C.

And I think for me at least, the point is that for those of us working on sufficiently core things rust therefore just adds work, no value. And I can't even really learn rust in the context of the code I'm maintaining, because I can't actually use it there myself! I'd totally be at the mercy of whoever adds the rust bindings, and then try to figure out what that does, but I don't think there'd be much of a chance if I'm not going to use it.

Do I think it's possible to learn rust? Sure. But I don't think it'll happen overnight (nor within a couple of weeks), but more importantly I don't think I'm going to learn rust and how to apply it to mac80211 without even being able to use it there.

(Given enough time I could of course write a toy driver for it or something, but in reality I don't see where to find the time for that. Most maintainers are already overworked. There's probably some promise in rust actually making that better, but that only applies when you can actually use rust for the work you need to do, which is nowhere near true yet for most of us.)

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 18:03 UTC (Sat) by mb (subscriber, #50428) [Link]

>So I have to maintain something I can't even _use_ myself.

What does "use" mean?
Does it mean "what the Intel driver uses"?

If one Rust Wifi driver were added, it would be used. Of course, the interface would not be added without a driver that uses it.

>Most maintainers are already overworked.

Yes. That is true.

But scaring away people with a hostile environment doesn't help that.

It has been a hostile environment 20 years ago and apparently (according to the article) it's still hostile today.
That doesn't improve things and doesn't relieve maintainers.

I'm quite happy that the code I'm still officially a maintainer of has basically become obsolete.
I won't jump in again to new areas as long as there are people potentially shouting at me for no reason.
No thanks.
I guess I'm not the only one.

(Just to avoid misunderstandings: I don't mean you personally. I like you and it was a lot of fun to work together with you. Most of what I am uncomfortable with is outside of the Wifi world)

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 22:45 UTC (Fri) by jgg (subscriber, #55211) [Link] (17 responses)

Bingo, I feel exactly the same way. I could have done iommufd in rust (and security justified it too!), but that is totally impractical. If I can't write any projects in Rust how am I expected to learn the language?

What?

Posted Aug 31, 2024 0:10 UTC (Sat) by rc00 (guest, #164740) [Link] (16 responses)

> If I can't write any projects in Rust how am I expected to learn the language?

Rustlings? But seriously, how would you learn any other programming language? Making demands of it to be supported in the Linux kernel? Or on your own time with your own resources and learning material. There are plenty of Rust projects that only want to do Rust, why not use one of those for a proving ground? Make the same argument with Nim or Odin or C3. These logical blind spots for a programming language are severe and very problematic.

What?

Posted Aug 31, 2024 21:57 UTC (Sat) by jgg (subscriber, #55211) [Link] (15 responses)

I've learned alot of languages in the past, they have all be hand in hand with a productive project. Today my full focus is on kernel work, I actually can't just stop and take a month off to go do something else. Other people depend on me to get their work done too!

If Linus were to declare that Janurary has no releases - a "learn rust month long holiday" and we could all get off the treadmill and get a head start on learning then sure maybe?

I understand the chicken and egg situation, but understanding it doesn't mean it is solved.

What?

Posted Sep 1, 2024 0:58 UTC (Sun) by rc00 (guest, #164740) [Link] (13 responses)

I can sympathize. Doesn't that add fuel to the bear case for Rust though? A language that is known for a steep learning curve and people who have limited reserves of extra time don't bode well for the long-term and probably don't mix well. I could see the argument for a different language that was simpler (especially with a better toolchain) but for the present, it seems like there's much more momentum in the opposition party to this experiment.

Either way, best of luck with all the work! We're all thankful for what you do! IOMMUFD is neat!

What?

Posted Sep 1, 2024 11:43 UTC (Sun) by jgg (subscriber, #55211) [Link] (12 responses)

I would be much more comfortable with the kernel rust project if I could see path from A -> B where myself and my hundred team mates could learn rust and *start using* it in a definable time frame. I don't see it, and I don't see people even talking about it yet.

FWIW, after sleeping on it, I really like the idea of a "Rust Holiday" declared by Linus. Not only does it help give people space to do something productive it is also a clear, but still soft, endorsement and message of the project's intent and direction. Might be a great rallying point.

Yes, I agree with your assesment. I think there is alot of energy in the "opposition party", and I'm not optimistic this can be overcome with the current approach. I'm am quite surprised by the forceful statements from Dave and Kees that this is done and decided. Definately not going to get involved in that fight when it comes. Let me know when it is over and we have a clear decision and workable plan please!

I'm glad you enjoy the projects! Thanks!

What?

Posted Sep 1, 2024 13:08 UTC (Sun) by rc00 (guest, #164740) [Link] (1 responses)

I can sense your enthusiasm but I struggle with the viability of the concept as useful for Rust. The main issue is that there are still many other challenges with the tooling, not just the language.* When there is adequate plumbing, then it makes sense to learn how to be a plumber. This experiment highlights much of what was already known: Rust and C are very different and incompatible languages, ignoring some of the community challenges as well. Rust is more often grouped with C++ even though there is a misnomer of it being an alternative to C. (To me, a viable alternative to C, should it ever exist, should embody C's simplicity and spirit of empowerment, but I digress.)

I do think there is value in the overall concept though! If some other language(s) were to prove viable and compatible with the Linux kernel project and effective (read: stable) infrastructure could be established, a development hiatus for upskilling and on-boarding would be tremendous for progress. To me, this as a takeaway could be the most valuable part of the Rust experimentation process.

* https://www.youtube.com/watch?v=OdMQwDtumJ8

What?

Posted Nov 28, 2024 20:06 UTC (Thu) by deepfire (guest, #26138) [Link]

> To me, a viable alternative to C, should it ever exist, should embody C's simplicity and spirit of empowerment

This makes me think that you still don't "get it" -- as mentioned by many, "C's simplicity" doesn't exist.

Tongue in cheek -- according to your definition of simplicity -- is assembly simpler or more complex?

If you agree, then why is your definition of simplicity a desirable goal?

If you disagree, then you must also disagree that C is simple.

What?

Posted Sep 1, 2024 20:35 UTC (Sun) by airlied (subscriber, #9104) [Link] (9 responses)

What do you think is missing from a workable plan?

You've been in the industry long enough to know how these things work, I find it unusal that anyone with so much industry experience can't see the writing on the wall.

You've described so many chicken/egg situations that just go away once the ball starts rolling down the hill, and often won't be solved in advance of getting over the top of the hill.

Objections like, rust in the kernel needs unstable toolchain features, will go away as people start to rely on rust in the kernel, not before. Of course as long as it is out of tree and has no hard deps in the tree, devs will push for newer toolchain features etc. Once the tree has a reason to lock down and not experiment that will all become harder.

Same for the we need support for ancient architectures, if it matters, someone will get to it, if it doesn't nobody will. I don't see the nvidia open driver supporting m68k for instance.

I would say the ball is 90% rolled up a hill, the big problem is the whole no code in the kernel without users, and we are solving it, but until that big first user lands in the kernel and people want to enable it in distros I don't think momentum will be achieved. Once all the basics for writing a driver and bindings to all the common things are in, then I expect people will have a lot happier time. I also only expect to have to drag Linus in when we get Ted style pushback from someone who matters.

I realised a year ago the value here for bringing on new developers, easier driver development and memory safety, before I'd even learned the language, I've only just seriously started writing code in rust in the past few weeks and I've brought up a bunch of nova in userspace across a lot of unstable firmware interfaces and it's much nicer than any C code I could have written.

What?

Posted Sep 2, 2024 0:50 UTC (Mon) by jgg (subscriber, #55211) [Link] (2 responses)

I don't know maybe I talk to different people than you? I've seen enough smaller fights to be sufficiently worried about of what is coming. This is *different*. The politics and discussions have a dangerous feel to them, reminds me of the code of conduct debate in some ways, IMHO.

So, I think, the tehnical side will reach 90% at some point and that will be the starting gun for the real fight. This is going to be chaotic and wild. Just look at the people who are already on-record being pretty significantly anti-rust. I think you see the same thing "Drag Linus in" is not exactly something you say when you anticipate a cordial walk in the park discussion. Seriously, count me out, call me when it is done.

We have rarely (ever?) seen the kernel community competently handle the choice of breaking someone's stuff vs allowing someone else's work, and it has very frequently leaned on the theme of "no regressions" for these kinds of arguments. Causally saying someone else will fix m68k, or not, is dangerously similar to what Ted suggested in that video, I don't see there will be alot of support for that idea.

So, a workable plan, is figure out some way to avoid or minimize the fighting! Come up with rules and agreement, in advance with timelines on when things will be allowed. There is a huge practical difference between merging something like drm_panic, or binder, and, say, me changing the iommu subsystem to require Rust. Figure out some kind of agreed framework for making these decisions.

For example, if we are at 90% today then that is enough to let things that can be disabled with minimal user impact be in Rust. It kind of feels like that is what we are doing informally today anyhow.

The next gate may be that Android has completed a full release cycle at 100% Rust support, then anything that is principally for Android/embedded can be in Rust.

Then maybe the next gate is RHEL/etc/etc have also completed a full cycle with 100% Rust support. Then Rust is fair game to use for general drivers.

And finally when we have full and complete support in all toolchains, architectures and distros for a year, then Rust is fair game to use anywhere.

Include some plan for training, etc. I like my Linus Rust Holiday idea quite a bit.

Have guidelines for rewriting/replacing existing code.

It needs to be written down, like the COC situation, where all kinds of people sign up to agree to it, and it needs to be on a *slow* timeline because we have a huge ecosystem that will take ages to swallow something this big.. Then, maybe, the fights would be small enough.. Communication is so important here.

I'm just making stuff up, to illustrate what I think a plan may sort of look like - I don't want to debate any of this (seriously, count me out). I really just want to read about the outcome in LWN :P

(I don't know anything about it, but I heard a story that someone sent obscure arch enablement patches to the NVIDIA driver's github. Since they are not part of the kernel community they have no obligation to act in any particular way, but if someone sent me m68k enablement patches for RDMA I would be expected to accept them as the duty of a maintainer)

What?

Posted Sep 3, 2024 19:36 UTC (Tue) by airlied (subscriber, #9104) [Link] (1 responses)

I really don't think we need to overthink this. We roll out the upstream support for a few things, see how it goes. Find where there is value in adding the next thing see how it goes. Creating a plan means a lot of people will expend a lot of time/energy on not doing the work of making rust work well in the kernel, for the benefit of very few people.

I'd rather stick with, "people who do the work, decide the plan" and let the plan evolve. I've never seen any plans for the Linux kernel either internally at companies or upstream work out in any fashion.

We do evolution really well, we should stick to our strengths.

I don't think we'll be taking "just because" rust rewrites, but again each maintainer needs to determine their own comfort level here. If we had a component with a history of memory safety issues then a rust rewrite might make it worth the investment. But from a point of view of upstreaming, I think we need novel drivers and features to start bringing in rust, not just virtual and ext2 rewrites, which are useful for finding a path, but not for driving upstream change.

What?

Posted Sep 4, 2024 18:32 UTC (Wed) by jgg (subscriber, #55211) [Link]

A plan is all about getting people on board to try to avoid so much fighting. It doesn't matter if the plan works out or not, it is the discussion and getting buy-in that is important here.

This isn't evolution. It is a step change, and it has built into it ongoing demands to other peoples time, not just "people who do the work pay the cost".

I don't think you will get novel drivers like this. People with a commercial interest will not get involved with this much uncertainty and backporting problems standing in the way. It is already hard to get C stuff merged, 'be the first rust in X' is 10x harder and more unpredictable.

What?

Posted Sep 2, 2024 10:13 UTC (Mon) by nim-nim (subscriber, #34454) [Link] (5 responses)

IMHO (which does not count for much) there are technical and infrastructural points (which are being addressed) and there are cultural points. It’s very hard to change collective behaviours once they have crystallised.

For Rust to be successful in the kernel Rust code writers need to interact well with C code writers because the C core is not going anywhere in the short term. Your legacy language sucks look how much better mine is let’s rewrite all your unsafe code is not a good template for (human) interaction. Some better customs needs to emerge.

Unfortunately all this is more easily said than done.

What?

Posted Sep 2, 2024 16:08 UTC (Mon) by Wol (subscriber, #4433) [Link] (4 responses)

> Rust code writers need to interact well with C code writers because the C core is not going anywhere in the short term.

> Unfortunately all this is more easily said than done.

And the majority of the problem seems to be that the C code writers don't want to interact :-(

Cheers,
Wol

What?

Posted Sep 2, 2024 16:59 UTC (Mon) by pizza (subscriber, #46) [Link] (3 responses)

> And the majority of the problem seems to be that the C code writers don't want to interact :-(

No, the majority of the problem is that the "C camp" and the "Rust camp" have (fundamentally?) different priorities, and there's no "higher authority" to disabuse one or both parties of their sense of self-importance.

What?

Posted Sep 3, 2024 4:53 UTC (Tue) by marcH (subscriber, #57642) [Link] (2 responses)

> disabuse one or both parties of their sense of self-importance.

Don't forget loud voices on the Internet are NOT representative.

It's really hard not to forget. I forget it every day. Everyone looking at TV news and social media does too. Every participant in this thread does.

What?

Posted Sep 3, 2024 11:20 UTC (Tue) by ms-tg (subscriber, #89231) [Link] (1 responses)

> Don't forget loud voices on the Internet are NOT representative.

While undoubtedly a good point and useful reminder, I have a follow-up question.

Are Ted T’so’s comments not representative of a set of kernel *maintainers*?

Wondering if, to address the question of representativeness directly, Linux might eventually need a list of (subsystem, maintainer, open to working with Rust for Linux Team to define interface semantics Y/N) tuples?

What?

Posted Sep 3, 2024 15:12 UTC (Tue) by marcH (subscriber, #57642) [Link]

How representative are maintainers? That sounds like a very interesting question but quite different from "people shout at and don't understand each other". You can have extremely polite maintainers that are not representative at all.

Like any other volunteer work, being a maintainer has an element of "trade": trading the work against dictating how it's done to some extent. This is summarized in the "BDFL" acronym. Again, all such "dictatorship" can be done very quietly and politely.

Of course this works only up to a point: no one wants a totally disconnected volunteer. But then:
1. How do you measure the disconnection?
2. Do you even have someone else to take over?

To measure 1. you could organize... a shouting match? Could not resist sorry :-)

What?

Posted Sep 1, 2024 1:35 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

> If Linus were to declare that Janurary has no releases - a "learn rust month long holiday" and we could all get off the treadmill and get a head start on learning then sure maybe?

That actually might be a great idea. He had one of this pauses during the introduction of git, after the BitKeeper fiasco.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 23:39 UTC (Fri) by roc (subscriber, #30627) [Link] (4 responses)

It's a chicken and egg problem. You don't want to learn Rust because you can't use it in the kernel. Meanwhile, you can't use it in the kernel because kernel developers don't know Rust.

Linux-for-Rust or Rust-for-Linux

Posted Aug 31, 2024 15:39 UTC (Sat) by johill (subscriber, #25196) [Link] (3 responses)

It's not really _want_ though, it's more like ... how many hours are there in a day. If I could use it for the job I need to do, great. But I can't.

And no, the reason I can't use it in the kernel is that it can't be in sufficiently core parts of the kernel, given the tools and architecture support story.

How could Rust improve?

Posted Sep 1, 2024 15:10 UTC (Sun) by gmatht (subscriber, #58961) [Link] (2 responses)

Hi, I haven't found a description of how Rust limitations affect kernel adoption. I understand that Linux developers are not the only ones who want better arch support. Do you prefer gcc-rs or compiling Rust to C? I have found cargo, etc., to be decent. How could Rust tooling be improved?

How could Rust improve?

Posted Sep 1, 2024 15:24 UTC (Sun) by johill (subscriber, #25196) [Link] (1 responses)

Well, honestly, I don't know all of the details. But it's clear that rust isn't there for all architectures, even LLVM isn't:

https://docs.kernel.org/kbuild/llvm.html#supported-archit...

And rust says:

https://docs.kernel.org/rust/arch-support.html

whereas the list of all architectures is currently

alpha, arc, arm, arm64, csky, hexagon, loongarch, m68k, microblaze, mips, nios2, openrisc, parisc, powerpc, riscv, s390, sh, sparc, um, x86, xtensa

So for something sufficiently core, you can't use rust.

I'm sure this will become better over time, but (IMHO) until it does there's a lot of stuff that's simply out of scope as I described.

Perhaps with the graphics driver(s?), NVMe driver and other things that have happened we should declare the rust experiment a success, and start focusing on the architecture story etc. (gccrs, code gen, whatever) so there can be hard dependencies. The extra dependency will probably still be hated by some (including some distributors perhaps), but at least it won't be a dependency that's literally impossible to satisfy.

I guess I just don't think it's fair to say it's all the fault of kernel maintainers, who are generally already overworked, have to support stuff on all architectures, have to (willing or not) learn a whole new thing, and then everyone says "oh but it works well for me so they're just being idiots." I mean, I guess we're used to being the scapegoats but still ... Both sides would do well with some empathy for the other side.

rustc_codegen_gcc announced it could compile Rust For Linux.

Posted Sep 3, 2024 13:14 UTC (Tue) by gmatht (subscriber, #58961) [Link]

I wasn't trying to criticize kernel developers. I was hoping that in addition to Linux better supporting Rust, Rust could better support Linux.

BTW, rustc_codegen_gcc announced just over a year ago that they can now compile Rust-for-Linux. Informal benchmarks suggest it is 3% slower than regular rust.

I also had a look at mrustc. I was impressed that it could compile my rust code without issues, but it doesn't seem to support some essential things like #![no_main]. The resulting C can be 200x the size of the input rust, but it seems to shrink during compiling, and the compiled executable was smaller than the rust executable. I guess that is rust's static linking for you. I am not sure if this means mrustc-generated C would introduce excessive overhead.

Linux-for-Rust or Rust-for-Linux

Posted Sep 4, 2024 6:31 UTC (Wed) by ebiederm (subscriber, #35028) [Link]

In my experience Ted Tso in maintaining ext4 has been the most serious of the filesystem developers in dealing with crafted images.

He has previously promised to fix all problems that are clearly identified. Finding all of them is the issue. Last I looked it was an open problem on how to do that.

I looked because it was a roadblock to my work.

Of course this was before the recent crop of fuzzers.

Linux-for-Rust or Rust-for-Linux

Posted Sep 2, 2024 23:38 UTC (Mon) by anthm (guest, #173221) [Link] (4 responses)

> There is not much opposition to Rust in the kernel community — far less than I had expected, honestly

But there is opposition, and it's loud and aggressive as evidenced by the video.

Loud aggressive and seemingly not particularly caring about the technical merit of their arguments.

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 0:32 UTC (Tue) by pizza (subscriber, #46) [Link] (3 responses)

> But there is opposition, and it's loud and aggressive as evidenced by the video.

There is "opposition" to nearly everything that is submitted for inclusion in Linux, for any number of reasons, technical or otherwise. Historically this opinionated maintainer gatekeeping activity has been lauded as a GoodThing(tm), and has been one of the primary contributors to steadily raising the overall quality bar for the past 30-odd years.

Meanwhile, on what basis do you assume that video is representative? Or is it just an example of confirmation bias?

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 7:24 UTC (Tue) by Wol (subscriber, #4433) [Link] (2 responses)

> Historically this opinionated maintainer gatekeeping activity has been lauded as a GoodThing(tm), and has been one of the primary contributors to steadily raising the overall quality bar for the past 30-odd years.

The problem, and history will also tell you this, is that "opinionated gatekeeping" from the "young turks" can be a damn good thing. The problem is when they become "old turks", they are a damn liability!!!

Robert Hooke (or was it Boyle?) is, iirc, a particularly good example, but it seems to be the norm. "Science advances one funeral at a time" or whatever the quote is. Why should Computer Science (and linux) be any exception?

Unfortunately, all the serious pushback from opinionated maintainers seems to be of the form "I don't like it so it's not going in". That's a VERY strong indicator that the maintainer in question should step down and retire! If the reaction was "I don't like this - I don't understand it but it's ringing alarm bells" then fair enough. That at least opens the door for dialog.

I'm afraid (hoping?) things could easily start going the way someone else suggested. All it will take is for Asahi to do what she's threatened - rewrite drmbuf or whatever it was. Linus then says "okay, it looks good, it's going in" - as he should if it lives up to its promises, and next minute all new graphic drivers will be in Rust.

At which point, all those opinionated gatekeeper maintainers will find their fiefdoms deprecated to "legacy" as new people come in with Rust versions and shove them aside. That, imho, would be an extremely successful manoeuvre for linux, even if tragic for the likes of Ted. (And there's plenty of alternative disk formats. No reason why it should always be ext2/3/4. The momentum for change already seems to have been building for a while ...)

Cheers,
Wol

Linux-for-Rust or Rust-for-Linux

Posted Sep 3, 2024 13:17 UTC (Tue) by pizza (subscriber, #46) [Link] (1 responses)

> Linus then says "okay, it looks good, it's going in" - as he should if it lives up to its promises, and next minute all new graphic drivers will be in Rust.

Linus has a lot of "soft power" but at the end of the day, he can't actually _force_ anyone to do anything. Other than enforcing the "Linux" trademark, anyway.

> Unfortunately, all the serious pushback from opinionated maintainers seems to be of the form "I don't like it so it's not going in".

Their objections have considerably more meat than that. Even Ted Tso's.

(And I recall numerous Rust folks having strong opinions about what should or shouldn't go in to stuff they maintain. So let's not pretend that opinionated maintainers are okay for me, but not for thee!)

> as new people come in with Rust versions and shove them aside.

(1) there isn't a legion of "Rust developers" just waiting at the gates, and (2) nearly everyone seems to vastly, vastly underestimate just how much _work_ these core maintainers actually do, day in and day out. Rust doesn't magically make most (if any) of that ongoing work become unnecessary.

Linux-for-Rust or Rust-for-Linux

Posted Sep 5, 2024 1:16 UTC (Thu) by raven667 (subscriber, #5198) [Link]

> nearly everyone seems to vastly, vastly underestimate just how much _work_ these core maintainers actually do, day in and day out

this seems to be a major sticking point, for someone who learned a Rust, understands the code and is past that experience it can be hard to understand the fear and trepidation from a maintainer who may feel like they are burnt out and just hanging on at the edge of their competence and time, who are told that they need to stretch and grow and spend even more mental effort learning when they don't feel like their current workload will take a holiday to let their brain relax and absorb new thoughts. It just looks like pain. Like being at mile 20 in a marathon and having someone start jogging alongside saying "what's wrong with you,why are you breathing so hard, can't you go faster".

The Rust design seems to do what it was designed to do well, which can clear up whole classes of hard to find bugs, but being right isn't the same as winning hearts and minds and its supremely frustrating to be right and still lose. It's one thing to be confident you are technically correct, it's another to make changes which affect millions of people when you are nagged by doubt that the house of cards will topple of you touch it wrong, from somewhere outside your awareness

Linux-for-Rust or Rust-for-Linux

Posted Aug 29, 2024 21:41 UTC (Thu) by pizza (subscriber, #46) [Link] (3 responses)

> Refactors like Folios are a significantly more invasive and far reaching than anything Rust has proposed so far.

It was also _incremental_ and didn't require fundamentally changing your tooling, workflows, and cognitive load.

> - Believing good decisions can only be made by fiat of strong leaders and not collaboratively by community consensus.

Both of those have some pathological failure conditions -- eg the first makes "wrong" decisions, and the latter never actually decides anything at all. Debian is a great example of this. (Meanwhile, there are other models, incluidng "those that do the _work_ make the decisions" and "you've earned your place as a leader by showing up to do the work consistently for a decade")

> - It is not worth it to design systems holistically to eliminate certain problems from the beginning.

Meanwhile, you can't make an omelette without breaking some eggs. Your original design is _always_ wrong. But you won't know how/what is wrong until it encounters the real world.

- So-called "soft-skills" like writing documentation should not be treated as essential parts of programming. Someone else will come along and do those. That works for the dishwasher after all.

You can "should" all you want, but at the end of the day, this won't change as long as folks paid to produce code, not documentation. (And if we're not paid at all, then what gives you any say what we should or shouldn't do?)

- The linux kernel is not just a piece of software like any other. Linux kernel developers are not just normal, average software developers. Learnings from outside the linux kernel, even from other kernels, are thus not applicable to us.

Yes, Yes, and very-much Yes. Objectively so, in both scope and scale.

And why do you presume that learnings can only go one way? Hubris, indeed.

- Tools should be built primarily around purely technical concerns, not around the humans that use them. Acknowledging their fallability and flaws is an insult and leads to worse software.

How about "different tools have different purposes"? (And a tool that cannot solve the technical problem doesn't even merit consideration!)

- And let's not ignore the spicy one: Having sufficient visibly female and queer people that you can't just dismiss them as quota fillers will doom a project, as they are by their nature less meritable.

WTF is this bullshit?

> However I'm ultimately more positive about the outlook there. Between AGX and Nova, we're about to be in a situation where 2/4 desktop graphic card drivers will run on Rust. I think that, even with the resistance to Rust, the kernel will not be able to afford to keep these things out of tree forever if people keep maintaining them.

...Just like Linux was unable to afford to keep the ginormous vendor Android trees (and their GPU drivers) out? How well did that work out?

...Again, Rust is only optional if nothing you need depends on it. And once you need it, that's a flag day, and you might as well make other stuff depend on it too because you're already paying the price, so you might as well maximize your benefits.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 0:52 UTC (Fri) by gerdesj (subscriber, #5446) [Link]

"- So-called "soft-skills" like writing documentation should not be treated as essential parts of programming. Someone else will come along and do those. That works for the dishwasher after all."

You are aware that big Jon is the kernel documenter-in-chief? ... and he runs LWN.

Describing LWN and #1 as "soft-skills" and hence non essential is not going to endear you to anyone. The Linux kernel is not a dishwasher and a lot of people spend a lot of time and effort documenting it.

I'll leave it there.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 9:26 UTC (Fri) by amacater (subscriber, #790) [Link] (1 responses)

Your comment on the community approach never doing anything at all and citing Debian - that may be a fair viewpoint from a long way out, but in practice Debian is a "do-ocracy" and the people that do the work get to make (the vast majority) of decisions.

Even when things get really protracted and discussions go on for years (systemd), it ends up that the people that do the work get to decide the technical and practical implementations of any ridiculously long debate. The number of huge discussions is decreasing and flamewars are largely out of fashion.

On visible diversity in any project - that's a problem in any significant project which doesn't reflect that talent is found in the widest community and actively seek it out.

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 11:51 UTC (Fri) by pizza (subscriber, #46) [Link]

> Your comment on the community approach never doing anything at all and citing Debian - that may be a fair viewpoint from a long way out, but in practice Debian is a "do-ocracy" and the people that do the work get to make (the vast majority) of decisions.

That my exact (if poorly-expressed) point -- It's the "strong leaders" that actually make the decisions by virtue of actually doing the damn work while "the community" endlessly bikesheds. This is pretty much true in all "communities".

> On visible diversity in any project - that's a problem in any significant project which doesn't reflect that talent is found in the widest community and actively seek it out.

At some point, one has to accept that "Individual talent" and "Individual interest" are not synonymous, especially in volunteer settings where a paycheck isn't involved.

Linux-for-Rust or Rust-for-Linux

Posted Aug 29, 2024 22:11 UTC (Thu) by rcampos (subscriber, #59737) [Link]

I do remember Mathew sending an email saying something like: fuck it, I can't take this anymore, don't merge this if you don't want.

(Not quoting, something along those lines is what I recall).

And not so much longer after that, they were merged.

Linux-for-Rust or Rust-for-Linux

Posted Aug 29, 2024 22:25 UTC (Thu) by Wol (subscriber, #4433) [Link] (1 responses)

> I'm sure you can think of more. Of course, the degree to which these beliefs are held in any one person will vary, and there are equally things that go the other way. But the general point is that views that extend far outside of the kernel and programming lead people to *really want* the project to either fail or succeed to prove or disprove those beliefs. Which makes things much harder than they need to be.

Focusing on the wrong metric?

My recent exchange has shown me that database designers focus far too heavily on worst case access. Because MultiValue cannot (and makes no attempt to) guarantee fast worst case response, it's dismissed. But in order to guarantee a worst case of O(log(n)), relational pretty much also guarantees a best case of O(log(n)).

So let's take a database with 100,000 rows. Relational pretty much guarantees *all* accesses are O(5). Compare that to MultiValue which guarantees (absent a pathological mess) 95,000 rows will be accessed O(1). It also guarantees to five 9s that all accesses will be O(5) or better! (which is why I picked 100K :-)

And then, assuming the relational database is First Normal Form it gets even worse - because MultiValue is 4th Normal Form, 100K records in MV are probably the same as half a mill FNF rows ...

Which would you prefer - a database that is slow and leaves you waiting forever (unless it's running on the latest and greatest electricity-guzzling monster), or a database that's maybe ten times faster, but has a slight risk of the odd query taking for ever? I would have thought a five 9s guarantee that MV is faster than relational would at least earn SOME interest...

Plus MV doesn't protect database designers from crap decisions - at least relational is an equal opportunities slowcoach :-) It peanalises novice and expert designers alike. What MV does have is plenty of tools to say "your worst case will be order X". And I've personally had experience of accidentally pathological tables - they're still damn fast.

Cheers,
Wol

Linux-for-Rust or Rust-for-Linux

Posted Aug 30, 2024 15:19 UTC (Fri) by ma4ris8 (subscriber, #170509) [Link]

C compiler is a tool, to convert source code into executable program.
Rust compiler is a tool, to convert source code into executable program.

Languages differ quite a bit. There is a common set of erroneous code that
are detected at compiler time. Other set of erroneous code is detected with linters.

Some programming models are shared with C and Rust.
Some programming models are C specific, some are Rust specific.

So both tools improve productivity, and prevent various incorrect programs.
Finding more error cases, might consume a bit more CPU time.

To take advantage of Rust's ability to detect more classes of errors at compile and lint time,
perhaps it could be that C code doesn't have to be changed,
or then to make life easier to both C and Rust developers, some
engineered solution might be needed to keep the implementation simple as a whole.

Interesting times

Posted Aug 30, 2024 23:58 UTC (Fri) by roc (subscriber, #30627) [Link] (6 responses)

If a lot of maintainers stick to the position that they cannot be expected to learn Rust, then it's hard to see Rust being used much until those maintainers retire. A wave must be coming in a decade or two, though... e.g. Ted T'so and Linus are in their mid-50s. It's an open question whether Linux will continue to thrive after those retirements. It seems to me there's a real opportunity for a new kernel to step up and prepare to fill that possible gap. Linux has accumulated a lot of overhanging issues that could be improved on with modern workflows, proper CI, Rust, microkernel for a sane TCB (yes it's time), everything_uring, fixing the worst APIs, building a community that welcomes contributions.

Interesting times ahead!

I don't really get the "cannot be expected to learn Rust" thing though. It's just not that hard compared to all the other things kernel developers are expected to know, especially given you get a lot more handholding from the tools.

Interesting times

Posted Aug 31, 2024 1:04 UTC (Sat) by airlied (subscriber, #9104) [Link] (5 responses)

Like do you want a developer who is willing to state in publc that they are not capable of learning a new language, being the main developer that you rely on for storing your disk content?

Like even just for future job prospects you'd think you'd refrain from publically stating your own inabilities.

Interesting times

Posted Aug 31, 2024 1:13 UTC (Sat) by roc (subscriber, #30627) [Link]

I was trying to take the more charitable interpretation of "I can't be bothered to learn Rust" rather than "I am not smart enough to learn Rust". But yes.

Interesting times

Posted Aug 31, 2024 23:20 UTC (Sat) by sashal (✭ supporter ✭, #81842) [Link] (3 responses)

Big difference between "not capable" and "not willing".

Interesting times

Posted Sep 1, 2024 20:25 UTC (Sun) by airlied (subscriber, #9104) [Link] (2 responses)

Not as big as you'd like to think, not willing speaks a worse attribute to have from a hiring perspective.

The ability to actually consider and embrace new ideas is usually consider valuable in engineers.

Interesting times

Posted Sep 2, 2024 3:04 UTC (Mon) by pizza (subscriber, #46) [Link]

> The ability to actually consider and embrace new ideas is usually consider valuable in engineers.

Eh, not really.

it's clearly better to hire someone that claims to have five years experience with a technology only two years old than it is to expend the time/money to train up an existing engineer.

Interesting times

Posted Sep 2, 2024 7:20 UTC (Mon) by LtWorf (subscriber, #124958) [Link]

> Not as big as you'd like to think, not willing speaks a worse attribute to have from a hiring perspective.

I'm sure he won't be homeless anytime soon -_-'

Disappointing but absolutely understandable

Posted Aug 31, 2024 11:51 UTC (Sat) by mohkale (guest, #159931) [Link] (2 responses)

Man, that filesystem discussion excerpt is just depressing. I don't know how to write rust and I don't have much of a motivation to atm but the hostile tone with which Almeida was addressed is not okay and the general attitude is flagrantly against any open or collaborative setting I'd expect of a true community project. I can't imagine any workplace or environment where a dev can say I'll work on this, but if I broke something else for someone else that's their problem and I'm not even going to try to fix it. Like for context I've worked on projects with niche, nebulous, customized build systems but never did I say "screw this, I don't care if my change breaks these other tests, merge it now" which is how I'm interpreting the comments here.

Disappointing but absolutely understandable

Posted Sep 2, 2024 7:24 UTC (Mon) by LtWorf (subscriber, #124958) [Link] (1 responses)

I've worked with plenty of people (with a job title of senior software engineer or above) who ping me because their code didn't compile and they're unwilling to read the output of the compiler.

So I guess in a professional setting it happens all the time.

Disappointing but absolutely understandable

Posted Sep 2, 2024 12:04 UTC (Mon) by Wol (subscriber, #4433) [Link]

> So I guess in a professional setting it happens all the time.

A! O! L!

Or something's obviously broke, so they carry on blindly following procedure and make the resulting mess ten times worse ...

Cheers,
Wol

Youth

Posted Sep 9, 2024 19:26 UTC (Mon) by 0x3333 (subscriber, #158599) [Link]

I like Almeida's work, but this clearly has issues that will never be resolved. Linux is C, and will always be. Rust is still too young for such an adventure with such old people.


Copyright © 2024, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds