A pair of Rust kernel modules
An NVMe driver
Andreas Hindborg was up first to talk about an NVM Express driver written in Rust. The primary reason for this project, he said, was to take advantage of the memory-safety guarantees that Rust offers and to gain some real-world experience with the language. His conclusions from this project include that Rust comes with a lot of nice tooling and that its type system is helpful for writing correct code. It is, he said, easier to write a kernel driver in Rust than in C.
![[Andreas
Hindborg]](https://static.lwn.net/images/conf/2022/kang/AndreasHindborg-sm.png "Andreas Hindborg")
Why write an NVMe driver when the kernel already has one that works well?
There are no problems with the existing driver, he said, but NVMe is a good
target for experiments with driver abstractions. NVMe itself is relatively
simple, but it has high performance requirements. It is widely deployed,
and the existing driver provides a mature reference implementation to
compare against.
Hindborg talked for a while about the internals of the NVMe interface; in short, communications between the interface and the computer go through a set of queues. Often the driver will configure an I/O queue for each core in the system if the interface can handle it. Creating data structures in Rust to model these queues is a relatively straightforward task. In the end, the Rust driver, when tested with the FIO tool, performs almost as well as the existing C driver. The difference, Hindborg said, is that the C driver has already been highly tuned, while the Rust driver has not; it should be able to get to the same level of performance eventually.
He concluded by saying that the Rust NVMe driver is still "a playground" and not production-ready at this point. To move things forward, he would like to create more abstractions that would allow the removal of the remaining unsafe blocks in the driver. It doesn't yet support device removal or the sysfs knobs for the nvme-cli tool. He would also like to look into using the Rust async model, which would "simplify a lot of things" in the driver, but possibly at the cost of performance.
At the end of Hindborg's talk, Paul McKenney asked if there was any available information on the relative bug rates between the C and Rust drivers. Hindborg answered that there have certainly been some bugs; building Rust abstractions around existing C code can be hard to do correctly. That work needs a lot of care and review, but once it works, drivers built on it tend to show few problems.
A 9P filesystem server
Last year, Linus Walleij suggested that, rather than writing drivers in Rust, developers should target areas with a higher attack surface — network protocols, for example. Wedson Almeida Filho has taken that advice and written an in-kernel server for the 9P filesystem protocol in the hopes that this project would demonstrate the productivity gains and security benefits that Rust can provide. Initially, he had started trying to replace the ksmbd server, but that turned out to not be an ideal project. The SMB protocol is too complex and the server needs some significant user-space components to work. He wanted something simpler; 9P fit the bill.
![[Wedson Almeida
Filho]](https://static.lwn.net/images/conf/2022/kang/WedsonAlmeidaFilho-sm.png "Wedson Almeida Filho")
The 9P file protocol, he said, comes from the Plan 9
operating system. The kernel has a 9P client, but no 9P server. There is
a 9P server in QEMU
that can be used to export host filesystems into a guest. The protocol is
simple, Almeida said, defining a set of only ten operations. His 9P server
implementation works now, in a read-only mode, and required just over 1,000
lines of code.
Almeida was also looking for a way to experiment with async Rust in the kernel. In the async model, the compiler takes thread-like code and turns it into a state machine that can be implemented with "executors" and "reactors", which are implemented in the kernel crate. He created an executor that can run async code in a kernel workqueue; anywhere such code would block, it will release the workqueue thread for another task. There is also a socket reactor that is called for socket-state changes; it will call Waker::wake() from the Rust kernel crate to get the appropriate executor going again.
There is, of course, plenty of work yet to be done. He would like to implement reactors for other I/O submission paths, including KIOCBs (asynchronous I/O), URBs (USB devices), and BIOs (block devices). Memory allocation can still use some work; it would be good if a GFP_KERNEL could give up its thread while waiting for the memory-management subsystem to do complicated things.
At the end, I asked whether the objective of demonstrating the security benefits of Rust had been achieved; has there been, for example, any fuzz testing of the server? Almeida answered that the Rust-based parsing interface makes a lot of mistakes impossible. No fuzz testing has been done — the server has only been working for a couple of weeks — but he will do it. He concluded that he will be interested to see how his server fares in such testing relative to the QEMU implementation.
[Thanks to LWN subscribers for supporting my travel to this event.]
| Index entries for this article | |
|---|---|
| Kernel | Development tools/Rust |
| Conference | Kangrejos/2022 |
Posted Sep 12, 2022 14:54 UTC (Mon)
by iustin (subscriber, #102433)
[Link]
Posted Sep 12, 2022 14:54 UTC (Mon)
by koverstreet (✭ supporter ✭, #4296)
[Link] (13 responses)
The async stuff in Rust works beautifully here; internally, the compiler is doing a CPS transformation. This is something (some of us) C programmers have known how to do for years, but in practice it's _tedious_, so when we do it in C it's always less than ergonomic and incomplete.
And the RAII stuff means a lot of tricky cleanup code just doesn't exist.
This the code I always wished I could write - and unlike in C++, we can write it and actually trust that it's correct.
Posted Sep 13, 2022 10:33 UTC (Tue)
by ncm (guest, #165)
[Link] (12 responses)
Coding it in Rust does not, in fact, guarantee it is correct. Coding C++, you need only choose known-correct primitives to get the same level of assurance. But you would then face rabid, unreasoned hostility from Linus for C++ features you may use in Rust code without approbation; and have your patch summarily rejected with, most likely, a rude remark.
The easy way to get C++ code into the kernel is via eBPF, which offers solid support for building from C++, and where there is nothing Linus can do to stop you. And, it is exactly as safe as Rust eBPF, but likely less annoying to code.
Posted Sep 13, 2022 11:08 UTC (Tue)
by gspr (subscriber, #91542)
[Link] (2 responses)
I believe that the parent commenter meant that doing exactly this is signficantly harder than it is in Rust.
Posted Sep 13, 2022 11:13 UTC (Tue)
by milesrout (subscriber, #126894)
[Link] (1 responses)
Posted Sep 13, 2022 11:16 UTC (Tue)
by corbet (editor, #1)
[Link]
Thank you all.
Posted Sep 14, 2022 10:56 UTC (Wed)
by xav (guest, #18536)
[Link]
To a certain extend, it does out of unsafe blocks, and way more than C++ can even dream of.
Posted Sep 15, 2022 19:14 UTC (Thu)
by nix (subscriber, #2304)
[Link] (7 responses)
Trying to write nontrivial C++ code and expecting it to turn to working eBPF without massive amounts of verifier agony is an exercise in futility at present. You can try if you like, but I wouldn't want to listen to the resulting cursing if you did. (Maybe this will change in time, but I very much suspect that turning the verifier into something that isn't full of arbitrary restrictions and can actually verify real code that wasn't written with 90% of one's attention on contorting things in unnatural ways to get it to verify will either slow it to unusability or just slam straight into the wall of Rice's theorem.)
Posted Sep 15, 2022 19:37 UTC (Thu)
by rahulsundaram (subscriber, #21946)
[Link] (1 responses)
Have you tried the Rust ones? https://aya-rs.dev/ or https://crates.io/crates/bcc
Posted Sep 15, 2022 21:13 UTC (Thu)
by nix (subscriber, #2304)
[Link]
Figuring out why required diving into the verifier source code, because there is of course nothing like a spec, and the verifier's verification contains enough holes where obviously valid code is rejected out of hand because nobody's written code to verify it (if you're lucky, there are comments noting the lack of said verification) that it frankly should be called a collection of holes with a little fabric connecting them :)
Posted Sep 16, 2022 8:51 UTC (Fri)
by nybble41 (subscriber, #55106)
[Link] (1 responses)
Strictly speaking, without taking into account knowledge of the internals of bpf_map_update_elem(), that loop might *not* terminate. You're passing in a pointer to the loop counter (id) and the function could potentially update the variable via that pointer to prevent it from incrementing. (Yes, even if the pointer argument is const-qualified; you can cast away the const qualifier on a pointer with const_cast as long you're not trying to store into a const object, and the variable id in this example isn't a const object.)
It would probably compile if you copied id into a separate local variable and passed a pointer to that variable to bpf_map_update_elem(). Then the compiler could easily prove that the function doesn't update id since it doesn't have its address (given the standard pointer provenance rules), which places a fixed upper bound on the number of loop iterations.
Posted Sep 16, 2022 9:22 UTC (Fri)
by Wol (subscriber, #4433)
[Link]
The standard explicitly permits moving the index into a register (which other code has no access to) and then it saves it to the variable whenever it increments or other fancy tricks.
Whatever, as far as the code *inside* the loop is concerned, the index variable is either read-only, or if you do try and modify it, it's undefined.
So, provided the naive loop terminates, you can guarantee that any loop will terminate.
Cheers,
Posted Sep 23, 2022 9:56 UTC (Fri)
by rep_movsd (guest, #100040)
[Link] (2 responses)
The compiler cannot guarantee that id doesnt change whilst you are looping.
So either the function takes a const pointer to int (which is braindead) and the compiler is brain dead not to see its not modifiable
OR
The function takes it by pointer (which may be braindead) and the compiler is right
Posted Sep 23, 2022 11:05 UTC (Fri)
by adobriyan (subscriber, #30858)
[Link]
BPF_CALL_4(bpf_map_update_elem,
Posted Sep 23, 2022 18:04 UTC (Fri)
by nybble41 (subscriber, #55106)
[Link]
Even if the function did take a const-qualified pointer, it wouldn't matter. It's perfectly legal to do something like this:
void f(int const *p) { *(int*)p = 7; }
... as long as the original *object* was not const-qualified. (Attempting to write into a *variable* which was const-qualified at the point of declaration is, of course, undefined behavior, whether or not the pointer being used is const-qualified.) Consequently, the compiler can't assume that the function won't write into a mutable object even if the pointer argument is const-qualified.
Also, the read-only pointer-to-int argument pattern isn't quite so "braindead" as you suppose. These functions are generic; in this case the key happens to be an int, but that isn't always true, and so the key is passed by reference rather than value. The comparison callback for the standard library's qsort() function uses the same pattern, for example, when sorting an int array.
Posted Sep 12, 2022 16:28 UTC (Mon)
by NHO (subscriber, #104320)
[Link]
Posted Sep 13, 2022 9:48 UTC (Tue)
by josh (subscriber, #17465)
[Link] (1 responses)
Posted Sep 13, 2022 15:38 UTC (Tue)
by MrWim (subscriber, #47432)
[Link]
Posted Sep 13, 2022 11:12 UTC (Tue)
by milesrout (subscriber, #126894)
[Link] (53 responses)
So in other words: oh yes, there are still bugs, but we can just blame them on C. Any issues with interfacing Rust and C? Obviously the fault of C. Even though clearly and logically any bugs introduced by having to interface two languages is only reasonably attributable to the language that came along later and seems intent on forcing its way into the kernel regardless of what anyone else wants.
And things like this:
> At the end, I asked whether the objective of demonstrating the security benefits of Rust had been achieved; has there been, for example, any fuzz testing of the server? Almeida answered that the Rust-based parsing interface makes a lot of mistakes impossible. No fuzz testing has been done — the server has only been working for a couple of weeks — but he will do it. He concluded that he will be interested to see how his server fares in such testing relative to the QEMU implementation.
In other words "no we haven't actually tested it but I'm sure the language prevents bugs, they're totally impossible". Yeah right. This is typical of the Rust community: huge promises, no evidence to back them up, all topped off with an "if it compiles it is correct" attitude that totally disregards that there are many more issues other than those the Rust people have decided count as 'safety'.
Anyone noticed that 'unsafe' as a general concept has suddenly been redefined to mean 'whatever Rust prevents' (if you don't use `unsafe`, which the Rust for linux code does all over the place)? When people involved in Rust's development realised just before the release of the language that the language was fundamentally unsound, allowing memory leaks, they quietly redefined 'safety' to exclude leak freedom, because they didn't have time to properly fix it before 1.0. This was despite months and years of telling people how wonderful Rust was because it prevented memory leaks, lol. This should come as no surprise from the same group of radicals that redefine words on a daily basis from what everyone in the world understood them to mean, inside and outside technology. 'Master branch' is an (attempted) victim of the same ideology - nothing, not even in the truth, will stand in their way.
The entire idea is horrible. I do not want a Rust compiler on my computer, I do not want to need a Rust compiler on my computer to be able to compile the kernel. Rust is 'trusting trust' on steroids, for one thing. It is incredibly slow. Compiling a kernel takes long enough as it is! It's specified as 'whatever rustc does'. It has one implementation, with no other implementations even remotely close to being ready (there are other "implementations", but they are nowhere near complete and cannot do borrow checking, the core feature of the language, so are practically useless.) It is an abhorrently complex language. Nobody has EVER explained what it provides to the kernel that Ada or better static analysis tools for C could not. The ONLY thing Rust has going for it is an incredibly pushy "community". They say "oh it will only be in a few drivers". Yeah, at first. But there's no real point doing it if there's not going to be a lot of it. And that means SLOW compile times and an inability to bootstrap reliably.
Anyway, let's be real: Rust is just not going to happen in the kernel. Rust can express legibly only one way of managing memory. Nobody has even managed to represent the Wayland memory model in Rust, and the kernel is far more complicated than that. The one serious attempt to do so failed after months of work because it required writing thousands upon thousands of lines of memory management boilerplate. If you want to represent anything more complex than Box or Rc you need to just about write a thesis.
And look at the code itself. There's a huge list of nightly (basically brand new!) compiler features that are required to build it. No way should there be code in the Linux kernel that requires anything other than stable language features. At least one of them is described as "Status: the current design is perma-unstable -- a new RFC is needed. The issue may be split." And generic associated types have been in progress for more than 5 years with no real sign that they're going to be stabilised any time soon.
And last I checked, 'async' in Rust is widely considered a failure of design.
Posted Sep 13, 2022 13:10 UTC (Tue)
by PengZheng (subscriber, #108006)
[Link] (5 responses)
I'd better invest that time learning C++20/23.
Posted Sep 13, 2022 14:04 UTC (Tue)
by reijoslav (guest, #98915)
[Link] (3 responses)
Posted Sep 13, 2022 14:45 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (1 responses)
Cheers,
Posted Sep 14, 2022 6:18 UTC (Wed)
by milesrout (subscriber, #126894)
[Link]
Posted Sep 14, 2022 6:19 UTC (Wed)
by milesrout (subscriber, #126894)
[Link]
Posted Sep 15, 2022 21:13 UTC (Thu)
by thecodedmessage (guest, #160911)
[Link]
Rust does not eliminate all bugs. It's true that Rust's safety features only prevent the behaviors that they're designed to prevent. Of course that's true -- no programming language can prevent all bugs, but Rust's safety features do prevent some very bad types of memory corruption that are infamous for causing problems in C and C++. Additionally, even 'unsafe' Rust does a better job at preventing those failures than C++, which in many cases doesn't even give you any tools to manage the problem, relying instead entirely on the programmer.
And yes, they tried to eliminate memory leaks in safe code and then realized that wasn't an achievable goal in line with their other goals. That doesn't mean that Rust is bad. Rust is much better at preventing memory leaks than C++, but it is still possible to leak memory. It's way harder to do by accident, though.
Rust is better than C++. The fact that it's not utopian or perfect doesn't mean that it's not better.
Posted Sep 13, 2022 15:06 UTC (Tue)
by rav (guest, #89256)
[Link] (6 responses)
Actually, under two hours after your comment, generic associated types were stabilised: https://github.com/rust-lang/rust/pull/96709#issuecomment...
Posted Sep 14, 2022 6:05 UTC (Wed)
by ssmith32 (subscriber, #72404)
[Link]
Posted Sep 14, 2022 6:21 UTC (Wed)
by milesrout (subscriber, #126894)
[Link] (4 responses)
Posted Sep 14, 2022 11:30 UTC (Wed)
by jezuch (subscriber, #52988)
[Link] (2 responses)
Python, on the other hand... ;)
Posted Sep 14, 2022 11:47 UTC (Wed)
by Wol (subscriber, #4433)
[Link] (1 responses)
Cleanliness and simplicity in design is just SO important! I go on about WordPerfect, about INFORMATION (sadly defunct but my favourite Pick), precisely *because* they set out to have simple logic / design and clean implementation.
I've been tripping over a Word bug for ages, and I've suddenly realised the problem - if a table extends over multiple pages it screws up printing! When printing labels, I often have grief with the printer so I get the first page and nothing else. If you try to start printing from page 2, it seems Word gets lost, assumes the entire table is on page 1, and prints nothing! WTF?
Cheers,
Posted Sep 14, 2022 15:42 UTC (Wed)
by rschroev (subscriber, #4164)
[Link]
It's sad of course that workaround like this are needed for any application, and double (or more) so for a high-profile word processor.
Posted Sep 14, 2022 15:21 UTC (Wed)
by khim (subscriber, #9252)
[Link]
Seriously? Modern, multithreaded OS kernels exist which means you should be able to create something like that in jiffy. Show me how, please. After we would benchmark it and compare to Linux and other popular kernels you would have a case. And in both cases it took more than a decade to develop these. I still remember time when rebind only existed in the standard, but not in G++. The fact that there are “huge amount of prior art” doesn't assert certain feature is easy, sorry.
Posted Sep 13, 2022 16:12 UTC (Tue)
by ssokolow (guest, #94568)
[Link] (17 responses)
No, the "leakpocalype" (There's your googleable keyword) was specifically about the realization that the original version of the scoped threads API that got re-added recently was unsound because it didn't account for Arc<T> and a reference cycle allowing you to leak memory and, as such, allowed code not marked with "unsafe" to create pointers that outlived what they pointed to. There was nothing "quietly" about it, and the thing they didn't have time to properly fix before 1.0 was the scoped threads API. It was already clear that preventing memory leaks in the the general case was a Rice's theorem problem. (Among other reasons, it's a good hint it might be when you can "leak memory" in the "same externally observed properties" sense in JavaScript just by losing track of which event handlers you've forgotten to unregister.) Ada is more focused on constraining integer types, which requires more runtime support, which makes it a worse fit for interoperating with an existing C kernel. Also, I don't have a citation for this, but I remember reading a comment that it's misleading to look at GCC's platform support and assume that GNAT has proper support for all those platforms. (And then there's the fact that Ada's more Wirth-style syntax feels more alien to the average 21st-century programmer than Rust's "Ocaml in a C++ trench coat" syntax does, and part of the interest in Rust is in heading off the "COBOL programmers get paid a ton because demand is outstripping supply" problem the kernel might face in the coming decades.) Citation, please. The people I've seen tend to say that it's an impressive exercise in pushing the boundaries for what can be achieved without tracing garbage collection.
Posted Sep 13, 2022 21:02 UTC (Tue)
by NYKevin (subscriber, #129325)
[Link] (16 responses)
https://doc.rust-lang.org/std/boxed/struct.Box.html#metho...
Posted Sep 14, 2022 1:54 UTC (Wed)
by milesrout (subscriber, #126894)
[Link] (1 responses)
Posted Sep 14, 2022 17:14 UTC (Wed)
by lambda (subscriber, #40735)
[Link]
mem::forget was the original variant of this, and it was stabilized as safe pre-1.0 https://doc.rust-lang.org/std/mem/fn.forget.html . It was originally marked as unsafe (in earlier pre-1.0 releases), but when it was realized that safety invariants couldn't depend on destructors being guaranteed to run, it was changed to be marked as safe in order to make that point more clear.
Box::leak is just a newer variant that provides a nicer API, allowing you to extract a reference to the leaked value at the same time.
It was always known that it was possible to leak memory by forming a reference counted pointer cycle, but it was fairly late in the pre-1.0 process that folks realized the interaction between this fact, and some proposed APIs that required blocking in a destructor in order to keep a stack frame alive, were incompatible. There was a long and reasoned discussion about it, and the Rust team decided that because it's not possible to categorically prevent resource exhaustion in a Turing complete language, that it was better to have the rules for what could be done in safe code include leaking memory (or otherwise failing to run destructors), and then code that provides safe abstractions has to keep that in mind, so you couldn't rely on blocking in a destructor to keep a stack frame alive.
It took some time in the development of Rust to figure out what the rules should be for what is marked as "safe" vs "unsafe"; you might imagine that any kind of undesirable behavior, like memory leaks, should be considered unsafe. But the distinction that the Rust project ended up on, and I think is the right decision, is that unsafe is only required for operations which could lead to undefined behavior. Rust provides tools to help prevent other kinds of undesirable behavior, like destructors that free memory for you when exiting a stack frame, but it won't refuse to compile code in which you set up a reference cycle and thus leak memory, as the behavior in that case is perfectly well defined, you just use more resources and may eventually run out.
Does this make sense to you? I feel like you are implying that there has been some massive rug-pull about what safety guarantees Rust provides, while Rust never guaranteed freedom from resource exhaustion, and during the run-up to 1.0 there was some learning about exactly the kind of guarantees you could make about when destructors were run, and some updates to the understanding of what unsafe code could rely on based on that and some engineering tradeoffs. If I'm misunderstanding your implication, or not explaining this clearly enough, let me know!
Posted Sep 14, 2022 3:08 UTC (Wed)
by khim (subscriber, #9252)
[Link]
If language is serious about memory leaks then it have to provide tools which leak memory. This may sound paradoxical, but it's not. If you define “memory leak” as something crazy and mostly useless (like tracing-GC languages do: it's not a memory leak if objects can be accesses by “live pointers”… even if program have no code which may actually ever access these objects) then you may “prevent them” but this wouldn't do anyone any good. Either your language allows memory leaks by losing track of which event handlers you've forgotten to unregister or it's not a general-purpose language. And if you admit that layman-definition memory leaks (as in: something which takes infinite amount of memory as time goes on is a memory leak, period) are impossible to prevent then having function which is specifically dedicated to creating persistent objects is obvious choice: this means that objects generated by that function can be treated specially by sanitizers, etc. Lisps typically have these these to implement interning, e.g.
Posted Sep 14, 2022 6:22 UTC (Wed)
by milesrout (subscriber, #126894)
[Link] (12 responses)
Posted Sep 14, 2022 6:35 UTC (Wed)
by zdzichu (guest, #17118)
[Link]
Posted Sep 14, 2022 9:10 UTC (Wed)
by wtarreau (subscriber, #51152)
[Link] (9 responses)
Are there really people who can parse this ? And how is that supposed to be pronounced ? That's still one of the biggest showstopper for me in this language, coding entirely with smileys. There are 16 symbols for 24 alphanumeric chars in this statement, I have no idea what this could mean at all nor whether that generates some code or only declares stuff. Go find a typo there... It's totally cryptic to me :-(
Posted Sep 14, 2022 10:28 UTC (Wed)
by ssokolow (guest, #94568)
[Link] (4 responses)
Hey, it's a lot easier to understand than C function pointer syntax and you get a lot more guarantees out of it too. For the record, that says: That "A" generic is the allocator and defaults to the configured global allocator, which defaults to libc's malloc, so the "where A: 'a" part is saying that it will return a mutable reference that must not outlive allocator that was used to allocate what it points to. It comes from the declaration for the Box struct as a whole: ("?Sized" is used to opt out of the default "only types with sizes known at compile time" constraint... which is particularly relevant here because, in type constraint terms, Box is a wrapper to convert an unsized type into a sized type.)
Posted Sep 14, 2022 15:16 UTC (Wed)
by milesrout (subscriber, #126894)
[Link] (3 responses)
If you have A, an array of pointers to functions that take an integer, a function taking int and returning int, and a pointer to a string, and which returns void*, how do you call it?
A[0] // get one of the function pointers
How do you write the type?
int (*fps[])(int, int(*)(int), const char *);
It's almost exactly the same as calling it. This is well known. It is not difficult or complicated, and is far easier to understand than the line noise that is Rust or Perl. In Rust or C++ you need to learn about a hundred different sublanguages to learn the language proper. In C, the declaration and use syntax are almost exactly the same. Once you know one, you know the other.
Posted Sep 14, 2022 16:15 UTC (Wed)
by khim (subscriber, #9252)
[Link]
And in LISP it's even more uniform thus easier, right? Everyone uses lisp, right? No? Not everyone? But why? Think about it. If you forget about the fact that for function which returns function (like already mentioned void (*signal(int, void (*)(int)))(int)) it doesn't work. That function have two arguments yet ends with Yes, C is simple if you ignore details. Which invariably bites you in the ass later. While Rust makes sure you wouldn't forget about details. Yes, it's different strategy, but it works much better for the large or huge projects where hundreds or thousands people work on the same codebase.
Posted Sep 14, 2022 19:27 UTC (Wed)
by ssokolow (guest, #94568)
[Link]
https://c-faq.com/decl/spiral.anderson.html
I never had to refresh my memory of how Rust type signatures worked back in the days before I started using it actively.
Posted Sep 20, 2022 22:25 UTC (Tue)
by amarao (guest, #87073)
[Link]
So when you mock rust for syntax, don't forget about !!a>>b%c&&--d--.
Posted Sep 14, 2022 11:04 UTC (Wed)
by excors (subscriber, #95769)
[Link] (1 responses)
I think it's not that complicated if you start learning Rust from a book that gradually introduces all the concepts and syntax (like https://doc.rust-lang.org/book/). But you can't expect to automatically know what it means without putting some effort into learning the language, even if you're already very familiar with e.g. C++. You can start with the basic syntax for declaring a struct, and an associated function that can be called with an argument of that type, which is pretty straightforward: Then you make it generic by replacing 'i32' with a type parameter T (similar to C++ templates): Instead of returning a copy of the value, you might want to return a reference. References are either shared ("&T") or exclusive ("&mut T"), where an exclusive reference cannot exist simultaneously with any other reference to the same object. Since Box::leak consumes the object and returns a single reference, it can be exclusive: But reference types have lifetimes (so the compiler can verify the reference won't outlive the object it's referring to). That's one of the major new concepts in Rust, so you need to spend time learning about that. Often it all works automatically with no extra syntax and you don't even need to think about it, but if you're doing something slightly tricky then you need to explain it to the compiler. For Box::leak we can safely return a reference with any lifetime (because that will always be shorter than the (infinite) lifetime of the leaked object), so we make the function generic over an unconstrained lifetime parameter 'a (similar to making it generic over a type T), and use that lifetime for the reference: There's also a potential issue if T contains references to other objects - even though the leaked T is given an infinite lifetime, it may refer to objects with shorter lifetimes, and it's not safe to access the T after those objects' lifetimes. So we add an extra constraint to say that T's lifetime (which (I think) is defined as the lifetimes of any references contained within T) must be as long as 'a: But that's not actually required for safety (the original patch says "Technically not needed, but kept to be explicit." and it was removed in later versions) - I think the borrow checker already prevents that potential issue. And that matches the original definition of Box::leak. (It doesn't quite match the GitHub issue title you quoted, but they just improvised syntax to make it more concise. And it doesn't quite match the modern definition of Box::leak because they added the allocator type parameter.) It looks intimidating at first, but I think it's okay when you go step by step with a willingness to learn. And this is a relatively complicated example - most Rust code doesn't need to be explicit about lifetimes, the default behaviour is usually sufficient.
Posted Sep 14, 2022 14:59 UTC (Wed)
by mbunkus (subscriber, #87248)
[Link]
Posted Sep 14, 2022 11:08 UTC (Wed)
by khim (subscriber, #9252)
[Link] (1 responses)
Sure. It's actually easier to parse than something like void (*signal(int, void (*)(int)))(int);. I mean: is it a function? Or pointer? Where are arguments? Heck, how many arguments are there? Rust code can be parsed from left to right, at least. How is the definition of And 13 symbols for 20 alphanumeric chars in C is, somehow, radically better? Well… if you don't know the language, of course it would be cryptic. But Rust syntax while ugly, serves a purpose: it makes code similar to C++. And, as I have already mentioned, is actually easier to parse than C or C++. Yes, it's punctuation-heavy, but you can blame C and C++ for that: most sigils have come into Rust from there. Only I still hate Rust syntax (much less elegant than it could have been), but it's useful uglyness: it makes Rust look superficially similar to C++. Which is important mimicry not to spook C++ developers before they would be hooked.
Posted Sep 14, 2022 19:33 UTC (Wed)
by Dr-Emann (guest, #136829)
[Link]
Posted Sep 14, 2022 22:25 UTC (Wed)
by NYKevin (subscriber, #129325)
[Link]
Posted Sep 13, 2022 18:23 UTC (Tue)
by lambda (subscriber, #40735)
[Link] (17 responses)
> So in other words: oh yes, there are still bugs, but we can just blame them on C. Any issues with interfacing Rust and C? Obviously the fault of C.
The fundamental unique feature of Rust is that it enables you to build safe abstractions around unsafe code (we'll get to what "safe" means in a bit). Whether you are interfacing with C, or defining your own primitives in Rust, doing that can be tricky, but once it's done, you have fairly strong guarantees on what can happen in the safe code that uses those abstractions.
So, this isn't "blaming bugs on C", but just an acknowledgement that Rust is not a silver bullet (and has never been intended as such), and that the fundamental work of building safe abstractions over unsafe code still requires care. Since there are a large number of abstractions that already exist in the Linux kernel, interoperating with them requires work, and there will be bugs in the process.
> In other words "no we haven't actually tested it but I'm sure the language prevents bugs, they're totally impossible".
You're attacking a strawman here. No one has ever said that bugs are totally impossible.
> Anyone noticed that 'unsafe' as a general concept has suddenly been redefined to mean 'whatever Rust prevents'
"Unsafe" in Rust means "can do things that could potentially be undefined behavior" or in other words, could potentially have executions which have no valid behavior under the language model. And as a Turing-complete, general purpose language, Rust is not intended to prevent you from making logic bugs, infinite loops, or resource leaks; it does have a number of design features that help make them more difficult to make (RAII, exhaustive match statements, etc), but it makes a hard distinction between things that it prevents categorically, and things that the language design helps with but doesn't prevent entirely.
And this definition is part of a contract between the compiler, standard library, and libraries that provide safe abstractions over unsafe code. There needs to be a definition of what rules such libraries need to follow, and what assumptions they can make, so that you can combine two different libraries which each provide safe abstractions over unsafe code, and the combination itself will continue to be safe.
So this particular set of rules are "whatever Rust prevents." As an example, pre-Rust 1.0, there were some API designs that were safe only if it was impossible to leak an object and return to the caller; but of course, reference counted pointers could be set up to cause a cycle and leak an object. There was a long involved discussion of this, with various proposals, but in the end, it was decided that categorically preventing leaks would impose too much burden on language and library design; for instance, it would require a much more complicated system for reference counted objects in order to prevent circular references, and impose similar design burdens many other kinds of data structures. So rather, it was decided that leaking objects wasn't something that would be categorically prevented in safe code, and libraries that provide safe abstractions over unsafe primitives can't depend on an object not being leaked for their safety guarantees.
So that's just part of the contract now; there were alternative possibilities for that contract, which would have had different tradeoffs, but this was the tradeoff chosen.
Effectively, what is forbidden in safe Rust is anything which could cause objects to be interpreted as the wrong type, accessed when they are not valid, or accessed in overlapping ways in space or time. This means no out of bounds access, use after free, data races (two threads accessing the same memory in ways that no linear interleaving could produce), iterator invalidation, etc. This is a category of bugs which are quite common in C and C++ programs, which are difficult to reason about because they effectively break the model of the programming language, and are quite commonly prone to exploitation by attackers.
Leaking an object, on its own, does not lead to arbitrary behavior; it's undesirable, and can lead to resource exhaustion, but in any Turing complete language you could write code which never halts or uses up an arbitrary amount of resources.
> because they didn't have time to properly fix it before 1.0
This is not true. It was not the case that "they didn't have time to properly fix it", it's that there is no way to statically prevent leaking of resources in a way that would not be overly burdensome on the API of data structures.
> This was despite months and years of telling people how wonderful Rust was because it prevented memory leaks, lol.
No one has ever claimed that Rust prevented memory leaks. Reference counted types have existed from the beginning, and everyone has always known that it's possible to create reference counted cycles and leak memory that way. Anyone who thought that Rust was supposed to prevent all memory leaks likely misinterpreted the term "memory safety", which is a term that predates Rust, and refers to preventing unsafe access to objects, not preventing leaking of objects.
> Rust is 'trusting trust' on steroids, for one thing.
There are at least two independent bootstrap paths currently possible for the Rust compiler; you can start from the original pre-1.0 compiler written in OCaml and bootstrap from release to release that way. Or there is an independent compiler, mrustc, written in C++ which is able to compile much more recent releases of rustc and start the bootstrap chain that way. This provides the diverse double compilation necessary to defeat trusting trust attacks.
> Nobody has EVER explained what it provides to the kernel that Ada or better static analysis tools for C could not.
Kernel developers are interested in writing kernel code in Rust, not Ada. Ada is far more foreign, and wasn't designed for the same kind of interop with C and encapsulation of unsafe code behind safe interface that Rust was.
A lot of people focus on the safety guarantees of Rust, because it has a unique approach to safety that no other language provides. But besides that, it's a modern language with a lot of nice features that many developers appreciate, and with a focus on balancing those safety goals with usability, performance, and interop with existing ecosystems like C.
"Better static analysis" can only get you so far. The language level support that Rust provides means that you design your APIs around the type system and lifetimes, which means that code can be statically checked independently and separately; the lack of such guarantees and annotations in C makes it far more difficult to add static checking that is anywhere near as robust as what Rust provides out of the box. It's not for lack of trying; people have been writing static checkers for C and C++ for decades, and yet memory safety errors still cause the majority of security vulnerabilities in codebases written in C and C++.
> It's specified as 'whatever rustc does'. It has one implementation, with no other implementations even remotely close to being ready
You realize that the same was true of C in the kernel until relatively recently, right? The kernel is not written in standard C; it's written in GCC C. The kernel has a memory model that is different than the standard C memory model. The kernel is now mostly able to be built with clang as well, but only by years of effort of adding GCC features to clang and modifying the kernel to not rely on them in quite as many places.
There is a Rust reference, and ongoing efforts on continuing to specify Rust, but this is mostly irrelevant to its usage in the kernel.
> There's a huge list of nightly (basically brand new!) compiler features that are required to build it. No way should there be code in the Linux kernel that requires anything other than stable language features.
An unstable feature in Rust is much like an implementation-specific feature in C; and the kernel uses plenty of GCC features. It's a way of providing some features without committing to supporting that feature in exactly that form indefinitely; there may be backwards incompatible changes in the future. Implementation-specific features in GCC may be replaced by standardized C features in the future, but that doesn't mean they aren't used in the kernel.
The entirety of the internal Linux kernel API is considered unstable, Linux only provides a stable interface to userspace; does that mean that no one writing code in one part of the kernel should depend on code in another part because it's unstable? No, it just means that changes to libraries within the kernel may need to be propagated to usages as well. The same is true of these unstable Rust features; you opt into a few, with the knowledge that you may need to make changes later on when upgrading compilers. It's something you shouldn't use willy-nilly, but using a few features where you may need to make changes in the future is not that big a deal.
These aren't being used with abandon; there is a tracking issue explaining all unstable features, and standard library config flags, what they are needed for, how essential they are, whether they're on track to stabilization, etc: https://github.com/Rust-for-Linux/linux/issues/2
Some of them will be stabilized in Rust, some will be removed from the kernel, and some will be lived with.
> (basically brand new!)
Ok, besides the fact that as someone else pointed out, this feature was agreed to be stabilized within hours of your post (it had already been in final comment period for a while), it's funny that you call these features "basically brand new" while also acknowledging that some of thems have been undergoing serious development for years.
> Nobody has even managed to represent the Wayland memory model in Rust, and the kernel is far more complicated than that. The one serious attempt to do so failed after months of work because it required writing thousands upon thousands of lines of memory management boilerplate. If you want to represent anything more complex than Box or Rc you need to just about write a thesis.
There are hundred of crates that provide memory management primitives beyond Box and Rc (https://lib.rs/memory-management), plus numerous examples of successful integration of Rust into C and C++ projects with custom memory management, including Firefox.
There was one project in which someone attempted to provide a general-purpose rust API over wlroots, a C Wayland library, and eventually decided that it wasn't worth it for their purposes. That doesn't mean it's impossible; just that one person tried one approach, didn't like it, and decided it wasn't what they wanted to spend their time doing.
> And last I checked, 'async' in Rust is widely considered a failure of design.
So much of a failure of design that it's being used for substantial production usage like an entire rewrite of the Tor client in Rust in two years: https://blog.torproject.org/arti_100_released/
There are gripes that folks have with aspects of async in Rust; but it's actually a very well received, widely used feature, that just happens to be a little bit young and has had some ecosystem growing pains.
Posted Sep 14, 2022 9:45 UTC (Wed)
by wtarreau (subscriber, #51152)
[Link] (11 responses)
But it could also be said that several other categories of bugs are avoided in C thanks to the language being quite primitive and reading fairly well and being suitable for peer reviewing. Do you have an estimate of the increased amount of logic bugs or algorithmic ones that could be caused by the language being significantly more difficult to use when it resists to your demands ? For example I've been caught many times adding bugs when trying to simply shut up an inappropriate gcc warning. When a compiler tries to force you to do something one way that doesn't match your need, the friction introduces new risks of bugs.
> > This was despite months and years of telling people how> wonderful Rust was because it prevented memory leaks, lol.
Note, the two of you said at least once "nobody shows" or "nobody claimed" etc. It's pointless to use such rhetoric. It doesn't add any value and needlessly increases tensions because anyone can have one personal counter example. I've personally heard someone tell me the point above for example, and that irritated me because I knew it was an absurd claim. Actually saying "no authoritative developer said/demonstrated/claimed", or even better "I've never heard any ..." would be easier to deal with for both parties in the discussion.
> > It's specified as 'whatever rustc does'. It has one implementation, with no other implementations even remotely close to being ready
I can understand this concern and I do share it as well. Not directly for the kernel in fact, rather for the language's life expectancy. 15 years ago I was told that Ruby was *the* language of the future, that prevented bugs etc... (hint: it just made them slower to appear). Now in 2022 can anyone cite any developer not working for Gitlab still using this language ? I do have the same concern about Rust: as long as it remains the self-defined input of rustc, it's not exactly a language and it can seriously fail over time. Serious implementations are absolutely required for it to survive. For sure Linux uses GCC C. But C is used everywhere and runs the whole internet, some built with gcc, some with any other compiler. It maintains an ecosystem afloat and forces implementations from various origins and use cases to exchange and evolve the standard. Rust does need to adopt a similar approach where there is no more *the* leading implementation and a few others trying to catch up like clang does with gcc or gnugo does with Go, but a set of slightly different implementations all following one standard to reach a 100% compatible code base. From there it's fine if some projects decide to only use one flavor for various reasons.
Posted Sep 14, 2022 10:53 UTC (Wed)
by Wol (subscriber, #4433)
[Link] (6 responses)
That is incredibly difficult to achieve. For two perfect examples from the database arena, SQL and DataBASIC. There's a whole bunch of subtle differences between SQL dialects, as many people here will be able to attest. Likewise, although far fewer people here are familiar with it, DataBASIC. Both have multiple competing implementations, and there are many corner cases where early design decisions collide badly with compatibility - my favourite DataBASIC statement
REM: REM = REM(6,3); REM this takes the remainder of 6 / 3
Every single usage of REM makes sense, and is legal in at least one DataBASIC compiler, but trying to support all four in this one statement is, well, tricky ... (I believe at least one does, probably OpenQM/ScarletDME.)
We're likely to end up with just the one implementation of rust just to get round the dialect problem. Like most C code is written to the "it compiles with gcc" standard for exactly the same reason.
Probably one of the big drivers pushing the kernel towards llvm/clang is too many developers are getting fed up with the breakages caused by the gcc developers attitude towards "undefined behaviour". And if we do get the kernel compiling successfully with llvm/clang we could rapidly hit a tipping point where new code supports the "well it compiles with llvm/clang" standard.
Cheers,
Posted Sep 14, 2022 15:42 UTC (Wed)
by khim (subscriber, #9252)
[Link] (5 responses)
I really like how Rust solved that crazy provenance business. Instead of trying to invent rules which would work for everyone (that's what C/C++ attempted, but failed to do and thus and still, after 20 years, doesn't have such rules, remember!) Rust just gives you rules which you can use! And then its developers go back to their blackboard to try to invent something better. What is surprising is that this is what was supposed to happen in the C land, too: Undefined behavior gives the implementor license not to catch certain program errors that are difficult to diagnose. It also identifies areas of possible conforming language extension: the implementor may augment the language by providing a definition of the officially undefined behavior. Only in Rust case they actually do that, instead of trying to find an excuse to justify yet-another-way which compiler is allowed to break your program. C++ once, long ago, was like that, too: it split C-style cast into But somehow in XXI century all that went out of the window. We can only hope Rust wouldn't repeat the same mistake.
Posted Sep 15, 2022 13:03 UTC (Thu)
by farnz (subscriber, #17727)
[Link]
The really nice thing about the Tower of Weakenings approach is that Rust is now able to have several layers of rules for provenance. Strict provenance is guaranteed to be correct for all implementations of Rust on all hardware that can support Rust; but because you have this portable set of rules, it's now possible to define rules like "for Rust on AArch64" or "for single-threaded Rust programs" that only apply if you're a special case.
In C and C++ standard terms, this has "strict provenance" as the rules that must apply, while permitting implementations to define relaxations of strict provenance that they will also accept as valid.
Posted Sep 15, 2022 19:34 UTC (Thu)
by Wol (subscriber, #4433)
[Link] (3 responses)
As I've said before, the C/C++ standards committee should be removing undefined behaviour. Placing the onus on the compiler writers to provide implementation-defined behaviour. Saying it's "whatever the hardware does". Whatever whatever but getting rid of all that crap.
And then there are things you can't define for whatever reason, where you admit that a definition is impossible.
The thing is, Rust has all three of those, and it clearly pigeonholes them. Safe Rust is supposedly *only* *defined* behaviour. And if undefined behaviour creeps into safe code it is defined as a BUG, a MUST-FIX.
I guess all that "hardware defined" stuff probably belongs in the "unsafe Rust" category, where the language can't reason because it doesn't have any idea what's going to happen behind its back.
And then there's the stuff you can't define, which is unsound, because there's some fault in the logic somewhere.
The important thing is, the programmer can REASON about all this lot, unlike C, where hardware behaviour triggers "undefined behaviour", and the C compiler makes a whole bunch of false assumptions and screws up your code (like deleting necessary safety checks, etc etc).
Cheers,
Posted Sep 15, 2022 20:19 UTC (Thu)
by khim (subscriber, #9252)
[Link] (2 responses)
But the thing is: this is what was supposed to happen with C and C++, too! Except for safe subset, but otherwise it was planned like that. I mean… the Rationale for
International Standard— Programming Languages— C says: Undefined behavior gives the implementor license not to catch certain program errors that are
difficult to diagnose. It also identifies areas of possible conforming language extension: the implementor may augment the language by providing a definition of the officially undefined behavior. This is your Tower of Weakenings right there! Note that before C89 was punished it actually worked that way: there was no standard but different implementations permitted different things and even if some were not so easy to implement (e.g. one-element-past-the-end-of-array means it becomes impossible to have simple 64KiB arrays on MS-DOS) they were added to standard where it made sense. I wonder how that stance turned into “if standard says something is undefined behavior then we have the carte blanche to destroy the program” and then “if standard doesn't say something is undefined behavior yet then we have the permission to destroy your program anyway”. I don't think there was some evil mastermind behind all these developments, but the end result sure is complete lack of trust. Periodic Linus outbursts and public complaints is not how you plan development of language which is used by millions!
Posted Sep 15, 2022 21:35 UTC (Thu)
by mathstuf (subscriber, #69389)
[Link] (1 responses)
Portability I would guess. Once more than one compiler could target a given platform (or one compiler could target more than one platform), "my compiler/platform is better than yours" creeps in and you start down the path of "what kinds of optimizations can we squeeze out here?" come up.
Today? Code that is written to work on multiple platforms from the same source. Here, the compiler saying "well, if it were $obscure_arch, this is different behavior, so we'll show it to you on your machine via UB-based optimizations (but not make any noise about it either)".
On one hand, a UB-less C would be "safer", but its portability would tank because "it worked on my x86_64" means diddly squat when you compile it for aarch64.
Posted Sep 15, 2022 22:33 UTC (Thu)
by Wol (subscriber, #4433)
[Link]
You've missed "implementation defined" and "hardware defined".
If something is "hardware defined" then yes, just because it works on x86_64, you can't expect the SAME code to work on aarch64, but firstly the programmer will KNOW that they need to check behaviour, and secondly they can put the ifdefs and whatever in there, and know that that IS DEFINED behaviour.
The *only* grounds for UB should be because "we can't define it because we can't get the logic to add up". There's no need for the C/C++ standard to define everything itself - it can defer the definition to something else - but all behaviour should be defined *somewhere*, if a definition is possible.
Take for example the size of a byte. In *PRACTICE* it's always 8-bit nowadays. I wouldn't be surprised if it's actually already implementation or hardware defined, but that's a perfect example of something that makes perfect sense as hardware-defined. In places, bytes are 6 bits, and if the programmer doesn't account for it it will cause a major problem if they're targetting an old platform. But the standard CAN, and SHOULD, address it.
Cheers,
Posted Sep 14, 2022 13:56 UTC (Wed)
by tialaramex (subscriber, #21167)
[Link]
If the warning is inappropriate in Rust, simply explain why in the source code
// We genuinely need Drop here, see https://some.example/url
And we can promote a warning in the opposite way
// We tried asking people nicely, it didn't work. If you write an overlapping range this won't compile. Learn to count.
Posted Sep 14, 2022 15:35 UTC (Wed)
by lambda (subscriber, #40735)
[Link]
I have not noticed any such tendency in Rust.
One of the advantages of Rust is that the more powerful type system, and a number of language design features, make a lot of things more explicit and possible for the compiler to reason about precisely. For instance, reference types and nullability are orthogonal, so you don't have to constantly add checks for null; the type tells you if a reference could possibly be null, and so there there can be fewer spurious compiler warnings due to the higher precision of the type system.
Another example would be warnings about use of uninitialized value, like the famous Debian SSH key bug that was introduced by trying to silence a warning about use of uninitialized values. Because this warning was found by someone later who wasn't the original author, they weren't as familiar with the code when trying to fix it, and they made a mistake and removed the actual source of entropy that was being used as well as the uninitialized value. In Rust, this is not a separate warning, but part of the language, so it's something that needs to be dealt with by the original author, rather than by someone else later on trying to silence warnings and not paying enough attention.
That's one of the major design goals of Rust; rather than having to rely on imprecise lints that can frequently lead to spurious warnings and dubious fixes, to have greater expressiveness in the language itself that allow these checks to be precise and enforced consistently, which leads to less confusion.
Usually, the kinds of workarounds that you need to do in cases where the compiler gets it wrong are to just be a little bit more explicit, possibly at the cost of being more verbose. I don't know of many cases where this has caused the introduction of bugs; I'm sure it could happen, but in my experience it seems like the additional expressiveness and precision of the type system far outweighs that, leading to many fewer of these kinds of bugs than you find in C.
> Note, the two of you said at least once "nobody shows" or "nobody claimed" etc. It's pointless to use such rhetoric. It doesn't add any value and needlessly increases tensions because anyone can have one personal counter example.
Sorry, fair point! This is a somewhat common misconception, so you're right, my rhetoric was probably too strong here.
> 15 years ago I was told that Ruby was *the* language of the future, that prevented bugs etc... (hint: it just made them slower to appear). Now in 2022 can anyone cite any developer not working for Gitlab still using this language ?
Ruby seems like a fairly different case than Rust, but off the top of my head, Homebrew, Vagrant, Discourse are all fairly widely used tools written in Ruby; and of course, Ruby on Rails is still a quite popular framework for writing web apps, though many of them are non-free, simply SaaS applications.
> I do have the same concern about Rust: as long as it remains the self-defined input of rustc, it's not exactly a language and it can seriously fail over time.
There are plenty of other successful, long-lived languages. Python has been around for as long as the Linux kernel, and it is defined by a single primary implementation, while also having alternate compatible implementations that are useful like PyPy, and Python is widely used for a large variety of software.
Rust is younger, and thus its alternate implementations are younger and not yet as complete, but it has one independent implementation mrustc which can be used for bootstrapping the compiler, it has another completely independent implementation in the gcc-rs project, and it has a GCC-based backend being added to rustc to supplement the LLVM based backend. The progress on these two implementations has been discussed on LWN recently: https://lwn.net/Articles/907405/
I've also heard rumors that there are other implementation projects that haven't yet been made public; of course those could never see the light of day, but there is a lot of active work in this field right now.
There is also the Rust reference, there's an extensive test suite, there's the entirety of crates.io which is used as an additional test suite, and there's a draft Ferrocene Language Specification https://spec.ferrocene.dev/ which is intended to provide a set of requirements that can be verified against for safety-critical applications.
> Rust does need to adopt a similar approach where there is no more *the* leading implementation and a few others trying to catch up like clang does with gcc or gnugo does with Go
I'm not sure I follow; as you're saying here, the situation for Rust is no different than the situation with C in the Linux kernel, where GCC is the leading implementation and clang is catching up. Are you saying that Rust needs to be held to a higher standard, where there are two independent implementations with feature parity before you can use it? I don't think that this is a reasonable requirement.
Yes, there is value in having multiple independent implementations, but there's also substantial cost in writing the new compiler and the standardization process itself. As mentioned, there is work in progress on all of these fronts (alternative implementations, and more detailed specifications/standards), but I don't think there's any reason to avoid using Rust before those are complete.
Posted Sep 15, 2022 19:02 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (1 responses)
Ruby has been overshadowed by Go and JS now that most complex webapps have frontends in JavaScript and the backend just provides a REST API. But back in the day, Ruby allowed tons of small companies to quickly build decent applications and get to market with them.
This list includes GitHub, AirBnB, Groupon, Zillow and many others. The company where I work is built on top of a Ruby app as well.
So it's fair to say that Ruby absolutely fulfilled its promise in the area of web apps. And it has never been really intended as a systems language or a language for desktop applications.
Posted Sep 15, 2022 19:39 UTC (Thu)
by rahulsundaram (subscriber, #21946)
[Link]
Yep, it hasn't advertised as one. Aside from the web arena, tools like Puppet or Chef uses it but that's quite different from being a systems language.
Posted Sep 14, 2022 9:51 UTC (Wed)
by Fabien_C (guest, #160870)
[Link] (4 responses)
Ada/C interoperability is on par with Rust/C interoperability as far as I can tell (not a Rust expert). See https://learn.adacore.com/courses/intro-to-ada/chapters/i...
Using Ada's formal verification subset (SPARK), one can even prove the correct use of a C API/Library at compile time. Very powerful in terms of software correctness, safety, and security.
Posted Sep 14, 2022 16:08 UTC (Wed)
by lambda (subscriber, #40735)
[Link] (2 responses)
Fair enough! I'm not an Ada expert either, so I can't necessarily speak to how the approaches compare.
I can say that I've seen a lot more work in the free software world to incrementally port portions of software to Rust, such as the original motivating example of Firefox, librsvg, curl, and this work in the Linux kernel, than I have in Ada. The mindshare in Ada seems to mostly be around safety-critical systems, while Rust seems to appeal to free software developers more as a general purpose programming language, which provides some better guarantees out of the box than C or C++ do, even when not doing a full formal verification process for safety critical systems.
I'd love to see examples where Ada has been used successfully to rewrite parts of free software to improve safety or maintainability, let me know if you know of any!
My comments about Ada were mostly to respond to why Rust over Ada or other static analysis tools, and while I don't know Ada well enough to do a detailed comparison, there just seems to be a lot more interest in using Rust for these kinds of use cases than Ada. If anyone has writeups on why Ada would be good for this kind of use case, I'd love to see them.
Posted Sep 14, 2022 22:02 UTC (Wed)
by khim (subscriber, #9252)
[Link] (1 responses)
The biggest problem of Ada IMO is that it was always supposed to be about safety, but it never addressed the most common source of bugs: pointer safety. Not even with SPARK. It's like discussing about how can you fortify the door in a house with three walls. I suspect they planned to solve it like everyone else (with tracing GC), but that never materialised (because most Ads users don't want tracing GC) thus was always kinda weird “safe” language which doesn't tackle the most common source of bugs. Finally, in year 2020, it solved that problem. By picking ideas from Rust, of course. But by that time momentum was lost and it would be very hard to overcome that “safe language without safety” stigma. In addition Ada very much likes to live in the world where it can dictate the rules thus Rust is much more suitable for the kernel IMO. It would be interesting to see how well Rust would do there. I'm not sure Rust would be able to push Ada from that niche, but it's also highly unlikely that Ada would be able to go into general-purpose computing. Mindsets of Ada programmers and general-purpose computing programmers are just too different. It wouldn't. Ada provides some additional facilities which Rust doesn't provide (such as range types), but these are not dependent types which are needed to express safety and thus add bloat to the language without improving safety much. This about it: the most famous example of range types are months and days… and yet, in Ada, you can not define type for day-of-month which is between 1 and 28 for February and 1 and 31 for January!
Posted Jan 16, 2024 13:56 UTC (Tue)
by yawaramin (guest, #169121)
[Link]
Yes you can:
```
Sure, it's not exactly trivial; but it's possible.
Posted Sep 15, 2022 19:10 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
SPARK doesn't support dynamically allocated objects well: https://docs.adacore.com/spark2014-docs/html/ug/en/source...
It's similar to Rust without lifetime annotations and with less static analysis for stack objects.
There are academic papers that try to fix that, but the outcome would basically look like Rust.
Posted Sep 13, 2022 19:44 UTC (Tue)
by atnot (subscriber, #124910)
[Link]
To quote David Gerard: "could" is a word that means "doesn't"
Posted Sep 14, 2022 8:54 UTC (Wed)
by em (subscriber, #91304)
[Link]
GATs have been stabilized yesterday, fyi: https://github.com/rust-lang/rust/pull/96709
Posted Sep 16, 2022 11:26 UTC (Fri)
by flussence (guest, #85566)
[Link]
Simply write a better language with measurably lower defects per hour invested and then people will use it.
This isn't rocket science (according to you), but it *is* science. Not religion. Code talks, bullshit walks, see you when you've done the work.
Posted Sep 20, 2022 22:09 UTC (Tue)
by amarao (guest, #87073)
[Link]
Posted Sep 13, 2022 13:20 UTC (Tue)
by david.a.wheeler (subscriber, #72896)
[Link]
Posted Sep 14, 2022 0:46 UTC (Wed)
by rsidd (subscriber, #2582)
[Link] (23 responses)
It boggles the mind.
Posted Sep 14, 2022 6:11 UTC (Wed)
by ssmith32 (subscriber, #72404)
[Link]
Posted Sep 14, 2022 8:16 UTC (Wed)
by marcH (subscriber, #57642)
[Link] (19 responses)
Posted Sep 14, 2022 9:20 UTC (Wed)
by milesrout (subscriber, #126894)
[Link] (18 responses)
Posted Sep 14, 2022 9:54 UTC (Wed)
by rsidd (subscriber, #2582)
[Link] (17 responses)
Many of your other comments -- such as they are, omitting the "yeah right", "anyone notice that..." etc -- have been amply rebutted by others above.
As far as the kernel is concerned, if there was a huge issue with Linux's memory model, Linus would have noticed by now, I think.
Posted Sep 14, 2022 10:37 UTC (Wed)
by Wol (subscriber, #4433)
[Link] (16 responses)
Isn't it interesting that milesrout wrote this? Because that is exactly what he is doing ...
Cheers,
Posted Sep 14, 2022 11:12 UTC (Wed)
by corbet (editor, #1)
[Link] (15 responses)
Posted Sep 14, 2022 12:36 UTC (Wed)
by nirbheek (subscriber, #54111)
[Link] (14 responses)
Posted Sep 14, 2022 12:41 UTC (Wed)
by mathstuf (subscriber, #69389)
[Link] (11 responses)
Posted Sep 14, 2022 13:03 UTC (Wed)
by nirbheek (subscriber, #54111)
[Link] (9 responses)
Posted Sep 14, 2022 13:17 UTC (Wed)
by Wol (subscriber, #4433)
[Link]
The difficulty is when you get articles that by their very nature are contentious, and debate can easily get heated. That, sadly, then attracts trolls like moths to a flame.
Which then encourages subscribers to subscribe to a level that permits them to apply their own moderation.
It's like democracy - "the worst form of government, apart from all the others we've tried". Moderation is a bad thing. Just is it better than the alternatives? For the most part, for this site at least, it clearly is not.
Cheers,
Posted Sep 14, 2022 13:21 UTC (Wed)
by corbet (editor, #1)
[Link] (6 responses)
Posted Sep 14, 2022 19:21 UTC (Wed)
by atnot (subscriber, #124910)
[Link] (2 responses)
My opinion is that a large part of the problem is that those two, very blunt weapons are the only ones available. It means that as long as one nominally follows the rules[1], one is completely free to (deliberately or accidentally) abuse LWNs perverse feedback loops to derail discussions: Write the most incendiary thing you think you can get away with, get it as far up the page as possible by being early, fight with every reply and watch your discussion take up vertical space on the page until nobody has the patience to scroll past it to read anything more substantive anymore. At that point you have won by being the loudest and stop replying to avoid getting chastized by Your Editor.
With no mechanism to counteract this, it's unsurprising it keeps happening. Softer measures like hiding, collapsing or locking a thread, sending it to the bottom of the sort order, showing limited reply depth or rate limiting back and forth replies are probably more effective there than hardline moderation.
> Most of the time, asking LWN readers to behave like adults works; I'm not sure what to do in cases where it doesn't.
I think it's remarkable how well this works, all things considered.
Posted Sep 15, 2022 18:03 UTC (Thu)
by marcH (subscriber, #57642)
[Link]
This.
Surprise, surprise, "analog" problems rarely ever have binary solutions (unless of course you read social media).
Posted Sep 16, 2022 3:32 UTC (Fri)
by milesrout (subscriber, #126894)
[Link]
And what would be the grounds for doing this? Is this JUST for criticism of Rust? Or does it include a broader category of messages than that?
In my opinion there is no issue. The discussion isn't even heated. I made a SINGLE comment and was immediately accused of trolling. It seems to me that ANY criticism of Rust immediately gets labelled as trolling. Yet lots of topics covered here are contentious and prompt back-and-forth discussions that often are not particularly productive. Only criticism of Rust seems to get immediately labelled as trolling, as far as I can see.
Posted Sep 15, 2022 7:05 UTC (Thu)
by ssmith32 (subscriber, #72404)
[Link]
It could be automated - other than ticking an "adult warning" on your post. The randomness is to prevent gaming - keeps you wondering if this time is the one.
I know, a bit naive, just thinking what would help me when I get too troll-y. Particularly in my younger years - I'd like to think I'm older and wiser then when I joined. Because I'm definitely older ;)
Posted Sep 15, 2022 13:24 UTC (Thu)
by farnz (subscriber, #17727)
[Link] (1 responses)
Two concrete proposals:
The idea behind the first option is to reduce the visibility of overheated threads, allowing you to indicate that you think that a given thread is non-productive (and discouraging people from either reading it or replying to it). The idea behind the second is to let you use time to cool off a thread that's become heated, or to increase the effort trolls and abusers have to put into actually being seen.
Finally, when I killfile a user, I should have the option to not only killfile that user, but also all immediate replies to them; that way, if I killfile a user who's good at trolling, I don't see the replies that they successfully troll for (but I do see the replies to replies, which may be interesting tangents from the troll).
Posted Sep 16, 2022 5:04 UTC (Fri)
by edomaur (subscriber, #14520)
[Link]
Posted Sep 16, 2022 12:01 UTC (Fri)
by flussence (guest, #85566)
[Link]
Incidentally a major subreddit became Main Character Of The Day on other social media this week for grotesque moderator overreach. I would say the reddit/voat/8chan/urbit model of expendable community and digital royalty built atop it has produced some of the worst results. That website has trench lines by the hundred and the tools are designed to only escalate.
Posted Sep 14, 2022 14:54 UTC (Wed)
by mbunkus (subscriber, #87248)
[Link]
Not sure what the best solution is here; maybe collapse those as well by default? Even if the top-most shown comment isn't by an ignored user? Or at least marking it in an obvious way?
Anyway, I'm very grateful to have that ignore list. For that & the excellent content in general I continue to be a happy subscriber.
Posted Sep 19, 2022 0:50 UTC (Mon)
by ras (subscriber, #33059)
[Link] (1 responses)
As a counter point, I've enjoyed the discussion milesrout has created. Some of the replies have been excellent - the best I've seen on this subject anywhere on the web. They're from people who clearly know the subject very well and also write clearly.
I think that's because they are passionate about the subject. They wear that passion on their sleeves, which I suspect is what you are objecting to - it's not a dry "authoritative" academic treatment of the subject. Yes, that passion could easily drag the discussion into the weeds, but without the passion they would not of invested the time and effort needed to craft such replies.
It's difficult to argue passionately about something, while maintaining the disciplined needed to address the just subject and not the people delivering it. Yet, that is mostly what has been achieved here, in what is effectively an open forum. I put it down to corbet's gentle steering. As other's have said, he limited tools available. His most effective one seems to be leading by example.
Posted Sep 19, 2022 3:24 UTC (Mon)
by marcH (subscriber, #57642)
[Link]
Posted Sep 14, 2022 22:28 UTC (Wed)
by NYKevin (subscriber, #129325)
[Link] (1 responses)
Posted Sep 15, 2022 0:50 UTC (Thu)
by rsidd (subscriber, #2582)
[Link]
A pair of Rust kernel modules
A pair of Rust kernel modules
Or, you could code it in C++, correctly, and be done.
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
Can we please try to avoid yet another round of silly language-advocacy postings here? We've all heard it. The article is about people showing real work, albeit at an early stage; let's focus on the actual work.
Please
A pair of Rust kernel modules
A pair of Rust kernel modules
The easy way to get C++ code into the kernel is via eBPF, which offers solid support for building from C++
Solid? Earlier this year I had a persistent failure getting this to verify (under 5.16):
uint32_t id;mp;
dt_bpf_specs_t zero;
__builtin_memset(&zero, 0, sizeof (dt_bpf_specs_t));
for (id = 1; id <= 16; id++) {
if (bpf_map_update_elem(&specs, &id, &zero,
BPF_NOEXIST) == 0)
return id;
}
A simple loop updating sixteen map values to 0 was rejected... because the verifier could not prove that this incredibly subtle and difficult loop would terminate. (I unrolled it, and everything was fine.)
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
Wol
A pair of Rust kernel modules
Why would a function need a primitive value passed in by pointer unless it intended to modify it?
A pair of Rust kernel modules
struct bpf_map *, map,
void *, key,
void *, value,
u64, flags)
{
WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held());
return map->ops->map_update_elem(map, key, value, flags);
}
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
Wol
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
Wol
A pair of Rust kernel modules
> There's no excuse for it to take this long.
A pair of Rust kernel modules
A pair of Rust kernel modules
Anyone noticed that 'unsafe' as a general concept has suddenly been redefined to mean 'whatever Rust prevents' (if you don't use `unsafe`, which the Rust for linux code does all over the place)? When people involved in Rust's development realised just before the release of the language that the language was fundamentally unsound, allowing memory leaks, they quietly redefined 'safety' to exclude leak freedom, because they didn't have time to properly fix it before 1.0. This was despite months and years of telling people how wonderful Rust was because it prevented memory leaks, lol.
It's specified as 'whatever rustc does'.
As opposed to "whatever GCC does" like the dialect of C that the Linux kernel requires, which LLVM is still working to perfectly replicate?
Nobody has EVER explained what it provides to the kernel that Ada or better static analysis tools for C could not.
And last I checked, 'async' in Rust is widely considered a failure of design.
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
As for that "where" clause, that's incorrect. The signature is actually
pub fn leak<'a>(b: Box<T, A>) -> &'a mut T
where
A: 'a,
pub struct Box<T, A = Global>(_, _)
where
A: Allocator,
T: ?Sized;
A pair of Rust kernel modules
(*A[0]) // dereference the pointer
(*A[0])(1, f, "hello") // call it
> In C, the declaration and use syntax are almost exactly the same. Once you know one, you know the other.
A pair of Rust kernel modules
(int) somehow. And about the fact that standalone variable declared as int x[3]; has entirely different type from argument of function declared in the same fashion. And about bazillion other similar quirks.A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
struct Foo {
val: i32, // i32 is the 32-bit signed integer type
}
impl Foo {
fn bar(f: Foo) -> i32 { // return type is i32
f.val
}
}
struct Foo<T> {
val: T,
}
impl<T> Foo<T> {
fn bar(f: Foo<T>) -> T {
f.val
}
}
fn bar(f: Foo<T>) -> &mut T {
// ...
}
fn bar<'a>(f: Foo<T>) -> &'a mut T {
// ...
}
fn bar<'a>(f: Foo<T>) -> &'a mut T
where T: 'a
{
// ...
}
A pair of Rust kernel modules
> Are there really people who can parse this ?
A pair of Rust kernel modules
signal is supposed to be pronounced?'a is new but Ada uses such syntax, too.
For anyone actually trying to parse the signal type, it helps a lot to have a typedef:
A pair of Rust kernel modules
typedef void (*sighandler_t)(int);
sighandler_t signal(int signum, sighandler_t handler);
The equivalent in rust is still pretty readable without a typedef:
fn signal(signum: c_int, handler: fn(int)) -> fn(int)
A pair of Rust kernel modules
A pair of Rust kernel modules
> generic associated types have been in progress for more than 5 years with no real sign that they're going to be stabilised any time soon.
A pair of Rust kernel modules
> No one has ever claimed that Rust prevented memory leaks.
> You realize that the same was true of C in the kernel until relatively recently, right? The kernel is not written in standard C; it's written in GCC C. The kernel has a memory model that is different than the standard C memory model. The kernel is now mostly able to be built with clang as well, but only by years of effort of adding GCC features to clang and modifying the kernel to not rely on them in quite as many places.
A pair of Rust kernel modules
Wol
> Probably one of the big drivers pushing the kernel towards llvm/clang is too many developers are getting fed up with the breakages caused by the gcc developers attitude towards "undefined behaviour".
A pair of Rust kernel modules
const_cast, dynamic_cast, reinterpret_cast and static_cast for similar reasons.A pair of Rust kernel modules
A pair of Rust kernel modules
Wol
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
Wol
A pair of Rust kernel modules
#[allow(drop_bounds)]
#[forbid(overlapping_range_endpoints)]
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
> I can say that I've seen a lot more work in the free software world to incrementally port portions of software to Rust, such as the original motivating example of Firefox, librsvg, curl, and this work in the Linux kernel, than I have in Ada.
A pair of Rust kernel modules
A pair of Rust kernel modules
procedure Adaproj is
type Month is (Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec);
type Month_Day (M : Month) is record
case M is
when Sep | Apr | Jun | Nov =>
Day1_30 : Integer range 1 .. 30;
when Feb =>
Day1_29 : Integer range 1 .. 29;
when Jan | Mar | May | Jul | Aug | Oct | Dec =>
Day1_31 : Integer range 1 .. 31;
end case;
end record;
begin
null;
end Adaproj;
```
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
Glad to see this!
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
It was a troll. For example, you wrote
A pair of Rust kernel modules
Nobody has even managed to represent the Wayland memory model in Rust, and the kernel is far more complicated than that. The one serious attempt to do so failed after months of work because it required writing thousands upon thousands of lines of memory management boilerplate.
In fact, the developer of wayland-rs rebutted this two years ago, and both wayland-rs and smithay (an alternative to wlroots -- not wayland -- in rust) are actively maintained today. Not in wide use (gnome/libweston dominates, kde and wlroots are a distant second/third, not much space for others). But it's there. Perhaps you are thinking of weston-rs.
A pair of Rust kernel modules
Wol
How about if we all stop calling each other names here? Please?
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
Wol
Out of curiosity, what would you propose that "the editors" do to make things better? Assuming we had the time and stomach to go deleting/blocking posts, what are the criteria we should use? Most of the time, asking LWN readers to behave like adults works; I'm not sure what to do in cases where it doesn't.
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
LWN user ignore list pros & cons
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules
A pair of Rust kernel modules