Huang: Rust: A Critical Retrospective [LWN.net]

Andrew 'bunnie' Huang has posted an extensive review of the Rust language derived from the experience of writing "over 100k lines" of code.

Rust is a difficult language for authoring code because it makes these "cheats" hard – as long as you have the discipline of not using "unsafe" constructions to make cheats easy. However, really hard does not mean impossible – there were definitely some cheats that got swept under the rug during the construction of Xous.
This is where Rust really exceeded expectations for me. The language's structure and tooling was very good at hunting down these cheats and refactoring the code base, thus curing the cancer without killing the patient, so to speak. This is the point at which Rust’s very strict typing and borrow checker converts from a productivity liability into a productivity asset.

to post comments

Huang: Rust: A Critical Retrospective

Posted May 19, 2022 17:58 UTC (Thu) by IanKelling (subscriber, #89418) [Link]

I enjoyed this blog post. I have a precursor. All I've done is reflashed it so far. While I haven't yet analyzed the source licenses in detail, it seems it is probably running entirely free software. The other modern computer that comes to mind which I actually know is entirely entirely free software is a raptor computer without a hard drive. It seems like a wonderful device to learn about hardware and rust.

Huang: Rust: A Critical Retrospective

Posted May 20, 2022 8:48 UTC (Fri) by kleptog (subscriber, #1183) [Link] (11 responses)

Very nice blog post with some valid criticisms but it's also impressive that they managed to build an OS on Rust. It's clearly not a well supported use-case but it does show what's possible.

Huang: Rust: A Critical Retrospective

Posted May 20, 2022 9:16 UTC (Fri) by pabs (subscriber, #43278) [Link]

There are other operating systems written in Rust:

https://www.redox-os.org/
https://en.wikipedia.org/wiki/Rust_(programming_language)#Operating_systems
https://github.com/flosse/rust-os-comparison

Huang: Rust: A Critical Retrospective

Posted May 20, 2022 13:55 UTC (Fri) by khim (subscriber, #9252) [Link] (9 responses)

There are, actually, a lot of work done in Rust to make it suitable for OS development.

Sure, it's not raison d'etre of Rust existence, and not the primary target of language development, but it's a pretty well-supported target.

Huang: Rust: A Critical Retrospective

Posted May 24, 2022 16:46 UTC (Tue) by jmalcolm (subscriber, #8876) [Link] (8 responses)

There is enough happening on the Rust OS front to show that Rust is certainly a viable choice for OS Dev. A very interesting development though is the SerenityOS folks not only rejecting Rust in favour of creating their own language ( Jakt ) but in fact choosing Rust to code the first implementation of it.

https://www.reddit.com/r/contextfree/comments/utqnsm/memo...

Huang: Rust: A Critical Retrospective

Posted May 24, 2022 19:05 UTC (Tue) by khim (subscriber, #9252) [Link] (5 responses)

It's not that interesting, really. And while I wish them luck this would mean that this OS would forever stay a love letter to '90s user interfaces with a custom Unix-like core and wouldn't turn into the next Linux.

I understand the temptation to create a specialized language for an OS development, this may even be right thing to do for the academic project, but as fate of Oberon, Singularity and many other OSes shows this means you OS would be a purely academic/hobbyist world experience. Simply because you wouldn't have a robust compiler for everything-and-anything (which GCC/LLVM provide and which Rust, with it's popularity, may, theoretically, support).

Also: while Jakt certainly sounds interesting it looks like it would try to be another Swift-like language (kinda-sorta-safe-but-not-really) and not a replacement for Rust.

Simply from their small blog post where they boldly proclaim that they would have OOP and classes it's easy to see that they have almost given up to the templation of supporting implementation inheritance which means they don't understand what correctness even mean.

Rust fight against it tooth-and-nail because, simply put, implementation inheritance is SUID-level mistake: easy hack to solve some simple problems in simple cases which leads to horrible mess later.

Linus was always adamantly against such things and against opaque void-pointers everywhere… and for the same reason Rust abhors implementation inheritance.

It would be interesting to see what other errors would Jakt catch besides memory errors (Rust uses it's ownership-and-borrow approach to do many things safely, e.g. scoped threads).

I suspect it would be yet-another-failed-attempt-to-sway-C++-programmers. Unsuccessful because it would be too little return for too much pain.

Huang: Rust: A Critical Retrospective

Posted May 26, 2022 7:42 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (3 responses)

Implementation inheritance is not a hack. It's entirely logically correct, provided that your code religiously complies with the Liskov substitution principle.

The problem is, the LSP is a semantic property. Deciding whether a specific implementation is LSP-compliant depends on the implementation's behavior, the behavior of the superclass, and the API to which both are written. In the general case, it's not only undecidable, but it depends on factors other than the code itself (which parts of the API are contractual?). And then Hyrum's law messes things up further from the other side of the API boundary. So if you want a language that is always provably correct, you can't do anything that depends on the LSP.

But then, all APIs have issues like this to some extent. There will always be fuzzy boundaries around the edges of the contract. If you *really* want provably correct code, you have to do formal verification, and there's a reason that nobody does that in practice.

Huang: Rust: A Critical Retrospective

Posted May 27, 2022 21:04 UTC (Fri) by khim (subscriber, #9252) [Link] (2 responses)

> Implementation inheritance is not a hack.

It is. If your superclass implements dozen of interfaces (not atypical for complex objects) and you don't explicitly document what methods call what other method, when and how, then you don't have any idea how changes to one interface in the subclass would affect other interfaces (if at all).

> It's entirely logically correct, provided that your code religiously complies with the Liskov substitution principle.

Which is only possible if you read and understand the implementation of your superclass. Which throws away all these nice “encapsulation” and “open-close principles”. Which is then fixed, in turn, with copious amount of other hacks (“single-responsibility principle”, “dependency inversion principle” are band-aids for this problem, there are many others invented, too).

C# and Java tried to fix that mess with default implementations of methods in C# 8 and Java 8, but, of course, then can not do what Rust did and just declare implementation inheritance forbidden.

> If you *really* want provably correct code, you have to do formal verification, and there's a reason that nobody does that in practice.

They do, they do. Google Wuffs is a pretty practical project. And Rust safety model was formally proven (even if safety of an actual Rust compiler wasn't).

I suspect that the only reason people are not doing it more is the fact that buggy, low-quality, barely working code was the historical norm and even code used for critical systems is often delivered with “no liabilities” licenses.

Situation is slowly changes, but we couldn't just, suddenly, add stiff penalties to the code: existing codebases would instantly become useless and we don't have anything better to replace them right now.

Huang: Rust: A Critical Retrospective

Posted Jun 2, 2022 22:54 UTC (Thu) by mrugiero (guest, #153040) [Link] (1 responses)

> I suspect that the only reason people are not doing it more is the fact that buggy, low-quality, barely working code was the historical norm and even code used for critical systems is often delivered with “no liabilities” licenses.
>
> Situation is slowly changes, but we couldn't just, suddenly, add stiff penalties to the code: existing codebases would instantly become useless and we don't have anything better to replace them right now.

I think it will continue to be the norm because extra work is extra time to market. While I do not support cutting corners that way (not everywhere anyway), there's simply very strong commercial incentives to ship sorta-kinda-works code fast rather than correct code a few months later. It's more or less while software keeps getting more and more inefficient.

Huang: Rust: A Critical Retrospective

Posted Jun 10, 2022 20:36 UTC (Fri) by matu3ba (guest, #143502) [Link]

> Situation is slowly changes, but we couldn't just, suddenly, add stiff penalties to the code: existing codebases would instantly become useless and we don't have anything better to replace them right now.

If you look at the code that is verified, then there are 2 approaches: 1. automatize a subclass of problems (Wuffs, Rust) or 2. formally verify the code (codegen or only logic with different performance and safety requirements etc).

> I think it will continue to be the norm because extra work is extra time to market.

Not really. Fundamental libraries make a small share of costs, but can play a huge risk on getting things wrong.
Though for Rust, it is more of a tradeoff than Wuffs.

That said, product analysis has initial costs, maintenance costs and risks vs gains. The product you describe sound like low trusted brand things (throw-away products or component can be restarted on failure with low chance + cost on data loss).

Huang: Rust: A Critical Retrospective

Posted May 27, 2022 19:41 UTC (Fri) by tsavola (subscriber, #37605) [Link]

> It's not that interesting, really. And while I wish them luck this would mean that this OS would forever stay a love letter to '90s user interfaces with a custom Unix-like core and wouldn't turn into the next Linux.

Failing to become the next Linux doesn't seem to be in violation with the purpose of the project.

> Simply from their small blog post where they boldly proclaim that they would have OOP and classes it's easy to see that they have almost given up to the templation of supporting implementation inheritance which means they don't understand what correctness even mean.

They have said that the reason for the OOPiness is that it works well for GUI application development, which is a central part of their work. It would be dumb if their custom language wouldn't cater for that.

> I suspect it would be yet-another-failed-attempt-to-sway-C++-programmers. Unsuccessful because it would be too little return for too much pain.

Their intended audience is SerenityOS developers.

Huang: Rust: A Critical Retrospective

Posted May 24, 2022 21:26 UTC (Tue) by kleptog (subscriber, #1183) [Link] (1 responses)

I got triggered a bit on this line:

> Q: Why ARC (automatic reference counting) instead of a borrow checker?

> ARC allows the language to feel lightweight without constantly asking the user to make decisions about memory management.

I'm of the opinion that if you don't know where your data lives and how long it lives, it's going to bite you in the ass. Yes, it's a bit of a pain that Rust is pretty strict about it, but I believe it makes you a better programmer and you get better programs.

Lazy memory management has its uses (see Python and other scripting languages for example) but they're writing an OS here.

Also, integrating reference counting into code written in other languages is tricky, I'm curious how they plan to handle that. Turning everything into a shared_ptr would work I suppose.

Huang: Rust: A Critical Retrospective

Posted May 24, 2022 21:40 UTC (Tue) by khim (subscriber, #9252) [Link]

> Yes, it's a bit of a pain that Rust is pretty strict about it, but I believe it makes you a better programmer and you get better programs.

Rust gives you optional reference counting and you can even build your whole program on top of Arc links. If you are not clever enough to use other tools and if you don't plan to squeeze every last drop of performance out of your code.

But it's visible in the code. And it doesn't feel “lightweight” because ARC is not lightweight. People did measurements and Swift is much slower than Rust or C++ for many code patterns because of ARC.

Sure, if you want to attract developers who are accustomed to C# or Java then ARC may be a good choice, but if you are doing low-level stuff then ARC use has to be explicit.

> Also, integrating reference counting into code written in other languages is tricky, I'm curious how they plan to handle that.

That part is fine. ARC is local, it's not like tracing GC which insists on violation of any and all good engineering principles to solve problem which doesn't exist.

Huang: Rust: A Critical Retrospective

Posted May 21, 2022 21:06 UTC (Sat) by chuckwolber (✭ supporter ✭, #138181) [Link] (3 responses)

"... about 5700 lines of code from various third parties that manipulates files, directories, and environment variables, and runs other programs on my machine every time I do a build."

"... one fix to this would be to re-configure our build system to run in some sort of a chroot environment or a virtual machine that fixes the paths in a way that almost anyone else could reproduce"

Those problems (and more) are why bitbake is such a powerful embedded build tool. It is inconceivable to me that Bunnie has not yet encountered and spent time evaluating Yocto/OpenEmbedded, so I am very curious what his objections are.

Huang: Rust: A Critical Retrospective

Posted May 22, 2022 18:47 UTC (Sun) by smoogen (subscriber, #97) [Link]

Actually it can be quite conceivable. I know lots of OS developers who even work in embedded who have not done anything deeper with Yocto than dropping in a .bb file someone else wrote. And then there the ones still doing things 'hardcore' embedded or look at it and say 'well yeah, but I already do that with my own crafted Gentoo'. The yocto method is very cool and has a lot of ways of doing things well for this usecase.. however, the method is different enough that a lot of 'complaints' usually are just having to bend your usual way of doing things into the build system.

Huang: Rust: A Critical Retrospective

Posted May 23, 2022 12:18 UTC (Mon) by NAR (subscriber, #1313) [Link]

"... about 5700 lines of code from various third parties that manipulates files, directories, and environment variables, and runs other programs on my machine every time I do a build."

I understand that ~~paranoid~~security-conscious developers don't like this. But this is not specific for these "new" languages. I mean in how does someone install software written in C? Downloads some .tar.gz from somewhere, untars it, then executes ./configure. The generated (i.e. write-only) configure script creates files, calls the compiler, executes random stuff, etc. Then make will also manipulate files, environment variables, etc. This is not a new problem - maybe the environment is new, but those generated Makefiles are as much opaque as the auto-executed code in those crates.

Huang: Rust: A Critical Retrospective

Posted May 23, 2022 13:18 UTC (Mon) by dskoll (subscriber, #1630) [Link]

I haven't worked with vanilla Yocto/OpenEmbedded, but I spent a painful 18 months dealing with Petalinux from Xilinx (based on Yocto) and I'm scarred for life. I'll be happy never to touch it again, which is very likely since I stopped doing embedded development in 2020.

Bitbake is indeed powerful... but the various recipes and ways of overriding them are badly-documented and filled with magic. Most vendors' layers are full of opaque recipes and zillions of variables whose meanings are not very clear.

Huang: Rust: A Critical Retrospective

Posted May 23, 2022 2:28 UTC (Mon) by ssmith32 (subscriber, #72404) [Link]

"Software people are quick to remind me that first and foremost, I make circuits and aluminum cases, not code, therefore I have no place ranting about software. "

Yikes, hopefully he never catches all the software developers (myself included) who regularly complain about circuits and aluminum cases ;)

Huang: Rust: A Critical Retrospective

Posted May 23, 2022 13:37 UTC (Mon) by scientes (guest, #83068) [Link]

Joy to read. Especially the analogy about pulling carefully bundled and labeled cables. Now try mixing aluminum and copper! (Not sure where I am going with that analogy besides trying to be cute)

Thanks bunnieh.