LWN: Comments on "NonStop discussion around adding Rust to Git"

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

kentborg — Sun, 29 Dec 2024 17:12:06 +0000

If the git community has technical reasons for using Rust in git, let them use Rust in git. Of course they should do it in an organized way, but otherwise go ahead.

The financial industry pays for NonStop itself, and they can afford to pay to add Rust support to NonStop.

Free software is not some constitutional right of big business.

The tune sounds familiar

ceplm — Mon, 16 Dec 2024 23:59:41 +0000

OK, there is such a project … https://github.com/rust-lang/libm/, but just from browsing the issue tracker, I won’t use it for calculating medications any time soon.

The tune sounds familiar

ceplm — Mon, 16 Dec 2024 23:52:27 +0000

It isn’t … that is in the Python. You still require Fortran for compiling scipy, for example.

There are just some Fortran modules which won’t be ever ported from Fortran. If you have a code which some PhD student spent her whole postgrad program working on (and there are such), there is no way that it could be rewritten in Rust any time soon.

How is that rewrite of libm to Rust going?

CVEs/LOC not a great metric

mathstuf — Sun, 15 Dec 2024 14:42:33 +0000

Memory safety is different than functional safety. PHP has the former but its APIs were woefully lacking to make the latter easy. Now there are better APIs more readily available, but its history definitely casts a long shadow in many minds.

> ...The largest data breaches in history are due do this sort of application-level logic flaw... implemented in "safe" languages.

No one here (AFAIK) is claiming *functional* safety (life, limb, correctness) here[1]. Proof systems are needed for that and Rust isn't there (nor do I know of anyone claiming such). However, when one has memory safety, the foundations for building up things like "encode requirements into types and let the compiler make sure it is fine" are far easier. Of course, some languages don't have sufficient mechanisms to teach their compiler (or interpreter) about such things, memory safe (Python, PHP, Bash) or not (C).

> numerous lackadaisical practices by folks developing with PHP

The same can be said for just about any software dysfunction. No one is perfect, but the level of vigilance and attention required to prevent such issues differs greatly between languages (and is not monotonic across the "memory safe" spectrum either). I know I'd much rather the team I'm working with have a steadfast gatekeeper for problems in the compiler before CI before review before customers. The quality of each filter is different between languages, teams, and cultures but (IMNSHO), finer filters earlier in the sequence are generally worth quite a lot more.

[1] Philosophical question: what constitutes "functionally safe" for a missile?

CVEs/LOC not a great metric

pizza — Sun, 15 Dec 2024 13:55:24 +0000

> I don't think I've ever heard anyone describe PHP as safe, most people who even have a concept of "safe" wouldn't touch PHP with a 20ft pole!

PHP has *always* been "safe".

Its poor reputation is due to numerous lackadaisical practices by folks developing with PHP -- primarily not sanitizing inputs and manually assembling database queries (and shell invocations) with those unsanitized inputs.

...The largest data breaches in history are due do this sort of application-level logic flaw... implemented in "safe" languages.

(Any language capable of concatenating strings together is "vulnerable" to this sort of thing)

CVEs/LOC not a great metric

sammythesnake — Sun, 15 Dec 2024 08:00:21 +0000

> people move to [...] PHP (also, technically, a memory safe language) [...] thinking the language safe [...]

I don't think I've ever heard anyone describe PHP as safe, most people who even have a concept of "safe" wouldn't touch PHP with a 20ft pole!

Having a Rust compiler is probably unavoidable

sunshowers — Sat, 14 Dec 2024 18:40:59 +0000

UB is shaken out better by alternative "midends" like Miri, which has had quite some success at doing so.

There are definitely benefits to another frontend, but they need to be balanced against the costs. I think the costs far outweigh the benefits right now.

Oh FFS!

sammythesnake — Sat, 14 Dec 2024 07:57:20 +0000

Sorry, fat-fingered edit there, the bracketed sentence "though the necessary changes can be made by automated handles" should have said "though the necessary changes _to support a new edition_ can be made by automated converters"

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

sunshowers — Sat, 14 Dec 2024 02:09:55 +0000

These days, tuned and safe Rust code can often be faster than highly-optimized C or C++ code. That is due to, among other things, Rust's constraints (e.g. noalias) making many optimizations possible.

See, for example, [1], where memory-safe Rust now outperforms C at decoding PNG images.

Writing Rust is also significantly nicer than writing C or C++. Not only are the error messages worlds better and you can *actually read them to understand what's going on* -- why would I want to do a convoluted pointer tagging thing at runtime when I can simply catch those issues at compile time?

[1] https://www.reddit.com/r/rust/comments/1ha7uyi/memorysafe...

Easier?

sunshowers — Sat, 14 Dec 2024 01:57:34 +0000

Interestingly, I started contributing to wasmtime's internal memory management recently. The details are quite subtle, and actually don't fully fit the Rust model -- some logically-borrowed memory is represented without lifetime parameters.

If wasmtime were in C or C++, it would have taken me a long time to wrap my head around all of the details. I'd probably had to have one of the maintainers walk me through it on a video call. But with Rust, I went from never having seen the code before to having a ~800 line prototype of a fairly major internal refactor within a day. I was contributing fixes and identifying bugs the very next day too.

Rust is great for novices to a codebase.

Oh FFS!

sammythesnake — Fri, 13 Dec 2024 19:51:32 +0000

They release new code pretty frequently, but in Rust land, an "Edition" is a specific thing[1] released very much less frequently[2] and which is stable and supported in all future versions of the compiler - no need to change any code (though the necessary changes can be made by automated handles

There are necessarily caveats to do with bugs and unavoidable backward incompatibility to handle security issues or whatever, but the only way to guarantee avoiding those is never release any new code at all...

[1] https://doc.rust-lang.org/edition-guide/editions/

[2] 2015, 2018, 2021 and 2024 so far, that's one every ~1100 days on average, so the orders of magnitude less often than you suggested.

gitoxide?

sammythesnake — Fri, 13 Dec 2024 15:01:22 +0000

"sub-" vs. "re-" is an important distinction. Perhaps the practical implications aren't much in many issues, but they're not the same thing and if you're not careful, you can find yourself accidentally crossing the border where the distinction starts to matter, potentially with very messy consequences...

"Safe C" variants like Fil-C vs. Rust

sammythesnake — Fri, 13 Dec 2024 14:34:36 +0000

At I understand it, the proof standard used by Rust is very conservative - making narrow promises and rejecting anything uncertain, and therefore pretty much bulletproof (modulo bugs in the bottle checker etc. of course)

It's the same mindset of approach as the eBPF verifier - there are cases where "safe" programs are rejected by the verifier/checker because it's not (yet) sophisticated enough to prove their safety. Adding to that capability is always an option (until you hit the halting problem, but you can asymptotically approach it until the correct-but-rejected case is meaninglessly small)

Sadly, "correct" is a far more nebulous (and *larger*) thing to prove than the subset of correct that we call memory safety. If we could somehow define "correct" in a general case and implement a checker (that runs in finite time!) then proving correctness would naturally also imply proving memory safety - it's "incorrect" to access the wrong memory, after all!

I imagine that Rust might eventually get to a point where "correctness" is the focus of the proof, with "memory safety" being a historical note as merely an early part of "correctness" to be implemented.

Defining correctness and implementing proof strategies for it can be done in incremental steps, possibly with the assistance of assertions and with optional aspects of proof opted into as the cost of the extra analysis is warranted...

Easier?

intelfx — Fri, 13 Dec 2024 03:46:59 +0000

> The barrier to contribution isn't "skilled in C" so much as "skilled in that particular codebase."

By virtue of having more high-level abstractions and generally a more modern/ergonomic design, Rust makes it easier to get up to speed within a given codebase[1]. It does not necessarily make you *skilled* faster, but it's likely to make you *productive* faster.

> Give a highly skilled person a more complex tool, and after some practice they may be able to achieve greater things. But you're not oging to get the same results by handing the same tool to someone that is barely literate.

That's not what GP was saying. In this analogy, C is the "more complex" tool.

[1] for a very simple reason: you spend less time on rediscovering how a particular codebase does fundamental things, because all of those fundamental things are baked into the language (or, worst case, implemented in highly popular libraries which are likely part of the dependency tree anyway).

Easier?

viro — Fri, 13 Dec 2024 02:44:38 +0000

Rhetorics about "building waves" ought to be treated as Godwin-equivalent, really. Regardless of the subject...

Easier?

pizza — Fri, 13 Dec 2024 01:17:53 +0000

> C is extraordinarily difficult to write correctly to a modern standard, even for people with decades of experience in it. Rust is not easy, but at least the compiler does a pretty good job helping you.

The barrier to contribution isn't "skilled in C" so much as "skilled in that particular codebase."

> There are entire communities of "hardcore" C/systems devs who have quietly moved to Rust—and don't really want to write C any more. There is a massive wave that's been building for a few years, and it's only going to get bigger over time.

Give a highly skilled person a more complex tool, and after some practice they may be able to achieve greater things. But you're not oging to get the same results by handing the same tool to someone that is barely literate.

Easier?

sunshowers — Fri, 13 Dec 2024 00:30:56 +0000

C is extraordinarily difficult to write correctly to a modern standard, even for people with decades of experience in it. Rust is not easy, but at least the compiler does a pretty good job helping you.

There are entire communities of "hardcore" C/systems devs who have quietly moved to Rust—and don't really want to write C any more. There is a massive wave that's been building for a few years, and it's only going to get bigger over time.

Having a Rust compiler is probably unavoidable

sunshowers — Thu, 12 Dec 2024 21:30:58 +0000

As a Rust developer with several popular crates to my name, I'm not really interested in supporting compiler frontends other than rustc. I think backends are the right way to go.

Note that Rust doesn't make breaking changes. There's not going to be a completely unchanging language for many years, because the language is still quite incomplete. Meanwhile, I'm going to continue being part of teams that ship projects and deliver value to users, in ways that are simply not possible in other languages.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

intelfx — Thu, 12 Dec 2024 18:23:43 +0000

That's called a capability-based architecture. We have a real-world example of that; it's called CHERI, and the problem is that you can't just compile any existing software for CHERI (precisely due to the fact that it fundamentally disallows many things C and C++ programmers take for granted).

So this "Fil-C" thing is... essentially, an emulation of capability-based architecture. More power to them, I guess, but make no mistake — you absolutely cannot just take arbitrary C/C++ software and compile it with FIl-C, for exactly same reasons.

The Rust something you are referring to is called "Tree Borrows" (or its predecessor, Stacked Borrows). Those are attempts at *formalization* of Rust's memory aliasing model (essentially a document that tells you what is or isn't UB with respect to unsafely accessing memory). They are not really related or equivalent to "capabilities for Rust". Instead, the idea is that if a piece of code complies with Stacked Borrows or Tree Borrows, then it will also conceptually comply with any reasonable capability-based architecture.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

sammythesnake — Thu, 12 Dec 2024 17:55:19 +0000

I quickly scanned through some of the website before coming back to see if everything I might have commented was already said by others (yeah, pretty much :-P) and I think the answer is as follows:

1. You use a union of pointer and the appropriate integer type.

2. You can access the bits of the pointer address by using the int member, including changing it.

3. The "InvisiCap" (essentially provenance plus size of accessible allocation) is tracked in the other half of the fat pointer that isn't available to the program, which means your can't change the address to something arbitrary and use it, you can only access the same stuff as before.

I remember reading about something in (proposed for?) Rust that sounded approximately equivalent , though with a somewhat different interface. ("Tree Borrow Checker"...?) Sorry for the vague description - I don't follow Rust all that closely :-P

Having a Rust compiler is probably unavoidable

sammythesnake — Thu, 12 Dec 2024 08:05:29 +0000

It strikes me that the best starting point for Microsoft would be to release documentation for their debug data format. That way, there's a good chance of support being worked on by others who would value it...

There's nothing stopping them from *also* adding rust support to MSVC, if that's something they want.

Multiple compilers supporting a given debug data format would be beneficial in much the same ways as multiple rust compilers, after all.

"Safe C" variants like Fil-C vs. Rust

Wol — Sun, 08 Dec 2024 08:45:03 +0000

> Or to put it briefly: people care far more about programs being *correct* than them being *safe*. A Safe C doesn't help at all in that respect.

And "safe" is a likely by-product of "correct". In fact, a "correct" program is far more likely to be safe than a "safe" one ... (as in it *doesn't* do anything wrong, rather than being allegedly incapable of doing anything wrong)

Cheers,
Wol

"Safe C" variants like Fil-C vs. Rust

kleptog — Sat, 07 Dec 2024 11:00:11 +0000

Speaking for myself: I'm not interested in a version of C that crashes more reliably when something goes wrong. We know that C crashes and it's annoying to debug because the crash happens far away from the actual bug, so fixing it is time consuming and frustrating.

Rust on the other hand prevents the errors happening in the first place, and the compiler points you to the exact place where the (potential) error is. This makes it far more pleasant to work with and gives you far more confidence in the end result.

Or to put it briefly: people care far more about programs being *correct* than them being *safe*. A Safe C doesn't help at all in that respect.

I'm putting more faith in some kind of bulk C->Rust conversion. From source would be hard, but I can image a kind of C interpreter that runs the program and analyses the runtime usage of objects to determine the appropriate types and lifetimes and then produces Rust source. Though I expect a lot a rewrites in the mean time, just because Rust is so much more pleasant to work with.

"Safe C" variants like Fil-C vs. Rust

anton — Fri, 06 Dec 2024 11:18:04 +0000

I remember several memory-safe C implementations being presented at the poster session of PLDI 2002; those worked with fat pointers and introduced quite a bit of slowdown compared to C without that checking. Fil-C seems to be further development of this work, that addresses some of the drawbacks, e.g., ABI compatibility.

At ETAPS 2006 (IIRC) I heard a talk by George Necula about Ccured. This work apparently later led to Ivy and its component Deputy (manual): He said that ABI incompatibility was the reason for the lack of uptake of the fat-pointer implementations of C, and proposed an alternative using an enhanced type system for C; to make an existing program safe, at some places the programmer would insert assertions that inform the type checker; the compiler would then either prove that the assertions hold, or would insert run-time checks.

This would allow the gradual conversion of C code to safe code without needing to convert the whole program at one time. Looks like a winner, no? However, it apparently was not taken up at all. After the research projects ended, the Ivy stuff even partially disappeared from the web, and the github link above is a ressurection of that source code (from 2008).

I can only guess at why there was so little interest in this work. My guess is: C programmers think that their programs are secure because of their programming prowess, so they don't want to waste time on putting the assertions in, and they don't accept the slowdown from run-time checking. Their managers don't welcome the cost of putting assertions in and on slowing down the programs, so they are willing to listen to the programmers when they say that the programs are secure (and, except for a few vulnerabilities, they are).

A few years later, Rust appeared and is also using a type-system-based approach to safety. It requires a complete rewrite, which seems to be at a grave disadvantage compared to the gradual-rewrite approach of Ccured/Ivy/Deputy. But in contrast to Ccured/Ivy/Deputy, Rust is a success. Maybe Ccured/Ivy/Deputy suffers from an uncanny-valley-like effect: does not interest C programmers for the reasons above, does not interest non-C programmers because it is too much like C.

So what does this mean for Fil-C? It seems to me that while Fil-C does not require adding assertions, it does suffer from slowdowns, more so than Ccured/Ivy/Deputy, so it probably will still see little love from C programmers and their managers.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

ssokolow — Thu, 05 Dec 2024 21:53:09 +0000

Sure, but that requires the maker of these types to think about that need.

That depends on what you mean by "think about that type of need".

Serde does have a workaround for "remote types" and, beyond that, you're bumping up against intentional Rust design decisions relating to API stability.

Having a Rust compiler is probably unavoidable

ssokolow — Thu, 05 Dec 2024 21:40:30 +0000

Getting a backend added to LLVM isn't the only possible route. rustc isn't locked into LLVM.

I'm aware of four backends already in various stages of completion. (LLVM, Cranelift, rustc_codegen_gcc, and rustc_codegen_clr.)

Having a Rust compiler is probably unavoidable

ssokolow — Thu, 05 Dec 2024 21:36:36 +0000

I could be wrong, but I don't remember seeing a promised timeline... just a (still ongoing) set of progress reports summarizing what is implemented and still to implement on a roughly monthly basis.

Having a Rust compiler is probably unavoidable

ssokolow — Thu, 05 Dec 2024 21:34:14 +0000

Which has been promised to be usable for at least two years now. It's still not usable.

According to the repo's Insights tab on GitHub, it wavers between a one-man project and a two-man project at any given time (depending on whether bjorn3 or GuillaumeGomez is contributing in addition to antoyo) and, as the progress reports show, they're also busy writing patches to be upstreamed into libgccjit to flesh out the API they depend on.

Open Source Projects Are Not Your Supply Chain

kpfleming — Thu, 05 Dec 2024 12:39:16 +0000

Well... the title says it all.

CVEs/LOC not a great metric

khim — Thu, 05 Dec 2024 11:58:38 +0000

I guess it helps a tiny bit (it's similar to the level of protection JavaScript const or Java's final provides), but if you are to use it like that, 99% of time, then why in 15 years of development nothing that doesn't force me to type the same thing twice was provided?

Note: I'm not saying that Go is worse than C (although it's, most likely, worse than C++), I'm just saying that for majority of the “memory safe” languages said “memory safety” doesn't make programs safer or more robust to a degree that people expect.

They add memory safety not to make them safer or more reliable, but, in 90% of cases, to make them “simpler”, or “more flexible”, etc.

And if you eliminate 70% of bugs but make remaining 30% of bugs 2-3 times easier to make… the net result is still positive, but it's not as large as many expect.

CVEs/LOC not a great metric

intgr — Thu, 05 Dec 2024 00:07:44 +0000

> JavaScript have const these days

TBF JavaScript's const is almost useless. It only prevents you from re-assigning the variable in the scope.

It does nothing to prevent mutation or side-effects: when the *value* referenced by variables is inadvertently mutated.

You can have a const Array variable and still mutate it many times in a function, pass it to another function that mutates the array etc.

CVEs/LOC not a great metric

Cyberax — Wed, 04 Dec 2024 21:18:34 +0000

> And does it help? If I replace newNubers with numbers problem remains, of course.

Yes, it helps. Because it's clear that you're mutating a passed-by-value object.

CVEs/LOC not a great metric

khim — Wed, 04 Dec 2024 20:32:01 +0000

And does it help? If I replace newNubers with numbers problem remains, of course.

The issue in that code is not that append is used incorrectly, but in the fact that append-using function is used incorrectly.

Which is precisely and exactly the same class of bugs that make C/C++ programs fail, most of the time!

You may as well say that malloc should be paired with free and anyone who doesn't do that is “holding it wrong”.

Whether it's a hashmap that someone incorrectly modifies, or changing a struct which is not supposed to be changed… it's the exact same pattern: something is changed when it shouldn't be changed, or, alternatively, something is not changed when it should be changed.

The latter case couldn't be handled by the language efficiently (that's firmly #1 in farnz classification), but the former could be handled, in many cases. And most serious languages before Go (like C/C++, Java, C#, etc) had a way to handled that to some degree (approaches are different, const in C doesn't work like final in Java and C# in/ref readonly have different scope from both of them), but only “improved C” language developed in XXI century decided that “you are holding it wrong” is the best answer to these concerns.

Sure, the languages that Go actually replaces (JavaScript, Python or Ruby) had no such mechanisms either… which partially explains why C++ developers don't flock to Go and most developers switch to Go from dynamically-typed languages. But even they are slowly growing them (JavaScript have const these days)!

But still it's kind of funny that Go developers were telling to themselves that they are doing a better alternative for C++, not better alternative for Python…

CVEs/LOC not a great metric

Cyberax — Wed, 04 Dec 2024 20:01:03 +0000

In the actual Go code this type of code is _very_ unusual. The typical pattern is `arr = append(arr, items...)`, so it's clear that you're mutating the array.

CVEs/LOC not a great metric

paulj — Wed, 04 Dec 2024 13:07:12 +0000

This particular "side effect" looks very confusing. Not a Go expert, but this kind of side-effect looks like a huge land-mine, that could be easy to trigger unwittingly.

CVEs/LOC not a great metric

paulj — Wed, 04 Dec 2024 13:04:11 +0000

Wol was doing so well the last while... He's actually managed to go quite a while recently without bringing up Pick. This is his first relapse of his pickaholicism in a while, TTBOM. Maybe he can last a bit longer again from this point forward. ;)

Classes of logic bug

farnz — Wed, 04 Dec 2024 11:37:03 +0000

Part of the fun here is that there's (at least) two classes of logic bug in play here:

Failure to correctly transpose constraints from the problem domain into the code. For example, if a route planner has you arrive at the airport after the gates for your flight have closed, but before the flight's scheduled departure.
Not respecting rules of the language that aren't compiler-enforced. For example, iterator invalidation when you delete items.

It's unreasonable to expect a language to prevent the first sort of logic bug completely (although it can give you the tools to write libraries that prevent them). But it's not unreasonable to expect that languages aim to reduce the second category of logic bugs to as close to zero as possible, and to at least require you to do something a bit strange in order to run into them (e.g. unusual syntax, or special marker keywords).

CVEs/LOC not a great metric

khim — Wed, 04 Dec 2024 01:18:37 +0000

> Go only guarantees that your program won't cause memory safety issues.

And then, only if there are no race conditions. Which is precisely my point: the goal of Go was never

> It's literally from the "don't do this" category.

Sure, but how would I even know if I'm supposed to do that or not?

Note, that:

Go doesn't offer any way to pass a read-only slice around (like not just C++, but even plain C could do)
Go doesn't offer separate types for the “view slice” and “owned array” (C also conflates them, but C++ handles them separately)
“Ideomatic” Go conflates many other different things (e.g. set is supposed to be handled via hashmap) for the “simplicity”

> Go's behavior is not something unusual

Maybe, but Go's creator's behavior is unusual: it's as if they deliberately combined all the worst sides of programming language design from the last century and created something that could fight with PHP and JavaScript for the title of “the most dangerous popular memory-safe language”.

Essentially the only thing that they added to not be able to take the prize in that contest is static typing (and then pretty weak one). This, indeed, pushed them somewhat from the winning position… but other than that… all the tried and failed ideas are picked up and implemented.

If that were the language like BASIC, only designed to support newbies who couldn't grasp all the concepts that “serious languages” are using – that would have been justified… or if that was supposed to be only used for a small scripts, like Python… maybe.

But Go is not positioned like this! It pretends to be usable for large-scale projects!

And there are even some people who are using it like that…

I guess it's justified by that “strange phenomenon” that even their creators noticed: Although we expected C++ programmers to see Go as an alternative, instead most Go programmers come from languages like Python and Ruby. Very few come from C++. Also note how it took us over a year to figure out arrays and slices – which means that abomination that we are discussing here is not some sort of omission, but something that their creators are proud of! That's just simply… unbelievable.

But still… now we are conflating all languages except for C/C++ in one huge lump of “memory safe” languages – and that's simply wrong.

Not all “memory safe” languages are equally safe… but given the fact that around 70% of bugs in C/C++ programs are memory safety bugs… we should consider themselves lucky if people would switch from C/C++ to Go, JavaScript and Python… although I hope languages like Ada and Rust would good some love, too.

CVEs/LOC not a great metric

Cyberax — Wed, 04 Dec 2024 00:33:53 +0000

> Isn't that the exact same mantra that C and, later, C++ programmers repeated for last half century: don't write programs with bugs – and everything would be all right?

Pretty much. Go's behavior is not something unusual, you're (possibly) mutating an object, and then depending on the side effects of this mutation. It's literally from the "don't do this" category.

Go only guarantees that your program won't cause memory safety issues.

CVEs/LOC not a great metric

mathstuf — Tue, 03 Dec 2024 21:16:17 +0000

Yes, there are definitely lints that warn about such things in Python. My caveman club-level of finesse with Golang is usually just happy to get things to work (though I've not dealt with its slices enough to know how any linters behave with them).