|
|
Subscribe / Log in / New account

NonStop discussion around adding Rust to Git

By Daroc Alden
November 22, 2024

The Linux kernel community's discussions about including Rust have gotten a lot of attention, but the kernel is not the only project wrestling with the question of whether to allow Rust. The Git project discussed the prospect in January, and then again at the Git Contributor's Summit in September. Complicating the discussion is the Git project's lack of a policy on platform support, and the fact that it does already have tools written in other languages. While the project has not committed to using or avoiding Rust, it seems like only a matter of time until maintainers will have to make a decision.

Taylor Blau opened the discussion in January by listing some of the advantages of Rust for the Git project: memory safety, lack of data races, easier refactoring (in the absence of unsafe code), and making it easier for more people to contribute to Git. "Given the allure of these benefits, I think it's at least worth considering and discussing how Rust might make its way into" Git's mainline. Blau also pointed out some questions that would need answering before the project could consider using Rust: how the language would affect platform support, which parts of Git would be easiest to migrate or extend (as is nature of sprawling mailing list discussions, the participants later touched on both), and how the change would interact with ongoing maintenance efforts.

Elijah Newren was excited about the idea, but suggested that one alternative would be to allow Rust only in optional components, in the same way that the kernel currently does. He volunteered to rewrite git-replay in Rust as a test case, noting that since (at the time of posting) it hadn't been included in a Git release, it wouldn't cause a regression. Newren was in favor of adopting Rust for different reasons than the ones Blau had listed in his message, however. He said that the Git developers often avoid making parts of the code execute in parallel, because of the difficulty of doing that correctly in C. Using Rust might allow for performance improvements from adding parallelism, and from switching to more efficient algorithms and data structures that are "quite onerous" in C. He specifically called out hash maps as a more performant alternative to the use of flat arrays in core Git code.

Other contributors were also open to the possibility. For example, brian m. carlson said "I'm very much in favour of this". He did think that there were serious obstacles to adopting Rust, however. He mentioned Rust's rapid release cycle as one such problem, but saw platform support as "probably the biggest stumbling point".

His main concern was with NonStop, a relatively niche platform primarily used in the financial sector for high-reliability computing. Earlier NonStop systems are based on Intel's Itanium architecture, but more recent systems have used x86_64. Both architectures run a proprietary operating system. The platform has its own C compiler, but many common tools, such as GCC, have not been ported. Git, however, has supported NonStop since Git 1.8.0 in 2012. That initial support was added by Joachim Schmitz, but Git doesn't have a policy for when and how that support should continue — as Emily Shaffer pointed out: "As far as I can tell, the support policy now is 'if you run make test on it and breaks, and you let us know, we'll try to fix it' - without much in the way of additional caveats."

Randall Becker, one of the most currently active NonStop maintainers, said that he thought Git's wide portability was actually one of its most important strengths. Git's policy of not relying on Linux-specific tools means that "we can take any developer from any platform and move them anywhere else without having to design new processes or teach them new processes for their workflows". In Becker's view, adopting Rust will "rapidly limit (or eliminate) git's long-term viability." He went on to suggest that the Git project should consider the impact of limiting platform support carefully, since it is such a fundamental piece of infrastructure:

What is the broad impact of dropping support for a platform that has a significant investment in git, where loss of support could have societal impact. There are platforms with 100 million to 1000 million lines of code managed by git today. This type of investment-related impact is specific to git compared to other Open-Source products. Leaving debit or credit card authorizers without a supported git would be, let's say, "bad".

Junio Hamano, the leader of the Git project since 2005, wasn't convinced, saying:

It depends but "investment-related impact" is unlikely cause for us to personally care. But those $CORPS who will feel the "investment-related impact" are welcome to hire quality developers and to these employed developers, the "impact" might become an issue they care more deeply about.

In another message, he did suggest that the project might consider treating niche platforms in the same way as platforms that package binary releases of Git — that is, requiring the platform to carry its own patches on top of stock Git. Newren listed the many problems he saw with establishing a platform-support policy that could guide the project in cases like this. Among other problems, he said that "we don't even know which platforms Git is used on". Newren also thought that any platform policy would need to address optional build components, including the several optional components that Git already has, such as Tcl support.

The idea of treating items written in rust as optional components, to be used only for non-essential uses, caused some doubts. Patrick Steinhardt expressed the opinion that once Rust had been permitted into any part of the Git code, it was likely to spread. He said that if Rust was accepted, "it will only be a matter of time before we want to start using it in central code paths".

Ultimately, the January discussion petered out with the topic unresolved. Blau implicitly reopened the topic in September when he posted the notes from the Git Contributor's Summit. Becker responded by noting that — while he does "not fundamentally object to Rust" — the contributors who are discussing it still have not answered whether or how Git could continue supporting NonStop as a platform if Rust were allowed. Porting Rust to NonStop is non-trivial, but beyond that, he said that depending on a single toolchain to determine which platforms are supported is "not responsible, in my view". Sean Allred pointed out that any platform Git supports already depends on outside toolchains. "Any platform-specific bugs in the various C compilers used are not in this project's control, either", he said.

Allred also thought that refusing to allow any new languages or libraries to be used for the project would essentially be a commitment to stop adding new features to Git:

It is hard to write correct, safe C. I worry that this difficulty will eventually translate to significant, safe contributions coming only from those with the resources to create them and not from a more diverse pool of contributors. As grateful as I am to large companies who fund Git contributions, I don't think those companies, given the choice in and of themselves, would think twice about dropping support for a platform they don't use — and that is probably a much smaller set than 'does the platform have GCC'.

Introducing Rust would be a pain for NonStop users, though, Allred acknowledged. Becker was not satisfied with that line of thinking. For one thing, he only objected to adding non-portable dependencies. For another: "Telling 10-20000 users that their core bit of infrastructure is insecure and not fixable is not a tenable position", he said. The fact that old versions of Git would remain usable on NonStop isn't good enough, when there's no guarantee or policy for how long it would take to backport security fixes, nor how long a version will remain supported.

Phillip Wood thought that it would always be possible for communities of interested users to backport their own security fixes, but agreed that adding Rust to the project was a tradeoff:

I believe that those advocating for using rust are doing so because they believe it will benefit both contributors and users. The problem we have to wrestle with is whether those benefits outweigh the cost to the relatively small proportion of users who do not have access to rust on their platform.

Becker has raised the possibility of porting Rust to NonStop with the platform vendor and promised to keep the mailing list updated, but any potential port will be at least a year away, assuming one is even possible. With this discussion failing to reach a firm conclusion either, it appears that the Git project is stuck with an ongoing tension between a small group of users, its legacy of broad support, and its role as core infrastructure on one hand, and increased security, performance, and ease of contribution on the other. Which way the project (and specifically Hamano, as its leader) will decide remains to be seen.



to post comments

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 15:52 UTC (Fri) by cesarb (subscriber, #6266) [Link] (32 responses)

It seems to me that, in the medium term, nearly every platform will end up having a Rust compiler targeting it, in the same way that nearly every platform ended up having a C compiler targeting it.

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 17:15 UTC (Fri) by willy (subscriber, #9762) [Link] (6 responses)

I saw a rumour that Microsoft are adding a Rust front end to their Visual compiler backend, allegedly because of code quality issues from LLVM. I somewhat doubt this story, but there may be a nugget of truth to it.

It's not clear to me whether it's easier to add a Rust frontend to Nonstop's compiler, or to add a Nonstop backend to LLVM.

At any rate, I don't see why it would be so bad for Nonstop users to have to wait a few years to get the latest version of git. In my experience with Enterprise Customers, they very much do not want to be on the leading edge of software ("I installed this system eight years ago and don't see why I should upgrade anything").

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 17:35 UTC (Fri) by pbonzini (subscriber, #60935) [Link] (2 responses)

It's about bad debug info quality from LLVM; that's not a surprise to be honest, since debugging formats are very complicated stuff and the native one for Windows is not very well documented.

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 17:38 UTC (Fri) by willy (subscriber, #9762) [Link] (1 responses)

Oh, that makes perfect sense. Whether it makes more sense for Microsoft to improve LLVM's debug output or to put a rust front end on Visual is a decision I don't have enough information to make, but now the rumour makes a lot more sense. Thanks!

Having a Rust compiler is probably unavoidable

Posted Dec 12, 2024 8:05 UTC (Thu) by sammythesnake (guest, #17693) [Link]

It strikes me that the best starting point for Microsoft would be to release documentation for their debug data format. That way, there's a good chance of support being worked on by others who would value it...

There's nothing stopping them from *also* adding rust support to MSVC, if that's something they want.

Multiple compilers supporting a given debug data format would be beneficial in much the same ways as multiple rust compilers, after all.

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 17:45 UTC (Fri) by NYKevin (subscriber, #129325) [Link] (2 responses)

> I saw a rumour that Microsoft are adding a Rust front end to their Visual compiler backend, allegedly because of code quality issues from LLVM. I somewhat doubt this story, but there may be a nugget of truth to it.

Source: This is mentioned in passing in a highly opinionated blog post, which I won't directly link to because it would certainly derail the thread, on the subject of C++ and safety (as well as several related topics). If you really want to go find it, it was written by Izzy Muerte. If you don't want to go find it, the TL;DR of this particular nugget is that, according to an unnamed source, Microsoft apparently dislikes the way that LLVM emits debug information and wants an MSVC Rust frontend so that they can do their own debugging thing instead.

(Yes, I have feelings about that blog post. No, I'm not going to discuss them here. It's clearly and unambiguously off-topic for this thread. If the LWN editors are willing to moderate that discussion, they can put up a brief article linking to it - otherwise, my assumption is that they don't want to deal with the flood of comments it would inevitably inspire. Let's not make their work any harder than it already is.)

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 20:02 UTC (Fri) by willy (subscriber, #9762) [Link] (1 responses)

I don't know what value this comment adds. You're confirming that there is a rumour, which isn't a particularly high value thing to add. It seems more likely to derail the conversation than add anything to it. So why bother saying anything?

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 21:02 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

I'm providing the source so that interested readers can look it up and judge for themselves whether they want to believe it.

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 17:24 UTC (Fri) by shironeko (subscriber, #159952) [Link]

Totally, IMO a proprietary system ought not to limit what improvements can or cannot be done in a free project.
There's value in keeping the existing users as well off as they used to be of course, but that just means the changes introduced need to have backwards compatibility in mind, e.g. keeping the existing C code in maintenance only mode in parallel to rust code, and handle missing new features gracefully.

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 19:19 UTC (Fri) by pizza (subscriber, #46) [Link] (13 responses)

> It seems to me that, in the medium term, nearly every platform will end up having a Rust compiler targeting it, in the same way that nearly every platform ended up having a C compiler targeting it.

Unlikely in the medium term, but even in the long term, this is less likely if only because Rust is LargeAndComplicated(tm) whereas C... is not.

C is often (usually?) chosen because it is the least common denominator, in no small part due to the relative simplicity of bringing it up on a "new" platform. Then there is also the matter of stability and maturity of C tooling. When you're writing infrastructure-type stuff that needs to be supportable for what can easily be decades, this matters.

Having a Rust compiler is probably unavoidable

Posted Nov 22, 2024 20:13 UTC (Fri) by pbonzini (subscriber, #60935) [Link] (6 responses)

I don't think the complexity of the Rust language is particularly important with respect to porting it. The frontend is free and permissive, and Rust does not impose on the backend any more requirements that C++.

Of course, writing a new frontend from scratch is a different story.

Having a Rust compiler is probably unavoidable

Posted Nov 25, 2024 9:17 UTC (Mon) by glaubitz (subscriber, #96452) [Link] (5 responses)

> I don't think the complexity of the Rust language is particularly important with respect to porting it. The frontend is free and permissive, and Rust does not impose on the backend any more requirements that C++.

The problem is that the Rust developers can agree on a stable language specification which makes implementing a compliant alternative compiler such gccrs nearly impossible.

Having a Rust compiler is probably unavoidable

Posted Nov 25, 2024 15:35 UTC (Mon) by matthias (subscriber, #94967) [Link] (4 responses)

This is exactly what pbonzini said. Development of a new frontend is hard, which is what gccrs is doing. If you just want a compiler on your architecture you should take the existing compiler and add another backend. This is what rustc-codegen-gcc is doing. Another bonus is, that once you have done that most of the changes on the frontend will not require any changes on the backend any more. Gccrs however while have to reimplement all new features on their own frontend.

Having a Rust compiler is probably unavoidable

Posted Nov 26, 2024 13:06 UTC (Tue) by glaubitz (subscriber, #96452) [Link] (3 responses)

> This is exactly what pbonzini said. Development of a new frontend is hard, which is what gccrs is doing.

It's hard because Rust upstream cannot agree on a stable language but decided to keep changing the language all the time.

> If you just want a compiler on your architecture you should take the existing compiler and add another backend. This is what rustc-codegen-gcc is doing.

Which has been promised to be usable for at least two years now. It's still not usable.

> Another bonus is, that once you have done that most of the changes on the frontend will not require any changes on the backend any more. Gccrs however while have to reimplement all new features on their own frontend.

Currently rustc requires LLVM support for a given architecture which is almost only possible for architectures backed by big corporations as LLVM upstream requires people actively supporting a backend.

The M68k backend is only possible because people (which includes me) have created a funding campaign through Patreon and OpenCollective.

Having a Rust compiler is probably unavoidable

Posted Nov 26, 2024 15:14 UTC (Tue) by kleptog (subscriber, #1183) [Link]

> It's hard because Rust upstream cannot agree on a stable language but decided to keep changing the language all the time.

ISTM that C/C++ aren't really stable either as they keep coming up with new revisions and compilers keep inventing new extensions. I'm guessing the problem is not the lack of stability, but the rate of change?

That does make it hard for competing back-ends to keep up, but I expect the rate to reduce as all the pending ideas are worked out. That should make it easier in the long term.

Having a Rust compiler is probably unavoidable

Posted Dec 5, 2024 21:34 UTC (Thu) by ssokolow (guest, #94568) [Link]

Which has been promised to be usable for at least two years now. It's still not usable.
According to the repo's Insights tab on GitHub, it wavers between a one-man project and a two-man project at any given time (depending on whether bjorn3 or GuillaumeGomez is contributing in addition to antoyo) and, as the progress reports show, they're also busy writing patches to be upstreamed into libgccjit to flesh out the API they depend on.

Having a Rust compiler is probably unavoidable

Posted Dec 12, 2024 21:30 UTC (Thu) by sunshowers (guest, #170655) [Link]

As a Rust developer with several popular crates to my name, I'm not really interested in supporting compiler frontends other than rustc. I think backends are the right way to go.

Note that Rust doesn't make breaking changes. There's not going to be a completely unchanging language for many years, because the language is still quite incomplete. Meanwhile, I'm going to continue being part of teams that ship projects and deliver value to users, in ways that are simply not possible in other languages.

Having a Rust compiler is probably unavoidable

Posted Nov 23, 2024 0:22 UTC (Sat) by comex (subscriber, #71521) [Link] (5 responses)

Nobody standing up a new operating system starts by writing their own C compiler. Well, I'm sure there are hobbyist projects that do. But for serious projects, projects where you want to run existing open-source code? You port GCC or you port LLVM. These days, usually LLVM. Both because it's easier, and because a C compiler isn't enough: you almost certainly need a C++ compiler too. And C++ is *quite* large and complicated.

And if you have an LLVM port, it's not much more work to port Rust – if you bother. Having important open-source projects depend on Rust is a great way to ensure platform vendors *will* bother.

(Hopefully Rust's GCC backend will get to production soon, so that the same ease of porting will also apply to platform vendors who choose GCC.)

…Though to be fair, Rust moves more quickly than C or C++. Even if the vendor does include a rustc port, if they don't either contribute it upstream or keep it up to date with upstream, then it will quickly become less useful as the Rust library ecosystem starts using functionality from newer releases.

Having a Rust compiler is probably unavoidable

Posted Nov 23, 2024 16:56 UTC (Sat) by Karellen (subscriber, #67644) [Link] (2 responses)

You port GCC or you port LLVM. These days, usually LLVM. Both because it's easier, and because a C compiler isn't enough: you almost certainly need a C++ compiler too.

Sorry, I'm not sure I'm reading this correctly. It looks to me like it's implying that GCC doesn't have a C++ compiler? Can you clarify?

Having a Rust compiler is probably unavoidable

Posted Nov 23, 2024 18:35 UTC (Sat) by raven667 (subscriber, #5198) [Link]

I read that as saying it's simpler to port GCC or LLVM to a new platform because you probably want more than a C compiler (that you could in theory write yourself) you want C++ and other languages which GCC and LLVM support, that would be prohibitively expensive to write a new independent implementation for on your new platform. If the path of least resistance is already porting GCC or LLVM and Rust can be compiled using LLVM and soon GCC then your platform can also get Rust support because much of the underlying work will have already been done, it's not as if anyone needs to write a new Rust compiler for your new platform.

Having a Rust compiler is probably unavoidable

Posted Nov 24, 2024 5:59 UTC (Sun) by comex (subscriber, #71521) [Link]

I could have worded that better. The “because” sentence was meant to explain why you would port GCC or LLVM as opposed to writing your own C compiler. I didn’t explain why it’s usually LLVM these days, but the answer is some combination of the permissive license and the codebase being seen as friendlier.

…Not that I can speak for everyone writing an OS. And while I’ve never heard of a recent professional project starting with a bespoke C compiler, there *are* lesser-known commercial C++ compilers still under active development, like EDG and maybe Green Hills. There’s probably someone who still thinks it’s a good idea to use one of those for a new project. My guess is not many someones, though. And those projects are almost certainly embedded thingies that will never need to run Git.

Having a Rust compiler is probably unavoidable

Posted Nov 25, 2024 9:19 UTC (Mon) by glaubitz (subscriber, #96452) [Link] (1 responses)

> (Hopefully Rust's GCC backend will get to production soon, so that the same ease of porting will also apply to platform vendors who choose GCC.)

The usability of the GCC backend has been promised for several years now but it's still not materializing. I have stopped putting my hopes on it.

Having a Rust compiler is probably unavoidable

Posted Dec 5, 2024 21:36 UTC (Thu) by ssokolow (guest, #94568) [Link]

I could be wrong, but I don't remember seeing a promised timeline... just a (still ongoing) set of progress reports summarizing what is implemented and still to implement on a roughly monthly basis.

Having a Rust compiler is probably unavoidable

Posted Nov 25, 2024 9:04 UTC (Mon) by glaubitz (subscriber, #96452) [Link] (9 responses)

> It seems to me that, in the medium term, nearly every platform will end up having a Rust compiler targeting it, in the same way that nearly every platform ended up having a C compiler targeting it.

I don't think this will ever happen.

For one, the Rust community is trying to fight alternative Rust frontends all the time because there can only be one and, secondly, with the community being unable to stabilize the language, alternative frontends will never be able to catch up.

Having a Rust compiler is probably unavoidable

Posted Nov 25, 2024 15:39 UTC (Mon) by matthias (subscriber, #94967) [Link] (5 responses)

Why do you need alternative fronends? What you need is a backend for your architecture.

Having a Rust compiler is probably unavoidable

Posted Nov 25, 2024 16:05 UTC (Mon) by pj (subscriber, #4506) [Link] (2 responses)

For a language to be stable, I'd say multiple implementations are a must; kind of a source-code interoperability test. With only one frontend, there's likely un-noticed UB at odd functionality conjunctions; another implementation that (purposely or not) makes different choices is likely to bring such things to light.

Having a Rust compiler is probably unavoidable

Posted Nov 25, 2024 20:38 UTC (Mon) by tialaramex (subscriber, #21167) [Link]

This "more implementations = more stable" thinking makes sense if by stability you mean it can't change in any way whatsoever, it's just sort of stuck there. Certainly C++ 98 feels more "stable" in this sense than say Visual BASIC

The problem is that the universe changes, so, you're going to need to change too and with this "more stability" you're condemned to go without. I'm sure Tandem's NonStop had a perfectly good C89 compiler, and in 1995 that was excellent. by 2005 that was tolerable. In 2025 it will be a pretty miserable situation.

Having a Rust compiler is probably unavoidable

Posted Dec 14, 2024 18:40 UTC (Sat) by sunshowers (guest, #170655) [Link]

UB is shaken out better by alternative "midends" like Miri, which has had quite some success at doing so.

There are definitely benefits to another frontend, but they need to be balanced against the costs. I think the costs far outweigh the benefits right now.

Having a Rust compiler is probably unavoidable

Posted Nov 26, 2024 13:08 UTC (Tue) by glaubitz (subscriber, #96452) [Link] (1 responses)

> Why do you need alternative fronends? What you need is a backend for your architecture.

Thanks for proving my point ;-).

Try adding a new backend to LLVM without financial support from a big company. Let me know how it goes.

Having a Rust compiler is probably unavoidable

Posted Dec 5, 2024 21:40 UTC (Thu) by ssokolow (guest, #94568) [Link]

Getting a backend added to LLVM isn't the only possible route. rustc isn't locked into LLVM.

I'm aware of four backends already in various stages of completion. (LLVM, Cranelift, rustc_codegen_gcc, and rustc_codegen_clr.)

Having a Rust compiler is probably unavoidable

Posted Nov 25, 2024 16:43 UTC (Mon) by mb (subscriber, #50428) [Link] (2 responses)

>the Rust community is trying to fight alternative Rust frontends

Do you have any back-ups for this huge claim?
Because what I can see the exact opposite is true between gccrs, mrustc and rustc.

Having a Rust compiler is probably unavoidable

Posted Nov 26, 2024 13:10 UTC (Tue) by glaubitz (subscriber, #96452) [Link] (1 responses)

> Do you have any back-ups for this huge claim?
> Because what I can see the exact opposite is true between gccrs, mrustc and rustc.

Which one of gccrs and mrustc can be used yet for building Rust code on an architecture not supported by rustc?

In theory gccrs and mrustc are great. However, the reality is that they are not usable yet after all these years.

Having a Rust compiler is probably unavoidable

Posted Nov 26, 2024 17:34 UTC (Tue) by intelfx (subscriber, #130118) [Link]

>>> the Rust community is trying to fight alternative Rust frontends
>>
>> [...] what I can see the exact opposite is true between gccrs, mrustc and rustc.
>
> Which one of gccrs and mrustc can be used yet for building Rust code on an architecture not supported by rustc?

So are we still arguing that "the Rust community is trying to **fight** alternative Rust frontends", or is it something else entirely now?

Really tired of people arguing in bad faith.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 22, 2024 17:17 UTC (Fri) by danpb (subscriber, #4831) [Link] (37 responses)

> Complicating the discussion is the Git project's lack of a policy on platform support,

This is an unfortunate trap to fall into, suffered by many projects I've been involved with. For C in particular code ends up with an ever growing set of #ifdefs, and endless debates about changing them, as no one is ever confident whether any can be removed - there may always be someone still using a platform that needs them. Or you want to take advantage of new APIs in a library, but can't be sure whether you can increase the minimum required version of the library, or again need to add #ifdefs for old platforms someone might be using. Not having a platform support policy puts maintainers in a difficult position, leaving them with impossible judgement calls, and an ever growing amount of likely unused & obsolete code. Defining a clear policy for the scenarios or timeframes in which a project is willing/intending to drop support for a platform is essential to keeping the codebase manageable, as well as giving a clear message to users of what they can expect to work. People will complain with a platform is dropped, but it is unreasonable to demand a project support every conceivable platform forever.

> His main concern was with NonStop, a relatively niche platform primarily used in the financial sector for high-reliability computing.

IMHO Rust is an inescapable part of the future software ecosystem for many projects that are currently written exclusively in C, as it can offer such compelling benefits to them. In general it is not satisfactory to withold the benefits of using Rust for the mainstream modern platforms, to satisfy desires to keep obsolete and/or niche platforms working.

I can accept the argument that Git is special due to its position as a fundamental infrastructure component and thus it could be reasonable to take a more conservative approach to excluding platforms. That's merely going to tweak the time frames, punting the Rust problem down the road a little. If Git doesn't trigger a need for Rust support today, sooner or later enough other software will require Rust that people interested in a platform will have their hand forced. Either they invest to support Rust, or accept that the platform will become increasingly excluded from new releases of existing software projects.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 22, 2024 17:55 UTC (Fri) by NYKevin (subscriber, #129325) [Link] (7 responses)

It would be nice if we had a dumb Rust-to-C transpiler. Then the legacy platforms could use the output of that, and Rust's borrow checking would still provide many compile-time safety invariants since you'd also be building for mainstream platforms (without using the transpiler).

By "dumb" I mean "does not attempt to take advantage of invariants that can only be represented in the Rust type system, unless they can be directly translated into restrict or another C feature, without extensive codegen." In other words, you turn references into pointers, Box into malloc/free, tuples and whatnot into structs, etc., and don't try to do anything clever unless it's extremely easy. You lose some performance, but it's probably much easier to maintain, which is a good tradeoff to make for a legacy platform where you can't get "proper" upstream Rust support.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 22, 2024 18:05 UTC (Fri) by daroc (editor, #160859) [Link] (4 responses)

There is mrustc, which is written in C++ and compiles Rust to C11 code. I haven't taken the time to really dig into it, so I'm not sure whether it meets your criteria for a simple translation. It was suggested in the Git discussion thread, but nobody mentioned whether they had actually tried it. Also, the project is designed to be a way to bootstrap rustc, not as a full general-purpose implementation, so it's plausible that it wouldn't actually work for compiling non-rustc code.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 22, 2024 18:22 UTC (Fri) by NYKevin (subscriber, #129325) [Link] (3 responses)

It's not clear to me whether mrustc is suitable, today, for compiling code that is not rustc (and is also separately compiled with a more normal Rust toolchain, so at least we know it's valid and we don't have to emit "good" error messages). The README seems unsure on this point.

As for whether it's "dumb" enough - all I really want is something that is maintained. If they're going to continue maintaining mrustc, and explicitly support using it to transpile non-rustc code, then that would be fine regardless of whether it is "dumb" or "smart." But if, for whatever reason, mrustc turns out not to be suitable, then it seems plausible that it would be easier to write and maintain a "dumb" transpiler than a "smart" transpiler.

mrustc compatible with Rust 1.54

Posted Nov 24, 2024 2:38 UTC (Sun) by gmatht (guest, #58961) [Link] (2 responses)

Well, it compiled my pet project "ZeroSum Accounts Reconciliation Tool" fine: https://www.dansted.org/zerosum/

The transpiled code wasn't exactly concise, but it worked. The transpiled code can make some assumptions about the target architecture so you might have to fiddle mrustc if you want to target something exotic. Also, mrustc seems to assume that you will compile the resulting code with -fwrapv (otherwise implementation defined signed integer overflow in Rust will become undefined in the resulting C).

I imagine the main limitation of mrustc is that it only supports up to Rust 1.54 at the moment. So, this excludes Rust-for-Linux which requires at least 1.78.0.

mrustc compatible with Rust 1.54

Posted Nov 24, 2024 15:43 UTC (Sun) by ralfj (subscriber, #172874) [Link] (1 responses)

One difficulty with transpiling Rust to C is that there is plenty of well-defined Rust code (even plenty of safe Rust code) around raw pointers that is Undefined Behavior in C. It's quite hard to compile Rust to standards-compliant C. It would be a lot easier to compile Rust to a dialect of C that removes a lot of the UB (no strict aliasing [this one has a flag in many compilers, but the rest of this list does not], no UB on out-of-bounds pointer arithmetic, no pointer-lifetime-end-zap -- this list is not exhaustive), but C compilers would first have to gain support for such a dialect.

I don't actually know what current Rust-to-C transpilers do in these cases. I assume they just generate C code with UB and hope for the best...

mrustc compatible with Rust 1.54

Posted Nov 26, 2024 5:18 UTC (Tue) by NYKevin (subscriber, #129325) [Link]

Cast to intptr_t and hope for the best (i.e. hope that the arch is kind enough to give you a working intptr_t, and is not CHERI or CHERI-like). Or, if you're trying to support arches with no intptr_t, cast (a pointer to the pointer) to char*, and treat it as a string of raw bytes.

Yes, you lose provenance. No, that is not UB under any version of C that I've ever heard of. The standard explicitly provides that we are allowed to round-trip through at least char*, and intptr_t if it exists. If that breaks provenance, oh well, your implementation is just expected to deal with it anyway.

There probably is a performance cost to doing this. But IMHO it's acceptable if the alternative is "I can't run this code at all."

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 22, 2024 18:38 UTC (Fri) by josh (subscriber, #17465) [Link] (1 responses)

Codegen is not the only problem. Rust-to-C does not solve the problem of having a standard library that works on the target platform, or knowing the properties of the target platform (e.g. sizes and alignment).

Also, someone *is* working on a Rust-to-C transpiler.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 22, 2024 19:35 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

I am assuming the target platform has a somewhat reasonable libc implementation, so you can transpile the high-level stuff like alloc and (most of) std, and replace references to core etc. with references to the corresponding C things such as <stdint.h>. Even if it is not POSIX-flavored, I would expect that it is at least ISO-conforming.

If the target platform is bare metal or non-standards-conforming (or just really old), then this is much harder, but for a project like Git, you've presumably already figured out how to write and compile C code for that platform (or else it wouldn't be supported in the first place), so this should be a solved problem on some level, and might be as simple as writing a few polyfills for things that would otherwise be provided by libc. Or it might be much harder, if e.g. the platform is still using K & R C and doesn't know that argument types go inside the parentheses.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 24, 2024 6:02 UTC (Sun) by linuxrocks123 (subscriber, #34648) [Link] (25 responses)

> IMHO Rust is an inescapable part of the future software ecosystem for many projects that are currently written exclusively in C, as it can offer such compelling benefits to them.

No, I think this project is likely to kill off Rust:

https://github.com/pizlonator/llvm-project-deluge/blob/de...

If we can have garbage-collected memory-safe C and C++ with less than 2x overhead, the value proposition of rewriting the world in Rust is approximately zero.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 24, 2024 6:06 UTC (Sun) by linuxrocks123 (subscriber, #34648) [Link]

Btw, a Gentoo system with everything built using this compiler would have very useful security properties.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 24, 2024 9:38 UTC (Sun) by tlamp (subscriber, #108540) [Link]

> If we can have garbage-collected memory-safe C and C++ with less than 2x overhead, the value proposition of rewriting the world in Rust is approximately zero.

Well, by the linked manifesto the compiled executables are 2x bigger and 1.5x slower in average, that's a dealbreaker for a lot of things already, and that just to get the memory safety part of rust, which is by far it's only advantage.

Heck, I'd use rust even without the borrow checker over C/C++, any day. And I do not dislike C at all and wrote quite a bit of it in the past and will do some fun projects in it (C++ might be another story, but that's partially subjective, and I do not want to write a dozen of paragraphs here).

Don't get me wrong, for some projects using a compiler like the one linked can be a great step forward, if it proves itself to be actually stable and work out, but definitively not a "rust killer".

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 24, 2024 10:18 UTC (Sun) by mb (subscriber, #50428) [Link] (11 responses)

> No, I think this project is likely to kill off Rust

It looks like an interesting approach to make some legacy code bases safer, but I don't see how this could kill Rust in new developments.

First of all the speed penalty due to the amount of runtime checks is an immediate show stopper. Why would I want to pay that for new projects?

And Rust is so much more than just memory safety.

For example, it tells you about sooo many programming errors at compile time. A panic-ing Rust program is something that is extremely rare in practice, because so many things are caught at compile time. Rust gives you the right tools to design so many things into your types. And that in turn enables compile time checks for many things beyond bare memory safety.

That's completely different from how Fil-C that does everything at runtime. Adding so many opportunities to panic is a bad thing. As far as I understand the description you linked, a mere thread data race can panic the program. That is a show stopper. How are you going to develop robust programs with that? You can never be quite sure whether you spotted and protected all of your objects. (In Rust the compiler will tell you)

That gives me Python feelings, where one can never be quite sure whether a line of code is going to throw an exception and crash the whole program or not.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 25, 2024 0:08 UTC (Mon) by linuxrocks123 (subscriber, #34648) [Link] (10 responses)

> First of all the speed penalty due to the amount of runtime checks is an immediate show stopper. Why would I want to pay that for new projects?

No speed penalty: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/gpp-rust.html

If Fil-C can get it down to its target 20% to 50% overhead, C++ will still win versus Rust much of the time.

Plus, you can turn off the checks when you're in an environment where you're not too concerned about security. Rust overhead is permanent overhead.

> That gives me Python feelings, where one can never be quite sure whether a line of code is going to throw an exception and crash the whole program or not.

I know what you mean, because I also dislike that about Python. However, I don't find myself writing memory corruption bugs too often, so I don't get much value out of catching them at compile time.

Also, it's not like Rust can't crash at runtime. Just use an array index out of bounds and it will crash the same as C or C++ usually does (and sometimes, annoyingly, doesn't, and instead does the weird stuff that Fil-C puts a stop to).

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 25, 2024 0:51 UTC (Mon) by dvdeug (guest, #10998) [Link] (3 responses)

> No speed penalty: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/gpp-rust.html

> If Fil-C can get it down to its target 20% to 50% overhead, C++ will still win versus Rust much of the time.

Huh? Looking naively at the best numbers, Rust took 66% of the time C++ did on regex-redux, and the best C++ did was 70% of Rust's time on k-nucleotide. Many of the others they were neck-and-neck. If, if mind you, they can get it to their targets, Fil-C will be 20% slower than Rust much of the time.

To be honest about these numbers, take a look at the listings and the code for each one. Yes, there's a C++ program that solves it in 2.01 seconds. There's also a C++ program that solves it in 6.96 seconds. If you're optimizing code to an inch of its life, then that best time might matter. Otherwise, the question is a lot more complex; which language makes natural code fastest? Which language can you get to be fastest in X hours? Which language can you get fast enough in the least time?

> Plus, you can turn off the checks when you're in an environment where you're not too concerned about security. Rust overhead is permanent overhead.

There's so many cases where you're not concerned about security (i.e. no untrusted files) and concerned about a 20-50% speed boost. Rust interoperates with the C ABI, so you can write this code you're concerned about in C or better yet assembly if you really need the boost.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 25, 2024 3:28 UTC (Mon) by linuxrocks123 (subscriber, #34648) [Link] (2 responses)

> There's so many cases where you're not concerned about security (i.e. no untrusted files) and concerned about a 20-50% speed boost.

Uh, yeah. There are. There are some systems and environments that do need high security, and there are many, many more that don't need it. A compute server behind an internal firewall, which is getting its jobs by pulling from a secure Git host, would almost certainly turn off Fil-C for everything except, maybe, sshd.

A 20%-50% slowdown in that environment translates to $50k+ in additional hardware costs. If Fil-C can be competitive with Rust's performance for the "I want security" people, then C/C++ will continue to dominate the world, because they and Fortran are the only languages that can deliver for the "I want everything to run as fast as it can because my computers cost a lot of money" people.

The only way Rust has an edge is if government tells the "I want performance" people to shove it on the basis that insecure software has negative externalities. Governments have been making noise about doing that, but Fil-C will shut them up. If you think large software vendors aren't willing to shell out some dough to touch up Fil-C and make it faster, to prevent having to choose between losing government and government-beholden customers and throwing away millions of man-hours of work so they can rewrite everything in the latest fad programming language, you are very mistaken.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 26, 2024 3:37 UTC (Tue) by dvdeug (guest, #10998) [Link]

> There are some systems and environments that do need high security, and there are many, many more that don't need it.

That's the problem. We've long since learned that virtually all systems are useful to attackers, even if they don't permit access to some poor schmuck's bank account or corporate systems, as a step closer to that access, or as part of a bot net. If we're looking at tens of thousands of dollars of compute costs, that's value for hacking. If it's targeted, flash drives are well known to cross firewalls with ease.

> would almost certainly turn off Fil-C for everything except, maybe, sshd.

Penny-wise, pound-foolish. Recompiling everything is going to take more CPU time then you will ever save from most programs. Optimize the hotspots, not everything.

> If Fil-C can be competitive with Rust's performance for the "I want security" people, then C/C++ will continue to dominate the world, because they and Fortran are the only languages that can deliver for the "I want everything to run as fast as it can because my computers cost a lot of money" people.

Again, you linked to some numbers on this. Optimized Rust routinely meets or beats optimized C++, and unoptimized Rust or C++ was 5 or 10 times slower. This idea that C/C++ are the only languages that can deliver is not backed up by the data you provided, and it's not backed up by reality. There are some people who want the equivalent of racing stripes on their computer, but the really cost-conscious know that optimization needs work and timing and $50K in additional hardware costs is cheaper than another programmer.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Dec 14, 2024 2:09 UTC (Sat) by sunshowers (guest, #170655) [Link]

These days, tuned and safe Rust code can often be faster than highly-optimized C or C++ code. That is due to, among other things, Rust's constraints (e.g. noalias) making many optimizations possible.

See, for example, [1], where memory-safe Rust now outperforms C at decoding PNG images.

Writing Rust is also significantly nicer than writing C or C++. Not only are the error messages worlds better and you can *actually read them to understand what's going on* -- why would I want to do a convoluted pointer tagging thing at runtime when I can simply catch those issues at compile time?

[1] https://www.reddit.com/r/rust/comments/1ha7uyi/memorysafe...

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 25, 2024 3:03 UTC (Mon) by lambda (subscriber, #40735) [Link] (4 responses)

I think that something you're ignoring is that besides memory safety, Rust is a much nicer, more modern language than C or C++.

If I had a choice between writing Rust and writing memory-safe C++, I'd choose Rust. The only reason to choose C++ would be if there were some large existing C++ code base that I needed to interoperate with.

And you're wrong about overhead in Rust, as well as how Rust handles out of bound memory access. You can opt in to unsafe in Rust, and do things like unchecked array access, if there's a particularly hot place where bounds checking is causing enough overhead to worry about and you're confident that you're not going to access past the end of an array.

But in normal, safe Rust, an access past the end of an array will panic, essentially equivalent to an exception (if uncaught, will halt your program, but can be caught and some fallback performed if there's some way to recover).

In C and C++, you just get undefined behavior; anything could happen, and frequently what happens is a mysterious crash in some other part of your code that's hard to debug, or frequently a security vulnerability that's easily exploitable.

The Fil-C project is cool! I absolutely support work on making C and C++ safer. But the current speed penalty is quite high, and I'm skeptical that they can get to their target. Rust works right now, and can be optimized to a comparable speed as C and C++.

And Rust provides so many features and niceties that you didn't get in C and C++. The borrow checker helps you to check that you get your references correct at compile time; Fil-C only catches errors at runtime. Rust has move semantics by default, C++ makes you explicitly move and you can accidentally use a moved value and then get a runtime error. Rust has nicer lambdas, a great trait system, match statement, etc.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 25, 2024 17:53 UTC (Mon) by khim (subscriber, #9252) [Link] (3 responses)

> If I had a choice between writing Rust and writing memory-safe C++, I'd choose Rust.

I would choose C++, it's much more mature and it's metaprogramming factilities are pure joy to use (out of top 20 languages only C, CSS, and Rust don't have any reflection support in the language… so much for “more modern language”).

But the issue is: today's C++ is not memory safe, today's C++ libraries wouldn't be memory-safe tomorow (even if memory safe C++ would actually materiailize) thus that infamous Linus's “Linux Rust wins heavily on points of being available now” means we would have to switch to Rust and not to a hypothetical “subset of a superset memory-safe C++”.

I may not like it, but let's face reality: advantage of said hypothetical “subset of a superset memory-safe C++” wouldn't be enough to sway people.

> Rust works right now, and can be optimized to a comparable speed as C and C++.

And that's essentially it. I hate many aspects of Rust, but it exists, it works, it delivers. That's enough. You don't have to be perfect to win, you just have to be better then other offers that actually exist.

Approximately 10 years ago, when Rust have just been released, people were ready to wait for the alternative. That's how vapourware works and, sometimes, wins.

But if vapourware is not delivered after 10 years then we usually conclude it wouldn't be delivered at all. Thus we are left with Rust.

It's as simple as that.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 25, 2024 18:06 UTC (Mon) by pbonzini (subscriber, #60935) [Link] (2 responses)

> only C, CSS, and Rust don't have any reflection support in the language

Is anybody using reflexpr in production?

Also it depends on what you mean by reflection. For Rust, serde provides serialization and deserialization that satisfies a lot of the usecases for reflection.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 25, 2024 18:48 UTC (Mon) by khim (subscriber, #9252) [Link] (1 responses)

> Is anybody using reflexpr in production?

I don't know. It's not yet available in a stable form, anyway.

But with C++20 you already have pretty decent reflection: could count number of fields in a struct (here's an explanation about how that could be achieved), enumerate enums, deal with various pointers to functions, etc.

But the most useful ability is simple std::is_same_v, which allows you to look on the type and do something special for one type or another. Incredibly useful in so many places when combined with if constexpr. And incredibly awkward in Rust.

> For Rust, serde provides serialization and deserialization that satisfies a lot of the usecases for reflection.

Sure, but that requires the maker of these types to think about that need. There are also #test which solves another issue that many other languages handle via reflection.

This covers around 90% of usecases and actually make Rust tolerable, sure.

But the remaining 10% of usecases still exist.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Dec 5, 2024 21:53 UTC (Thu) by ssokolow (guest, #94568) [Link]

Sure, but that requires the maker of these types to think about that need.
That depends on what you mean by "think about that type of need".

Serde does have a workaround for "remote types" and, beyond that, you're bumping up against intentional Rust design decisions relating to API stability.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 26, 2024 18:19 UTC (Tue) by roc (subscriber, #30627) [Link]

Fil-C essentially turns every pointer into a "fat" double-word pointer. This is fundamentally less efficient than Rust where references and pointers to most types are a single word. That is a permanent performance disadvantage. OTOH, comparing the performance of a few small, hand-tuned toy benchmarks written in different languages as the basis for predicting a permanent performance gap between those languages on real large applications is totally invalid. I see no reason to believe Fil-C on idiomatic C++ will ever be competitive in performance with idiomatic Rust.

Don't get me wrong, Fil-C seems pretty cool and something like it could well be very useful for large legacy C++ codebases whose operators are willing to accept a significant performance hit for more security (when sandboxing is inadequate).

Fil-C, like similar efforts before it going back decades (e.g. CCured), has the problem of being ABI-incompatible with existing C code. All software running in an address space needs to be compiled with Fil-C, and JITs and other tools would need to be Fil-C aware. That limits its utility quite a lot compared to Rust.

And as others have noted, Rust provides a lot more safety than just memory safety. Fil-C isn't protecting against data races, for example.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 25, 2024 5:53 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (5 responses)

> No, I think this project is likely to kill off Rust

Sigh. No, it won't.

It can't compile most of the complicated software unchanged because it's no longer possible to case ints to pointers. Other than that, it has no ways to stop the mutator threads for collection. The code also scales poorly to multiple threads, as it requires essentially an atomic barrier on the first object access.

Avoiding this barrier would require black magic, extremely familiar to people working with JVMs in finance. It _is_ possible, but I don't think it's feasible for regular unmanaged code.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 26, 2024 3:04 UTC (Tue) by linuxrocks123 (subscriber, #34648) [Link] (4 responses)

The documentation is somewhat ambiguous on whether int-to-pointer works, but, if that doesn't work yet, that's unfortunate.

It's kind of weird if that really doesn't work, because all that has to be done is to just treat size_t, intptr_t, and uintptr_t like pointer types. I expect that solution will be implemented at some point.

Eventually, I'll do it if he doesn't :)

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 26, 2024 9:29 UTC (Tue) by jwakely (subscriber, #60262) [Link] (1 responses)

But size_t and intptr_t are just typedefs for something like unsigned long and long. How do you treat them as pointers, without treating all integers as pointers?

You can't rely on the typedefs always being used or visible in a given piece of code.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 26, 2024 16:02 UTC (Tue) by linuxrocks123 (subscriber, #34648) [Link]

If someone is casting a pointer to int and back and then dereferencing, that someone is doing something very bad and deserves to crash. You at least need to use size_t to make that sane.

It may be necessary to modify clang not to autodecay the typedefs, don't know about that, but he's definitely already modifying clang.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Dec 12, 2024 17:55 UTC (Thu) by sammythesnake (guest, #17693) [Link] (1 responses)

I quickly scanned through some of the website before coming back to see if everything I might have commented was already said by others (yeah, pretty much :-P) and I think the answer is as follows:

1. You use a union of pointer and the appropriate integer type.

2. You can access the bits of the pointer address by using the int member, including changing it.

3. The "InvisiCap" (essentially provenance plus size of accessible allocation) is tracked in the other half of the fat pointer that isn't available to the program, which means your can't change the address to something arbitrary and use it, you can only access the same stuff as before.

I remember reading about something in (proposed for?) Rust that sounded approximately equivalent , though with a somewhat different interface. ("Tree Borrow Checker"...?) Sorry for the vague description - I don't follow Rust all that closely :-P

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Dec 12, 2024 18:23 UTC (Thu) by intelfx (subscriber, #130118) [Link]

That's called a capability-based architecture. We have a real-world example of that; it's called CHERI, and the problem is that you can't just compile any existing software for CHERI (precisely due to the fact that it fundamentally disallows many things C and C++ programmers take for granted).

So this "Fil-C" thing is... essentially, an emulation of capability-based architecture. More power to them, I guess, but make no mistake — you absolutely cannot just take arbitrary C/C++ software and compile it with FIl-C, for exactly same reasons.

The Rust something you are referring to is called "Tree Borrows" (or its predecessor, Stacked Borrows). Those are attempts at *formalization* of Rust's memory aliasing model (essentially a document that tells you what is or isn't UB with respect to unsafely accessing memory). They are not really related or equivalent to "capabilities for Rust". Instead, the idea is that if a piece of code complies with Stacked Borrows or Tree Borrows, then it will also conceptually comply with any reasonable capability-based architecture.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 29, 2024 4:36 UTC (Fri) by ssmith32 (subscriber, #72404) [Link]

Hmm.. less than 2x is a pretty big window. 1.999x the hardware costs... is still a pretty penny in many situations.

Less than 1.10x ... then you're talking about something interesting.

"Safe C" variants like Fil-C vs. Rust

Posted Dec 6, 2024 11:18 UTC (Fri) by anton (subscriber, #25547) [Link] (3 responses)

I remember several memory-safe C implementations being presented at the poster session of PLDI 2002; those worked with fat pointers and introduced quite a bit of slowdown compared to C without that checking. Fil-C seems to be further development of this work, that addresses some of the drawbacks, e.g., ABI compatibility.

At ETAPS 2006 (IIRC) I heard a talk by George Necula about Ccured. This work apparently later led to Ivy and its component Deputy (manual): He said that ABI incompatibility was the reason for the lack of uptake of the fat-pointer implementations of C, and proposed an alternative using an enhanced type system for C; to make an existing program safe, at some places the programmer would insert assertions that inform the type checker; the compiler would then either prove that the assertions hold, or would insert run-time checks.

This would allow the gradual conversion of C code to safe code without needing to convert the whole program at one time. Looks like a winner, no? However, it apparently was not taken up at all. After the research projects ended, the Ivy stuff even partially disappeared from the web, and the github link above is a ressurection of that source code (from 2008).

I can only guess at why there was so little interest in this work. My guess is: C programmers think that their programs are secure because of their programming prowess, so they don't want to waste time on putting the assertions in, and they don't accept the slowdown from run-time checking. Their managers don't welcome the cost of putting assertions in and on slowing down the programs, so they are willing to listen to the programmers when they say that the programs are secure (and, except for a few vulnerabilities, they are).

A few years later, Rust appeared and is also using a type-system-based approach to safety. It requires a complete rewrite, which seems to be at a grave disadvantage compared to the gradual-rewrite approach of Ccured/Ivy/Deputy. But in contrast to Ccured/Ivy/Deputy, Rust is a success. Maybe Ccured/Ivy/Deputy suffers from an uncanny-valley-like effect: does not interest C programmers for the reasons above, does not interest non-C programmers because it is too much like C.

So what does this mean for Fil-C? It seems to me that while Fil-C does not require adding assertions, it does suffer from slowdowns, more so than Ccured/Ivy/Deputy, so it probably will still see little love from C programmers and their managers.

"Safe C" variants like Fil-C vs. Rust

Posted Dec 7, 2024 11:00 UTC (Sat) by kleptog (subscriber, #1183) [Link] (2 responses)

Speaking for myself: I'm not interested in a version of C that crashes more reliably when something goes wrong. We know that C crashes and it's annoying to debug because the crash happens far away from the actual bug, so fixing it is time consuming and frustrating.

Rust on the other hand prevents the errors happening in the first place, and the compiler points you to the exact place where the (potential) error is. This makes it far more pleasant to work with and gives you far more confidence in the end result.

Or to put it briefly: people care far more about programs being *correct* than them being *safe*. A Safe C doesn't help at all in that respect.

I'm putting more faith in some kind of bulk C->Rust conversion. From source would be hard, but I can image a kind of C interpreter that runs the program and analyses the runtime usage of objects to determine the appropriate types and lifetimes and then produces Rust source. Though I expect a lot a rewrites in the mean time, just because Rust is so much more pleasant to work with.

"Safe C" variants like Fil-C vs. Rust

Posted Dec 8, 2024 8:45 UTC (Sun) by Wol (subscriber, #4433) [Link] (1 responses)

> Or to put it briefly: people care far more about programs being *correct* than them being *safe*. A Safe C doesn't help at all in that respect.

And "safe" is a likely by-product of "correct". In fact, a "correct" program is far more likely to be safe than a "safe" one ... (as in it *doesn't* do anything wrong, rather than being allegedly incapable of doing anything wrong)

Cheers,
Wol

"Safe C" variants like Fil-C vs. Rust

Posted Dec 13, 2024 14:34 UTC (Fri) by sammythesnake (guest, #17693) [Link]

At I understand it, the proof standard used by Rust is very conservative - making narrow promises and rejecting anything uncertain, and therefore pretty much bulletproof (modulo bugs in the bottle checker etc. of course)

It's the same mindset of approach as the eBPF verifier - there are cases where "safe" programs are rejected by the verifier/checker because it's not (yet) sophisticated enough to prove their safety. Adding to that capability is always an option (until you hit the halting problem, but you can asymptotically approach it until the correct-but-rejected case is meaninglessly small)

Sadly, "correct" is a far more nebulous (and *larger*) thing to prove than the subset of correct that we call memory safety. If we could somehow define "correct" in a general case and implement a checker (that runs in finite time!) then proving correctness would naturally also imply proving memory safety - it's "incorrect" to access the wrong memory, after all!

I imagine that Rust might eventually get to a point where "correctness" is the focus of the proof, with "memory safety" being a historical note as merely an early part of "correctness" to be implemented.

Defining correctness and implementing proof strategies for it can be done in incremental steps, possibly with the assistance of assertions and with optional aspects of proof opted into as the cost of the extra analysis is warranted...

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 26, 2024 23:11 UTC (Tue) by tome (subscriber, #3171) [Link] (1 responses)

> IMHO Rust is an inescapable part of the future software ecosystem for many projects that are currently written exclusively in C, as it can offer such compelling benefits to them.

Agree, heartily.

Git is currently Rust-free. That's a feature. Git should announce the deprecation of Rust-free Git and allow some time for those impacted to adjust. HP has the motivation and the resources to fix it. Inevitably they would fund Rust support in LLVM or GCC or both, and use one of those to build Git for NonStop.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Nov 28, 2024 13:41 UTC (Thu) by foom (subscriber, #14868) [Link]

NonStop's users may want git to continue to work on NonStop, but IMO that would likely not be enough motivation for HP to actually invest the (possibly considerably expensive) work required to port Rust...

I agree, they probably will do a rust port _eventually_, just...not because of git. And it might happen a decade from now.

Supporting Rust is inevitable for platforms that want to remain viable for modern software releases

Posted Dec 29, 2024 17:12 UTC (Sun) by kentborg (guest, #70128) [Link]

If the git community has technical reasons for using Rust in git, let them use Rust in git. Of course they should do it in an organized way, but otherwise go ahead.

The financial industry pays for NonStop itself, and they can afford to pay to add Rust support to NonStop.

Free software is not some constitutional right of big business.

"Telling 10-20000 users that their core bit of infrastructure is insecure and not fixable is not a tenable position"

Posted Nov 22, 2024 17:19 UTC (Fri) by Wol (subscriber, #4433) [Link] (3 responses)

And who cares?

He who pays the piper calls the tune. If those users are not contributing back to the project, then in the FLOSS way they have no say in what happens. If they're not prepared to support that "lone developer in Nebraska" then tough on them.

There's also that other side to it, if their "core bit of infrastructure IS insecure and not fixable", then NOT telling them is not a tenable position, either.

How much *would* it cost for those 10/20K users to get a back-end to LLVM or gcc written? Given that it's x86_64, I guess it would be a drop in the ocean? And given that Non Stop apparently has a "Unix-like layer", it shouldn't be too hard to port stuff? If Stratus VOS can do it (as per wikipedia), why not HP Non-Stop?

Cheers,
Wol

"Telling 10-20000 users that their core bit of infrastructure is insecure and not fixable is not a tenable position"

Posted Nov 22, 2024 18:22 UTC (Fri) by tux3 (subscriber, #101245) [Link] (2 responses)

Yes, indeed. And I'm a little surprised how easily the community is being roped in a support job for a platform so secretive and proprietary that not even the binaries are available. Let alone the source.

The starving developers at big financial companies must not have the means to support their own secret, closely held platform choice!
Git must do the work to support them, since their usecase is so large and commercially important.

Although, not quite important enough for them to invest in their own platform support. That's what open source labor is for :)

"Telling 10-20000 users that their core bit of infrastructure is insecure and not fixable is not a tenable position"

Posted Nov 23, 2024 10:00 UTC (Sat) by tchernobog (guest, #73595) [Link]

Also, can't a small proprietary platform just keep using an old version of git, and add security patches on top? A lot of these people still use LTS toolchain with enterprise support.

There sure is a market for enterprise support to a git LTS version.

The financial sector uses ancient toolchains anywy, and have deep pockets by definition.

Do they really need the extra speed and extra features at this point?

"Telling 10-20000 users that their core bit of infrastructure is insecure and not fixable is not a tenable position"

Posted Nov 26, 2024 9:35 UTC (Tue) by jwakely (subscriber, #60262) [Link]

The article says "Randall Becker, one of the most currently active NonStop maintainers". So there are multiple people maintaining git for that platform. I'm sure they're doing it because of their jobs, not because they have NonStop systems in their closet as a hobby.

So that means those users (or their employers) *are* investing in it.

rust is a reasonable requirement

Posted Nov 22, 2024 17:34 UTC (Fri) by jhoblitt (subscriber, #77733) [Link] (4 responses)

Rust has tier 2/3 support for fairly niche systems like Nintendo 3DS. Clearly, if there is a will, there is a way.

If HPE isn't willing to, or doesn't have the resources to support, porting rust to NonStop the platform is in trouble.

Ultimately, the choice of implementation language should be solely up to the devs doing the work (and their employers).

rust is a reasonable requirement

Posted Dec 1, 2024 5:35 UTC (Sun) by riking (guest, #95706) [Link] (3 responses)

The primary blocker to adding a NonStop to Rust mainline is likely this line in the target tier policy about "onerous legal requirements".

https://doc.rust-lang.org/rustc/target-tier-policy.html?h...

rust is a reasonable requirement

Posted Dec 1, 2024 9:33 UTC (Sun) by intelfx (subscriber, #130118) [Link] (2 responses)

> The primary blocker to adding a NonStop to Rust mainline is likely this line in the target tier policy about "onerous legal requirements".

It does not have to be *in mainline*. If the platform is secretive enough that it has its own proprietary C toolchain, its owners might just as well fund a proprietary fork of the Rust toolchain or whatever. They don't even have to build one from scratch.

All in all, I concur with the general idea expressed in many comments here — the free software ecosystem probably isn't supposed to bend over backwards to benefit a secretive proprietary silo.

rust is a reasonable requirement

Posted Dec 1, 2024 11:42 UTC (Sun) by pizza (subscriber, #46) [Link] (1 responses)

> All in all, I concur with the general idea expressed in many comments here — the free software ecosystem probably isn't supposed to bend over backwards to benefit a secretive proprietary silo.

Git is currently a "very" portable C tool. Introducing Rust *instantly* collapses that wide platform portability to the subset supported by a *single* rapidly-changing toolchain with a deeply engrained "use the latest or GTFO" culture.

The question then is, are the benefits of adopting Rust into Git worth giving that portability (and ability to not live on the bleeding edge of tooling) up?

The thing is, git is *infrastructure*. Its portability policy is of general benefit to everyone, free software ecosystem included. So while it just so happens that NonStop benefits from the current policy of portability, that was a happy coincidence.

...Historically, especially when LLVM/Clang was benefiting from "portability improvements" said portability was widely proclaimed to be a good thing for the health of the free software ecosystem. Now that the shoe is on the other foot, suddenly it doesn't matter any more. I find that quite telling.

rust is a reasonable requirement

Posted Dec 1, 2024 12:36 UTC (Sun) by intelfx (subscriber, #130118) [Link]

> Introducing Rust *instantly* collapses that wide platform portability to the subset supported by a *single* rapidly-changing toolchain with a deeply engrained "use the latest or GTFO" culture.

If you're going to be taking preferences of one group of people and ascribing them to another group of people while using derogatory words...

> when LLVM/Clang was benefiting from "portability improvements" said portability was widely proclaimed to be a good thing for the health of the free software ecosystem. Now that the shoe is on the other foot, suddenly it doesn't matter any more. I find that quite telling.

...then taking a vague statement of unspecified genesis (dressing it in weasel words like "widely proclaimed") and putting that statement into your correspondent's mouth, and then insinuating hypocrisy on that basis, I don't think your participation will lead this discussion anywhere productive.

Easier?

Posted Nov 22, 2024 17:42 UTC (Fri) by ceplm (subscriber, #41334) [Link] (24 responses)

> making it easier for more people to contribute to Git …

???

Do you mean to say, with a straight face, that there are more Rust developers than the C ones?

Easier?

Posted Nov 22, 2024 18:08 UTC (Fri) by NYKevin (subscriber, #129325) [Link] (3 responses)

The argument is probably some combination of the following:

* While there are more C than Rust developers now, in 10 or 20 years that may no longer be the case.
* Once you are proficient in both Rust and C, it is easier to come up to speed on a Rust codebase than a C codebase (because the Rust compiler will enforce the correct ownership model and many other invariants that the C compiler does not understand).
* C is a simpler language, but that does not make it an easier language. By forcing you to write provably correct code from the beginning, the Rust compiler avoids the pitfalls of memory corruption and "impossible" crashes that many C beginners struggle with (before eventually learning many of the same lessons anyway, but without explicit syntax to describe what they have learned). Rust may have more complexity to learn up-front, but in the long run, it is an easier language to use, and ultimately easier to master, specifically because it forces you to deal with these issues instead of ignoring them until they bite you.

Easier?

Posted Nov 23, 2024 9:19 UTC (Sat) by ceplm (subscriber, #41334) [Link] (2 responses)

> * While there are more C than Rust developers now, in 10 or 20 years that may no longer be the case.

Do you believe, with the speed of the IT development, anybody will care for either of these programming languages in twenty years? Do you believe that anybody will care for git? And even if we do and the proportion of Rust/C-C++ changes tenfold, I guess git development could change (and the rewrite of git to Rust would already happen anyway).

Easier?

Posted Nov 23, 2024 14:44 UTC (Sat) by pbonzini (subscriber, #60935) [Link]

Most of the language currently in use have been around for at least 30 years (for example C, C++, Python, Lua; Java and PHP are at 29). Of all the languages that were in use in the late 80s and early 90s the only somewhat surprising demise is Pascal, which is probably due to the disappearance of its two main proponents (Borland and pre-NeXT Apple).

Perl is probably the least healthy of the various languages in use today. Maybe in 20 years people will treat C like they treat COBOL or Fortran, but the lesson of COBOL and Fortran is exactly that C and C++ *will* be around in 10 or 20 years just for the sheer amount of code that's written using those languages. Others like Rust or Go or Ruby maybe not have the same amount of code to protect them, but of the three I would bet on Rust the most because it's pretty much the only language to provide important advantages in the field occupied by C/C++.

Easier?

Posted Nov 24, 2024 5:41 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

> Do you believe that anybody will care for git?

Git _itself_ is 20 years old! People are still using RCS in some niche software, and it's 40 years old.

I can say for certain, that Rust absolutely will be around in 20 years.

Easier?

Posted Nov 22, 2024 18:15 UTC (Fri) by geofft (subscriber, #59789) [Link] (4 responses)

That seems totally plausible to me in the sense of people who can make a drive-by contribution - there are probably a lot more programmers who do not actively write either C or Rust and feel more confident making a small change in Rust and seeing if it compiles than trying to figure out what's going on with pointers in C.

Easier?

Posted Dec 3, 2024 9:01 UTC (Tue) by LtWorf (subscriber, #124958) [Link] (3 responses)

By that logic, python or js would be orders of magnitude better.

Easier?

Posted Dec 3, 2024 9:53 UTC (Tue) by intelfx (subscriber, #130118) [Link] (2 responses)

> By that logic, python or js would be orders of magnitude better.

Yes, except that Rust offers comparable (or better) performance to C code, while at the same time offering a significant amount of statically verified guarantees about the program's correctness (under the umbrella term of "safety", which is emphatically NOT just about "memory safety", read here for details). None of this is available in Python or JS.

So, unfortunately, it appears that you've entirely missed the point of "that logic".

Easier?

Posted Dec 3, 2024 10:27 UTC (Tue) by LtWorf (subscriber, #124958) [Link] (1 responses)

I didn't, but I think I was not explicit enough in my reply for you to not miss my meaning.

1. The increased complexity of building it would be a burden to the project

2. The quality of the developers is not the same

3. More developers doesn't imply more contributors

I hope it's clearer now.

Easier?

Posted Dec 3, 2024 12:30 UTC (Tue) by intelfx (subscriber, #130118) [Link]

The entire discussion happens in a context, which you have missed. All of the "increased complexity" that you cite is offset by the benefits Rust brings in terms of increased ergonomics *while simultaneously* not reducing performance and upholding a baseline of safety, which means that the contributor base can grow without significant detriment to code quality and performance. Neither of Python or JS meets this objective.

So, no, I did not miss "your meaning". If you had any other meaning, you should have said something that conveys that meaning.

Easier?

Posted Nov 23, 2024 1:45 UTC (Sat) by khim (subscriber, #9252) [Link] (4 responses)

> Do you mean to say, with a straight face, that there are more Rust developers than the C ones?

No, but anyone may say, with a straight face, that the vast majority of developers, these days, have no idea how to write code in any language which have piles of UB and which doesn't notify you if you do mistakes.

You can give Rust compiler to TypeScript or Python developer and while they may not like it… they would be able to write something working.

You can not give them a C compiler and expect them to write code that works.

Because they implicitly expect that code either works or breaks and they get an error message – that's true for Rust without unsafe and very explicitly not true for C.

Easier?

Posted Nov 23, 2024 13:25 UTC (Sat) by pizza (subscriber, #46) [Link] (3 responses)

> while they may not like it… they would be able to write something working.

No.

They may/will be able to write something without UB, but that doesn't mean it actually *works*.

The *works* is the critical point. It doesn't matter if git is written in rust or javascript or C or algol or whatever. You still have to have at least some understanding of what and how git does things -- core concepts, data structures, algorithms, etc -- if there is going to be any chance of producing something that works.

And sure you can just bash against the compiler until your rust code compiles, but again, that doesn't mean that the resulting binary _works_ for its intended purpose.

Easier?

Posted Nov 23, 2024 15:56 UTC (Sat) by khim (subscriber, #9252) [Link] (2 responses)

> And sure you can just bash against the compiler until your rust code compiles, but again, that doesn't mean that the resulting binary _works_

It kida does, though. C/C++ is more-or-less unique among popular languages in that you may easily write code that compiler would accept and that would works, for some time, but may then explode if you change something in entirely different place.

In most other popular languages if code compiles – then it works… somehow. It may not do what you want, it may even fail (and most likely would fail on first try), but it fails in predictable fashion!

> for its intended purpose.

That's different story altogether. Once your code works but doesn't do what you want it to do you start the “genetic programming”: start mutating code semi-randomly while picking better and better behaving specimens till it would do what it needs to do. StackOverflow is a great thing at this point, ChatGPT or Gemini are even better.

> It doesn't matter if git is written in rust or javascript or C or algol or whatever.

It does, though. Very much.

> You still have to have at least some understanding of what and how git does things -- core concepts, data structures, algorithms, etc -- if there is going to be any chance of producing something that works.

Well… that's precisely what makes is so hard to contribute, isn't it? The algorithm explained above that the vast majority of modern programmers employ… it just simply doesn't work with C/C++!

Because in C/C++ wrong code may actually work by accident in your tests, but then it would fail in larger program and random mutations don't bring you closer and closer to something you may declare as “finished” but just make it harder and harder for anyone to fix it.

Easier?

Posted Nov 23, 2024 16:29 UTC (Sat) by pizza (subscriber, #46) [Link] (1 responses)

> In most other popular languages if code compiles – then it works… somehow. It may not do what you want, it may even fail (and most likely would fail on first try), but it fails in predictable fashion!

...Are you seriously asserting that something can still "work" even if it "it fails" ?

(Well, if by "works" you mean "compiler accepts it and it produces a runnable binary" then sure. But most would consider that a necessary, but wholly insufficient milestone)

Easier?

Posted Nov 23, 2024 16:45 UTC (Sat) by khim (subscriber, #9252) [Link]

> But most would consider that a necessary, but wholly insufficient milestone

Sure, but in most language that perfectly fine basis for the iterable attempt to fix your code. config is undefined – oh, damn, I forgot to read the config file, ok, done panic: index 24325462 is out of bounds – wow, I guess I mess up with my calculations, somehow… better to fix that, and so on.

In C/C++… that approach doesn't work at all: usually your code still fails but without any breadcrumbs which allow you to go and tweak the code randomly till error messages would stop occuring.

Because in C/C++… very often the actual problem that makes you code fail (some wild pointer or buffer overflow or any other silly problem) is happening in entirely different piece of code, not where something actually crashes!

That's such a crazy wild disconnect from what the most contributors (the ones who don't know C/C++, but know JavaScript, C#, Go, or any other popular language) expect that C and C++ form it's own wild subculture entirely disconnected from all other languages.

Sure, there are also a lot of infigthing withing that subculture, too, but it's entirely separated from the other languages – and that division also reduces number of potential contributors.

In C/C++ people consider it the norm that you would first try to understand core concepts, data structures, algorithms, etc of the project – and only then would attempt to contribute… but in all other languages it's the opposite: you first contribute something and as you add more and more code to the project you, gradually, learn about core concepts, data structures, algorithms, etc of the project.

Contributing to the project written in Rust is the same as contributing to the project written in any other popular language and thus open up your project to a much wider audience.

Easier?

Posted Nov 23, 2024 2:07 UTC (Sat) by dvdeug (guest, #10998) [Link] (4 responses)

I have no intention of working with a programming language that doesn't have basic data structures builtin, and expect, like C does, for you build them out of arrays and pointers. While I'm at it, proper modularity would be nice. I have to work with C sometimes, but I'd really rather not.

C++ is terrifying; it's like four layers of experimental language stacked on a weak foundation.

Rust? It's more low-level than I like, but it's a reasonably well designed language of the 2010s, not a reasonably well designed language of 1970s, or several decades of experimental languages piled on top of that language of the 1970s.

Easier?

Posted Nov 23, 2024 9:13 UTC (Sat) by ceplm (subscriber, #41334) [Link] (3 responses)

Notice, that I haven’t tried for a second to discuss the merits of any language whatsoever (I don’t consider myself to be a professional programmer in neither C nor Rust). I was just commenting that to say it would be “[…] easier for more people to contribute to Git […]” if it was in Rust, is completely insane in the situation that (according to the TIOBE index; I know it is wildly inaccurate, but it is the best what we have) there is roughly ten times more C++ programmers (who can be assumed to be at least somehow competent in plain C) plus nine times more C programmers than of the Rust programmers.

That is all I was saying.

Easier?

Posted Nov 23, 2024 11:45 UTC (Sat) by khim (subscriber, #9252) [Link] (2 responses)

According to that logic we should use Chinese and not English here, because there are way more Chinese speakers in the world than English speakers.

Now… why don't we do that? Hmm?

Easier?

Posted Nov 23, 2024 12:06 UTC (Sat) by aragilar (subscriber, #122569) [Link] (1 responses)

https://en.wikipedia.org/wiki/List_of_languages_by_total_... would suggest English no (arguably, we actually care what language has the most people who can read/write it, given this is a textual medium, rather than a audible medium, but I wouldn't expect the numbers to be that different)?

Easier?

Posted Nov 23, 2024 12:42 UTC (Sat) by khim (subscriber, #9252) [Link]

Sigh. Please read the very first sentence of the very page that you linked to: For example, Chinese and Arabic are sometimes considered single languages, but each includes several mutually unintelligible varieties, and so they are sometimes considered language families instead. Conversely, colloquial registers of Hindi and Urdu are almost completely mutually intelligible, and are sometimes classified as one language, Hindustani.

> arguably, we actually care what language has the most people who can read/write it, given this is a textual medium, rather than a audible medium, but I wouldn't expect the numbers to be that different

Indeed, there are big difference. English have significantly less speakers, but if you add up all the people who may write something in it (even if in a horrible broken form) you would arrive at about the same number as for Chinese (around 1.5 billion for both).

And similarly with languages: if you look on the graph you'll see that number of projects that include Rust code is almost the same as number of projects that include C code and significantly more than number of projects that include a Dart code… in fact there are more code in Rust on Github than in, e.g., Kotlin, but Kotlin is considered way more popular than Rust because there are more questions in Stack Overflow!

And with programming language there are also big difference between the ability to understand the language and ability to contribute code in the language.

C++ have both more code and more questions and there are, arguably, more people who may understand C code than Rust code right now… but would such people really be ready and willing to contribute to the project in the language where even simplest bast data structures are not standardized and you have to learn what bespoke solution was used in this particular piece of code?

Easier?

Posted Dec 13, 2024 0:30 UTC (Fri) by sunshowers (guest, #170655) [Link] (4 responses)

C is extraordinarily difficult to write correctly to a modern standard, even for people with decades of experience in it. Rust is not easy, but at least the compiler does a pretty good job helping you.

There are entire communities of "hardcore" C/systems devs who have quietly moved to Rust—and don't really want to write C any more. There is a massive wave that's been building for a few years, and it's only going to get bigger over time.

Easier?

Posted Dec 13, 2024 1:17 UTC (Fri) by pizza (subscriber, #46) [Link] (2 responses)

> C is extraordinarily difficult to write correctly to a modern standard, even for people with decades of experience in it. Rust is not easy, but at least the compiler does a pretty good job helping you.

The barrier to contribution isn't "skilled in C" so much as "skilled in that particular codebase."

> There are entire communities of "hardcore" C/systems devs who have quietly moved to Rust—and don't really want to write C any more. There is a massive wave that's been building for a few years, and it's only going to get bigger over time.

Give a highly skilled person a more complex tool, and after some practice they may be able to achieve greater things. But you're not oging to get the same results by handing the same tool to someone that is barely literate.

Easier?

Posted Dec 13, 2024 3:46 UTC (Fri) by intelfx (subscriber, #130118) [Link] (1 responses)

> The barrier to contribution isn't "skilled in C" so much as "skilled in that particular codebase."

By virtue of having more high-level abstractions and generally a more modern/ergonomic design, Rust makes it easier to get up to speed within a given codebase[1]. It does not necessarily make you *skilled* faster, but it's likely to make you *productive* faster.

> Give a highly skilled person a more complex tool, and after some practice they may be able to achieve greater things. But you're not oging to get the same results by handing the same tool to someone that is barely literate.

That's not what GP was saying. In this analogy, C is the "more complex" tool.

[1] for a very simple reason: you spend less time on rediscovering how a particular codebase does fundamental things, because all of those fundamental things are baked into the language (or, worst case, implemented in highly popular libraries which are likely part of the dependency tree anyway).

Easier?

Posted Dec 14, 2024 1:57 UTC (Sat) by sunshowers (guest, #170655) [Link]

Interestingly, I started contributing to wasmtime's internal memory management recently. The details are quite subtle, and actually don't fully fit the Rust model -- some logically-borrowed memory is represented without lifetime parameters.

If wasmtime were in C or C++, it would have taken me a long time to wrap my head around all of the details. I'd probably had to have one of the maintainers walk me through it on a video call. But with Rust, I went from never having seen the code before to having a ~800 line prototype of a fairly major internal refactor within a day. I was contributing fixes and identifying bugs the very next day too.

Rust is great for novices to a codebase.

Easier?

Posted Dec 13, 2024 2:44 UTC (Fri) by viro (subscriber, #7872) [Link]

Rhetorics about "building waves" ought to be treated as Godwin-equivalent, really. Regardless of the subject...

it can go both way!

Posted Nov 22, 2024 20:51 UTC (Fri) by ballombe (subscriber, #9523) [Link] (5 responses)

If one allows developers to replace some C code to rust, as a matter of fairness one should allow developers to port some rust code back to C!

it can go both way!

Posted Nov 23, 2024 1:51 UTC (Sat) by dvdeug (guest, #10998) [Link] (4 responses)

Why? The point of replacing C with Rust is that it's more reliable and easier to maintain, while being reasonably as fast. There's no fairness involved; changes between languages should be for rational reasons, not territorial ones.

it can go both way!

Posted Nov 23, 2024 8:52 UTC (Sat) by Wol (subscriber, #4433) [Link] (3 responses)

I think you're missing the tongue-in-cheek.

There are probably a lot more developers (famliar with Rust) who would like to contribute but are crap at C, than there are good C developers who would like to port from Rust to C.

Cheers,
Wol

it can go both way!

Posted Nov 23, 2024 20:27 UTC (Sat) by jhoblitt (subscriber, #77733) [Link] (2 responses)

In that vein...

The size of one of those groups is zero because there aren't any devs that aren't crap at C. There are, at best, devs who don't understand that they are crap at C.

it can go both way!

Posted Nov 23, 2024 21:51 UTC (Sat) by pizza (subscriber, #46) [Link] (1 responses)

> The size of one of those groups is zero because there aren't any devs that aren't crap at C. There are, at best, devs who don't understand that they are crap at C.

The same can be said about *any* human endeavor.

...Meanwhile, continue to crap all over those who write the code you depend on. I'm sure that will convince them of your non-crappiness.

it can go both way!

Posted Nov 23, 2024 23:10 UTC (Sat) by jhoblitt (subscriber, #77733) [Link]

I think you're missing the tongue-in-cheek.

The tune sounds familiar

Posted Nov 22, 2024 21:50 UTC (Fri) by smoogen (subscriber, #97) [Link] (5 responses)

For some reason, the 'can't move to Rust' seems very familiar to other new languages. Sometimes it is the right call and other times it turns into trying to stop an avalanche. The ones I can remember are:

* Can't move this <Fortran|Snoball|Algol> code to <C, C++, Java> because we need it to work on our platform and there isn't a compiler for that.
* Can't move this C code to C++, Java, Go and I guess now Rust because ...

The first one was a problem until the gcc toolset gained widescale adoption. Some code like Fortran scientific code took much longer to move to a different language and most I know is now in Python but I expect there is a lot still around. I am really hoping the Algol/Pascal/Snoball code got moved or ended in the 90's.

The second one has come up many times when a new C-like language comes into play. The new language has its proponents who either actually code in it or just want to play with it and think it will solve ALL the problems they currently have. One of the the first complaints are that the new language doesn't have full support on some subset of systems.

A better comment is "Are you really interested in doing that code change or are you just wanting others to do it." With any new language there seem to be proponents who will talk about ALL the great things the new language has and why it should be used instead of the current language. However, they aren't really interested in doing much of the work themselves or will only do it if enough other people will do it also. [I was one of these people in the late 80's with Fortran to C on some projects... ]

Where I have seen these sorts of things work is where someone just goes and does a good set of work already and can say "Ok I moved this code to <new language>. The old code took X lines, this amount to compile, this amount of time to test and this amount of time to run. The new code takes Y lines, etc etc etc. This work usually gets more movement in a group of other people who might have been 'interested' to start helping or show enough that it wasn't worthwhile.

The tune sounds familiar

Posted Nov 23, 2024 9:03 UTC (Sat) by Wol (subscriber, #4433) [Link] (2 responses)

> Some code like Fortran scientific code took much longer to move to a different language and most I know is now in Python but I expect there is a lot still around.

Given that for much of FORTRAN, the language's speed was a major selling point, you could be right. Okay, we've now got much faster computers, but back when tomorrow's weather forcast took 24hrs to run give or take, porting to a slower toolchain was a no-brainer - DON'T!

It's amazing as computers have got faster, the time available for tea-breaks has got longer :-) (My colleagues in the section next door now run their models twice a day, not once an hour ...)

Cheers,
Wol

The tune sounds familiar

Posted Dec 16, 2024 23:52 UTC (Mon) by ceplm (subscriber, #41334) [Link] (1 responses)

It isn’t … that is in the Python. You still require Fortran for compiling scipy, for example.

There are just some Fortran modules which won’t be ever ported from Fortran. If you have a code which some PhD student spent her whole postgrad program working on (and there are such), there is no way that it could be rewritten in Rust any time soon.

How is that rewrite of libm to Rust going?

The tune sounds familiar

Posted Dec 16, 2024 23:59 UTC (Mon) by ceplm (subscriber, #41334) [Link]

OK, there is such a project … https://github.com/rust-lang/libm/, but just from browsing the issue tracker, I won’t use it for calculating medications any time soon.

The tune sounds familiar

Posted Nov 23, 2024 11:52 UTC (Sat) by aragilar (subscriber, #122569) [Link] (1 responses)

There's still lots of Fortran written, and usually what I've seen is it being wrapped rather than be rewritten (and rewrites have tended to be for features rather than language).

I think if you're going to propose rust (unlike other languages people might suggest), specifying what's your proposed MRSV policy is should be one of the details provided, as given Rust's fast release cycle (every 6 weeks), and enough crates chasing the latest version, it would be better not to shock people when they need to upgrade.

The tune sounds familiar

Posted Nov 23, 2024 15:20 UTC (Sat) by pbonzini (subscriber, #60935) [Link]

> what I've seen is it being wrapped rather than be rewritten

The same will happen for large C programs, where you can have Rust wrappers that allow adding new leaf functionality in Rust.

Git in this sense has an advantage because some of the wrappers already exist (there is Rust code out there that interacts with git repositories).

NonStop should just port Rust

Posted Nov 22, 2024 21:55 UTC (Fri) by mcon147 (subscriber, #56569) [Link]

Git shouldn't be held back by a non-open-source OS, if NonStop doesn't have rust support they should pay to make that happen. The community shouldn't feel beholden to proprietary projects.

gitoxide?

Posted Nov 22, 2024 22:58 UTC (Fri) by gren (subscriber, #3954) [Link] (8 responses)

I was a bit surprised not to see any mention of gitoxide - https://github.com/GitoxideLabs/gitoxide

Apache or MIT licenced, not GPL - so I'm guessing there can't be much (any?) cross pollination and although it seems plenty active it isn't there yet but interesting all the less.

The README talks about it being a good project for learning Rust and as I'm interested in git as a user and in learning Rust it has been on my some day radar.

gitoxide?

Posted Nov 23, 2024 0:19 UTC (Sat) by hunger (subscriber, #36242) [Link] (4 responses)

Oh, git can take gitoxide code and relicense that to GPL just fine. The other way around is not an option.

gitoxide?

Posted Nov 23, 2024 7:43 UTC (Sat) by mb (subscriber, #50428) [Link]

>can take gitoxide code and relicense that to GPL just fine.

No?
A GPL project can integrate and use MIT/Apache2 code, but it can't relicense it to GPL.

gitoxide?

Posted Nov 24, 2024 15:12 UTC (Sun) by draco (subscriber, #1792) [Link] (2 responses)

That's not true as written, though it's a common misunderstanding.

They can't relicense the code, but they can combine it with GPL code and offer the combination under the GPL (while continuing to comply with the license requirements of the code that was taken).

The code that was taken can be taken again from the GPL project without complying with the GPL as long as they don't grab any of the new GPL bits.

So the existing license isn't changed. If you modify the files to strip out the copyright information and other things the license says you can't do, you will violate the license.

gitoxide?

Posted Nov 24, 2024 15:26 UTC (Sun) by intelfx (subscriber, #130118) [Link] (1 responses)

> They can't relicense the code

They most certainly can, in the colloquial meaning of the word. Both MIT and Apache-2.0 licenses (that Gitoxide is offered under) do grant the right to sublicense.

gitoxide?

Posted Dec 13, 2024 15:01 UTC (Fri) by sammythesnake (guest, #17693) [Link]

"sub-" vs. "re-" is an important distinction. Perhaps the practical implications aren't much in many issues, but they're not the same thing and if you're not careful, you can find yourself accidentally crossing the border where the distinction starts to matter, potentially with very messy consequences...

gitoxide?

Posted Nov 23, 2024 0:53 UTC (Sat) by tialaramex (subscriber, #21167) [Link] (1 responses)

Yes, it is interesting. In my experience Rust projects are much easier to "get into" than all but the most welcoming and easy to learn Free software projects in say C. Gitoxide does seem like a great long term project because by default Cargo will give you a git repo (assuming git is available) for each new project you make unless all your projects already live inside a single git repo.

As you noted, Gitoxide is not GPL. But actually I think Git is a big enough deal that it would be fine if there were two different popular free software Git implementations, and that as for say PNG or Opus or zlib this is more valuable if everybody can do it, when it comes to the core facilities at least, not like a fancy GUI or whatever.

gitoxide?

Posted Nov 23, 2024 9:10 UTC (Sat) by Wol (subscriber, #4433) [Link]

> As you noted, Gitoxide is not GPL. But actually I think Git is a big enough deal that it would be fine if there were two different popular free software Git implementations, and that as for say PNG or Opus or zlib this is more valuable if everybody can do it, when it comes to the core facilities at least, not like a fancy GUI or whatever.

And isn't the core of git the data structures, not the code on top? Provided the structures don't change (and by now they're probably set somewhat firmly in stone), who cares how many front ends we have? As I understand it, there are three distinct layers to git - the data, the plumbing, and the porcelain.

Port the plumbing to Rust (I gather there's actually not a lot), and then who cares what language is used for the porcelain?

Cheers,
Wol

gitoxide?

Posted Nov 26, 2024 22:50 UTC (Tue) by MrWim (subscriber, #47432) [Link]

Yeah, gitoxide is an amazing project. The author seems really dedicated to correctness - in terms of hunting down edge cases and compatibility with git. And it's fast too, both in terms of single-core efficiency and multi-core scaling.

The author posts monthly progress updates here: https://github.com/GitoxideLabs/gitoxide/discussions

Now that's a name I haven't heard in a long time... a long time.

Posted Nov 23, 2024 0:23 UTC (Sat) by tialaramex (subscriber, #21167) [Link] (6 responses)

Thirty years ago I worked for a company which no longer exists, Tandem Computers. I was fresh out of school (it was a "year out" programme designed to provide management skills to engineers, which, in hindsight, is a bad idea but oh well) before University.

Tandem NonStop is very old but it had some unique capabilities which render it especially attractive if you're extremely risk averse, as some banks and other financial sector organisations are. Unfortunately the other thing about banks when it comes to buying hardware (as opposed to paying staff evidently) is that they don't like to give you any money, and I believe this meant Tandem began to struggle financially and was sold.

I guess I'm not surprised that the platform lives on even though I doubt many people involved in designing and building it - never mind the hardware it ran on originally - are still active. Risk averse people weren't very interested in buying a new machine but can be relied on to pay for your Extended Support, and your Further Extended Support and your No Really This Platform Is Obsolete Please Migrate Now support.

However, I do wonder if maybe the correct way to view the situation is that support for git, the software, on NonStop is not necessarily a good goal, while support for git the _version control system_ on NonStop is absolutely worthwhile. It sounds as though NonStop people do not have a modern C++ compiler, don't have Go, don't have Rust, actually it sounds though they don't yet have C17, which isn't even the current version of C, it's the previous version. None of which is a reason they shouldn't have Git's version control system, but it is a reason that they might not get (and indeed it seems already they're beginning not to get) shiny new things.

Now that's a name I haven't heard in a long time... a long time.

Posted Nov 23, 2024 0:53 UTC (Sat) by NYKevin (subscriber, #129325) [Link]

The folks in these industries will simply continue to use the obsolete tech until it is physically impossible to repair or maintain. In 2019, the US Air Force announced that it had figured out how to launch a nuclear missile without 8" floppy disks. I'm not sure exactly when manufacture of those disks ceased, but it was certainly well before 2019, so either they continued to reuse old disks until they stopped working, or they had to buy used disks. If the latter, then I'm sure they had a fun conversation with their infosec people about low-probability-high-impact risks.

Source: https://www.nytimes.com/2019/10/24/us/nuclear-weapons-flo...

Now that's a name I haven't heard in a long time... a long time.

Posted Nov 23, 2024 9:20 UTC (Sat) by Wol (subscriber, #4433) [Link] (2 responses)

> Thirty years ago I worked for a company which no longer exists, Tandem Computers.

Same timeframe, a friend of mine worked for Stratus. I think they used a "better" - aka more expensive - approach to fault-tolerant computers, and Tandem pretty much put them out of business. But one of my memories is of my friend opening a case to a computer, pointing out the CPU board, and saying "if I pull that out, nobody will notice anything has happened - except us, because that's the main fault reporting computer which will go mad telling itself it's failed". He also said it was the norm for customers not to know their computers had suffered a major failure until the engineer turned up with replacement parts ...

I don't think upgrading those systems would be that complicated, although it probably would be expensive, especially given that there are probably few customers left ...

Cheers,
Wol

Now that's a name I haven't heard in a long time... a long time.

Posted Nov 23, 2024 13:30 UTC (Sat) by tialaramex (subscriber, #21167) [Link] (1 responses)

Yes, at Tandem the offered resilience was belt + suspenders + sky hook. If one of the local CPUs, or disks, or interconnect buses fails, those processes which must live on just have a recent checkpoint started on the other CPU, or disk, or interconnect (you can say whether your process is important in this way) and then if your entire computer fails, processes which are even more important just restart from a recent checkpoint on another machine at that site. Finally, if the entire site goes offline (e.g. meter destroys it, massive building fire) the absolutely critical processes automatically continue from a recent checkpoint in another data centre.

My understanding is that doing this in hardware is significantly more expensive and the practical benefit is very dubious, which means that over time "NonStop" goes from a solution requiring expensive custom hardware to a software system running on commodity hardware, without substantial difference in the results. I can't say how true that is in practice.

Now that's a name I haven't heard in a long time... a long time.

Posted Nov 24, 2024 0:50 UTC (Sun) by Wol (subscriber, #4433) [Link]

> and the practical benefit is very dubious

What happens to all the transactions that happened between the checkpoint and the failure?

Okay, I suspect that can be programmed around, but that probably means everything has to be coded in a batch manner, with checkpoints between each batch. Then it's down to the users to spot problems as the data is fed into the system, but once that's happened, it's guaranteed to flow through the system.

Cheers,
Wol

Now that's a name I haven't heard in a long time... a long time.

Posted Nov 26, 2024 7:56 UTC (Tue) by jpfrancois (subscriber, #65948) [Link] (1 responses)

What I fail to understand, is why they can't use git on the machine they use to develop as opposed to the machine the code runs on.
I mean, I have been developping code for ARM machine for 20 years, yet all my development is done on x86 or x86_64.
So what are this NonStop users doing with git ? Keeping track of their machine configuration ?

Now that's a name I haven't heard in a long time... a long time.

Posted Nov 26, 2024 9:50 UTC (Tue) by farnz (subscriber, #17727) [Link]

It's not uncommon for configurations to be stored in git, and for systems to read configuration from a checkout of the git repo. If you remove git from the supported toolset, then instead of configuration being "check in a config to your trusted git server, automation on the NonStop host will do the equivalent of git pull && git checkout protected/config && restart-services on a known schedule", it becomes "set up a system with remote access to the NonStop filesystem; have it do the git pull && git checkout protected/config, then synchronise the resulting checkout with the NonStop filesystem, then run restart-services on the NonStop system".

That's quite a lot of extra complexity (a whole system extra, plus redoing the code git already has for synchronising state from repo to filesystem, plus having to add new code for verifying that the git checkout on the NonStop system matches the repo state, instead of reusing git's existing code for comparing a working copy to the repo).

Oh FFS!

Posted Nov 24, 2024 16:44 UTC (Sun) by mirabilos (subscriber, #84359) [Link] (42 responses)

Rust is an inside job to kill FOSS.

It is getting close to succeeding to turning me off of FOSS, and I’ve been fucking *living* FOSS for about ⅔ of my life (and not limited to software either).

Oh FFS!

Posted Nov 24, 2024 17:12 UTC (Sun) by intelfx (subscriber, #130118) [Link] (5 responses)

> It is getting close to succeeding to turning me off of FOSS

If Rust is turning you off of FOSS, then you probably weren’t attracted to it in the first place. You were merely attracted to the status quo.

Oh FFS!

Posted Nov 24, 2024 17:48 UTC (Sun) by mirabilos (subscriber, #84359) [Link] (4 responses)

Nonsense.

(Meanwhile, the IRC channel where this popped up has more people expressing disbelief in this “cult”.)

Oh FFS!

Posted Nov 24, 2024 18:52 UTC (Sun) by intelfx (subscriber, #130118) [Link]

> Nonsense.

Your original posting did not carry much sense either.

> (Meanwhile, the IRC channel where this popped up has more people expressing disbelief in this “cult”.)

If you happen to dislike something, or dislike those who disagree with you, that does not make it a cult. So far you have provided zero rational reasons for your displeasure, which makes _your_ position much closer to a “cult” than the one you argue against.

Oh FFS!

Posted Nov 25, 2024 8:44 UTC (Mon) by Wol (subscriber, #4433) [Link] (2 responses)

But WHAT is turning you off FLOSS?

Is it Rust itself, or is it all the luddites throwing up the barricades against Rust?

(And for the record, from what I'm picking up, most of the key people aren't what we think of as luddites - although they may have a lot in common with the original Luddites who were skilled workers facing watching their skills become obsolete - and I think a lot of them would welcome it if it weren't for the high *personal* cost associated with it.)

Cheers,
Wol

Oh FFS!

Posted Nov 25, 2024 12:06 UTC (Mon) by pizza (subscriber, #46) [Link] (1 responses)

> Is it Rust itself, or is it all the luddites throwing up the barricades against Rust?

You left out "Or the obnoxious rust in everything and everything is magically better in rust!!!!!111" advocates?

Oh FFS!

Posted Nov 25, 2024 13:46 UTC (Mon) by Wol (subscriber, #4433) [Link]

Yep. There seems to be a lot of that.

But as I see it, most of the people who are actually USING rust seem to be the friendly "come in and join us" types.

And most of the people USING C are just "oh no not even MORE work that I'm expected to do without getting paid" types.

It's all the empty vessels that are the problem (that and the odd personality clash).

Cheers,
Wol

Oh FFS!

Posted Nov 24, 2024 18:35 UTC (Sun) by jrtc27 (subscriber, #107748) [Link] (35 responses)

All current Rust compilers, including the reference/official one, and the standard library are under FOSS licenses, and any Rust code you write can be under the exact same FOSS license as existing C or C++. As with any technology there are legitimate criticisms you can make of it, but "an inside job to kill FOSS" is just not in any way true.

Oh FFS!

Posted Nov 24, 2024 19:13 UTC (Sun) by viro (subscriber, #7872) [Link] (6 responses)

I can't speak for mirabilos, obviously, but my impression is that the problem is not with technology - it's with the fanboys. Evangelists tend to induce a gag reflex, regardless of the subject they cover with, er, sticky evidence of their devotion...

Oh FFS!

Posted Nov 24, 2024 19:49 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link] (4 responses)

> Evangelists tend to induce a gag reflex

Like, those Linux hippies?

Oh FFS!

Posted Nov 24, 2024 21:09 UTC (Sun) by viro (subscriber, #7872) [Link] (3 responses)

I don't know about hippies, but if you are talking about Linux advocates - yes, absolutely. Starting with the regular sight of comp.os.linux.advocacy threads cross-posted to hell and back, unfortunately including c.o.l.development.*, full of the holy warriors with not a single clue between them judging by the quality of information they'd been posting. Gag reflex - you bet.

FWIW, I carefully keep my impression regarding Rust separate from anything induced by Rust advocates/awareness raisers/evangelists/etc. It takes conscious efforts (revulsion tends to spread along the associations), but that's the only way to keep sanity, IME.

Oh FFS!

Posted Nov 25, 2024 6:54 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

I don't mind over-enthusiastic fanboys/fangirls (fanpersons?). They help to move the industry forwards, even if their enthusiasm is often misguided and/or naïve (cue the crypto folks). And sometimes they DO end up being right.

And hey, I do like a good flame once in a while.

Oh FFS!

Posted Nov 25, 2024 8:19 UTC (Mon) by josh (subscriber, #17465) [Link]

> FWIW, I carefully keep my impression regarding Rust separate from anything induced by Rust advocates/awareness raisers/evangelists/etc.

That's greatly appreciated. Rust folks don't like the RESF either, and have on multiple occasions said "please stop doing that, it isn't helping".

Oh FFS!

Posted Nov 25, 2024 18:10 UTC (Mon) by khim (subscriber, #9252) [Link]

> FWIW, I carefully keep my impression regarding Rust separate from anything induced by Rust advocates/awareness raisers/evangelists/etc. It takes conscious efforts (revulsion tends to spread along the associations), but that's the only way to keep sanity, IME.

I could only agree with that. What I hate most about fanboys/fangirls (fanpersons?) is their tendency to completely ignore failures of something try try to “promote” – yet the very moment when said failures are overcome they suddenly turn into a selling point.

I'm not capable of doing that. I hated certain decisions that Rust did and still hate some aspects of it, but they fixed it's deficiencies well enough that I have to admit that Rust is currently the best language for low-level work – even in spite of colored functions, lack of reflections, complicated metaprogramming and everything…

Is it perfect? Absolutely not. But exists, it works, it delivers. And I can live with it's deficiencies, even if just barely, in some cases.

But try to tell that to a fanboy/fangirl (fanperson?)! They would immediately tar you with feathers, declare that you don't know enough of the theory to judge that perfect creation, etc.

I, sometimes, feel that fanboys/fangirls (fanpersons?) could be the biggest imediment to the Rust adoption… but the cure is simple: just talk to actual people who develop Rust… these guys and gals very often would accept you complains and would explain why certain things are done in the way you hate… and you may even understand why this or that compromise was made.

Apropos the local Victorian Sufi Buddha

Posted Nov 24, 2024 20:07 UTC (Sun) by tux3 (subscriber, #101245) [Link]

Here's what I see when I try to post a comment:

>Enter your comment text below. Before posting, though, please consider:
>
> - Is your comment polite, respectful, and informative?
> - Are you saying something new that extends the conversation?
> - Have you provided a useful subject line?

I suppose different sites have different comment policy. Some places ask for kind, truthful, and necessary. The local version says polite, respectful, and informative.

Was it really necessary for people, on this Sunday evening, to be subject to this kind of tawdry comparison.

Oh FFS!

Posted Nov 24, 2024 21:27 UTC (Sun) by mirabilos (subscriber, #84359) [Link] (27 responses)

It’s great for corporate monoculture FOSS, I’m sure.

“Here’s the multi-million-LoC codebase which depends on another multi-million-LoC codebase (LLVM, see below), surely you have no problems using it, it’s open source after all!”

Too bad when you’re a hobbyist OS (not even mine, but I know the founder personally) that wants to add a small patch, which doesn’t even touch other platforms, to LLVM to add support for his OS, because if you’re in that situation you get rejected unless you shell over money in the ballpark of 300 $/month from your hobbyist budget so that LLVM can run CI on your OS. (And he was “lucky” enough that that would even have been possible and this cheap, as it’s on amd64; imagine wanting to write an OS for SPARC or MMIX or whatever that *doesn’t* have quick VMs).

Now go back to when Torvalds wrote Linux. Do you think it could have gotten off the ground having to meet such insane requirements?

And while not all hobbyist OSes bring as much to the table as Linux, evidently some quite do. (I’m seeing lots of Plan 9 influence recently, in rather many places as well, to name one.)

Sometimes, feasibility studies are being made as hobbyist OSes. I’m sure that both quite some of the performance improvements Linux got over the years as well as some of the security improvements were pioneered elsewhere first.

By keeping the core requirements to do realistically doable, fun, FOSS work small enough to be understandable (port *one* assembler and linker, port *one* C compiler, don’t even bother wigth C++ or threads or SMP), and the amount of ressources *required* for that low enough (sure could do GCC but some targets do better with something like pcc at first), creativity *outside* of amd64/arm64 Linux/Windows/OSX monoculture can happen.

Oh FFS!

Posted Nov 24, 2024 22:05 UTC (Sun) by viro (subscriber, #7872) [Link]

Hadn't Linux early development been done with cross-builds on Minix, anyway? No idea how many patches to gcc had been Linux-specific in the first few years; H.J.Lu might be the right person to ask - IIRC, he'd done quite a bit of toolchain-related work...

Cross-builds are less of an issue these days - getting a kernel off the ground on bare metal may be an interesting exercise, but if you are going to do something interesting with it, the sane approach is to use KVM/qemu/whatnot at least for early development, so there's already a host infrastructure within easy reach...

Oh FFS!

Posted Nov 25, 2024 0:29 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (25 responses)

> Too bad when you’re a hobbyist OS

Well, then you just carry the patch yourself. If you're writing a freaking OS, then what's a big deal with one more patch?

Also your whole argument is disingenuous. If you want your OS, then you need a C++ compiler for it because both LLVM and gcc need it. And you for sure are not going to write a C++ compiler yourself. Adding Rust to the mix doesn't change the amount of work required.

> Now go back to when Torvalds wrote Linux. Do you think it could have gotten off the ground having to meet such insane requirements?

Indeed, back then Torvalds and other developers had to port binutils, X11, emacs, vim, and other programs. It was WAY more complicated than today.

These days, you just implement a bunch of interfaces, and a lot of infrastructure just falls in place. It's simply ridiculous how easy it is to develop operating systems now: https://gitlab.redox-os.org/redox-os/redox/

Oh FFS!

Posted Nov 25, 2024 1:48 UTC (Mon) by mirabilos (subscriber, #84359) [Link] (8 responses)

Don’t assume every hobbyist OS is “just Linux” though. The interesting experiments deviate, and they are where the undue burden is applied worst.

And no, without Rust, you absolutely do NOT need CFrustFrust. There’s many C compilers out there that don’t need it, and all the usual infrastructure as well. (For my hobbyist OS, not to be confused with the one I was referencing above, I got rid of GNU groff as last core component needing it by replacing a better alternative.)

Porting Emacs is harsh (due to its absolutely insane and unportable internal design), but nobody needs to port Emacs to have an (one) OS.

And, having worked my way from the bottom to the top because _my_ conclusion comes from here: you obviously have no idea of the burden that carrying local patches (as opposed to upstreaming them) is, nor that all the upstreams generally want patches upstreamed as well. Your FOSS experience is definitely not broad enough. Also, this is not the first time I see your name in… questionable replies; therefore, I ask you to please DO NOT INTERACT with my posts any further.

Oh FFS!

Posted Nov 25, 2024 5:38 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (7 responses)

> And no, without Rust, you absolutely do NOT need CFrustFrust. There’s many C compilers out there that don’t need it, and all the usual infrastructure as well.

So let me recap. You want the entire field of software development to bend over backwards to accommodate hypothetical hobbyist OSes that don't have support for C++ (and so no FireFox, Chrome, etc.).

You do realize how ridiculous it is? Especially given that porting LLVM/gcc to a new platform is a task doable within a month of work.

Oh FFS!

Posted Nov 25, 2024 6:17 UTC (Mon) by viro (subscriber, #7872) [Link] (6 responses)

For fairness sake, for e.g. prototyping a research project, Chromium or Firepox are hardly a priority - at about the same level of interest as ffmpeg et.al.

Said that, llvm requirements from the host are pretty minimal and unless the target is _really_ weird, getting a backend for it wouldn't be harder than comparable-quality pcc targeted to the same thing, so that really shouldn't be a big deal.

Getting decent target-specific optimizations can be interesting, of course, but then the same would go for pcc or gcc...

What kind of local llvm patch had that been? I'm really curious; the cost of keeping a local branch can, of course, be unpleasant, but that depends upon the area being modified. Could you (mirabilos, that is) drop it someplace public and post a link (along with commit sha1 of llvm version it's been against)?

Oh FFS!

Posted Nov 25, 2024 6:20 UTC (Mon) by mirabilos (subscriber, #84359) [Link] (4 responses)

@cyberax: what part of D.N.I. do you not understand?

@viro: not my code, and I’m not in the mood of wading through an unfamiliar codebase, probably still on svn of all things, to look for it, sorry.

Oh FFS!

Posted Nov 25, 2024 6:50 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

Feel free to mute me if you don't want to see my posts.

Oh FFS!

Posted Nov 25, 2024 22:53 UTC (Mon) by mirabilos (subscriber, #84359) [Link] (1 responses)

There is no muting on LWN (for lynx and eMail).

What the fuck about DNI do you not understand? This is a basic consent thing.

Oh FFS!

Posted Nov 25, 2024 23:01 UTC (Mon) by corbet (editor, #1) [Link]

OK, cool it, please.

You posted a comment that can really only be seen as a troll. People fed the troll. You had no right to expect anything else, and have no right to tell others they cannot post on the site. Please just calm down and, perhaps, think a bit more before you post if you do not want people to disagree with you.

Oh FFS!

Posted Nov 25, 2024 7:22 UTC (Mon) by viro (subscriber, #7872) [Link]

llvm has moved to git 5 years ago, so presumably that patch is at least that old and I've enough software coproarchaeology on the kernel side...

Oh FFS!

Posted Nov 25, 2024 7:09 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

> Said that, llvm requirements from the host are pretty minimal and unless the target is _really_ weird, getting a backend for it wouldn't be harder than comparable-quality pcc targeted to the same thing, so that really shouldn't be a big deal.

I was trying to think of a realistic example, and the only one I can think of, is that specialized C compiler (romcc) in Coreboot that used only registers for its memory because it works before the DRAM itself is initialized.

Oh FFS!

Posted Nov 25, 2024 8:54 UTC (Mon) by Wol (subscriber, #4433) [Link] (15 responses)

> Also your whole argument is disingenuous. If you want your OS, then you need a C++ compiler for it because both LLVM and gcc need it. And you for sure are not going to write a C++ compiler yourself. Adding Rust to the mix doesn't change the amount of work required.

WTF?

You're assuming a new OS springs out fully formed, like that greek goddess who sprang full grown out of her father's head when someone chopped it open (can't remember which legend, sorry).

If you're writing an OS from scratch, you start small! Personally, I'd probably use Forth as the most appropriate starting language, but ...

When you're a mouse, you don't worry about the elephants. Wait until you've grown a bit!

Don't get into circular arguments. Who cares if LLVM and gcc need a C++ compiler because C++ needs LLVM or gcc. Just don't use C++! What do you *NEED* it for? YOU DON'T.

Cheers,
Wol

Oh FFS!

Posted Nov 25, 2024 12:14 UTC (Mon) by pizza (subscriber, #46) [Link] (3 responses)

> Don't get into circular arguments. Who cares if LLVM and gcc need a C++ compiler because C++ needs LLVM or gcc. Just don't use C++! What do you *NEED* it for? YOU DON'T.

In the short term, you're cross-compiling anyway.

But it's a lot simpler to just port the parts of GCC/LLVM necessary for cross-compilation of C (including a very minimal libc) than it is to _also_ port the inftastructure neeed for C++ (and Rust) including their respective standard/runtime libraries.

Once you have the basics going, then you can go back and port a full-featured libc, then move on to the infrastructure needed for C++, Rust, and so forth as needed.

Oh FFS!

Posted Nov 25, 2024 18:04 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

> But it's a lot simpler to just port the parts of GCC/LLVM necessary for cross-compilation of C (including a very minimal libc) than it is to _also_ port the inftastructure neeed for C++ (and Rust) including their respective standard/runtime libraries.

Oh, absolutely. Interestingly, adding Rust is _easier_ than C++. You don't need to worry about a lot of gnarly stuff like multiple inheritance, member pointers, and so on.

Oh FFS!

Posted Nov 30, 2024 14:35 UTC (Sat) by mathstuf (subscriber, #69389) [Link] (1 responses)

What platform specifics are there for multiple inheritance and member pointers? ABI decisions? I'd expect most to just import the Itanium ABI for such things which would be minimal consideration for porting. But if one wants to describe their own ABI…yes, Rust probably has fewer decision points than C++.

Oh FFS!

Posted Nov 30, 2024 21:01 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

> What platform specifics are there for multiple inheritance and member pointers?

Virtual table layouts, virtual tables during object construction, and so on. For member pointers: the way you represent pointers to virtual functions (via offsets in a vtable). Also various small things, like allocating storage for the array length so that delete[] can properly run destructors.

Nothing too complicated, but still a fair bit of work. And I assume that you don't want to just re-use the existing ABI for some reason, because it equally applies to Rust as well as C++.

Oh FFS!

Posted Nov 25, 2024 17:48 UTC (Mon) by wittenberg (subscriber, #4473) [Link] (1 responses)

Athena sprung fully grown from Zeuss's head

Oh FFS!

Posted Nov 25, 2024 18:16 UTC (Mon) by Wol (subscriber, #4433) [Link]

Thanks.

Cheers,
Wol

Oh FFS!

Posted Nov 25, 2024 22:57 UTC (Mon) by mirabilos (subscriber, #84359) [Link] (8 responses)

(The offturning thing is that Rust, a language too immature to even be specified, is effectively required for even basic touching a computer soon, apparently.)

Yes, precisely, starting small… but at the same time, once the initial bootstrapping is done, stay self-hosting, because only by dogfooding you can even run into all the bugs, so all the development will be moved to the new OS as soon as possible.

Oh FFS!

Posted Nov 25, 2024 23:24 UTC (Mon) by Wol (subscriber, #4433) [Link] (5 responses)

> (The offturning thing is that Rust, a language too immature to even be specified, is effectively required for even basic touching a computer soon, apparently.)

And?

To quote Ecclesiastes, "there is nothing new under the sun" ...

That was true of Basic, C, PL/1, etc etc etc.

(The OS I cut my teeth on was originally written in FORTRAN, and slowly got transmogrified into PL/1, PLP, SPL, C, ...)

And hasn't Rust been specified for ages? Just because it's not ISO, or IEEE, or "insert your quango here" of the day ... you just say "this code needs edition 2018", and there's your spec.

Cheers,
Wol

Oh FFS!

Posted Nov 25, 2024 23:51 UTC (Mon) by pizza (subscriber, #46) [Link]

>And hasn't Rust been specified for ages?

The specification is "how [the latest version of] the official (and only) implementation does it"

Oh FFS!

Posted Nov 26, 2024 0:18 UTC (Tue) by mirabilos (subscriber, #84359) [Link] (2 responses)

AIUI it has not been specified other than “what rustc does”, which is a problem for the gccrs people.

And it wouldn’t be that bad if you could say edition 2018, and there’s an edition or two a decade.

Not multiple editions a day.

Oh FFS!

Posted Dec 13, 2024 19:51 UTC (Fri) by sammythesnake (guest, #17693) [Link] (1 responses)

They release new code pretty frequently, but in Rust land, an "Edition" is a specific thing[1] released very much less frequently[2] and which is stable and supported in all future versions of the compiler - no need to change any code (though the necessary changes can be made by automated handles

There are necessarily caveats to do with bugs and unavoidable backward incompatibility to handle security issues or whatever, but the only way to guarantee avoiding those is never release any new code at all...

[1] https://doc.rust-lang.org/edition-guide/editions/

[2] 2015, 2018, 2021 and 2024 so far, that's one every ~1100 days on average, so the orders of magnitude less often than you suggested.

Oh FFS!

Posted Dec 14, 2024 7:57 UTC (Sat) by sammythesnake (guest, #17693) [Link]

Sorry, fat-fingered edit there, the bracketed sentence "though the necessary changes can be made by automated handles" should have said "though the necessary changes _to support a new edition_ can be made by automated converters"

Oh FFS!

Posted Nov 26, 2024 11:12 UTC (Tue) by khim (subscriber, #9252) [Link]

> And hasn't Rust been specified for ages?

Nope. Rust reference says that in very red rectangle: Warning: This book is incomplete. Documenting everything takes a while. See the GitHub issues for what is not documented in this book.

Ferrocene have specification. That one is good enough to present to various bureaucrats, but it's very imprecise.

> you just say "this code needs edition 2018", and there's your spec.

Rust doesn't work like that. I can use “Rust edition 2015” and yet still use facilities that weren't available in year 2023, easily.

Rust edition is for breaking changes, not for supporting frozen-in-time toolchains.

Rust community stance for the demand to provide support for ancient compilers in the various crates was always: “you invent artificial limitations, you support artificial limitations… could fork old versions of old crates or do whatever you want, that's not our problem”.

Oh FFS!

Posted Nov 30, 2024 15:38 UTC (Sat) by mathstuf (subscriber, #69389) [Link] (1 responses)

Someone go tell K&R that they should have specified C before using it for Unix back in 1968… Or anyone who used Python in some system-level required software before Jython/IronPython/etc. existed (or even today…what spec exists?) are committing similar offenses? Maybe so, but I argue that that's a way to also get nowhere as bootstrapping with waterfall development practices (as any such formal standardization turns out to be) sounds…painful and unproductive.

Oh FFS!

Posted Nov 30, 2024 20:19 UTC (Sat) by viro (subscriber, #7872) [Link]

Ugh... So many things wrong with that...

1) no C back in '68 (or Unix either, for that matter; they hadn't started that until '69, and back in early days B had been "we might want to use it someday" thing - C had started as an implementation language for one of the rewrites of B compiler several years later, with "looks like it can be used for a lot of things we wanted to use B for" coming shortly after that).

2) the _total_ size of v7 source ('79, with most of the early C changes already behind) is 3.4Mb (and not quite all of that C either - ~0.2Mb is assembler, and there's some data as well). Yes, by v7 time there'd already been quite a bit of C sources around (2BSD, for example), but nowhere near the size of Rust codebase these days. Impact of language changes does depend upon the amount of programs that need to be updated, so that comparison is more than slightly disingenuous.

3) while we are at it, "somebody go tell K & R" is somewhat in a poor taste - dmr died 13 years ago ;-/

Yes, grandparent posting by mirabilos reeks with advocacy, but that's not a reason to stoop down to the same.

Advocate: n. Pack animal, tends to produce a lot of noise, especially when two groups greet each other with projectile vomit and long loud screams. Can imitate speech better than parrots, but can't be used as pets due to guano problems.

A bit Ironic

Posted Nov 25, 2024 17:59 UTC (Mon) by wittenberg (subscriber, #4473) [Link] (45 responses)

Am I the only one who finds it ironic that the company objecting to increasing the security of its tools is one that specializes in highly reliable/secure systems?

A bit Ironic

Posted Nov 26, 2024 5:47 UTC (Tue) by mirabilos (subscriber, #84359) [Link] (44 responses)

Rust is not a pancake, it’s even been shown that, thinking the language safe, people make more of the other kinds of mistakes (which you still *can* do) when writing in those new so-called “safe” languages.

Additionally, research has shown that the benefit of those new languages, even for its proponents, is in new code only, not in replacing/rewriting/encroaching on old code, as old code generally has *those* bugs already fixed by now.

Finally, we all know how well rewriting codebasēs works… *sigh…*

A bit Ironic

Posted Nov 26, 2024 6:39 UTC (Tue) by mb (subscriber, #50428) [Link] (40 responses)

>it’s even been shown that, thinking the language safe, people make more of the other kinds of mistakes

Another bold claim that completely contradicts my experience.
Can you please back this up with some actual data? Who has shown that where?

A bit Ironic

Posted Nov 26, 2024 11:40 UTC (Tue) by khim (subscriber, #9252) [Link] (39 responses)

> Who has shown that where?

Just count number of CVEs per line of code for things written in “memory safe” languages like PHP or Ruby to number of CVEs per line of code in something like SQLite.

That the well-known phenomenon, but I'm not sure if it can be extended to Rust, specifically.

Most “memory safe” languages are not safe at all! They are only “safe” in a sense that no matter what you do your mistakes wouldn't affect the runtime system and bugs wouldn't cause these “nasal daemons” to fly.

It's as if you would have achieved safety in real world by putting everything in the rubber sack and ensuring that a hyperactive kid would be able to break arms and legs as many times as it want, all the pieces would be kept inside. Tracing GC is prime example of that attitude.

Rust approach to safety is different and I suspect it would produce different results.

Heck, Rust ditched tracing GC (it was in the initial plan, remember?) because it's completely useless if your goal is not to “keep all the mangled and torn organs in the sack in safety”, but to “ensure that kid would actually stay in one piece with unbroken arms and legs”!

CVEs/LOC not a great metric

Posted Nov 29, 2024 1:58 UTC (Fri) by gmatht (guest, #58961) [Link] (38 responses)

Java got lots of CVEs because people kept finding ways malicious code could break the memory safety model, whereas writing `free(x);free(x)` in C got nary a peep. Safe languages tend to also have less severe CVEs, where C would give the attack complete access to the system, a safe langauge is likely to panic leading only to a DOS vulnerability. Also I presume Rust is more concise than C. I am not sure how many CVEs/LOC Rust code has, but according to [1], two thirds of vulnerabilities in Rust are Memory or Concurrency vulnerabilities. This is a little bit meaningless without knowing their definition of Concurrency and Vulnerability, but since these are they types of things Rust tries to prevent in safe code, it is a hint that Rust isn't encouraging bugs elsewhere.

[1] https://dl.acm.org/doi/10.1145/3624738

CVEs/LOC not a great metric

Posted Nov 29, 2024 12:47 UTC (Fri) by khim (subscriber, #9252) [Link] (37 responses)

> Java got lots of CVEs because people kept finding ways malicious code could break the memory safety model, whereas writing `free(x);free(x)` in C got nary a peep.

Note that I haven't told anything about Java. Java tried (and utterly failed) to do something else: use in-language restrictions as part of security boundaries.

This just doesn't work and, e.g., Android (which takes security seriously) doesn't even try to use Java for that.

But all that excitement about “memory safe languages” lumps together languages like Python and Ruby, on one hand, and Ada and Rust, on the other hand. Just take a loon on the list in the government report: Python®, Java®, C#, Go, Delphi/Object Pascal, Swift®, Ruby™, Rust®, and Ada. Thanks god noone told them that the most popular language 1980th is also “memory-safe” (if you forget about PEEK/POKE, but then most other “memory safe” languages also have such “escape hatches”)!

> Safe languages tend to also have less severe CVEs

Have you forgot about xkcd 1200? In what way Log4Shell is “less severe” than buffer overflow in kernel? Sure, they don't give you the ability to install unawthorized driver, but only give you a way to stole passwords and your money…

> Also I presume Rust is more concise than C

Rust is almost as strict about safety and correctness as Ada. C# also comes close. Haskell is probably even safer than Rust. And, lo and behold: all these four languages have affine types today! Rust got them first, then C# added simplified (and restricted) version, then Ada picked themHaskell even have linear types!

But because the push, today, is not to move to “safe” langauges, but to “memory safe” languages… people move to Python, Ruby and PHP (also, technically, a memory safe language) – and if you look on CVE numbers for these… there that phenomenon of thinking the language safe, people make more of the other kinds of mistakes is very well observed.

> This is a little bit meaningless without knowing their definition of Concurrency and Vulnerability, but since these are they types of things Rust tries to prevent in safe code, it is a hint that Rust isn't encouraging bugs elsewhere.

Most CVEs in Rust ecosystem are for violation of the soundness pledge. IOW: they are similar to Java CVEs and cover cases where incorrect use of library code may allow one to break through unsafe boundary and make unsafe code misbehave.

Only Java ever tried to treat these as a security vulnerability – and only for Sun (later Oracle) produced code for Java Runtime (and privileged Java classes). Most Rust CVE bugs wouldn't even be considered bugs in most other languages (including Java, outside of runtime).

Rust is extremely serious about safety… which is, ironically, how it lost the tracing GC. Because having tracing GC in the language makes it easier to achieve “memory safety”, but harder to deal with other kinds of safety!

And there are research for how dependent typing can make things even safer in the future, in some kind of Rust successor… but these are at the experiments stage, only Wuffs is used in production, and it's not a general-purpose language.

P.S. And I absolutely agree that going to “memory safe” languages would be good first step. If someone would have tried to make all these JavaScript and Python coders to do a 100 feet high jump and switch to Haskell… there would have been riots and no one would have done anything. Making sure C and C++ wouldn't be used, though… that people can buy.

CVEs/LOC not a great metric

Posted Dec 2, 2024 19:43 UTC (Mon) by mathstuf (subscriber, #69389) [Link] (33 responses)

Not too far beyond "memory safety" one gets into things like "formalized logic" and such. We (as humans) typically don't spec software out enough to "prove" them correct in any "logically safe" sense. There's also the question of what "safe" means for the software on a missile or in some weapon system. It's certainly not safe for anyone within the collateral damage radius of the weapon (even assuming the intended target "deserves" it in some way…which is certainly debatable most of the time). Getting memory safety, to me, is mostly about getting the code to do what is intended while also guarding against unintended behaviors due to assumptions made within the computing environment at use (i.e., the abstract machine the language uses to represent its semantics that are then translated into actual hardware instructions in some equivalent way). Basically, it's about the code doing what it says on the tin rather than having to also guard against own-goals like "oh, your stack got smashed because someone wrote outside of an array and now any localized reasoning you had about the code is useless".

CVEs/LOC not a great metric

Posted Dec 2, 2024 20:37 UTC (Mon) by khim (subscriber, #9252) [Link] (32 responses)

> Not too far beyond "memory safety" one gets into things like "formalized logic" and such.

Yes, and no. It's true that originally, the first “memory safe” language, LISP was created to reason about things like "formalized logic" and such.

But the vast majority of popular “memory safe” languages today were born to go in the precisely opposite direction! They are not using garbage collector as part of the “formalized logic”, but for the exact opposite reason: to enable “flexibility” and “velocity”. To make sure one may write code without writing formal spec and thinking about what could would be doing too much.

IOW: “memory safety” is not used in these languages as a step on the path to the correct program, nope. It's the opposite: “memory safety”, achieved via tracing GC, is used as “padded walls” for the crazy programs!

Hey, we made it impossible to touch arbitrary memory and that means your program couldn't touch our runtime and that means it couldn't touch unallocated memory and now it no longer matters what crazy stuff you are doing on top of that – runtime system would keep you safe!

That's the idea behind the majority of “memory safe” languages in today's world. Not to make software more robust, but to make wildly unsafe code somewhat more predictable and easier to debug.

> Getting memory safety, to me, is mostly about getting the code to do what is intended

IOW: for you memory safety is one step on the path toward the program safety and correctness… and that's what some other language do with it, too: Haskell, Ada, Rust…

But in the majority of “memory safe” languages that exist these days that's not the primary (and often not even secondary) goal. “Flexibility” and “velocity” matters, not safety.

It's just so happened that to achieve “flexibility” and “velocity” (where developers combine random pieces of code without thinking and then test the result till they would be satisfied) “memory safety” is useful tool.

It's even true in Java, to a great extend: the majority of these “industrial” programmers simply couldn't write anything in a language with UB, where tests may pass then your “well-tested” code may blow up in your face without any good reason.

And you can only achieve so much “safety” in a language when the user of the language is not concerned about safety at all, but is just conducting genetic experiments and looks for the specimen that would look kinda-sorta-working.

> Basically, it's about the code doing what it says on the tin rather than having to also guard against own-goals like "oh, your stack got smashed because someone wrote outside of an array and now any localized reasoning you had about the code is useless".

Yeah, but “memory safety” is not guarding your program in most memory safe languages, today. It guards solely and exclusively runtime, it doesn't even try to protect your program.

Consider the latest and greatest one, Go:

func appendAndChange(numbers []int) {
	newNumbers := append(numbers, 42)
	newNumbers[0] = 666
	fmt.Println("inside", newNumbers)
}

func main() {
	slice := []int{1, 2, 3}
	fmt.Println("before", slice)
	appendAndChange(slice)
	fmt.Println("after ", slice)

	fmt.Println("original slice is intact")
	fmt.Println("------")

	slice = append(slice, 5)
	fmt.Println("before", slice)
	appendAndChange(slice)
	fmt.Println("after ", slice)
	fmt.Println("original slice is modified")
}
The output of the program above would be:
before [1 2 3]
inside [666 2 3 42]
after  [1 2 3]
original slice is intact
------
before [1 2 3 5]
inside [666 2 3 5 42]
after  [666 2 3 5]
original slice is modified

Can you explain why first slice wasn't modified, but second slice was? How the whole thing works? Is it even guaranteed to work?

The answer is “no”, and “you are using it wrong”. And that's “memory safe” language! The latest one!

The only difference between C/C++ and these “memory safe” languages is in the fact that error in your code may generate “nasal daemons” in C/C++ but in “memory safe” languages behavior is restricted, predictable and thus debuggable.

If you care about safety and correctness then sooner or later your language would get affine types, maybe linear types, and, at some point, dependent typing is not out of the question. You may even drop tracing GC at some point, because affine and linear types work without it!

Like Rust did.

But if you are using “memory safety” as a gateway to “flexibility” and “velocity” then sure, you can protect yourself against some common C/C++ bugs, but that doesn't mean that your could wouldn't include other, even easier to exploit, bugs!

After all, if you investigate stories about how people break into various web servers… 90% of time (if not 99% of time) they don't bother to use buffer overflow in these huge and complicated parts (like Linux kernel, PostreSQL or NGINX), but simply find exploitable vulnerability in relatively simple and small scripts that people are creating in these “memory safe” languages like PHP or JavaScript!

CVEs/LOC not a great metric

Posted Dec 2, 2024 23:53 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (26 responses)

> The answer is “no”, and “you are using it wrong”. And that's “memory safe” language! The latest one!

It's actually easy to explain: the array reserves space exponentially on appends. There is nothing surprising here. You can't use any of this behavior to violate memory safety in Go.

Although you can do that if you have multiple threads, Go doesn't guarantee the memory safety in the presence of multithreading. FWIW, Java actually does guarantee it.

CVEs/LOC not a great metric

Posted Dec 3, 2024 0:24 UTC (Tue) by khim (subscriber, #9252) [Link] (12 responses)

> It's actually easy to explain: the array reserves space exponentially on appends.

And how could I predict the effect of that behavior? When would my “outer” variable be corrupted and when would it survive intact?

> There is nothing surprising here.

Seriously? This variable is maybe mutable or maybe not mutable and maybe would return modified result or maybe not return it… now that's called “nothing surprising here”?

> You can't use any of this behavior to violate memory safety in Go.

And that's precisely what I meant when I mentioned that xkcd 1200 strip.

Can such “easy to explain” behavior lead to corruption of user data? Yes, sure. Lots. Can it be used to bypass security checks? Of course. Lots. Can it be used to cause “memory unsafety”? No, no, that's sacred, of course not!

What good does “memory safety” buys me if my security locks are broken, passwords and databases are stolen and data is replaced with some garbage? What's the point of it?

> FWIW, Java actually does guarantee it.

Yeah, C# and Java are probably the most sane of popular “memory safe” languages. They even try to actually provide safety in a sense of getting the code to do what is intended to do, honestly! Not much and they mostly steal features from Rust, Haskell or ML and adapt them in half-backed form… but they at least try to care about safety is a sense of getting the code to do what is intended to do.

Go, JavaScript, PHP, Python, Ruby? Nope, or at least they are not really serious about it. They try to add some optional, opt-in features in recent versions… but security doesn't work if it's opt-in. It only works if it's opt-out. Only in that case you can actually guarantee something.

That's the biggest issues with TypeScript: it contains decent safety guards… but they all can be easily broken because of compatibility features – that have to be there or else it wouldn't be useful in the world where majority of web developers use JavaScript and not TypeScript.

P.S. Nonetheless push to adopt memory safe languages is a good first step. It doesn't buys us too much and a lot of people who are using horribly unsafe (yet genuinely “memory safe” languages) would be feeling smug for no good reason… but that's definitely only first step. Heck, think about it: Microsoft BASIC from almost half-century ago with no subroitines, only global variables and only two letters for globals… technically it's “memory safe” if you don't use PEEK/POKE. Would you want to rewrite something like Linux kernel in it? Would it become more robust after such rewrite? If no – then why no? It's “memory safe” language, it should be better than C! But at least this tells us that “you are holding it wrong” is not the sole reason for fact that our programs are, essentially, a Swiss cheese form the security POV. And castigating developers for the mistakes they sometimes do is not a constructive way to achieve safety and security. Perhaps 10 or 20 years down the road people would warm up to the idea that not all “memory safe” languages are born equal… but first we have to step up from C/C++… like we stepped up from assembler 30 (or 40?) years ago.

CVEs/LOC not a great metric

Posted Dec 3, 2024 1:50 UTC (Tue) by mirabilos (subscriber, #84359) [Link]

Great points! Especially thanks for the example in issue9… that would totally trip me up if I had to peek at some code written in it without having learnt the language inside out (which I haven’t).

I also considered BASIC and scripting languages, funnily enough.

CVEs/LOC not a great metric

Posted Dec 3, 2024 20:35 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (10 responses)

> And how could I predict the effect of that behavior? When would my “outer” variable be corrupted and when would it survive intact?

How do you remember not to use index "8" if you delete 10 elements from an array with the size of 11?

It's literally a logical bug. Don't depend on the underlying storage length, unless you know what you're doing.

And no, it's not a memory safety issue. At worst, you'll get a bounds check panic.

CVEs/LOC not a great metric

Posted Dec 3, 2024 21:08 UTC (Tue) by khim (subscriber, #9252) [Link] (8 responses)

> It's literally a logical bug. Don't depend on the underlying storage length, unless you know what you're doing.

Isn't that the exact same mantra that C and, later, C++ programmers repeated for last half century: don't write programs with bugs – and everything would be all right?

Note that bug is easy to spot here only because I have explicitly called function appendAndChange. If it would have been called in a fashion that doesn't attract attention to the fact that it may change content of the slice that is passed into it – then it could be used for a long time before someone would notice that it corrupts some data it shouldn't be corrupting.

Because it usually doesn't change array into it, if it's created in static fashion, and even simple and naïve test wouldn't reveal any issues.

You have to prepare your slice in a way that is both unnatural for the testing environment and common for the actual code that works with dynamic arrays.

Perfect combo to lead to long hours of debugging if you don't know precisely undocumented rules about how Go slices work. Almost as dangerous as Python's desire for “simplicity” and arrays used as default arguments (although there the behavior is documented, thus a tiny bit less dangerous).

> At worst, you'll get a bounds check panic.

Nope. That's the best outcome. And very much not guaranteed outcome. Normal, typical, most common outcome would be silent data corruption somewhere.

Because, again, it's only called appendAndChange here to make it easier to understand the issue.

In real-world code that would be some function that is not supposed to change it's argument and wouldn't change it in tests… but would change it in production code deployed in some far away place.

Yes, that's not a memory safety issue, it's just almost indistinguishable from the “memory unsafety”.

And wouldn't exist in any “complicated” language like C# or Java (with String and StringBuilder), Rust or even C++!

Yet “modern” Go language in it's quest “for the simplicty” does the exact same mistake that C did half-century ago.

But C have an excuse: it was designed in on computer with 16KiB of RAM, it couldn't add many complicated niceties… what's the Go excuse?

CVEs/LOC not a great metric

Posted Dec 4, 2024 0:33 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (7 responses)

> Isn't that the exact same mantra that C and, later, C++ programmers repeated for last half century: don't write programs with bugs – and everything would be all right?

Pretty much. Go's behavior is not something unusual, you're (possibly) mutating an object, and then depending on the side effects of this mutation. It's literally from the "don't do this" category.

Go only guarantees that your program won't cause memory safety issues.

CVEs/LOC not a great metric

Posted Dec 4, 2024 1:18 UTC (Wed) by khim (subscriber, #9252) [Link]

> Go only guarantees that your program won't cause memory safety issues.

And then, only if there are no race conditions. Which is precisely my point: the goal of Go was never

> It's literally from the "don't do this" category.

Sure, but how would I even know if I'm supposed to do that or not?

Note, that:

  1. Go doesn't offer any way to pass a read-only slice around (like not just C++, but even plain C could do)
  2. Go doesn't offer separate types for the “view slice” and “owned array” (C also conflates them, but C++ handles them separately)
  3. “Ideomatic” Go conflates many other different things (e.g. set is supposed to be handled via hashmap) for the “simplicity”
> Go's behavior is not something unusual

Maybe, but Go's creator's behavior is unusual: it's as if they deliberately combined all the worst sides of programming language design from the last century and created something that could fight with PHP and JavaScript for the title of “the most dangerous popular memory-safe language”.

Essentially the only thing that they added to not be able to take the prize in that contest is static typing (and then pretty weak one). This, indeed, pushed them somewhat from the winning position… but other than that… all the tried and failed ideas are picked up and implemented.

If that were the language like BASIC, only designed to support newbies who couldn't grasp all the concepts that “serious languages” are using – that would have been justified… or if that was supposed to be only used for a small scripts, like Python… maybe.

But Go is not positioned like this! It pretends to be usable for large-scale projects!

And there are even some people who are using it like that…

I guess it's justified by that “strange phenomenon” that even their creators noticed: Although we expected C++ programmers to see Go as an alternative, instead most Go programmers come from languages like Python and Ruby. Very few come from C++. Also note how it took us over a year to figure out arrays and slices – which means that abomination that we are discussing here is not some sort of omission, but something that their creators are proud of! That's just simply… unbelievable.

But still… now we are conflating all languages except for C/C++ in one huge lump of “memory safe” languages – and that's simply wrong.

Not all “memory safe” languages are equally safe… but given the fact that around 70% of bugs in C/C++ programs are memory safety bugs… we should consider themselves lucky if people would switch from C/C++ to Go, JavaScript and Python… although I hope languages like Ada and Rust would good some love, too.

CVEs/LOC not a great metric

Posted Dec 4, 2024 13:07 UTC (Wed) by paulj (subscriber, #341) [Link] (5 responses)

This particular "side effect" looks very confusing. Not a Go expert, but this kind of side-effect looks like a huge land-mine, that could be easy to trigger unwittingly.

CVEs/LOC not a great metric

Posted Dec 4, 2024 20:01 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (4 responses)

In the actual Go code this type of code is _very_ unusual. The typical pattern is `arr = append(arr, items...)`, so it's clear that you're mutating the array.

CVEs/LOC not a great metric

Posted Dec 4, 2024 20:32 UTC (Wed) by khim (subscriber, #9252) [Link] (3 responses)

And does it help? If I replace newNubers with numbers problem remains, of course.

The issue in that code is not that append is used incorrectly, but in the fact that append-using function is used incorrectly.

Which is precisely and exactly the same class of bugs that make C/C++ programs fail, most of the time!

You may as well say that malloc should be paired with free and anyone who doesn't do that is “holding it wrong”.

Whether it's a hashmap that someone incorrectly modifies, or changing a struct which is not supposed to be changed… it's the exact same pattern: something is changed when it shouldn't be changed, or, alternatively, something is not changed when it should be changed.

The latter case couldn't be handled by the language efficiently (that's firmly #1 in farnz classification), but the former could be handled, in many cases. And most serious languages before Go (like C/C++, Java, C#, etc) had a way to handled that to some degree (approaches are different, const in C doesn't work like final in Java and C# in/ref readonly have different scope from both of them), but only “improved C” language developed in XXI century decided that “you are holding it wrong” is the best answer to these concerns.

Sure, the languages that Go actually replaces (JavaScript, Python or Ruby) had no such mechanisms either… which partially explains why C++ developers don't flock to Go and most developers switch to Go from dynamically-typed languages. But even they are slowly growing them (JavaScript have const these days)!

But still it's kind of funny that Go developers were telling to themselves that they are doing a better alternative for C++, not better alternative for Python…

CVEs/LOC not a great metric

Posted Dec 4, 2024 21:18 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

> And does it help? If I replace newNubers with numbers problem remains, of course.

Yes, it helps. Because it's clear that you're mutating a passed-by-value object.

CVEs/LOC not a great metric

Posted Dec 5, 2024 11:58 UTC (Thu) by khim (subscriber, #9252) [Link]

I guess it helps a tiny bit (it's similar to the level of protection JavaScript const or Java's final provides), but if you are to use it like that, 99% of time, then why in 15 years of development nothing that doesn't force me to type the same thing twice was provided?

Note: I'm not saying that Go is worse than C (although it's, most likely, worse than C++), I'm just saying that for majority of the “memory safe” languages said “memory safety” doesn't make programs safer or more robust to a degree that people expect.

They add memory safety not to make them safer or more reliable, but, in 90% of cases, to make them “simpler”, or “more flexible”, etc.

And if you eliminate 70% of bugs but make remaining 30% of bugs 2-3 times easier to make… the net result is still positive, but it's not as large as many expect.

CVEs/LOC not a great metric

Posted Dec 5, 2024 0:07 UTC (Thu) by intgr (subscriber, #39733) [Link]

> JavaScript have const these days

TBF JavaScript's const is almost useless. It only prevents you from re-assigning the variable in the scope.

It does nothing to prevent mutation or side-effects: when the *value* referenced by variables is inadvertently mutated.

You can have a const Array variable and still mutate it many times in a function, pass it to another function that mutates the array etc.

Classes of logic bug

Posted Dec 4, 2024 11:37 UTC (Wed) by farnz (subscriber, #17727) [Link]

Part of the fun here is that there's (at least) two classes of logic bug in play here:

  1. Failure to correctly transpose constraints from the problem domain into the code. For example, if a route planner has you arrive at the airport after the gates for your flight have closed, but before the flight's scheduled departure.
  2. Not respecting rules of the language that aren't compiler-enforced. For example, iterator invalidation when you delete items.

It's unreasonable to expect a language to prevent the first sort of logic bug completely (although it can give you the tools to write libraries that prevent them). But it's not unreasonable to expect that languages aim to reduce the second category of logic bugs to as close to zero as possible, and to at least require you to do something a bit strange in order to run into them (e.g. unusual syntax, or special marker keywords).

CVEs/LOC not a great metric

Posted Dec 3, 2024 10:53 UTC (Tue) by paulj (subscriber, #341) [Link] (12 responses)

> It's actually easy to explain: the array reserves space exponentially on appends.

Can you expand on that for non-Go people? The slice that append returns is a different type of slice to the original []int slice?

CVEs/LOC not a great metric

Posted Dec 3, 2024 11:05 UTC (Tue) by mathstuf (subscriber, #69389) [Link] (11 responses)

It looks like bad copy-on-write semantics to me. The first call's `append` has to allocate new memory, so `newNumbers[0]` is completely separate from the input array's backing memory. The second call uses the existing allocation (the extra space the first call to `append` reserved) and writes "see through" to the passed array and modifications affect it. Feels kind of like the Python behavior where `def foo(arr=[]): arr.append(1)` ends up modifying the default argument's backing store if `arr` is not passed.

CVEs/LOC not a great metric

Posted Dec 3, 2024 12:40 UTC (Tue) by khim (subscriber, #9252) [Link] (8 responses)

Correct. And the most amusing issue there is the reaction to that problem from language developers: just don't do that.

Come on, guys! If just don't do that (and write your code without bugs, dummy) would have been a good, actionable, answer then that push for the “memory safe” languages wouldn't have happened in the first place!

If you write code without mistakes then you can write correct, working, program in C, C++ or even assembler: D. J. Bernstein did that (or almost did that depending on you are asking), this approach works!

But it doesn't scale – and that raises the question: why should we accept such answer in “memory safe” languages… we have just abandoned “memory unsafe” precisely because that's bad answer!

P.S. This doesn't mean that Ada or Rust never say just don't do that. Sometimes it's just simply unavoidable. E.g. if you hash functions doesn't always return the same value for the same object (which may happen if you use address of some objects in hash and then object is moved) then Rust's HashSet is not guaranteed to find element in that set. Kind of unfortunate, but hard to offer better alternative. But to design language in XXI century and conflate dynamic array and slice “for simplicity”? Really? Sure, you “simplified” the language… by shoving all that complexity into my head… how is that a good trade-off? I like “simple” languages as much as the next guy – till that “simplicity” is not bought by shoving the intrinsic complexity into my head! And most “memory safe” languages that are popular today are like that: they are “simple” in a sense that their reference guide is short – but then all that complexity have to live in the head of the language user… because where else could it go?

CVEs/LOC not a great metric

Posted Dec 3, 2024 13:05 UTC (Tue) by Wol (subscriber, #4433) [Link] (7 responses)

> Really? Sure, you “simplified” the language… by shoving all that complexity into my head… how is that a good trade-off?

Einstein: "Make things as simple as possible - But No Simpler!"

If you make *PART* of the problem space simpler, by pushing the complexity elsewhere (where it doesn't belong), you make the overall solution more complex.

(Don't get me started on Relational and SQL :-)

Cheers,
Wol

CVEs/LOC not a great metric

Posted Dec 3, 2024 13:20 UTC (Tue) by khim (subscriber, #9252) [Link] (6 responses)

> (Don't get me started on Relational and SQL :-)

Lol. That's actually surprisingly relevant (if sad) example. Sure, SQL is horrible paradigm, but would anything else have a chance in a world where so many people know SQL and so few know anything else?

That push for safer, more robust programs (and push for “memory safe” langauges) have started not because what Rust offers is better than what C++ offers, no.

It have only started when losses started being measured in trillions.

Does the [ab]use of SQL have a chance to lead to losses of that magnitude? Does it cause anything but 10x (or maybe 100x?) excess resource consumption in some rare situations?

I don't know, in all the applications where I use SQL 100x speedup of database access would affect the work of the whole system only very marginally.

CVEs/LOC not a great metric

Posted Dec 3, 2024 14:14 UTC (Tue) by Wol (subscriber, #4433) [Link] (5 responses)

> I don't know, in all the applications where I use SQL 100x speedup of database access would affect the work of the whole system only very marginally.

And if it sped up DEVELOPMENT time by the same one or two orders of magnitude?

One of my war stories (that I'm not sure if it rings true ... the database side rings true, it's the rest of it that doesn't) the Australian breweries when Australia introduced GST.

Of the six breweries, five ran a relational-based accounts system. One of these realised that by leasing barrels to the pubs rather than selling and buying back, they could save a lot of tax. They updated their systems and caught the other five breweries on the hop.

The other four relational breweries took six months to catch up - and lost a lot of market share. Made worse because the sixth brewery modified their Pick-based system in two months - so for the last four months they had TWO breweries taking market share because they were paying less tax...

(Oh - and where seconds count, like 999-response systems (that's 112 or 911 to non-brits), would a "results almost before I hit "return" " response time make much difference? :-)

Cheers,
Wol

CVEs/LOC not a great metric

Posted Dec 3, 2024 16:02 UTC (Tue) by khim (subscriber, #9252) [Link] (4 responses)

> that I'm not sure if it rings true ... the database side rings true, it's the rest of it that doesn't

But it's the rest of it that matters.

> The other four relational breweries took six months to catch up - and lost a lot of market share. Made worse because the sixth brewery modified their Pick-based system in two months - so for the last four months they had TWO breweries taking market share because they were paying less tax...

And the whole thing just falls to pieces at this point. My sister is an accountant and she faced cases like these few times. Usually it just takes two or three days to cobble together something working using some Excel sheets and maybe VBA scripts. And accounting people just do papers by hand. Then later, when the need is not as pressing one makes it automatic.

But sure that only works for a small companies that are not under extra-strict pressure… but these companies that do have the need to do everything in a “certified” fashion… who the hell allowed them to use Pick in place of SAP?

In my experience if something takes not days, but months or years that's not the result of deficiency of SQL or something like that, but the need to jump through lots of paper barriers – and I fail to see how Pick can improve anything there.

> Oh - and where seconds count, like 999-response systems (that's 112 or 911 to non-brits), would a "results almost before I hit "return" " response time make much difference? :-)

Most likely no. Simply because no one would even allow you to use anything without pile of certification papers… that Pick-based system wouldn't have (and most SQL-based systems wouldn't have either).

In the end some horribly inefficient, but certified solution would be picked, no matter what you do.

CVEs/LOC not a great metric

Posted Dec 3, 2024 17:03 UTC (Tue) by Wol (subscriber, #4433) [Link] (2 responses)

> In my experience if something takes not days, but months or years that's not the result of deficiency of SQL or something like that, but the need to jump through lots of paper barriers – and I fail to see how Pick can improve anything there.

In my experience (and I'm currently living the hell ...) writing one single query in SQL can take days. The same query in Pick can be knocked up, tested, and verified! in five minutes! (The query I'm currently trying to debug, and falling over corner cases, and discovering that I'm querying data that doesn't exist or is stored somewhere else, and and and - has taken me months and I've just had a message saying it's screwed up AGAIN! I think I know why, some attribute I don't give a monkeys about has an unexpected value ... :-(

> In the end some horribly inefficient, but certified solution would be picked, no matter what you do.

True :-(

So basically, you're saying that 90% of the work is not programming, but administration ... but there's a lot of places where that isn't true. I work for an (ex)FTSE100, and I'm trying to introduce Pick to get rid of our Excel database. But yes, I know about paperwork - it's supposedly dead easy to introduce "new" technology, there's just a whole bunch of hoops I don't have a clue how to navigate :-( Once it's in, it won't be a problem any more ...

Cheers,
Wol

CVEs/LOC not a great metric

Posted Dec 3, 2024 17:24 UTC (Tue) by daroc (editor, #160859) [Link] (1 responses)

I think we've wandered pretty far off topic, at this point. Let's leave this discussion here.

CVEs/LOC not a great metric

Posted Dec 4, 2024 13:04 UTC (Wed) by paulj (subscriber, #341) [Link]

Wol was doing so well the last while... He's actually managed to go quite a while recently without bringing up Pick. This is his first relapse of his pickaholicism in a while, TTBOM. Maybe he can last a bit longer again from this point forward. ;)

CVEs/LOC not a great metric

Posted Dec 3, 2024 17:08 UTC (Tue) by Wol (subscriber, #4433) [Link]

> But sure that only works for a small companies that are not under extra-strict pressure… but these companies that do have the need to do everything in a “certified” fashion… who the hell allowed them to use Pick in place of SAP?

The accountants? Because chances are, the accounting system was written by accountants who knew what an accounting system was supposed to do, and not by programmers who filled it with bugs because they didn't have a clue?

(Okay, this IS a seriously double-edged sword!)

Cheers,
Wol

CVEs/LOC not a great metric

Posted Dec 3, 2024 18:09 UTC (Tue) by mirabilos (subscriber, #84359) [Link] (1 responses)

Except that the Python behaviour is consistent (and documented, not sure about the issue9 behaviour wrt. that).

CVEs/LOC not a great metric

Posted Dec 3, 2024 21:16 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

Yes, there are definitely lints that warn about such things in Python. My caveman club-level of finesse with Golang is usually just happy to get things to work (though I've not dealt with its slices enough to know how any linters behave with them).

CVEs/LOC not a great metric

Posted Dec 3, 2024 13:06 UTC (Tue) by kleptog (subscriber, #1183) [Link] (4 responses)

> And you can only achieve so much “safety” in a language when the user of the language is not concerned about safety at all, but is just conducting genetic experiments and looks for the specimen that would look kinda-sorta-working.

The results of genetic programming is basically determined by the quality of your fitness function. A strict compiler, test cases and code review are all things that make genetic programming more effective. Only the first of those can't be skipped, you can't skip the compiler.

The test is: how close can your code come to *looking* like it's supposed to work while not actually being correct? Writing code in Rust that simultaneously looks correct, compiles but still does the wrong thing is actually tricky. This I believe is the saving grace of Python: the syntax is simple enough that that it almost always does what it looks like its doing, together with PEP8 and a particular coding culture makes it mostly work. The rampant use of monkey patching in Ruby drove me nuts (not sure if it's still like that) and makes things much less obvious.

So far there's only been one Underhanded Rust contest with only 4 entries. It would be great to see if people can improve on those submissions.

CVEs/LOC not a great metric

Posted Dec 3, 2024 14:05 UTC (Tue) by khim (subscriber, #9252) [Link]

> Writing code in Rust that simultaneously looks correct, compiles but still does the wrong thing is actually tricky.

I also believed that… till I saw what people who simply connect output of ChatGPT with the compiler are producing.

Believe me, if you concentrate on keeping your head entirely clean without any understanding of what you are doing… just feeding compiler advice into ChatGPT often leads you to something totally crazy that still compiles and runs… just doesn't work.

The trick is to not know anything about programming at all and not even thinking about the task that you are trying to solve.

Then you can create really powerful headache for the reviewers.

CVEs/LOC not a great metric

Posted Dec 3, 2024 14:19 UTC (Tue) by Wol (subscriber, #4433) [Link] (1 responses)

> The test is: how close can your code come to *looking* like it's supposed to work while not actually being correct?

This is actually a pretty neat description of the sexual arms race :-)

Male profligacy attracts females. How does a male APPEAR profligate, while not actually expending much in resources? How can a female be sure he is actually being profligate (because he can) and so would make a good partner?

As the above shows, too many compilers are bad sexual partners, because they're faking it ... :-)

Cheers,
Wol

Digression

Posted Dec 3, 2024 14:51 UTC (Tue) by corbet (editor, #1) [Link]

This seems like an entirely unnecessary, and perhaps misogynistic, digression. It would really be nicer if you could resist the temptation to do this sort of thing here. Please?

CVEs/LOC not a great metric

Posted Dec 3, 2024 15:26 UTC (Tue) by farnz (subscriber, #17727) [Link]

The test is a little more complex than that - you need the code to "look" like it's supposed to work, to compile, and to work in the cases the programmer tests by hand. Code that looks like it should work, that compiles, but doesn't work when tested by the programmer is code that gets reworked further until it does work for the programmer's test cases.

No language is completely resistant to this - anyone can make the code more complicated until it basically has N cases for the things the programmer tests by hand, and then a general case that doesn't work - but it does make it more likely that your fitness function is going to determine that this code is "too complicated" and reject it on that basis.

CVEs/LOC not a great metric

Posted Dec 15, 2024 8:00 UTC (Sun) by sammythesnake (guest, #17693) [Link] (2 responses)

> people move to [...] PHP (also, technically, a memory safe language) [...] thinking the language safe [...]

I don't think I've ever heard anyone describe PHP as safe, most people who even have a concept of "safe" wouldn't touch PHP with a 20ft pole!

CVEs/LOC not a great metric

Posted Dec 15, 2024 13:55 UTC (Sun) by pizza (subscriber, #46) [Link] (1 responses)

> I don't think I've ever heard anyone describe PHP as safe, most people who even have a concept of "safe" wouldn't touch PHP with a 20ft pole!

PHP has *always* been "safe".

Its poor reputation is due to numerous lackadaisical practices by folks developing with PHP -- primarily not sanitizing inputs and manually assembling database queries (and shell invocations) with those unsanitized inputs.

...The largest data breaches in history are due do this sort of application-level logic flaw... implemented in "safe" languages.

(Any language capable of concatenating strings together is "vulnerable" to this sort of thing)

CVEs/LOC not a great metric

Posted Dec 15, 2024 14:42 UTC (Sun) by mathstuf (subscriber, #69389) [Link]

Memory safety is different than functional safety. PHP has the former but its APIs were woefully lacking to make the latter easy. Now there are better APIs more readily available, but its history definitely casts a long shadow in many minds.

> ...The largest data breaches in history are due do this sort of application-level logic flaw... implemented in "safe" languages.

No one here (AFAIK) is claiming *functional* safety (life, limb, correctness) here[1]. Proof systems are needed for that and Rust isn't there (nor do I know of anyone claiming such). However, when one has memory safety, the foundations for building up things like "encode requirements into types and let the compiler make sure it is fine" are far easier. Of course, some languages don't have sufficient mechanisms to teach their compiler (or interpreter) about such things, memory safe (Python, PHP, Bash) or not (C).

> numerous lackadaisical practices by folks developing with PHP

The same can be said for just about any software dysfunction. No one is perfect, but the level of vigilance and attention required to prevent such issues differs greatly between languages (and is not monotonic across the "memory safe" spectrum either). I know I'd much rather the team I'm working with have a steadfast gatekeeper for problems in the compiler before CI before review before customers. The quality of each filter is different between languages, teams, and cultures but (IMNSHO), finer filters earlier in the sequence are generally worth quite a lot more.

[1] Philosophical question: what constitutes "functionally safe" for a missile?

A bit Ironic

Posted Nov 26, 2024 7:05 UTC (Tue) by mjg59 (subscriber, #23239) [Link]

I think you're referring to https://security.googleblog.com/2024/09/eliminating-memor... when you say "Additionally, research has shown that the benefit of those new languages, even for its proponents, is in new code only"? The takeaway from that research is that for existing projects we should stop writing any new code in memory-unsafe languages, and that concentrating on this is more useful than rewriting old code in the project. Notably, projects that follow this advice will still end up depending on compilers for memory-safe languages, so I'm not sure it supports your general argument - it's explicit that transitioning new development to memory-safe languages improved the overall security of Android by a significant degree.

A bit Ironic

Posted Dec 2, 2024 0:48 UTC (Mon) by eean (subscriber, #50420) [Link] (1 responses)

and we're talking about a development tool - do people really develop software for NonStop on NonStop? What is git doing on these systems?

But yeah I think financial regulators take a dim view of languages like C these days.

A bit Ironic

Posted Dec 2, 2024 1:03 UTC (Mon) by pizza (subscriber, #46) [Link]

> and we're talking about a development tool - do people really develop software for NonStop on NonStop? What is git doing on these systems?

Git is used for _far_ more than "development" purposes; it's routinely used for configuration management [1] and deployment purposes, for example.

[1] https://etckeeper.branchable.com/ among others.

Open Source Projects Are Not Your Supply Chain

Posted Dec 5, 2024 12:39 UTC (Thu) by kpfleming (subscriber, #23250) [Link]

Well... the title says it all.


Copyright © 2024, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds