LWN: Comments on "Cranelift code generation comes to Rust"

LLVM ist a mess

Curan — Thu, 11 Apr 2024 07:39:42 +0000

Yeah, the full switch to ACO is not yet there for radeonsi. And, given corporate constraints, might never really happen.

llvmpipe I do not mind. That is a very distinct driver and allows testing for extensions. That would be easy not to build in a production environment with hardware acceleration around. llvmpipe was also very important for virtual machines (these days less). But the core Khronos and/or hardware driver stuff with LLVM dependencies I really wish I could avoid.

LLVM ist a mess

daenzer — Thu, 11 Apr 2024 07:24:26 +0000

Also, radeonsi hasn't switched the default from LLVM to ACO yet, and it'll probably keep the LLVM backend for some time after that. E.g. AFAIK the radeonsi ACO backend doesn't work with the rusticl frontend yet.

And that's just one driver, Mesa uses LLVM in many other places, the elephant in the room being llvmpipe.

LLVM ist a mess

Curan — Mon, 25 Mar 2024 07:34:50 +0000

Yeah, it certainly looks that way. Though ACO was, AFAIR a decision driven purely by performance. ACO is just faster than the LLVM path.

LLVM ist a mess

donio — Sat, 23 Mar 2024 23:08:14 +0000

I think there might be a very slowly growing momentum of projects moving away from LLVM. Mesa's ACO shader compiler is the first example that comes to mind, I believe this was motivated primarily by compilation performance. Zig is also working a new backend that won't require LLVM.

Go has famously sidestepped this by doing its own thing (but that was relatively early days for LLVM too). They were criticized for it at the time but on the long run I think it was a wise decision.

Benchmark results

sdumitriu — Thu, 21 Mar 2024 18:32:49 +0000

I see, thanks for the link. It looks like each set consists of several tens of very simple rules of a similar nature, e.g. they all rewrite bit operations into equivalent forms that may or may not be more efficient.

Benchmark results

daroc — Thu, 21 Mar 2024 17:35:34 +0000

Unfortunately, there is no direct comparison between LLVM's optimization passes and Cranelift's ISLE files. By analogy, you might consider comparing the number of source files in a project. The number of source files does not actually tell you anything absolute about the complexity of a project, because a program could throw everything in one large source file or split it out into many small ones. But it is still a useful measure of complexity, because most reasonable programmers tend not to go to either of those extremes. Likewise, LLVM optimization passes and ISLE files are not equivalent, but they are the unit of organization which the developers of each project have chosen.

If you are curious, you can read Cranelift's optimizations here[0], and judge for yourself how similar one of their ISLE files is to one of LLVM's optimization passes.

[0]: https://github.com/bytecodealliance/wasmtime/tree/main/cr...

Benchmark results

sdumitriu — Thu, 21 Mar 2024 16:44:54 +0000

I don't get it... This achieved the incredible feat of speeding up builds from the unbearably high time of 37 seconds, down to a mere... 30 seconds. Based on the hype I was expecting something more like 3 seconds or less.

Unrelated to that, it's ambiguous when saying that Cranelift has "ten sets of related optimizations", compared to "96 optimization passes" for LLVM. What is a set? How big are those sets? Are these 10 sets each made of 10 different optimizations, or are there like 20 (or merely the 5 listed in the article) total optimizations, but grouped in 10 different ways?

LLVM ist a mess

khim — Thu, 21 Mar 2024 10:06:16 +0000

> At least make the minor versions work across the board all the time.

Why should they do that and was there such a promise ever mentioned on their web site?

> Still get the some fails there, when some piece of software bundles their own LLVM and the system has a different one.

Yes, LLVM is designed around the idea that it would be bundled with a frontend. If you have other ideas then it's your responsibility to support them.

> NB: LLVM/clang is the standard compiler for eg. Mac OS via XCode.

Yes. And that pair have stable outer interfaces AFAIK.

> So either there needs to be a big fat warning at the top of all of LLVM that says "don't use my for production, I am a test environment" or LLVM needs to play ball.

It does do that if you use it according to it's design.

For a long time LLVM wasn't even designed to be used as shared library, but at some point Apple decided to change that. And they did. Now it's easier to embed LLVM into external projects as a shared library, but there are still no promises beyond that.

If you want something more than that then it's your responsibility to offer such solution.

> I want that commitment from LLVM. I do not care what they do internally. But their interfaces need to be stable enough.

But who would do the work to ensure that? That's non-trivial amount of work and AFAIK no one ever volunteered.

Cranelift code generation comes to Rust

andreashappe — Thu, 21 Mar 2024 07:23:19 +0000

Just disable them:

> LWN subscribers at the "professional hacker" level and above can disable ads by going into the account management area and selecting "Customization."

LLVM ist a mess

Curan — Wed, 20 Mar 2024 22:44:45 +0000

>> if you need to break core concepts this often, you probably made a lot of mistakes in the past
>
> Nope. Linux kernel breaks internal interfaces pretty often, too. The only problem of LLVM is that it's advertised as something of a separate project while in reality it's only half of many projects.
>
> If all these projects would have lived in one repo and people would have changed everything in sync it wouldn't have even been visible.

I do not care about internal interfaces. I do care about what they break in their official interfaces. Especially in their C interface (which gets worse every release, it seams like, and forces you to use the even worse C++ interface).

And offering a libllvm/libclang means you have to take some responsibility. At least make the minor versions work across the board all the time. Still get the some fails there, when some piece of software bundles their own LLVM and the system has a different one.

>> a lot of LLVM feels like it is a test environment to try out new things for the compiler space
>
> Which is precisely what LLVM was designed for. Just open Wikipedia and read: LLVM was originally developed as a research infrastructure to investigate dynamic compilation techniques for static and dynamic programming languages. From what you are saying LLVM works and acts like it was designed to work and act so why is that an issue?

First of all – as others pointed out too – that is not what LLVM claims itself these days. (NB: LLVM/clang is the standard compiler for eg. Mac OS via XCode.) So either there needs to be a big fat warning at the top of all of LLVM that says "don't use my for production, I am a test environment" or LLVM needs to play ball.

>> but then it shouldn't be the basis of anything else
>
> Build “better basis for anything else”, isn't that the right solution? Maybe as LLVM fork or write from scratch.
>
> I was told in no-uncertain terms in somewhat tangetially related discussion just over there that you have zero right to complain since LLVM is free.

I have no issue with somebody attempting to build something better (no matter the language or the licensing model). What I do have an issue with is my stuff breaking because of some library. The glibc makes sure my oldest programs still work (even though there will be not much of a chance to get a new version of them for me). I want that commitment from LLVM. I do not care what they do internally. But their interfaces need to be stable enough.

LLVM ist a mess

Curan — Wed, 20 Mar 2024 22:24:53 +0000

Oh my goodness no. LLVM needs to decide what it wants to be, IMHO. If they are a compiler, then each version can do, what it likes – as long as it produces executable programs without any bugs due to its internals.

If they want to offer a libllvm (and all the other wonderful libraries like libclang), with a SONAME, that indicates it is stable, they need to get their act together. Or make it impossible (or at least a manual downstream patch) to do dynamic linking (which they do not). Apart from that: even static linking caused issues in the past. When I linked LLVM statically, I still ended up having issues with executed workloads, that brought their own (sometimes statically linked) versions of LLVM. I have to admit, that I never went for the deep dive into these issues, since I am not given time for that during my day job. But I do have to keep the workloads running. So far that meant either manually "fixing" the LLVM builds (I will not claim these to be universally applicable fixes) or pinning the deployed LLVM version until we could work something out with our various upstreams.

(Side note: building LLVM across platforms is not fun. LLVM constantly breaks one build or another. I am sure Sylvestre (who's behind apt.llvm.org) and others beyond myself could attest to that. So even your "just bundle it" solution falls flat on its face right away.)

LLVM ist a mess

ringerc — Wed, 20 Mar 2024 20:14:09 +0000

You can always make the course more interesting.

I had a comp sci course on concurrency proofs and theory. The tool they used for it sucked so I updated it from the ancient RH4 target it required and replaced the build system. Then fixed some bugs and memory issues. Improved the error messages and generally made the tool nicer to use.

LLVM ist a mess

Cyberax — Tue, 19 Mar 2024 18:11:29 +0000

> 2) every such project's build output was bit-for-bit reproducible (and if no one ever used different compiler flags or versions).

That's actually how Nix works (compiler flags and environment are also a part of the content's hash). Some newer languages like Go also have this baked into the module system.

LLVM ist a mess

paulj — Tue, 19 Mar 2024 12:28:02 +0000

Unless some header has even a slightly different length between one object file and another, and then every block has different contents, despite (say) 99+% of the content being the same.

LLVM ist a mess

intelfx — Tue, 19 Mar 2024 12:24:53 +0000

This could only have a _chance_ of working as intended if either 1) everyone was pinning and bundling from the same blessed set of binary artifacts, or 2) every such project's build output was bit-for-bit reproducible (and if no one ever used different compiler flags or versions).

I think it's safe to say that the real world is very far from achieving either scenario.

LLVM ist a mess

Cyberax — Tue, 19 Mar 2024 05:56:29 +0000

I actually promptly forgot it. I always just spam the code with braces to disambiguate anything that is more complex than 2*2+1.

LLVM ist a mess

adobriyan — Tue, 19 Mar 2024 05:40:38 +0000

And you'll learn operator precedence table for FREE!

LLVM ist a mess

buck — Tue, 19 Mar 2024 04:47:57 +0000

Sorry. Let me stand corrected by myself:

Quoth https://bellard.org/jslinux/news.html:

2020-07-05:

Added the Alpine Linux distribution. Many packages are included such as gcc, Clang, Python 2.7 and 3.8, Node.js, Ruby, PHP, ... The more adventurous (and patient) people can also try to run Wine or Firefox.
Added SSE2 support to the x86 emulator
Added dynamic resizing of the terminal

LLVM ist a mess

buck — Tue, 19 Mar 2024 04:42:09 +0000

> I was just pointing out the humour in the irony of making that point via an example written by an author who has a (prodigious) habit of solving difficult problems. ;)

Well, I'm with you: a compiler written by Fabrice Bellard is not your run-of-the-mill hobby project.

But, I'm really more just pointing out that this still blows me away:

https://bellard.org/jslinux/index.html

which i think saves this comment from being (rightly) criticized for being OT, since it's more Linux-y than the article, and this is LWN after all, dang it.

Well, to get this right back on topic, i can actually just point out that JSLinux features tcc and gcc but not clang:

localhost:~# cat readme.txt
Some tests:

- Compile hello.c with gcc (or tcc):

gcc hello.c -o hello
./hello

- Run QuickJS:

qjs hello.js

- Run python:

python3 bench.py
localhost:~#

b/c the performance win:

[`time gcc -o hello -c hello.c -O0` output elided to spare my old laptop's feelings]

LLVM ist a mess

NYKevin — Mon, 18 Mar 2024 23:49:41 +0000

Meh, just use block-layer deduplication and/or a content-addressed filesystem, and then all of the logical duplication works out to relatively little physical duplication.

LLVM ist a mess

NYKevin — Mon, 18 Mar 2024 23:48:17 +0000

> I too, like most students who've done a CS degree, have had to write some kind compiler for a class assignment (a toy front-end with type-checker for a subset of a typed JS-ish language).

Fun fact: If your professor is sufficiently insane, it is possible that you will end up having to write an interpreter for the (untyped) lambda calculus. So count yourself lucky that you got a language that actually looked vaguely modern.

OTOH, I must admit that the lambda calculus is much, *much* easier to implement than most real languages. It only has 2½ rules, or 1½ if you use De Bruijn indexing. But I would've liked to do a real language, or at least something resembling a real language. I often feel that the most difficult courses were the only ones that actually taught me anything useful.

Much faster!

proski — Mon, 18 Mar 2024 21:01:16 +0000

I tried to compile alacritty (a terminal program) using Cranelift. It compiles much faster. It really makes the difference for the development, when I want to test my changes quickly.

The resulting binary would panic on startup. The backtrace points to some unsafe code in `fontconfig-rs`. It could actually be a bug in that code. It's nice to have another tool that can find issues in unsafe code. Miri is a tool specifically for that purpose, but it has too many limitations.

LLVM ist a mess

Cyberax — Mon, 18 Mar 2024 19:57:06 +0000

Writing a "toy C" compiler had been a course project in my university. It was C with most of its functionality, except things that are baffling, like syntactic ambiguity between function pointers and definitions, dangling else, etc.

It really is not a hard problem. Annoying and somewhat long, but not hard.

LLVM ist a mess

paulj — Mon, 18 Mar 2024 18:15:34 +0000

It was mostly humour. I too, like most students who've done a CS degree, have had to write some kind compiler for a class assignment (a toy front-end with type-checker for a subset of a typed JS-ish language). I've written compilers for simple DSLs - in AWK even ;). A compiler itself is not /that/ hard - completely agreed.

I was just pointing out the humour in the irony of making that point via an example written by an author who has a (prodigious) habit of solving difficult problems. ;)

I agree though that, even if a basic compiler is simple, there is a /lot/ more to making a _good_ C/C++ compiler.

LLVM ist a mess

farnz — Mon, 18 Mar 2024 18:02:16 +0000

I've written more than one compiler, and I would not count myself as a Fabrice Bellard level developer. You should be able to write a simple optimizing C compiler following a book like this in about 3 months full-time effort if you're a competent developer (less if you're willing to reuse tools like gas and ld rather than doing everything yourself).

LLVM ist a mess

willy — Mon, 18 Mar 2024 16:28:14 +0000

You've completely missed the point of that comment, but to refute the uninteresting part you're quibbling with:

https://student.cs.uwaterloo.ca/~cs444/

Team of four students builds a compiler in three months.

LLVM ist a mess

paulj — Mon, 18 Mar 2024 15:18:52 +0000

"writing a [blah] is not a hard problem" -> links to something written by one of the most prodigious, genius authors of Free Software. Hmm... ;)

LLVM ist a mess

intelfx — Mon, 18 Mar 2024 12:43:58 +0000

> If each user of LLVM bundled it, these problems would go away

Ah yes. I believe it is time for a revised version of the "layers of abstraction" maxim: every releng problem may be solved by pinning and bundling, except for the problem of too much pinning and bundling.

LLVM ist a mess

khim — Mon, 18 Mar 2024 09:08:40 +0000

> Modern CPUs, at least for Intel and Arm, have an architecturally defined data independent timing mode that you can enable in a status register bit when you want to execute this kind of crypto code, and which then guarantees that execution timing of a specified subset of instructions is not dependent on the data they are operating on.

They still would depend on alignment of you data and code, on speculative properties of code which was executed before and after you call that “well crafted” code and so on.

Just look on continuous struggle to guarantee that SGX is useful for something. With another vulnerability revealed less than week ago.

Ultimately the solution would be the same as with memory security in C: solution that was obvious on the day one would be applied… but only after everything else would be unsuccessfully tried.

LLVM ist a mess

pm215 — Mon, 18 Mar 2024 09:01:41 +0000

Modern CPUs, at least for Intel and Arm, have an architecturally defined data independent timing mode that you can enable in a status register bit when you want to execute this kind of crypto code, and which then guarantees that execution timing of a specified subset of instructions is not dependent on the data they are operating on. So I think the situation is not so bleak as you suggest: there's now a defined set of "stay within these boundaries and things won't change in future designs or microcode updates" rules.

Constant-time cryptography

khim — Mon, 18 Mar 2024 08:55:25 +0000

> Hardware crypto engines are nice, but they are not at all a substitute for constant time guarantees for software operations.

Oh, sure. Hardware works. “Constant time guarantees” are a snake oil you may lucratively sell. Completely different products with different properties and target audience.

> That requires application changes, not just library ones.

So you can't even change apps, yet, somehow, pretend that they are not leaking your precious key in some other way except for operations being of different speeds depending on source?

You keys are not leaking (or maybe leaking but you just don't know that) because nobody targets you. It's as simple as that.

Constant-time cryptography

DemiMarie — Mon, 18 Mar 2024 07:05:21 +0000

This is not the first time I have seen this suggestion. It is also completely non-viable in practice. The security core will be much slower than the other cores, which will ruin performance. One can avoid that by using a hardware accelerator instead of a slow core, but then one needs to (a) patch all of the existing applications and libraries to use the accelerator and (b) deal with the fact that hardware accelerators, especially for symmetric cryptography, require an asynchronous API to get good performance. That requires application changes, not just library ones.

Hardware crypto engines are nice, but they are not at all a substitute for constant time guarantees for software operations.

LLVM ist a mess

DemiMarie — Mon, 18 Mar 2024 06:55:48 +0000

I think part of the problem is that distributions and downstream try to use LLVM as a system library, instead of having each downstream bundle a known-good version. If each user of LLVM bundled it, these problems would go away, at the expense of significantly longer build times.

I know that Rust has a fork of LLVM. I believe this is partly because sometimes they need to fix miscompiles and can’t wait for upstream to take the patch.

LLVM ist a mess

khim — Sun, 17 Mar 2024 22:07:08 +0000

> One compiler application which feels intuitively useful to me (but I'm not a language designer) would be to have a _non-optimising_ compiler which can translate from a suitable language to dependable constant time machine code for some N architectures where N > 1

You do realize that for modern CPUs “architecture”, here, would include not just CPU vendor, but stepping, version of microcode, etc? One trivial example: when Intel implemented BMI instructions in 2013 they had nice, constant, execution time, but AMD turned them into nice let's leak all your data to everyone to see version after four years and every microcode update (on both AMD and Intel) may do the same to any instruction — to patch some other vulnerability.

> In this application we actually don't want ordinary optimisation, so I suspect some (many?) optimisation strategies are invalid and it may be faster to begin from almost nothing.

Before you may even begin attempting something like this you would need to define what do you want in the end. Given the fact that give enough samples you may even distinguish between (xor %eax,%eax and mov $1,%eax (they affect flags and one is 2bytes while other is is 5bytes) first you would need to define some metric which would say if timings are “sufficiently similar” or not.

The whole thing looks like an incredible waste of manpower: instead of trying to achieve something that's not possible to, realistically, achieve on modern CPUs we should ensure that non-ephemeral keys are generated on dedicated core. Adding tiny ARM core (Cell-style) would be much easier and more robust than attempts to create such compiler.

LLVM ist a mess

tialaramex — Sun, 17 Mar 2024 20:25:56 +0000

One compiler application which feels intuitively useful to me (but I'm not a language designer) would be to have a _non-optimising_ compiler which can translate from a suitable language to dependable constant time machine code for some N architectures where N > 1

The purpose would be to wean ourselves off machine code for writing core cryptographic libraries. It would be nice if the sort of people who enter NIST competitions could write this rather than C but it's not crucial.

In this application we actually don't want ordinary optimisation, so I suspect some (many?) optimisation strategies are invalid and it may be faster to begin from almost nothing.

LLVM ist a mess

roc — Sun, 17 Mar 2024 19:56:01 +0000

"Writing a compiler is not a hard problem" is too ambiguous to be useful. The gulf between a minimal C compiler (TCC) and an optimizing cross-platform C++ compiler with all the bells and whistles (good error messages, sanitizers, etc etc etc etc) is so vast you're not talking about the same thing at all.

LLVM ist a mess

farnz — Sun, 17 Mar 2024 19:53:24 +0000

Implementation language isn't at the root of this; the underlying issue is that LLVM IR's semantics aren't (yet) formally defined, but instead rely on informal reasoning throughout LLVM. As a consequence, it's intractable to verify that LLVM actually implements the claimed semantics, and it's not reasonable to write test cases that validate the LLVM IR semantics are met in edge cases, since we don't actually know what the edge cases are.

There's efforts afoot to fully define LLVM IR semantics formally, and one of the biggest outputs those efforts are having (at the moment) is finding bugs in existing LLVM functionality, where existing LLVM code assumes opposing meanings (that both fit the informally defined semantics) for the same LLVM IR construct in different places.

LLVM ist a mess

Wol — Sun, 17 Mar 2024 19:09:22 +0000

Is LLVM written in C++? Should they re-write it in Rust?

:-))

Cheers,
Wol

LLVM ist a mess

Wol — Sun, 17 Mar 2024 18:57:08 +0000

> that means everyone who uses it just have to get their act together and fork

Drop the "and fork", please. Okay, you can if you want, but the BEST approach (which is what the Rust guys did, I assume) is to *branch* it, fix it, and send pull requests upstream.

If upstream ignores them, then you have to decide what you want to do about it, but what you do NOT do is launch a self-entitled petulant shit-storm at upstream because their priorities are different from yours. If it ends with a full fork, then that's sad, but then you have two - hopefully friendly - projects sharing code, but with different priorities and aims. So be it ...

Cheers,
Wol

LLVM ist a mess

farnz — Sun, 17 Mar 2024 17:49:23 +0000

And even when they didn't assume C++, there's often been bugs that boil down to "Clang doesn't use this much, therefore it's not routinely tested and there's lot of lurking bugs"; see the fun Rust has had trying to use noalias on references, where because the matching Clang feature (the C99 restrict type qualifier) is rarely used correctly, miscompilations by LLVM traceable to noalias in LLVM IR kept blocking Rust from using it for Rust references (which definitionally can't alias each other).