LLVM ist a mess

Posted Mar 18, 2024 6:55 UTC (Mon) by DemiMarie (subscriber, #164188)
In reply to: LLVM ist a mess by Curan
Parent article: Cranelift code generation comes to Rust

I think part of the problem is that distributions and downstream try to use LLVM as a system library, instead of having each downstream bundle a known-good version. If each user of LLVM bundled it, these problems would go away, at the expense of significantly longer build times.

I know that Rust has a fork of LLVM. I believe this is partly because sometimes they need to fix miscompiles and can’t wait for upstream to take the patch.

to post comments

LLVM ist a mess

Posted Mar 18, 2024 12:43 UTC (Mon) by intelfx (subscriber, #130118) [Link] (4 responses)

> If each user of LLVM bundled it, these problems would go away

Ah yes. I believe it is time for a revised version of the "layers of abstraction" maxim: every releng problem may be solved by pinning and bundling, except for the problem of too much pinning and bundling.

LLVM ist a mess

Posted Mar 18, 2024 23:49 UTC (Mon) by NYKevin (subscriber, #129325) [Link] (3 responses)

Meh, just use block-layer deduplication and/or a content-addressed filesystem, and then all of the logical duplication works out to relatively little physical duplication.

LLVM ist a mess

Posted Mar 19, 2024 12:24 UTC (Tue) by intelfx (subscriber, #130118) [Link] (1 responses)

This could only have a _chance_ of working as intended if either 1) everyone was pinning and bundling from the same blessed set of binary artifacts, or 2) every such project's build output was bit-for-bit reproducible (and if no one ever used different compiler flags or versions).

I think it's safe to say that the real world is very far from achieving either scenario.

LLVM ist a mess

Posted Mar 19, 2024 18:11 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

> 2) every such project's build output was bit-for-bit reproducible (and if no one ever used different compiler flags or versions).

That's actually how Nix works (compiler flags and environment are also a part of the content's hash). Some newer languages like Go also have this baked into the module system.

LLVM ist a mess

Posted Mar 19, 2024 12:28 UTC (Tue) by paulj (subscriber, #341) [Link]

Unless some header has even a slightly different length between one object file and another, and then every block has different contents, despite (say) 99+% of the content being the same.

LLVM ist a mess

Posted Mar 20, 2024 22:24 UTC (Wed) by Curan (subscriber, #66186) [Link]

Oh my goodness no. LLVM needs to decide what it wants to be, IMHO. If they are a compiler, then each version can do, what it likes – as long as it produces executable programs without any bugs due to its internals.

If they want to offer a libllvm (and all the other wonderful libraries like libclang), with a SONAME, that indicates it is stable, they need to get their act together. Or make it impossible (or at least a manual downstream patch) to do dynamic linking (which they do not). Apart from that: even static linking caused issues in the past. When I linked LLVM statically, I still ended up having issues with executed workloads, that brought their own (sometimes statically linked) versions of LLVM. I have to admit, that I never went for the deep dive into these issues, since I am not given time for that during my day job. But I do have to keep the workloads running. So far that meant either manually "fixing" the LLVM builds (I will not claim these to be universally applicable fixes) or pinning the deployed LLVM version until we could work something out with our various upstreams.

(Side note: building LLVM across platforms is not fun. LLVM constantly breaks one build or another. I am sure Sylvestre (who's behind apt.llvm.org) and others beyond myself could attest to that. So even your "just bundle it" solution falls flat on its face right away.)