No compilation, please? [LWN.net]

No compilation, please?

Posted Jul 11, 2024 20:03 UTC (Thu) by atnot (subscriber, #124910) [Link] (1 responses)

I've used NixOS and Nix for a few years now and the only time it's compiled anything from source was when I enabled MGLRU in the kconfig to see if it would help on a low memory device. As long as you don't start messing with build options, it may as well be binary.

No compilation, please?

Posted Jul 11, 2024 20:19 UTC (Thu) by jhe (subscriber, #164815) [Link]

My experience was the opposite - the amount of local recompilation and hosages due to full disks (back then with btrfs...) was what made me drop nix in 2017. This was around the glibc 2.26 update which apparently required a full rebuild of the world.

No compilation, please?

Posted Jul 11, 2024 20:32 UTC (Thu) by ericproberts (guest, #139553) [Link] (1 responses)

My reading of the nix philosophy pretty emphatically points to binary caches as the solution. You could use fetchurl and friends, but then you lose the connection between build instructions result. A nixpkgs of binary packages seems like having a cache but with extra steps.

Maybe there's something I'm missing. Is there a specific problem you're trying to solve?

No compilation, please?

Posted Jul 11, 2024 20:36 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

I want replicable environments, but I'm fine with not getting a fully flexible specification. Something like Debian snaphsots, but more flexible.

No compilation, please?

Posted Jul 12, 2024 8:55 UTC (Fri) by spacefrogg (subscriber, #119608) [Link] (12 responses)

I use NixOS as a daily driver since 2015 and there is no such thing as recompile the World under normal usage. Whoever told you this was setting you up. If, however, you come frome Gentoo-like systems, you have to accept that you can neither touch the kernel nor libc or GCC build flags, otherwise, you DO end up in endless recompile the World scenarios. That is true for any other distribution platform in existence, though. So, I am not sure where you are coming from. Recompilation just does not happen massively, only small things here and there, on a released nixpkgs channel.

No compilation, please?

Posted Jul 12, 2024 11:18 UTC (Fri) by ms (subscriber, #41272) [Link]

Yep, I'm also daily-driver of NixOS on several machines. I have a few custom bits and pieces, e.g. a tweak to the configuration of sqlite-as-used-in-audacity so whenever either of those change, I get a recompile _of just those bits_. I have once or twice (in a few years) triggered a kernel recompile, but even that didn't result in a rebuild-the-world. I would certainly not dare touch things like libc though.

No compilation, please?

Posted Jul 12, 2024 14:23 UTC (Fri) by intelfx (subscriber, #130118) [Link] (10 responses)

> you have to accept that you can neither touch the kernel nor libc or GCC build flags, otherwise, you DO end up in endless recompile the World scenarios. That is true for any other distribution platform in existence, though.

Well, that is just blatantly untrue. In any other distribution platform in existence, changing kernel configuration, or patching the libc, or even changing gcc build flags will never result in rebuilding the world — so long as you maintain the ABI of the component in question, which is pretty much always.

The only distribution that has a problem with it is NixOS (and its derivatives/equivalents). And *that* is the problem with Nix{,OS}.

No compilation, please?

Posted Jul 12, 2024 15:39 UTC (Fri) by atai (subscriber, #10977) [Link] (6 responses)

> The only distribution that has a problem with it is NixOS (and its derivatives/equivalents). And *that* is the problem with Nix{,OS}.

Really? Gentoo should have the same "problem" and unrelated to NixOS or derivatives

No compilation, please?

Posted Jul 12, 2024 18:03 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

It is not a problem for Gentoo, because "I want to compile everything locally" is Gentoo's entire raison d'être (i.e. it's not a bug, it's a feature). This is not the case for NixOS, because Nix is all about containerization and isolation, which does not (in principle) require compiling things locally.

No compilation, please?

Posted Jul 12, 2024 18:17 UTC (Fri) by intelfx (subscriber, #130118) [Link] (4 responses)

> Really? Gentoo should have the same "problem" and unrelated to NixOS or derivatives

No, because Gentoo does not require rebuilding anything **as a consequence** of patching the libc or changing toolchain build flags.

As NYKevin says, when Gentoo _does_ require rebuilding things, it's because you deliberately chose to use a source-first distribution. Or you might choose to use Gentoo's binary cache, which is (unlike Nix's) not predicated on never touching or altering the dependencies of packages involved. So it's a feature (that you might or might not choose to use), not a problem.

Nix, on the other hand, will force you to rebuild the world exactly **as a consequence** of changing some definitions at the bottom of the dependency graph, even if it is not technically necessary.

No compilation, please?

Posted Jul 12, 2024 19:10 UTC (Fri) by NightMonkey (subscriber, #23051) [Link] (3 responses)

+1. I'll just add: I've been a satisfied Gentoo user for the last 18 years. Early on, yes, there was a good amount of world rebuilding needed. For production, this meant investing in creating temporary systems to handle the build and storage of binaries was worth it.

But, the times I've had to rebuild world all at once in the last 5 years has been zero. The work to create centralized binary caches at Gentoo HQ and making binary-only versions of some of the most brittle builds (major web browsers, office suites, multimedia players, etc.) has taken the pain out of everyday upgrades.

And, if you have a farm of dozens, hundreds or thousands of machines, you can easily create the infrastructure to reduce the pain of recompiling to a temporary build farm that creates a local binary cache, and distribute the built binaries to the fleet. Modern CPUs and ramdisks handle this task well.

Gentoo is a *metadistribtution*. This means that it can be used to create *your own distribution*. It has the tools to enable you to do so. With the care and attention, it can reward you greatly.

Let's not slag Gentoo reflexively when we want to just complain about inefficiency generally.

Cheers!

No compilation, please?

Posted Jul 12, 2024 19:59 UTC (Fri) by Wol (subscriber, #4433) [Link] (2 responses)

> But, the times I've had to rebuild world all at once in the last 5 years has been zero. The work to create centralized binary caches at Gentoo HQ and making binary-only versions of some of the most brittle builds (major web browsers, office suites, multimedia players, etc.) has taken the pain out of everyday upgrades.

You haven't upgraded your profile recently?

Mind you, that was the upgrade from 2017 to 2024, and it was also the upgrade where you were sort-of forced to merged-usr, and it was sort-of the upgrade where you felt pushed forward to get the distro into the third decade.

I think it was only my second profile upgrade in however many years (Istr the one before was 2014), and this upgrade the instructions did say "finish with 'emerge -e @world' " ie "rebuild everything", But how long have I been using gentoo? And this is the first time.

Cheers,
Wol

No compilation, please?

Posted Jul 12, 2024 21:45 UTC (Fri) by NightMonkey (subscriber, #23051) [Link] (1 responses)

Ah, yes, you're right, Wol - I did have to do that migration-related rebuild of world. I was able to skip the merged-usr, tho, with the "split-usr" profiles, as I'm using OpenRC. So, not zero. Once, across all my local systems. I do have distcc and other optimizations set up so it makes doing so less painful in terms of time.

No compilation, please?

Posted Jul 12, 2024 22:26 UTC (Fri) by Wol (subscriber, #4433) [Link]

I didn't go to merged-usr, because both my systems are reasonably recent, and systemd. So they were merged-usr from the start.

I went systemd for the same reason I chose gentoo - it's a deliberate choice of a learning curve. My old OpenRC system was just that - too old. I used to have distcc, but I just haven't got round to setting my new systems up the way I would like. I want to share the build directories over the network, and then I'd uprade the faster system (I build binary packages as a matter of course), and then upgrade the slower system with the "use packages if they're there" option.

Cheers,
Wol

No compilation, please?

Posted Jul 12, 2024 21:07 UTC (Fri) by bauermann (subscriber, #37575) [Link] (1 responses)

> so long as you maintain the ABI of the component in question, which is pretty much always.

The aim of Nix and its derivatives/equivalents is to do away with the "pretty much" part of the sentence, because "pretty much" isn't reliable enough. It's very hard to guarantee that an ABI hasn't been broken, and subtle changes in the input can affect a package build in unexpected ways.

The principle of Nix/Guix/etc is that the inputs to a package build are painstakingly tracked so that if anything changes in any of the inputs (or the inputs' inputs, etc down to the last turtle) then a new build is made. Which is why changes deep in the dependency graph trickle down to everything. Eelco Dolstra's PhD thesis has the details. It's an interesting read.

This is the cornerstone of the functional package manager's reproducibility and rollback guarantees.

Guix has the graft mechanism to replace a dependency without propagating the changes to the dependant packages, assuming that the ABI is the same. It's used to apply security fixes without requiring world rebuilds or downloads of updated cached binaries for everything. But it's a practical concession that goes against the underlying model.

No compilation, please?

Posted Jul 12, 2024 22:33 UTC (Fri) by Wol (subscriber, #4433) [Link]

> The principle of Nix/Guix/etc is that the inputs to a package build are painstakingly tracked so that if anything changes in any of the inputs (or the inputs' inputs, etc down to the last turtle) then a new build is made. Which is why changes deep in the dependency graph trickle down to everything. Eelco Dolstra's PhD thesis has the details. It's an interesting read.

I haven't looked ... but that is a plentiful source of gentoo bugs I suspect. I'm sure I've been bitten by missed dependencies in the ebuilds, I've seen bug reports ... and of course, they're a right pain to actually track down because they're obscure and don't bite that many people.

Cheers,
Wol

No compilation, please?

Posted Jul 13, 2024 9:05 UTC (Sat) by JanSoundhouse (subscriber, #112627) [Link]

It all comes down on how dependencies are managed. Your build system has to be pretty sophisticated in order to avoid rebuilding the world when some as basic as the libc changes (c makes it pretty hard to determine what actually changes). With something like bazel its possible to avoid the unnecessary rebuilds completely. But thats because every dependency has to be declared - on the consumer and provider side and its actually enforced by building everything in isolation.
For example if a flag for the libc is changed, the depending packages must be able to infer whether that change needs a rebuild on their end. You cannot rely on make to do it for you, as it only looks a file dates and not the actual content.
For a human it might be possible to tell that a certain flag added or removed will result in a compatible binary, but if you want your builds to be always correct (and you dont have a perfect declaration of _all_ dependcies) rebuilding is the safest bet.

No compilation, please?

Posted Jul 12, 2024 13:42 UTC (Fri) by RaitoBezarius (subscriber, #106052) [Link]

FWIW, cache.nixos.org contains already more than 700TB of cached packages. Unfortunately, the number of possibilities and combinations using all of nixpkgs necessarily contains more than 700TB of cached packages (across revisions), there's a balance to strike. The philosophy is to aim for a "95 % of the time, you will get a cache hit".