A look at Nix and Guix

Posted Mar 11, 2024 12:34 UTC (Mon) by Foxboron (subscriber, #108330)
In reply to: A look at Nix and Guix by Nebucatnetzer
Parent article: A look at Nix and Guix

A 4 page article claiming "some buildsystem is reproducible in this experiment" isn't particularly interesting when actual Linux distros are reproducing ~90% of their published packages in the real world.

A look at Nix and Guix

Posted Mar 11, 2024 12:44 UTC (Mon) by gioele (subscriber, #61675) [Link] (2 responses)

> A 4 page article claiming "some buildsystem is reproducible in this experiment" isn't particularly interesting when actual Linux distros are reproducing ~90% of their published packages in the real world.

This is a misreading of the paper.

The paper results are:

> Our results show that we can achieve 99.99% reproducibility of build environments over 7 010 516 packages coming from 200 historical revisions of Nixpkgs, the Nix package set. Additionally, we were able to rebuild 99.94% of the packages from a 6-year-old Nixpkgs revision, demonstrating that reproducibility of build environments is actually useful for software rebuildability.

"This experiment" is rebuilding 99.94% of all the packages of a 6-year-old revision of a distro. "Actual Linux distros", cannot reproducibly build that many packages today, nor are able to do that for older revisions.

A look at Nix and Guix

Posted Mar 14, 2024 10:36 UTC (Thu) by Foxboron (subscriber, #108330) [Link] (1 responses)

I re-read the paper and this is wrong as well. This paper is just misusing the "Reproducible Builds" definition from the Reproducible Builds project. It's checking if you can reproduce the build paths which has nothing to do with the build outputs.

From RQ1:

> Therefore, we adjust RQ1.2 slightly into: assessing whether we can reproduce identical output paths for all the jobs (as we can always retrieve historical Hydra output paths).

From RQ2:

>Besides, for now, we do not check for *bit-by-bit* reproducible builds [ 16 ]. Extending our evaluation to build more revisions and checking for reproducible builds are both part of our future plans

So they have redefined "reproducible builds" to mean "can we reproduce the build-paths" and schewed the Reproducible Builds definition to "bit-by-bit reproducible builds" which is just misleading and extremely dissapointing.

A look at Nix and Guix

Posted Mar 14, 2024 10:46 UTC (Thu) by Foxboron (subscriber, #108330) [Link]

I should read better. It's not really using "Reproducible Builds" at all an is only talking about build environments, which is very much what the static build-paths represents.

Equating this paper with reproducible builds is the mistake here, not them misusing any words.