Adding package information to ELF objects

Posted Nov 5, 2021 7:24 UTC (Fri) by zuki (subscriber, #41808)
In reply to: Adding package information to ELF objects by pabs
Parent article: Adding package information to ELF objects

> The tool version number doesn't matter at all, what matters is behavior of the tool, which is often identical across versions of the tool. Artificially changing the behavior of the tool (by embedding the version number in the output) with every single change in the version of the tool reduces reproducibility.

You're conflating two things: behaviour and labelling of a package. This proposal has no effect whatsoever on behaviour of tools and their output.

Once again: since this proposal produces predictable output, it is reproducible in the sense of https://reproducible-builds.org/docs/definition/ . If you want to create some further definition, please do so, but give it a different name and a clear explanation.

> This is incorrect, build-ids are meant to be deterministic given identical inputs, and package versions are mostly not inputs to package builds, only source code is.

In practice, build-ids change almost always, e.g. see the investigation in https://fedoraproject.org/wiki/Changes/Package_informatio... .

Adding package information to ELF objects

Posted Nov 5, 2021 8:02 UTC (Fri) by pabs (subscriber, #43278) [Link] (1 responses)

If I have a package foo with versions 1, 2, 3, where each version has foo.c with identical contents. In our current world, /usr/bin/foo will be identical for each version of the package. Under the Fedora plan, /usr/bin/foo will change hashes. This will duplicate foo's object in ostree and other content-based storage systems.

If I build version 1 of that package on Debian and on three Debian based distributions that do not change any of the build dependencies of foo. In our current world, /usr/bin/foo will be identical for each of those distributions. Under the Fedora plan, /usr/bin/foo will have different hashes again.

If I have foo.c in package foo version 1 and in version 2 I run clang-format over the code, changing the source hashes. In our current world, /usr/bin/foo will likely be identical for both versions. Under the Fedora plan, /usr/bin/foo will have different hashes again.

The history and philosophy of reproducible builds is a lot more nuanced than the summary on the website makes it out to be. There is some more of that sort of thing in this document:

https://salsa.debian.org/reproducible-builds/specs/buildi...

Adding package information to ELF objects

Posted Nov 5, 2021 9:35 UTC (Fri) by zuki (subscriber, #41808) [Link]

OK, let's call this property "repeatable binaries" for ease of reference.

> If I have a package foo with versions 1, 2, 3, where each version has foo.c with identical contents. In our current world, /usr/bin/foo will be identical for each version of the package.

Yes, this is theoretically possible. But in practice, at least in the practical examples I have looked into, it doesn't hold.

Trivially, many programs include the package version in output (for purposes of identification), e.g. clang, gcc, qemu, the kernel.

More subtly, in Fedora, binaries include a .gnu_debuglink section that includes the package version:

$ readelf -W -p .gnu_debuglink /usr/bin/true
...
[ 0] true-8.32-31.fc35.x86_64.debug

$ readelf -W -p .gnu_debuglink /usr/bin/false
...
[ 0] false-8.32-31.fc35.x86_64.debug

AFAICT, this section is included in the build-id calculation, so the build ids also vary when the package version changes.

There is a lot of moving parts here… In the examples I looked at, binaries are not "repeatable". But I'll say it once again: if somebody has an example where it is true, please show it. Right now people bring up theoretical considerations which are trivially shown to be false in real packages.

> If I build version 1 of that package on Debian and on three Debian based distributions that do not change any of the build dependencies of foo. In our current world, /usr/bin/foo will be identical for each of those distributions. Under the Fedora plan, /usr/bin/foo will have different hashes again.

I don't know the details of how Debian&derivs build packages, but at least in case of Fedora, those binaries would already be different when rebuilt in a derivative distro, as shown above.

> run clang-format over the code, changing the source hashes. In our current world, /usr/bin/foo will likely be identical for both versions.

For the sake of argument, let's say that the build process is such that you really get identical binaries in this case. Does this have any practical value? It would matter only if people reformat their code and release a new package version. People have better things to do. And if I was a maintainer and I saw release notes that say that comments were reformatted and absolutely no other changes were done, I'd just ignore that version. So please, stop with the theoretical examples and show a case that actually has an iota of practical effect.