|
|
Subscribe / Log in / New account

Scanning for secrets

Scanning for secrets

Posted Apr 10, 2021 1:17 UTC (Sat) by pabs (subscriber, #43278)
In reply to: Scanning for secrets by mathstuf
Parent article: Scanning for secrets

I don't think every project should need to guard against environment variables like CFLAGS, LD_PRELOAD or PYTHONPATH, since those are intended to change the build process.

At some point, "it works in CI" is basically the same as "it works on my computer". A better approach in case of build environment related problems is to record the two build environments, compare them and bisect the differences to find out which change causes the problem.

Some build environment related examples:

The Debian "buildd from hell", which compared packages built in a clean chroot with those built in a chroot containing as many -dev packages as possible to install at the same time. The mail below contains Message-IDs for related discussions, which you can put into the Debian lists msgid-search form to find the archives.

https://lists.debian.org/msgid-search/351842f7a4da3cff7ee...
https://lists.debian.org/msgid-search/

The Reproducible Builds folks deliberately vary build environments in various ways in order to detect parts of the build system that introduce non-determinism. Some of that may (in the past, now or in future) include LD_PRELOAD of various things, including faketime and or cowbuilder, which has a copy-on-write preload. The buildinfo.rst link below contains some of the philosophy that lead to this approach. Of course, the set of variations could be expanded and the set of tested build environments will never achieve the level of variation that random users trying to reproduce builds could achieve.

https://reproducible-builds.org/
https://tests.reproducible-builds.org/debian/index_variat...
https://salsa.debian.org/reproducible-builds/reprotest
https://reproducible-builds.org/docs/perimeter/
https://reproducible-builds.org/docs/recording/
https://salsa.debian.org/reproducible-builds/specs/buildi...

The Bootstrappable Builds project is aiming to get to a full Linux distro from < 1000 bytes of audited machine code plus all the necessary source code. Their approach is slightly different, instead of recording the build environment, they aim to *create* the build environment from scratch, but they will still encounter build environment differences, due to hardware differences and non-determinism (but they plan to eventually push the bootstrap process deeper into the hardware layer). They also desire build environment diversity though, they want to be able to do this for any arch and from any arch and on a variety of hardware of the same arch.

https://bootstrappable.org/
https://github.com/fosslinux/live-bootstrap/blob/master/p...
https://github.com/oriansj/talk-notes/blob/master/live-bo...
https://github.com/oriansj/talk-notes/blob/master/live-bo...


to post comments

Scanning for secrets

Posted Apr 12, 2021 13:00 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

> A better approach in case of build environment related problems is to record the two build environments, compare them and bisect the differences to find out which change causes the problem.

I agree with all of that. However, we're lacking a suitable "diff" tool to first get the diff we need to bisect. Unfortunately, not everyone is aware of effects that their "quick fixes" actually have and so when asking for differences, the important details don't even come up until you've already changed the code a few times, rebuilt, then finally asked for `LD_DEBUG=libs` output to be provided showing that none of the changes even mattered.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds