LWN: Comments on "Lots of progress for Debian's reproducible builds"

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

indolering — Fri, 29 May 2015 01:45:34 +0000

Perfect security is a myth, the name of the game is increasing the attack cost. Deterministic builds allow us to dramatically increase the cost of certain attacks, which is a net gain in terms of security.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

nix — Tue, 17 Feb 2015 19:42:14 +0000

It instruments arcs between basic blocks, so it's deterministic.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

cesarb — Wed, 04 Feb 2015 08:55:55 +0000

Does profile-guided optimization add profiling code to all branches, or does it use a timer (and a signal) to sample the basic blocks? If it uses a timer, it won't be deterministic.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

nix — Tue, 03 Feb 2015 22:29:11 +0000

Of course this is wrong. Profile-guided optimizations *that depend on nondeterministic test runs* are out of the question. If the test run is deterministic (thus the branches taken are unchanging across multiple runs), then of course profile-guided optimizations are not incompatible with reproducible builds.

Lots of progress for Debian's reproducible builds

robbe — Thu, 29 Jan 2015 20:02:45 +0000

I stand corrected (and am not disappointed)! Both projects are important work.

Lots of progress for Debian's reproducible builds

Lunar^ — Thu, 29 Jan 2015 17:58:06 +0000

Sorry to disappoint you, but I have been mostly active in the Tor Project. My contributions to Tails have been sporadic at best. But you are right, contributing back to Debian is one of Tails' focus.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

nix — Thu, 29 Jan 2015 15:55:03 +0000

I also wonder what kinds of compiler optimizations have to be turned off to make sure that the code compiles the same

Thanks to -frandom-seed, none to speak of. You just need to provide the same seed. (Well, none to speak of in the deterministic set of default optimizations. I presume that profile-guided optimizations are also out of the question, unless you distribute the .gcno files, etc, which is unlikely to happen!)

Lots of progress for Debian's reproducible builds

robbe — Thu, 29 Jan 2015 07:33:31 +0000

A bit of background for those that don't know it: Jérémy is a very active Tails¹ developer. Tails being one of the major anonymity environments² utilizing (and complementing) Tor, and is a Debian derivative.

It's great that Tails is not sitting on its changes, but contributing them upstream to Debian and beyond. Thanks to Jérémy and the others!

¹ https://tails.boum.org/
² Whonix is the other big one

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

paulj — Wed, 28 Jan 2015 18:27:25 +0000

When Debian finish making their build reproducible, they can then try work on reproducible cross-compiles.

Though, I don't see why it would be impossible for an ELF virus to contain payloads for a variety of target architectures (especially those known to be used by the reproducible, cross-compile, build checker), and select the appropriate one to infect newly created files. So, I'm not sure what that would achieve.

Reproducible builds: Moving Beyond Single Points of Failure for Software Distribution

jburgess777 — Tue, 27 Jan 2015 00:40:49 +0000

(presented at 31C3)

Description: http://events.ccc.de/congress/2014/Fahrplan/events/6240.html
Slides: http://events.ccc.de/congress/2014/Fahrplan/system/attach...
Video: http://media.ccc.de/browse/congress/2014/31c3_-_6240_-_en...

Watch this one next Saturday

MarkVandenBorre — Tue, 27 Jan 2015 00:15:32 +0000

https://fosdem.org/2015/schedule/event/stretching_out_for...

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

Cyberax — Mon, 26 Jan 2015 22:23:59 +0000

So simply add a couple of cross-compilations to the mix. That also applies to make and other low-level utilities.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

paulj — Mon, 26 Jan 2015 11:43:47 +0000

Even just restricting Thompson's attack to compilers (which is artificially restricting his point), it's not complicated at all, because in modern systems all those compilers come in standardised binary formats with some well-known places you can hook your own code into.

Though, I think some people here have unreasonable assumptions about how isolated different compiler authors are from each other. E.g. another article this very week is discussing GCC AST exports for Emacs, and contains a link to a GCC thread where Sun compiler people are posting patches to GCC. Personally, I think it's highly unreasonable to assume that people who develop very in-depth expertise in compilers will only ever work on one implementation. Not my anecdotal experience at all!

If you don't artificially restrict Thompson's attack, then you've still got the rest of the system to verify: all the software, firmware, microcode and hardware.

Good luck with that.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

tao — Mon, 26 Jan 2015 11:43:35 +0000

You could always hack binutils instead... Or make...

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

epa — Mon, 26 Jan 2015 10:11:25 +0000

I think the point is that while in principle Thompson's attack will apply, in practice if you have a big tower of compilers (pcc -> clang -> gcc -> gcc v1.0 -> clang -> etc) it will not be possible for the evil code to be propagated all the way through that chain because it's simply too complicated a problem.

As the other poster said you would need a "superintelligent agent" to write a program that's somehow capable of recognizing and Trojaning code from all these different compilers, plus goodness knows how many others that might be thrown into the mix.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

smoogen — Fri, 23 Jan 2015 19:06:47 +0000

I didn't put this in originally and should have. Nothing I am saying is to mean that I don't think that verifiable build systems aren't needed or have good uses. I just think that the amount of "This solves ...." is being over stated and quickly leads to -funroll-loops level of blindness.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

smoogen — Fri, 23 Jan 2015 19:03:25 +0000

I would expect that people looking to attack repeatable builds would figure on where people are most likely to put blind trust in something and exploit that.

Easiest? If the .changes files has to do certain things to make sure the archive is altered to "meet matching criteria" Stick something there which will get run every time that sticks in the exploit.

Hard? Everyone looks at the compiler to "deal with Thompson attack" but no one looks at m4, yacc, lex, Makefiles etc where it is easier to stick in some line noise that no one understands and then explain it away as an accident if they do.

Harder? If the system clock or other parts of the build system have to be derandomized to make sure that parts of various code are built exactly correct.. figure out which settings in such a build system are beneficial for your in-code exploit.

In any case, I could actually see where this sort of build system actually makes trust attacks easier because people will blindly say "well it built the same as the first one... must be clean." and not look at what actually was done to make it build exactly like the other part. [I also wonder what kinds of compiler optimizations have to be turned off to make sure that the code compiles the same.]

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

paulj — Fri, 23 Jan 2015 18:40:04 +0000

If only binaries came in standardised formats, and had general ways to inject code, and lots of well-known hooks to allow that code to execute (well-knowns hooks supplied by the format, by the runtime, and by the specific compiler code). Oh wait, they do.

In other news, "viruses" are a *lot* more sophisticated since Thompson's POC.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

paulj — Fri, 23 Jan 2015 18:37:02 +0000

Your DDC work is cool, and people working on doing reproducible builds of entire distros is cool, but, really, it's *reinforcing* Thompson's point about trust-roots, not countering. Anyway....

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

gnb — Fri, 23 Jan 2015 12:44:19 +0000

The work of backdooring the compiler scales linearly with the number of compilers, but I wouldn't expect the difficulty of making sure each of the relevant compilers ships with the backdoor included to be linear: you have to patch each of N compilers without any of these attempts being noticed.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

PaXTeam — Thu, 22 Jan 2015 22:56:58 +0000

it doesn't matter how many compilers you chain together, you have to start with a trusted one or Thompson's attack will apply. also pattern matching one compiler's AST is about the same work as matching another (read: scales linearly with the number of compilers), it's nothing to do with likeliness.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

Cyberax — Thu, 22 Jan 2015 22:41:01 +0000

You can use multiple compilers (PCC to compile CLang to compile GCC). It's theoretically possible for a sufficiently advanced superintelligent agent to write a software that can recognize the source of these compilers, but it's exceedingly unlikely.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

PaXTeam — Thu, 22 Jan 2015 22:33:35 +0000

DDC doesn't counter the trusting trust attack. it's only a method to produce a trusted toolchain by another, but it does *not* help producing the initial trusted toolchain - you have to do it the hard way, which was Thompson's point.

Awesome! Fedora also, and this is a step towards countering "Trusting Trust" toolchain issues

david.a.wheeler — Thu, 22 Jan 2015 22:07:51 +0000

This is really awesome. I've been following the reproducible (deterministic) build work with interest, and I'm really impressed with the speed of progress. Let's face it, 80% of the huge Debian repository, in such a short time, is quite an accomplishment. I'd like to add a few notes.

First, Debian's not the only one. Fedora is also working on reproducible builds, and I believe there are others.

Second, reproducible builds are also a step towards countering the trusting trust attack (attacks on toolchains). My approach for countering the trusting trust attack, called diverse double-compiling (DDC), first requires that the toolchain portions you care about have a reproducible (deterministic) build. If the whole toolchain is reproducible, it's suddenly much easier to use DDC to counter the trusting trust attack.

Lots of progress for Debian's reproducible builds

Lunar^ — Thu, 22 Jan 2015 16:53:59 +0000

We are also tracking the status of specific package sets so we can focus our efforts to what matters to most users. Another aspect is that until now, it has been a priority of very few Debian contributors. If every maintainers start to pay attention to build reproducibility, it will also help quite a bit for remaining packages.

Lots of progress for Debian's reproducible builds

dgm — Thu, 22 Jan 2015 13:46:36 +0000

> The Debian Reproducible Builds project has recently gotten more than 80% of packages to build reproducibly

That number.

Well, if it has taken 1.5 years to this point, the remaining 20% of the work will be done in 6 years, more or less.

Lots of progress for Debian's reproducible builds

pabs — Thu, 22 Jan 2015 13:37:55 +0000

FYI the .changes file is a set of instructions for how the archive software (dak) should alter the Debian archive.