User: Password:
|
|
Subscribe / Log in / New account

Fedora/mock situation can be fixed

Fedora/mock situation can be fixed

Posted Jun 25, 2013 18:30 UTC (Tue) by david.a.wheeler (subscriber, #72896)
In reply to: Fedora/mock situation can be fixed by nim-nim
Parent article: Van den Oever: Is that really the source code for this software?

"The end result is that it is essentially impossible to get reproducible builds of a large software environment without either freezing it (and accepting bugs, including security bugs, won't be fixed) or rebuilding everything all the time (and then you get a build anyone can reproduce for the few days it will live before being replaced by a new build)."

The good news is that we've got a lot more cycles to throw at the problem, and a lot of this is embarrassingly parallel. You can force rebuilds so that on release everything was built with the current dev tools. You want to do that anyway, because that means that all the packages use the current optimizations, etc. There's no reason you have to wait to the very end to do this; you could recompile lower-level dev tools, then slowly move up the tree. Regression tests can catch a lot, since recompilation of unchanged source code should not be changing functionality at all.

"Even getting a bootstrapable distro (something that can be built from scratch once, without depending on binaries of suspect origin) is non-trivial due to how many software bits accumulated cyclic dependencies over time. When A depends on B that depends on C which depends on A what is the build order exactly you are supposed to follow so people can reproduce your stack ?"

It's annoying, but for bootstrapping you can break the dependencies. There are lots of ways to to do it. And in the longer term, you can work to break the dependencies; a lot of these cycles are inadvertent.


(Log in to post comments)

Fedora/mock situation can be fixed

Posted Jun 25, 2013 19:24 UTC (Tue) by nim-nim (subscriber, #34454) [Link]

"The good news is that we've got a lot more cycles to throw at the problem"

The limiting factor is not the build farm cycles. The limiting factor is your deployment process. What good your beefed up build farm that can rebuild the universe in a blink will be, if you can't push the result to users fast enough? Rebuilding for security problems is not here to look good on the builder, to be useful it must be deployed as fast as possible. The more stuff you rebuild to have a perfectly coherent universe the more bits you have to push user-side.

The current systems balance reproducibility with deployment ease.

> It's annoying, but for bootstrapping you can break the dependencies.
> There are lots of ways to to do it. And in the longer term, you can work
> to break the dependencies; a lot of these cycles are inadvertent.

And new cycles will be added faster than you can break them. Distros have tried for years to break the cycles with no luck. Developers keep adding them. They don't care about the system costs, they are not system people. An academic from-scratch project could try to eradicate them, but in the real world, Apple (OSX), Google (Android), Sony (PS4) all chose to reuse off-the-shelf software. Restarting from zero is not an realistic scenario nowadays, you get to live with preexisting software, warts included.

Fedora/mock situation can be fixed

Posted Jun 25, 2013 20:27 UTC (Tue) by david.a.wheeler (subscriber, #72896) [Link]

"The limiting factor is your deployment process. What good your beefed up build farm that can rebuild the universe in a blink will be, if you can't push the result to users fast enough?" - The obvious answer is to rebuild before a major release. For example, before releasing Fedora 32, recompile from the ground up (twice) so that Fedora 32, when it compiles itself, will produce itself. You can do this in a rolling process to avoid making it a "big bang". There are lots of other options too.

"new cycles will be added faster than you can break them." - Yes, those are annoying. But they can be handled.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds