| |||||||||||||
Creating a Fedora ARM distribution part 2: BootstrappingCreating a Fedora ARM distribution part 2: BootstrappingPosted Oct 21, 2011 21:59 UTC (Fri) by alvieboy (subscriber, #51617)Parent article: Creating a Fedora ARM distribution part 2: Bootstrapping > but in the case of a complete bootstrap there are no pre-built dependencies, so they must be created from scratch. Not only that, but build dependencies must be built using a minimal "bootstrap" configuration that avoids circular source dependencies on packages not built yet, and also provides enough basic functionality to rebuild the full set of dependencies using standard tools Unless this is about GCC + libc dependency, it does not make much sense. Bootstrapping an unknown architecture might require more than this *once*, but we're speaking about same OS, a Linux OS, which supports cross compiling easily. The only circular dependency I see is this one. The one that might prevent you to have a build system up and running. I've developed a proprietary linux distro, with all dependencies very much like you see in debian. I have so far managed to get a cross compiling environment ready for a new architecture in less than 2 days. Having circular dependencies is not only wrong, as it might get you in deep trouble. So called "weak" dependencies are used to avoid this (like Suggests, Recommends, so on). Still, you don't actually need this, do you? And yes, our distro is based on chroot environment. And no circular dependencies at all [our system did not cope with those, so we definitely did not have them until recently we had to import python modules, and use their depicted deps - we did have to handle them, but only then]. Sorry for being a PITA here, but I often think that RPM based systems not only did not evolve much, but started with the wrong foot from the beginning [and still paying the cost]. One thing that might help you is to decouple the build environment from the deployment environment. Meaning that you can have a set of packages meant for development, and another set for publishing. Right now I can install a fully dev environment just by "mkdir DIR" and [real command ommited here] apt-get --chroot DIR --install dev-system --from-scratch. Rebuilding it, differently from the main stream, is absolutely not needed. But we did think about this in a very early stage. Alvie
(Log in to post comments)
Creating a Fedora ARM distribution part 2: Bootstrapping Posted Oct 21, 2011 22:48 UTC (Fri) by xxiao (subscriber, #9631) [Link]
recalled a few years back when i needed to bootstrap debian for a new cpu, it's painful and near impossible(cross compiling, perl, cross-dependencies,etc, i eventually gave up), I wish one day debian can have something like this too, it's purely black magic.
Creating a Fedora ARM distribution part 2: Bootstrapping Posted Oct 22, 2011 4:17 UTC (Sat) by pabs (subscriber, #43278) [Link]
There was a session about that at DebConf11:
http://penta.debconf.org/dc11_schedule/events/745.en.html
Seems like the Emdebian folks are working on it.
Creating a Fedora ARM distribution part 2: Bootstrapping Posted Oct 22, 2011 7:53 UTC (Sat) by jcm (subscriber, #18262) [Link]
Right, so Debian are working on adding explicit Bootstrap deps. In fact, I'd argue we (other distros) have some good things to feed back into our process from their experiences. Take that as a round about way of me say kudos to the Debian guys for an awesome job on v7 so far :)
Creating a Fedora ARM distribution part 2: Bootstrapping Posted Oct 22, 2011 7:58 UTC (Sat) by jcm (subscriber, #18262) [Link]
Fedora, like many other distros does not ever build its packages with cross-compilation. Sure, it "works" but it's more than just compilation we care about in building on target. Runtime tests, etc. can all be made to work using various projects that are out there to glue a cross-compiler into running tests on target - all of this can be done. But we don't. It's just a bigger-than-ARM specific policy of Fedora that everything is build on target.
So. Given that we needed to build the world, on the target, this is what we did and how we did it. And even bootstrapping Debian I happen to know is a great PITA too. Wookey and others have done some good work there, and I'm genuinely impressed with the multi-arch work that feeds into some of this (and which we don't currently have plans to implement, but that could change if there's more than ARM specific interest in doing it within Fedora).
All of this stuff is hard. Bootstrapping happens relatively infrequently but we decided it was better to know how to do this. The next trick will be to have it nicely documented for the next time someone needs to do this.
Creating a Fedora ARM distribution part 2: Bootstrapping Posted Nov 8, 2011 22:15 UTC (Tue) by oak (subscriber, #2786) [Link]
There are also tools intended for bootstrapping & cross compiling distros.
Scratchbox1 (SB1) & Scratchbox2 (SB2) are such beasts. First one requires maintaining your own built tools "distro" which can easily bitrot & get out of sync with your distro, the latter can map tools to wherever you want to, typically this would be a directory containing x86 versions of your target packages (e.g. x86 debootstrap when cross-building Debian).
If your x86 host distro is the same one that you're trying to build for some other architecture, using SB2 so that it maps missing things to your host and letting everything else be run with Qemu, makes it much easier to build bootstrap stuff.
MeeGo has also some support from cross-compiling RPMs, but that's mainly to accelerate building, not AFAIK for bootstrapping purposes.
Here's some discussion on RPM-based MeeGo's OBS cross-build accelerators and SB2 usage:
SB2 seems to have just gotten some RPM support:
Creating a Fedora ARM distribution part 2: Bootstrapping Posted Oct 23, 2011 19:26 UTC (Sun) by kleptog (subscriber, #1183) [Link]
A lot of the circular dependencies come from non-essential parts of packages. For example: there are packages which build their manpages from TeX documents. To build TeX you need even more packages, but the manpages are non-essential and can be skipped for bootstrapping.
As pointed out elsewhere, Debian is working explicit bootstrap dependencies, which for each package will indicate the minimal dependencies required to build a useful (though not necessarily Debian-compliant) package. Hopefully these can be made almost completely loop-free so eventually bootstrapping can be automated.
Circular dependencies Posted Oct 24, 2011 9:13 UTC (Mon) by epa (subscriber, #39769) [Link]
Circular dependencies are not just a problem for cross-compiling. They can make life difficult for anyone building their own packages. I came across this when building a newer set of perl packages for Fedora. Although the packages from upstream (CPAN) do not have circular deps, the Fedora source RPMs do, because of the practice of adding in all possible dependencies as hard deps, even if they are not needed to build the module but only for optional bits of the test suite. (Or perhaps not even that - sometimes a module was not needed for 'make test' but only for 'make author_test', which downstream users and packagers should not need to run at all.).
It would be better to have a distribution policy to avoid circular hard build dependencies whenever possible. Admittedly they are sometimes unavoidable.
|
|||||||||||||