The ARM architecture is growing in popularity and is expected to expand its
reach beyond the mobile and "small embedded" device space that it currently
occupies. Over the next few
years, we are likely to see ARM servers and, potentially, desktops.
Fedora has had at least some ARM support for the last few years, but
always as a secondary architecture (SA), which meant that the support lagged
that of the two primary architectures (32 and 64-bit x86) of the
distribution. Recently, though, there has been discussion of "elevating"
ARM to a primary architecture (PA), but, so far, there is lots of resistance to
a move like that.
The subject came up at a meeting of the
Fedora engineering steering committee (FESCo) on March 19. Adding ARM as a primary
architecture for Fedora 18 was a late addition to the agenda, which annoyed some, but
the discussion was largely to "start the ball rolling and collect feedback from
everyone", as Kevin Fenzi put it.
There will be many other opportunities to discuss the idea, he said.
log bears that out as the only vote taken (or even proposed) was to ask
for input from various teams (QA, release engineering, kernel, and
infrastructure) about the impact of a change like that.
The difference between primary and secondary architectures for Fedora is
rather large. Releases cannot be made without all of the packages building
and working for each primary architecture, whereas secondary architecture
packages can languish. In fact, the current release of Fedora for ARM is
based on Fedora 14—though there are alphas of Fedora 15 and
17—which is past its end-of-life on x86.
The meeting discussion focused mostly on the motivation for making ARM a
PA and why the project's goals couldn't be met, at least for now, by
remaining as an SA. Much of the motivation, it would seem, is for Fedora
to get out ahead of the curve on ARM support. Making ARM a "first class
citizen" would increase its visibility and put the full weight of the
Fedora community behind the effort. Therein, it seems, lies part of the
One could argue that Fedora has already fallen behind the curve with
respect to ARM given what Ubuntu and Debian are doing to support the
architecture. There is a lot going on with Linux on ARM, and Fedora may
well find itself becoming less relevant if support for ARM does not
improve. But the question seems to be whether that support needs to
improve as an SA before even considering whether it can be a new PA.
Based on the discussion in the meeting, Matthew Garrett posted an RFC draft of the requirements to
promote an architecture to a PA. So far, there has been no architecture
that has transitioned from an SA to a PA, so some kind of ground rules need
to be established.
Garrett lists seven potential criteria,
Promoting an architecture to primary architecture status is a
significant responsibility. It implies that the port is sufficiently
mature that little in the way of further architecture-specific changes
or rebuilds will be required, and also that it has enough development
effort to avoid it delaying the development of other primary
architectures. Further, it means that the architecture becomes part of
the overall Fedora brand. Fedora is an integrated Linux distribution
rather than a technology collection, and as such there are various
expectations that the overall Fedora experience will be consistent over
all primary architectures.
Much of the response to that posting concerns the amount of
time it (currently) takes to build ARM packages (vs. the time for x86 and
other architectures). Jakub Jelinek noted that GCC builds for 64-bit x86 are on
the order of two hours, while building for ARM takes much longer. He
followed that up with actual numbers from
GCC 4.7 builds for Fedora 17, which ranged from one-and-a-half hours for
x86_64 to more than 26 hours for armv5tel (and more than 24 for armv7hl).
Brendan Conoboy pointed out that plans for newer "enterprise
would cut those ARM build times in half, but that still leaves a
Slow builds are not just an annoyance, as there are some impacts for the
distribution if package building takes "too long".
Josh Boyer lists
two. If a package builds for the x86 family, but then fails to build
for ARM, the x86 build will have to be resubmitted after the ARM problem is
fixed. In addition, when trying to do an update (for a security issue for
example), it has to wait for the slowest build to finish before the update
can go out. Adam Williamson also notes
another problem that could arise in the release verification process:
So it's not unusual for me to be bugging, say, the kernel team to give
us a new kernel build that fixes a blocker bug, so we can do a new
release candidate, so we can test the release candidate in twelve hours,
so we can make the go/no-go meeting deadline the next morning.
If builds get significantly slower, that could have a concrete impact on
the release validation process: it's plausible that we'd either need to
extend the validation period somewhat - earlier freezes - or we would
have to eat a somewhat higher likelihood of release slippages.
Build speed is a technical issue that can presumably be overcome
(eventually) with faster hardware. Other possibilities like cross-compiling
on faster x86_64 servers or parallelizing the Koji build system (perhaps
using something like distcc)
seem to have been ruled out by Fedora release engineering or the Fedora ARM
team. While some remain unconvinced, Conoboy is adamant that cross-compilation is not a good
Look, even x86_64 is
topping out on speed and moving to a more-core and more-systems-per-rack
model. Cross compilation solves yesterday's problem, not tomorrow's. If
build speed truly is a fundamental issue to becoming PA the answer is to
harness multiple systems for a single build, not to use a somewhat faster
system to make up for the speed of a somewhat slower system. Scaling across
more cores than fit in a single SMP Linux environment is the only sensible
approach to future build speedups. Though [it] is an interesting challenge, it
is completely beyond the scope of primary architecture requirements.
But some question the wisdom of even having criteria for promoting SAs to
PAs, whether it makes sense for Fedora to even consider ARM for a PA, or
both. Kevin Kofler is definitely in the last category as he believes that the current list of PAs "should be set in stone unless a MAJOR change in hardware
landscape happens". Some would argue that the change is already
happening. But he is concerned that additional PAs put a
burden on all of the package maintainers, so that it should require an
extraordinary event (like "x86 gets discontinued by the hardware manufacturers
and everyone uses ARM instead") before any change like that is even
considered. He continues:
In the current state of things, I don't see a sufficient demand for making
ARM (or even less any other secondary architecture) a primary architecture.
If ARM is really the future as its proponents claim, we can revisit that in
a few years. Not now.
The focus should be on finding ways to make secondary architecture releases
more timely (i.e. it's not acceptable that e.g. the stable ARM release is
still Fedora 14 which doesn't even get security updates anymore), not to
cheat around the problem by making ARM a primary architecture (which does
not help all the other secondary architectures).
Kofler harps on the same points throughout the thread,
belittling the ARM market share (at least in the market segments that he
thinks should be targeted) and finding the build times for ARM packages to
be untenable. He considers the large
existing base of ARM devices to be unsuitable for installing Fedora, at
least at this
point. But, as Richard W.M. Jones points
out, that is changing rapidly:
It's a matter of time, and not very much time at that.
My £400 tablet has plenty enough power, storage and whatever else to
run Fedora. Fedora works pretty well on £200 Trim Slice servers.
Fedora is going to be shipped with £25 Raspberry Pi devices in the
Others were also skeptical of the current ARM hardware being a good target
for Fedora, but Williamson points out that
getting Fedora ARM running does more than just target
those devices. The ARM project is looking toward the future, both on servers
and mobile devices. Getting the distribution running on one is a big step
toward having it available for the other.
But the speed of the build system is just one symptom of the problems that
another PA will bring. One of the bigger questions, which remains largely
unanswered as yet, is what making ARM a PA would do for Fedora as a
distribution. It's reasonably clear why it would help the Fedora ARM
project to have ARM as a primary, but the advantages to the distribution,
at least at this point, are less clear. As Garrett put it:
primary architecture isn't meant to be a benefit to the port - it's
meant to be a benefit to Fedora. Adding arm to the PA list means you'll
have to take on a huge number of additional responsibilities, deal with
more people who are unhappy about the impact upon their packages and so
on. You get very little out of it except that there's more people to
tell you that something's broken. The question is whether making arm a
primary architecture makes Fedora a better distribution, and yes, in an
ideal world arm would demonstrate that it was just as functional as x86
before we had to make that decision.
The only reward you'll get from being a primary architecture is basking
in the knowledge that the project thinks your work is good enough to be
a primary architecture. The better you can demonstrate that in advance,
the easier the process will be.
Jones Robinson outlined many of the advantages
that the Fedora ARM team sees in another fedora-devel thread. Essentially,
it would spread the load of responsibility throughout the Fedora
community. That is, of course, the underlying concern of many posters in the
threads. But Jones sees it this way:
I'm fully aware that Primary Arch isn't the perfect panacea and that
once we're there the ARM team can't go and sit of the beach and do
nothing but it does spread the load, automate a lot of things because
it's part of core infra and processes and it then allows the ARM team
to concentrate on fixing corner case packages, working with various
components of the project, optimising the way things are done for ARM,
working with upstreams for HW etc and generally making ARM even better
rather than constantly chasing our tail trying to keep up with basic
things as building packages, running infrastructure, composing the
repos, dealing with branching and tags and targets in koji and all
those other things. Those advantages of being a primary arch can't be
There is, of course, nothing stopping the ARM team from achieving most of
its goals while staying as secondary architecture. It will be more
difficult and likely require more volunteers, but the Fedora project as a
whole needs to be convinced of the
advantages of taking on the "burden" of ARM as a primary. So far, the ARM
project doesn't seem to have made a convincing case for that, but, given
the importance of the architecture going forward, one might guess that the
situation may change in the next year or two. In the meantime, setting
some goal posts for any secondary architecture that wants to be promoted
seems like a good first step.
to post comments)