A possible change of direction for Foresight Linux

Rune asked me to share some comments from a discussion we had.  He
noted that things have seemed slow in Foresight lately, and I think
it's obvious to everyone why -- despite having several people who
are formally recognized as Foresight developers, a lot of the core
has remained a one-man operation.  Ken and António both have shown
nearly boundless energy and commitment as core developers, and are
responsible for a lot of Foresight's success, such as it has been.
A few somewhat more specialized contributors have made truly
major ongoing contributions -- one of the easy examples to cite is
Pavel's ongoing OpenOffice.org work.  For several years, Justin
provided excellent kernel support.  And so on.  Most of the existing
Foresight developers fit into this second category.  I've been rather
on the fringe, honestly.  Perhaps that gives me more perspective.

What Foresight doesn't have is pure OS distro specialists.  This is
showing a lot right now, when nothing has shown up in a released
group on fl:2 for months.  António has made up for many manpower
deficiencies in the past by burning the candle at all five ends,
but that is not sustainable.

I'd like to propose a radical return to what Foresight has been
good at in the past, which necessarily implies deciding to walk
away from some of the things that the Foresight development
community has desired to become (sometimes with more success,
sometimes with less).

Originally, Foresight was about getting the latest versions of
cool new software to users quicker.  Since the point of the latest
versions is that they are changing rapidly, this necessarily implies
that the target users are those who accept and relish this change.
The technology that enabled Foresight was Conary: with Conary,
Foresight could be built on top of another distribution.  rPath
Linux provided that base on top of which Foresight could focus on
the new bits.

However, the fact that Foresight is rolling to new versions at
any time and rPath Linux is maintained as a classic stable
distribution, and that rPath has not found strategic value in
maintaining a constant stream of development updates for a set
of rapid releases of new versions of rPath Linux, has created
an immense amount of duplicated work in Foresight, outside of
the core competence and purpose of Foresight.  We saw this with
fl:1; it got better when fl:2 was closely based on rpl:2, and
now that fl:2 is barely related to rpl:2 we're back to progress
being very hard to attain.

I think it's time to propose a change.

Because I'm rPath's Director of Operating Systems, in charge of
rPath Linux, this may come as a big shock, but perhaps I'm the one
in the best place to say this: rPath Linux is not the right base
OS for Foresight.  rPath Linux is a great OS for the purpose for
which it was built, and delivers great value to rPath's customers
for building server-oriented application stacks that include a
versioned operating system -- in fact, it is based on demand from
those customers that rPath has concentrated on doing incremental
improvements to a stable OS base rather than new OS versions.
The development model of rPath Linux is too divergent from the
development model of Foresight to make it an appropriate long-term
base for Foresight.

Most of you know that rPath now provides multiple OS bases
by importing existing OSes built with other technology into a
Conary repository.  We have automatically-updated imports of SLES
10 and SLES 11 (both available by subscription only), CentOS 5,
and Scientific Linux 5.  We have imported a version of Ubuntu
Hardy as a functional proof of concept that we can improve based
on customer demand.  We have done some work on importing a few
other OS variants, too.  We do this with a tool that we can make
available as Open Source so that Foresight is not dependent on
rPath to maintain the import of another more frequently-updated
OS as a base.

I think that Foresight needs to be based on an upstream distro that
is regularly fully updated and refreshed, and that is maintained by
distro specialists with experience and expertise that is just plain
missing within the Foresight development community.  That distro
needs to be imported into a Conary repository; that will allow
Foresight to continue to use Conary to manage the process of building
a set of consistent modifications relative to that upstream distro,
providing a true rolling release.  That would allow Foresight
developers to concentrate on only the problems inherent in
integrating the very latest development source against a recent
base that is relatively close to the basis on which the software
is maintained.

If I were to make the decision of which distro to use as a base, I
would choose Fedora, not because I was the original Fedora leader,
but because rPath Linux has followed many Red Hat conventions, and
therefore the change would cause less turmoil on installed systems
and would require less ancillary work (for example, fewer changes
would be required to anaconda for installable ISO images).  However,
it's not infeasible to choose another distro that is rapidly updated,
such as Ubuntu; it would just take more work (*lots* of anaconda
development, a good bit of development on the OS importer tool; and
I don't know who has the time to step up to do that work).  For the
purposes of this proposal, I'll just say "Fedora" as a reasonable
proxy for whatever upstream the Foresight developers might choose.

I want to be clear that part of what I'm proposing is that Foresight
be built on a *binary import* of Fedora into a Conary repository,
updated with all security updates that are released for Fedora, with
only things that are clearly important for Foresight's unique vision
built separately for Foresight.  I suggest that to accomplish that,
the Fedora import should stand alone -- it should be useful per se.
That would mean that a user could install a Conary-managed Fedora
system, migrate to Foresight based on the same Fedora version,
migrate back to Fedora, and so on.  That would make Foresight's
differentiating value clearer, and it would be very easy to
understand what's different between Fedora and Foresight.

It would also mean that Foresight would be *rebased* to every
new Fedora version.  It wouldn't have to happen immediately on
the release of a new Fedora version, but when deciding whether to
deviate from upstream Fedora in Foresight, a specific Foresight
developer would either have to successfully argue that the deviation
is required only for a single upstream Fedora version, or explicitly
volunteer to maintain the deviation indefinitely.

In the Fedora-based Foresight universe I imagine, there would be
at least four sources for binaries that are included on a Foresight
system:
1) Packages included in a stable release of the base Fedora OS
2) Specific rawhide packages, either imported as binaries or rebuilt
3) Specific packages from third party repositories built for Fedora
   that are useful for Foresight but for whatever reason are not
   acceptable for inclusion in Fedora
4) Specific Conary packages that are newer versions of packages from
   Fedora or are unique to Foresight for whatever reason

I'd love this to be worked as a cooperative relationship with the
upstream project.  I see this as symbiotic, not parasitic -- the sort
of thing I would have wanted to encourage when I was the Fedora lead.

Am I crazy?