|| ||Robert Schweikert <rjschwei-AT-suse.com> |
|| ||osuse-factory <opensuse-factory-AT-opensuse.org> |
|| ||Fiddling with our development model |
|| ||Tue, 30 Jul 2013 13:08:50 -0400|
|| ||Article, Thread
This is NOT board intervention, this is just me ;)
While at present things appear to be reasonably smooth sailing some
(including me) think that there is trouble brewing. Thus, before things
move into a more troublesome direction it is probably a good thing to
think about changes that could avoid wreckage.
Behind the "smooth sailing" appearance is a ton of work that is ever
increasing and getting ever more complex. The current push model leads
to problems we experience every time a new gcc version shows up, a bunch
of stuff breaks and only 2 or 3 people fix things. While the use of
staging projects "hides" the problems from Factory users, the problems
remain the same. The work for the most part, ends up with the release
team as they are the integrators. Partially the behavior of non release
team members is understandable as for the most part getting your package
build broken by others is annoying at best, and it generally happens at
an inconvenient time. Partially the problem may also be that people
plain and simply do not get the information about their package being
broken in factory.
With this as a short background I'd like to propose a change in
development model for our beloved distribution. Everything needs a
catchy title, thus I'll call it "Integration of independently released
components with pull option", very catchy I know ;) .
The idea is that we define components, this is going to be a chunk of
work, and for each component we define it's dependencies. The component
model is not set in stone and would get refined over time to reduce
potential overlap and improve interoperability. A component is developed
by a team and you can think of a component as a "mini factory replica"
(factory as we know it today.) The component team works on releases of
their component according to some common and some component specific
guidelines we'd have to define and publish. Any component consists of
one or more packages and may get fed by devel projects or development
may take place in component branches. Each component advances based on
it's own release cycle and the component team decides which release of
it's dependent components to build against. Some tooling will be needed
to help sort out the combinatorial problem.
How does this work in more concrete terms? Lets say we have a "Core"
component that includes the kernel, glibc, the toolchain and a few other
things, but no fluff ;) . This component then gets released every time a
new kernel is released and every time a new gcc or glibc are released,
or based on some other criteria the core team determines and publishes.
Every release gets a number. When a new release of the core component
happens nothing breaks because by default nothing builds against the new
release yet. Component teams higher up the stack decide against which
release of a dependent component they want to build. This puts the
"adopt a new toolchain" decision into the court of the component teams
rather than having it in the realm of the release team, i.e. the
integrators, or the realm of the toolchain team. Therefore, a component
team decides on it's time scale when to build against a new release of a
dependent component. This concept works whether a given component
depends on one or many other components. Obviously there needs to be
some notification mechanism that lets dependent component owners know
when a new release of one of their dependent components is available.
There also needs to be some tagging mechanism for releases and other
mechanisms of notification and tracking.
A simple example:
Lets say component A depends on component B. Component A is released as
version 1 (A-1) and component B is also released as version 1 (B-1). Now
component team B decides to have release number 2 (B-2) and thus
component team A gets notified. Component team A can now create a branch
of component A and start building against B-2. If the changes to get A-1
to build against B-2 are trivial and A-1 can work with both B-1 and B-2,
component team A merges the changes back from the branch and tags A-1 as
working with B-1 and B-2. If the changes to A are extensive component
team A may decide to have a new release, tag it as release 2 (A-2) and
tag it to be working with B-1 and B-2 or only with B-2 if that is the case.
These same principles apply for components with multiple dependencies,
with the caveat that things get a bit more complex, still less
cumbersome of course than the battles the release team has to fight day
in and day out in today's model.
This change does imply the end of factory as we know it today. Factory
may be "empty" for a good chunk of the release cycle. The replacement
for "living on the edge" developers is a set of component selections
that can keep you in "alpha mode" all the time. When a release
approaches,the release team picks a version of the core component that
will be the base for the next openSUSE release. This version is used to
populate factory and may be the only component that is in factory for a
little while. As other component teams integrate and build against the
version of their dependencies that end up in factory and make their own
component releases the openSUSE release team can fill factory by pulling
more and more per-integrated components. With things per-integrated at
the released component level the work of the openSUSE release team
becomes easier and sustainable.
The underlying assumption is that component teams have an interest in
having their component be part of the openSUSE release and thus
component teams will have an interest in building against new releases
of dependent components. This underlying assumption is not much
different from today's.
What do we gain from all of this? Interesting work of course ;) , but we
also switch to a model that is sustainable in the long run. I am not the
only one arguing that the work the release team is doing in its present
form cannot be sustained with a growing distribution. The work has to be
distributed in some way, and more people doing the same thing that is
being done today is probably not the answer. We also gain a unique and
interesting development model, that with some additional tooling will
allow developers to generate a distribution that is tailored to exactly
their need. This in turn should stir interest give us some nice
publicity and help us find more fingers on the keyboard to help grow the
developer community and with it the distribution.
With the independent release of components one can easily imagine an
interface where a developer picks a component at a given release level,
gets choices of versions of dependent components to select and so on. In
the end we can relatively easily generate a tailored "release". This
also works the other way around, pick a released version of the core and
the interface shows you other components that match this core and so on
up the stack.
Lots of work, yes, but in the long run we have to make some changes.
Thus, the sooner we start the easier it will be.
Robert Schweikert MAY THE SOURCE BE WITH YOU
SUSE-IBM Software Integration Center LINUX
To unsubscribe, e-mail: firstname.lastname@example.org
To contact the owner, e-mail: email@example.com
to post comments)