|
|
Subscribe / Log in / New account

OSTree for Fedora

By Jake Edge
January 23, 2014

Linux distributions are typically delivered either in piecemeal fashion via packages or as a single bundle like Android or Chrome OS. Colin Walters's OSTree project is taking something of a middle course. It is delivering a kind of bundle, but with a twist: the tool makes it easy to switch between multiple independent operating systems that are all simultaneously installed on the system.

We first looked at OSTree as part of our coverage of GUADEC 2012. Since that talk, though, Walters has not been idle. His gnome-continuous project has been building OSTree versions of GNOME using continuous integration for some time. Beyond that, he recently announced an OSTree repository for Fedora: fedostree:

I've often struggled with explaining OSTree to people - but for the audience here, I want to emphasize that OSTree is designed to be *complementary* to package systems like rpm/dpkg. While OSTree does take over some roles from RPM such as handling /etc, if you study it carefully, I think you'll come to agree.

The overall vision is to change Fedora (and really its prominent downstream) into a less flexible, but more reliable set of products, tested and delivered *much* *much* faster.

Rather than break things up into packages, OSTree builds fully assembled, versioned filesystem trees that hold what would have gone into a set of package (e.g. RPM) files. All of the binaries, libraries, documentation files, and so on are gathered together into the OSTree repository and can be retrieved using the ostree command. The fedostree installation instructions make it clear that the model Walters used was Git; many of the commands are clearly patterned after Git's. In fact, the project's home page calls it "git for operating system binaries".

For Fedora, one can install multiple different kinds of trees, based on either Fedora 20 or Rawhide, and switch between them at boot time. Each of the trees will have a focus on a particular project's output, such as GNOME, Docker, FreeIPA, or others. Those can be combined with a tree that provides the minimal F20 or Rawhide base to produce a system that can boot and run the project's output, and can do so repeatedly as each new version gets built from newer upstream code. Developers on those projects will then have a tool to quickly and easily switch between multiple versions of the project. That can make bisecting problems much easier, for example. Beyond the project documentation and Walters's announcement, he also describes some of the details of the tree switching in a blog post.

Each of the OSTree-installed systems will share the /etc directory; changes will be merged with those made by the other OSTrees. Also, /var will be shared between the instances. Beyond that, OSTree does quite a bit of filesystem rearranging to enable it to switch between the different trees. In the blog post, Walters gives some details:

Let’s try it; say you’ve installed the “fedostree/20/x86_64/base/minimal” ref. Now you want to change to the tree containing @standard along with docker-io. With the latest OSTree, this is just
# ostree admin switch fedostree/20/x86_64/server/docker-io

Semantically, this is somewhat like doing yum install @standard docker-io – except the transition is fully atomic. Furthermore, we get an exact replication of the tree from the build server; if for example we were transitioning from say the fedostree/20/workstation/gnome/core tree, then we wouldn’t get a mix of GNOME packages and docker-io. Our running GNOME tree would be untouched, still on disk, still running – when we reboot, then we’d be in the new server/docker-io tree. If we then did from inside the booted docker-io tree:

# ostree admin switch fedostree/20/workstation/gnome/core

Nothing would be downloaded (unless the ref was updated), and we’d be back to GNOME (again, atomically). All along this process, our changes to /etc will have a 3 way merge applied, and our data in /var will be carried along.

In addition to bisecting problems in broken umbrella projects (like GNOME or Mesa) that may not be easy to find otherwise, there are a number of other possible uses that Walters lists on the fedostree page. Installing RHEL and Fedora in parallel, or Fedora and Rawhide, would be one possibility. That would make switching between them far easier, while still keeping each up to date. It would also make testing Rawhide much safer since the only real risk is a filesystem-corrupting kernel bug. Making an Android-like recovery partition with a minimal install is another idea.

As he notes in the announcement (and elsewhere), fedostree (and OSTree itself) are rather "bleeding edge" at this point. He is looking for brave souls to evaluate the ideas, tools, and repositories to help him find problems. There is also a list of "development areas" that need to be addressed in the near future.

OSTree/fedostree is clearly a project that is just getting off the ground. OSTree has proven itself useful for GNOME continuous integration, so Walters plans to bring that to fedostree; trees would get rebuilt (and booted as a smoketest) immediately after a change is made that affects any package relevant to that tree. Before much of that can happen, though, some testing and shaking things out is required. It should be interesting to see where things go from here.



to post comments

OSTree for Fedora

Posted Jan 23, 2014 10:34 UTC (Thu) by paulj (subscriber, #341) [Link] (3 responses)

This sounds a very interesting, high potential project. :) Looking forward to seeing it develop and playing with it.

OSTree for Fedora

Posted Jan 23, 2014 14:06 UTC (Thu) by walters (subscriber, #7396) [Link] (2 responses)

Thanks for a good article! Just two comments:

First it is worth re-emphasizing this is quite early code. But when I got to the point of being able to usefully have Red Hat Enterprise Linux 7 beta in the / on my laptop, and dynamically multiboot Fedora in /ostree/deploy/fedostree so I can track development, I knew it had reached the point where it could be useful for quite a lot of people. So I do appreciate the early article =)

Second, I'd prefer to avoid the term "bundle" as it has negative connotations, and leads one in the wrong mental picture of how things actually work.

All of the trees transparently share storage in /ostree/repo via hardlinks. When you download a new tree, you're quite simply only downloading some metadata plus the files you don't have.

As an example, I could easily make a new tree shipped by the build server:

fedostree/20/x86_64/base/minimal-and-strace

Which contains @core and "strace".

If you downloaded this tree, you'd only end up doing approximately 11 HTTP requests for the content of the strace package (binares, manpage, files in /usr/share/doc), plus say about 20 very tiny HTTP requests for metadata.

Obviously, the potential number of trees to construct of this form are far, far too large to be shipped by a vendor like Fedora.

But this dynamic transparent shared storage between operating system allows scenarios like the following:

Imagine a downstream of Fedora taking the package set and creating their own custom trees (in a pre-OSTree world, images) to deploy to their servers (or workstations). Obviously, this happens a lot today with packages.

So this downstream puts up their own repository easily enough, and replicate that content to their own servers or clients.

Now imagine that they have a problem in the field, for example with the kernel, and come complain to upstream. It's very easy for them to replicate and boot into a "pure upstream" tree (say just the base/minimal tree), and try that out, without breaking their custom OS install.

Maybe it turns out that they had a mismatched version of some proprietary kernel driver, or some bad code overwrote the copy of /usr/bin/bash, or whatever. This sort of thing can be diagnosed much more easily, as you could tell that the minimal tree worked.

This type of thing is why I feel "bundling" is a bad term. I admit "bootable, immutable, versioned filesystem trees that share storage" is more of a mouthful though =)

OSTree for Fedora

Posted Jan 23, 2014 15:48 UTC (Thu) by jhoblitt (subscriber, #77733) [Link] (1 responses)

I've only skimmed the manual for a few minutes but OSTree sounds a lot like EUPS (https://github.com/RobertLuptonTheGood/eups -- probably unknown outside of HPC/scientific circles) scaled up to the complete OS.

Is it possible to install package updates on top of deployment or is there some sort of read only bind mount magic that would prevent that?

OSTree for Fedora

Posted Jan 24, 2014 4:21 UTC (Fri) by walters (subscriber, #7396) [Link]

Hi,

I'm finding the TeX hard to read by hand and I'm too lazy to process it locally, but from a quick skim, I suspect this system falls into the same category as another project from the HPC/scientific space, which is "CernVM fs". I just added a link/comparison to that one here:

https://wiki.gnome.org/Projects/OSTree/RelatedProjects

Basically if you don't *boot* off of it, it's in a different space compared to OSTree - it's an "application mechanism". Application mechanisms are obviously very useful in *combination* with OSTree, but OSTree does not come with or mandate any particular one.

Similar to olpc-update

Posted Jan 23, 2014 16:11 UTC (Thu) by martin.langhoff (subscriber, #61417) [Link] (1 responses)

OLPC uses olpc-update on its laptops running a custom Fedora Linux, an it is very similar to what's outlined here. Atomicity and low memory/cpu consumption are its main benefits.

As BTRFS matures, a path opens. The obvious thing is to snapshot before package installation, and roll back if it goes wrong. However, that upgrade is not atomic. With the same toolchain you can create a writable snapshot (effectively, "branch" your FS), apply the update there, and then switch root, atomically.

Similar to olpc-update

Posted Jan 23, 2014 17:42 UTC (Thu) by walters (subscriber, #7396) [Link]

I've added some text on olpc-update to: https://wiki.gnome.org/Projects/OSTree/RelatedProjects

(Please anyone feel free to jump in and tweak it for further accuracy)

One question - I don't see anything in olpc-update with respect to /etc. Does OLPC support writing to /etc?

Adding /usr/etc (holding the OS defaults) and doing a 3 way merge to compute the new /etc was a pretty essential thing to make OSTree still feel like Unix, I think.

If you're only keeping the original /etc, then you're not going to get new default config files, which is going to cause problems.

OSTree for Fedora

Posted Feb 2, 2014 4:13 UTC (Sun) by sitaram (guest, #5959) [Link] (2 responses)

I have to ask: why does the website *require* Javascript to show the text?

OSTree for Fedora

Posted Feb 2, 2014 4:49 UTC (Sun) by lsl (subscriber, #86508) [Link] (1 responses)

Because, apparently, without Javascript there is no website[1]. It's assembled by the Javascript code as you hit the page.

It gets the actual content from [2] and [3]. Opening those links directly gives me the content without JS. Actually, apart from minor font differences, it looks exactly the same, but with broken hyperlinks.

Probably I'm just missing something because no one would load nearly 1 MB of Javascript library code without it doing something very awesome. ;-)

Now, seriously: It's probably just that it's a prototype and no one thought about clients without JS yet.

[1] https://github.com/cgwalters/rpm-ostree/tree/master/fedos... (website source)
[2] http://rpm-ostree.cloud.fedoraproject.org/partials/home.html
[3] http://rpm-ostree.cloud.fedoraproject.org/partials/instal...

OSTree for Fedora

Posted Feb 2, 2014 18:36 UTC (Sun) by walters (subscriber, #7396) [Link]

It's because the website is a fork of gnome-continuous, and I plan to do the same sort of dynamic UI here that I do there ( http://build.gnome.org/#/gnome-continuous ), with build status, screenshots, etc.

Yes, a goal of rpm-ostree is to take screenshots even of say the base/minimal tree:

http://rpm-ostree.cloud.fedoraproject.org/local/smoketest...


Copyright © 2014, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds