OSTree for Fedora
Linux distributions are typically delivered either in piecemeal fashion via packages or as a single bundle like Android or Chrome OS. Colin Walters's OSTree project is taking something of a middle course. It is delivering a kind of bundle, but with a twist: the tool makes it easy to switch between multiple independent operating systems that are all simultaneously installed on the system.
We first looked at OSTree as part of our coverage of GUADEC 2012. Since that talk, though, Walters has not been idle. His gnome-continuous project has been building OSTree versions of GNOME using continuous integration for some time. Beyond that, he recently announced an OSTree repository for Fedora: fedostree:
The overall vision is to change Fedora (and really its prominent downstream) into a less flexible, but more reliable set of products, tested and delivered *much* *much* faster.
Rather than break things up into packages, OSTree builds fully assembled,
versioned filesystem trees that hold what would have gone into a set of
package (e.g. RPM) files. All of the binaries, libraries, documentation
files, and so on are gathered together into the OSTree repository and can
be retrieved using the ostree
command.
The fedostree
installation instructions make it clear that the model Walters used was
Git; many of the commands are clearly patterned after Git's. In fact, the
project's home page calls it "git for operating system binaries
".
For Fedora, one can install multiple different kinds of trees, based on either Fedora 20 or Rawhide, and switch between them at boot time. Each of the trees will have a focus on a particular project's output, such as GNOME, Docker, FreeIPA, or others. Those can be combined with a tree that provides the minimal F20 or Rawhide base to produce a system that can boot and run the project's output, and can do so repeatedly as each new version gets built from newer upstream code. Developers on those projects will then have a tool to quickly and easily switch between multiple versions of the project. That can make bisecting problems much easier, for example. Beyond the project documentation and Walters's announcement, he also describes some of the details of the tree switching in a blog post.
Each of the OSTree-installed systems will share the /etc directory; changes will be merged with those made by the other OSTrees. Also, /var will be shared between the instances. Beyond that, OSTree does quite a bit of filesystem rearranging to enable it to switch between the different trees. In the blog post, Walters gives some details:
# ostree admin switch fedostree/20/x86_64/server/docker-io
Semantically, this is somewhat like doing yum install @standard docker-io – except the transition is fully atomic. Furthermore, we get an exact replication of the tree from the build server; if for example we were transitioning from say the fedostree/20/workstation/gnome/core tree, then we wouldn’t get a mix of GNOME packages and docker-io. Our running GNOME tree would be untouched, still on disk, still running – when we reboot, then we’d be in the new server/docker-io tree. If we then did from inside the booted docker-io tree:
# ostree admin switch fedostree/20/workstation/gnome/core
Nothing would be downloaded (unless the ref was updated), and we’d be back to GNOME (again, atomically). All along this process, our changes to /etc will have a 3 way merge applied, and our data in /var will be carried along.
In addition to bisecting problems in broken umbrella projects (like GNOME or Mesa) that may not be easy to find otherwise, there are a number of other possible uses that Walters lists on the fedostree page. Installing RHEL and Fedora in parallel, or Fedora and Rawhide, would be one possibility. That would make switching between them far easier, while still keeping each up to date. It would also make testing Rawhide much safer since the only real risk is a filesystem-corrupting kernel bug. Making an Android-like recovery partition with a minimal install is another idea.
As he notes in the announcement (and elsewhere), fedostree (and OSTree itself) are rather "bleeding edge" at this point. He is looking for brave souls to evaluate the ideas, tools, and repositories to help him find problems. There is also a list of "development areas" that need to be addressed in the near future.
OSTree/fedostree is clearly a project that is just getting off the ground. OSTree has proven itself useful for GNOME continuous integration, so Walters plans to bring that to fedostree; trees would get rebuilt (and booted as a smoketest) immediately after a change is made that affects any package relevant to that tree. Before much of that can happen, though, some testing and shaking things out is required. It should be interesting to see where things go from here.
Posted Jan 23, 2014 10:34 UTC (Thu)
by paulj (subscriber, #341)
[Link] (3 responses)
Posted Jan 23, 2014 14:06 UTC (Thu)
by walters (subscriber, #7396)
[Link] (2 responses)
First it is worth re-emphasizing this is quite early code. But when I got to the point of being able to usefully have Red Hat Enterprise Linux 7 beta in the / on my laptop, and dynamically multiboot Fedora in /ostree/deploy/fedostree so I can track development, I knew it had reached the point where it could be useful for quite a lot of people. So I do appreciate the early article =)
Second, I'd prefer to avoid the term "bundle" as it has negative connotations, and leads one in the wrong mental picture of how things actually work.
All of the trees transparently share storage in /ostree/repo via hardlinks. When you download a new tree, you're quite simply only downloading some metadata plus the files you don't have.
As an example, I could easily make a new tree shipped by the build server:
fedostree/20/x86_64/base/minimal-and-strace
Which contains @core and "strace".
If you downloaded this tree, you'd only end up doing approximately 11 HTTP requests for the content of the strace package (binares, manpage, files in /usr/share/doc), plus say about 20 very tiny HTTP requests for metadata.
Obviously, the potential number of trees to construct of this form are far, far too large to be shipped by a vendor like Fedora.
But this dynamic transparent shared storage between operating system allows scenarios like the following:
Imagine a downstream of Fedora taking the package set and creating their own custom trees (in a pre-OSTree world, images) to deploy to their servers (or workstations). Obviously, this happens a lot today with packages.
So this downstream puts up their own repository easily enough, and replicate that content to their own servers or clients.
Now imagine that they have a problem in the field, for example with the kernel, and come complain to upstream. It's very easy for them to replicate and boot into a "pure upstream" tree (say just the base/minimal tree), and try that out, without breaking their custom OS install.
Maybe it turns out that they had a mismatched version of some proprietary kernel driver, or some bad code overwrote the copy of /usr/bin/bash, or whatever. This sort of thing can be diagnosed much more easily, as you could tell that the minimal tree worked.
This type of thing is why I feel "bundling" is a bad term. I admit "bootable, immutable, versioned filesystem trees that share storage" is more of a mouthful though =)
Posted Jan 23, 2014 15:48 UTC (Thu)
by jhoblitt (subscriber, #77733)
[Link] (1 responses)
Is it possible to install package updates on top of deployment or is there some sort of read only bind mount magic that would prevent that?
Posted Jan 24, 2014 4:21 UTC (Fri)
by walters (subscriber, #7396)
[Link]
I'm finding the TeX hard to read by hand and I'm too lazy to process it locally, but from a quick skim, I suspect this system falls into the same category as another project from the HPC/scientific space, which is "CernVM fs". I just added a link/comparison to that one here:
https://wiki.gnome.org/Projects/OSTree/RelatedProjects
Basically if you don't *boot* off of it, it's in a different space compared to OSTree - it's an "application mechanism". Application mechanisms are obviously very useful in *combination* with OSTree, but OSTree does not come with or mandate any particular one.
Posted Jan 23, 2014 16:11 UTC (Thu)
by martin.langhoff (subscriber, #61417)
[Link] (1 responses)
As BTRFS matures, a path opens. The obvious thing is to snapshot before package installation, and roll back if it goes wrong. However, that upgrade is not atomic. With the same toolchain you can create a writable snapshot (effectively, "branch" your FS), apply the update there, and then switch root, atomically.
Posted Jan 23, 2014 17:42 UTC (Thu)
by walters (subscriber, #7396)
[Link]
(Please anyone feel free to jump in and tweak it for further accuracy)
One question - I don't see anything in olpc-update with respect to /etc. Does OLPC support writing to /etc?
Adding /usr/etc (holding the OS defaults) and doing a 3 way merge to compute the new /etc was a pretty essential thing to make OSTree still feel like Unix, I think.
If you're only keeping the original /etc, then you're not going to get new default config files, which is going to cause problems.
Posted Feb 2, 2014 4:13 UTC (Sun)
by sitaram (guest, #5959)
[Link] (2 responses)
Posted Feb 2, 2014 4:49 UTC (Sun)
by lsl (subscriber, #86508)
[Link] (1 responses)
It gets the actual content from [2] and [3]. Opening those links directly gives me the content without JS. Actually, apart from minor font differences, it looks exactly the same, but with broken hyperlinks.
Probably I'm just missing something because no one would load nearly 1 MB of Javascript library code without it doing something very awesome. ;-)
Now, seriously: It's probably just that it's a prototype and no one thought about clients without JS yet.
[1] https://github.com/cgwalters/rpm-ostree/tree/master/fedos... (website source)
Posted Feb 2, 2014 18:36 UTC (Sun)
by walters (subscriber, #7396)
[Link]
Yes, a goal of rpm-ostree is to take screenshots even of say the base/minimal tree:
http://rpm-ostree.cloud.fedoraproject.org/local/smoketest...
OSTree for Fedora
OSTree for Fedora
OSTree for Fedora
OSTree for Fedora
Similar to olpc-update
Similar to olpc-update
OSTree for Fedora
OSTree for Fedora
[2] http://rpm-ostree.cloud.fedoraproject.org/partials/home.html
[3] http://rpm-ostree.cloud.fedoraproject.org/partials/instal...
OSTree for Fedora