Day: GNOME OS [LWN.net]

Day: GNOME OS

Posted Aug 8, 2012 13:28 UTC (Wed) by walters (subscriber, #7396)
In reply to: Day: GNOME OS by nix
Parent article: Day: GNOME OS

Let's be clear: for nearly all of the stuff on the wiki, "someone" is me =)

The page you're referring to is the *long term* plan. It's doable over time.

There are a variety of shorter term approaches, and I have an OSTree branch that in fact started on a "stupid merge" strategy like what you're describing.

Anyways, if you are interested in coding instead of commenting in a textarea, you can also join ostree-list@gnome.org - thanks!

(Log in to post comments)

Day: GNOME OS

Posted Aug 8, 2012 16:33 UTC (Wed) by cmccabe (guest, #60281) [Link]

I realize you have good intentions, but do you really think creating another competitor to RPM and DEB is a good idea? I think it's just going to increase fragmentation.

It seems like the feature you want (atomic updates) could be retrofitted to the existing systems. btrfs supports the concept of transactions at the filesystem level, which is exactly what you need here.

I'm pretty sure that the RPM guys would be overjoyed to have someone sane take over as the upstream maintainer. Please, please, please, think before you reinvent the wheel.

Day: GNOME OS

Posted Aug 8, 2012 16:53 UTC (Wed) by walters (subscriber, #7396) [Link]

I describe the BTRFS issues here: https://live.gnome.org/OSTree/RelatedProjects

Basically however you slice it, you need a way to have two operating systems installed. Whether they're squirreled away in a BTRFS snapshot or unpacked as hard links is really an implementation detail.

The *real* challenges are around configuration management, how you present this to the system administrator, etc. For example, it's clearly desirable if the system software management tool can tell you stuff like "you have N OS images installed, the additional snapshots are taking up XX megabytes of disk space compared to the current".

To do that kind of thing, you need something aware of *both* the rpm/deb state and the BTRFS state.

Furthermore, one huge advantage of the current OSTree design (that I need to promote more on the wiki) is that it parallel installs inside your existing distribution - you don't need to switch disk formats. All you sacrifice is disk space.

You don't have access to that package of bioinformatics software in the latest GNOME OS? No problem, reboot into your regular host distribution and use it. Switch back when you want to see the latest GNOME.

Day: GNOME OS

Posted Aug 8, 2012 21:10 UTC (Wed) by cmccabe (guest, #60281) [Link]

There are two issues here:
1. Some package managers don't atomically put their new sets of files into place. This could cause problems in the event of an unexpected power cut.
2. Current package managers update application files while those applications are still open.

#1 can (probably) be solved just by batching everything up that rpm or deb does into a single btrfs transaction.

#2 is a little more subtle. Most update systems that I'm aware of don't even try to solve this problem. For example, Android forces you to restart your apps after you've upgraded them. (Application restarts are less of a big deal in Android, though, because of how they designed process lifecycle.)

In general, it seems sufficient to restart most applications. Maybe a flag could be introduced indicating whether the application needs to be auto-restarted, or whether it chooses to handle updates at the application level (by using inotfy, or by keeping a bunch of fds open like Chrome does).

Please don't bother re-inventing the chroot wheel for GNOME. I know how to set up chroots and virtual machines, thanks very much.

Day: GNOME OS

Posted Aug 8, 2012 17:01 UTC (Wed) by walters (subscriber, #7396) [Link]

Also, my GUADEC 2012 presentation covers some of this: http://people.gnome.org/~walters/guadec-2012-building-gnome/

(It's now on the wiki too)

Day: GNOME OS

Posted Aug 8, 2012 19:54 UTC (Wed) by nix (subscriber, #2304) [Link]

[Warning: this is a rant.]

As far as I can see, the 'stupid merge' strategy -- or a smarter one with custom merge drivers -- is the only sane approach. What's the alternative? As far as I can see, there is none other than some bizarre intent to avoid all systemwide configuration forever, which, as I said, is just not going to fly with virtually any upstream you don't already control.

Any system that retains configuration state in any way will retain the problem of merging it -- and systemwide state isn't the largest problem in any case, it's per-user state, which is far more mutable and often contains references to data that must not be lost (be that emails or financial info). What are you going to do, wipe everyone's dotfiles whenever a downgrade takes place? Keep the config with the earlier trees, so if you make a change after upgrading it's *lost* on downgrade? Neither seems even slightly sane to me.

What OSTree is proposing is somewhat unclear, but appears to require rebooting on *every single package upgrade* so as to switch into a new chroot containing that package. That means several times a day for me. Not a bloody chance am I letting something like *that* near any of my systems outside a VM: it is transparently ridiculous and optimizing for a very few programs that might need to take extra measures to avoid being broken by updates happening underneath them, like Chromium, yeah, already *did*, so it is clearly not impossible: the zygote is also not exactly rocket science, weighing in at well under a thousand lines of code).

Now maybe laptops don't mind being rebooted this often, but a server with NFS-mounted home and scads of daemons and remote users relying on it, no way. (And that's where I do most of my work, upgrading and downgrading everything but the kernel when necessary with nary a hiccup, and no reboot-associated state loss. Yes, I know this setup is 'obsolete' now in the brave new Web 3.0 world where everyone uses horrible proprietary web services rather than anything like shared filesystems, but it meant I and some friends could band together and buy one big RAIDed box with ECCRAM that none of us could otherwise afford, and it means lower power costs than each of us owning a medium-big box.)

What's bizarre is that you reference nixos, which does most of this right (and which I am not associated with in any way, despite the coincidence of names: nixos *does* do some things wrong, including, last time I looked, paying no attention to the fact that most shared library upgrades do not break compatibility) -- and then you go on to propose a system which is as far as I can tell worse in every way in which it differs from nixos. Why?!

Day: GNOME OS

Posted Aug 8, 2012 20:05 UTC (Wed) by walters (subscriber, #7396) [Link]

So you're just wrong about the systemwide configuration bit; concretely, it will absolutely be possible to change say /etc/passwdqc.conf and have that apply to every root you boot into.

On the "live updates" part: Absolutely nothing precludes having a mechanism which attempts to apply "live" updates from the new root to the currently running system. In fact, one could trivally reproduce the semantics that dpkg/RPM provide by just doing a "rsync --delete /ostree/trees/new/ /ostree/trees/current/" underneath the bind mount.

But the point is that comes *afterwards*. Make operations safe by default, and optimize later. But remember - there's lots of evil race conditions that happen on "live" updates as package managers do today.

Also remember that I fully intend to allow having a hybrid OSTree+package model, where you have a base system (basically ostree+dependencies) that must be updated atomically, but everything else can be mutated at runtime, if you like.