LWN.net Logo

Various notes on /usr unification

Various notes on /usr unification

Posted Feb 28, 2012 12:20 UTC (Tue) by Sho (subscriber, #8956)
In reply to: Various notes on /usr unification by pj
Parent article: Various notes on /usr unification

Actually I consider preupgrade a much better way to go. To recap, preupgrade works by downloading a bootable installer image, performing a dry-run and then adding the installer as the default option to the bootloader configuration. Then it reboots into the installer, performs the upgrade and reboots again.

The reason this is better is that the system that is being upgraded is offline during the upgrade. A lot of software doesn't support being upgraded while it's running Things can go subtly wrong when files get swapped out underneath running instances, or config files get rewritten by upgrade processes to fit the new version and then the old version overwrites the changes when it's flushing state to disk while quitting. "Being replaced while running" is a scenario very few projects test for.


(Log in to post comments)

Various notes on /usr unification

Posted Feb 28, 2012 13:06 UTC (Tue) by khim (subscriber, #9252) [Link]

"Being replaced while running" is a scenario very few projects test for.

Every rule has an exception. Google Chrome (and Chromium) are quite explicitly developed for such usage. Of course "silent upgrades" are their core design decision thus they have to support them. Other projects... are less forgiving. Our Linux support team likes to push upgrades every time you connect [loaned] laptop to a corp network. Firefox often becomes unusable as a result :-( Usually restart fixes it, but once or twice (in last five years) it corrupted profile beyond repair.

Various notes on /usr unification

Posted Feb 28, 2012 13:25 UTC (Tue) by Sho (subscriber, #8956) [Link]

One large contingent of software that potentially has problems with in-place upgrades is any KDE application. This is only very, very rarely a real problem in practice, though, so there's no reason to panic. Still, it's good to be aware of it:

Basically, when code evolves, the format of its configuration file will sometimes change. One way to handle existing user config files using the old format is to retain support for reading the old format, and then write out the new format later, and that's indeed what apps often do. However, the KDE platform also contains an alternate mechanism called "kconf_update". With kconf_update, the application developer supplies scripts that will be run to transform a config file from one format into another (either in a simple DSL or something free-form specifying the interpreter), and a section in the config file is used to keep track of what scripts have already been run.

The problem lies in the "will be run" part. KDE sessions run a process called kded, the KDE daemon, which plays host to lightweight plugins that perform a variety of tasks, from holding binary caches of things like the app db to disk space warnings, etc. One of those plugins also watches directories for newly-installed kconf_update scripts and runs them immediately as it discovers them.

However, KDE apps also tend to flush their config out to disk while quitting.

You can see where this goes:
1. The package manager installs a new version of $app that ships a kconf_update script while the user is logged into KDE and has $app running.
2. The kconf_update plugin in kded notices the new script and runs it, reformatting $app's config file.
3. The user quits the currently-running old version, which flushes its config out to disk as it shuts down, potentially overwriting changes made by the script.
4. The user now runs the new version which may now run into problems reading its config file.

The reason this is very rarely a problem in practice is because config file format changes are rare to begin with, and the changes that do happen often include changing of the config key names, so the number of actual conflicts that occur is very low. But it's a possibility.

And yes, this is definitely a case of bad design.

Various notes on /usr unification

Posted Feb 29, 2012 11:53 UTC (Wed) by Lennie (subscriber, #49641) [Link]

I would change the path of the config file and only look at the new path first in the newer version of the program.

Because otherwise things will just go wrong.

Especially if you have several machines with /home on NFS or whatever which don't get upgraded at the exact same time.

Various notes on /usr unification

Posted Feb 28, 2012 13:08 UTC (Tue) by drago01 (subscriber, #50715) [Link]

Yeah I don't see preupgrade as worse but actually as a better solution.

Updating the whole OS while it is running sounds to error prone to me (it can work but it is kinda fragile).

Various notes on /usr unification

Posted Feb 28, 2012 15:17 UTC (Tue) by sorpigal (subscriber, #36106) [Link]

It may sound error prone but in practice I've had incredibly few issues, not with servers or workstations. I do make sure I shut down outside access for servers to prevent user access during an upgrade. Sometimes you get warnings like "You should shut down X first," which I ignore unless a video driver is involved, and even doing this have had no more than a small handful of issues in the last decade.

It seems like we accidentally managed to make this work nearly without flaw.

Various notes on /usr unification

Posted Feb 28, 2012 15:21 UTC (Tue) by Sho (subscriber, #8956) [Link]

Thing is, though, you're power user who is equipped to deal with the breakage in the event that it does arise, so it's a calculated risk for you. A distro may legitimately decide to care about serving users who aren't equipped for it. And in the face of the possibility of breakage, an upgrade process that tries to avoid it sounds like a good idea.

Various notes on /usr unification

Posted Feb 28, 2012 23:37 UTC (Tue) by cmccabe (guest, #60281) [Link]

If things get broken by the upgrade, users would simply reboot. And if the issue was the version skew you're worried about, that would fix it.

Does knowing how to press the power button make you a "power user"? If so, I weep for the future of computing.

Various notes on /usr unification

Posted Feb 29, 2012 0:57 UTC (Wed) by mgedmin (subscriber, #34497) [Link]

If the upgrade crashes halfway through, there is a chance the machine won't boot.

A power user might fix it (dpkg --configure -a; apt-get install -f; apt-get dist-upgrade).

Various notes on /usr unification

Posted Mar 4, 2012 6:25 UTC (Sun) by cmccabe (guest, #60281) [Link]

And if a Boeing 747 crash-lands on top of the computer, that would probably be bad too. But how is it related to what we were talking about?

Just to be clear, the issue we were talking about was version skew in libraries causing applications to misbehave during the upgrade.

Various notes on /usr unification

Posted Feb 29, 2012 13:21 UTC (Wed) by sorpigal (subscriber, #36106) [Link]

You're not wrong, but I find myself wondering whether, if we've come this far without really trying, it might not be worthwhile to test for this usage and make the "unequipped" users able to do it in relative safety.

Various notes on /usr unification

Posted Mar 4, 2012 17:45 UTC (Sun) by dirtyepic (subscriber, #30178) [Link]

> A lot of software doesn't support being upgraded while it's running

I'm going to have to call bullshit on that one. See every rolling distro ever.

> or config files get rewritten by upgrade processes to fit the new version
> and then the old version overwrites the changes when it's flushing state
> to disk while quitting

Well, first of all, don't overwrite config files. Store the new config somewhere else and give the user an interactive diff tool. Second, any service that writes state to a file that would get overwritten on a reinstall is broken. Any service that can't recognize state information from its previous version is broken.

> "Being replaced while running" is a scenario very few projects test for.

Because it rarely causes problems. The majority of Gentoo systems are upgraded continually while in use. The biggest problem I had after upgrading and recompiling every package installed on a system that hadn't been updated in six months, which included major gcc and glibc updates, was Firefox wouldn't open links sent to it from external programs until I restarted it.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds