Distributions

Distributors ponder a systemd change

Linux users tend to pride themselves on their position at the leading edge of a fast-moving development community. But, in truth, much of what we do is rooted in many decades of Unix tradition, and we tend to get grumpy when young developers show up and start changing things around. A recent change of default in systemd represents such a change and the kind of response that it brings out; as a result, Linux distributors are going to have to make a decision on whether they should preserve the way things have always worked or make a change that, while potentially disruptive to users, is arguably a step toward more predictable, controllable, and secure behavior.

Killing processes on logout

Systemd makes extensive use of control groups to manage processes on the system; because control groups contain processes and any subprocesses they may create, systemd has a clear view of the full set of processes that belong to each service or session. That allows it to, for example, kill all of the processes belonging to a given service, regardless of whether those processes were started directly by systemd. When it comes to user sessions, systemd (or, rather, the logind component) has, for some time, had the ability to kill all of a user's running processes when that user logs out.

That capability is controlled by the KillUserProcesses configuration option to logind; by default in older systemd releases, that option was set to "no". The systemd 230 release announcement included, among many other changes, the news that the default had been changed to "yes," so that, absent other measures, users cannot leave processes running after they log out of the system. This change quickly found its way into the faster-moving development distributions; the unhappy cries from users were not far behind.

The problem, from the point of view of these users, is that systemd 230's behavior represents a significant change in how Linux systems work, and that this change will surprise people in unpleasant ways. Prior to this change, processes could easily be made to persist after the user logs out; it was just a matter of blocking the SIGHUP signal that is delivered when the controlling terminal goes away. That could be done by running the process in the background on C shell variants, or with the nohup command for Bourne shell variants. A program could also take control of its own fate by managing SIGHUP directly.

In the new scheme of things, life is not so simple. Blocking SIGHUP will no longer work, since systemd does not use that signal; instead, it sends SIGTERM followed by an unblockable SIGKILL. Thus, programs that have traditionally set themselves up to survive a logout (screen and tmux are often-cited examples) can no longer do so and will not be able to perform their intended function.

There are ways around this problem, of course. One is for the program to tell systemd directly that it needs to persist. That involves accepting the systemd library as a dependency. Not all projects are willing to add this kind of systemd-specific code; see this tmux tracker entry, for example. So it may be some time before even the programs that are explicitly intended to run after logout are able to work transparently in this manner.

Beyond that, most programs are not designed for this kind of persistence, but can be used that way anyway; it is not uncommon to place a long-running task in the background and expect it to complete after logging out. In the new systemd world, these programs need to be run with the systemd-run command. Creating a version of nohup that invokes systemd-run should not be a hard thing for a distributor to do, so there should be relatively little disruption in cases where users explicitly ask for persistence already.

The only remaining snag is that the user account must have "lingering" enabled for persistence requests to actually be honored. This is done with the "loginctl enable-linger" command, but is turned off by default. Enabling lingering can be configured as an unprivileged operation, or it can be restricted to system administrators. In the end, the ability to run processes that persist after logout has not gone away, but the way things behave by default has changed.

Arguments pro and con

It has been suggested that this change has been made to cope with GNOME applications that won't go away without a SIGKILL-sized hint. There may be some truth to that, but it also appears that all desktop environments have at least some problems with unwanted persistent processes. Such processes don't just clog the system; they can also delay the availability of the console for another login or slow down the shutdown process. It is easy to say that such programs should simply be fixed, but, in the real world, sometimes one has to stop playing whack-a-mole and just pave the field instead.

Beyond that, systemd creator Lennart Poettering sees process persistence as a security issue. Allowing somebody to log into a machine should not imply allowing that user to run code when they are not logged in; that is a separate decision that should be reserved for the administrator. Even if the change creates some trouble, he believes that somebody needs to do it:

One need not look far to find criticism of Lennart's code, attitude, ancestry, or hair style, but one would be hard put to find accusations that he (or the systemd development community as a whole) is unwilling to dare to make disruptive changes. The community was certainly up to the challenge this time, and so the change was made.

This change has clearly already surprised a number of users; see this Debian bug or this fedora-devel thread for examples. Some of the discontent can be seen as the sort of "but that's not how we've always done it" screaming that seems to follow everything that happens in the systemd space, but that is not the whole of it.

Most people who have looked at how systemd works in this area seem to think that the ability to ensure that no user processes persist after logout is useful. But many object to turning it on by default, especially in a setting where most programs that need to be persistent have not been updated to work in the new regime. As Chris Adams put it in the Fedora thread:

It seems fairly clear that there would be less resistance to this change if common patterns still worked without the need for users to notice the new rules and make explicit changes. On the other hand, without the pressure represented by this change, the work to make programs operate properly might never actually get done. On yet another hand, there is little evidence of the systemd developers submitting patches to fix popular programs before making this change.

Distributor response

In the Linux world, upstream projects do not normally have the last word when it comes to system behavior; that privilege (and responsibility) falls to distributors. So the logout behavior seen by users may differ from that chosen by the systemd development community. The default as shipped by that community can be changed by distributors in a couple of ways. One is to build logind with the --without-kill-user-processes flag to restore the default to "no"; that is what Arch Linux and Gentoo have chosen to do, for example. The other is to set the KillUserProcesses option to "no" in a distribution-supplied logind.conf file; that would appear to be the outcome of the Debian discussion. [Correction: in truth, Debian seems to have taken the build-time flag approach as well.]

Other distributions that use systemd will still need to make a decision on when — and if — they will pass this change on to their users. The Fedora discussion has had few clear outcomes so far, with one exception: this change certainly will not appear in Fedora 24, which, having slipped three times, is now due on June 14. It would appear that the chances of this change showing up in Fedora 25 are fairly high, though. Some of the other community-oriented distributions have yet to hold their discussion on this issue, but, if past history is any guide, the lengthy threads will eventually appear.

Enterprise-oriented distributions have more time to think about it, of course; by the time they have to make a decision, it may well be that most of the known problems have been fixed. This may be the setting where the default behavior matters the most, though. In 2016, logging out of a desktop system is relatively rare; users are far more likely to lock the screen, power down entirely, or suspend the system. Logging into a server, starting a long-running process, and logging out is a much more common pattern. So the enterprise Linux user base, which is arguably more focused on servers, may care more about this issue.

For all the fuss, one might well argue that the development community is working as it should. Upstream projects should push toward their view of a better world, making changes where they can even if those changes sometimes go against longstanding tradition. Distributors, instead, are responsible for delivering stable, working systems to their users; most of them would prefer to avoid shipping unpleasant surprises. So it is fitting that systemd pushes things in a direction that its developers see as being cleaner and more secure, but it is also fitting that distributors hold off on shipping changes until they work well. The process is noisy, but the results are often good.

Comments (275 posted)

Distribution quotes of the week

-- Holger Levsen

-- Patrick Lauer (Thanks to Alexander E. Patrakov)

Comments (1 posted)

Open Build Service 2.7 released

Open Build Service 2.7 has been released. "Three large features around the topic of integrating external resources made it into this release. We worked on automatic tracking of moving repositories of development versions like Fedora Rawhide, distribution updates or rolling Linux releases like Arch. A change to the OBS git integration to enable developers to work on continuous builds. And last but not least an experimental KIWI import that can be used to easily migrate your images from SUSE studio."

Full Story (comments: 1)

Maru OS now freely available

The Maru OS handset distribution (reviewed here in April) has moved out of the beta-test period and is now freely downloadable without an invitation. Maru functions as both an Android handset and an Ubuntu desktop (when connected to an external monitor). For now, it remains limited to Nexus 5 handsets. "Now that the beta program is over, I’m finally turning my attention to the open-source project so we can expand device support with the help of the community. Let’s get Maru in the hands of a lot more people!"

Comments (2 posted)

Debian stable updates released

Debian 7.11 is the eleventh (and final) update of its oldstable distribution Debian 7 "wheezy". The Debian Long Term Support Team will continue to support Wheezy until May 31, 2018.

Debian 8.5 is the fifth update of its stable distribution Debian 8 "jessie". "This update mainly adds corrections for security problems to the stable release, along with a few adjustments for serious problems. Security advisories were already published separately and are referenced where available."

Comments (8 posted)

Distribution newsletters

• Debian Misc Developer News (#41) (June 6)
• DistroWatch Weekly, Issue 664 (June 6)
• Lunar Linux weekly news (June 3)
• openSUSE Tumbleweed – Review of the Week (June 3)
• Ubuntu Kernel Team newsletter (June 7)
• Ubuntu Weekly Newsletter, Issue 468 (June 5)

Comments (none posted)

Alpine Linux Desktop (DZone)

DZone takes a look at Alpine Linux on an older laptop. "The default installation is reasonably functional for console use. Even vi is present, though it is provided by Busybox and therefore doesn't contain all the glorious features to which I'm accustomed. I did have to immediately go back through setting up the wireless, as expected, but fortunately the changes are permanent. The most interesting part of getting started was to decide what to install. The notion of performing installations of bash, grep, awk, sed, and other common tools seem a little strange since we just automatically expect them to be present in non-Busybox form. The best part about the whole experience is that it feels like using Linux from five (or twenty) years ago, but with completely new and modern versions of things. It reminds me of my first Slackware install back in the mid-90s, but instead of feeding floppies I'm installing Git 2.6 from a fast Internet connection over a wireless network."

Comments (none posted)