Thanks [LWN.net]

Thanks

Posted Nov 23, 2014 7:31 UTC (Sun) by smurf (subscriber, #17840)
In reply to: Thanks by mgb
Parent article: Today's Debian technical committee resignation: Ian Jackson

How does socket-activating sshd increase any likelihood of problems?

I'd assume that this would decrease it, since this way you do not risk starting it too early (/var/log not yet mounted, or whatever else might cause sshd to fail to start), nor too late (i.e. not at all, if something earlier in the boot sequence fails)

to post comments

Thanks

Posted Nov 23, 2014 7:42 UTC (Sun) by mchapman (subscriber, #66589) [Link] (18 responses)

> How does socket-activating sshd increase any likelihood of problems?

If the system is close to being out of memory, then I would expect a socket-activated SSH to be less likely to get you to a shell than a non-socket-activated SSH.

If your only remote access to the system is with SSH, and remotely rebooting it is infeasible or impractical, then this could be a concern.

Thanks

Posted Nov 23, 2014 9:01 UTC (Sun) by mgb (guest, #3226) [Link] (12 responses)

Also for handling upgrade problems.

Once your sshd process exists you don't want it to disappear unexpectedly.

Thanks

Posted Nov 23, 2014 9:29 UTC (Sun) by mchapman (subscriber, #66589) [Link]

No, I don't think socket-activation changes anything there. When you upgrade SSH, the package's post-installation scripts should cause it to be restarted if it's running as a non-socket-activated service.

Thanks

Posted Nov 23, 2014 16:37 UTC (Sun) by flussence (guest, #85566) [Link] (8 responses)

Reliably upgrading sshd doesn't have much in common with socket activation. You can still do the normal "restart daemon, log in on new sshd to verify it works, log out of old sshd when it's safe" routine, though that of course renders this micro-optimization pointless.

On the other hand, a system using cgroups to indiscriminately purge entire process ancestries will create exactly the problem you're describing... by design!

Thanks

Posted Nov 23, 2014 16:57 UTC (Sun) by mgb (guest, #3226) [Link] (5 responses)

> On the other hand, a system using cgroups to indiscriminately purge entire process ancestries will create exactly the problem you're describing... by design!

Exactly.

A hypothetical modular portable dependency-based init could have been a useful tool but systemd is the opposite of a tool - it is a restrictive imposition. Systemd's short-sighted design is extraordinarily counter-productive and this becomes glaringly obvious as systemd attempts to move from its desktop niche to the critical server and embedded spaces.

Thanks

Posted Nov 23, 2014 17:53 UTC (Sun) by zuki (subscriber, #41808) [Link] (4 responses)

>> Reliably upgrading sshd doesn't have much in common with socket
>> activation. You can still do the normal "restart daemon, log in on new
>> sshd to verify it works, log out of old sshd when it's safe" routine,
>> though that of course renders this micro-optimization pointless.
Not if exit-on-idle is implemented. If it is, you can do a test login after upgrade, and still have the process go away after while.

I agree that for a server this is pointless, but let's say that you are running containers or lightweight VMs, in multiple instances. Then avoiding starting the process can be a nice optimization.

>> On the other hand, a system using cgroups to indiscriminately purge
>> entire process ancestries will create exactly the problem you're
>> describing... by design!
> Exactly.
I don't grok. When an SSH session is started, the ssh instance and user process forked off it are moved to a separate scope unit, which of course also means a different subtree in the cgroup hierarchy. So individual sessions are completely independent of the ssh service. As for the sshd service itself, yes, all processes in it are stopped during restart. That pretty much describes the basic functionality of systemd achieved with cgroups. If you think something is wrong with "purging" all processes of a service when it is stopped, please explain.

[snip the rant]

Thanks

Posted Nov 23, 2014 18:06 UTC (Sun) by mgb (guest, #3226) [Link] (3 responses)

> If you think something is wrong with "purging" all processes of a service when it is stopped, please explain.

You should try pre-systemd Debian Stable which doesn't kill existing sshd connections during upgrades. It's great.

Thanks

Posted Nov 23, 2014 18:15 UTC (Sun) by zuki (subscriber, #41808) [Link] (2 responses)

>> If you think something is wrong with "purging" all processes of a
>> service when it is stopped, please explain.
> You should try pre-systemd Debian Stable which doesn't kill existing sshd
> connections during upgrades. It's great.

Once again: existing sshd connections *are* *not* *part* of sshd.service.
After they are established they are independent and are not touched when sshd.service is stopped or restarted.

(It is possible that there's a bug in your Debian package or setup or whatever... I can only say that it works for me and apparently for most people, and of course is *designed* to work this way. If it doesn't work for you, please provide the details and we'll work on a fix. Probably best to do this on the distribution bugtracker rather than here though.)

Thanks

Posted Nov 23, 2014 18:43 UTC (Sun) by mpr22 (subscriber, #60784) [Link] (1 responses)

Asking mgb for details about specific failures of systemd-based systems is a waste of electrons; they have explicitly stated on this site a refusal to use systemd or help debug systemd.

Thanks

Posted Nov 25, 2014 16:31 UTC (Tue) by nix (subscriber, #2304) [Link]

It's also clear from this subthread that mgb doesn't bother to actually read posts to which it responds: rather, it scans them for things that can be attacked, and ignores everything else, including inconvenient facts that might contradict its rants.

Thanks

Posted Nov 23, 2014 23:28 UTC (Sun) by mchapman (subscriber, #66589) [Link] (1 responses)

> On the other hand, a system using cgroups to indiscriminately purge entire process ancestries will create exactly the problem you're describing... by design!

Except it doesn't. There are two reasons for this.

First, pam_systemd will move the SSH child process into a different cgroup.

Second, even if it didn't do this, the sshd.service contains KillMode=process, which means only the main process is killed upon stop or restart. Other processes in the sshd.service cgroup are unaffected.

Thanks

Posted Nov 24, 2014 20:16 UTC (Mon) by smurf (subscriber, #17840) [Link]

This did happen, once upon a time. It was a bug. It was fixed rather quickly, for obvious reasons.

Too bad (from the point of view of the anti-systemd crowd, that is).

Thanks

Posted Nov 24, 2014 20:18 UTC (Mon) by smurf (subscriber, #17840) [Link] (1 responses)

It won't. Socket-activated processes don't get killed because they're idle. systemd ignores the socket while the process is running, so how could it know?

Thanks

Posted Nov 24, 2014 20:48 UTC (Mon) by mgb (guest, #3226) [Link]

> Socket-activated processes don't get killed because they're idle.

The subthread to which you are responding is concerned with zuki's suggestion http://lwn.net/Articles/622790/ that sshd should exit on idle.

Thanks

Posted Nov 24, 2014 13:02 UTC (Mon) by james (guest, #1325) [Link] (1 responses)

I'd expect a socket-activated sshd that is activated from init to be more likely to work (perhaps after a minute or two), simply because init won't be killed by the OOM killer.

Thanks

Posted Nov 25, 2014 16:32 UTC (Tue) by nix (subscriber, #2304) [Link]

sshd also takes precautions against being OOM-killed -- but for most other daemons your point is valid.

Thanks

Posted Nov 24, 2014 18:33 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

It's quite likely for OOM killer to kill your sshd, especially if it's inactive. Sure, you can play with OOM killer settings to make sure that your sshd (which is usually dropbear ssh) is not killed.

But of course, it's easier to do with systemd and socket-activation is useful because it will restart your daemon in case of failures...

Thanks

Posted Nov 24, 2014 19:27 UTC (Mon) by rodgerd (guest, #58896) [Link] (1 responses)

Of course, systemd makes it trivial to mark OOM settings on a per-service basis, too.

Thanks

Posted Nov 24, 2014 20:00 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

I'm actually thinking that combining socket activation and adjusting OOM killer score to make it _more_ likely to kill sshd is an even better strategy. This way, sshd will be killed first and will free up RAM that is likely wasted.

There's a slight possibility of a livelock when sshd is killed immediately after it's restarted by systemd. Perhaps the "OOMAdjustDelay" setting can be added to systemd?