Thanks [LWN.net]

Thanks

Posted Nov 23, 2014 1:29 UTC (Sun) by zuki (subscriber, #41808)
In reply to: Thanks by rodgerd
Parent article: Today's Debian technical committee resignation: Ian Jackson

I think socket-activated sshd makes a lot of sense... Not the extreme of running a separate instance for each connection (Accept=yes in systemd parlance), but a single instance with delayed activation. For many notebooks, workstations, and dedicated servers sshd is used never or rarely, so not starting it unconditionally is a nice optimization. For this to really work nicely, sshd should learn to exit-on-idle. Then you get the best of both worlds.

to post comments

Thanks

Posted Nov 23, 2014 3:05 UTC (Sun) by mgb (guest, #3226) [Link] (20 responses)

You save a few bytes of swap space and increase the likelihood of having to drive to the data center at zero dark thirty.

Not for me, thanks.

Thanks

Posted Nov 23, 2014 7:31 UTC (Sun) by smurf (subscriber, #17840) [Link] (19 responses)

How does socket-activating sshd increase any likelihood of problems?

I'd assume that this would decrease it, since this way you do not risk starting it too early (/var/log not yet mounted, or whatever else might cause sshd to fail to start), nor too late (i.e. not at all, if something earlier in the boot sequence fails)

Thanks

Posted Nov 23, 2014 7:42 UTC (Sun) by mchapman (subscriber, #66589) [Link] (18 responses)

> How does socket-activating sshd increase any likelihood of problems?

If the system is close to being out of memory, then I would expect a socket-activated SSH to be less likely to get you to a shell than a non-socket-activated SSH.

If your only remote access to the system is with SSH, and remotely rebooting it is infeasible or impractical, then this could be a concern.

Thanks

Posted Nov 23, 2014 9:01 UTC (Sun) by mgb (guest, #3226) [Link] (12 responses)

Also for handling upgrade problems.

Once your sshd process exists you don't want it to disappear unexpectedly.

Thanks

Posted Nov 23, 2014 9:29 UTC (Sun) by mchapman (subscriber, #66589) [Link]

No, I don't think socket-activation changes anything there. When you upgrade SSH, the package's post-installation scripts should cause it to be restarted if it's running as a non-socket-activated service.

Thanks

Posted Nov 23, 2014 16:37 UTC (Sun) by flussence (guest, #85566) [Link] (8 responses)

Reliably upgrading sshd doesn't have much in common with socket activation. You can still do the normal "restart daemon, log in on new sshd to verify it works, log out of old sshd when it's safe" routine, though that of course renders this micro-optimization pointless.

On the other hand, a system using cgroups to indiscriminately purge entire process ancestries will create exactly the problem you're describing... by design!

Thanks

Posted Nov 23, 2014 16:57 UTC (Sun) by mgb (guest, #3226) [Link] (5 responses)

> On the other hand, a system using cgroups to indiscriminately purge entire process ancestries will create exactly the problem you're describing... by design!

Exactly.

A hypothetical modular portable dependency-based init could have been a useful tool but systemd is the opposite of a tool - it is a restrictive imposition. Systemd's short-sighted design is extraordinarily counter-productive and this becomes glaringly obvious as systemd attempts to move from its desktop niche to the critical server and embedded spaces.

Thanks

Posted Nov 23, 2014 17:53 UTC (Sun) by zuki (subscriber, #41808) [Link] (4 responses)

>> Reliably upgrading sshd doesn't have much in common with socket
>> activation. You can still do the normal "restart daemon, log in on new
>> sshd to verify it works, log out of old sshd when it's safe" routine,
>> though that of course renders this micro-optimization pointless.
Not if exit-on-idle is implemented. If it is, you can do a test login after upgrade, and still have the process go away after while.

I agree that for a server this is pointless, but let's say that you are running containers or lightweight VMs, in multiple instances. Then avoiding starting the process can be a nice optimization.

>> On the other hand, a system using cgroups to indiscriminately purge
>> entire process ancestries will create exactly the problem you're
>> describing... by design!
> Exactly.
I don't grok. When an SSH session is started, the ssh instance and user process forked off it are moved to a separate scope unit, which of course also means a different subtree in the cgroup hierarchy. So individual sessions are completely independent of the ssh service. As for the sshd service itself, yes, all processes in it are stopped during restart. That pretty much describes the basic functionality of systemd achieved with cgroups. If you think something is wrong with "purging" all processes of a service when it is stopped, please explain.

[snip the rant]

Thanks

Posted Nov 23, 2014 18:06 UTC (Sun) by mgb (guest, #3226) [Link] (3 responses)

> If you think something is wrong with "purging" all processes of a service when it is stopped, please explain.

You should try pre-systemd Debian Stable which doesn't kill existing sshd connections during upgrades. It's great.

Thanks

Posted Nov 23, 2014 18:15 UTC (Sun) by zuki (subscriber, #41808) [Link] (2 responses)

>> If you think something is wrong with "purging" all processes of a
>> service when it is stopped, please explain.
> You should try pre-systemd Debian Stable which doesn't kill existing sshd
> connections during upgrades. It's great.

Once again: existing sshd connections *are* *not* *part* of sshd.service.
After they are established they are independent and are not touched when sshd.service is stopped or restarted.

(It is possible that there's a bug in your Debian package or setup or whatever... I can only say that it works for me and apparently for most people, and of course is *designed* to work this way. If it doesn't work for you, please provide the details and we'll work on a fix. Probably best to do this on the distribution bugtracker rather than here though.)

Thanks

Posted Nov 23, 2014 18:43 UTC (Sun) by mpr22 (subscriber, #60784) [Link] (1 responses)

Asking mgb for details about specific failures of systemd-based systems is a waste of electrons; they have explicitly stated on this site a refusal to use systemd or help debug systemd.

Thanks

Posted Nov 25, 2014 16:31 UTC (Tue) by nix (subscriber, #2304) [Link]

It's also clear from this subthread that mgb doesn't bother to actually read posts to which it responds: rather, it scans them for things that can be attacked, and ignores everything else, including inconvenient facts that might contradict its rants.

Thanks

Posted Nov 23, 2014 23:28 UTC (Sun) by mchapman (subscriber, #66589) [Link] (1 responses)

> On the other hand, a system using cgroups to indiscriminately purge entire process ancestries will create exactly the problem you're describing... by design!

Except it doesn't. There are two reasons for this.

First, pam_systemd will move the SSH child process into a different cgroup.

Second, even if it didn't do this, the sshd.service contains KillMode=process, which means only the main process is killed upon stop or restart. Other processes in the sshd.service cgroup are unaffected.

Thanks

Posted Nov 24, 2014 20:16 UTC (Mon) by smurf (subscriber, #17840) [Link]

This did happen, once upon a time. It was a bug. It was fixed rather quickly, for obvious reasons.

Too bad (from the point of view of the anti-systemd crowd, that is).

Thanks

Posted Nov 24, 2014 20:18 UTC (Mon) by smurf (subscriber, #17840) [Link] (1 responses)

It won't. Socket-activated processes don't get killed because they're idle. systemd ignores the socket while the process is running, so how could it know?

Thanks

Posted Nov 24, 2014 20:48 UTC (Mon) by mgb (guest, #3226) [Link]

> Socket-activated processes don't get killed because they're idle.

The subthread to which you are responding is concerned with zuki's suggestion http://lwn.net/Articles/622790/ that sshd should exit on idle.

Thanks

Posted Nov 24, 2014 13:02 UTC (Mon) by james (guest, #1325) [Link] (1 responses)

I'd expect a socket-activated sshd that is activated from init to be more likely to work (perhaps after a minute or two), simply because init won't be killed by the OOM killer.

Thanks

Posted Nov 25, 2014 16:32 UTC (Tue) by nix (subscriber, #2304) [Link]

sshd also takes precautions against being OOM-killed -- but for most other daemons your point is valid.

Thanks

Posted Nov 24, 2014 18:33 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

It's quite likely for OOM killer to kill your sshd, especially if it's inactive. Sure, you can play with OOM killer settings to make sure that your sshd (which is usually dropbear ssh) is not killed.

But of course, it's easier to do with systemd and socket-activation is useful because it will restart your daemon in case of failures...

Thanks

Posted Nov 24, 2014 19:27 UTC (Mon) by rodgerd (guest, #58896) [Link] (1 responses)

Of course, systemd makes it trivial to mark OOM settings on a per-service basis, too.

Thanks

Posted Nov 24, 2014 20:00 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

I'm actually thinking that combining socket activation and adjusting OOM killer score to make it _more_ likely to kill sshd is an even better strategy. This way, sshd will be killed first and will free up RAM that is likely wasted.

There's a slight possibility of a livelock when sshd is killed immediately after it's restarted by systemd. Perhaps the "OOMAdjustDelay" setting can be added to systemd?

Thanks

Posted Nov 23, 2014 9:32 UTC (Sun) by rodgerd (guest, #58896) [Link] (4 responses)

For some cases. Of course, OpenSSH isn't very traditional Unix. It's one project that replaces telnet, rlogin, rsh, rcp, and FTP. Why have a big monolithic blob? Shouldn't it be run as a set of loosely-coupled projects with seperate repositories?

The proper Unix way would be to have all those traditional services spawn out of inetd and use tools like tcpwrappers for access control; if you really need security you should be using something like stunnel rather than building it all into the basic tool. All that code living together is obviously a much bigger attack service.

And have you seen the attitude of the maintainer? He rejects portability patches and keeps saying rude things about other platforms!

All of this is against traditional Unix principles and will be a disaster for Unix, mark my words.

Thanks

Posted Nov 23, 2014 18:35 UTC (Sun) by dlang (guest, #313) [Link] (3 responses)

actually, the fact that ssh is telnet + ftp + vpn is an ongoing problem for security people who would like to allow some of this capability without allowing it all.

Thanks

Posted Nov 24, 2014 20:22 UTC (Mon) by smurf (subscriber, #17840) [Link] (2 responses)

It is, like, _so_ difficult to turn the unwanted features off in sshd_config, no?

Thanks

Posted Nov 25, 2014 2:22 UTC (Tue) by dlang (guest, #313) [Link] (1 responses)

yes, it's extremely hard to turn features off for some users while allowing it for others.

Thanks

Posted Nov 25, 2014 16:33 UTC (Tue) by nix (subscriber, #2304) [Link]

The keyword you're looking for is 'Match'. You can PermitTunnel on a group-by-group, IP-by-IP, user-by-user, or even local-port-by-port (?!) basis.