Another daemon for managing control groups [LWN.net]

Another daemon for managing control groups

Posted Dec 17, 2013 1:52 UTC (Tue) by mathstuf (subscriber, #69389)
In reply to: Another daemon for managing control groups by skissane
Parent article: Another daemon for managing control groups

AIUI, systemd's API is higher level (it works on the service/scope/user level) while cgmanager is closer to a 1:1 of the kernel API. Any translation layer is going to need to implement systemd's logic to DTRT. It's not just a matter of calling a different DBus method.

to post comments

Another daemon for managing control groups

Posted Dec 17, 2013 4:49 UTC (Tue) by dlang (guest, #313) [Link] (27 responses)

if your description is correct, it should be simple to make systemd use cgmanager, at which point we have one master daemon that manages userspace, and it isn't systemd, that should keep everyone (except the people who think systemd should be in control of everything) happy.

Another daemon for managing control groups

Posted Dec 17, 2013 4:59 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (16 responses)

an init system shouldn't depend on another daemon. that doesn't make anybody happy.

Another daemon for managing control groups

Posted Dec 17, 2013 5:20 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (13 responses)

SHOULD is not MUST. An optional ability to use an external manager would do systemd only good.

Another daemon for managing control groups

Posted Dec 17, 2013 5:27 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (12 responses)

It is only optional if systemd has its own implementation of similar functionality and at that point, there is even less of a reason to depend on another daemon. external dependencies for a init system must be as minimal as possible. standardize on external interfaces if needed but not a specific implementation itself.

Another daemon for managing control groups

Posted Dec 17, 2013 6:17 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (11 responses)

Right now the whole systemd and single-tree cgroups API is an abomination that should be burned alive and then have its ashes scattered over an active volcano.

So another API is sorely needed. And if another library forces systemd to adopt something sane then it might be the only solution.

Another daemon for managing control groups

Posted Dec 17, 2013 6:57 UTC (Tue) by mchapman (subscriber, #66589) [Link] (10 responses)

> So another API is sorely needed.

Why? How is that going to help anything?

Maybe I've missed something in this whole argument, but I fail to see why there must be only one implementation of a cgroups manager. Yes, you may only be able to *run* one at any time due to kernel limitations, but that doesn't mean multiple projects can't choose to implement the functionality in different ways if they have different goals.

Isn't that the whole point of FOSS?

Another thing I don't really understand is why systemd should be "forced" to adapt to use some other manager. systemd doesn't need another manager; it does enough management of cgroups for itself already. What compelling reason is there for the systemd developers to throw all that out?

Another daemon for managing control groups

Posted Jan 3, 2014 18:23 UTC (Fri) by aigarius (guest, #7329) [Link] (9 responses)

Those are two competing points right there - currently systemd developers want to manage cgroups inside systemd and do not want to support delegating this management to any other outside daemon or library. For any other implementation of cgroups management to be able to work on a systemd system it would need systemd cooperation as above.

Therefore there are two options - either systemd must be relegated to be a tiny minority init version that can easily be replaced with something else on any distro, so that you can actually use a different cgroups manager or any other cgroups managers will be unusable without a very major hassle on many popular distros.

The refusal of systemd developers to cooperate in a flexible solution (combined with the marketing drive to make systemdbe default in major distros) is the very thing that is poisonous to alternative cgroup management implementations.

Another daemon for managing control groups

Posted Jan 3, 2014 18:57 UTC (Fri) by raven667 (guest, #5198) [Link] (8 responses)

> The refusal of systemd developers to cooperate in a flexible solution (combined with the marketing drive to make systemdbe default in major distros) is the very thing that is poisonous to alternative cgroup management implementations.

No, what is poisonous to other competing implementations is that systemd is technically advanced and comprehensive so that there is little need for other re-implementations because it works so well. That is the reason it has gained so much uptake, do you really think that all the people who build distributions are technically incompetent and are switching to systemd components because of some "marketing drive" or because these components solve the problems they are trying to solve better than the systems which came before?

It's true that the systemd developers refuse to re-design their system to have cgroup management handled by an external process, the "flexability" you talk about, but there are very rational technical reason for that which you have not addressed. Do you understand the kernel cgroups api and your workload and the systemd configuration knobs enough to be able to tell us where the systemd cgroup managment fails in your workload that another cgroup manager would be able to do better that couldn't be implemented in systemd? Do you have any examples or is this an entirely philosophical conversation with no real backing in reality?

Another daemon for managing control groups

Posted Jan 3, 2014 19:15 UTC (Fri) by dlang (guest, #313) [Link] (7 responses)

even the systemd developers admit that there are linux systems out there that should not run systemd (embedded type systems for example), but those systems may still want to use some of the features that systemd is trying to take over and kill off the competition in.

how would you use cgroups on a wireless access point where you don't want to run systemd but still want to limit some set of processes?

Another daemon for managing control groups

Posted Jan 3, 2014 20:03 UTC (Fri) by raven667 (guest, #5198) [Link] (6 responses)

It's a tautology really, if you don't want to run systemd for whatever reason, then don't run systemd.

> ... features that systemd is trying to take over and kill off the competition in.

I'm not sure what "take over" means in this context if you aren't running systemd on a system because you don't want to than what systemd does is not of relevance to you except as an example. The only way they are "killing the competition" is by making a high-quality, feature-rich offering which people want to use, they are out-competing the competition. I hope we can all agree that they shouldn't hobble the implementation and make it less useful just to make the competition look better...

> how would you use cgroups on a wireless access point where you don't want to run systemd but still want to limit some set of processes?

You do whatever you want to do, within the confines of the kernel interface, which in the future will want a single-writer (although compatibility with the existing interface will be maintained for some time as I understand it). So you can make your own cgroup manager daemon, being advised that there are fundamental problems with restarts, race conditions, etc. and that this can only manage a subset of processes (ones that it starts), or you write your custom manager into pid 1 either from scratch or patching one of the existing inits to have the logic you want.

There are probably going to be a handful of small cgroup managers that you can start from which are dedicated to specific tasks like HPC or Google, although if systemd incorporates whatever configuration toggles these workloads need then many will probably just use that rather than re-implement something themselves, but there is always someone out there who wants to do it themselves and is distrustful of any third party code 8-)

Really, how is this different from today where if you want to use this kernel feature you have to have your software talk to the kernel and twiddle it. How is it different than iproute2 or iptables?

Another daemon for managing control groups

Posted Jan 3, 2014 21:44 UTC (Fri) by dlang (guest, #313) [Link] (4 responses)

> The only way they are "killing the competition" is by making a high-quality, feature-rich offering which people want to use, they are out-competing the competition.

not quite, they also campaign to get the competition removed from the distro (or at least the default installation of the distro), rsyslog vs journald in Fedora is an example of this in practice.

Another daemon for managing control groups

Posted Jan 3, 2014 21:48 UTC (Fri) by mchapman (subscriber, #66589) [Link] (3 responses)

> they also campaign to get the competition removed from the distro (or at least the default installation of the distro)

Is that so surprising? Clearly they think they have a better solution. They *should* be campaigning for it, if that's what they really think.

I'm sure distro maintainers are smart enough to judge the various options available on their merits.

Another daemon for managing control groups

Posted Jan 3, 2014 21:54 UTC (Fri) by dlang (guest, #313) [Link] (2 responses)

they can campaign to have their system added, but what's the justification for removing the other option, especially when it can co-exist with theirs and provide capabilities that they don't offer?

Another daemon for managing control groups

Posted Jan 3, 2014 21:59 UTC (Fri) by mchapman (subscriber, #66589) [Link]

> they can campaign to have their system added, but what's the justification for removing the other option, especially when it can co-exist with theirs and provide capabilities that they don't offer?

For Rsyslog in Fedora specifically?

According to the Change page [1]:

1. Less disk and runtime footprint
2. Fewer packages in the default install
3. Faster booting

Now, you can argue whether those are *sufficient* justification for the removal of syslog from the default install. I'm sure people did.

At the end of the day, though, the vote was in that this Change should go ahead.

[1] https://fedoraproject.org/wiki/Changes/NoDefaultSyslog

Another daemon for managing control groups

Posted Jan 3, 2014 22:02 UTC (Fri) by raven667 (guest, #5198) [Link]

But they haven't removed rsyslog, and in fact rsyslog is required if you want to do central logging using syslog, systemd and the journal will pass through log messages to it. What they aren't doing is installing both logging systems by default. There is also no default sendmail in Fedora either.

Other distros make different decisions but rsyslog integration is still an important part of the journal, heck its even an important part of rsyslog which added support for the journal on-disk data format as I recall.

Another daemon for managing control groups

Posted Jan 3, 2014 21:46 UTC (Fri) by dlang (guest, #313) [Link]

> Really, how is this different from today where if you want to use this kernel feature you have to have your software talk to the kernel and twiddle it. How is it different than iproute2 or iptables?

iproute2 and iptables are happy to sit on a system and only be used for the parts that you want them to be used for. They don't try really hard to force you to do things a specific way and push you to replace a bunch of other tools if you use them.

Another daemon for managing control groups

Posted Dec 17, 2013 5:36 UTC (Tue) by dlang (guest, #313) [Link] (1 responses)

so systemd really isn't modular because if you replaced any of the binaries, then it would be depending on another daemon.

you can't have it both ways

(yes I'm done trolling now :-)

Another daemon for managing control groups

Posted Dec 17, 2013 5:48 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]

I was referring only to the init system part. systemd itself is a collection of programs, many of them optional and some of them not. If you introduce extra dependencies on the non-init system parts, then it is not much of a problem.

Another daemon for managing control groups

Posted Dec 17, 2013 11:33 UTC (Tue) by mathstuf (subscriber, #69389) [Link] (9 responses)

OK, so in this case, how does systemd manage cgmanager? Cgroups is out since cgmanager hasn't started. Or systemd has special logic for cgmanager. But then why have the logic in two places? Plus, cgmanager would probably end up as PID 2(ish)[1]. Sorry, I don't see how making systemd ask anything else about cgroups when it is such a core mechanism makes the overall system simpler and cleaner.

[1]Or 3; journald is pretty early too...but what is more important: logs or management? If logs, there's another thing that needs special group logic. Wait, cgmanager is over DBus, so add dbus-daemon to the list. What's the saying? "Do something once, hack it out; twice, use history; thrice, write a goddamn script" (or something). So if systems has special code to put the journal, DBus (modulo kdbus), and cgmanager, why the hell would you not just put all the logic in systemd to start with?

Another daemon for managing control groups

Posted Dec 17, 2013 13:23 UTC (Tue) by Jonno (guest, #49613) [Link] (8 responses)

Without having written any prototype code, but only looked at the problem architecturally, I agree that pushing cgroup management out of PID 1 will likely require *more* code in PID 1 than simply doing the cgroups management directly in systemd in the first place. But even so you don't need to drag dbus-daemon into the picture. cgmanager could use the same trick as systemd does, and initially only listen on a private socket for root-only direct libdbus communication, and then register itself on the dbus-daemon system bus whenever it becomes available.

That way cgmanager can start as PID 3 (after only systemd and journald), and the only early boot special handling in PID 1 would be to first start journald and cgmanager and then add them to their respective cgroups after the fact, but before continuing with any other units. Obviously that adds a bit of serialisation before going of and doing everything else in parallel, so will likely increase boot time by a second or two, but it's not like getting it to work is rocket science.

The trickier question is how to deal with cgmanager dying unexpectedly. PID 1 enjoys a privileged position and the kernel makes sure it never ever dies for any reason whatsoever, but PID 3 has no such protection. If systemd depends on it to do regular service management, it can't just restart cgmanager like it does any other service. Instead it would need special recovery code for that rare-but-inevitable case, code that likely won't get much testing, a perfect recipe for eventual disaster...

Another daemon for managing control groups

Posted Dec 17, 2013 18:07 UTC (Tue) by hummassa (guest, #307) [Link] (7 responses)

> PID 1 enjoys a privileged position and the kernel makes sure it never ever dies for any reason whatsoever

AFAICT the kernel dies and locks the machine out if PID 1 dies.

Another daemon for managing control groups

Posted Dec 17, 2013 18:14 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

I think it "just" panics. At that point, PID numbers are pretty useless I'd think :) .

Systemd's behavior in the face of a fatal error is to enter an infinite loop rather than dying[2].

[1]http://cgit.freedesktop.org/systemd/systemd/tree/src/core...
[2]http://cgit.freedesktop.org/systemd/systemd/tree/src/shar...

Another daemon for managing control groups

Posted Dec 17, 2013 19:23 UTC (Tue) by Jonno (guest, #49613) [Link]

> AFAICT the kernel dies and locks the machine out if PID 1 dies.
The kernel will panic if PID 1 exits on its own, but it will protect it from all outside interference (including blocking the normally unblockable SIGKILL and SIGSTOP), so as long as you handle all normal signals and never call exit() or return from main() you are golden.

As noted above systemd will enter an infinite loop (always yielding it's time slot back to the scheduler as to not monopolize the CPU) in the signal handler for any error signal it can't handle more gracefully, but that is only to protect from internal bugs, and not a normal operational condition, which a dying cgmanager would be.

Another daemon for managing control groups

Posted Dec 18, 2013 9:45 UTC (Wed) by anselm (subscriber, #2796) [Link] (4 responses)

The kernel won't lock the machine if PID 1 dies; it will complain but keep running by itself.

The legend goes that there used to be people whose PID 1 would set up a bunch of iptables rules and associated config and then exit – which would give you a firewall/router that was absolutely free from outside interference if a bit inconvenient to manage.

Another daemon for managing control groups

Posted Dec 18, 2013 15:14 UTC (Wed) by raven667 (guest, #5198) [Link]

I've had that happen, but not intentionally 8-) I've had devices too where the block device died but everything that was commonly used an in RAM/cache was still accessible, that machine actually ran for a year or two in that state doing its job before it could be decommissioned.

Another daemon for managing control groups

Posted Dec 18, 2013 18:33 UTC (Wed) by dlang (guest, #313) [Link]

I did that once, as much to show I could as anything else.

Another daemon for managing control groups

Posted Jan 17, 2014 10:27 UTC (Fri) by etienne (guest, #25256) [Link] (1 responses)

> ... let PID 1 exit to have a secure router ...

I have a new question to an old answer, to secure a system after it has been correctly configured:
Can you get rid of /bin/sh (i.e. any sort of shell: bash, ksh, zsh...) and still have a booting computer if you use systemd?
That would be very difficult to try to compromise a computer if the shell has been removed...

Another daemon for managing control groups

Posted Jan 17, 2014 10:36 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

Yes, it's absolutely possible to boot a computer without any shell using systemd.