LWN: Comments on "Systemd programming part 2: activation and language issues"

Systemd programming part 2: activation and language issues

Duncan — Wed, 28 May 2014 23:00:57 +0000

[Not that many will see this so long after original publishing, but for the benefit of people googling, etc...]

This.

When invoking systemctl enable/disable, systemctl/systemd only has to look at [install], other sections can be ignored.

Similarly, when systemd is constructing its dependency tree, it only has to look at the [unit] section, [install] and [service], etc, sections can be ignored. And when it's actually invoking the unit, only the [service], [socket], etc, sections as appropriate, must be considered, [install] and [unit] can be ignored.

While I've no idea whether systemd is actually coded this way, from back in my MS days I know their ini-file APIs were setup to work exactly that way. The API allowed you to open a particular ini file, then specify section and key (IIRC they used the term "value", which always confused me) to return or set, with its value (which they termed "data", as "value" referred to the key) then returned or set. Other sections might as well not have existed at all, and in fact, could belong to entirely different applications (as was the case in MS Windows 2.x I believe, when all sorts of apps had sections in win.ini, before 3.x introduced dedicated application ini files).

So access is (at least conceptually) by section and then key, and if key names don't overlap between sections, it's only to avoid human confusion, not because they couldn't, as to the application, if they're in different sections they are (or should be) entirely different things, regardless of what they're named.

Duncan

Systemd programming part 2: activation and language issues

cg909 — Thu, 20 Feb 2014 17:54:49 +0000

I think the separation into the sections is more for clearifying what the directives apply to.

Directives in [Unit] apply to the dependeny tree and define general properties of the unit.
Directives in [Install] are only used when running systemctl enable/disable and have no other direct effect.
Directives in [Service], [Socket], ... define the behaviour when the unit is run. And as the allowed directives are different between unit types the sections also are named differently. This little redundancy may also help catching copy&paste errors between unit files.

Sections like [Exec], [Kill], ... would only group some related directives without providing context. Currently it would also split the execution environment directives and the Exec, ExecStartPre, etc. directives into different sections.

Systemd programming part 2: activation and language issues

cg909 — Thu, 20 Feb 2014 17:03:41 +0000

Systemd has another way to explicity disable a service:
systemctl mask $service
That tells systemd that this service should never run.

In the systemd concept every service that is known to systemd *may* be started if its needed (and is not masked).

systemctl enable $service can be read as "Make sure that $service *will* be run."

So with systemd you have in fact more flexibility: You can leave a service disabled, so that it will run only if required by another unit; you can enable it to let it run in any case and you can mask it to disable it completely.

Systemd programming part 2: activation and language issues

mathstuf — Thu, 20 Feb 2014 14:30:11 +0000

Hmm? Could you be more specific? Sometimes dependencies are configuration-related. For example, I run nginx in front of any service which uses a web interface. How would flyspray know that its configuration needs to start nginx first to actually work? In systemd, I could just enable it by symlinking nginx into fastcgiwrap@flyspray.service.wants without modifying any service files (though in reality I'd use socket activation). How would you do such a dependency in other systems?

Systemd programming part 2: activation and language issues

kevinm — Thu, 20 Feb 2014 11:12:37 +0000

It seems unfortunate that the concept of whether a service is administratively disabled or enabled is apparently conflated with codifying the dependencies between services.

The dependencies are quite static and rarely changed, presumably set by upstream and tweaked by the builder of your distribution. However services may be enabled and disabled frequently as a matter of administrative fiat.

Systemd programming part 2: activation and language issues

mbunkus — Sat, 15 Feb 2014 17:29:54 +0000

> Clearly Exec{Start,Stop}{Pre,Post} (and multiple others such as WorkingDirectory) are permitted in both the [Service] section and the [Socket] section. However they have the same meaning in both sections, and no unit file can have both sections, so there would still be no ambiguity if the section headings were absent.

My guess is that it's more about future extensibility without breaking compatibility. While it may be true that there are no conflicts today doesn't mean they won't invent features that allow multiple sections with the same option names. These can be added easily with new sections, but if there were no sections then they might be forced to chose awkward names or prefix quite a lot of those new options, neither or which would improve the ease of use.

Systemd programming part 2: activation and language issues

neilbrown — Fri, 14 Feb 2014 21:29:46 +0000

Hi,
thanks for highlighting that. It is a relevant example that I hadn't fully explored.

Clearly Exec{Start,Stop}{Pre,Post} (and multiple others such as WorkingDirectory) are permitted in both the [Service] section and the [Socket] section. However they have the same meaning in both sections, and no unit file can have both sections, so there would still be no ambiguity if the section headings were absent. The precise meaning is determined by the suffix of the unit file.

(and I did wonder why those "multiple others" were not in an [Exec] section, seeing they are documented in systemd.exec - there is some fancy m4 preprocessor code to duplicate a collection of Directives across multiple sections in src/core/load-fragment-gperf.gperf.m4)

COME FROM

giraffedata — Fri, 14 Feb 2014 21:15:00 +0000

The dependency declaration looks backwards only if you look at the meaning of "multiuser" backwards.

The Upstart philosophy is the "multiuser" unit means the user has authorized or requested multiuser service. NFS is appropriate only in multiuser service, so NFS depends upon multiuser. The Systemd philosophy is that the multiuser unit refers to the system providing the multiuser service. You can't claim to providing multiuser service if you aren't providing NFS, so multiuser depends upon NFS.

The Systemd definition of multiuser seems more useful and easy to understand to me.

Here's an analogy: a house is inhabitable only if the heat is on, but it's wasteful to heat an uninhabited house. So you could say inhabitability depends upon the heat being on and therefore the landlord should turn on the heat before signing a lease. Or you could say the appropriateness of the heat being on depends upon the house being inhabited, so the furnace should turn on only when it detects that a lease has been signed.

Systemd programming part 2: activation and language issues

mathstuf — Fri, 14 Feb 2014 18:45:14 +0000

> One issue that stuck me as odd, though there is some room for justification, is the existence of section headings such as [Unit] or [Service] or [Install].

I think the root reason is because not all sections are allowed in all unit files. .socket files don't have a [Service] section, so anything that belongs under it doesn't belong because it doesn't make sense.

Also, from systemd.socket(5):

> Socket files must include a [Socket] section, which carries information about the socket or FIFO it supervises. A number of options that may be used in this section are shared with other unit types. These options are documented in systemd.exec(5) and systemd.kill(5). The options specific to the [Socket] section of socket units are the following:

So it does appear that the certain directives are valid in multiple sections.

Systemd programming part 2: activation and language issues

ocharles — Thu, 13 Feb 2014 21:39:20 +0000

W.r.t the logic stuff, any logical expression can be converted to conjunctive (or disjunctive) normal form. I wonder if that means you can get the aforementioned troublesome conditionals?

Systemd programming part 2: activation and language issues

zuki — Tue, 11 Feb 2014 21:46:10 +0000

I stand corrected. Indeed WantedBy is only created implicitly.

> I don't agree with your description of how the syslinks in the .wants
> directory are handled. It is best to understand them as being exactly like
> drop-in files - they are even handled by code in load-dropin.c. A symlink
> in the .wants directory is *extemely* similar to a .conf file in the ".d"
> drop-in directory containing "Wants=foo"
I guess I wasn't very clear. You're right that the symlink is very similar and could be replaced by a drop-in. I was trying to underline the fact that you need the symlink (or the equivalent drop-in), because the configuration has to become a part of the "wanting" unit, even if is installed together with the "wanted" unit. No disagreement, just a different vantage point.

Systemd programming part 2: activation and language issues

Siosm — Tue, 11 Feb 2014 20:47:52 +0000

> incorporate the functionality of portmap/rpcbind within systemd's PID 1 itself

That's absolutely not what the author meant :
> bind an arbitrary port, register that port with rpcbind

This would be one time process for "rpc-sockets" which is really close to the already include socket activation. This could be added as an option to the current socket unit I think.

Systemd programming part 2: activation and language issues

cesarb — Tue, 11 Feb 2014 18:38:12 +0000

Clearly, what is needed is to incorporate the functionality of portmap/rpcbind within systemd's PID 1 itself. We could call this "portmap activation".

(I might or might not be joking.)

Systemd programming part 2: activation and language issues

Siosm — Tue, 11 Feb 2014 16:38:50 +0000

> So systemd would need to do this too: bind an arbitrary port, register that port with rpcbind, and start rpc.statd when traffic arrives on that socket. (...) Whether it is worth adding this functionality to systemd for the two or maybe three services that would use it is hard to say.

Well, only upstream can answer that. You should ask on the systemd-devel mailing list.

COME FROM

hummassa — Tue, 11 Feb 2014 13:34:19 +0000

Exactly that. :)

Not that COME FROM is totally useless (no GUI-based programming without it).

COME FROM

cesarb — Tue, 11 Feb 2014 12:46:20 +0000

> There is a key difference here, though, and it isn't really about events or dependencies, but about causality. In upstart, a job can declare "start on" to identify which event it should start on. So each job declares the events which cause it to run. Systemd, despite its rich dependency language, has no equivalent to "start on", an omission that appears to be deliberate. Instead, each event — the starting or stopping of a unit — declares which jobs (or units) need to be running. The dependency language is exactly reversed. With upstart, each job knows what causes it to start. With systemd, each job knows what it causes to start.

Let's see if I understood this correctly. Upstart's behavior is like COME FROM, while systemd's is more like GO TO.

Systemd programming part 2: activation and language issues

johannbg — Tue, 11 Feb 2014 11:10:11 +0000

I must say I'm a bit astound the unit creation you are proposing here on the article most notably splitting things into several targets like that and why you simply did not chose to use these unit [1] which should serve as bases for units used by upstream and distribution.

The main problem with NFS in Fedora is and always has been it's maintainer Steve which chose on several occasion to ignore comments made by upstream systemd developer ( like Kay's first comment on that bug ) submitted units and even half implement provided units ( he cherry picked units from the provided set of units which left the entire dependency chain broken ) .

1. https://bugzilla.redhat.com/show_bug.cgi?id=769879

Systemd programming part 2: activation and language issues

gerdesj — Tue, 11 Feb 2014 11:00:09 +0000

Great articles - thanks.

You have performed a near miracle in getting NFS to work within systemd-ese. I have struggled to get my Gentoo laptop running everything correctly for similar reasons that you mention but alas I lack your insight and skills (I can manage man pages but the source makes my eyes bleed)

That said, it works really well and I binned OpenRC on it a few months ago. I also find the short descriptive units easy to get on with. In many cases nearly all the same functionality of the original takes a few simple lines rather than the monster rc scripts. Plus socket/port activation means that I only start stuff when I use it, which is really nice and a lot neater than using xinetd.

Again, thanks for explaining some concepts that I was struggling with.

Cheers
Jon

Systemd programming part 2: activation and language issues

neilbrown — Tue, 11 Feb 2014 01:51:32 +0000

> Normally, nfs.service would have [Unit] Requires=rpcbind.service, After=rpcbind.service

Clearly that section could have been clearer - it was even less clear before the lwn editorial team helped out.

The context was intended to imply that "systemctl enable nfs" was being run with "nfs" as a SysVinit script (/etc/init.d/nfs), which is how nfs is currently started on openSUSE-13.1. systemd can read SysVinit scripts and parse the LSB headers, but there is no way for those headers to have the equivalent of "Require=rpcbind.service". Only the "After=rpcbind.server" bit.

So yes: nfs.service would have those lines, but /etc/init.d/nfs cannot and so the compatibility between systemd and SysVinit is not as great as I originally hoped.

> There are two places where WantedBy/RequiredBy can be used.

"Citation needed". I checked the man pages, read the source code, and performed as simple test and I find that "WantedBy" is only permitted in "Install", never in "Unit".

However ... the internal implementation details of systemd involves creating reflections of some (or maybe all) relationships. When one unit "Wants" another, then the internal representation of the other gets a "WantedBy" annotation referring to the first. Similarly there is "ConsistsOf" which is a reflection of "PartOf". These are internal only and not part of the language. "ConsistsOf" is not permitted anywhere in unit files, and "WantedBy" is only permitted in the [Install] section with the meaning I outline.

I don't agree with your description of how the syslinks in the .wants directory are handled. It is best to understand them as being exactly like drop-in files - they are even handled by code in load-dropin.c. A symlink in the .wants directory is *extemely* similar to a .conf file in the ".d" drop-in directory containing "Wants=foo"

Thanks for your comments.

Systemd programming part 2: activation and language issues

zuki — Tue, 11 Feb 2014 01:09:46 +0000

Nice article, but there are a few statements which seem to be wrong (or at least misleading):

> "systemctl enable nfs" works, but a subsequent "systemctl start nfs" fails
> because rpcbind (which is the new name for portmap) isn't running
Normally, nfs.service would have [Unit] Requires=rpcbind.service, After=rpcbind.service, which would cause rpcbind.service to be started before nfs.service is started. This applies both when nfs.service is started manually, and also when nfs.service is started because it is requires by some target (e.g. nfs.target, which is in turn wanted by multi-user.target, which is in turn wanted by default.target).

> "WantedBy" and "RequiredBy" directives, which are quite different from the
> "Requires" and "Wants" etc. dependency directives. "WantedBy" plays no role
> in determining when to start (or stop) any service. Instead, it is an
> instruction on how to enable a specific unit.
There are two places where WantedBy/RequiredBy can be used. In the [Install] section, they behave like you describe, i.e. they are used to create links when 'systemctl enable' is invoked. In the [Unit] section, they are simply Wants/Requires in the reverse direction. For this latter use, there's no difference between having Wants=B.service in A.service, having WantedBy=A.service in B.service, or having a symlink to B.service created as /etc/systemd/system/A.service.wants/B.service when systemd has loaded all the units.

The reason that those (overlapping) ways of adding dependencies exist is a combination of two factors. Because it often the "wanted" or "required" service that knows about the dependency. E.g. in case of targets, the list of services that the target "wants" is completely arbitrary, so it cannot be part of the T.target configuration and we want to add something like WantedBy=T.target as part of the service file. But systemd loads units lazily, so if we simply install the unit file, we must make sure that systemd will always load it. This is achieved by creating a symlink in the .wants directory, which will cause systemd to also load the service that the symlink points to when it reads the unit that the .wants directory is for. So the equivalency between different ways of expressing dependencies stated in the previous paragraph is only true when those units have actually been loaded.

Also, without actually looking at the units, it's strange that you use "Requisite=nfs-secure.target" and not "Requires=nfs-secure.target" or maybe "RequiredBy=rpc-gssd.service".

I agree totally with your critique of the multitude of systemd dependency types. They have grown somewhat organically over the years, and it would be good to simplify them at some point. Nevertheless, they were not added lightly, but rather in response to usecases found in the wild which were hard to satisfy with existing vocabulary.