|
|
Log in / Subscribe / Register

The Grumpy Editor's guide to surviving the systemd debate

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 17, 2014 1:33 UTC (Mon) by neilbrown (subscriber, #359)
In reply to: The Grumpy Editor's guide to surviving the systemd debate by fandingo
Parent article: The Grumpy Editor's guide to surviving the systemd debate

> Why do you speak in ... veiled examples in your several replies?

Sometimes too much detail can distract people from thinking about the point you are trying to make. So including it at the wrong time might not help. But I'm happy to share.

These are two issues that I recently hit. They are *not* examples which try to say "systemd is horrible and doomed can never work", they are meant as examples that say "systemd does have some real complexity, does have steep regions on the learning curve, isn't entirely transparent". It could be that they are problems for me only because I haven't learned enough yet - which just points to the learning-curve issue.

Firstly, I really like that using systemd makes it a lot easier for upstream packages to provide the start-up config file: providing init scripts in upstream was always problematic.
But if a distro has a pre-existing practice of using a particular name for a service, and upstream chooses a different name, then there is an issue to be resolved. I would like to support both names, at least during a transition period. That means we need an effective alias, so "foo-daemon" and "food" can be used interchangeably.

systemd *has* alias functionality using syslinks. If you create "food.service" as a link to "foo-daemon.service" then you can "start" and "stop" and "reload" and request "status" using either name. Which is great.
But you cannot "enable" food.service. Is this by design or is it a bug? From what I know of systemd I cannot really decide and the documentation only seems to mention aliases in asides - I cannot find a section "how aliases work".

So I created 'food.service' as a separate service with (hopefully) enough "Requires" and "PartOf" directives pointing at "foo-daemon.service" that systemd will make sure they are either both running or neither are.

Secondly, I had some fun with dracut (not that dracut is much fun ... I find it particularly difficult even remembering what it is called!! Whose idea was it to name key system software after towns in New England? And will they please stop it!?!)

Dracut uses systemd (which seems appropriate) and has a bunch of shell scripts which are run by a queue manager (I can't decide if there is something ironic about that.... can systemd not manage the queue? should it?).

Anyway in a particular circumstance an md/raid array will *not* be assembled when all the devices have appeared, and this is correct. There needs to be a timeout to give up waiting for more devices - then the array is assembled. If that array contains an LVM volume and the LVM volume contains the root filesystem, then dracut needs to:
1/ wait the timeout
2/ assemble the md array
3/ activate the LVM volume

all of which it does correctly. During this time, systemd is waiting for the root device to appear.
How long does it wait? According to "systemd.unit", "JobTimeoutSec" defaults to zero - except for devices units. It doesn't tell me what the default is for device units :-( It turns out to be 90 seconds - DefaultTimeoutStartSec in /etc/systemd/system.conf. But the timeout in step '1' above in 120 seconds. How can this ever work?
I'm not entirely sure, but it does is most cases. In the above case, systemd times out between steps 2 and 3. So if the md array contained the root device directly it would work. If not, it doesn't because systemd doesn't wait the extra second it takes to activate the volume.

If I increase DefaultTimeoutStartSec to 120 .. it still doesn't work. 130? No. 300? Yes. Now it works.

I'm sure there is a good explanation for this. It might even make sense. But "simple" "transparent" "gentle learning curve" aren't the words I'm focussing on just now.

Maybe I need to enable some tracing somewhere.... old habits of adding "echo" or "printk" as just going to have to change...


to post comments

The Grumpy Editor's guide to surviving the systemd debate

Posted Mar 13, 2015 0:37 UTC (Fri) by neilbrown (subscriber, #359) [Link]

For the record, the problem with dracut was that it has a systemd 'generator' which only creates the important files the first time it is run: rootfs-generator.sh

When you "systemctl daemon-reload", systemd will remove everything from /run/systemd/generator and then re-run all the generators. As rootfs-generator.sh only created files in that directory the first time it is run, a subsequent 'systemctl daemon-reload' will revert the effect of that generator.

The key effect of the generator is to disable systemd timeouts on the device which holds the root filesystem - as dracut has its own timeout mechanism. So a 'daemon-reload' will re-instate the default timeout.

When dracut does timeout waiting for the md array to assemble, and forces the assembly of all degraded md array, it also runs "daemon-reload" (as part of giving up on resume-from-disk). This will cause systemd to give up on the root device immediately, typically a second or so before it is actually ready.

Does that make sense? Anyway, the primary problem was not a problem with systemd ... except...

The real goal of dracut here is to generally disable the systemd timeouts which wait for devices. Lots of systemd is configurable at runtime using systemctl, but this thing isn't. It can only be configured in /etc/systemd/system.conf before systemd starts. 'daemon-reload' doesn't reload system.conf.

So dracut has this complex (and buggy) code to over-ride timeouts that systemd doesn't let it configure in a sensible fashion. So maybe systemd is partly to blame, after all...


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds