GNU Shepherd 1.0.0 released

[Posted December 10, 2024 by corbet]

Version 1.0.0 of the GNU Shepherd service manager has been released after a mere 21 years of development.

This 1.0.0 release is published today because we think Shepherd has become a solid tool, meeting user experience standards one has come to expect since systemd changed the game of free init systems and service managers alike. It's also a major milestone for Guix, which has been relying on the Shepherd from a time when doing so counted as dogfooding.

Comparison available?

Posted Dec 10, 2024 16:16 UTC (Tue) by david.a.wheeler (subscriber, #72896) [Link] (25 responses)

Is there a comparison of shepherd to other tools like systemd?

Comparison available?

Posted Dec 10, 2024 17:27 UTC (Tue) by dave_malcolm (subscriber, #15013) [Link] (24 responses)

One aspect that's of interest to me: if GNU Shepherd's config files are in Scheme, does that mean that they can execute arbitrary code with the permissions of the manager process?

Am I right in thinking that in systemd that .service files are deliberately not Turing complete, so that they can be statically analyzed ? (e.g. mechanically verified to comply with site-wide policies)

Comparison available?

Posted Dec 10, 2024 17:55 UTC (Tue) by quotemstr (subscriber, #45331) [Link] (21 responses)

It'd be straightforward to write a static analyzer for Shepherd configurations --- you could probably do it nice and declaratively with a sexp pattern matcher, like Emacs pcase. (Doesn't Guile support running Emacs Lisp?) Of course it'd be hard or impossible to verify invariants of *arbitrary* Shepherd programs, but a linter is allowed to say "Sorry, this is too complex for me to understand so I'm going to bail out with an error", so halting problem concerns don't apply.

I'm a fan in general of using general purpose programming languages as configuration languages.

Comparison available?

Posted Dec 10, 2024 20:59 UTC (Tue) by dankamongmen (subscriber, #35141) [Link] (18 responses)

interesting. i regard systemd's departure from turing-complete service configs as one of its most fundamental advantages over sysv, but this is surely a debate which has been had ten thousand times, and won't be resolved here.

Comparison available?

Posted Dec 10, 2024 22:05 UTC (Tue) by rweikusat2 (subscriber, #117920) [Link] (16 responses)

Well, the inevitable outcome of that is that people start stacking service start scripts on top of systemd. This is, for instance, being done by the Debian ntp package (at least in Debian 12).

Comparison available?

Posted Dec 11, 2024 0:51 UTC (Wed) by Wol (subscriber, #4433) [Link] (2 responses)

> Well, the inevitable outcome of that is that people start stacking service start scripts on top of systemd.

Wasn't that how almost all the original systemd units were written - as just a wrapper for the SysV shell scripts?

Cheers,
Wol

Comparison available?

Posted Dec 11, 2024 15:37 UTC (Wed) by rweikusat2 (subscriber, #117920) [Link]

This is a script specifically written for systemd (as far as I can tell).

Comparison available?

Posted Dec 15, 2024 15:46 UTC (Sun) by arsen (subscriber, #161285) [Link]

to my awareness, that is a debianism, and other vendors did not follow suit

Comparison available?

Posted Dec 11, 2024 6:28 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (10 responses)

I don't see it in my ntpd?

BTW, systemd supports dynamic units via generators. It's a nice escape hatch, and it's also nice that you need to explicitly use it, making it conspicuous.

Comparison available?

Posted Dec 11, 2024 15:35 UTC (Wed) by rweikusat2 (subscriber, #117920) [Link] (9 responses)

The Debian 12 ntpsec package uses /usr/libexec/ntpsec/ntp-systemd-wrapper as ExecStart command in the ntpsec.service unit which is a script implementing actual ntpd start.

NB: I didn't spend any time looking more closely at this, I just noticed it. This means I have no idea if this could also be implemented in a different way.

Comparison available?

Posted Dec 11, 2024 18:24 UTC (Wed) by raven667 (subscriber, #5198) [Link] (4 responses)

I haven't looked at it either (just finishing lunch) but I'm not sure it matters, there is a service startup and the wrapper does whatever it needs to do, there isn't an intrinsic value in "purity" where everything is done the "one true way" with ExecStart/ExecStartPre or Wants/WantedBy dependencies, unless that solves some other maintenance or availability problem. Maybe some day the wrapper can be retired, maybe there are corner cases which can't be handled with a strictly declarative config, in the end the service can be started and monitored, sandboxed and logged, so all is good.

Comparison available?

Posted Dec 11, 2024 19:14 UTC (Wed) by rweikusat2 (subscriber, #117920) [Link] (3 responses)

It matters insofar it demonstrates that systemd did not "do away" with service start scripts written in a general-purpose programming language. And that's despite it already (extracted¹ from Debian 12 documentation) supports about 317 different types of service declarations.

¹ Via man systemd.something | perl -ne 'print("$_\n") for /\w+=/g' | sort -u for each something documenting configuration directives.

Comparison available?

Posted Dec 11, 2024 19:34 UTC (Wed) by zdzichu (guest, #17118) [Link]

It demonstrates systemd still makes it possible for shot one in the foot. And Debian's reputation of overcomplicating things is well-deserved.

I took a quick glance on this wrapper script:
– pidfile and lockfile are unnecessary with systemd
– /etc/default/ntpsec can be imported using EnvironmentFile= directive
– using different configuration for DHCP could be done with service instances (if it's really needed; I doubt that)
– User=/Group= exists

Wrapper is excessive.

Comparison available?

Posted Dec 11, 2024 19:37 UTC (Wed) by bluca (subscriber, #118303) [Link]

Out of 269 service files on my machine, there are only 4 that point to a script in ExecStart, so I'd say a single case where some extremely old stuff was lazily ported doesn't prove much of anything.

Comparison available?

Posted Dec 11, 2024 22:05 UTC (Wed) by himi (subscriber, #340) [Link]

I think that's a pretty unreasonable argument to make - I deal with packages that do things this way regularly (openvswitch with its ovs-ctl script, and OVN with its ovn-ctl script), and while both of those control scripts are kind of like a SysV init script if you squint a bit, they're more like a wrapper around the underlying package binaries than an init script. Sure, with enough work you could move all the functionality they provide into a set of systemd units, but then you couldn't use them to provide the additional runtime management features they support (switching a backup ovsdb server to an active state, and various things like that), or you'd need to have the control script /as well as/ the systemd unit.

SysV init scripts did often incorporate that kind of thing - hell, the old openvswitch init scripts did a lot of what ovs-ctl does, and a lot of that functionality was clearly migrated from the init scripts to ovs-ctl with the move to support systemd. But it was always a better and more flexible model to have the separate control script and an init script/unit file that made use of it.

Comparison available?

Posted Dec 11, 2024 19:38 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

I'm not clear why this wrapper needs to exist. It looks like it's completely redundant.

Is the reason "# Protect the service startup against concurrent ntpdate ifup hooks"?

Comparison available?

Posted Dec 11, 2024 20:16 UTC (Wed) by rweikusat2 (subscriber, #117920) [Link] (2 responses)

I have no idea and don't really care as that's besides the point I was making. It doesn't even really matter if the script is being used because there's no other sensible way to accomplish whatever it's doing or because someone just believed this to be the case. It exists. And hence, service start scripts written in general-purpose programming languages are still with us.

Comparison available?

Posted Dec 11, 2024 20:18 UTC (Wed) by bluca (subscriber, #118303) [Link]

Sure, but that looks like a bug to solve mroe than anything else

Comparison available?

Posted Dec 11, 2024 20:30 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

> I have no idea and don't really care as that's besides the point I was making

You're not making it well, because your example is more of a half-assed porting that is specifically highlighted by it being explicitly annotated.

I actually used dynamic systemd units myself, generated from an internal service definition language. So I know that the need exists. But it's a situation that is similar to `unsafe` in Rust: you sometimes need it, but whenever it's used, it must be explicitly annotated. It also should not be as ubiquitous as in C, the safe code should be sufficient for the vast majority of use-cases.

Comparison available?

Posted Dec 12, 2024 8:28 UTC (Thu) by beagnach (guest, #32987) [Link] (1 responses)

From the systemd 257 release notes:

> Support for System V service scripts is deprecated and will be
> removed in v258. Please make sure to update your software
> *now* to include a native systemd unit file instead of a legacy
> System V script to retain compatibility with future systemd releases.

https://lwn.net/Articles/1001657/

Comparison available?

Posted Dec 12, 2024 8:33 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

You can just copy/paste the SysV-script based unit generator from the previous releases of systemd.

Comparison available?

Posted Dec 11, 2024 0:17 UTC (Wed) by raven667 (subscriber, #5198) [Link]

While I very much appreciate the standardization, documentation and comprehensiveness of systemd it's always good to have a vibrant ecosystem of people trying different ways to build and manage a system because there is more than one valid way to do it, some are better than others but some are just different preferences and that's ok. Whether the config is a complete executable language is has pros and cons and tradeoffs, so one way isn't always better than another, or at least you can't prove it without evidence of operations using different schemes to see how it works in real life.

Congrats on their 1.0, I'm sure Guix admins are happy.

Comparison available?

Posted Dec 10, 2024 23:19 UTC (Tue) by motk (subscriber, #51120) [Link]

If I need static analysis for config files for system init then I know I have Gone Too Far and it's time for me to turn off the machine and retire.

Comparison available?

Posted Dec 14, 2024 19:43 UTC (Sat) by lispwitch (subscriber, #175059) [Link]

(Doesn't Guile support running Emacs Lisp?)

Yes, but only the core language and a small standard library (not including pcase), just enough for bootstrapping Guile-Emacs. Guile has its own pattern-matching library, (ice-9 match), that would be more useful for this task.

Comparison available?

Posted Dec 10, 2024 21:45 UTC (Tue) by gmw (subscriber, #122071) [Link] (1 responses)

> if GNU Shepherd's config files are in Scheme, does that mean that they can execute arbitrary code with the permissions of the manager process?

Sort of.
Scheme has a couple of features that, when combined, clamp down on arbitrary code execution:
1. First class environments. You can build an environment that contains all of the identifiers the evaluated code is allowed to see.
2. Lexically scoped everything. "Everything" is not just variables or functions, but also what other languages call keywords. Of particular note is the "import" keyword (or "special form" in scheme parlance).

This means you can build an eval environment with a limited set of functions (like no arbitrary file I/O) and also no ability to get access to extra functions.

Comparison available?

Posted Dec 13, 2024 18:10 UTC (Fri) by dannyobrien (subscriber, #25583) [Link]

Also, the throw-away comment at the end mentioning Spritely points to a capability-based security system in the future that would only grant scripts explicit functionality (including networked scripts).