|
|
Log in / Subscribe / Register

Init system support in Debian

By Jake Edge
October 31, 2018

The "systemd question" has roiled Debian multiple times over the years, but things had mostly been quiet on that front of late. The Devuan distribution is a Debian derivative that has removed systemd; many of the vocal anti-systemd Debian developers have switched, which helps reduce the friction on the Debian mailing lists. But that seems to have led to support for init system alternatives (and System V init in particular) to bitrot in Debian. There are signs that a bit of reconciliation between Debian and Devuan will help fix that problem.

The Devuan split was acrimonious, much like the systemd "debate" that preceded it. Many bits were expended in describing the new distribution as a waste of time (or worse), while the loudest Devuan proponents declared that systemd would cause the end of Debian and Linux as a whole. Over time, that acrimony has mostly been reduced to random potshots (on both sides); there is clearly no love lost between the pro and anti sides (whether those apply to systemd, Devuan, or both). Some recent developments have shown that perhaps a bit of thawing in relations is underway—that can only be a good thing for both sides and the community as a whole.

Holger Levsen alerted the debian-devel mailing list that the Debian "Buster" (i.e. Debian 10) release was in danger of shipping with only partial support for running without systemd. The problem is that two packages needed for running with System V init (sysvinit-core and systemd-shim) are not really being maintained. The shim is completely unmaintained and sysvinit-core has been languishing even though it has two maintainers listed.

While Thorsten Glaser at first downplayed the problems, he changed his tune somewhat after Ben Hutchings pointed out that the sysvinit package has "many open bugs with patches that have not been applied or answered". The problem goes even deeper than that, according to Andreas Henriksson:

I think sysvinit is getting closer and closer to being removed. Usually the debian way is to say that whoever reaps the benefits gets to do the work. That hasn't been the case for sysvinit for [at least] several years now. The only reason sysvinit is still around is because people who don't use it and largely don't care about it keeps doing the work to keep it afloat (while people who use it keep repeating "[everything] is fine, it's mature" and stick their heads in the sand).

And, as Petter Reinholdtsen explained, the sysvinit "team" is really just him at this point. He is "lacking both the required spare time and interest to do [a] good job, but still try to fix the gravest problems while waiting for someone with time and interest to [adopt] the packages". He too is concerned that the packages will be removed before long. So Jonathan Dowland suggested looking elsewhere for help:

Is it worth interested parties reaching out to the Devuan project regarding person-power for sysvinit maintenance? As a derivative distribution, I imagine their lives would become much harder if we did drop sysvinit; they would have to pay the cost of maintaining the sysvinit package themselves (which is what I am proposing they do now) *as well* as a rapidly growing delta of sysvinit-support/initscripts in lots of other packages, as they steadily rotted in Debian.

That set off a fairly predictable round of Devuan bashing, along with concerns that inviting Devuan developers to work in Debian would bring on a return of the mailing list battles of old. But it also resulted in a reply from Enzo Nicosia (also known as "KatolaZ"), who is one of the Devuan "Caretakers" team:

The problem that spurred this thread is that sysvinit needs a maintainer. That's why some of us are here: our intention is to help with maintaining sysvinit in Debian if possible, since we will keep maintaining it in Devuan nevertheless. You can still decide you don't want this kind of help because we stink or you find Devuan "toxic" (even if this would be against some of the principles we should instead agree upon). That could be fine, if motivated by a solid reasoning, and not just by a flame.

In the last four years there has been hatred from both camps (Debian vs Devuan), and there is no doubt that most of that could/should have been avoided on both parts. Grepping through email archives and [resurrecting] posts from 3 or 4 years ago won't help to move on, though.

I am not interested in chit-chat or flames, because those don't get packages released. The only reason I am here is that sysvinit is effectively getting kicked off Debian, and I think I can help avoiding that.

Several Debian developers were ready to let bygones be bygones and welcomed any effort toward keeping sysvinit alive in Debian. Ian Jackson invited Nicosia to a new debian-init-diversity mailing list. That mailing list has been expressly set up to avoid the hostility on (some) Debian mailing lists, Jackson said. It would appear to be the place where non-systemd init systems will be discussed and developed moving forward.

Back in debian-devel, there were other sysvinit proponents (such as Benda Xu) who did not see any real problems with the sysvinit package. But Reinholdtsen was quick to point out that attitude as helping to push sysvinit out of Debian. Beyond that, as Martin Pitt noted, the problems are far more widespread than simply being confined to the sysvinit package:

[...] the *real* maintenance is in all the packages that *ship* SysV init scripts.

SysV init leaves all the really hard problems to these, as it cannot really do much by itself. That's a fact that people that keep yelling "but SysV init was so easy!" keep finessing..

So "how many RC bugs does sysvinit have" is a completely useless metric IMHO.

Bernd Zeimetz brought up a related issue: "the typical package maintainer won't test initscripts". Many of them won't even have access to a system that runs sysvinit, he said, so they can't test them well. In another part of the long thread, though, Philipp Kern asked: "Could someone reiterate about what the current state of init diversity is supposed to be?" Russ Allbery had some thoughts on that:

I think a package of a daemon that does not inherently require any systemd-specific features and would work straightforwardly with sysvinit, but has only a systemd unit file and no init script, is not only buggy but RC-buggy. That's what Policy currently says.

It is quite easy to write a sysvinit init script for most daemons that will basically work. I don't think the maintainer is obligated to do much more than that (for instance, I don't think you need to try to duplicate systemd hardening configuration or other features that are quite challenging to do under sysvinit without more tool support, although some of that may be coming in start-stop-daemon).

He suggested that maintainers could test the init scripts by moving the systemd unit file aside and have systemd use the init script. "For nearly all daemons that don't involve tight system integration, this will be more than adequate." In another message, he explained that providing an init script is, at least partly, a gesture of goodwill within the Debian community:

My personal concern is more about the social community of the project than about the technical details. I consider providing an init script even if I don't personally use sysvinit to be extending the hand of community and solidarity to other Debian community members who use it. To say to them that their concerns have been heard and supported, even if I don't agree with their concerns. Personally, I find that extremely important, a principle that's as important as the technical quality bar we try to reach in our packages.

Overall, the dialog was relatively positive and the outcome may well lead to better maintenance of sysvinit for both Debian and Devuan moving forward. Given a little more time (and water under the bridge), Devuan's users could provide the testbed for the init scripts, which will obviously help Devuan, but will also be a boon for any Debian users of sysvinit. There is still likely to be something of chasm between the two distributions, but any rapprochement should be welcome news to most. In truth, what separates the two is pretty trivial in the grand scheme of things—the flames and acrimony notwithstanding.


to post comments

NO TO SYSVINIT - or initscripts, rather!

Posted Oct 31, 2018 21:58 UTC (Wed) by fartman (guest, #128226) [Link] (84 responses)

Here's a small - or rather not so small - suggestion to all invested parties (context: I don't use Debian, or Devuan at all, I run Gentoo):

Stop pushing sysvinit! Instead invest in communities that are trying to bring benefits systemd has in a way that does not require daemons to couple with them. There are alternatives to sysvinit that are well maintained, have an author upstream who works on bug fixes, and responds to community issues. I have been using the said project for a fair amount of time now and everything works. Everytime the sysvinit vs systemd debate is brought up, I die a little inside.

sysvinit is not comparable to systemd, init scripts are not comparable to the systemd-suite as a whole. You need proper supervision, things like runit and s6 provide it.

I use s6[1] therefore I would request the Debian/Devuan people to come forth, and more prominently, evaluate it against sysvinit, ditch sysvinit, and invest efforts in something considerably modern to today's standards. The s6 suite has a lot of the things you have in systemd (except the complex transactional dependency engine, which systemd and SMF are the only known implementations of), be it startup notifications, file descriptor holding, and so on. It can be used with an RC, such as *OpenRC* (which might be considered a choice for inclusion along with s6), but it comes with an RC of its own. The author works hard on making things work, but needs some help, it's been a one man show, but I think they're going in the right direction and not beating the old dead horse over and over. I won't make this too long and request you to look into it

Also, as far as systemd-shim is concerned, please, there is a decoupled version of logind called elogind already available and integrated in non-systemd distros (namely Void Linux and Gentoo Linux). As far as timesyncd, timedated, and hostnamed are concerned, they can already be built from systemd sources and made to work with minimal patching (and now consider this, s6 comes with a *compat binary for daemons that use the systemd notification protocol*, so you don't even need to patch out that part, win!).

What's more, this new combination of s6+s6-rc/OpenRC can be used in Debian/kFreeBSD (for all the 10 users there) and other Debian ports as they're portable to most Unices.

This is a massive change, but you cannot cling on to sysvinit for the rest of the future. I would like to see some improvement, in that, things like s6, nosh, runit be considered, and even shipped by default. They're easily replaceable hence if anyone wants to go back to sysvinit, they can!

Summarising, I wish to see:

* Debian and Devuan to come together and let go sysvinit. Instead, choose a modern alternative like s6, runit, or nosh (In my opinion, preferably s6, as it will be the least painful of all three options, and does things better than the other two, supports more things when the entire suite is used together (again, nothing depending on each other at runtime, but just sharing a common library), but that is something debatable and I'd be happy if it is debated).

* Make the selected one the default, and ship with it (considering there are enough reasons I listed above that would help its adoption as the default).

* systemd-shim begone and replaced with elogind+timedated+hostnamed from upstream, each of which are maintained properly.

* Integrating sd_notify daemons with the s6 suite has a less painful path even when the said daemon uses its notification protocol (as s6 comes with compatibility code for it [3]). It has provision to perform file descriptor stashing for daemons without putting them in PID1, like systemd does.[2]

* The s6 suite also has "socket activation"[4] related tools (a marketing term for the UCSPI toolset, well known among daemontools users), and is a complete alternative solution to much of the problems systemd tries to solve, but in an arguably better way.

* There's a small s6-sudod[6] daemon to allow you to run things in a pristine context like systemd-run would (since PID1 forks off the service), moreover, a small accessrules library for access control over its socket, an idempotent database those rules are compiled to and loaded, an alternative to policykit.

* You can easily configure s6 to run as a user session manager, an alternative to systemd --user. elogind already supports the user bus, with some glue, it should be possible to support proper user session (or with enough will :) ).

* Small codebase, less things to do, less bugs, less maintainence.

* Sandboxing is modifying the execution context of the service before executing into it, something easily done with execline, it's accompanying tools, and if need be, Linux specific tools like unshare/nsenter from util-linux/busybox. This is how it happens in systemd as well, a child process is forked off from PID1 which takes care of setting up the execution context of the service before executing into it. Moving into cgroups and applying resource limits is just writing a few values to /sys/fs/cgroup and related files.

* s6 bundles are concise to write and maintain, come with a small shell of their own to minimise boilerplate, avoid forking, save resources[5] and this burden can be shifted to a team that maintains them for all packages (because asking people to learn how to write s6 bundles will unnecessarily over power all other benefits and lead to it being ruled out, I'm afraid).

Note that I am not enforcing any of these, I am in favor on discussion happening on what to choose, but please, PLEASE, don't make the mistake of choosing sysvinit again (or more precisely, initscripts, s6 works just fine with sysvinit - the init). There are better alternatives, they need some polish and integration and they will end up working as well as systemd does, giving you most - if not all features (and flexible enough to implement the missing yourself).

Here are some links:
[1]: http://skarnet.org/software/s6/
[2]: http://skarnet.org/software/s6/s6-fdholder-daemon.html
[3]: https://skarnet.org/software/misc/sdnotify-wrapper.c
[4]: http://skarnet.org/software/s6/socket-activation.html
[5]: http://skarnet.org/software/execline/
[6]: http://skarnet.org/software/s6/s6-sudo.html

NO TO SYSVINIT - or initscripts, rather!

Posted Oct 31, 2018 22:02 UTC (Wed) by fartman (guest, #128226) [Link]

and by shipping as default, I mean shipping it as the default alternative to systemd - replacing sysvinit, not switching the default itself.

NO TO SYSVINIT - or initscripts, rather!

Posted Oct 31, 2018 22:08 UTC (Wed) by rweikusat2 (subscriber, #117920) [Link] (39 responses)

The problem with "real world initscripts" is that they implement way to much things they shouldn't be implementing and that there's a lack of support tools for common process management tasks. Both problems can be solved: I happen to have a set of such support tools I wrote because I needed them[*] (totalling 4143 lines of C code). These exclude "dependencies" and "readiness notification" and other nonsense people who don't understand TOCTOU races are wont to believe to be solutions to problems which don't exist outside of their respective imagination.

[*] I'm one of the people Mr Bradley Kuhn considers to be "morally reprehensible" in that I work as developer based on a "standard" software industry employment contract --- everything I write which could conceivably be useful for anything automatically becomes "proprietary IP" of my employer. Hence, this is (unfortunatety) all "secret code".

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 3, 2018 14:03 UTC (Sat) by drag (guest, #31333) [Link] (38 responses)

The problems of 'writing real init scripts' have been solved... over and over and over and over again. They have been solved by thousands of different people in a thousand different ways. It's been solved in shell, solved in python, solved in perl, solved in C. Sometimes people rewrote the same programs multiple times in different languages.

People have been solving these problems for decades now. Over and over again.

And eventually somebody came along, solved them, wrote it all in C, created a formal project and got all of the major distributions to adopt it. They call it Systemd.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 4, 2018 19:56 UTC (Sun) by rweikusat2 (subscriber, #117920) [Link] (37 responses)

Did you read the text you claim to be replying to?

systemd is one of half a gazillion replacement inits written in $compiled_language which roughly all provide about the same feature set (and make the same errors), there's nothing special about it beyond "it's the RedHat replacement init!". It's arguably an improvement compared to the mess RedHat, SuSE, Debian etc used to ship in /etc/init.d. So is upstart, runit, s6, monit, $younameit etc, they just never became "mainstream" because RedHat didn't adopt any of them.

But the "mess in /etc/init.d" RH, SuSE, etc used to ship wasn't created by some force of nature, it just represents about 20 years of "organic shell script growth" based on lack of sensible support tools for process management and Bourne shell ad hocery. A large part of this codeheap isn't really good for anything and could just be deleted (I still fondly remember a comment in one SuSE init script complaining that it was necessary to work around bugs in another SuSE init script --- why not just fix them?). Another large part can be condensed into a set of support tools written in C which can be reused in other init scripts (or simliar situations).

That's not much different from what systemd and other members of the zoo offer, it's just more flexible in that it retains a fully capable high-level, standardized programming language as backbone engine instead of some non-extensible, declaratory language intepreted by a dedicated interpreter which has some random set of "process management support code" aggregated in one (or some) fairly large program(s).

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 4, 2018 22:00 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link] (20 responses)

> systemd is one of half a gazillion replacement inits written in $compiled_language which roughly all provide about the same feature set (and make the same errors)
Nope. systemd's flash of brilliance was the use of cgroups to isolate daemons, which allowed it to actually reliably track them. For the first time ever. No other init system did this before.

Everything else simply follows. If you can isolate daemons then you don't need PID files. You can place limits on their resource use and reliably redirect logs.

If you are aware of any other pre-systemd init system that does the same, I'm all ears.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 20:07 UTC (Mon) by rweikusat2 (subscriber, #117920) [Link] (7 responses)

Control groups have existed since at least 2004 and their very purpose is "aggregating/ partitioning sets of tasks". It's not particularly brilliant to use them for this. They're also not really needed unless one has to deal with misbehaving or even actively hostile software[*]. For the usual case that this is not the case, normal parent-child relationship and ensuring that managed processes don't background themselves are entirely sufficient. The lxc-project has been using them for containers since before systemd existed (Wikipedia puts its beginnings at 2010. In 2010, I was using lxc as base for virtualized IPsec VPN servers for mobile clients).

"PID files" are a grotty, traditional UNIX hack to prevent multiple instances of a program from running (a very dubious practice to begin with) and to enable some measure of process control by sending signals to the process whose pid is recorded in a pid file (which may or may not be the process actually supposed to get these signals). A well-behaved program won't 'escape' (or at least can be asked to avoid doing so) from its parent. Hence, the parent can reliably send signals to it and can provide an interface enabling other software to do so. It can also provide "only one running instance" feature reliably and can set resource limits for processes which either aren't privlileged or at least don't actively sabotage management.

[*] So far, I haven't encountered the case of a program which actively resisted management. Hence, I had no reason to create a tool for supporting this.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 20:18 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (6 responses)

> Control groups have existed since at least 2004 and their very purpose is "aggregating/ partitioning sets of tasks". It's not particularly brilliant to use them for this.
I don't remember other init systems doing this, though. It's brilliantly simple like many other inventions.

Socket activation and fully asynchronous design are also new.

> For the usual case that this is not the case, normal parent-child relationship and ensuring that managed processes don't background themselves are entirely sufficient. The lxc-project has been using them for containers since before systemd existed (Wikipedia puts its beginnings at 2010. In 2010, I was using lxc as base for virtualized IPsec VPN servers for mobile clients).
Uhm, systemd was released in 2010. At best this is an example of convergent evolution.

> Hence, the parent can reliably send signals to it and can provide an interface enabling other software to do so. It can also provide "only one running instance" feature reliably and can set resource limits for processes which either aren't privlileged or at least don't actively sabotage management.
The problem here is that SysV (and its ilk) doesn't really have a parent that can be used to manage its children. For this to work you need a specialized persistent process that stays up at all time, because if it dies all of its children will also be killed.

The earliest example of such process is DJB's daemontools. It follows the same ideology as systemd, it has declarative service description and it uses traditional UNIX process groups to track processes. It doesn't have dependencies or socket activation, though. It's also not asynchronous so a badly-written script can lock it up.

Upstart attempted to do better but its design honestly sucks. It also uses hacks like ptrace() to detect double-forking daemons.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 20:42 UTC (Tue) by rweikusat2 (subscriber, #117920) [Link] (3 responses)

Cgroups were created to aggregate groups of processes for control/ management purposes. It's not particularly brilliant when something which is intentionally not portable beyond Linux uses them for this exact purpose, especially considering that there were even technical precedents in userspace: The lxc version I originally used in 2010 was 0.7.3 and it had been in development for some years at that time already. There's also an intentional design choice here: Cgroups are used to solve the problem of controlling misbehaving or intentionally hostile other processes, ie, programs written such that they cannot be managed with traditional UNIX facilities built around the notion of cooperating processes. I doubt this was ever a design objective for daemontools. It's certainly not a problem I have encountered in practice so far (my efforts in this respect being strictly restricted to providing process management facilities I demonstrably need).

Socket activation with the intent to reduce system boot time and provide better overall system performance saw the light of the day with the 4.3BSD inetd.

Fully asynchronous, that is, event-driven servers have also existed for a long time. Dan Kegel's original C10K paper mentioned the concept. The (older) racoon IKEv1 daemon also works in this way. This is again a design choice: Manage more than one process with the same process management process and accomodate buggy or intentionally hostile "other software". One could argue that buggy other software should rather be fixed and that one shouldn't be using intentionally hostile one.

The sysvinit program actually supports (some very limited form) of process management: It's possible to define per-runlevel commands which will be started automatically whenever a runlevel is entered and restarted if they ever terminate. Traditionally, this is unused except for gettys (to support user logins on all kinds of text terminals, virtual, physical, serial console etc). But the initscripts/ runlevel management system is completely independent of that: All init knows about this is that it's supposed to execute the command /etc/init.d/rc with the runlevel number as argument (both configurable). This means it's possible to use the sysinit program (or any other kind of "init init" which restricts itself to this role) together with any other process management program/ tool. Eg, the one I'm using is just a command which is executed during service start. Basically, it will execute another command with certain arguments in a subprocess. Should this other command terminate, it will be respawned. Further, the command creates an AF_UNIX stream socket for external process control. This can be used to query the status of the managed command, to initiate a stop or restart and to send abitrary signals to it.

If the process serving as init ever terminates, the kernel will panic. But ordinarily, death of a parent process doesn't affect its children. They'll be inherited by the init process which will eventually collect their exit status to prevent them from turning into zombies. Otherwise, backgrounding a program by forking and terminating the parent process wouldn't be possible.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 21:51 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

Again, I'm not saying that systemd invented anything new. It just combined multiple obvious ideas in a brilliant way.

I'm not aware of any other init system that does this, even to this day. There were scattered bits and pieces here and there, but not a coherent design.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 7, 2018 17:04 UTC (Wed) by rweikusat2 (subscriber, #117920) [Link] (1 responses)

As I already wrote: systemd is nothing but daemontools reimplemented once again, just based on a somewhat different set of assumptions about the technical issues such a system should handle. Like C string libraries, init systems are essentially rites-de-passage programming: Given enough time, everybody will have implemented one which solved (or sort-of solved) a particular problem the concept could be applied to (more or less forcibly).

But this again starts to drift from a technically interesting topic (technically interesting to me at least) towards marketing statements ("This shoesine is brilliant!") ... :-)

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 7, 2018 17:47 UTC (Wed) by pizza (subscriber, #46) [Link]

> systemd is nothing but daemontools reimplemented once again,.

That's like saying that Linux is just a reimplementation of UNIX.

(Along with the implication that there nothing new, different, or otherwise improved between the two)

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 12, 2018 2:44 UTC (Mon) by fest3er (guest, #60379) [Link] (1 responses)

"The problem here is that SysV (and its ilk) doesn't really have a parent that can be used to manage its children. For this to work you need a specialized persistent process that stays up at all time, because if it dies all of its children will also be killed."

I'll put forth that sysvinit *is* the parent that manages its children. It simply has never been used that way. Instead, the old method of using RC scripts (did this come fom Berzerkely?) was used (or continued). The very useful part of sysvinit (most of what's in inittab) was relegated to the status of a curious-looking red-headed stepchild useful only for running getty.

If sysvinit was extended to, say, 128 run levels to handle dependencies, actually used as intended, and possibly extended to facilitate enabling/disabling/activating/deactivating daemons without needing to edit inittab, sysvinit would do everything a lot of people need without a lot of complexity.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 12, 2018 9:39 UTC (Mon) by anselm (subscriber, #2796) [Link]

If sysvinit was extended to, say, 128 run levels to handle dependencies, actually used as intended, and possibly extended to facilitate enabling/disabling/activating/deactivating daemons without needing to edit inittab, sysvinit would do everything a lot of people need without a lot of complexity.

Sysvinit can actually deal with more runlevels than the customary six. It's just that people seldom bother to define more of them.

That still wouldn't give you the consistency and convenience that systemd provides. Remember that much of the advantage that systemd confers, from the POV of system management, is in configuration, e.g., the obvious division between system-provided and locally-provided configuration settings which makes it easier to install a new upstream version of a software package without stomping on local customisations. You could obviously come up with a management layer for sysvinit that generates the real /etc/inittab file from a set of snippets in various directories, but the simple fact is that that hasn't yet been done, and now that systemd exists (and can be pared way down if required) the need for it is much less than it may have been earlier.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 21:28 UTC (Mon) by luto (subscriber, #39314) [Link] (11 responses)

I would personally rather see systemd use subreapers for this. If needed, the kernel could gain whatever mechanisms were needed.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 23:02 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (8 responses)

I guess you can replicate cgroups process group functionality using subreapers. But why?

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 23:12 UTC (Mon) by luto (subscriber, #39314) [Link] (7 responses)

To avoid using cgroups! This has two major benefits. First, if I want to save resources, I can run on a kernel without cgroups. Second, and more importantly, it would let me use systemd without using systemd as a cgroup controller, which would mean that I could arrange other cgroup policies that have nothing to do with systemd.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 23:17 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (6 responses)

I think you can simply use systemd without any controllers in a separate tree and manage your tree with controllers yourself.

Saving kernel size is a valid concern but I'm not sure how much you're going to save if you disable all cgroups controllers.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 23:24 UTC (Mon) by luto (subscriber, #39314) [Link] (5 responses)

But what if I actually *want* systemd to manage my services?

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 23:28 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (4 responses)

It will be able to do it, just without any resource controllers. And you can have a separate cgroups tree with controllers attached.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 12, 2018 17:22 UTC (Mon) by nix (subscriber, #2304) [Link] (3 responses)

I thought that in these days of cgroupsv2 there was a single unified tree, which systemd insists on managing?

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 12, 2018 19:54 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

You can still have multiple trees. The central tree with controllers attached and multiple v1-like trees without controllers. This is enough for systemd to manage the processes.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 13, 2018 0:18 UTC (Tue) by nix (subscriber, #2304) [Link] (1 responses)

Oh, so the only restriction with v2 is that *controllers* can only appear on one unified tree? You can have lots of other trees for the sole purpose of defining relationships between processes?

That's... useful. As in, I think I'll have a use for it in my own work in the next week. :) I wish I'd realised v2 worked this way years ago!

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 13, 2018 0:21 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

Correct. You can still use a thin tree only for processing tracking.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 23:19 UTC (Mon) by anselm (subscriber, #2796) [Link]

The subreapers would give you the opportunity to track processes that exit, but you'd still need to use cgroups to do resource control.

NO TO SYSVINIT - or initscripts, rather!

Posted Mar 13, 2019 16:30 UTC (Wed) by nopsled (guest, #129072) [Link]

Have you seen https://www.freebsd.org/cgi/man.cgi?query=procctl&sek...

It would be nice if Linux had the PROC_REAP_KILL stuff. I think one could improve the FreeBSD interface a tad bit when incorporating it, but the concept looks sound.

systemd ofcourse has one supervisor per context (system or user), but this will be especially useful for supervisors that have one monitor per service (s6, runit style), and that these can be nested supervisors (so an option to not cross subreaper boundaries where each supervisor is a subreaper, or a way to kill inherited children only, and a way to kill direct children only, and OR the flags to do both). You could get creative, but if done right, this could solve a lot of woes with process management.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 13:17 UTC (Mon) by anselm (subscriber, #2796) [Link] (15 responses)

systemd is one of half a gazillion replacement inits written in $compiled_language which roughly all provide about the same feature set (and make the same errors), there's nothing special about it beyond "it's the RedHat replacement init!".

It makes more sense to think of systemd as “basic plumbing” rather than “init”. Systemd does include an init process but it tries to address many low-level OS issues that the other replacement inits don't. One advantage of this is that systemd offers a unified approach to configuration that actually extends the same features of, e.g., subprocess creation to wherever subprocesses are created in systemd. With sysvinit or, for that matter, the other replacement inits, this sort of thing is still handled in an ad-hoc manner in a variety of tools that have been cobbled together over decades with no cohesive design.

In theory it would of course be perfectly possible to come up with a set of reusable “support tools written in C” that init systems could avail themselves of, and make these popular across distributions, but nobody except Lennart Poettering and Kay Sievers actually condescended to doing that in practice. Now, from a practical POV, that ship has sailed, since the adoption of systemd by virtually all mainstream distributions has placed the bar for basic-plumbing replacement a lot higher than it used to be when sysvinit+cron+at+inetd+… was the “state of the art”. Anything put forward now must not just provide a significant advantage over sysvinit+…, but over systemd, and with the amount of clever engineering that has gone into systemd over the last decade or so, that is a considerably more difficult task.

Also note that nobody forced Red Hat, SUSE, Debian, … into adopting systemd. Out of the existing init system replacements at the time it just happened to be the one with the most enticing value proposition. (It's also worth reiterating, since people here have been calling systemd “Red Hat's init”, that Lennart Poettering and Kay Sievers started systemd on their own time; nobody from Red Hat went to them and said “write an init replacement for Red Hat Linux”.)

That's not much different from what systemd and other members of the zoo offer, it's just more flexible in that it retains a fully capable high-level, standardized programming language as backbone engine instead of some non-extensible, declaratory language

Systemd lets you call out to shell scripts if you need to. But in general, adding the shell to the mix only makes things more complicated for those many if not most cases which are really quite simple and straightforward. The declarative configurations of systemd are easier to read, understand, reason about, standardise upstream, maintain, and teach, than shell scripts and you can still write and use old-fashioned sysvinit-style init scripts with systemd if you absolutely must.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 20:09 UTC (Mon) by rweikusat2 (subscriber, #117920) [Link] (14 responses)

> In theory it would of course be perfectly possible to come up with a set of reusable “support tools written in C” that init systems
> could avail themselves of, [...] but nobody except Lennart Poettering and Kay Sievers actually condescended to doing that in
> practice.

The set of "open source developers" on this planet is a proper subset of the set of developers.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 21:23 UTC (Mon) by anselm (subscriber, #2796) [Link] (13 responses)

You conveniently neglected to quote an important part of my sentence, namely

to come up with a set of reusable “support tools written in C” that init systems could avail themselves of, and make these popular across distributions, but nobody except Lennart Poettering and Kay Sievers actually condescended to doing that

As you say, init replacements are a dime a dozen, but none of them really achieved a lot of traction (with the possible exception of Canonical's Upstart). It is to Lennart's and Kay's credit that they managed to come up with something so convincing that virtually all major Linux distributions have rallied around it, and the resulting network effects are a major win for the platform.

(In the non-open-source area, practically every provider of proprietary Unix had come up with its own replacement for sysvinit even before systemd got started – Linux used to be the last holdout of sysvinit – but that is neither here nor there.)

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 22:12 UTC (Mon) by rweikusat2 (subscriber, #117920) [Link] (12 responses)

> You conveniently neglected to quote an important part of my sentence, namely

[...]

I've ignored that on purpose because while I'm somewhat interested in discussing the technology, I have zero interest to engage in a discussion of the surrounding politics and other social mechanics as that's bound to degrade into an entirely pointless flamewar of a very familiar shape quickly.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 23:17 UTC (Mon) by anselm (subscriber, #2796) [Link] (11 responses)

For the record, I agree that there have been enough flame wars about systemd. But it is important to get one's basic facts right, including that (a) systemd was not started as a Red Hat project, and certainly by now includes active developers from many parts of the community, and (b) other distributions (whose developers, on the whole, aren't gullible incompetents either) adopted systemd because they felt it was technically the Right Thing to do, not because they were somehow (how exactly?) coerced to do so by Red Hat. Remember that Debian in particular did not take the decision to default to systemd at all lightly.

The fact that most major Linux distributions did adopt systemd as their default init system differentiates systemd from all of the other init system replacements (except Upstart), and as such it is reasonable to ask what the other init system replacements “did wrong” if they're as clearly superior to systemd as their proponents claim but yet, unlike systemd, didn't manage to convince multiple mainstream Linux distributions of their merits. It's easy to blame “politics and other social mechanics” but we should not altogether discount the fact that systemd, too, includes a good dash of clever engineering.

In any case, no matter what people think about systemd, it's not going away anytime soon. It is certainly not going away unless somebody comes up with something that the major distributions consider obviously better than systemd, and by now that is a pretty high bar to clear (especially since being “only” an init system replacement is no longer enough; to compete successfully with systemd there needs to be some proposition for the overall basic-plumbing aspect of things). It's still not entirely inconceivable, but somebody will eventually have to start writing code, rather than LWN comments about how terrible systemd is.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 8:41 UTC (Tue) by ras (subscriber, #33059) [Link] (8 responses)

> is reasonable to ask what the other init system replacements “did wrong”

As a Debian Developer watching on when this happened, what they "did wrong" had nothing to do with the merits of the init system. ConsoleKit and a few other things were bit rotting badly at the time, and logind, dbus and a few other things got moved into the same source ball as systemd. I remember wondering consolekit and logind did that was so important - it turned out the most important thing is gnome needed something to give the local user (ie the one running the X server) permission to access local resources (like the sound card, and the keyboard), and gnome decided to use what systemd provided. (As I recall gnome didn't have a whole pile of alternatives to choose from.)

Gnome was Debian's default Window manager, so it was game over.

I use lxde on my laptop, and administer a whole pile of servers. Systemd works, but what has to happen is so trivial I don't have a clue what that 10MB of code in systemd could possibly be doing. I'm pretty sure I would of been happier with something much simpler. In fact I needed to boot a Debian chroot on a system without systemd once, and as this article points out the sysv scripts are indeed bit rotting so I wrote a shell script to interpret systemd's .service files for a headless server. It was under 300 lines. It makes the 1/2 a million lines in systemd a bit of a puzzle.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 10:37 UTC (Tue) by lkundrak (subscriber, #43452) [Link] (6 responses)

> I don't have a clue what that 10MB of code in systemd could possibly be doing

It's more like 1 MB:

$ size /usr/lib/systemd/systemd
   text	   data	    bss	    dec	    hex	filename
1238373	 259368	    416	1498157	 16dc2d	/usr/lib/systemd/systemd

If you also count other tools from the systemd repository then their file name usually gives a good clue about what the tool does. See: systemd-resolved -- a resolver. systemd-timedated -- manages time and date. systemd-hostnamed -- manages host name. Et cetera.

Also, both the init daemon and the supplementary tools are thoroughly documented. If you're really curious I wholeheartedly recommend trying out the man utility. It could turn out useful in your "administering a pile of servers" endeavour.

In an unlikely case the documentation won't be sufficient, you should know that the source is available for free and is perhaps one of more reasonable C programs that you'll find running on a typical Linux system.

Hope this helps.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 22:36 UTC (Tue) by ras (subscriber, #33059) [Link] (5 responses)

> It's more like 1 MB

I was referring to the installed size reported by Debian. If you are going to use size, you should at least include the additional 1.5MB in libsystemd-shared-232.so. If you want just count binary size, you should probably be include everything in /lib/systemd, which is over 5MB. That misses a bit of it installed elsewhere of course. The single figure is maybe less accurate, but is a damned sight easier to come by and I doubt it effects the error margin overly.

But if you insist on using systemd + libsystemd-shared's binary size, it's 2.5MB versus 34K for a sysV init I have lying around.

> Hope this helps.

Not really. You''re talking to someone who has an accepted pull request for the systemd documentation. You are, right there is a *lot* of documentation, and it's very good. But don't underestimate the value of only something that does a similar job that only needs a few pages of documentation.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 22:39 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (4 responses)

> But if you insist on using systemd + libsystemd-shared's binary size, it's 2.5MB versus 34K for a sysV init I have lying around.
But then you need bash/dash and a ton of other utilities (sed, awk, ...) for the initscripts. You can mostly get away with busybox these days, which is around 2MB.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 23:18 UTC (Tue) by ras (subscriber, #33059) [Link] (3 responses)

> But then you need bash/dash and a ton of other utilities (sed, awk, ...)

True, but these are usually there anyway, and on the systemd side of the ledger I didn't count libc or systemd's other 20 shared dependencies.

The Debian installed size is so much easier. Looking on an old machine, the installed size of sysV init and ancillary stuff is about 900KB. Adding inetd and cron increases it by 0.4MB. 1/10th the size does seem about right.

This stuff is not that hard. I wrote an "house keeping" system for a container as a dash script. It booted, started and stopped services, provided timers, did scheduled backups, rotated log files, maintained a status web page for monitoring and provided a framework for building containers using it. It was 3K lines and would do a fine job of managing my laptop.

Not all init system's can be simple. An init system for clusters like kubernetes is complex for reasons. But for a single machine with a couple of tasks, it's damned hard to see what added value systemd's complexity brings.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 23:41 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

I acutally did an embedded system with systemd and musl about 5 years ago. It didn't have busybox but did have a Dropbear SSH daemon. The total size was less than 1.5Mb for systemd and dropbear.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 23:58 UTC (Tue) by ras (subscriber, #33059) [Link] (1 responses)

> The total size was less than 1.5Mb for systemd and dropbear.

How did you pull that off? On Debian stretch systemd + libsystemd-shared-232.so alone are 3.3MB.

Hmmm. Looking at the debian archives the systemd v44 .orig.tar.gz ball dates from 6 years ago, and is 860K. systemd v239 .orig.tar.gz is 6.8MB. Have you noticed a 8 fold increase in functionality?

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 11, 2018 11:24 UTC (Sun) by mpr22 (subscriber, #60784) [Link]

That comparison is not nearly as valid as you seem to want it to be, because the v239 source tarball has broader scope than the v44.

Most trivially, v239 includes udev (whose tarball at the time of the merger in v183 was about 70% the size of the systemd tarball) and the udev HWDB (which was merged later, and is itself pretty huge).

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 11, 2018 10:13 UTC (Sun) by ms_43 (subscriber, #99293) [Link]

logind was part of the systemd project since its inception[1], and DBus (the user-space daemon) has never been[2] part of the systemd project.

There was an opportunity to contribute the necessary maintenance and bug fixes to ConsoleKit to keep it working, but instead the anti-systemd camp decided to troll the Debian technical committee decision process. I'm very happy that this turned out not to be a successful strategy.

[1] https://github.com/systemd/systemd/compare/v29...v30
[2] https://packages.debian.org/sid/dbus

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 6, 2018 17:44 UTC (Tue) by rweikusat2 (subscriber, #117920) [Link] (1 responses)

I'm totally serious about not getting dragged into this kind of discussion, minus pointing out that "appeal to popularity" (ad populum) is a well-known term.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 7, 2018 1:13 UTC (Wed) by anselm (subscriber, #2796) [Link]

That would be to the point if the claim was that systemd is good because it is popular. A more likely proposition, however, is the converse: that systemd is popular because – at least in the eyes of the developers in most mainstream Linux distributions who decide what init system to default to – it is good.

NO TO SYSVINIT - or initscripts, rather!

Posted Oct 31, 2018 22:18 UTC (Wed) by jccleaver (guest, #127418) [Link] (27 responses)

This is a lot of text that completely and totally misses the point.

s6, and any other service manager that doesn't require being executed as PID 1 for its operation, such as DJB's daemontools, or even xinetd/inetd, can function as a completely distinct add-on to any sysv-style init system.

When I say "I want sysvinit", I don't mean "ban all other forms of service management"; I mean "I don't want systemd running the boot sequence (before services have begun), and I sure as hell don't want it as PID 1".

I've run daemontools for 15 years on RH systems, fired off from /etc/init.d/svscan with a simple /sbin/chkconfig svscan on + /sbin/service svscan start. Works great.

I can't speak for the Debian side either, and I'm aware that their init system as a whole (prior to the whole systemd mess everywhere) wasn't quite as clean as the Red Hat initscripts environment, but saying "NO TO SYSVINIT" misses the point about this completely.

All I want is a reliable PID1; a reliable, *deterministic* boot sequence; and a simple environment from which to launch other things in userland.

If you need a service manager, either because your daemon doesn't know how to behave as a proper *nix daemon, or because you need more comprehensive restart and logging capabilities, then install a service manager of your choosing (be it xinetd, s6, daemontools, or -yes- even systemd-not-as-PID1) and run that daemon through that. Otherwise, a traditional init script serves its purpose well and gets out of the way of a system administrator quickly and easily.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 2:06 UTC (Thu) by milesrout (subscriber, #126894) [Link] (25 responses)

> When I say "I want sysvinit", I don't mean "ban all other forms of service management"; I mean "I don't want systemd running the boot sequence (before services have begun), and I sure as hell don't want it as PID 1".

'I want sysvinit' does not mean 'I don't want systemd'. That's a false dichotomy. There are lots of options that aren't systemd and aren't sysvinit. Comparing systemd to sysvinit is pretty unfair to sysvinit. I can understand why someone would choose systemd if their only other option was sysvinit. I can't understand why someone would choose systemd if their other options include OpenRC.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 4:18 UTC (Thu) by mchehab (subscriber, #41156) [Link] (15 responses)

> Comparing systemd to sysvinit is pretty unfair to sysvinit.

That pretty much depends on the use case.

For my most common use case it is just the opposite: systemd is a very poor replacement for SysV.

I have here several machines here that I use just for testing - most of the time Kernel patches.

When they start, I want two things to happen real quick (no more than a couple of seconds):

- enable network and sshd.
- open a shell at the serial console;

I also need a real quick reboot that won't wait for daemons to shut down, doing just sync and unmount - and, if unmount fails, reboot anyway - as the Kernel might have OOPSed.

Also, I don't have any usage for journald, as I'm using rsyslogd anyway, in order to export logs to some other machine.

Never found a way to do that with systemd.

Starting with logs, I never found a way to disable journald on modern systemd-based distros.

Also, during boot, sometimes systemd spends several minutes to start network on a system - and debugging the culpit that it is slowing down the network to start earlier is really hard. Last time I had to do that, I lost *several hours* and ended by uninstalling lots of stuff from the machine and by hacking several systemd service files.

Finally, systemd only enables shell at console after initializing everything else (changing this behavior is possible, but not trivial).

With sysV, implementing such use case is trivial: just remove links from rc3.d or rename them, in order to change the startup priority.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 4:42 UTC (Thu) by fartman (guest, #128226) [Link] (13 responses)

Here are some clues:

> When they start, I want two things to happen real quick (no more than a couple of seconds):
> - enable network and sshd.
> - open a shell at the serial console;

Define a target, make it pull these things, and make systemd boot to it on the kernel command line with systemd.unit=myspecialtarget.target. man systemd.target, systemd.unit, systemd (for debug shell).

> I also need a real quick reboot that won't wait for daemons to shut down, doing just sync and unmount - and, if unmount fails, reboot anyway - as the Kernel might have OOPSed.

systemctl reboot --force, https://www.freedesktop.org/software/systemd/man/systemct...
Though you always have sysrq as the good old hammer.
Also, kill -RTMIN+15 1, https://www.freedesktop.org/software/systemd/man/systemd....

> Also, I don't have any usage for journald, as I'm using rsyslogd anyway, in order to export logs to some other machine.
> Never found a way to do that with systemd.
They have https://www.freedesktop.org/software/systemd/man/systemd-... and https://www.freedesktop.org/software/systemd/man/systemd-...

Although I myself just configure it to log locally and pass things on to rsyslog.

> Starting with logs, I never found a way to disable journald on modern systemd-based distros.

You can't, you loose kernel logs, you lose stdout/stderr, you lose audit in the journal, etc. PID1 is tightly coupled with journald.
That's what upstream says, you really can, but then need to set DefaultStandardOutput=/DefaultStandardError= in /etc/systemd/system.conf to whatever socket/fifo/etc you might want to pick those up from, or syslog. Also, in this case you end up losing native journal messages (unless you write something that forwards them to syslog).

Better idea for it to maintain a tight buffer in volatile storage and pass things on to rsyslog (this way systemctl status works fine too).

> Also, during boot, sometimes systemd spends several minutes to start network on a system - and debugging the culpit that it is slowing down the network to start earlier is really hard. Last time I had to do that, I lost *several hours* and ended by uninstalling lots of stuff from the machine and by hacking several systemd service files.

Sounds like the respective *-wait-online service is not enabled and not activating network-online.target, and/or you're not depending on that from your unit file. Could be both, could be just one of those. Also, network filesystems automatically gain a dep (or use _netdev as an option in fstab).

> Finally, systemd only enables shell at console after initializing everything else (changing this behavior is possible, but not trivial).

https://www.freedesktop.org/software/systemd/man/kernel-c...
You want systemd.debug_shell (started as early as possible, root shell).

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 11:57 UTC (Thu) by mchehab (subscriber, #41156) [Link] (12 responses)

> Define a target, make it pull these things, and make systemd boot to it on the kernel command line with systemd.unit=myspecialtarget.target. man systemd.target, systemd.unit, systemd (for debug shell).

I actually did the opposite: I disabled several services at the multi-user target, and had to patch some *.service files in order to reduce timeout and do some cleanups at the dependency chain.

Anyway, adjusting the target is the easiest part. The main problem is how to get rid of the long chain of dependencies that the network service has. See, for example, the time required to initialize network manager on an Intel i7-5557U CPU with M.2 SSD disks (on a machine whose I used lots of tricks to optimize it):

NetworkManager.service +1.241s
└─dbus.service @47.065s
└─basic.target @46.210s
└─sockets.target @45.940s
└─lxd.socket @42.127s +3.542s
└─sysinit.target @39.893s
└─systemd-update-utmp.service @37.305s +2.312s
└─systemd-tmpfiles-setup.service @30.462s +1.908s
└─systemd-journal-flush.service @23.994s +3.461s
└─systemd-remount-fs.service @2.192s +9.791s
└─systemd-fsck-root.service @584542y 2w 2d 20h 1min 37.360s +821ms
└─systemd-fsckd.socket @5.262s
└─system.slice
└─-.slice

With systemd, the minimal time I was able to have network working is around 45-60 seconds. When using a mechanical HD, it takes *a lot* longer time.

The beauty of sysV is that it doesn't have a dependency chain: the scripts just follow the order defined by the admin. Optimizing the chain order is a way simpler.

Thanks for the tip with journald. I may try to play with some options there. If you see the above log, journald slowed network start by ~3.5 seconds.

> Sounds like the respective *-wait-online service is not enabled

I had to play with those. My main goal here is not to wait: it is just the opposite. Ideally, I would really love to have the network started even before mounting the disks, in order to be able to get the logs at the remote machine as soon as possible - and on some machines even use a network console.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 13:42 UTC (Thu) by pizza (subscriber, #46) [Link] (11 responses)

> With systemd, the minimal time I was able to have network working is around 45-60 seconds. When using a mechanical HD, it takes *a lot* longer time.

Does that include the BIOS/GRUB delays?

At home I have four systems with spinning rust that don't even take anywhere near long to get all the way from to the graphical login prompt. And yes, networking (wifi!) is up at that point. (Meanwhile, two servers each take longer than that to get through the POST...)

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 17:09 UTC (Thu) by flussence (guest, #85566) [Link] (9 responses)

It would make sense if it included cold power on time…

I just timed power-on to network up with a stopwatch. 13 seconds in the BIOS, 19 more seconds to desktop, 10 seconds more until `ip mon` came to life, so about 42 seconds total. This is on a 9 year old netbook with runit.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 17:45 UTC (Thu) by mchehab (subscriber, #41156) [Link] (8 responses)

> I just timed power-on to network up with a stopwatch. 13 seconds in the BIOS, 19 more seconds to desktop, 10 seconds more until `ip mon` came to life, so about 42 seconds total. This is on a 9 year old netbook with runit.

BIOS/GRUB time a little better... 11 seconds on BIOS, 17 seconds for the first Kernel messages to appear.

However, network comes a lot late here:

1:23 for ping to answer; 1:35 for ssh to log into it.

This machine is a 3 years old NUC (Bios identifies it as NUC5i7RYB) with a 5th gen i7core. It was not supposed to take that 1,5 minutes for ssh do succeed on it. When booting the same machine with Debian and SysV (right now, it is booting Ubuntu/systemd), it takes about 4 times less for ssh to start working.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 18:14 UTC (Thu) by pizza (subscriber, #46) [Link]

> This machine is a 3 years old NUC (Bios identifies it as NUC5i7RYB) with a 5th gen i7core. It was not supposed to take that 1,5 minutes for ssh do succeed on it. When booting the same machine with Debian and SysV (right now, it is booting Ubuntu/systemd), it takes about 4 times less for ssh to start working.

Be careful, you're not exactly comparing apples to apples here. At the very least, both systems have to be configured to start up the same services before you can make any direct comparison.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 18:47 UTC (Thu) by seyman (subscriber, #1172) [Link] (6 responses)

> It was not supposed to take that 1,5 minutes for ssh do succeed on it.

That's certainly a long time. Can you run "systemd-analyse blame"? It might reveal something.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 19:55 UTC (Thu) by mchehab (subscriber, #41156) [Link] (5 responses)

Yes, I did, but doesn't help much:

26.999s lvm2-monitor.service
25.765s dev-mapper-localhost\x2d\x2dvg\x2dubuntu.device
10.902s resolvconf.service
9.526s keyboard-setup.service
9.497s systemd-modules-load.service
8.397s kmod-static-nodes.service
8.387s systemd-remount-fs.service
8.067s binfmt-support.service
7.825s sys-kernel-debug.mount
...

The top two things there are not related to network, so shouldn't prevent network to start earlier (it might prevent ssh, though).

The third item (the DNS resolver) took 10 seconds to start, and the only thing to blame would be systemd itself (as it is starting systemd-resolved):

resolvconf.service +10.902s
└─systemd-journald.socket
└─system.slice
└─-.slice

Anyway, while I'm enjoying this discussion, as it helps me to try optimize booting, we're getting a little OOT here: the point is that systemd seems to be just randomly adding delays to network start for no good reason, and there's no direct way to control it, making network start as early as possible.

In other words, with systemd, the system admin cannot tell it what are the priority services that should be started first.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 0:25 UTC (Fri) by jccleaver (guest, #127418) [Link] (1 responses)

> In other words, with systemd, the system admin cannot tell it what are the priority services that should be started first.

In a nutshell, systemd is a developer's boot stamping on the face of a sysadmin/syseng... begging the question with their design choices on so many paradigm shifts that the administrator gives up trying to fight it, and eventually gives up trying to grok it at all.

NO TO SYSTEMD

Posted Nov 2, 2018 19:24 UTC (Fri) by naptastic (guest, #60139) [Link]

> In a nutshell, systemd is a developer's boot stamping on the face of a sysadmin/syseng... begging the question with their design choices on so many paradigm shifts that the administrator gives up trying to fight it, and eventually gives up trying to grok it at all.

You've hit the nail on the head. However much of systemd is NIH syndrome, or enforcing vendor lock-in, or adding complexity so people will pay for support, or just obliviousness to reality, systemd is just... well, I remove it as fast as I can wherever I can, and every time I do, life gets better. My systems boot faster, shut down faster, have fewer random problems, and are easier to troubleshoot.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 0:45 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

> In other words, with systemd, the system admin cannot tell it what are the priority services that should be started first.
You can whack it with a hammer and delete the dependencies of the network service. But your startup times look really unhealthy anyway, 10 seconds for resolvd is unbelievable.

I wonder if you have an unrelated issue in your systems somewhere.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 1:32 UTC (Fri) by zlynx (guest, #2285) [Link]

Yeah. I've got a home server running Fedora 29 now, with 30 TB of disk, and it's longest job is 13.6 seconds to mount the six drive btrfs array. After that nothing holds it up for longer than 2 seconds. Total boot time after the BIOS to gdm, is about 33 seconds.

There is definitely something wrong. If it's reading from an M.2 drive benchmark that thing, because it's looking broken.

I believe my last NAS with dual Bobcat cores at 1.6 GHz booted faster than that.

I think Raspberry Pi's reading from slow sdcards are quicker.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 19:11 UTC (Fri) by mchehab (subscriber, #41156) [Link]

> I wonder if you have an unrelated issue in your systems somewhere.

Yeah, there was indeed an issue: I build kernels with KASAN enabled. That, together with all stuff systemd tries to load and then unload at the initrd disk, made it really slow (it was spending ~30 seconds at initrd). I did a clean custom.target with the minimum stuff, with improved it a lot:

$ ls /etc/systemd/system/custom.target.wants/
anacron.service dbus.service networking.service rsyslog.service systemd-resolved.service thermald.service atd.service inetd.service NetworkManager.service ssh.service systemd-update-utmp-runlevel.service ttyUSB0.service cron.service irqbalance.service ondemand.service systemd-logind.service systemd-user-sessions.service

Also removed an open-iscsi package (with alone saved almost 15 seconds).

After lots of cleanups, I was able to reduce the time for network to start to "just" 1 minute, with KASAN enabled.

Without KASAN, using a vanilla Ubuntu Kernel, it is now booting really quick:

Startup finished in 777ms (kernel) + 2.416s (initrd) + 8.385s (userspace) = 11.579s
custom.target reached after 8.383s in userspace

Discovering it costed me an entire day of work. Doing the same with sysV would be trivial.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 17:26 UTC (Thu) by mchehab (subscriber, #41156) [Link]

> Does that include the BIOS/GRUB delays?

Nope. Just after systemd starts:

$ systemd-analize

Startup finished in 5.095s (kernel) + 24.631s (initrd) + 54.179s (userspace) = 1min 23.906s
multi-user.target reached after 53.796s in userspace.

$ systemd-analyze critical-chain multi-user.target

multi-user.target @53.796s
└─mpd.service @52.285s +1.509s
└─network.target @52.258s
└─systemd-resolved.service @52.060s +197ms
└─tmp.mount @39.178s +6.959s
└─dev-localhost\x2dvg-tmp.device @37.992s

> At home I have four systems with spinning rust that don't even take anywhere near long to get all the way from to the graphical login prompt. And yes, networking (wifi!) is up at that point. (Meanwhile, two servers each take longer than that to get through the POST...)

Systemd services setup seem to be written to prioritize starting the GUI. Here, GUI comes quick, but all the rest takes a lot more time. Usually, the *last* thing systemd starts are the text consoles (just after network).

On some of my machines, it may take up to 10 minutes to have them started - that was actually happening on that specific machine and forced me to actually remove packages from it (like lxc/lxd/snapcraft/libvirt/...).

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 3:18 UTC (Fri) by tytso (✭ supporter ✭, #9993) [Link]

I've automated building a minimal Debian system that uses systemd, brings up networking, and spawns a shell on a serial console. Oh, and it installs rsyslog for the same reason why you don't like journald --- I want a simple log file that I can pull off of the test system.

I normally use it as as part of a VM image, but if you need to run it on a real x86 hardware that shouldn't be all that difficult. In any case, it satisfies your requirement that it should come up in seconds. So it can definitely be done:

https://github.com/tytso/xfstests-bld/blob/master/kvm-xfs...

Documentation:

https://github.com/tytso/xfstests-bld/blob/master/Documen...
https://github.com/tytso/xfstests-bld/blob/master/Documen...

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 14:48 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link] (8 responses)

> I can understand why someone would choose systemd if their only other option was sysvinit.

This statement makes no sense. I'm tolerably familiar with the sysvinit code because I'm maintaining a fork of it. It contains some rudimentary process management which is virtually unused and presumably mainly exists (or came to exist) for compatibility with the really SysV init. Apart from that, it waits for run-level change requests and runs a (configurable) script in response to these. It also handles the exit status of processes it inherited because their parent processes exited while the children were still running. And that's it. That's a basic component which (very flexibly) enables implementing a variety of process manageing schemes on top of it, including the 'legacy' Linux distribution process not-management-system with all its horror workarounds for lack of sensible additional tools, pid files, pgrep, start-stop-daemon etc.

Considering this, your statement is somewhat like "I understand why someone would fit a combine harvester to a wheelbarrow after the wheel broke. After all, it has four them, meaning, greater reliabilty through redundant hardware!".

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 14:56 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link] (7 responses)

Maybe clarify this is a little: systemd is certainly a huge improvement over the "init system" Linux distributions used to ship on top of sysvinit. But this didn't come into being because of "evil radiation" going out from this program and "improvments" had also been possible without systemd.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 18:56 UTC (Fri) by jccleaver (guest, #127418) [Link] (6 responses)

> Maybe clarify this is a little: systemd is certainly a huge improvement over the "init system" Linux distributions used to ship on top of sysvinit.

I'm not sure about the "certainly" there. Arguably? Yes. Certainly? No.

I can count the number of unexpected boot hangs traced back to a problem with the rc.sysinit or related bootup scripts in non-systemd systems going back to 2000 on maybe one hand, with the only regular issue being sendmail hanging the boot on a lack of DNS resolution of the local hostname. Virtually every other hang has been a result of hardware, or daemon bugs.

systemd is too clever for its own good in trying to parallellize boot, meaning the "init system" is much more likely to fail in unexpected and hard to trace ways. upstart, of course, is capable of "parallellizing" things too... but, crucially, in RHEL 6 this wasn't actually used. The vast majority of the boot logic was in /etc/rc.sysinit, not /etc/init/*. If systemd did the same a significant number of problems would be avoided. (Even if other philosophical problems remained.)

systemd proposes, and then implements, a paradigm shift among things that used to be discrete, including "the init system", as opposed to PID 1. I think there's room for reductionist debate at all levels.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 19:45 UTC (Fri) by mpr22 (subscriber, #60784) [Link] (5 responses)

Debian parallelized the launch of sysvrc scripts (using 'startpar') long before it adopted systemd as the init daemon.

AFAIK systemd doesn't do anything particularly "clever" when it comes to parallelizing system startup, but rather just says that by default, any pair of units which do not state themselves to require serialization with respect to each other shall be assumed parallelizable.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 20:28 UTC (Fri) by xtifr (guest, #143) [Link] (4 responses)

> Debian parallelized the launch of sysvrc scripts (using 'startpar') long before it adopted systemd as the init daemon.

And yet, when Debian switched over to systemd as a default, it still reduced my boot time by an order of magnitude!

(Note that I do not really have any strong opinions about init systems one way or the other. But despite switching to systemd as part of Debian's automatic updates, and, more recently, because of a re-install of my system partitions from scratch, the only difference I've observed is a faster, cleaner boot. Heck, even my log files are still working the way I'm used to. Which rather surprised me, since "I hate binary log files" is one of the few complaints I've heard about systemd that actually had me at all worried.)

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 20:56 UTC (Fri) by lkundrak (subscriber, #43452) [Link] (3 responses)

> "I hate binary log files" is one of the few complaints I've heard

I have no idea how could I have lived without journalctl --since '5 minutes ago'.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 21:50 UTC (Fri) by jccleaver (guest, #127418) [Link] (2 responses)

> I have no idea how could I have lived without journalctl --since '5 minutes ago'.

tail -200 /var/log/messages ?

(and maybe adjust your number if you guessed your log velocity wrong)

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 23:39 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

Won't list logs that live in their own files.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 11:57 UTC (Mon) by jond (subscriber, #37669) [Link]

Not really comparable I'm afraid

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 7:23 UTC (Thu) by gfernandes (subscriber, #119910) [Link]

Are you really seriously suggesting determinism and bash scripts in the same breath?

Really?

Honestly?

Come on my friend. That's a bit like eating chalk pretending it's cheese.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 2:04 UTC (Thu) by milesrout (subscriber, #126894) [Link] (14 responses)

It certainly does bother me a lot when people say things like 'systemd is great, because initscripts is not', it's a symptom of this common misunderstanding that there are two options: what we had and what we have now, or what we have and what someone is proposing.

Maybe a few years ago the options were sysvinit vs systemd, and the decision to go with systemd was correct. But today, there is a wide variety of options, and the people that continue to act like the only alternative to sysvinit is systemd (which is bad, they say, so we should use sysvinit instead) or vice versa are being very misleading IMO.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 2:19 UTC (Thu) by anselm (subscriber, #2796) [Link] (3 responses)

There may be a “wide variety of options” today but package maintainers can't be expected to know and support them all. Many of them restrict themselves to systemd and sysvinit, which is what policy asks them to do.

So therefore, if you want to promote a new (non-default) init system in Debian you should be prepared to contribute support for that to most of the relevant packages. It is unlikely that you will get all of their maintainers to come up with the required configuration files from scratch, although they should in theory be willing to accept patches.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 4:48 UTC (Thu) by zlynx (guest, #2285) [Link] (2 responses)

Indeed. Writing service start definitions or scripts is annoying and people only want to do it once.

To win them over to some other system write an automatic conversion tool that takes a source (like systemd unit files) and produces whatever configuration the other system needs.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 17:10 UTC (Thu) by jccleaver (guest, #127418) [Link] (1 responses)

One ironic benefit out of this is that systemd unit files actually do do a decent job at identifying the various bits that need to happen. For functions that are supported, this makes automated conversion from a system unit file for a basic daemon back to a basic init script pretty easy, except for the need to figure out how to remove the various "--no-daemon" -type options that would get passed in.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 5, 2018 13:21 UTC (Mon) by mjthayer (guest, #39183) [Link]

> One ironic benefit out of this is that systemd unit files actually do do a decent job at identifying the various bits that need to happen. For functions that are supported, this makes automated conversion from a system unit file for a basic daemon back to a basic init script pretty easy, except for the need to figure out how to remove the various "--no-daemon" -type options that would get passed in.

This sub-thread struck me as a very constructive part of the discussion. Debian policy around unit files to keep them easily convertible to other init systems could be helpful.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 7:27 UTC (Thu) by gfernandes (subscriber, #119910) [Link] (7 responses)

Systemd is certainly not the *only* alternative to SysV init.

But it must certainly is the *best*, most comprehensive, and well thought out solution there is.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 7:28 UTC (Thu) by gfernandes (subscriber, #119910) [Link]

/must/most

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 1, 2018 21:24 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link] (5 responses)

Systemd supports "readiness notification". This is an inherently broken concept because of a TOCTOU-situation: That something happened to be ready at the time the notification was emitted doesn't mean it'll still be ready by the time it would be needed. Systemd also supports restarting programs which terminated unexpectedly. This amounts to the tacit admission that "readiness notification" is indeed a broken concept because it's code supposed to handle something suddenly becoming "unready" again. Internal contradictions of this kind are not a sign of something being particularly "well though out".

Granted, that's not specific to systemd, that's a defect it shares with a lot of other "Jonathans very own init replacement"-programs.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 1:13 UTC (Fri) by zlynx (guest, #2285) [Link] (2 responses)

Without it you get hacks like tail -f logfile | grep -c 1 "Giant Java App: Ready to Serve\|Giant Java App: Error"

Wrapped in a retry loop.

You really cannot do anything about an app that crashes after it reports ready. It's not like you can force it to keep running.

I'm not actually sure what systemd would do there. It would have been ready, so it would have started dependent units. But then it would have noticed the service crash which makes it not-ready again. I don't think it actually stops the dependents. They just have to survive on their own I guess.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 19:51 UTC (Fri) by rweikusat2 (subscriber, #117920) [Link] (1 responses)

I've seen enough code like this (the tail-pipeline) but this just illustrates the 'skills' (or lack thereof) of the person who wrote this code.

However, it doesn't really matter in which way something which cannot possibly work is implemented. And "readiness notification" is such a something as it communicates information about a state at a time of check but there's nothing which stops this state from changing in the time between a somewhat later time of use. Check-and-use would need to be an atomic operation and this operation would need to be performed every time use is supposed to happen. The very fact that check is done by some system daemon and use by "something else" prevents this.

The underlying problem here (if any) is really a bug (or some bugs) in application desiring to do the use which cannot cope with the service they're trying to use being unavailable for some time. And the "readiness notification" is just a doomed attempt to work around this bug (or these bugs).

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 20:07 UTC (Fri) by zlynx (guest, #2285) [Link]

Well, if you are designing your services in a good way you can use systemd socket activations. Then all of the services can start at the same time, all of them connecting to the service sockets that they depend on. Then once the server is running it picks up those sockets and serves.

The only problem is if the server startup time (giant Java apps) is so long that the clients time out before the server wakes up. Then you're back to needing readiness notifications.

Smart clients would retry connections on connection failure. Or they'd immediately exit, but their activation unit would restart them with some retry delay.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 19:32 UTC (Fri) by mpr22 (subscriber, #60784) [Link] (1 responses)

A program which, having become "ready", subsequently becomes "unready" without actually exiting or getting a fatal signal is a problem whether you're using a service supervisor or not.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 2, 2018 19:56 UTC (Fri) by rweikusat2 (subscriber, #117920) [Link]

I don't remember writing about something a service manager couldn't fix (or try to fix) by restarting a program (or some programs). I guess this must have been because my statement wouldn't make any sense otherweise :->

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 3, 2018 4:32 UTC (Sat) by ThinkRob (guest, #64513) [Link] (1 responses)

At some point the size of a userbase and the secondary effects becomes a significant factor. Maybe even the most significant.

systemd almost certainly has the largest install base of any of the Linux init options at the moment, by a large margin. That alone makes it attractive: there's more support for it, documentation/help is easier to find, and it gets a lot more testing. Something else could take its place, but it would have to be a LOT better to overcome that inertia. (That doesn't mean it can't happen; that's how systemd won in the first place!)

So IMHO systemd is "great" because it's the most widely used tool that does the job.

NO TO SYSVINIT - or initscripts, rather!

Posted Nov 3, 2018 13:49 UTC (Sat) by drag (guest, #31333) [Link]

The systemd project has done a good job standardizing the 'linux plumbing' of common distributions. This is very good because there is a limited pool of development resources.

Sysvinit does not address most of the issues that low-level-but-above-kernel OS design requires and in the past this meant a lot of ad-hoc development work being forced on distribution maintainers, application developers, and sysadmins who really are not that interested in designing operating systems.

This ended up with thousands of people facing the same basic set of issues trying to solve them independently to significantly different degrees of success. How many tens of thousands of different variations of 'start/stop/status httpd' scripts does the world really need? How many hundreds daemon helper programs that only 10 people is going to make the world a better place?

The ultimate purpose of operating systems is to make applications easier to write and easier to consume. It is telling that people that are doing the actual important work of making services/applications easier to use on Debian don't use sysvinit except though some sense of obligation.

And, sure, nobody questions that there are not alternatives to doing what systemd does. Systemd isn't magic, it's just software. And like any other program it can all be done in a billion different variations with a billion different designs any of which can be perfectly viable.

It's not the design details that matters so much. It's the amount of work people put into making it work. And if nobody is able to put the same amount of work into any systemd alternatives and offer it up to wide-spread consumption then there really isn't any benefit to doing it that way for most people and most distributions.

Init system support in Debian

Posted Nov 1, 2018 19:24 UTC (Thu) by mads (subscriber, #55377) [Link] (1 responses)

Just a comment on the systemd debate in general - it irks me that when Gentoo is mentioned in the same breath as systemd, it seems to attract a lot of people that regards Gentoo as some kind of anti-systemd bastion.

I just want to mention that systemd works great in Gentoo, it's easy to download a stage3 from gentoo.org that has a systemd profile already enabled, and it's easy to switch between an openrc based profile and a systemd based profile on an already built system. Can gladly recommend it, I've been using it for years on all of my Gentoo systems.

Init system support in Debian

Posted Nov 8, 2018 13:36 UTC (Thu) by Wol (subscriber, #4433) [Link]

As somebody who runs gentoo - this system I'm typing on is gentoo - I would beg to differ.

I tried to install gentoo/systemd on my laptop. I gave up - it's now running my second-choice distro, SUSE.

I am left with the *very* strong impression that there are several anti-systemd developers at the core of gentoo. And it shows in that there are a fair few references *to* systemd documentation, but *no pointers*, and the documentation is (apparently deliberately) quite hard to find.

So yes, I think gentoo is, sadly, one of the anti-systemd distros, even if it is quite easy (allegedly, I couldn't) to get a systemd version running.

Cheers,
Wol

Init system support in Debian

Posted Nov 2, 2018 19:36 UTC (Fri) by joey (guest, #328) [Link] (1 responses)

many of the vocal anti-systemd Debian developers have switched

Who are those many developers who switched? The only one I know of is Roger Leigh.

Otherwise, none of the names on https://devuan.org/os/team/ have been in the Debian keyring. Perhaps some of the anonymous/pseudonymous people are DDs (the original fork announcement claimed they had two other anonymous DDs), but this seems a thin thread for LWN to hang that statement on.

Init system support in Debian

Posted Nov 5, 2018 11:58 UTC (Mon) by jond (subscriber, #37669) [Link]

I'd agree.

Roger Leigh is the only one I know of, and I do miss him, almost as much as I miss Joey Hess.

Init system support in Debian

Posted Nov 3, 2018 19:04 UTC (Sat) by dps (guest, #5725) [Link] (8 responses)

I really don't like installing systemd on servers, especially internet facing ones. On these systems dbus provides no benefits whatsoever, and therefore should be eliminated.

There are problems, which I have not solved, with systemd like pointless start jobs on disks which are already mounted which take 90s to fail. Just not mounting thar device early is not an option because that device contains /usr.

Things like NetorkMamager are a *huge* step backwards if you need simple and predictable networkong - doing it with nmcli is very hard and old style network configuration scripts work much better. Nothing in a dmz should even think of trusting dhcp,

It is possible to eliminate systemd, dbus and udev but this requires nontrivial effort. Removing udev, which obviously implements anti-features on some systems, breaks most supported init systems without changes which are hard to do because the system won't boot.

IMHO those that want simple, low dependency and bloat free init should be able to get it more simply. Static /dev might be out of fashion but I actively don't want automatic device creation so udev does not solve the problem.

P.S. I remember the days is SLS 1.03 (kernel version 0.99pl11), when udev and dbus did not exist and everybody used static /dev. If think that the dhcp rfc had not been written at that time.

Init system support in Debian

Posted Nov 3, 2018 19:15 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

> There are problems, which I have not solved, with systemd like pointless start jobs on disks which are already mounted which take 90s to fail. Just not mounting thar device early is not an option because that device contains /usr.
Add a device dependency to these jobs.

Init system support in Debian

Posted Nov 3, 2018 19:38 UTC (Sat) by zdzichu (subscriber, #17118) [Link] (1 responses)

Don't bother. The poster thinks udev creates devices in /dev, it's the lost cause. You won't be able to educate this one.

Init system support in Debian

Posted Nov 5, 2018 13:56 UTC (Mon) by nix (subscriber, #2304) [Link]

Well, it used to. He just hasn't used udev since the days when udev 141 or thereabouts was new.

Init system support in Debian

Posted Nov 3, 2018 19:38 UTC (Sat) by mpr22 (subscriber, #60784) [Link] (4 responses)

> IMHO those that want simple, low dependency and bloat free init should be able to get it more simply.

Perhaps some of those that want that should team up and do the work to provide tools for arranging it, so that the rest of those who want it can get it.

Init system support in Debian

Posted Nov 4, 2018 15:45 UTC (Sun) by jospoortvliet (guest, #33164) [Link] (3 responses)

More importantly, so that those of us sick of the whining can point and laugh when the simple solution doesn't work out. Every frickin sysadmin and engineer always think they can do better. Very rarely are they right - they usually simply don't understand the problem and at best can create a 'solution' that handles a tiny subset of the problems the technology they want to replace dealt with.

Sometimes I wonder if the complainers are just jealous of Lennart who actually does manage to build BETTER solutions. After all his technology gets adopted widely and that wouldn't happen if it wasn't superior right? Isn't open source a meritocracy?

Init system support in Debian

Posted Nov 5, 2018 15:56 UTC (Mon) by dps (guest, #5725) [Link] (1 responses)

If you have a laptop which is likely to need to handle multiple networks and random USB devices, for which udev *does* create devices nodes in /dev for me, then systemd et al is genuinely good. The comment was specifically about firewall machines, bastion hosts and internet facing servers. There is no need to handle mode than a single fixed set of hardware.

Firewalls are not configured like regular boxes and what is desirable is different. A lot things which are better for regular systems are worse for firewalls and bastion hosts, which actively don't want most features. Some features are security features and obviously a good idea, and others may be required to do whatever the box is supposed to do. Anything else is anti-feature and should be resisted.

Those that have used Linux 0.90pl11 know how to do things without the modern features that depend on what is an anti-features on a paranoid host. System V init could handle this problem. Those things where the simple solution does not work out are almost all features that are useless on paranoid hosts in noisy data centres.

Init system support in Debian

Posted Nov 5, 2018 16:50 UTC (Mon) by anselm (subscriber, #2796) [Link]

I don't think that if you're paranoid you would want to use anything on a firewall or bastion host that is as needlessly complicated as sysvinit.

Init system support in Debian

Posted Nov 8, 2018 13:48 UTC (Thu) by Wol (subscriber, #4433) [Link]

> More importantly, so that those of us sick of the whining can point and laugh when the simple solution doesn't work out. Every frickin sysadmin and engineer always think they can do better. Very rarely are they right - they usually simply don't understand the problem and at best can create a 'solution' that handles a tiny subset of the problems the technology they want to replace dealt with.

The problem is those people who want (or need, let's face it, a lot of us DON'T need a big complicated init) a simple init usually aren't the people with the skill to understand the problem.

Those of us who need a complicated init will design an init to suit our needs, and those of us who want a simple init will have to put up with it because we don't have the skills to do it ourselves.

It's like distros - your average distro is a jack-of-all-trades and any one user probably would describe 90% of it as unwanted crap. Problem is, my unwanted crap is your can't live without application, and vice versa.

So you need an expert in the complicated system to strip out the 90%, to make a dedicated single purpose distro or init system.

Cheers,
Wol

Thanks!

Posted Nov 8, 2018 9:43 UTC (Thu) by oldtomas (guest, #72579) [Link] (1 responses)

Thanks for this article: it gives some perspective on how a community might find a way to resolve a seemingly insoluble conflict.

Skimming the comment section is a sad experience and still reveals how deep that conflict is. The tendency to denigrate the "other side" seems still... irresistible.

Luckily, the tone within Debian has (mostly) reached a cooperative, almost friendly level.

To all of you (on any of both sides): re-read the Russ Allbery quotes up there in the article. This is the example to follow (it has been from the hottest phase in the systemd "war", btw).

Thanks!

Posted Nov 10, 2018 9:42 UTC (Sat) by HelloWorld (guest, #56129) [Link]

> To all of you (on any of both sides)
"Both sides"? Is that you, Donald?

Init system support in Debian

Posted Nov 12, 2018 20:13 UTC (Mon) by dkg (subscriber, #55359) [Link]

As a debian package maintainer, i'll speak up and say that i am one of the people guilty of starting to drop sysvinit-style initscripts from my packages. This makes me a bit sad because i really would like to support non-Linux kernels in Debian better, and systemd is Linux-only, of course.

I am not doing this out of antipathy toward sysvinit or its users; i'm doing it because i can't responsibly maintain them. even if i ran systems that had sysvinit as pid 1 that i could use for testing, systemd offers so many better ways to ship tuned, constrained unit files that for moderately complex services (ones that need customized permissions or OS constraints, for example, or that use socket activation), i can't reliably assert that:

  • the necessary dependencies will be handled correctly
  • any supporting data (e.g., /run/foo/cache or /var/lib/foo) will be correctly accessible
  • the kernel capabilities will be appropriately scoped
  • the daemon will be restarted reliably on failure
  • local system administrator edits or configuration will be reliably preserved (e.g. if i add /etc/default/foo to make /etc/init.d/foo configurable, does that mean i now have to source /etc/default/foo from somewhere in my systemd unit file as well, or is it only supposed to affect sysvinit systems?)
  • etc…

shipping a sysvinit file responsibly means i need to help people use it, and test and analyze the ways that it fails. I have limited cycles, and i don't want to maintain a lot more code, especially if there is some other codepath that is already established and i can relatively simply express what i want common services to do in a declarative manner. And i definitely don't want local admins who have the package installed using sysvinit having radically different active system semantics. for example, it would be pretty bad if a daemon runs as root under sysvinit, but as user foo under systemd.

Consequently, i have opened bugs asking for help with sysvinit maintenance (e.g. #833854 and #855653), but have gotten very little concrete help in making them work reliably.

If i can't offer something that is reliable, i think i'm doing debian a disservice in shipping it. Certainly, i'd never complain if an administrator wants to write their own initscripts (for sysvinit) or servicedirs (for runit) or any other system integration. And i don't want to accidentally clobber a locally-customized sysvinit script either! But if anyone wants to step up to offer to co-maintain that part of the package so that it has comparable support to the systemd unit files, i'd welcome it.

but i'm going to focus my Debian package maintenance time on things that seem more useful and efficient and widely applicable to the Debian ecosystem. Today, that's systemd.


Copyright © 2018, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds