Debian reconsiders init-system diversity
The new systemd discussion had been simmering for some time before flaring up in late 2018 when it became clear that the project had been unable to fully support systemd alternatives due to a lack of developer interest. The current discussion, instead, is a direct result of Debian project leader Sam Hartman's Bits from the DPL posting on October 29. Therein, he stated his belief that a new general resolution on init-system support is necessary, and that he would be proposing one in the near future.
The immediate motivation for a reconsideration would appear to be the proposed addition of elogind, a standalone fork of the systemd-logind daemon, to Debian. Elogind would provide support for systemd's D-Bus-based login mechanism — needed to support small projects like the GNOME desktop — without the need for systemd itself. The addition of elogind has been controversial; it is a difficult package to integrate for a number of reasons. Much of the discussion has evidently been carried out away from the mailing lists, but some context on the problem can be found in this bug report. In short: merging elogind appears to be complex enough that it would be hard to justify in the absence of a strong commitment to the support of non-systemd init systems. It seems possible that this commitment no longer exists across the distribution as a whole; the purpose of a general resolution would be to determine whether that is the case or not.
Feature pressure
Unsurprisingly, Debian developers have a variety of opinions on this issue.
This response
from Russ Allbery is worth reading in its entirety. He argues that the
2014 decision (of which he was a part) never really nailed down the
project's position toward other init systems. That was a necessary
compromise at the time, he said, but it is causing stress now: "while
I feel
somewhat vindicated by the fact that this didn't immediately fall apart
and has sort of worked, I think it's becoming increasingly
untenable
". The time has come, he says, to make the project's
position clear so that developers can and will do the right thing:
Josh Triplett zeroed in on one of the issues that is testing the init-system peace now. There is, he said, an increasingly long list of features that are only available with systemd, and application developers want to use those features:
Today, people can't use systemd persistent timers in place of cron (and in place of anacron's "wake up periodically" approach). [...]
Systemd user sessions, socket activation, sysusers, dynamic users, systemd-homed, temporary directory setup, transient units, anything talking to the slice or control group APIs, containerization, firstboot, systemd's whole "preset" system-wide configuration and policy mechanism for admins to say "what services do I want launched when installed and what services do I want to leave stopped until I configure them", "stateless system" capabilities, and I'm probably forgetting another dozen.
The responses to this argument took a couple of different approaches. Ted
Ts'o described those
features as "the 'embrace, extend, and extinguish' phenomenon of
systemd which caused so much fear and loathing
". Debian has avoided
much of this stress until now, he said, because it ships a relatively old
version of systemd, but that won't last forever. Upstream developers are
going to use newer systemd features, and Debian will have to accept that if
it wants to package and ship their code. "I don't think we have a
choice but acknowledge reality and accept that some packages may simply be
incompatible with alternative init systems
". The situation doesn't
make him happy, he said, but there is little to be done about it.
Allbery, instead, suggested that the project could decide on a set of interesting systemd features, then create non-systemd implementations of those features where necessary. That is essentially what is being done with elogind, for example. All init systems could be made to parse systemd unit files, and there would be a minimal set of directives they would be expected to support. There would, in effect, be a definition of the ABI an init system must support, rather than any rules about specific systems. Done properly this approach could ease the feature pressure, but there is one little problem:
Some, like Ian Jackson, claimed that much of this functionality already exists and works. Svante Signell also argued that the systemd alternatives are well maintained and that Debian should remain open to contributions from developers who are working on those alternatives.
Others, including Triplett, said that not only do the developers of other init systems lack the desire to implement systemd features, but they argue that those features should not exist in the first place. That, Allbery added, is the question that the project has to answer: what will be Debian's policy toward software that requires features that will be specific to systemd? Should there be a specific subset of systemd features that Debian software will be allowed to depend on, with the idea that alternative init systems will eventually gain implementations of those features?
The first GR draft
Hartman posted a draft general resolution on November 7 that does not directly address Allbery's question. It focused strongly on init systems, rather than support for advanced systemd features; there are three alternatives:
- "Affirm init diversity": running init systems other than systemd is an important project goal, and provision of init scripts would remain mandatory.
- "Systemd but we support exploring alternatives": systemd would be the preferred init system, but alternatives would remain important; elogind is given as an example of the sort of project that should be supported. Hartman posted another message with a detailed description of what this option might imply.
- "Systemd without diversity as a priority": there would be no requirement to support anything but systemd in Debian.
Hartman thus appears to have left out Allbery's suggestion of defining an init-system ABI entirely. It should be noted, though, that this is explicitly a draft; it is likely to evolve considerably before it reaches the point where the project will vote on it.
There is one final bit of context to consider as part of this vote: according to
Marco d'Itri, it has already happened. "We have already voted
with apt: less than 1% of new installs use sysvinit
". That suggests
a relatively low level of interest in systemd-free Debian. Perhaps users
with that interest have moved to Devuan
but, to all outside appearances,
progress there has slowed considerably since its developers fell out with each other in April. Even
if Debian chooses to continue to support multiple init systems, it is not
clear that the development effort to make that support work will be there.
The free-software community values diversity and choice in its systems. But we also value standardization and the progress that can be made when we all focus our efforts on a single system. Many parts of the Linux system are essentially monocultures: examples include the kernel, the C library, and, until relatively recently, the compiler. There are strong arguments in favor of both standardization and diversity. In the case of GCC, the lack of competition is often cited as a cause for a long period of relative stagnation. If one looks at desktop environments, instead, it is often said that the scattering of developer effort has prevented the success of the Linux desktop as a whole.
It would thus seem that there is no obvious answer to the question of
init-system diversity, regardless of how one feels about systemd in
particular. Most community distribution projects have long since decided
that they lack the resources (and maybe the desire) to support multiple
init systems. Most of those, especially the larger ones, have settled
on systemd, even if they feel that a viable alternative to systemd would
be good to have. Debian has, so far, been an exception to this trend. It
will be interesting to see whether the project has the will and the
development resources to continue on that course.
Posted Nov 12, 2019 7:04 UTC (Tue)
by ncm (guest, #165)
[Link] (14 responses)
The ABI idea is kind of appealing just because I'm sure that once there is a target, a low-overhead version could be cranked out in short order. It would be based on programs that start up, run to completion, and terminate, freeing up all the resources used instead of squatting on them until kingdom come.
Posted Nov 12, 2019 7:34 UTC (Tue)
by mebrown (subscriber, #7960)
[Link]
Posted Nov 12, 2019 11:04 UTC (Tue)
by smurf (subscriber, #17840)
[Link] (12 responses)
Posted Nov 13, 2019 4:48 UTC (Wed)
by ncm (guest, #165)
[Link]
Posted Nov 13, 2019 15:09 UTC (Wed)
by IanKelling (subscriber, #89418)
[Link] (10 responses)
Posted Nov 13, 2019 16:31 UTC (Wed)
by pizza (subscriber, #46)
[Link] (5 responses)
(With embedded or RAM constrained systems you're presumably going to turn off systemd's features you don't need or care about. Of the three systems I have immediately in front of me, the RSSes for PID1 are 17.1MB, 8.5MB, and 5.4MB respectively.. Even the 5.4MB one still pulls in the likes of libselinux, libapparmour, libaudit, and libpam...)
What really matters in that router (or other embedded systems) is peak RAM usage. Sure, sysvinit on your router might only be 840k, but for it to do anything useful it has to invoke a shell, which in turn invokes subshells and a pile of various other tools along the way. If you have a higher baseline but your peak is lower, then you've come out ahead.
(Back when I was still writing router BSPs, I had to put a lot of effort into auditing this sort of thing..)
Posted Nov 13, 2019 21:36 UTC (Wed)
by hmh (subscriber, #3838)
[Link] (4 responses)
Posted Nov 14, 2019 12:04 UTC (Thu)
by pizza (subscriber, #46)
[Link] (3 responses)
Meanwhile, the triple-band "gaming" AC monstrosity at the work lab has a quad-core ARM under its gaudy exterior with 13 wired and 1 wireless client, and 399/1024MB free (according to its web UI) But it's also doing routing/dnsmasq/openvpn, plus a whole lot of other crap we don't actually use.
Posted Nov 14, 2019 17:32 UTC (Thu)
by hmh (subscriber, #3838)
[Link] (1 responses)
With 128MiB they are supposed to survive, and still have some RAM left to do something other than pushing packets around, running the control plane, and hosting the [buffers with] the uncompressed image of the O.S...
Posted Nov 14, 2019 18:36 UTC (Thu)
by farnz (subscriber, #17727)
[Link]
Just to put some numbers on this; a single PHY-level frame in 802.11ac VHT mode can be up to 1 megabyte long, and is targeted at a single receiver. That means that for full throughput, you need 2 megabytes per client buffering in your AP - one frame being transmitted now, one frame being prepared for the next time that client gets airtime.
Posted Nov 18, 2019 22:35 UTC (Mon)
by rahvin (guest, #16953)
[Link]
Posted Nov 14, 2019 2:59 UTC (Thu)
by rbrito (guest, #66188)
[Link] (1 responses)
Besides that, since this thing doesn't have a physical terminal, forcing an fsck at boot time with systemd is hard, since you have to pass it as a (kernel) command line parameter, which is not really as simple as entering GRUB and setting the right option (fsck.mode=force).
(With other systems, I don't mind running systemd, but with some, I do).
Posted Nov 14, 2019 17:48 UTC (Thu)
by derobert (subscriber, #89569)
[Link]
Posted Nov 14, 2019 3:23 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Nov 14, 2019 23:48 UTC (Thu)
by Darakian (guest, #96997)
[Link]
Posted Nov 12, 2019 7:26 UTC (Tue)
by flussence (guest, #85566)
[Link] (16 responses)
More to the point, why are they still insistent on sysvinit? Nobody in living memory has ever used its full feature set (corrections welcome - does your laptop send SIGPWR at the right time?), they'd be much better off with OpenRC now that it provides a PID1.
Posted Nov 12, 2019 7:45 UTC (Tue)
by andrewsh (subscriber, #71043)
[Link] (5 responses)
Posted Nov 12, 2019 13:06 UTC (Tue)
by BirAdam (guest, #132170)
[Link] (4 responses)
Posted Nov 12, 2019 13:18 UTC (Tue)
by mpr22 (subscriber, #60784)
[Link] (3 responses)
kFreeBSD is a more compelling argument, since it involves work happening within Debian, but leads to the obvious question: How much work would it actually be to make openrc play nicely on kFreeBSD? (I've never looked; I have no idea.)
Posted Nov 12, 2019 22:39 UTC (Tue)
by sjj (guest, #2020)
[Link] (1 responses)
It's insanity that this monstrosity that somebody somehow jammed into "Debian" now has to be considered in discussions about Debian's future directions.
Posted Nov 13, 2019 4:36 UTC (Wed)
by ncm (guest, #165)
[Link]
Posted Nov 13, 2019 23:07 UTC (Wed)
by flussence (guest, #85566)
[Link]
Posted Nov 13, 2019 14:41 UTC (Wed)
by Deleted user 129183 (guest, #129183)
[Link] (9 responses)
Because most people ‘concerned’ about systemd are tech conservatives – they want things done ‘as they always were done’. Hence we have a lot of people who complain about systemd being ‘a bloated mess’ while at the same time defending X.org, an actual bloated mess, just because they don’t want the change to Wayland.
Posted Nov 14, 2019 15:21 UTC (Thu)
by bferrell (subscriber, #624)
[Link] (8 responses)
Posted Nov 14, 2019 16:11 UTC (Thu)
by pizza (subscriber, #46)
[Link]
s/worked/kinda sorta worked most of the time, except when it randomly didn't/
(It's rather disingenuous to blame systemd for having the audacity to actually catch and report errors that the prior mudball blithely ignored..)
*shrug*
Posted Nov 14, 2019 17:16 UTC (Thu)
by smurf (subscriber, #17840)
[Link] (6 responses)
My personal favorite init-related breakage is the far-from-empty set of daemons that refuse to start up if they find a PID file that now points to some unrelated process. But hey, the pid is valid so refuse to run, eh? (Not to mention that absolutely nobody wites these files correctly, i.e. race condition free, in the first place.)
Surprise, systemd (a) finally pushed us towards a RAM file system for /run and (b) made that kind of nonsense obsolete anyway.
Posted Nov 14, 2019 18:34 UTC (Thu)
by HenrikH (subscriber, #31152)
[Link]
Posted Nov 22, 2019 10:55 UTC (Fri)
by Wol (subscriber, #4433)
[Link] (4 responses)
It required the network service to be running when it shut down, else it would get stuck in an infinite loop. If your system happened to shut down networking before bind ... if it was in a remote co-lo you were SOL.
Oh and I think this took *years*, *if ever* to get fixed in the official repo.
Cheers,
Posted Nov 24, 2019 20:59 UTC (Sun)
by flussence (guest, #85566)
[Link] (3 responses)
Posted Nov 27, 2019 17:01 UTC (Wed)
by anselm (subscriber, #2796)
[Link]
BIND 9 actually borders on the usable. In addition, while there are loads of purported “BIND replacements” they often implement only that part of the functionality of a DNS server that their authors like or agree with. If one desires a DNS server which actually does everything that the RFCs say a DNS server should do, the number of feasible substitutes for BIND is suddenly not all that large anymore.
Which is not to say that there aren't BIND replacements that are useful in many contexts (I'm a happy user of PowerDNS and dnsmasq, for example) – it's just that the “BIND replacements” tend to come with their own sets of issues. Hence, caveat emptor.
Posted Nov 28, 2019 11:13 UTC (Thu)
by jezuch (subscriber, #52988)
[Link] (1 responses)
Posted Nov 29, 2019 4:04 UTC (Fri)
by flussence (guest, #85566)
[Link]
Posted Nov 12, 2019 7:58 UTC (Tue)
by idealista (guest, #121682)
[Link] (25 responses)
Posted Nov 12, 2019 13:26 UTC (Tue)
by jezuch (subscriber, #52988)
[Link] (18 responses)
Systemd was adopted so widely so quickly mostly because, for everyone except a group of die-hards, it was a huge relief from the pain of existing init infrastructure. Lack of alternatives may be simply a result of the fact that systemd did things so well that there is not much of a reason to do anything differently.
Also, I am a long-time Debian user and I wish for Debian to not become a proving grounds for someone's pet peeve that prevents the whole system from progressing. If there is a way to provide init system diversity without placing an unwanted burden on people doing unrelated work - then fine, do it. But if it's not possible, then I'm perfectly fine with systemd's monopoly.
Posted Nov 12, 2019 18:14 UTC (Tue)
by bkw1a (subscriber, #4101)
[Link] (17 responses)
My understanding was that systemd was adopted widely because Gnome added it as a dependency.
Posted Nov 12, 2019 22:45 UTC (Tue)
by tchernobog (guest, #73595)
[Link]
No GNOME there.
Rather, a very appealing feature set for service unit handling that improves security, logging and enforcing usage limits.
As a system architect of large installations (we are talking several thousands of container instances), I couldn't do without systemd anymore.
Posted Nov 12, 2019 22:58 UTC (Tue)
by rahulsundaram (subscriber, #21946)
[Link]
GNOME added as a dependency to some interfaces. Those interfaces have alternative implementations. This isn't the reason why major distributions have adopted systemd
Posted Nov 13, 2019 13:13 UTC (Wed)
by anselm (subscriber, #2796)
[Link]
I think the more probable explanation is that distribution engineers realised that a standardised plumbing layer à la systemd was likely to save them a whole lot of work down the road.
Posted Nov 14, 2019 21:46 UTC (Thu)
by ms_43 (subscriber, #99293)
[Link] (13 responses)
https://www.freebsd.org/gnome/newsflash.html
See also:
https://blogs.gnome.org/mclasen/2014/02/19/on-portability/
Posted Nov 14, 2019 22:14 UTC (Thu)
by mgb (guest, #3226)
[Link] (12 responses)
No serious application depends on systemd because you'd be locking out 98-99% of your potential user base.
I've heard that Gnome multi-seat might depend on systemd but I don't know anyone who uses it.
Posted Nov 14, 2019 22:18 UTC (Thu)
by mpr22 (subscriber, #60784)
[Link] (6 responses)
Posted Nov 15, 2019 19:43 UTC (Fri)
by tpo (subscriber, #25713)
[Link] (4 responses)
Some data:
http://tpo.sourcepole.ch/articles/194%20popularity-of-des...
Posted Nov 16, 2019 1:05 UTC (Sat)
by karkhaz (subscriber, #99844)
[Link] (2 responses)
Posted Nov 16, 2019 5:15 UTC (Sat)
by rahulsundaram (subscriber, #21946)
[Link] (1 responses)
It seems a little odd that file managers don't correspond to the desktop environment in these stats. What's the explanation for that?
Posted Nov 16, 2019 17:45 UTC (Sat)
by karkhaz (subscriber, #99844)
[Link]
Posted Nov 18, 2019 22:56 UTC (Mon)
by rahvin (guest, #16953)
[Link]
Posted Nov 22, 2019 10:59 UTC (Fri)
by Wol (subscriber, #4433)
[Link]
Which is why my make.conf has "-gtk -gnome" in it ... (gentoo make options, for those who don't know). In other words, "disable them if you can".
Cheers,
Posted Nov 14, 2019 22:28 UTC (Thu)
by pizza (subscriber, #46)
[Link]
Hard dependency, no -- but any serious application will already have a non-trivial amount of platform-specific code, even when using portability-enhancing frameworks like Qt.
Posted Nov 15, 2019 11:10 UTC (Fri)
by tao (subscriber, #17563)
[Link] (3 responses)
... Oh, wait...
Oh, but all iPhone software is written to be compatible with Android, right? Depending on iOS-specific features would lock out too big of a potential user base.
Oh, and there's no server software written specifically for Windows Server, because the largest market is Linux...
Besides, most Linux distros use systemd. Your realistic potential user base uses Linux. This isn't an issue. GDM (one of the components in GNOME that benefits from systemd) isn't ever gonna be used on MacOS or Windows.
Posted Nov 15, 2019 14:55 UTC (Fri)
by mgb (guest, #3226)
[Link] (2 responses)
As a software engineer I also oppose the EEE aspects of systemd-EEE and the resulting Linux monoculture as counter-productive to software evolution for similar reasons that I for decades have opposed Windows EEE.
The push for systemd-EEE is driven by packagers working for a corporation which monetizes the work of FLOSS authors for its own benefit.
The unnecessary tying of FLOSS applications to systemd-EEE is a harmful varnish applied to FLOSS applications by packagers.
Posted Nov 15, 2019 16:49 UTC (Fri)
by smurf (subscriber, #17840)
[Link] (1 responses)
(a) what is EEE?
(b) The push for systemd is driven by people like me who want it because if makes our job (regardless of whether we get paid for it) a whole damn lot easier, not to mention people like Lennart who realized that the then-current state of fractured broken tools (including upstart) is unsustainable and decided to effing do something about it. I can only assume that the same holds for RedHat. Kindly explain what could possibly be bad about that.
(c) If you don't want other people to profit from your work (NB, you profit from theirs too) then don't publish your code under a free license. Or at all. This is 100% orthogonal to systemd-or-not.
(d) almost no program depends on systemd-as-init. Gnome for one does because it saved them from reimplementing a whole load of compatibility crap and/or from depending on broken unmaintained libraries like consolekit. You don't like that? fine, then kindly help elogind become a fully-supported alternate implementation – instead of lamenting that nobody does free work to cater to your sensibilities.
Posted Nov 15, 2019 19:39 UTC (Fri)
by geert (subscriber, #98403)
[Link]
"Embrace, Extend, Extinguish", is what Google Search told me.
Posted Nov 12, 2019 16:01 UTC (Tue)
by ale2018 (guest, #128727)
[Link] (4 responses)
In fact, Linux is a niche user case itself. If we reason by that logic, we can install Windows and stop worrying.
Servers are niche cases too. In many cases, servers need no GUI. If needed, LXDE is more than enough. Most often, games, fancy word processors, GUI network managers and such are not needed too.
I'd rather switch to Devuan than systemd.
Posted Nov 12, 2019 16:23 UTC (Tue)
by farnz (subscriber, #17727)
[Link] (3 responses)
Once you are in a niche use case (such as Linux desktops), the critical bit becomes finding people who will actually do the work to keep that niche viable.
In that regard, it's a shame that Devuan (a) appears to be in the process of imploding, and (b) wasn't able to function within the Debian project itself, keeping non-systemd users happy and providing Debian with the people needed to keep alternatives to systemd alive in the Debian world.
Posted Nov 12, 2019 16:49 UTC (Tue)
by ale2018 (guest, #128727)
[Link] (2 responses)
AFAIK it was. There is a debian-init-diversity mailing list where Debian developers like Dmitry Bogatov and Devuan developers like KatolaZ cooperate to maintain SysV code.
Posted Nov 12, 2019 16:58 UTC (Tue)
by seyman (subscriber, #1172)
[Link]
I believe the list you're thinking of is this one: http://www.chiark.greenend.org.uk/mailman/listinfo/debian...
Posted Nov 12, 2019 17:01 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
The very existence of Devuan as a separate project, rather than the people involved in Devuan working to ensure that you can continue to replace systemd with a.n.other init system in a standard Debian install (complete with appropriate tweaks to d-i so that you can choose to never run systemd on your Debian install) was a sign that Devuan as a whole was not able to function inside Debian - they needed a separate project to do what they wanted to do.
I'm not surprised that some Devuan developers were able to function inside Debian, and I'm glad that they could, but I can't help but think that we wouldn't be here today if the Devuan project had just been like-minded Debian Developers working inside Debian to keep non-systemd init systems usable.
Posted Nov 13, 2019 14:53 UTC (Wed)
by Deleted user 129183 (guest, #129183)
[Link]
I think that nosh <https://jdebp.eu/Softwares/nosh/> could be a good start. It was written with being compatible with systemd units in mind, and is of a much cleaner design.
Posted Nov 12, 2019 10:34 UTC (Tue)
by arekm (guest, #4846)
[Link] (6 responses)
Posted Nov 12, 2019 11:07 UTC (Tue)
by smurf (subscriber, #17840)
[Link] (5 responses)
Posted Nov 15, 2019 2:53 UTC (Fri)
by ncm (guest, #165)
[Link] (4 responses)
Posted Nov 15, 2019 3:42 UTC (Fri)
by pizza (subscriber, #46)
[Link]
"footprint" is code for "reject the basic premise of the entire project"
As for "code quality", bug reports are taken seriously and patches are welcomed.
(Which is a far cry from the barely-maintained swampy morass of the former mudball. "Quality code" it was not, and its highly inconsistent "footprints" are the reason we are even having this conversation...)
Posted Nov 15, 2019 11:31 UTC (Fri)
by tao (subscriber, #17563)
[Link] (1 responses)
And that's still with a distro with rather strict policies for everything.
Compare this to .service files. Simple dependencies, simple logging, simple rules for start/stop/restart etc.
There certainly are a few really complex .service files (timesync1 comes to mind), typically to lock things down. Most of such functionality cannot even be had from initscripts though.
So tell me again, what code quality issues do you see with systemd that sysvinit + a myriad of scripts doesn't suffer from?
Posted Nov 26, 2019 12:02 UTC (Tue)
by immibis (subscriber, #105511)
[Link]
FUSE drivers can afford to be far less stable than in-kernel filesystem drivers - now imagine importing that instability into the kernel without cleaning it up.
Posted Nov 16, 2019 7:34 UTC (Sat)
by ThinkRob (guest, #64513)
[Link]
That stands in stark contrast to numerous init scripts I've read over the years, which (in addition to following virtually no standards across products + distros) are often filled with incredibly optimistic (i.e. minimal-to-non-existent) error-handling, mix-and-match language selection (to work around shell-scripting's many weaknesses) and frequently ignore race conditions altogether.
In fact, the only consistent thing I've seen about init scripts is that the code quality seems to range from "incredibly complex because the distro re-wrote/re-factored them all" to "welp, it worked on our dev's machine and our software's too enterprisey for you not to buy, so that's good enough for us!" And in the case of the former, sure it's "consistent"... for that distro. Hope you don't have to deal with another, because now that "standard" set of init helper functions is totally different, as is the naming, as are the conventions for error reporting, and also everything else.
I don't know if systemd actually does use more resources, but if it does, well... getting rid of that inconsistent duct-taped mess would be worth an extra couple megs of RAM!
Posted Nov 12, 2019 10:37 UTC (Tue)
by kilobyte (subscriber, #108024)
[Link] (12 responses)
There are attempts to remove well-working pieces of software just because they're useful with modular inits rather than systemd (example: #930869 -- a RC bug with refusal to name any actionable problems), while "breaks the whole system, and requires a near-complete reinstall to switch inits" (#930105 instead got insta-downgraded+closed+WONTFIXed.
Within Debian's current model, it takes just a single hostile maintainer to block system-wide functionality.
Posted Nov 12, 2019 13:47 UTC (Tue)
by bigon (subscriber, #57617)
[Link] (9 responses)
IMVHO the problem lies in this comment elogind is developed by gentoo and in the gentoo world recompiling is not a problem. In binary distributions like debian it's a problem. And allowing to replace library packages is calling for troubles and will not make a distribution more robust. I'm still thinking that it's elogind daemon that should stub/mock the behavior of logind
Posted Nov 12, 2019 14:14 UTC (Tue)
by kilobyte (subscriber, #108024)
[Link] (6 responses)
Posted Nov 12, 2019 14:34 UTC (Tue)
by bigon (subscriber, #57617)
[Link] (4 responses)
Posted Nov 12, 2019 18:04 UTC (Tue)
by smurf (subscriber, #17840)
[Link] (3 responses)
elogind and systemd do their cgroups layout in a completely different way; teaching one library / daemon to interpret / layout these data in a way that the other can read it is not going to happen. Elogind isn't going to implement systemd's cgroups features, systemd cannot use elogind's layout even if it wanted to because it doesn't support its feature set, neither library reads the other's data, and AFAIK nobody is doing any work to implement any of that.
Simply put, these two implementations cannot co-exist. libelogind can be used as a libsystemd stub but this will only work on a system running elogind.
The only way to get this to work in some stable kind of way (that I can see, anyway) would be a stub init that checks which real init should run, re-links the libsystemd0 symlink to point to either libsystemd0.whatever-version.so or libelogind.some-other-version.so, and execs the real init. Good luck doing that if [/usr]/lib should happen to be readonly. Oh yes, you'd also need to prevent ldconfig to overwrite the link, prevent dpkg to remove the "wrong" library if the "wrong" init is running, and probably some other problems I haven't thought of.
Summary: it's time to throw in the towel. Just declare systemd to be *the* supported init and be done with this mess. Anybody who still wants to run current Debian on non-current-Linux kernels would need to do some sort of fork, their problem (unfortunately!).
The amount of effort to accommodate non-systemd Debian machines has been staggering, but IMHO we've learned that it's too much effort for too little gain to be sustainable.
Posted Nov 13, 2019 7:13 UTC (Wed)
by zuki (subscriber, #41808)
[Link]
My conclusion is similar to the debian release team: there is no sane way to try to substitute libsystemd that would not break programs linking to it. Trying to do it live on a running system is madness. That library implements dbus and device queries and logging and an event loop, and putting in stubs is never gonna fly.
On the other hand, the original design to keep libsystemd, but replace the part behind the dbus boundary, i.e. logind with elogind, will not work either, because various libsystemd functions look at the cgroup layout.
The only solution that would IMHO work would be to make elogind use compatible layout of cgroup paths. This sounds complicated, but the systemd cgroup layout is simple and almost stable. Then libsystemd0 and elogind packages could be installed in parallel at any time, and they would both work. Using an incompatible layout would be OK if this was a completely independent project, but for a purported drop-in replacement, that is a gratuitous and deal-breaking incompatibility.
(In the infrequent scenario that systemd introduces a cgroup layout change, elogind would need to be adjusted. But Debian doesn't need to allow any version of systemd to work with any version of elogind, so it could have elogind declare Conflicts:systemd>nnn, until it is known that nnn+1 is still compatible or elogind is adjusted.)
People keep bringing up that when systemd was being introduced it was parallel installable on existing systems and could be turned on and off by setting init= kernel command line. That design choice made the initial introduction vastly easier. Too bad that the elogind people didn't go for the same thing.
(systemd developer here)
Posted Nov 13, 2019 17:20 UTC (Wed)
by khim (subscriber, #9252)
[Link] (1 responses)
I think I miss something really important here: you have to call mount once to bind-mount either libsystemd0.whatever-version.so or libelogind.some-other-version.so on top of libsystemdo.so... and that kinda implies root privileges... but PID1 always have them so... what's the problem?
Posted Nov 13, 2019 21:13 UTC (Wed)
by NYKevin (subscriber, #129325)
[Link]
Posted Nov 16, 2019 10:48 UTC (Sat)
by intelfx (subscriber, #130118)
[Link]
Would you mind providing some nontrivial examples of such functionality?
Posted Nov 12, 2019 19:50 UTC (Tue)
by qyliss (subscriber, #131684)
[Link] (1 responses)
From the project’s README:
> Elogind has been developed for use in GuixSD, the OS distribution of GNU Guix.
Posted Nov 12, 2019 20:55 UTC (Tue)
by bandrami (guest, #94229)
[Link]
Posted Nov 12, 2019 16:14 UTC (Tue)
by ebiederm (subscriber, #35028)
[Link] (1 responses)
The only small regression I have is network-manager winds up depending upon
Posted Nov 12, 2019 17:43 UTC (Tue)
by zainryan (guest, #131584)
[Link]
Posted Nov 12, 2019 13:38 UTC (Tue)
by tlamp (subscriber, #108540)
[Link] (41 responses)
I loathe the time I had to maintain some init scripts my self, or wanted to see what one exactly executed when. Naturally one could write relatively nice and simple ones, but that did not happen often, and it was still a procedural way of describing something which should clearly be described in schema-based configurations.
I'd guess that it's easier to make a backward comaptible switch from a systemd based one to a new, keeping the configs, than it was switching from sysv to systemd.. Albeit, one needs to support quite some more features, or ship parts of systemd until it's fully replaced.
And for me, that is one of the biggest advantages systemd made, not the faster (parallel) boot, or the pile of additional features.. Just the simple config files where I can *describe* my unit's relation and start/stop behavior in a maintainable way.
Posted Nov 12, 2019 16:33 UTC (Tue)
by BirAdam (guest, #132170)
[Link] (40 responses)
That said, most of this is irrelevant to why most people dislike SystemD (ime). The most common arguments I have heard revolve around security, complexity, and Poettering himself. A lesser heard but still somewhat salient argument has been: Re Hat imperialism. I do not personally find the argument compelling, but I do think it's an argument to be heard. The centralization of so much code into the hands of one guy who clearly doesn't value other people, refers to what came before as "child's tinker toys" and doesn't understand why things were built the way they were isn't a good selling point. Nor is it a good selling point to then say that this is software now largely controlled by a single mega corp.
I say that those aren't selling points, because they aren't. The actual selling point to me for SystemD is that I at least now have some standardization across distributions, which has made things "simpler" from a certain point of view. It's "nice" to be able to use journalctl and systemctl and the like, and have things (for the most part) just work. It's just way worse now when it does not "just work".
The older I get the more I realize that early software systems were built around failure. The idea was that software was shit and therefore you had to expect failure and make it as easy as possible to troubleshoot and fix. Newer software seems to be written from the opposite point of view, make it easier for people to work when it is working and hope it never fails or that those responsible for it when it fails are experts.
As to which of those routes is best is probably better answered by the companies and other organizations that will be consuming the product than by those of us in the industry who apparently just enjoy endlessly arguing about this stuff. Even this though is less relevant than it used to be. People are largely trying to move to outsourcing everything to cloud vendors and having completely managed everything. This then involves more automation, less customization, less performance tuning, etc... I can easily see where the operating system and all of its stuff just "doesn't matter" anymore. Oddly enough, this may also mean that in the end, it's busybox's init that becomes the winner, as that's what Alpine uses, and everyone seems to love containers for deployment.
Posted Nov 12, 2019 16:39 UTC (Tue)
by seyman (subscriber, #1172)
[Link] (3 responses)
Given that the two commands do not do the same things, I seriously doubt that people say that.
Posted Nov 12, 2019 16:41 UTC (Tue)
by BirAdam (guest, #132170)
[Link] (2 responses)
Posted Nov 15, 2019 4:15 UTC (Fri)
by jccleaver (guest, #127418)
[Link] (1 responses)
Coming from RedHat-land, that's one thing that's always bugged me about Debian's system. RedHat had long had the chkconfig and service scripts, which were preferred as the interfaces for both starting and stopping services (via script) as well as canonically turning them on and off on boot.
Among other things, having that interface allowed single locations for expanding featuresets, patching in weird stuff, or dealing with security context switches. Way cleaner than tweaking initscript perms by hand or adding "exit 0"'s in the scripts the way some Debian folks do it. (Of course, that's always there as an option during an emergency -- an important benefit over systemd -- but that's not the preferred way of dealing with services.)
Posted Nov 15, 2019 5:21 UTC (Fri)
by anguslees (subscriber, #7131)
[Link]
The difference I guess is that most *humans* on Debian just invoked the init scripts directly.
Posted Nov 12, 2019 17:17 UTC (Tue)
by jkingweb (subscriber, #113039)
[Link]
It's not about systemctl (after all, service xyz <start|stop|reload> has made a lot of what systemctl offers to users easy for years), but rather what drives it. Given a process with no init script and no systemd unit file that I want to run as a service, there's no question which if those two missing pieces is simpler to write myself. It's not surprising people used to init.d find it simpler and easier, but I wonder how many people familiar with no init systems at all find systemd harder to work with.
Posted Nov 12, 2019 18:24 UTC (Tue)
by smurf (subscriber, #17840)
[Link]
Personally I don't care a whit why sysV init was built the way it was, that was 30 years ago and we might possibly have learned a thing or two in the meantime. I care about whether the new kid on the block does what I need it to do in order to make my job easier. Surprise: systemd does that, very well IMHO. A whole lot of small programs which each replicate one single slice of its features (most of what systemd does isn't exactly new …), each of which needs to be hacked into every daemon's init script, do not.
Posted Nov 12, 2019 23:47 UTC (Tue)
by SEJeff (guest, #51588)
[Link]
Or how about when the process started up and daemonized, but didn't write its pid file out properly or the status method is written poorly (vendor software much?) so now the init script can't stop it? Old versions of tomcat used to be very problematic due to this. In the systemd land, it creates a control group for each service. When it is time to see the status or to stop a service, systemd simply terminates all pids in that control group, a feature native to the kernel. There are no bugs in this because the kernel keeps track of the pids in a given control group, unlik the pid file under /var/run with the wrong pid in it in the olden days.
The restart bits are also super nice, but many other non-sysvinit systems have this. There are so many areas as a systems engineer who has managed tens of thousands of bare metal linux servers over my career that systemd has made it easier. It has caused some problems, it has added some bugs, but overall, it is a step in the right direction. Despite comments like yours, systemd "won" because it is a technically better solution and absolutely no one has written something better to replace it with.
Posted Nov 13, 2019 3:02 UTC (Wed)
by mathstuf (subscriber, #69389)
[Link] (4 responses)
First, I doubt the claim that Poettering doesn't understand why things were the way they were with init systems. And even if so, why would that be a reason to *not* do systemd? Sure, things may have been fast and loose before, but we're in a different time now. I like reliable service process management. I like being able to not have a long-running email sync step on itself because cron is too dumb to know the previous instance is still running (yeah, lock files, but now I have two problems; OnUnitInactiveSec=15m that knows when I manually checked too is just too damn convenient).
As an analogy, sure, I can understand why C is everywhere the past 50 years. I can see why it can still be useful today. But would I write a file format parser in C today? Hell no. I know myself better than that. Sure, choosing something like Rust loses you things like more obscure architecture support with the current state of the toolchain, but I don't have access to test on those things *anyways* and any C code I'd be writing would be "eh, hope they're normal" hopes and prayers. And C is the kind of language that does not answer kindly to hopes and prayers.
Posted Nov 26, 2019 12:06 UTC (Tue)
by immibis (subscriber, #105511)
[Link] (3 responses)
Posted Nov 26, 2019 13:55 UTC (Tue)
by farnz (subscriber, #17727)
[Link] (2 responses)
Yes. After all, we're going from one monopoly (SysV style init), to another (systemd style init), with a small phase in the middle where distros used multiple new inits (upstart, openrc, systemd) in SysV style compatibility mode instead of their native modes.
It's not like SysV style init was something with multiple compatible implementations for most of its life, after all - why is the SysV monopolization good, while the newer styles are bad?
Posted Nov 27, 2019 17:09 UTC (Wed)
by jwilk (subscriber, #63328)
[Link] (1 responses)
https://sources.debian.org/src/debian-policy/3.1.1.1/poli...
Posted Nov 27, 2019 17:33 UTC (Wed)
by farnz (subscriber, #17727)
[Link]
That's not what that document says, though - it says that SysV init supports two different ways to manage initscripts, and they are only documenting one of the two.
Posted Nov 13, 2019 4:30 UTC (Wed)
by tau (subscriber, #79651)
[Link]
I dunno, I get the opposite impression. Old UNIX systems and mainframes were built to try to deal with every possible failure case, likely because any sort of useful hardware had to be big and expensive back then. Instances were unique and extremely precious. Decade-long uptimes were considered something to be proud of. If the hardware or software failed anyway, well, restore from backup. Hopefully that too is a contingency plan that you've tested recently.
Modern systems look like Kubernetes. Which is also a horribly complex beast, but in different ways. Everything is cheap, redundant, and disposable. You know you can deal with failure because it is constantly happening. Distributed state is managed by consensus, network topology is virtualized, program instances are redundant. It is still possible to induce a system-wide failure severe enough to jeopardize the whole system though, and good luck quickly getting to the bottom of what's going on when that happens.
Posted Nov 13, 2019 5:34 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (1 responses)
I recently had a misfortune to debug a (virtualized) system that depended on a daemon taking at least 1 second to start. It was not a problem on 90-s era hardware, but a virtual system suddenly can start it in less than a tenth of a second.
Early Unix systems were crap. At least all Unix and BSD-based systems, I don't have much experience with proprietary ones. They depended on the system being static, on launch timings remaining stable and on tons of other things.
These days we're moving away from this. Today's software is designed to be built from smaller modules that try to encode their assumptions explicitly and deal with the dynamic environment (with all of its failures). Heck, we are even finally getting a reliable process management API in Linux!
Posted Nov 13, 2019 13:40 UTC (Wed)
by pizza (subscriber, #46)
[Link]
Relatedly, a sizeable portion of the "systemd caused failure in X/Y/Z" reports were actually due to systemd properly checking return codes and statuses instead of blindly carrying on. In other words, the old scripts were already broken, but just happened to function well enough (except of course when they randomly didn't) for most folks to not notice.
Meanwhile, those "failures" were actually much easier to diagnose and fix than they would have been before, because systemd also captured *all* stdout/stderr/syslog into a single log stream and made it easy to isolate the messages (via systemctl status or journalctl -u unitname).
Posted Nov 13, 2019 8:18 UTC (Wed)
by pixelpapst (guest, #55301)
[Link] (2 responses)
Wow, I'm amazed at how well you too made this exact same point in my head with the above commend. Because I too grew up with init-scripts, and in my circles, *the* canonical way to disable services always was stopping the service, then editing the init file and inserting an "exit 0" at the appropriate place, ideally with a comment on the reasoning ¹. Which, of course, is not declarative at all.
Up until reading your comment just now, I had never even considered using a chmod -x for disabling a service. Or that some other admin might have done so. So yeah, much better indeed.
(I'm tempted to insert some rambling on why you'd want a failure to be logged in this situation etc., but the main point is made so well, I'll leave it as is.)
¹ Plus Debian's way of some services being disableable in /etc/default/xyz.
Posted Nov 14, 2019 17:38 UTC (Thu)
by derobert (subscriber, #89569)
[Link]
Don't forget the third way of disabling a service — change around the symlinks in /etc/rc?.d so that they're K* instead of S* symlinks (update-rc.d and several other programs could do this for you on Debian, RedHat had its own programs to do it, so did everyone else.). And the fourth, chmod -x /usr/sbin/daemon ... which thankfully I've only seen on other people's systems :-/ systemd thankfully only has two ways (disable and mask). And systemctl status will tell you about both.
Posted Nov 19, 2019 16:36 UTC (Tue)
by jccleaver (guest, #127418)
[Link]
On RedHat/Fedora/RHEL/CentOS/Scientific Linux all of those other mechanisms were deprecated LONG ago, and you were supposed to use the /sbin/service and /sbin/chkconfig commands for a) controlling current state, and b) controlling boot and S/K## ordering. Initscripts were supposed to be immutable (replaced on update) and all config happened in /etc/sysconfig/<scriptname> or /etc/chkconfig.d/<scriptname>, which were treated as config files and remained on update. I'm not sure when these were introduced but it's got to have been at least 20 years now... Certainly there by the Cartman RedHat Linux release.
All these other mechanisms for doing something, other than as a 3am hack, strike me as incredibly dirty. Can't believe other ecosystems hadn't fixed it by now.
Posted Nov 13, 2019 13:06 UTC (Wed)
by anselm (subscriber, #2796)
[Link] (14 responses)
The sad fact is that there really is no deep philosophical “why” there to understand. The traditional init infrastructure was basically cobbled together out of various pieces that happened to float around the community at the time – the init system itself was cribbed from System V, inetd was adopted from BSD, and so on. There simply was no common overarching design to the whole setup, so it is no surprise that the result is a big mess where no two configuration files have the same syntax, there are four or five unrelated ways in which service processes are started, etc. Contrast this to systemd which for its faults at least has a reasonably unified design and consistent features across the board.
Of course if you've grown up with the big traditional mess it's possible that you're inured to it to a point where you don't notice that it is actually a big mess. But if you're new to Linux then systemd is a lot easier to teach and learn (I used to teach Linux administration for a living so I know what I'm talking about; among my students I have yet to meet anyone who when presented with both the traditional setup and systemd didn't prefer systemd).
If I was at all cynical I'd say that the people who have sunk a lot of time into understanding the traditional system hate systemd mostly because all the cruft associated with the traditional setup goes away. You used to have to work really hard and accept a lot of odd things on faith in order to be admitted to the priesthood and now suddenly it's all reasonably straightforward, internally consistent, well-documented, and easily maintainable? Can't have that.
Posted Nov 13, 2019 13:28 UTC (Wed)
by smurf (subscriber, #17840)
[Link]
There's nothing cynical about that, it's basic human nature to act that way.
The sad part is that it happens on a subconscious level, thus the inability of most people to recognize this mechanism in themselves.
Posted Nov 13, 2019 13:46 UTC (Wed)
by pizza (subscriber, #46)
[Link] (5 responses)
Actually, there is -- the UNIX philosophy is first and foremost about simplicity of implementation, at the potential expense of performance, correctness, consistency, and of course, general user friendliness.
(In other words, the "Do One Thing" mentality -- Not to be confused with "Do One Thing _Well_", because "Well" typically requires much more complex implementations)
Posted Nov 13, 2019 15:31 UTC (Wed)
by anselm (subscriber, #2796)
[Link] (4 responses)
I personally would be reluctant to use the words “System V init” and “simplicity of implementation” in the same sentence, but of course you're right in principle.
The main point, really, is that the traditional setup was built from pieces that all purported to “do one thing” each, but nobody ever took the time or trouble to ensure that the various things being done are configured sanely, or even that if the same thing is being done in various places it is being done in roughly the same way. (Which of course is an issue that is endemic in Unix – look at the various command-line options for specifying field separator characters in sort, cut, awk, join, …)
By contrast, systemd builds on a common design and infrastructure which ensures, for example, that the same features and mechanisms are available regardless of whether a service process is started by (in terms of the traditional setup) /etc/inittab, an init script, inetd, cron, or at – which is something that the traditional setup does not address at all. In this case, systemd is much more about “do one thing well” than the traditional setup ever was, mostly because it includes library support to do that one thing which is used wherever that one thing needs to be done.
Posted Nov 13, 2019 19:27 UTC (Wed)
by rgmoore (✭ supporter ✭, #75)
[Link] (3 responses)
My impression is that SysV init shows one of the major drawbacks of the Unix philosophy. Unix is supposed to provide a nice toolbox of "do one thing and do it well" programs that let sysadmins hack together quick and dirty solutions to their problems when they need to. Over time those quick and dirty solutions become critical components of the system in their own right, but there's no guideline in the Unix philosophy about when it's time to rewrite the hacked together shell script with a properly written C program. Even worse, as your example of init, at, cron, inetd, etc. shows, there's no guideline about recognizing when there are multiple programs doing related things that need to have their shared functionality merged. The net result is that hacks like SysV init scripts hang around long past the point when they should have been replaced by a better written solution.
Posted Nov 13, 2019 19:48 UTC (Wed)
by anselm (subscriber, #2796)
[Link] (2 responses)
I think this has less to do with the “Unix philosophy” and more with the fact that for a number of its crucial early years Unix wasn't a “product” in the sense that other operating systems are “products”; not unlike Linux in the early 1990s, Unix in the 1970s was a fun side project of a few people that they graciously shared with other interested folks. If you compare Unix with, say, VAX/VMS, which is roughly of the same vintage as Unix and, unlike Unix, used to be an actual supported product, you will find that the good people at Digital Equipment, for example, expended a huge amount of effort to make command-line options align between different commands in a way that the Unix community never managed.
When the “Unix toolbox” was assembled in the mid-to-late 1970s, the guiding principle was more along the lines of “here's another neat program”, people didn't really worry too much about consistency, and there was no product manager, “benevolent dictator”, or written human-interface guide to ensure that command-line options that do the same thing are actually identical between different commands.
Posted Nov 13, 2019 22:27 UTC (Wed)
by rgmoore (✭ supporter ✭, #75)
[Link] (1 responses)
And my gut feeling is that this kind of anarchy has become part of the Unix philosophy. The idea of using small "do one thing and do it well" tools fits in perfectly with disconnected developers working without oversight from a master architect. Maybe you can say that this is part of Unix culture rather than Unix philosophy, but it's clear that separate tools developed by people who aren't always talking to each other is deeply embedded in the way Unix is done.
There's a reason it was Unix, the system initially developed by people having fun in their spare time and writing loosely coupled tools, that was eventually replaced by Free Software rather than one of the top-down designed systems. There's no way the GNU tools could have been developed if it weren't practical to rip individual components out of Unix and replace them with alternative implementations. And things like Linux and systemd show that this is possible even with critical components like the OS kernel and init system.
I think this is an underappreciated part of what people who object to systemd don't like about it. You can argue all you like about the systemd project consisting of a bunch of separate tools that just happen to share a repository, but it is still being developed by a group that coordinates closely and has built its tools to cooperate closely. That's very different from the traditionally anarchic Unix approach, and importantly, that close coordination makes it more difficult for somebody to take just the parts they want and replace the ones they don't.
Posted Nov 13, 2019 23:12 UTC (Wed)
by anselm (subscriber, #2796)
[Link]
I don't think that's actually the case. The various pieces of systemd, their interactions and interdependencies are in fact documented quite well (certainly more so than those of other Unix-based tools). I do agree with Lennart elsewhere in this discussion that it's probably not entirely trivial to “replace the ones they don't want” because there's a real risk of reinventing subtle bugs that the official components have fixed a long time ago, and in practice there's likely not a whole lot to be gained from attempting that sort of thing.
In any case I don't think anyone would have objected if the authors of sort, awk, etc. had at some point got together and decided to standardise the names and meanings of their common command-line options. It would definitely have made Unix stronger. So close coordination is not necessarily a bad thing – as a user and educator I would certainly prefer it over gratuitous diversity.
Posted Nov 14, 2019 1:03 UTC (Thu)
by sorpigal (guest, #36106)
[Link] (6 responses)
Am I alone in thinking that this is a feature and not a bug?
A system cobbled together is malleable (you can always cobble something different.) All of the discussion in comments here about how complicated it is to replace logind with elogind in a compatible way is, to me, evidence that the whole systemd thing isn't "cobbled together from parts" *enough*.
I like the feel of a system made out of parts; I'm also a big fan of legos. Maybe it was unintentional, but the messy system that evolved out of the 80s exhibited behaviors I liked even if there's nothing about the "design" that particularly recommends it. A well designed system which preserved the "just a pile of bricks you can unstick and restick any which way" feel of that would be most appealing.
Posted Nov 14, 2019 8:46 UTC (Thu)
by dvdeug (guest, #10998)
[Link] (2 responses)
You can't unstick basic components from Linux or any other major system and restick them any which way. To replace any basic tool with a fundamentally similar tool is going to find problems where the new tool is not 100% compatible with the original, including bugs. To do something interestingly new is going to require lots and lots of work, and ongoing work if you can't convince the world to switch to your new way.
Posted Nov 18, 2019 17:05 UTC (Mon)
by jezuch (subscriber, #52988)
[Link] (1 responses)
Well, it took Debian literally years to make it possible to use dash as the default /bin/sh instead of bash. There was a huge campaign to rid all the scripts of "bashisms" - in fact I don't even know if it has been completed :)
Oh, and the kfreebsd architecture in Debian... Is not faring well, I think.
Posted Nov 19, 2019 8:50 UTC (Tue)
by tao (subscriber, #17563)
[Link]
Posted Nov 14, 2019 11:50 UTC (Thu)
by tao (subscriber, #17563)
[Link]
The traditional UNIX userspace is more like older style Meccano. You can still fit all the pieces together,
The worst trouble I encounter when building LEGO is typically that a piece I need isn't available in that particular shade (yet).
Having different designers isn't a bad thing if they try for consistency on a UI level. Having to memorise
Posted Nov 18, 2019 17:09 UTC (Mon)
by jezuch (subscriber, #52988)
[Link] (1 responses)
It's also brittle. Undocumented. Not discoverable. It fossilizes quickly and a cargo cult develops around it ;)
Posted Nov 26, 2019 12:08 UTC (Tue)
by immibis (subscriber, #105511)
[Link]
Posted Nov 13, 2019 22:52 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link] (5 responses)
Well, so you know it's one thing telling me I don't understand stuff, but I'd really prefer if you'd not put quotes in my mouth I am pretty sure I never said. I googled for "child's tinker toys" now, because it's not really in my active vocabulary, and I hence was curious where you picked up that supposed quote of mine, and all I found was this LWN comment of yours.
Or should I claim that BirAdama refers to his left hand as "Mr. Noseypokey"?
Thank you,
Lennart
Posted Nov 14, 2019 1:33 UTC (Thu)
by BirAdam (guest, #132170)
[Link] (4 responses)
Posted Nov 14, 2019 8:32 UTC (Thu)
by hunger (subscriber, #36242)
[Link]
Posted Nov 14, 2019 10:31 UTC (Thu)
by edomaur (subscriber, #14520)
[Link]
Posted Nov 14, 2019 13:28 UTC (Thu)
by mezcalero (subscriber, #45103)
[Link] (1 responses)
Posted Nov 14, 2019 14:23 UTC (Thu)
by zdzichu (subscriber, #17118)
[Link]
Now, let's forfeit nitpicking and return to technical discussion.
Posted Nov 15, 2019 5:29 UTC (Fri)
by anguslees (subscriber, #7131)
[Link]
With the old sysvinit way of doing things, you only had a global sort-ordering of scripts, and anything else you had to actually execute the script to discover.
Amongst other things, the config file approach gives you all the information you need to "easily" generate a sysvinit script and ordering (if you wanted to migrate in that direction) - whereas the opposite is basically impossible.
If we ignore all the rest, moving the ecosystem to systemd unit files (or something similar) gives us a much more useful description of our system.
Posted Nov 12, 2019 16:51 UTC (Tue)
by rweikusat2 (subscriber, #117920)
[Link] (1 responses)
Newsflash: The world has moved on, guys. Please revive your zombie debate in a movie about it.
Posted Nov 12, 2019 17:57 UTC (Tue)
by penguin666 (guest, #135483)
[Link]
what?
Posted Nov 12, 2019 17:37 UTC (Tue)
by penguin666 (guest, #135483)
[Link] (39 responses)
systemd increases complexity and attack surface massively. (`systemd-homed` next?)
traditional init systems like sysvinit are easier to sanity check and secure, UNIX philosophy of minimality and essentialism.
... is the init system doing what it is supposed to be doing??? With sysvinit there are just a few things to check, a few things to lockdown and secure, sanity checking doesn't drive you insane.
with systemd, apart from core complexity how many systemd-blahd tentacles are there? more added how often? are we expected to have a comprehensive total understanding of all the tentacles at all times, as they mutate and grow neoplasms?
i don't know ...
Posted Nov 12, 2019 18:23 UTC (Tue)
by apoelstra (subscriber, #75205)
[Link] (1 responses)
I really really cannot understand how someone could make either claim about shell scripts.
Posted Nov 26, 2019 12:09 UTC (Tue)
by immibis (subscriber, #105511)
[Link]
Posted Nov 12, 2019 18:54 UTC (Tue)
by rgmoore (✭ supporter ✭, #75)
[Link] (1 responses)
The "bigger attack surface" argument only works if you compare systemd to SysV init alone. If you compare systemd to SysV init plus all the scripts it needs to do its work, plus all the other programs you need to replicate systemd's full functionality (e.g. cron, at, inetd, etc.) the balance swings the other way very quickly. That's especially true when you consider the typical quality of init scripts, which tend to be among the worst available examples of cut-and-paste cargo cult programing.
Posted Nov 12, 2019 20:50 UTC (Tue)
by smurf (subscriber, #17840)
[Link]
I rarely see code with that many "assert" and "if(error) return log_error()" statements, not to mention their 10k lines of tests – which, while far from enough if you want complete coverage of 250k lines of source code, is still way more than most other init-related tools out there have.
Posted Nov 12, 2019 19:20 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link] (14 responses)
Debian, for example, shipped for many years with a BIND9 script that could hang the system on shutdown indefinitely because of a small mistake. I had to run to our DC at night to power-cycle a server because of this once.
Posted Nov 12, 2019 20:53 UTC (Tue)
by wahern (subscriber, #37304)
[Link] (13 responses)
This has caused me problems diagnosing issues with remote management scripts, though no midnight trips to the DC, yet.
I'm sure there are ways to fix this behavior. But my point is that systemd hasn't just solved problems, it's created them, too. Whether it's caused more problems than it has solved depends on your context, but in general systemd seems to prefer convenience and short cuts (particularly for interactive and desktop use cases) over correctness and playing nice with the wider ecosystem.
Posted Nov 12, 2019 22:13 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Nov 13, 2019 22:40 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link] (11 responses)
We send SIGTERM and wait for the processes to react to that. It's how this works on UNIX. Apps can catch that if they want, and do what they need to do. After a time-out we will eventually kill the process with SIGKILL though, like everything that refuses to terminate. It's what process babysitting means. What else would you expect us to do? Not send SIGTERM and just hope that programs exit? I am not sure such indeterminism would be a good idea.
Or is this a comment about ordering? systemd tries to run as many jobs as quickly as it can, which means: when the system goes down it will SIGTERM these sessions as one of the first things, before it terminates daemons or even unmounts anything, simply because sessions generally require daemons and mounts to work, and not the other way round. sysvinit had not sensible understanding of ordering or requirement, hence it generally ended up killing ssh children at arbitrary places, typically after running most service init scripts for stop. However, I'd argue that's deeply problematic, since that way user code gets services it might rely on pulled under its feet, and that's really not a good idea.
Hence, I'd argue that systemd really does everything right here. We follow UNIX semantics of sending SIGTERM and we do it properly ordered.
Lennart
Posted Nov 14, 2019 6:52 UTC (Thu)
by smurf (subscriber, #17840)
[Link] (10 responses)
On the other hand, sysVinit runs a shell script for each and every step on the way, which mostly prevents this problem. (Except when it conveniently forgets to kill your session …)
Inserting a half-second delay after killing your sessions should fix this.
Posted Nov 15, 2019 7:14 UTC (Fri)
by xnox (guest, #63320)
[Link] (9 responses)
Yes, the point is to shut down fast, because one is billed for it. And come back up quickly to reduce nonproductive maintainanance window.
In DC setting, one would have ipmi serial console to monitor shutdown, and power status. Not derive whether or not shutdown is started or not via SSH output.
If you need to perform shutdown tasks and longer timings, use facilities to do that with inhibitors and shutdown binaries that can do that. And one can keep networking up with critical connections up.
Just because something is operating differently, does not imply that previous and current behaviour was/is race free, and there are tools to guarantee desired behaviour in a declarative manner, instead of by-luck.
I understand that when something happens quickly, and one doesn't understand the sequence of events it feels unknown and foreign. Learning to capture, debug, order, inflect shutdown tasks and inhibitors builds knowledge and confidence in that particular aspect of modern operating systems. Without learning, one will not make peace with the unknown.
Posted Nov 16, 2019 2:05 UTC (Sat)
by flussence (guest, #85566)
[Link] (1 responses)
Posted Nov 16, 2019 7:14 UTC (Sat)
by smurf (subscriber, #17840)
[Link]
Posted Nov 18, 2019 4:20 UTC (Mon)
by cas (guest, #52554)
[Link] (6 responses)
yes, they do.
That's why it pisses me off that on some of my servers running systemd, it can take tens of minutes - or longer - to shutdown or reboot.
It often goes into a seemingly endless wait-loop - I say "seemingly" because I don't have hours or days to wait for the machine to finish shutdown and reboot, and frequently have to just physically power-cycle the machine just so I can reboot it sometime today rather than tomorrow. or next year.
And "hours" is NOT an exaggeration - I have occasionally left machines in that wait-loop state for half a day or a day just to see if it will eventually give up and reboot like I told it to. That patience is not rewarded, the wait is only resolved with a power cycle.
I have NFI why it does that, but if I watch the console while a server is shutting down, I see systemd repeatedly increasing the timeout while it's waiting for something (like stopping a service or unmounting a filesystem or de-configuring a network interface) to stop. and for each repeat it increases the timeout - from tens of seconds to several minutes to tens of minutes and even hours.
why even bother having a timeout on a stop action if you're just going to repeat the wait with an ever-increasing timeout?
I **never** had that problem with sysvinit. My machines still running sysvinit give everything a reasonable timeout to stop cleanly - and then reboot anyway. Which is **exactly** what a server **should** do when an zombie process or whatever won't die - otherwise it'll get stuck waiting forever.
in my experience, systemd does not cope well with error conditions and takes ages to either boot up, shutdown, or reboot if there are any problems - especially with drives or networks. and when shit like that happens, that's exactly the time you want the machine to reboot quickly so you can find the problem and fix it. not wait 10s of minutes or hours for every reboot.
if everything goes well, systemd can boot up in 5 seconds rather than 15. amazing! that's really worth writing home about. if not, systemd might finish booting in hours rather than seconds.
---
And booting up with systemd is no better - on one of my machines, booting it will sometimes trigger an endless (?) loop of adding the same half-dozen routes to the routing table, ending up with MILLIONS of copies of the same half-dozen routes in the routing table (as shown by "ip route list > ip-route-list.txt" - the last time I tried that, ip-route-list.txt was over 15 million lines long...and it was only that "short" because I got bored of waiting for "ip route list" to finish, and hit ^C to kill it). e.g.
# wc -l /root/ip-route-list.txt
To stop it, I have to login on the console, down the interface(s), and then manually bring them back up again. and then I have to manually restart the services that were bound to specific IP addresses (such as the ones I only want listening on an internal LAN address rather than a live internet address. The ones that listen on all addresses generally didn't need to be manually restarted). Obviously, this is a completely shit thing to have to do on a server - it's supposed to boot up and start providing its services without manual intervention.
Again, I have NFI why that happens. Neither systemd-analyse nor systemctl status nor journalctl have any useful information to offer.
At least with sysvinit i'd be able to find out which script had a broken loop. At worst, i could add debugging printf statements in likely-suspect scripts. with systemd, it just does what it wants to do when it wants to do it and fuck you if you want to find out why or, cthulhu forbid, override it.
I've got to the point with this infuriating bug that I'm going to P2V clone the entire system into a VM so that I can experiment enough to find out WTF is going on without having my server down for days on end. I'll be able to reboot the VM as often as I need without disturbing the other machines that depend on the services it provides (NFS, DNS cache, dhcpd, squid proxy, and more).
my current guess is that it'll probably turn out to be some stupid race condition - but I don't know where or what's causing it.
and on another of my machines (my mythtv box at home), it boots perfectly with sysvinit but whenever I try to boot it with systemd it fails to boot at all. every single time.
Posted Nov 18, 2019 5:03 UTC (Mon)
by sjj (guest, #2020)
[Link] (3 responses)
Posted Nov 18, 2019 7:56 UTC (Mon)
by cas (guest, #52554)
[Link] (2 responses)
Right, I'll remember that in future. Your amazing and incredibly useful insight completely overshadows my 40+ years of experience in diagnosing and solving computer problems. Ignore problems, and don't expect software to work properly, otherwise you're just being "butthurt". Wonderful.
Or, here's a thought: you so strongly believe that systemd can do no wrong that any evidence that it's less than perfect triggers a defensive reaction of blowhard bullshit lest the cognitive dissonance blow your tiny mind.
BTW, there's little or no point in submitting systemd-related bug reports to debian because most of my fellow DDs have caught the "fuck you, you're doing it wrong, it's your fault" disease from systemd developers. because, of course, systemd can do no wrong.
In any case, there's nothing useful to report because I don't know what the cause is yet. I have some vague suspicions about what the source of the problem might be but as far as reportable facts, I don't have much more than "it's broken (sometimes)", which is about the most useless bug report possible....a complete waste of time for everyone concerned.
Have A Nice Day.
Posted Nov 18, 2019 8:40 UTC (Mon)
by zdzichu (subscriber, #17118)
[Link]
But if it still not enough, I'd expect maintainer to provide targeted debugging instructions. If you feel there's no point in submitting bug reports, I'd strongly suggest to change your provider. There are many distributions to choose from.
Posted Nov 21, 2019 13:04 UTC (Thu)
by mathstuf (subscriber, #69389)
[Link]
At some point, this is still useful to file. Maybe the developers have seen it before and know where to point you for the fix and/or discussion of a fix. Sitting in your corner with your puzzle you won't show anyone complaining that it's "impossible" doesn't do much to convince anyone that it's actually an "impossible" problem. There's no shame in saying "well crap, I'm out of ideas, anyone else have a thought about this problem?".
Posted Nov 18, 2019 12:10 UTC (Mon)
by pizza (subscriber, #46)
[Link] (1 responses)
Every time I've seen systemd stuck waiting on something, I've seen it get reported to the console, and start/complete times are logged. I've not personally seen any shutdown task without a hard upper time limit (and that limit is configurable globally) but I suppose one could exist elsewhere, perhaps as part of some distro/daemon/application-provided unit.
I might add that this is probably not a "systemd bug". The underlying problem is that something is not shutting down cleanly, and the system has been explicitly configured to wait indefinitely for it. In other words, it's most likely due to how your distribution (or software supplier, or admin) integrated the troublesome component.
Have you at least reported this bug? Clearly this is not desired behaviour.
Meanwhile, with respect to your ip routing bug. You didn't say how networking was configured; networkmanager? systemd-networkd? distro (which one?) ifup/down scripts? script that manually invokes ifconfig/ip tools? dhcp or static configuration? Are these routes statically/explcitily configured, or automagic with respect to the networking infrastructure? Details matter, and supplying that (here or as part of a bug report) doesn't require setting up VMs or whatnot.
(FWIW, this sounds a lot like what happened when I wrote an incorrect networkmanager hook script to selectively enable a VPN -- It worked, except when it didn't, ending up with multiple VPN instances stepping on each other's toes with ping times that would grow to 4-5s before the whole thing came crashing down and restarted itself..)
Posted Nov 21, 2019 13:07 UTC (Thu)
by mathstuf (subscriber, #69389)
[Link]
I agree. It sounds like some dependency specification problem. I see it as akin to "parallel build is busted, but the serial build works". Well, this means your rules aren't listing their dependencies properly. The right fix is to find that missing link and fix it, not say "well, then just use `-j1` and be happy". Granted, systemd is at times tight lipped about some of these things, but it takes some analysis of the logs of services and the systemd service event logs to figure out.
Posted Nov 13, 2019 5:26 UTC (Wed)
by mebrown (subscriber, #7960)
[Link] (9 responses)
Each systemd-* component is... wait for it... a separate binary. Systemd is quite highly modular, so it doesn't follow that just because you have one git repo that compiles to a few dozen binaries that it's more complex or a higher attack surface. It's all modular and you use the pieces you need to use. How does being co-located in a repo make this more complicated than the maze of twisty tarballs we had before?
It was most definitely NOT easier to sanity check older init systems. Trust me, I had to do that on several projects. Auditing all the scripts and, more importantly, keeping the uninformed developers from adding crap where it shouldn't be was the biggest pain. Not to mention when a new technique came along and I had to go update dozens of init scripts. It took tons of hours to ensure that we were secure, had proper ulimits, umask, user/group, no tmpfile vulnerabilities, etc for a small embedded system. That's not even to mention trying to parallelize boot or make it "boot faster"... that was a daunting task that just made me give up on sysv init. The race conditions and holes trying to get everything going was too complex.
I, for one, welcome our new systemd overlords. My init hell is now a cozy hot-tub in comparison to what it was. Boot time on our first systemd shipping system was 30s compared to almost 2 minutes previously. And we dropped from ~10k lines of init scripting to <500 lines of helper setup scripts. It was a *huge* jump in maintainability and I'll never go back.
Posted Nov 26, 2019 12:11 UTC (Tue)
by immibis (subscriber, #105511)
[Link] (1 responses)
Posted Nov 26, 2019 14:09 UTC (Tue)
by mathstuf (subscriber, #69389)
[Link]
- libesystemd is *not* ABI/API compatible?
How are either of these problems libsystemd's fault rather than limitations of the supposed replacement and dpkg/apt-get bugs?
Posted Nov 26, 2019 20:42 UTC (Tue)
by flussence (guest, #85566)
[Link] (6 responses)
If that's your metric for success, you should have used runit (which predates systemd by a decade).
~ $ find /etc/service/ -follow -name run -exec wc -l '{}' + 2>/dev/null | tail -1
Posted Nov 26, 2019 21:14 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link] (5 responses)
Posted Nov 28, 2019 2:39 UTC (Thu)
by flussence (guest, #85566)
[Link] (4 responses)
Posted Nov 28, 2019 7:18 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
runit can't handle classic double-forking daemons. Fortunately, because of systemd most daemons these days have a foreground mode.
Posted Nov 28, 2019 8:09 UTC (Thu)
by smurf (subscriber, #17840)
[Link]
Happily the once-common failure mode of
* daemonize, which involves …
is a thing of the past. Don't ask me how often I found that kind of thing with strace.
Posted Nov 29, 2019 3:55 UTC (Fri)
by flussence (guest, #85566)
[Link] (1 responses)
Posted Nov 29, 2019 6:12 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Nov 13, 2019 16:14 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link] (9 responses)
Posted Nov 13, 2019 20:29 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (8 responses)
I know that musl misses some needed functions, but can you just provide a musl_compat.c file for them? It would not affect much of anything else and musl's maintainer is even more... err.. opinionated than you are.
Posted Nov 13, 2019 21:13 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link] (7 responses)
I mean, for example qsort_r() as I understand is not available in musl. It's a truly useful API though, and life without it is very hard. If the musl people want to argue that it's evil because it's not in POSIX then I must say I really don't believe they know what they are doing, and don't understand that a libc's primary purpose is to provide services to applications, and not in being true to the one theoretic religion of POSIX.
We could of course ship compat implementations of qsort_r() and all that stuff, but I genuinely don't want to maintain it. And if everyone ships their own, undermaintained reimplementations of these common sense calls then you just increase footprint on disk, in memory and most importantly in maintenance, and I don't want that.
Hence, fix the stuff in musl once, don't put the burden on all apps to ship compat kludges for it, in a myriad of individual projects. i.e. if you can patch the same reasonably API code into musl once and solve it for everyone, or you can patch it into every single app using it and only solve it for that one app then, at a price on resources and maintainance burden, then I am clearly on the side of saying it must be musl that carries that patch for this.
Also note that we previously have merged various patches for musl compat into systemd that were about including headers and such, i.e. where one could make the point that including header x is more in line with POSIX than including y, if both x an y get your symbol z and both work on glibc and one only with musl. And we'll continue to do so.
Anyway, please talk to the musl people. (I hear they don't like systemd very much, but I am not sure we should fix that social problem by reimplementing parts of libc in systemd for you...)
Hope that makes some sense, though it's disappointing I figure.
Lennart
Posted Nov 13, 2019 23:26 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (6 responses)
> We could of course ship compat implementations of qsort_r() and all that stuff, but I genuinely don't want to maintain it.
Realistically it's not even a big amount of code to maintain, pretty much all of it can be isolated in a couple of files. Would you at least consider it if it's submitted to systemd, with a disclaimer that it's not an officially supported and tested combination?
Posted Nov 13, 2019 23:28 UTC (Wed)
by mjg59 (subscriber, #23239)
[Link] (3 responses)
Posted Nov 13, 2019 23:34 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (2 responses)
Systemd also pokes into some glibc-dependent internals (like cpuset) which require small patches into systemd's internals. A third-party library would require inclusion of something like "compat.h" in several systemd c-files.
Posted Nov 14, 2019 9:30 UTC (Thu)
by mezcalero (subscriber, #45103)
[Link] (1 responses)
Posted Nov 14, 2019 19:56 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Nov 14, 2019 9:23 UTC (Thu)
by mezcalero (subscriber, #45103)
[Link] (1 responses)
Well, again, fixing this in musl is one patch that makes many apps happy. Working around this in all those apps individually means many many patches, making only musl happy... It's pretty obvious what the more rational side here I think, no?
And no, I don't think we should reimplement qsort, it's not really the business we are in, I don't want to see that in systemd... I mean, patch it in downstream if you must, but I'd advise you to patch it in downstream into musl, really, it's saves you a lot of work...
Posted Nov 14, 2019 19:10 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
> Well, again, fixing this in musl is one patch that makes many apps happy. Working around this in all those apps individually means many many patches, making only musl happy... It's pretty obvious what the more rational side here I think, no?
Most free software avoids using non-portable features of glibc exactly because it wants to be portable to BSDs and other OSes. So musl works great for it. Heck, even monsters like Java SDK work on musl.
Posted Nov 12, 2019 18:02 UTC (Tue)
by jmclnx (guest, #72456)
[Link]
The best thing about Linux is you do not need to stick to just one distro, there are plenty not using systemd, so if you want to use a system without it.
Posted Nov 13, 2019 16:08 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link] (6 responses)
But most importantly: even stuff such as tmpfiles.d/ is inherently really really hard to get right, even if from a view far away it looks trivial since all it does is a bit of mknod()+chown()+chmod(), right? I know that it is hard because it took us long to get it right. Here's the thing: UNIX doesn't make writing a tool such as tmpfiles.d/ easy — at least not if you want to do so securely. Why? Because all those issues with concurrent execution on Linux: people might replace files underneath you, there are symlinks and hardlinks, and you only have very limited control what a specific path really means at a specific moment. In our tmpfiles implementation we nowadays use openat() for these things, and follow symlinks entirely manually to fix all these races (and thus potential vulnerabilities) as much as we can. Of course, people are good at ignoring these complexities, because they don't think about this. For example gentoo came up with a shell script to replace it: https://github.com/OpenRC/opentmpfiles/blob/master/tmpfil... – there are more security vulnerabilities (and benign races) in it then I can count, it's just baaaad. I mean, I am not saying one couldn't reimplement this as safe as the original (or even safer), it's just a much harder, much more time consuming project than people could ever believe.
And don't get me started with the idea to write another executor for unit files, like Allbery has been suggesting: this stuff is much more complex even.
Yes I too would like more competition, because there's nothing really we can compare us with anymore. Everything interesting (SMF, launchd) kinda stopped progressing. But I am pretty sure just copying 1:1 what systemd came up with is pointless. Take our code, it's free, fork it if you must, but why reimplement the exact same thing in a crappier way again? If you do something new, then doing something properly new and don't wast your time in just driving the evil daemons out of codebases you think are possessed...
Lennart
Posted Nov 13, 2019 20:45 UTC (Wed)
by HenrikH (subscriber, #31152)
[Link] (5 responses)
I know that you don't get to hear this enough Lennart, but from me and the whole team of developers and admins where I work, and from the bottoms of our hearts: Thank You! The world in where both pulse and systemd exists is a better and brighter one.
Posted Nov 13, 2019 21:14 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link]
Posted Nov 14, 2019 4:17 UTC (Thu)
by rra (subscriber, #99804)
[Link] (3 responses)
Replacing sysvinit with something better was quite difficult because there actually wasn't a single implementation of "sysvinit." It was more of an idea than a program. There was a giant ecosystem of random shell scripts and utilities and conventions and tools that parsed comments to discover ordering and a mess of problems that were solved and resolved by every package or every distribution independently (creating users, creating temporary directories, and so forth). The transition was therefore quite painful because it wasn't replacing one package with another, it was replacing a bunch of quite hairy and undocumented components with truly esoteric behavioral interactions and wildly varying implementations across distributions with a standardized set of configuration files and implementations largely shared across distributions.
I understand that some folks are not happy that this has happened, but this is largely what has happened, whatever one's feelings about whether it was a good idea.
Like systemd or not, what it's done is turn a bunch of ad-hoc, tiny programs in a full if irritating programming language into an API based on configuration files that clearly specifies what a particular program wants to have happen. This is huge, because (as someone else pointed out earlier in the comments) this makes replacing systemd far, far easier, should anyone really want to do it, than replacing sysvinit.
The point that I am trying to make is that if you believe in init system diversity, the path forward is not back to an ad hoc mess of shell scripts. This makes no sense to anyone who comes at the problem from the outside. The path forward is to recognize that systemd is giving you a great gift, namely an implementation-agnostic API which you can reimplement in a way that's transparent to the consumer of that API and then add your own unique spin to.
I'm trying to get init system diversity supporters to realize that a standardized API is a *boon* to their cause, not some sort of nefarious attack on it, because standardized APIs are exactly what permit multiple competing implementations, and because there's nothing in the API that forecloses any of the things that folks who don't like systemd are arguing for. You can implement that API perfectly well with a variety of independently-developed programs that do only one piece of the problem, for instance.
Full credit to Lennart and the rest of the systemd team for the work that they've done, but another team of people absolutely could write a new implementation of that API from scratch and possibly even do it better. It's just months and months and months of work. I agree with the idea that, all things being equal, taking advantage of the properties of free software and forking systemd rather than starting from scratch and writing all new bugs would make a lot of practical sense. But everyone reading this certainly knows that free software developers aren't necessarily practical, and sometimes that's one of the wonderful things about our community. For example, I fully expect someone to eventually write a new init system in Rust just because they can, and there's no obvious reason to start such an effort by forking systemd.
But there is an obvious reason to start that effort by implementing systemd's configuration API. Because that's what makes diversity actually practical.
Posted Nov 15, 2019 5:35 UTC (Fri)
by ncm (guest, #165)
[Link]
Posted Nov 19, 2019 23:40 UTC (Tue)
by rgmoore (✭ supporter ✭, #75)
[Link] (1 responses)
To put this slightly differently, the biggest problem with SysV init is that it fails to separate policy from mechanism. An init script is inherently about both what is supposed to happen and how to do it. Since the "how" is hard and involves lots of details to get it right, it tends to overwhelm the "what". This leads to two problems:
systemd deals with both these problems by separating policy (unit files) from mechanism (systemd itself). Unit files are much easier to understand than init scripts, solving problem 1. And mechanisms only need to be written and debugged once in systemd for every program it's starting to benefit, solving problem 2.
Posted Nov 20, 2019 0:38 UTC (Wed)
by pizza (subscriber, #46)
[Link]
Exhibits 1-$VeryBigNumber: improper handling of variables, arguments, or data that contains spaces, leading to all sorts of joy.
Posted Nov 15, 2019 16:27 UTC (Fri)
by mirabilos (subscriber, #84359)
[Link] (4 responses)
1. elogind is not new, it’s been there since before buster *and* is included in the buster release, but the other packages that had a hard dependency on it need to adjust their Depends. (I’ve done this successfully on a buster system by editing two lines in /var/lib/dpkg/status to the effect that even network-manager works.) It’s been usable as-is for some time already, just the “can substitute for systemd” part is new (and almost solved in unstable for a while).
2. “merging” is completely the wrong phrasing. elogind is an alternative, and it must be installed by hand and does not affect the default systemd configuration.
This directly bleeds over into…
3. Md is telling nonsense as usual: the new installations use systemd *because* there are dependencies on it from certain packages many people install. People did not have the *option* to use elogind without going through extreme hoops until very recently in unstable (and even then, switching is… interesting). So the statistics are utterly skewed.
Posted Nov 15, 2019 17:25 UTC (Fri)
by smurf (subscriber, #17840)
[Link] (2 responses)
Posted Nov 15, 2019 21:58 UTC (Fri)
by mirabilos (subscriber, #84359)
[Link] (1 responses)
And hmm yes, that was, in reflection, badly worded. Sorry about that.
Posted Nov 16, 2019 7:12 UTC (Sat)
by smurf (subscriber, #17840)
[Link]
Posted Nov 17, 2019 12:45 UTC (Sun)
by lobachevsky (subscriber, #121871)
[Link]
Posted Nov 18, 2019 4:17 UTC (Mon)
by cas (guest, #52554)
[Link] (5 responses)
Up to this point, systemd is pretty good. Certainly not objectionable. It works well enough, and even though shell scripts don't scare me off sysvinit or other alternatives, I'm happy enough to use it. I don't care enough about init for it to be a big deal as long as it does its job and stays out of the way.
The problem is that systemd tries to do a lot more than that.
The problem is that it's not just "systemd as init", it's that it comes with an enormous amount of undesirable baggage - all the "systemd-borgd" crap providing half-arsed implementations of cron, dns resolver, ntp, syslog, network configuration, login session management, and much more.
None of this stuff belongs in init. And none of it should be so tightly integrated with init.
None of these "systemd-borgd" built-ins are useful for machines that **don't** closely resemble the authors' personal desktop machines. They're not capable of being used for anything that doesn't match the devs' personal needs. and, worse, the devs aren't at all interested in catering to anything they don't personally need.
And, worse, every time systemd borgs another common service it breaks whatever was previously providing that service and I have to spend time and effort figuring out that The Borg has struck again and then more time and effort figuring out how to unfuck what it fucked.
and THERE is the problem.
If the systemd devs want systemd to be uncontroversial, they can get rid of all that shit.
And if they think they can do a better DNS resolver, a better cron, a better log service, a better ntpd, a better anything-that-isn't-actually-init then they can write a standalone daemon to prove it. There's no need - and no justification at all - to do Microsoft-style monopolistic bundling to force every everyone to use their implementation.
It doesn't even matter much if distros use the systemd versions of common services by default. What matters is that the systemd devs' implementations:
1. aren't privileged with special access to init's internals, hard-coded dependencies, or deliberate sabotage of alternatives within systemd
2. are easy to disable - without breakage of other stuff running on the system
3. are reasonably easy to replace with other alternative/competing implementations.
---
PS: i'm sick and tired of seeing this stupid argument presented as "systemd vs sysvinit", when it's really "monopoly vs diversity", and i'm sick and tired of seeing arguments against all the non-init stuff that systemd attempts to do being dismissed with goalpost-shifting bullshit hand-waving about "but it's a nice init system". Nobody's complaining about the init aspects of systemd. The complaints are about its Borg nature.
Posted Nov 18, 2019 11:50 UTC (Mon)
by pizza (subscriber, #46)
[Link] (3 responses)
You do realize you're commiting the same "I don't care about those use cases, therefore they're not valid for anyone" hubris that you're accusing the systemd cabal of doing? (Meanwhile, the systemd developers have demonstrated time and again that they care about use cases beyond their personal needs...)
> And if they think they can do a better DNS resolver, a better cron, a better log service, a better ntpd, a better anything-that-isn't-actually-init then they can write a standalone daemon to prove it.
And they did just that; resolver, cron, logging, ntpd, etc etc are standalone daemons, not part of PID1.
> None of this stuff belongs in init. And none of it should be so tightly integrated with init.
Except for logging (which has special considerations) "so tightly integrated" is actually nothing more than "resides in the same source tree and relies on a shared library."
> 1. aren't privileged with special access to init's internals, hard-coded dependencies, or deliberate sabotage of alternatives within systemd
Except for journald, all three are already the case -- at least with respect to systemd and its authors.
Now distributions downstream may make, as a matter of policy, choices that prevent (2) and (3). Similarly, application/framework developers may decide, as a matter of policy, to rely on systemd-provided functionality.
Personally, I'm having a hard time seeing the decisions of third parties to use systemd features is somehow the fault of systemd's authors.
Posted Nov 18, 2019 12:32 UTC (Mon)
by pizza (subscriber, #46)
[Link]
whoops, I mistyped this; 'cron' functionality is part of the core. And that's a good thing. (Yay, no more long-runing cron tasks stepping on each other's toes, no more buggy dependency/prereq checking, no more inconsistent logging, etc..)
Posted Nov 26, 2019 12:29 UTC (Tue)
by immibis (subscriber, #105511)
[Link] (1 responses)
This is actually a problem, because it makes it harder to not install one of them than it should be. I've worked on embedded projects where we ended up installing all sorts of crap because nobody noticed it was being installed alongside something we actually needed. Most likely with systemd we would've ended up building and installing everything, then deleting the unit files for the bits we didn't want, and leaving the binaries where they are.
When you want a package you:
and you shouldn't have to
When it's a distribution, this effect is amplified because if the package maintainer doesn't do it, then none of the distribution's users can do it even if they want to. Basically, when you make something the default, you ensure that 98% of people are stuck with it. "Defaults are arguably the most important design decisions you'll ever make as a software developer": https://blog.codinghorror.com/the-power-of-defaults/
This is great for increasing your market share. You can truthfully say "Well 98% of our users use systemd-resolved" without considering that many of those users don't even know they're running systemd-resolved because the build script just assumed they would and installed it anyway. And then you can say "Since 98% of our users use systemd-resolved anyway, we won't run the unit tests with it disabled." or "Since 98% of our users use systemd-resolved anyway, we'll refactor the build system with the side effect of making it mandatory in the next version, and those who complain are just whiny boomers."
Also, did you look at the other comments indicating how difficult it is to replace that shared library? Not-systemd needs to replace the library with one that talks to not-systemd so that applications can use it. But presumably, all of systemd's components also use it as a utility library. So they won't run on not-systemd unless not-systemd also implements all of the utilities.
Posted Nov 26, 2019 13:01 UTC (Tue)
by pizza (subscriber, #46)
[Link]
Embedded developers too lazy (or to be blunt, incompetent) to handle tasks that basic cannot be trusted to exercise the considerable diligence to write classic-style init scripts of the quality/reliability needed for embedded work.
> Also, did you look at the other comments indicating how difficult it is to replace that shared library? Not-systemd needs to replace the library with one that talks to not-systemd so that applications can use it. But presumably, all of systemd's components also use it as a utility library. So they won't run on not-systemd unless not-systemd also implements all of the utilities.
So... you're saying that the libsytemd authors, by following well-established best practices with respect to code reuse, should have instead re-implemented helper/utility functions into each and every component, and required all third-party users to do the same?
(and for the record, systemd-as-pid1 is emphatically *not* a requirement for the use of libsystemd functions)
Posted Nov 18, 2019 13:54 UTC (Mon)
by smurf (subscriber, #17840)
[Link]
Feel free to ignore each and every one of them if you want to make your life more difficult than necessary. In the meantime I happen to be ecstatic about a cron replacement that finally doesn't fill my machine with 4718 hung jobs (by design, not because every cron job script author remembered how to correctly use "shlock"), a log system that finally doesn't lose data and can be queried without going insane, an ntp client that hasn't been designed for a machine that does nothing but NTP, and a "monopoly" that frankly doesn't exist.
You want a distribution without systemd? fine. Use one that didn't switch to systemd, or create your own. Then at least you can't blame somebody else for your broken system.
Or you might stop to ask whether there might be actual real-world usecases that require systemd to do logging or cron or networking or DNS – usecases that didn't work with the previous heap of independent services. It's not as if they sat down and decided to usurp /etc/network/interfaces (or whatever) from the blackness of their ugly little hearts, fer crissake!
NB: You might want to enlighten us WRT exactly which usecase you have in mind that's so incompatible with systemd that it requires special "catering".
Posted Nov 28, 2019 19:11 UTC (Thu)
by jrigg (guest, #30848)
[Link]
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Depending on your filesystem, one of the FS-specific tools might be able to replace the flag file. E.g., tune2fs -E force_fsck for ext4.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
And how many new Windows XP installs used Netscape?
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Wol
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
I miss an alternative to reimplement systemd from scratch letting to people fix that what are being criticizim these years, wich is a huge effort, but Debian can be a good place to grow and probably attract a lot of people to it.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
My understanding was that systemd was adopted widely because Gnome added it as a dependency.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
I prefer these graphs from the Debian Popularity Contest: Debian reconsiders init-system diversity
Debian - SysV vs. systemd popularity as total number installed
Debian - SysV vs. systemd popularity as a Percentage of Installs
SysV use in Debian at this point is a rounding error.
Debian reconsiders init-system diversity
Wol
Debian reconsiders init-system diversity
> No serious application depends on systemd because you'd be locking out 98-99% of your potential user base.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
The problem is not the lack of people willing to do the work, it's people actively refusing to merge ready, tested patches that don't fit within a maintainer's wishes. For example: a comprehensive set for patches for logind/policykit have been around since January 2018 (based on consolekit, later elogind), yet despite repeated requests there was no real response until just before Buster's freeze, when suddenly we received a demand for a thorough rework (so instead of co-installable libelogind it had to be made to reimplement full libsystemd ABI). Which then was rejected based on the change being too big and complex. In the result, Buster is mostly unusable in a GUI.
Actively blocking fixes
Actively blocking fixes
Actively blocking fixes
Actively blocking fixes
Actively blocking fixes
Actively blocking fixes
Actively blocking fixes
The only way to get this to work in some stable kind of way (that I can see, anyway) would be a stub init that checks which real init should run, re-links the libsystemd0 symlink to point to either libsystemd0.whatever-version.so or libelogind.some-other-version.so, and execs the real init. Good luck doing that if [/usr]/lib should happen to be readonly.
Actively blocking fixes
Actively blocking fixes
Actively blocking fixes
Actively blocking fixes
XFCE works well without systemd
I was surprised to learn that XFCE for my uses actually works better.
something related to logind and so I have to edit my network configuration by hand.
XFCE works well without systemd
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
No, it was not. If we're talking about initscripts here, then pretty much all early systems had some kinds of race conditions in them for example.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
doesn't understand why things were built the way they were
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
When the “Unix toolbox” was assembled in the mid-to-late 1970s, the guiding principle was more along the lines of “here's another neat program”, people didn't really worry too much about consistency, and there was no product manager, “benevolent dictator”, or written human-interface guide to ensure that command-line options that do the same thing are actually identical between different commands.
Debian reconsiders init-system diversity
I think this is an underappreciated part of what people who object to systemd don't like about it. You can argue all you like about the systemd project consisting of a bunch of separate tools that just happen to share a repository, but it is still being developed by a group that coordinates closely and has built its tools to cooperate closely. That's very different from the traditionally anarchic Unix approach, and importantly, that close coordination makes it more difficult for somebody to take just the parts they want and replace the ones they don't.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
There are still a lot of scripts requiring bash, but at least any script using /bin/sh should run equally well (except faster) with /bin/sh pointing to /bin/dash.
Debian reconsiders init-system diversity
but sometimes you need to do a bit of bending to get things to fit. An even better simile might be Geomags.
At first everything fits together nicely, but eventually you either end up in a situation where you cannot quite
achieve the angles you want without horrible kludges, or situations where your design collapses under its own weight.
whether I get help from a binary using "-?" "-h", "--help", "help", "--usage", "-u", "-usage", "-help", "/h" or whatever
else just scratches the surface of frustrating. Some people not even bothering to write manual pages for their software
(hi GNU!) makes things even worse.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Whut? Shellscripts are nothing but secure.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
15426932 /root/ip-route-list.txt
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
- dpkg/apt-get is mis-behaving because it can't replace implementations of libraries it links to?
Debian reconsiders init-system diversity
Here's how much code it takes to boot my desktop and server, with parallel startup and service dependency ordering:
174 total
~ $ ssh $server !!
327 total
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
* redirecting stdout+stderr to /dev/null
* the daemon or one of its libraries runs into a problem
* part/all of the problem/crash report is written to, you guessed it, standard error
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
They actually have a compelling argument here - qsort_r() in BSD has a different argument order, and they don't want to pick sides until it's standardized.
And musl doesn't want it either, so there's a deadlock.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
The most popular Linux platform in existence uses bionic libc, and busybox is probably the second one.
No, not quite. systemd is quite unusual in its enthusiastic embrace of glibc-specific features.
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
There was a giant ecosystem of random shell scripts and utilities and conventions and tools that parsed comments to discover ordering and a mess of problems that were solved and resolved by every package or every distribution independently (creating users, creating temporary directories, and so forth).
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
Debian reconsiders init-system diversity
systemd is an init-system? i wish.
systemd is an init-system? i wish.
> 2. are easy to disable - without breakage of other stuff running on the system
> 3. are reasonably easy to replace with other alternative/competing implementations.
systemd is an init-system? i wish.
systemd is an init-system? i wish.
* Download the package
* Build the package
* Install the package
* Review the package didn't automatically install a bunch of other packages that you now have to remove
* Delve into the package's build system to figure out how to disable those packages because it's not well documented
* Give up figuring out how to make it not build those packages, and just delete them after installing
systemd is an init-system? i wish.
systemd is an init-system? i wish.
Debian reconsiders init-system diversity