|
|
Subscribe / Log in / New account

The "Devuan" Debian fork and the fuss about systemd

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 1, 2014 16:30 UTC (Mon) by paulj (subscriber, #341)
In reply to: The "Devuan" Debian fork and the fuss about systemd by dlang
Parent article: The "Devuan" Debian fork

+1 to dlang's post on describing the systemd debate.

I happen to mostly like what systemd is doing wrt cleaning up init. Giving the init system a tight grip on processes through cgroups is a good thing. It was high time for a better process management API than the standard Unix one, and an init system that used it to avoid some of the nasty cases the traditional API allowed.

However, some of the initial claims of the systemd folks (person even, back then?) turned out to be naïve and somewhat self-serving. E.g, it was easy to predict that socket activation would not in fact be the answer to everything, and there were comments on LWN about this. Service files today often contain a number of hard-coded dependencies on other services, and on things that must be run before or after the running of the service itself. Indeed, systemd even has directories of symlinks to encode dependencies. So that's not quite that different to the previous world, of SysV init services, chkconfig, etc., which the systemd folks had decried.

Also, trying to support everything via declarative config files has led to quite a plethora of options (see systemd.{unit,service,directives}, and I'm starting to wonder if this is always really easier to understand than a shell script with a clean set of library functions available to it (e.g. see OpenWRT's init). (Note: this is *not* advocating for SysV init + traditional Unix process control).

I'm left wondering if a good amount of bath-water was thrown out, only to have to refill the bath with fresh, functionally identical, but different water.

Next, the systemd project appears to be on a mission to do more than just rewrite init. It appears to be on a mission to subsume all of the core Linux userspace, either by absorbing utilities or else by rewriting them. I'm sure the systemd project people view this as a good thing, that they view that the decoupled nature of that user-space held back progress. The systemd project also seems to value progress above compatibility (backward and platform). Finally, when they rewrite stuff, their self-confidence doesn't always match their ability or the (accumulated over time) abilities of those who wrote the existing implementations.

So, to summarise, for me:

- A modern init that can use modern APIs to fully control services, and provide better socket activation? Yay!

- The "overselling" of what socket activation could achieve: Hmm...

- Decrying hard-coded dependencies and sets of symlink directories, only to replace it with another set of the same but different: Hmmm...

- Replacing "complicated" init scripts with nearly as (if not more?) complicated declarative config files? Hmmmmm...

- Systemd becoming the katamari damacy of Linux user-space, the "One True Way"? Not like.


to post comments

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 2, 2014 22:47 UTC (Tue) by kleptog (subscriber, #1183) [Link] (40 responses)

> Next, the systemd project appears to be on a mission to do more than just rewrite init. It appears to be on a mission to subsume all of the core Linux userspace, either by absorbing utilities or else by rewriting them.

The thing is, a lot of these things needed work. Look at for example resolveconf. You have a number of different programs that can setup an internet connection and to decide which DNS to use you get scripts to dump files in a directory and then one is selected. I had non-working DNS for a while because some program had dumped a file there which overrode everything. Try tracking that down. This is so obviously a hack.

Now systemd has a daemon which does DNS by asking all the of the provided DNS servers, thus transparently merging all the DNS namespaces. If an interface goes down the DNS goes away, without relying on some fragile scripting. This is so obviously the correct solution I don't understand why people are complaining.

(This is actually a manifestation of the platform problem. When the given multiple DNS servers, since the glibc resolver works with resolv.conf, the answer chosen was to select one source to put in resolv.conf. Whereas the correct answer should have been, fix the resolver to handle multiple independent DNS servers).

Why do you have to run a complete NTP server daemon just to keep the system time accurate? Why would you still have cron, which can only mail output, when you have an init which support many more options? From my point of view systemd is finally providing correct solutions for problems that have been endlessly worked around.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 2, 2014 22:50 UTC (Tue) by dlang (guest, #313) [Link] (14 responses)

asking every DNS server for the answer and merging the results is very obviously NOT the correct answer if one of those is an internal server and one is an external server for the same name.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 2:33 UTC (Wed) by pizza (subscriber, #46) [Link] (13 responses)

> asking every DNS server for the answer and merging the results is very obviously NOT the correct answer if one of those is an internal server and one is an external server for the same name.

How is a resolver supposed to know what names are "internal" vs "external"?

How is the resolver supposed to tell if "somehost.internaldomain" or "somehost.externaldomain" is correct when the user asks to resolve "somehost"? Or heck, what if I type "somehost.internaldomain" and the request is sent to an external server that can't resolve it?

(This happens when I'm VPN'd into the office; I lose the ability to resolve my home network)

I'm sorry, asking all servers is a better solution, and you can use the domain search order for each network interface as a filter to figure out what servers' responses can be trusted -- but when you don't ask for a FQDN, you're going to have problems if the same hostname can be legally resolved in different namespaces.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 4:08 UTC (Wed) by raven667 (subscriber, #5198) [Link]

> you're going to have problems if the same hostname can be legally resolved in different namespaces.

Not necessarily, there could be a conflict resolution mechanism, for example pick the name that has an address with the most specific route entry, so if you have split tunneling with your office that response will probably be more specific than the one coming through the default route. That isn't always guaranteed so you will need another level to be a tie breaker but a few simple rules will get it right most of the time, and have an option for manual config if you need it.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 4:36 UTC (Wed) by mgb (guest, #3226) [Link] (11 responses)

> How is a resolver supposed to know what names are "internal" vs "external"?

Wrong question. The resolver should use the the hosts files, DNS servers, and NIS servers specified by the sysadmin. The resolver should not query random servers or merge records from all servers.

BIND views can help somewhat to prevent systemd from giving wrong answers but why waste effort working around a systemd design that just doesn't understand DNS?

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 5:51 UTC (Wed) by pizza (subscriber, #46) [Link] (10 responses)

> Wrong question. The resolver should use the the hosts files, DNS servers, and NIS servers specified by the sysadmin. The resolver should not query random servers or merge records from all servers.

Au contraire, that is absolutely the question, and you can't just handwave it away by regurgitating the status quo that demonstrably does the wrong thing in a dynamic network environment.

What if I'm in a coffee shop, and have two VPN tunnels up, one to my home systems and the other to the office? Which of the three completely legit sets of DNS servers (one set from the hotspot via DHCP, and one set from each of the VPN links) should the resolver use, given that using the wrong one will result in false NXDOMAIN results?

If you have a solution to this that doesn't require running a hand-tweaked DNS server on my laptop, I'm all ears -- besides, this half-baked solution won't work on many public hotspots due to restrictive firewall rules, and what happens if the VPN tunnel is down?

(And for the record, I have written DNS clients and servers, so I'm actually fairly clued in how DNS-the-protocol, DNS-the-system, and the glibc resolver works. You, on the other hand, appear to be lacking on all fronts)

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 7:07 UTC (Wed) by mgb (guest, #3226) [Link] (9 responses)

> Which of the three ... DNS servers ... should the resolver use ... ?

Whichever the sysadmin specified.

Clients on our VPNs use our DNS servers for security reasons. Your sysadmin may have different but equally valid requirements.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 7:23 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (8 responses)

I'm connected to a VPN and I have a media-center in my home network. Which DNS server should resolve its name?

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 8:26 UTC (Wed) by mgb (guest, #3226) [Link] (7 responses)

> I'm connected to a VPN and I have a media-center in my home network. Which DNS server should resolve its name?

If you were a client on one of our business VPNs the answer while you were connected would be that no DNS server should resolve the name of your media center for you.

Your sysadmin may have different requirements but it's unlikely he or she would want you to use possibly malicious DNS servers that systemd hears about from DHCP.

DNS server priority and VPNs (was: The "Devuan" Debian fork and the fuss about systemd)

Posted Dec 3, 2014 9:35 UTC (Wed) by cesarb (subscriber, #6266) [Link] (6 responses)

I don't know much about how VPNs are configured in the real world, and this seems like a good opportunity to learn. So far, I've only used "toy" VPNs with vtun, and I don't recall using alternate DNS servers with them.

How does a VPN server tell the client which DNS server to use? Is it DHCP over the virtual link? Is it IKEv2's configuration attributes? Is it PPP's negotiation? Something else?

In none of the options I just mentioned, AFAIK, there's a "override other DNS servers" flag (it wouldn't even make sense - what if the user were to connect to more than one VPN setting the same "override other DNS servers" flag?). So there must be some other mechanism by which the VPN server administrator tells the client to use the VPN's DNS servers and only the VPN's DNS servers. Do you know where I could read more about that mechanism?

Because without such a mechanism, the most logical response from the VPN client software would be to dump all learned DNS servers from all links in the /etc/resolv.conf file, where they would be queried either in random order or in sequential order. In the latter case, the ordering between "work" and "home" VPNs (pizza's and Cyberax's examples both have two VPNs) would depend on which one was connected to last.

DNS server priority and VPNs (was: The "Devuan" Debian fork and the fuss about systemd)

Posted Dec 3, 2014 9:58 UTC (Wed) by kleptog (subscriber, #1183) [Link] (5 responses)

Well, what used to happen is that the VPN servers/DHCP clients would dump all over the settings in resolv.conf and if you were lucky it would put them back after disconnect. The DNS settings are part of the setup packet, just like the routes and gateways. If you had multiple VPNs and/or DHCP your DNS would work about 50% of the time. Basically you had to configure it manually.

So instead a bit of glue called resolveconf was created and each program that wanted to setup DNS would dump its settings in a directory and another script would look at its priority list and select the one with the highest prio.

This mostly works, although it can in no way merge the results. You get one of the choices, but you have some control over the choice.

What happened in my case is that one of those files got created and had a high prio (NetworkManager is dead last in prio) but the DHCP client had gone. Hence broken DNS. Ofcourse, these files are stored in /var/run so a reboot would have fixed it but that's the lame solution. You can edit /etc/resolv.conf by hand, but it just gets overwritten again next time you start a VPN.

Once you have a daemon that knows the relationship between the DNS settings and the VPN server/DHCP client (like in systemd) it can make sure the settings are correct even if the server dies abnormally.

You could have done this by making resolv.conf smarter, but sysvinit doesn't provide a way to determine if something is still running. Note that NetworkManager has some smarts internally so if you're only using that it works better. But if you use stuff besides that you're left with the glory of resolveconf.

DNS server priority and VPNs (was: The "Devuan" Debian fork and the fuss about systemd)

Posted Dec 3, 2014 12:44 UTC (Wed) by mgb (guest, #3226) [Link] (4 responses)

> Once you have a daemon that knows the relationship between the DNS settings and the VPN server/DHCP client (like in systemd) it can make sure the settings are correct even if the server dies abnormally.

Except that systemd doesn't do that.

Poettering did propose an extension to handle the xhamster problem[1] but the existing Domains="..." option and the new Domains="*" syntax are the opposite of VPN DNS security. Systemd also lacks the dynamic reconfiguration which resolvconf provides.

Without a clean functional design systemd is forever adding more and more keywords to the systemd hairball and it's still not even close to implementing the functionality provided but existing trusted solutions.

[1] http://lists.freedesktop.org/archives/systemd-devel/2014-...

DNS server priority and VPNs (was: The "Devuan" Debian fork and the fuss about systemd)

Posted Dec 4, 2014 21:19 UTC (Thu) by paulj (subscriber, #341) [Link] (3 responses)

Yeah, the ever growing number of options for service/unit/etc. files are starting to make even the more complicated shell scripts in older init systems looksensible. :(

DNS server priority and VPNs (was: The "Devuan" Debian fork and the fuss about systemd)

Posted Dec 4, 2014 21:41 UTC (Thu) by anselm (subscriber, #2796) [Link] (2 responses)

At least the systemd options are documented in very visible man pages. Most of the clever hacks in the more complicated init shell scripts of the traditional setup aren't.

DNS server priority and VPNs (was: The "Devuan" Debian fork and the fuss about systemd)

Posted Dec 4, 2014 22:05 UTC (Thu) by mgb (guest, #3226) [Link] (1 responses)

> At least the systemd options are documented in very visible man pages. Most of the clever hacks in the more complicated init shell scripts of the traditional setup aren't.

To boot a traditional sysvinit server you just need to get the sequence numbers in order - something that your distro almost always gets right and you never need to change.

There's no more need for you to try to understand how shell scripts are written than there is for you to be able to disassemble ELF files in your head.

Systemd is much more complicated - and much more prone to random boot failure - because it is so much harder to express every one of the O(N^2) dependencies and it's not feasible to prove whether your dependency graph is complete.

DNS server priority and VPNs (was: The "Devuan" Debian fork and the fuss about systemd)

Posted Dec 5, 2014 0:07 UTC (Fri) by mchapman (subscriber, #66589) [Link]

> Systemd is much more complicated - and much more prone to random boot failure - because it is so much harder to express every one of the O(N^2) dependencies and it's not feasible to prove whether your dependency graph is complete.

Oh, puhlease. Very few units need inter-service dependencies (on my system, it appears to be about a fifth of them). Those that do are usually bundled together anyway (e.g. sshd.service wants sshd-keygen.service). The remaining dependencies are mostly along the lines of "I need the network" (and the service is not smart enough yet to work before the network has been brought up). Most dependencies are "wants" rather than "requires" anyway, which makes it very *hard* to break boot.

The sysadmin simply doesn't need to worry themselves about this stuff. A simple "systemctl list-dependencies multi-user.target" proves that systemd can solve this problem completely on its own.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 2, 2014 23:39 UTC (Tue) by cesarb (subscriber, #6266) [Link] (3 responses)

> Why do you have to run a complete NTP server daemon just to keep the system time accurate?

Because a complete NTP daemon disciplines the clock, so not only the time is correct, but also the frequency is correct, and because a complete NTP daemon has a selection algorithm which compares the time returned from all the servers and discards falsetickers, instead of trusting the first server which answers.

I will not use systemd-timesyncd on any of my systems until it becomes more than a dumb SNTP client. In the meantime, I'll use chrony or ntpd.

OTOH, systemd-resolved looks interesting, but can it work when the network is managed by NetworkManager instead of systemd-networkd? I don't think systemd-networkd can manage a laptop's wireless card as well as NetworkManager (i.e. choosing the SSID and entering the PSK from a GUI within KDE or Gnome), AFAIK it's more of a replacement for e.g. Debian's /etc/network/interfaces.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 10:15 UTC (Wed) by kleptog (subscriber, #1183) [Link]

> Because a complete NTP daemon disciplines the clock, so not only the time is correct, but also the frequency is correct, and because a complete NTP daemon has a selection algorithm which compares the time returned from all the servers and discards falsetickers, instead of trusting the first server which answers.

You do have a small point there, which is why I said "NTP server daemon". We just need a smarter client. I'm not sure if it's important enough for me though. What we need is something smarter than timesyncd but not as complicated as ntpd. Actually disciplining the clock is not hard, might fix that myself if it bugs me enough.

> I will not use systemd-timesyncd on any of my systems until it becomes more than a dumb SNTP client. In the meantime, I'll use chrony or ntpd.

Chrony looks interesting but appears to solve a different problem. It's still a server though.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 11:25 UTC (Wed) by anselm (subscriber, #2796) [Link] (1 responses)

Because a complete NTP daemon disciplines the clock, so not only the time is correct, but also the frequency is correct, and because a complete NTP daemon has a selection algorithm which compares the time returned from all the servers and discards falsetickers, instead of trusting the first server which answers.

Actually, systemd-timesyncd does discipline the clock. Right now it does talk only to one NTP server at a time, so it will work best syncing against a set of local time servers that run a full NTP server implementation.

I will not use systemd-timesyncd on any of my systems until it becomes more than a dumb SNTP client. In the meantime, I'll use chrony or ntpd.

And there is nothing wrong with that. In fact, systemd's documentation explains how to do this.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 13:29 UTC (Wed) by cesarb (subscriber, #6266) [Link]

> Actually, systemd-timesyncd does discipline the clock.

I took a quick peek at its source code, and it does not seem to use ADJ_FREQUENCY. Does it really adjust the clock's frequency?

> Right now it does talk only to one NTP server at a time, so it will work best syncing against a set of local time servers that run a full NTP server implementation.

That can work for a server or desktop usecase, but it will not work as well for a laptop or similar usecase, where a "local" time server might not exist at the moment.

And even on a controlled network, there's the risk that one server has gone insane; the recommendations I have seen is to have three or four NTP servers in your network, so a NTP client can detect and ignore the falseticker.

> And there is nothing wrong with that. In fact, systemd's documentation explains how to do this.

Does it even need documentation? Do a "systemctl disable systemd-timesyncd", a "systemctl enable chrony", and you're done.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 0:12 UTC (Wed) by mgb (guest, #3226) [Link] (17 responses)

> From my point of view systemd is finally providing correct solutions for problems that have been endlessly worked around.

From a sysadmin's point of view, systemd is providing *incorrect* answers - megabytes of config files with hundreds of poorly documented options, logging bottlenecks, incorrect DNS results, and time jumps.

What fraction of RHEL sysadmins have actually bought into the hype and deployed systemd in production servers? 1%? And how many of them are already regretting it?

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 7:14 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

I have a simple question for you: are you a bash script?

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 10, 2014 17:19 UTC (Wed) by nix (subscriber, #2304) [Link]

No, this is Serdar Argic, except someone's changed the trigger string to 'systemd' and altered the templates it sprays out when triggered.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 10, 2014 21:03 UTC (Wed) by mathstuf (subscriber, #69389) [Link]

Seeing as he pretty much evaporated since the discussion about him offering nothing new to the threads he comments in, the indication that he's basically got no standing from his declarations that he never has nor will try systemd personally, and the lack of a denial here, I wonder if a "trap" condition got triggered in his source code.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 11:11 UTC (Wed) by anselm (subscriber, #2796) [Link]

From a sysadmin's point of view, systemd is providing *incorrect* answers - megabytes of config files with hundreds of poorly documented options, logging bottlenecks, incorrect DNS results, and time jumps.

You must be talking about a different systemd than what most people are using. Most of the points you raise have been debunked over and over again here and elsewhere.

As far as systemd's time synchronisation service is concerned, there are two points worth noting:

  • systemd-timesyncd supports SNTP, which is a simplified version of NTP. This means that you should use it in a context where you have a set of local time servers that run a full-blown NTP server implementation, and you don't want to run an NTP server on every single machine. If you distribute the names of your NTP servers by DHCP, systemd-timesyncd does not require any local configuration at all.
  • Systemd makes it easy to deploy a different NTP implementation such as chrony or ntpd, which are safer on standalone machines without a local time server elsewhere on the LAN. This is documented in systemd's timedatectl(1) man page.

What fraction of RHEL sysadmins have actually bought into the hype and deployed systemd in production servers?

Those who have upgraded to RHEL 7, presumably.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 13:13 UTC (Wed) by cesarb (subscriber, #6266) [Link]

> [...] and time jumps.

In case you are referring to my complaint about systemd-timesyncd, you're wrong.

Like any decent NTP client, systemd-timesyncd *slews* the clock. The Linux kernel can be told "slew the clock by 10ms" and it will gradually shift the clock until it matches, without any jumps.

And yeah, if the difference is too large, every NTP client will step (i.e. jump) the clock. That includes chrony and ntpd.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 20:54 UTC (Thu) by sjj (guest, #2020) [Link] (11 responses)

Huh, we don't have megabytes of config files here:
$ cat /etc/redhat-release ; du -sh /usr/lib/systemd/system
CentOS Linux release 7.0.1406 (Core)
748K /usr/lib/systemd/system

What system has megabytes?

And poorly documented? How about apropos systemd; man systemd.unit; man systemd.service. I *wish* other parts of the system were so "poorly" documented!

As my first foray into systemd, I needed to run a special configuration script for openvpn server before it starts. Steps: copy service file from /usr/lib to /etc, add ExecStartPre=, and... done. Extra nice that since the config files are .ini format, and ansible knows how to edit those natively in-place, this is now automatic.

Sure, I could hack that into the openvpn startup script, or put in a new init.d script to run before openvpn, but that's brittle. And now I have a pattern - I can use the exact same pattern for other services instead of a service-specific hack (have you ever read an HP or Oracle etc "Enterprise" init script???).

Consistency and easy automation of changes in service initialization? This is a win-win-win for me as a sysadmin.

Please provide an example (preferably with reproducible test case or perfomance numbers) of logging bottleneck - I definitely would like to see if this could be problem.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 21:06 UTC (Thu) by dlang (guest, #313) [Link] (4 responses)

> Please provide an example (preferably with reproducible test case or perfomance numbers) of logging bottleneck - I definitely would like to see if this could be problem.

this again?

you want a test case, ok here's a simple test

take any largeish text file

run logger -f against that file and see how many messages per second you can process with systemd delivering the messages to syslog and going directly to syslog

to eliminate other possible distractions, I would suggest simplifying your syslog config to just write all logs to a single file. If you are using rsyslog, I would suggest using a template that has %timegenerated% in it so that you get the timestamp of when rsyslog sees the message as opposed to the timestamp that the journal puts on it.

While you do this, look at the time/cpu taken by logger to deliver the messages, and also the time/cpu used by the journal and syslog daemons to process the messages

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 22:49 UTC (Thu) by sjj (guest, #2020) [Link] (3 responses)

This was an honest question. Since you've been banging this drum for weeks here, I would expect you to have some examples. Care to share?

I don't have time to setup much testing right now (maybe next week) - a quick test in a CentOS7 VM (KVM under Ubuntu latest on an MBAir) gives me about 12000 msgs/sec with /var/log on either tmpfs or xfs-on-KVM-on-ext4-on-SSD (this makes me wonder about the test setup...)

If a dinky little VM can do 12000 msg/sec, I'm fairly confident this isn't going to spoil my day. Even when I rebuild the central log server, but I'll be looking at the options at that time. Anybody have comparison numbers between stock RHEL/CentOS 6 vs 7 logging performance?

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 8, 2014 19:21 UTC (Mon) by dlang (guest, #313) [Link] (2 responses)

a moderate VM should be able to do at least 10x the log volume that you are reporting (a couple hundred thousands of messages/sec), so I don't know what's going on with your test.

I don't have any systemd based systems to run tests on, so I can't give you a direct comparison from my own tests.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 9, 2014 2:57 UTC (Tue) by mathstuf (subscriber, #69389) [Link] (1 responses)

Just out of curiosity, what application(s) are pumping out that volume of data? What do you do with a fire hose of something like 10 megs a second (assuming 100 bytes per message)? What tools do you use to analyze the logs? Do you have a dedicated network interface for the logs? Eating up 10% of your gigabit bandwidth doesn't sound like the best use of it or the CPU work involved. Assuming that you have users which are generating the data interactively (and not batch jobs).

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 9, 2014 3:25 UTC (Tue) by dlang (guest, #313) [Link]

I normally only see that sort of volume on my log aggregation boxes, not on individual boxes generating the logs. But you would be surprised at how much traffic a firewall or webserver can generate at a company that has multiple gigabit Internet connections. Yes, Gig-E throughput is a limiting factor on some of my systems (a limit I haven't hit outside of testing, yet. But it's in sight)

As far as best use of the CPU goes, it's better spent getting the logs off the box so that other systems can analyze them than trying to do all that analysis locally (not to mention that some things require analysis on logs across systems)

> Assuming that you have users which are generating the data interactively (and not batch jobs).

when you have 10's of millions of users using your services interactively, you can generate a LOT of log data.

for what it's worth, the reported rsyslog record is 1 million logs/sec processed by a single VM (on a 10G network with a good disk subsystem, but I think it was a 2-core VM)

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 22:45 UTC (Thu) by rodgerd (guest, #58896) [Link] (2 responses)

> copy service file from /usr/lib to /etc, add ExecStartPre=, and... done.

Even better: source it and add the ExecStartPre= and not worry that upgrades will skew the source and your copy.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 23:08 UTC (Thu) by cortana (subscriber, #24596) [Link] (1 responses)

Oblig. even better again... create /etc/systemd/foo.service.d/local.conf in which you can put _just_ the ExecStartPre= line. :)

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 3:09 UTC (Fri) by sjj (guest, #2020) [Link]

D'oh, of course, nice. Thanks!

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 0:00 UTC (Fri) by mchapman (subscriber, #66589) [Link] (2 responses)

> $ cat /etc/redhat-release ; du -sh /usr/lib/systemd/system
CentOS Linux release 7.0.1406 (Core)
748K /usr/lib/systemd/system

And if you stick an --apparent-size in there as well, so you get the size of the content only, it drops by a factor of five.

That's what you get when you have lots of very small files. I'd much rather have that than a few large ones.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 3:13 UTC (Fri) by sjj (guest, #2020) [Link] (1 responses)

Thanks for the clarification. I had this small insistent voice at the back of my head during the drive home that something still wasn't right since this was only 200+ files.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 10:22 UTC (Fri) by cortana (subscriber, #24596) [Link]

You can make it even more accurate with something like:

find /lib/systemd/system /etc/systemd/system -type f -exec du --apparent-size -hc {} + | sort -h

Which comes to 107k on my system.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 9:16 UTC (Wed) by paulj (subscriber, #341) [Link] (2 responses)

I wouldn't dispute that things could be improved. I am not saying there should be no progress.

The point is purely about how systemd has a goal of moving core Linux user-space into a single project. That doesn't make me feel good. I'll be honest, for me that's not so much about the single project aspect, but more about the benevolent dictator in charge of it. I fear his attitude on compatibility and bugs is likely to lead to even more pain for me than it already has (on other, less important projects).

I just don't like the idea of core Linux user-space becoming tightly coupled with lots of interdependencies, and requiring very specific kernel versions that mean you need flag days for upgrades (that aren't documented very well!) where you can't go back to the previous kernel.

Then there's the tendency to re-implement stuff. Or sometimes base those re-implementations on new crypto schemes that havn't been published, never mind peer-reviewed (but hey, it's his brother, so it must be ok!). That kind of thing doesn't fill me with confidence.

Nor does the kind of coding methodology the project is happy to accept for highly-security sensitive parsing of network input inspire confidence. That's one remote-exploit CVE already. The code concerned, even after that CVE, will not leave anyone with confidence in that projects code review practices on any highly sensitive code.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 9:50 UTC (Wed) by niner (subscriber, #26151) [Link]

" I'll be honest, for me that's not so much about the single project aspect, but more about the benevolent dictator in charge of it."

What makes you think that there's a dictator in the systemd project? Looking at http://cgit.freedesktop.org/systemd/systemd/log/ I find at least 6 differenct committers in the past few days. The highest number of commits was by Zbigniew Jędrzejewski-Szmek, who to the best of my knowledge has never been mentioned in any of these systemd discussions.

"and requiring very specific kernel versions that mean you need flag days for upgrades (that aren't documented very well!) where you can't go back to the previous kernel."

Not very specific kernel versions. Just new enough kernels. And you don't need flag days, you just have to make sure, you have the newer kernel before the systemd version that depends on this kernel. Really just like with any other software and its dependencies.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 13:37 UTC (Wed) by johannbg (guest, #65743) [Link]

" I'll be honest, for me that's not so much about the single project aspect, but more about the benevolent dictator in charge of it. "

Lennart is not an benevolent dictator of the project, he is however an lead developer along with Kay, Tom, Zbyszek and others and there are certain areas he is better qualified to make "judgment calls" about just like Kay, Tom, Zbyszek and others are as well.

I however dont understand where this perception is coming from since things are being openly discussed on the mailinglist, hackfests etc. and atleast up to this point the most technical sound solution always has prevailed regardless if Lennart has been with it to begin with or against it.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 17:30 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (15 responses)

> Service files today often contain a number of hard-coded dependencies on other services, and on things that must be run before or after the running of the service itself.
Yes, just like any other init system. Dependencies are actually NOT hardcoded - you can override them (and only them!) using custom units in /etc.

>Indeed, systemd even has directories of symlinks to encode dependencies
Incorrect. Systemd does not use symlinks to encode dependencies in any meaningful way.

> - Decrying hard-coded dependencies and sets of symlink directories, only to replace it with another set of the same but different: Hmmm...
BS.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 3, 2014 23:56 UTC (Wed) by mchapman (subscriber, #66589) [Link] (14 responses)

> Incorrect. Systemd does not use symlinks to encode dependencies in any meaningful way.

I think you'll find the wants.d/ and requires.d/ directories that "systemctl enable/disable" manages use symlinks to represent dependencies. Each symlink is exactly equivalent to a Wants or Requires directive in the corresponding unit's configuration file.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 3:17 UTC (Thu) by raven667 (subscriber, #5198) [Link] (2 responses)

I'm looking at a system and the only thing I see are target.wants directories which correspond to which services are enabled at each run target, I don't see any symlinks encoding what services depend on each other, service dependencies are dynamically generated at runtime as far as I know.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 6:21 UTC (Thu) by mchapman (subscriber, #66589) [Link] (1 responses)

You certainly can have some.service.wants/ directories. They work exactly the same as some.target.wants/ directories.

In other words, just as having a "multi-user.target.wants/foo.service" symlink means "in order to start multi-user.target, foo.service must also be started", you can also have a "foo.service.wants/bar.service" symlink to mean "in order to start foo.service, bar.service must also be started".

There's a few reasons why you're probably not seeing this on your system.

First, most service-to-service dependencies are static, so there's no need for the dynamism that symlinks provides. The author of the unit file just defines the dependencies when they're writing it. Using symlinks for dependencies in the {/usr,}/lib/systemd directory is discouraged.

Second, symlinks can only be used to define Wants and Requires directives. When you have service-to-service dependencies, you often want After as well.

Finally, the symlinks managed by "systemctl enable/disable" ultimately come from the [Install] sections in unit files, and most of the time "enabling" a unit only involves having it depended upon by a target.

For these reasons, it is quite likely you only have .wants/ and .requires/ directories for target units.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 16:56 UTC (Thu) by raven667 (subscriber, #5198) [Link]

Right but the original statement which you were disagreeing with was

> Systemd does not use symlinks to encode dependencies in any meaningful way.

Which I think you are now agreeing with, the inter-service dependancies are not statically defined by symlinks by the admin before the system boots (like with S## and K## scripts), service dependancies are calculated at runtime by reading config files for explicit Wants/Requires/After statements, socket activation, udev and dbus events, etc. in an attempt to reach a particular target which is statically defined by the admin using symlinks before the system boots. While you can define Wants and Requires using symlinks as well that is not a common case as my running system doesn't have any service dependancies defined that way.

Maybe we are just talking around each other and don't actually disagree?

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 18:54 UTC (Thu) by paulj (subscriber, #341) [Link] (10 responses)

and chkconfig used to read init scripts to determine which SysV init runlevel links to create for services.

I don't have a problem with this.

I'm just noting we have ended up with somewhat the same but different means of hard-coded services dependencies. Somewhat counter to some of the arguments as to why systemd was so necessary in earlier days.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 21:36 UTC (Thu) by anselm (subscriber, #2796) [Link] (9 responses)

With sysvinit, there are either no dependencies at all (and the distributor and/or local admin must figure out a working order when they're setting up the rc?.d symlinks by hand) or else the dependencies are encoded in the LSB headers of the various init scripts, where they are a hassle to change – especially in the face of upgrades by the distribution. The dependencies you can express are also quite limited.

Systemd to a large degree gets by without explicit dependencies, and it offers a mechanism that allows a local administrator to override those explicit dependencies that do exist, without having to tweak any files that are provided by the distribution. This makes updates a lot easier. It also supports command line tools to enact changes, which is useful for shell scripts (sysvinit requires the administrator to actually edit the init scripts, which is considerably less convenient).

From a system maintenance POV, systemd is way ahead of sysvinit in this respect.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 22:00 UTC (Thu) by mgb (guest, #3226) [Link] (8 responses)

> From a system maintenance POV, systemd is way ahead of sysvinit in this respect.

Occasional random boot failures due to O(N^2) missing dependencies?

No thanks.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 22:08 UTC (Thu) by anselm (subscriber, #2796) [Link] (6 responses)

Hasn't happened to me yet, on a variety of systems. Systemd happens to handle that sort of thing really well as far as I'm concerned – certainly a lot better than the traditional setup, which has no notion of a successfully started service in the first place. (It can see when an init script exits, but will then blithely continue trying to start stuff that depends on that service regardless of what happened with the init script, not to mention the service itself.)

In the absence of concrete evidence to the contrary I'll consider your claim FUD.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 22:14 UTC (Thu) by dlang (guest, #313) [Link] (5 responses)

so, rephrasing it "I haven't seen that problem so I don't believe it exists"

haven't we just seen the systemd opponents accused of holding this belief? (when they don't question that the problems exist for some people, they just disagree on it being enough to require it to be a default)

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 4, 2014 23:36 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (4 responses)

No, because here, anselm is asking for proof-positive of a claimed issue. The responses you get for your claims are for the non-existence of issues you don't see (and others, presumably, have had personal experience with such issues, so they don't believe that you have evidence to say they're wrong). Now, you can argue how *important* the issues are, but that's a different discussion.

Here, I basically discount anything mgb claims without evidence because there hasn't been much to indicate that he has actually tried systemd and instead just states the same FUD over and over.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 11:59 UTC (Fri) by mpr22 (subscriber, #60784) [Link] (3 responses)

there hasn't been much to indicate that he has actually tried systemd

Even better, mgb has in fact clearly indicated in comments on this very site that they have no intention of ever running systemd under any circumstances.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 15:47 UTC (Fri) by raven667 (subscriber, #5198) [Link] (2 responses)

> Even better, mgb has in fact clearly indicated in comments on this very site that they have no intention of ever running systemd under any circumstances.

That's so weird, maybe not the most effective way to evaluate technology choices, and more similar to Ford vs. Chevy fanboisim or sports team loyalty, or religion.

Also, the Chicago Bears still suck. 8-)

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 17:30 UTC (Fri) by anselm (subscriber, #2796) [Link] (1 responses)

The real danger for somebody like mgb is that they become so invested in their crusade against systemd that they end up literally afraid to actually try systemd, or to learn anything factual about it at all, because they might find out that it is really not that bad, and the cognitive dissonance would be too much to bear. It is a lot less dangerous (to their own sanity, anyway) to keep repeating the FUD with increased zeal, and so they just continue doing that instead.

This type of psychology is otherwise often seen, for example, in many young-Earth creationists, who will actively resist looking at evidence for evolution because they are scared that it might shake their faith in the literal truth of the biblical creation myth that is central to their religion.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 17:35 UTC (Fri) by raven667 (subscriber, #5198) [Link]

I suppose what you describe is a fundamentally very weak faith, or ego, if it needs to be protected so strongly.

The "Devuan" Debian fork and the fuss about systemd

Posted Dec 5, 2014 20:56 UTC (Fri) by flussence (guest, #85566) [Link]

That, somewhat ironically, describes most people's experience with OpenRC in parallel mode.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds