Another daemon for managing control groups [LWN.net]

Another daemon for managing control groups

Posted Dec 5, 2013 1:55 UTC (Thu) by Baylink (guest, #755)
Parent article: Another daemon for managing control groups

Am I really the *only* person who hasn't drunk the koolaid?

Does not lumping *everything your Linux system does* inside the opaque ball of systemd violate nearly 35 years of Unix Philosophy pretty violently?

Do one thing. Do it well.

WWBKKRS&SD?

to post comments

Another daemon for managing control groups

Posted Dec 5, 2013 3:46 UTC (Thu) by neilbrown (subscriber, #359) [Link] (42 responses)

Next you'll be telling us that we should re-write Linux as a micro-kernel with a herd of daemons, 'cause as it is Linux does *lots* of things.

You might have noticed that Unix has faded. Maybe its philosophy is wasn't sufficient.

The Linux philosophy (i.e. what Linus regularly rants of favour of) is pragmatism: Do what works.
Plus competitive evolution: Do several conflicting and incompatible things and may the best code win.

"Do one thing and do it well" certainly has its place, but that place isn't everywhere.

Another daemon for managing control groups

Posted Dec 5, 2013 5:02 UTC (Thu) by luto (subscriber, #39314) [Link] (25 responses)

Except that you won't even be able to try the competing solution on a Red Hat, Fedora, or other systemd-using distribution.

At the risk of harping on something I've said before: there needs to be a way to tell systemd to stay the $*!& away from the cgroup hierarchy.

If this can't be done today, let's fix it.

Another daemon for managing control groups

Posted Dec 5, 2013 8:07 UTC (Thu) by ovitters (guest, #27950) [Link]

Seems like you like to tell other people what to do. Maybe try convincing them. Because with telling, you're very likely going to be ignored. And no, you cannot tell people what to do and expect them to actually listen to you. "Fixing that" is not possible, nor wanted.

Another daemon for managing control groups

Posted Dec 5, 2013 13:29 UTC (Thu) by mezcalero (subscriber, #45103) [Link] (23 responses)

You can also turn this around: You won't be able to try the systemd solution on systems using that new Ubuntu cgroup manager either. There can only be one cgroup manager, and you have to choose between systemd or the Ubuntu one. You cannot use both. And that's not because systemd was evil, that's because the kernel iface is the way it is and the only secure way to allow multiple managers live side by side in userspace fiddling around in the same tree...

Another daemon for managing control groups

Posted Dec 7, 2013 0:43 UTC (Sat) by isilmendil (guest, #80522) [Link] (22 responses)

Actually, turning this around is a good excercise:
You can't use the systemd solution on a Ubuntu system obviously (since systemd would also conflict with upstart and maybe udev), but you can swap the cgroup management for a (not currently existing) different solution, and you won't be forced to remove upstart or any other program.

I'm not saying that systemd is evil -- it just forces an all or nothing approach. Essentially it's a new Linux stack: additionally to GNU/Linux and Android/Linux we now have GNU/systemd/Linux...

Another daemon for managing control groups

Posted Dec 8, 2013 3:55 UTC (Sun) by cas (guest, #52554) [Link] (21 responses)

this "new stack", GNU/systemd/Linux is not like unix, and not like linux - it is the complete opposite of the traditional small-tools, do one job and only one job but do it extremely well approach. it's a jack-of-all-trades master-of-none that tries to do too much, tries to do everything, in fact.

call it what it is, Lennartix, or Lennix. It's certainly not Linux. or Unix.

Another daemon for managing control groups

Posted Dec 8, 2013 5:19 UTC (Sun) by neilbrown (subscriber, #359) [Link] (19 responses)

"Linux" is just a kernel, so stuff in user-space certainly isn't "Linux".
It also certainly isn't "Unix" as that is fairly dead. It met needs in the 60s and 70s and even to some extent the 80s quite well. But the needs we have today are very different.
And it clearly isn't "GNU/Linux" as that was largely an attempt to re-implement UNIX is free software, and add more options. So it maybe carried us to the 90s.
I think calling it "Lennartix" gives way too much credit (or otherwise) to Lennart. He certainly instigated systemd and a few other bits, but the whole userspace that Fedora or openSUSE enjoy today is much more than any one person's innovation. Trying to name the whole after any one person is a mistake.

One of the big weaknesses of the "do one job and do it well" approach is that those individual tools didn't really combine very well. sort, join, cut, paste, cat, grep, comm etc make a nice set of tools for simple text-database work, but they all have slightly different ways of identifying and selecting fields and sort orders etc. You can sort-of stick them together with pipes and shell scripts, but it is rather messy and always error prone.

I remember being severly disillusioned by this in my early days. I read some article that explained how a "spell" program can be written to report the spelling errors in a file. It uses 'tr' to split into words, then "sort" and "uniq" to get a word list, then "comm" to find the differences. "cool" I thought. Then I looked at the actual "spell" program on my university's Unix installation. It used a special 'dcomm' (or something like that) which knew about "dictionary ordering" (Which ignores case - sometimes). Suddenly the whole illusion came shattering down. Lots of separate tools only do 90% of the work. To do really complete work, you need real purpose-built tools. "do one thing and do it well" is good for prototypes, not for final products.

One thing Unix never gave us was a clear big picture. It was always lots of bits that could mostly be stuck together to mostly work. I spent a good many years as a Unix sysadmin at a University and I got to see a lot of the rough edges and paper over some of them.

Systemd provides us with a clear big picture for one fairly important part of the sysadmin story: daemon process control. It may not be a perfect picture, but it is a whole lot better than no real picture at all. Back when I was being a sysadmin I would have loved something that would give me real control over all the various deamons: status, control, auto-restart, dependency etc. I even wrote something. It was nowhere near as ambitious as systemd but as far as it went, it was designed along very similar lines.

There certainly may be technical details about systemd that can be honestly debated, and there may be social/community issues that not everyone feels comfortable with (but then ambitious people often come across as intimidating), but to say that systemd is bad because it is not the "Unix way" is to simply have your head in the sand. The core idea behind systemd of unifying system service management into a single infrastructure is something that Unix has badly needed for years. It's about time we got it.
The thing that annoys me most about systemd is that I didn't write it first!

Another daemon for managing control groups

Posted Dec 8, 2013 5:59 UTC (Sun) by cas (guest, #52554) [Link] (18 responses)

> "Linux" is just a kernel, so stuff in user-space certainly isn't "Linux".
> It also certainly isn't "Unix" as that is fairly dead

wow, thanks for that startling revelation. Next you'll be telling me that water is wet, or that nobody ever actually drinks the kool-aid because that's a specific brand-name that's only available in certain countries(*)

Followed, no doubt, by the conclusion that nobody wants to (or should ever want to) combine water + flavouring agent because that's too messy and inconvenient and far too difficult so it's easier to just buy a can of some fizzy drink.

pure pedantic-literalist genius!

(*) US only? i dunno. i've never seen a kool-aid brand of anything, ever. i'm not even entirely sure of what it is - some kind of cordial or powdered flavour concentrate, i believe. but i know what someone means when they use a phrase like "drink the kool-aid". hint: it's not a literal statement about imbibing a flavoured liquid.

> One of the big weaknesses of the "do one job and do it well" approach
> is that those individual tools didn't really combine very well. sort,
> join, cut, paste, cat, grep, comm etc
> [ ... blah blah blah, i don't understand it, it's too hard ... ]

thank you for demonstrating that you don't understand unix.

that's the trouble with systemd advocates - they don't understand what it is that they're pushing to replace and, worse, universally think that their ignorance is a compelling argument for dismissing any relevant concerns by those who do know how the unix small-tools modular approach works.

Another daemon for managing control groups

Posted Dec 8, 2013 12:38 UTC (Sun) by pizza (subscriber, #46) [Link]

> that's the trouble with systemd advocates - they don't understand what it is that they're pushing to replace and, worse, universally think that their ignorance is a compelling argument for dismissing any relevant concerns by those who do know how the unix small-tools modular approach works.

BTW, That argument turns out to work quite well in reverse, as the systemd detractors (as a whole) spend very little time discussing the technical [de]merits of systemd (usually rather incorrectly) and prefer to just bash Lennart instead.

Another daemon for managing control groups

Posted Dec 8, 2013 23:19 UTC (Sun) by mathstuf (subscriber, #69389) [Link]

> hint: it's not a literal statement about imbibing a flavoured liquid.

While the phrase doesn't mean it literally, it is a reference to an actual event where Kool-Aid was used[1] for one of those murder-suicide cult things.

[1]https://en.wikipedia.org/wiki/Drinking_the_Kool-Aid#Jones...

Another daemon for managing control groups

Posted Dec 9, 2013 1:39 UTC (Mon) by nix (subscriber, #2304) [Link]

thank you for demonstrating that you don't understand unix.

Oh, thank you. Unintentional funny of the week. Neil Brown doesn't understand Unix?! Who are you going to accuse of not understanding Unix next, Roland McGrath? (GNU Make has far too many features and so does glibc, they are not in the Unix spirit!). Rob Pike? Ken Thompson?

Another daemon for managing control groups

Posted Jan 2, 2014 11:29 UTC (Thu) by Rudd-O (guest, #61155) [Link] (14 responses)

You are talking to Neil Brown. Where is your respect? You look like an idiot accusing him of "not understanding UNIX". Apologize.

Another daemon for managing control groups

Posted Jan 3, 2014 23:02 UTC (Fri) by cas (guest, #52554) [Link] (13 responses)

I really don't care who he is. what people say and do is more important than what their name is.

anyone who says that sort, grep, tr, comm etc 'don't combine together well' and '"do one thing and do it well" is good for prototypes, not for final products' does not understand unix. read his post - the entire thing was "waah! unix toosl are too hard and complicated, systemd saves us from that!".

perhaps he was just indulging in some exaggeration - it's a popular systemd pusher's tactic to exaggerate the alleged 'difficulty' or 'inadeqacy'of all previous/competing methods so that systemd's meager benefits seem greater by comparison. in this particular case, he has to grossly exaggerate the difficulty and inadequacy and just plain awfulness of the small-tools approach.

the truth is that systemd just isn't good enough to be worth the price of loss of modularity and replacability (nothing could be good enough to be worth that price), especially when the minor benefits that systemd does provide could easily be provided without a monolithic mono-culture monstrosity absorbing every low-level userland function in sight. init systems really don't have to be that complicated, they don't have to absorb syslog and cron and automount and everything else.

Another daemon for managing control groups

Posted Jan 3, 2014 23:47 UTC (Fri) by dlang (guest, #313) [Link] (3 responses)

ahh, but they aren't trying to just create a new init system. They are defining a new standard linux userspace. That is enough to justify anything.

I just wish they would do this on their own distro.

Another daemon for managing control groups

Posted Jan 4, 2014 2:27 UTC (Sat) by raven667 (guest, #5198) [Link]

Well this helps standardize service startup and login session management, the CLI and GUI shell userspace is not affected in any way

Another daemon for managing control groups

Posted Jan 25, 2014 20:47 UTC (Sat) by Baylink (guest, #755) [Link] (1 responses)

That's the part that I really don't understand.

I can get that Lennart doesn't understand the Unix Philosophy, or why it's more than "just a philosophy"; I can get that he doesn't care about why binary log files are bad, or why not being able to see *which* component of the core system went runaway on ps and kill just it is bad, or why being unable to replace those components with the specific ones you need for your job is bad. Or why optimizing for something that the vast majority of non-developer users of Linux don't care about -- booting at ludicrous speed -- is bad.

Why, in short: *requiring me to UTSL* just to do admin work, is bad.

What I *cannot* understand is how *entire committees* of people who drive the designs of distributions drank the Flavor Aid.

It is effectively impossible now to run sysVinit, because the people who package things for dnew Tweets istributions *no longer include, much less support* the components necessary to do that. And unless you're google-scale (in which case you're probably building your distro from scratch anyway, with a dedicated department of staffers), you can't fix that.

And that's not to mention the *literally decades* of sysadmin reflexes that have been flushed down the drain by this decision on the part of distro managers -- that's the part that really frosts my ass. I have better things to do than to relearn the basics all over again, guys, really.

I got into Unix because I am *lazy*. I want to learn small things once, and leverage them into big things, to make my life easier.

No matter how much better it might be at the actual work, as a sysadmin, systemd continues to make my life harder and more frustrating, at every single turn.

My philosophy of life is that our morale is proportional not to the size of our wins and losses, but to their *number*; that's why I use Linux.

systemd just made it go the other way.

Thanks, guys.

Another daemon for managing control groups

Posted Jan 27, 2014 15:07 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

If you're so lazy, why are you even trying to follow everyone else? Is sticking with Slackware not the laziest thing one can do these days anymore?</snark>

Maybe others aren't so lazy to not see the benefits it brings to the table and that's why it has been adopted? Making my user setup use unit files took maybe a day or two (and that includes a *lot* of unit files; at least 30) and I just got hung up on needing suid to start X. Systems does bring a lot to the table. Have you even tried it or did "service" not working prevent you from digging deeper?

Anyways, an anecdote from dealing with FreeBSD this weekend: I tried getting a taskd server running, but unfortunately I get no indication that it has failed to start, logs as to why it might have happened, or what the init system even attempted to do. And BSD init scripts are easier to deal with than sysvinit. I really would rather have systemd with all the tools it gives me to deal with such problems (so I know whether I should go poke upstream).

Another daemon for managing control groups

Posted Jan 4, 2014 2:12 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

>anyone who says that sort, grep, tr, comm etc 'don't combine together well' and '"do one thing and do it well" is good for prototypes, not for final products' does not understand unix. read his post - the entire thing was "waah! unix toosl are too hard and complicated, systemd saves us from that!".

Frankly, I'm sick of total ignoramuses who repeat this very argument.

Get the freaking systemd and see what it can do. Now try to replicate it with "do no thing well" tools like SysV Init.

Another daemon for managing control groups

Posted Jan 4, 2014 3:19 UTC (Sat) by neilbrown (subscriber, #359) [Link]

> I really don't care who he is. what people say and do is more important than what their name is.

Well said!

As for the rest of your post, I guess we'll have to agree to disagree.

(and no, I didn't mean to exaggerate. I love Unix tools and use them all the time, but I also believe in choosing the right tool for the job)

Another daemon for managing control groups

Posted Jan 4, 2014 14:50 UTC (Sat) by anselm (subscriber, #2796) [Link] (6 responses)

the truth is that systemd just isn't good enough to be worth the price of loss of modularity and replacability (nothing could be good enough to be worth that price)

Any »modularity« in System-V init is there purely by accident. Actual »modularity« would imply a certain amount of unity of design and basic concepts, and communication between the components by means of well-defined and documented interfaces. None of these apply to System-V init and its associated tools such as inetd, cron, etc.

It's not as if System-V init had been consciously designed the way it is. It is a hodgepodge of stuff from different sources that were developed separately from each other, don't interact well (if at all) and are each configured completely differently from the rest. If there is a long-overdue opportunity to replace this utter mess by something that has actually been properly thought out as a complete system, works a lot better in real life, and is easier to understand, develop for, maintain, teach, and learn, then I'm all for it.

Another daemon for managing control groups

Posted Jan 4, 2014 17:56 UTC (Sat) by raven667 (guest, #5198) [Link]

In some ways this is like Plan9 again, a re-thinking of the UNIX concepts with an eye toward completeness and consistency with a lot of operational experience behind it that didn't exist when many of these systems were first developed.

Another daemon for managing control groups

Posted Jan 25, 2014 20:50 UTC (Sat) by Baylink (guest, #755) [Link] (4 responses)

No, the modularity comes from *the pieces not all being one big piece with no visibility or lines of separation*.

Yes, the design of sysVinit is a *bit* rough. But it works, and it's easy to understand if you know basic Unix, and you don't have to read thousands of lines of source to figure out what it's doing.

And if one part of it falls over, ps will tell you, and you can kill just that one part.

And you can grep the damn logs.

Another daemon for managing control groups

Posted Jan 26, 2014 1:20 UTC (Sun) by HelloWorld (guest, #56129) [Link]

> Yes, the design of sysVinit is a *bit* rough. But it works,
It *doesn't work*. Stop denying it, sysvinit can't even stop a service reliably and correctly.

> and you don't have to read thousands of lines of source to figure out what it's doing.
Yeah, you just have to read thousands of lines of shell. With systemd otoh, you read *documentation*. You know, this stuff that is actually meant to be read by humans. Because unlike the sysvinit quagmire, systemd actually *has* documentation that's worth mentioning.

> And if one part of it falls over, ps will tell you, and you can kill just that one part.
Yeah, like systemd somehow magicall broke ps.

> And you can grep the damn logs.
You can do that with journalctl too. But actually most people don't do it, because it *sucks* and just using the domain-specific features of journalctl is easier, faster and more robust.

Another daemon for managing control groups

Posted Jan 27, 2014 19:38 UTC (Mon) by anselm (subscriber, #2796) [Link] (2 responses)

The current init/service activation infrastructure (System V init, inetd, cron, …) does not have anything worth calling »a design«. It is cobbled together out of random pieces from different sources and the amazing thing is that it works at all. The pieces don't talk to one another and are all configured differently. If it had in fact been »designed« you would expect at least a minimum of consistency and cooperation between the pieces.

If somebody proposed this setup today as a new system they would be laughed out of the lecture theatre. The only reason this is still in actual use at all is that for the longest time nobody has dared come up with an alternative. Most of the other Unix-like systems have replaced it long ago.

People always cite »the Unix philosophy« as »do one thing and do it well«, and point to System V init as a prime example of this approach. This disregards the obvious fact that while SysV init is surely doing something, there is no way that one could claim it is doing it well.

Another daemon for managing control groups

Posted Jan 27, 2014 20:45 UTC (Mon) by khim (subscriber, #9252) [Link] (1 responses)

This disregards the obvious fact that while SysV init is surely doing something, there is no way that one could claim it is doing it well.

It does pretty decent work on scaring newbies away from Linux. Perhaps that is the thing is was designed for? Of course it's kinda overenginered for such a use case, you can create a much smaller and simpler mess which will be much scarier.

Another daemon for managing control groups

Posted Jan 27, 2014 22:40 UTC (Mon) by nix (subscriber, #2304) [Link]

Someone needs to implement an init system as an IOCCC entry, clearly.

Another daemon for managing control groups

Posted Dec 8, 2013 13:00 UTC (Sun) by anselm (subscriber, #2796) [Link]

it's a jack-of-all-trades master-of-none that tries to do too much, tries to do everything, in fact.

Systemd is really a collection of co–operating tools, each of which tries to do one thing well (including but not limited to service process configuration, control, and management).

Another daemon for managing control groups

Posted Dec 6, 2013 13:13 UTC (Fri) by roblucid (guest, #48964) [Link] (15 responses)

Whilst appreciating Neil's point, it also seems to a more open philosophy to have program packages focussed on related tasks eg) setting up & managing raid whilst leaving FS layout, backup and so on to other tools. That makes it easier to evolve with time, because new features or projects improving on past work, only need to replicate a limited set of functionality.

The monolithic approach is disliked, because it leads to stagnation and lock in. Perhaps a new kernel feeature, would in practice require support by systemd, which in might not be of interest to that project; yet other projects would be faced with a daunting task to replicate a whole host of systemd compatible features, because of it's wide remit. This is why it could be so hard to replace MS Exchange servers in business; they weren't just doing email, but group meeting planning and so on.

systemd, merging init(8) and inetd(8) makes good sense, it's less clear what the benefit is of multiple incompatible implementations of cgroup managament to me. systemd's implementation is going to be rather experimental, which is very different from integrating in a better more modern implementation init & inetd; they may not have the best ideas but because of systemd becoming ubiquitous we could be lumbered with something satisfactory, rather than the best.

Another daemon for managing control groups

Posted Dec 6, 2013 15:00 UTC (Fri) by anselm (subscriber, #2796) [Link] (13 responses)

it also seems to a more open philosophy to have program packages focussed on related tasks eg) setting up & managing raid whilst leaving FS layout, backup and so on to other tools. That makes it easier to evolve with time, because new features or projects improving on past work, only need to replicate a limited set of functionality.

On the other hand, the latest fashion in file systems seems to be all-in-one approaches like ZFS and Btrfs, which combine file, volume, and RAID management, snapshots, etc. in one single implementation instead of relying on existing system features such as LVM and Linux kernel RAID.

It is interesting to note that the backlash against these new file systems is a lot less emotional than the one against systemd, even though the underlying idea of coming up with a radical new integrated design to replace a number of loosely-connected existing but sub-optimal infrastructure elements is quite similar. (But then again, Btrfs wasn't invented by Lennart Poettering.)

Another daemon for managing control groups

Posted Dec 6, 2013 16:27 UTC (Fri) by dlang (guest, #313) [Link] (10 responses)

> It is interesting to note that the backlash against these new file systems is a lot less emotional than the one against systemd

The reason for this is very easy to understand, nobody is forcing anyone to use the new filesystem.

Even if a distro were to change to use one of these new filesystems by default, the distro would not remove support for the existing filesystems, and would be extremely unlikely to make it so that if you selected one of the other filesystems you would have lots of other things you had to change in the distro as well.

It's really easy to have your system using different filesystems for different tasks.

It's really hard to have multiple init systems in your distro, even as options. And it's impossible to have a new init program on your system that you only use for some testing things.

Another daemon for managing control groups

Posted Dec 6, 2013 16:50 UTC (Fri) by anselm (subscriber, #2796) [Link] (6 responses)

nobody is forcing anyone to use the new filesystem.

Nobody is »forcing« anyone to use the new init system, either. It may be the case that a Linux distribution or a project like GNOME decides to take advantage of the new features and better integration that something like systemd has to offer, but if you don't want to come along for the ride there's always the likes of Slackware. The beauty of free software means that for as long as enough people want to stick with SysV init desperately enough, there will be a distribution (or distributions) around that is based on SysV init.

As far as file systems are concerned, the most recent SUSE distributions, for example, do »force« you onto Btrfs if you want to avail yourself of advanced tools like Snapper – and SUSE's enterprise-class distribution doesn't even support Ext4 (which arguably is the file system one would want to use if one didn't want to use Btrfs). Hence the situation isn't all that different, it's just that with file systems the »out with the old, in with the new« phenomenon isn't as widespread yet. This may also be to do with the fact that an init system is conceptually a lot simpler than a modern advanced file system, and that, for the time being, people (and in particular distribution makers) appear to be more inclined to trust systemd over SysV init than Btrfs over Ext4.

Another daemon for managing control groups

Posted Dec 6, 2013 18:09 UTC (Fri) by dlang (guest, #313) [Link]

sure, you can abandon the distro that you have been using when they change init systems.

you then have to listen to people harass other distros because they are being 'backwards' and not using the new init system.

people are far more careful about changing filesystems because a filesystem bug will eat their data, and distros that eat people's data tend to disappear :-)

Another daemon for managing control groups

Posted Dec 6, 2013 23:19 UTC (Fri) by nix (subscriber, #2304) [Link] (4 responses)

I note that people who write filesystems are paranoid bastards who know what backward compatibility is for. The history of udev and pulseaudio and all of Lennart's projects makes me think that he wouldn't know backward compatibility if it shook him very hard by the throat. PA only settled down and stopped breaking things for all its users on almost every upgrade when Lennart was no longer doing most of the maintenance. I see no sign that udev (which I long gave up trying to upgrade at despair at its constant pointless disruptive changes) and systemd are ceasing in their ongoing attempts to break the world.

After all, adapting to all that breakage is up to the *distribution*! And, oh yes, if you're not using Fedora and/or otherwise *precisely* the same platform Lennart is and experience problems you can bugger off. Or that's what I've seen Lennart saying over and over and over to quite tiresome lengths. (And, no, I'm *not* going to use a distro with a six-month lifespan for anything important. I need a greater degree of stability, without being stuck in the six-year-old software mire of the 'enterprise' distros, which apparently makes me a niche market nobody cares about.)

Another daemon for managing control groups

Posted Dec 7, 2013 0:20 UTC (Sat) by pizza (subscriber, #46) [Link] (3 responses)

Comments:

udev was primarily developed by GregKH. Though udev is now part of the systemd tree it is an independently-developed component that is one of the fundamental building blocks of a Linux system.

Meanwhile, Pulseaudio's user-facing problems were threefold -- Bugs in hardware drivers, bugs in ALSA's user plugin system, and horrendously buggy applications. Oh, and it was still a vast, vast improvement over what came before. (I say this as someone who has written a decent amount of audio code over the years..)

This is not unlike NetworkManager, which exposed the inconsistent behaviour of wifi drivers supposedly implementing the same Wireless Extensions interface (and the even more inconsistent event interface) and how every single WEXT client had to use device-specific hacks to ensure consistent behaviour.

I find it sad that the folks who put forth the effort to sanitize core infrastructure bits are blasted for all of the bugs they uncover (and fix!) in the process, while those same voices complain about how Linux isn't ready because things aren't transparently integrated and fully tested like $somecommercialos. This work doesn't just happen, and there are frightfully few people with the technical chops and skin thick enough to accomplish this.

Another daemon for managing control groups

Posted Dec 9, 2013 2:40 UTC (Mon) by ThinkRob (guest, #64513) [Link] (2 responses)

> Meanwhile, Pulseaudio's user-facing problems were threefold -- Bugs in hardware drivers, bugs in ALSA's user plugin system, and horrendously buggy applications. Oh, and it was still a vast, vast improvement over what came before. (I say this as someone who has written a decent amount of audio code over the years..)

Wasn't one of the somethings that came before something that had per-application mixing, a consistent cross-platform API (and cross-platform implementations), existing application support, and precise, lower-latency mixing? i.e. most of pulse's features, sans network streaming?

Another daemon for managing control groups

Posted Dec 9, 2013 13:44 UTC (Mon) by mathstuf (subscriber, #69389) [Link] (1 responses)

If you're thinking of JACK, it is a little on the high end and complicated for just getting mplayer to play audio at the same time as notifications. Pulseaudio works with JACK for handoff IIRC, so they can cooperate. If you're thinking of OSS4 or aRTS, I think those are pretty much dead in Linux-land.

Another daemon for managing control groups

Posted Dec 10, 2013 5:56 UTC (Tue) by ThinkRob (guest, #64513) [Link]

I was thinking of OSS4, which is both still developed for, and still quite usable in Linux. It's just (sadly) no longer the default, in large part due to self-inflicted wounds.

Another daemon for managing control groups

Posted Dec 10, 2013 19:06 UTC (Tue) by BenHutchings (subscriber, #37955) [Link] (2 responses)

Even if a distro were to change to use one of these new filesystems by default, the distro would not remove support for the existing filesystems,

It certainly might remove support for existing filesystems as root. For example, the package manager might be changed to require transaction support. (I would expect to hear a similar outcry if this actually happens in a mainstream distro.)

Another daemon for managing control groups

Posted Dec 10, 2013 22:12 UTC (Tue) by anselm (subscriber, #2796) [Link] (1 responses)

For example, the package manager might be changed to require transaction support. (I would expect to hear a similar outcry if this actually happens in a mainstream distro.)

OpenSUSE and SLES are apparently moving in that direction. Recent incarnations of YaST use a tool called »Snapper« to make snapshots of the root file system before and after configuration changes; if you want to avail yourself of that functionality you are pretty much forced to use Btrfs. So far people don't seem to get as worked up about this as they do over systemd (which to be fair was introduced to openSUSE some time ago, seemingly without much of a fuss either).

Another daemon for managing control groups

Posted Jan 1, 2014 12:27 UTC (Wed) by jospoortvliet (guest, #33164) [Link]

Note that some work is ongoing to support this on non-btrfs filesystems...

Another daemon for managing control groups

Posted Dec 9, 2013 2:36 UTC (Mon) by ThinkRob (guest, #64513) [Link]

<quote> It is interesting to note that the backlash against these new file systems is a lot less emotional than the one against systemd, even though the underlying idea of coming up with a radical new integrated design to replace a number of loosely-connected existing but sub-optimal infrastructure elements is quite similar. (But then again, Btrfs wasn't invented by Lennart Poettering.) </quote>

AFAIK, no distros have plans to adopt a single filesystem as the One True filesystem and stop supporting all others. The same cannot be said for systemd.

The upshot of this is that while you're always free to run whatever FS you choose despite the default choice of your distro's installer, the same cannot be said for the ease with which one can switch init systems.

Another daemon for managing control groups

Posted Dec 10, 2013 8:29 UTC (Tue) by zdzichu (subscriber, #17118) [Link]

Memory fades fast. There was "blatant layering violation" as a nickname for ZFS for some time.

Another daemon for managing control groups

Posted Dec 7, 2013 0:26 UTC (Sat) by rgmoore (✭ supporter ✭, #75) [Link]

it's less clear what the benefit is of multiple incompatible implementations of cgroup managament to me.

The requirement of a single manager for cgroups is coming from the kernel side, not from systemd. Since systemd makes very heavy use of cgroups for process management, it makes sense for systemd to be that manager in distributions that use it. Otherwise, you have the init system depending on another process to do a truly essential part of its job, which doesn't seem any simpler than having an integrated system.

Another daemon for managing control groups

Posted Dec 5, 2013 4:03 UTC (Thu) by daniels (subscriber, #16193) [Link]

inb4 ... damn, too late.

Another daemon for managing control groups

Posted Dec 5, 2013 6:55 UTC (Thu) by zdzichu (subscriber, #17118) [Link] (7 responses)

systemd is becoming *the* Linux userspace.
Just like glibc is *the* Linux system libc. There are other contenders, but they occupy their small niches.

Another daemon for managing control groups

Posted Dec 5, 2013 11:03 UTC (Thu) by rwmj (subscriber, #5474) [Link] (6 responses)

It's not the same thing. glibc provides a broad API, but with definite limits, and you can easily replace bits of glibc. Need to use a GUI? Link with gtk. Need a different way of handling memory? Link with talloc. You can even replace (for example) all str* functions with stricter versions from an external library.

systemd is a fine way to manage daemons, but there's no reason why it couldn't use a library or separate process for cgroup management.

Breaking up monolithic daemons into separate processes and libraries with well-defined interfaces between them is both the Unix philosophy AND a better way to develop more reliable software.

Another daemon for managing control groups

Posted Dec 5, 2013 11:17 UTC (Thu) by hummassa (guest, #307) [Link]

Actually, the only pro-systemd argument I heard that makes a lot of sense seems to contradict you and goes in the "yo dawg, I'll make init, that is a daemon, to init and manage the deamon that will init and manage the other daemons so you can init and manage the deamons" direction. If systemd/sysv-init will manage one deamon-managing process, isn't it the same as managing all the deamons? I really don't have an answer.

Another daemon for managing control groups

Posted Dec 5, 2013 13:03 UTC (Thu) by mezcalero (subscriber, #45103) [Link] (4 responses)

Same thing with systemd. You don't like logind? Turn it off at compilation, use ConsoleKit instead. Don't like journald? Just run syslog, too. Don't like it's SELinux support, then turn it off. Don't like its bootchart implementation, then turn it off, maybe use another one. Don't like its readahead implementation? Then turn it off, use another one. There are 35 things you can turn/off with configure switches, to tailor it exactly to your needs. And distros can split up systemd into multiple RPMs/DEBs if they desire so and make use of them independently.

However, there are a few things you cannot turn off: That's PID1's process management of which cgroups is part of. That's udev device management and that's the journal (but you can turn off the journal's storage on disk, we however need it to pass stdout/stderr of all daemons through).

And yes, there are tons of reasons why you want cgroup management in PID1: because it's trivially easy there and simple, and it is not if you do it outside of PID1. If you do it out-of-process, you need to replicate pretty much the entire state of your service manager in your cgroup manager, since the entities that are resource managed are in 95% of the cases exactly the same entitites that are service managed. So splitting them up means you need some form of IPC that constantly replicates the entire tree of services from PID1 in that other cgroup daemon. That's fragile and messy, and a lot of unnecessary code. IPC always is. Then, there is the issue of cyclic deps: a good PID 1 knows at any time securely and reliably of each process to which service it belongs. That's easy to do with cgroups. However, if cgroup management is done outside of PID1 in a different process, then PID 1 suddenly becomes a client to that other process, while that other process is also a managed process PID 1 wants to use cgroups to manage for. To start and manage that other daemon that will allow you to deal with cgroups you need cgroups in the first place. And that's just broken.

Beyond that: When doing cgroups you need an execution queue of some kind and you need a dependency tree between your groups, so that you can always rebuild your tree from the root down, in the right order, with each property set before you go to the next child. You also need to dynamically react to devices coming and going and rebuild/refresh your tree since many cgroup props take device major/minor pairs which are pretty much dynamically assigned to devices these days. You also need to be able to deal with groups coming, and going, and possible propagating changes then to other groups on the same level. Soooo, an execution queue that is based on a dependency tree that is influenced by services/cgroups coming and going and devices coming and going, that's pretty much exactly what systemd *is*. If you avoid doing this in PID 1 then you are just recreating systemd a second time in a different process. A lot more code to write, to test, to spend resources on. I am sorry, but I am just not going to write or help building such a needlessly complex system. You can have it easy and small and today. I am really not interested in having it complicated, redundant, larger and one later day.

(And no, a library certainly doesn't work at all for this, you need to cross the privilege boundary and a single active queue dispatcher, which doesn't really make a library such a good choice...)

Another daemon for managing control groups

Posted Dec 5, 2013 19:03 UTC (Thu) by butcher (guest, #856) [Link] (3 responses)

I believe it's good to have both, systemd/cgroups and cgroupd, but I'll probably be buildroot-ing images with systemd/cgroups. It seems like a more appropriate factoring of functionality.

Thanks all involved for the ability to choose.

Another daemon for managing control groups

Posted Dec 5, 2013 20:18 UTC (Thu) by drag (guest, #31333) [Link] (2 responses)

> I believe it's good to have both, systemd/cgroups and cgroupd,

Why is it good to have both?

In what way is systemd's solution inadequate?

Another daemon for managing control groups

Posted Dec 5, 2013 20:55 UTC (Thu) by jake (editor, #205) [Link] (1 responses)

> In what way is systemd's solution inadequate?

I think primarily the problem that some people have with systemd's solution is that it requires running systemd as the system's init, which some do not want to do. According to Lennart in http://lwn.net/Articles/575847/ there are "lots of cases where you don't need systemd", but if you want to use cgroups you (will) need some kind of manager.

systemd's solution may well be perfectly adequate (or, in my mind, way better than 'adequate'), but it does require systemd.

jake

Another daemon for managing control groups

Posted Dec 6, 2013 18:06 UTC (Fri) by drag (guest, #31333) [Link]

> but if you want to use cgroups you (will) need some kind of manager.

Yes, but only one can be used at a time. So any solution that will provide cgroups management will have the same problem as systemd.

No really good way to fix this I expect.

Another daemon for managing control groups

Posted Dec 5, 2013 8:05 UTC (Thu) by ovitters (guest, #27950) [Link] (3 responses)

Being able to setup tcp connections from bash is probably terribly evil in your book? Let's replace bash then, because that's what bash provides (built in, no external tools required).

Anyway, a philosophy has a certain purpose. Blindly following some philosophy is not useful. Calling things unneeded names like "opaque ball" is vague. Further, "everything your Linux system does" is factually wrong.

Try being specific: why *must* *everything* be reimplemented multiple times?

Further, if you continue to twist words like you did before I'll call you out on it. Good discussion ok, but twisting of words is poor form.

Another daemon for managing control groups

Posted Dec 5, 2013 12:28 UTC (Thu) by NAR (subscriber, #1313) [Link] (1 responses)

Try being specific: why *must* *everything* be reimplemented multiple times?

Exactly: why is an operating system (Linux) implemented when there were (and are) multiple other operating systems?

More seriously: one answer is that specifications can be "verified" by implementing them multiple times by different people and see if the result interoperates. Of course, my gut feeling is that the systemd and cgroup developers are not interested in interoperability or portability or even stable APIs because they hinder progress. Whether one agrees with this position or not, that's an other question.

Another daemon for managing control groups

Posted Dec 5, 2013 12:41 UTC (Thu) by ovitters (guest, #27950) [Link]

You're not being specific, you're just answering one question. Where is the specification for every software available on Debian? Where are the multiple implementations for every software?

I was asking why *everything must* be implemented multiple times.

Also your answer implies that systemd has a specification which it is being written against. I'm not aware of such specification. Systemd did create various specifications.

Suggest thinking about my question a little bit more.

Another daemon for managing control groups

Posted Dec 6, 2013 2:38 UTC (Fri) by rgmoore (✭ supporter ✭, #75) [Link]

Try being specific: why *must* *everything* be reimplemented multiple times?

I'm not sure if everything must be reimplemented multiple times, but everything should be capable of being reimplemented multiple times so that components can be torn out and replaced with better ones. That has been a significant way that Unix/Linux has made progress. Of course it's helpful to have standardized interfaces so that new implementations can be drop-in compatible with old ones. It may even be helpful to have multiple implementations of critical infrastructure so that it can be replaced quickly if it's ever discovered to have critical flaws.

Another daemon for managing control groups

Posted Dec 5, 2013 9:53 UTC (Thu) by Karellen (subscriber, #67644) [Link] (1 responses)

"Do one thing. Do it well."

I don't use systemd (yet) but I do see it doing one thing well. It manages the other processes (or groups of processes) on your system.

That means starting them up, monitoring them, re-starting them if necessary, and shutting them down. Whether those processes start on system boot, on receipt of a network connection, on a timer, or on some other event, it makes sense to me that there's one process that knows the whole policy, and can make sure it's being followed as intended.

And that to me means that if you want to set up a special environment for a new process, that daemon is the place to do it. Want to specify which user/group the process runs as? Be able to configure that in the process-managing deamon. Want to specify resource limits for the new processes (rlimits/ulimits)? Configure it in the process-managing daemon. Want to specify an alternate rootfs - configure it in the process-managing daemon.

Want to manage the control groups that new processes are started in, or move processes between control groups? Why not do it in the process-management daemon.

Another daemon for managing control groups

Posted Dec 6, 2013 23:23 UTC (Fri) by nix (subscriber, #2304) [Link]

That means starting them up, monitoring them, re-starting them if necessary, and shutting them down.

Shame it's trying to do so many other things too.

Another daemon for managing control groups

Posted Dec 5, 2013 19:57 UTC (Thu) by drag (guest, #31333) [Link]

> Does not lumping *everything your Linux system does* inside the opaque ball of systemd violate nearly 35 years of Unix Philosophy pretty violently?

No. Because that is not what is happening.

> Do one thing. Do it well.

Cgroups is about managing resources of processes in a hierarchical/grouped manner from what I can tell. Linux/Unix processes are already arranged in a hierarchical manner due to the parent child process. The parent process is PID 1, which is the Init.

Makes sense to me that you'd want to be able to manage things from the top down instead of having a entire separate process you have reprogram your applications deal with via dbus.

Another daemon for managing control groups

Posted Dec 12, 2013 8:19 UTC (Thu) by oldtomas (guest, #72579) [Link]

> Am I really the *only* person who hasn't drunk the koolaid?

No, you aren't the only one. "Me Too".

I've managed to avoid systemd up to now, and hope I'll manage in the future.

It's just that the discourse with its very vocal proposers isn't fun anymore.