LWN: Comments on "Tumbleweed backs off on systemd for now"

pulseaudio digression [Tumbleweed backs off on systemd for now]

sfeam — Fri, 23 Sep 2011 22:35:21 +0000

I've had similar aggravations ever since pulseaudio appeared. In my case there is no bluetooth involved, but pa randomly loses devices and settings across suspend/resume, shutdown/reboot, and even at less well defined times. I have found no real solution, but for me it mostly worked to get an acceptable state in place and then do "chmod -R a-w ~/.pulse". That obviously has other drawbacks, but at least it stopped losing track of the devices altogether. And the instantaneous settings mostly got pinned as the default state.

Tumbleweed backs off on systemd for now

nevets — Fri, 23 Sep 2011 21:58:22 +0000

I want sound to default to my speakers. I have a single app (twinkle) that should go to the headset. Note, this headset has a mic and is for calling. Twinkle can select which source dependent from other sound apps. I have yet to figure out how to consistently make it default to the speakers.

Tumbleweed backs off on systemd for now

jspaleta — Fri, 23 Sep 2011 18:37:36 +0000

To be clear, what is the expected default scenario on boot from your perspective?

With the usb headset plugged in you want the sound to be routed to both the speakers and the usb headset as the default configuration?

-jef

Tumbleweed backs off on systemd for now

nevets — Fri, 23 Sep 2011 18:21:31 +0000

Actually, my desktop can't play sound anymore after I did an upgrade. PulseAudio for some reason just wants to use my USB headset, and ignores the speakers. Every so often I tweak a bunch of things to get it working, but after a reboot, it's gone again.

Two years ago, my desktop sound worked out of the box. Today? Not so much.

Is that due to a constraint on kernel version/features or safe "upgradibility"?

zuki — Fri, 23 Sep 2011 10:49:25 +0000

> In all cases, being able to boot the previous kernel helps a lot,
> since my distro doesn't permit "un-upgrading" userland (is that possible on some other distros ?).

According to your previous comment, you are using debian. Check out e.g. 'apt-get install libc6/squeeze -V'. Sometimes it's a bit of manual labour, but downgrading works.

Is that due to a constraint on kernel version/features or safe "upgradibility"?

herodiade — Thu, 22 Sep 2011 17:52:36 +0000

> if your system is that "stable" why do you need/want systemd?

Well, it may not be that "stable" as you guessed ;). But more concretely, I was thinking mostly about all my difficult upgrades experiences, like "oops, I broke the grub entry for the new kernel" or "oops, nvidia hadn't provided anything yet for the new kernel", "oops, my block driver now require a firmware from an external package", ...

In all cases, being able to boot the previous kernel helps a lot, since my distro doesn't permit "un-upgrading" userland (is that possible on some other distros ?).

As to why upgrading, well, for the desktop it seems obvious (new and maintained releases of, say, firefox, are more important than new kernels I think). For the server, in my case, the upgrades are triggered by developers needing recent versions of php and python, not by a new init system; but on the other hand, I don't expect distributions to provide both systemd and systemd-less upgrades with equal robustness.

> You seem to be assuming a newer userland that immediately drops support for sysvinit and yet doesn't have other kernel incompatibilities.

Not at all, but each new incompatibility makes upgrades a bit more dangerous. Not enough to throw away a valuable tool like systemd though.

I got bitten by udev during system upgrades. I should have (according to my distro upgrade guide, but forgot to) upgraded the userland first, then kernel. I guess in that case the kernel may have missed forward compatibility; but what would happen if, at the same time, the new userland (ie. a systemd daemon) required a newer kernel?

> As for a long term plan to make systemd compatible with older kernels, wouldn't such a goal likely be obsolete by the time it was achieved?

Yes probably. But it could be something like "a new systemd release works on kernels released at least X years before the current systemd release" maybe (ie. to support kernels from the previous release of "enterprise" distributions, and permit smooth upgrades ?). Or just "the kernel 3.X has everything needed for a good init system, so even if we won't test olds kernels in the future, we stop moving dependency on features from newer and newer kernels, [minus oversight/bugs that vendors have to find and fix themselves for their needed versions]". Or (more likely ?) it may happen without formal decision, almost magically, when more users will use systemd ;).

But granted, I have no idea about the trade-off for systemd developers, the difficulties to overcome, and I don't know what makes older (yet cgroups-enabled) kernels hard to support.

Is that due to a constraint on kernel version/features or safe "upgradibility"?

maderik — Thu, 22 Sep 2011 16:11:06 +0000

I'm not sure I understand: if your system is that "stable" why do you need/want systemd? You seem to be assuming a newer userland that immediately drops support for sysvinit and yet doesn't have other kernel incompatibilities. As for a long term plan to make systemd compatible with older kernels, wouldn't such a goal likely be obsolete by the time it was achieved?

It's possible that this itch will be scratched by someone -- if in the future there are systems that need both systemd advantages and still run older kernels -- but it seems a lot of effort for that use case w/o a good estimate it's importance.

Is that due to a constraint on kernel version/features or safe "upgradibility"?

herodiade — Thu, 22 Sep 2011 11:46:44 +0000

While I'm pretty much eager to get systemd as default from my distribution of choice (Debian, but I'm patient ;), and find the project almost perfect as is, having a good dozen of killer features, almost all compromises (including the lack of support for non-linux platforms) reasonable, and admire Lennart sane attitude and zen facing (sometime) unfair criticisms (when not insults), I don't feel at ease with the lack of support for older linux kernels.

This will imply flag days, dangerous upgrades (where you absolutely have to keep kernel and base userland in sync). And we loose the option to run a modern distros with fresh userland but somewhat older kernel (ie. when some driver or component - say xen, openvz, oracle, ... - isn't supported for newer releases), or want a solidly supported system if you encounter a bug after an upgrade and need to switch back to the previous kernel, ...
I'm not sure if this change in OpenSUSE is related to those constraints on kernel versions ?

And I wonder whether upstream account this difficult as I do (ie. like in https://bugzilla.redhat.com/show_bug.cgi?id=628004#c12 , except the specific cause of this reply was addressed at the end of the day).

More specifically, this requirement on recent kernels is dangerously sliding to very recent versions (will upstream decide upon a definitive "minimal kernel release" ?). See for instance
http://cgit.freedesktop.org/systemd/commit/?id=a63599edcc... :

REQUIREMENTS:
- Linux kernel >= 2.6.30 (with devtmpfs, cgroups; optional but strongly recommended: autofs4, ipv6)
+ Linux kernel >= 2.6.39 (with devtmpfs, cgroups; optional but strongly recommended: autofs4, ipv6)

Which means, we probably can't run a really reliable/supported systemd on a 2.6.32 kernel as provided by many stable distributions (?).

Do you know if there's a plan (even for the long term) to polish systemd in this respect?

Tumbleweed backs off on systemd for now

smurf — Thu, 22 Sep 2011 06:33:03 +0000

+1 again.

systemd may not yet be at the stage where it needs to be, but I'm pretty sure it'll get there.

Just like PulseAudio, in fact. Did you ever try to sync to a new Bluetooth headset and play your music through it? With pulse, that stuff just works. Today.
Two years ago? Not so much.

Tumbleweed backs off on systemd for now

jcm — Thu, 22 Sep 2011 03:13:45 +0000

I get "too many lines". Too many lines is defined as more than /proc, a tmpfs or two, and my filesystems. When I have to sift through dozens of lines of useless mounts to get to the crux of what I'm using, then it's too many and it's unusable. Sure, it's not systemd's fault that mount, df, and friends don't make this easier, but it was systemd that changed the world to the point that this became a problem. As part of integration, this stuff should be addressed (and that means other than by adding a GUI for df).

Tumbleweed backs off on systemd for now

vonbrand — Thu, 22 Sep 2011 02:05:02 +0000

/me runs for cover...

Tumbleweed backs off on systemd for now

HelloWorld — Wed, 21 Sep 2011 17:44:22 +0000

Those incompatibilities only bite scripts that are broken begin with. If you, say, rely on sysvinit ignoring your broken LSB headers, you deserve to be punished.

Tumbleweed backs off on systemd for now

NAR — Wed, 21 Sep 2011 17:22:46 +0000

Probably yes. The systemd documentation does mention some incompatibilities - the fact that they don't seem to bite me doesn't mean it can't bite somebody else.

Tumbleweed backs off on systemd for now

rgmoore — Wed, 21 Sep 2011 16:31:50 +0000

And my impression is that one of the design goals of systemd is to maintain significant backward compatibility with SysVinit. For example, it can use existing init scripts to manage services rather than its native configuration files. It can take advantage of cgroups for process management, but it's capable of working on systems without cgroup support. And so forth. You don't get the full power of the new system when you run it using the backward compatibility features, but it does work.

Tumbleweed backs off on systemd for now

martinfick — Wed, 21 Sep 2011 15:47:45 +0000

I think you just described sysvinit.

Tumbleweed backs off on systemd for now

tzafrir — Wed, 21 Sep 2011 15:32:28 +0000

Maybe what you actually need is df -T #?

Tumbleweed backs off on systemd for now

HelloWorld — Wed, 21 Sep 2011 14:33:10 +0000

> I'm certain that systemd will also get there, eventually. The problem is that it's not there yet,
This is, again, just meaningless FUD. If you see an actual problem with systemd, then explain what it is, instead of nebulously asserting that it's "not there yet".

> and the system it tries to replace, even with all his warts, is solid and well understood.
Yeah right, except that it isn't.
Quoting from http://lwn.net/Comments/459725/ :
> If anyone, and I mean anyone, claims to "understand" the current init system - that person is full of crap. The current system is the nightmare everyone is claiming systemd is; it is buggy, complex, slow, and *extremely* fragile.

Tumbleweed backs off on systemd for now

nix — Wed, 21 Sep 2011 14:27:20 +0000

Why waste a perfectly good cgroup (or something else) management when it is available?

Incoming ballistic Viro expected shortly to explain.

Tumbleweed backs off on systemd for now

HelloWorld — Wed, 21 Sep 2011 13:32:48 +0000

So you're saying that you can understand people who don't want to "embrace" systemd, and yet in the same posting you explain why there's absolutely no reason not to? Am I the only one to whom this seems weird?

Tumbleweed backs off on systemd for now

rahulsundaram — Wed, 21 Sep 2011 13:08:15 +0000

In Fedora 14, systemd was optional. In Fedora 15, it was the default with a smaller number of services converted. In Fedora 16, a lot more will be and so on. This is incremental progress

Tumbleweed backs off on systemd for now

whitemice — Wed, 21 Sep 2011 11:28:05 +0000

> While the primary reason cited for pulling it was to allow developers to
> focus on 12.1 bug reports, gkh also rightly pointed out on the openSUSE
> list that Tumbleweed is supposed to be openSUSE with stable updates. The
> systemd in tumbleweed was not stable. So, it shouldn't be there.

Reasonable decision made; nothing to see here, move along.

Tumbleweed backs off on systemd for now

whitemice — Wed, 21 Sep 2011 11:25:29 +0000

I've been a professional UNIX/LINUX System Administrator for ~20 years...

> I personally came at systemd without a preconceived notion that it was
> wrong or a failure and instead read the design docs and was excited. It
> also helps that many years ago the team I was with gave up on sysvinit and
> unmanaged daemons and tried to only run services under daemontools.

Yep. The claims that systemd is complex and not well understood - in relation to the pile-o-hacks that is the current script based init system?

"Captain! The bogosity field is approaching maximum containment!"

If anyone, and I mean anyone, claims to "understand" the current init system - that person is full of crap. The current system is the nightmare everyone is claiming systemd is; it is buggy, complex, slow, and *extremely* fragile. I know how to *use* and modify the current init system to accomplish common goals - but there is no understanding that thing as there is very little to anything of a "design". It has been built via the uh-oh-we-need-to-deal-with-that-now method. The method it uses to determine initialization order can barely be described as a method.

> I don't understand all this hostility at all,

And on my laptop Pulse Audio works perfectly, and has since almost the beginning. Much like the Beagle search tool a couple of buggy initial releases have been perpetually held against by a small but loud group of users [who generally just like to hate things, IMNSHO]. I predict that systemd will face the same fate - it will be adopted but there will be some initial issues and a segment of users will spend the next decade talking about how much systemd sucks - while everyone else forgets about it because it just-works. And their systems boot so much faster they don't have to watch it roll-by every time they power on.

Tumbleweed backs off on systemd for now

neilbrown — Wed, 21 Sep 2011 11:06:06 +0000

One thing that bugs me about systemd is that it is presented as all-or-nothing. flag-day changes are always painful and should be avoided when possible and minimised when essential.

So I would much rather that in the first instance, systemd was 'just another daemon' like inetd and it was given some simple services to manage. Then people could become familiar with it without the risk of paralyzing their machine.

Then services could be gradually migrated across until eventually the only thing that the sysvinit started was systemd.

Then a mini-flag day when we discard sysvinit and run systemd as pid-1.

Incremental development has a long tradition of working well in Linux circles...

Tumbleweed backs off on systemd for now

quintesse — Wed, 21 Sep 2011 09:30:23 +0000

And since when is a FLOSS project ever finished??

Tumbleweed backs off on systemd for now

NAR — Wed, 21 Sep 2011 09:05:45 +0000

Please enumerate what things were possible with sysvinit and can't be done today. Then explain when you would use each of them

I'm aware of Poettering's views on portability, nevertheless my requirement is to make an application that works on Solaris and on a specific brand of Linux, so I don't like any extra incompatibility introduced, because that's more work for me. For example currently we use the very same SystemV init script on both Solaris and Linux. Based on my understanding of the systemd documentation we can keep on using the same script, so probably systemd (when it will be introduced at our customers in about 5 years time) won't cause any headaches for us, but I can understand other application developers who might not want to embrace systemd.

Tumbleweed backs off on systemd for now

dgm — Wed, 21 Sep 2011 08:58:42 +0000

I mine they spell "unfinished project".

Tumbleweed backs off on systemd for now

dgm — Wed, 21 Sep 2011 08:38:24 +0000

> Of course I'm one of those persons who hasn't had any problem with Pulse Audio for ages now and think it's actually quite good so I might not feel the need to rile against another Poettering project ;)

I'm certain that systemd will also get there, eventually. The problem is that it's not there yet, and the system it tries to replace, even with all his warts, is solid and well understood. And let's not forget that we are talking about a critical part of the system.

Tumbleweed backs off on systemd for now

rahulsundaram — Wed, 21 Sep 2011 07:31:02 +0000

Its not about what you said. The problem is with the way you said it. It was not constructive.

Tumbleweed backs off on systemd for now

quintesse — Wed, 21 Sep 2011 07:17:40 +0000

"What I see is a lot of code, a lot of scope creep, a lot of code changes... and these things usually spell disaster"

Weird, in my book they spell "successful project".

Tumbleweed backs off on systemd for now

dsommers — Wed, 21 Sep 2011 06:50:56 +0000

> I think you can at least leave the H out of you're IMHOs ;)

Unless that H is for Honest ...

Tumbleweed backs off on systemd for now

cowsandmilk — Wed, 21 Sep 2011 01:00:19 +0000

While the primary reason cited for pulling it was to allow developers to focus on 12.1 bug reports, gkh also rightly pointed out on the openSUSE list that Tumbleweed is supposed to be openSUSE with stable updates. The systemd in tumbleweed was not stable. So, it shouldn't be there.

Tumbleweed backs off on systemd for now

rgmoore — Tue, 20 Sep 2011 23:04:24 +0000

My concern is that systemd does require a contemporary kernel

This seems like a nonsensical criticism. Replacing an init system is a far more intrusive change than updating a kernel. Anyone who's willing and able to rip out SysVinit and replace it with systemd should be more than up to the task of updating their kernel to one with the features needed to support it. Which Linux users are interested in updating to systemd but are holding back because they don't want to update to kernels with cgroup support?

It's a doubly ridiculous criticism when aimed at a distribution that provides its own kernels along with the rest of the system. If openSUSE or Fedora decides that systemd is a good idea, it's no problem at all for them to ensure their kernels support its features. They're going to be shipping a contemporary kernel whether they include systemd or not, so the need for a contemporary kernel is not a real-world hindrance to adopting it.

The only way this criticism makes any sense is if you don't like the kernel features that systemd depends on and want to compile your kernel without them, or you think it should support other Unix-like systems that lack those features. But if you want to make those points you should make them directly.

Tumbleweed backs off on systemd for now

HelloWorld — Tue, 20 Sep 2011 20:29:41 +0000

> We have a unusually high level of signal compared to noise in this site and we must maintain this collectively.
That's why I told cjcox to produce something more signal-like or at least stop making noise.

Tumbleweed backs off on systemd for now

jmayer — Tue, 20 Sep 2011 20:17:22 +0000

While the article mentions the fact that there was discussion whether and with which scope to "publish" this information (apart from the ml post that had already occurred) it does not elaborate the reason why it was done. Tumbleweed tries to provide only "stable" packages. Right now one of the big problems that systemd in Tumbleweed seems to experience is the interaction with other packages (surprise). Updating one without the other may lead to inconsistencies or simply trigger bugs. To make it easier for the developers that are working on getting systemd ready for production use for 12.1 (factory right now) removing it from Tumbleweed allows them to focus on one set (or combination) of packages to debug/stabilize instead of two.

Tumbleweed backs off on systemd for now

jonabbey — Tue, 20 Sep 2011 20:09:42 +0000

This.

It's far better to pull the complexity in towards a well engineered core than to require the complexity to be smeared out across hundreds or thousands of applications.

It'll be a long time before daemon authors can trust that systemd will be there, but I'd rather write to systemd than sysv init.

Tumbleweed backs off on systemd for now

imitev — Tue, 20 Sep 2011 19:27:48 +0000

>> As jspaleta points out user space may evolve and maybe distros will begin to ship helper scripts or patched mount

replying to myself: /me will have to check new commands when upgrading (thank you juergbi for the pointer to findmnt)

Tumbleweed backs off on systemd for now

mezcalero — Tue, 20 Sep 2011 19:24:56 +0000

systemd does not require rewriting daemons. Not at all. No patching required. You can patch your daemons if you wish, to integrate things better with systemd, but that's strictly optional, and usually trivial and far far far from the word "rewriting".

And note that our compatibility with SysV goes quite far, and where it ends we have documented this pretty comprehensively:

http://www.freedesktop.org/wiki/Software/systemd/Incompat...

Please be aware that when we deviate from SysV we generally do that for good reasons, and to make things easy we document the incompatibilities. We really try our best to design a sane system on one hand but to keep the conversion from SysV as hassle-free as possible on the other. Sometimes these goals contradict. Hence there will be hassles, there's no doubt on that, and we'll not lie about that.

Tumbleweed backs off on systemd for now

imitev — Tue, 20 Sep 2011 19:19:31 +0000

>> "In any case, what do you want to read the full output for"

You mean filtering the output ? Reading quickly through a few short lines is easier / takes less time than thinking of a search pattern: oh, what was the fs type again ? where was it mounted ? what are the mount arguments, ...
And then: hmm, no match, did I make a mistake with the regex ? In which case it's safer to use grep -v '\(/sys\|systemd\|tmpfs\|mqueue\|hugetlbfs\)' but you'll agree it's not so convenient when you forgot to set it as an alias or you're too lazy to hack/deploy a little filtering script.

[Note: as I'm getting used to systemd - which I like - I'll just get used to have 40+ lines of mount output. I'm just outlining it used to be a bit easier before, and that for various reasons people still need to go through mount's output (I often do). As jspaleta points out user space may evolve and maybe distros will begin to ship helper scripts or patched mount...]

Tumbleweed backs off on systemd for now

juergbi — Tue, 20 Sep 2011 19:16:47 +0000

Have you tried findmnt? Its output is much easier to read.

Tumbleweed backs off on systemd for now

vonbrand — Tue, 20 Sep 2011 19:11:12 +0000

Please enumerate what things were possible with sysvinit and can't be done today. Then explain when you would use each of them

As for exploiting new kernel features, why, they are there to be used. Why waste a perfectly good cgroup (or something else) management when it is available? Even sysvinit required some special kernel tweaks (passing runlevels around, at the very least). Daemon rewriting, especially if it means getting rid of repetitive and error-prone drop privileges, daemonize, manage children code, is welcome (like it is a pleasure to be able to write a network service under {,x}inetd instead of directly accessing the 'net). Ditto for getting rid of hundreds of "init library" shell lines, and (again) long, repetitive, error-prone, hard to test shell scripts for managing services.