The Grumpy Editor's guide to surviving the systemd debate [LWN.net]

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 13, 2014 3:37 UTC (Thu) by mgb (guest, #3226)
In reply to: The Grumpy Editor's guide to surviving the systemd debate by bferrell
Parent article: The Grumpy Editor's guide to surviving the systemd debate

Init. Logging. Process Monitoring. Udev. Networking. Login. Audio. Printing. Desktop. Security. NTP. DNS. KDbus. Distribution. Packaging.

Then they came for your beloved kernel, Jon --- and there was no one left to speak for you.

This debate is not about technology. There's nothing even remotely new about dependency based boot, daemon management, and utility libraries.

This debate is about control. This debate is about freedom.

to post comments

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 13, 2014 3:52 UTC (Thu) by jonnor (guest, #76768) [Link] (2 responses)

No, this debate is about fear, unwillingness to work constructively and take responsibility.

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 22, 2014 18:34 UTC (Sat) by thedevil (guest, #32913) [Link]

"No, this debate is about fear"

Sometimes fear is justified.

I fear the next US national election, because I have lived long enough
to see a trend (and not just a statistical trend but a trend in the
opinions of people I talk to). My fear of systemd and projects
associated with it is somewhat similar. The similar aspects include the
fact that nominally I can influence the outcome in both cases (by voting
or contributing code), yet my influence will make no difference in the
face of the trend, with a 99.999% probability.

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 28, 2014 5:43 UTC (Fri) by blujay (guest, #39961) [Link]

> unwillingness to work constructively

A common refrain of the systemd advocates in Debian is, "You can't force people to do work they don't want to do." Yet here a systemd advocate seems to be complaining about other people not doing work they don't want to (have to) do.

> unwillingness to take responsibility.

Responsibility for what? Fixing breakage caused by systemd?

Perhaps I misunderstand you, but this seems like hypocrisy. If systemd worked constructively with existing software and other distros, and if it took responsibility for fixing things it breaks (instead of, "This is your wake-up call: fork it, reimplement it, or have no more udev. Good luck."), we wouldn't be having such vehement arguments.

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 13, 2014 12:15 UTC (Thu) by motk (guest, #51120) [Link]

The debate is about ethics in gaming^Wlinux journalism.

Not really. Sheesh, overreach.

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 13, 2014 12:55 UTC (Thu) by cortana (subscriber, #24596) [Link] (21 responses)

I think you just went full Godwin. *plonk*

Nobody expects the spanish plonk squad!

Posted Nov 20, 2014 1:35 UTC (Thu) by ksandstr (guest, #60862) [Link] (20 responses)

That's funny: I didn't think there were killfiles on LWN. What is the reason you go out of your way to state that you won't be listening in the future?

In the absence of an answer, I'll assume that you're utilizing an excuse not to engage, though it is a weird thing to do as a first post in a thread. However there are worse readings, so I'll go with this best-faith one.

Nobody expects the spanish plonk squad!

Posted Nov 20, 2014 1:59 UTC (Thu) by mpr22 (subscriber, #60784) [Link]

I didn't think there were killfiles on LWN.

Subscribers at the "professional hacker" level and above have had access to a simple "filter all comments from this list of users" feature for quite some time now; the addition of the feature was announced in an LWN article at the time. A list of filtered users looks a lot like a killfile to me.

Nobody expects the spanish plonk squad!

Posted Nov 21, 2014 0:53 UTC (Fri) by flussence (guest, #85566) [Link] (18 responses)

> What is the reason you go out of your way to state that you won't be listening in the future?

It's the human equivalent of a "too many errors, aborting" message from the compiler.

A message like that *is* snippy and rude on the surface, but it gives the plonk-ee at least the courtesy of knowing they've gone off-course far enough to provoke such a reaction, and leaves them an opening to do better in future.

It's more than a throwaway remark, IMHO. Staying to face someone down and tell them why they've screwed up badly enough to end up on one's killfile — under the aggravation that leads to that kind of decision — can be quite a lot harder to do than simply throwing them on there and being done with it.

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 6:14 UTC (Fri) by blujay (guest, #39961) [Link] (12 responses)

Your comment presumes that the person in question "screwed up" and needs to "do better" in the first place. I see no such thing. What I see is an argument based on principle, one which many people not only do not seem to comprehend, but become actively angry about. The response is then equivalent to a drive-by, "la la la, I can't hear you!" In contrast, to "face someone down and tell them why they've screwed up badly" would mean engaging in a rational discussion.

The accuracy of the principled argument is made quite clear by LP's "wakeup call" to Gentoo about udev. Whether or not they are actively trying to take a dominant position in the Linux "ecosystem" to force changes downstream, the end result of their "we broke it, you fix it" attitude is the same.

And I'm mystified as to why so many people don't understand this. Without a cooperative, you-scratch-my-itch-and-I-scratch-yours attitude, the whole free software ecosystem and community falls apart, or at least splinters into incompatible sects.

What is the underlying issue? Is it a fundamental lack of respect for others, a selfish refusal to accommodate others' needs and desires? That's how it looks to me, and if so, it will eventually come back to bite these people when they have conflict among themselves, when they eventually have divergent needs or desires.

Another possibility is a simple lack of patience. How long have distros been using non-systemd tools successfully? Why do these changes which so many people oppose have to be forced through NOW? The "we have to do SOMETHING, and we have to do it NOW" attitude is responsible for a lot of conflict and mistakes and problems throughout history.

It strongly resembles politics: people suddenly get the idea that "something must be done," and so they try to ram something through before they run out of political capital (or before their party is out of office), even if it's not a good solution or if many people are opposed to it.

And it resembles politics so much that I think it strongly suggests that, behind the scenes, this whole conflict is primarily not a technical one. Good technical solutions sell themselves, especially in FOSS. For this to be forced through in spite of such opposition suggests a lot about the motives of those pushing it through.

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 13:52 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (10 responses)

> In contrast, to "face someone down and tell them why they've screwed up badly" would mean engaging in a rational discussion.

And they feel like this has been done already with no change whatsoever.

> The accuracy of the principled argument is made quite clear by LP's "wakeup call" to Gentoo about udev. Whether or not they are actively trying to take a dominant position in the Linux "ecosystem" to force changes downstream, the end result of their "we broke it, you fix it" attitude is the same.

I read that as more of "we aren't going to maintain this old code [netlink] anymore, it are going to use kdbus instead; if this bothers you, do something about it".

> Why do these changes which so many people oppose have to be forced through NOW? The "we have to do SOMETHING, and we have to do it NOW" attitude is responsible for a lot of conflict and mistakes and problems throughout history.

They don't have to be done now; Debian is already 3 years behind other distros in adopting it. But waiting *another* 3 or 4 years to do it is probably not in their interest either. If Debian does find issues in it, don't you think it'd be easier to change now than in 2017 when other distros have had 6+ years of experience with it?

As stated before, there has been no active breakage of non-systemd init systems (at least that anyone has bothered to point out rather than just handwave about it) and as long as folks who don't want to use it contribute patches so that things still work (who are also best to get such things anyways), nothing is stopping you except, apparently, that you think the maintainers should have to keep N installs to test their packages with their init systems. *That* is forcing work on maintainers.

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 22:06 UTC (Fri) by dlang (guest, #313) [Link] (9 responses)

>> The accuracy of the principled argument is made quite clear by LP's "wakeup call" to Gentoo about udev. Whether or not they are actively trying to take a dominant position in the Linux "ecosystem" to force changes downstream, the end result of their "we broke it, you fix it" attitude is the same.

> I read that as more of "we aren't going to maintain this old code [netlink] anymore, it are going to use kdbus instead; if this bothers you, do something about it".

Well, that's a large part of the problem. Deliberately breaking existing users because you don't want to bother maintaining the code any longer.

This is the opposite of the stable external API guarantee that the kernel team maintains where they only remove something if nobody is using it or if it's got serious security holes they can't fix.

This is the attitude towards compatibility that makes people grumble and declare that Linux will never be a serious desktop.

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 22:40 UTC (Fri) by anselm (subscriber, #2796) [Link] (8 responses)

Is udev's netlink-based communication protocol actually part of an "external API" of udev? Or is it essentially an internal implementation choice, the sort of thing a kernel developer would feel free to change at very short notice if any? Which non-udev programs make use of it?

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 22:47 UTC (Fri) by dlang (guest, #313) [Link] (5 responses)

well, if it's used by things outside of udev, then it's an "external API", and since LP is calling out that Gentoo is going to have to change things as a result, it sure seems like things other than udev are affected.

On the other hand, if you ignore the claims about systemd being modular and consider udev just an internal component of systemd, and users have to go all-or-nothing with systemd, then it could be an "internal API" of systemd.

But that would make the claims that systemd isn't forcing you to use everything a lie.

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 23:05 UTC (Fri) by anselm (subscriber, #2796) [Link] (4 responses)

well, if it's used by things outside of udev, then it's an "external API", and since LP is calling out that Gentoo is going to have to change things as a result, it sure seems like things other than udev are affected

I think Lennart called out Gentoo because they forked udev (IIRC mostly because they didn't like that the udev and systemd repositories were merged). AFAIK the netlink-based interface is what the kernel uses to talk to udev, so once that is replaced by kdbus then the Gentoo version of udev will also have to start using kdbus (either by adopting the userspace kdbus binding that is part of systemd, which does not force you to take on anything else from systemd, or by implementing another kdbus binding), or else the Gentoo people will need to maintain their own version of the kernel side of things, too.

On the other hand, if you ignore the claims about systemd being modular and consider udev just an internal component of systemd, and users have to go all-or-nothing with systemd, then it could be an "internal API" of systemd.

I don't think that systemd's degree of modularity has anything to do with the problem at hand. Udev is really quite independent of systemd, and AFAIK the netlink-based communications protocol is used by udev but by nothing else in systemd. As long as this is only used by udev and the kernel, it remains an internal interface, and it is reasonable to change it if that makes technical sense. Hence my question whether anything outside of udev uses it (and Gentoo's udev fork doesn't really count).

Nobody expects the spanish plonk squad!

Posted Nov 29, 2014 0:39 UTC (Sat) by mgb (guest, #3226) [Link]

> Hence my question whether anything outside of udev uses it (and Gentoo's udev fork doesn't really count).

Will your proposed kernel changes break eudev?

Nobody expects the spanish plonk squad!

Posted Dec 10, 2014 16:32 UTC (Wed) by nix (subscriber, #2304) [Link] (2 responses)

Sorry, last I encountered it, udev was userspace, right? And there are other things that do what udev does (Gentoo's fork, eudev, perhaps mdev?). That makes this a userspace/kernel ABI in my eyes, with multiple users, making breaking it verboten. What's worse they are multiple boot-critical users!

Nobody expects the spanish plonk squad!

Posted Dec 10, 2014 20:53 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (1 responses)

It's backwards. My understanding: the kernel and udev currently communicate over netlink. Newer udev will only talk over kdbus (netlink isn't going away in the kernel) and kdbus requires some userspace process to set up the bus (which systemd currently does). Gentoo can fix this by writing their own kdbus setup tool (which is what the warning was about).

Nobody expects the spanish plonk squad!

Posted Dec 11, 2014 17:04 UTC (Thu) by nix (subscriber, #2304) [Link]

Ah, so this is 'newer udev will require a bleeding-edge kernel yet again', not 'we're going to break everyone using older udev and force them to upgrade'. So I'm not as annoyed as I might be, and Al is probably more annoyed...

Nobody expects the spanish plonk squad!

Posted Nov 29, 2014 0:10 UTC (Sat) by viro (subscriber, #7872) [Link] (1 responses)

You still don't get it. Incompatible changes in udev did happen. _Exactly_ because somebody thought that there is such a thing as application-private kernel API. And it was a fucking disaster.

These days any kernel developer who would even try to pull off something like that would get painfully educated on the reasons why You Don't Fucking Do That(tm).

It doesn't matter how many programs use it. One boot-critical is more than enough. Quite a few of us avoided udev for years after that stunt, exactly because it made our lives very, very unpleasant *and* udev developers tried to pull off that kind of indignant "but it's just a private API, how can you say that we don't get to change it at whim?" shit, making everyone suspect that they fundamentally didn't get it.

We can keep several kernel images and boot one of them. Easily. We can not do the same with the userland programs. And you don't get to assume that your Great! Shiny! Change! renders all previous history instantly obsolete and uninteresting.

Nobody expects the spanish plonk squad!

Posted Nov 29, 2014 3:15 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

As I understand, nobody is going to rip off the netlink API from the kernel. They simply plan to rip it from udev. Your old boot system will still work.

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 15:58 UTC (Fri) by foom (subscriber, #14868) [Link]

> Another possibility is a simple lack of patience.

Well, debian was patient -- and therefore skipped the whole upstart dead end. At some point the advantages of the new system outweigh the desire to wait and you take a risk with moving.

I've been waiting for a less sucky init system since I became aware of the possibility of something better when launchd came out in 2005. Isn't it time YET?

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 15:15 UTC (Fri) by ksandstr (guest, #60862) [Link] (4 responses)

>(...) it gives the plonk-ee at least the courtesy of knowing they've gone off-course far enough to provoke such a reaction, (...)

However, the original plonker up there only came, plonked, and then vanished. No reason was given, not even a cursory "I disagree, but cannot articulate my reasons"; and clearly there's at least one poster, you, reading it in some other way.

>(...) and leaves them an opening to do better in future.

That's exactly what a killfile doesn't do.

>Staying to face someone down and tell them why they've screwed up badly enough to end up on one's killfile — under the aggravation that leads to that kind of decision — can be quite a lot harder to do than simply throwing them on there and being done with it.

And a throwaway display of passive-aggression is more difficult and time-consuming than simply plopping them in there and being done with it, sans comments. Minimization of effort is clearly not the goal here, and neither is avoidance of posting under aggravation -- anyone can go cool down for about an hour anytime they've got occasion to be reading LWN comment chains.

Nobody expects the spanish plonk squad!

Posted Nov 28, 2014 22:24 UTC (Fri) by mgb (guest, #3226) [Link] (3 responses)

> However, the original plonker up there only came, plonked, and then vanished.

SD believers have been ad-homineming the post you refer to as a plonk for more than two weeks now without actually addressing the issues.

It has been fascinating watching you echo each other but why should any of us experienced software professionals waste LWN bandwidth responding to you?

http://lwn.net/Articles/620159/

Nobody expects the spanish plonk squad!

Posted Nov 29, 2014 16:29 UTC (Sat) by ksandstr (guest, #60862) [Link] (2 responses)

>It has been fascinating watching you echo each other but why should any of us experienced software professionals waste LWN bandwidth responding to you?

The short version: because failing to respond displays an outright absence of professionalism regardless of supposed degrees of experience. Every reader can confirm the former with his/her own eyes, whereas the other is a matter of taking someone's word for it.

The long version: assuming that it's at all possible, the anti-systemd concerns can be answered with a reference to a FAQ document where previous answers are listed. If those concerns aren't in the FAQ, an answer should be written and then added to the FAQ. Doing this would take as much time and effort as answering someone who isn't supposedly a nobody, and makes the pro-systemd position more solid. If it isn't possible to answer a particular anti-systemd concern, then there must be a proper argument for why that is so. The definition of proper argument excludes Mr. Pöttering's "systemd myths" page, his "claim to sacred victimhood" G+ posting, and various journalistic threat narratives concerning the Devuan (nee debianfork.org) project.

This'd move the argument into one of dueling FAQs, so I'll jump the gun on the next salvo with this <URL:http://judecnelson.blogspot.fi/2014/09/systemd-biggest-fa...> list of fallacious pro-systemd arguments and the counterarguments demonstrating them as such. Note that the document was first posted in September 2014 which makes it two months old at this time.

However instead of a civilized discussion we've seen various forms of playing to the crowd, ad-hominems, discussion-stoppers, and outright playground-style sneering. "Neener-neener, aren't you just a butt-hurt neck-beard troll! Ban his stupid ass. I'm being victimized by white males in their 30s and 40s, and I'm not going to engage in any further discussion about this post. lololo." On the pro-systemd end there are people, even Debian developers, who'd rather be publicly seen throwing their toys out of the pram than engage criticism at a level that excludes dogma.

It's my sincere opinion that this discussion must move forward regardless of the non-arguments in their various forms that we've seen so far, and especially despite the "this matter has been decided, please disperse, move along nothing to see here, go back to your homes" theme of article seen in e.g. the LWN headline two weeks ago. As it stands right now the discussion is lagging the impact of systemd and its tentacles by about three years: even commenters on a cutting-edge Linux news site such as this right here will rehash "just an init system", "socket activation makes it worthwhile on its own", "modular architecture means it isn't monolithic", and other demonstrably ill-founded quips.

I'm not one to put my faith in credibility games; they only exist to make perceived truth conform to the best player's ideology and not to empirically verifiable reality and logically equivalent derivatives therefrom. However, when open source luminaries such as Bruce Perens are getting on systemd's ass about not just technology but also communication, then it's certainly time to stop, take a deep breath, and consider just what the hell's going on outside of the pre-chewed conclusions which the pro-systemd camp is offering.

Nobody expects the spanish plonk squad!

Posted Nov 29, 2014 16:49 UTC (Sat) by mgb (guest, #3226) [Link] (1 responses)

Thank you for putting the time and effort into an exemplary post. I agree we must move forward but respectfully disagree on the details.

The opportunity for preventing harm to Debian has passed. It is now time to code around the systemd blockage - whether in blends or derivatives or forks or Gentoo/Funtoo or the BSDs.

And exploring the various technical alternatives is *much* more interesting than trying to convert systemd believers.

Nobody expects the spanish plonk squad!

Posted Nov 30, 2014 1:43 UTC (Sun) by dlang (guest, #313) [Link]

> And exploring the various technical alternatives is *much* more interesting than trying to convert systemd believers.

I don't think anyone is trying to convert systemd believers to prefer anything other than systemd.

The only think I am seeing is people asking the systemd believers to be more accepting to other users, including ones who don't want to run systemd, or who have a need to not always run the latest (which includes kernel developers who are developing the next, because they need to compare to older versions)