LWN: Comments on "Devuan, April Fools, and self-destruction"

Devuan, April Fools, and self-destruction

spongy — Tue, 04 Jun 2019 07:32:46 +0000

With all of the suspicious excuses surrounding the initial lockout situation at Devuan, followed by those suspicious April Fools excuses, I could never trust that the Devuan distro was not pwned.

Instead, were I the CSO at any site running Devuan, I would quietly(*) start a migration off of all of that gear running Devuan at my facility. I would install an entirely new distro, one system at a time. And I would start using file integrity systems like Tripwire, integrit or Aide to monitor any and all changes to the release software base of my new distro.

(*) quietly, meaning keep the new file monitoring system absolutely private and quiet. That way, if any insiders unwittingly attempt to poison your new software base, you will be able to identify those persons and deal with them appropriately.

Devuan, April Fools, and self-destruction

rickmoen — Wed, 22 May 2019 08:08:23 +0000

Devuan servers were compromised.

No.

You have long been aware of this claim being flat-out incorrect, but keep repeating it. We were both there. (I'm not a Devuan Project insider, but am a longtime sympathetic participant with no horse in this race otherwise, as you probably recall.) As you are fully aware, exactly zero Devuan servers were compromised. One of the caretaker pretended, as the substance of a meticulously implemented, hilarious, and deeply unwise prank that they had been, and then revealed the prank within the customary one-day period, and then apologised for the unwise choice of prank framing (inside a supposed security breach).

The biggest damage was then done by, to be blunt, you and a few other people who flew off the handle and did the Internet-maximal-noise dance at great length and to stupefying effect. If you were hoping to be thanked for that, I fear you will be a long time waiting.

Rick Moen
rick@linuxmafia.com

Devuan, April Fools, and self-destruction

anselm — Sat, 04 May 2019 00:13:50 +0000

My head is full of WTF that anyone could possibly have thought this a good idea for even a microsecond.

Remember that these are the people who think it's a good idea to fork an entire Linux distribution just to get rid of an inconsequential library they don't like. Personally I'm not in the least surprised.

Devuan, April Fools, and self-destruction

julian67 — Fri, 03 May 2019 02:00:08 +0000

The amazing thing to me is not a technical issue, nor exactly a personal trust issue (though closely related). It's that responsible people deliberately undermined their own brand. I know "anti-corporate this, hacker ethos that" etc. but still Devuan *is* a brand. It started as a response, gained momentum as a cause, established itself by product or actions, and identified itself and invested its ethos in its brand name, Devuan. To deliberately undermine all of this for some self gratification suggests not understanding that "brand" is *not* just corporate language, it is a representation and symbol of ethos and trust built over time. That is a valuable asset, often the most valuable asset an entity possesses.

Anyhow, I'm not a convert. I found systemd painful and frustrating as it got integrated into Debian testing but once it got mature I really have no complaints. As a boring and small time end user I like it. Even though it was a little like learning esperanto at first I am now truly grateful that I will never have to write or understand another init script. I like the logging options too. For what it's worth I run multiple Debian (and armbian) systems, some of which serve as audio and video streamers, and while I'm very happy with systemd I prefer plain old ALSA and an appropriate ~/.asound.conf over pulseaudio, so I think I can claim not to be an undiscriminating consumer of all things Poettering/Red Hat/<insert wicked name here>.

Devuan, April Fools, and self-destruction

nix — Thu, 02 May 2019 13:41:51 +0000

ONE Devuan admin compromised some Devuan servers. All other Devuan admins and devs were locked out, thought the attack was real, reported that Devuan had been pwned, and were doing what they could to isolate other Devuan infrastructure from the compromised systems.

That's something I hadn't grasped, and makes this much closer to an actual insider attack by a single privileged entity. It's as much an attack as, say, an admin wiping systems before he's fired (though a less destructive one). My head is full of WTF that anyone could possibly have thought this a good idea for even a microsecond.

Devuan, April Fools, and self-destruction

ale2018 — Thu, 02 May 2019 12:32:18 +0000

You don't say "the prime of April" in English, do you?

And, I guess, you neither say "first numbers". Hm...

Devuan, April Fools, and self-destruction

rahvin — Tue, 30 Apr 2019 17:07:07 +0000

At best you've got a contribute locking everyone else and all he admins out of the servers without a word. And they left everyone locked out for a significant period of time while pretending it was a real outside hacking event.

You might still trust that person, but anyone who sensibly relies on those services can't trust that nothing it compromised without an outside audit. That's just a rule of the business world. The only question that I see is, Is Devuan a professional distribution with standards or is it a toy where a lockout like this can be undertaken without an external audit to verify it was just a joke.

The person who instituted the joke at this point should not be trusted, they locked out the entire administration staff. Maybe you are having difficulty seeing this outside point of view because you know and trust the person involved. Ask yourself this question:

If you worked at a company with a handful of admin's and one of them locked everyone else out of the servers and pretended they'd been externally compromised for an extended period of time how would that go down? Would the company laugh it off as a good joke?

Devuan, April Fools, and self-destruction

mgb — Tue, 30 Apr 2019 14:57:52 +0000

> it was not a compromise, because the people who openly stated that they took it offline were Devuan's own admins

ONE Devuan admin compromised some Devuan servers. All other Devuan admins and devs were locked out, thought the attack was real, reported that Devuan had been pwned, and were doing what they could to isolate other Devuan infrastructure from the compromised systems.

This continued for 24 hours.

The "prankster" then left the project but Devuan management refused to audit or rebuild the compromised servers and simply declared them uncompromised.

To this date nobody knows whether the "prankster" accidentally or deliberately left any compromises, or whether unrelated black hats were able to gain access during the compromise.

The "prank" was stupid but Devuan could have recovered from it. The management response was inexcusable.

Devuan, April Fools, and self-destruction

nix — Tue, 30 Apr 2019 14:29:18 +0000

You keep calling this a compromise, yet the very article you are responding to says, in the second line:

the Devuan web site looked like it had been taken over by attackers, which was worrisome to many, but it was all a prank

That is to say, it was not a compromise, because the people who openly stated that they took it offline were Devuan's own admins. If you think this was a compromise, then I recommend you go to the island of San Seriffe for your next holiday, because that clearly exists as well. This whole prank was a terrible idea, but I see no more reason to believe Devuan was compromised after it than before it. (I do see it as a reason to believe that Devuan's administrators are not people I would trust to administer a public resource, and thus that it is more likely that it was compromised long ago than I had last month -- but this prank is not itself a sign of a systems compromise happening at the same time.)

Actual compromises require a different set of responses to terribly ill-judged poor jokes, and complaining that post-compromise responses were not implemented in response to a really badly-judged prank is like complaining that release announcements are not properly sent out after a compromise (equally inappropriate, because a compromise is not a software release).

Devuan, April Fools, and self-destruction

amarao — Mon, 29 Apr 2019 22:16:46 +0000

We should fork Devuan as a new anti-systemd anti-joke disto. Boruan?

But I feel that drama disto got it's own queen time.

Devuan, April Fools, and self-destruction

bigon — Mon, 29 Apr 2019 10:07:49 +0000

> You people don't understand the true spirit beyond Devuan.

What I understand is that FOSS is based on trust. Start claiming that your infrastructure has been hacked is a breach of this trust. So, sure, distributions have no or limited legal liabilities, but having users means that you have people who trust and/or rely on your work.

> So much happiness and joy and freedom...

I always considered that there is a difference between freedom and doing irrational stuff...

But in the end I also don't care about Devuan, so....

Devuan, April Fools, and self-destruction

jezuch — Mon, 29 Apr 2019 10:06:48 +0000

There was a time when "hacking" was almost synonymous with "vandalism". This was before criminals realized that there is serious money to be made from it. But in those times "we're the hackers using green monochrome text monitors and have taken over the web to replace it with gopher, as it was always meant to be" would totally be a thing that crackers would put on your vandalized web page. So... Not so obvious. On any date.

Devuan, April Fools, and self-destruction

pizza — Fri, 26 Apr 2019 22:24:59 +0000

...You do realize that the parent article contains many, many links to actual messages posted on the devuan-devel mailing list, and that the drama is still ongoing?

Devuan, April Fools, and self-destruction

roc — Fri, 26 Apr 2019 22:21:51 +0000

Of course, devuan.org explicitly saying "DEVUAN.ORG HAS BEEN PWNED" is entirely different from just "not looking professional".

Devuan, April Fools, and self-destruction

rweikusat2 — Fri, 26 Apr 2019 20:07:23 +0000

This autorepeat-FUD based on nothing but thin air is getting a bit tiresome.

Devuan, April Fools, and self-destruction

mgb — Fri, 26 Apr 2019 19:29:47 +0000

> If your sysadmin was depending on the website looking professional to determine whether the distro was compromised or not, well let's just say you have bigger issues.
'
When a distro's own sysadmins and developers are locked out of their own compromised servers I would say that is not a good sign.

Devuan, April Fools, and self-destruction

perennialmind — Fri, 26 Apr 2019 19:13:02 +0000

There are minimalist websites that I hold in high regard. LWN, for one. If LWN started pulling unprofessional the-sky-is-falling pranks, it would behoove me to rely less on the accuracy of their reporting. Thankfully, when LWN authors poke fun, they do let the audience in on the joke.

Conscientious admins come in small scale shops and solo acts too. Sure, beyond-the-basics, fancy intrusion detection systems are available for those who can devote the necessary resources for defense in depth. But even for them, given the prevalence of scrupulous, painstaking curation in distros like Debian, RedHat, and others, why consider Devuan when such carelessness is on display?

Message matters. The message here: toy, not tool.

Devuan, April Fools, and self-destruction

pizza — Fri, 26 Apr 2019 19:00:27 +0000

It wasn't just a replaced web page. If it was, nobody would have really cared after April 2nd. Instead, a whole bunch of infrastructure was taken offline, and weeks later, at least some of it is _still_ down.

There seem to be two logical explanations:

* They were genuinely (and cleverly) hacked, and are lying to cover it up while trying to restore services
* This was a prank that was made without any heads-up to other core team members, and was taken _way_ too far, to the point where weeks later services still aren't fully restored.

Either way, the way it's been handled does not exactl instil confidence in Devuan's competence or professionalism, and I would expect "Veteran Unix Administrators" to be quite aware that those qualities are high on the list of "reasons to use Distribution X for anything remotely important"

Devuan, April Fools, and self-destruction

nivedita76 — Fri, 26 Apr 2019 17:44:21 +0000

If your sysadmin was depending on the website looking professional to determine whether the distro was compromised or not, well let's just say you have bigger issues.

Devuan, April Fools, and self-destruction

nivedita76 — Fri, 26 Apr 2019 17:40:31 +0000

This is also an interesting attitude to take, because in the real world there are exactly zero operating systems that can prove they have not been compromised. No linux distro gives you any sort of proof that its servers haven't been compromised. If that's the level of trust you need, then you need to use something homegrown, built from source code and with an audit team going over that source code to make sure there are no compromises. I'll bet even the NSA isn't that paranoid.

Devuan, April Fools, and self-destruction

nivedita76 — Fri, 26 Apr 2019 17:36:45 +0000

The point is that in this case it was irrelevant. The prank was perpetrated by someone who ALREADY had full access to the servers. There was nothing to compromise.

Devuan, April Fools, and self-destruction

rweikusat2 — Fri, 26 Apr 2019 17:21:44 +0000

There was no 'event like that'.

A member of the core team of some distribution temporarily replaced a web page/ set of web pages on some set of servers belonging to the distribution. This happened on April 1st, was meant to be an April Fools joke and was pretty clearly recognizable as such due to the nature of the replacement page (efficient text-only gopher vs the bloated WWW being a holy war of the 1990s --- do we perhaps need a warning sign "You may encounter people over 35 here. If they do something you absolutely don't understand, please consider asking about it before panicking and jumping to wild conclusion"?). Revealing this as the joke it was supposed to be ought to be entirely sufficient to 'prove' nothing was compromised here.

Devuan, April Fools, and self-destruction

hunger — Fri, 26 Apr 2019 13:47:08 +0000

... and ci.devuan.org is still down -- one week after the call for the responsible person to resign.

I would expect my distribution to fix central infrastructure first and discuss consequences afterwards.

Devuan, April Fools, and self-destruction

asbesto — Fri, 26 Apr 2019 13:02:14 +0000

...a THOUSAND of them will blossom elsewhere. (This is already happening :) )

All this... for an April's fool. (The BEST April's fool EVER, IMHO)
MIT doesn't teach anything to all of you? In all those years? :)

MEH

You people don't understand the true spirit beyond Devuan.
So much happiness and joy and freedom...

I'm so sorry for that :( Eventually, you will get it. Eventually not. We don't care.

Because Devuan is skyrocketing. Get a life! LAUGH! and CODE! LOVE!

Devuan, April Fools, and self-destruction

mgb — Fri, 26 Apr 2019 12:30:09 +0000

> This seems inconsistent with your position here, that just stopping using it is enough.

Devuan servers were compromised. Devuan devs were locked out. Devuan devs believed their servers had been hacked. Devuan devs took steps to disconnect other servers from the compromised servers.

What happened next was entirely Devuan's choice. They could have sought advice from their lawyers or the police. They could have audited or rebuilt their servers to ensure their integrity. Their choice was to do nothing.

The choice for Devuan users is different. If they no longer trust Devuan they can stop using it. That is the choice we made.

Everyone makes their own choices. Choices have consequences.

Devuan, April Fools, and self-destruction

nix — Fri, 26 Apr 2019 11:42:46 +0000

your best bet is to switch to Microsoft products, as you are likely to be able to get an indemnification agreement

Really? With Microsoft, as with any other large software company, you're as likely to be able to get them to indemnify you against flaws as you are to get them to ship you a live unicorn. Small software companies that you have over the barrel (as by far their largest customer) might be convinced to do it, as the last alternative before an outright takeover, but anyone else? Not a chance.

Devuan, April Fools, and self-destruction

nix — Fri, 26 Apr 2019 11:40:29 +0000

Sorry, wasn't it you who said:

> I know nothing of Italian law but whether or not the incident
> should be referred for criminal prosecution is a question you
> should already be discussing with your lawyers or the police.

This seems inconsistent with your position here, that just stopping using it is enough.

Devuan, April Fools, and self-destruction

nilsmeyer — Fri, 26 Apr 2019 08:56:55 +0000

> Well, demanding that a community project kick out one of its core team and sue him to boot.. is a little excessive. That Bird guy is cuckoo.

People who threaten lawsuits usually can't afford an attorney.

Devuan, April Fools, and self-destruction

nilsmeyer — Fri, 26 Apr 2019 08:55:15 +0000

Or on St. Patrick's Day.

Devuan, April Fools, and self-destruction

mvdwege — Fri, 26 Apr 2019 08:13:47 +0000

Especially on the nose considering Nicosia's 'apology' came down to "I'm sorry you're too stupid to get the joke".

Devuan, April Fools, and self-destruction

sml — Thu, 25 Apr 2019 23:49:33 +0000

> Which raises the question -- For an end-user, what exactly is Devuan's value proposition over Debian?

The major value is that the noisy anti-systemd crowd has removed themselves from Debian mailing lists in favour of their own little echo chamber. This removes a major distraction from Debian and results in more time to concentrate on fixing bugs.

Devuan, April Fools, and self-destruction

roc — Thu, 25 Apr 2019 21:16:39 +0000

It's easy for anyone to see themselves as meek and whoever they don't like as obnoxious. So this is a license to be nasty to whoever you want.

Devuan, April Fools, and self-destruction

roc — Thu, 25 Apr 2019 21:13:58 +0000

It's reasonable to assume that if the site is compromised, there is a significant chance other project assets are also compromised, especially in a small project where the same people likely have access to both.

Devuan, April Fools, and self-destruction

rweikusat2 — Thu, 25 Apr 2019 20:54:39 +0000

Ignorance can't be cured with speculation.

Devuan, April Fools, and self-destruction

perennialmind — Thu, 25 Apr 2019 20:47:10 +0000

A responsible admin might question whether the time of "pwning" coincides with the announcement thereof.

Devuan, April Fools, and self-destruction

edomaur — Thu, 25 Apr 2019 20:40:33 +0000

I understand his point : How can you prove that they are, indeed, figments of imagination ?

The whole point here is that after any event like that, you need to do a security assessment, otherwise how can you be _REALLY_ sure that nothing is amiss ? Today, Linux distros are somewhat central in the Internet world. If one of those is not able to prove that it has really not been compromised, then it is only a toy and not a tool.

Devuan, April Fools, and self-destruction

rweikusat2 — Thu, 25 Apr 2019 20:35:40 +0000

There's no magic connection between servers operated by the Devuan project and installations of the distribution. The only thing a "responsible" admin would need to do here is "don't install software from there until more information becomes available".

Devuan, April Fools, and self-destruction

roc — Thu, 25 Apr 2019 20:13:34 +0000

This is exactly right.

A responsible admin for production systems can't just take the approach "I think this apparent breach is *probably* a prank therefore I am going to do nothing".

A responsible project owner would know this and not create such a dilemma for their users.

Demonizing the concerns of responsible admins by portraying them as the wrong side of the false dichotomy of "passionate hackers working long nights" vs "team of serious white collars in suit and scarf doing 9-to-5" is even more contemptful of those users. (I particularly dislike this trope --- some very skilled, very passionate hackers have families and other responsibilities that make it necessary and right to limit their work hours.)

Devuan, April Fools, and self-destruction

rgmoore — Thu, 25 Apr 2019 19:56:35 +0000

Something can only be considered a prank if the target thinks its funny

I disagree strongly. An important use of pranks is a way for the meek and powerless to puncture the overinflated egos of the powerful and obnoxious. That type of prank is rarely appreciated by its victim, and it can be quite nasty. That doesn't mean it isn't a prank; it's just a mean spirited one.

Devuan, April Fools, and self-destruction

rweikusat2 — Thu, 25 Apr 2019 19:33:03 +0000

I used to be active on the dng list for a few months some years ago (I'm still subscribed to it) despite it was very noisy and very trolly. I stopped this because some of the trolls became seriously aggressive and because of the usual assumptions of bad faith which are routinely lobbed at people using language in a which differs from the way people who consider themselves the embodiment of 'normal' happen to use it[*]. I don't really know if this changed meanwhile but I suspect not. Hence, without doing some sort of analysis of some poster's past communication record, I wouldn't put much trust in the genuineness of any individual message (in the sense of it being an honest attempt communicating something and not some sort of disruptive missile).

[*] Devuan being the last OSS project I cared for beyond "convenient software limestone quarry".