A backdoor in xz

Source code is defined as the preferred format to use when you want to work on the code in question. I submit that these days the number of upstream authors of nontrivial packages whose workflow consist of "while buggy: edit compile debug", followed by "edit version# && make clean && tar cfz && upload", is essentially zero.

Thus it seems like a good idea to use the code that's actually checked into upstream's version control as a basis for automated building of a distribution's binaries *and* the sources it needs to provide, for copyleft-right-and-center reasons if nothing else. Instead of ignoring upstream's git archive and basing the build on some unsigned tarball somebody created somehow.

Mostly-seamless conversion from upstream-plus-patches to Debian-branch-of-git-archive and back is pretty straightforward these days, thanks to dgit and related tools.

It's tricky, though, because one of the ways to get more quality is to realize that some of the foundations are shaky and built on a bunch of poorly-thought-out principles. But if you rebuild them, that is a form of speeding up, since it requires changes by everyone else to move to the new thing.

I use a variety of software on a daily basis that, so far as I can tell, has worked correctly for years. In some cases I've been tempted to adopt it and maintain it as part of that effort in slowing down and improving quality. But probably 95% of the time when I look at the source to some old program that I have come to rely on, it's written in janky C with global variables all over the place, static buffers, no comments, and a code flow that I find difficult to follow. I have rescued some of my own software from such a state through the power of sheer embarrassment, but I've yet to have the oomph to rescue someone else's software.

We have an enormous maintenance problem, and I'm not sure what slowing down and writing less code with more quality looks like in the face of that maintenance problem. It's one of the reasons why I'm somewhat sympathetic to the folks who would prefer to rewrite the world. It looks like speeding up and writing more code, the opposite of what you're correctly advocating, but the advantage of a clean slate is that it's a lot easier to add standards for slower work and higher quality when starting from scratch than it is to retrofit them to an existing community, or even an orphaned code base.

(And this is all apart from the fact that slower and higher quality is more expensive, and we lack any effective mechanism to fund resilience, sustainability, and going slower. Not just in software, but in most human endeavor. Most social and political forces are pushing in exactly the opposite direction, as you correctly note.)

Would for example Kenyan devs be “Western” enough for you?

I live in a country that is theoretically a democracy but has something called "National Security Letters" that are substantially similar to this description, which makes me dubious the line is quite this clear-cut.

Huge scandals? Crypto AG, by the time it came out, was a minor historical note. I'd bet most Americans would consider it a good move that let us spy on foreign enemies, and many wouldn't bat an eye at the amount of spying done on allied powers. The NSA stunt annoyed some in the field of computer security, but few outside it.

> In dictatorships such stuff is business as usual.

I'm not sure. China isn't going to play around with Dual_EC_DRBG; you use their keys for SSL or else. They openly put taps in, and wouldn't bother with something nobody is going to trust. This skullduggery is part and parcel of Western intelligence agencies.

> Developers do not automatically risk jail or death or worse if they refuse a "proposition" from the intelligence agency of a democracy.

https://arstechnica.com/tech-policy/2013/04/wikipedia-edi... There's a risk of jail time. Nor do I think US intelligence agencies are above blackmail, though the CIA might reserve that for people outside the US.

> Nothing's black and white here but there can be a huge differences between "light grey" and "dark grey"

Yes, but at the same time, intelligence agencies, democracy or otherwise, are very clandestine and have a history of this type of stuff.

As another note, a new developer from the Western world could be a genuine newbie, but could also be someone who found a job posting on the dark web when looking to buy the "Easy Home Kit for Cooking Meth", and now gets paid for his identity, quite possibly with no idea who is paying him.

> Yet Western business is nowadays required to distance itself more and more from powerful dictatorships which is costing a LOT. Guess why.

More and more? If we're talking about China, it's got Cold War parallels, two large powers fighting over control of the world. Russia has even more Cold War parallels, and while pulling out of Russia is bad for business, so is letting Russia take over Eastern Europe. There's moral elements, and fears that Chinese products have backdoors. I'm really not sure what you're getting at here. I certainly don't see Western business having much trouble dealing with dictatorships that haven't pissed off the West.

This already happened a while ago but that wasn't my main point. My main point is: how was 20% better compression enough to "lure" projects into adding some bad dependency? _This_ is when projects should "slow down" and take the time to weight all the pros and cons, including future maintenance headaches
https://cacm.acm.org/practice/surviving-software-dependen...

> The amount of collective effort to remove xz from production dwarfs the effort of maintaining xz. By multiple orders of magnitude.

This is comparing apples and oranges because it's not the same people. There's no "collective" either: each project should manage its dependencies independently.

I'm surprised that switching to a different compression is so expensive because many projects offer such a choice at configuration time which hints at comparable APIs.

> "maintained" doesn't mean it can be trusted.

Indeed but "unmaintained" cannot be trusted _for sure_. Same as testing: it can only "prove the existence of bugs, not their absence". Quality is not black or white.

> Meanwhile, as this mess demonstrates,

If anything, this mess demonstrated a lack of maintenance.

What does "verified" mean in this context?