Verify the identity of developers [LWN.net]

Verify the identity of developers

Posted Mar 30, 2024 15:39 UTC (Sat) by marcH (subscriber, #57642)
In reply to: Verify the identity of developers by epa
Parent article: A backdoor in xz

> ... and accepting the fact that we will never have enough reviewers to check every commit, ...

This is the main problem right here and the "solution" is to just SLOW DOWN. No xz maintainer? Fine, no xz software at all. The world will keep spinning.

This site is never short of very valid criticism of greedy corporations exploiting exhausted maintainers. But sometimes it would also be good to look in the mirror and at the attitude of some maintainers. Scratching your own itch and sharing your random and insecure musings on the Internet is great. But when that pet project becomes popular hubris kicks in. Then comes the desire to become popular and successful too and to avoid a fork at all costs: someone else more "active" and more lax could fork and reap all the fame! Quick, quick, let's merge all these great new features that I have barely the time to look at. Who knows: someone somewhere could be interested in them?

The number 1 job of a good maintainer is to say "no" or even (the horror!) completely ignore some ideas and submissions. The Linux kernel is not perfect but actually decent at this. The rest of what runs on a Linux desktop... not so much.

It's not just corporations: many consumers, developers and generally _people_ generally don't want to pay for quality. We just want more and more for less and less money. With just one exception: security. Because a device that grants criminals access to your data and passwords is much scarier than a device that stops working. So security has been saving quality and it will keep slowing things down. That's a good thing.

Verify the identity of developers

Posted Mar 30, 2024 15:53 UTC (Sat) by pizza (subscriber, #46) [Link] (3 responses)

> This is the main problem right here and the "solution" is to just SLOW DOWN. No xz maintainer? Fine, no xz software at all. The world will keep spinning.

Ok, so widely-user xz stops being maintained. The amount of collective effort to remove xz from production dwarfs the effort of maintaining xz. By multiple orders of magnitude.

Meanwhile, as this mess demonstrates, "maintained" doesn't mean it can be trusted.

Verify the identity of developers

Posted Mar 30, 2024 17:04 UTC (Sat) by marcH (subscriber, #57642) [Link] (2 responses)

> Ok, so widely-user xz stops being maintained.

This already happened a while ago but that wasn't my main point. My main point is: how was 20% better compression enough to "lure" projects into adding some bad dependency? _This_ is when projects should "slow down" and take the time to weight all the pros and cons, including future maintenance headaches
https://cacm.acm.org/practice/surviving-software-dependen...

> The amount of collective effort to remove xz from production dwarfs the effort of maintaining xz. By multiple orders of magnitude.

This is comparing apples and oranges because it's not the same people. There's no "collective" either: each project should manage its dependencies independently.

I'm surprised that switching to a different compression is so expensive because many projects offer such a choice at configuration time which hints at comparable APIs.

> "maintained" doesn't mean it can be trusted.

Indeed but "unmaintained" cannot be trusted _for sure_. Same as testing: it can only "prove the existence of bugs, not their absence". Quality is not black or white.

> Meanwhile, as this mess demonstrates,

If anything, this mess demonstrated a lack of maintenance.

Verify the identity of developers

Posted Mar 31, 2024 21:19 UTC (Sun) by calumapplepie (guest, #143655) [Link] (1 responses)

> My main point is: how was 20% better compression enough to "lure" projects into adding some bad dependency?

Because it wasn't a bad dependency, and 20% is a big improvement. xz has (had?) a website, multiple contributors, maintainers, and more. There were heated debates over the advantages of it compared to other tools. Eventually, it gained wide use, because 20% faster downloads, 20% less storage, 20% lower bandwidth fees, and a 20% better algorithm was an upgrade.

> This is comparing apples and oranges because it's not the same people. There's no "collective" either: each project should manage its dependencies independently.

Then we will have 60% of projects continue using XZ for years after it is abandoned. Besides, the idea of projects as a bunch of little silos that all know each other and work together isn't realistic; lots of contributions are drive-by, and lots of contributors are involved in many projects.

> If anything, this mess demonstrated a lack of maintenance.

XZ was and is maintained. It was subjected to continuous fuzzing as a part of Googles OSS-fuzz. It was vendored into the Linux kernel, an "essential" part of Debian, and more. There isn't a maintainer on the planet who would have rejected its inclusion because of reputational fears.

Do we need more auditing? Yes. Maybe some clever person can think of linker patches that will catch this particular exploit technique, or maybe we need to have more people running with the options that exist to catch it, or maybe even someone should write some automated tests. But rogue maintainers was and will remain a problem.

Verify the identity of developers

Posted Apr 1, 2024 0:04 UTC (Mon) by marcH (subscriber, #57642) [Link]

> xz has (had?) a website, multiple contributors, maintainers, and more. [...] XZ was and is maintained.

Did you read the news?

> Then we will have 60% of projects continue using XZ for years after it is abandoned. Besides, the idea of projects as a bunch of little silos that all know each other and work together isn't realistic; lots of contributions are drive-by, and lots of contributors are involved in many projects.

I don't get the point you're trying to make here.

In any case adding and managing dependencies is a benefit-risk assessment and it's up to each project to make its own decisions in a decentralized manner; there is simply no alternative besides not making this assessment.

"Decentralized" does not mean "in a vacuum": more popular dependencies are of course more likely to be picked up or forked by someone who really needs them if the current maintainer(s) fail.