LWN: Comments on "Arch Linux and (the lack of) package signing"

Arch Linux and censorship

i3839 — Sat, 09 Apr 2011 11:25:42 +0000

Years ago I used to help out people on the Arch forum and even made it to moderator. I'm still using Arch, but you won't find me on the forums any more. There are some very immature people with too much power that love to censor anything they don't like personally. After that happened to me and it became clear that I was the only one thinking that censoring is plain wrong I just left. My post that got deleted was quoting the crap some people were posting on the public TU or AUR mailinglist (I forgot the details). I even left out the personal insults, but they were not amused anyway. As I was a mod at the time, I could probably have restored my post, but it's the principle of censoring stuff you don't like which is very wrong.

I also wrote a patch adding the ability to upgrade only packages that were x days old (the only sensible way to add some stability to a rolling release package system IMHO), and a bunch of buffer overflow fixes for Pacman, but they didn't make it in either as far as I know.

I know others that left the forum/Arch for similar reasons. All very disheartening. Sad that not much changed since I left.

Security of a git tree

jrn — Sun, 03 Apr 2011 07:37:00 +0000

Presumably he has, since git refuses to fetch over HTTP without it.

Perhaps the servers you've been connecting to use the (relatively) new "smart" HTTP support, which negotiates which objects to send using a CGI script.

Security of a git tree

smurf — Sun, 03 Apr 2011 06:17:39 +0000

I don't think you've heard of "git update-server-info".
It creates a few index files which speed up the job considerably.
(It's typically run from the post-update hook in the shared repository.)

Security of a git tree

vapier — Sun, 03 Apr 2011 02:52:38 +0000

i dont think you've ever used git over http. the performance is downright awful for even small repos.

Arch Linux and (the lack of) package signing

Lord_REL — Thu, 31 Mar 2011 20:35:22 +0000

signing support has been added in pacman 3.5.0 and relevant packaging scripts

Security of a git tree

smurf — Wed, 30 Mar 2011 02:08:08 +0000

You don't need a git server for mirriring a git archive.
That works quite well with http.

Security of a git tree

vapier — Tue, 29 Mar 2011 18:34:41 +0000

signing a SHA1 doesnt increase confidence in SHA1 in any way. it's still a SHA1.

you missed the "resources" aspect. high compression means significantly higher cpu/mem usage which makes scaling up much harder. plus, our mirrors now have to run a git daemon to do mirroring ? it just doesnt work out.

as a developer, you can mirror the VCS tree yourself.

Arch Linux and (the lack of) package signing

jschrod — Mon, 28 Mar 2011 16:26:55 +0000

I read both, and I disagree. The article presents one viewpoint and makes that clear. The blog with the disagreeing opionion is linked. The article facts don't seem to disagree with the Arch developer's blog, only their interpretation. (This guy comes away as very very sensitive, by the way.)

For the record: I have never used Arch Linux, don't intend do, and don't know any of the involved persons.

I assume that you are as uninvolved in Arch Linux as I am, otherwise you would have surely noted that when calling for censorship of an opinionated article.

Security of a git tree

nye — Mon, 28 Mar 2011 11:01:20 +0000

>SHA1 is broken WRT collisions, i.e. you can find (with a lot of effort) two "random" bytestrings which hash to the same SHA1.

In principle yes, but nobody's ever actually done it with full SHA1 - until it gets a bit more broken than it currently is, going beyond proof-of-concept attacks on much reduced versions of SHA1 would still require more computing power than is currently feasible.

>But as SHA1 is considered "broken enough" that it should be phased out

True, it would be a bad choice for something new, but things aren't so terribly bad for SHA-1 yet - hell, there aren't even any pre-image attacks for *MD5* yet AFAIK and that's been considered utterly broken for *years*.

Security of a git tree

smurf — Sat, 26 Mar 2011 20:39:19 +0000

SHA1 is broken WRT collisions, i.e. you can find (with a lot of effort) two "random" bytestrings which hash to the same SHA1.

That's not the same as finding a bytestring which hashes to a given SHA1, which is still easier than to find a bytestring with, together with a given ASCII pre- and postamble, will match that given SHA1.

I don't think there's a feasible attack for the latter. But as SHA1 is considered "broken enough" that it should be phased out, AFAIK current efforts on one-way hashes are more focused on trying to break the several candidates for SHA's replacement, than to break SHA1 'even more'.

Arch Linux and (the lack of) package signing

nicooo — Fri, 25 Mar 2011 23:44:51 +0000

I agree. Also, there are some books at the local library that are wrong and the sensible thing for the municipality to do is burn them.

SHA1

alex — Fri, 25 Mar 2011 15:54:49 +0000

I was going from memory but it seems so. Having said that there is a long road from generating collisions to a feasible attack. However from a crypto point of view you are no longer being protected by the maths ;-)

Security of a git tree

foom — Fri, 25 Mar 2011 14:04:02 +0000

> Manufacturing a SHA1 collision is certainly doable.

Really? Isn't it still only theoretical?

Security of a git tree

alex — Fri, 25 Mar 2011 10:10:41 +0000

Manufacturing a SHA1 collision is certainly doable. Doing it in a way that allows you to modify an individual commit in a git tree while also not breaking git's internal consistency with respect to history is an attack I've not seen done yet.

And of course you can sign tags with GPG for extra confidence.

As far as bandwidth and diskspace is concerned it would be worth doing some tests before ruling it out as less efficient than rsync. The over the air update protocol is fairly efficient and the .git directory is often smaller than the checked out tree as it's fairly heavily compressed. Plus of course as a developer it's really useful to have the whole history of an ebuild available to you when diagnosing issues or trying to understand why something was done.

Funtoo

alex — Fri, 25 Mar 2011 10:03:57 +0000

I did look at Funtoo, unfortunately the git repo (or at least the gentoo mirror side) was just a daily snapshot of the CVS tree. That doesn't give you any confidence that the mirror hasn't been compromised.

Really you want each change to the metadata to be a discreet verifiable commit.

Arch Linux and (the lack of) package signing

BradReed — Fri, 25 Mar 2011 09:44:10 +0000

I fully agree with what you said. I know nothing about this first-hand, and have never even tried Arch Linux. I just saw McGee's post on reddit and thought it might be worthwhile to link to it here. It definitely expressed a different view on things.

The truth is probably somewhere in the middle.

Arch Linux and (the lack of) package signing

nonoitall — Fri, 25 Mar 2011 05:42:20 +0000

Kudos to LWN for helping to bring this to light. When I heard you guys had posted this article I just had to subscribe to give a little thanks. :-)

Arch Linux and (the lack of) package signing

nicooo — Fri, 25 Mar 2011 02:53:41 +0000

It seems like there are some issues that still need to be resolved.

Arch Linux and (the lack of) package signing

IgnorantGuru — Fri, 25 Mar 2011 02:46:00 +0000

Due to your curious message, I just found one of Allan's comments in the spam folder - he used so many links Wordpress nailed it as spam. I will restore it. He never informed me of the missing comment, and this is the first time the spam filter has ever nailed a legit comment. My apologies. I do not edit or delete reader's comments.

Arch Linux and (the lack of) package signing

nlucas — Fri, 25 Mar 2011 02:40:15 +0000

I don't think the article is in any way offensive or wrong.
It may be biased because an author not directly related to the story may catch the wrong viewpoint, but that doesn't make it wrong - only out of context when more information is available later.

As it is, even after reading the other side of the story, I don't see anything wrong with it. No impartial person can deny that even if IgnorantGuru may have gone beyond reason, his main point was still valid, and not answered as it should.

P.S. I don't have any reason to think badly of ArchLinux (nor had in the past or will start now). I still think it's an interesting distro, even if I never used it and maybe never will. To me this just means there are times where perfectly fine people just have a bad moment.

Arch Linux and (the lack of) package signing

randomguy3 — Fri, 25 Mar 2011 01:50:58 +0000

Disclaimer: I am a happy Arch user, but have no particular familiarity with the Arch development community.

The impression I get from IgnorantGuru's blog post is of someone who has stamped his foot and yelled and not got his way. I agree with him insofar as I believe that package signing should be implemented, and should be given a higher priority than the pacman developers have apparently given it. However, from what I have read (summaries of the situation by IgnorantGuru and Dan, and some of the bug reports), IgnorantGuru appears to be ignorant (ha ha) of how to go about getting something implemented in an open source project.

Protip 1: simply telling the developers that something is very important for them to do (and even that it's "easy") will not get them on your side.

Protip 2: being antagonistic (which is how IG comes across in https://bugs.archlinux.org/task/23101, for example) will not get them on your side.

In fact, IgnorantGuru's whole approach to this seems to be a textbook example of how not to get a feature you want implemented. He has been antagonistic and demanding, and has failed to get involved in the development process (in the manner generally accepted by the project) while at the same time claiming that implementing signing is easy.

In fact, the main sticking point in this whole issue seems to be that there is no-one who cares sufficiently about package signing that is both able and willing to write and maintain the code, and see the feature through to completion, in a way that is acceptable to the maintainers. And, like it or not, that's what it takes to get something done in a relatively small volunteer-run open source project like Archlinux.

All this is not to absolve Dan and Allan of any wrongdoing, however. But I am not in the least bit surprised that IgnorantGuru has not acheived his goal, given how he went about it.

Arch Linux and (the lack of) package signing

wonder — Fri, 25 Mar 2011 00:19:30 +0000

> Below is my brief reply to Dan McGee. I posted this on his blog but given > the Arch way of doing things, he'll probably just delete it

Look who's talking. the guy who deliberate block Allan's comments on his blog.

Dan would never do that.

Arch Linux and (the lack of) package signing

IgnorantGuru — Thu, 24 Mar 2011 23:52:15 +0000

This is not the Arch Forums.

Arch Linux and (the lack of) package signing

IgnorantGuru — Thu, 24 Mar 2011 23:51:14 +0000

Below is my brief reply to Dan McGee. I posted this on his blog but given the Arch way of doing things, he'll probably just delete it. I notice Arch devs are now attacking LWN and trying to get them to delete their story. What's with these guys? This has been their approach to this issue for years - silence it. I still see no indication that their users' security is of any importance to them. Just ego.

LWN should be applauded for taking the heat for bringing this issue forward with integrity, and not buying the spent Arch dev arguments that no one has been willing to contribute. That is false - I have also heard privately from many devs who told me they also tried to get things done and hit the same brick wall. And I have been thanked by many Arch users for making them aware of this issue. LWN has their priorities right - they are informing their readers of a serious security problem. Silence and censorship is not the solution. Don't shoot the messenger.

As for package signing being 'almost done' - we'll see. They said this in 2008.

My reply to Dan McGee:
http://igurublog.wordpress.com/2011/03/24/lwn-picks-up-on...

Gentoo Mitigations?

blitzkrieg3 — Thu, 24 Mar 2011 23:32:51 +0000

So what? It doesn't mean the packages are signed.

Arch Linux and (the lack of) package signing

giraffedata — Thu, 24 Mar 2011 22:09:38 +0000

Dan McGee's explanation doesn't cover the censorship, except to mention in passing that it happened and calling the triggering post a "rant that looked more like a blog post." IgnorantGuru says that he was told the problem was that the post was "trolling." I read it, and I don't think this moderator knows what the word means. IgnorantGuru also says the moderator said he was warned multiple times, but he doesn't remember being warned.

McGee makes a good case that IgnorantGuru's criticism was unfair, but I'm still troubled by the silencing of it.

McGee's blog post, by the way, shows him to be hypersensitive and highly defensive with respect to the LWN article. The LWN article is unbalanced, but not dishonest. As a neutral reader, I was aware throughout that I was seeing a report of IgnorantGuru's beliefs. The article's wording is careful enough that it is in fact consistent with McGee's version.

Arch Linux and (the lack of) package signing

jiu — Thu, 24 Mar 2011 21:54:48 +0000

I think after reading both this article and Dan McGhee's response, the sensible thing for LWN would be to delete this article altogether. No point propagating baseless accusations of the arch devs.

Arch Linux and (the lack of) package signing

BradReed — Thu, 24 Mar 2011 20:28:10 +0000

You may want to hear the opposing side of the story:

http://www.toofishes.net/blog/real-story-behind-arch-linu...

Arch Linux and (the lack of) package signing

vapier — Thu, 24 Mar 2011 19:48:23 +0000

hmm, should be trivial to start enforcing manifest signing in the main tree. it's trivial to setup keys after all.

$ find *-* -maxdepth 2 -name Manifest | wc -l
14438
$ find *-* -maxdepth 2 -name Manifest -exec grep -l 'BEGIN PGP SIGNATURE' {} + | wc -l
6032

that is fairly abysmal ...

Gentoo Mitigations?

vapier — Thu, 24 Mar 2011 19:47:40 +0000

except for when the Manifest itself is signed

anyone who truly cares about security is not going to rely on SHA1 checksums. claiming "use git" is a fix is ridiculous.

plus, you're now syncing the git tree which balloons overhead considerably since the entire history is mirrored and not just the latest set of files. i'm sure the mirrors giving up free space/bandwidth/resources will love that.

Gentoo Mitigations?

quentin.casasnovas — Thu, 24 Mar 2011 19:35:33 +0000

You may want to take a look at Funtoo, a "Gentoo Linux variant personally developed by Daniel Robbins, creator of Gentoo Linux" : it uses git instead of rsync to update the portage tree.

Arch Linux and (the lack of) package signing

hickinbottoms — Thu, 24 Mar 2011 16:46:35 +0000

> You are many years out of date :)

I suspected as much! However, I looked into this feature today and it doesn't appear to be enabled in the currently marked-stable portage (on my system, at least):

> # emerge ...whatever...
> FEATURES variable contains unknown value(s): gpg
> Portage 2.1.9.42 (default/linux/x86/10.0, gcc-4.4.5, glibc-2.11.3-r0, 2.6.35-gentoo-r12 i686)

Still, as you say it counts for little if most of the software isn't signed by the mai

Gentoo package signing

alex — Thu, 24 Mar 2011 16:39:02 +0000

So I assume if old packages aren't signed portage will either allow them or refuse to install them depending on the level of the feature?

/me makes a note to enable the gpg feature.

Arch Linux and (the lack of) package signing

tetromino — Thu, 24 Mar 2011 15:50:50 +0000

> Unless I'm out of date I believe Gentoo has also always suffered this, and continues to do so.

You are many years out of date :) Gentoo's portage has had the ability to use GPG to sign and verity package manifests since 2004: http://www.gentoo.org/news/20041021-portage51.xml

What is true is that there seems to be no policy requiring Gentoo developers to sign manifests, and as a result, many developers never bother to do so and thousands of packages remain unsigned.

Gentoo Mitigations?

hickinbottoms — Thu, 24 Mar 2011 13:36:00 +0000

Indeed, but of course there's no private key signature element to the manifest, and the manifest is held on the same mirror as the ebuild ('recipe'), which includes the location to download the source from.

This makes it a trivial matter to replace packages without end-user detection if you have write access to a mirror (and there are lots of mirrors). This is the same problem reported in the original article on Arch.

I'm sure you knew this but wanted to point it out since I don't think the presence of the manifest does anything other than to aid detection of corruption of the source tarball during or after download - it doesn't help with indicating package authenticity at all.

I believe there have been discussions over a git-based Portage but it's not got anywhere significant as yet, and GPG-based package signing seems to have been discussed for many years but still isn't stable as far as I can tell.

Gentoo Mitigations?

alex — Thu, 24 Mar 2011 12:28:04 +0000

Gentoo has manifest checksums which need to be updated when the recipe is. This is vulnerable to compromise and of course assumes the developer has checked their copy of the upstream tarball hasn't been comprised.

One mitigation would be move from rsync'ing the portage tree (which I believe is still held in CVS) to using a git tree instead. At least this way you can be sure* the metadata hasn't been tampered with between syncs. Of course you still depend on the developer doing due diligence on the first tree.

I'm not sure if the nature of the distribution (being source based) widens or narrows the attack surface.

* YMMV, I don't believe anyone has managed an attack on a git repo yet.

Arch Linux and (the lack of) package signing

hickinbottoms — Thu, 24 Mar 2011 08:28:41 +0000

Unless I'm out of date I believe Gentoo has also always suffered this, and continues to do so.

Arch Linux and (the lack of) package signing

bo — Thu, 24 Mar 2011 07:38:03 +0000

...and here is the commit log for pacman's master branch grepped for "signature": http://projects.archlinux.org/pacman.git/log/?showmsg=1&...

Note that Allan is the author of a number of these patches.

Arch Linux and (the lack of) package signing

wonder — Thu, 24 Mar 2011 06:44:51 +0000

FYI package signing was merged in master couples of days ago.

Arch Linux and (the lack of) package signing

joey — Thu, 24 Mar 2011 06:10:16 +0000

> Almost all prominent Linux distributions use package signing to allow
> users and system administrators to verify the integrity of packages.
> Typically, package uploaders use a private GPG key to sign the package
> before it is pushed to the release server

That approach is not entirely typical -- Debian and presumably most or all of its derivatives use the second approach, where there's a checksum-based trust path to a central file that is signed.

This broadly reminds me of discussions in Debian about how to provide a trust path to packages. Some wanted to embed the signatures inside each package, while others preferred that a central list of packages in the distribution (and checksums) be signed, and some preferred both. There were plusses and minuses to both approaches (for example, if packages are individually signed by their uploaders, how to handle key revocation? if packages are centrally signed, how to handle the checksum path becoming cryptographically broken?), and implementing both methods and deciding between them took time in which there remained little strong security.

Of course, that was many years ago, and I can't remember anyone doing quite the ostrich imitation described in this article.