Removal of any signatures system [LWN.net]

Removal of any signatures system

Posted Apr 12, 2018 16:54 UTC (Thu) by sumanah (guest, #59891)
In reply to: Removal of any signatures system by anarcat
Parent article: A new package index for Python

Hi! I'm the author of the article.

I may not have been as clear as I wanted to be. You said "I'm worried that the current rearchitecture just completely drops the PGP signatures altogether" but, in my view, Warehouse does not do that.

Warehouse retains existing signatures for the existing releases it hosts.
Project maintainers can still attach signatures as they upload new packages (for example, with Twine).
Package signatures are still available alongside downloadable files in PyPI's Simple Project API, complying with PEP 503.

Folks other than me can discuss the merits of Python package signing, what ought to be done in the future, and so on. But I hope this clarifies the current situation.

to post comments

Removal of any signatures system

Posted Apr 12, 2018 17:25 UTC (Thu) by mmerickel (guest, #117211) [Link] (4 responses)

So we can sign releases with the keys we already added to our accounts but we can't change, upload or view keys on the web ui. We're stuck with whatever we put into the system before the switch in a couple weeks.

Removal of any signatures system

Posted Apr 12, 2018 18:06 UTC (Thu) by jwilk (subscriber, #63328) [Link]

Wait, there was UI to "change, upload or view keys" in the old PyPI?

As far I could see, the only thing related to OpenPGP key management was a field where you could put 32-bit(!) key ID. But this field wasn't used for anything.

Removal of any signatures system

Posted Apr 12, 2018 18:34 UTC (Thu) by dstufft (guest, #93456) [Link] (2 responses)

This is incorrect.

There were two distinct features that dealt with PGP signatures on PyPI:

1. The ability to associate a PGP key identity with your user account.
2. The ability to upload a detached signature alongside the file during upload, and having that available for download, and displayed in the UI.

These two features were, as far as the code itself was concerned, entirely unrelated. You could (and still can) submit a detached signature using any key that you possess regardless of what PyPI thinks your public key is via the mechanism in (1). The only purpose to the mechanism in (1) is to allow people to discover what your public key is, theoretically to them allow them to validate the signatures if you used that public key.

So what's changed in the new PyPI?

Well (1) is gone, because it is almost entirely pointless. The only way to add or get the information stored in that field is using a HTTP request protected by TLS. If you rely on trusting that field, then the security of the system you've created devolves into trusting TLS, and if you're trusting TLS then you might as well trust TLS for everything and not just one part of it.

(2) still exists and works, you can still sign files with whatever key you possess and upload a detached signature as part of the file upload. The only thing that has changed in regards to (2) is that we will no longer indicate in the UI if a file has been PGP signed or not, nor provide a link in the UI to download it. The file is still available for download (using the URL specified in PEP 503) and it is still possible to upload signatures.

Removal of any signatures system

Posted Apr 12, 2018 18:52 UTC (Thu) by anarcat (subscriber, #66354) [Link] (1 responses)

What's interesting with (1) is that it does make the pypi x509 a trust anchor that can be used to fetch OpenPGP keys that can then be used in a TOFU manner. By removing this, that task is relegated to the keyservers which we know to have hostile material because of their open nature...

So I don't think (1) was completely pointless: more channels to verify OpenPGP keys is always a good thing... I understand you're taking the direction of TUF to fix that problem, but I figured that *removing* the feature wasn't exactly necessary in order to get there...

Of course, maybe it's the reverse and the feature was never implemented in Warehouse in which case I understand better the decision: no need implementing a feature in a "wrong" way that will be implemented in another "better" way later. :)

Thanks for the clarification!

Removal of any signatures system

Posted Apr 12, 2018 19:04 UTC (Thu) by dstufft (guest, #93456) [Link]

> Of course, maybe it's the reverse and the feature was never implemented in Warehouse in which case I understand better the decision: no need implementing a feature in a "wrong" way that will be implemented in another "better" way later. :)

Yea, that feature was never implemented in Warehouse (other than modeling the database tables needed to support it so our auto migration scripts wouldn't try to remove them).

Removal of any signatures system

Posted Apr 12, 2018 17:43 UTC (Thu) by anarcat (subscriber, #66354) [Link] (2 responses)

I understand. I did not quite phrase my comment correctly. :) What I was trying to say is that I'm worried about the trend of removing that certification mechanism while the newer ones are not in place yet. I understand that the system still somewhat works, but it's getting less and less useful and we don't have a good replacement ready.

For example, others have pointed out that the system is already sort of broken if you can't update the keys: for example, my certification key expires once a year and i need to update it in a few places like this to keep it working...

Right now, I feel the only integrity/authentication system that is effectively in place is really HTTPS and the X509 cartel, and the trust that we have in the integrity of the host(s) running PyPI. If any single component in there gets compromised, it's basically game over///

Removal of any signatures system

Posted Apr 12, 2018 18:44 UTC (Thu) by dstufft (guest, #93456) [Link] (1 responses)

> What I was trying to say is that I'm worried about the trend of removing that certification mechanism while the newer ones are not in place yet. I understand that the system still somewhat works, but it's getting less and less useful and we don't have a good replacement ready.

The system still works almost entirely the same as it always has, the only differences are we're not exposing the signature in the UI anymore, and you no longer have a little text field to publish what your public key identity is for a specific user.

We've removed the PGP signatures from the UI in an attempt to de-emphasize them. They are largely pointless in their current implementation because they lack a coherent trust model that applies to the packaging domain. You could build a secure package signing protocol ontop of PGP, but you'd do so by effectively throwing out the WOT portions of PGP. Personally I'd rather remove them entirely, because I think they are 99% security theater in their current implementation, but folks argued against doing that until the replacement was in place, and the code was already written to support them, so I conceded.

As far as the removal of the little text field to publish your public key, that is gone in the new PyPI because it was 100% pointless. I don't even think it was even exposed to end users anywhere, but if it was, you couldn't actually use it for anything. If you're trying to design a secure crpytosystem ontop fo the features that PyPI has, the only point in that field would be to make a HTTPS request to PyPI to ask what the author's public key is so you could verify that the signature was made by an authorized key. However at that point you're trusting HTTPS to tell you who to trust to sign a package, and if you assume HTTPS is not trustworthy, then a malicous attacker could just tell you to use their own key rather than the author's key. So at that point any system which used that, was effectively as secure as relying only on HTTPS.

Removal of any signatures system

Posted Apr 12, 2018 21:43 UTC (Thu) by anarcat (subscriber, #66354) [Link]

In the Debian packaging workflow, OpenPGP certifications are useful. The workflow goes a bit like this:

1. software gets packaged in Debian
2. linting tools warn that PGP signatures could be checked
3. maintainer checks if upstream tarballs have a signature
4. if they do, the public key responsible for the signature is added to the Debian package
5. future updates to the package will verify the tarball with the signature, using a TOFU model

In step 4, it is useful to have the key available from PyPI instead of fishing it outside. Inciting maintainers to publish their keys on PyPI also helps in making that model work.

But again, I understand where you're coming from and I am very thankful and happy for the new PyPI. It seems you have done an awesome job with a huge project, and I didn't mean to nitpick on this pet peeve of mine. ;) So: congrats, and I'm curious to see what TUF for in PyPI in the future!

Removal of any signatures system

Posted Apr 12, 2018 17:59 UTC (Thu) by mikemol (guest, #83507) [Link] (2 responses)

If I read that Simple Project API documentation correctly, those aren't signatures, those are hashes. There's no key ID, so there's no way to tie an identity to the hash.

Don't get me wrong; the hashes are useful and carry value, but they're not _signatures_, they're just tamper checks.

Removal of any signatures system

Posted Apr 12, 2018 21:36 UTC (Thu) by sumanah (guest, #59891) [Link]

mikemol, thanks for the heads-up. That's not clear enough and we need to fix the docs. The signatures are available, but the API response does not specifically mention the signature filenames and the user has to concatenate a .asc suffix onto a filename to check whether that signature exists. So, we host the signatures but don't make it easy to retrieve them or check whether a particular release is associated with a signature or not. This is, I will admit, very inconvenient for people who want to check for signatures.

Removal of any signatures system

Posted Apr 13, 2018 22:33 UTC (Fri) by sumanah (guest, #59891) [Link]

Fixed the docs. Thanks again.

Removal of any signatures system

Posted Mar 10, 2024 22:23 UTC (Sun) by LtWorf (subscriber, #124958) [Link]

Well this aged like milk since in the end they did get removed.