Nightly PyTorch builds compromised

Interesting. My first reaction was to question instead if any package should be pulled in from pypi that automatically.

You won't get that.

Instead you'll get colourful fireworks on how to fix a social problem with technical means :-)

Happy New Year!

What is interesting here is that it it seems the attackers have searched CI logs for that kind of situation...
Using the internet during CI build is always dangerous.

Exactly. Having multiple repositories where the repository name does not prefix the namespace is pretty bonkers. Or better yet, there should be no central repository and just use URLs of each repo as the name. I.e. the Go design is basically right.

Pip does allow matching on sha256sum and failing if it doesn't match. It's however not the default and not exactly user friendly either. There are other options controlling repositories but it's not very helpful.

Python package repositories weren't created with an actual design, so this kind of thing wasn't really considered.

There isn't much in the way of control of precedence in pip, unfortunately. It's a requested feature [1] but there has been relatively litttle work to make it work. PyPA does define a standard (.pypirc) for configuring indexes that gives very good control over precedence, but pip has zero support for it (honestly I don't know what *does* support it)

The gold standard (imo) for avoiding this kind of mistake is to set up your own index that is capable of falling back onto PyPI, and use `--index-url` instead. One of the pip maintainers publishes the tool `simpleindex` [2] for doing this, letting you specify explicitly which packages you want from your own index, and falling back to PyPI for the rest. There's also `devpi` [3] but it's substantially more complicated to operate.

Honestly, it feels like a huge mistake for pip to keep the `--extra-index-url` feature. It's hard to use safely and I think a big reason that pip hasn't grown a better way to do it is because it's "good enough" if you're willing to overlook the massive attack vector it brings along for the ride with any internal packages.

[1] e.g. https://github.com/pypa/pip/issues/6045 and https://github.com/pypa/pip/issues/4263
[2] https://github.com/uranusjr/simpleindex
[3] https://github.com/devpi/devpi

This would most likely work just fine. When you add such a repository, be it an APT repo for Debian-based systems or an RPM repository for RPM-based distros such as Fedora/RHEL/openSUSE, you pretty much always add a GPG key that the repo signs it package lists (APT repos) or the packages themselves (RPM packages) with to the list of trusted GPG keys. See e.g. installation instructions on my MKVToolNix home page at https://mkvtoolnix.download/downloads.html#debian

Then it's just a matter of adding a package called "bash" with a slightly higher version number to that repository, and a subsequent manual package upgrade should pick it up.

That being said, it will likely not be installed automatically. In the Debian-based world there's the "unattended-upgrades" mechanism/package that takes care of installing updates automatically. However, it's pretty much always configured to only download updates from specific APT sections (e.g. from the "security" section). Though I'm not sure how easy it is to fake it.

Both apt & dnf will show where packages are downloaded from; therefore you might spot that "bash" is coming from a server you don't necessarily expect it from. It might also just be overlooked if the number of downloaded packages is big.

It's minimally harder to set up an APT/dnf repository than it is to provide a malicious shell script & a sudo-curl-bash one-liner. But there's no real security there.