Trust and mirrors

A recent look at attacks on package managers has much of interest. None of the attack methods are particularly new at some level, but applying them to the update process is. When the mechanism that is used to keep one's system updated with respect to security vulnerabilities is itself susceptible, it is definitely worth a look.

Much of the problem stems from the fact that many community distributions rely on volunteer mirrors to distribute updates. These mirrors could be malicious which would allow them to distribute bad code to systems that are checking for updates. In addition, mirrors are perfectly placed to notice which machines are updating for particular vulnerabilities—information that could be used in attacks.

The study looked at ten of the most popular Linux and BSD package management systems and found all of them to be vulnerable to one or more of the flaws they identified. Package managers track metadata—information about what package versions and dependencies there are—as well as the packages themselves in formats like .rpm or .deb. Typically, the packages are cryptographically signed (using GPG for example) so that they can be verified as genuine by client systems. Some package managers also sign the metadata, but some do not, which allows for additional attacks.

The biggest issue with mirrors is the information that they gain. When a client requests a certain package, it is pretty easy to guess that it is probably vulnerable to whatever security flaw is being fixed in that new package. A malicious mirror—or one that has been subverted—could try to attack the client machine via the flaw being fixed. A suitable vulnerability could be used to completely compromise the client machine.

Once a particular chunk of data, either package or metadata, has been signed, it is valid more or less forever. This can be used by malicious mirrors in two ways: serving up old metadata that points clients at known vulnerable package versions or serving up old packages that are known to have flaws. In both cases, it is a kind of "replay" attack, using old, valid data for malicious purposes.

In most cases, package managers will not downgrade to previous package versions unless explicitly instructed to, so machines that have already upgraded are not generally vulnerable to a package replay. However, if a client reliably contacts a particular mirror for metadata, that mirror can continue serving an older version until an exploit of interest comes along. By knowing that the client has not upgraded—because it has been held back by the mirror-served metadata—an attacker can exploit the newly-discovered vulnerability at their convenience.

Mirrors can also perform "endless data" attacks where the data transfer for the package or metadata is never terminated. The mirror keeps sending more and more data until it fills the client disk. This is likely to "only" cause a denial of service on the machine that is being updated, but that can still be a serious result, especially when the update process is automated.

Unsigned metadata can allow for several other kinds of attacks. Manipulating the dependencies that are provided or needed by a package can lead to various kinds of problems. A dependency on a non-existent package will stop the update from happening, while a dependency on a package of the attacker's choosing can lead to complete compromise.

There is not a lot that can be done to solve the information gathering problem. Subscription-based distributions generally provide their own servers and do not rely upon mirrors to avoid this problem. For community distributions, there really is no central authority that has the resources to do that. Also, controlling all the mirrors only goes so far; if any are compromised, the same kinds of attacks are possible. Downloading the packages to a non-vulnerable host is probably the best avoidance technique, but is difficult to do in practice.

The lessons from this study are clear. Metadata should be signed and only downloaded from "trusted" servers. If there is a concern about man-in-the-middle attacks, an encrypted connection should be used between the clients and servers with certificates being checked to ensure the connection is going where expected.

In the end, it comes down to trusting the mirrors that one uses. It is not terribly surprising that mirrors can cause these kinds of problems, but the study authors did an excellent job pulling together the different kinds of attacks. The picture that they paint is not particularly pretty, but it is one we needed to see.