Protecting update systems from nation-state attackers

The list of companies that have suffered successful attacks on their update systems is long, Cappos began; it is something that happens all too frequently. Often these attacks are carried out by governments; he listed compromises that have been attributed to North Korea and Russia. The Stuxnet attack exploited the Windows update service as well, he said. Nation-state attackers can launch complex attacks; if you are defending against them, you have to worry about holding off a dedicated team of professionals — the best attackers in the world — who command massive resources and who are focused on your company in particular. It is a scary scenario, he said.

It is even scarier when one is dealing with the software that makes a modern automobile run. An attacker who gains the ability to install new software on cars could create no end of mayhem, up to and including large-scale loss of life. Clearly, we all want our cars to be well defended against even the most sophisticated intrusion attempts.

Protecting update systems

There are four general approaches to addressing these attacks. Prevention, obviously, is focused on making attacks harder to carry out. Detection, instead, is about knowing quickly when a compromise happens. The automotive industry, he said, has emphasized these two approaches, but there is far more that can be done.

The third approach is transferring the risk to somebody else, described as "have insurance and claim that all regulations were followed". Transferring risk is not really an option in the automotive industry, though, where the price tag for a compromise can start at $100 million — and go up from there. He cited the case of Fiat Chrysler being penalized $300 million for emissions cheating, even though nobody was directly hurt; imagine what would happen in a case involving a loss of life. Nobody will want to insure against losses at that level.

Finally, there is mitigation — reducing the impact of a compromise, preferably to something close to zero. That is where Uptane comes in, he said. Other update systems "give up everything" in the case of a compromise; Uptane will often be able to hold the line and prevent even a successful attacker from loading hostile software onto an automotive system.

Traditional solutions, he said, do not provide this sort of compromise resistance. For example, one common approach is for the target systems to authenticate the repository using TLS before accepting any updates from there. But that means that the private key has to be on the repository server itself; if that server is compromised, then the game is over.

Another common approach is to sign the updates themselves, keeping the key away from the repository. That requires the client systems to trust the key, which must be used for every update and thus must live on the build farm somewhere. If an attacker gets into that system, they can sign malevolent updates themselves. This has happened to some prominent Linux distributors in the past, he said.

In 2008, Cappos created The Update Framework (TUF) to address software-distribution issues in the cloud. It is now widely used, and has been adopted by the Cloud-Native Computing Foundation. TUF is built on the assumption that people will manage to break into servers and steal keys; its goal is to reduce the amount of damage that can be done when that happens.

TUF is built on the idea of separation of responsibilities. One responsibility, for example, would be to indicate that a given update is valid; that could require developers and quality-assurance personnel all to sign-off on it. But that is separate from informing clients that a new update is available; that is done by somebody else with a different key. In such a system, stealing a single key is not enough to get a client to accept an update. Key revocation is yet another role, separate from the others. The needed keys do not have to (and indeed should not) live in the same place, so multiple compromises would be required to obtain them all.

TUF also has a robust mechanism for key revocation. Cappos asserted that revocation in systems like PGP is "a mess"; it was not designed with any thought about how things should actually work. The same is true of the "bolted on" revocation mechanism in X.509. TUF, instead, had revocation designed in from the beginning. The "implicit revocation" mechanism will automatically revoke keys that have not been explicitly refreshed within a given time window. Explicit revocation of known-compromised keys is also supported as a first-class feature.

Uptane

TUF, though, is intended for cloud distribution; the requirements for automobiles are a bit different. Meeting those needs led to the creation of Uptane as an enhancement to TUF. The code on the server side is identical, he said, but the client side has to be able to work on "tiny embedded systems" that lack direct connectivity to the update server. A typical automobile has a primary client that obtains updates from the server and stores them locally. The other, secondary clients then obtain those updates from the primary.

One cannot assume that the automobile network is secure; indeed, an attacker having access inside the car is a common case. They can get in by way of the car's network connectivity, or just plug into the OBD socket inside the car itself. This sort of compromise is "just going to happen", so Uptane has to take it into account.

Deploying Uptane involves starting with the existing update mechanism used by the manufacturer. A list of update types is created, and a set of signature requirements is created for each type. This can, for example, require signatures from both the supplier and the original equipment manufacturer. The in-car units must then perform the validation of updates to ensure that the required signatures are there.

These units are divided into "strong" and "weak" categories, with only the strong ones having the CPU power to fully validate updates. Weak units perform a lower-level of validation that exposes them to more risk. Still, Cappos said, the system is strong enough to tolerate an attacker having access to a car's network with "no severe consequences". Server compromises can lead to more unpleasant consequences, but nothing that might, for example, lead to a loss of life. The only time that severe problems are possible is when multiple, offline keys have been compromised.

There are multiple open-source implementations of Uptane available. It has now been mandated by several manufacturers, but he was not allowed to name them. It meets or surpasses all of the existing proposals for update security, including upcoming regulations that require compromise resistance. There is a standardization effort around Uptane that is funded by the US Department of Homeland Security, rather than by the vendors. The system has been through a number of security audits as well. Uptane has been integrated with in-toto, a mechanism for supply-chain security that has been adopted widely, including by Debian, Arch Linux, and the reproducible builds project.

This code, he said, can be expected to ship in about one-third of all new cars on US roads in the near future.

Cappos closed by saying that, regardless of the work he and others have done, some groups will use insecure designs and car companies will put lives at risk. Attacks will happen, and appeals to weak regulations for cover will not suffice; people will die and (seemingly worse for manufacturers) big lawsuits will result. Systems like Uptane are meant to prevent that from happening.

[Your editor thanks the Linux Foundation for supporting his travel to the event.]