Securing Git repositories with gittuf

The so-called software supply chain starts with source code. But most security measures and tooling don't kick in until source is turned into an artifact—a source tarball, binary build, container image, or other method of delivering a release to users. The gittuf project is an attempt to provide a security layer for Git that can handle key management, enforce security policies for repositories, and guard against attacks at the version-control layer. At Open Source Summit North America (OSSNA), Aditya Sirish A Yelgundhalli and Billy Lynch presented an introduction to gittuf with an overview of its goals and status.

Lynch began the talk with an overview of the state of the world for security tools. He observed that there are many tools used to generate provenance and attestation documents for artifacts, as well as tools to address threats against software at run time. "But we don't often talk about the very first part of this. How do we start securing our source code and source repositories [...] where everything originates?"

More and more frequently, he said, organizations are pulling source directly into continuous-integration / continuous-delivery (CI/CD) pipelines without the kind of protections we have for, say, container images or software packages. But a compromise in the repository itself could snowball into a compromise of everything built from the source code. "So it's equally important to protect and [use] all the same protection mechanisms" for repositories.

Different projects will need different policies, so there needs to be a way to define policies for common scenarios. For example, Lynch said that projects may want to enforce a policy that requires code to be reviewed and pass CI before being merged into a main branch. Applying a tag to the repository might require more stringent policies, since they often correspond with releases and things people are most likely to use. For organizations that have a monorepo, it might be necessary to have a folder-level policy to ensure that certain teams or people review and approve changes to specific parts of the repository.

Current state

Git, today, does not provide these features. Lynch pointed out that Git has "some amount of integrity checks" as well as commit and tag signing, but it is "fairly simple" overall. There are many other operations on Git repositories that need validation beyond what Git provides on its own.

The various forges may layer additional features on top of Git, including security features such as protected branches. But these, he said, "are forge-specific features and not actually part of the repository". Even if a project or organization uses one of the forges exclusively, the question becomes "is that enough?"

Especially when it comes to security metadata you know it's very nice to be able to say "hey, if we have a commit we can associate that with a pull request but how do we actually know that that pull request has gone through all the checks?" And, more importantly, how do we know six months down the line, a year down the line, how can we look back and verify and ensure that all of those checks have happened in the past? Sometimes that can be very difficult.

Even when a forge like GitHub provides more information about how teams have interacted with a repository, that information may not be visible to people outside the organization. Ideally, he said, "we want to get to a state where anyone can verify this metadata". Perhaps just as importantly, how do users verify that policies were enforced for a repository without taking a forge's word for it? This led to thinking about "what sort of security properties do we care about, or might people care about, when consuming Git repositories?"

Goals

Next Lynch moved into a discussion of security goals that have guided the project. First, verification that policies have been followed should be possible by any party, and not just members of an organization or users of specific Git forges. It should also be possible to verify the state of the repository at any point in its development. "It shouldn't just be what is latest, we should be able to go back even to the first commit, ideally."

Signing-key distribution is another hard problem that the project hopes to solve. How do users make sure they're getting the right keys, or which keys they should be checking against? On top of that, "how do we rotate and revoke keys?"

Of course, a security tool needs to be flexible and has to prevent the possibility of internal threats. Lynch said that feature branches might have more lax policies, while main and release branches have stricter policies that require multi-user signoffs to account for possible insider attacks. "We want to make sure that you know whenever a security policy changes, multiple people have to sign off on it. Even if one account is compromised, it's harder to compromise the entire repository."

Finally, any tool that works on top of Git is going to need to provide backward compatibility. "If somebody is using more security tools on top of an existing repo, that shouldn't break the workflow of everyone else using that repo." It has to be possible to adopt stricter security policies "in an incremental way, without having to completely annihilate the history of the repo and all the metadata" prior to using the new tool. All of those goals, Lynch said, brought them to the idea behind gittuf. Here, he handed the talk over to Yelgundhalli, to talk about the project and the implementation so far.

Gittuf

Yelgundhalli said that gittuf takes concepts from The Update Framework (TUF), a Cloud Native Computing Foundation (CNCF) project that provides a specification for securing software-update systems. "It gets a lot of things right in [the] context of handling key distribution, rotation, and revocation" as well as providing a model for delegating trust from one user to another.

Another important concept for gittuf is the reference state log (RSL). This idea comes from a 2016 paper on preventing Git metadata tampering. The RSL is similar to the Git reflog, but "actually embedded in the repository" and "authenticated using signatures on each individual entry". These are implemented in gittuf using a custom git namespace in a separate Git refs under refs/gittuf. The RSL is stored under refs/gittuf/reference-state-log and the policy metadata is stored under refs/gittuf/policy. Every time a reference state changes in Git, new entries are added in these logs. This means that the RSL records not only "the main branch went from commit A to commit B" but also when owners of a repository update policies.

Git servers need not be aware of gittuf, since its metadata is stored under the custom namespace. If the server is gittuf-enabled, then it can perform verification on a change when is pushed to the server. "This is great because now you have the ability to reject changes from making their way to other clients" even if those clients aren't using gittuf. But if they are, "they get the changes on the branch as well as the signed statement" of changes that they can verify.

Gittuf is using the in-toto project to provide attestations, which allows a project to make claims about the software. For example, he said that Git only allows a commit to be "meaningfully" signed by a single person, but using in-toto it is possible to record multiple signatures. It would also be useful for answering other questions like "did this test run, and pass" or other policy questions that are not covered by Git itself or even Git forges. The project is also considering how it can authenticate users who are not using gittuf, so that they are still able record evidence that they have authenticated.

Ultimately, Yelgundhalli said that step one in a project's or organization's security policy could be to require source code to have an attestation from gittuf to verify that it has followed policies and is suitable to be "plugged into other parts of our supply chain". As a CI pipeline receives changes to source code, it could inspect each change to ensure that it meets policy.

With the basics of gittuf covered, Yelgundhalli moved on to a short demo of the tool. This included using gittuf to list policy rules for the main branch of a repository that requires two authorized signatures to push to the branch, and demonstrating what happens when a commit meets or violates that policy.

As expected, if a commit has the two required signatures, gittuf reports success. If not, the gittuf command-line tool will emit an error. The demo also illustrated that using gittuf adds several steps to the process to record and commit the reference log. For those who would like to follow along at home, the project provides a demo repository to test out gittuf.

What's next

After the demo, Lynch took the lead again to talk about the status of gittuf and its roadmap. Currently it is considered alpha status. The project has "only just joined" the OpenSSF as a sandbox project in the Supply Chain Integrity Working Group. So far, the project has been focused on functionality but making it easier to use is on the agenda as well:

There's all these multiple refs that we have to worry about, ideally that should just be one command, and ideally that should have command compatibility with existing Git commands so you don't really have to think about it.

He said that the project is looking at "things like repository hooks" to help automate gittuf operations "so you can just use your normal Git workflow". Another feature that Lynch mentioned for the future is signed pushes, but he said that "is not the top priority" because it is not yet supported by any of the Git forges. The project also has a roadmap that mentions a number of interesting features planned for gittuf. This includes adding support for roles and teams, so that policies can require things like "a change must be signed off by two members of a development team and one member of a security team". And, of course, the project should "dogfood" itself by using gittuf to protect the project's source code thus demonstrating its viability for use with other projects.

During the Q&A an attendee asked about Kubernetes CI. The audience member said that CI for the project cost "between $100,000 and $200,000 a month"; they wondered whether gittuf could allow developers to run CI tests locally and just submit proof that they had been run to save on costs. Yelgundhalli said that it is "something we've been talking about but it's early days". Actually collecting proof that CI jobs had run would involve a lot of moving parts, but it would be possible to allow developers to attest that they had run the jobs locally if a project is willing to extend trust that far.

Even though gittuf is not yet ready for prime time, the tool and thinking behind it show a lot of promise. If the project can build out the planned functionality and improve usability, it may well find its way into securing source code for many projects and organizations. (The video of the talk is available on the Linux Foundatoin's YouTube channel.)

[Thanks to the Linux Foundation, LWN's travel sponsor, for supporting our travel to this event.]