Git evolve: tracking changes to changes

Some patches are applied to a project's repository soon after being written, but other take more work. Consider, for example, support for stackable security modules, which has been through (at least) 38 revisions over many years. If and when this work lands in the Linux kernel mainline, it will bear little resemblance to what was initially posted years ago. Each revision will have undergone changes that will have rippled through much of the 39-part patch set. Git can support iteration on a series like that, but it can be a bit awkward, leading many developers to use other tools (such as Quilt) to manage in-progress work.

Commits, meta-commits, and changes

The proposed evolve functionality for Git adds a new level of tracking for "meta-commits", which can be thought of as versions of a specific commit. The meta-commits describing the history of a given commit are stored in a special branch that is called a "change". The documentation tosses the "meta-commit" and "change" terms around almost as if they were interchangeable and, for the most part, they can be thought of as the same. Meta-commits simply hold the history of a change — the evolution that a given commit has gone through over time.

Consider an extended example: if a developer does some work and commits it, the result will be the new commit itself (identified by its hash — we'll call it A in this case) and a meta-commit stored in a new change branch with a name like metas/mc1 (the naming of changes is a subject of its own, with the obligatory hook so that users can add scripts to generate their own names). The result is a structure that, given the limits of your editor's diagramming skills, can be represented as:

Here we see the new commit A on the trunk branch in the local repository; the change branch metas/mc1 contains a meta-commit with reference to the hash of that commit.

Now imagine that this commit, like many, is not perfect in its initial form; it will need to be improved. If, later on, this developer uses a command like git commit --amend to change this commit, Git will update the metas/mc1 change to refer to the hash of the amended commit (B here), but also to note that this commit "obsoletes" commit A:

The old commit A will remain in the repository and can be consulted if, later, somebody wants to see what changed between A and B.

If our developer adds another commit C (without --amend) to the same branch, the result will be another change branch, call it metas/mc2, referring to this new commit:

A large patch series will thus have a number of active change branches, one for each commit in the series. Notably, the mechanism described above ensures that the change name for each commit in the series remains stable, even as the commits themselves are changed. The first commit in the series is metas/mc1, even as that commit itself evolves over time and its hash changes. There is a set of commands to list the known change, and a simple git reset or git checkout command can be used to reset the branch to a given change.

Now suppose that commit B needs further changes; our developer has inexplicably forgotten to use reverse Christmas-tree ordering for their variable declarations and has been called out on it. They can use git reset to go back to that commit — the one described by metas/mc1 — to fix this unacceptable state of affairs. A bit of editing and a new git commit with --amend will yield a new commit D, and metas/mc1 will be updated to reflect the fact that commit B has been obsoleted.

The first commit in the series has been updated, but now our second commit in the series (C), the one described by metas/mc2, still has the old commit B as its parent, so the sequence has been split. If the developer now runs git evolve, though, all changes that were based on metas/mc1 (in any version) will be rebased, recreating the full change history.

The commit formerly known as C has been rebased on top of D, restoring the full patch series. The git evolve command can also be used to update a set of changes to a new base in the repository — rebasing all of the changes to reflect changes merged elsewhere.

More than rebase

Thus, git evolve can be used somewhat like git rebase, but there are some differences. Perhaps most significant is that commits can be modified in various places in the stream, then all evolved together at the end. A developer can, for example, make changes to patches 3, 7, and 9 of a 12-part series, each isolated from the other, then use git evolve to stitch the sequence back together at some future time.

Another difference is that the change history might not be strictly linear. As a simple example, imagine a repository with a single commit; the developer could amend that commit to create a new change, like the commit B shown above. If, then, our developer uses git reset to get back to the pre-amend commit (A) and amends it again, there will now be two changes, each of which obsoletes commit A. The documentation calls this "divergence"; the change history for a patch series can contain any number of divergences and changes built upon them. A divergence could be caused by trying alternative fixes for a problem, for example.

Git will be able to track that divergence indefinitely, but there will come a point when things need to be resolved. For example, if the developer runs git evolve, Git will need to know how to resolve the divergence so that it can rebase the rest of the series. The usual resolution at that point is to do a merge of the diverging changes, but it is also possible to simply pick one side.

Since changes are Git branches in their own right, they can be pushed and pulled between repositories. So developers can share the current state of their work — and how it got to that state — with other developers or with some sort of change-tracking system. Anybody who can access a change can review the various versions of the patch and see the direction in which the work is heading.

Finally, changes are ephemeral, in that they are really only relevant until the work they described is finalized and committed to a trunk branch. At that point, the change is presumably perfect and the story of how it got to its current state is no longer of interest. So, whenever a git evolve command sees that the commit described by a change has been merged, it will automatically delete the changes themselves. So a developer's set of active changes will normally reflect the work that is actually in progress at any given time.

An evolving story

The above description was mostly taken from this document describing the proposed feature. The document is thorough and detailed, but a bit challenging. Your editor only had to read it a dozen times or so, though, to get a superficial understanding of what is going on.

The git evolve patches are not new; indeed, they have been through a fair amount of evolution themselves. An initial design for the feature was posted by Stefan Xenos in late 2018, and the first implementation patches came out in January 2019. The most recent version of these patches, as of this writing, was posted by Christophe Poucet in early October. There has been interest in the patches over the years, but the complexity of the feature also arguably makes it hard for others to properly review.

As a result, it is still not clear whether git evolve will find its way into the Git mainline or not. There are some clear use cases for git evolve, and each version of the patch set has evoked active discussion. Whether the benefits of the feature justify the added complexity will be something for the Git maintainers to evaluate, though. If the evolve functionality can clear the bar, it could enhance Git with features that developers currently must seek in other tools. Some more complexity evolved into Git here might thus simplify life overall.