Written by dap
on 

My workflow for stacked PRs on GitHub

tl;dr

I like stacked pull requests (PRs), but they’re hard to do on GitHub if you also use squashing to keep a linear history on main. Here’s what I do:

  • I use merge commits in all PR branches when sync’ing up with their upstreams (including main).

  • I do this one weird trick to fix up PR branches after their parent was squashed onto main:

    • First, I make sure the PR is sync’d up with the tip of its parent (the commit that was squashed) using git merge.

    • Then, I sync up with the commit on main prior to the squash commit, also using git merge.

    • Finally, I sync up with the squash commit with git merge -X ours. 😱

  • I check my work by diff’ing the diffs. I’ll do:

    git diff --merge-base origin/main my/branch/before > before.txt
git diff --merge-base origin/main my/branch/after > after.txt
vimdiff before.txt after.txt

    There should be very little different between these. I should only see the result of manual conflict resolution and anything that changed in main in the few lines around something in my changes.

This seems to work pretty well. I only ever have to resolve intrinsic conflicts that I haven’t already resolved. (A confession: actually, I have sometimes been surprised to find other conflicts. I think this is because I’ve messed up executing this procedure, but it’s possible that the procedure itself isn’t quite right. I plan to debug further next time it happens.)

Introduction

I really like stacking pull requests. It helps me keep individual changes small without losing focus on the bigger stream of work. Small changes have lots of advantages: they’re easier to review, they’re easier to reason about in the Git history, they’re easier to backport, they often unblock other work, etc. But it’s also easy to feel blocked on landing each pull request before starting the next one. If you do that, there’s a real cost to keeping changes small. Stacking PRs gets around this: it helps me keep my changes small while also letting me stay focused and keep my pipeline full.

For reasons that don’t seem particularly good, stacking PRs on GitHub is pretty tough, at least if you also care about these other things:

  • keeping a linear history on the main branch (i.e., you use squash commits)

  • keeping PR branches in sync with main, without having to resolve the same conflicts multiple times

  • code review, especially incremental review, and especially the idea that:

    • you might have multiple reviewers

    • they might each wind up looking at different versions of the PR at different times

    • at any time, each of them ought to be able to see the delta between the PR today and what they last reviewed

I don’t care about the tidiness of the commits in each PR branch. I like to commit early and often so that I have a lot of checkpoints (and, frankly, so I don’t have to think about organizing commits on my in-progress branch). And I push them all up to GitHub, both as a backup and to share my work. The individual commits are meaningless to anyone but me.[1]

I’ve used this general approach for my whole career so I’m used to it. Some tools like Gerrit make all this feel very natural. GitHub’s gotten better, but it’s still tricky.

Note
 I wrote this post to document what I do. I’m not trying to convince anybody else to do it. If you don’t like this, that’s okay!

Shortest possible explanation of my workflow

Each PR is (of course) its own branch.

  • The PRs always have a logical order: the order they will wind up landing in main. (If there’s no ordering like this, then there’s no dependency between them, and I don’t need any of this. These are just ordinary PRs. On the other hand, all of this works with a tree of dependencies (rather than a line) as well.)

  • The base branch (sometimes called parent branch) of each PR is either main (if it’s currently first in line among the open PRs) or the branch of the PR ahead of it in line. So PR #2’s base branch is the branch for PR #1, etc.

  • All ordinary work is done with regular commits. No rebases and no force pushes.

  • Upstream changes (e.g., from PR #1) are pulled into downstream branches (e.g., into PR #2) using git merge.

  • Upstream changes are only ever pulled directly into the immediate downstream branch.

  • Changes from downstream branches never land in upstream branches.

  • PRs are landed onto main using a squash commit.

None of this should be too surprising. This is one of several typical ways to use Git on GitHub: using merge commits for most merges except the main branch, which uses squash commits.

It’s easy enough to set up the PRs like this. Here I have PR #1 using branch1 and PR #2 using branch2:

10 simple

After a bunch of iteration in the branches and merging from upstream, I might have something like this:

80 branch2 sync branch1 again

Now, it’s also easy to land the first PR in line, just like any other PR. Where this goes awry is that after each PR lands on main, you have to fix up the next in-flight PR carefully. If I just git merge origin/main into branch2, I’ll wind up pulling in the squash commit, which will have a raft of conflicts with commits A1-A3 (since they’re the same changes). These are not intrinsic conflicts. I shouldn’t have to resolve these at all. These are just an artifact of how all the merging was done up to this point.

Here’s what I do instead. Continuing the example above, PR #1 was branch1, whose tip commit was A4. A4 landed on main as squash commit M4:

120 landed as squash

To fix up PR #2, in branch2:

  1. git merge origin/branch1 (or git merge A4, if GitHub already deleted branch1 after the PR landed). This syncs up with any last changes in the parent branch. The only conflicts I can have here are intrinsic conflicts between branch1 and branch2 that I have not already resolved.

  2. git merge M3. This syncs up with main up to (but not including) where the parent branch landed. The only conflicts I can have here are intrinsic conflicts between any commits on main that I haven’t already sync’d up with and branch2. There can’t be any conflicts between commits on main that branch1 hadn’t sync’d up with and branch1 (A4) because GitHub wouldn’t have let me land the squash commit without resolving those conflicts first.

    One might choose to write this as git merge M4^, which better reflects the meaning "the commit before the one that branch1 landed in".

  3. git merge -Xours M4. This is the magic step. This pulls M4 into branch2, but automatically resolves all the conflicts that squash commit would have with all the branch1 and branch2 commits by taking the version currently in branch2.

    Why is this correct? Because I’ve already resolved these conflicts correctly in one of the earlier steps. Put differently: after the previous step, I already have (1) everything in main, and (2) everything that was in branch1 before it landed. I’m not pulling anything new with M4.

This leaves me with:

130 updated branch2

Now branch1 is obsolete (that PR landed). branch2 is at the front of the line and can be landed with a regular squash.

Once I’ve fixed up the new front-of-line PR, I fix up the rest by just sync’ing them up with their immediate upstream, like usual.

That’s it. The next section goes into more detail about this whole process. After that I compare this approach to various others.

Step-by-step explanation

Feel free to skip this section if this all made sense already.

Stacking the PRs themselves is the easy part. Let’s start simply and precisely:

  • main is at commit M1.

  • To create PR #1, I create branch branch1, branched from main at M1. I add commit A1. I push this and create a pull request to merge it into main.

  • To create PR #2, I create branch branch2, branched from branch1 at A1. I add commit B1. I push this and create a pull request to merge it into branch1 (not main).

It looks like this:

10 simple

This reflects that the logical sequence of work is PR #1, then PR #2. That’s the order they will land on main. Then it will look like this:

20 simple merged

I use separate clones for branch1 and branch2. You don’t have to do this. You can do it all in one local clone and switch branches back and forth. I use separate clones mainly to keep incremental rebuilds of each branch quick.

Iterating on PRs

Suppose I get feedback on PR #1. Easy: I push new commits to branch1. Say I add A2. It looks like this:

30 simple more

Of course, I can iterate on branch2 with more commits, too. Suppose I add A3 to branch1 and B2 and B3 to branch2:

50 simple iterate

What about review? You could imagine three different reviewers who looked at A1, A2, and A3. Regardless of where they started, they can always look at "changes since last review" — that’s just the delta between the commit they reviewed and the current tip, A3.

It’s the same on branch2. Reviewers only see the delta relative to A1 (the merge base of branch2 with its base branch, branch1). This is exactly what happens for PRs whose base branch is main.

Sync’ing up branch2 with changes in branch1

[2]

Now, branch2 is supposed to be logically after branch1, so I generally want to pull changes from branch1 into branch2 as I go. I just use a standard git merge for this.

In the branch2 clone:

$ git fetch
$ git merge origin/branch1

This is where I’ll resolve any conflicts between branch1 and branch2. More precisely, this would cover conflicts between A2-A3 and B1-B3. The result looks like this:

60 branch2 sync branch1

I never want to go the other direction. Changes in branch2 don’t belong in branch1.

Sync’ing up with main

Sync’ing either branch up with main works the same way, but it’s a little more complicated. Let’s assume main has advanced to commit M2. First, I’ll sync up branch1.

In the branch1 clone:

$ git fetch
$ git merge origin/main

This is where I resolve conflicts between the new changes on main (M2) and the changes in my branch since the last main commit that’s in my branch (that’s M1, and the relevant changes in my branch are A2-A3). The result:

70 branch1 sync main

I sync up branch2 with main by sync’ing it up branch1, not main directly. It’s exactly as above. In the branch2 clone:

$ git fetch
$ git merge origin/branch1

The key here is that the only conflicts I see here are the ones between the new changes (so, just A4, which is just M2) and the changes in branch2 (so, B1-B3). I won’t see the conflicts with A1-A3 again because those were resolved in A4. The result looks like this:

80 branch2 sync branch1 again

If I instead tried to sync up branch2 with main directly, I would have to resolve any conflicts between M2 and A1-A3 again. Those were resolved in A4 already and that’s why I want to pull that into branch2. (Plus, I don’t want stuff from main that’s not in branch1 anyway, since then it will look like PR #2 is trying to land those commits into branch1 — i.e., they’ll show up in the delta in the PR, which I don’t want.)

Dealing with more than two pull requests

I’ve now covered all the cases so that this can be extended to more than two pull requests. In fact, you can think of main in my diagrams as branch1, branch2 as branch3, and branch2 as branch3:

90 three branches

The key principles are:

  • All ordinary work is done with regular commits. No rebases and no force pushes.

  • Upstream changes are pulled into downstream branches using git merge.

  • Upstream changes are only ever pulled directly into the immediate downstream branch.

  • Changes from downstream branches never land in upstream branches.

Landing PRs

This all sounds terrific until it’s time to start landing the pull requests. That’s where it can get tricky.

Let’s review my two-PR example:

80 branch2 sync branch1 again

To make it more general, let’s imagine that main has evolved further to M3. Let’s assume there are no merge conflicts between M3 and A4 though. (If there were any, I’d have to manually sync up branch1 with main, and we know how to do that.) So it looks like this:

100 before landing

Landing the first PR

Landing the first PR is easy. I hit the big "Squash and Merge" button. (Recall that I do this because I want a linear history on main.) This creates a new commit M4 with the same contents as if I’d merged A4 into main. Logically, it looks like this:

110 landed as merge

But in fact, as far as the Git tree is concerned, there is no connection between M4 and A4. It really looks like this:

120 landed as squash

Why does that matter? Because of PR #2. First of all, the base of PR #2 was branch1. GitHub is smart enough to know that since PR #1 landed, the base branch of PR #2 should be changed to main. That is: before, PR #2 was something I was going to land into PR #1. Now that PR #1 has landed on main, PR #2 is going to land into main, too.

But branch2 includes ordinary commits A1-A3 as well as B1-B3. And A1-A3 are not in main. So it looks to Git and GitHub like PR #2 is trying to land all of that — including commits A1-A3 again. In GitHub, the PR delta will be much larger than it should be — indeed, much larger than it was before PR #1 landed (which of course isn’t what I want). I’ll very likely have a raft of merge conflicts here because commits A1-A3 will absolutely conflict with M4 — they’re the same changes. I don’t want to have to resolve these. These aren’t real conflicts. They’re an artifact of the process I used to get here.

It’s similarly tricky to sync up branch1 with main. If I just tried to pull in M4, again, I’d get a raft of pointless merge conflicts. I don’t want this.

The key step

Here’s what does work. In the branch2 clone, first, I make sure that branch2 is sync’d up with the tip of branch1, the same commit that got landed (as a squash) onto main.[3]

git fetch
git merge origin/branch1

This is exactly like the above examples of sync’ing up with the upstream branch. The only conflicts that could show up here are any intrinsic conflicts between branch1 and branch2 that I haven’t resolved yet. In my example, there were no commits on branch1 that I hadn’t already merged, so this step will do nothing.

Next, I sync up with main up through the commit prior to the one where PR #2 landed. In my case, that’s M3:

git merge M3

Here, I’ll resolve any conflicts between M3 and B1-B3. Note that there cannot be any conflicts between M3 and A1-A3 because if there were, I wouldn’t have been able to land PR #1 without first sync’ing it up and resolving these conflicts. So again, the only conflicts here are real conflicts that I haven’t already resolved.

Critically, at this point, I know:

  • branch2 contains all of the changes in branch1 and has resolved any conflicts (because I sync’d up with it)

  • branch2 contains all of the changes in main up through M3 and has resolved any conflicts (because I sync’d up with that, too)

Thus, the contents of branch2 is exactly what would logically be on main if I landed branch2 right after M4: it’s M4 plus the changes that were specific to branch2 (B1-B3). The only problem is that as far as Git can tell, branch2 doesn’t have M4.

I can fix that by merging in M4, but with the ours option:[4]

git merge -Xours M4

This says: merge commit M4 into this branch, but anywhere they conflict take the change in my branch, not M4. This is an automerge that always takes what’s in my branch. This would normally be shady, but here, I know that I’ve fully, correctly merged all these conflicts. So I’m telling Git to ignore them. The only purpose of this operation is to get M4 into my commit history without changing the contents of my tree.

And voilà! branch2 is now sync’d up with M4:

130 updated branch2

The net result is:

  • I had multiple stacked PRs.

  • I kept a linear history on "main".

  • I never had to resolve the same conflicts twice (or any non-intrinsic conflicts).

  • I never force-pushed, so I never broke incremental review.

Hooray!

Comparison with other approaches

Use merge commits in main (rather than squashing)

Some projects use merge commits in main rather than squashing. If you go that route, I imagine you don’t have to worry about most of this. The problem at the crux of this post results directly from the fact that PR #2 has to pull in the squash commit, which Git doesn’t have a way of knowing is the same contents as something that’s already been pulled in.

That said, the linear history is extremely valuable to me. The way I think of it is: the PR process itself is messy, with lots of random commits that aren’t meaningful. By the time the PR itself lands, it’s one logical change. Let’s pay the cost at merge-time (which is trivial, since this is fully automated) to create an easier-to-understand history for our future selves, rather than forever having to deal with whatever messy process we went through during the PR process. (Again, others disagree, and that’s fine.)

Iterate on PRs using rebase / force push

Another common practice is to use rebase, at least to sync up with "main" and maybe even for all changes that you make to the PR. In the latter case, the PR might only ever have one commit, but it keeps changing. Each change requires a force push.

If you force push a PR branch, that blows out of the water any other PRs based on this PR. Concretely, going with my example, you’ve got:

  • PR #1 branch branch1 with commit A1

  • PR #2 branch branch2 with commit B1, whose parent is A1. PR #2’s base branch is branch1.

Now you force push branch1 so that you have A1', not A1.

PR #2 is immediately a mess. It still has B1, which means it also has A1, but that’s not in the parent branch. So it’ll look like PR #2 is adding both A1 and B1, which will almost certainly have lots of conflict with B1. This is probably easy to fix up: in branch2, you git rebase -i A1' (onto the new tip of branch1) and only include commit B1 (i.e., you drop A1).

But this approach has a few problems:

  • Every time you change any PR in the sequence, all subsequent ones get immediately broken. You have to fix them all individually. (By contrast, if you just keep pushing new commits like I do, the other PRs are unaffected. You’ll have to sync them up eventually, but in the meantime, they’re fine.)

  • Every time you do fix up any of these PRs, you’re rebasing this whole branch, which means re-resolving any conflicts you’ve already resolved. (By contrast, with the approach I use, I only ever have to resolve any given conflict once.)

I gather tooling like jj can automate a lot of this. That leaves the second major issue: force pushes break GitHub incremental review. GitHub has gotten better about this, but it’s still a mess. That’s understandable because it’s tricky! Suppose:

  1. You have PR #1 branch branch1 with commit A1. Alice and Bob review this.

  2. From Alice’s feedback, you force-push the branch so it only has A2.

    • Alice and Bob can both do an incremental review (showing only the delta from what they last looked at) by looking at how the branch changed. You can see this with the "Compare" link that shows up in the PR conversation view (on the far right here):

      github force push

      Also, if they go to "Files changed" (to view all the diffs), any files that haven’t changed across the force push are collapsed and still marked "Viewed".

  3. Suppose Alice does do a review, you incorporate more feedback, and you force-push A3. Bob never looked at A2.

At this point, how does Bob do an incremental review? He wants to see the difference between A1 and A3. There’s no way to show this in GitHub (as far as I know). And if you could, it likely wouldn’t correctly omit changes that came in via sync’ing up with an upstream branch.

On the other hand, if you just keep pushing commits to the PR without force pushing, GitHub gives you a handy way to show only changes since your last review, which just compares two commits:

github changes commits

and this skips merge commits so you don’t see changes that came in from upstream. But here you’re choosing from a subset of the commits currently on the PR. I don’t believe there’s a way to do this when the commit you saw before isn’t on the PR any more.

Conclusion

See tl;dr.

I’m sure I’ve got some of this wrong. Let me know if so!

1. I know this is different from how a lot of other people work. That’s okay.
2. I have long loved this usage note in illumos panic.c, referencing the Oxford Guide to English Usage. But I just can’t agree that "syncked"/"syncking" is better than "sync’d"/"sync’ing" in practice.
3. GitHub may have deleted branch1 after it landed. This is configurable. If you get an error here about origin/branch1 not existing, just go to the page for PR #1, find the last commit in that PR, and use git merge COMMIT_ID instead. Or un-delete the branch in the GitHub UI, run git fetch, then delete the branch again. You’ll be able to use git merge origin/branch1 locally.
4. I believe it should also work to use the "ours" merge strategy (-s ours), which is a different thing but should be equivalent in this case.