Skip to content

fix: one bump pull request per repo, and a run that fails when one cannot merge - #24

Merged
fatherlinux merged 3 commits into
mainfrom
fix/bump-consumers-one-pr-per-repo
Nov 5, 2026
Merged

fatherlinux merged 3 commits into
mainfrom
fix/bump-consumers-one-pr-per-repo

Conversation

@fatherlinux

Copy link
Copy Markdown
Member

v0.4.0 and v0.5.0 released two minutes apart and left 10 repos with a bump PR that can never merge — data-collector#42, mcp-pcloud#64, mcp-systemd#46, mcp-trove#104, mcp-workboard#96, petit#115, rotv#852, shiva#35, tig#35, trentina#717. Crunchtools 3 is rebasing those by hand; this is the cause.

The cause

Per-version branches — chore/python-sdlc-v0.4.0, chore/python-sdlc-v0.5.0 — look tidier and guarantee this. Both get cut from a default branch that still pins the old version, so both rewrite the same rev: line. When the older one merges it moves that line out from under the newer branch, which goes CONFLICTING.

Rebasing before opening would not have helped. The conflict appears afterwards, created by the other PR landing.

One branch per repo, chore/python-sdlc, force-updated each release with its PR reused in place. A second release then supersedes the first on the same branch instead of racing it — which is correct, since the later image contains the earlier one's code. The clone is always the current default branch, so the rewrite is always against what the repo pins now.

The damage, which is the part worth fixing properly

A CONFLICTING pull request runs no pull_request workflows at all, so its required check is ABSENT rather than red. Auto-merge never fires, nothing reports a failure, and the PR sits looking green indefinitely. All 10 would have.

That is the same failure this entire release has been about — a check that did not run reading exactly like a check that passed. So the script now asks GitHub whether each PR can actually merge and fails the run on CONFLICTING, naming the repos to rerun. UNKNOWN is not a failure: GitHub computes mergeability asynchronously and a release must not go red because the API was slow.

bump-consumers also gets a fixed concurrency group, like promote's. One branch per repo means two releases at once would fight over it and the loser's version is what the branch would be left pointing at.

Verification

--dry-run against the fleet: 33 repos pin python-sdlc, 23 already on v0.5.0, and the 10 it names are exactly the 10 that conflicted — found independently of the list above.

No version bump: release automation only, and it does not change the image. It takes effect on the next release.

🤖 Generated with Claude Code

https://claude.ai/code/session_01R7Y1jfBgAyvK9xarny6EHx

…nnot merge

v0.4.0 and v0.5.0 released two minutes apart and left 10 repos with a bump
pull request that can never merge. The branch name was the cause and the
silence was the damage.

Per-version branches -- `chore/python-sdlc-v0.4.0`, `chore/python-sdlc-v0.5.0`
-- look tidier and guarantee this. Both are cut from a default branch that
still pins the old version, so both rewrite the same `rev:` line. When the
OLDER one merges it moves that line out from under the newer branch, which goes
CONFLICTING. Rebasing before opening would not have helped: the conflict
appears afterwards, created by the other pull request landing.

One branch per repo, `chore/python-sdlc`, force-updated each release and its
pull request reused in place. A second release then supersedes the first on the
same branch rather than racing it -- which is correct, since the later image
contains the earlier one's code. The clone is always the current default
branch, so the rewrite is always against what the repo pins now.

The silence is the part worth fixing properly. A CONFLICTING pull request runs
no `pull_request` workflows AT ALL, so its required check is ABSENT rather than
red: auto-merge never fires, nothing reports a failure, and it sits looking
green indefinitely. All 10 would have. That is the same failure this whole
release has been about -- a check that did not run reading exactly like a check
that passed -- so the script now asks GitHub whether each pull request can
actually merge and FAILS the run on CONFLICTING, naming the repos to rerun.
UNKNOWN is not a failure: GitHub computes mergeability asynchronously and a
release must not go red because the API was slow.

`bump-consumers` also gets a fixed concurrency group, like `promote`'s. One
branch per repo means two releases at once would fight over it and the loser's
version would be what the branch was left pointing at.

Verified with `--dry-run` against the fleet: 33 repos pin python-sdlc, 23
already on v0.5.0, and the 10 it names are exactly the 10 that conflicted.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gatehouse found 1 issues (1 medium)

Full review.

1 finding(s) dropped: their evidence is not in the diff.

Comment thread scripts/bump_consumers.py Outdated
not go red because the API was slow. CONFLICTING is, because nothing else
will ever report it -- see the module docstring.
"""
for attempt in range(6):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MEDIUM (Performance Check): Each consumer PR can trigger six sequential GitHub CLI/API calls, with five 5-second sleeps when mergeability remains UNKNOWN. Because this check runs per repo, a fleet of 10 repos can add roughly five minutes of waiting, plus request latency, to a release run.

Suggestion: Reduce the per-repo polling budget or perform mergeability checks in a batched pass after updating all repos, so UNKNOWN results do not impose repeated serial waits on the whole run.

Evidence:

for attempt in range(6):
    if attempt:
        time.sleep(5)
    state = gh(
        "pr", "view", url, "-R", f"{ORG}/{repo}", "--json", "mergeable", "--jq", ".mergeable"
    ).strip()

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d00f047, and the suggestion was better than what you reviewed — the batched pass, not a smaller budget.

The point I had missed: GitHub computes mergeability asynchronously, so asked immediately after a push the answer is always UNKNOWN. That makes the first ask guaranteed wasted and the waiting the normal case rather than the exception, which is exactly your 5-minutes-per-10-repos. Shrinking the budget would have traded the wait for a weaker guard.

Now it is one pass after every repo is bumped, and each round asks only the repos still unanswered — three rounds, five seconds apart. By then the rest of the fleet has been bumped, so the answer is usually there on the first ask and costs nothing. Same guard, same failure behaviour.

It also reads better than what you flagged: conflicts are reported together as a fleet state rather than one repo at a time as a side effect of bumping it.

…ce per repo

Gatehouse was right, and its suggestion was better than what it reviewed.

The mergeability check sat inside bump(), so each repo got up to six questions
with five-second waits between them. GitHub computes mergeability
asynchronously, so asked immediately after a push the answer is ALWAYS
UNKNOWN -- which means the first ask is guaranteed to be wasted and the waiting
is the normal case, not the exception. Across 33 repos that is minutes of a
release run doing nothing at all, serially, for an answer that arrives for free
once the rest of the fleet has been bumped.

Now it is one pass after the loop, and each round asks only the repos still
unanswered. Three rounds, five seconds apart. Same guard, same failure
behaviour, without spending a release's time on a question that answers itself.

It also reads better: the conflicts are reported together, as a fleet state,
rather than one repo at a time as a side effect of bumping it.

And a counting bug of my own in the same block: the summary said
`len(opened) - len(failed)` bumped, but `failed` also carries repos whose
config could not be READ and which were never opened, so a single unreadable
repo would have understated the bump count. It subtracts the conflicts, which
is what it meant.

Verified with --dry-run: 33 repos pin python-sdlc, 32 already on v0.5.0 as the
rebases land, and an unreadable repo still reports honestly rather than
silently.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gatehouse found 1 issues (1 medium)

Reviewed 1 commit since 5693c04 (incremental).

3 finding(s) dropped: their evidence is not in the diff.

Comment thread scripts/bump_consumers.py Outdated
if attempt:
time.sleep(PAUSE)
for repo, url in list(pending.items()):
state = gh(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MEDIUM (General Review): A failure while querying mergeability now escapes the per-repository error handling in the main loop. The resulting exception does not identify the repository being checked, making the failure harder to diagnose and preventing the normal summary from being printed.

Suggestion: Handle subprocess.CalledProcessError around the query, add the repository and pull-request URL to the error context, and let the caller report the failure consistently.

Evidence:

state = gh(
    "pr",
    "view",
    url,
    "-R",
    f"{ORG}/{repo}",
    "--json",
    "mergeable",
    "--jq",
    ".mergeable",
).strip()

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f974ba2. You caught a regression I introduced one commit earlier, and the diagnosis is exact.

Moving the check out of the per-repo loop moved it outside that loop's error handling too. gh() raises on a non-zero exit, and from conflicting() nothing catches it, so one refused API call would have ended the run with a traceback and printed no summary — which is the specific failure this script already carries a comment about (RT#1553): a run that could not finish looking like a job that half worked. Reintroducing it in the same change that adds a guard against silent failure would have been funny in the wrong way.

A repo the API will not answer for is now named and treated as unanswered, the same as UNKNOWN, and not as a failure — a release must not go red because GitHub was briefly unavailable. Both kinds of non-answer print by name, which matters more here than usual: a guard that silently answered for nobody is exactly the shape it exists to catch.

… the run

Gatehouse caught a regression I introduced in the previous commit. Moving the
check out of the per-repo loop also moved it outside that loop's error
handling: `gh()` raises on a non-zero exit, and from `conflicting()` there is
nothing to catch it, so one refused API call would have ended the whole run
with a traceback and printed no summary at all.

That is the specific failure this script already carries a comment about
(RT#1553): a run that could not finish its job looking like a job that half
worked. Reintroducing it while adding a guard against silent failure would
have been funny in the wrong way.

A repo the API will not answer for is now named and treated as unanswered,
the same as UNKNOWN. Not a failure: a release must not go red because GitHub
was slow or briefly unavailable. Both kinds of non-answer are printed by name,
so a guard that quietly answered for nobody is visible rather than assumed --
which matters more here than usual, since a guard that silently does nothing is
exactly the shape it exists to catch.

Verified with --dry-run: the fleet is now 33 of 33 pinned at v0.5.0.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gatehouse found no issues.

Reviewed 1 commit since d00f047 (incremental).

1 finding(s) dropped: their evidence is not in the diff.

@fatherlinux
fatherlinux merged commit 0cdf0b1 into main Nov 5, 2026
16 checks passed
@fatherlinux
fatherlinux deleted the fix/bump-consumers-one-pr-per-repo branch November 5, 2026 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant