Skip to content

ci(adoption-wall): land the weekly refresh through a bot-merged PR - #573

Merged
jpr5 merged 1 commit into
mainfrom
ci/adoption-wall-pr-flow
Oct 22, 2026
Merged

jpr5 merged 1 commit into
mainfrom
ci/adoption-wall-pr-flow

Conversation

@jpr5

@jpr5 jpr5 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Why

During the 10/21 incident, copilotkit-bot (app 1108748) was removed from ruleset blorpathupap (13465930). That ruleset protects main and requires a PR with 1 approval. The bot was the only identity that could push the weekly adoption-wall refresh straight to main, so the next safe refresh fails with GH013.

The bot is now back on the ruleset as a pull-request-only bypass actor. It can merge a PR without the approval, but it cannot push to main directly. This PR changes the auto-land step to match:

  • Push a bot/adoption-wall-<date>-<run>-<attempt> branch.
  • Open a PR into the run's ref.
  • Merge the PR with gh pr merge --merge --admin. The ruleset allows merge commits only. --auto waits for the approval forever.
  • The app token now also requests pull-requests: write.

The needs-review path is unchanged. Comments and the Slack text no longer say the step pushes to main.

Red (real surface, copilotkit-bot installation token, before the ruleset change)

=== RED (retry) 2026-10-21 06:46:34 PM PDT
--- ruleset 13465930 as copilotkit-bot:
{"bypass_actors":null,"current_user_can_bypass":"never","name":"blorpathupap"}
--- rules applying to main, as copilotkit-bot:
deletion ruleset=13465930
pull_request ruleset=13465930
non_fast_forward ruleset=13465930
--- workflow direct push line (main):
502:            if git push origin "HEAD:${GITHUB_REF_NAME}"; then
--- direct push attempt to main as copilotkit-bot (empty commit on top of origin/main):
probe commit 4837d8c55c352608a2fa96ad614c4ab481f3885c
remote: error: GH013: Repository rule violations found for refs/heads/main.        
remote: Review all repository rules at https://github.com/CopilotKit/aimock/rules?ref=refs%2Fheads%2Fmain        
remote: 
remote: - Changes must be made through a pull request.        
remote: 
To https://github.com/CopilotKit/aimock.git
 ! [remote rejected]   4837d8c55c352608a2fa96ad614c4ab481f3885c -> main (push declined due to repository rule violations)
error: failed to push some refs to 'https://github.com/CopilotKit/aimock.git'
push exit=1
--- main after attempt:
95a63adcca126fdc3902d5ba713d525f3ace5116	refs/heads/main

Ruleset change + green for the push half

After the change, the bot reads current_user_can_bypass: pull_requests_only, and a direct push is still rejected:

=== post-PUT 2026-10-21 06:46:51 PM PDT
{"current_user_can_bypass":"pull_requests_only","name":"blorpathupap"}
--- direct push still rejected in pull_request mode?
remote: error: GH013: Repository rule violations found for refs/heads/main.        
remote: Review all repository rules at https://github.com/CopilotKit/aimock/rules?ref=refs%2Fheads%2Fmain        
remote: 
remote: - Changes must be made through a pull request.        
remote: 
To https://github.com/CopilotKit/aimock.git
 ! [remote rejected]   4837d8c55c352608a2fa96ad614c4ab481f3885c -> main (push declined due to repository rule violations)
error: failed to push some refs to 'https://github.com/CopilotKit/aimock.git'
push exit=1
95a63adcca126fdc3902d5ba713d525f3ace5116	refs/heads/main

Probe of the PR half (bot-authored PR #572, closed unmerged)

=== GREEN probe 2026-10-21 06:48:08 PM PDT
--- push branch bot/adoption-wall-probe-20261021184809 as bot
remote: 
remote: Create a pull request for 'bot/adoption-wall-probe-20261021184809' on GitHub by visiting:        
remote:      https://github.com/CopilotKit/aimock/pull/new/bot/adoption-wall-probe-20261021184809        
remote: 
To https://github.com/CopilotKit/aimock.git
 * [new branch]        1988db407bca0e6fc34660cb7dcebe5680cda8e4 -> bot/adoption-wall-probe-20261021184809
push exit=0
--- gh pr create as bot
https://github.com/CopilotKit/aimock/pull/572
--- PR #572 as seen BY THE BOT:
{"author":{"login":"copilotkit-bot"},"mergeStateStatus":"BLOCKED","reviewDecision":"REVIEW_REQUIRED","viewerCanMergeAsAdmin":false}
--- gh pr merge WITHOUT --admin as bot (expect refusal: approval required):
X Pull request CopilotKit/aimock#572 is not mergeable: the base branch policy prohibits the merge.
To have the pull request merged after all the requirements have been met, add the `--auto` flag.
To use administrator privileges to immediately merge the pull request, add the `--admin` flag.
exit=1
--- gh pr merge --auto as bot (expect: queued, NOT merged):
exit=0
{"auto":true,"state":"OPEN"}
--- close probe PR unmerged + delete branch
✓ Closed pull request CopilotKit/aimock#572 ([probe] adoption-wall bot bypass check (do not merge))
✓ Deleted branch bot/adoption-wall-probe-20261021184809
{"mergedAt":null,"state":"CLOSED"}
95a63adcca126fdc3902d5ba713d525f3ace5116	refs/heads/main

The bot can push a branch, open a PR into main, and enable auto-merge. A plain merge is refused because the approval is required. viewerCanMergeAsAdmin is false for the bot, so the --admin merge through the PR-only bypass is not yet proven. See the next section: it is now proven.

Green: bot admin-merge through the PR-only bypass (option B probe, 6:56 PM PDT)

Ruleset 13465930 was briefly widened to also cover a throwaway ruleset-probe/bot-merge branch. As copilotkit-bot, the probe pushed a bot/probe-* branch, opened PR #574 into that base and ran the exact command this PR uses. The ruleset was then restored to the exact pre-probe JSON (diffed). Both probe branches are gone.

=== option B probe 2026-10-21 06:56:03 PM PDT
main=95a63adcca126fdc3902d5ba713d525f3ace5116
created refs/heads/ruleset-probe/bot-merge @ 95a63adcca126fdc3902d5ba713d525f3ace5116
widened include=["~DEFAULT_BRANCH","refs/heads/ruleset-probe/bot-merge"] bypass=["null:always","2:pull_request","2740:always","1108748:pull_request"]
--- rules on probe base (as jpr5):
deletion ruleset=13465930
pull_request ruleset=13465930
non_fast_forward ruleset=13465930
head branch bot/probe-20261021185645
bot view: pull_requests_only
--- control: bot direct push to protected probe base (expect GH013):
remote: error: GH013: Repository rule violations found for refs/heads/ruleset-probe/bot-merge.        
remote: Review all repository rules at https://github.com/CopilotKit/aimock/rules?ref=refs%2Fheads%2Fruleset-probe%2Fbot-merge        
remote: 
remote: - Changes must be made through a pull request.        
remote: 
To https://github.com/CopilotKit/aimock.git
 ! [remote rejected]   88569654f9f6e289f58db31ffea08c7c5710ed22 -> ruleset-probe/bot-merge (push declined due to repository rule violations)
error: failed to push some refs to 'https://github.com/CopilotKit/aimock.git'
exit=1
--- bot push head branch:
 * [new branch]        88569654f9f6e289f58db31ffea08c7c5710ed22 -> bot/probe-20261021185645
PR: https://github.com/CopilotKit/aimock/pull/574
bot view of PR: {"author":{"login":"copilotkit-bot"},"mergeStateStatus":"BLOCKED","reviewDecision":"REVIEW_REQUIRED","viewerCanMergeAsAdmin":false}
--- EXACT #573 command as bot: gh pr merge "bot/probe-20261021185645" --merge --admin
gh exit=0
final: state=MERGED mergedBy=app/copilotkit-bot commit=764f186b0b9c09259257a8345021d35b19964897
--- probe base tip:
764f186b0b9c09259257a8345021d35b19964897	refs/heads/ruleset-probe/bot-merge
N=574 B=bot/probe-20261021185645
--- rule-suite for the merge:
2026-10-21T18:56:59-07:00 actor=copilotkit-bot[bot] result=bypass after=764f186b0b9c09259257a8345021d35b19964897
2026-10-21T18:56:46-07:00 actor=copilotkit-bot[bot] result=fail after=88569654f9f6e289f58db31ffea08c7c5710ed22
=== RESTORE 06:57:12 PM PDT
restored include=["~DEFAULT_BRANCH"]
DIFF vs pre-probe: IDENTICAL
--- delete probe branches:
deleted ruleset-probe/bot-merge
{"message":"Reference does not exist","documentation_url":"https://docs.github.com/rest/git/refs#delete-a-reference","status":"422"}gh: Reference does not exist (HTTP 422)
(no LEFTOVER lines = clean)
=== post-restore main check 06:57:24 PM PDT
bot view: pull_requests_only
remote: error: GH013: Repository rule violations found for refs/heads/main.        
 ! [remote rejected]   4837d8c55c352608a2fa96ad614c4ab481f3885c -> main (push declined due to repository rule violations)
95a63adcca126fdc3902d5ba713d525f3ace5116	refs/heads/main
  • gh pr merge --merge --admin merged [probe] bot admin-merge through PR-only bypass #574 as app/copilotkit-bot, even though viewerCanMergeAsAdmin was false.
  • The rule-suites API records the merge as result=bypass.
  • The bot's direct push to the same protected base was rejected (result=fail, GH013).
  • After the restore, a bot direct push to main is still rejected.

After merge

The first safe + changed + confined run exercises this path on main. A workflow_dispatch takes that path only when the wall verdict is safe.

actionlint: clean. prettier: clean. commitlint: clean.

The default branch ruleset no longer lets copilotkit-bot push to main
directly; it is now a pull-request-only bypass actor. Open a PR from a
bot/adoption-wall-* branch and merge it with --admin (merge commit)
through that bypass instead of pushing to main.
@pkg-pr-new

pkg-pr-new Bot commented Oct 22, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@573

commit: 60fe76c

@jpr5

jpr5 commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Calibration at 6:49 PM PDT on this PR: viewerCanMergeAsAdmin is true for jpr5 (OrganizationAdmin, always) and false for copilotkit-bot (pull_request mode). So GraphQL does not confirm that the bot's --admin merge will be accepted. GitHub's docs say pull-request bypass lets the actor merge PRs. The first safe run after merge is the proof. If the merge is refused, the step fails and leaves the bot PR open.

@jpr5
jpr5 merged commit 8206984 into main Oct 22, 2026
17 checks passed
@jpr5
jpr5 deleted the ci/adoption-wall-pr-flow branch October 22, 2026 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant