Repository navigation
[probe] adoption-wall bot bypass check (do not merge) - #572
Closed
copilotkit-bot[bot] wants to merge 1 commit into
Closed
copilotkit-bot[bot] wants to merge 1 commit into
copilotkit-bot[bot] wants to merge 1 commit into
Conversation
Contributor
Author
|
Probe complete; closing unmerged. |
jpr5
added a commit
that referenced
this pull request
Oct 22, 2026
) ## Why During the 10/21 incident, `copilotkit-bot` (app 1108748) was removed from ruleset `blorpathupap` (13465930). That ruleset protects `main` and requires a PR with 1 approval. The bot was the only identity that could push the weekly adoption-wall refresh straight to `main`, so the next safe refresh fails with GH013. The bot is now back on the ruleset as a **pull-request-only** bypass actor. It can merge a PR without the approval, but it cannot push to `main` directly. This PR changes the auto-land step to match: - Push a `bot/adoption-wall-<date>-<run>-<attempt>` branch. - Open a PR into the run's ref. - Merge the PR with `gh pr merge --merge --admin`. The ruleset allows merge commits only. `--auto` waits for the approval forever. - The app token now also requests `pull-requests: write`. The needs-review path is unchanged. Comments and the Slack text no longer say the step pushes to `main`. ## Red (real surface, copilotkit-bot installation token, before the ruleset change) ``` === RED (retry) 2026-10-21 06:46:34 PM PDT --- ruleset 13465930 as copilotkit-bot: {"bypass_actors":null,"current_user_can_bypass":"never","name":"blorpathupap"} --- rules applying to main, as copilotkit-bot: deletion ruleset=13465930 pull_request ruleset=13465930 non_fast_forward ruleset=13465930 --- workflow direct push line (main): 502: if git push origin "HEAD:${GITHUB_REF_NAME}"; then --- direct push attempt to main as copilotkit-bot (empty commit on top of origin/main): probe commit 4837d8c remote: error: GH013: Repository rule violations found for refs/heads/main. remote: Review all repository rules at https://github.com/CopilotKit/aimock/rules?ref=refs%2Fheads%2Fmain remote: remote: - Changes must be made through a pull request. remote: To https://github.com/CopilotKit/aimock.git ! [remote rejected] 4837d8c -> main (push declined due to repository rule violations) error: failed to push some refs to 'https://github.com/CopilotKit/aimock.git' push exit=1 --- main after attempt: 95a63ad refs/heads/main ``` ## Ruleset change + green for the push half After the change, the bot reads `current_user_can_bypass: pull_requests_only`, and a direct push is still rejected: ``` === post-PUT 2026-10-21 06:46:51 PM PDT {"current_user_can_bypass":"pull_requests_only","name":"blorpathupap"} --- direct push still rejected in pull_request mode? remote: error: GH013: Repository rule violations found for refs/heads/main. remote: Review all repository rules at https://github.com/CopilotKit/aimock/rules?ref=refs%2Fheads%2Fmain remote: remote: - Changes must be made through a pull request. remote: To https://github.com/CopilotKit/aimock.git ! [remote rejected] 4837d8c -> main (push declined due to repository rule violations) error: failed to push some refs to 'https://github.com/CopilotKit/aimock.git' push exit=1 95a63ad refs/heads/main ``` ## Probe of the PR half (bot-authored PR #572, closed unmerged) ``` === GREEN probe 2026-10-21 06:48:08 PM PDT --- push branch bot/adoption-wall-probe-20261021184809 as bot remote: remote: Create a pull request for 'bot/adoption-wall-probe-20261021184809' on GitHub by visiting: remote: https://github.com/CopilotKit/aimock/pull/new/bot/adoption-wall-probe-20261021184809 remote: To https://github.com/CopilotKit/aimock.git * [new branch] 1988db4 -> bot/adoption-wall-probe-20261021184809 push exit=0 --- gh pr create as bot #572 --- PR #572 as seen BY THE BOT: {"author":{"login":"copilotkit-bot"},"mergeStateStatus":"BLOCKED","reviewDecision":"REVIEW_REQUIRED","viewerCanMergeAsAdmin":false} --- gh pr merge WITHOUT --admin as bot (expect refusal: approval required): X Pull request #572 is not mergeable: the base branch policy prohibits the merge. To have the pull request merged after all the requirements have been met, add the `--auto` flag. To use administrator privileges to immediately merge the pull request, add the `--admin` flag. exit=1 --- gh pr merge --auto as bot (expect: queued, NOT merged): exit=0 {"auto":true,"state":"OPEN"} --- close probe PR unmerged + delete branch ✓ Closed pull request #572 ([probe] adoption-wall bot bypass check (do not merge)) ✓ Deleted branch bot/adoption-wall-probe-20261021184809 {"mergedAt":null,"state":"CLOSED"} 95a63ad refs/heads/main ``` The bot can push a branch, open a PR into `main`, and enable auto-merge. A plain merge is refused because the approval is required. `viewerCanMergeAsAdmin` is `false` for the bot, so the `--admin` merge through the PR-only bypass is **not yet proven**. See the next section: it is now proven. ## Green: bot admin-merge through the PR-only bypass (option B probe, 6:56 PM PDT) Ruleset 13465930 was briefly widened to also cover a throwaway `ruleset-probe/bot-merge` branch. As copilotkit-bot, the probe pushed a `bot/probe-*` branch, opened PR #574 into that base and ran the exact command this PR uses. The ruleset was then restored to the exact pre-probe JSON (diffed). Both probe branches are gone. ``` === option B probe 2026-10-21 06:56:03 PM PDT main=95a63adcca126fdc3902d5ba713d525f3ace5116 created refs/heads/ruleset-probe/bot-merge @ 95a63ad widened include=["~DEFAULT_BRANCH","refs/heads/ruleset-probe/bot-merge"] bypass=["null:always","2:pull_request","2740:always","1108748:pull_request"] --- rules on probe base (as jpr5): deletion ruleset=13465930 pull_request ruleset=13465930 non_fast_forward ruleset=13465930 head branch bot/probe-20261021185645 bot view: pull_requests_only --- control: bot direct push to protected probe base (expect GH013): remote: error: GH013: Repository rule violations found for refs/heads/ruleset-probe/bot-merge. remote: Review all repository rules at https://github.com/CopilotKit/aimock/rules?ref=refs%2Fheads%2Fruleset-probe%2Fbot-merge remote: remote: - Changes must be made through a pull request. remote: To https://github.com/CopilotKit/aimock.git ! [remote rejected] 8856965 -> ruleset-probe/bot-merge (push declined due to repository rule violations) error: failed to push some refs to 'https://github.com/CopilotKit/aimock.git' exit=1 --- bot push head branch: * [new branch] 8856965 -> bot/probe-20261021185645 PR: #574 bot view of PR: {"author":{"login":"copilotkit-bot"},"mergeStateStatus":"BLOCKED","reviewDecision":"REVIEW_REQUIRED","viewerCanMergeAsAdmin":false} --- EXACT #573 command as bot: gh pr merge "bot/probe-20261021185645" --merge --admin gh exit=0 final: state=MERGED mergedBy=app/copilotkit-bot commit=764f186b0b9c09259257a8345021d35b19964897 --- probe base tip: 764f186 refs/heads/ruleset-probe/bot-merge N=574 B=bot/probe-20261021185645 --- rule-suite for the merge: 2026-10-21T18:56:59-07:00 actor=copilotkit-bot[bot] result=bypass after=764f186b0b9c09259257a8345021d35b19964897 2026-10-21T18:56:46-07:00 actor=copilotkit-bot[bot] result=fail after=88569654f9f6e289f58db31ffea08c7c5710ed22 === RESTORE 06:57:12 PM PDT restored include=["~DEFAULT_BRANCH"] DIFF vs pre-probe: IDENTICAL --- delete probe branches: deleted ruleset-probe/bot-merge {"message":"Reference does not exist","documentation_url":"https://docs.github.com/rest/git/refs#delete-a-reference","status":"422"}gh: Reference does not exist (HTTP 422) (no LEFTOVER lines = clean) === post-restore main check 06:57:24 PM PDT bot view: pull_requests_only remote: error: GH013: Repository rule violations found for refs/heads/main. ! [remote rejected] 4837d8c -> main (push declined due to repository rule violations) 95a63ad refs/heads/main ``` - `gh pr merge --merge --admin` merged #574 as `app/copilotkit-bot`, even though `viewerCanMergeAsAdmin` was `false`. - The rule-suites API records the merge as `result=bypass`. - The bot's direct push to the same protected base was rejected (`result=fail`, GH013). - After the restore, a bot direct push to `main` is still rejected. ## After merge The first `safe` + changed + confined run exercises this path on `main`. A `workflow_dispatch` takes that path only when the wall verdict is safe. actionlint: clean. prettier: clean. commitlint: clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated probe of copilotkit-bot's pull-request bypass on ruleset 13465930. Closed without merging.