Repository navigation
Competitor feature-watch report - #581
Merged
Merged
Conversation
jpr5
added a commit
that referenced
this pull request
Oct 26, 2026
…erged PR (#583) ## Problem The weekly competitor feature-watch report PR is opened with the workflow `GITHUB_TOKEN`. GitHub does not start workflows from that token's events, so no CI runs on the PR and it waits for a human push. #581 is today's example. ## Change A run whose only change is `scripts/competitive-watch-state.json` now lands the way the adoption wall does (60fe76c): - `Verify the auto-land diff is confined to the state file`: auto-land only when `git status --porcelain -- docs/ scripts/competitive-watch-state.json` is exactly ` M scripts/competitive-watch-state.json`. - `Mint app token`: same action SHA, app `1108748`, `secrets.DEVOPS_BOT_PRIVATE_KEY`, `contents`/`pull-requests` write only, minted only on the auto-land path. - `Auto-land via bot PR`: per-command auth through `GIT_CONFIG_*` env, `copilotkit-bot[bot]` identity, `bot/competitive-watch-*` branch, stages that one file, `gh pr merge --merge --admin` into `GITHUB_REF_NAME`. If the merge fails, the PR stays open and the job fails. - Any `docs/` change (homepage or migration pages) still goes to the existing review PR on the workflow token. - Visibility: the report stays in the PR body, is now also written to the job summary, and the Slack digest (`SLACK_WEBHOOK`) gets a "merged via a bot PR" line. - Also as in adoption-wall: `HUSKY=0` and `--no-verify` on the bot commit, and `cancel-in-progress: false`. No new secrets. No new actions. ## Red / green RED, workflow: `gh pr checks 581` returns `no checks reported on the 'auto/competitive-matrix-20261026-40297803406' branch` (exit 1). The check-runs API gives `total_count: 0`. RED, Slack contract test (`src/__tests__/competitive-matrix-slack.test.ts`, which runs the real `Notify Slack` step) against the old workflow: ``` × says the bot merged the report, with the digest, when the state auto-landed Tests 1 failed | 4 passed (5) ``` GREEN, same test against this workflow: ``` Tests 5 passed (5) ``` Workflow and competitive tests: 24 files, 1410 passed. Lint: `actionlint` exit 0. `zizmor` 1.26.1 with `.github/zizmor.yml`, offline and online: exit 0, no findings. The end-to-end check is a `workflow_dispatch` run after this merges.
Contributor
|
Closing and reopening to start CI: this PR was opened with GITHUB_TOKEN, so no checks ran. The bot-merge fix (#583) is merged, but the regenerating run fails today because every mock-server.com page returns HTTP 403. |
commit: |
jpr5
added a commit
that referenced
this pull request
Oct 26, 2026
) ## What Follow-up to #583 (and the same bug #581 had). When a refresh needs a human, the review PR was still pushed and opened with `GITHUB_TOKEN`. GitHub does not start workflows from `GITHUB_TOKEN` events, so CI never ran on that PR. Before, in `update-competitive-matrix.yml`: ```yaml - name: Configure git credentials git config --local url."https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/".insteadOf "https://github.com/" - name: Create PR env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` `update-adoption-wall.yml` had the same gap: ```yaml - name: Open review PR env: GH_TOKEN: ${{ github.token }} ``` ## Change - Both review paths push and open their PR with the copilotkit-bot App token. The token comes from the existing "Mint app token" step (same app, same pinned action, same secret, same `contents`/`pull-requests` write scopes). No new secret. - The review PR is **never merged** by the workflow. It waits for a human, as before. - The mint condition is now the union of the two steps that use the token: - competitive matrix: `changed == 'true'` - adoption wall: `exists && changed && (status == 'safe' || status == 'needs-review')` - Least privilege for the workflow token: `pull-requests: write` is gone from both jobs. The competitive-matrix job is now `contents: read`, because its workflow token only reads the API now. - The "Configure git credentials" step is removed. It wrote `GITHUB_TOKEN` into `.git/config`. The review step now passes the App token through git's environment config, like the auto-land step. ## `if:` trace Competitive matrix: | case | mint | auto-land | review PR | |---|---|---|---| | no change | skip | skip | skip | | docs/ changed (confine skipped) | run | skip (`docs_changed`) | run (`confined != 'true'`) | | state only, confined | run | run | skip | | state only, not confined | run | skip | run | Adoption wall: auto-land (`exists && safe && changed && confined`) and review (`exists && changed && (needs-review || (safe && !confined))`) are both subsets of the mint condition. A clean run, a run with no change, or a run with no state branch mints nothing. ## Red-green proof Contract test added to `src/__tests__/competitive-matrix-slack.test.ts` (`review PRs are opened with the App token`), for both workflows. It checks that the review step uses `steps.app-token.outputs.token` for push and PR, that no workflow-token git credential remains, that the review step never merges, and that the mint condition covers the review path. RED (new test against the `origin/main` workflows): ``` ⎯⎯⎯⎯⎯⎯⎯ Failed Tests 4 ⎯⎯⎯⎯⎯⎯⎯ FAIL src/__tests__/competitive-matrix-slack.test.ts > review PRs are opened with the App token > update-competitive-matrix.yml: "Create PR" pushes and opens the PR on the App token FAIL src/__tests__/competitive-matrix-slack.test.ts > review PRs are opened with the App token > update-competitive-matrix.yml: the App token is minted for the review path too FAIL src/__tests__/competitive-matrix-slack.test.ts > review PRs are opened with the App token > update-adoption-wall.yml: "Open review PR" pushes and opens the PR on the App token FAIL src/__tests__/competitive-matrix-slack.test.ts > review PRs are opened with the App token > update-adoption-wall.yml: the App token is minted for the review path too Tests 4 failed | 7 passed (11) ``` GREEN (same test against this branch): ``` Tests 11 passed (11) ``` Full suite: 483 files, 19670 tests passed. `actionlint` exit 0. `zizmor --offline` exit 0 (no findings). typecheck, eslint and prettier clean. ## End-to-end proof Not triggered by hand: mock-server.com currently returns 403, so a run would fail before these steps. The end-to-end proof is the next scheduled Monday run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No homepage competitive matrix changes this week. Detections below need a manual check.
Feature watch changes need a review; see the Feature watch section.
Migration Page Rows To Check By Hand
The competitor's cell in these rows shows no, but the scan does not flip it: a combined row covers several capabilities, and an unsupported cell is not in the page's usual cross shape. A no-row entry is a homepage change whose migration page has no row for the capability: add the row or map it in buildMigrationRowPatterns.
docs/migrate-from-mock-llm/index.htmlRow-Less Detections (Manual Follow-Up)
These rules have no homepage row. The last column says what the competitor's migration page did with each. Check them by hand.
Feature watch
The scan never changes a homepage cell from these checks. For each source that is not "no change", re-check its claims in the MCP spec's claims table (14.2), then edit the homepage by hand.