From 74cfbc5c553e9e21e424576cabd203201e701e52 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 22 Oct 2026 00:16:59 +0000 Subject: [PATCH] docs: explain how CI picks tests Adds a How CI picks tests section to docs/contributing.md and a task map row in docs/agent-architecture.md (spec 023 T048, T049). Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Claude --- docs/agent-architecture.md | 1 + docs/contributing.md | 39 +++++++++++ .../briefs/docs.md | 68 +++++++++++++++++++ .../023-change-based-test-selection/tasks.md | 6 +- 4 files changed, 111 insertions(+), 3 deletions(-) create mode 100644 specs/023-change-based-test-selection/briefs/docs.md diff --git a/docs/agent-architecture.md b/docs/agent-architecture.md index 02e2a47e5..3617b9cf6 100644 --- a/docs/agent-architecture.md +++ b/docs/agent-architecture.md @@ -47,6 +47,7 @@ This document is the **first stop** for AI agents modifying or extending Gamepla | Task | Read | | --- | --- | | Translations, language packs, localized server or module messages (i18n) | `docs/translating.md`, `specs/019-localization-language-packs/contracts/` (`pack-format.md`, `server-messages.md`, `i18n-api.md`, `module-locales.md`, `check-tool.md`), and the Internationalization section of `web/specs.md` | +| CI test selection: which jobs, packages and tests a PR runs (`hack/ci_scope.py`, `hack/check_changed_coverage.py`, `test/e2e/suite-components.json`, the `changes` job in `.github/workflows/ci.yaml`) | `docs/contributing.md` ("How CI picks tests"), `specs/023-change-based-test-selection/contracts/` (`ci-scope-cli.md`, `suite-components.md`, `changed-coverage.md`) | ## How to use this index 1. Identify the domain of your task (e.g., "Add an auth provider"). diff --git a/docs/contributing.md b/docs/contributing.md index affec7301..a73c7e67c 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -72,6 +72,45 @@ These jobs block merging. Each gate with a fixture has a `hack/*_test.sh` self-t - **`codegen drift`** runs `make generate manifests` and `make tidy`, then fails if `git diff` is not empty. Run the same commands locally and commit what they change (CLAUDE.md rule 7). - **`submodule freshness`** runs `hack/check-submodule-freshness.sh`. It fails when the `modules` or `website` pointer names a commit that is not on that repo's `main`, when a gitlink is missing, or when the script cannot fetch that repo's `main`. For a pointer that is not on `main`, merge the submodule PR first, then bump the pointer to the merged commit. Otherwise restore the gitlink, or fix the fetch error, and re-run. +### How CI picks tests + +On a pull request, CI runs only the work your change can affect (spec 023). The +`detect changes` job decides, and its job summary has a **Test selection** table that +lists every suite as `run` or `skip`, with the changed files that chose it or the reason +everything ran. + +What a partial run narrows: + +- **Go unit tests** run only the packages that own a changed file, plus every package + that imports them, across modules. A module's own `go.mod`, `go.sum` or + `.testcoverage.yml` reruns that whole module. The module coverage gate then becomes a + changed-lines gate: the lines you add must meet the module's own minimum + (`hack/check_changed_coverage.py`). +- **Web unit tests** run `vitest related` on the changed `web/src` files, with the same + changed-lines gate at the Vitest `lines` threshold. Lint and build always run whole. + Shared inputs (`package.json`, the lockfile, the Vite/Vitest/TS/ESLint configs, + `src/test/**`, `i18n/locales/**`) run the full suite. +- **Go e2e tests** run only the tests whose components changed, per + `test/e2e/suite-components.json`. Jobs with no selected test are skipped. +- **`ci.yaml` edits** run only the jobs whose lines changed, plus the jobs they need. + `.golangci.yml` runs only the lint legs. Other workflow files run only `workflow lint`. + +Everything still runs on every push to `master`, and when a selection input changes +(`hack/ci_scope.py`, `hack/check_changed_coverage.py`, their tests, `Makefile`, +`go.work`, `go.work.sum`, `.github/actions/**`, `test/e2e/buckets.sh`, +`test/e2e/suite-components.json`). It also runs everything for a `ci.yaml` edit outside +any job or in `changes`/`report`, a path no rule maps, and any selection error. + +To force a full run on your PR, add the label `ci: full-run` (read on the next push), or +start the `ci` workflow by hand on your branch (`workflow_dispatch`). + +When you add an e2e test or a component, keep `test/e2e/suite-components.json` in step: +add the test name pattern under `suites` with the components and paths it exercises, or +list the component under `unexercised` with a reason. The `e2e bucket coverage` job +(`ci_scope.py verify-suites`) fails when a registered test matches no entry or a +component is missing. The full rules are in +`specs/023-change-based-test-selection/contracts/suite-components.md`. + ## Testing Run the whole suite: diff --git a/specs/023-change-based-test-selection/briefs/docs.md b/specs/023-change-based-test-selection/briefs/docs.md new file mode 100644 index 000000000..b1eeaafa0 --- /dev/null +++ b/specs/023-change-based-test-selection/briefs/docs.md @@ -0,0 +1,68 @@ +# Brief: PR 5, docs (T048–T050) + +Branch `claude/project-thread-63yei6-pr5`, stacked on PR 4. Apply exactly; touch nothing else. + +## 1. `docs/contributing.md`: new subsection + +Insert right before the line `## Testing` (it follows the `### Security and drift gates` +list), keeping one blank line before and after: + +```markdown +### How CI picks tests + +On a pull request, CI runs only the work your change can affect (spec 023). The +`detect changes` job decides, and its job summary has a **Test selection** table that +lists every suite as `run` or `skip`, with the changed files that chose it or the reason +everything ran. + +What a partial run narrows: + +- **Go unit tests** run only the packages that own a changed file, plus every package + that imports them, across modules. A module's own `go.mod`, `go.sum` or + `.testcoverage.yml` reruns that whole module. The module coverage gate then becomes a + changed-lines gate: the lines you add must meet the module's own minimum + (`hack/check_changed_coverage.py`). +- **Web unit tests** run `vitest related` on the changed `web/src` files, with the same + changed-lines gate at the Vitest `lines` threshold. Lint and build always run whole. + Shared inputs (`package.json`, the lockfile, the Vite/Vitest/TS/ESLint configs, + `src/test/**`, `i18n/locales/**`) run the full suite. +- **Go e2e tests** run only the tests whose components changed, per + `test/e2e/suite-components.json`. Jobs with no selected test are skipped. +- **`ci.yaml` edits** run only the jobs whose lines changed, plus the jobs they need. + `.golangci.yml` runs only the lint legs. Other workflow files run only `workflow lint`. + +Everything still runs on every push to `master`, and when a selection input changes +(`hack/ci_scope.py`, `hack/check_changed_coverage.py`, their tests, `Makefile`, +`go.work`, `go.work.sum`, `.github/actions/**`, `test/e2e/buckets.sh`, +`test/e2e/suite-components.json`). It also runs everything for a `ci.yaml` edit outside +any job or in `changes`/`report`, a path no rule maps, and any selection error. + +To force a full run on your PR, add the label `ci: full-run` (read on the next push), or +start the `ci` workflow by hand on your branch (`workflow_dispatch`). + +When you add an e2e test or a component, keep `test/e2e/suite-components.json` in step: +add the test name pattern under `suites` with the components and paths it exercises, or +list the component under `unexercised` with a reason. The `e2e bucket coverage` job +(`ci_scope.py verify-suites`) fails when a registered test matches no entry or a +component is missing. The full rules are in +`specs/023-change-based-test-selection/contracts/suite-components.md`. +``` + +## 2. `docs/agent-architecture.md`: task map row (T049) + +old: +```markdown +| Translations, language packs, localized server or module messages (i18n) | +``` +Keep that whole row unchanged and add this row directly below it: +```markdown +| CI test selection: which jobs, packages and tests a PR runs (`hack/ci_scope.py`, `hack/check_changed_coverage.py`, `test/e2e/suite-components.json`, the `changes` job in `.github/workflows/ci.yaml`) | `docs/contributing.md` ("How CI picks tests"), `specs/023-change-based-test-selection/contracts/` (`ci-scope-cli.md`, `suite-components.md`, `changed-coverage.md`) | +``` + +## 3. `specs/023-change-based-test-selection/tasks.md` + +Change `- [ ] T048`, `- [ ] T049` and `- [ ] T050` to `- [X]`. + +## 4. Checks + +Only check that the links and paths named above exist (`ls`). No link checker, no lint. diff --git a/specs/023-change-based-test-selection/tasks.md b/specs/023-change-based-test-selection/tasks.md index fc581fc20..af1ebd1bc 100644 --- a/specs/023-change-based-test-selection/tasks.md +++ b/specs/023-change-based-test-selection/tasks.md @@ -157,9 +157,9 @@ Repository root of `GameplanePanel/Gameplane`. Main files: `hack/ci_scope.py`, ` ## Phase 7: Polish & Docs — PR 5 -- [ ] T048 [P] Add a "How CI picks tests" section to `docs/contributing.md`: what a partial run skips, the `ci: full-run` label and manual run, how to add a `suite-components.json` entry, where to read the run summary -- [ ] T049 [P] Add the new selection files to the task map in `docs/agent-architecture.md` -- [ ] T050 Compile and link checks only; open PR 5 with labels `type: docs`, `area: shared` +- [X] T048 [P] Add a "How CI picks tests" section to `docs/contributing.md`: what a partial run skips, the `ci: full-run` label and manual run, how to add a `suite-components.json` entry, where to read the run summary +- [X] T049 [P] Add the new selection files to the task map in `docs/agent-architecture.md` +- [X] T050 Compile and link checks only; open PR 5 with labels `type: docs`, `area: shared` - [ ] T051 After the first `master` push following PR 5's merge, confirm every job ran (FR-006, SC-006) and record it in `validation.md` - [ ] T052 After 30 merged PRs, measure median runner-minutes against the 30 before PR 2 via the Actions API and record SC-002 in `validation.md`; for 60 days review each red `master` run for a selection miss (SC-004)