ci: map ci.yaml edits to the jobs they sit in (spec 023 PR 4)

docs/agent-architecture.md

@@ -47,6 +47,7 @@ This document is the **first stop** for AI agents modifying or extending Gamepla
 | Task | Read |
 | --- | --- |
 | Translations, language packs, localized server or module messages (i18n) | `docs/translating.md`, `specs/019-localization-language-packs/contracts/` (`pack-format.md`, `server-messages.md`, `i18n-api.md`, `module-locales.md`, `check-tool.md`), and the Internationalization section of `web/specs.md` |
+| CI test selection: which jobs, packages and tests a PR runs (`hack/ci_scope.py`, `hack/check_changed_coverage.py`, `test/e2e/suite-components.json`, the `changes` job in `.github/workflows/ci.yaml`) | `docs/contributing.md` ("How CI picks tests"), `specs/023-change-based-test-selection/contracts/` (`ci-scope-cli.md`, `suite-components.md`, `changed-coverage.md`) |
 
 ## How to use this index
 1. Identify the domain of your task (e.g., "Add an auth provider").

docs/contributing.md

@@ -72,6 +72,45 @@ These jobs block merging. Each gate with a fixture has a `hack/*_test.sh` self-t
 - **`codegen drift`** runs `make generate manifests` and `make tidy`, then fails if `git diff` is not empty. Run the same commands locally and commit what they change (CLAUDE.md rule 7).
 - **`submodule freshness`** runs `hack/check-submodule-freshness.sh`. It fails when the `modules` or `website` pointer names a commit that is not on that repo's `main`, when a gitlink is missing, or when the script cannot fetch that repo's `main`. For a pointer that is not on `main`, merge the submodule PR first, then bump the pointer to the merged commit. Otherwise restore the gitlink, or fix the fetch error, and re-run.
 
+### How CI picks tests
+
+On a pull request, CI runs only the work your change can affect (spec 023). The
+`detect changes` job decides, and its job summary has a **Test selection** table that
+lists every suite as `run` or `skip`, with the changed files that chose it or the reason
+everything ran.
+
+What a partial run narrows:
+
+- **Go unit tests** run only the packages that own a changed file, plus every package
+  that imports them, across modules. A module's own `go.mod`, `go.sum` or
+  `.testcoverage.yml` reruns that whole module. The module coverage gate then becomes a
+  changed-lines gate: the lines you add must meet the module's own minimum
+  (`hack/check_changed_coverage.py`).
+- **Web unit tests** run `vitest related` on the changed `web/src` files, with the same
+  changed-lines gate at the Vitest `lines` threshold. Lint and build always run whole.
+  Shared inputs (`package.json`, the lockfile, the Vite/Vitest/TS/ESLint configs,
+  `src/test/**`, `i18n/locales/**`) run the full suite.
+- **Go e2e tests** run only the tests whose components changed, per
+  `test/e2e/suite-components.json`. Jobs with no selected test are skipped.
+- **`ci.yaml` edits** run only the jobs whose lines changed, plus the jobs they need.
+  `.golangci.yml` runs only the lint legs. Other workflow files run only `workflow lint`.
+
+Everything still runs on every push to `master`, and when a selection input changes
+(`hack/ci_scope.py`, `hack/check_changed_coverage.py`, their tests, `Makefile`,
+`go.work`, `go.work.sum`, `.github/actions/**`, `test/e2e/buckets.sh`,
+`test/e2e/suite-components.json`). It also runs everything for a `ci.yaml` edit outside
+any job or in `changes`/`report`, a path no rule maps, and any selection error.
+
+To force a full run on your PR, add the label `ci: full-run` (read on the next push), or
+start the `ci` workflow by hand on your branch (`workflow_dispatch`).
+
+When you add an e2e test or a component, keep `test/e2e/suite-components.json` in step:
+add the test name pattern under `suites` with the components and paths it exercises, or
+list the component under `unexercised` with a reason. The `e2e bucket coverage` job
+(`ci_scope.py verify-suites`) fails when a registered test matches no entry or a
+component is missing. The full rules are in
+`specs/023-change-based-test-selection/contracts/suite-components.md`.
+
 ## Testing
 
 Run the whole suite:

hack/ci_scope.py

@@ -22,8 +22,8 @@
 ]
 # Mirrors the paths-filter `ci` filter in .github/workflows/ci.yaml; keep both in sync.
 FORCE_ALL_FILES = {
-    ".github/workflows/ci.yaml", "hack/ci_scope.py", "hack/test_ci_scope.py", "Makefile",
-    "go.work", "go.work.sum", "test/e2e/buckets.sh", ".golangci.yml",
+    "hack/ci_scope.py", "hack/test_ci_scope.py", "Makefile", "go.work", "go.work.sum",
+    "test/e2e/buckets.sh", "test/e2e/suite-components.json",
     "hack/check_changed_coverage.py", "hack/test_check_changed_coverage.py",
 }
 FORCE_ALL_PREFIXES = (".github/actions/",)
@@ -33,7 +33,7 @@
 SUITES_FILE = "test/e2e/suite-components.json"
 # Paths no Go e2e test reads. Their own jobs keep their paths-filter gates.
 NO_E2E_FILES = {
-    "design.pen", "LICENSE", ".claudeignore", ".coderabbit.yaml", ".editorconfig",
+    "design.pen", "LICENSE", ".claudeignore", ".coderabbit.yaml", ".editorconfig", ".golangci.yml",
     ".gitattributes", ".gitignore", ".gitmodules", ".hadolint.yaml",
     "deploy/kind/up.sh", "deploy/kind/down.sh",
     "test/e2e/joincoverage.sh", "test/e2e/joincoverage_test.sh",
@@ -63,6 +63,49 @@
 # Web files that change every Vitest run (research R6).
 WEB_ALL_FILES = {"web/package.json", "web/package-lock.json", "web/vitest.config.ts",
                  "web/vite.config.ts", "web/eslint.config.js"}
+CI_FILE = ".github/workflows/ci.yaml"
+# ci.yaml jobs that compute or report the selection; an edit there forces full scope (R8).
+CI_SELECTION_JOBS = {"changes", "report"}
+CI_JOB_KEY = re.compile(r"^  ([A-Za-z0-9_-]+):\s*(?:#.*)?$")
+CI_TOP_KEY = re.compile(r"^[A-Za-z_][\w-]*:")
+# e2e jobs whose own switch output a ci.yaml edit inside them turns on (mode e2e).
+E2E_JOB_SWITCHES = {
+    "e2e-multicluster": "e2e-multicluster", "e2e-multicluster-arm64": "e2e-multicluster",
+    "e2e-upgrade": "e2e-upgrade", "e2e-upgrade-arm64": "e2e-upgrade",
+    "e2e-web-live": "e2e-web-live", "e2e-web-live-arm64": "e2e-web-live",
+    "e2e-capture-overhead": "e2e-capture-overhead",
+}
+# Summary row prefixes and the ci.yaml jobs behind them; the first match wins.
+ROW_JOBS = (
+    ("lint (", ("lint",)),
+    ("go (test/e2e)", ("go-e2e-unit",)),
+    ("go (", ("go",)),
+    ("api (postgres)", ("api-postgres",)),
+    ("web e2e (mock)", ("web-e2e-mock",)),
+    ("web", ("web",)),
+    ("helm lint", ("helm",)),
+    ("chart render", ("chart-template",)),
+    ("standalone Compose smoke", ("standalone-compose",)),
+    ("workflow lint", ("workflow-lint",)),
+    ("docs and repository checks", ("docs",)),
+    ("codegen drift", ("codegen-drift",)),
+    ("submodule freshness", ("submodule-freshness",)),
+    ("vuln (go)", ("vuln-go",)),
+    ("capture-sidecar CAP_NET_RAW survival", ("capture-sidecar-setcap-proof",)),
+    ("dependency review", ("dependency-review",)),
+    ("telemetry default gate", ("telemetry-default-gate",)),
+    ("build e2e images", ("build-images", "build-images-arm64")),
+    ("vuln (images)", ("vuln-images",)),
+    ("e2e bucket coverage", ("e2e-buckets",)),
+    ("e2e multicluster", ("e2e-multicluster", "e2e-multicluster-arm64")),
+    ("e2e upgrade", ("e2e-upgrade", "e2e-upgrade-arm64")),
+    ("e2e postgres", ("e2e-postgres",)),
+    ("e2e game bot", ("e2e-game-bot",)),
+    ("e2e web live", ("e2e-web-live", "e2e-web-live-arm64")),
+    ("e2e capture overhead", ("e2e-capture-overhead",)),
+    ("ingress-nginx smoke", ("e2e-kind-ingress-smoke",)),
+    ("e2e ", ("e2e-go", "e2e-go-arm64")),
+)
 
 
 def is_unmapped(path, modules):
@@ -113,9 +156,10 @@ def changed_paths(root, base, head, *, history=False):
 def affected_modules(root, paths):
     """Select changed modules and the reverse dependency closure, including tests."""
     modules = workspace_modules(root)
-    shared = {"Makefile", "go.work", "go.work.sum", ".golangci.yml", "test/e2e/buckets.sh"}
+    # ci.yaml and .golangci.yml map to their own jobs through ci_jobs() (OD-3).
+    shared = {"Makefile", "go.work", "go.work.sum", "test/e2e/buckets.sh"}
     if paths is None or any(
-        p in shared or p.startswith((".github/", "hack/ci_scope", "hack/test_ci_scope",
+        p in shared or p.startswith((".github/actions/", "hack/ci_scope", "hack/test_ci_scope",
                                      "hack/check_changed_coverage",
                                      "hack/test_check_changed_coverage"))
         or Path(p).name in {"go.mod", "go.sum"}
@@ -607,6 +651,120 @@ def unit_outputs(root, paths, reason):
     }
 
 
+def ci_blocks(text):
+    """Return ({line number: job id or None}, {job id: needed job ids}) for a workflow."""
+    owner, needs = {}, {}
+    job, in_jobs, collecting = None, False, None
+    for number, line in enumerate(text.splitlines(), 1):
+        if CI_TOP_KEY.match(line):
+            in_jobs, job, collecting = line.split(":", 1)[0] == "jobs", None, None
+        elif in_jobs and CI_JOB_KEY.match(line):
+            job, collecting = CI_JOB_KEY.match(line)[1], None
+            needs.setdefault(job, set())
+        elif job and line.startswith("    needs:"):
+            collecting = job
+            line = line[len("    needs:"):]
+        elif re.match(r"^    \S", line):
+            collecting = None
+        owner[number] = job
+        if collecting:
+            needs[collecting].update(re.findall(r"[A-Za-z0-9_][A-Za-z0-9_-]*",
+                                                line.split("#", 1)[0]))
+    return owner, needs
+
+
+def changed_ci_lines(root, base, head):
+    """Return ([(old line, text)], [(new line, text)]) for ci.yaml lines changed base..head."""
+    diff = subprocess.run(["git", "diff", "-U0", "--no-color", base, head, "--", CI_FILE],
+                          cwd=root, check=True, capture_output=True, text=True).stdout
+    removed, added, old, new, started = [], [], 0, 0, False
+    for line in diff.splitlines():
+        hunk = re.match(r"^@@ -(\d+)(?:,\d+)? \+(\d+)(?:,\d+)? @@", line)
+        if hunk:
+            old, new, started = int(hunk[1]), int(hunk[2]), True
+        elif started and line.startswith("-"):
+            removed.append((old, line[1:]))
+            old += 1
+        elif started and line.startswith("+"):
+            added.append((new, line[1:]))
+            new += 1
+    return removed, added
+
+
+def git_text(root, revision, path):
+    """Return a file's text at revision, or "" when it does not exist there."""
+    result = subprocess.run(["git", "show", f"{revision}:{path}"], cwd=root,
+                            capture_output=True, text=True, check=False)
+    return result.stdout if result.returncode == 0 else ""
+
+
+def ci_jobs(root, base, head, paths):
+    """Map shared CI file changes to ci.yaml job ids (research R8, OD-3).
+
+    Returns (jobs, forced). jobs holds every job whose ci.yaml lines changed, `lint` for
+    .golangci.yml, `workflow-lint` for any .github/ or .golangci.yml change, and every job
+    they need. forced is True when a changed ci.yaml line that is not blank or a comment
+    sits outside any job or inside the changes or report job.
+    """
+    jobs, forced, needs = set(), False, {}
+    if any(p.startswith(".github/") or p == ".golangci.yml" for p in paths):
+        jobs.add("workflow-lint")
+    if ".golangci.yml" in paths:
+        jobs.add("lint")
+    if CI_FILE in paths:
+        removed, added = changed_ci_lines(root, base, head)
+        for revision, changes in ((base, removed), (head, added)):
+            owner, found = ci_blocks(git_text(root, revision, CI_FILE))
+            needs.update(found)
+            for number, text in changes:
+                if not text.strip() or text.strip().startswith("#"):
+                    continue
+                job = owner.get(number)
+                if job is None or job in CI_SELECTION_JOBS:
+                    forced = True
+                else:
+                    jobs.add(job)
+    queue = list(jobs)
+    while queue:
+        for name in needs.get(queue.pop(), ()):
+            if name not in jobs:
+                jobs.add(name)
+                queue.append(name)
+    return sorted(jobs - CI_SELECTION_JOBS), forced
+
+
+def scoped(env, base, head):
+    """Return (paths, reason, ci.yaml jobs): the diff, why everything runs, edited jobs."""
+    event_reason = full_scope_reason(env, [])
+    paths = None if event_reason else changed_paths(ROOT, base, head)
+    reason = full_scope_reason(env, paths)
+    if reason:
+        return paths, reason, []
+    jobs, forced = ci_jobs(ROOT, base, head, paths)
+    return paths, ("selection-rule change" if forced else None), jobs
+
+
+def widen_e2e(values, full, jobs):
+    """Run every test of each e2e job whose ci.yaml block changed (research R8)."""
+    if {"e2e-go", "e2e-go-arm64"} & set(jobs):
+        values["e2e-run"] = full["e2e-run"]
+    for job, key in E2E_JOB_SWITCHES.items():
+        if job in jobs:
+            values[key] = True
+    if "e2e-game-bot" in jobs:
+        values["bot-matrix"], values["bots-any"] = full["bot-matrix"], full["bots-any"]
+    return values
+
+
+def widen_units(values, jobs):
+    """Run every package or web test of a go or web job whose ci.yaml block changed."""
+    if "go" in jobs:
+        values["go-packages"] = {m: ["./..."] for m in values["go-packages"]}
+    if "web" in jobs:
+        values["web-tests"] = "all"
+    return values
+
+
 def verify_suites(suites, buckets, modules):
     """Return the violations of contracts/suite-components.md rules 1, 2 and 5."""
     errors = []
@@ -650,6 +808,7 @@ def summarize(env, paths, reason):
     bots = load_json(env, "BOT_MATRIX")
     e2e = load_json(env, "E2E_OUTPUTS")
     units = load_json(env, "UNITS_OUTPUTS")
+    edited_jobs = set(jlist(env, "CI_JOBS"))
     go_selected = decoded(units, "go-packages", {})
     web_selected = decoded(units, "web-tests", "all")
     # Without the e2e step's outputs every e2e suite follows the coarse e2e gate.
@@ -706,6 +865,9 @@ def go_note(module):
 
     def add(suite, run, feed=(), run_reason="", skip_reason="not affected", note=""):
         """Append one selection row with a run or skip decision and its reason."""
+        jobs = next((found for prefix, found in ROW_JOBS if suite.startswith(prefix)), ())
+        if not run and edited_jobs & set(jobs):
+            run, run_reason = True, "ci.yaml: lines in this job changed"
         if run and reason:
             why = f"full: {reason}"
         elif run and run_reason:
@@ -805,7 +967,7 @@ def bot_matrix(names):
 def main():
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument("mode", choices=("modules", "images", "dockerfiles", "bots", "summary",
-                                         "e2e", "verify-suites", "units"))
+                                         "e2e", "verify-suites", "units", "jobs"))
     parser.add_argument("--base", default="")
     parser.add_argument("--head", default="HEAD")
     args = parser.parse_args()
@@ -853,25 +1015,32 @@ def main():
     if args.mode == "e2e":
         env = os.environ
         try:
-            event_reason = full_scope_reason(env, [])
-            paths = None if event_reason else changed_paths(ROOT, args.base, args.head)
-            values = e2e_outputs(ROOT, paths, full_scope_reason(env, paths))
+            paths, reason, jobs = scoped(env, args.base, args.head)
+            values = e2e_outputs(ROOT, paths, reason)
+            if jobs:
+                values = widen_e2e(values, e2e_outputs(ROOT, None, "ci.yaml job edit"), jobs)
         except Exception as exc:  # any doubt selects every e2e test (FR-002)
             print(f"::warning::e2e selection failed, running every e2e test: {exc}")
             values = e2e_outputs(ROOT, None, f"error: {exc}")
         emit(values)
         return
+    if args.mode == "jobs":
+        try:
+            _, reason, jobs = scoped(os.environ, args.base, args.head)
+        except Exception as exc:  # any doubt selects the full suite (FR-002)
+            print(f"::warning::ci.yaml job mapping failed, running everything: {exc}")
+            reason, jobs = f"error: {exc}", []
+        emit({"all": bool(reason), "reason": reason or "", "ci-jobs": jobs})
+        return
     if args.mode == "units":
         env = os.environ
         try:
-            event_reason = full_scope_reason(env, [])
-            paths = None if event_reason else changed_paths(ROOT, args.base, args.head)
-            reason = full_scope_reason(env, paths)
+            paths, reason, jobs = scoped(env, args.base, args.head)
             modules = workspace_modules(ROOT)
             unmapped = [p for p in paths or [] if is_unmapped(p, modules)]
             if reason is None and unmapped:
                 reason = f"unmapped path {unmapped[0]}"
-            values = unit_outputs(ROOT, paths, reason)
+            values = widen_units(unit_outputs(ROOT, paths, reason), jobs)
         except Exception as exc:  # any doubt runs every unit test (FR-002)
             print(f"::warning::unit selection failed, running every unit test: {exc}")
             values = unit_outputs(ROOT, None, f"error: {exc}")
@@ -881,14 +1050,15 @@ def main():
     # label is the full-suite backstop.
     env = os.environ
     try:
-        event_reason = full_scope_reason(env, [])
-        paths = None if event_reason else changed_paths(ROOT, args.base, args.head)
+        paths, reason, jobs = scoped(env, args.base, args.head)
+        if paths is not None and reason and full_scope_reason(env, paths) is None:
+            paths = None  # a ci.yaml edit outside any job or in changes/report (R8)
         if paths is not None and any(is_unmapped(p, workspace_modules(ROOT)) for p in paths):
             paths = None  # mode e2e reports the path; every module runs (FR-002)
         selected = affected_modules(ROOT, paths)
     except Exception as exc:  # any doubt selects the full suite (FR-002)
         print(f"::warning::test selection failed, running everything: {exc}")
-        paths = None
+        paths, jobs = None, []
         selected = sorted(workspace_modules(ROOT))
     modules = sorted(workspace_modules(ROOT))
     emit({
@@ -897,8 +1067,11 @@ def main():
         "lint": bool(selected),
         "go": bool(set(selected) - {"test/e2e"}),
         "e2e-unit": "test/e2e" in selected,
-        "lint-exclude": [{"module": m} for m in modules if m not in selected],
-        "go-exclude": [{"module": m} for m in modules if m not in selected and m != "test/e2e"],
+        # An edit inside the lint or go job of ci.yaml runs that job on every module.
+        "lint-exclude": [{"module": m} for m in modules
+                         if m not in selected and "lint" not in jobs],
+        "go-exclude": [{"module": m} for m in modules
+                       if m not in selected and m != "test/e2e" and "go" not in jobs],
     })