diff --git a/.claude/agents/squad-code-officer.md b/.claude/agents/squad-code-officer.md index 5f5f1766..6a68e543 100644 --- a/.claude/agents/squad-code-officer.md +++ b/.claude/agents/squad-code-officer.md @@ -31,10 +31,12 @@ issue and the project's documentation are not needed for format and analyzer fix Limits: no change to behavior, signatures used across files, control flow, test assertions or test data. A new guard, branch, early return, null check or a changed assertion counts as structural: do not make it, hand it back with file, line and rule id to the Dev (production code) or Tester (tests). Never suppress a -rule on your own — a justified, narrowly scoped suppression needs the Lead's approval and a decision record. +rule on your own — a justified, narrowly scoped suppression needs the Lead's approval and a decision record. A finding that +contradicts a documented style rule (`AGENTS.md`, *Code style*; `.squad/stack.md`) is reported as a conflict with +that rule, with the `.editorconfig` severity as the proposed fix, and the code is not restructured around it. -You also receive the findings of the *Sonar gate* (step 7) and, after the PR is open, of the CI code analysis -(`sonar-check.py pr `); treat them like findings of the analyzer gate. For a rule no local analyzer +After the PR is open you also receive the findings of the *Sonar gate*, the CI code analysis of the draft pull +request (`sonar-check.py pr `); treat them like findings of the analyzer gate. For a rule no local analyzer reported, add one line to the *Analyzer gate* section of `.squad/stack.md` (the rule id, what it flags, how to avoid it while writing code) in the product PR (*Scope of a product PR* in `.squad/routing.md`, *Squad lessons* in `.squad/orchestration.md`); no other file under `.squad/` is edited. diff --git a/.claude/agents/squad-dev.md b/.claude/agents/squad-dev.md index eb0aabab..34750dfe 100644 --- a/.claude/agents/squad-dev.md +++ b/.claude/agents/squad-dev.md @@ -55,11 +55,16 @@ test documentation, the area documents with their index rows, `docs/ARCHITECTURE *Coverage gate* passes — for a feature after the last task group, in your `implement` agent. Write each statement from the code as it now stands (`git diff `, the working tree included), not from the plan's wording, and re-read every sentence of the touched sections that describes the changed behavior; run `decision-check.py` when -you touched `docs/areas/`. A statement that would contradict a decision record goes to the Lead in your report. +you touched `docs/areas/`. A sentence you write that states a number (a count, a limit, a size, a budget), an order +or a completeness ("only", "all", "never", "covered by …") names in your report the code line, test or measurement +it rests on; one you cannot trace is narrowed to what the code shows or left out. A statement that would contradict +a decision record goes to the Lead in your report. Before handing over, run *Format* (`.squad/team.md`, *Shared rules*). Do not chase style diagnostics unless the Code Officer hands one back; run the *Analyzer gate* once and fix the findings in your production files -that need a code change, so they do not come back later as a structural hand-back. +that need a code change, so they do not come back later as a structural hand-back. A finding that contradicts a +documented style rule (`AGENTS.md`, *Code style*; `.squad/stack.md`) is not worked around: report it as a conflict +with that rule, name the `.editorconfig` severity as the proposed fix, and leave the code as the rule has it. Tests stay the Tester's beyond the mechanical call-site adaptations above (*Loop limits* in `.squad/routing.md`): a test you believe is wrong, or a plan that does not work, goes back as a report for the Lead, and a deviation from the plan is stated in your report. diff --git a/.claude/agents/squad-tester.md b/.claude/agents/squad-tester.md index d42ab4c4..d0e4f382 100644 --- a/.claude/agents/squad-tester.md +++ b/.claude/agents/squad-tester.md @@ -33,7 +33,9 @@ Mode `tests-first`: For a documented shell procedure derive the cases from the plan's list of what happens when the lines are pasted at once or one at a time, one fails or a variable is lost; a test that reads a repository document normalizes `\r\n` (`docs/UNIT_TESTS.md`); for a script only CI can run, run the local smoke substitute - (*Writing tests* in `.squad/stack.md`) before you hand back. + (*Writing tests* in `.squad/stack.md`) before you hand back. For a case you cannot run locally (a container, a + CI-only script), read *Known pitfalls* in `.squad/stack.md` first and write its commands for the environment they + run in, not for the host's tools; process their output on the host. For a feature, the criteria are `spec.md`'s and their tests the plan's *Tests per criterion*: every field and value a criterion names, and the scenarios and edge cases it lists, are asserted by a test; name one no test asserts in your report. diff --git a/.claude/skills/create-pr/SKILL.md b/.claude/skills/create-pr/SKILL.md index 1d45b13c..96f49d8f 100644 --- a/.claude/skills/create-pr/SKILL.md +++ b/.claude/skills/create-pr/SKILL.md @@ -40,10 +40,9 @@ repository. unformatted code or outstanding analyzer diagnostics. This step is the gate before the PR; CI is not meant to find anything here. 5. **Commit** as `AGENTS.md` (*Commit messages and pull requests*) says, - staging only the files that belong to this task. Then run the *Sonar gate* - from `.squad/stack.md` — SonarQube Cloud's analysis of the committed - branch; fix every finding it lists (and commit again). NOT CHECKED (no - SonarQube credential in this environment) leaves the check to step 9. + staging only the files that belong to this task. The *Sonar gate* from + `.squad/stack.md` reads the CI code analysis of the draft pull request, so it + runs in step 9. 6. **Run the internal review loop** (see below) and resolve what it finds. This happens *before* the push, so the pull request opens on a reviewed change instead of collecting review rounds afterwards. diff --git a/.claude/skills/squad-issue/SKILL.md b/.claude/skills/squad-issue/SKILL.md index c619a94a..61e79f2a 100644 --- a/.claude/skills/squad-issue/SKILL.md +++ b/.claude/skills/squad-issue/SKILL.md @@ -55,7 +55,7 @@ every Git and GitHub action yourself, including follow-up issues the Lead decide **Logging.** Only you write `log.md`, one row per step and per launch, always with `python3 .squad/tools/squad-log.py issue- "" "" ""` — never by hand. - - Write a launch's row as soon as its report is in, with `--agent squad-` alone (usage pending), a continued one too (`--continued` only on the amend). When + - Write a launch's row as soon as its report is in, with `--agent squad-` alone (usage pending; `--launch` for a launch *Launch model* overrides), a continued one too (`--continued` only on the amend). When its task notification arrives, add the usage with `--amend-last "" --agent squad- --tokens … --tool-uses … --seconds …` (`--step ""` for a row that is no longer the member's last). - **Every number is copied from the usage block of a task notification you have in front of you** — @@ -105,7 +105,7 @@ every Git and GitHub action yourself, including follow-up issues the Lead decide signatures and the existing test call sites the plan assigns to the Dev. A plan that declares the change free of production and test code skips steps 4 and 5 (*Changes without production or test code*): log the skipped steps and run the plan's *Verification without tests* instead. -5. **Tests first** (not for `trivial`). Launch `squad-tester` in mode `tests-first`. Confirm yourself that the +5. **Tests first** (not for `trivial`). Launch `squad-tester` in mode `tests-first`, with the model *Launch model* names. Confirm yourself that the new tests compile and fail on the current code (unless the Tester justified why one cannot); note the tests it hands back as **deferred**. A fix without a reproducing test is acceptable only when the bug genuinely needs a live external system — then the PR says so. @@ -124,9 +124,9 @@ every Git and GitHub action yourself, including follow-up issues the Lead decide reported as not unit-testable go to `squad-lead` in mode `decide`. 7. **Code check.** `git fetch origin main` and merge `main` first when it moved (*Merge conflicts*). Run the gates of step 7 (*Pipeline*; for a change without code and for `patch` their sections) with the commands from `.squad/stack.md`, - once they pass, commit, then run the *Sonar gate*; keep the output for step 8. When *Format check* fails, run *Format*, + once they pass, commit (the *Sonar gate* runs on the draft PR in step 11). When *Format check* fails, run *Format*, `git restore -- ` every file it changed outside the change, and check again. Launch - `squad-code-officer` with the base ref only for analyzer diagnostics, Sonar findings or style findings that + `squad-code-officer` with the base ref only for analyzer diagnostics or style findings that need reading; structural items it hands back go to `squad-dev` (or `squad-tester`), a scope finding to its owner, a file it formatted outside the change you restore. Run the gates again until they pass. 8. **Review.** Launch `squad-reviewer` (round 1, full) and — on tier `security` — `squad-security` in mode diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index fd359615..ae0d54cd 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -1,7 +1,7 @@ --- name: Bug report about: Report a problem -title: "[BUG] " +title: "" labels: bug assignees: '' diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 2d8e2502..276f8a83 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -1,7 +1,7 @@ --- name: Feature request about: Suggest an idea for this project -title: "[Feature] " +title: "" labels: enhancement assignees: '' diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 48e1fedb..0bf62a23 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -192,7 +192,7 @@ and the round semantics. The squad's members (`.squad/team.md`) run as subagents commits, pushes and posts. The pipeline, its limits and escalations are in [`.squad/routing.md`](/.squad/routing.md), the project's guarantees, security areas and integration surface in [`.squad/project.md`](/.squad/project.md). Working records stay under `specs/` on the work branch. A product PR never changes the squad or these -instructions (`.squad/stack.md`, `.squad/project.md` and the project blocks of this file excepted); squad lessons become `squad` issues where they can be fixed (`.squad/orchestration.md`, *Squad +instructions (`.squad/stack.md`, `.squad/project.md`, `.squad/tools/squad_settings.py` and the project blocks of this file excepted); squad lessons become `squad` issues where they can be fixed (`.squad/orchestration.md`, *Squad lessons*). The user acts as Product Manager and is asked only when the Lead escalates. The reasoning behind code decisions — why something was built the way it was — is recorded by the Lead diff --git a/.squad/orchestration.md b/.squad/orchestration.md index 83f963b7..d890f586 100644 --- a/.squad/orchestration.md +++ b/.squad/orchestration.md @@ -57,6 +57,14 @@ template — is filed in the template repository, which the session attaches fir to create the issue, e.g. `add_repo`); only if attaching is refused it is filed here with the label `squad-upstream`, the user is told to move it, and it is never worked here. +## Launch model + +Every member is launched with the model and effort of its agent file, except: every `squad-tester` launch of a run +whose `plan.md` names tier `security` gets `model: opus` (effort unchanged); a continued agent keeps the model it +was launched with. Log such a launch with `--launch opus/` instead of `--agent squad-tester`, on the +pending row and on its amend; `squad-log.py` refuses `--agent` alone there, so the summary shows the model the +launch really ran on. + ## Continuing a member A fix round or a delta round of the same member within the same run continues that member's last launch @@ -82,7 +90,8 @@ detached HEAD is switched back to it first (`git switch `, then `git mer commits made on the detached HEAD) and logged with the launch that left it. Push without `-q` and check after every push that `git rev-parse origin/` equals `git rev-parse HEAD`: a push that did not move the branch is a failure, not a success. After the PR every push re-triggers CI and the code analysis, so edits are -batched and committed once per round. +batched and committed once per round, and a tests-first commit that fails on purpose is pushed only together +with the implementation that makes it pass: the head of an open PR never fails by design. ## The stop hook diff --git a/.squad/routing.md b/.squad/routing.md index f3bd4ef8..e5d0481c 100644 --- a/.squad/routing.md +++ b/.squad/routing.md @@ -53,7 +53,8 @@ Squad lessons are filed in step 12 (*Squad lessons*), never fixed in the product Reviewer in step 8, and a finding is blocking. One exception: `.squad/stack.md`, `.squad/project.md` and the project blocks of `AGENTS.md` describe the -product, not the squad. A product PR updates them when the change itself makes them untrue — a new build command, +product, not the squad. `.squad/tools/squad_settings.py` is the repository's own settings file and a product PR may +change it too (a coverage exclusion, *Code a unit test cannot cover*; a threshold is never lowered). A product PR updates them when the change itself makes them untrue — a new build command, a new security area, a new guarantee or coupling point — without an escalation, and the Reviewer treats a stale entry there like stale documentation. A project block of `AGENTS.md` changes identically in `.github/copilot-instructions.md` (`scope-check.py` compares them). @@ -159,10 +160,21 @@ generated file, process startup glue — is found in the plan, not after the PR: `plan.md` with the reason, the Tester confirms or contests the list in its report, and the Lead's decision is a decision record. Such a file is excluded from the coverage **measure** only, in every place that computes it (`sonar.coverage.exclusions` for the CI code analysis, and `COVERAGE_EXCLUDES` in -`.squad/tools/squad_settings.py` where `COVERAGE_PATHSPECS` includes the file; *Coverage exclusions* in -`.squad/stack.md` names where the repository sets them) — never from the analysis (`sonar.exclusions`, a +`.squad/tools/squad_settings.py` where `COVERAGE_PATHSPECS` includes the file — the product PR edits that file +itself; *Coverage exclusions* in `.squad/stack.md` names where the repository sets them) — never from the analysis (`sonar.exclusions`, a suppression): its rules still apply, and those no local gate reports arrive in step 11. +### Early draft PR + +CI runs on pull requests only, so a change whose **CI-only criteria** (*Criteria verified only in CI*) or **dry +run** (*Operator environments and documented procedures*) needs a CI result before the review, and a PR opened in +step 10 comes too late for it. For such a change the plan says so, and the orchestrator opens the PR as a draft +right after step 7 (`create-pr` step 8, `specs/` still in the diff, no working record comment yet), then waits for CI +and reads `sonar-check.py pr ` before step 8; findings are worked as in step 11. The *Sonar gate* alone is no +reason for an early PR: every push to an open PR runs all of CI again, and step 11 reads the analysis anyway. The dry run runs on that head, before the documentation is +reviewed. Step 9 additionally requires the green CI run, the dry-run result and the Reviewer's round on the same +head; step 10 then only posts the working record, removes `specs/` and marks nothing ready (*Draft until green*). + ## Pipeline | # | Step | Owner | Exit condition | @@ -173,10 +185,10 @@ suppression): its rules still apply, and those no local gate reports arrive in s | 4 | Skeleton | Dev | Only when the plan adds or changes API: compile-only signatures built as *Skeleton* in `.squad/stack.md` describes, *Build* passes | | 5 | Tests first | Tester | Only the tests the plan names for its acceptance criteria (one or a few per criterion); they compile and **fail** on the current code or the skeleton. A plan-named test that cannot fail yet is *deferred* to step 6; other guards, pins of existing behavior, determinism and bound tests are not written | | 6 | Implementation + coverage | Dev, Tester | All tests green; *Coverage gate* passes; then the plan's doc updates, area documents included, written by the Dev against the code as built (`squad-dev.md`, *Documentation*). The Tester's coverage pass adds tests only for uncovered lines, a resource bound the plan names, a deferred or missing plan-named test, or a test a reviewer asks for | -| 7 | Code check | Orchestrator, Code Officer | `main` merged first when it moved (*Merge conflicts*). The orchestrator runs *Format check*, *Analyzer gate*, *Test*, *Coverage gate* and `scope-check.py`, commits, then runs the *Sonar gate* (NOT CHECKED — no SonarQube credential, or no scanner download — is logged and leaves it to step 11). It clears a *Format check* failure itself (*Format*, then `git restore` of every file outside the change); analyzer diagnostics, Sonar findings and style findings that need reading launch the Code Officer (structural items go to the Dev or Tester), then the gates run again. Exit: every gate passes on the head | +| 7 | Code check | Orchestrator, Code Officer | `main` merged first when it moved (*Merge conflicts*). The orchestrator runs *Format check*, *Analyzer gate*, *Test*, *Coverage gate* and `scope-check.py`, then commits; the *Sonar gate* runs in step 11 on the draft PR. It clears a *Format check* failure itself (*Format*, then `git restore` of every file outside the change); analyzer diagnostics and style findings that need reading launch the Code Officer (structural items go to the Dev or Tester), then the gates run again. Exit: every gate passes on the head | | 8 | Review | Reviewer (+ Security's diff review on `security`, and on an unlisted security area) | They get the code SHA and the orchestrator's gate output of step 7 (they do not re-run the gates); later commits during the round may touch only `specs/` and do not make it stale. No blocking findings → 9; blocking → owner fixes (Dev: code and documentation, Tester: tests), back to 6, then a mandatory delta round — except a verbatim documentation fix (*Verbatim documentation fixes*) | -| 9 | PR approval | Orchestrator, Lead when needed | *PR approval* below | -| 10 | Pull request | Orchestrator | Working record posted as comment (a long `plan.md` may be given as a permalink to the last commit that contains it plus a summary), `specs//` removed and `scope-check.py --no-specs` clean, PR opened **as a draft** (*Draft until green*; merged later with *Squash and merge*) | +| 9 | PR approval | Orchestrator, Lead when needed | *PR approval* below; with an *Early draft PR* also green CI, the dry-run result and the Reviewer's round on the same head | +| 10 | Pull request | Orchestrator | Working record posted as comment (a long `plan.md` may be given as a permalink to the last commit that contains it plus a summary), `specs//` removed and `scope-check.py --no-specs` clean, PR opened **as a draft** (*Draft until green*; merged later with *Squash and merge*) — or, after an *Early draft PR*, only the record and the removal | | 11 | After the PR | Dev, Code Officer, Reviewer | *Draft until green* holds and the PR is marked ready for review; review comments worked | | 12 | Wrap-up | Orchestrator | Squad lessons filed as one issue per destination (*Squad lessons*), or "no lessons" logged; the run metrics of `squad-log.py --summary` (exit 0) appended to the working record exactly as printed and reported to the user | @@ -275,6 +287,11 @@ PR is opened, the branch stays without one, and the user is told so. cover*) is a plan defect the Lead records with the decision. - **After the PR (step 11):** CI or quality-gate failures and review comments are fixed on the same branch through steps 7–8 again (delta review), at most 2 fix rounds per failure; then the Lead decides. +- **New scope on the open PR (step 11):** a Product Manager request on the open pull request that adds or + changes behavior beyond the spec's acceptance criteria (not a criterion the change misses) goes to the Lead + before any edit. The Lead decides, recorded in `log.md`: a follow-up issue (the default), or this change grows, + and then the spec gets the new criteria and the plan a new task group, which runs steps 3–8 like the others + (plan challenge and Security plan review as the tier asks) before step 9 approves again. - **Safeguard stop (any member, any step):** an API `invalid_request` of category `cyber` is not an API error: relaunching unchanged loops, and rephrasing the request is a workaround. The orchestrator escalates to the Product Manager at once, naming the finding the member had not reviewed as unverified. diff --git a/.squad/stack.md b/.squad/stack.md index 170129e4..1e5000a6 100644 --- a/.squad/stack.md +++ b/.squad/stack.md @@ -40,7 +40,7 @@ profile and owned by this repository: keep it true when the build changes. | *Test with coverage* | `rm -rf TestResults && dotnet test DockerUpdateGuard.slnx -c Release --no-build --collect:"XPlat Code Coverage" --results-directory ./TestResults` (clears earlier runs: the coverage gate merges every report it finds) | | *Coverage gate* | `python3 .squad/tools/coverage-check.py` | | *Analyzer gate* | `python3 .squad/tools/analyzer-check.py` | -| *Sonar gate* | `python3 .squad/tools/sonar-check.py branch -- bash -c 'dotnet sonarscanner begin /k:"LarsLaskowski_DockerUpdateGuard" /o:"networlddev" ${SONAR_TOKEN:+/d:sonar.token="$SONAR_TOKEN"} /d:sonar.exclusions="docs/**" /d:sonar.branch.name="$(git branch --show-current)" /d:sonar.projectBaseDir="$PWD" && dotnet build DockerUpdateGuard.slnx -c Release --no-incremental && dotnet sonarscanner end ${SONAR_TOKEN:+/d:sonar.token="$SONAR_TOKEN"}'` (after *Test with coverage*; needs a SonarQube credential and the `dotnet-sonarscanner` tool, see *Sonar gate* below; the token is passed only when `SONAR_TOKEN` is set) | +| *Sonar gate* | `python3 .squad/tools/sonar-check.py pr ` (step 11, on the draft pull request once its CI has finished; see *Sonar gate* below) | | *Image build check* | `docker build -f src/DockerUpdateGuard/Dockerfile .` where Docker runs; otherwise the restore-order reproduction under *Image build check* below (`.squad/routing.md`, *Criteria verified only in CI*) | ## Analyzer gate @@ -57,8 +57,7 @@ underscores in member names, a region for an interface implementation named afte SonarQube Cloud's own quality profile can still report `S####` rules the local default profile does not, and its non-Roslyn checks (hotspots, taint analysis, duplication) have no local equivalent: the *Sonar gate* -below is the first to report their issues and hotspots, and squad step 11 reads the quality gate, -duplication included. +below reports their issues and hotspots and the quality gate, duplication included, on the draft pull request. One such C# rule that bit (`csharpsquid:S2589`, "condition always evaluates to True"): a nullable local that is assigned a value already checked as non-null and then tested `is not null` in a loop condition (`while (requestUri is not null)`). @@ -84,26 +83,15 @@ version follows the base image's repository and a pin breaks the build when it m `# hadolint ignore=DL3018` line above it, with the reason. Keep that line when the Dockerfile changes; any other hadolint finding is fixed, not ignored. -*Sonar gate* runs SonarQube Cloud's own analysis on the work branch before the pull request (squad step 7): -`sonar-check.py` runs the scanner with the branch name, waits for the analysis and fails on every open issue or -security hotspot on a line the branch changed, so the rules no local analyzer has stop surfacing only after the -pull request. It runs on the committed change (the scanner uploads the working tree, so it refuses uncommitted or -untracked files other than the coverage reports) and needs a SonarQube credential: a network secret of the cloud environment for `sonarcloud.io` and -`*.sonarcloud.io` (the agent proxy attaches the token to those requests, so no session or member sees it), or -else `SONAR_TOKEN` in the environment, which every process of the session can read. The scanner also needs -network access to its download hosts (`api.sonarcloud.io` and `scanner.sonarcloud.io`, which the secret's -hosts cover, and the package registry); without them it reports NOT CHECKED, which on a branch without a pull request yet is not final: step 11 reads the draft pull -request with `python3 .squad/tools/sonar-check.py pr ` instead, and that result counts. The SessionStart hook restores -`dotnet-sonarscanner` from the tool manifest and puts `~/.dotnet/tools` on the `PATH`; a failed `sonar-check.py` read names the request and -the response body (HTTP status included), so the cause of an HTTP error can be told from the output. A finding of a rule no local analyzer reports is noted in this section, in the same change, instead of a +*Sonar gate* reads the CI code analysis of the draft pull request (squad step 11, *Draft until green* in +`.squad/orchestration.md`): `sonar-check.py pr ` checks that the analysis belongs to the pushed head and +fails on a failed quality gate, on every open issue and on every security hotspot to review. The work branch is +not scanned before the pull request: SonarQube Cloud does not let every organization read the data of non-main +branches through its API. A private project needs a SonarQube credential: a network secret of the cloud +environment for `sonarcloud.io` and `*.sonarcloud.io` (the agent proxy attaches the token to those requests, so no +session or member sees it), or else `SONAR_TOKEN` in the environment, which every process of the session can +read. A finding of a rule no local analyzer reports is noted in this section, in the same change, instead of a squad lesson. -The .NET scanner is the `dotnet-sonarscanner` tool, pinned in the tool manifest `.config/dotnet-tools.json`, which -`dotnet tool restore` must have restored before the gate in a fresh cloud session: the SessionStart hook does it, run it -by hand where the hook did not; its `begin` -arguments mirror the CI step, plus the branch name and the repository root as base -directory, and it needs the full, non-incremental build between `begin` and `end`. The local coverage report is -Cobertura, which the scanner does not read, so the branch analysis carries no coverage figure: the *Coverage -gate* covers that. ## Image build check @@ -143,8 +131,7 @@ step of `.github/workflows/ci.yml` (today `.squad/**`, `.claude/**` and `src/DockerUpdateGuard.Data/Migrations/**`). A file no unit test can cover (`wwwroot/js/**`, a generated file) is added to both with a decision record — never to `sonar.exclusions`, which would drop it from the analysis too. `sonar.exclusions` holds only `docs/**` (concept mockups, screenshot scripts and the deployment examples -`docker-compose.*.yml` and `.env.example`, no product code), set in the same -step of `ci.yml` and in the *Sonar gate* command. +`docker-compose.*.yml` and `.env.example`, no product code), set in the same step of `ci.yml`. ## Writing code diff --git a/.squad/team.md b/.squad/team.md index 18a5a43d..52e12719 100644 --- a/.squad/team.md +++ b/.squad/team.md @@ -29,9 +29,8 @@ specific to this repository lives in two files the members read first: | Product Manager | — | answers escalations (the human user) | Model and effort are set per role in each agent file's front matter. The orchestrator may override a single -launch through its own `model` and `effort` parameters; the one override by default is that every `squad-tester` -launch of a tier-`security` run gets `model: opus` (a continued agent keeps its model), and `xhigh` is not -used. A change of model or effort for a role is made in the template, on the evidence of the runs' +launch through its own `model` and `effort` parameters; the overrides it makes by default are in +`orchestration.md` (*Launch model*), and `xhigh` is not used. A change of model or effort for a role is made in the template, on the evidence of the runs' `squad-log.py --summary` tables. Where production and test code live is defined in `stack.md` (*Layout*). The **Lead** decides everything inside the squad, including approving plans; the pull request is approved @@ -43,7 +42,9 @@ Product Manager*). `AGENTS.md`, `docs/ARCHITECTURE.md`, `docs/CONTRIBUTING.md`, `docs/UNIT_TESTS.md`, `.squad/stack.md` and `.squad/project.md` are binding: unit tests for all new code until the *Coverage gate* passes, and the code -conventions from `stack.md` while writing. +conventions from `stack.md` while writing. In a repository with more than one profile a code convention binds +only the code of its own language: a rule under **Profile ``** in `AGENTS.md`, or one `stack.md` states for +one language, is never carried over to another (the .NET profile's `== false` is no rule for Go code). **Format.** The member that changed files — Dev, Tester, Code Officer, or the orchestrator for its own edits — runs *Format* from `stack.md` before handing back and keeps its result to those files: every other file the diff --git a/.squad/template.json b/.squad/template.json index 4dc28506..b6cb8e78 100644 --- a/.squad/template.json +++ b/.squad/template.json @@ -1,6 +1,6 @@ { "repository": "LarsLaskowski/Squad-Spec-Repository-Template", - "commit": "cee6347c97940443a9638ec2a4fb3f47a6c400af", + "commit": "94a7016687caeb831a5541bf83af6acc5fe07758", "profile": "dotnet", "profiles": [ "dotnet" diff --git a/.squad/tools/decision-check.py b/.squad/tools/decision-check.py index a46a2653..f6f14f74 100644 --- a/.squad/tools/decision-check.py +++ b/.squad/tools/decision-check.py @@ -236,6 +236,10 @@ def assigned_area(number, record, assignments, released, errors): errors.append(f"{rel}: no {AREA_FIELD} (an area from {AREAS_DIR}/README.md, or {NO_AREA} for a " f"record about no area)") return None + if released is None and not assignments: + errors.append(f"{rel}: possibly released (release tags unknown: no local tags or no history); add an " + f"{AREA_FIELD} (an area from {AREAS_DIR}/README.md, or {NO_AREA}) if it is unreleased") + return None if not assignments: errors.append(f"{rel}: released without a {AREA_FIELD} and no '## Area assignment' table in a " f"later record names it") diff --git a/.squad/tools/plan-check.py b/.squad/tools/plan-check.py index 19b7feff..16553f34 100644 --- a/.squad/tools/plan-check.py +++ b/.squad/tools/plan-check.py @@ -81,6 +81,8 @@ SKELETON_ROW = re.compile(r"^skeleton\b", re.I) NO_NEW_API = re.compile(r"\bno new API\b", re.I) SPAN = re.compile(r"`([^`\n]+)`") +# The receiver of a Go method (`func (o *Options) Kernel() string`): its parentheses are no call. +GO_RECEIVER = re.compile(r"^\s*func\s*\([^)]*\)\s*") DECLARATION_KEYWORDS = {"func", "def", "fn", "public", "private", "protected", "internal", "static", "const", "readonly", "final", "async", "export", "function", "type", "class", "struct", "record", "interface", "enum", "var", "let", "val", "abstract", "override"} @@ -291,10 +293,11 @@ def signatures_section(plan): def signature_names(body): - """The identifier of every backticked declaration in the plan's *Signatures* section (`Options.Kernel` and - `func (o *Options) Kernel() string` give `Kernel`); a "none" section gives none.""" + """The identifier of every backticked declaration in the plan's *Signatures* section (`Options.Kernel` and the + Go method `func (o *Options) Kernel() string` give `Kernel`); a "none" section gives none.""" names = [] for span in SPAN.findall(strip_comments(body or "")): + span = GO_RECEIVER.sub("", span) call = re.search(r"([A-Za-z_]\w*)\s*\(", span) words = [w for w in re.findall(r"[A-Za-z_][\w.]*", span) if w not in DECLARATION_KEYWORDS] name = words[0].split(".")[-1] if words else "" diff --git a/.squad/tools/scope-check.py b/.squad/tools/scope-check.py index 4e6573e2..bcc7c202 100644 --- a/.squad/tools/scope-check.py +++ b/.squad/tools/scope-check.py @@ -5,8 +5,8 @@ Checks, over every file changed since the merge base with BASE_REF (working tree and untracked files included; BASE_REF from `.squad/tools/squad_settings.py`, default origin/main): -- no squad or instruction file is changed: `.squad/` except `stack.md` and `project.md`, `.claude/`, - `CLAUDE.md` (`.squad/routing.md`, *Scope of a product PR*; squad lessons become issues); +- no squad or instruction file is changed: `.squad/` except `stack.md`, `project.md` and, outside a fork, + `tools/squad_settings.py`, `.claude/`, `CLAUDE.md` (`.squad/routing.md`, *Scope of a product PR*; squad lessons become issues); - a marked file (`AGENTS.md` and its Copilot mirror `.github/copilot-instructions.md`, `docs/CONTRIBUTING.md`, `docs/ARCHITECTURE.md`, the bug report and pull request templates, the decision and area index) is changed only inside its `` blocks; `AGENTS.md` and its mirror are neither added nor deleted, and a project @@ -54,6 +54,9 @@ from analyzer_common import BASE_REF, control_characters, git, merge_base, settings # noqa: E402 (shared helpers next to this script) SQUAD_FILES_ALLOWED = {".squad/stack.md", ".squad/project.md"} +# The repository's own settings (coverage exclusions, thresholds): a product PR may change them, a fork's never +# (the file is executed Python). +SETTINGS_FILE = ".squad/tools/squad_settings.py" INSTRUCTION_FILES = {"CLAUDE.md"} RULES, MIRROR = "AGENTS.md", ".github/copilot-instructions.md" # The rules every agent reads: a product PR of the squad may change their project blocks (identically in both), a @@ -82,7 +85,7 @@ def changed_files(base): def is_squad_file(path, fork=False): - if path in SQUAD_FILES_ALLOWED: + if path in SQUAD_FILES_ALLOWED or (path == SETTINGS_FILE and not fork): return False return path.startswith((".squad/", ".claude/")) or path in INSTRUCTION_FILES or (fork and path in RULE_FILES) diff --git a/.squad/tools/sonar-check.py b/.squad/tools/sonar-check.py index df4623fe..26836b7d 100644 --- a/.squad/tools/sonar-check.py +++ b/.squad/tools/sonar-check.py @@ -1,35 +1,21 @@ #!/usr/bin/env python3 -"""Sonar gate: the CI code analysis (SonarQube Cloud) read as a check, so its findings are a gate result -instead of a surprise after the pull request, and a rule the local analyzers do not know is no squad lesson. - -Two modes: - -- `branch -- ` (step 7, before the pull request): needs a SonarQube credential and a - committed work branch (the scanner uploads the working tree; an uncommitted scratch file would be published - with a public project) — only the coverage reports of *Test with coverage* may lie around untracked. Runs - the scan command (*Sonar gate* in `.squad/stack.md`: the stack's scanner, analysing the work branch as a - branch of the project), waits for SonarQube to process the report, and lists every open issue and security - hotspot on a line the branch added or changed since the merge base with BASE_REF. Without a credential, or - when the scan fails (a network policy that blocks the scanner's download hosts, for example), the result is NOT - CHECKED, and the pull request mode runs on the draft pull request. -- `pr ` (step 11, on the draft pull request): reads the analysis of that pull request (with - `SONAR_TOKEN` when it is set, which a private project needs) and checks that it belongs to the current head - (`HEAD`, or `--head `); an analysis of an earlier head, or one without a quality gate yet, is PENDING, - never a pass. Then lists the quality gate status, the open issues and the security hotspots to review. - -The credential is a network secret of the cloud environment for the SonarQube hosts (the agent proxy attaches -it to every request to them, so the session never sees the token; `api/users/current` tells whether one is -attached), or else `SONAR_TOKEN` in the environment, which every process of the session can read. +"""Sonar gate: the CI code analysis (SonarQube Cloud) of a pull request read as a check, so its findings are a +gate result instead of a comment nobody reads, and a rule the local analyzers do not know is no squad lesson. + +`pr ` (step 11, on the draft pull request) reads the analysis of that pull request and checks that it +belongs to the current head (`HEAD`, or `--head `); an analysis of an earlier head, or one without a quality +gate yet, is PENDING, never a pass. Then it lists the quality gate status, the open issues and the security +hotspots to review. The work branch itself is not scanned before the pull request: SonarQube Cloud does not let +every organization read the data of non-main branches through its API, and such a scan only delayed step 7. + +A private project needs a credential: a network secret of the cloud environment for the SonarQube hosts (the +agent proxy attaches it to every request to them, so the session never sees the token), or else `SONAR_TOKEN` in +the environment, which every process of the session can read. The project comes from `SONAR_PROJECT_KEY` in `.squad/tools/squad_settings.py`, else `sonar.projectKey` in `sonar-project.properties`, else the `/k:` argument of the scanner in a workflow under `.github/workflows/`; the -server from `SONAR_HOST` (default https://sonarcloud.io); the scanner's report file from `SONAR_REPORT_TASK` -(default `.scannerwork/report-task.txt`). - -Two reads have a second endpoint for a credential that may browse the project but not use the first one (a -403 from the proxy-authenticated session): the pull request's commit comes from `project_analyses/search` when -`project_pull_requests/list` is refused, and the scan's processing status from `ce/activity` when `ce/task` is -refused. +server from `SONAR_HOST` (default https://sonarcloud.io). The pull request's commit comes from +`project_analyses/search` when a credential that may browse the project is refused `project_pull_requests/list`. A read that SonarQube answers with an HTTP error is NOT CHECKED with the request (path and parameters, never the token) and the response body, so a rejected parameter can be told from a missing project or the proxy. @@ -39,39 +25,31 @@ change), not filed as a squad lesson. Usage, from anywhere in the repository: - python3 .squad/tools/sonar-check.py branch -- python3 .squad/tools/sonar-check.py pr [--head ] -Exit code 0 clean, 1 findings or a failed quality gate, 2 NOT CHECKED (no credential, no project key, a dirty tree, -the scan or SonarQube failed), 3 PENDING (the analysis of this head is not there yet: wait and run again), -4 wrong usage (a mistyped command line never reads as a skipped gate). +Exit code 0 clean, 1 findings or a failed quality gate, 2 NOT CHECKED (no project key, SonarQube or git could not +be read), 3 PENDING (the analysis of this head is not there yet: wait and run again), 4 wrong usage (a mistyped +command line never reads as a skipped gate). """ import argparse import base64 -import fnmatch import glob import json import os import re -import shutil import subprocess import sys -import time import urllib.error import urllib.parse import urllib.request sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) -from analyzer_common import BASE_REF, changed_line_map, git, settings # noqa: E402 (shared helpers next to this script) +from analyzer_common import git, settings # noqa: E402 (shared helpers next to this script) CLEAN, FINDINGS, NOT_CHECKED, PENDING, USAGE = 0, 1, 2, 3, 4 HOST = getattr(settings, "SONAR_HOST", "https://sonarcloud.io").rstrip("/") -REPORT_TASK = getattr(settings, "SONAR_REPORT_TASK", ".scannerwork/report-task.txt") ERROR_BODY_CHARS = 500 -WAIT_SECONDS = 600 -POLL_SECONDS = 5 PAGE_SIZE = 500 -FILES_PER_QUERY = 50 KEY_ARGUMENT = re.compile(r'/k:"?([\w.:-]+)"?') @@ -130,15 +108,6 @@ def api(path, params, token=None): raise ApiError(f"HTTP {error.code} for GET {url}: {body or 'empty response body'}", error.code) from error -def proxy_credential(): - """True when SonarQube accepts requests that carry no token of their own: a network secret the agent proxy - attaches to the requests for the SonarQube hosts.""" - try: - return api("users/current", {}).get("isLoggedIn") is True # `authentication/validate` is true anonymously too - except (OSError, ValueError): # URLError is an OSError - return False - - def paged(path, params, field, token=None): """Every item of a paged list endpoint.""" items, page = [], 1 @@ -163,15 +132,6 @@ def finding_line(item, key, kind): return f"{rule} {path}:{line} {item.get('message', '')}".rstrip() -def on_changed_line(item, key, lines): - """True when the finding sits on a changed line, or is a file-level finding of a changed file.""" - path = file_of(item.get("component", ""), key) - if path not in lines: - return False - line = item.get("line") or item.get("textRange", {}).get("startLine") - return line is None or line in lines[path] - - def report(findings, gate=None): """Print the result and return the exit code.""" for finding in findings: @@ -221,138 +181,9 @@ def check_pr(key, number, head, token=None): + [finding_line(h, key, "hotspot") for h in hotspots], gate) -def ce_task_id(): - if os.path.isfile(REPORT_TASK): - with open(REPORT_TASK, encoding="utf-8") as handle: - for line in handle: - name, _, value = line.strip().partition("=") - if name == "ceTaskId": - return value - return None - - -def ce_task(task, token): - """The compute engine task; `ce/activity` answers for a credential that `ce/task` refuses.""" - try: - return api("ce/task", {"id": task}, token)["task"] - except ApiError as error: - if error.status != 403: - raise - key = project_key() - for candidate in api("ce/activity", {"component": key, "ps": 20}, token).get("tasks", []): - if candidate.get("id") == task: - return candidate - return {"id": task, "status": "PENDING"} - - -def wait_for_task(task, token): - """Wait until SonarQube has processed the uploaded report; return the task (its status says how it ended).""" - deadline = time.monotonic() + WAIT_SECONDS - while True: - result = ce_task(task, token) - if result["status"] not in ("PENDING", "IN_PROGRESS"): - return result - if time.monotonic() >= deadline: - return {**result, "status": "TIMEOUT"} - time.sleep(POLL_SECONDS) - - -def scan(command): - """Run the scan command; return the CE task id, or None when the scan failed or could not start.""" - output_dir = REPORT_TASK.split("/", 1)[0] - left_behind = not os.path.exists(output_dir) - try: - result = subprocess.run(command, check=False) # the command line of *Sonar gate* in .squad/stack.md - return ce_task_id() if result.returncode == 0 else None - except OSError as error: - print(f"The scan command could not start: {error}") - return None - finally: - if left_behind: # leave no untracked scanner output behind - shutil.rmtree(output_dir, ignore_errors=True) - - -def coverage_reports(): - """The globs of the coverage reports *Test with coverage* writes: untracked output the gate expects.""" - pairs = getattr(settings, "COVERAGE_REPORTS", None) or [ - (getattr(settings, "COVERAGE_FORMAT", ""), getattr(settings, "COVERAGE_REPORT_GLOB", ""))] - return [pattern for _, pattern in pairs if pattern] - - -def unexpected_files(): - """Changed tracked files, and untracked files other than the coverage reports, which the scanner would upload.""" - changed = git("diff", "--name-only", "HEAD").splitlines() - reports = coverage_reports() - untracked = [path for path in git("ls-files", "--others", "--exclude-standard").splitlines() - if not any(fnmatch.fnmatch(path, pattern) or path.startswith(pattern.split("/", 1)[0] + "/") - for pattern in reports)] - return [path for path in changed + untracked if path.strip()] - - -def branch_preconditions(): - """The current branch, when branch mode may scan it.""" - branch = git("branch", "--show-current").strip() - base = BASE_REF.split("/", 1)[1] if BASE_REF.startswith("origin/") else BASE_REF - if not branch or branch == base: - raise NotChecked("branch mode scans a work branch only (not a detached HEAD or the base branch, whose " - "analysis it would replace)") - unexpected = unexpected_files() - if unexpected: - raise NotChecked("uncommitted or untracked files the scanner would upload — commit the change first (no " - "push needed) and remove scratch files: " + ", ".join(unexpected[:10])) - return branch - - -def findings_on_branch(key, branch, token): - """Open issues and hotspots of the analysed branch on the lines the branch changed. The issues are read for - the whole project and filtered by path here: a `componentKeys` list mixing source files (`FIL`) and test - files (`UTS`) is refused with HTTP 400.""" - lines = changed_line_map(merge_base_or_raise()) - paths = sorted(lines) - issues = paged("issues/search", {"componentKeys": key, "branch": branch, "resolved": "false"}, "issues", token) - found = [finding_line(i, key, "issue") for i in issues if on_changed_line(i, key, lines)] - for start in range(0, len(paths), FILES_PER_QUERY): - chunk = paths[start:start + FILES_PER_QUERY] - hotspots = paged("hotspots/search", {"projectKey": key, "branch": branch, "files": ",".join(chunk), - "status": "TO_REVIEW"}, "hotspots", token) - found += [finding_line(h, key, "hotspot") for h in hotspots if on_changed_line(h, key, lines)] - return found - - -def merge_base_or_raise(): - result = subprocess.run(["git", "merge-base", BASE_REF, "HEAD"], capture_output=True, text=True, check=False) - if result.returncode != 0: - raise NotChecked(f"no merge base with {BASE_REF}: fetch it first (git fetch origin )") - return result.stdout.strip() - - -def check_branch(key, command, token): - if not command: - print("Usage: branch mode needs the scan command after `--` (*Sonar gate* in .squad/stack.md)") - return USAGE - branch = branch_preconditions() - task = scan(command) - if not task: - raise NotChecked("the scan failed or wrote no report (output above); check the draft pull request with " - "`sonar-check.py pr ` in step 11 instead") - result = wait_for_task(task, token) - if result["status"] != "SUCCESS": - raise NotChecked(f"SonarQube processed the report with status {result['status']}") - try: - findings = findings_on_branch(key, result.get("branch") or branch, token) - except ApiError as error: - if "non main branch" not in str(error): - raise - raise NotChecked("the organization cannot read data of non-main branches through the API; read the " - "analysis on the draft pull request with `sonar-check.py pr ` in step 11") from error - return report(findings) - - def parse(argv): parser = Usage(prog="sonar-check.py", description="SonarQube Cloud findings as a gate result.") modes = parser.add_subparsers(dest="mode", required=True, parser_class=Usage) - branch = modes.add_parser("branch", help="scan the work branch (needs a SonarQube credential)") - branch.add_argument("command", nargs=argparse.REMAINDER, help="-- followed by the *Sonar gate* scan command") pr = modes.add_parser("pr", help="check the analysis of a pull request") pr.add_argument("number", type=int) pr.add_argument("--head", help="commit SHA the analysis must belong to (default: HEAD)") @@ -365,19 +196,12 @@ def run(args): raise NotChecked("no SonarQube project key (SONAR_PROJECT_KEY in .squad/tools/squad_settings.py, " "sonar.projectKey in sonar-project.properties or /k: in a workflow)") token = os.environ.get("SONAR_TOKEN", "") - if args.mode == "pr": - head = args.head or git("rev-parse", "HEAD").strip() - ahead = subprocess.run(["git", "rev-list", "--count", "@{u}..HEAD"], capture_output=True, text=True, - check=False).stdout.strip() - if not args.head and ahead not in ("", "0"): - print("Note: HEAD has commits that are not pushed; the pull request cannot have been analysed at it.") - return check_pr(key, args.number, head, token or None) - if not token and not proxy_credential(): - raise NotChecked("no SonarQube credential: neither a network secret for the SonarQube hosts nor " - "SONAR_TOKEN in this environment; check the draft pull request with " - "`sonar-check.py pr ` in step 11 instead") - command = args.command[1:] if args.command[:1] == ["--"] else args.command - return check_branch(key, command, token or None) + head = args.head or git("rev-parse", "HEAD").strip() + ahead = subprocess.run(["git", "rev-list", "--count", "@{u}..HEAD"], capture_output=True, text=True, + check=False).stdout.strip() + if not args.head and ahead not in ("", "0"): + print("Note: HEAD has commits that are not pushed; the pull request cannot have been analysed at it.") + return check_pr(key, args.number, head, token or None) def main(argv=None): diff --git a/.squad/tools/squad-log.py b/.squad/tools/squad-log.py index dedffa3a..6857449d 100644 --- a/.squad/tools/squad-log.py +++ b/.squad/tools/squad-log.py @@ -13,7 +13,9 @@ - a control character (Unicode categories Cc, Cf, Zl, Zp other than tab and line break) is written as its escape (`U+202E`), never as the character itself; a tab becomes a space; - for a subagent launch, `--agent squad-` (model and effort read from `.claude/agents/squad-.md`, - so nothing is typed from memory) or `--launch MODEL/EFFORT`, with `--tokens N --tool-uses N --seconds N` from + so nothing is typed from memory) or `--launch MODEL/EFFORT` (a launch that overrode the agent file; a default + override of `.squad/orchestration.md`, *Launch model*, read against the *Tier* line of the folder's `plan.md`, is + refused with `--agent`), with `--tokens N --tool-uses N --seconds N` from the usage block of the launch's task notification, appends a trailer `(opus/high · 37,445 tokens · 6 tool uses · 57 s)` the summary can read back; - the first row replaces the template's heading `# Log: ` with the work folder's @@ -36,7 +38,9 @@ prints a warning, unless its step cell names a fix, delta or revise round: the first launch of another mode (`skeleton` to `implement`, `tests-first` to `coverage`, `approve-pr`, `decide`) is a fresh launch, not a continuation. `--continued` and `--continued-from` are only for continuing the same agent: a fresh launch of - the same role at another step is logged without them; + the same role at another step is logged without them. A continuation of an agent that was already continued + five times or reported 300,000 tokens is logged with a warning, and the summary names it under *Past the + continuation cap* (`.squad/orchestration.md`, *Continuing a member*: a fresh launch with a hand-over was due); - the usage block often arrives only after the member's hand-back: `--amend-last MEMBER` with the same options adds the trailer to the last row of that member afterwards, so the row can be written as soon as the report is in. A row gets its trailer once (a pending one is completed); a second amend is refused. When a @@ -142,7 +146,14 @@ SPECS_PREFIX = "specs/" FOLDER_NAME = re.compile(r"\w[\w.-]*") FIX_STEP = re.compile(r"^\s*11\b") +# Continuing a member (.squad/orchestration.md): after this many continuations, or once the agent's reported +# tokens reach this figure, the next round is a fresh launch with a hand-over. +CONTINUATION_CAP = 5 +CONTINUATION_TOKENS = 300_000 FRONT_MATTER_KEY = re.compile(r"^(model|effort):\s*(\S+)\s*$", re.M) +PLAN_TIER = re.compile(r"^Tier:\s*`?(\w+)", re.M) +# The launches the orchestrator overrides by default (.squad/orchestration.md, *Launch model*): agent -> (tier, model). +LAUNCH_OVERRIDES = {"squad-tester": ("security", "opus")} def escape_controls(text): @@ -185,37 +196,77 @@ def metrics(match): return (int(match.group("tokens").replace(",", "")), int(match.group("tools")), int(match.group("seconds"))) -def previous_tokens(path, member, skip_last=False, step=None, origin=None): - """The tokens the continued agent had used before the round being logged: its fresh launch plus the - continuations logged for it. Without `origin` that agent is the member's last fresh launch, with `origin` the - member's last fresh launch whose step cell is that text; a continuation row belongs to the fresh launch its - trailer names (`· continued from `), or else to the member's last fresh launch before it. None when the - member has no row with usage (with `origin`: no fresh launch of that step). With `skip_last` the member's last - trailer is left out (it is the one being replaced); with `step`, only the rows before the member's last row of - that step count (that row is the one amended or replaced).""" +def agent_chains(rows, member): + """The agents of `member` (an escaped cell) in log order, as {"step", "tokens", "continuations", "over"}: a fresh + launch starts one, a continuation row joins the fresh launch its trailer names (`· continued from `), or + else the member's last fresh launch before it. "tokens" is the agent's reported total, "over" the steps of its + continuations launched after the cap (`CONTINUATION_CAP`, `CONTINUATION_TOKENS`) was already reached; a + continuation without any fresh launch before it starts a chain with no step. Also the index of the member's + last fresh launch and {fresh step: index} as the third and second item.""" + chains, fresh, current = [], {}, None + for _, row_step, row_member, result in rows: + match = TRAILER.search(result) if row_member == member else None + if not match: + continue + tokens = metrics(match)[0] + if not match.group("continued"): + current = fresh[row_step] = len(chains) + chains.append({"step": row_step, "tokens": tokens, "continuations": 0, "over": []}) + continue + own = match.group("origin") + index = fresh.get(own) if own else current + if index is None: + index = current = len(chains) + chains.append({"step": None, "tokens": 0, "continuations": 0, "over": []}) + chain = chains[index] + if at_cap(chain): + chain["over"].append(row_step) + chain["tokens"] += tokens + chain["continuations"] += 1 + return chains, fresh, current + + +def at_cap(chain): + """True when the agent may not be continued again (`.squad/orchestration.md`, *Continuing a member*).""" + return chain["continuations"] >= CONTINUATION_CAP or chain["tokens"] >= CONTINUATION_TOKENS + + +def continued_chain(path, member, skip_last=False, step=None, origin=None): + """The agent a continuation of `member` continues, as `agent_chains` gives it: with `origin` the member's last + fresh launch whose step cell is that text, else its last fresh launch. None when the member has no row with + usage (with `origin`: no fresh launch of that step). With `skip_last` the member's last trailer is left out + (it is the one being replaced); with `step`, only the rows before the member's last row of that step count + (that row is the one amended or replaced).""" with open(path, encoding="utf-8-sig") as handle: rows = parse_rows(handle.read()) wanted = cell(member) if step is not None: rows = rows_before(rows, wanted, cell(step)) - matches = [(row_step, TRAILER.search(result)) for _, row_step, row_member, result in rows if row_member == wanted] - matches = [(row_step, match) for row_step, match in matches if match] - if skip_last and step is None: - matches = matches[:-1] - chains, fresh, current = {}, {}, None - for index, (row_step, match) in enumerate(matches): - tokens = metrics(match)[0] - if not match.group("continued"): - current, fresh[row_step] = index, index - chains[index] = tokens - continue - own = match.group("origin") - chain = fresh.get(own) if own else current - chains[chain] = chains.get(chain, 0) + tokens - if origin is not None: - target = fresh.get(cell(origin)) - return None if target is None else chains[target] - return chains.get(current) + elif skip_last: + last = [index for index, row in enumerate(rows) if row[2] == wanted and TRAILER.search(row[3])] + rows = rows[:last[-1]] + rows[last[-1] + 1:] if last else rows + chains, fresh, current = agent_chains(rows, wanted) + index = fresh.get(cell(origin)) if origin is not None else current + return None if index is None else chains[index] + + +def previous_tokens(path, member, skip_last=False, step=None, origin=None): + """The tokens the continued agent had used before the round being logged: its fresh launch plus the + continuations logged for it (`continued_chain`, which takes the same arguments); None when there is none.""" + chain = continued_chain(path, member, skip_last, step, origin) + return None if chain is None else chain["tokens"] + + +def cap_warning(path, member, skip_last=False, step=None, origin=None): + """A warning, or None, for a continuation of an agent that had already been continued `CONTINUATION_CAP` times + or reported `CONTINUATION_TOKENS` tokens (the arguments as for `continued_chain`).""" + chain = continued_chain(path, member, skip_last, step, origin) + if chain is None or not at_cap(chain): + return None + return (f"WARNING: {member}'s agent was already continued {chain['continuations']} time(s) at " + f"{chain['tokens']:,} tokens: from {CONTINUATION_CAP} continuations or {CONTINUATION_TOKENS:,} tokens " + "on, the next round is a fresh launch with a short hand-over (.squad/orchestration.md, *Continuing a " + "member*)") def rows_before(rows, member, step): @@ -458,6 +509,16 @@ def member_totals(rows): return per_member +def over_cap(rows): + """(member, steps) of every member with continuations launched past the cap (`agent_chains`).""" + found = [] + for member in dict.fromkeys(row[2] for row in rows): + steps = [step for chain in agent_chains(rows, member)[0] for step in chain["over"]] + if steps: + found.append((member, steps)) + return found + + def usage_notes(rows): """The notes under the table: launches whose usage never arrived, and launch rows still without usage.""" text = "" @@ -466,6 +527,12 @@ def usage_notes(rows): names = ", ".join(f"`{step} · {member}`" for step, member in unreported) text += (f"\n\n**Without reported usage:** {len(unreported)} launch(es) whose task notification never " f"arrived — counted in the launches, not in the totals: {names}") + over = over_cap(rows) + if over: + names = ", ".join(f"`{member} · {', '.join(steps)}`" for member, steps in over) + text += (f"\n\n**Past the continuation cap:** continuations launched after the agent had been continued " + f"{CONTINUATION_CAP} times or had reported {CONTINUATION_TOKENS:,} tokens, where a fresh launch with " + f"a hand-over was due (`.squad/orchestration.md`, *Continuing a member*): {names}") missing = missing_usage(rows) if missing: names = ", ".join(f"`{step} · {member}`" for _, step, member in missing) @@ -603,14 +670,40 @@ def unreported_launch(parser, model, args): return (model, None, None, None) -def launch_metrics(parser, root, args): +def plan_tier(log_path): + """The tier the *Tier* line of the work folder's `plan.md` names, or None (no plan: tiers `docs` and `patch`, + or a log read from standard input after step 10).""" + path = os.path.join(os.path.dirname(log_path), "plan.md") + if not os.path.isfile(path): + return None + with open(path, encoding="utf-8-sig") as handle: + found = PLAN_TIER.search(handle.read()) + return found.group(1) if found else None + + +def default_override(log_path, agent, launch): + """The `model/effort` the orchestrator launches `agent` with by default in this run instead of `launch` from + its agent file, or None when no override applies.""" + override = LAUNCH_OVERRIDES.get(os.path.basename(agent or "").removesuffix(".md")) + if override is None or launch is None or plan_tier(log_path) != override[0]: + return None + return f"{override[1]}/{launch.split('/', 1)[1]}" + + +def launch_metrics(parser, root, args, path): """(model/effort, tokens, tool uses, seconds) from the options, or None when no metric option is given; an - incomplete set is a usage error, an unknown agent file ends the run with exit code 1.""" + incomplete set is a usage error, an unknown agent file ends the run with exit code 1. A launch the orchestrator + overrides by default is refused with `--agent` alone, so the override is logged as launched.""" if not (args.agent or args.launch or has_numbers(args)): return None model = args.launch or (agent_launch(root, args.agent) if args.agent else None) if args.agent and not args.launch and model is None: sys.exit(f"No .claude/agents/{args.agent}.md with a model and an effort") + expected = None if args.launch else default_override(path, args.agent, model) + if expected: + parser.error(f"{args.agent} is launched with model {expected.split('/')[0]} in a tier-" + f"{plan_tier(path)} run (.squad/orchestration.md, *Launch model*): log the launch with " + f"--launch {expected} instead of --agent, or with the model it really ran on") if args.unreported: return unreported_launch(parser, model, args) if model and is_pending_row(args): @@ -650,15 +743,20 @@ def amend_command(parser, path, folder, member, launch, replace, continued, step own_step, current = target_row(path, member, step) current = current or "" # Only a replaced trailer with real numbers is part of the earlier usage a continuation subtracts. - launch = share_of_round(path, member, launch, continued, replace and bool(TRAILER.search(current)), step, origin) + replaces_usage = replace and bool(TRAILER.search(current)) + launch = share_of_round(path, member, launch, continued, replaces_usage, step, origin) if launch is None: return 1 - warning = mode_change_warning(path, member, own_step, origin, True, step) if continued and own_step else None + warnings = [] + if continued: + warnings = [cap_warning(path, member, replaces_usage, step, origin)] + if own_step: + warnings.insert(0, mode_change_warning(path, member, own_step, origin, True, step)) row = amend_last(path, member, launch, replace=replace, step=step) if row is None: print(no_row_message(folder, member, replace, step)) return 1 - if warning: + for warning in filter(None, warnings): print(warning) print(row) return 0 @@ -686,8 +784,9 @@ def row_command(parser, path, args, launch): launch = share_of_round(path, args.member, launch, args.continued, origin=args.continued_from) if args.continued and launch is None: return 1 - warning = mode_change_warning(path, args.member, cell(args.step), args.continued_from) if args.continued else None - if warning: + warnings = [mode_change_warning(path, args.member, cell(args.step), args.continued_from), + cap_warning(path, args.member, origin=args.continued_from)] if args.continued else [] + for warning in filter(None, warnings): print(warning) print(append_row(path, args.step, args.member, args.result, launch=launch)) return 0 @@ -719,7 +818,7 @@ def run(parser, args, root, path): if target is not None and is_not_launch(target) and uses_metrics: parser.error(f"'{target}' is no launch: its rows are written without any metric option and never " "amended, and the summary asks no usage for them") - launch = launch_metrics(parser, root, args) + launch = launch_metrics(parser, root, args, path) args.continued = args.continued or args.continued_from is not None if args.amend_last and args.replace_last: parser.error("--amend-last and --replace-last exclude each other") diff --git a/.squad/tools/squad_settings.py b/.squad/tools/squad_settings.py index ed127507..e29ecc3d 100644 --- a/.squad/tools/squad_settings.py +++ b/.squad/tools/squad_settings.py @@ -10,9 +10,6 @@ # Solution or project file the analyzer gate builds. SOLUTION = "DockerUpdateGuard.slnx" -# Sonar gate (.squad/tools/sonar-check.py): the report file the .NET scanner writes. -SONAR_REPORT_TASK = ".sonarqube/out/.sonar/report-task.txt" - # Coverage gate (.squad/tools/coverage-check.py): line coverage on new/changed production lines, and overall. # COVERAGE_OVERALL_THRESHOLD may start below COVERAGE_THRESHOLD in a repository adopted with a coverage debt; # it is never lowered and is raised towards COVERAGE_THRESHOLD as coverage improves. diff --git a/AGENTS.md b/AGENTS.md index 56450155..90d720ea 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -194,7 +194,7 @@ and the round semantics. The squad's members (`.squad/team.md`) run as subagents commits, pushes and posts. The pipeline, its limits and escalations are in [`.squad/routing.md`](/.squad/routing.md), the project's guarantees, security areas and integration surface in [`.squad/project.md`](/.squad/project.md). Working records stay under `specs/` on the work branch. A product PR never changes the squad or these -instructions (`.squad/stack.md`, `.squad/project.md` and the project blocks of this file excepted); squad lessons become `squad` issues where they can be fixed (`.squad/orchestration.md`, *Squad +instructions (`.squad/stack.md`, `.squad/project.md`, `.squad/tools/squad_settings.py` and the project blocks of this file excepted); squad lessons become `squad` issues where they can be fixed (`.squad/orchestration.md`, *Squad lessons*). The user acts as Product Manager and is asked only when the Lead escalates. The reasoning behind code decisions — why something was built the way it was — is recorded by the Lead diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index 02b4c5f2..b8292efb 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -70,7 +70,7 @@ squash-merging in [`decisions/`](decisions/README.md)). For PR naming use the following convention: `[area] Description` (no period at the end). -- For the area, use one of the areas listed below, capitalized. +- For the area, use one of the areas listed below, capitalized. Issue titles use the same `[area]` prefix. - For the description, do not reference an issue number in there. A clear, short summary of what the change entails is enough; there is room to elaborate in the description. @@ -100,8 +100,8 @@ is agreed as acceptance criteria before it is built, and the maintainers own tho - **Out of scope for an outside pull request:** the AI squad and agent instructions (`.squad/` except `stack.md` and `project.md`, `.claude/`, `AGENTS.md`, `CLAUDE.md`, `.github/copilot-instructions.md`), the text outside the `` blocks of the template's marked files, and the workflows under - `.github/workflows/`. They come from a shared template and are changed there; the *Fork scope* check fails - on the first two. Open an issue instead. + `.github/workflows/`. The first two come from a shared template and are changed there, the workflows are the + repository's own; the *Fork scope* check fails on the first two. Open an issue instead. - **Checks on a fork:** pull requests from forks run without repository secrets, so the analysis that needs one (e.g. SonarQube Cloud) does not run there; a maintainer runs it before merging. Run the quality gates below locally so the pull request arrives clean. The CI run of a first-time contributor's fork waits for a maintainer's diff --git a/docs/decisions/0017-product-prs-may-change-the-squad-settings-file.md b/docs/decisions/0017-product-prs-may-change-the-squad-settings-file.md new file mode 100644 index 00000000..c3cd42ca --- /dev/null +++ b/docs/decisions/0017-product-prs-may-change-the-squad-settings-file.md @@ -0,0 +1,33 @@ +# 0017: Product PRs may change the squad settings file + +- **Status:** Accepted +- **Date:** 2026-11-01 +- **Area:** — +- **Source:** Squad refreshed from Squad-Spec-Repository-Template +- **Supersedes:** [0003](0003-squad-working-records-off-main.md) (amends its list of squad files a product PR never changes: `.squad/tools/squad_settings.py` is no longer one of them; 0003 is not edited) + +## Context + +Record 0003 lets an issue or feature PR change no squad file except `.squad/stack.md`, `.squad/project.md` and the +project blocks of `AGENTS.md` and its Copilot mirror. `.squad/tools/squad_settings.py` holds this repository's own +settings for the squad tools: the coverage thresholds, the paths the gates read and `COVERAGE_PATHSPECS`. When a +product change adds a file no unit test can cover, *Code a unit test cannot cover* in `.squad/routing.md` has the +same pull request exclude it from the coverage measure in that file, which 0003 forbade. + +## Options considered + +1. **Keep the file a squad file**: the exclusion goes to a separate squad-maintenance pull request, and the product + pull request fails its *Coverage gate* until that one is merged. +2. **Treat the file like `stack.md` and `project.md`**: project knowledge a product pull request may change, while + the scripts that import it stay template-managed. `scope-check.py` allows it (a fork pull request still may not). + +## Decision + +Option 2: an issue or feature PR may change `.squad/tools/squad_settings.py` as well; every other rule of 0003 stands. + +## Consequences + +- A change that adds code no unit test can cover sets its coverage exclusion in the same pull request, together + with the decision the Lead records for it. +- A change to the settings file is reviewed like the rest of the diff; the scripts under `.squad/tools/` stay + template-managed and are never edited in a product pull request. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 2bc98027..06203bc8 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -69,4 +69,5 @@ once in the [area document](../areas/README.md) of the record's area, and the re | [0014](0014-registry-tag-names-and-digests-are-validated-where-they-are-read.md) | Remote tag names, digests, image references and names are validated or bounded where they are read | Accepted | 2026-10-28 | | [0015](0015-the-trivy-scan-target-follows-an-end-of-options-marker.md) | The Trivy scan target follows an end-of-options marker | Accepted | 2026-10-29 | | [0016](0016-the-bundled-trivy-skips-its-update-check-and-telemetry.md) | The bundled Trivy skips its update check and telemetry | Accepted | 2026-10-29 | +| [0017](0017-product-prs-may-change-the-squad-settings-file.md) | Product PRs may change the squad settings file | Accepted | 2026-11-01 |