Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions docs/ticket/0yb2pgw-rustqual-baselines-go-stale-when-main-moves.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,3 +114,16 @@ set to `github.event.pull_request.head.sha` for `qual` only, leaving `test` and
the other tasks on the merge ref where testing the merge result is the point.
Needs one thing verified first — whether `actions/checkout` treats an empty
`ref:` as unset, which is what the other matrix entries would pass.

-----

- **From**: jp
- **Date**: 2026-10-06T21:46:15Z

Decided: option 3, with no committed baseline.
Findings are compared one by one by `(rule, file, symbol)` rather than
per-category counts, and the comparison lives in a new `qual` crate (T-0yh5frr).
CI compares the merge ref against `HEAD^1` in the same job (T-0yh5npc), so the
measured tree and the base always match and the interim `checkout_ref`
workaround isn't needed.
This ticket closes when T-0yh5npc lands.
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Add a qual crate that diffs rustqual findings by identity

- **Status**: Todo
- **Kind**: Feature
- **Authors**: jp
- **Date**: 2026-10-06
- **Label**: domain=tooling
- **Label**: type=task

A new internal crate, `crates/internal/qual`, that turns rustqual reports into
individual findings, compares two reports, and renders the result.
CI, `cargo_check` and the burn-down loop harness all call it, so the gate and
its messages behave the same in every place it runs.

## Why

The gate today compares per-category counts in `.config/rustqual/baseline.json`
and `baseline-dry.json`.
Those files record a location only for IOSP violations (`violation_details`).
Every other finding (about 2,000 of the ~3,200 distinct ones) is stored as a
count.
That is why CI can say `srp_module_warnings: 207 -> 208` but not which file.
A committed baseline also goes stale as soon as `main` moves (see T-0yb2pgw).

## Scope

- **Parse** both passes (`config.toml` and `dry.toml`) into findings keyed by
`(rule, file, symbol)`.
Line numbers are not part of the key, so editing code above a function causes
no churn.
Duplicate keys count as a multiset.
IOSP appears in both passes, so de-duplicate it.
This ticket picks the input format (`json`, `sarif` or `ai-json`): whichever
gives a stable rule ID and symbol most cleanly.
- **Diff** a base report against a head report and return the new findings.
Before comparing, map renamed files using `git diff -M` between the two
revisions.
- **Base analysis:** run rustqual on `git worktree add --detach <base>`.
Cache the report, keyed by base sha, rustqual version and config hash.
A run takes a few seconds, so the cache is a convenience, not a requirement.
- **Policy**, read from `.config/rustqual/` (exact file decided here):
- `exempt`: rules that never gate.
IOSP goes here, because rustqual has no way to turn it off.
- `enforced`: rules that have reached zero on `main`.
Any finding of an enforced rule fails, whether or not it appears in the
diff.
Rules are only ever added to this list; it never shrinks.
- **Render** the same findings as plain text, GitHub annotations (`::error
file=…,line=…::`), a Markdown table for `$GITHUB_STEP_SUMMARY`, and a
compact AI-oriented format.
Each finding carries rustqual's message plus the guidance from
`.config/rustqual/guidance/<RULE>.md` when that file exists.
- **CLI:**
- `qual diff --base <rev> [--format …]` exits non-zero on new or enforced
findings.
- `qual report` lists every current finding (the burn-down picker needs this).
- **Library API:** jp-tools depends on the crate the same way it depends on
`ticket`.

## Fail closed

A report that can't be parsed, or a rustqual run that fails, is an error, never
a pass.
This replaces the canary in `qual-ci`: unit tests prove that a new finding fails
the gate and that a malformed report is an error.

## Tests

Keep the core pure: parsing, rename mapping, diff, policy and rendering are
tested against fixed fixture reports.
Only the thin shell around them spawns rustqual and git.

## Out of scope

Wiring into CI, `cargo_check` or the harness.
Those are separate tickets that depend on this one.
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Extract a reusable loop harness from bug-hunt

- **Status**: Todo
- **Kind**: Chore
- **Authors**: jp
- **Date**: 2026-10-06
- **Label**: domain=tooling
- **Label**: type=task

Split the unattended bug-hunt script into a generic loop harness plus a
`bug-hunt` mission, with no change in behaviour.
The rustqual burn-down becomes a second mission on the same harness.

The script currently lives untracked at `bug-hunt/run.sh` in the `rustqual-next`
worktree.
Its header still refers to `.bug-hunt/`.

## Harness (shared)

Everything that already works and isn't specific to bugs:

- the loop, the wall-clock deadline (`--hours` / `--until`) and the iteration
limit;
- conversation rotation by commits, events or consecutive failures;
- rate-limit detection, capacity probes and waiting;
- network-stall detection and backoff;
- per-turn timeouts and interrupt handling;
- the commit probe and the refusal to run on `main` or with a staged index;
- facts measured from git after each turn (commits made, dirty files, gate
result);
- `RUNLOG.md`, with per-run log and snapshot directories.

## Mission (per directory)

- **Settings:** model, rotation limits, `--cfg` persona and skill flags,
compaction spec.
- **Prompt templates:** mission, continue, handover, repair.
- **`next-target` executable**, optional.
When a mission has one, the harness runs it before each turn and inserts its
output into the prompt; empty output means no targets remain and ends the run.
When a mission has none, the model picks its own target, as bug-hunt does
today.
- **`gate` executable:** runs after each turn and reports pass or fail with
output.
For bug-hunt this is today's `cargo check --workspace --all-targets`.
- **Discard on gate failure**, opt-in per mission: the harness resets its own
branch to the iteration's starting commit. bug-hunt keeps this off and keeps
today's "next turn repairs" behaviour.

## Constraints

- The first commit adds `run.sh` unchanged, so the extraction reads as a diff
against the original.
- Stays in bash.
Rewriting it in Rust at the same stage invites the second-system effect.
- Commit only the harness and mission definitions.
Ledgers, run logs, logs and snapshots stay gitignored.
- No behaviour change for bug-hunt.
Verify with a `--iterations 1` smoke run before and after, and compare the run
logs and prompts.
- Pick a location in the tree (for example under `.config/`) and record why.
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# File upstream rustqual issues found during adoption

- **Status**: Todo
- **Kind**: Chore
- **Authors**: jp
- **Date**: 2026-10-06
- **Label**: domain=tooling
- **Label**: type=task

rustqual problems found while adopting it, to report at
https://github.com/SaschaOnTour/rustqual/issues (pinned at v1.8.2 in the
`justfile`).
Reproduce each one on the current version before filing, and drop any that a
newer release has fixed.

1. **`SRP-002` stops counting at the first `#[cfg(test)]` anywhere in a file.**
`count_production_lines` in `src/adapters/analyzers/srp/module.rs` breaks on
the first line starting with `#[cfg(test)]`, so a test-only helper partway
down a file hides all the production code below it.
2. **`--compare --fail-on-regression` misses new findings.** It treats a
regression as `quality_score` dropping.
Adding compliant code alongside a new finding dilutes the score enough to
hold it level.
A baseline that can't be parsed reports "not regressed" and exits 0 under
`--no-fail`.
Details are in the `qual-ci` recipe comment in the `justfile`.
3. **Dimensions that can't be disabled.** IOSP has no `enabled` key, and
`MAGIC_NUMBER` still fires under `[complexity] enabled = false`.
Disabling a dimension then reports its matching `qual:allow` markers as
orphaned (`ORPHAN_SUPPRESSION`), even though the findings keep firing.
See `.config/rustqual/dry.toml`.
4. **`DRY` is all-or-nothing.** Turning off `[duplicates]` also turns off
dead-code (`DRY-002`) and dead-type detection.
A per-rule switch would remove the need for a second pass.
5. **`ai-json` lacks the stable rule ID and the threshold/actual values** (for
example `length=72 > 60`).
Agents need both to fix a finding without a second lookup.
6. **Possible IOSP false positive.** `exponential_backoff`
(`crates/jp_llm/src/retry.rs`) appears as a violation although it seems to
call only std methods and do arithmetic.
Confirm with `--verbose` that a method name matched a project function before
filing.
62 changes: 62 additions & 0 deletions docs/ticket/0yh5m6r-record-the-rustqual-rule-policy-in-config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Record the rustqual rule policy in config

- **Status**: Todo
- **Kind**: Chore
- **Authors**: jp
- **Date**: 2026-10-06
- **Label**: domain=tooling
- **Label**: type=task

Decide, for each rustqual rule, whether JP enforces it, tunes it, or exempts it.
Record each decision and its reason as a comment in `.config/rustqual/`, in the
style the existing configs already use.
The decisions belong to the maintainers.

**For an assistant picking this up:** apply the items under *Decided* as
written.
For each item under *To decide*, don't choose.
Draft the entry instead: the current finding count, three or four real examples
from the workspace, and the options with what each one costs.
Then stop and ask the user to decide before writing it into the config.
The PR is where the decisions get reviewed.

Depends on T-0yh5frr for the `exempt` policy list.

This has to land before the burn-down mission runs, so the loop doesn't refactor
code to satisfy rules JP never adopted.

## Decided

- **IOSP: exempt (option A).** rustqual can't disable it, so it goes in the
`qual` crate's `exempt` list rather than behind suppressions (862 violations,
6.2% of functions, which is over the 5% `max_suppression_ratio`).
Revisit if, after the tier 2 burn-down (complexity, length, nesting), reviews
still turn up functions within those limits that mix orchestration and logic
badly.
The two alternatives:
- **B:** gate IOSP on new code only;
- **C:** burn it down fully, with `allow_recursion = true`.
- **`[srp] file_length = 800`.** rustqual counts only code lines: blank lines,
`//`, `///` and block comments don't count, and counting stops at the first
`#[cfg(test)]`.
The default of 300 is rustqual's own house style.
Note that `srp_module_warnings` also covers the independent-cluster cohesion
check, so it won't drop to zero from the length change alone.

## To decide

- **`[tests]` thresholds.** `*_tests.rs` files have no `#[cfg(test)]`, so every
code line counts and they inherit 800 and the 60-line function limit.
JP's tests are deliberately self-contained, which makes them long.
- **`allow_expect`.** `EventId::random` uses `.expect()` with a documented `#
Panics` section, a deliberate pattern.
Decide whether `.expect()` is allowed while `.unwrap()` keeps firing (194
error-handling findings in total).
- **`unsafe`** (74 findings).
Suppress audited FFI and syscall sites with `qual:allow(unsafe)`, which
doesn't count toward the suppression ratio, or exempt the rule.
- **`[boilerplate] accepted_display_idioms`.** Declare JP's house style for
trivial `Display` impls, so `BP-002` enforces that style instead of firing on
it.
- **Any rule** whose findings the maintainers consider noise after looking at
samples.
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Gate CI on new rustqual findings against the base commit

- **Status**: Todo
- **Kind**: Feature
- **Authors**: jp
- **Date**: 2026-10-06
- **Label**: domain=tooling
- **Label**: type=task

Replace the committed-baseline gate in `just qual-ci` with `qual diff` from
T-0yh5frr.
CI analyses the tree it builds and its base in the same job, and reports only
the findings that are new.

Closes T-0yb2pgw: a baseline produced in the same run can't go stale, and it
always shares a base with the measured tree.

## Base revision

- `pull_request`: the checkout is `refs/pull/N/merge`, so the base is `HEAD^1`,
the same parent the `changes` job already diffs against.
- `push` to `main`: the base is `github.event.before`.
On a new branch, where `before` is all zeros, there's nothing to compare:
analyse only, and fail only on enforced rules.
- The `qual` task's checkout needs enough history to reach the base.
`fetch-depth: 2` covers pull requests; on push, fetch the `before` sha
explicitly.

## Output

- `::error` annotations for new findings only, so GitHub's display cap no longer
hides the regression behind existing findings.
- The full new-findings table, with guidance, in `$GITHUB_STEP_SUMMARY`.
- On success, one line: the head and base counts per pass.

## Remove

- `.config/rustqual/baseline.json`, `baseline-dry.json` and `regressions.jq`.
- The `qual-baseline` recipe.
- The canary in `qual-ci`.
The fail-closed behaviour is covered by unit tests in the `qual` crate.
- Check the `justfile`, the configs and `docs/` for any other references to the
baselines.
Keep `just qual` as the exploratory entry point.
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Report new rustqual findings from cargo_check

- **Status**: Todo
- **Kind**: Feature
- **Authors**: jp
- **Date**: 2026-10-06
- **Label**: domain=tooling
- **Label**: type=task

Make the `cargo_check` tool (`.config/jp/tools/src/cargo/check.rs`) report the
rustqual findings that CI would fail on, so an assistant can find out before
pushing.
It uses the `qual` crate from T-0yh5frr as a library, so it gives the same
verdict and the same messages as CI.

## Behaviour

- Add a `QualCheck` step next to `ComfortCheck`, with the same `Clean` /
`Findings(note)` / `Failed(stderr)` shape.
- Run it only after clippy succeeds: findings on code that doesn't compile are
noise.
- The head is the working tree, including uncommitted changes.
The base is `merge-base(HEAD, origin/main)`, falling back to `main` when
there's no remote, and its report is cached by `qual`.
- The `package` parameter filters which findings are *reported*, not what is
analysed.
Dead-code and duplicate detection need the whole workspace.
- Render in the AI-oriented format with per-rule guidance, and list the allowed
escape hatches (see the escape-hatch ticket).
- Update the tool summary in `.jp/mcp/tools/cargo/check.toml`, and the
`rust-development` skill description if it lists what `cargo_check` covers.

## Tests

Follow `check_tests.rs`: cover clean, findings and failure with
`MockProcessRunner`, plus the case where clippy fails and the qual step is
skipped.
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Surface rustqual suppressions and config edits in the gate

- **Status**: Todo
- **Kind**: Feature
- **Authors**: jp
- **Date**: 2026-10-06
- **Label**: domain=tooling
- **Label**: type=task

With no committed baseline, the gate can only be silenced in two ways: a
`qual:allow` marker in the code, or an edit under `.config/rustqual/`.
Make both visible wherever the gate reports.
Builds on T-0yh5frr.

## Gate behaviour (`qual diff`)

- List `qual:allow`, `qual:api` and `qual:test_helper` markers added since the
base, in their own section, with file, line and reason.
- List changed files under `.config/rustqual/` in their own section.
- Fail on any added `qual:allow` that has no `reason:`.
- Neither section fails the gate otherwise.
They exist so a reviewer sees them, and so CI and `cargo_check` show them to
the author.

## Assistant instructions

Update the `dev` and `rfd-implementor` personas, and the `coding` skill if it
covers verification:

- Fix the finding.
Don't add a suppression, change a threshold, or edit `.config/rustqual/`
unless the user asked for it in this conversation.
- If a finding looks wrong, say so and stop; don't route around it.
Loading
Loading