[bpamiri]

Summary

tools/rustcfml/baseline.json's first _notes entry records where the baseline came from: the run, commit, engine version and totals. run-suite.sh --write-baseline copied _notes over unchanged, so every regenerated baseline kept the previous run's note. This includes every auto-bump PR. #4577 still cited the v0.693.0 run until it was fixed by hand.

In write mode, the script now:

• drops any existing provenance note: the generated form, or a hand-written Linux baseline from ... one, including the wording used on chore(rustcfml): bump engine pin to v0.732.0 #4577;
• puts a fresh note first, with:
  • the Actions run URL (from GITHUB_SERVER_URL/GITHUB_REPOSITORY/GITHUB_RUN_ID), or a local run outside CI;
  • the short commit (GITHUB_SHA, or git rev-parse HEAD locally);
  • the engine version and platform;
  • pass / fail / error / skipped totals;
• keeps the other, human-maintained notes (upstream issue links) in their order.

Example, from a dry run against develop's baseline.json:

Baseline written by run-suite.sh --write-baseline from a local run (9fcc17c, v0.732.0, macOS): 7273 pass / 0 fail / 0 error / 177 skipped. totalSpecs is the sum of those reported categories.

Tests

tools/rustcfml/test_run_suite.py gains WriteBaselineTest, which has 7 cases. They were written first and seen failing:

• a CI note names the run, commit, version and totals;
• a legacy note is replaced, not kept;
• a note hand-written on a bump PR is replaced;
• rewriting twice leaves a single provenance note;
• human notes keep their order;
• a local run says so and links no run;
• a baseline without _notes gets the note.

All four tools/rustcfml/test_*.py files pass, and rustcfml-ci.yml runs them. A dry run against develop's real baseline.json replaced the stale v0.693.0 note and kept the 3 human notes in order.

Refs #4577

🤖 Generated with Claude Code

[bpamiri]

rev1-r3 review @ f4a3c10: APPROVE

The old-note detection can't drop today's human notes. It removes only string notes that start with "Baseline written by run-suite.sh" (this PR's own form) or "Linux baseline from" (the earlier hand-written forms). It keeps every other note, and non-string entries, in order.

• The current baseline.json has 4 notes. _notes[0] is purely provenance: run URL, 1bd2eef95, v0.693.0, "5850 pass / 0 fail / 0 error / 29 skipped". It's already stale, since chore(rustcfml): bump engine pin to v0.730.0 #4545 moved the pin to v0.730.0.
• The other three (the RustCFML#431 fix, the migrationSpec/StorageSpec history, the SystemUtilSpec JVM note) don't start with either prefix, so they survive unchanged.
• The match is a prefix, so a human note that only mentions "Linux baseline" mid-sentence is kept too.
• An unreadable or missing baseline yields just the provenance note.

The totals come from the same result data. The note interpolates the totals dict built at L174 from the parsed run (data). That dict is also the one written to payload["totals"] and printed in the summary, so the note can't disagree with the baseline it sits in. The source is $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID plus GITHUB_SHA in CI, or "a local run" plus git rev-parse HEAD, cut to 9 characters.

The 7 new tests are meaningful.

• _parser_source() runs the real heredoc from run-suite.sh, with GITHUB_* stripped from the environment so CI and local runs are tested separately.
• They cover:
  • CI note contents;
  • replacing the legacy note;
  • the chore(rustcfml): bump engine pin to v0.732.0 #4577 hand-written wording;
  • idempotence across two rewrites;
  • human-note order;
  • the local run;
  • a baseline without notes.
• Mutation check: with the prefix filter disabled (if True), 4 of them fail (…replaced_not_kept, …hand_written…, …rewriting_twice…, …keep_their_order). They really pin the behaviour.
• Locally on this head, test_run_suite.py is 15/15 OK, and test_check_version.py and test_bump_pin.py are OK.

PR CI: 12 pass, 3 skipping, 0 fail. Up to date with develop, MERGEABLE.