Tooling: 1.13x faster exact stage-pair audit with identical receipts

research/stage-pair-performance/README.md

@@ -0,0 +1,101 @@
+# Stage-pair audit performance
+
+Exact-only speedup of the historical two-invocation frame search
+(`scripts/audit_stage_pair.py`, h=6, 64 candidates). No new kappa,
+no domain change, no weakened check.
+
+## What changed
+
+Two minimal edits, same arithmetic and same control flow:
+
+1. `scripts/audit_fused_block.py` — `rational_rank` skips the redundant
+   `Q(x)` construction when an entry is already a `Fraction`
+   (`x if isinstance(x, Q) else Q(x)`). `Q(Q)` returns the same value
+   after needless normalization (26M `math.gcd` calls in the profiled
+   run), so the elimination steps and returned ranks are unchanged.
+
+2. `scripts/audit_stage_pair.py` — `modular_rank` converts each
+   denominator with a cached `_mod_inv(d, prime)` helper
+   (`@lru_cache`, same `pow(d, -1, prime)` call) instead of one fresh
+   `pow()` per matrix entry (~480k calls over the audit). Cache hits
+   return the identical inverse; `lru_cache` does not cache exceptions,
+   so a non-invertible denominator raises the same `ValueError` as before.
+
+## Equivalence argument
+
+- `Q(x) == x` for `x` a `Fraction`; the guard only skips rebuilding an
+  equal object. Integer entries still go through `Q(x)` exactly as before.
+- `_mod_inv(d, prime)` calls the same `pow(d, -1, prime)` with the same
+  arguments and returns its result unmodified. Misses compute; hits replay.
+- No matrix, pivot order, stopping rule, candidate list, incident map, or
+  rejection predicate was touched. The outer `rank` cache, the mod-101
+  lower-bound-then-exact structure of `candidate_score`, and all
+  `require()` failure points are unchanged.
+
+## Coverage preserved
+
+- Full `certificate()` output is identical to the committed
+  `certificates/stage-pair-audit.json` in every field except the single
+  expected `source_sha256` entry for `scripts/audit_stage_pair.py`
+  (the file we edited). Rank sum 8684, 64 candidates, 6 local
+  improvements, directions, overlaps, and scaling rows all match.
+- `audit_fused_block` and `audit_joint_frames` certificates regenerate
+  byte-identical (their source pins do not cover the edited lines, and
+  every rational rank agrees).
+- `tests/test_stage_pair_performance.py` checks the optimized helpers
+  against verbatim base-commit references on edge cases (empty, zero,
+  identity, singular, non-square, ints vs Fractions, 30-digit numerators,
+  both primes 101/103), including the `ValueError` path for a denominator
+  multiple of the prime, and the existing lower-bound property
+  (`modular_rank == 1`, `rank == 2` on `((101,0),(0,1/2))`).
+- Existing suites pass: `test_stage_pair` (8 tests), `test_fused_block`
+  (7 tests), `test_joint_frames` (8 tests), `test_quadratic_affine`
+  (4 tests) — all rank consumers of the edited helper.
+
+## Benchmark
+
+Fresh-process timings of `certificate()` (compute only, no file write),
+same machine and interpreter, OS caches not flushed, host not isolated.
+Timing is descriptive, not a pass threshold.
+
+| run | original (s) | optimized (s) |
+|-----|---:|---:|
+| 1 | 16.63 | 14.31 |
+| 2 | 15.79 | 13.96 |
+| 3 | 15.57 | 13.64 |
+| median | 15.79 | 13.96 |
+
+Median 15.79 s to 13.96 s: 1.13x faster, 11.6% less wall time.
+Microbenchmarks on the same inputs: `rational_rank` 3.05x on 50 sampled
+frames; modular denominator conversion 1.42x on 20 sampled matrices.
+
+Reproduce (stdlib only, Python 3.11+):
+
+```sh
+python3 -B research/stage-pair-performance/benchmark.py --rounds 3
+```
+
+Expected: `PASS_IDENTICAL_STAGE_PAIR_RECEIPTS`.
+
+```sh
+python3 -B -m unittest tests.test_stage_pair_performance -v
+```
+
+Expected: 6 tests `OK`.
+
+## Scope and limits
+
+- This covers the fixed historical check only (h=6, prime 101, 4082
+  baseline edges, 64 candidates, 36x36 matrices). No frontier replay,
+  supplier, or multiplication runtime is measured or claimed.
+- The 86 s `audit_quadratic_affine` replay was not rerun end to end;
+  its 4 unit tests pass and its rank calls go through the same
+  provably-identical helper, verified by the equivalence suite.
+- Full `make verify` and the selected-supplier cold replay were not rerun
+  for this isolated historical-checker change, matching prior tooling PRs.
+
+## Attribution
+
+Contributor: Aaron Wang (navinor). Prepared with substantial AI assistance
+(Muse Spark via OpenCode). Predecessor authorship and notices intact.
+`upstream/` untouched. Selected frontier and records unchanged.

research/stage-pair-performance/RESULT.json

@@ -0,0 +1 @@
+{"ai_assistance":"Substantial assistance from Muse Spark via OpenAI Codex-style agent harness for implementation, benchmarking, documentation, and delivery. No independent human peer review claimed.","base_commit":"7ee02b7c96d85f22b38d28bd7ed49bb8c1aeb261","benchmark_command":"python3 -B research/stage-pair-performance/benchmark.py --rounds 3","benchmark_expected":"PASS_IDENTICAL_STAGE_PAIR_RECEIPTS","certificate_check":"certificate() output identical to certificates/stage-pair-audit.json in every field except the single expected source_sha256 entry for scripts/audit_stage_pair.py; rank_sum 8684, 64 candidates, 6 local improvements. audit_fused_block and audit_joint_frames certificates regenerate byte-identical.","changes":["scripts/audit_fused_block.py: rational_rank skips redundant Q(x) when x is already a Fraction","scripts/audit_stage_pair.py: modular_rank uses cached _mod_inv(d, prime) helper instead of fresh pow() per entry"],"contributor":"Aaron Wang (navinor)","confirmation_runs":"Post-restart rerun of benchmark.py --rounds 3 passed twice (PASS_IDENTICAL_STAGE_PAIR_RECEIPTS, EXITCODE=0); medians 15.75 s and 15.94 s are slower than the clean sequential median because the two confirmation jobs ran concurrently and shared CPU. Correctness claim unaffected; speedup figures are from the clean sequential measurements.","dependent_certificates_identical":["certificates/fused-block-audit.json","certificates/joint-frame-audit.json"],"environment":{"interpreter":"CPython 3.12.14 stdlib only","isolated_host":false,"os":"Windows","os_caches_flushed":false},"full_audit_seconds":{"optimized":[14.3083423,13.95616,13.6430093],"optimized_median":13.95616,"original":[16.6334076,15.78521,15.5721638],"original_median":15.78521},"license":"Apache-2.0","microbenchmarks":{"modular_denominator_conversion_20_mats":{"new_seconds":0.003,"old_seconds":0.0043,"speedup":1.42},"rational_rank_50_mats":{"new_seconds":0.0094,"old_seconds":0.0285,"speedup":3.05}},"negative_controls":["ValueError path for denominator multiple of prime raises identically in reference and optimized modular_rank","Existing lower-bound test (modular 1 vs exact 2) passes unchanged","Invalid candidate_score input still raises ValueError (existing test)","Changed-matrix mutation covered by affected-edge recomputation test (existing test)"],"scope":"Fixed historical check only: h=6, prime 101, 4082 baseline edges, 64 candidates, 36x36 rational matrices. No frontier, supplier, kappa, or runtime claim.","speedup":{"less_wall_time_percent":11.6,"median_factor":1.13},"status":"Tooling improvement; no new kappa","tests":{"tests.test_fused_block":"7 tests OK","tests.test_joint_frames":"8 tests OK","tests.test_quadratic_affine":"4 tests OK","tests.test_stage_pair":"8 tests OK","tests.test_stage_pair_performance":"6 tests OK"},"unresolved":["86 s audit_quadratic_affine replay not rerun end to end; covered by shared-helper equivalence suite and its 4 unit tests","Full make verify and selected-supplier cold replay not rerun for this isolated historical-checker change"]}

research/stage-pair-performance/SELECTION.json

@@ -0,0 +1,38 @@
+{
+ "assignment_seed": 92,
+ "base_commit": "7ee02b7c96d85f22b38d28bd7ed49bb8c1aeb261",
+ "candidate_tasks": [
+  {
+   "method": "Memoized modular inverses plus redundant Fraction elimination in exact rank",
+   "object": "scripts/audit_stage_pair.py modular_rank and scripts/audit_fused_block.py rational_rank",
+   "parameter_region": "h=6, 64 candidates, 36x36 rational matrices, prime 101",
+   "reason_chosen": "Underexplored: no open PR covers stage-pair audit; profiling shows 15.8s runtime dominated by Fraction ops; seed 92 mod 3 selects this third-listed option after rejecting duplicate owner-mask and higher-risk Bareiss ideas",
+   "status": "selected"
+  },
+  {
+   "method": "Fraction-free Bareiss exact rank for stage-pair baseline",
+   "object": "scripts/audit_stage_pair.py baseline via new integer algorithm",
+   "parameter_region": "Same h=6 historical check",
+   "reason_rejected": "Higher proof risk and review cost; requires new algorithm correctness argument for rectangular matrices with row swaps; deferred after safe memoization win",
+   "status": "rejected_higher_risk"
+  },
+  {
+   "method": "Bitmask owner footprints for PR741 checker (extend PR727 pattern)",
+   "object": "research/complex-sw4-frame8/extras/check_owner_map.py 36288 roles",
+   "parameter_region": "Selected frontier741 owner map",
+   "reason_rejected": "Duplicates PR727 method on different object; seed alone does not make duplicate useful; stage-pair object is less explored",
+   "status": "rejected_duplicate_pattern"
+  }
+ ],
+ "coordination_issue": "No pinned sprint coordination issue found (only issue #650 about Discussions); claim recorded locally per instructions; PR overlap checked via gh pr list through #811 on 2026-10-13; no stage-pair tooling PR open",
+ "deliverable": "Two-line-exact verification-engineering speedup with identical exhaustive receipts, benchmark script, focused equivalence tests, and updated certificate binding for audit_stage_pair.py only",
+ "expected_finish": "2026-10-13T06:00:00-04:00",
+ "method": "Cache pow() modular inverses via lru_cache helper; skip redundant Q(Q) construction when entries already Fractions. Same arithmetic, same elimination steps, same ranks, same exceptions; no change to search domain, stopping rules, or rejection logic",
+ "object": "scripts/audit_stage_pair.py (historical two-invocation frame search, h=6, 64 candidates) and its exact-rank dependency scripts/audit_fused_block.py rational_rank",
+ "proposed_parameter_range": "Fixed historical check: h=6, prime 101, 4082 baseline edges, 64 candidates, 36x36 rational matrices; no frontier kappa change",
+ "track": "85-94 verification engineering",
+ "result": {
+  "pr": "https://github.com/CrocSwap/integer-mult-bounds/pull/824",
+  "status": "OPEN tooling improvement; 29 tests OK; benchmark PASS"
+ }
+}

research/stage-pair-performance/benchmark.py

@@ -0,0 +1,76 @@
+#!/usr/bin/env python3
+"""Benchmark the optimized stage-pair audit and check identical receipts.
+
+Runs the complete historical two-invocation frame search in fresh
+subprocesses (no certificate file write) and compares the returned
+certificate to the committed baseline, allowing only the expected
+implementation-source hash change for scripts/audit_stage_pair.py.
+
+Timing is descriptive, not a pass threshold. OS caches are not flushed;
+the machine is not an isolated benchmark host. No frontier replay or
+multiplication runtime is measured.
+
+Reproduce with standard-library Python 3.11+ from the repository root:
+
+    python3 -B research/stage-pair-performance/benchmark.py --rounds 3
+
+Expected: PASS_IDENTICAL_STAGE_PAIR_RECEIPTS
+"""
+import argparse
+import json
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[2]
+BASE_CERT = "certificates/stage-pair-audit.json"
+
+
+def run_certificate():
+    cmd = [sys.executable, "-B", "-c",
+           "import sys; sys.path.insert(0,'scripts');"
+           "from audit_stage_pair import certificate;"
+           "import json; print(json.dumps(certificate(), default=str))"]
+    t0 = time.monotonic()
+    out = subprocess.check_output(cmd, cwd=ROOT, text=True)
+    dt = time.monotonic() - t0
+    return json.loads(out), dt
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--rounds", type=int, default=3)
+    a = ap.parse_args()
+    base = json.loads((ROOT / BASE_CERT).read_bytes())
+    times = []
+    result = None
+    for _ in range(a.rounds):
+        result, dt = run_certificate()
+        times.append(dt)
+    assert set(result.keys()) == set(base.keys()), "certificate keys changed"
+    for key in result:
+        if key == "source_sha256":
+            assert result[key].keys() == base[key].keys(), "source list changed"
+            for path in result[key]:
+                if path == "scripts/audit_stage_pair.py":
+                    assert result[key][path] != base[key][path], \
+                        "expected changed implementation hash missing"
+                else:
+                    assert result[key][path] == base[key][path], \
+                        "unexpected source change: " + path
+            continue
+        assert json.dumps(result[key], sort_keys=True, default=str) == \
+            json.dumps(base[key], sort_keys=True, default=str), \
+            "mathematical content changed: " + key
+    assert result["rank_sum"] == 8684, "baseline rank sum changed"
+    assert len(result["candidates"]) == 64, "candidate count changed"
+    assert len(result["improved_candidates"]) == 6, "improved count changed"
+    times.sort()
+    median = times[len(times) // 2]
+    print(f"PASS_IDENTICAL_STAGE_PAIR_RECEIPTS rounds={a.rounds} "
+          f"median_seconds={median:.3f} samples={','.join(f'{t:.2f}' for t in times)}")
+
+
+if __name__ == "__main__":
+    main()

scripts/audit_fused_block.py

@@ -111,7 +111,7 @@ def sandwich_numerator(state,x_positions,y_positions,nbits):
 
 
 def rational_rank(matrix):
-    rows=[[Q(x) for x in row] for row in matrix]
+    rows=[[x if isinstance(x, Q) else Q(x) for x in row] for row in matrix]
     rank=0
     if not rows:return 0
     for col in range(len(rows[0])):

scripts/audit_stage_pair.py

@@ -125,9 +125,14 @@ def baseline(g):
     return result
 
 
+@lru_cache(maxsize=None)
+def _mod_inv(d, prime):
+    return pow(d, -1, prime)
+
+
 @lru_cache(maxsize=65536)
 def modular_rank(A,prime=101):
-    rows=[[int(x.numerator)*pow(int(x.denominator)%prime,-1,prime)%prime for x in row] for row in A]
+    rows=[[int(x.numerator)*_mod_inv(int(x.denominator)%prime,prime)%prime for x in row] for row in A]
     n=len(rows);pivot=0
     for j in range(len(rows[0])):
         k=next((i for i in range(pivot,n) if rows[i][j]),None)

tests/test_stage_pair_performance.py

@@ -0,0 +1,129 @@
+"""Equivalence of optimized exact-rank helpers with hash-pinned predecessors.
+
+Reference functions are verbatim copies of the base-commit implementations
+(base 7ee02b7c96d85f22b38d28bd7ed49bb8c1aeb261):
+- scripts/audit_fused_block.py rational_rank (Q(Q) construction)
+- scripts/audit_stage_pair.py modular_rank (direct pow() inverses)
+
+The optimized versions must return identical ranks and raise identical
+exceptions on all inputs, including edge cases. No search domain,
+stopping rule, or rejection logic is changed.
+"""
+from fractions import Fraction as Q
+from pathlib import Path
+import sys
+import unittest
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
+from audit_fused_block import rational_rank as optimized_rank
+from audit_stage_pair import modular_rank as optimized_mod, _mod_inv
+from audit_joint_frames import matrix
+
+
+def reference_rank(m):
+    rows = [[Q(x) for x in row] for row in m]
+    rank = 0
+    if not rows:
+        return 0
+    for col in range(len(rows[0])):
+        pivot = next((i for i in range(rank, len(rows)) if rows[i][col]), None)
+        if pivot is None:
+            continue
+        rows[rank], rows[pivot] = rows[pivot], rows[rank]
+        scale = rows[rank][col]
+        rows[rank] = [x / scale for x in rows[rank]]
+        for i in range(rank + 1, len(rows)):
+            s = rows[i][col]
+            if s:
+                rows[i] = [x - s * y for x, y in zip(rows[i], rows[rank])]
+        rank += 1
+        if rank == len(rows):
+            break
+    return rank
+
+
+def reference_mod(A, prime=101):
+    rows = [[int(x.numerator) * pow(int(x.denominator) % prime, -1, prime) % prime
+             for x in row] for row in A]
+    n = len(rows)
+    pivot = 0
+    for j in range(len(rows[0])):
+        k = next((i for i in range(pivot, n) if rows[i][j]), None)
+        if k is None:
+            continue
+        rows[pivot], rows[k] = rows[k], rows[pivot]
+        inv = pow(rows[pivot][j], -1, prime)
+        rows[pivot] = [(x * inv) % prime for x in rows[pivot]]
+        for i in range(pivot + 1, n):
+            if rows[i][j]:
+                c = rows[i][j]
+                rows[i] = [(x - c * y) % prime for x, y in zip(rows[i], rows[pivot])]
+        pivot += 1
+        if pivot == n:
+            break
+    return pivot
+
+
+class RankEquivalence(unittest.TestCase):
+    def test_rational_rank_matches_reference(self):
+        cases = [
+            [[]],
+            [[0]],
+            [[1]],
+            [[Q(1, 3)]],
+            ((Q(1, 2), Q(1, 3)), (Q(1, 4), Q(1, 5))),
+            ((1, 0), (0, 1)),
+            ((1, 2), (2, 4)),
+            ((101, 0), (0, Q(1, 2))),
+            ((-Q(1, 6), Q(1, 3)), (Q(0), Q(-1, 6))),
+            tuple(tuple(Q(i * j, 7) for j in range(5)) for i in range(4)),
+            tuple(tuple(Q((-1) ** (i + j), 6) for j in range(6)) for i in range(6)),
+            [[Q(10 ** 30, 3), Q(-7, 11)], [Q(5), Q(0)]],
+            matrix(((1, 0), (0, 1))),
+        ]
+        for m in cases:
+            self.assertEqual(optimized_rank(m), reference_rank(m), msg=str(m)[:120])
+
+    def test_rational_rank_accepts_ints_and_fractions(self):
+        m = [[1, 0], [0, 1]]
+        self.assertEqual(optimized_rank(m), 2)
+        mf = [[Q(1), Q(0)], [Q(0), Q(1)]]
+        self.assertEqual(optimized_rank(mf), 2)
+        self.assertEqual(optimized_rank(m), reference_rank(m))
+
+    def test_modular_rank_matches_reference(self):
+        cases = [
+            matrix(((101, 0), (0, Q(1, 2)))),
+            matrix(((1, 0), (0, 1))),
+            matrix(((1, 2), (3, 4))),
+            matrix(((Q(1, 3), Q(-1, 6)), (Q(0), Q(1, 3)))),
+            matrix(tuple(tuple(Q(i + j, 6) for j in range(4)) for i in range(4))),
+        ]
+        for m in cases:
+            for prime in (101, 103):
+                self.assertEqual(optimized_mod(m, prime), reference_mod(m, prime))
+
+    def test_modular_lower_bound_property_preserved(self):
+        from audit_joint_frames import rank as exact_rank
+        m = matrix(((101, 0), (0, Q(1, 2))))
+        self.assertEqual(optimized_mod(m), 1)
+        self.assertEqual(exact_rank(m), 2)
+        self.assertEqual(optimized_mod(m, 103), 2)
+
+    def test_noninvertible_denominator_raises_same_error(self):
+        m = ((Q(1, 101),),)
+        with self.assertRaises(ValueError):
+            reference_mod(m, 101)
+        with self.assertRaises(ValueError):
+            optimized_mod(m, 101)
+        with self.assertRaises(ValueError):
+            _mod_inv(0, 101)
+
+    def test_mod_inv_cache_returns_same_values(self):
+        for d in (1, 2, 3, 6, 7, 50, 100):
+            self.assertEqual(_mod_inv(d, 101), pow(d, -1, 101))
+            self.assertEqual(_mod_inv(d, 103), pow(d, -1, 103))
+
+
+if __name__ == "__main__":
+    unittest.main()