Reduce complete literal header-census time by 47.30% (no κ)

research/low-rank-header-conjugation/BENCHMARK.json

@@ -0,0 +1,61 @@
+{
+  "compiler": "Apple clang version 21.0.0 (clang-2100.4.1.1)\nTarget: arm64-apple-darwin27.0.0\nThread model: posix\nInstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin\n",
+  "errors": 0,
+  "flags": [
+    "-O2",
+    "-std=c11",
+    "-Wall",
+    "-Wextra",
+    "-Werror",
+    "-pedantic"
+  ],
+  "implementation_sha256": {
+    "benchmark.py": "a7cbcac08facd2f6221e3196a13fbbb837b14a80dd1ce4afd2de3fda043acb5e",
+    "build.py": "87f3feee3662dcea125fe32a13ba75992c9244c8b4daa57ce8af9d59f031a6bc",
+    "conjugate.h": "3c4a32beaf738c579ebec766f42d79e999609bcac0a4b2350b0b1b9fb48b516a"
+  },
+  "median_ns": [
+    9341309958,
+    4922998958
+  ],
+  "order": "baseline,candidate for each pair",
+  "platform": "macOS-27.0.1-arm64-arm-64bit",
+  "python": "3.12.15 (main, Oct 14 2026, 00:55:37) [Clang 23.1.3 ]",
+  "range_ns": [
+    [
+      9005408583,
+      10028481167
+    ],
+    [
+      4758782667,
+      5112260125
+    ]
+  ],
+  "reduction_percent": 47.29862321093532,
+  "rounds": 5,
+  "samples_ns": [
+    [
+      10028481167,
+      9341309958,
+      9517116167,
+      9142645000,
+      9005408583
+    ],
+    [
+      4983895209,
+      5112260125,
+      4858205709,
+      4922998958,
+      4758782667
+    ]
+  ],
+  "schema": "complete-header-benchmark/1",
+  "source_sha256": "ab0b0c9f4b296ee4cac6d7b9d84713e16cc61ef0fb3e03eb88d3fbcc412ed4ee",
+  "stdout": "q=3 m=4 GL=24261120 one=7705152 pair=1989504 oneway=2270016 k3=550848 k4=125184 laminar=4366656 split=1247616\n",
+  "stdout_sha256": "c1b871db92194b1de14ceece168353c2c3b3af76f22f9095080892783c4f8cc5",
+  "timed_region": "fresh census process through complete output; compilation excluded",
+  "workload": [
+    3,
+    4
+  ]
+}

research/low-rank-header-conjugation/README.md

@@ -0,0 +1,156 @@
+# Faster literal header census by low-rank conjugation
+
+This implementation was produced by Nick Petalas. Apache-2.0.
+
+The complete `q=3,m=4` command of Ryanson Jonathan's
+[PR4345](https://github.com/CrocSwap/integer-mult-bounds/pull/4345) takes
+47.30% less local wall time with a direct low-rank request kernel.
+Five alternating fresh-process pairs give median times of 9.341 seconds
+and 4.923 seconds. The ranges are 9.005 to 10.028 seconds and 4.759 to
+5.112 seconds. All ten commands finish the same 24,261,120-header census
+with zero errors and byte-identical output. [BENCHMARK.json](BENCHMARK.json)
+is the generated source-bound measurement record.
+
+Claim type: scoped implementation-preservation proof with finite
+conformance and local performance evidence. Existing counts and counting
+theorems are cited context, not new results or audit deliverables.
+No new kappa, supplier admission or selected-frontier change is claimed.
+
+## What changes
+
+The retained engine forms each request as `G R G^-1` with two dense matrix
+products. The eight fixed requests have rank one or two, or are identity
+minus a rank-one matrix. The new kernel forms the conjugate directly from
+columns of G and rows of its already-computed inverse. It changes that
+arithmetic from O(m^3) to O(m^2) per request.
+
+The adapter authenticates the exact retained source before generating C.
+It replaces the dense multiply helper with the new kernel and connects the
+request index through three uniquely matched fixed substitutions. Every matrix
+enumeration, inverse, guard-elimination statement, short-circuit evaluation,
+counter and output statement retains its original source text.
+Source changes reject rather than silently applying a stale adapter.
+No predecessor file is edited.
+
+## Preservation proof
+
+Work over any field and let G be invertible. Write c_t=G e_t and
+r_t=e_t^T G^-1. Then `G I G^-1=I` and
+`G (u v^T) G^-1=(G u)(v^T G^-1)` give the entire request table:
+
+| Request index | Original request | New conjugate |
+| --- | --- | --- |
+| t=0,1,2,3 when t<m | I-e_t e_t^T | I-c_t r_t |
+| 4 | I-e_0(e_0+e_1)^T | I-c_0(r_0+r_1) |
+| 5,6 | e_t e_t^T, t=0,1 | c_t r_t |
+| 7 | e_0 e_0^T+e_1 e_1^T | c_0 r_0+c_1 r_1 |
+
+These are identities for every entry, including odd characteristic and
+extension fields. Thus every guard receives the identical matrix at every
+invertible header. Its unchanged elimination and short-circuit logic return
+the same Boolean result. Singular matrices are skipped by the unchanged
+inverse routine on both sides. The complete enumerations and all output
+counters therefore agree. No formula for a count, conjugacy quotient,
+sampling reduction or cached scientific result is used for the speedup.
+
+The identities are classical. The contribution is this specialized engine
+kernel, its source-bound adapter, and the measured complete-command saving.
+
+## Measurement and limiter
+
+The host is macOS 27.0.1 arm64 with 18 logical CPUs and Apple clang 21.0.0.
+Both binaries use `-O2 -std=c11 -Wall -Wextra -Werror -pedantic`.
+The timed region includes a fresh process, the entire matrix scan, all
+inversions, guards and final output. Compilation is excluded. Neither side
+reuses state between processes. Other research processes were active;
+alternation puts both sides under the same changing host load.
+
+A separate three-second `/usr/bin/sample` run supplied 2,572 main-thread
+samples. Exactly 1,474 landed at the dense conjugation call on retained
+`headers.c:70`. Removing that work can explain the observed reduction.
+The unchanged matrix scan, inverse and guard elimination limit the gain.
+Profiling samples are excluded from the five timing pairs. The benchmark
+covers the complete census command, not compilation or an entire research
+package workflow, and is host-descriptive rather than a CI timing gate.
+
+## Reproduction
+
+From the repository root, use CPython 3.12 and a C11 `cc` compiler:
+
+```sh
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/low-rank-header-conjugation/verify.py
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -O -W error -B research/low-rank-header-conjugation/verify.py
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package research/low-rank-header-conjugation
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/check_research_commit.py --package research/low-rank-header-conjugation --check-source-history
+/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/low-rank-header-conjugation/benchmark.py
+```
+
+Another CPython 3.12 installation can replace the absolute interpreter path.
+Set `TMPDIR` to an external scratch directory to choose where generated C
+and binaries live. Temporary builds are removed on success or failure.
+The default verifier and benchmark write only to their temporary directories
+and stdout. The benchmark needs at least five pairs; its timings are optional
+and are not part of registered correctness acceptance.
+
+The verifier compares all eleven complete historical commands over F2, F3,
+F4 and F5, including GL5(F2), GL4(F3), GL3(F4) and GL3(F5).
+A separate C harness compares every conjugated matrix entry in eight complete
+smaller header domains, before any guard can hide a discrepancy. It checks
+14,746,536 entries in 1,515,444 request matrices on 213,714 headers. Four actual
+kernel mutations must fail: a missing odd-field sign, an incomplete one-way
+dual, transposed row/column access and a missing rank-two summand. Altered
+retained source bytes also reject. Checks use explicit exceptions and return
+codes and remain active with optimized Python.
+
+The direct marker is `PASS_LOW_RANK_HEADER_CONJUGATION`. Its generated
+stdout receipt binds the package files by SHA256. The generic runner separately
+authenticates declared source and historical metadata.
+
+## Limits and engineering notes
+
+The preservation proof applies to these eight requests wherever the retained
+engine executes correctly. The executable scope is the eleven bounded
+historical domains, not arbitrary sizes accepted by its legacy CLI. In
+particular, this package does not repair its large-size counter-overflow or
+input-parsing limitations. Those retained paths are unnecessary for the
+measured command and finite conformance domains.
+
+New C functions use fixed stack matrices, no recursion and bounded loops.
+The comparison harness has at most 4^9 raw matrices per domain and matrices
+of order at most four. The complete-command check also runs the unchanged
+binary order-five and ternary order-four scans. Python subprocesses have
+explicit timeouts; functions remain under 60 lines. Temporary directories
+own all generated sources and binaries. Warning-strict C builds and Python
+replays supply diagnostics. Managed Python allocation, the fixed source
+generation adapter and the retained engine's global field tables and macro
+are scoped Power of Ten deviations. Live work has fixed fixture and file-size
+bounds; no general code-generation framework or shared writer is added.
+
+No counting theorem, all-size native operation, physical compiler, selected
+cold replay, Lean or Docker build was audited or run. The five inherited
+framework assumptions retain their status. Independent human and formal
+proof-kernel review are unclaimed.
+
+## Open next questions
+
+Can an actual caller supply low-rank request factors directly and use this
+kernel without constructing dense requests? Start with one complete caller,
+preserving its field representation, arbitrary inputs and every guard.
+That caller and its benchmark remain missing. No further optimization was
+attempted after this first measured improvement.
+
+## Credit and source status
+
+Credit Ryanson Jonathan / pinterbanget for PR4345's complete literal engine,
+field implementation, request suite and original source notice. Credit Nick
+Petalas's PR4259/4261/4273/4274/4275 for the cited header models and counting
+laws, and retain the earlier BIT/header and normalizer lineage. None of those
+mathematical results is newly audited here.
+
+[SOURCES.json](SOURCES.json) pins canonical main
+`7c49fe32c55de02c2c873655c804b16e4c3fef0a`, the current card and the exact C
+input. The original claim was changed before implementation in the
+[scope update](https://github.com/CrocSwap/integer-mult-bounds/discussions/1166#discussioncomment-19360966).
+Live open PRs, recent account work, handoffs, local reservations, merged
+searches and main README terms were checked. Sparse notes, arbitrary diffs
+and unpublished work remain coverage limits. Only this new package is changed.

research/low-rank-header-conjugation/SOURCES.json

@@ -0,0 +1,24 @@
+{
+  "schema_version": 1,
+  "base_commit": "7c49fe32c55de02c2c873655c804b16e4c3fef0a",
+  "files": [
+    {
+      "path": "certificates/selected-result.json",
+      "usage": "provenance",
+      "git_blob": "d59b5035e7b4cbaa9d688a1d32dd0d6eb79c466a",
+      "sha256": "9b8d0eea95b3bf0dfcb7938a732a6439b87a2685dec2a5ab9e525b3b34debbfa"
+    },
+    {
+      "path": "docs/research/agent-task-catalog.json",
+      "usage": "provenance",
+      "git_blob": "3404fbdd5e7529a0674eef26114b5071e4586cec",
+      "sha256": "ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81"
+    },
+    {
+      "path": "research/full-guard-header-census/headers.c",
+      "usage": "input",
+      "git_blob": "42a826da296fc777f0734523afe9df36eed3e4c5",
+      "sha256": "ab0b0c9f4b296ee4cac6d7b9d84713e16cc61ef0fb3e03eb88d3fbcc412ed4ee"
+    }
+  ]
+}

research/low-rank-header-conjugation/TASK.json

@@ -0,0 +1,81 @@
+{
+  "schema_version": 1,
+  "task_id": "replay-runtime",
+  "priority_revision": 8,
+  "base_commit": "7c49fe32c55de02c2c873655c804b16e4c3fef0a",
+  "original_card": {
+    "id": "replay-runtime",
+    "question": "Can one measured replay bottleneck be sped up without dropping coverage?",
+    "starting_points": [
+      "scripts/run_isolated_tests.py",
+      "scripts/verify_replay_coverage.py",
+      "certificates/selected-result.json"
+    ],
+    "methods": [
+      "Profile one isolated replay and optimize the measured bottleneck.",
+      "Design source-bound reuse with explicit dependency invalidation and negative controls."
+    ],
+    "acceptance_gate": "Report before/after wall time on the same fixture, retained checks and invalidation behavior.",
+    "first_step": {
+      "command": [
+        "python3",
+        "-B",
+        "scripts/research_selected_baseline.py"
+      ],
+      "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY",
+      "inspect": "Resolve the current selected review, separate complex/ordinary BIT/admitted BIT/outer budgets, and compare the card against live handoffs before starting. Historical ceilings require exact source/model equivalence.",
+      "scope": "Source-binding orientation only; not a scientific replay or all-size proof. Materialize selected inputs if this checkout is sparse."
+    },
+    "first_experiment": "Measure one unchanged fixture, preserve check coverage, and test one optimization with invalidation controls.",
+    "bottleneck_link": "Faster trustworthy replays increase review throughput and let mechanism candidates be checked sooner.",
+    "allowed_variations": [
+      "Profile one stage or design one source-bound cache boundary."
+    ],
+    "scope_change_triggers": [
+      "Skipping coverage or comparing different fixtures to claim speedup."
+    ],
+    "overlap_terms": [
+      "replay runtime",
+      "cache invalidation",
+      "isolated tests"
+    ]
+  },
+  "scope_status": "refined",
+  "actual_scope": "Speed the complete PR4345 C header census by replacing dense conjugation of its eight canonical requests with exact rank-one/rank-two formulas. Preserve all header enumeration, inversion, guards and output fields.",
+  "scope_change_reason": "The implementation bottleneck is measured in the complete q=3,m=4 command. Existing counts are conformance outputs, not new mathematical or audit results.",
+  "bottleneck_link": "Verifier engineering only. The optimization reduces complete literal-census replay cost without changing the covered mathematical domain or the selected frontier.",
+  "budget_minutes": 60,
+  "stop_rule": "Ship the first measured improvement after complete conformance checks; do not extend the census domain or optimize another stage.",
+  "outcome": "completed",
+  "evidence": "Written conjugation identity, generated alternating benchmark, full-command comparisons and entrywise finite conformance with semantic mutations.",
+  "author": "Nick Petalas",
+  "ai_assistance": "Independent human and formal proof-kernel review are unclaimed.",
+  "open_questions": [
+    {
+      "question": "Can a consumer use the same low-rank request kernel without rebuilding dense requests?",
+      "why_it_matters": "The present adapter demonstrates one complete historical census improvement.",
+      "first_step": "Identify one actual caller with explicitly supplied low-rank factors and retain every guard and input contract.",
+      "blocked_by": "No further caller or benchmark has been supplied; this result stops at the measured engine."
+    }
+  ],
+  "claim": "https://github.com/CrocSwap/integer-mult-bounds/discussions/1166#discussioncomment-19360902",
+  "scope_update": "https://github.com/CrocSwap/integer-mult-bounds/discussions/1166#discussioncomment-19360966",
+  "source_pr": 4345,
+  "source_head": "e99128a6785b562dd7a73ee550bf32ff06ac5b4a",
+  "upstream_checkpoint": {
+    "commit": "7c49fe32c55de02c2c873655c804b16e4c3fef0a",
+    "priority_revision": 8,
+    "checked_utc": "2026-11-02T21:16:10Z",
+    "selected_kappa": "781451/250000000"
+  },
+  "preserved_checkout": {
+    "branch": "research/w15-1102a-frontier-step",
+    "head": "f9e68f40b0da7236a7fad415f7172218c0666d27"
+  },
+  "overlap": {
+    "source": "PR4345",
+    "source_author": "pinterbanget",
+    "checked": "2026-11-02",
+    "coverage": "Live open PR bodies, 319 account PRs created since October 26, recent handoffs, local reservations, main README searches and targeted merged searches. No duplicate low-rank conjugation implementation found. Sparse local cache, arbitrary historical diffs and unpublished work remain limits."
+  }
+}

research/low-rank-header-conjugation/VERIFY.json

@@ -0,0 +1,23 @@
+{
+  "schema_version": 1,
+  "status": "ready",
+  "python": "3.12",
+  "requirements": [],
+  "inputs": [
+    "research/full-guard-header-census/headers.c"
+  ],
+  "commands": [
+    {
+      "argv": [
+        "{python}",
+        "-I",
+        "-W",
+        "error",
+        "-B",
+        "{package}/verify.py"
+      ],
+      "timeout_seconds": 240,
+      "checks": "Compile authenticated baseline and low-rank adapter, compare all eleven complete historical census commands, compare all matrix entries in eight finite domains, reject four semantic kernel mutations and altered baseline bytes."
+    }
+  ]
+}