Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion certificates/dyadic-central-factor.json
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@
"scripts/binary_phase_frames.py": "4148cfa313cf050ed25296e8db36d35871520b8318fd086c7d1facf0c6af67d7",
"scripts/complex_compression.py": "9b6f758a5985ba5d9fdd726fe6556b3d3b4bd9d98aaf8ba06a7bb58093c4c504",
"scripts/exclusion_circuit.py": "f9fbcd6e85e442cbd0926ac0887414e20735b2dc923d2d5822c803ceb46726d3",
"scripts/experiments/complex_centers.py": "3744396587d729253d80980f6310b56e7bf0ea21b165bd178d0d6d4c49faf2e2",
"scripts/experiments/complex_centers.py": "39cfcf869dc4a00dd30a1e91efb986074371621dad14b390cf1053f6aeea79bf",
"scripts/experiments/disjoint_tensor.py": "085600fc922ae167b5981505e1a5caba4d1c1df15482f61d83d2c086fa89eac1",
"scripts/general_network_guard.py": "790a2064449afdbffd6dbd4adbcf4a28eaf41931ce7134e4a4e8766f3764270c"
},
Expand Down
19 changes: 19 additions & 0 deletions research/central-factor-production/NOTICE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Attribution and license

Prepared by Wentao Li ([VictorLiwentao](https://github.com/VictorLiwentao),
[@Is__Victor](https://x.com/Is__Victor), wentaol4@illinois.edu) with substantial
AI assistance for orientation, overlap search, profiling, production wiring,
packaging, benchmarking and writing. Not independent human peer review.
Apache-2.0; see the repository root `LICENSE`.

The injective packed-row algorithm and its radix argument were introduced by
**Ross Richardson (@RossRichardson)** in the opt-in adapter
`research/central-factor-packed/` (see that package's NOTICE.md). This package
wires an equivalent check into the production
`scripts/experiments/complex_centers.py:verify_factor` path used by
`make verify-certificates`.

The original dyadic factor, structural validation, receipt schema, side circuit,
phase argument and cost ledger are by **Douglas R. Colkitt**, retained at
`d55bcd62ceb8f322bcf311a5a66e94c3ac7724f0`. Upstream material retains its own
source notices and license.
94 changes: 94 additions & 0 deletions research/central-factor-production/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Production packed dyadic central-factor check

**Tooling improvement; no new kappa.** At base
`0fdd7d82de03e43c9e5a1000dfbcd380d1b687d2`, `scripts/experiments/complex_centers.py:verify_factor`
still used the historical nested entrywise loop over all scatter/gather pairs.
Ross Richardson's opt-in adapter `research/central-factor-packed/` already proved
an injective packed-row algorithm with identical receipts. This package wires
that equivalent check into the **production** source used by
`scripts/audit_complex_centers.py` / `make verify-certificates`.

Only the source pin for `scripts/experiments/complex_centers.py` changes inside
`certificates/dyadic-central-factor.json`. Every coefficient is still checked;
`circuit`/`factor` generators, side/phase obligations and the retained historical
`2^-31` language are unchanged. Selected frontier untouched.

## Reproduce

Python **3.12**, standard library only, from the repository root:

```sh
python3.12 -I -B research/central-factor-production/verify.py
python3.12 -I -O -B research/central-factor-production/verify.py
python3.12 -B scripts/research_package.py --package research/central-factor-production
python3.12 -I -B research/central-factor-production/benchmark.py --rounds 5
```

Expected:

```text
PASS_CENTRAL_FACTOR_PRODUCTION heights=[8, 10, 12, 18, 26] ... no_new_kappa
PASS_REGISTERED_RESEARCH_PACKAGE research/central-factor-production outcome=completed
PASS_IDENTICAL_CENTRAL_FACTOR_PRODUCTION_RECEIPTS
```

Default verify is read-only. Use `--write-certificate` only after editing the
production source. Timing is host-descriptive and is not a correctness gate.

On Linux / CPython 3.12.3, five alternating fresh-process pairs gave median
**factor-only 2.615 → 0.059 s (44.06×)** and **complete audit 5.784 → 3.356 s
(1.72×)**. Samples are in [`RESULT.json`](RESULT.json).

## Why the packed check is equivalent

For each scatter row `u(S)` and gather column `v(T)` the historical obligation is

```text
sum_i u(S)[i] v(T)[i] = |S ∩ T| - 1.
```

Gather entries are in `{-1,0,1}` and appear at most once per column, so
`|actual - expected| <= ||u(S)||_1 + 2`. Choose
`bits = (max_S ||u(S)||_1 + 2).bit_length()` and place `B = 2**bits`. Packing
every gather/incidence column as `sum_j coeff_j B**j` and comparing packed
actual/expected rows is an injective encoding of the difference vector: the
packed equality holds if and only if every coefficient matches. This is the
same argument as `research/central-factor-packed/README.md`; structural and
scalar-charge gates are unchanged.

## Overlap and credit

Closest priors:

- [Ross Richardson / central-factor-packed](../central-factor-packed/) — opt-in
packed adapter; this package is the production wiring of that algorithm.
- [CrocSwap/integer-mult-bounds#756](https://github.com/CrocSwap/integer-mult-bounds/pull/756)
— identical-receipt tooling quality bar.
- Open leave-alone tooling on other objects (#1922 joint frames, #2054/#2150
shared/mixed-point, #2100 ternary, #2117 DisjointTensor.frame_data, #1861
parity-side, #2155 exclusion embedding residual) was not reopened.

Douglas R. Colkitt authored the dyadic central factor and original checker.
Ross Richardson authored the packed algorithm. Prepared by Wentao Li
([VictorLiwentao](https://github.com/VictorLiwentao),
[@Is__Victor](https://x.com/Is__Victor)) with substantial AI assistance, under
Apache-2.0.

## Open next questions

1. **Question:** After #2117's `DisjointTensor.frame_data` lands, what is the
residual wall time of `audit_complex_centers.py` with packed `verify_factor`
already in production?
**Why it matters:** `frame_data` still dominates the complete audit after the
coefficient check shrinks.
**First step:** Replay `benchmark.py --rounds 5` on a tree that also contains
the merged #2117 head.
**Blocked by:** #2117 is leave-alone / open.

2. **Question:** Should `verify-certificates` adopt a microbenchmark regression
gate for `verify_factor(h=26)`?
**Why it matters:** A future edit could restore the nested loop without
failing equivalence tests.
**First step:** Decide whether `RESULT.json` medians belong in CI or remain
descriptive.
**Blocked by:** Maintainer CI-budget preference.
142 changes: 142 additions & 0 deletions research/central-factor-production/RESULT.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
{
"ai_assistance": "Substantial AI assistance for orientation, overlap search, profiling, implementation, packaging, benchmarking and writing. Not independent human peer review.",
"base_commit": "0fdd7d82de03e43c9e5a1000dfbcd380d1b687d2",
"complete_audit": {
"median_baseline_seconds": 5.784228472999985,
"median_optimized_seconds": 3.356052378000186,
"median_ratio": 1.7235215132270096,
"rounds": 5,
"samples": [
{
"baseline_seconds": 5.760912544999883,
"optimized_seconds": 3.3874211810002635,
"order": [
"baseline",
"optimized"
],
"ratio": 1.7006779603647506,
"round": 0
},
{
"baseline_seconds": 5.784228472999985,
"optimized_seconds": 3.356052378000186,
"order": [
"optimized",
"baseline"
],
"ratio": 1.7235215132270096,
"round": 1
},
{
"baseline_seconds": 5.861394950999966,
"optimized_seconds": 3.3083781870000166,
"order": [
"baseline",
"optimized"
],
"ratio": 1.7716822623337942,
"round": 2
},
{
"baseline_seconds": 5.726917460000095,
"optimized_seconds": 3.339230219000001,
"order": [
"optimized",
"baseline"
],
"ratio": 1.715041217408225,
"round": 3
},
{
"baseline_seconds": 6.1704699699998855,
"optimized_seconds": 3.517462851000346,
"order": [
"baseline",
"optimized"
],
"ratio": 1.7542388452645747,
"round": 4
}
],
"scope": "audit"
},
"engines": {
"baseline": "research/central-factor-production/baseline.py",
"optimized": "scripts/experiments/complex_centers.py"
},
"environment": {
"cpu_affinity_set": false,
"executable": "/usr/bin/python3.12",
"implementation": "CPython",
"os_caches_flushed": false,
"platform": "Linux-6.12.94+-x86_64-with-glibc2.39",
"python": "3.12.3 (main, Aug 31 2026, 10:18:26) [GCC 13.3.0]"
},
"factor_only": {
"median_baseline_seconds": 2.615371209999921,
"median_optimized_seconds": 0.05935716699968907,
"median_ratio": 44.06159091139948,
"rounds": 5,
"samples": [
{
"baseline_seconds": 2.646467106000273,
"optimized_seconds": 0.061992203000045265,
"order": [
"baseline",
"optimized"
],
"ratio": 42.690321974819,
"round": 0
},
{
"baseline_seconds": 2.538598202999765,
"optimized_seconds": 0.060868976000165276,
"order": [
"optimized",
"baseline"
],
"ratio": 41.70594561986507,
"round": 1
},
{
"baseline_seconds": 2.518348651999986,
"optimized_seconds": 0.05801914999983637,
"order": [
"baseline",
"optimized"
],
"ratio": 43.40547305513935,
"round": 2
},
{
"baseline_seconds": 2.615371209999921,
"optimized_seconds": 0.05935716699968907,
"order": [
"optimized",
"baseline"
],
"ratio": 44.06159091139948,
"round": 3
},
{
"baseline_seconds": 2.6376186860002235,
"optimized_seconds": 0.05685944800006837,
"order": [
"baseline",
"optimized"
],
"ratio": 46.38839768541285,
"round": 4
}
],
"scope": "factor"
},
"no_new_kappa": true,
"predecessor": "research/central-factor-packed/",
"source_sha256": {
"baseline.py": "3744396587d729253d80980f6310b56e7bf0ea21b165bd178d0d6d4c49faf2e2",
"research/central-factor-packed/packed.py": "42f244d6e81dc5ea038a9dea19ccc803872603a94d5e405de14af8a47a11b9f6",
"scripts/experiments/complex_centers.py": "39cfcf869dc4a00dd30a1e91efb986074371621dad14b390cf1053f6aeea79bf"
},
"status": "PASS_IDENTICAL_CENTRAL_FACTOR_PRODUCTION_RECEIPTS"
}
54 changes: 54 additions & 0 deletions research/central-factor-production/SOURCES.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"schema_version": 1,
"base_commit": "0fdd7d82de03e43c9e5a1000dfbcd380d1b687d2",
"files": [
{
"path": "docs/research/agent-task-catalog.json",
"usage": "provenance",
"git_blob": "e8870e5917057d59b730eb14721a778639eca050",
"sha256": "d46eb6d5b3e279428753d770653663fbdf1063e55c255ea073d768f344927ef7"
},
{
"path": "certificates/selected-result.json",
"usage": "provenance",
"git_blob": "ae6a99f28aa468649eb66a361e764d769363bf11",
"sha256": "10b9e340fd8e354a37a0c4813c90e5de1dd42044b5f6aca4a3fc5a42b685fa32"
},
{
"path": "certificates/dyadic-central-factor.json",
"usage": "provenance",
"git_blob": "5bd4e583cd61b0e5183d74799e886afb57690960",
"sha256": "3f46000e645c338a583e0f2ab669a7c681980743d03335548eeb838072f1d458"
},
{
"path": "docs/research/dyadic-central-factor.md",
"usage": "provenance",
"git_blob": "033537f01d3fced8515fac9e2c10b7048d351df3",
"sha256": "e97b1dfe11b5eaebacf3f910131e13bcad586ea5fc1af05db23c00ae0eea5c11"
},
{
"path": "scripts/audit_complex_centers.py",
"usage": "provenance",
"git_blob": "3d985ee3eb1fd43ab5cc07a8570b438bf080e4a4",
"sha256": "166b18f560cd0d520d23fa29ee8147e0ea056358c4268a466ec458195a97a960"
},
{
"path": "scripts/experiments/complex_centers.py",
"usage": "provenance",
"git_blob": "13be871d501b7ebaaaec7e755ff4834c6cc49dc7",
"sha256": "3744396587d729253d80980f6310b56e7bf0ea21b165bd178d0d6d4c49faf2e2"
},
{
"path": "research/central-factor-packed/packed.py",
"usage": "input",
"git_blob": "379b6cc4a5933ec6c0b6cbe73273e5d097fb1335",
"sha256": "42f244d6e81dc5ea038a9dea19ccc803872603a94d5e405de14af8a47a11b9f6"
},
{
"path": "research/central-factor-packed/README.md",
"usage": "provenance",
"git_blob": "baac43252038396da941185b2e49c5a5002d95c5",
"sha256": "b91678e94e1fa6828ecbe66a623e6164765599d54f2bcdc9c9e006a5338926a3"
}
]
}
70 changes: 70 additions & 0 deletions research/central-factor-production/TASK.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
{
"schema_version": 1,
"task_id": "replay-runtime",
"priority_revision": 4,
"base_commit": "0fdd7d82de03e43c9e5a1000dfbcd380d1b687d2",
"chooser_seed": "23e9e450f1bf96b41368bf1bef28dd8e",
"original_card": {
"id": "replay-runtime",
"question": "Can one measured replay bottleneck be sped up without dropping coverage?",
"starting_points": [
"scripts/run_isolated_tests.py",
"scripts/verify_replay_coverage.py",
"certificates/selected-result.json"
],
"methods": [
"Profile one isolated replay and optimize the measured bottleneck.",
"Design source-bound reuse with explicit dependency invalidation and negative controls."
],
"acceptance_gate": "Report before/after wall time on the same fixture, retained checks and invalidation behavior.",
"first_step": {
"command": [
"python3",
"-B",
"scripts/research_baseline.py"
],
"expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY",
"inspect": "Read the selected reproduction command and choose a single replay stage for profiling.",
"scope": "Positive control / source-bound orientation only; this is not a new construction or full cold replay."
},
"first_experiment": "Measure one unchanged fixture, preserve check coverage, and test one optimization with invalidation controls.",
"bottleneck_link": "Faster trustworthy replays increase review throughput and let mechanism candidates be checked sooner.",
"allowed_variations": [
"Profile one stage or design one source-bound cache boundary."
],
"scope_change_triggers": [
"Skipping coverage or comparing different fixtures to claim speedup."
],
"overlap_terms": [
"replay runtime",
"cache invalidation",
"isolated tests"
]
},
"scope_status": "refined",
"actual_scope": "Can the production dyadic central-factor checker (scripts/experiments/complex_centers.py:verify_factor, exercised by scripts/audit_complex_centers.py / certificates/dyadic-central-factor.json) adopt the injective packed-row algorithm from research/central-factor-packed/ so the shipped verify-certificates path is faster while every coefficient check and receipt field stays identical?",
"scope_change_reason": "The catalogue card asks for one measured replay bottleneck with before/after wall time and retained checks. Expertise override selected verifier-engineering after profiling make verify-certificates poles. Leave-alone objects (joint frames, shared-point, mixed-point verify, ternary frames, parity-side, exclusion embedding residual, inplace compile-scan) were skipped. Ross Richardson's opt-in packed adapter already proved the algorithm; production still used the nested O(n^2) entrywise loop. This session wires that equivalent check into the shipped source and certificate pin.",
"bottleneck_link": "Side result on review throughput: dyadic-central-factor is part of verify-certificates. A faster identical coefficient check does not change kappa or the binding complex supplier.",
"budget_minutes": 60,
"stop_rule": "Ship an equivalence-preserving production verify_factor change with frozen baseline, identical factor and audit payloads, rejection/temp-corruption controls, certificate pin discipline, registered replay and a measured before/after median. Do not invent kappa or reopen leave-alone objects.",
"outcome": "completed",
"evidence": "PASS_CENTRAL_FACTOR_PRODUCTION via research/central-factor-production/verify.py (also under -O); PASS_IDENTICAL_CENTRAL_FACTOR_PRODUCTION_RECEIPTS via benchmark.py; certificates/dyadic-central-factor.json differs from base only in the complex_centers.py source pin.",
"author": "Wentao Li",
"github": "VictorLiwentao",
"email": "wentaol4@illinois.edu",
"ai_assistance": "Substantial AI assistance for orientation, overlap search, profiling, production wiring, packaging, benchmarking and writing. Not independent human peer review.",
"open_questions": [
{
"question": "After #2117's DisjointTensor.frame_data lands, what is the residual wall time of audit_complex_centers.py with packed verify_factor already in production?",
"why_it_matters": "frame_data still dominates the complete audit after the coefficient check shrinks; measuring the stacked gain decides whether further central-audit work is worth another session.",
"first_step": "Replay research/central-factor-production/benchmark.py --rounds 5 on a tree that also contains the merged #2117 head and compare complete_audit medians.",
"blocked_by": "#2117 is leave-alone / open; do not edit shared frame_data here."
},
{
"question": "Should verify-certificates adopt a microbenchmark regression gate for verify_factor(h=26)?",
"why_it_matters": "A future edit could restore the nested loop without failing equivalence tests.",
"first_step": "Decide whether RESULT.json medians belong in CI or remain descriptive.",
"blocked_by": "Maintainer CI-budget preference."
}
]
}
Loading
Loading