PR212's frame floor on PR217's butterfly word: κ < 6.968e-4 banked, +1.77% of frame room left (no new κ)

research/butterfly-word-frame-floor/README.md

@@ -0,0 +1,150 @@
+# The frame floor of PR217's butterfly word
+
+**No new κ, no certificate change.** PR212 bounded the bit coarse saving of one *word* --
+#200's physical bit word, shared by #205, #206 and #207 -- over every admissible choice of
+operation frames. PR217 builds a **different word**: 440 isolated four-input XOR
+reassociations (880 graph nodes and scalar controls) on top of PR211's cascade frames and
+PR200's construction. This note runs PR212's method unchanged on that word.
+
+## Result
+
+At 1,200 Lagrangian rounds, the round count of PR212's published run, on PR217's head
+`b739fc226a54a4a507989aa1bef092d491287451`:
+
+| Ledger | PR200's word (PR212) | PR217's butterfly word (here) |
+|---|---|---|
+| own: floor on `C` | 2,804,210 | 2,800,700 |
+| own: coarse saving < | 6.904e-4 | **6.913e-4** |
+| own: κ < | 6.900e-4 | 6.908e-4 |
+| completed banks: floor on `C` | 2,780,144 | 2,776,634 |
+| completed banks: coarse saving < | 6.964e-4 | **6.973e-4** |
+| completed banks: κ < | 6.959e-4 | 6.968e-4 |
+
+The like-for-like pair at 400 rounds moves together as well (own 6.909e-4 vs 6.918e-4,
+banked 6.969e-4 vs 6.978e-4), so the comparison does not depend on the round count.
+
+The word's own numbers, recomputed here from its certificate:
+
+| quantity | PR200's word | PR217's butterfly word |
+|---|---|---|
+| certified coarse saving, own ledger | 6.7777395e-4 | 6.7938424e-4 |
+| `D/C` of the certified layout, own | 6.7837287e-4 | 6.7998615e-4 |
+| certified coarse saving, completed banks | 6.8366e-4 (PR207) | 6.8516596e-4 |
+| `D/C` of the certified layout, banks | 6.8427e-4 (PR207) | 6.8578301e-4 |
+| κ of the composition | 6.831904550365e-4 (PR207) | 6.84696826673891e-4 |
+
+**So the butterflies raised the ceiling and the achieved value by about the same amount,
+and the room left is slightly smaller than before.** On the completed-bank ledger the
+distance from the achieved `D/C` to the ceiling is +1.68% here against +1.77% on #200's
+word; on the κ basis, +1.77% against +1.86%. Changing the DAG is not a way to enlarge the
+frame slack -- it is a different word with its own, slightly higher, ceiling. PR212's own
+scope statement already said that non-frame changes are unbounded by it; this note only
+confirms that the bounded part still binds after the change.
+
+## Can PR216's admitted cuts be carried onto this word?
+
+PR216 (`research/coordinated-crossover-pr200-v2`, head `34abdc5`) is a third layout on
+this same lineage: it keeps PR211's cascade frames, adds 220 further operation-frame
+changes, and admits them natively (κ = 6.838643131376e-4). Because the ceiling above
+bounds *every* frame layout of PR217's word, the natural next question -- combine the
+two -- has a bounded answer, and `transfer_check.py` checks its structural side against
+the two packages themselves:
+
+* **The two words are one operation-index space.** Both have 41,288 operations, and the
+  parts that fix the DAG are identical: 11,856 arcs, 25,900 order entries, 1,760 reuse
+  pairs, 3,960 gauges, 6,624 root roles, 17,688 phase-1 and 7,728 plain operations, and
+  the same 1,760 sources and reads. Exactly **880 operation contents** differ -- PR217's
+  reassociation set -- and 7,068 operation frames differ.
+* **PR216's admitted frames are keyed by that same index.** Its records are
+  `[op_index, [basis vectors …]]` pairs: 6,191 current bases, 6,258 override records and
+  2,268 canonical subspaces, all with indices inside the shared list. Only **3 of the
+  6,258** override records land on a rewritten operation, and **none of the 2,268**
+  genuinely different subspaces does.
+* **So a merge is well defined at the index level, and that is the whole of what is
+  free.** PR216 admits its cuts with "scalar operations, source/target contracts, …"
+  fixed, and PR217's change is exactly in those operations: a merged layout is a new
+  submission that needs fresh global admission and a fresh column replay, not a
+  transplant of an existing certificate.
+* **And the prize is capped.** Any frame layout of this word gives κ < 6.968e-4, i.e. at
+  most **+1.77%** over the 6.84696826673891e-4 record, while PR216's own layout on the
+  plain word sits **0.122% below** that record -- the merge would have to recover the
+  butterflies' whole contribution before it recovers anything. For scale, the bank
+  absorption of #219 claims +2.30% over the same record with an accounting ledger, which
+  is the route this comparison actually favours.
+
+## How the word was obtained and checked
+
+1. PR217's own constructor, `research/butterfly-coordinated-bit-211/candidate.py`, was
+   called with `--bit-root` at PR202's head `8d8d67bcf69c5ea67d3a29dbc64ba588156d6e8d`
+   (fetched from `pull/202/head`). It rebuilds the changed word from PR200's frozen word,
+   PR211's pinned frame payload (`opframe-bases.json.gz.b64`) and the frozen butterfly
+   overlay, checking each of those hashes itself.
+2. The three emitted payloads reproduce PR217's own declared hashes exactly:
+   `graph_p12.json` `2bb271df49780365…`, `word_p12.json` `ef77a7d30390a21c…`,
+   `frames_p12.json` `4e0fce06f07a044e…`, and the word rebuilt here has 41,288
+   operations, 18,908 roles, 1,760 reuse pairs, 3,960 gauges and **11,671 changed
+   operation frames**, the number its `expected-bit.json` declares. That match is what
+   ties this note's word to PR217's.
+3. The tree handed to `ceiling.py --tree` carries PR217's emitted `graph_p12.json`,
+   `word_p12.json` and `frames_p12.json`; PR202's unchanged `kchron_p12.json`,
+   `sinks.json`, `profile_p12.json` and vendored PR168-v4 checker; and a certificate
+   shim whose `bit` record is PR217's own post-terminal profile plus the pre-sink profile
+   restored by the terminal record (`{3, 21} x -102`, i.e. +34 children of width 3 and
+   +34 of width 21 and +34 roles, giving `W = 20,668` and rank mass 1,486,160).
+4. `ceiling.py` accepted the tree and passed every self-test on this word: the certified
+   layout lies inside every derived joint frame bound, **its slot chains reproduce
+   PR217's certified ledger child for child** (`W 20,668`, rank 1,486,160) and the
+   terminal substitution is exactly `[3, 21]` per sink and stage, with every per-slot
+   floor below the actual chain cost. Fact 1 reproduces the certificate to eight digits
+   (`6.7938424e-4 < D/C = 6.7998615e-4`, gap 0.089%).
+
+## Reproduce
+
+`rebuild_word.py` rebuilds the emitted payloads and checks them against PR217's
+`expected-bit.json`; `build_tree.py` assembles the `--tree` input; then run PR212's
+script unmodified:
+
+```sh
+python3 rebuild_word.py --package <pr217 checkout>/research/butterfly-coordinated-bit-211 \
+                        --bit-root <pr202 checkout> --out <fresh dir>
+python3 build_tree.py   --emit <fresh dir> --bit-root <pr202 checkout> \
+                        --package <pr217 checkout>/research/butterfly-coordinated-bit-211 \
+                        --tree <fresh tree>
+python3 <pr212 checkout>/research/bit-frame-ceiling-200/ceiling.py --tree <fresh tree> --rounds 1200
+```
+
+`transfer_check.py` compares PR217's emitted word with PR216's, from the same emit
+directory:
+
+```sh
+python3 transfer_check.py --emit <fresh dir> \
+    --pr216-package <pr216 checkout>/research/coordinated-crossover-pr200-v2
+```
+
+## Scope and limits
+
+* This bounds **frame layouts of PR217's word only**, and only in the sense PR212's floor
+  has: joint per-operation frame bounds, an exact per-slot dynamic programme and
+  Lagrangian coupling between the two registers of an operation, with the certified
+  layout as its own control.
+* The butterflies themselves are part of the word. Nothing here bounds further
+  reassociation, other circuits, other reuse, gauges or sinks -- the escapes PR212 lists.
+* The pre-sink profile in the shim is reconstructed, not read from PR217's own receipt:
+  the delta is exact arithmetic and `ceiling.py`'s child-for-child self-test is the
+  check, but a run of PR217's own `run_bit.py` would settle it directly. That run needs
+  Linux (`resource` limits and `signal.alarm`) and the pinned numpy 2.3.5 / scipy 1.17.0;
+  this machine has numpy 2.4.4 / scipy 1.18.1 and Windows, so the emission was driven
+  through the package's own `candidate.py` instead, which is the stage that produces the
+  word.
+* All κ remain conditional on the interfaces each submission states.
+
+## Credits
+
+PR212 (Chafik Boukhalfa) supplies the method and the script, unmodified; PR217 (hcg890)
+supplies the word, its overlay and `candidate.py`; PR211 (Rohan Arun) supplies the pinned
+cascade frames; PR200/PR168 v4 (Chafik Boukhalfa, eumemic, icekylinx) supply the base
+word and the vendored checker; PR216 (Dugongue) supplies the cut package whose admitted
+frames are compared above -- that comparison reads PR216's own witness files, its native
+admission was not run here. This note adds only the run, the shim, the comparison and the
+reading.
+Prepared with Anthropic Claude assistance; Apache-2.0.

research/butterfly-word-frame-floor/build_tree.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+"""Assemble a PR212-shaped `--tree` input for PR217's changed bit word.
+
+The tree is a faithful shim: the word, frames and graph are PR217's emitted payloads
+(their hashes were checked against the package's `expected-bit.json` by
+`rebuild_word.py`), the partner chronology, sinks, base profile and the vendored
+PR168-v4 checker come from the pinned PR202 checkout, and `certificate.json['bit']`
+carries PR217's own post-terminal profile plus the pre-sink profile restored by the
+terminal record (`{3, 21} x -102`, exactly as PR200's certificate records it, i.e. +34
+children of width 3 and +34 of width 21 and +34 roles).
+
+Every field of that shim is re-derived by `ceiling.py`'s own self-tests: it rebuilds the
+children multiset from the word and requires it to equal the shim's pre-sink histogram,
+requires the sink record to be one width-3 and one width-21 child per sink and stage, and
+requires `2v + roles - pairs` and the rank mass to match.  A wrong shim fails loudly
+instead of quietly lowering the floor.
+
+    python3 build_tree.py --emit <dir from rebuild_word.py> --bit-root <pr202 checkout> \
+                          --package <checkout>/research/butterfly-coordinated-bit-211 \
+                          --tree <fresh tree dir>
+"""
+import argparse
+import gzip
+import json
+import shutil
+import sys
+from pathlib import Path
+
+PKG_REL = Path('research/paired-cube-diagonal-bit-168')
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument('--emit', type=Path, required=True, help='directory written by rebuild_word.py')
+    ap.add_argument('--bit-root', type=Path, required=True, help='the #202 checkout used for the rebuild')
+    ap.add_argument('--package', type=Path, required=True, help='the butterfly package directory')
+    ap.add_argument('--tree', type=Path, required=True, help='fresh tree directory to create')
+    args = ap.parse_args()
+    sys.set_int_max_str_digits(0)
+    pr202 = (args.bit_root.resolve() / PKG_REL)
+    package = args.package.resolve()
+    tree = args.tree.resolve() / PKG_REL
+    bit = tree / 'selected' / 'bit'
+    if tree.exists():
+        shutil.rmtree(tree)
+    bit.mkdir(parents=True)
+    shutil.copytree(pr202 / 'references', tree / 'references')
+    for name in ('kchron_p12.json', 'sinks.json', 'profile_p12.json'):
+        shutil.copy2(pr202 / 'selected' / 'bit' / name, bit / name)
+    for name in ('graph_p12.json', 'word_p12.json', 'frames_p12.json'):
+        payload = (args.emit.resolve() / name).read_bytes()
+        (bit / name).write_bytes(payload)
+        (bit / (name + '.gz')).write_bytes(gzip.compress(payload, compresslevel=9, mtime=0))
+    post = json.loads((package / 'expected-bit.json').read_text())['profile']
+    pre = dict(post)
+    pre['child_histogram'] = dict(post['child_histogram'])
+    pre['child_histogram']['3'] = pre['child_histogram'].get('3', 0) + 102
+    pre['child_histogram']['21'] = pre['child_histogram'].get('21', 0) + 102
+    pre['rank_per_vertex'] = sum(int(r) * n for r, n in pre['child_histogram'].items())
+    pre['W_per_vertex'] = post['W_per_vertex'] + 34
+    pre['physical_target_histogram'] = dict(post['physical_target_histogram'])
+    pre['physical_target_histogram']['21'] = pre['physical_target_histogram'].get('21', 0) + 34
+    cert = json.loads((package / 'certificate.json').read_text())
+    shim = {'bit': {
+        'overridden_profile': pre,
+        'profile': post,
+        'terminal': {'child_delta': {'3': -102, '21': -102}},
+        'coarse': {'coarse_saving': cert['bit_coarse']},
+        'kappa': cert['kappa'],
+        'packed_profile': cert['packed_profile'],
+        'shim_note': 'pre-sink profile restored from the post-terminal profile by the {3,21} x -102 sink record',
+    }}
+    (tree / 'certificate.json').write_text(json.dumps(shim, indent=1, sort_keys=True) + '\n')
+    print('tree:', tree)
+    print('pre-sink W %s rank %s deficit %s'
+          % (pre['W_per_vertex'], pre['rank_per_vertex'], pre['deficit_per_vertex']))
+    print('post-terminal W %s rank %s deficit %s'
+          % (post['W_per_vertex'], post['rank_per_vertex'], post['deficit_per_vertex']))
+
+
+if __name__ == '__main__':
+    main()

research/butterfly-word-frame-floor/rebuild_word.py

@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+"""Rebuild PR217's changed bit word through PR217's own constructor.
+
+Calls `candidate(bit_root)` from the butterfly package (the stage that produces the
+changed graph, word and frames), writes the three emitted payloads into --out, and
+checks them against the package's own `expected-bit.json` hashes.  Nothing in the
+butterfly package or the PR202 checkout is modified, and no network is used.
+
+    python3 rebuild_word.py --package <checkout>/research/butterfly-coordinated-bit-211 \
+                            --bit-root <pr202 checkout> --out <fresh dir>
+"""
+import argparse
+import hashlib
+import importlib.util
+import json
+import sys
+import time
+from pathlib import Path
+
+
+def load(name, path):
+    spec = importlib.util.spec_from_file_location(name, path)
+    mod = importlib.util.module_from_spec(spec)
+    sys.modules[name] = mod
+    spec.loader.exec_module(mod)
+    return mod
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument('--package', type=Path, required=True, help='the butterfly package directory')
+    ap.add_argument('--bit-root', type=Path, required=True, help="a checkout of #202's head 8d8d67b")
+    ap.add_argument('--out', type=Path, required=True, help='fresh directory for the emitted payloads')
+    args = ap.parse_args()
+    sys.set_int_max_str_digits(0)
+    start = time.monotonic()
+    pkg = args.package.resolve()
+    if str(pkg) not in sys.path:
+        sys.path.insert(0, str(pkg))
+    cand = load('bfly_candidate', pkg / 'candidate.py')
+    word, emitted, meta = cand.candidate(args.bit_root.resolve())
+    print('[%.1fs] candidate built' % (time.monotonic() - start), flush=True)
+    expected = json.loads((pkg / 'expected-bit.json').read_text())['emitted_sha256']
+    got = {k: hashlib.sha256(v).hexdigest() for k, v in emitted.items()}
+    ok = True
+    for k in sorted(got):
+        same = got[k] == expected.get(k)
+        ok &= same
+        print('  %-18s %s  %s' % (k, got[k][:16], 'matches expected-bit.json' if same else 'DIFFERS'))
+    if not ok:
+        raise SystemExit('emitted payloads differ from the package\'s own expected-bit.json')
+    print('word: ops %d roles %d v %d h %d pairs %d gauges %d changed frames %d'
+          % (len(word.ops), word.R, word.v, word.h, len(word.w.get('pairs', [])),
+             len(word.w.get('gauges', [])), len(word.changed_frames)))
+    args.out.mkdir(parents=True, exist_ok=True)
+    for name, payload in emitted.items():
+        (args.out / name).write_bytes(payload)
+    (args.out / 'word-meta.json').write_text(json.dumps(
+        {'emitted_sha256': got, 'changed_operation_frames': len(word.changed_frames),
+         'source_head': meta.get('source_head') if isinstance(meta, dict) else None},
+        indent=1, sort_keys=True) + '\n')
+    print('wrote', args.out)
+
+
+if __name__ == '__main__':
+    main()