diff --git a/research/helper-handoff-r18/README.md b/research/helper-handoff-r18/README.md new file mode 100644 index 0000000000..07167aa089 --- /dev/null +++ b/research/helper-handoff-r18/README.md @@ -0,0 +1,177 @@ +# Successor hand-off of productive conduit helpers: a verified proof of concept (no κ claim) + +This package introduces a native operation pattern, the **successor hand-off**. It demonstrates the pattern exactly +on a scaled-down instance and prices what it is worth on the selected #5413 profile. It also shows exactly why +the current #4610 word admits no hand-off. **No κ is claimed.** The mechanism only gains inside a new native word, +and this package does not build one. + +> **Addendum (follow-up, `addendum-r19-neutrality/`): a local hand-off is gain-neutral once re-flagging is priced.** +> Between two anchors a register's paid rank mass is fixed. Routing any register through a mid frame F therefore costs +> g(p,q) = f(p) + f(q) − f(p+q), the same concave function as the hand-off gain. +> - **Screen (exact rows):** all 408 helpers of the #4610 word, every event and every mid frame, both orientations, +> about 1.88M points per orientation, with holder and cancellation lifts priced. **0 net-positive.** Wherever all +> holders can be lifted, the carried data contains port inputs held only by registers on dim 5–8 frames not +> contained in F, which climbs can never bring back. +> - **Designed F2^9 conduit blocks (exact replay):** a hand-off is **exactly neutral** (dM = 0) when the helper starts +> at its load frame. With a bottom holder, a later entrance with no hand-off dominates by at least 0.004192. +> - **Consequence:** the 0.0035309 ceiling below counts the hand-off gain without the re-flagging cost, so it is +> **not** an achievable estimate. A gain needs a new *main* transform, whose port flags pass through the helpers' +> mid frames with the carried data unchanged; a local helper block is not enough. +> +> `addendum-r20-p19-repricing/` separately prices the p19 line of #5270/#5440 with the reviewed costs charged. The +> corrected complex saving is 0: the 4,256 missing letter-Q rank-1 jobs per unit are about 3.5× the 1,202 tolerance, +> and merging cannot rescue it (≥ 4,179). This agrees with zxv's and Matarener's numbers and adds the merge-robust +> bound, the idle-Q count and the literal middle-idle pricing. + +Labels used below: +- **checked**: an exact computation in this package, reproduced by `verify.sh`. +- **screen**: an exact computation whose interpretation covers only the stated rewrite family. +- **argued**: a written argument that is not machine-checked. + +## The mechanism + +In the productive-helper paradigm (#4359), a helper's arbitrary input that climbs from dim d0 to dim d1 earns one +rank-(d1−d0) target. By concavity of x^(1−C), splitting that one target into a + b at the **same** paid climbs is +a strict gain: f(a) + f(b) − f(a+b) > 0. R16 (#5469) looked only for free *pure* split points and found none. + +The successor hand-off produces the split constructively, with a second helper: + +1. A successor helper `s` (a fresh arbitrary input) **starts at the mid frame F** and stays idle there. +2. **Co-framed cancellation.** At time t, conduit helper `h` and the port data D it carries meet at F, with D + unchanged. Free ADDs at F make `h` pure (`h −= x`) and load `s` (`s += x`). `h` retires at F. +3. **Renaming.** Every later operation on `h` acts on `s` instead. +4. **Successor pre-subtraction.** Each consumer above F used to pre-subtract `h_in` and later receive + `h_in + D` through the conduit. It now pre-subtracts `s_in` during its own early pass through F, while `s` is + still idle and pure. + +Paid climbs are unchanged and the target mass is unchanged. The single target d1−d0 becomes (dim F − d0) + (d1 − dim F). + +**Hand-off condition (argued in general, checked on the gadget).** Let τ swap the inputs `h_in` and `s_in`. +Suppose free scalar edits up to time t bring every register to τ(its original row), with two exceptions: `h` holds +exactly `h_in`, and `s` holds τ(original row of h). Then every later operation is linear and frame-identical. So +the final state is τ(original final state): `s` ends holding `s_in` at h's old final frame, `h` ends holding `h_in` +at F, and every other register is unchanged. + +Prior art, stated honestly: +- R16 split screen (#5469): pure point and no other holder. Here the condition is generalised to co-framed + cancellation, deferred cancellation and pre-subtraction. +- #4441 (helper retiming) and #4470 (entrance/exit lattice). A hand-off at a pure point with no outside holder is + only a later helper entrance. +- #3039 (block-climb obstruction, older cost alphabet). +- #5462 (relations; a hand-off that is not free leaves coupled outputs). +- "Relay" in the P104 packages means value delivery paths. That is unrelated. + +We found no PR that hands a conduit to a successor helper that starts mid-flag. + +## 1. Constructive proof of concept: `gadget.py` (checked) + +The instance lives on F₂⁵ and uses the conduit motif of #4610 helper threads (compare h300, which reads x19 at +dim 1, carries it on another flag, delivers it, and uncomputes at dim 9). Its registers: +- port x: frames A → Xb → F → U; +- consumers m1 (at B) and c (at G); +- helper h; +- for the hand-off, successor s, which starts at F. + +`nreplay.py` is the interpreter. It enforces exactly the rules of `native_extensions.literal_replay`, generalised +to any register count. `control_4610.py` checks it against the original: it reproduces `literal_replay` on the +#4610 word, with identical rows on all 632 registers plus C0, both orientations, and profile +{1:2201, 2:412, 3:510, 4:127, 5:40, 8:8}. + +| word | exact on all arbitrary columns | paid profile | helper climbs | productive targets | +|---|---|---|---|---| +| baseline (conduit uncomputed at the top) | yes (4 columns) | {1:12, 2:2, 3:1} | h: 5 | [5] | +| successor hand-off at F | yes (5 columns) | {1:12, 2:2, 3:1} | h: 3, s: 2 | [3, 2] | + +Three mutation controls are each rejected: +- no successor pre-subtraction (wrong endpoint row); +- no co-framed cancellation (wrong endpoint row); +- cancellation attempted before x reaches F (frame violation). + +**Gain per unit.** One hand-off adds dM = f(a) + f(b) − f(a+b) at the root. On the selected #5413 profile +(`price_handoff.py`, Decimal at 50 digits; raw roots, not the strict compiler): + +| hand-off | dM each | raw root, one per local in all 18 locals | +|---|---:|---:| +| 9 → 4+5 | 0.018612 | 0.0030286465 (+1.36e-6) | +| 9 → 3+6 | 0.017248 | 0.0030285469 | +| 9 → 1+8 | 0.009464 | 0.0030279787 | +| a single hand-off anywhere | ≥ 0.0084 | ≥ 0.0030273222 (> U = 0.00302728825) | + +The ceiling, with every rank-9 helper handed off once at 4+5, is 0.0035309 (gain-only; see the addendum: re-flagging costs cancel it locally). + +The table shows that a **single** legal hand-off would raise the raw root by about 3.3×10⁻⁸. That is far more +than U's 6×10⁻¹⁴ gap and than the roughly 10⁻¹¹ that the strict compiler reserves below the raw root. This is +argued, not funded. + +**Scope of the gadget.** The conduit above F is needed in the gadget only because consumer c must leave F before x +arrives there. That ordering stands in for the dependency structure of a real word. If c could wait at F, reading +x directly would dominate. + +## 2. The #4610 word admits no hand-off (`screens/`, exact screens) + +Base word: `research/kappa-round3-warp-20261022/ACTUAL-WORD.json.gz`. Rows are replayed exactly over ℚ in the free +group ring of warp words. The pool covers helpers at an intermediate frame (start dim < dim F) with climbs ahead. + +| screen | forward | backward | +|---|---|---| +| necessary span condition h_in ∈ span(rows at F) (`spanb.py`) | 172 helpers (168 at pure points just after the first climb) | 65 helpers (all pure points) | +| co-framed cancellation with all h_in holders at F (`relay.py`) | 0 | 0 | +| pure point + deferred cancellation by the rest of the program (`relay2.py`, 257 / 455 tests) | 0 | 0 | +| pure candidates whose outside h_in holders never visit F, so pre-subtraction is impossible (`presub.py`) | 165 of 168 (3 partial) | 65 of 65 | +| fewest foreign input variables left in h after co-framed cancellation, where (a) holds (`relay_def.py`) | 6 | 2 (67 helpers at dim 3) | + +**Reading.** The only points where a hand-off is linear-algebraically possible come right after a helper's first +climb, before its first read. At every such point, the helper's input has already been spread at the entrance into +consumers, and those consumers recollect it only at the top frame. Away from those points, the carried port data +never shares a frame with a copy of itself. The helper flags and the port flags cross only at dim 1 and dim 9. +So the existing word gives no hand-off. This is consistent with the conclusion of #5465 and #5469 that fixed +operation sets are exhausted. + +## 3. The other candidate mechanisms (argued) + +- **A. Field packing / mixed recursion sizes.** #4359's balance Σ p_r r^(1−C) counts paid arrays. A rank-r job on + a pack of k fields is k standard rank-r jobs, so packing changes nothing. Transforming the pack axis would turn + free scalar mixing into paid rank. In a volume-weighted invoice, the rank of a job grows only by merging + consecutive climbs of the *same* data, which is lever B. A multi-size recursion is admissible only in that + volume-weighted form, and it brings no gain. +- **B. Pre-gathered helper reads.** In the native grammar, data changes frame only through paid climbs or + registered warps. Staging in owned C0 costs paid centre transports with zero credit; #3039 is the analogue in + the older alphabet. Defragmentation at a fixed operation set is closed (#5426, #5465). B therefore needs new + flags in which helpers and port lines cross at mid frames. That is the same requirement as the hand-off, which + converts such a crossing into a split target. +- **C. Product-output helpers.** The successor hand-off **is** a concrete form of C, with the 9 → a+b split + realised by two independent helpers and no relation decoding. + +## What remains to scale it to a full supplier + +1. Build a new native local word in which conduit helpers meet the port lines they carry at a mid frame, with the + data unchanged (flag crossings). Consumers above that frame must pass it before the hand-off. +2. Re-run the full admission chain: + - all-632-column replay in both interpreters; + - Clifford admission of any new G maps; + - helper boundaries and the controller; + - the owned relations of #5383, which are pinned to helper rows and must be re-derived; + - the #5402/#5413 caller; + - the strict compiler and the 54 outer predicates. +3. Hand-offs must not add paid work. Any extra rank-1 job costs about 1, which outweighs the ≤ 0.0186 a hand-off + earns. + +## Reproduce + +```sh +sh research/helper-handoff-r18/verify.sh # PASS_HELPER_HANDOFF_R18 (~60 s, python3.12, stdlib only) +``` + +Expected outputs (`*.txt`) are stored next to each script. + +**Credits.** +- Cao Yifan (#4610 word and native semantics). +- zxv (#4359 productive-helper paradigm; native interpreter `literal_replay`). +- Clear1nsight (#5413 caller and profile). +- #5409 (ceiling U). +- Matarener (#5422/#5426/#5433). +- aliromman (#5383 relations). +- #4441/#4470 (helper retiming and lattice). +- Aaron Wang (#3039). + +AI disclosure: Claude Opus 5.5 (Anthropic) via Claude Code, directed by Joel Pulikkan (GamingPuzzled). Apache-2.0. diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/PROGRESS.md b/research/helper-handoff-r18/addendum-r19-neutrality/PROGRESS.md new file mode 100644 index 0000000000..750bb4aae9 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/PROGRESS.md @@ -0,0 +1,20 @@ +# R19 checkpoint log (hand-off-enabled native block) +## M0 setup +- worktree ~/imb-r19 branch r19-handoff-block (from r18-helper-handoff ce96fe974). Work dir ~/imb-harness/r19. +- Key accounting observation: per register, paid rank mass of climbs between two anchors is fixed (dim N - dim P); + re-flagging a register through a mid frame F only splits a jump: cost g(a,b)=f(a)+f(b)-f(a+b), the SAME + concave function as the hand-off gain. g(1,1)=1.39C, g(1,2)=1.91C, g(1,8)=3.14C, g(4,5)=6.18C (C~0.003). + => a hand-off nets positive iff the forced lifts' g-sum < hand-off g. Plan: priced-lift relay screen on + #4610 word (generalises R18 relay/presub), then construct+replay any positive candidate exactly. +## M1 priced-lift screen on #4610 word (tools/lift.py, exact rows; float pricing) +- Every helper (224..631), every event, every mid frame F on its flag; holders of h_in not at F must be LIFTED + (re-chosen climb chain P->F->N between anchors, cost g); then (h_in,()) in span(R_F + lifted) with extra lifts. +- maxmand=6 run: 0 net-positive both orientations (lift_forward/backward.txt). All points that pass holders fail span. +- DIAG anatomy (diag_*.txt, anatomy.py): residual of D = port inputs (e.g. L188,L193 / L3,L8,L95,L370..) held only by + registers whose current frames are dim 5-8 and NOT contained in F (P<=F False): data has moved up incomparable flags. + Unliftable for any budget in the lift family (forward 69/69, backward 174/174 (helper,dF) pairs). +## M2 designed blocks (tools/block.py, nreplay exact, F2^9 flag frames, 56 params x 2 families) +- plain conduit (h starts at its load frame): HAND - BASE = 0 exactly (x lift costs g(b-a,9-b) = gain). +- bottom-holder conduit (h starts at 0, presub y): HAND beats BASE by up to +0.002907, but LATER ENTRANCE (no hand-off) + dominates: best hand-off word - best non-hand-off word <= -0.004192 on all 56. +- rerun full screen without holder cap (liftall_*.txt) in progress. diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/anatomy.py b/research/helper-handoff-r18/addendum-r19-neutrality/anatomy.py new file mode 100644 index 0000000000..9deefaa60d --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/anatomy.py @@ -0,0 +1,32 @@ +"""R19: anatomy of the unliftable residual at a hand-off point (side, helper, event).""" +import sys +from lift import * +side=sys.argv[1]; H=('L',int(sys.argv[2])); J=int(sys.argv[3]); back=side=='backward'; prog=w['programs'][side] +anch,clb=timelines(prog) +rows={('L',i):v.identity(('L',i)) for i in range(632)}; rows['C',0]=v.identity(('C',0)) +cur={('L',i):fm for i,fm in enumerate(c['start'])}; cur['C',0]=None +for j,e in enumerate(prog[:J+1]): + k=e[0] + if k=='scalar': + gg,frame=e[1:]; t=tuple(gg[1]) + if gg[0]=='clear': rows[t]={}; cur[t]=frame + elif gg[0]=='copy': rows[t]=dict(rows[tuple(gg[2])]) + elif gg[0]=='add': v.add(rows[t],rows[tuple(gg[2])],Q(gg[3],gg[4])) + else: q=Q(gg[2],gg[3]); rows[t]={a:q*x for a,x in rows[t].items()} + elif k=='warp': + _,r,op,of,nf=e; t=tuple(r); rows[t]=v.warp(rows[t],op,back); cur[t]=nf + elif k=='climb': cur[('L',e[2])]=e[4] + elif k=='center': cur[('C',e[1])]=e[3] +F=cur[H]; print('h',H,'at frame',F,fr[F],'dim',dim(F)) +atF=[r for r in cur if cur[r]==F]; print('R_F',atF) +piv=[] +for r in atF: addpiv(rows[r],piv) +res=reduce({(H,()):Q(1)},piv) +print('residual', {(str(a),str(b)):str(x) for (a,b),x in res.items()}) +for (var,wd) in res: + hold=[r for r in rows if any(vv==var for (vv,ww) in rows[r])] + print(' var',var,'word',wd,'holders',len(hold)) + for r in hold[:8]: + A=anch[r]; idx=[a[0] for a in A]; p=bisect.bisect_right(idx,J) + P=A[p-1][2] if p else c['start'][r[1]]; N=A[p][1] if pU directly (no pass-through visits anywhere) + HAND successor hand-off at F = B: x lifted A->B->U, h retires at B, s starts at B + LATE (only with y) later entrance: h starts at A, y pre-subtracts at A (y lifted 0->A->G) + LATE+HAND hand-off at B on LATE +Each word is replayed exactly on all arbitrary-input columns (identity/restoration semantics: x, h, s, y +restored; m = m_in + x_in; c = c_in + x_in; y = y_in + x_in), and dM = sum f(targets) - sum f(paid).""" +import sys, itertools +from decimal import Decimal as D, getcontext +from fractions import Fraction as Q +from pathlib import Path +getcontext().prec = 50 +sys.path.insert(0, str(Path.home()/'imb-r19/research/helper-handoff-r18')) +from nreplay import replay, v, need +CROOT = D('0.0030272882499395587812724555013848588705741016191552') # selected #5413 raw root +def f(x): return D(0) if x == 0 else (D(x).ln() * (1 - CROOT)).exp() + +FR = [[1 << i for i in range(k)] for k in range(10)] # flag H_0 < H_1 < ... < H_9 (frame index = dim) +X, M, CC, H, Y, S = range(6) +def climb(i, a, b): return ('climb', 'L', i, a, b, b - a) +def add(t, s, num, fr): return ('scalar', ['add', ['L', t], ['L', s], num, 1], fr) +CLR = ('scalar', ['clear', ['C', 0]], 0) + +def word(a, b, gg, holder, late, hand): + U = 9; hs = a if (late or not holder) else 0 + st = {X: a, M: b, CC: gg, H: hs, Y: 0, S: b}; fi = {X: U, M: U, CC: U, H: U, Y: U, S: U} + p = [CLR] + if holder: + if late: p += [climb(Y, 0, a), add(Y, H, -1, a)] + else: p += [add(Y, H, -1, 0), climb(H, 0, a)] + p += [add(H, X, 1, a), climb(H, a, b), add(M, H, 1, b)] + yfrom = a if late else 0 + if hand: + p += [climb(X, a, b), add(H, X, -1, b), add(M, H, -1, b)] # h pure; m recollects h_in + if holder: p += [climb(Y, yfrom, b), add(Y, H, 1, b), add(Y, S, -1, b)] # y: h_in -> s_in + p += [add(S, X, 1, b), climb(S, b, gg), add(CC, S, 1, gg)] + if holder: p += [climb(Y, b, gg), add(Y, S, 1, gg)] + p += [climb(S, gg, U), climb(X, b, U), add(S, X, -1, U), climb(CC, gg, U), add(CC, S, -1, U), + climb(M, b, U)] + fi[H] = b; helpers = [H, S] + else: + p += [climb(H, b, gg), add(CC, H, 1, gg)] + if holder: p += [climb(Y, yfrom, gg), add(Y, H, 1, gg)] + p += [climb(H, gg, U), climb(X, a, U), add(H, X, -1, U), climb(M, b, U), add(M, H, -1, U), + climb(CC, gg, U), add(CC, H, -1, U)] + helpers = [H] + if holder: p += [climb(Y, gg, U)] + regs = [X, M, CC, H] + ([Y] if holder else []) + ([S] if hand else []) + n = max(regs) + 1 + start = [st[r] if r in regs else 0 for r in range(n)]; final = [fi[r] if r in regs else 0 for r in range(n)] + return n, regs, start, final, p, helpers + +def check(a, b, gg, holder, late, hand): + n, regs, start, final, prog, helpers = word(a, b, gg, holder, late, hand) + rows, prof, climbs = replay(n, FR, start, final, prog) + for r in regs: + want = v.identity(('L', r)) + if r in (M, CC, Y): v.add(want, v.identity(('L', X)), Q(1)) + need(rows['L', r] == want, 'endpoint row of register %d' % r) + targets = sorted(final[i] - start[i] for i in helpers) + dM = sum(f(k) for k in targets) - sum(cnt * f(k) for k, cnt in prof.items()) + return prof, targets, dM + +if __name__ == '__main__': + print('flag frames H_k = ; f(x)=x^(1-C) at the selected raw root C=%s' % str(CROOT)[:14]) + best = {} + for holder in (False, True): + for a, b, gg in itertools.product(range(1, 8), range(2, 9), range(3, 9)): + if not a < b < gg < 9: continue + res = {} + for late, hand in [(False, False), (False, True)] + ([(True, False), (True, True)] if holder else []): + res[(late, hand)] = check(a, b, gg, holder, late, hand) + base = res[(False, False)][2] + bestbase = max(res[k][2] for k in res if not k[1]); besthand = max(res[k][2] for k in res if k[1]) + key = 'holder' if holder else 'plain' + best.setdefault(key, []).append((float(res[(False, True)][2] - base), float(besthand - bestbase), a, b, gg)) + if (a, b, gg) in [(1, 4, 6), (3, 4, 5), (2, 5, 7)]: + for k, (prof, t, dm) in res.items(): + print(f'{key:6s} a={a} F=B={b} G={gg} {"LATE" if k[0] else "BASE"}{"+HAND" if k[1] else " "} ' + f'exact rows OK; paid {prof}; helper targets {t}; block dM {float(dm):+.6f}') + for key, rows in best.items(): + rows.sort(reverse=True) + mx = max(rows, key=lambda r: r[1]) + print(f'{key}: {len(rows)} parameter sets; max dM(HAND - BASE) = {rows[0][0]:+.6f} at a,b,G={rows[0][2:]}; ' + f'max dM(best hand-off word - best non-hand-off word) = {mx[1]:+.6f} at a,b,G={mx[2:]}') + print('PASS_BLOCK_R19') diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/diag_backward.txt b/research/helper-handoff-r18/addendum-r19-neutrality/diag_backward.txt new file mode 100644 index 0000000000..f7600a9ebc --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/diag_backward.txt @@ -0,0 +1,29 @@ +backward points 1875133 {'points': 1875133, 'span check needs extra': 315, 'span fails within budget': 315, 'too many holders': 937468, 'holder not liftable': 881828, 'nontrivial word': 2392} mandatory-lift histogram {0: 52322, 1: 154098, 2: 173659, 3: 125975, 4: 134129, 5: 106379, 6: 191103} +backward DIAG per (helper,dF): min total lift cost to complete the span (99 = unliftable residual) + h273 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 6 extra 0 event 12478 + h275 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 6 extra 0 event 12489 + h289 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 6 extra 0 event 12532 + h291 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 6 extra 0 event 12543 + h305 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 6 extra 0 event 12588 + h321 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 6 extra 0 event 12640 + h586 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 2 event 13418 + h587 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 2 event 13420 + h588 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13422 + h589 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13426 + h590 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 2 event 13454 + h592 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13471 + h591 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 1 event 13456 + h593 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13462 + h594 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 3 event 13492 + h595 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 1 event 13496 + h596 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13495 + h597 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13501 + h602 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 2 event 13569 + h603 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 2 event 13571 + h604 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13573 + h605 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13577 + h629 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 1 event 13768 + h630 dF=4 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 2 extra 0 event 13770 + h311 dF=3 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 7 extra 0 event 9335 +backward DIAG count 174 unliftable 174 +backward helpers with a net-positive priced hand-off: 0 {} diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/diag_forward.txt b/research/helper-handoff-r18/addendum-r19-neutrality/diag_forward.txt new file mode 100644 index 0000000000..b95b4d8101 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/diag_forward.txt @@ -0,0 +1,29 @@ +forward points 1875116 {'points': 1875116, 'too many holders': 1242338, 'holder not liftable': 480365, 'span check needs extra': 73, 'span fails within budget': 73, 'nontrivial word': 106} mandatory-lift histogram {0: 151671, 1: 215973, 2: 88694, 3: 57410, 4: 73824, 5: 14228, 6: 30978} +forward DIAG per (helper,dF): min total lift cost to complete the span (99 = unliftable residual) + h544 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 7 extra 0 event 13355 + h545 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 8 extra 0 event 13358 + h556 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 7 extra 0 event 13361 + h557 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 8 extra 0 event 13364 + h546 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 8 extra 0 event 13379 + h547 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 7 extra 0 event 13382 + h558 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 8 extra 0 event 13385 + h559 dF=5 gain 0.01861 holder-cost 0.00000 total 99.00000 net -98.98139 residual-vars 7 extra 0 event 13388 + h507 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 10 extra 0 event 13157 + h508 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 10 extra 0 event 13160 + h496 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 8 extra 0 event 13181 + h497 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 9 extra 0 event 13184 + h474 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 7 extra 0 event 13199 + h475 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 8 extra 0 event 13202 + h486 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 9 extra 0 event 13205 + h487 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 10 extra 0 event 13208 + h484 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 8 extra 0 event 13223 + h485 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 8 extra 0 event 13226 + h476 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 8 extra 0 event 13229 + h477 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 7 extra 0 event 13232 + h516 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 8 extra 0 event 13295 + h517 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 8 extra 0 event 13298 + h530 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 7 extra 0 event 13301 + h531 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 6 extra 0 event 13304 + h566 dF=6 gain 0.01725 holder-cost 0.00000 total 99.00000 net -98.98275 residual-vars 13 extra 1 event 13307 +forward DIAG count 69 unliftable 69 +forward helpers with a net-positive priced hand-off: 0 {} diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/lift.py b/research/helper-handoff-r18/addendum-r19-neutrality/lift.py new file mode 100644 index 0000000000..49363a29c6 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/lift.py @@ -0,0 +1,189 @@ +"""R19 priced-lift relay screen on the #4610 word. + +Generalises R18 relay.py: a hand-off of helper h at frame F (time j) needs every register holding h_in at F +and (h_in,()) in the Q-span of rows at F. R18 required this with the EXISTING frames (0 hits). Here a +register r that is not at F may be LIFTED: between r's previous anchor (scalar op / warp, frame P) and its +next anchor (frame N) r has only climbs, which can be re-timed and re-chosen; if span(P) <= F <= span(N) +the chain becomes P -> F -> N. Its paid rank mass is unchanged; cost = f-sum(new chain) - f-sum(old chain) +(0 when the old chain already visits F). Net = g(dF-d0, d1-dF) - sum lift costs, f(x) = x^(1-C). +Usage: lift.py forward|backward [maxmand]""" +import sys, gzip, json, bisect, math +from collections import Counter, defaultdict +from fractions import Fraction as Q +from pathlib import Path +R = Path.home()/'imb-r19' +sys.path.insert(0, str(R/'research/kappa-causal-continuation/buffer-borrowing')) +import verify_buffers as v +w = json.loads(gzip.decompress((R/'research/kappa-round3-warp-20261022/ACTUAL-WORD.json.gz').read_bytes())) +c = w['canonical']; fr = c['frames']; SP = [frozenset(v.span(f)) if f else frozenset({0}) for f in fr] +C = 0.00302728825 +import os; DIAG = os.environ.get('DIAG') == '1' +def f(x): return x ** (1 - C) if x > 0 else 0.0 +def g(a, b): return f(a) + f(b) - f(a + b) +def dim(F): return len(fr[F]) + +def timelines(prog): + anch = defaultdict(list); clb = defaultdict(list) + for j, e in enumerate(prog): + k = e[0] + if k == 'scalar': + gg, frame = e[1:] + for r in gg[1:3]: + if isinstance(r, list): anch[tuple(r)].append((j, frame, frame)) + elif k == 'warp': + anch[tuple(e[1])].append((j, e[3], e[4])) + elif k == 'climb': + clb[('L', e[2])].append((j, e[3], e[4], e[5])) + elif k == 'center': + anch[('C', e[1])].append((j, e[2], e[3])) + return anch, clb + +def lift_cost(r, j, F, anch, clb): + """cost to have r at frame F right after event j (None if impossible).""" + if r[0] != 'L': return None + i = r[1]; A = anch[r]; idx = [a[0] for a in A] + p = bisect.bisect_right(idx, j) + P = A[p - 1][2] if p else c['start'][i] + N = A[p][1] if p < len(A) else c['final'][i] + lo = A[p - 1][0] if p else -1; hi = A[p][0] if p < len(A) else 10**9 + chain = [x for x in clb[r] if lo < x[0] < hi] + visited = {P} | {x[2] for x in chain} + if F in visited: return 0.0 + if not (SP[P] <= SP[F] <= SP[N]): return None + old = sum(f(x[3]) for x in chain) + return f(dim(F) - dim(P)) + f(dim(N) - dim(F)) - old + +def reduce(x, piv): + x = dict(x) + for k, rr in piv: + a = x.get(k) + if a: + for kk, vv in rr.items(): + nv = x.get(kk, Q(0)) - a * vv + if nv: x[kk] = nv + else: x.pop(kk, None) + return x + +def addpiv(vec, piv): + x = reduce(vec, piv) + if not x: return False + k = next(iter(x)); a = x[k]; x = {kk: vv / a for kk, vv in x.items()} + for k2, r2 in piv: + b = r2.get(k) + if b: + for kk, vv in x.items(): + nv = r2.get(kk, Q(0)) - b * vv + if nv: r2[kk] = nv + else: r2.pop(kk, None) + piv.append((k, x)); return True + +def screen(side, maxmand=4): + back = side == 'backward'; prog = w['programs'][side] + anch, clb = timelines(prog) + rows = {('L', i): v.identity(('L', i)) for i in range(632)}; rows['C', 0] = v.identity(('C', 0)) + cur = {('L', i): fm for i, fm in enumerate(c['start'])}; cur['C', 0] = None + inv = defaultdict(set) + for r, row in rows.items(): + for (var, wd) in row: inv[var].add(r) + atF = defaultdict(set) + for r, fm in cur.items(): atF[fm].add(r) + def setf(r, fm): atF[cur[r]].discard(r); cur[r] = fm; atF[fm].add(r) + climbs_left = Counter(e[2] for e in prog if e[0] == 'climb') + helpers = [('L', i) for i in range(224, 632)] + seen = set(); diag = {}; hver = Counter() + best = {} # (i, dF) -> (net, j, mandatory, extra) + mandhist = Counter(); stats = Counter() + for j, e in enumerate(prog): + k = e[0] + if k == 'scalar': + gg, frame = e[1:]; t = tuple(gg[1]); hver[t[1]] += 1 + old = {var for (var, wd) in rows[t]} + if gg[0] == 'clear': rows[t] = {}; setf(t, frame) + elif gg[0] == 'copy': rows[t] = dict(rows[tuple(gg[2])]) + elif gg[0] == 'add': v.add(rows[t], rows[tuple(gg[2])], Q(gg[3], gg[4])) + else: q = Q(gg[2], gg[3]); rows[t] = {a: q * x for a, x in rows[t].items()} + new = {var for (var, wd) in rows[t]} + for var in old - new: inv[var].discard(t) + for var in new - old: inv[var].add(t) + elif k == 'warp': + _, r, op, of, nf = e; t = tuple(r); rows[t] = v.warp(rows[t], op, back); setf(t, nf) + elif k == 'climb': + setf(('L', e[2]), e[4]); climbs_left[e[2]] -= 1 + elif k == 'center': + setf(('C', e[1]), e[3]) + else: + continue + if j + 1 < len(prog) and prog[j + 1][0] == 'climb' and prog[j + 1][2] == e[2] if k == 'climb' else False: + pass + for h in helpers: + i = h[1] + if climbs_left[i] <= 0: continue + F = cur[h]; dF = dim(F); d0 = dim(c['start'][i]); d1 = dim(c['final'][i]) + if not d0 < dF: continue + gain = g(dF - d0, d1 - dF) + mand = inv[h] - atF[F] + stats['points'] += 1 + if len(mand) > maxmand: stats['too many holders'] += 1; continue + mandhist[min(len(mand), 10)] += 1 + cost = 0.0; ok = True + for r in mand: + lc = lift_cost(r, j, F, anch, clb) + if lc is None: ok = False; break + cost += lc + if cost >= gain: break + if not ok: stats['holder not liftable'] += 1; continue + if cost >= gain: stats['holder lifts exceed gain'] += 1; continue + if any(wd != () for r in inv[h] for (var, wd) in rows[r] if var == h): stats['nontrivial word'] += 1; continue + key = (i, dF) + if key in best and best[key][0] >= gain - cost: continue + dkey = (i, F, hver[i], frozenset(mand)) + if DIAG and dkey in seen: continue + seen.add(dkey) + cmand = cost + piv = [] + for r in atF[F] | mand: addpiv(rows[r], piv) + target = {(h, ()): Q(1)} + res = reduce(target, piv); extra = [] + if res: + stats['span check needs extra'] += 1 + vars_ = {var for (var, wd) in res} + cands = set() + for var in vars_: cands |= inv[var] + cands -= atF[F] | mand + priced = [] + for r in cands: + lc = lift_cost(r, j, F, anch, clb) + if lc is not None and (DIAG or cost + lc < gain): priced.append((lc, r)) + priced.sort() + for lc, r in priced: + if cost + lc >= gain and not DIAG: break + if addpiv(rows[r], piv): + extra.append(r); cost += lc + res = reduce(target, piv) + if not res: break + if DIAG: + nv = len({var for (var, wd) in res}) + dk = (i, dF); val = (cost if not res else 99.0, nv, len(extra), j, gain, cmand) + if dk not in diag or val < diag[dk]: diag[dk] = val + if res or cost >= gain: stats['span fails within budget'] += 1; continue + stats['NET POSITIVE'] += 1 + best[key] = (gain - cost, j, sorted(mand), extra, gain, cost) + print(side, 'points', stats['points'], dict(stats), 'mandatory-lift histogram', dict(sorted(mandhist.items())), flush=True) + if DIAG: + print(side, 'DIAG per (helper,dF): min total lift cost to complete the span (99 = unliftable residual)') + hist = Counter() + for (i, dF), (tc, nv, ne, j, gain, cm) in sorted(diag.items(), key=lambda kv: kv[1][0] - kv[1][4]): + hist[(dF, 'net<=-%.3f' % (round((tc - gain) * 1000) / 1000) if tc < 90 else 'residual-unliftable')] += 1 + best_rows = sorted(diag.items(), key=lambda kv: kv[1][0] - kv[1][4])[:25] + for (i, dF), (tc, nv, ne, j, gain, cm) in best_rows: + print(' h%d dF=%d gain %.5f holder-cost %.5f total %.5f net %.5f residual-vars %d extra %d event %d' % (i, dF, gain, cm, tc, gain - tc, nv, ne, j)) + print(side, 'DIAG count', len(diag), 'unliftable', sum(1 for x in diag.values() if x[0] > 90)) + tot = Counter((dim(c['start'][i]), dF, dim(c['final'][i])) for (i, dF) in best) + print(side, 'helpers with a net-positive priced hand-off:', len({i for i, _ in best}), dict(sorted(tot.items()))) + for key, val in sorted(best.items(), key=lambda kv: -kv[1][0])[:40]: + print(' ', key, 'net %.6f gain %.6f cost %.6f event %d mand %s extra %s' % (val[0], val[4], val[5], val[1], val[2], val[3])) + return best + +if __name__ == '__main__': + side = sys.argv[1]; mm = int(sys.argv[2]) if len(sys.argv) > 2 else 4 + screen(side, mm) diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/lift_backward.txt b/research/helper-handoff-r18/addendum-r19-neutrality/lift_backward.txt new file mode 100644 index 0000000000..134a441b83 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/lift_backward.txt @@ -0,0 +1,2 @@ +backward points 1875133 {'points': 1875133, 'span check needs extra': 53445, 'span fails within budget': 53445, 'too many holders': 937468, 'holder not liftable': 881828, 'nontrivial word': 2392} mandatory-lift histogram {0: 52322, 1: 154098, 2: 173659, 3: 125975, 4: 134129, 5: 106379, 6: 191103} +backward helpers with a net-positive priced hand-off: 0 {} diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/lift_forward.txt b/research/helper-handoff-r18/addendum-r19-neutrality/lift_forward.txt new file mode 100644 index 0000000000..c4a0fa44b6 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/lift_forward.txt @@ -0,0 +1,2 @@ +forward points 1875116 {'points': 1875116, 'too many holders': 1242338, 'holder not liftable': 480365, 'span check needs extra': 152307, 'span fails within budget': 152307, 'nontrivial word': 106} mandatory-lift histogram {0: 151671, 1: 215973, 2: 88694, 3: 57410, 4: 73824, 5: 14228, 6: 30978} +forward helpers with a net-positive priced hand-off: 0 {} diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/liftall_backward.txt b/research/helper-handoff-r18/addendum-r19-neutrality/liftall_backward.txt new file mode 100644 index 0000000000..f6ac861fd4 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/liftall_backward.txt @@ -0,0 +1,2 @@ +backward points 1875133 {'points': 1875133, 'span check needs extra': 53445, 'span fails within budget': 53445, 'holder not liftable': 1819296, 'nontrivial word': 2392} mandatory-lift histogram {0: 52322, 1: 154098, 2: 173659, 3: 125975, 4: 134129, 5: 106379, 6: 191103, 7: 149139, 8: 121351, 9: 79164, 10: 587814} +backward helpers with a net-positive priced hand-off: 0 {} diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/liftall_forward.txt b/research/helper-handoff-r18/addendum-r19-neutrality/liftall_forward.txt new file mode 100644 index 0000000000..c661b68171 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/liftall_forward.txt @@ -0,0 +1,2 @@ +forward points 1875116 {'points': 1875116, 'holder not liftable': 1722703, 'span check needs extra': 152307, 'span fails within budget': 152307, 'nontrivial word': 106} mandatory-lift histogram {0: 151671, 1: 215973, 2: 88694, 3: 57410, 4: 73824, 5: 14228, 6: 30978, 7: 15279, 8: 58026, 9: 44287, 10: 1124746} +forward helpers with a net-positive priced hand-off: 0 {} diff --git a/research/helper-handoff-r18/addendum-r19-neutrality/trace.py b/research/helper-handoff-r18/addendum-r19-neutrality/trace.py new file mode 100644 index 0000000000..879b96ef3e --- /dev/null +++ b/research/helper-handoff-r18/addendum-r19-neutrality/trace.py @@ -0,0 +1,21 @@ +"""trace which registers hold helper H's input over time (exact rows).""" +import sys +from lift import * +side=sys.argv[1]; H=('L',int(sys.argv[2])); back=side=='backward'; prog=w['programs'][side] +rows={('L',i):v.identity(('L',i)) for i in range(632)}; rows['C',0]=v.identity(('C',0)) +cur={('L',i):fm for i,fm in enumerate(c['start'])}; cur['C',0]=None +def hv(r): return {wd:x for (var,wd),x in rows[r].items() if var==H} +for j,e in enumerate(prog): + k=e[0] + if k=='scalar': + gg,frame=e[1:]; t=tuple(gg[1]); before=hv(t) + if gg[0]=='clear': rows[t]={} + elif gg[0]=='copy': rows[t]=dict(rows[tuple(gg[2])]) + elif gg[0]=='add': v.add(rows[t],rows[tuple(gg[2])],Q(gg[3],gg[4])) + else: q=Q(gg[2],gg[3]); rows[t]={a:q*x for a,x in rows[t].items()} + cur[t]=frame + if hv(t)!=before: print(j,gg,'dim',dim(frame),'->',{str(a):str(b) for a,b in hv(t).items()}) + elif k=='warp': + _,r,op,of,nf=e; t=tuple(r); rows[t]=v.warp(rows[t],op,back); cur[t]=nf + if hv(t): print(j,'WARP',t,'holds') + elif k=='climb': cur[('L',e[2])]=e[4] diff --git a/research/helper-handoff-r18/addendum-r20-p19-repricing/README.md b/research/helper-handoff-r18/addendum-r20-p19-repricing/README.md new file mode 100644 index 0000000000..aa916d52a7 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r20-p19-repricing/README.md @@ -0,0 +1,76 @@ +# R20: the p19 coded-reverse invoice (#5270 / #5440) with the reviewed costs charged + +**Result: the claim collapses. No positive saving remains.** The missing `Q` on the stage letters is enough on +its own to kill it. No κ claim is made here. + +Base: DaysSky's package `research/coded-reverse-word-p19` at #5440 head `c0f52b5b1`, worktree `~/imb-r20`. +(#5270 head `225a19a18` has the same `invoice()`.) Prices come from the package's own `invoice()` and +`listing()`, a 60-digit Decimal moment M(C) = Σ_T n k^(1−C) − Σ_P n r^(1−C), and the exact concave-order hinge +test from `scripts/compare_recursive_invoices.py`. + +## How much the original invoice can absorb +- Raw root: 0.007578535190. +- Rank-mass surplus: 2016 out of 265,533. +- Spare moment at C = 0.0034 (BIT): 1103.28. Spare moment at the selected frontier 0.003027288235: 1202.6683. At + our 0.0030272882489783 it is also 1202.6683. +- So the invoice can absorb at most **1202** extra rank-1 jobs and still beat the frontier. + +## Charges (all exact counts taken from the listing) +| variant | added cost | mass at C=0 | M(U_sel) | root | +|---|---|---:|---:|---:| +| V1: `Q` on every stage letter. The native call supplies `D`, but the listing applies `R = D·Q`. 19 LETTER gates × 2 roles × 112 owners | **+4256 rank-1** | −2240 | **−3053.33** | **0** (dead for every r^t, min hinge 2240) | +| V2: V1 plus R-type idles in the fresh-block geometry. Q^|S| on the central line for odd |S|. Charged most favourably as merges 8k→8k+1 at k ∈ {1,3,5,7,9}, 448 fields each | +2240 rank | −4480 | −5263.21 | 0 | +| V3: V2 plus the 18 middle idles, priced as the listing actually executes them (R on 224 fields, rank 9; the invoice prices them at 0) | +4032 × rank 9 | −40768 | −41310.6 | 0 | +| VB: letter `D` instead, with the Q-conjugated terminal gauge decoded by one paid `Q` per owner on the 19 coded banks; credit 8k instead of 1+8k | +2128 rank-1, −2128 credit | −2240 | −3023.90 | 0 | +| VB0: terminal decoder only, all credit kept (lower bound for the VB route) | +2128 rank-1 | −112 | −925.33 | 0 | + +**Merge-robust bound.** Suppose the 4256 `Q` axes are not paid as separate jobs but merged into existing +paid jobs. +- Adding one Gaussian axis to a job of rank r (with r+1 ≤ 153, the ambient rank) costs + (r+1)^(1−C) − r^(1−C) ≥ (1−C)·153^(−C). At C = U_sel this is ≥ 0.981905 H1-equivalent. +- So the 4256 `Q` axes cost ≥ 4179 against a budget of 1202.67. At most 1224 such axes fit. +- Merging therefore cannot rescue the invoice. +- Assumption: no existing paid job already acts Gaussianly on the central (query) axis. PHYSICAL.md supports + this: the native backward call is `diag(D,D)·upper(−QC)`, and its `D` is rank 8 on the stage block. + +**The charge that kills it: the missing letter `Q`** (#5440 reviews by Matarener, zxv and gvconly). +- It overshoots the budget by 4256 − 1202 = 3054 rank-1 jobs, i.e. 3.5× the available margin. +- Even the cheapest identified alternative overshoots: the terminal-gauge decoder alone (2128) is 1.8× the + budget. +- Compiler constants (jwr404) do not matter here. They only move κ below the raw root. #5440 already uses the + shape-correct h = 1/64 majorant, and it closes. +- The distinct-axis issue (zxv, #5349, gvconly) only adds more charges (V2/V3). It also explains why the + 9-axis replay missed them: every listed idle power (2, 4, …, 18) is even, and R^even is monomial on the same + nine axes. + +## What a rescue would need (not attempted) +Either of these would change the conclusion: +- a coded realization of `controller()` itself: letter `D`, fresh blocks, exact, with every `Q` charged; +- a gauge repair costing ≤ 1202 rank-1-equivalents per p=19 unit, i.e. ≤ 63 per thick stage against 224 + independent fields. + +R17's one-vector-per-paid-job lemma suggests that ≥ 224 jobs per stage are needed when the 224 fields are +formally independent. That puts the second option out of reach unless a new native call supplies `R` with no +additional rank mass. At C = 0 that is impossible: rank mass is additive, so mass = −2240 for any realization. + +## Reproduce +```sh +cd ~/imb-harness/r20/tools && OPENBLAS_NUM_THREADS=1 /opt/homebrew/bin/python3.12 -B corrected_invoice.py # ~8 s +``` +Outputs: `corrected_invoice.txt`, `corrected_invoice.json`. + +Credits: +- DaysSky: the p19 word, invoice and package (#5270, #5440). +- Reviewers: + - jwr404 / James Reed: compiler constants and the #5349 follow-up. + - #5349: standard() ≠ controller(). + - zxv: same nine axes, unpriced middle idles, the 4256 repair and the concave-order audit. + - Matarener / Mark: the Q-conjugated terminal gauge, letter R vs D, 4256, and the 1103/1202 tolerances. + - Clear1nsight / Chris R: the IDLE stage-list decoding bug and the native C_jj = 0 counterexample. + - gvconly / G. V. Chudnovsky: the independent simulator and the fresh-block coded replay failures. +- Lineage: PR4940 controller (Cayden Burkart / SethBurkart123), PR4929 producer (Dugongue), PR4933 code + (Dillon Amburgey), Cinna Davis / 1337isnot1337. + +This note re-derives their numbers independently from the package's own invoice and adds the merge-robust +bound and the idle-Q census. Prepared with Claude Opus 5.5 (Anthropic) via Claude Code, directed by Joel +Pulikkan (GamingPuzzled). diff --git a/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.json b/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.json new file mode 100644 index 0000000000..b31d40abbb --- /dev/null +++ b/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.json @@ -0,0 +1,83 @@ +{ + "V0_as_claimed": { + "root": "0.0075785351898977172184061023773293009528326601478040322023062", + "mass_surplus": 2016, + "T": { + "153": 224, + "9": 6642, + "17": 448, + "25": 448, + "33": 448, + "41": 448, + "49": 448, + "57": 448, + "65": 448, + "73": 448, + "4": 532, + "7": 133, + "8": 893 + }, + "P": { + "8": 448, + "16": 448, + "24": 448, + "32": 448, + "40": 448, + "48": 448, + "56": 448, + "64": 448, + "72": 448, + "1": 44059, + "2": 7828, + "3": 9690, + "4": 2413, + "5": 760 + } + }, + "V1 +4256 Q (separate rank-1 jobs)": { + "root": "0", + "mass_surplus": -2240, + "concave_order_dead": true, + "min_hinge": 2240, + "M_at_selected": "-3053.331700119581570060843295649075308127191248030812583270", + "M_at_ours": "-3053.331703850258500042997953072684935484850295420669406493" + }, + "V2 = V1 + odd-idle Q merged (8k->8k+1)": { + "root": "0", + "mass_surplus": -4480, + "concave_order_dead": true, + "min_hinge": 4480, + "M_at_selected": "-5263.210321592354228253883416951270358600908180336463428603", + "M_at_ours": "-5263.210325184867075590416286659810210562407145748287632670" + }, + "V3 = V2 + literal middle R idles (rank 9)": { + "root": "0", + "mass_surplus": -40768, + "concave_order_dead": true, + "min_hinge": 40768, + "M_at_selected": "-41310.636885483840433823028493019076101796883132005119969620", + "M_at_ours": "-41310.636887969211931961526283058141665945398867869323205668" + }, + "VB letter D + terminal Q decoder (2128), credit 8k": { + "root": "0", + "mass_surplus": -2240, + "concave_order_dead": true, + "min_hinge": 2240, + "M_at_selected": "-3023.896764301696821019069457353975252228084019162571473304", + "M_at_ours": "-3023.896767897383214116149818615021473204823286332391857749" + }, + "VB0 terminal decoder only, credit kept (2128 rank-1)": { + "root": "0", + "mass_surplus": -112, + "concave_order_dead": true, + "min_hinge": 112, + "M_at_selected": "-925.331700119581570060843295649075308127191248030812583270", + "M_at_ours": "-925.331703850258500042997953072684935484850295420669406493" + }, + "merge_robust": { + "per_q_lower": "0.981905246825783343043287321966235032879161316323358459605322", + "budget_at_selected": "1202.668299880418429939156704350924691872808751969187416730", + "max_affordable": 1224, + "needed": 4256 + } +} \ No newline at end of file diff --git a/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.py b/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.py new file mode 100644 index 0000000000..4558035d4a --- /dev/null +++ b/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.py @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 +"""R20: re-price DaysSky's p19 coded-reverse invoice (#5270 / #5440) with the reviewed costs charged. + +Reads the package at ~/imb-r20 (PR #5440 head c0f52b5b1). Uses the package's own `invoice()` +(p19_claim.py) and listing generator (p19_word.py) and prices with a 60-digit Decimal engine +(R16 price16.py style) plus the repository's exact concave-order comparator +(scripts/compare_recursive_invoices.py). Standard library only. + +Pricing: M(C) = sum_T n k^(1-C) - sum_P n r^(1-C); the complex supplier is the raw root C*. +Frontiers: selected 0.003027288235 (605457647/200000000000), ours 0.0030272882489783, BIT 0.0034. +""" +import importlib.util, json, sys +from collections import Counter +from decimal import Decimal as D, getcontext +from fractions import Fraction +from pathlib import Path + +getcontext().prec = 60 +PKG = Path.home()/'imb-r20/research/coded-reverse-word-p19' +REPO = Path.home()/'imb-r20' +U_SEL = D('0.003027288235') +U_OURS = D('0.0030272882489783') +BIT = D('0.0034') + + +def load(name, path): + spec = importlib.util.spec_from_file_location(name, path) + mod = importlib.util.module_from_spec(spec); spec.loader.exec_module(mod) + return mod + + +def pw(x, e): return (D(x).ln()*e).exp() + + +def M(T, P, C): + e = 1-D(C) + return sum(n*pw(k, e) for k, n in T.items() if n) - sum(n*pw(r, e) for r, n in P.items() if n) + + +def root(T, P): + """Largest-sign-change bisection on (0, 0.05); returns 0 if M <= 0 on a fine grid (no break-even).""" + grid = [D(i)/D(10000) for i in range(0, 500)] + vals = [M(T, P, c) for c in grid] + if all(v <= 0 for v in vals): return D(0) + i = max(j for j, v in enumerate(vals) if v > 0) + lo, hi = grid[i], grid[i]+D('0.0001') + for _ in range(120): + m = (lo+hi)/2 + if M(T, P, m) > 0: lo = m + else: hi = m + return lo + + +def concave_order_dead(T, P): + """Exact: K(k) = sum_r (P[r]-T[r]) min(r,k) > 0 at every knot => M(C) < 0 for all C in (0,1).""" + d = Counter(P); d.subtract(T) + knots = sorted(set(T) | set(P)) + K = [sum(n*min(r, k) for r, n in d.items()) for k in knots] + return all(x > 0 for x in K), min(K) + + +def tolerance(T, P, limit): + """Max extra rank-1 paid jobs keeping the raw root above `limit` (M(limit) / 1).""" + return int(M(T, P, limit) // 1) + + +def add(h, delta): + out = Counter(h) + for k, v in delta.items(): out[k] += v + return +out + + +def main(): + claim = load('p19claim', PKG/'p19_claim.py') + word = load('p19word', PKG/'p19_word.py') + p, pat = 19, 'I'*9 + 'T'*9 + thick = frozenset(range(1, 2*p, 2)) + T0, P0 = claim.invoice(p, pat, thick, claim.P_NEW) + out = {} + mass = lambda h: sum(k*n for k, n in h.items()) + r0 = root(T0, P0) + out['V0_as_claimed'] = dict(root=str(r0), mass_surplus=mass(T0)-mass(P0), T=dict(T0), P=dict(P0)) + print('V0 package invoice: root %.12f mass surplus %d of %d' % (r0, mass(T0)-mass(P0), mass(T0))) + for name, lim in (('BIT 0.0034', BIT), ('selected', U_SEL), ('ours', U_OURS)): + print(' tolerance (extra rank-1 jobs) above %-10s: %d [M = %s]' % (name, tolerance(T0, P0, lim), + str(M(T0, P0, lim))[:14])) + + # ---- charge 1: missing Q on every letter (native call supplies D, listing applies R = D Q) ---- + g = word.listing(p, 'R') + kinds = Counter(x[0] for x in g) + letters = [x for x in g if x[0] == 'LETTER'] + owners = word.OWNERS + q_letter = sum(len(x[1:3]) for x in letters)*owners + assert len(letters) == p and len(thick) == p and q_letter == 4256, q_letter + print('listing gate kinds', dict(kinds)) + print('charge 1: %d LETTER gates x 2 roles x %d owners = %d Q jobs (rank 1)' % (len(letters), owners, q_letter)) + + # ---- charge 2: R-type idles in the invoiced fresh-block geometry ---- + # An idle over the thick stages S is prod_{s in S} R_s = (prod D_s) Q^{|S|}; Q^4 = I, Q^2 a signed + # address swap (monomial, free); so odd |S| needs one extra Gaussian rank on the central line. + # The invoice charges the idle as one rank-8|S| job per field; most favourable: merge -> 8|S|+1. + idle_merge = Counter() + for i, ch in enumerate(pat, 1): + S = (frozenset(range(0, 2*i)) if ch == 'I' else frozenset(range(2*i+1, 2*p+1))) & thick + if len(S) % 2: idle_merge[8*len(S)] += 224 + print('charge 2: retained idles with odd thick count:', dict(idle_merge), '->', sum(idle_merge.values()), + 'rank increments (merged into the idle job)') + # listing check: the listing's idle powers count ALL stages (2i, 36-2j), always even -> R^even is + # monomial in the collapsed 9-axis replay, which is why the replay never saw these costs. + pows = [x[3] for x in g if x[0] == 'IDLE' and x[4] != 'middle'] + middle = [x for x in g if x[0] == 'IDLE' and x[4] == 'middle'] + print(' listing idle powers (non-middle):', sorted(pows), '(all even)') + print(' middle idles in the listing: %d executing R^1 on 2x%d fields; invoice prices them at rank 0' + % (len(middle), owners)) + + variants = {} + variants['V1 +4256 Q (separate rank-1 jobs)'] = (T0, add(P0, {1: q_letter})) + P2 = add(P0, {1: q_letter}) + for r, n in idle_merge.items(): P2[r] -= n; P2[r+1] += n + variants['V2 = V1 + odd-idle Q merged (8k->8k+1)'] = (T0, +P2) + P3 = add(P2, {9: len(middle)*2*owners}) + variants['V3 = V2 + literal middle R idles (rank 9)'] = (T0, +P3) + # terminal-gauge alternative (letter D, no letter Q): Matarener outcome B -- decode the 19 coded + # output banks with one paid Q per owner, credit 8k instead of 1+8k + outs_even = [1+8*len(s & thick) for s in + [frozenset(range(2*p+1))] + [frozenset(range(0, 2*i+1)) if ch == 'I' else + frozenset(range(2*i, 2*p+1)) for i, ch in enumerate(pat, 1)]] + TB, PB = Counter(T0), Counter(P0) + for r in outs_even: TB[r] -= 112; TB[r-1] += 112; PB[1] += 112 + variants['VB letter D + terminal Q decoder (2128), credit 8k'] = (+TB, +PB) + variants['VB0 terminal decoder only, credit kept (2128 rank-1)'] = (T0, add(P0, {1: 2128})) + + for name, (T, P) in variants.items(): + r = root(T, P) + dead, kmin = concave_order_dead(T, P) + ms = mass(T)-mass(P) + out[name] = dict(root=str(r), mass_surplus=ms, concave_order_dead=dead, min_hinge=kmin, + M_at_selected=str(M(T, P, U_SEL)), M_at_ours=str(M(T, P, U_OURS))) + print('%-55s root %.12f mass %+6d M(U_sel) %+.3f dead-for-all-t %s (min hinge %d)' + % (name, r, ms, M(T, P, U_SEL), dead, kmin)) + + # ---- merge-robust bound: any placement of the 4256 Q ranks into existing jobs ---- + # Adding one Gaussian axis to a job of rank r costs (r+1)^(1-C) - r^(1-C) >= (1-C)(r+1)^(-C); + # no job exceeds the ambient rank 153, so each Q costs >= (1-U) 153^(-U) at C = U. + per_q = (1-U_SEL)*pw(153, -U_SEL) + budget = M(T0, P0, U_SEL) + print('merge-robust: each Q costs >= %.6f at U_sel; 4256 Q cost >= %.1f vs budget M0(U_sel) = %.3f' + % (per_q, 4256*per_q, budget)) + print(' max Q ranks affordable at U_sel even with free merging: %d (need 4256; with letter Q only)' + % int(budget/per_q)) + out['merge_robust'] = dict(per_q_lower=str(per_q), budget_at_selected=str(budget), + max_affordable=int(budget/per_q), needed=4256) + (Path(__file__).resolve().parent/'corrected_invoice.json').write_text(json.dumps(out, indent=1, default=str)) + + +if __name__ == '__main__': + main() diff --git a/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.txt b/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.txt new file mode 100644 index 0000000000..408a5937d5 --- /dev/null +++ b/research/helper-handoff-r18/addendum-r20-p19-repricing/corrected_invoice.txt @@ -0,0 +1,16 @@ +V0 package invoice: root 0.007578535190 mass surplus 2016 of 265533 + tolerance (extra rank-1 jobs) above BIT 0.0034: 1103 [M = 1103.276122268] + tolerance (extra rank-1 jobs) above selected : 1202 [M = 1202.668299880] + tolerance (extra rank-1 jobs) above ours : 1202 [M = 1202.668296149] +listing gate kinds {'GAUGE_LOAD': 19, 'FORWARD': 20, 'REVERSE': 19, 'LETTER': 19, 'IDLE': 36, 'BRIDGE': 108, 'GAUGE_UNLOAD': 19, 'SWAP': 19} +charge 1: 19 LETTER gates x 2 roles x 112 owners = 4256 Q jobs (rank 1) +charge 2: retained idles with odd thick count: {8: 448, 24: 448, 40: 448, 56: 448, 72: 448} -> 2240 rank increments (merged into the idle job) + listing idle powers (non-middle): [2, 2, 4, 4, 6, 6, 8, 8, 10, 10, 12, 12, 14, 14, 16, 16, 18, 18] (all even) + middle idles in the listing: 18 executing R^1 on 2x112 fields; invoice prices them at rank 0 +V1 +4256 Q (separate rank-1 jobs) root 0.000000000000 mass -2240 M(U_sel) -3053.332 dead-for-all-t True (min hinge 2240) +V2 = V1 + odd-idle Q merged (8k->8k+1) root 0.000000000000 mass -4480 M(U_sel) -5263.210 dead-for-all-t True (min hinge 4480) +V3 = V2 + literal middle R idles (rank 9) root 0.000000000000 mass -40768 M(U_sel) -41310.637 dead-for-all-t True (min hinge 40768) +VB letter D + terminal Q decoder (2128), credit 8k root 0.000000000000 mass -2240 M(U_sel) -3023.897 dead-for-all-t True (min hinge 2240) +VB0 terminal decoder only, credit kept (2128 rank-1) root 0.000000000000 mass -112 M(U_sel) -925.332 dead-for-all-t True (min hinge 112) +merge-robust: each Q costs >= 0.981905 at U_sel; 4256 Q cost >= 4179.0 vs budget M0(U_sel) = 1202.668 + max Q ranks affordable at U_sel even with free merging: 1224 (need 4256; with letter Q only) diff --git a/research/helper-handoff-r18/control_4610.py b/research/helper-handoff-r18/control_4610.py new file mode 100644 index 0000000000..5e747ac6b2 --- /dev/null +++ b/research/helper-handoff-r18/control_4610.py @@ -0,0 +1,20 @@ +"""Positive control: nreplay reproduces the official literal_replay on the #4610 round-three word +(both orientations): identical rows for all 632 registers + C0 and identical paid profile.""" +import gzip, json, importlib.util, sys +from pathlib import Path +from nreplay import replay, v, ROOT +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path); m = importlib.util.module_from_spec(spec) + sys.modules[name] = m; spec.loader.exec_module(m); return m +X = load(ROOT/'research/kappa-rebuild/complex/native_extensions.py', 'r18_literal') +w = json.loads(gzip.decompress((ROOT/'research/kappa-round3-warp-20261022/ACTUAL-WORD.json.gz').read_bytes())) +c = w['canonical'] +for side in ('forward', 'backward'): + back = side == 'backward' + rows0, rec = X.literal_replay(v, c, w['programs'][side], back) + rows1, prof, _ = replay(632, c['frames'], c['start'], c['final'], w['programs'][side], back) + same = rows0 == rows1 and {int(k): x for k, x in rec['paid_profile'].items()} == prof + ident = all(rows1['L', r] == v.expected(('L', r), [], back, False) for r in range(632)) + print(side, 'nreplay == literal_replay:', same, '| all 632 endpoints expected:', ident, '| profile', prof) + assert same and ident +print('PASS_NREPLAY_CONTROL_4610') diff --git a/research/helper-handoff-r18/control_4610.txt b/research/helper-handoff-r18/control_4610.txt new file mode 100644 index 0000000000..975527fb91 --- /dev/null +++ b/research/helper-handoff-r18/control_4610.txt @@ -0,0 +1,3 @@ +forward nreplay == literal_replay: True | all 632 endpoints expected: True | profile {1: 2201, 2: 412, 3: 510, 4: 127, 5: 40, 8: 8} +backward nreplay == literal_replay: True | all 632 endpoints expected: True | profile {1: 2201, 2: 412, 3: 510, 4: 127, 5: 40, 8: 8} +PASS_NREPLAY_CONTROL_4610 diff --git a/research/helper-handoff-r18/gadget.py b/research/helper-handoff-r18/gadget.py new file mode 100644 index 0000000000..30515f1fb6 --- /dev/null +++ b/research/helper-handoff-r18/gadget.py @@ -0,0 +1,97 @@ +"""Constructive proof of concept: SUCCESSOR HAND-OFF of a productive conduit helper (R18). + +Scaled-down native instance on F_2^5 (h = 5 axes). Registers: + 0 x port line start A=<1> final U (full) + 1 m1 main consumer start 0 final U wanted row m1_in + x_in + 2 c main consumer start 0 final U wanted row c_in + x_in + 3 h helper (arbitrary dirty input, must be restored) + 4 s successor helper (hand-off version only; arbitrary dirty input, starts at F) +The helper h is a CONDUIT, the same motif as #4610's helper threads (e.g. h300: read a port line at a +low frame, carry it along a different flag, deliver it to consumers there, uncompute at the top): +x's data must reach m1 at frame B=<1,2> and c at frame G=<1,2,4,8>; x itself travels A -> Xb=<1,4> -> F. + +BASELINE: h carries x from A through B, F, G to U; consumers pre-subtract h_in while h is pure at A; +the uncompute h -= x happens at the top. h climbs 0>A>B>F>G>U: five rank-1 jobs, ONE rank-5 target. + +HAND-OFF: identical paid climbs. At F, where x meets the conduit with unchanged data, two free ADDs +make h pure (h -= x) and load the successor (s += x); h retires at F with a rank-3 target and s (which +sat idle at its start frame F) carries the conduit F>G>U with a rank-2 target. The consumer above F +(c) pre-subtracts s_in instead of h_in, during its own early pass through F while s is still pure. +Every register ends with its wanted row on ALL arbitrary input columns (exact replay, nreplay.py = +literal_replay rules). Targets 5 -> 3 + 2 at the same paid profile.""" +import sys +from fractions import Fraction as Q +from nreplay import replay, v, need +sys.path.insert(0, str(__import__('pathlib').Path(__file__).resolve().parent)) + +FR = [[], [1], [1, 2], [1, 4], [1, 2, 4], [1, 2, 4, 8], [1, 2, 4, 8, 16]] +Z, A, B, XB, F, G, U = range(7) +X, M1, CC, H, S = 0, 1, 2, 3, 4 +def climb(i, a, b): return ('climb', 'L', i, a, b, len(FR[b]) - len(FR[a])) +def add(t, s, num, frame, den=1): return ('scalar', ['add', ['L', t], ['L', s], num, den], frame) + +CLR = ('scalar', ['clear', ['C', 0]], Z) # owned C0 is unused; cleared as the interpreter requires +def baseline(): + return [CLR, climb(H, Z, A), + climb(M1, Z, A), add(M1, H, -1, A), # m1 -= h_in (h pure at A) + climb(CC, Z, A), add(CC, H, -1, A), # c -= h_in + add(H, X, 1, A), # conduit loads x + climb(CC, A, F), climb(CC, F, G), # c passes F early (before x arrives there) + climb(H, A, B), climb(M1, A, B), add(M1, H, 1, B), # m1 = m1_in + x_in + climb(X, A, XB), climb(X, XB, F), climb(H, B, F), # x and h meet at F + climb(H, F, G), add(CC, H, 1, G), # c = c_in + x_in + climb(H, G, U), climb(X, F, U), add(H, X, -1, U), # uncompute at the top + climb(M1, B, U), climb(CC, G, U)] + +def handoff(presub=True, cancel=True, cancel_frame=F): + p = [CLR, climb(H, Z, A), + climb(M1, Z, A), add(M1, H, -1, A), + climb(CC, Z, A)] + ([] if presub else [add(CC, H, -1, A)]) + [ + add(H, X, 1, A), + climb(CC, A, F)] + ([add(CC, S, -1, F)] if presub else []) + [ # c -= s_in (s idle & pure at F) + climb(CC, F, G), + climb(H, A, B), climb(M1, A, B), add(M1, H, 1, B), + climb(X, A, XB)] + if cancel_frame == XB: # control: try the cancellation before x reaches F + p += [add(H, X, -1, XB)] + p += [climb(X, XB, F), climb(H, B, F)] + if cancel and cancel_frame == F: + p += [add(H, X, -1, F)] # h pure: retires at F + p += [add(S, X, 1, F), # successor takes the conduit + climb(S, F, G), add(CC, S, 1, G), + climb(S, G, U), climb(X, F, U), add(S, X, -1, U), + climb(M1, B, U), climb(CC, G, U)] + return p + +def wanted(n): + out = {} + for r in range(n): + row = v.identity(('L', r)) + if r in (M1, CC): v.add(row, v.identity(('L', X)), Q(1)) + out[r] = row + return out + +def run(name, n, start, final, prog, helpers): + rows, prof, climbs = replay(n, FR, start, final, prog) + want = wanted(n) + ok = all(rows['L', r] == want[r] for r in range(n)) + need(ok, name + ': endpoint rows') + targets = sorted(len(FR[final[i]]) - len(FR[start[i]]) for i in helpers) + print(f'{name:9s} all {n} arbitrary input columns exact: {ok}; paid profile {prof}; ' + f'helper climbs {[climbs[i] for i in helpers]}; productive helper targets {targets}') + return prof, targets + +if __name__ == '__main__': + p0, t0 = run('baseline', 4, [A, Z, Z, Z], [U, U, U, U], baseline(), [H]) + p1, t1 = run('hand-off', 5, [A, Z, Z, Z, F], [U, U, U, F, U], handoff(), [H, S]) + need(p0 == p1, 'same paid profile'); need(sum(t0) == sum(t1), 'same target mass (rank-mass invariant)') + # mutation controls: each must FAIL + ctl = {'no successor pre-subtraction': dict(presub=False), + 'no co-framed cancellation': dict(cancel=False), + 'cancellation before the frames meet': dict(cancel=False, cancel_frame=XB)} + for name, kw in ctl.items(): + try: + run('control', 5, [A, Z, Z, Z, F], [U, U, U, F, U], handoff(**kw), [H, S]); raise SystemExit('control passed: ' + name) + except AssertionError as e: + print(f'control rejected ({name}): {str(e)[:60]}') + print('PASS_HANDOFF_GADGET targets', t0, '->', t1, 'paid', p0) diff --git a/research/helper-handoff-r18/gadget.txt b/research/helper-handoff-r18/gadget.txt new file mode 100644 index 0000000000..91a778dc9b --- /dev/null +++ b/research/helper-handoff-r18/gadget.txt @@ -0,0 +1,6 @@ +baseline all 4 arbitrary input columns exact: True; paid profile {1: 12, 2: 2, 3: 1}; helper climbs [5]; productive helper targets [5] +hand-off all 5 arbitrary input columns exact: True; paid profile {1: 12, 2: 2, 3: 1}; helper climbs [3, 2]; productive helper targets [2, 3] +control rejected (no successor pre-subtraction): control: endpoint rows +control rejected (no co-framed cancellation): control: endpoint rows +control rejected (cancellation before the frames meet): Exact scalar target frame ('scalar', ['add', ['L', 3], ['L', +PASS_HANDOFF_GADGET targets [5] -> [2, 3] paid {1: 12, 2: 2, 3: 1} diff --git a/research/helper-handoff-r18/nreplay.py b/research/helper-handoff-r18/nreplay.py new file mode 100644 index 0000000000..d09ec99641 --- /dev/null +++ b/research/helper-handoff-r18/nreplay.py @@ -0,0 +1,64 @@ +"""Generalised literal native replay (R18). + +Same rules as research/kappa-rebuild/complex/native_extensions.py::literal_replay, but for any number of +persistent registers and any frame lattice F_2^h: + * scalar add/copy/scale need the target (and source) at the event's exact frame index; + * clear and copy may only target the owned register ('C',0); + * a climb is a strictly ascending subspace inclusion, paid at its dimension jump (rank checked); + * warps change the frame and append a word symbol (verify_buffers.warp); + * final frames must match and owned C0 must be cleared at return. +Rows are exact (Fraction) formal combinations of (input, warp word).""" +import sys +from collections import Counter +from fractions import Fraction as Q +from pathlib import Path +ROOT = Path(__file__).resolve().parents[2] +sys.path.insert(0, str(ROOT/'research/kappa-causal-continuation/buffer-borrowing')) +import verify_buffers as v + +def need(ok, msg): + if not ok: raise AssertionError(msg) + +def span(basis): + pts = {0} + for x in basis: pts |= {p ^ x for p in tuple(pts)} + return frozenset(pts) + +def replay(n, frames, start, final, program, backward=False): + rows = {('L', i): v.identity(('L', i)) for i in range(n)}; rows['C', 0] = v.identity(('C', 0)) + cur = {('L', i): f for i, f in enumerate(start)}; cur['C', 0] = None + sp = [span(f) for f in frames]; profile = Counter(); climbs = Counter() + for event in program: + kind = event[0] + if kind == 'scalar': + g, frame = event[1:]; t = tuple(g[1]) + if g[0] == 'clear': + need(t == ('C', 0), 'Only owned C0 can be cleared'); rows[t] = {}; cur[t] = frame + else: + need(cur[t] == frame, 'Exact scalar target frame %r' % (event,)) + if g[0] in ('add', 'copy'): + s = tuple(g[2]); need(cur[s] == frame, 'Exact scalar source frame %r' % (event,)) + if g[0] == 'copy': + need(t == ('C', 0), 'Only owned field overwritten by copy'); rows[t] = dict(rows[s]) + else: + need(t != s, 'Distinct reversible ADD arguments'); v.add(rows[t], rows[s], Q(g[3], g[4])) + else: + need(g[0] == 'scale', 'Known scalar action'); q = Q(g[2], g[3]) + rows[t] = {k: q*x for k, x in rows[t].items()} + elif kind == 'warp': + _, r, op, old, new = event; t = tuple(r) + need(cur[t] == old, 'Warp input frame'); rows[t] = v.warp(rows[t], op, backward); cur[t] = new + elif kind in ('climb', 'center'): + if kind == 'climb': + _, part, i, old, new, rank = event; t = ('L', i) + need(sp[old] < sp[new], 'Ascending borrowed climb %r' % (event,)); climbs[i] += 1 + else: + _, i, old, new, rank = event; t = ('C', i) + need(sp[old] < sp[new] or sp[new] < sp[old], 'Nested center transport') + need(cur[t] == old and rank == abs(len(frames[new]) - len(frames[old])), 'Paid rank and source frame %r' % (event,)) + cur[t] = new; profile[rank] += 1 + else: + need(kind in ('phase', 'adapter', 'residual_adapter'), 'Known inert native marker') + need([cur['L', i] for i in range(n)] == list(final), 'Final native frames') + need(not rows['C', 0], 'Owned C0 cleared at return') + return rows, dict(sorted(profile.items())), climbs diff --git a/research/helper-handoff-r18/price16.py b/research/helper-handoff-r18/price16.py new file mode 100644 index 0000000000..0af5d25092 --- /dev/null +++ b/research/helper-handoff-r18/price16.py @@ -0,0 +1,28 @@ +"""R16 float/Decimal pricer for the selected #5413 profile and structural lever screens. +M(C) = sum_T n k^(1-C) - sum_P n r^(1-C). Numerical screens only (no outward rounding).""" +from decimal import Decimal as D, getcontext +import json, sys +getcontext().prec = 50 +P0 = {1:41154,2:7416,3:9180,4:2286,5:720,8:144,16:448,24:448,32:448,40:448,48:448,56:448,64:448,72:224} +T0 = {4:504,7:126,8:990,9:5724,17:448,25:448,33:448,41:448,49:448,57:448,65:448,73:224,145:224} +def pw(x, e): return (D(x).ln()*e).exp() +def M(T, P, C): + e = 1-D(C) + return sum(n*pw(k,e) for k,n in T.items() if n) - sum(n*pw(r,e) for r,n in P.items() if n) +def root(T=T0, P=P0, lo='0', hi='0.05'): + lo, hi = D(lo), D(hi) + if M(T,P,lo) <= 0: return D(0) + while M(T,P,hi) > 0: hi *= 2 + for _ in range(80): + m = (lo+hi)/2 + if M(T,P,m) > 0: lo = m + else: hi = m + return lo +def add(h, d): + h = dict(h) + for k, v in d.items(): h[k] = h.get(k,0)+v + return h +if __name__ == '__main__': + r = root(); print('selected root', r) + eps = D('1e-12'); print('dM/dC', (M(T0,P0,r+eps)-M(T0,P0,r-eps))/(2*eps)) + print('mass surplus', sum(k*n for k,n in T0.items())-sum(k*n for k,n in P0.items())) diff --git a/research/helper-handoff-r18/price_handoff.py b/research/helper-handoff-r18/price_handoff.py new file mode 100644 index 0000000000..65d04bcaf0 --- /dev/null +++ b/research/helper-handoff-r18/price_handoff.py @@ -0,0 +1,18 @@ +"""Price successor hand-offs on the selected #5413 profile (Decimal, 50 digits; numerical screen). +A hand-off splits one rank-k productive helper target into a + (k-a) at an unchanged paid profile. +Columns: per-hand-off moment gain dM = f(a)+f(k-a)-f(k) at the selected root, and the raw root when one +such hand-off is added in each of n locals. U = 0.00302728825 is the fixed-profile ceiling (#5409).""" +from price16 import * +U = D('0.00302728825'); base = root(); e = 1 - base +print('selected raw root', base) +f = lambda x: pw(x, e) +for k, a in [(9, 1), (9, 2), (9, 3), (9, 4), (8, 4), (7, 3), (4, 2), (5, 2)]: + dM = f(a) + f(k - a) - f(k) + line = f'k={k} -> {a}+{k-a}: dM per hand-off {float(dM):.6f}' + for n in (1, 9, 18): + r = root(add(add(T0, {k: -n, a: n}), {k - a: n}), P0) + line += f' | n={n:2d} root {r:.17f} (beats U: {r > U})' + print(line) +# ceilings: every rank-9 helper handed off once at 4+5; every helper split at every climb (R16 levers.py) +print('all 5724 rank-9 targets handed off once at 4+5:', f"{root(add(T0, {9: -5724, 4: 5724, 5: 5724}), P0):.10f}") +print('PASS_PRICE_HANDOFF') diff --git a/research/helper-handoff-r18/price_handoff.txt b/research/helper-handoff-r18/price_handoff.txt new file mode 100644 index 0000000000..f293073a01 --- /dev/null +++ b/research/helper-handoff-r18/price_handoff.txt @@ -0,0 +1,11 @@ +selected raw root 0.0030272882499395587812724555013848588705741016191552 +k=9 -> 1+8: dM per hand-off 0.009464 | n= 1 root 0.00302732660083306 (beats U: True) | n= 9 root 0.00302763344304756 (beats U: True) | n=18 root 0.00302797871507213 (beats U: True) +k=9 -> 2+7: dM per hand-off 0.014359 | n= 1 root 0.00302734643976947 (beats U: True) | n= 9 root 0.00302781203907800 (beats U: True) | n=18 root 0.00302833600978172 (beats U: True) +k=9 -> 3+6: dM per hand-off 0.017248 | n= 1 root 0.00302735814804349 (beats U: True) | n= 9 root 0.00302791744923437 (beats U: True) | n=18 root 0.00302854691044092 (beats U: True) +k=9 -> 4+5: dM per hand-off 0.018612 | n= 1 root 0.00302736367664294 (beats U: True) | n= 9 root 0.00302796722574580 (beats U: True) | n=18 root 0.00302864650650212 (beats U: True) +k=8 -> 4+4: dM per hand-off 0.016699 | n= 1 root 0.00302735592235097 (beats U: True) | n= 9 root 0.00302789741072923 (beats U: True) | n=18 root 0.00302850681705823 (beats U: True) +k=7 -> 3+4: dM per hand-off 0.014402 | n= 1 root 0.00302734661373588 (beats U: True) | n= 9 root 0.00302781360527997 (beats U: True) | n=18 root 0.00302833914332083 (beats U: True) +k=4 -> 2+2: dM per hand-off 0.008367 | n= 1 root 0.00302732215684232 (beats U: True) | n= 9 root 0.00302759343950518 (beats U: True) | n=18 root 0.00302789869082438 (beats U: True) +k=5 -> 2+3: dM per hand-off 0.010149 | n= 1 root 0.00302732937705650 (beats U: True) | n= 9 root 0.00302765843433960 (beats U: True) | n=18 root 0.00302802870954285 (beats U: True) +all 5724 rank-9 targets handed off once at 4+5: 0.0035309323 +PASS_PRICE_HANDOFF diff --git a/research/helper-handoff-r18/screens/presub.py b/research/helper-handoff-r18/screens/presub.py new file mode 100644 index 0000000000..57aec1eab6 --- /dev/null +++ b/research/helper-handoff-r18/screens/presub.py @@ -0,0 +1,28 @@ +"""For each pure-window hand-off candidate (relay2), list the outside holders E of h_in and whether each +ever visits the hand-off frame F (needed for successor pre-subtraction / post-correction).""" +from relay2 import * +def visits(prog): + vis = defaultdict(set) + for r, f in enumerate(c['start']): vis['L', r].add(f) + for e in prog: + if e[0] == 'climb': vis['L', e[2]].add(e[4]) + elif e[0] == 'warp': vis[tuple(e[1])].add(e[4]) + elif e[0] == 'scalar': vis[tuple(e[1][1])].add(e[2]) + elif e[0] == 'center': vis['C', e[1]].add(e[3]) + return vis +for side in ('forward', 'backward'): + back = side == 'backward'; prog = w['programs'][side]; vis = visits(prog) + win = windows(side); at = defaultdict(list) + for h, a, b in win: at[a].append(h) + rows = {('L', i): v.identity(('L', i)) for i in range(632)}; rows['C', 0] = v.identity(('C', 0)) + cur = {('L', i): f for i, f in enumerate(c['start'])}; cur['C', 0] = None + stat = Counter(); nE = Counter() + for j, e in enumerate(prog): + step(rows, cur, e, back) + for h in at.get(j, ()): + F = cur[h] + E = [r for r in rows if cur[r] != F and any(k[0] == h for k in rows[r])] + nv = sum(1 for r in E if F in vis[r]) + stat['all E-holders visit F' if nv == len(E) else ('some' if nv else 'none visit F')] += 1 + nE[len(E)] += 1 + print(side, 'candidates', len(win), dict(stat), 'E-size histogram', dict(sorted(nE.items()))) diff --git a/research/helper-handoff-r18/screens/presub.txt b/research/helper-handoff-r18/screens/presub.txt new file mode 100644 index 0000000000..9baf7f4468 --- /dev/null +++ b/research/helper-handoff-r18/screens/presub.txt @@ -0,0 +1,2 @@ +forward candidates 168 {'none visit F': 165, 'some': 3} E-size histogram {1: 73, 2: 14, 3: 9, 4: 37, 5: 2, 6: 4, 8: 4, 9: 6, 14: 4, 17: 2, 18: 4, 23: 2, 24: 5, 33: 2} +backward candidates 65 {'none visit F': 65} E-size histogram {1: 16, 2: 42, 3: 5, 11: 2} diff --git a/research/helper-handoff-r18/screens/relay.py b/research/helper-handoff-r18/screens/relay.py new file mode 100644 index 0000000000..bfdb8f18e5 --- /dev/null +++ b/research/helper-handoff-r18/screens/relay.py @@ -0,0 +1,113 @@ +"""R18 relay screen (generalises R16 split.py). + +Relay lemma. At a time t, helper h (input variable h_in) sits at frame F strictly between its +start and final dims with climbs ahead. Let R_F = registers whose current frame index is F. +If (a) every register whose row mentions h_in (any warp word) is in R_F, and + (b) h_in occurs only with the trivial word and (h_in, ()) lies in the Q-span of the rows of R_F, +then a FRESH arbitrary helper h' that STARTS at frame F can be inserted, and free scalar ops at F +realise the substitution tau = swap(h_in, h'_in) on the whole state. Every later op is linear, so the +final state is tau(original final): register h ends holding h'_in (transformed F -> final) and the +new register ends at F holding h_in (transformed start -> F). Paid climbs are unchanged; the +rank-k productive target splits into (dimF - d0) + (d1 - dimF). +This script only SCREENS for such points (exact rows over Q).""" +import sys, gzip, json +from collections import Counter, defaultdict +from fractions import Fraction as Q +from pathlib import Path +R13 = Path(__file__).resolve().parents[3] +sys.path.insert(0, str(R13/'research/kappa-causal-continuation/buffer-borrowing')) +import verify_buffers as v +w = json.loads(gzip.decompress((R13/'research/kappa-round3-warp-20261022/ACTUAL-WORD.json.gz').read_bytes())) +c = w['canonical']; fr = c['frames'] +def dim(f): return len(fr[f]) if f is not None else -1 + +def in_span(target, vecs): + """exact: is target (dict) in Q-span of vecs (list of dicts)?""" + piv = [] # list of (pivot key, row dict) reduced + def reduce(x): + x = dict(x) + for k, r in piv: + a = x.get(k) + if a: + for kk, vv in r.items(): + nv = x.get(kk, Q(0)) - a*vv + if nv: x[kk] = nv + else: x.pop(kk, None) + return x + for vec in vecs: + x = reduce(vec) + if x: + k = next(iter(x)); a = x[k]; x = {kk: vv/a for kk, vv in x.items()} + # keep piv fully reduced w.r.t. new pivot + for i, (k2, r2) in enumerate(piv): + b = r2.get(k) + if b: + for kk, vv in x.items(): + nv = r2.get(kk, Q(0)) - b*vv + if nv: r2[kk] = nv + else: r2.pop(kk, None) + piv.append((k, x)) + return not reduce(target) + +def screen(side, relax=False): + back = side == 'backward'; prog = w['programs'][side] + rows = {('L', i): v.identity(('L', i)) for i in range(632)}; rows['C', 0] = v.identity(('C', 0)) + cur = {('L', i): f for i, f in enumerate(c['start'])}; cur['C', 0] = None + inv = defaultdict(set) + for r, row in rows.items(): + for (var, wd) in row: inv[var].add(r) + atF = defaultdict(set) + for r, f in cur.items(): atF[f].add(r) + def setf(r, f): + atF[cur[r]].discard(r); cur[r] = f; atF[f].add(r) + climbs_left = Counter(e[2] for e in prog if e[0] == 'climb') + helpers = [('L', i) for i in range(224, 632)] + found = defaultdict(set); first = {}; nspan = 0; npass_a = Counter() + for j, e in enumerate(prog): + k = e[0] + if k == 'scalar': + g, frame = e[1:]; t = tuple(g[1]) + old = {var for (var, wd) in rows[t]} + if g[0] == 'clear': rows[t] = {}; setf(t, frame) + elif g[0] == 'copy': rows[t] = dict(rows[tuple(g[2])]) + elif g[0] == 'add': v.add(rows[t], rows[tuple(g[2])], Q(g[3], g[4])) + else: q = Q(g[2], g[3]); rows[t] = {a: q*x for a, x in rows[t].items()} + new = {var for (var, wd) in rows[t]} + for var in old - new: inv[var].discard(t) + for var in new - old: inv[var].add(t) + elif k == 'warp': + _, r, op, old_f, new_f = e; t = tuple(r); rows[t] = v.warp(rows[t], op, back); setf(t, new_f) + elif k == 'climb': + i = e[2]; setf(('L', i), e[4]); climbs_left[i] -= 1 + elif k == 'center': + setf(('C', e[1]), e[3]) + else: + continue + # check every helper (cheap filter (a) first) + for h in helpers: + i = h[1] + if climbs_left[i] <= 0: continue + F = cur[h]; d = dim(F); d0 = dim(c['start'][i]) + if not d0 < d: continue + hold = inv[h] + if not hold <= atF[F]: continue + npass_a[i] += 1 + key = (i, d) + if key in found and not relax: continue + # (b) trivial words only, span + if any(wd != () for r in hold for (var, wd) in rows[r] if var == h): continue + nspan += 1 + if in_span({(h, ()): Q(1)}, [rows[r] for r in atF[F]]): + found[i].add(d); first.setdefault(key, (j, sorted(atF[F]), len(hold))) + tot = Counter() + for i, ds in found.items(): + for d in ds: tot[(dim(c['start'][i]), d, dim(c['final'][i]))] += 1 + print(side, 'helpers with a relay point:', len(found), 'options', sum(tot.values()), dict(sorted(tot.items())), + '| (a)-passes helpers', len(npass_a), 'span checks', nspan, flush=True) + for key, val in sorted(first.items())[:40]: + print(' ', key, 'event', val[0], 'holders', val[2], 'R_F', val[1][:8], len(val[1])) + return found, first + +if __name__ == '__main__': + for side in sys.argv[1:] or ('forward', 'backward'): + screen(side) diff --git a/research/helper-handoff-r18/screens/relay.txt b/research/helper-handoff-r18/screens/relay.txt new file mode 100644 index 0000000000..25948cde61 --- /dev/null +++ b/research/helper-handoff-r18/screens/relay.txt @@ -0,0 +1,2 @@ +forward helpers with a relay point: 0 options 0 {} | (a)-passes helpers 65 span checks 151573 +backward helpers with a relay point: 0 options 0 {} | (a)-passes helpers 172 span checks 49934 diff --git a/research/helper-handoff-r18/screens/relay2.py b/research/helper-handoff-r18/screens/relay2.py new file mode 100644 index 0000000000..ddda36938c --- /dev/null +++ b/research/helper-handoff-r18/screens/relay2.py @@ -0,0 +1,96 @@ +"""Relay with deferred cancellation (R18). +Candidate: helper h pure (row == h_in, trivial word) at an intermediate frame F (start dim < dim F) +with climbs ahead, at time t. Rewrite: new register R' starting at F; at t, for every register r at +F holding a*h_in (trivial word): r -= a*h ; r += a*R' (free ADDs at F); every later op on h is renamed +to R'; h retires at frame F. Valid iff the h_in-content held OUTSIDE F at time t (vector E) is driven +exactly to 0 by the remaining program (the rest of the renamed program acts on E linearly). +Gain: rank (d1-d0) target -> (dF-d0) + (d1-dF), paid climbs unchanged.""" +import sys +from collections import Counter, defaultdict +from relay import * + +def step(rows, cur, e, back): + k = e[0]; touched = None + if k == 'scalar': + g, frame = e[1:]; t = tuple(g[1]); touched = t + if g[0] == 'clear': rows[t] = {}; cur[t] = frame + elif g[0] == 'copy': rows[t] = dict(rows[tuple(g[2])]) + elif g[0] == 'add': v.add(rows[t], rows[tuple(g[2])], Q(g[3], g[4])) + else: q = Q(g[2], g[3]); rows[t] = {a: q*x for a, x in rows[t].items()} + elif k == 'warp': + _, r, op, o, n = e; t = tuple(r); touched = t; rows[t] = v.warp(rows[t], op, back); cur[t] = n + elif k == 'climb': + touched = ('L', e[2]); cur[touched] = e[4] + elif k == 'center': cur['C', e[1]] = e[3] + return touched + +def propagate(prog, j0, E, back): + E = {r: dict(x) for r, x in E.items() if x} + for e in prog[j0+1:]: + if not E: break + k = e[0] + if k == 'scalar': + g = e[1]; t = tuple(g[1]) + if g[0] == 'clear': E.pop(t, None) + elif g[0] == 'copy': + s = tuple(g[2]); + if s in E: E[t] = dict(E[s]) + else: E.pop(t, None) + elif g[0] == 'add': + s = tuple(g[2]) + if s in E: + d = E.setdefault(t, {}); v.add(d, E[s], Q(g[3], g[4])) + if not d: E.pop(t) + elif t in E: + q = Q(g[2], g[3]); E[t] = {a: q*x for a, x in E[t].items()} + elif k == 'warp': + _, r, op, o, n = e; t = tuple(r) + if t in E: E[t] = v.warp(E[t], op, back) + return E + +def windows(side): + back = side == 'backward'; prog = w['programs'][side] + rows = {('L', i): v.identity(('L', i)) for i in range(632)}; rows['C', 0] = v.identity(('C', 0)) + cur = {('L', i): f for i, f in enumerate(c['start'])}; cur['C', 0] = None + left = Counter(e[2] for e in prog if e[0] == 'climb'); open_ = {}; win = [] + for j, e in enumerate(prog): + t = step(rows, cur, e, back) + if e[0] == 'climb': left[e[2]] -= 1 + if t and t[0] == 'L' and t[1] >= 224: + i = t[1] + if t in open_: win.append((t, open_.pop(t), j - 1)) + if left[i] > 0 and dim(c['start'][i]) < dim(cur[t]) and rows[t] == {(t, ()): Q(1)}: + open_[t] = j + return win + +def screen(side, samples=6): + back = side == 'backward'; prog = w['programs'][side] + win = windows(side) + times = defaultdict(list) + for h, a, b in win: + ts = sorted({a, b} | {a + (b - a) * s // samples for s in range(samples + 1)}) + for t in ts: times[t].append(h) + rows = {('L', i): v.identity(('L', i)) for i in range(632)}; rows['C', 0] = v.identity(('C', 0)) + cur = {('L', i): f for i, f in enumerate(c['start'])}; cur['C', 0] = None + res = []; why = Counter() + for j, e in enumerate(prog): + step(rows, cur, e, back) + for h in times.get(j, ()): + F = cur[h] + inF = {r: {k: x for k, x in rows[r].items() if k[0] == h} for r in rows if cur[r] == F and r != h} + if any(wd != () for x in inF.values() for (var, wd) in x): why['nontrivial word at F'] += 1; continue + E = {r: {k: x for k, x in rows[r].items() if k[0] == h} for r in rows if cur[r] != F} + E = {r: x for r, x in E.items() if x} + Ef = propagate(prog, j, E, back) + ok = not Ef + why['valid' if ok else 'E survives'] += 1 + res.append((h[1], j, dim(c['start'][h[1]]), dim(F), dim(c['final'][h[1]]), len(E), ok, sum(1 for x in inF.values() if x))) + print(side, 'pure windows', len(win), 'tests', sum(why.values()), dict(why), flush=True) + good = sorted({(r[0], r[2], r[3], r[4]) for r in res if r[6]}) + print(side, 'VALID relays (helper, d0, dF, d1):', len(good), good[:60], flush=True) + return res + +if __name__ == '__main__': + import pickle + out = {s: screen(s) for s in (sys.argv[1:] or ('forward', 'backward'))} + diff --git a/research/helper-handoff-r18/screens/relay2.txt b/research/helper-handoff-r18/screens/relay2.txt new file mode 100644 index 0000000000..793a8561e4 --- /dev/null +++ b/research/helper-handoff-r18/screens/relay2.txt @@ -0,0 +1,4 @@ +forward pure windows 168 tests 257 {'E survives': 257} +forward VALID relays (helper, d0, dF, d1): 0 [] +backward pure windows 65 tests 455 {'E survives': 455} +backward VALID relays (helper, d0, dF, d1): 0 [] diff --git a/research/helper-handoff-r18/screens/relay_def.py b/research/helper-handoff-r18/screens/relay_def.py new file mode 100644 index 0000000000..1d81cf1e4d --- /dev/null +++ b/research/helper-handoff-r18/screens/relay_def.py @@ -0,0 +1,123 @@ +"""R18 relay screen (generalises R16 split.py). + +Relay lemma. At a time t, helper h (input variable h_in) sits at frame F strictly between its +start and final dims with climbs ahead. Let R_F = registers whose current frame index is F. +If (a) every register whose row mentions h_in (any warp word) is in R_F, and + (b) h_in occurs only with the trivial word and (h_in, ()) lies in the Q-span of the rows of R_F, +then a FRESH arbitrary helper h' that STARTS at frame F can be inserted, and free scalar ops at F +realise the substitution tau = swap(h_in, h'_in) on the whole state. Every later op is linear, so the +final state is tau(original final): register h ends holding h'_in (transformed F -> final) and the +new register ends at F holding h_in (transformed start -> F). Paid climbs are unchanged; the +rank-k productive target splits into (dimF - d0) + (d1 - dimF). +This script only SCREENS for such points (exact rows over Q).""" +import sys, gzip, json +from collections import Counter, defaultdict +from fractions import Fraction as Q +from pathlib import Path +R13 = Path(__file__).resolve().parents[3] +sys.path.insert(0, str(R13/'research/kappa-causal-continuation/buffer-borrowing')) +import verify_buffers as v +w = json.loads(gzip.decompress((R13/'research/kappa-round3-warp-20261022/ACTUAL-WORD.json.gz').read_bytes())) +c = w['canonical']; fr = c['frames'] +def dim(f): return len(fr[f]) if f is not None else -1 + +def in_span(target, vecs): + """exact: is target (dict) in Q-span of vecs (list of dicts)?""" + piv = [] # list of (pivot key, row dict) reduced + def reduce(x): + x = dict(x) + for k, r in piv: + a = x.get(k) + if a: + for kk, vv in r.items(): + nv = x.get(kk, Q(0)) - a*vv + if nv: x[kk] = nv + else: x.pop(kk, None) + return x + for vec in vecs: + x = reduce(vec) + if x: + k = next(iter(x)); a = x[k]; x = {kk: vv/a for kk, vv in x.items()} + # keep piv fully reduced w.r.t. new pivot + for i, (k2, r2) in enumerate(piv): + b = r2.get(k) + if b: + for kk, vv in x.items(): + nv = r2.get(kk, Q(0)) - b*vv + if nv: r2[kk] = nv + else: r2.pop(kk, None) + piv.append((k, x)) + return reduce(target) + +def screen(side, relax=False): + back = side == 'backward'; prog = w['programs'][side] + rows = {('L', i): v.identity(('L', i)) for i in range(632)}; rows['C', 0] = v.identity(('C', 0)) + cur = {('L', i): f for i, f in enumerate(c['start'])}; cur['C', 0] = None + inv = defaultdict(set) + for r, row in rows.items(): + for (var, wd) in row: inv[var].add(r) + atF = defaultdict(set) + for r, f in cur.items(): atF[f].add(r) + def setf(r, f): + atF[cur[r]].discard(r); cur[r] = f; atF[f].add(r) + climbs_left = Counter(e[2] for e in prog if e[0] == 'climb') + helpers = [('L', i) for i in range(224, 632)] + best = {}; seen = set(); found = defaultdict(set); first = {}; nspan = 0; npass_a = Counter() + for j, e in enumerate(prog): + k = e[0] + if k == 'scalar': + g, frame = e[1:]; t = tuple(g[1]) + old = {var for (var, wd) in rows[t]} + if g[0] == 'clear': rows[t] = {}; setf(t, frame) + elif g[0] == 'copy': rows[t] = dict(rows[tuple(g[2])]) + elif g[0] == 'add': v.add(rows[t], rows[tuple(g[2])], Q(g[3], g[4])) + else: q = Q(g[2], g[3]); rows[t] = {a: q*x for a, x in rows[t].items()} + new = {var for (var, wd) in rows[t]} + for var in old - new: inv[var].discard(t) + for var in new - old: inv[var].add(t) + elif k == 'warp': + _, r, op, old_f, new_f = e; t = tuple(r); rows[t] = v.warp(rows[t], op, back); setf(t, new_f) + elif k == 'climb': + i = e[2]; setf(('L', i), e[4]); climbs_left[i] -= 1 + elif k == 'center': + setf(('C', e[1]), e[3]) + else: + continue + # check every helper (cheap filter (a) first) + for h in helpers: + i = h[1] + if climbs_left[i] <= 0: continue + F = cur[h]; d = dim(F); d0 = dim(c['start'][i]) + if not d0 < d: continue + hold = inv[h] + if not hold <= atF[F]: continue + npass_a[i] += 1 + key = (i, d) + if key in found and not relax: continue + if (i, d, len(atF[F])) in seen: continue + seen.add((i, d, len(atF[F]))) + # (b) trivial words only, span + if any(wd != () for r in hold for (var, wd) in rows[r] if var == h): continue + nspan += 1 + res = in_span({(h, ()): Q(1)}, [rows[r] for r in atF[F]]) + if not res: + found[i].add(d); first.setdefault(key, (j, sorted(atF[F]), len(hold))) + else: + # deficiency: residual of h's own row against the OTHER registers at F + rr = in_span(rows[h], [rows[r] for r in atF[F] if r != h]) + dv = len({var for (var, wd) in rr} - {h}) + if dv < best.get(i, (99,))[0]: best[i] = (dv, d, j, len(atF[F])) + tot = Counter() + for i, ds in found.items(): + for d in ds: tot[(dim(c['start'][i]), d, dim(c['final'][i]))] += 1 + print(side, 'helpers with a relay point:', len(found), 'options', sum(tot.values()), dict(sorted(tot.items())), + '| (a)-passes helpers', len(npass_a), 'span checks', nspan, flush=True) + for key, val in sorted(first.items())[:40]: + print(' ', key, 'event', val[0], 'holders', val[2], 'R_F', val[1][:8], len(val[1])) + print(side, 'deficiency histogram (min #foreign input vars left in h after cancelling with co-framed rows):', Counter(b[0] for b in best.values())) + for i, b in sorted(best.items(), key=lambda kv: kv[1])[:15]: print(' h', i, 'deficiency,dim,event,|R_F|', b) + return found, first + +if __name__ == '__main__': + for side in sys.argv[1:] or ('forward', 'backward'): + screen(side) diff --git a/research/helper-handoff-r18/screens/relay_def.txt b/research/helper-handoff-r18/screens/relay_def.txt new file mode 100644 index 0000000000..3b544e6e4e --- /dev/null +++ b/research/helper-handoff-r18/screens/relay_def.txt @@ -0,0 +1,34 @@ +forward helpers with a relay point: 0 options 0 {} | (a)-passes helpers 65 span checks 198 +forward deficiency histogram (min #foreign input vars left in h after cancelling with co-framed rows): Counter({8: 22, 13: 12, 7: 12, 12: 6, 10: 4, 9: 4, 6: 3, 11: 2}) + h 531 deficiency,dim,event,|R_F| (6, 6, 13304, 2) + h 532 deficiency,dim,event,|R_F| (6, 6, 13349, 2) + h 535 deficiency,dim,event,|R_F| (6, 6, 13370, 2) + h 544 deficiency,dim,event,|R_F| (7, 5, 13355, 2) + h 556 deficiency,dim,event,|R_F| (7, 5, 13361, 2) + h 547 deficiency,dim,event,|R_F| (7, 5, 13382, 2) + h 559 deficiency,dim,event,|R_F| (7, 5, 13388, 2) + h 474 deficiency,dim,event,|R_F| (7, 6, 13199, 2) + h 477 deficiency,dim,event,|R_F| (7, 6, 13232, 2) + h 530 deficiency,dim,event,|R_F| (7, 6, 13301, 2) + h 529 deficiency,dim,event,|R_F| (7, 6, 13322, 2) + h 519 deficiency,dim,event,|R_F| (7, 6, 13328, 2) + h 520 deficiency,dim,event,|R_F| (7, 6, 13343, 2) + h 533 deficiency,dim,event,|R_F| (7, 6, 13352, 2) + h 523 deficiency,dim,event,|R_F| (7, 6, 13376, 2) +backward helpers with a relay point: 0 options 0 {} | (a)-passes helpers 172 span checks 347 +backward deficiency histogram (min #foreign input vars left in h after cancelling with co-framed rows): Counter({2: 67, 5: 20, 6: 18, 7: 17, 4: 13, 21: 4, 60: 4, 45: 4, 16: 4, 37: 4, 8: 4, 3: 3, 24: 2, 36: 2, 57: 2, 41: 1, 13: 1, 22: 1, 32: 1}) + h 525 deficiency,dim,event,|R_F| (2, 3, 11813, 1) + h 524 deficiency,dim,event,|R_F| (2, 3, 11864, 1) + h 537 deficiency,dim,event,|R_F| (2, 3, 11867, 1) + h 502 deficiency,dim,event,|R_F| (2, 3, 11892, 1) + h 549 deficiency,dim,event,|R_F| (2, 3, 11905, 1) + h 548 deficiency,dim,event,|R_F| (2, 3, 11914, 1) + h 561 deficiency,dim,event,|R_F| (2, 3, 11927, 1) + h 560 deficiency,dim,event,|R_F| (2, 3, 11936, 1) + h 513 deficiency,dim,event,|R_F| (2, 3, 11955, 1) + h 536 deficiency,dim,event,|R_F| (2, 3, 12113, 1) + h 472 deficiency,dim,event,|R_F| (2, 3, 12492, 2) + h 473 deficiency,dim,event,|R_F| (2, 3, 12497, 2) + h 482 deficiency,dim,event,|R_F| (2, 3, 12548, 2) + h 483 deficiency,dim,event,|R_F| (2, 3, 12553, 2) + h 493 deficiency,dim,event,|R_F| (2, 3, 12609, 2) diff --git a/research/helper-handoff-r18/screens/spanb.py b/research/helper-handoff-r18/screens/spanb.py new file mode 100644 index 0000000000..0b5dbe7d12 --- /dev/null +++ b/research/helper-handoff-r18/screens/spanb.py @@ -0,0 +1,24 @@ +"""Necessary condition for any helper hand-off inserted at time t into the #4610 word with the prefix +unchanged: h_in (trivial word) in the Q-span of the rows of the registers sharing h's frame F at t, +with h at an intermediate frame (start dim < dim F) and climbs ahead. Checked exactly whenever a +register at F changes (row or arrival).""" +from relay2 import * +def screen(side): + back = side == 'backward'; prog = w['programs'][side] + rows = {('L', i): v.identity(('L', i)) for i in range(632)}; rows['C', 0] = v.identity(('C', 0)) + cur = {('L', i): f for i, f in enumerate(c['start'])}; cur['C', 0] = None + left = Counter(e[2] for e in prog if e[0] == 'climb'); hits = defaultdict(set); checks = 0 + for j, e in enumerate(prog): + t = step(rows, cur, e, back) + if e[0] == 'climb': left[e[2]] -= 1 + if not t: continue + F = cur[t]; RF = [r for r in rows if cur[r] == F] + for h in RF: + if h[0] != 'L' or h[1] < 224: continue + i = h[1] + if left[i] <= 0 or not dim(c['start'][i]) < dim(F): continue + checks += 1 + if in_span({(h, ()): Q(1)}, [rows[r] for r in RF]): hits[i].add((dim(c["start"][i]), dim(F), dim(c["final"][i]))) + print(side, 'span checks', checks, 'helpers satisfying the necessary span condition:', len(hits), dict(list(hits.items())[:10]), flush=True) +if __name__ == '__main__': + for s in ('forward', 'backward'): screen(s) diff --git a/research/helper-handoff-r18/screens/spanb.txt b/research/helper-handoff-r18/screens/spanb.txt new file mode 100644 index 0000000000..4cf18d714d --- /dev/null +++ b/research/helper-handoff-r18/screens/spanb.txt @@ -0,0 +1,2 @@ +forward span checks 12089 helpers satisfying the necessary span condition: 172 {224: {(0, 1, 9)}, 240: {(0, 1, 9)}, 256: {(0, 1, 9)}, 505: {(0, 3, 9)}, 400: {(0, 1, 9)}, 404: {(0, 1, 9)}, 405: {(0, 1, 9)}, 413: {(0, 1, 9)}, 416: {(0, 1, 9)}, 420: {(0, 1, 9)}} +backward span checks 12243 helpers satisfying the necessary span condition: 65 {320: {(0, 8, 9)}, 304: {(0, 8, 9)}, 507: {(0, 6, 9)}, 508: {(0, 6, 9)}, 496: {(0, 6, 9)}, 497: {(0, 6, 9)}, 474: {(0, 6, 9)}, 475: {(0, 6, 9)}, 486: {(0, 6, 9)}, 487: {(0, 6, 9)}} diff --git a/research/helper-handoff-r18/verify.sh b/research/helper-handoff-r18/verify.sh new file mode 100644 index 0000000000..10f39ecbc5 --- /dev/null +++ b/research/helper-handoff-r18/verify.sh @@ -0,0 +1,19 @@ +#!/bin/sh +# Reproduce every output of research/helper-handoff-r18 and compare with the stored expected outputs. +# Run from the repository root: sh research/helper-handoff-r18/verify.sh (python3.12, stdlib only; ~45 s) +set -e +PY=${PY:-python3.12} +D=research/helper-handoff-r18 +T=$(mktemp -d) +cd $D +OPENBLAS_NUM_THREADS=1 $PY -B control_4610.py > $T/control_4610.txt +OPENBLAS_NUM_THREADS=1 $PY -B gadget.py > $T/gadget.txt +OPENBLAS_NUM_THREADS=1 $PY -B price_handoff.py > $T/price_handoff.txt +cd screens +for f in relay relay2 spanb presub; do OPENBLAS_NUM_THREADS=1 $PY -B $f.py > $T/$f.txt; done +OPENBLAS_NUM_THREADS=1 $PY -B relay_def.py forward backward > $T/relay_def.txt +cd .. +for f in control_4610 gadget price_handoff; do cmp $T/$f.txt $f.txt; done +for f in relay relay2 spanb presub relay_def; do cmp $T/$f.txt screens/$f.txt; done +grep -q PASS_NREPLAY_CONTROL_4610 control_4610.txt && grep -q PASS_HANDOFF_GADGET gadget.txt && grep -q PASS_PRICE_HANDOFF price_handoff.txt +echo PASS_HELPER_HANDOFF_R18