diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index da91d3c62..6d29b5806 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,837 @@ +## 2026-09-16 (14) — E-THE-NET-ARM-RANKED-ON-A-PARTIAL-SUM-AND-ITS-ONLY-APPARENT-SIGNAL-WAS-THAT-BUG-1 — entry (13)'s NET row is re-measured; the arm is a census on BOTH arms, not one + +**Status:** MEASURED — re-run of `D-HXP-8` arm 1 after a one-line ranking fix in +`crates/perturbation-sim/examples/tictactoe_raumgewinn.rs`. **Confidence:** HIGH +(the fix is a derivation from `TierFloors::stack_early_exit`'s own return value, +and the re-measured numbers are self-consistent with entry (13)'s structural +result rather than against it). + +### The defect + +`score()` ranked BOTH arms by `StackResult.stacked`. That field is the +**cumulative sum up to and including the exit tier only** — it equals the true +full stack iff `exit_tier == 3`. Entry (13)'s own meter note already said a +partial is not a bound under signed terms, and measured that early exit changed +the top move in **10.13 %** of positions. The prose was right; the ranking code +did not honour it. So for `Arm::Net`, F1 was being scored against a quantity that +is a genuine partial sum in ~1 position in 10 — and with signed per-tier terms a +later negative tier can pull the true total below what an earlier positive +partial suggested. + +Fixed: `Arm::Agreement` keeps ranking by the early-exit value (its terms are +non-negative, so the partial IS a bound); `Arm::Net` ranks by the full stack. +`tied` and `distinct` follow the same match, since they must be computed over +whatever F1 actually ranked by. **F2 is deliberately untouched and stays +arm-independent** — it compares the same two values for both arms by +construction. + +### What the re-measurement says, and it is the uncomfortable half + +| quantity | entry (13) NET | re-measured NET | AGREEMENT (unchanged) | +|---|---|---|---| +| F1 det / tie-aware | 0.5677 / 0.5655 | **0.5865 / 0.5797** | 0.5865 / 0.5797 | +| mean distinct stacked | **1.157** | **1.000** | 1.000 | +| F3 degree-1 drop | +0.0152 | **+0.0011** | +0.0004 | +| null shuffle, 20 seeds | 0.5768 [0.5617, 0.5899] | **0.5797 [0.5797, 0.5797]** | 0.5797 [0.5797, 0.5797] | +| F2 | 0.8987 | 0.8987 | 1.0000 | + +**The `1.157 → 1.000` is the finding.** Entry (13) reported the mean number of +distinct FULL-stack values as `1.000` on both arms — correctly — but the NET +arm's *scored* quantity showed `1.157`, i.e. it looked like the NET arm had at +least a little discrimination where AGREEMENT had none. It did not. That 0.157 +was entirely the partial-sum artifact, and with the fix NET is exactly the +census, identically to AGREEMENT. + +Two corroborating collapses, neither of which was arranged: the null shuffle +degenerates to a **single point** (shuffling the rails changes nothing, because +the full stack is the board census regardless of rail topology — which is the +F0-degenerate claim restated from a direction entry (13) did not test), and F1 +lands exactly on the random-move baseline `0.5797`. + +### What this supersedes + +⊘ Entry (13)'s NET row and every number derived from it. The structural verdict +— **F0 DEGENERATE, F1/F2/F3 are DATA not verdicts** — is UNCHANGED and is in +fact strengthened: it now holds identically on both arms instead of holding on +one while the other showed a small unexplained spread. + +⊘ The same-day `⊘ UNPINNED` block proposing a flat-vs-DROP tolerance for NET's +`+0.0152`. That block did honest work on a number that no longer exists: the +re-measured drop is **+0.0011**, flat by any tolerance, so the question it was +opened to adjudicate is closed by measurement rather than by a threshold. The +tolerance proposal itself stays on file as unpinned and unused. + +### Why it was invisible + +`0.5677` is not an implausible number next to a `0.5797` baseline, and `1.157` +reads as "barely any discrimination" — which is the same story the entry was +already telling. **A bug whose output agrees with your conclusion is the hardest +kind to see**, and nothing in the suite could have caught it: there was no test +that distinguished "ranks by early-exit" from "ranks by full stack" for the NET +arm at all. There is now (`net_arm_ranks_by_the_full_stack_not_the_early_exit_partial`, +disable-verified: reverting `top` to `top_early` turns it red), plus 7 more unit +tests on the probe's own primitives (negamax exact values, the symmetry-class +relation, the three `Horizon` cases, the rail shuffle's permutation invariants). + +**Cross-refs:** entry (13) (`E-RAUMGEWINN-NEEDS-A-HORIZON-SMALLER-THAN-THE-BOARD-…`) +— structural result stands, NET numbers superseded; `TierFloors::stack_early_exit` +in `crates/perturbation-sim/src/rolling_floor.rs` (the return value the fix is +derived from); D-HXP-8 arm 2 (a board larger than the rails' horizon) remains +the unblocked next arm, unchanged by this. + +## 2026-09-15 (13) — E-RAUMGEWINN-NEEDS-A-HORIZON-SMALLER-THAN-THE-BOARD-TIC-TAC-TOE-HAS-NONE-SO-ARM-1-IS-F0-DEGENERATE-NOT-A-KILL-1 — D-HXP-8 arm 1 ran; the pre-registration lacked a fixture-validity gate, and the fixture failed it + +**Status:** MEASURED — `D-HXP-8` arm 1 (tic-tac-toe) RUN. Probe +`crates/perturbation-sim/examples/tictactoe_raumgewinn.rs` +(`cargo run --manifest-path crates/perturbation-sim/Cargo.toml --example tictactoe_raumgewinn --release`), +committed with the F0 gate that this entry adds to the pre-registration of (12). +**Confidence:** HIGH on the structural result — it is a derivation confirmed by +measurement (mean distinct FULL-stack values per position `1.000` on both arms); HIGH +on the meter note (measured `0.1013`); the next-arm criterion is a proposal. + +### What ran + +Full negamax over the 4520 reachable non-terminal positions (627 classes up to +symmetry; the "765" in (12) counts the terminal classes too — the probe scores the +627 that have a move); a move is optimal when it preserves the value. Rails = Chebyshev +rings 1..4 (rings 3 and 4 empty on 3×3), floors preheated on every (position, empty +cell) pair with `k = 2`, `stack_early_exit` on a clone per candidate, rank by the +stacked value. Random-move baseline `0.5797`. + +| arm | F1 det / tie-aware | F2 early == full | all tied | distinct stacked / FULL | F3 degree-1 tie-aware (drop) | shuffled-rail null, 20 seeds | +|---|---|---|---|---|---|---| +| AGREEMENT (pre-registered) | 0.5865 / **0.5797** | 1.0000 | **1.0000** | 1.000 / 1.000 | 0.5800 (+0.0004) | 0.5797 [0.5797, 0.5797] | +| NET own − opp (exploratory) | 0.5677 / 0.5655 | **0.8987** | 0.8434 | 1.157 / 1.000 | 0.5808 (+0.0152) | 0.5768 [0.5617, 0.5899] | + +Read naively, the pre-registered arm sits EXACTLY on the baseline and the null collapses +to a point — the KILL rule of (12) ("F1 at chance") would fire. It does not fire, and +the reason is the finding. + +### F0 — the horizon exhausts the board, so the stack is a census + +On 3×3 every cell's rings 1 ∪ 2 reach all 8 other cells (measured: `reach 8..=8 of 8`, +`horizon exhausts the board on 9/9 cells`). Both arms are ring-ADDITIVE — tier `r` sums +a per-cell term `f(b[j])` over `j ∈ ring_r(m)` — so the full stack of candidate `m` is +`Σ_{j≠m} f(b[j]) = Σ_j f(b[j]) − f(E)`, the same number for every empty `m`. The full +stack cannot rank anything; it is the board census. Measured, not assumed: mean distinct +FULL-stack values per position `1.000` on both arms, every candidate tied in `1.0000` +of positions on AGREEMENT. F1 = baseline, F2 = 1 and F3 flat are all the census +signature — not a KILL, not a pass, not a reading. **The fixture is degenerate.** + +This is (11)'s elephant : Wal at board scale: popcount is position-blind, and when the +horizon reaches the whole world, position-blindness is total blindness. Raumgewinn is a +LOCALITY property — it exists only where a cell's rails reach strictly less than the +board. The number that decides a fixture is `reach / (board − 1)`; the gate is computed +from the rails alone before any position is scored, and its silent twin is the degree-1 +rails (reach 1 of 8 — the probe asserts the gate stays silent there, so it discriminates). + +**F0 joins the pre-registration from here on.** A readable arm needs the board diameter +strictly greater than twice the deepest ring, so that not even the centre sees everything: +Gobang 15×15 (Chebyshev diameter 14 > 8 ✓), Go 9×9 (Manhattan 16 > 8 ✓), Hex ≥ 7×7 (hex +diameter 12 > 8 ✓). Hex 5×5 fails it (diameter 8 — the centre reaches every cell) and +must not be the next arm. Tic-tac-toe is retired as a Raumgewinn falsifier; it remains +the ground-truth harness the larger boards reuse (negamax, symmetries, the F1/F2/F3 +scoring are board-size-agnostic in the probe). + +### The meter note — early exit presumes non-negative stacking + +`stack_early_exit` returns the PARTIAL sum at the exit tier — +`crates/perturbation-sim/src/rolling_floor.rs:239` (`if crossed || band == FloorBand::Alarm {`) +returns `stacked` as accumulated so far — and its doc promises +`rolling_floor.rs:220` (`decision is confident, the finer tiers need not be computed`). +That promise holds only when the remaining tiers cannot LOWER the reading, i.e. for +non-negative intensity (the shipped caller, `weyl_over_fiedler`, is non-negative). The NET +arm is signed (own − opp): a partial is then not a bound on the full stack, and the early +exit changed the top move in **10.13 %** of positions (F2 `0.8987`) while AGREEMENT held +`1.0000`. On this fixture that spread is the ONLY spread (FULL stack `1.000`), which is +what makes the mechanism unambiguous. Filed in `TECH_DEBT.md` (the premise is unstated +in the doc, not a bug in the shipped non-negative use). Consequence for D-HXP-8: the +pre-registered arm stays AGREEMENT; a signed arm runs full-stack, or the meter grows a +signed-safe exit (exit only when the remaining tiers' maximum magnitude cannot reverse +the ranking — a bound, not a guess). + +### Process + +The pre-registration in (12) had a KILL rule that assumed the fixture could read F1. +Every falsifier from here on carries F0 before F1: **can this fixture distinguish the +candidates at all?** — measured as the distinct-value count of the quantity being ranked, +before the ranking is read. The `all candidates tied 1.0000` line existed only because +the anti-vacuity counters were added after the first run showed F1 == baseline to four +decimals; had the KILL been read at face value, a correct claim would have been retired by +a fixture that could not test it. + +Home: `hexagon-plasticity-v1.md` §12a (appended), `STATUS_BOARD` `D-HXP-8` (In +progress — arm 1 RUN, F0 degenerate; next Gobang 15×15 / Hex ≥ 7×7 / Go 9×9), +`LATEST_STATE` (10), `TECH_DEBT` 2026-09-15 (early-exit premise). + +## 2026-09-15 (12) — E-POPCOUNTS-UPPER-RANGE-SIMILARITY-IS-THE-HEXAGONS-RAUMGEWINN-AND-BOARD-GAMES-MAKE-IT-FALSIFIABLE-1 — the counterweight to (11): the same position-blindness is territory on the hexagon substrate, the toolkit for it ships under the operator's own words, and the games give it ground truth + +**Status:** RULING — operator, verbatim, two messages: *"Der Vorteil von HDR popcount +stacking early exit Belichtungsmesser statistical confidence interval thresholds +preheating rolling floor bucket assignment ist jedoch daß es für hexagon Substrate +hilfreich ist — der sprichwörtliche Raumgewinn beim Go boardgame."* and *"Und +witzigerweise müsste tiktaktoe gobbang, go damit sogar falsifiable sein."* The census is +a read of the tree; the pre-registration at the end is mine, on the operator's proposal. +**Confidence:** HIGH on the census and on the `head2head` mapping (shipped code); the +falsifier is pre-registered, NOT run — and the board already holds one measured +counterexample it must beat (E-Q8). + +### The pair, in shipped vocabulary + +(11) said popcount finds *elephant : Wal* because it is position-blind. This entry says: +on the hexagon that is the point. The operator's Go framing is already in the contract — +`lance_graph_contract::head2head::WinnerCriterion` (D-H2H-1, 2026-05-31): **infight** ≈ +`DissonanceMin` (the tightest match wins), **Raumgewinn** ≈ `SupportSpread` (the widest +distinct support wins). Read against (11): lcp asks how DEEP one agreement goes — +infight; stacked popcount asks how MANY neighbours agree — Raumgewinn. Elephant : whale +is territory: the class's influence reaches the whale. Neither metric is wrong; they are +`head2head`'s two criteria, and the substrate already competes them. + +### The toolkit, by name — every word in the operator's list is a shipped surface + +- **HDR bands, Belichtungsmesser, CI thresholds, recalibration** — + `ndarray::hpc::cascade::Cascade::{calibrate, expose, observe, recalibrate}`: + `calibrate` takes a distance sample to μ/σ, `expose(distance) -> Band` IS the light-meter + reading, `observe` returns a `ShiftAlert`, `recalibrate` takes it; `adaptive_resolution` + picks the band from query entropy × corpus CV; `PackedDatabase::cascade_query` runs it. +- **Popcount stacking, early exit, preheating, rolling floor, bucket assignment** — + `perturbation_sim::rolling_floor::{RollingFloor::{preheat, observe, threshold, z, band}, + TierFloors::{preheat, stack_early_exit}}`: the L1..L4 tiers as an HDR popcount-stacking, + early-exit cascade over a self-calibrating μ + kσ floor; `band` → `FloorBand`; the + bf16-hhtl-terrain knowledge doc names it *bucket-ROUTING (the rolling floor), NOT + reconstruction*. +- `belichtungsmesser()` — 7 sample points `[0,19,41,59,79,101,127]` → (mean, sd), the SD + entropy gate (`agi-stack-cross-repo.md`); `holo.rs`: batch Wasserstein search with an + early-exit cascade. +- The operator's exact phrase is already a doctrine line: + `observer-effect-tfpn-doctrine.md` — *"early exit, statistical confidence-interval + thresholds, preheating + rolling floor bucket — the Belichtungsmesser reading."* + +Nothing to build for the toolkit. What is unbuilt is the EVIDENCE that it helps the +hexagon. + +### The caveat the board already holds + +`E-Q8-THE-SIX-DOES-NO-WORK-A-DEGREE-ABLATION-COLLAPSES-THE-HEX-OVERLAYS-ENTIRE-ADVANTAGE-1` +(2026-08-31): B beat A and the RAND null on every metric — and at degree 1 it scored +identically to four decimals; *"the six does nothing; B is a bigram successor table."* +That is not a refutation of six-neighbourness; it is the finding that THAT task never +consulted more than the first neighbour. "Helpful for hexagon substrates" is therefore, +on the board today, a claim with one measured non-result behind it — and the operator's +second message is the answer: pick tasks whose ground truth NEEDS the neighbours. + +### The games as the falsifier — pre-registered, not run + +Why games: exact ground truth (solved values, legal territory); rules that ARE neighbour +relations; and the degree-ablation twin built in — a line needs aligned neighbours, a +liberty count needs all four, a Hex connection needs six. E-Q8's failure mode cannot +pass silently here: on these tasks degree 1 is provably insufficient. + +Lattices, stated so the six is not oversold: tic-tac-toe and Gobang are square with 8 +directions; Go is square with 4-adjacency (territory = 4-connected flood fill); **Hex is +the six-neighbour game** — the exact match for six rails as six neighbour pointers, with +the square games as 4- or 8-subgraphs. Order: tic-tac-toe (solved, a draw; 765 positions +up to symmetry), Hex on small boards (first-player win by strategy stealing, explicit +solutions on small boards), Gobang (free-style 15×15 a first-player win, Allis 1994), Go +(small boards solved; end-position territory is pure flood-fill ground truth). + +The claim, operationalised: each cell a unit whose six rails are its neighbours, the +stone colour in the payload; a position evaluated by popcount stacking over the rails' +agreement ring by ring (the HDR stack), `TierFloors::stack_early_exit` deciding when the +reading is settled, floors preheated from a position sample. + +- **F1 — correctness.** Tic-tac-toe: the top-ranked move is value-preserving in ≥ 95 % of + the 765 positions, chance level measured by a shuffled-rail null, not assumed. Go end + positions: stacked territory == flood-fill scoring — an equality, no tolerance. + +> ⊘ **CORRECTED, same day, entry (13).** "The 765 positions" is not the population F1 can +> score — a terminal class has no move to rank, so F1 is undefined on it. Entry (13) runs +> the probe over the 4,520 reachable non-terminal positions (627 classes up to symmetry) +> and says so explicitly: *"the '765' in (12) counts the terminal classes too — the probe +> scores the 627 that have a move."* Read every F1 percentage in this entry, and in (13), +> against 627, not 765. + +- **F2 — economy.** Early exit changes NO verdict (equality against the full stack) and + the mean exposed tiers is below the full depth; the fraction is measured and stated. +- **F3 — the degree ablation, mandatory.** At degree 1, F1 must DROP. Flat = the task did + not exercise the six, and the probe proves nothing (E-Q8 as a gate, not a memory). +- **KILL:** F1 at chance on tic-tac-toe, or F3 flat. + +> ⊘ **UNPINNED, flagged same day.** Neither "DROP" nor "flat" carries a numeric tolerance +> or a rounding rule above. Entry (13) measured AGREEMENT's degree-1 tie-aware F3 at +> 0.5800 against the un-ablated tie-aware value 0.5797 (Δ = +0.0004) and read the whole +> arm through F0 (the fixture is a census, so F1/F2/F3 all read as the census signature) +> rather than against an independent flatness threshold. A rule derivable from the +> fixture's own size (n = 4,520 non-terminal positions, p ≈ 0.58): binomial standard +> error √(p(1−p)/n) ≈ 0.0073, so treat |Δ| < 1 SE (~0.007) as flat and require |Δ| ≥ 2 SE +> (~0.015) to call a genuine DROP toward the 0.5797 chance baseline. Under that reading +> AGREEMENT's +0.0004 is flat; NET's +0.0152 sits at the 2-SE edge but moves AWAY from +> chance, not toward it, so it is not a DROP either. This is proposed here, not +> pre-registered before the run — treat it as UNPINNED until a non-degenerate arm (F0 +> passes) lets the ranking variance, not census noise, decide the tolerance. + +Home: `hexagon-plasticity-v1.md` §12 (appended), `STATUS_BOARD` `D-HXP-8` (Queued). +Precedent for the method: `E-SF-AWARENESS-OPPONENT-ARC-1` ran the operator's Go +Raumgewinn-vs-infight design inputs as five gated chess probes on stockfish-rs — same +discipline, different board. + +## 2026-09-15 (11) — E-POPCOUNT-FINDS-ELEPHANT-WHALE-BECAUSE-IT-IS-POSITION-BLIND-THE-TREES-METRIC-IS-LZCNT-AND-THE-BOARD-ALREADY-FILED-IT-1 — the operator's caveat on (10), and it lands on an open issue + +**Status:** RULING — operator, verbatim: *"Der 'Nachteil' beim popcount ist daß +Ähnlichkeit auch elephant : Wal findet — Ähnlichkeit im oberen Bereich."* The reading +below is mine; the census is a read of the tree; the issue it lands on is already filed +(`ISS-SHARED-PREFIX-TIERS-IS-TIER-COARSE-AND-BRANCHES`). +**Confidence:** HIGH — the ranking inversion is arithmetic (worked below), and the dual +instruction is the one the board's own re-scope already names. + +### Why popcount finds the whale + +A root→leaf code RANKS its positions: the bit at depth 3 outweighs every bit below it. +Popcount does not know that — it counts disagreements wherever they are. Take elephant +and whale sharing `animal · mammal` and parting at the order nibble by ONE bit (`0001` +vs `0011`), and two elephant species sharing everything down to the leaf nibble, where +they differ in all four (`0000` vs `1111`). Popcount says whale 1, sibling 4: the cousin +is "closer" than the sibling. The similarity it reports is real — the shared upper tiers +ARE shared ancestry, *im oberen Bereich* — but it answers *how much do we share*, and +the tree asks *how deep do we agree*. The quotes around "Nachteil" are right: as +generalisation (find the cousin) it is the feature; as retrieval (find the sibling) it is +the defect. + +### The tree's metric is one instruction, and it is the same XOR + +Longest common prefix = the depth of the FIRST disagreement = `lzcnt(u ⊕ self)` in +root→leaf bit order — one instruction, one cycle, next to popcount's one. `>> 2` is the +level, `>> 4` the tier: the canon's *"tier-of-level = level >> 2 — a shift, never a +branch."* On the example: whale parts at depth 8 (nibble 3), the sibling at depth 12 — +sibling nearer, as the tree says. + +The substrate already names this exact measure: `NiblePath::common_prefix_depth` +(`lance-graph-contract`, `hhtl.rs`) — *"the radix-trie nearest-neighbor measure"*, +`E-PANCAKES-IS-RADIX-IS-HHTL` — and the board already carries its branchless form as an +open item, `ISS-SHARED-PREFIX-TIERS-IS-TIER-COARSE-AND-BRANCHES` (re-scope): the shipped +function is a `while … match` nibble walk running per row inside `mailbox_scan`, and the +one-liner is `((a.path ^ b.path).leading_zeros() >> 2).min(a.depth.min(b.depth))`. The +operator's caveat is the SEMANTIC reason that issue matters, beyond the branch count. + +### How (9), (10) and this compose + +| question | metric | instruction | selection | ships | +|---|---|---|---|---| +| how deep do we agree (tree) | `lcp = lzcnt(u ⊕ self)` | LZCNT | `lcp ≥ d` ≡ `(u ⊕ self) ∧ care(d) == 0` — (9), k = 0 | threshold-to-mask: yes (ternary match); per-row depth vector: no | +| how much do we share (exchangeable bits) | `popcount((u ⊕ self) ∧ care)` | POPCNT | `≤ k` — (10) | k = 0: yes; k > 0 fused: no | + +So on the tree rails the `(self, d)` prefix IS the predicate, and popcount's k > 0 is +not wanted there — which narrows (10)'s gap to the carriers whose positions are +exchangeable (planes, bipolar identities): the `I-VSA-IDENTITIES` fence, seen from the +other side. What the tree rails lack is the VECTORISED lcp per row — `lzcnt` over +`u ⊕ self` in the strided 12-byte form, yielding a depth vector (`u8` per row) for +RANKING, nearest = deepest — where the threshold-to-mask form already ships. ndarray has +no `lzcnt` / `leading_zeros` primitive today (census, this session), and a per-row +`u8`-out shape is new: not a mask, not a count, a sibling of `hamming_batch_raw`. + +A middle ground exists: `heel_weighted_hamming` (ndarray) weights popcount per plane; +with weights falling by level it approaches the lexicographic order lcp gives exactly. +Recorded, not recommended — lcp is one instruction and exact. + +### Consequence + +- (10)'s gap is re-scoped: the fused `popcount ≤ k` predicate is for exchangeable-bit + carriers; on tree rails the missing primitive is per-row `lzcnt` (a depth vector), + and its threshold form is already the ternary match. +- `ISS-SHARED-PREFIX-TIERS-IS-TIER-COARSE-AND-BRANCHES` gains its motivation: not merely + branch-free, but the metric under which the sibling outranks the cousin. +- Phase 7: no new arm — `lzcnt` costs what `popcnt` costs; the k > 0 arm stays for the + carriers it applies to. + +> ⊘ **Per (12), same day:** the popcount side is not demoted by this — it is the hexagon's +> Raumgewinn, and lcp / stacked popcount are `head2head`'s two criteria (infight / +> SupportSpread). The games falsifier (`D-HXP-8`) is where the pair gets ground truth. + +## 2026-09-15 (10) — E-POPCOUNT-TIMES-SELF-THE-EXACT-PREFIX-IS-THE-K-EQUALS-ZERO-HAMMING-BALL-AND-THE-FUSED-ROW-PREDICATE-IS-THE-GAP-1 — the operator's one-line generalisation of (9), what of it ships, and where I-VSA-IDENTITIES fences it + +**Status:** RULING — operator, verbatim: *"You could even say it's popcount × self."* +The two readings below are mine and labelled; the census of primitives is a read of the +tree (every name verified); the fence is the substrate's own iron rule, applied. +**Confidence:** HIGH on the census and on the k = 0 identity; MEDIUM on the second +reading; the fence is a consequence of `I-VSA-IDENTITIES`, not a new ruling. + +### Reading 1 — per row: the exact prefix is the k = 0 Hamming ball + +(9)'s selection `(u ⊕ self) ∧ care(d) == 0` is `popcount((u ⊕ self) ∧ care(d)) ≤ 0`. +Generalise the 0 to k and the triple `(self, care, k)` is one predicate family: + +- `care = care(d), k = 0` — the exact stepless prefix: tree distance ≤ d from root, (9); +- `care = all 96, k > 0` — the Hamming ball of radius k around self; +- `care = care(d), k > 0` — a Hamming ball inside the prefix. + +"Distance from root" and "Hamming distance" are the same popcount over different +care masks, both 0..=96. Per row: XOR, AND, POPCNT, CMP. + +**What ships** (ndarray, names as in the tree): `ternary_match_u32_to_mask`, +`ternary_match_u64_to_mask`, `ternary_match_strided_to_mask` — the k = 0 form, one +pass, a mask out. `hamming_distance_raw`, `hamming_batch_raw(query, database, +num_rows, row_bytes) -> Vec`, `hamming_top_k_raw` — per-row DISTANCES, a `Vec`, +not a mask. `masked_popcount_batch(words, mask) -> Vec` — per-word +`popcount(w ∧ mask)`, one XOR short of the row predicate on a u64 lane. mask-risc: +`Pred::MatchU32` / `Pred::MatchU64` (k = 0 only); `Terminal::Count` = +`popcount_batch_u64(mask)`. + +**The gap:** the fused one-pass `popcount((row ⊕ pattern) ∧ care) ≤ k → mask` — a +`hamming_le_*_to_mask` kernel in ndarray (per lane and 12-byte strided) and a +`Pred::HammingLe { lane, pattern, care, k }` in mask-risc. Today k > 0 is two passes +through a 64k-entry distance vector (512 KiB) plus an allocation — exactly the shape the +masking floor exists to remove. Named, not built. + +> ⊘ **Per (11), same day:** scoped to exchangeable-bit carriers (planes, bipolar +> identities). On tree rails popcount is position-blind — it ranks a cousin above a +> sibling — and the missing primitive there is a per-row `lzcnt` depth vector; the +> threshold form is already the ternary match. + +### Reading 2 — per mask: `popcount(mask(self, d)) = |ball(self, d)|` + +The cardinality of self's neighbourhood at depth d is `Terminal::Count` over the (9) +mask — the probe's radius sweep already prints it as `rows` (2^(56−d) on the +perfect-tree lane). Swept over d it is a thought's specificity profile: how many units +share its first d bits. One popcount per 64 rows; nothing to build. + +### The fence — `I-VSA-IDENTITIES`, applied + +Reading 1 with k > 0 is sound only where bit-Hamming IS a distance: fingerprint planes, +bipolar identities, and the tree through `care(d)` with k = 0. It is NOT a distance over +the L4 `palette256²` rails or any CAM-PQ code: two centroid INDICES that differ in every +bit may be neighbours, and the substrate's distance there is the 256×256 LUT (bgz17 +lineage), never a popcount. `I-VSA-IDENTITIES` already forbids superposing content +codes; the same register-loss argument forbids Hamming over them. So "popcount × self" +is exact for k = 0 on every carving, and for k > 0 only on Hamming-meaningful bits — +which bits those are is the ClassView's to say. + +### Phase 7, second arm — pre-registered, not run + +Beside (9)'s cycles/row for the k = 0 strided match: the k > 0 fused predicate. Pass: +≤ 1 cycle/row with `vpopcntq` (AVX-512 VPOPCNTDQ); expect 2–3 cycles/row on AVX2 +through the nibble-LUT popcount. If the fused kernel does not exist by then, the +two-pass form is what gets measured, and the delta IS the cost of the gap. + +## 2026-09-15 (9) — E-A-THOUGHT-MASKS-ITSELF-BY-ITS-DISTANCE-FROM-ROOT-THE-V3-FACET-IS-THE-MASK-AND-THE-RADIUS-IS-STEPLESS-1 — the operator's favourite masking variant; it is already the shape of `MatchU64`, ndarray ships its 12-byte strided form, and the probe measured it stepless + +**Status:** RULING — operator, verbatim: *"Meine Lieblingsvariante ist V3 Format. Jeder +beliebige Gedanke kann sich selbst × Abstand from root 0–96 bit maskieren und somit +eine exakte stufenlose Auswahl treffen mit close to 1 CPU cycle."* — plus FINDING for +the radius sweep (re-runnable). The lowering shape and the one-cycle mechanism below +are derivations; the timing claim is a pre-registered Phase 7 falsifier, NOT measured +here. +**Confidence:** HIGH on the sweep; HIGH that the primitive exists — ndarray ships it; +MEDIUM on the root→leaf bit order inside the 12 bytes, which is the ClassView's +carving and lives in the care table, not in a shift. + +### The variant, as the substrate sees it + +A mask is a pair `(self, d)`: the unit's own 96-bit payload as the pattern, its +distance from root `d ∈ 0..=96` as the radius, `care(d)` = the first d bits in +root→leaf order, and the selection is every unit `u` with +`(u.payload ^ self.payload) & care(d) == 0`. Three properties, each literal: + +- **exact** — a ternary equality; no threshold, no approximation, no 8 KiB object + unless the caller wants the result materialised; +- **stepless** — every d is a distinct selection: d = 0 is the class, d = 96 the unit + itself. Nibble boundaries are where the codebook's centroid cells sit (OGAR: *1 + nibble = 1 level of the 16-ary tree*) — a fact about MEANING; the mask cuts anywhere; +- **~1 cycle** — one AND and one compare per row, both vectorised. + `ndarray::simd::ternary_match_strided_to_mask(bytes, first_offset, stride_bytes, + count, pattern: &[u8; 12], care: &[u8; 12], out_words)` IS this call over a + 16-byte-strided table: 12-byte pattern, 12-byte care, one mask word per 64 rows. + Shipped. mask-risc's `LaneRef::U64` doc names the strided `Operand` as the IR's + own gap (PR4/PR5); `Pred::MatchU64` / `MatchU32` are the same operation on one lane, + and `Filter::prefix_u64(col, self, d)` is `(self, d)` on that lane today. + +The root→leaf order of the 96 bits is not memory order — each rail is LE `hi:lo`, and +which bits are "closer to root" is the ClassView's carving — so `care(d)` is a +97-entry table per carving, not `!0 << (96 − d)`. The ternary match does not care +which, and that is the point: any bit order, any d, one instruction. + +### What the sweep measured + +`examples/adaptive_order_probe.rs`, radius d on the 16 row bits of `i << 8` +(d = 40 is every row, d = 56 is one), the prefix leading the clustered conjunction: + +| d | rows | words skipped | blocks skipped | +|---|---|---|---| +| 40 | 65 536 | 0.00 % | 0.00 % | +| 41 | 32 768 | 50.00 % | 50.00 % | +| 44 | 4 096 | 93.75 % | 93.75 % | +| 47 | 512 | 99.22 % | 99.22 % | +| 48 | 256 | 99.61 % | **99.61 %** | +| 49 | 128 | 99.80 % | 99.61 % | +| 50 | 64 | **99.90 %** | 99.61 % | +| 51…56 | 32…1 | 99.90 % | 99.61 % | + +- Every d selects exactly 2^(56−d) rows — asserted per step. Stepless and exact. +- The skip is a step function of the UNIT, not of d: words gain until one live word + (d = 50), blocks until one live block (d = 48); between 48 and 50 the units part + ways. `/50` — which (7) called illegal and (8) "not a tile cell" — is the radius at + which the word-skip saturates. Legal, exact, one of 97. +- The (5) §3 family `1 − 2^(50−P)/1024`, labelled CONJECTURE there and withdrawn in + (7), is measured across 40..=56 and holds for words down to the word floor; its block + twin saturates two bits earlier. Restored as FINDING. + +### Corrections this makes + +- (7), (8): *"legal prefixes are nibble-multiples"* — nibble alignment is codebook + structure, not a legality condition on selection. `/48` stays in the probe's + clustered regime as a representative radius, not the only legal one; the sweep now + carries every d. +- (8) sub-reading (ii), *six per-rail prefixes*: the operator's variant is ONE radius + over the whole payload in root→leaf order, self-referential. Sub-reading (i), six + needles, is a different object — a survivor set — and stays as the sparse arm's + shape. + +### Pre-registered falsifier — Phase 7, not run + +`ternary_match_strided_to_mask` over a 64k × 16 B table, release build, ≥ 10 runs, +cycles per row from `rdtsc`. **Pass:** ≤ 1 cycle per row sustained (the SIMD kernel +should land well under: four u64 lanes per compare on AVX2). **Fail:** > 1 cycle per +row — then "close to 1 CPU cycle" is true of the primitive and not yet of the +substrate end to end, and the strided `Operand` in mask-risc is what closes it. + +## 2026-09-15 (8) — E-THE-RAIL-IS-A-NEEDLE-NOT-A-MASK-256-BY-256-IS-THE-EXACT-ROW-ADDRESS-AND-A-MASK-OVER-THE-AREA-IS-ANOTHER-OBJECT-1 — operator clarification; (7)'s "hi byte = skip unit" is withdrawn as the ruling's meaning, and its measurement is kept as data + +**Status:** RULING — operator, verbatim (2026-09-15, five lines): *"Ich meine 64k sind +2 byte. 256:256 sind 2 byte für die exakte SoA inna given table. Das gilt nur für needle +in a haystack x table. Für Maske über 64k als Fläche bräuchte es entsprechend mehr. Eine +Mögliche Lesart für masking wäre 256:256⁶, also genau 96 bit, oder bitpacked 64k."* +Everything below the ruling is my reading of it, labelled as such. +**Confidence:** HIGH that (7)'s derivation was not what was meant — the operator says so +in the first word. MEDIUM on the two sub-readings of `256:256⁶`: the operator called it +*eine mögliche Lesart*, and it stays open here. + +### What the ruling says, read plainly + +1. **The rail is an address, not a mask.** `u8:u8` = 2 bytes = the exact SoA row in a + given 64k table. 256 × 256 = 65 536: every value a row, every row a value. THAT is + "kein Rest" — a bijection between rail values and rows — not a tiling of the mask + into skip units. It holds for the needle-in-a-haystack × table case: one value, + one row. +2. **A mask over the 64k AREA is a different, larger object.** Bitpacked it is 65 536 + bits = 8 KiB = 1 024 words = 256 four-word blocks, and it tiles with no remainder in + either unit. The rail does not dictate which unit an executor skips in; nothing in + the ruling does. +3. **`256:256⁶` = 96 bits is the facet payload — the operator's candidate for a masking + reading.** Two ways to read it, both recorded, neither ruled: (i) six exact + needles — a sparse survivor set of at most six rows, in the 12 bytes the facet + already has; (ii) six per-rail prefixes — a product cell in the six-rail tile space, + a mask given by predicate rather than by bits. Reading (i) is the sparse arm the A1 + falsifier is missing, in the substrate's own register: D-GTM-0n's *below ~0.1 % + active, switch to sparse* is below 65 rows of 64k, and six needles are 0.009 %. + +### What (7) got wrong, and what of it stands + +- **Wrong as the ruling's meaning:** *"the rail's unit is its hi byte"*, *"a quarter + block is a remainder"*, *"count in the unit the address is carved in"*. The address + is not carved into the mask at all. Withdrawn; (7) is ⊘-regraded in place. The same + sentences had been pushed as canon in `d80b802` — in the probe header, `lib.rs`'s + `and_by_skip` doc, the prefix test's comment and mask-risc's `MaskOp::Pred` doc — and + are corrected in this commit to the reading above. +- **Stands as data:** the two-unit measurement. 64-row words are the executor's unit; + 256-row blocks are the 2-nibble prefix cell of the OGAR tier tile (`OGAR/CLAUDE.md` + "Tier interpretation — 256×256 CENTROID TILE": a 4-ary hierarchy per byte) — a + legitimate coarser skip unit, not the rail's. Clustered `/48`: 99.61 % in both + units; selective: 80.66 % words / 61.91 % blocks, written order 0 blocks; the ramps. + The `/48` cut stays: it is nibble-aligned under the tile canon, which is a separate + and older ruling; `/50` (2.5 nibbles) is not. +- **Stands, restated:** the probe's `N == 256 * 256` assert now says what it is — the + table is exactly 2-byte addressable — rather than "no fractional block". +- **Stands from (5)/(7):** the clustered regime at `/48` sits above the 0.1 % bound + (0.177 %); the selective regime (0.055 %) is under it and the probe has no sparse arm. + +### Consequence + +The missing sparse arm now has a shape and a home. Shape: a needle list of `u16` row +ids — the very thing a rail value is. Home: Phase 7's *very-sparse* density arm +(task #6), measured against the bitpacked sweep at 6, 36 and 116 survivors, where six +is the count the facet register itself can hold. Not built here; recorded so the arm +is built against the operator's reading and not against mine. + +> ⊘ **Same day, entry (9):** the operator's favourite is neither sub-reading — it is +> `(self, d)`, ONE stepless radius over the whole 96-bit payload in root→leaf order, +> self-referential; sub-reading (i), six needles, stays as the sparse arm's shape. + +## 2026-09-15 (7) — E-256-BY-256-IS-EXACTLY-64K-THE-RAILS-SKIP-UNIT-IS-ITS-HI-BYTE-AND-A-QUARTER-BLOCK-IS-A-REMAINDER-1 — operator-ruled; the `/50` cut read across `u8:u8`, and re-measured on the byte boundary the clustered regime moves ABOVE the density bound + +**Status:** RULING — operator, verbatim: *"256:256 is exactly 64k. Es darf gar keinen +Rest geben."* — plus FINDING for everything measured below: the probe now prints both +units and the ramp, re-runnable. Reading the ruling as *the rail's hi byte is the skip +unit* is my derivation and is labelled as such. +**Confidence:** HIGH on the numbers. The derivation is the only reading under which +"no remainder" and the `u8:u8` canon (two separate bytes, never widened — +`E-V1-TAIL-FORBIDDEN-V3-IS-CONTENT-BLIND-1`) are satisfied by one skip unit. + +> ⊘ **Superseded the same day by the operator's own clarification — entry (8).** The +> reading *hi byte = skip unit, quarter block = remainder* was mine, and it is not what +> was meant: the rail is the exact row ADDRESS of a 64k table (2 bytes ↔ 65 536 rows — +> that bijection is the "no remainder"), not a mask; a mask over the area is a larger +> object, and the rail dictates no skip unit. The measurements below stand as data in +> two units (words; 256-row blocks = the tier tile's 2-nibble cell), the `/48` cut +> stands under the tile canon, and the "rule" at the end is withdrawn. +> ⊘ And per (9): `/48` is a representative radius, not the only legal one — the +> operator's variant makes the radius STEPLESS; `/50` is where the word-skip saturates. + +### The ruling, and what it rules out + +A rail is `u8:u8`: 256 × 256 = 65 536 rows — exactly the 64k slab, exactly the A1 +probe's `N`. Its hi byte addresses 256 blocks of 256 rows; a block is four 64-row +words, one 256-bit vector. "No remainder" cuts two ways: + +- **Addressing.** A prefix on a rail is a whole number of hi-byte cells or it is not a + rail address. `/48` on the probe's `i << 8` lane pins the hi byte — one block. The + `/50` the (5) entry measured pins two more bits by reading across the two bytes as + if they were a `u16`, and selects a QUARTER block. That quarter is the remainder. +- **Counting.** A skip counted on a rail is counted in blocks. The 64-row word is the + facade's machine unit — `MaskOp::Pred`'s doc already lets an executor skip coarser + chunks with an identical result — but as an ADDRESSING unit it straddles the lo byte + (6 bits against a nibble cascade), and a block with one live word is live, not + three-quarters dead. + +### Re-measured on the byte boundary (`crates/lance-graph-quack/examples/adaptive_order_probe.rs`, both units, ramp printed) + +| regime | survivors | words, worst → best | 256-row blocks, worst → best | +|---|---|---|---| +| selective | 36 (0.055 %) | 5.66 % → 80.66 % | **0.00 % → 61.91 %** | +| moderate | 14 311 (21.8 %) | 0 → 0 | 0 → 0 | +| permissive | 61 777 (94.3 %) | 0 → 0 | 0 → 0 | +| clustered, `/48` | **116 (0.177 %)** | 0.00 % → **99.61 %** | 0.00 % → **99.61 %** | + +Three things the byte boundary changes: + +1. **99.61 % = 1 − 1/256, predicted before the run, held.** Words and blocks agree + exactly on the clustered regime (4 080 / 4 096 and 1 020 / 1 024): one live block, + four live words. The ramp is `[4080, 3060, 2040, 1020, 0]` words / + `[1020, 765, 510, 255, 0]` blocks — monotone, linear, in both units. The (5) + entry's 99.90 % was the quarter-block cut's number: correct for that cut, and that + cut is not a rail address. + +> ⊘ **CORRECTED, same day.** The printed totals do not match this entry's own unit +> definitions. Above: "a block is four 64-row words, one 256-bit vector," on +> N = 256 × 256 = 65,536 rows. That arithmetic gives **1,024** total 64-row words +> (65,536 / 64) and **256** total 256-row blocks (65,536 / 256) — not 4,096 and 1,024. +> A one-live-block result under THOSE totals is **1,020 / 1,024 words skipped** and +> **255 / 256 blocks skipped**, both = 99.61 % = 1 − 1/256, matching this section's own +> headline claim. The printed pair — "4,080 / 4,096" labelled words, "1,020 / 1,024" +> labelled blocks — has totals (4,096 and 1,024) that are one granularity finer than +> each label: 4,096 is the total for a 16-row unit (not defined anywhere else in this +> entry), and 1,024 is the total for the entry's own 64-row WORD definition, not its +> 256-row BLOCK definition. Both mislabeled pairs reduce to the identical 99.61 % ratio +> (4,080/4,096 = 1,020/1,024 = 255/256), which is why the qualitative claim — words and +> blocks agree, ceiling = 1 − 1/256 — survives; the printed raw counts and their column +> labels do not. Cite the percentage, not these counts, until the labels are checked +> against `adaptive_order_probe.rs`'s actual column definitions. +2. **The clustered regime sits ABOVE D-GTM-0n's 0.1 % bound, not under it.** 116 + survivors fill one block: 0.177 % active. The (5) entry's "both lever regimes sit + under the bound" was an artifact of `/50` (31 survivors). What survives: the + SELECTIVE regime (0.055 %) is under the bound and the probe still has no sparse arm + — the missing-arm finding stands, narrowed to that one regime. +3. **In the rail's unit the selective regime's written order skips NOTHING.** 232 dead + words of 4 096 in the written order — and 0 dead blocks: scattered survivors leave + no 256-row block empty until the two selective conjuncts have run. Best drops + 80.66 → 61.91 %. The word count flattered the lever by 19 points on the scattered + regime; on the contiguous regime it was exact. + +Sharper than before, too: the clustered regime has three times the selective regime's +survivors and skips MORE — 116 vs 36 rows, 99.61 vs 80.66 % of words, 99.61 vs 61.91 % +of blocks. Ranked by selectivity the two come out backwards. That replaces the (5)-era +"near-identical counts, 19 points apart" argument, which was also `/50`'s. + +### Where it landed + +- The probe: `dead_blocks`, `skipped → (words, blocks, count)`, `/48`, a + `const _: () = assert!(N == 256 * 256 && BLOCKS * BLOCK_WORDS == WORDS)` that says + "no remainder" in code, and the ramp printed so the quoted figure is a measurement. +- `lib.rs`: the `and_by_skip` doc table carries both units and the new figures; the + crate doc's 99.90 → 99.61; the prefix-halving test keeps `/49` and `/50` as + COMPARATOR arithmetic (a ternary match pins any care mask) and says which prefixes + are rail cells. +- mask-risc `MaskOp::Pred` doc: one paragraph naming the rail's unit. +- Board: (5) ⊘-regraded in place at both affected sections; matrix §8a and D-QCK-9 + ⊘-annotated; `LATEST_STATE` 2026-09-15 (4). + +### The rule + +**Count in the unit the address is carved in.** A skip fraction measured in a unit +finer than the rail's cell reports a saving the rail cannot address — and on scattered +populations overstates it. The word is how the executor tests; the block is what the +rail can promise. + +## 2026-09-15 (6) — E-THE-SLOWEST-GATE-IS-THE-ONE-YOUR-OWN-PUSH-CADENCE-CANCELS-1 — 33 runs, 19 cancelled; the fix waited 75 minutes for a verdict, and three of its four heads were cancelled by my own next push + +**Status:** FINDING. Every number is the Actions API ledger for `rust-test.yml` filtered +to this branch, re-fetchable. +**Confidence:** HIGH. + +### The ledger + +`Rust Tests` on `claude/clone-repositories-71a5sw`: **33 runs — 19 cancelled, 7 success, +7 failure.** A verdict on 14 of 33 heads. For #1235's fourteen runs on 2026-09-15: 7 +failures (`d73c742` 17:25Z → `36646a2` 18:43Z, each finishing 2.9–4.5 minutes after +start — the compile-failure signature; the aws-smithy cause was verified by log on +`36646a2`, `E-A-CHECK-THAT-CANNOT-RUN-IS-INDISTINGUISHABLE-FROM-A-CHECK-THAT-PASSES-1`), +**6 cancelled, 1 success** — `d77cd4e`, 19 minutes wall, the only time the workflow +completed on this PR. + +The mechanism is a concurrency group keyed on the PR number with +`cancel-in-progress: true` (`.github/workflows/rust-test.yml:13-15`), on a workflow that +takes 19 minutes. Correct CI +economy. The failure is the reader's, and the timeline is the finding: + +| head | committed | run created | outcome | +|---|---|---|---| +| `947753d` (the fix) | 19:09Z | — | no run: the PR was conflicted | +| `99479a5` | 19:27Z | 19:28Z | cancelled 19:32Z by the next push, after 4 min | +| `188d6b3` | 19:31Z | 19:32Z | cancelled 19:48Z, after 16 min | +| `e43d12f` | 19:47Z | 19:48Z | cancelled 20:06Z, after 18 min — about a minute short of a full run | +| `d77cd4e` | 20:05Z | 20:05Z | **success 20:24Z** | + +75 minutes from the fix to its first test verdict, and nothing but my own cadence in +between. + +### What I read as confirmation meanwhile, and what it actually confirmed + +`Build` and `Style Check` went green on `188d6b3`. `build.yml` is `cargo build` / +`cargo test` with `--manifest-path crates/lance-graph/Cargo.toml` — **one crate**. It +proved the aws fix compiles the core crate and runs its tests; it never builds +`lance-graph-quack` or `lance-graph-mask-risc` and cannot run their suites. "The fix is +confirmed" was true for the compile and unsupported for the tests until 20:24Z. + +The check-in prompt I wrote for myself compounded it: it asked the next wake to confirm +that `member-tests` reaches its `cargo test -p lance-graph-quack` line. **There is no such +line.** The quack step is step 17 of the `test` job, `Run quack tests` = +`cargo test --manifest-path crates/lance-graph-quack/Cargo.toml` +(`.github/workflows/rust-test.yml:134-135`); the member-tests job enumerates other +crates by manifest path and never names quack. Verified on `d77cd4e` by log: `Running unittests +src/lib.rs (…lance_graph_quack-…)` → `test result: ok. 14 passed; 0 failed`. A wake that +trusted the prompt would have grepped the wrong job's log, found nothing, and reported the +suite as never run — a false negative primed thirty minutes ahead by its own author. + +### The rules + +1. **When you are waiting on the slowest gate, a push is a cancellation.** Hold the push + until the verdict, or accept that the verdict will be for the next head. The tell in + the ledger: a run whose `updated_at` matches the next run's `created_at` to the second. +2. **A cancelled run leaves no verdict, and the greens beside it are the fast workflows.** + Read the slow workflow's ledger (`actions_list`, branch-filtered), not the PR page. +3. **Name a job from its step list, never from memory.** `get_workflow_job` returns the + steps; thirty seconds against thirty minutes. + +Cross-ref: `E-A-CHECK-THAT-CANNOT-RUN-IS-INDISTINGUISHABLE-FROM-A-CHECK-THAT-PASSES-1` is +the conflicted-PR half of the same day — no run at all. This entry is the other half: a +run that starts and is killed. + +## 2026-09-15 (5) — E-THE-SKIP-LEVER-LIVES-ONLY-BELOW-THE-DENSITY-WHERE-D-GTM-0N-SAYS-SWITCH-TO-SPARSE-AND-THE-CLUSTERED-99-90-IS-PREFIX-ARITHMETIC-1 — both readers dropped the density bound; both A1 lever regimes sit under it; the 99.90 % is 1023/1024 by construction + +**Status:** FINDING for the density collision and for the P = 50 arithmetic — both +re-checkable from two board rows, two source lines and the §8a table. CONJECTURE for +the prefix-length family below P = 50: derived, not run; its falsifier is named. +**Confidence:** HIGH on every number quoted. The *competitive* claim — that a sparse +arm beats the gated sweep at 0.05 % active — is deliberately NOT made. The probe has no +sparse arm, and that absence is the finding. + +### 1. The bound two readers dropped + +`STATUS_BOARD` row `D-GTM-0n / P3` carries two bounds *"that ride with the number and +may not be dropped when it is cited"*: the win is L2-residency-contingent, **and mask +loses to sparse below 0.1 % active**. The crosswalk entry restates it — *"only above +~0.1 % active — a crosswalk hop that thins the survivors below that must switch to +sparse"* (its "bounds that ride with the claim" paragraph). The parallel session's second +feedback cited D-GTM-0n by its L2 bound and its K = 1 control and omitted the density +bound; earlier the same day I had cited D-GTM-0m without its one-fixture / upper-bound +caveat (⊘ in `E-FUSING-FORFEITS-THE-SKIP-AND-ADAPTIVEFILTER-FAILS-IN-TWO-PLACES-NOT-ONE-1`). +Same move, two readers, two rows: the memorable bound survives the citation and the +inconvenient one does not. + +### 2. Why it bites: the lever's only live regimes are under the bound + +§8a (`.claude/plans/duckdb-to-v3-translation-matrix-v1.md`) — 65 536 rows, 5 conjuncts, +all 120 permutations: + +| regime | survivors | active | spread | +|---|---|---|---| +| selective | 36 | **0.055 %** | 75.00 pts | +| clustered | 31 | **0.047 %** | 99.90 pts | +| moderate | 14 311 | 21.8 % | 0 | +| permissive | 61 777 | 94.3 % | 0 | + +The two regimes where ordering moves the skip fraction sit at roughly half of 0.1 %. The +two regimes where the mask arm is the right arm by D-GTM-0n's own bound have spread 0. +So `Filter::and_by_skip` is measurable exactly where the substrate's bound assigns the +work to the sparse representation, and inert everywhere the mask representation is the +right one. + +What that changes: the A1 verdict (*ADAPT conditionally*) was reached by comparing +orderings of the gated sweep **to each other**. The competitor D-GTM-0n names at this +density is a survivor list with per-row evaluation of the remaining conjuncts — on the +order of 36 rows × 4 conjuncts against 1 024 gated words × 4 — and the probe never runs +it. The falsifier is missing an arm, and it is not the fused arm (Phase 7): that one is +order-independent by construction (the (4) entry above) and settles a different question. +**The A1 row must carry the 0.1 % bound the way D-GTM-0n's row does.** + +> ⊘ **Corrected the same day (operator ruling, entry (7)).** The clustered row above is +> the `/50` cut's — a quarter block. On the rail's byte boundary (`/48`) the clustered +> regime has **116 survivors = 0.177 %**, ABOVE the bound; only the selective regime +> (0.055 %) sits under it. The missing-arm finding stands for that regime. + +### 3. The clustered 99.90 % is arithmetic, not a property of the data + +`crates/lance-graph-quack/examples/adaptive_order_probe.rs:186` — `(i as u64) << 8`; +`:322` — `Filter::prefix_u64(ADDR, addr[N / 4], 48)` (it read `50` when this entry was +measured; see the ⊘ below). A 50-bit prefix on a u64 leaves 14 +low bits free: 8 are the shift, **6 are log₂ 64 — the word.** The prefix pins `i`'s top +10 bits, which IS the 64-row word index (N = 2¹⁶ rows = 1 024 words); exactly the 64 rows +of one aligned word satisfy it. Gated on that accumulator, every later conjunct evaluates +1 word of 1 024. The ramp `[4092, 3069, 2046, 1023, 0]` is 1 023 × (conjuncts after the +prefix), and 4 092 / 4 096 = **99.90 % = 1023/1024**. + +The parallel session named the general form the *10-bit handoff*: on a 16-bit rail the +low 6 bits address inside a word and the top 10 select it, so any prefix predicate of +length ≥ 10 on an address-ordered rail is word-granular **by construction**. On this +lane a prefix of P bits leaves 2^(50−P) live words — skip = 1 − 2^(50−P)/1024: 99.90 % at +50, 99.80 % at 49, 99.61 % at 48, 0 % by 40. §8a measured the P = 50 point of that function +and tabulated it beside three regimes that ARE properties of the data. "Clustered" should +be read as *the predicate is a word address*, and 99.90 % as the mechanism's ceiling, not +as an observation about clustering. + +Two consequences. The ceiling is reachable only on an ADDRESS-ORDERED lane — the blocker +the parallel session already named for the V3-native seed; on an unsorted lane a prefix's +survivors scatter and the function above does not apply. And a V3 `6×(u8:u8)` rail is 16 +bits wide, so the handoff sits at exactly 10 bits for every rail: above that length the +prefix skip and the word skip are one mechanism, below it they are two. + +**Falsifier (not run):** the probe with the prefix at 50, 49, 48 must report 99.90 / +99.80 / 99.61 %. A departure falsifies §3; a match promotes the family from CONJECTURE. + +> ⊘ **Corrected the same day — the handoff is at the BYTE, not at 10 bits.** Operator: +> *"256:256 is exactly 64k. Es darf gar keinen Rest geben."* The rail's unit is its hi +> byte — 256 blocks of 256 rows, four words each. `/50` pins two bits of the lo byte, +> reading across `u8:u8`, and selects a quarter block; the 1023/1024 is that cut's +> arithmetic. On the byte boundary the ceiling is 1 − 1/256 = **99.61 %**, measured in +> both units (entry (7); the `/48` point of the falsifier above ran and matched). The +> prefix-length family below the boundary is withdrawn as a family of rail addresses — +> legal prefixes are nibble-multiples and the skip unit is the block. The word-level +> figures above remain correct as facade arithmetic for the cut they describe. + +### What to carry + +- Any citation of D-GTM-0n carries the 0.1 % bound; any citation of §8a's lever regimes + carries their densities (0.055 %, 0.047 %) beside it. + +> ⊘ **STALE, same day — superseded by entry (7) above (earlier in this file's reading +> order).** `0.047 %` here is the `/50` quarter-block cut's clustered density. Entry +> (7)'s byte-boundary (`/48`) re-measurement gives the clustered regime **116 +> survivors = 0.177 %** — ABOVE the 0.1 % bound, not sitting beside the selective +> regime under it. Only the selective regime (0.055 %) still sits under the bound. +> Cite (0.055 %, 0.177 %), not (0.055 %, 0.047 %). + +- The A1 falsifier's missing arm is SPARSE, not fused. Filed as the operator's call in + `LATEST_STATE` 2026-09-15 (3). +- No code changes here; `and_by_skip` stays as shipped. + ## 2026-09-15 (4) — E-FUSING-FORFEITS-THE-SKIP-AND-ADAPTIVEFILTER-FAILS-IN-TWO-PLACES-NOT-ONE-1 — the fused lowering is order-independent BY CONSTRUCTION, two readers derived it from source because the crate doc does not say so, and DuckDB's A1 turns out to have a dead seed as well as an unrunnable loop **Status:** FINDING. Convergent — derived independently in two sessions from the same diff --git a/.claude/board/ISSUES.md b/.claude/board/ISSUES.md index dc6a214e8..5f26a3ce0 100644 --- a/.claude/board/ISSUES.md +++ b/.claude/board/ISSUES.md @@ -373,6 +373,11 @@ differing in the FIRST nibble and a pair differing in the LAST must not be confu > `(path: u64, depth: u8)` root-first, so the branchless form is one line: > `((a.path ^ b.path).leading_zeros() >> 2).min(a.depth.min(b.depth))`. The `morton48()`-unused > observation in perturbation-sim stands as written. +> ⊘ **2026-09-15, later — the semantic reason (operator):** popcount over the same XOR is +> position-blind and finds *elephant : Wal* — a cousin parting by one bit at the order level +> outranks a sibling parting by four at the leaf. The branchless `leading_zeros` form above is +> the tree's own metric, not only a branch-free rewrite; see +> `E-POPCOUNT-FINDS-ELEPHANT-WHALE-BECAUSE-IT-IS-POSITION-BLIND-THE-TREES-METRIC-IS-LZCNT-AND-THE-BOARD-ALREADY-FILED-IT-1`. **The shipped join is 4× coarser than the canon's own level granularity and uses a branch chain diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index 508d9169c..f5ce1dea0 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,205 @@ +## 2026-09-15 (10) — D-HXP-8 arm 1 RAN: tic-tac-toe is F0-degenerate (rails reach the whole board → the stack is a census), not a KILL; the early-exit meter presumes non-negative stacking + +- **State consumers should know:** the Raumgewinn falsifier's first arm is + committed (`crates/perturbation-sim/examples/tictactoe_raumgewinn.rs`) and + ran clean: 4520 positions / 627 classes, F1 tie-aware `0.5797` = the + random-move baseline, null `[0.5797, 0.5797]`. That is the CENSUS + signature, not chance: on 3×3 every cell's rings reach all 8 others, so + the full stack of any ring-additive intensity is identical for every + candidate (measured: distinct FULL-stack values per position `1.000`). + A new **F0 fixture-validity gate** now runs before any scoring and the + verdict prints `F0 DEGENERATE — F1 not read`. Next readable arm needs board + diameter > 2 × deepest ring: Gobang 15×15, Go 9×9, Hex ≥ 7×7 (Hex 5×5 fails). +- **Meter note:** `TierFloors::stack_early_exit` returns the partial sum at + the exit tier; with SIGNED tiers (the exploratory NET arm) that is not a + bound and the early exit changed the top move in 10.13 % of positions + (F2 `0.8987` vs `1.0000` on the non-negative arm). Doc premise unstated — + `TECH_DEBT` 2026-09-15. +- **Mississippi Queen "im Anschluss":** two readings, neither buildable + without a decision — hexagon W2 (D-HXP-3/4, "rail in the loop on the + river") is DATA-blocked (no MQ docs / river fixture in any repo or git + history; not fabricated), and ndarray `gemm-ternlog-mask-consolidation-v1.md` + §9 M1–M3 (`pack_a_masked_f32`, tile cache, ±1 ladder, coal budget) has spec + but no code — a first-writer implementation. Operator picks the reading. +- Entry: + `E-RAUMGEWINN-NEEDS-A-HORIZON-SMALLER-THAN-THE-BOARD-TIC-TAC-TOE-HAS-NONE-SO-ARM-1-IS-F0-DEGENERATE-NOT-A-KILL-1`; + plan §12a; `D-HXP-8` → In progress. + +## 2026-09-15 (9) — operator: the popcount side is the hexagon's Raumgewinn, its toolkit ships under the operator's own words, and tic-tac-toe / Gobang / Go / Hex make it falsifiable — `D-HXP-8` queued + +- **State consumers should know:** (11)'s lcp and (12)'s stacked popcount are + `head2head::WinnerCriterion::{DissonanceMin, SupportSpread}` — infight vs + Raumgewinn — already shipped (D-H2H-1). The Belichtungsmesser toolkit is + `ndarray::hpc::cascade::Cascade::{calibrate, expose, observe, recalibrate}` + + `perturbation_sim::rolling_floor::{RollingFloor, TierFloors::stack_early_exit}`; + nothing to build there. What is unbuilt is the evidence that it helps the + hexagon — E-Q8 is the one measured non-result (degree 1 sufficed). +- **New D-id:** `D-HXP-8` — board games as the falsifier (F1 agreement with + the solved value, F2 early-exit economy at unchanged verdict, F3 mandatory + degree-1 ablation that must DROP; KILL at chance or flat). Hex is the + six-neighbour game; the square games are its 4-/8-subgraphs. Queued; plan + §12 appended. Nothing built. +- Entry: + `E-POPCOUNTS-UPPER-RANGE-SIMILARITY-IS-THE-HEXAGONS-RAUMGEWINN-AND-BOARD-GAMES-MAKE-IT-FALSIFIABLE-1`; + (11) carries one ⊘ line. + +## 2026-09-15 (8) — operator: popcount also finds *elephant : Wal* — position-blind; the tree's metric is `lzcnt(u ⊕ self)`, and the board had already filed its branchless form + +- **State consumers should know:** on a root→leaf code popcount ranks a + cousin that parts by one bit at the order level above a sibling that + parts by four bits at the leaf. The tree's metric is the longest common + prefix — `lzcnt(u ⊕ self)`, `>> 2` = level — which the substrate already + names as `NiblePath::common_prefix_depth` (the radix NN measure) and whose + branchless one-liner is filed in + `ISS-SHARED-PREFIX-TIERS-IS-TIER-COARSE-AND-BRANCHES`. The (9) prefix + selection is its threshold form and ships; the per-row depth VECTOR + (`u8` per row, for ranking) is the missing vectorised piece — ndarray has + no `lzcnt` primitive today. (10)'s fused `popcount ≤ k` gap is thereby + scoped to exchangeable-bit carriers (planes, bipolar identities). +- Nothing built; entry + `E-POPCOUNT-FINDS-ELEPHANT-WHALE-BECAUSE-IT-IS-POSITION-BLIND-THE-TREES-METRIC-IS-LZCNT-AND-THE-BOARD-ALREADY-FILED-IT-1`; + (10) and the issue each carry one ⊘ line. + +## 2026-09-15 (7) — operator: *"You could even say it's popcount × self"* — the (self, d) prefix is the k = 0 Hamming ball; the fused k > 0 row predicate is a named gap in ndarray AND mask-risc; nothing built + +- **State consumers should know:** `(self, care, k)` with + `popcount((u ⊕ self) ∧ care) ≤ k` is one predicate family. k = 0 ships + (`ternary_match_{u32,u64,strided}_to_mask`; `Pred::MatchU32/U64`). k > 0 + does not ship as a mask builder: ndarray has per-row distances + (`hamming_batch_raw -> Vec`) and per-word masked popcounts + (`masked_popcount_batch`), mask-risc has no `Pred::HammingLe`. Two passes + and a 512 KiB vector today; the fused one-pass kernel + predicate is the + gap, named in the entry, not built. +- **Fence:** Hamming with k > 0 is a distance only on Hamming-meaningful bits + (planes, bipolar identities, the tree via `care(d)`) — never over + `palette256²` rails or CAM-PQ codes, whose distance is the 256×256 LUT + (`I-VSA-IDENTITIES`). +- **Already visible:** the probe sweep's `rows` column is + `popcount(mask(self, d))` = |ball(self, d)| — reading 2 of the entry. +- Entry: + `E-POPCOUNT-TIMES-SELF-THE-EXACT-PREFIX-IS-THE-K-EQUALS-ZERO-HAMMING-BALL-AND-THE-FUSED-ROW-PREDICATE-IS-THE-GAP-1`. + Phase 7 gains a second pre-registered arm (k > 0 cycles/row). + +## 2026-09-15 (6) — operator: *"Meine Lieblingsvariante ist V3 Format"* — a thought masks itself × distance from root, 0–96 bit, stepless, ~1 cycle; measured stepless on the probe, primitive already shipped in ndarray + +- **What changed (this commit):** the A1 probe gained a stepless radius sweep + (`d = 40..=56` on the address lane: rows = 2^(56−d) asserted per step; + words saturate at d = 50 / 99.90 %, blocks at d = 48 / 99.61 %); the + three doc sites that read as "only nibble-aligned prefixes are legal" now + say the radius is stepless and nibble boundaries are codebook cells. No + lowering behaviour changed. +- **State consumers should know:** the operator's variant `(self, d)` — + `(u.payload ^ self.payload) & care(d) == 0` — is exactly + `ndarray::simd::ternary_match_strided_to_mask` (12-byte pattern + care + over a 16-byte stride, shipped) and, per lane, `Pred::MatchU64` / + `Filter::prefix_u64`. What is missing is the strided `Operand` in + mask-risc (its own `LaneRef::U64` doc names it, PR4/PR5) and a 97-entry + `care(d)` table per ClassView carving. The "close to 1 CPU cycle" claim is + a pre-registered Phase 7 falsifier (cycles/row over 64k × 16 B), not + measured. +- Entry: + `E-A-THOUGHT-MASKS-ITSELF-BY-ITS-DISTANCE-FROM-ROOT-THE-V3-FACET-IS-THE-MASK-AND-THE-RADIUS-IS-STEPLESS-1`; + (7) and (8) carry one ⊘ line each. + +## 2026-09-15 (5) — operator clarification: the rail is the exact row ADDRESS (2 bytes ↔ 64k rows), not a mask — (4)'s "counts in the rail's unit" is withdrawn, the two-unit measurement stays + +- **What changed (this commit, docs only):** the probe header, `lib.rs`'s + `and_by_skip` doc, the prefix test's comment and mask-risc `MaskOp::Pred`'s + doc no longer say the rail's hi byte is the skip unit. They say: a `u8:u8` + rail is the exact SoA row address of a 64k table (256 × 256, every value a + row — that is the "no remainder"); a mask over the 64k area is a larger + object, bitpacked 1 024 words = 256 four-word blocks; the 256-row block is + the OGAR tier tile's 2-nibble cell, a coarser skip unit an executor may use. + No number moved; the probe reproduces `[4080, 3060, 2040, 1020, 0]` / + `[1020, 765, 510, 255, 0]`. +- **The operator's candidate reading for masking, recorded not built:** + `256:256⁶` = the 96-bit facet payload — six exact needles (a sparse + survivor set of ≤ 6 rows in the register the facet already has) or six + per-rail prefixes; or bitpacked 64k. The sparse arm the A1 falsifier lacks + is the needle list; it lands in Phase 7's very-sparse density arm. +- Entry: + `E-THE-RAIL-IS-A-NEEDLE-NOT-A-MASK-256-BY-256-IS-THE-EXACT-ROW-ADDRESS-AND-A-MASK-OVER-THE-AREA-IS-ANOTHER-OBJECT-1`; + (7) ⊘-regraded in place. + +## 2026-09-15 (4) — operator ruling *"256:256 is exactly 64k. Es darf gar keinen Rest geben."* — the A1 probe counts in the rail's unit, and two figures in (3) moved + +- **What changed (this commit, no lowering behaviour touched):** + `crates/lance-graph-quack/examples/adaptive_order_probe.rs` reports 64-row + words (the executor's unit) AND 256-row blocks (the rail's hi byte, four + words, one 256-bit vector), cuts the clustered prefix on the byte boundary + (`/48` — one whole block; the earlier `/50` selected a quarter block by + reading across `u8:u8`), const-asserts `N == 256 * 256` with no remainder, + and prints the ramp its doc quotes. `lib.rs` docs and the prefix-halving + test's message follow; mask-risc `MaskOp::Pred`'s doc names the rail's unit. +- **Figures, re-measured:** clustered 99.90 → **99.61 %** (= 1 − 1/256, in + both units), survivors 31 → **116 (0.177 %, ABOVE the 0.1 % bound)**; + selective in blocks **0.00 → 61.91 %** (its written order skips no block + at all); moderate and permissive still 0 in both units. Ramp + `[4080, 3060, 2040, 1020, 0]` words / `[1020, 765, 510, 255, 0]` blocks. +- **Consequence for (3)(c) below:** the missing SPARSE arm stands for the + selective regime only; the clustered regime is above the bound on the byte + boundary. Entry: + `E-256-BY-256-IS-EXACTLY-64K-THE-RAILS-SKIP-UNIT-IS-ITS-HI-BYTE-AND-A-QUARTER-BLOCK-IS-A-REMAINDER-1`; + the (5) epiphany is ⊘-regraded in place. +- ⊘ **Same day, entry (5) above:** "the rail's unit" was my reading, not the + ruling — the rail is a row address, not a mask. Numbers unchanged. + +## 2026-09-15 (3) — PR #1235 merged (`e8d3c19`): `lance-graph-quack` is on `main` and the AWS SDK is opt-in on `main` — one new workspace member, NO contract inventory delta + +- **Added on `main`:** `crates/lance-graph-quack` — the DuckDB-shaped surface + whose operators lower to `lance-graph-mask-risc` programs (`Filter`/`Cmp`/ + `Agg`/`Query`, `lower` + `lower_fused`, `Filter::Plane`, the survivor skip + with accumulator gating and `hoist_gate_subset`, `Filter::and_by_skip`, + `lower_group_by`, `LowerError: std::error::Error`); its only in-tree + dependency is mask-risc, no `ndarray`, by design. + `examples/adaptive_order_probe.rs` (the A1 falsifier, matrix §8a, RUN); + `.claude/harvest/duckdb-headers/README.md` (123 methods / 1 622 events — the + harvest §6 had recorded as FAILED, repaired against the headers); + `.claude/audits/nars-34-substrate-audit.md`; CI lines for the member + (`rust-test.yml` `Run quack tests`, `style.yml` clippy + fmt). 16 files, + +4 227 / −11, 25 commits; merged 20:25:15Z by the operator. +- **Contract inventory delta: NONE.** The one `lance-graph-contract` file in + the diff is `src/recipes.rs` — two citations moved from classical + Berry-Esseen to Jirak (`I-NOISE-FLOOR-JIRAK`) plus the guard that keeps a + third from appearing. No type added or changed. +- **Dependency state consumers should know:** `lance` is + `default-features = false` plus its own default list minus `aws`; + `lance-graph` gained the opt-in `aws-sdk = ["lance/aws"]` (`947753d`, + operator ruling *"make it optional so that later we fork 1.7 and fix it"*). + `rust-publish.yml` passes an explicit feature list that excludes `aws-sdk` + AND `lancedb-sdk` — the latter because `lancedb 0.38.0` cannot compile + without its `remote` feature (`ISS-LANCEDB-038-NEEDS-REMOTE-TO-COMPILE`). + Both stay declared in the manifest; whether a published crate (0.5.4, + 89 276 downloads) keeps declared-but-unbuildable features is the + operator's call and was not picked here. +- **Status board:** D-QCK-0..6, 8, 9 Shipped; D-QCK-10 shipped on the + lance-graph-java side; D-QCK-7 Blocked on mask-risc PR5 (no `hop` op). + Also corrected in this commit: `D-MRX-1..6` still read *In PR (PR3)* + although #1226 merged on 2026-09-14 (`0b1ebaa`) — status cells flipped, + nothing else in those rows touched. +- **CI on the merged head, by log, not by badge:** `Rust Tests` completed on + `d77cd4e` only — the one completion in this PR's fourteen runs; `test` job + step 17 `Run quack tests` → `14 passed; 0 failed`. Ledger and lesson: + `E-THE-SLOWEST-GATE-IS-THE-ONE-YOUR-OWN-PUSH-CADENCE-CANCELS-1`. +- **Open, and the operator's to decide:** (a) the two published-but- + unbuildable features above; (b) the citation-decay unique-basename resolver + — 92 of 3 216 citations verify today, 1 499 bare basenames resolve to + exactly one tracked file, so the resolver would take coverage to ~1 670, + gated on sampling the anchor heuristic's precision (2 of the 3 new decays + it surfaced were false positives); (c) the A1 falsifier's MISSING ARM is + the sparse arm, not the fused one — both lever regimes (0.055 %, 0.047 % + active) sit under D-GTM-0n's *mask loses to sparse below 0.1 %* bound, + which both readers had dropped + (`E-THE-SKIP-LEVER-LIVES-ONLY-BELOW-THE-DENSITY-WHERE-D-GTM-0N-SAYS-SWITCH-TO-SPARSE-AND-THE-CLUSTERED-99-90-IS-PREFIX-ARITHMETIC-1`); + the fused arm stays gated on Phase 7 as before; (d) + `ISS-QUACK-LOWER-FUSED-IS-SUPERLINEAR-AND-DEEP-FILTERS-ABORT` wants a depth + budget in `gate_walk`, and its sibling half lives in mask-risc; (e) + `ruff#115`, the only real cross-repo conflict of the day, still open. +- **Branch:** `claude/clone-repositories-71a5sw` restarted from `main` at + `e8d3c19` (a fast-forward — it held only merged history); the #1235 + check-in routine deleted. +- Arc entry: `PR_ARC_INVENTORY.md` under PR #1235. + ## 2026-09-15 (2) — PR #1234 merged (`74f6302`): three retractions on `main` — NO contract inventory delta, and two rules locked - **Nothing added.** Five commits, four board files, zero code, zero types, diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index 222babe47..8ee6c2dc2 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -1,3 +1,92 @@ +## 2026-09-15 (3) — lance-graph PR #1235 (merged `e8d3c19`, branch `claude/clone-repositories-71a5sw`) — `lance-graph-quack`: the DuckDB-shaped surface whose operators ARE masking ops, and A1's falsifier run + +- **Added:** the workspace member `crates/lance-graph-quack` (builds mask-risc + `Program`s and never evaluates one — no expression interpreter, no row + iterator, no per-operator kernel library, no `dyn Operator` chain, no + validity bitmap beside the data, no hash table for `GROUP BY`; each is a + lowering); the survivor skip with the DROP law (a comparison vanishes with + the gate, an `AND` if ANY child does, an `OR` if EVERY child does, a `NOT` + never) and accumulator gating made sound by `hoist_gate_subset`; + `Filter::and_by_skip` (the caller's measured score orders an `AND` — no + decay, no intervals, no warm-up counters, because the crate cannot measure); + `examples/adaptive_order_probe.rs` (65 536 rows × 5 conjuncts × all 120 + permutations × 4 regimes); the repaired DuckDB header harvest; the + 34-recipe substrate audit with two `recipes.rs` citations Jirak-corrected; + the AWS-optional dependency change and the explicit publish feature list; + three CI lines. 16 files, +4 227 / −11, 25 commits. +- **Locked:** *a plan is a `Program` — one flat op list, one terminal*; a + `match` in quack that computed anything would be the duplicate evaluator + this arc exists to remove. *The DROP does not pass through `NOT`.* *The + accumulator gate is sound only inside the plane*, which the rotation + guarantees — found twice, nested one level down + (`E-THE-ACCUMULATOR-GATE-OUTRANKED-THE-PLANE-AND-SILENTLY-DROPPED-IT-1`). + *Only `lower` can skip; `lower_fused` is order-independent by construction* + (`E-FUSING-FORFEITS-THE-SKIP-AND-ADAPTIVEFILTER-FAILS-IN-TWO-PLACES-NOT-ONE-1`, + convergent with a parallel session). +- **Measured — and what the measurement is now known to lack.** Ordering + moves the skip fraction only in the two sparse regimes (75.00 pts at 0.055 % + active, 99.90 pts at 0.047 %), never in the two dense ones, so A1 is ADAPT + conditionally and DuckDB's hill-climb is not ported. Post-merge re-read: + both lever regimes sit under D-GTM-0n's *mask loses to sparse below 0.1 %* + bound and the probe has no sparse arm; the clustered 99.90 % is 1023/1024 by + prefix arithmetic + (`E-THE-SKIP-LEVER-LIVES-ONLY-BELOW-THE-DENSITY-WHERE-D-GTM-0N-SAYS-SWITCH-TO-SPARSE-AND-THE-CLUSTERED-99-90-IS-PREFIX-ARITHMETIC-1`). + The verdict stands as an ORDERING result; it is not yet a REPRESENTATION + result. +- **Review: CodeRabbit twice, codex once — nothing disputed.** The codex P1 + was a real wrong answer (a nested `Plane` dropped under an accumulator not + inside it), fixed with two falsifiers; round 2 re-found the same P1 + independently and reversed my own fmt decline (`LATEST_STATE`, the two + 2026-09-15 #1235 review entries). Six CodeRabbit findings verified and + fixed in `e43d12f`: [1] the §8a summary lines credited 75.00 pts / 14.2× + (selective) as the maximum when clustered's 99.90 pts is; [2] the plan + still called `hoist_gate_subset` a correctness requirement the code had + already retracted — ⊘-annotated to match; [3] `Query::and_by_skip` → + `Filter::and_by_skip` in `lib.rs:699` and `ISSUES.md` (the file disagreed + with itself and the intra-doc link did not resolve); [4] `LowerError` gains + `std::error::Error`, matching eight workspace precedents — mask-risc's own + `FuseError`/`ExecError` lack it, noted and not touched; [5] a board + citation by LINE (`EPIPHANIES:19221`, already decayed to style-table + content) → the entry name; [9] my own entry described `rust-publish.yml` as + PRESENTLY passing `--all-features` after `c2b4bc7` had already replaced it + — rewritten as historical, and its "no branch could ever have gone red" + narrowed to *no push-triggered run* (`workflow_dispatch` can aim at any + branch). Then `188d6b3`: four more citations by section instead of line. + Two findings deferred with reasons (the `gate_walk` depth budget; the + published-feature question — operator's), one unverified (whether the + cam96 passage over-ranks `HelixResidue`). One reply comment discharged the + rest. +- **CI, honestly:** red from the PR's opening (17:25Z) to the fix (`947753d`, + 19:09Z) on the aws-smithy 0.63 / 1.7 mismatch, with NO run at all on the + fixing head because the PR was conflicted + (`E-A-CHECK-THAT-CANNOT-RUN-IS-INDISTINGUISHABLE-FROM-A-CHECK-THAT-PASSES-1`); + after the rebase, `Rust Tests` completed ONCE in the PR's fourteen runs — + on the final head — and that completion ran the quack suite (`14 passed`). + The other thirteen outcomes are seven compile failures and six + cancellations, three of the latter by my own pushes after the fix + (`E-THE-SLOWEST-GATE-IS-THE-ONE-YOUR-OWN-PUSH-CADENCE-CANCELS-1`). + `citation-decay`, `append-only`, `supersession-index`, `Build`, `Style + Check` green on the merged head. +- **Deferred, named:** D-QCK-7 (the join — blocked on mask-risc PR5's `hop`); + the SPARSE arm of the A1 falsifier (new); the fused arm (Phase 7); the depth + budget for `gate_walk` + (`ISS-QUACK-LOWER-FUSED-IS-SUPERLINEAR-AND-DEEP-FILTERS-ABORT`); the two + declared-but-unbuildable published features + (`ISS-LANCEDB-038-NEEDS-REMOTE-TO-COMPILE`, + `ISS-PUBLISH-FEATURE-LIST-CAN-DRIFT`) — operator's call; the citation-decay + resolver — gated on precision sampling; `ruff#115`. +- **Docs:** the crate doc's "two lowerings, one meaning" section now names the + skip asymmetry; `rust-publish.yml` carries a comment explaining both + exclusions and their drift cost; matrix §8a corrected (99.90 pts clustered, + 75.00 pts + 14.2× selective) and its `hoist_gate_subset` paragraph + ⊘-annotated; the D-QCK-9 row corrected in place (the "dead words" and + "wrong landscape" reasons were measured false — the ramp is monotone). +- **Confidence:** high on the lowerings — every test differential against a + per-row oracle that never sees a `Program`, anti-vacuity `0 < selected < n`, + five arms agreeing on the 64k slice, and the suite run in CI on the merged + head; medium on A1's standing, for the reason under *Measured*; high on the + ledger numbers, which are the API's, not mine. + ## 2026-09-15 (2) — lance-graph PR #1234 (merged `74f6302`, branch `claude/ladybug-transcoding-plan-q5zbrs`) — three retractions, and ONE MOVE found six times, three of them inside the correction - **Added: nothing.** Zero types, zero plans, zero deliverables, zero code — diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 585b7d289..b7fab254b 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -31,19 +31,19 @@ evaluates one; `execute` stays the consumer's call on a scratch it owns. | D-QCK-7 | the join — `src_mask → hop → dst_mask` | **Blocked** on mask-risc's PR5: there is no `hop` op to lower to | — | | D-QCK-10 | the lowering convergence — `lgj-abi`'s `plan_lower` (a flat op list) and `quack::lower` (a Boolean tree) pinned equal. **A differential, deliberately NOT a delegation:** quack is a DEV-dependency of `lgj-abi`, because the membrane must not depend on a consumer of the IR it serves | Shipped (lance-graph-java `8ad1a1b` + `e9bf3aa`, `native/lgj-abi/src/exports/tests/lowering_convergence.rs`). Five disables red. Two dead fixtures found: `LT_I32(500)` on a `-150..=361` lane made the whole 16-vector arity-4 arm inert (15 → 21 of 28) and made a real `LE_I32 → LtI32` mis-map INVISIBLE (measured: red at operand 300, green at 500) | `the_two_lowerings_agree_on_every_combine_vector` (28 vectors, `assert_eq!(non_degenerate, 21)`), `the_all_rows_shortcut_is_the_same_condition_on_both_sides` (two-sided: 3 `AllRows`, 25 not), `every_opcode_maps_to_the_same_predicate_in_both_lowerings` (`assert_eq!(proper_subset, 9)` + the TCAM half-swap) | | D-QCK-8 | accumulator gating — an `AND` gates each later child under the accumulator built so far, not only under a resident plane. The same rule `lgj-abi`'s `plan_lower` already had; the two lowerings now implement one law | Shipped. Disable-verified 3×: gate-never-on-accumulator, an `OR` gating its children on its own accumulator, and `hoist_gate_subset` never rotating — each reddens | `the_gate_reaches_every_comparison_and_is_dropped_only_where_it_vanishes` via the new `gate_shape` helper (all-gated / on-accumulator / dropped, three facts not two); `every_lowered_filter_agrees_with_an_independent_per_row_reading` is what catches the DROP going unsound | -| D-QCK-9 | A1 `Filter::and_by_skip` — the caller's measured skip score orders an `AND`'s conjuncts. DuckDB's hill-climb NOT ported (see the corrected reason) | Shipped, with the matrix's A1 falsifier RUN (`examples/adaptive_order_probe.rs`, 65 536 rows × 5 conjuncts × all 120 permutations × 4 regimes). Order moves the skip fraction, so A1 is **ADAPT conditionally** — inert in 2 of the 4 regimes, and the pre-registered *representative predicate stream* half was never run (matrix §9). ⊘ **Corrected 2026-09-15:** this row read "the control signal is DEAD WORDS, not selectivity" — the probe never ranks by either signal and never calls `and_by_skip`, so only the between-regime diagnostic is supported; and the "hill-climb explores the wrong landscape" reason was MEASURED FALSE (`[4092, 3069, 2046, 1023, 0]`, deltas all −1023: a monotone ramp). The real reason is that this crate never executes | `skip_ordering_moves_the_work_and_never_the_answer`: the reordering agrees with the oracle (not with the other lowering), the emitted program's first predicate is the highest-scored, and descending/no-sort disables both redden. **The probe itself is run by no gate** — `cargo test` compiles an example and never executes it — so the four-row table is a measured-once observation, not a pinned one | +| D-QCK-9 | A1 `Filter::and_by_skip` — the caller's measured skip score orders an `AND`'s conjuncts. DuckDB's hill-climb NOT ported (see the corrected reason) | Shipped, with the matrix's A1 falsifier RUN (`examples/adaptive_order_probe.rs`, 65 536 rows × 5 conjuncts × all 120 permutations × 4 regimes). Order moves the skip fraction, so A1 is **ADAPT conditionally** — inert in 2 of the 4 regimes, and the pre-registered *representative predicate stream* half was never run (matrix §9). ⊘ **Corrected 2026-09-15:** this row read "the control signal is DEAD WORDS, not selectivity" — the probe never ranks by either signal and never calls `and_by_skip`, so only the between-regime diagnostic is supported; and the "hill-climb explores the wrong landscape" reason was MEASURED FALSE (`[4092, 3069, 2046, 1023, 0]`, deltas all −1023: a monotone ramp). The real reason is that this crate never executes | `skip_ordering_moves_the_work_and_never_the_answer`: the reordering agrees with the oracle (not with the other lowering), the emitted program's first predicate is the highest-scored, and descending/no-sort disables both redden. **The probe itself is run by no gate** — `cargo test` compiles an example and never executes it — so the four-row table is a measured-once observation, not a pinned one ⊘ **2026-09-15, operator ruling (E-256-BY-256-IS-EXACTLY-64K-THE-RAILS-SKIP-UNIT-IS-ITS-HI-BYTE-AND-A-QUARTER-BLOCK-IS-A-REMAINDER-1):** the clustered figures were the `/50` quarter-block cut's; re-measured on the byte boundary (`/48`, the rail's hi byte) → 116 survivors (0.177 %), 99.61 % in both words and 256-row blocks; selective in blocks 0.00 → 61.91 %. The probe now prints both units and the ramp. ⊘ Same day (E-THE-RAIL-IS-A-NEEDLE-NOT-A-MASK-…-1): "the rail's hi byte" as skip unit was my reading — a rail is a 2-byte row ADDRESS, not a mask; the block is the tile's 2-nibble cell; numbers unchanged. ⊘ Same day (E-A-THOUGHT-MASKS-ITSELF-BY-ITS-DISTANCE-FROM-ROOT-…-1): the radius is STEPLESS — the probe sweeps d = 40..=56, 2^(56−d) rows each; words saturate at d = 50, blocks at d = 48 | ## mask-risc-executor (PR3 — D-ids minted 2026-09-14, `.claude/plans/mask-risc-executor-v1.md`) | D-id | scope | status | gate / falsifier | |---|---|---|---| | D-MRX-0 | ndarray T1: ten `*_to_mask_under` gated predicates on one `pack_under` engine, parity arm `0xAxx` | Shipped (ndarray #307, merged `854924a`) | closure-count can-it-fire: 20 of 40 with the skip, 40 without | -| D-MRX-1 | `exec.rs` — `Scratch`, `execute`, one facade call per op, generated ternlog dispatch | In PR (PR3; F-X3 + every aliasing shape disable-verified. F-X1 is a COST property — no differential can separate the two routings, they are semantically identical — and `count_probe`'s sparse-gate pair is its instrument: gated 3526 ns vs `pred + and` 10594 ns at the same count of 1741. Printed, never asserted) | F-X1 gated-vs-`mask_and` op count; F-X3 odd-immediate tail law on `n_rows % 64 != 0` | -| D-MRX-2 | `reference.rs` — scalar oracle with zero facade tokens | In PR (PR3; also owns the ONE `validate` both sides call). F-R2 as RUN: `And` wired to `mask_or` reddens the two-input differential (AGENT_LOG 2026-09-14 (3)) — not the `AND2`/`AND3` swap the plan first named | F-R1 source grep; F-R2 a wrong immediate is caught on seeded planes | -| D-MRX-3 | `fuse.rs` — ≤3-leaf Boolean tree → one `Ternlog{imm}` | In PR (PR3; wider trees reduce larger-child-first) | F-B1 `mask_passes() == 1`; F-B4 immediate equals bit-serial evaluation | -| D-MRX-4 | `ternlog_dispatch.rs` — generated 256-arm match, regenerate-and-diff gated | In PR (PR3; CI `--check` line wired) | all 256 immediates equal the bit-serial reference | -| D-MRX-5 | the differential suite at 8 row counts × every op × both CI arms | In PR (PR3; `x86-64-v3` in CI, `-v4` locally on 2026-09-14 — NEON/WASM/scalar unexercised) | anti-vacuity `survivors * 3 < n_rows` for the 8 non-`Ne` predicates, UNGATED, at `n_rows >= 3` (the two `Ne` shapes are selective by complement, and `n_rows ∈ {0, 1}` are execution-only — the bound is `0 < 0` there); the gate itself is pinned selective (`gated < ungated`) at `n_rows >= 64` | -| D-MRX-6 | `examples/count_probe.rs` — reference / interpreted / fused, 0 B per execute | In PR (PR3; four arms agree on the COUNT — 4855 at 64k rows; `interpreted` and `fused` gated at 0 B; timings printed from one run, not pinned) | counting allocator reads 0 after warm-up; three arms agree | +| D-MRX-1 | `exec.rs` — `Scratch`, `execute`, one facade call per op, generated ternlog dispatch | Shipped (#1226 `0b1ebaa`; PR3; F-X3 + every aliasing shape disable-verified. F-X1 is a COST property — no differential can separate the two routings, they are semantically identical — and `count_probe`'s sparse-gate pair is its instrument: gated 3526 ns vs `pred + and` 10594 ns at the same count of 1741. Printed, never asserted) | F-X1 gated-vs-`mask_and` op count; F-X3 odd-immediate tail law on `n_rows % 64 != 0` | +| D-MRX-2 | `reference.rs` — scalar oracle with zero facade tokens | Shipped (#1226 `0b1ebaa`; PR3; also owns the ONE `validate` both sides call). F-R2 as RUN: `And` wired to `mask_or` reddens the two-input differential (AGENT_LOG 2026-09-14 (3)) — not the `AND2`/`AND3` swap the plan first named | F-R1 source grep; F-R2 a wrong immediate is caught on seeded planes | +| D-MRX-3 | `fuse.rs` — ≤3-leaf Boolean tree → one `Ternlog{imm}` | Shipped (#1226 `0b1ebaa`; PR3; wider trees reduce larger-child-first) | F-B1 `mask_passes() == 1`; F-B4 immediate equals bit-serial evaluation | +| D-MRX-4 | `ternlog_dispatch.rs` — generated 256-arm match, regenerate-and-diff gated | Shipped (#1226 `0b1ebaa`; PR3; CI `--check` line wired) | all 256 immediates equal the bit-serial reference | +| D-MRX-5 | the differential suite at 8 row counts × every op × both CI arms | Shipped (#1226 `0b1ebaa`; PR3; `x86-64-v3` in CI, `-v4` locally on 2026-09-14 — NEON/WASM/scalar unexercised) | anti-vacuity `survivors * 3 < n_rows` for the 8 non-`Ne` predicates, UNGATED, at `n_rows >= 3` (the two `Ne` shapes are selective by complement, and `n_rows ∈ {0, 1}` are execution-only — the bound is `0 < 0` there); the gate itself is pinned selective (`gated < ungated`) at `n_rows >= 64` | +| D-MRX-6 | `examples/count_probe.rs` — reference / interpreted / fused, 0 B per execute | Shipped (#1226 `0b1ebaa`; PR3; four arms agree on the COUNT — 4855 at 64k rows; `interpreted` and `fused` gated at 0 B; timings printed from one run, not pinned) | counting allocator reads 0 after warm-up; three arms agree | ## bbb-nars-lowering (D-id minted 2026-09-07 with the operator ruling) @@ -1944,3 +1944,4 @@ Plan path: `.claude/plans/polyglot-container-query-membrane-v1.md`. Research gro | D-HXP-5 | W3 — ARM promotion: cue → validated `⟨f,c⟩` path (`Dataset`/`CandidateRule`/`arm_to_truth_u8`, zero production callers today) | Queued | PASS promoted rails beat a **same-count shuffled** promotion on held-out futures — the H5c shape, at the bar H5c failed (0.48 % vs 0.77 % permuted). Disable: promote by `cooccur` alone; if it still passes, confidence is decoration | | D-HXP-6 | W4 — the flow channel: promotion yield vs challenge/skill (`mul::FlowState`, `#[repr(u8)]`, D-CSV-13b) | Queued | PASS yield peaks in the middle escape bucket (0 % / ~50 % / 85 % = Boredom / Flow / Anxiety). Two-sided kills: yield monotone in escape → the flow model is wrong; yield flat → `FlowState` carries no information here and must not be cited as a mechanism | | D-HXP-7 | **EWA concentration gate** — replaces the struck W0 signal gate (§10). `ewa_sandwich` + `pillar_5plus_bound`, zero-dep in the contract | Queued | PASS PSD ≥ 0.999 AND measured CV ≤ 1.75 × bound at **n ≥ 10** (bound 0.7483 at n=5 → 0.5715 at n=12). **KILL** CV exceeds slack at n ≤ 5. BETWEEN: the largest holding `n` becomes W3's measured path budget. **Anti-vacuity mandatory** — the unstructured fixture must FAIL at low n, else this gate is struck like its predecessor. Disable: plain additive accumulation must go red at n ≥ 10 | +| D-HXP-8 | **Board games as the Raumgewinn falsifier** (operator, 2026-09-15: *"tiktaktoe gobbang, go … falsifiable"*) — cells as units, the six rails as neighbours (Hex exact; Go 4 of 6; tic-tac-toe / Gobang 8 directions), positions evaluated by HDR popcount stacking through `TierFloors::stack_early_exit`, winner by `head2head::SupportSpread`; plan §12 | In progress — arm 1 (tic-tac-toe) RUN 2026-09-15: **F0 DEGENERATE**, not a KILL (every cell's rails reach all 8 others → the full stack is the board census, distinct values `1.000`; F1 tie-aware `0.5797` = baseline is the census signature). F0 fixture-validity gate added (plan §12a); next readable arm: Gobang 15×15 / Go 9×9 / Hex ≥ 7×7 (board diameter > 2 × deepest ring; Hex 5×5 fails). Meter note: signed tiers break the early exit (10.13 %), `TECH_DEBT` 2026-09-15 | F1 agreement with the solved value (tic-tac-toe: top move value-preserving ≥ 95 % of 765 positions, shuffled-rail null for chance; Go end positions: stacked territory == flood-fill scoring, an equality); F2 early exit changes NO verdict and exposes fewer tiers (fraction measured); **F3 mandatory degree-1 ablation must DROP F1** — flat = the task did not exercise the six (E-Q8) and the probe proves nothing. KILL: F1 at chance on tic-tac-toe, or F3 flat | diff --git a/.claude/board/SUPERSESSION-INDEX.md b/.claude/board/SUPERSESSION-INDEX.md index 0a26f6bd9..bb9b9ce47 100644 --- a/.claude/board/SUPERSESSION-INDEX.md +++ b/.claude/board/SUPERSESSION-INDEX.md @@ -85,7 +85,7 @@ a licence to act on it. | **READ** | `deepnsm-v3-convergence-v1` | `StepMask` | PROPOSED (doc-only). Extends `v3-convergence | 5/5 | | **READ** | `epistemic-quadrant-materialization-v1` | `MergeMode` | PROPOSED.** Operator direction 2026-07-29: * | 4/4 | | **READ** | `graphrag-doc-retrieval-soa-integration-v1` | `GateDecision` | DESIGN + FIRST CODE. **v1.2 (2026-07-17):** | 7/10 | -| **READ** | `hexagon-plasticity-v1` | `MergeMode` | ACTIVE — landed 2026-09-14 on operator go. * | 4/9 | +| **READ** | `hexagon-plasticity-v1` | `MergeMode` | ACTIVE — landed 2026-09-14 on operator go. * | 5/10 | | **READ** | `integration-actionhandler-rbac-orchestration-v1` | `GateDecision` | HARDENING (5+3 in progress). | 0/0 | | **READ** | `mask-algebra-revision-read-v1` | `StepMask` | DRAFT, awaiting operator ruling on §5 | 2/3 | | **READ** | `mul-calibration-not-verdict-v1` | `GateDecision` | PROPOSAL (unbuilt) — 2026-08-26. PLAN/BOARD | 6/12 | diff --git a/.claude/board/TECH_DEBT.md b/.claude/board/TECH_DEBT.md index 7bec0d9fd..975b4f02f 100644 --- a/.claude/board/TECH_DEBT.md +++ b/.claude/board/TECH_DEBT.md @@ -1,3 +1,25 @@ +## 2026-09-15 — `TierFloors::stack_early_exit` promises a settled reading; that is true only for non-negative intensity, and the doc does not say so + +- **The premise, unstated.** `crates/perturbation-sim/src/rolling_floor.rs:220` + (`decision is confident, the finer tiers need not be computed`) — the early + reject returns the PARTIAL sum at the exit tier + (`rolling_floor.rs:239`, `if crossed || band == FloorBand::Alarm {`). A partial + is a lower bound on the full stack only when every remaining tier is + ≥ 0. The shipped caller (`weyl_over_fiedler`) is non-negative, so nothing + shipped is wrong; the CONTRACT is narrower than the signature (`[f64; 4]` + admits signed tiers) and the doc does not fence it. +- **Measured (D-HXP-8 arm 1, `tictactoe_raumgewinn.rs`):** on the signed NET + arm (own − opp) the early exit changed the top-ranked candidate in + **10.13 %** of positions (F2 `0.8987`); on the non-negative AGREEMENT arm + `1.0000`. On that fixture the full stack is constant per position, so the + partial-sum spread is the only spread — the mechanism is unambiguous. +- **Options, not chosen here:** (a) document the premise and `debug_assert!` + non-negativity; (b) a signed-safe exit — exit only when the remaining tiers' + maximum possible magnitude (a preheated per-tier bound) cannot reverse the + ranking; (c) callers with signed tiers run the full stack. D-HXP-8 takes (c) + for its exploratory arm and keeps the pre-registered arm non-negative. + Board: `EPIPHANIES` (13) 2026-09-15. + ## 2026-09-14 — PR3 (mask-risc executor) hands three items to PR4 - **Two runtime-immediate → const-generic ternlog bridges now exist.** diff --git a/.claude/plans/duckdb-to-v3-translation-matrix-v1.md b/.claude/plans/duckdb-to-v3-translation-matrix-v1.md index 9c912529b..0ec2f1e58 100644 --- a/.claude/plans/duckdb-to-v3-translation-matrix-v1.md +++ b/.claude/plans/duckdb-to-v3-translation-matrix-v1.md @@ -611,6 +611,21 @@ positions of one ordering (term 0 is the ungated seed). | permissive | 61 777 (94.3 %) | 0.00 % | 0.00 % | 0.00 % | 0 | | clustered (one conjunct is an ADDRESS PREFIX) | 31 (0.047 %) | **99.90 %** | 0.00 % | 99.90 % | 99.90 pts | +> ⊘ **2026-09-15, operator ruling +> (`E-256-BY-256-IS-EXACTLY-64K-THE-RAILS-SKIP-UNIT-IS-ITS-HI-BYTE-AND-A-QUARTER-BLOCK-IS-A-REMAINDER-1`):** +> the clustered row is the `/50` cut's — a quarter of a 256-row block, read across the +> rail's `u8:u8`. On the byte boundary (`/48`, the rail's hi byte): **116 survivors +> (0.177 %)**, 0.00 % → **99.61 %** in words AND in 256-row blocks (1 − 1/256); the +> selective row in blocks is 0.00 % → 61.91 % (its written order skips no block). The +> probe now prints both units and the ramp. ⊘ Same day, operator clarification +> (`E-THE-RAIL-IS-A-NEEDLE-NOT-A-MASK-…-1`): "the rail's hi byte" as a skip unit was +> my reading — a rail is the exact 2-byte row address of a 64k table, not a mask; the +> 256-row block is the tier tile's 2-nibble cell. Numbers unchanged. +> ⊘ And per `E-A-THOUGHT-MASKS-ITSELF-BY-ITS-DISTANCE-FROM-ROOT-…-1`: the radius is +> STEPLESS — the probe's sweep over d = 40..=56 selects exactly 2^(56−d) rows at every +> step; the word-skip saturates at d = 50 (99.90 %), the block-skip at d = 48 (99.61 %). +> `/50` was never illegal; it is the word-skip's saturation point. + **Verdict: ADAPT, not ELIMINATE — but not DuckDB's algorithm.** Three findings, each of which changes what should be built: @@ -665,7 +680,9 @@ each of which changes what should be built: Measured, instrumenting the probe's own `skipped_words` model with the prefix term at each index: **`[4092, 3069, 2046, 1023, 0]`, adjacent deltas - all exactly `−1023`.** The prefix term's mask is one live word of 1024, and + all exactly `−1023`.** (⊘ On the byte boundary, `/48`, the printed ramp is + `[4080, 3060, 2040, 1020, 0]` words / `[1020, 765, 510, 255, 0]` blocks — + still monotone and linear; see the ⊘ under the §8a table.) The prefix term's mask is one live word of 1024, and `skipped_words` charges `dead_words(acc)` once per gated position, so skip is `(4 − p) · 1023` in the term's index `p` — a monotone linear ramp. Every single forward adjacent swap improves it by the same amount. That is the diff --git a/.claude/plans/hexagon-plasticity-v1.md b/.claude/plans/hexagon-plasticity-v1.md index a438c60fc..4d77049ae 100644 --- a/.claude/plans/hexagon-plasticity-v1.md +++ b/.claude/plans/hexagon-plasticity-v1.md @@ -724,3 +724,69 @@ re-run from this repository.** Every measurement depends on the ~10 GB `r2harves the `ore-full-v2` build, which live in an ephemeral session scratchpad, are not committable, and have no immutable revision to cite. What is committed is the instrument and the result; what is missing is the input. Said plainly rather than implying reproducibility this repo cannot offer. + +## 12. D-HXP-8 — board games as the Raumgewinn falsifier (operator proposal 2026-09-15 — appended, nothing above rewritten) + +Operator, verbatim: *"Der Vorteil von HDR popcount stacking early exit Belichtungsmesser +statistical confidence interval thresholds preheating rolling floor bucket assignment ist +jedoch daß es für hexagon Substrate hilfreich ist — der sprichwörtliche Raumgewinn beim Go +boardgame."* — *"Und witzigerweise müsste tiktaktoe gobbang, go damit sogar falsifiable +sein."* Board entry: +`E-POPCOUNTS-UPPER-RANGE-SIMILARITY-IS-THE-HEXAGONS-RAUMGEWINN-AND-BOARD-GAMES-MAKE-IT-FALSIFIABLE-1`. + +**Why this is the falsifier E-Q8 asked for.** E-Q8 (§9's lineage) found the six doing no +work because the task consulted only the first neighbour. A game's ground truth NEEDS the +neighbours — a line, a liberty count, a Hex connection — so the degree-1 ablation is a +real gate here, not a memory. + +**Encoding.** Each cell a unit; its six rails its neighbours (Hex: exact; Go: 4 of 6; +tic-tac-toe / Gobang: 8 directions as two rings); stone colour in the payload. Evaluation += popcount stacking over rail agreement, ring by ring, through +`perturbation_sim::rolling_floor::TierFloors::stack_early_exit`, floors preheated from a +position sample; the winner criterion is `head2head::WinnerCriterion::SupportSpread`. + +**Pre-registered gates.** +- **F1 correctness** — tic-tac-toe: top-ranked move value-preserving in ≥ 95 % of the + 765 positions, chance measured by a shuffled-rail null. Go end positions: stacked + territory == flood-fill scoring, an equality. +- **F2 economy** — early exit changes no verdict (equality vs the full stack); mean + exposed tiers < full depth, fraction stated. +- **F3 degree ablation, mandatory** — at degree 1, F1 must DROP; flat ⇒ the probe proves + nothing. +- **KILL** — F1 at chance on tic-tac-toe, or F3 flat. + +**Order.** tic-tac-toe → Hex (small boards) → Gobang → Go. Not started. + +### 12a. Arm 1 RUN (2026-09-15) — tic-tac-toe is F0-degenerate; F0 joins the gates (appended, nothing above rewritten) + +Probe: `crates/perturbation-sim/examples/tictactoe_raumgewinn.rs`. 4520 reachable +non-terminal positions, 627 classes up to symmetry (the "765" above counts terminal +classes too), empty-board value 0, random-move baseline `0.5797`. + +| arm | F1 det / tie-aware | F2 | all tied | distinct stacked / FULL | F3 degree-1 (drop) | null (20 seeds) | +|---|---|---|---|---|---|---| +| AGREEMENT | 0.5865 / 0.5797 | 1.0000 | 1.0000 | 1.000 / 1.000 | 0.5800 (+0.0004) | 0.5797 [0.5797, 0.5797] | +| NET | 0.5677 / 0.5655 | 0.8987 | 0.8434 | 1.157 / 1.000 | 0.5808 (+0.0152) | 0.5768 [0.5617, 0.5899] | + +**Verdict: `F0 DEGENERATE — F1 not read`.** On 3×3 rings 1 ∪ 2 reach all 8 other cells +from every cell (`reach 8..=8 of 8`, `9/9 cells`), so the full stack of any ring-additive +intensity is the board census — identical for every candidate (measured: distinct +FULL-stack values `1.000`). F1 = baseline, F2 = 1 and F3 flat are the census signature, +not a KILL. Entry: +`E-RAUMGEWINN-NEEDS-A-HORIZON-SMALLER-THAN-THE-BOARD-TIC-TAC-TOE-HAS-NONE-SO-ARM-1-IS-F0-DEGENERATE-NOT-A-KILL-1`. + +**Gate added — F0 fixture validity, read FIRST.** Computed from the rails alone before +any position is scored: every cell's reach must be < board − 1, and the ranked quantity +must take > 1 distinct value per position. Silent twin: the degree-1 rails (reach 1), +asserted in the probe. A readable arm needs board diameter > 2 × deepest ring so that not +even the centre sees everything. + +**Meter note.** `stack_early_exit` returns the partial sum at exit; with signed tiers +(NET) that is not a bound, and early exit changed the top move in 10.13 % of positions. +The pre-registered arm stays non-negative (AGREEMENT); a signed arm runs full-stack. +`TECH_DEBT` 2026-09-15. + +**Order, revised.** tic-tac-toe RUN (F0 degenerate; retired as falsifier, kept as the +harness) → Gobang 15×15 (Chebyshev diameter 14 > 8 ✓) or Go 9×9 (Manhattan 16 > 8 ✓) or +Hex ≥ 7×7 (12 > 8 ✓; **Hex 5×5 fails F0**, diameter 8). F1/F2/F3 and KILL unchanged for +a fixture that passes F0. diff --git a/crates/lance-graph-mask-risc/src/ir.rs b/crates/lance-graph-mask-risc/src/ir.rs index af3eff8df..6f362f419 100644 --- a/crates/lance-graph-mask-risc/src/ir.rs +++ b/crates/lance-graph-mask-risc/src/ir.rs @@ -92,6 +92,14 @@ pub enum MaskOp { /// executor is free to skip coarser chunks; the result is identical by /// construction, since a skipped chunk is an all-zero gate). Compare cost /// follows the gate's live words; the per-word gate test is still ∝ rows/64. + /// + /// On a V3 table the coarser chunk has a natural size: a 256-row block — + /// four words, one 256-bit vector, the 2-nibble prefix cell of the OGAR + /// tier tile — and a 64k table's mask is exactly 1 024 words or 256 such + /// blocks, no remainder in either unit. The rail (`u8:u8`) is not a unit + /// of the mask at all: it is the exact row ADDRESS of a 64k table, 256 × + /// 256 = every row and nothing else (operator, 2026-09-15). Which unit an + /// executor skips in is its own choice; the result is identical. Pred { pred: Pred, under: Option, diff --git a/crates/lance-graph-quack/examples/adaptive_order_probe.rs b/crates/lance-graph-quack/examples/adaptive_order_probe.rs index 7579d73b4..3a62910c5 100644 --- a/crates/lance-graph-quack/examples/adaptive_order_probe.rs +++ b/crates/lance-graph-quack/examples/adaptive_order_probe.rs @@ -35,6 +35,29 @@ //! OPPORTUNITIES. The honest reading of the output is "how much work becomes //! avoidable", not "how much time is saved". //! +//! # Two granularities — and the rail is neither +//! +//! The 64-row word is the FACADE's unit — the `u64` the executor tests before +//! it loads a chunk. A 256-row block (four words, one 256-bit vector) is the +//! 2-nibble prefix cell of the OGAR tier tile — a coarser unit an executor may +//! skip in, with an identical result. The probe reports BOTH, because on +//! scattered survivors they disagree: a block with one live word is live. +//! +//! What a rail is NOT is a unit of either. Operator, 2026-09-15: *"64k sind 2 +//! byte. 256:256 sind 2 byte für die exakte SoA in a given table — needle in a +//! haystack × table. Für Maske über 64k als Fläche bräuchte es entsprechend +//! mehr."* A `u8:u8` rail is the exact row ADDRESS of a 64k table: 256 × 256 = +//! 65 536 = this fixture's `N`, every value a row and every row a value — that +//! is the "no remainder". A mask over the same 64k is a different, larger +//! object (bitpacked: 1 024 words = 256 blocks, tiling with no remainder in +//! either unit), and the rail says nothing about which unit it is skipped in. +//! The clustered regime's prefix is `/48` — the hi byte of the row index, one +//! representative radius. The radius itself is STEPLESS: the operator's V3 +//! variant (2026-09-15) masks a unit by its own facet × its distance from +//! root, 0–96 bits, exact and stepless, and the sweep at the end walks every +//! d from 40 to 56. An earlier run used `/50`; its figures (99.90 % = +//! 1023/1024) are the point where the word-skip saturates. +//! //! Run: `cargo run -p lance-graph-quack --example adaptive_order_probe --release` use lance_graph_mask_risc::{MaskOp, Operand}; @@ -46,6 +69,16 @@ const ADDR: Col = Col(2); const N: usize = 1 << 16; const WORDS: usize = N / 64; +/// Rows per block: the 2-nibble prefix cell of the tier tile — 256 rows, four +/// 64-row words, one 256-bit vector. A coarser skip unit than the word; not a +/// property of the rail, which is a row address (see the module doc). +const BLOCK_ROWS: usize = 256; +const BLOCK_WORDS: usize = BLOCK_ROWS / 64; +const BLOCKS: usize = N / BLOCK_ROWS; +const _: () = assert!( + N == 256 * 256 && BLOCKS * BLOCK_WORDS == WORDS, + "a 2-byte rail addresses exactly N rows, and N is whole words and whole blocks — no remainder" +); /// One conjunct: a label, the filter, and the rows it selects. struct Term { @@ -74,6 +107,16 @@ fn dead_words(mask: &[u64]) -> usize { mask.iter().filter(|w| **w == 0).count() } +/// Blocks of `mask` (four words each) that are entirely zero — a coarser skip +/// unit an executor may use. A block with one live word is LIVE, not +/// three-quarters dead, so block counting is stricter than word counting on +/// scattered survivors. +fn dead_blocks(mask: &[u64]) -> usize { + let (blocks, rest) = mask.as_chunks::(); + assert!(rest.is_empty(), "a 64k mask is whole blocks — no remainder"); + blocks.iter().filter(|c| c.iter().all(|w| *w == 0)).count() +} + fn and_into(acc: &mut [u64], other: &[u64]) { for (a, b) in acc.iter_mut().zip(other) { *a &= *b; @@ -84,20 +127,27 @@ fn popcount(mask: &[u64]) -> u32 { mask.iter().map(|w| w.count_ones()).sum() } -/// Total words SKIPPED across a conjunction evaluated in this order. +/// Total words and blocks SKIPPED across a conjunction evaluated in this order. /// /// Term 0 is ungated and skips nothing — it is the seed. Term `i` for `i > 0` /// is gated on the accumulation of `0..i`, and skips that accumulation's dead -/// words. -fn skipped_words(terms: &[&Term]) -> (usize, u32) { +/// words (the executor's unit) and dead blocks (the 256-row block unit). +fn skipped(terms: &[&Term]) -> (usize, usize, u32) { + let masks: Vec<&[u64]> = terms.iter().map(|t| t.mask.as_slice()).collect(); + skipped_masks(&masks) +} + +/// The same accounting over bare masks, for the radius sweep below. +fn skipped_masks(masks: &[&[u64]]) -> (usize, usize, u32) { let mut acc = vec![u64::MAX; WORDS]; - and_into(&mut acc, &terms[0].mask); - let mut skipped = 0usize; - for t in &terms[1..] { - skipped += dead_words(&acc); - and_into(&mut acc, &t.mask); + and_into(&mut acc, masks[0]); + let (mut words, mut blocks) = (0usize, 0usize); + for m in &masks[1..] { + words += dead_words(&acc); + blocks += dead_blocks(&acc); + and_into(&mut acc, m); } - (skipped, popcount(&acc)) + (words, blocks, popcount(&acc)) } /// EXHAUSTIVE, not sampled — which is what lets the probe report a true @@ -263,11 +313,15 @@ fn main() { Scenario { name: "clustered (one conjunct is an ADDRESS PREFIX)", terms: vec![ + // /48 on `i << 8` pins the row index's hi byte — one 256-row + // block, a representative radius. Any d is a legal radius (the + // sweep below walks 40..=56); /50 is where the word-skip + // saturates. build( - "addr prefix /50 (contiguous)", - Filter::prefix_u64(ADDR, addr[N / 4], 50), + "addr prefix /48 (hi byte: one block)", + Filter::prefix_u64(ADDR, addr[N / 4], 48), &|r| { - let care = u64::MAX << (64 - 50); + let care = u64::MAX << (64 - 48); (addr[r] ^ addr[N / 4]) & care == 0 }, ), @@ -295,15 +349,19 @@ fn main() { }, ]; - println!("N = {N} rows, {WORDS} words per mask, 5 conjuncts per scenario"); println!( - "\nskipped = 64-row words a gated `Pred` does not evaluate, summed over the\n {} gated positions of one ordering (term 0 is the ungated seed).\n", - 4 * WORDS + "N = {N} rows = {BLOCKS} blocks x {BLOCK_ROWS} = {WORDS} words x 64, 5 conjuncts per scenario" + ); + println!( + "\nskipped = 64-row words (the executor's unit) and 256-row blocks (the tile's\n 2-nibble cell) a gated `Pred` does not evaluate, summed over the {} word /\n {} block gated positions of one ordering (term 0 is the ungated seed).\n", + 4 * WORDS, + 4 * BLOCKS ); for sc in &scenarios { let refs: Vec<&Term> = sc.terms.iter().collect(); - let gated_positions = (sc.terms.len() - 1) * WORDS; + let gated_words = (sc.terms.len() - 1) * WORDS; + let gated_blocks = (sc.terms.len() - 1) * BLOCKS; // The model is tied to the SHIPPED lowering rather than assumed: the // conjunction is really lowered, and every term after the first must @@ -337,17 +395,18 @@ fn main() { ); let perms = permutations(&refs); - let mut results: Vec<(usize, u32, Vec<&'static str>)> = perms + // (words, blocks, survivors, order) + let mut results: Vec<(usize, usize, u32, Vec<&'static str>)> = perms .iter() .map(|p| { - let (skipped, count) = skipped_words(p); - (skipped, count, p.iter().map(|t| t.label).collect()) + let (words, blocks, count) = skipped(p); + (words, blocks, count, p.iter().map(|t| t.label).collect()) }) .collect(); - let answer = results[0].1; + let answer = results[0].2; assert!( - results.iter().all(|r| r.1 == answer), + results.iter().all(|r| r.2 == answer), "{}: orderings disagree on the count; the probe is measuring a defect", sc.name ); @@ -361,8 +420,9 @@ fn main() { results.sort_by_key(|r| r.0); let worst = &results[0]; let best = results.last().expect("non-empty"); - let (written, _) = skipped_words(&refs); - let pct = |s: usize| 100.0 * s as f64 / gated_positions as f64; + let (written_w, written_b, _) = skipped(&refs); + let pct = |s: usize| 100.0 * s as f64 / gated_words as f64; + let pct_b = |s: usize| 100.0 * s as f64 / gated_blocks as f64; println!( "=== {} — {answer} survivors ({:.3}%)", @@ -373,22 +433,20 @@ fn main() { println!(" {:<30} sel {:.4}", t.label, t.selectivity); } println!( - " {:<12} {:>9} {:>9}", - "ordering", "skipped", "of gated" + " {:<12} {:>7} {:>8} {:>7} {:>8}", + "ordering", "words", "of gated", "blocks", "of gated" ); - println!( - " {:<12} {:>9} {:>8.2}%", - "as written", - written, - pct(written) - ); - println!( - " {:<12} {:>9} {:>8.2}%", - "worst", - worst.0, - pct(worst.0) - ); - println!(" {:<12} {:>9} {:>8.2}%", "best", best.0, pct(best.0)); + for (name, w, b) in [ + ("as written", written_w, written_b), + ("worst", worst.0, worst.1), + ("best", best.0, best.1), + ] { + println!( + " {name:<12} {w:>7} {:>7.2}% {b:>7} {:>7.2}%", + pct(w), + pct_b(b) + ); + } // 0/0 is not an infinite ratio, it is NO SPREAD — the `moderate` and // `permissive` regimes skip nothing in ANY order, and printing `infx` // beside `spread 0.00` made the line contradict itself. Those two rows @@ -406,13 +464,96 @@ fn main() { best.0 as f64 / w as f64 ), } - println!(" best order: {}", best.2.join(" < ")); + println!(" best order: {}", best.3.join(" < ")); + // Term 0 walked through every index with the rest in written order — + // the ramp `Filter::and_by_skip`'s doc quotes. Printed, so the quoted + // figure is a re-runnable measurement rather than a one-off + // instrumentation. + let ramp: Vec<(usize, usize)> = (0..refs.len()) + .map(|p| { + let mut order: Vec<&Term> = refs[1..].to_vec(); + order.insert(p, refs[0]); + let (w, b, _) = skipped(&order); + (w, b) + }) + .collect(); + println!( + " term 0 at index 0..={}: words {:?} blocks {:?}", + refs.len() - 1, + ramp.iter().map(|r| r.0).collect::>(), + ramp.iter().map(|r| r.1).collect::>() + ); + // The 256-row block unit, ranked on its own: the best-by-blocks ordering + // can differ from the best-by-words one, and the spread in blocks is the + // number a block-granular skip can actually realise. (A rail is an exact + // ROW ADDRESS, never a unit of the mask — see `Filter::prefix_u64`'s note.) + let (wb, bb) = ( + results.iter().map(|r| r.1).min().expect("non-empty"), + results.iter().map(|r| r.1).max().expect("non-empty"), + ); + println!( + " blocks: worst {wb} ({:.2}%) best {bb} ({:.2}%) spread {:.2} percentage points", + pct_b(wb), + pct_b(bb), + pct_b(bb) - pct_b(wb) + ); println!(); } + // The operator's V3 masking variant (2026-09-15): a unit masks ITSELF by its + // distance from root — `care = the first d bits`, d ∈ 0..=96 over the + // facet; here 0..=56 over this lane's 16 row bits (bits 8..=23 of `i << 8`, + // so d = 40 is every row and d = 56 is one). Any d is a legal radius — the + // selection is STEPLESS; nibble boundaries are where the codebook's + // centroid cells sit, which is a fact about meaning, not about the mask. + // The sweep leads the clustered conjunction with each radius and reports + // what it selects and skips: the ceiling family §8a's clustered row is one + // point of, in both units — and the point where the units part ways. + let clustered = scenarios + .iter() + .find(|s| s.name.starts_with("clustered")) + .expect("the clustered scenario exists"); + let others: Vec<&[u64]> = clustered.terms[1..] + .iter() + .map(|t| t.mask.as_slice()) + .collect(); + let base = addr[N / 4]; + println!( + "=== stepless radius d on the address lane, prefix first, then the clustered conjuncts" + ); + println!( + " {:>3} {:>7} {:>7} {:>8} {:>7} {:>8}", + "d", "rows", "words", "of gated", "blocks", "of gated" + ); + for d in 40..=56u32 { + let care = u64::MAX << (64 - d); + let mut prefix = vec![0u64; WORDS]; + let mut rows = 0usize; + for r in 0..N { + if (addr[r] ^ base) & care == 0 { + prefix[r / 64] |= 1u64 << (r % 64); + rows += 1; + } + } + assert_eq!( + rows, + 1usize << (56 - d), + "a radius of d bits selects 2^(56-d) rows" + ); + let mut masks: Vec<&[u64]> = vec![prefix.as_slice()]; + masks.extend(others.iter().copied()); + let (w, b, _) = skipped_masks(&masks); + println!( + " {d:>3} {rows:>7} {w:>7} {:>7.2}% {b:>7} {:>7.2}%", + 100.0 * w as f64 / (4 * WORDS) as f64, + 100.0 * b as f64 / (4 * BLOCKS) as f64 + ); + } + println!(); + println!( "A1's falsifier: if term order does not move the skipped fraction, the row is\n\ ELIMINATE and DuckDB's hill-climb must not be ported. These are counts of\n\ - AVOIDABLE word-evaluations, not timings." + AVOIDABLE word- and block-evaluations, not timings." ); } diff --git a/crates/lance-graph-quack/src/lib.rs b/crates/lance-graph-quack/src/lib.rs index 829dc6779..320866766 100644 --- a/crates/lance-graph-quack/src/lib.rs +++ b/crates/lance-graph-quack/src/lib.rs @@ -62,9 +62,10 @@ //! the 64 values, so a dead word's column read never happens. //! //! This matters because it is the whole of what survives from DuckDB's -//! `AdaptiveFilter` (matrix row A1 / §8a). Ordering is worth up to 99.90 -//! percentage points of skipped words on a clustered conjunction under -//! [`lower`] — and exactly zero under [`lower_fused`], on the same query. +//! `AdaptiveFilter` (matrix row A1 / §8a). Ordering is worth up to 99.61 +//! percentage points of skipped words — and of skipped 256-row blocks, the +//! tier tile's 2-nibble cell — on a clustered conjunction under [`lower`], and +//! exactly zero under [`lower_fused`], on the same query. //! [`Filter::and_by_skip`]'s lever is therefore alive in one configuration: //! gated lowering, plane-free conjunction, contiguous survivors. Under a //! plane it is inert too (`ISS-QUACK-AND-BY-SKIP-IS-INERT-UNDER-A-PLANE`). @@ -128,7 +129,8 @@ //! than unfinished: it never executes, so there is nothing for it to measure. //! //! Order is still a real cost lever here — under the survivor skip a conjunct -//! whose survivors die in whole WORDS shrinks every later predicate's live +//! whose survivors die in whole WORDS — or whole 256-row blocks — +//! shrinks every later predicate's live //! count — so [`Filter::and_by_skip`] takes the ordering decision as an INPUT. //! The measurement that licensed even that much is //! `examples/adaptive_order_probe.rs`; read [`Filter::and_by_skip`] for what it @@ -310,16 +312,29 @@ impl Filter { /// first literally shrinks the input. In V3 a predicate sweep costs the /// full column wherever it sits, so ordering can only pay by AVOIDANCE: /// the survivor skip drops a 64-row WORD when the gate has no survivor in - /// it. The matrix (row A1) said so and required the measurement before any - /// port. `examples/adaptive_order_probe.rs` is that measurement, over - /// 65,536 rows, five conjuncts, all 120 orderings, four regimes: + /// it — and a 256-row block (four words, the OGAR tier tile's 2-nibble + /// cell) is the coarser unit an executor may skip in; on scattered + /// survivors the two units disagree, so the probe reports both. A `u8:u8` + /// rail itself is neither: it is the exact row ADDRESS of a 64k table — + /// 256 × 256, every value a row, no remainder — not a mask and not a skip + /// unit (operator, 2026-09-15: *"64k sind 2 byte … für Maske über 64k als + /// Fläche bräuchte es entsprechend mehr"*). The matrix (row A1) said so + /// and required the measurement before any port. + /// `examples/adaptive_order_probe.rs` is that measurement, over 65,536 + /// rows, five conjuncts, all 120 orderings, four regimes, in both units: /// - /// | regime | survivors | skipped, worst → best order | - /// |---|---|---| - /// | permissive | 94.3 % | **0.00 % → 0.00 %** | - /// | moderate | 21.8 % | **0.00 % → 0.00 %** | - /// | selective | 0.055 % | 5.66 % → **80.66 %** (14.2×) | - /// | clustered (an address prefix) | 0.047 % | 0.00 % → **99.90 %** | + /// | regime | survivors | words, worst → best order | 256-row blocks, worst → best | + /// |---|---|---|---| + /// | permissive | 94.3 % | **0.00 % → 0.00 %** | 0.00 % → 0.00 % | + /// | moderate | 21.8 % | **0.00 % → 0.00 %** | 0.00 % → 0.00 % | + /// | selective | 0.055 % | 5.66 % → **80.66 %** (14.2×) | 0.00 % → **61.91 %** | + /// | clustered (an address prefix: one block) | 0.177 % | 0.00 % → **99.61 %** | 0.00 % → **99.61 %** | + /// + /// (An earlier run cut the clustered prefix at `/50` — the radius at which + /// the word-skip saturates: one live word, 31 survivors, 99.90 % = + /// 1023/1024. The radius is stepless — the operator's V3 variant masks a + /// unit by its own facet × a distance from root of 0–96 bits — and the + /// probe's sweep walks 40..=56.) /// /// Two findings, and a third that is a correction rather than a result. /// @@ -333,12 +348,15 @@ impl Filter { /// words. /// /// **Which means selectivity cannot tell you WHETHER reordering is worth - /// anything.** The selective and the clustered regimes have almost - /// identical survivor counts — 36 and 31 rows — and differ by 19 - /// percentage points of achievable skip (best against best: 80.66 % vs - /// 99.90 %), because one conjunct's survivors are contiguous and the - /// other's are scattered. A selectivity-only cost model cannot separate - /// those two cases. That is also why V3 has this lever at all: an address + /// anything.** The clustered regime has three times the selective + /// regime's survivors — 116 against 36 rows — and skips MORE, not less: + /// 99.61 % against 80.66 % of words, 99.61 % against 61.91 % of blocks, + /// because one conjunct's survivors fill one block and the other's are + /// scattered. Ranked by selectivity the two come out backwards; a + /// selectivity-only cost model cannot separate those two cases. And in + /// the 256-row block unit the selective regime's written order skips NOTHING + /// (232 dead words, 0 dead blocks): scattered survivors leave no block + /// empty until the two selective conjuncts have run. That is also why V3 has this lever at all: an address /// prefix selects a contiguous subtree ([`Filter::prefix_u64`]), which is /// the clustered row of that table. /// @@ -362,14 +380,15 @@ impl Filter { /// reason first given here was measured FALSE, so it is worth stating /// correctly.** The claim was that "the quantity being optimised is a step /// function of clustering … so a local search over adjacent swaps is - /// exploring the wrong landscape". Instrumenting the probe's own - /// `skipped_words` model with the clustered regime's prefix term at each - /// index gives `[4092, 3069, 2046, 1023, 0]` — adjacent deltas all - /// exactly `-1023`, a monotone linear ramp, because the prefix term's - /// mask is one live word of 1024 and each gated position past it skips - /// the other 1023. Every forward adjacent swap improves it by the same - /// amount. That is the friendliest possible hill-climb landscape, not the - /// wrong one. + /// exploring the wrong landscape". The probe walks the clustered + /// regime's prefix term through every index (its `term 0 at index` line) + /// and prints `[4080, 3060, 2040, 1020, 0]` words — adjacent deltas all + /// exactly `-1020` — and `[1020, 765, 510, 255, 0]` blocks, deltas + /// `-255`: a monotone linear ramp in both units, because the prefix + /// term's mask is one live block of 256 (four live words of 1024) and + /// each gated position past it skips the other 255 blocks. Every forward + /// adjacent swap improves it by the same amount. That is the friendliest + /// possible hill-climb landscape, not the wrong one. /// /// The step-like behaviour is BETWEEN regimes (does this conjunction /// contain a clustered term at all); the search space is WITHIN one @@ -2051,10 +2070,19 @@ mod tests { } previous = Some(rows.len()); } + // 48 is the byte boundary — the row index's hi byte, one 256-row + // block, a 2-nibble cell of the OGAR tier tile. 46, 47, 49 and 50 are + // not nibble-aligned: not cells of the codebook's 4-ary-per-byte + // cascade, but every one a legal, exact selection radius — the V3 + // variant masks a unit by its own facet × a STEPLESS distance from + // root (operator, 2026-09-15). The comparator is a ternary match on + // any care mask; the halving is its arithmetic and holds through + // every boundary. assert_eq!( previous, Some(64), - "the 50-bit prefix pins the row index down to a 64-row subtree" + "the 50-bit prefix pins the row index down to 64 rows — a quarter of \ + the 256-row block that /48 selects" ); // The widest prefix is a single address; the empty one is every row. diff --git a/crates/perturbation-sim/Cargo.toml b/crates/perturbation-sim/Cargo.toml index 5e8b0b6cd..ba3f437ce 100644 --- a/crates/perturbation-sim/Cargo.toml +++ b/crates/perturbation-sim/Cargo.toml @@ -128,3 +128,7 @@ path = "examples/family_basin_weyl_multihop.rs" [[example]] name = "chaoda_surge_epicenter" path = "examples/chaoda_surge_epicenter.rs" + +[[example]] +name = "tictactoe_raumgewinn" +path = "examples/tictactoe_raumgewinn.rs" diff --git a/crates/perturbation-sim/examples/comma_awareness.rs b/crates/perturbation-sim/examples/comma_awareness.rs index e6b8e0509..7a80b4138 100644 --- a/crates/perturbation-sim/examples/comma_awareness.rs +++ b/crates/perturbation-sim/examples/comma_awareness.rs @@ -149,7 +149,7 @@ fn dither(guid: u64, level: usize, comma: bool) -> f64 { } else { 0 }; - let phase = (hash2(guid, 0xC0FF_EE) % M + step) % M; + let phase = (hash2(guid, 0x00C0_FFEE) % M + step) % M; phase as f64 / M as f64 - 0.5 } diff --git a/crates/perturbation-sim/examples/tictactoe_raumgewinn.rs b/crates/perturbation-sim/examples/tictactoe_raumgewinn.rs new file mode 100644 index 000000000..f50385db2 --- /dev/null +++ b/crates/perturbation-sim/examples/tictactoe_raumgewinn.rs @@ -0,0 +1,967 @@ +//! D-HXP-8, arm 1 — tic-tac-toe as the falsifier of "popcount Raumgewinn". +//! +//! Operator (2026-09-15): *"Popcount Raumgewinn — und Mississippi Queen im +//! Anschluss."* Pre-registration: `EPIPHANIES.md` +//! `E-POPCOUNTS-UPPER-RANGE-SIMILARITY-IS-THE-HEXAGONS-RAUMGEWINN-AND-BOARD-GAMES-MAKE-IT-FALSIFIABLE-1`, +//! `hexagon-plasticity-v1.md` §12. The claim under test: a position evaluated by +//! popcount STACKING over a cell's rails (its neighbours), ring by ring, through +//! the rolling-floor Belichtungsmesser (`TierFloors::stack_early_exit`), ranks +//! moves in agreement with the solved game — territory, not lines. +//! +//! # What is fixed before the first number +//! +//! - **Ground truth** — full negamax over every position reachable from the +//! empty board; a move is *optimal* when it preserves the position's value. +//! - **Encoding** — each cell a unit; its rails are its neighbours at Chebyshev +//! ring 1 and ring 2 (tic-tac-toe is the 8-direction lattice; on 3×3 rings 3 +//! and 4 are empty, so tiers 2 and 3 of the stack contribute zero — the API is +//! the four-tier one so a 15×15 Gobang board runs unchanged). +//! - **The stack** — for a candidate move by P, tier r = popcount of P's stones +//! among the ring-r rails (the pre-registered AGREEMENT arm); a second, +//! exploratory arm uses the signed Zobrist influence own − opponent (NET). +//! Floors are preheated on the population of every (position, empty cell) +//! pair with `k = 2`, then `stack_early_exit` runs per candidate on a CLONE of +//! the preheated floors, so the ranking inside a position is order-independent. +//! Rank = descending, ties to the lowest index — but the RANKED QUANTITY is +//! arm-dependent: AGREEMENT's per-tier terms are non-negative popcounts, so +//! the early-exit *partial* stacked sum is a sound monotone bound and F1 +//! ranks on it directly; NET's per-tier terms are SIGNED (own − opp), so a +//! partial sum is NOT a bound on the full stack (a later tier can be +//! negative and pull the total below what an earlier positive partial +//! suggested — see the "meter note" printed at the verdict site), and F1 +//! ranks NET candidates on the FULL stack instead. F2 (below) is unaffected +//! by this split either way — it always compares the EARLY-EXIT top move +//! against the FULL-STACK top move, regardless of which one F1 used. +//! - **F0 fixture validity, read FIRST** — the rails' horizon must be smaller +//! than the board. If every cell's rings reach every other cell, the FULL stack +//! of any ring-additive intensity (both arms sum a per-cell term over ring +//! members) is the board census — identical for every candidate — and the +//! meter can separate candidates only by early-exit PARTIAL sums. Such a +//! fixture cannot read F1 or F2, and F1 landing exactly on the random-move +//! baseline is the census signature, not a KILL. The gate is computed from the +//! rails alone, before any position is scored; the degree-1 rails (reach 1 of +//! 8) are its can-it-stay-silent twin. +//! - **F1 correctness** — top-ranked move ∈ optimal set. Reported with the +//! deterministic tie-break AND tie-aware (share of the tied top set that is +//! optimal). Chance is MEASURED twice: the random-move baseline (mean share of +//! optimal moves per position) and the shuffled-rail null (every cell's rails +//! rewired to random cells of the same count, 20 seeds). +//! - **F2 economy** — the early-exited top move must equal the full-stack top +//! move (an equality), and the mean exposed tiers is reported. This +//! comparison is independent of which value F1 ranks by (see "The stack" +//! above): it always pits the early-exit top against the full-stack top. +//! - **F3 degree ablation, mandatory** — every cell keeps ONE rail (its first +//! ring-1 neighbour); F1 must DROP, or the task never exercised the six (E-Q8). +//! +//! Run: `cargo run --manifest-path crates/perturbation-sim/Cargo.toml --example tictactoe_raumgewinn --release` + +use perturbation_sim::rolling_floor::TierFloors; +use std::collections::{HashMap, HashSet}; + +const N: usize = 9; +const K_SIGMA: f64 = 2.0; +const NULL_SEEDS: u64 = 20; +const F1_BAR: f64 = 0.95; + +#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] +enum Cell { + E, + X, + O, +} + +type Board = [Cell; N]; + +const LINES: [[usize; 3]; 8] = [ + [0, 1, 2], + [3, 4, 5], + [6, 7, 8], + [0, 3, 6], + [1, 4, 7], + [2, 5, 8], + [0, 4, 8], + [2, 4, 6], +]; + +fn other(c: Cell) -> Cell { + match c { + Cell::X => Cell::O, + Cell::O => Cell::X, + Cell::E => Cell::E, + } +} + +fn winner(b: &Board) -> Option { + LINES + .iter() + .find(|l| b[l[0]] != Cell::E && b[l[0]] == b[l[1]] && b[l[1]] == b[l[2]]) + .map(|l| b[l[0]]) +} + +fn terminal(b: &Board) -> bool { + winner(b).is_some() || b.iter().all(|c| *c != Cell::E) +} + +/// Negamax value for the side to move and the set of value-preserving moves. +fn solve( + b: &Board, + to_move: Cell, + memo: &mut HashMap<(Board, Cell), (i8, Vec)>, +) -> (i8, Vec) { + if let Some(v) = memo.get(&(*b, to_move)) { + return v.clone(); + } + let result = if winner(b).is_some() { + // The previous mover completed a line: the side to move has lost. + (-1, vec![]) + } else if b.iter().all(|c| *c != Cell::E) { + (0, vec![]) + } else { + let mut best = -2i8; + let mut moves = Vec::new(); + for m in 0..N { + if b[m] != Cell::E { + continue; + } + let mut nb = *b; + nb[m] = to_move; + let v = -solve(&nb, other(to_move), memo).0; + if v > best { + best = v; + moves = vec![m]; + } else if v == best { + moves.push(m); + } + } + (best, moves) + }; + memo.insert((*b, to_move), result.clone()); + result +} + +/// Every non-terminal position reachable from the empty board, X to move first. +fn reachable() -> Vec<(Board, Cell)> { + let mut seen: HashSet<(Board, Cell)> = HashSet::new(); + let mut out = Vec::new(); + let mut stack = vec![([Cell::E; N], Cell::X)]; + while let Some((b, p)) = stack.pop() { + if !seen.insert((b, p)) || terminal(&b) { + continue; + } + out.push((b, p)); + for m in 0..N { + if b[m] == Cell::E { + let mut nb = b; + nb[m] = p; + stack.push((nb, other(p))); + } + } + } + out.sort_by_key(|(b, p)| (encode(b), *p == Cell::O)); + out +} + +fn encode(b: &Board) -> u32 { + b.iter().fold(0u32, |acc, c| { + acc * 3 + + match c { + Cell::E => 0, + Cell::X => 1, + Cell::O => 2, + } + }) +} + +/// The eight symmetries of the square, as index maps. +fn symmetries() -> Vec<[usize; N]> { + let rot = |b: [usize; N]| -> [usize; N] { + let mut o = [0; N]; + for (i, v) in b.iter().enumerate() { + let (r, c) = (i / 3, i % 3); + o[c * 3 + (2 - r)] = *v; + } + o + }; + let flip = |b: [usize; N]| -> [usize; N] { + let mut o = [0; N]; + for (i, v) in b.iter().enumerate() { + let (r, c) = (i / 3, i % 3); + o[r * 3 + (2 - c)] = *v; + } + o + }; + let id: [usize; N] = std::array::from_fn(|i| i); + let mut out = vec![id]; + let mut cur = id; + for _ in 0..3 { + cur = rot(cur); + out.push(cur); + } + let f = flip(id); + out.push(f); + let mut cur = f; + for _ in 0..3 { + cur = rot(cur); + out.push(cur); + } + out +} + +fn canonical(b: &Board, syms: &[[usize; N]]) -> u32 { + syms.iter() + .map(|s| { + let mut o = [Cell::E; N]; + for (i, &j) in s.iter().enumerate() { + o[j] = b[i]; + } + encode(&o) + }) + .min() + .expect("eight symmetries") +} + +/// A cell's rails: neighbours by Chebyshev ring (index 0 = ring 1). Four rings +/// so the stack is four tiers; on 3×3 rings 3 and 4 are empty. +type Rails = Vec<[Vec; 4]>; + +fn lattice_rails() -> Rails { + (0..N) + .map(|i| { + let (r, c) = ((i / 3) as i32, (i % 3) as i32); + let mut rings: [Vec; 4] = Default::default(); + for j in 0..N { + if j == i { + continue; + } + let (rj, cj) = ((j / 3) as i32, (j % 3) as i32); + let d = (r - rj).abs().max((c - cj).abs()) as usize; + if (1..=4).contains(&d) { + rings[d - 1].push(j); + } + } + rings + }) + .collect() +} + +/// F0 — the rails' horizon: how many of the other `N − 1` cells each cell's +/// rings reach in total. When every cell reaches every other cell the full stack +/// is the board census (see the module doc) and F1/F2 are not readable. +struct Horizon { + min_reach: usize, + max_reach: usize, + exhausted_cells: usize, +} + +impl Horizon { + fn of(rails: &Rails) -> Self { + let mut min_reach = usize::MAX; + let mut max_reach = 0; + let mut exhausted_cells = 0; + for rings in rails { + let reach: HashSet = rings.iter().flatten().copied().collect(); + min_reach = min_reach.min(reach.len()); + max_reach = max_reach.max(reach.len()); + if reach.len() == N - 1 { + exhausted_cells += 1; + } + } + Self { + min_reach, + max_reach, + exhausted_cells, + } + } + + /// **Any** cell whose rings reach every other cell makes the ranked + /// population contaminated — not only the all-cells case. + /// + /// The first version of this gate tested `exhausted_cells == N`, which is + /// right for a uniform board like 3×3 and wrong for a board where only the + /// CENTRE is covered by all rings (Hex 5×5 is the intended next fixture). + /// There `exhausted_cells` is non-zero but less than `N`, the old + /// predicate declared the fixture readable, and census-valued candidates — + /// ones whose stacked score carries no ranking information at all — went + /// straight into F1/F2 alongside real ones. Reported by codex on PR #1239. + /// + /// Strictly-any is deliberately conservative: it reports contamination + /// rather than trying to weigh it. The probe already computes + /// `f1_tie_nondegenerate` (F1 over the uncontaminated positions only), so + /// a partially-exhausted board is not lost — it is read from that field + /// instead of from `f1_tie`, and the verdict says so. + fn exhausts_board(&self) -> bool { + self.exhausted_cells > 0 + } +} + +/// F3 — degree 1: every cell keeps only its first ring-1 rail. +fn degree_one(rails: &Rails) -> Rails { + rails + .iter() + .map(|rings| { + let mut one: [Vec; 4] = Default::default(); + if let Some(&first) = rings[0].first() { + one[0].push(first); + } + one + }) + .collect() +} + +fn lcg(seed: &mut u64) -> u64 { + *seed = seed + .wrapping_mul(6364136223846793005) + .wrapping_add(1442695040888963407); + *seed >> 11 +} + +/// The shuffled-rail null: same rail COUNTS per cell and ring, targets drawn at +/// random from the other cells — the topology is scrambled, nothing else. +fn shuffled_rails(rails: &Rails, seed: u64) -> Rails { + let mut s = seed; + rails + .iter() + .enumerate() + .map(|(i, rings)| { + let mut out: [Vec; 4] = Default::default(); + let mut pool: Vec = (0..N).filter(|&j| j != i).collect(); + // Fisher–Yates over the pool, then deal it out ring by ring. + for k in (1..pool.len()).rev() { + let j = (lcg(&mut s) % (k as u64 + 1)) as usize; + pool.swap(k, j); + } + let mut cursor = 0; + for (r, ring) in rings.iter().enumerate() { + out[r] = pool[cursor..cursor + ring.len()].to_vec(); + cursor += ring.len(); + } + out + }) + .collect() +} + +#[derive(Clone, Copy)] +enum Arm { + /// Popcount of P's stones among the ring's rails — the pre-registered arm. + Agreement, + /// Signed Zobrist influence: own − opponent per ring — exploratory. + Net, +} + +fn intensity(b: &Board, p: Cell, m: usize, rails: &Rails, arm: Arm) -> [f64; 4] { + let mut out = [0.0; 4]; + for (r, ring) in rails[m].iter().enumerate() { + let own = ring.iter().filter(|&&j| b[j] == p).count() as f64; + let opp = ring.iter().filter(|&&j| b[j] == other(p)).count() as f64; + out[r] = match arm { + Arm::Agreement => own, + Arm::Net => own - opp, + }; + } + out +} + +/// Preheat one floor per tier on the cumulative stacked value through that tier, +/// over every (position, empty cell) pair — the population the meter will read. +fn preheat(positions: &[(Board, Cell)], rails: &Rails, arm: Arm) -> TierFloors { + let mut samples: Vec> = vec![Vec::new(); 4]; + for (b, p) in positions { + for m in 0..N { + if b[m] != Cell::E { + continue; + } + let inc = intensity(b, *p, m, rails, arm); + let mut acc = 0.0; + for (t, x) in inc.iter().enumerate() { + acc += x; + samples[t].push(acc); + } + } + } + let mut floors = TierFloors::new(K_SIGMA); + floors.preheat(&samples); + floors +} + +struct Scored { + f1_det: f64, + f1_tie: f64, + f2_equal: f64, + exposed_tiers: f64, + early_frac: f64, + /// Anti-vacuity: positions where EVERY candidate carries the same stacked + /// value — there the ranking is empty and tie-aware F1 is the random-move + /// baseline by construction. + all_tied_share: f64, + /// Mean number of distinct stacked values among a position's candidates. + distinct_mean: f64, + /// Mean distinct FULL-stack values per position; 1.000 = the census (F0). + full_distinct_mean: f64, + /// Tie-aware F1 over the NON-degenerate positions only (None if there are none). + f1_tie_nondegenerate: Option, +} + +/// Rank every position's candidates by the stacked reading and score against +/// the solved optimal set. +fn score( + positions: &[(Board, Cell)], + optimal: &HashMap<(Board, Cell), Vec>, + rails: &Rails, + floors: &TierFloors, + arm: Arm, +) -> Scored { + let (mut hit_det, mut hit_tie, mut equal, mut tiers, mut early) = (0.0, 0.0, 0.0, 0.0, 0.0); + let mut candidates_total = 0usize; + let (mut all_tied, mut distinct_sum, mut nondeg_hit, mut nondeg_n) = + (0usize, 0usize, 0.0, 0usize); + let mut full_distinct_sum = 0usize; + for (b, p) in positions { + let opt = &optimal[&(*b, *p)]; + // (cell, stacked-with-early-exit, full stack, exit tier, early) + let mut rows: Vec<(usize, f64, f64, usize, bool)> = Vec::new(); + for m in 0..N { + if b[m] != Cell::E { + continue; + } + let inc = intensity(b, *p, m, rails, arm); + let mut mine = floors.clone(); + let res = mine.stack_early_exit(inc); + rows.push((m, res.stacked, inc.iter().sum(), res.exit_tier, res.early)); + } + candidates_total += rows.len(); + // Deterministic ranking: descending, then lowest index. `top_early` + // always ranks by the early-exit stacked value (index 1); `top_full` + // always ranks by the full stack (index 2) — F2 below compares these + // two UNCONDITIONALLY, regardless of arm. F1's own ranking (`top`) is + // arm-dependent: AGREEMENT's per-tier terms are non-negative, so the + // early-exit partial is a sound bound and F1 ranks on it directly + // (== `top_early`, unchanged from before this split existed); NET's + // per-tier terms are signed, so a partial is NOT a bound on the full + // stack (see the module doc's "The stack" + the "meter note" below), + // and F1 must rank NET candidates on the full stack (== `top_full`). + let top_early = rows + .iter() + .copied() + .max_by(|a, b| a.1.partial_cmp(&b.1).expect("finite").then(b.0.cmp(&a.0))) + .expect("non-terminal has a move"); + let top_full = rows + .iter() + .copied() + .max_by(|a, b| a.2.partial_cmp(&b.2).expect("finite").then(b.0.cmp(&a.0))) + .expect("non-terminal has a move"); + let top = match arm { + Arm::Agreement => top_early, + Arm::Net => top_full, + }; + if opt.contains(&top.0) { + hit_det += 1.0; + } + let tied: Vec = match arm { + Arm::Agreement => rows.iter().filter(|r| r.1 == top.1).map(|r| r.0).collect(), + Arm::Net => rows.iter().filter(|r| r.2 == top.2).map(|r| r.0).collect(), + }; + let tied_opt = tied.iter().filter(|m| opt.contains(m)).count(); + let tie_score = tied_opt as f64 / tied.len() as f64; + hit_tie += tie_score; + let mut distinct: Vec = match arm { + Arm::Agreement => rows.iter().map(|r| r.1.to_bits()).collect(), + Arm::Net => rows.iter().map(|r| r.2.to_bits()).collect(), + }; + distinct.sort_unstable(); + distinct.dedup(); + distinct_sum += distinct.len(); + let mut distinct_full: Vec = rows.iter().map(|r| r.2.to_bits()).collect(); + distinct_full.sort_unstable(); + distinct_full.dedup(); + full_distinct_sum += distinct_full.len(); + if tied.len() == rows.len() { + all_tied += 1; + } else { + nondeg_hit += tie_score; + nondeg_n += 1; + } + // F2's own question — does early exit change the top move — is + // asked the SAME way regardless of which quantity F1 ranked by. + if top_early.0 == top_full.0 { + equal += 1.0; + } + for r in &rows { + tiers += (r.3 + 1) as f64; + if r.4 { + early += 1.0; + } + } + } + let n = positions.len() as f64; + Scored { + f1_det: hit_det / n, + f1_tie: hit_tie / n, + f2_equal: equal / n, + exposed_tiers: tiers / candidates_total as f64, + early_frac: early / candidates_total as f64, + all_tied_share: all_tied as f64 / n, + distinct_mean: distinct_sum as f64 / n, + full_distinct_mean: full_distinct_sum as f64 / n, + f1_tie_nondegenerate: (nondeg_n > 0).then(|| nondeg_hit / nondeg_n as f64), + } +} + +fn main() { + let positions = reachable(); + let mut memo = HashMap::new(); + let mut optimal: HashMap<(Board, Cell), Vec> = HashMap::new(); + for (b, p) in &positions { + let (_, moves) = solve(b, *p, &mut memo); + assert!( + !moves.is_empty(), + "a non-terminal position has an optimal move" + ); + optimal.insert((*b, *p), moves); + } + let syms = symmetries(); + let classes: HashSet<(u32, Cell)> = positions + .iter() + .map(|(b, p)| (canonical(b, &syms), *p)) + .collect(); + let empty_value = solve(&[Cell::E; N], Cell::X, &mut memo).0; + assert_eq!(empty_value, 0, "tic-tac-toe is a draw"); + + // The random-move baseline: what an uninformed pick scores, per position. + let random_move: f64 = positions + .iter() + .map(|(b, p)| { + let empties = b.iter().filter(|c| **c == Cell::E).count() as f64; + optimal[&(*b, *p)].len() as f64 / empties + }) + .sum::() + / positions.len() as f64; + + println!( + "tic-tac-toe: {} reachable non-terminal positions, {} classes up to symmetry, value of the empty board {empty_value}", + positions.len(), + classes.len() + ); + println!( + "random-move baseline (mean share of value-preserving moves per position): {:.4}\n", + random_move + ); + + let lattice = lattice_rails(); + let horizon = Horizon::of(&lattice); + let horizon_d1 = Horizon::of(°ree_one(&lattice)); + // The gate's own two sides, on non-trivial rails: it must fire on the 3×3 + // lattice (rings 1 ∪ 2 = every other cell) and stay silent at degree 1. + assert!( + !horizon_d1.exhausts_board(), + "F0 must stay silent on degree-1 rails (reach {} of {})", + horizon_d1.max_reach, + N - 1 + ); + // F0 has TWO independent ways to be unreadable, and the topology is only + // the first. A board can be locally bounded — every cell's horizon smaller + // than the board — and STILL give every candidate the same full-stack + // value under the chosen intensity, in which case there is nothing to rank + // and an F1 PASS/KILL would be meaningless. The second half is measured, + // not derived, so it can only be applied after `score` runs; see + // `score_is_degenerate` at the verdict site. Reported by codex on #1239. + let topology_degenerate = horizon.exhausts_board(); + println!( + "F0 fixture validity — ring coverage: each cell's rails reach {}..={} of {} other cells; horizon exhausts the board on {}/{N} cells (degree-1 rails reach {} — the gate stays silent there)", + horizon.min_reach, + horizon.max_reach, + N - 1, + horizon.exhausted_cells, + horizon_d1.max_reach + ); + if topology_degenerate { + println!( + " => FIXTURE DEGENERATE (topology): {}/{N} cells reach every other cell, so their full stack is the board census and carries no ranking information. F1/F2/F3 below are DATA, not verdicts; the readable arm needs a board larger than the rails' horizon.\n", + horizon.exhausted_cells + ); + } else { + println!(" => topology readable: every cell's horizon is smaller than the board (the SCORE is checked separately, per arm, below).\n"); + } + for (arm, name) in [ + (Arm::Agreement, "AGREEMENT (pre-registered)"), + (Arm::Net, "NET own-opp (exploratory)"), + ] { + println!("=== arm: {name}"); + let floors = preheat(&positions, &lattice, arm); + for (t, f) in floors.floors.iter().enumerate() { + println!( + " tier {t} floor: mu {:.3} sigma {:.3} threshold {:.3}", + f.mu(), + f.sigma(), + f.threshold() + ); + } + let s = score(&positions, &optimal, &lattice, &floors, arm); + println!( + " F1 top move optimal: {:.4} (deterministic tie-break) {:.4} (tie-aware) bar {F1_BAR}", + s.f1_det, s.f1_tie + ); + println!( + " F2 early-exit top == full-stack top: {:.4} exposed tiers {:.3} of 4 early-exit share {:.4}", + s.f2_equal, s.exposed_tiers, s.early_frac + ); + println!( + " anti-vacuity: all candidates tied in {:.4} of positions; mean distinct stacked values {:.3} (FULL stack {:.3}; 1.000 = census); F1 tie-aware on non-degenerate positions {}", + s.all_tied_share, + s.distinct_mean, + s.full_distinct_mean, + s.f1_tie_nondegenerate + .map(|v| format!("{v:.4}")) + .unwrap_or_else(|| "n/a".into()) + ); + let d1 = degree_one(&lattice); + let f1 = preheat(&positions, &d1, arm); + let s1 = score(&positions, &optimal, &d1, &f1, arm); + println!( + " F3 degree-1 ablation F1: {:.4} (det) {:.4} (tie-aware) drop {:+.4}", + s1.f1_det, + s1.f1_tie, + s1.f1_tie - s.f1_tie + ); + let mut nulls = Vec::new(); + for seed in 1..=NULL_SEEDS { + let sr = shuffled_rails(&lattice, seed); + let fr = preheat(&positions, &sr, arm); + nulls.push(score(&positions, &optimal, &sr, &fr, arm).f1_tie); + } + let mean = nulls.iter().sum::() / nulls.len() as f64; + let (lo, hi) = nulls + .iter() + .fold((f64::MAX, f64::MIN), |(l, h), &x| (l.min(x), h.max(x))); + println!( + " null shuffled rails, {NULL_SEEDS} seeds, F1 tie-aware: mean {:.4} range [{:.4}, {:.4}]", + mean, lo, hi + ); + // The measured half of F0: if the ranked quantity has one distinct + // value per position, the ranking is empty whatever the topology says. + // `full_distinct_mean` is the mean number of distinct FULL-stack values + // per position; 1.000 means every candidate tied everywhere. + let score_degenerate = s.full_distinct_mean <= 1.0 + 1e-9; + let degenerate = topology_degenerate || score_degenerate; + if score_degenerate && !topology_degenerate { + println!( + " F0 (score): every candidate carries the SAME full-stack value (mean distinct {:.3}) although the topology is bounded — the ranking is empty and F1 below is not a verdict", + s.full_distinct_mean + ); + } + let verdict = if degenerate { + "F0 DEGENERATE — F1 not read (F1 at the baseline is the census signature, not a KILL)" + } else if s.f1_tie >= F1_BAR { + "F1 PASS" + } else if s.f1_tie <= mean + (hi - lo) { + "F1 KILL — at the null" + } else { + "F1 BETWEEN — above the null, below the bar" + }; + let f3 = if degenerate { + "F3 not read (a degree-1 F1 against a census F1 compares nothing)" + } else if s1.f1_tie < s.f1_tie - 1e-9 { + "F3 real (degree 1 drops)" + } else { + "F3 FLAT — the task did not exercise the neighbours" + }; + let f2 = if degenerate { + "not read (the full stack is constant per position; any spread is early-exit partial sums)" + } else if s.f2_equal >= 1.0 - 1e-12 { + "equality holds" + } else { + "verdict CHANGES under early exit" + }; + println!(" verdict: {verdict}; {f3}; F2 {f2}"); + if s.f2_equal < 1.0 - 1e-12 { + println!( + " meter note: early exit changed the top move in {:.4} of positions — `stack_early_exit` reports the PARTIAL sum at the exit tier, and with signed tiers (NET) a partial is not a bound on the full stack; the early-reject premise is monotone non-negative stacking", + 1.0 - s.f2_equal + ); + } + println!(); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + // ---- negamax --------------------------------------------------------- + + #[test] + fn negamax_forced_win_returns_exact_value_one() { + // X to move, top row two-thirds full: cell 2 completes a line right + // now. Whatever else is true of the position, having an immediately + // winning move pins the negamax value at the domain's maximum, 1 — + // no deeper search can ever beat "win this move." + #[rustfmt::skip] + let b: Board = [ + Cell::X, Cell::X, Cell::E, + Cell::O, Cell::O, Cell::E, + Cell::E, Cell::E, Cell::E, + ]; + assert!(winner(&b).is_none(), "fixture must not already be decided"); + let mut memo = HashMap::new(); + let (value, moves) = solve(&b, Cell::X, &mut memo); + assert_eq!( + value, 1, + "an immediate winning move must yield the maximum negamax value" + ); + assert!( + moves.contains(&2), + "cell 2 completes the top row and must be among the optimal moves" + ); + } + + #[test] + fn negamax_forced_draw_returns_exact_value_zero() { + // The empty board, X to move: the single best-established fact about + // tic-tac-toe — perfect play from the start draws. `main()` leans on + // this exact same fact (`assert_eq!(empty_value, 0, ...)`); pinning + // it here as a dedicated unit test means a `solve`/`winner`/ + // `terminal` regression is caught by `cargo test`, not only by a + // one-off assertion buried inside `main`. + let mut memo = HashMap::new(); + let (value, moves) = solve(&[Cell::E; N], Cell::X, &mut memo); + assert_eq!( + value, 0, + "tic-tac-toe from the empty board is a draw under perfect play" + ); + assert!( + !moves.is_empty(), + "a non-terminal position must report at least one optimal move" + ); + } + + // ---- reachable-position filtering ------------------------------------- + + #[test] + fn reachable_and_symmetry_classes_stay_linked() { + // Deliberately NOT pinned against the classic external combinatorics + // trivia for tic-tac-toe (the well-known ~5478/~765 figures) — those + // numbers were not independently re-derived here and hard-coding a + // misremembered one would be worse than no test at all. Instead this + // pins a relationship between `reachable()` and the up-to-symmetry + // class count that is TRUE BY CONSTRUCTION whenever the underlying + // code is correct, so the two can never drift apart silently even + // without trusting an external number. + let positions = reachable(); + assert!( + positions.contains(&([Cell::E; N], Cell::X)), + "the empty board is the BFS seed and is itself non-terminal" + ); + for (b, _) in &positions { + assert!( + !terminal(b), + "reachable() must never report an already-decided board" + ); + } + let syms = symmetries(); + let classes: HashSet<(u32, Cell)> = positions + .iter() + .map(|(b, p)| (canonical(b, &syms), *p)) + .collect(); + // Canonicalization can only ever coarsen: `classes` is the image of + // `positions` under `canonical`, so its cardinality cannot exceed the + // domain's. + assert!( + classes.len() <= positions.len(), + "canonicalization must not manufacture positions: {} classes over {} positions", + classes.len(), + positions.len() + ); + // Every orbit under the eight square symmetries has size at most 8, + // so no canonical class can be the image of more than 8 positions — + // a bug that OVER-merges distinct positions (e.g. an `encode` + // collision, or a broken `canonical` that isn't actually a per- + // symmetry minimum) would drive `classes.len()` below this floor. + assert!( + classes.len() * 8 >= positions.len(), + "an orbit of size <= 8 cannot make {} classes cover {} positions", + classes.len(), + positions.len() + ); + // Anti-vacuity: the two bounds above both hold even if `classes.len() + // == 1` (total collapse) or `classes.len() == positions.len()` (no + // reduction at all) — neither of which is what symmetry reduction on + // a real board should do. Assert the reduction is genuine and partial. + assert!( + classes.len() > 1 && classes.len() < positions.len(), + "canonicalization must produce a real, non-trivial reduction: {} classes from {} positions", + classes.len(), + positions.len() + ); + } + + // ---- horizon detection ------------------------------------------------- + + #[test] + fn horizon_detects_the_full_lattice_as_exhausted() { + let lattice = lattice_rails(); + let h = Horizon::of(&lattice); + assert!(h.exhausts_board()); + assert_eq!( + h.exhausted_cells, N, + "every cell's ring1+ring2 rails reach all 8 of the other cells on 3x3" + ); + assert_eq!(h.max_reach, N - 1); + } + + #[test] + fn horizon_stays_silent_on_degree_one_rails() { + let d1 = degree_one(&lattice_rails()); + let h = Horizon::of(&d1); + assert!(!h.exhausts_board()); + assert_eq!(h.exhausted_cells, 0); + assert_eq!(h.max_reach, 1); + } + + #[test] + fn horizon_detects_partial_exhaustion_from_a_single_cell() { + // Built directly as a `Rails` value (not derived from a board or from + // `lattice_rails`): cell 0's single ring reaches every one of the + // other 8 cells (exhausted), while every other cell keeps exactly one + // harmless ring-1 rail back to cell 0 (nowhere near exhausted). This + // is exactly the case the `exhausts_board` doc comment's fix + // addresses: the OLD predicate (`exhausted_cells == N`) would have + // read this as fine (1 != 9), while the fixed, strictly-any predicate + // (`exhausted_cells > 0`) must flag it. + let mut rails: Rails = vec![Default::default(); N]; + rails[0][0] = (1..N).collect(); + for i in 1..N { + rails[i][0] = vec![0]; + } + let h = Horizon::of(&rails); + assert_eq!(h.exhausted_cells, 1, "exactly cell 0 is exhausted"); + assert_eq!(h.max_reach, N - 1); + assert_eq!(h.min_reach, 1); + assert!( + h.exhausts_board(), + "ANY exhausted cell must contaminate the board, not only ALL cells" + ); + } + + // ---- rail shuffling ------------------------------------------------------ + + #[test] + fn shuffled_rails_permutes_targets_and_preserves_per_ring_counts() { + // A hand-built `Rails` with distinct, easily-checked ring sizes per + // cell (3/3/2/0), independent of `lattice_rails`'s own correctness, + // so the shuffle's ring-by-ring "deal it out" step has real structure + // to preserve. + let mut rails: Rails = vec![Default::default(); N]; + for i in 0..N { + let others: Vec = (0..N).filter(|&j| j != i).collect(); + rails[i][0] = others[0..3].to_vec(); + rails[i][1] = others[3..6].to_vec(); + rails[i][2] = others[6..8].to_vec(); + } + let shuffled = shuffled_rails(&rails, 7); + for i in 0..N { + for r in 0..4 { + assert_eq!( + rails[i][r].len(), + shuffled[i][r].len(), + "cell {i} ring {r}'s rail COUNT must survive the shuffle" + ); + } + let mut before: Vec = rails[i].iter().flatten().copied().collect(); + let mut after: Vec = shuffled[i].iter().flatten().copied().collect(); + before.sort_unstable(); + after.sort_unstable(); + assert_eq!( + before, after, + "cell {i}'s shuffled targets must be a PERMUTATION of the originals, \ + never an invention or a drop" + ); + assert!( + !after.contains(&i), + "a cell must never rail to itself after shuffling" + ); + } + // Anti-vacuity: every assertion above would also pass for an + // implementation that returns `rails` unchanged. Assert the shuffle + // actually moved something. + let moved = (0..N).any(|i| rails[i] != shuffled[i]); + assert!( + moved, + "shuffled_rails(seed=7) must not be the identity permutation" + ); + } + + // ---- signed-arm ranking -------------------------------------------------- + + #[test] + fn net_arm_ranks_by_the_full_stack_not_the_early_exit_partial() { + // A hand-built board + rails where cell 0's tier-0 term alone (+3, + // all-own) trips the Alarm on a virgin floor and stops the cascade + // before its own negative tiers 1..3 ever get added, while cell 1 + // sails through all four tiers on net-zero terms and only shows its + // value at the leaf. Every number below is hand-traced against + // `intensity`'s own own-opp formula and `stack_early_exit`'s own + // preheat/observe mechanics — nothing here is asserted "because the + // compiler said so." + #[rustfmt::skip] + let b: Board = [ + Cell::E, Cell::E, Cell::X, + Cell::X, Cell::O, Cell::O, + Cell::X, Cell::O, Cell::X, + ]; + let p = Cell::X; + let mut rails: Rails = vec![Default::default(); N]; + // Candidate A (cell 0): tier 0 alone is +3 (own-dominant, all-X + // ring), tiers 1..3 are each a single opposing stone (-1 apiece). + rails[0][0] = vec![2, 6, 8]; + rails[0][1] = vec![4]; + rails[0][2] = vec![5]; + rails[0][3] = vec![7]; + // Candidate B (cell 1): every non-leaf tier nets to zero (one X, one + // O each); only the leaf tier carries a net +1. + rails[1][0] = vec![2, 4]; + rails[1][1] = vec![3, 5]; + rails[1][2] = vec![6, 7]; + rails[1][3] = vec![8]; + + let floors = TierFloors::new(K_SIGMA); + let inc_a = intensity(&b, p, 0, &rails, Arm::Net); + let inc_b = intensity(&b, p, 1, &rails, Arm::Net); + assert_eq!(inc_a, [3.0, -1.0, -1.0, -1.0]); + assert_eq!(inc_b, [0.0, 0.0, 0.0, 1.0]); + let res_a = floors.clone().stack_early_exit(inc_a); + let res_b = floors.clone().stack_early_exit(inc_b); + let full_a: f64 = inc_a.iter().sum(); + let full_b: f64 = inc_b.iter().sum(); + // The scenario's whole point: the two readings DISAGREE on which + // candidate is best. + assert!( + res_a.stacked > res_b.stacked, + "A's early-exit partial ({}) must beat B's ({}) — that is the OLD \ + ranking this test would otherwise vindicate", + res_a.stacked, + res_b.stacked + ); + assert!( + full_b > full_a, + "B's full stack ({full_b}) must beat A's ({full_a}) — that is the \ + signal a signed arm's F1 ranking must use" + ); + assert!( + res_a.early, + "A must exit early, before its own negative tiers land" + ); + assert!( + !res_b.early, + "B must run to the leaf, where stacked == full" + ); + + let mut optimal = HashMap::new(); + optimal.insert((b, p), vec![1usize]); + let positions = vec![(b, p)]; + let s = score(&positions, &optimal, &rails, &floors, Arm::Net); + assert_eq!( + s.f1_det, 1.0, + "Arm::Net must rank by the full stack and pick cell 1, not the \ + early-exit-favoured cell 0 (reverting the fix flips this to 0.0)" + ); + } +}