Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions .claude/board/EPIPHANIES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,96 @@
## E-NO-FOLD-REPORTS-AN-O-POPULATION-COST-1 (2026-09-18) — a "fold" that materializes a population- or lane-sized buffer is a sweep wearing a fold's name

**The rule, stated once:** a fold's cost is a function of its ANSWER's size,
never of the population it was searched over. An allocation sized to the lane,
or a per-row predicate — even a narrowed one — is a materialization, and a
measurement that includes one is not measuring a fold.

**What happened.** The first commit-2 draft of the D-DIAMOND-1 probe measured
two things and called both "folds" while neither was one:

- **P2's write** allocated `dst = vec![0u64; words_for(n_rows)]` — sized to the
WHOLE lane — before calling `mask_set_range(&mut dst, lo, hi)`.
`mask_set_range` writes every word of whatever slice it is given (zero
before, ones inside, zero after), so the cost was O(n_rows) regardless of
how narrow `[lo, hi)` was. The bound search itself (`partition_point` × 2)
was a real fold; the write immediately after it silently reintroduced an
O(N) term and the report did not distinguish the two.
- **P3's "fold" arm A** called `ternary_match_u64_to_mask` — the shipped
per-row ternary-match SWEEP — narrowed to the bound's row range. Narrowing
the RANGE a sweep runs over does not change that it is still a sweep: every
row in the narrowed range is still individually compared. It was reported
next to "arm B: two sweeps + AND" as though it were a different KIND of
operation; it was the same kind, just over fewer rows.

**The diagnosis, stated generally:** a materialization test for "is this
actually a fold" is not "does it look like one API call" — it is **does its
cost scale with the population touched, or only with the answer's own size**.
A bound's answer is `(lo, hi)` — two integers — and a fold-shaped consumer of
it (a narrowed AND, a popcount over `hi − lo` bits) costs O(range width) or
O(1), never O(N). The moment a fold's output is unconditionally turned into a
buffer sized by the LANE rather than by the RANGE, or into a call that visits
every row rather than every match, the O(N)/O(sweep) term is back — just
hidden one call deeper than the reviewer looked.

**The fix, both instances (D-DIAMOND-1 commit 2, `crates/d-diamond-1-probe`):**

- P2: `touched_write(lo, hi)` returns `(w0, dst)` — a base word index
`w0 = lo / 64` and a buffer of `words_for(hi) - w0` words, so cost is
O((hi − lo) / 64) for a range at ANY position. `n_rows` never appears in its
signature. Flat 20.5–22.6 ns across N = 1K → 1M at a fixed range (old
whole-lane buffer over the same range: 34 → 4,620 ns, ~134×) AND flat
20.5–21.7 ns across positions 500 → 3,999,900 at a fixed 100-row width. The
full-lane sweep is kept ONLY as an explicitly separate `reference_sweep_ns`
column, never summed into the fold's own total.

**This bullet is itself the second correction, and the sharper half of the
lesson.** Its first version sized the buffer to `words_for(hi)` and wrote
from word 0 — so `mask_set_range` zeroed every word BEFORE `lo` and the cost
was O(hi), the range's END POSITION in the lane. That is still a
population-shaped cost for a range near the lane's end, and the flatness
falsifier could not see it, because holding `(lo, hi)` at a FIXED ABSOLUTE
position across N holds `hi` constant by construction. Fixing the answer and
varying N is NOT sufficient; the answer has a position as well as a size, and
a cost proportional to position passes every N-sweep unchallenged. Falsifier:
`f_touched_write_is_position_independent` — fixed width, moving position;
red against the old shape, green against the fix.
- P3: the fold arm is rebuilt around a `JointIndex` — a Morton-interleaved
joint key over BOTH lanes, sorted once (a real, separately-timed cost), then
bounded with two `partition_point`s and NOTHING ELSE. Verified structurally,
not by report: `ternary_match_u64_to_mask` appears in the probe crate ONLY
inside the `reference_sweep_ns` timing block — zero occurrences in
`JointIndex`. Bound alone: 69–79 ns. **Bound + `materialize_rows`: 89–98 ns**
— and that is the number to quote, because the comparator produces a full
original-ordinal mask while a bound alone produces two offsets into the
JOINT index's own order. Against 745,473–797,268 ns for two sweeps + AND:
**8,135×–8,376×**. The remap is O(kept), so it is a legitimate fold cost, but
omitting it compares inequivalent outputs and inflated the ratio to
~10,700×. Conditional on a prebuilt `JointIndex` (≈61 ms per 1M rows).

**The reusable check, for any future "we measured a fold" claim:** name the
quantity the reported cost is a function of. If it is a function of N, or of
the number of rows touched by a predicate rather than the number of rows in
the ANSWER, it is not a fold measurement — it is a narrowed sweep or a
population-sized buffer, whatever the code calls it.

And then vary the answer along EVERY axis it has, not just its size. Three
tests, not two, are the shape this check demands:
`p2_touched_write_cost_does_not_scale_with_lane_size` (fix the answer, vary N),
`p2_touched_write_beats_the_old_whole_lane_sized_buffer` (the old shape is
measurably worse), and `f_touched_write_is_position_independent` (fix the
answer's SIZE, move its POSITION). The third exists because the first two were
both green over a cost that was still O(end position) — a benchmark that varies
one parameter certifies exactly one parameter.

Cross-ref: `three-prefix-fold-carriers.md` §1 (*"masking wins when the slice
is GRANULAR, PEEK wins when the slice is ADDRESSED"* — this entry adds the
third case: **BOUND wins when the population is ORDERED, and its cost is the
answer's size, never the lane's**); D-DIAMOND-1 plan §5 (the full corrected
numbers); `E-BYTES-ARE-STORED-INTEGERS-ARE-PROJECTED-1` (the sibling
discipline one layer down — a fold's OUTPUT can be a projection too: `(lo,
hi)` stored as two integers, materialized into a mask only at the point a
consumer genuinely needs one).

## 2026-09-18 — E-BYTES-ARE-STORED-INTEGERS-ARE-PROJECTED-1 — byte-agnosticism is the STORAGE superpower and little-endian is the COMPUTE superpower; the bug is always a stored projection

**Status:** OPERATOR-RULED (the framing is the operator's: *"byte is storage
Expand Down
17 changes: 17 additions & 0 deletions .claude/board/INTEGRATION_PLANS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,20 @@
## 2026-09-18 (1) — d-diamond-1-dual-fold-substrate-v1 — can one 8×2×8-shaped carrier carry both point-peek and population-mask traversal → `.claude/plans/d-diamond-1-dual-fold-substrate-v1.md`

**Status:** ACTIVE (operator-directed 2026-09-18; one probe arc; verdict fixed in
advance as PROVEN / BOUNDED / FALSIFIED). Two rulings locked and shipped first, with
tests before any optimization code: **R1** tile 0 is canon — the whole-facet
`shared_prefix_tiles` lens counted APP_PREFIX before the concept because the LE image
stores `custom` at bytes 0..2; the projection is corrected, the image is not touched
(`ISS-SHARED-PREFIX-TILES-CLASSID-INVERSION`). **R2** the ordering witness — «numeric
projection order over the canonical LE image», «storage-attested, planner-consumed»;
no witness → sweep, false witness → the bound is unavailable. Then four measured
arms over 1M skewed synthetic keys: P1 point (8-tile ancestry, six pair classes),
P2 field (witnessed bound + `mask_set_range` vs `MatchU64` sweep, crossover N),
P3 fold intersection over one ordinal, P4 sealed reader under an open writer.
Explicitly NOT authorized: GridLake landing, address-derived placement, `NodeGuid` /
`CausalEdge64` changes, the JC clippy fix, DAG folding, Hamming folds, value-slab
decoding, planner cost-model work.

## 2026-09-17 (1) — three-carrier-blast-radius-v1 — how far do the three prefix-fold carriers reach, and where do they touch → `.claude/plans/three-carrier-blast-radius-v1.md`

**Status:** PROPOSAL. Read-only census + seam map; gates only, no code
Expand Down
65 changes: 65 additions & 0 deletions .claude/board/ISSUES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,68 @@
## ISS-WITNESSED-RANGE-DOES-NOT-ATTEST-PLANE-ORDER (2026-09-18) — OPEN (D-DIAMOND-1, found in review of #1250)

**What it is.** `Filter::prefix_facet`'s `Bound` lowering emits `Cmp::Range { lo, hi }`
where `lo`/`hi` are **ordinals in the sealed lane's order**. `SealedFacetLane::seal`
sorts its own private key vector. Nothing ties that order to the order of the `Planes`
the program actually executes over: `lane_col` is provenance only, `Pred::Range` reads
no lane, and a `Planes` value carries no order identity. If the planes are in their
original order — or belong to any other same-sized lane — the range selects or
aggregates unrelated rows, and no layer can tell.

**Why it is not closed by this PR.** Closing it needs one of two substrate changes,
neither of which belongs in a probe arc:

- `seal` exposes its permutation, and the caller is required to apply it to every
aligned plane (making the planes definitionally the lane's order); or
- the execution surface gains a row-order identity that a witness can attest against,
so the check is mechanical rather than a caller promise.

**What was done instead.** The precondition is now stated explicitly on
`Filter::prefix_facet` as a caller obligation, and `PrefixLowering::Bound` carries the
lane's `version` and order-sensitive `digest` so an execution layer that DOES know its
own row order can reject a mismatch one level up. A test asserts the evidence actually
reaches the caller. That makes the gap detectable and recorded; it does not make it
enforced.

**Falsifier when it is closed.** Build planes in a deliberately different order from
the sealed lane, run a witnessed prefix, and assert the mismatch is refused — not that
it returns a plausible wrong answer. No such test can be written today, which is the
issue.

## ISS-SHARED-PREFIX-TILES-CLASSID-INVERSION (2026-09-18) — RESOLVED at the lens (D-DIAMOND-1 R1, commit 1); recorded because it was latent, not because it was live

**What it was.** `FacetCascade::shared_prefix_tiles` — the whole-facet 8-tile prefix
lens, "classid tiles 0–1 first, then the 6 cascade tiers" — ran `trailing_zeros / 16`
straight over `as_u128()` of the LE image. Since the canon-high flip
(`ogar_codebook::ClassidOrder::CanonHigh => (canon << 16) | custom`,
D-CLASSID-CANON-HIGH-FLIP), the LE image holds **`custom` (APP_PREFIX) at bytes `[0..2)`
and `canon` (the concept) at `[2..4)`**, so the lens read tile 0 = app, tile 1 = concept:

- **the semantic prefix is canon-first** — the concept is the shared, coarse thing; the
app prefix is the fine, per-render thing (OGAR `OGAR-CONSUMER-BEST-PRACTICES.md`);
- **the raw LE byte order is custom-first at the classid boundary** — a property of
storing a `u32` little-endian, not a traversal order anyone chose;
- **zero current callers** — verified by reading (`shared_prefix_tiles` /
`prefix_distance` appear only in `facet.rs` and its tests), so nothing shipped read
the inverted count;
- **latent until whole-facet traversal used the lens** — which is exactly what
D-DIAMOND-1 proposes, and how it was found: `shared_prefix_tiles(account.move@odoo,
account.move@medcare)` was `0`, `shared_prefix_tiles(account.move@odoo,
res.partner@odoo)` was `1`.

**What changed.** Only the projection: the two classid tiles of the XOR are swapped
(`rotate_left(16)` on the low 32 bits) before counting. **The stored LE image, the ABI and
every serialized form are unchanged** — F1 asserts the bytes byte-for-byte before and after.
One existing test expectation flipped (`redout_is_granularity_free_and_orthogonal`: flipping
image bit 0 flips `custom`, so the corrected lens reports 1 shared tile, not 0), annotated
in place. Red-first: F1 and the F5 depth test failed against `a2a51012` before the fix.

**Why it is worth an entry though nothing shipped read it.** It is the −32 smell of
`three-prefix-fold-carriers.md` §2 one layer down: a coordinate system (LE integer
storage) leaking into an operation that wanted a different one (coarse→fine hierarchy).
The cure is the same doctrine as #1248 — bytes are stored, integers are projected — and
the R2 witness is built on the projections (`semantic_tiles`, `cmp_numeric_projection`),
never on byte order, so the inversion cannot recur on the bound path.

## ISS-EDGE-BLOCK-WAS-A-SECOND-TYPE-FOR-THE-SAME-FACET (2026-09-17) — RESOLVED at the type; the readers that still split at 12 are the named residue

**Operator ruling (verbatim, 2026-09-17):** *"It's forbidden for the edge block to even
Expand Down
21 changes: 21 additions & 0 deletions .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,24 @@
## D-DIAMOND-1 — dual fold substrate over the shipped `FacetCascade` (D-ids minted 2026-09-18, plan `.claude/plans/d-diamond-1-dual-fold-substrate-v1.md`)

Operator-directed probe arc from `main` `a2a51012`: «Can one canonical 8×2×8-shaped carrier
support both point-peek and population-mask traversal, with semantic hierarchy reduced to
prefix/bound folds, while async writes remain invisible to sealed readers?» Verdict vocabulary
fixed in advance: PROVEN / BOUNDED / FALSIFIED. Fence: no GridLake, no placement, no JC
cleanup, no planner redesign, no new graph abstraction.

| D-id | scope | status | gate / falsifier |
|---|---|---|---|
| D-DMD-R1 | tile 0 is canon: fix the semantic projection in `FacetCascade::shared_prefix_tiles` (classid tile swap on the XOR); stored LE image unchanged | **Shipped (commit 1).** `ISS-SHARED-PREFIX-TILES-CLASSID-INVERSION` recorded; one existing expectation flipped and annotated | F1 seen RED against `a2a51012` (lens said 0 shared tiles for same-concept/different-app), GREEN after; F5 depth test likewise |
| D-DMD-R2 | the ordering witness — «numeric projection order over the canonical LE image», «storage-attested, planner-consumed»: `ordered_lane::{OrderedLaneWitness, SealedFacetLane, WitnessError}`, `FacetCascade::{semantic_tiles, from_semantic_tiles, cmp_numeric_projection, semantic_u64_halves}`, `SemanticPrefix` | **Shipped (commit 1).** contract 1356 → 1367 tests | R2 tuple order == semantic-tile lexicographic, ≠ byte-wise image order; F2 shuffled lane unattestable + unwitnessed bound caught; F3 forged/stale/re-sealed witness rejected before any bound |
| D-DMD-L | witnessed prefix → bound lowering: quack `Cmp::Range` + `PrefixLowering` + `Filter::prefix_facet` (no witness → `MatchU64` sweep; rejected witness → sweep + reason) | **Shipped (commit 1).** quack 14 → 17 tests | bound and sweep lower the SAME predicate at every depth 0..=8 (differential vs oracle); a rejected witness emits no `Range` leaf |
| D-DMD-L2 | R2 gains a `SemanticLens`: storage is a content-blind ordinal, "sorted" only means something under a named projection — the witness now carries `lens` and `bound()` rejects a mismatch before searching | **Shipped (commit 2).** Forced mid-arc: operator rejected a first probe draft that put a per-row sweep on P3's "fold" arm and an O(N) mask allocation on P2's write — see `E-NO-FOLD-REPORTS-AN-O-POPULATION-COST-1` | contract 1367 → 1368 tests; one lens variant shipped (`CanonHighTiles8`); `WitnessError::LensMismatch` |
| D-DMD-P1 | point universe: 8-tile `is_ancestor` latency, six pair classes, corrected tzcnt lens vs peek chain | **Shipped (commit 2).** All three arms cluster 1.7–4.2 ns; the 1.72 ns axis-chain number does NOT transfer to the whole-facet 8-tile cell | oracle-first verified; `is_ancestor` checked against all three arms + the class's expected LCP before timing |
| D-DMD-P2 | field universe: witnessed bound + `touched_write(lo,hi) -> (w0, dst)` (base word offset; cost O((hi−lo)/64) at any position, never `words_for(n_rows)`) vs a `reference_sweep_ns` comparator (never summed into the fold total) | **Shipped (commit 2, corrected in review).** Flat 22.0–23.1 ns across N=1K→1M (1000×) AND flat 20.5–21.7 ns across positions 500→3,999,900 at fixed 100-row width; old whole-lane-sized buffer grew 34→4,620 ns (~134×). Speedup 119×–707× at 1M | F4 anti-vacuity on every timed range; flatness falsifier across N; `f_touched_write_is_position_independent` across position (disable-verified red against the first fix, which was O(end position)) |
| D-DMD-P3 | fold intersection over one ordinal via a probe-only `JointIndex` (Morton-interleaved joint key, equal depths only, `JOINT_MAX_DEPTH=4`) — zero sweep calls in the fold path, verified structurally (`ternary_match_u64_to_mask` appears ONLY inside `reference_sweep_ns`) | **Shipped (commit 2, corrected in review).** Build 61.2 ms (1M rows, once — the win is CONDITIONAL on this index existing); bound 69–79 ns thereafter, **89–98 ns including `materialize_rows`** (the comparable column: the comparator emits a full original-ordinal mask) vs 745,473–797,268 ns for two sweeps + AND — **8,135×–8,376×** (timing the bound alone inflated this to ~10,700×). Tenant lane confirmed unattestable over the ontology ordinal (`WitnessError`, inversion at row 1) | F4: `kept ∩ >= 32` (INTERSECTION_FLOOR), neither side a subset; no join structure, no shipped second lens |
| D-DMD-P4 | sealed reader under an open writer: peek/bound distributions (min/median/p90) ± writer, seal-sort, attest, publish | **Shipped (commit 2).** No perturbation beyond noise; pinned lane's digest and validation unchanged while the writer published a strictly higher version | «open-lane producer arrival order must not perturb reads from the sealed image» — held |
| D-DMD-F | the five falsifiers F1–F5 | **All shipped and green** — F1/F2/F3/F5 at contract+lowering level (commit 1), F4 enforced at runtime throughout P2/P3 (commit 2) | see plan §3 |
| D-DMD-V | **Verdict: BOUNDED.** The dual peek/mask substrate works and the bound/fold wins materially in its region, but the region has named edges: P1's 1.72 ns does not transfer to the 8-tile whole-facet cell; P2's crossover is real (≈N=256–512) and depth-dependent; P3's ~8,200× fold (materialized, the comparable output) is CONDITIONAL on a prebuilt `JointIndex` (~61 ms/1M rows, amortized over queries) that a second, independently-ordered lane needs by construction, since one sequence is monotone under one lens at a time | **Landed.** Full narrative: plan §5 | none — this is the terminal ruling for D-DIAMOND-1 |

## three-carrier prefix folds (D-ids minted 2026-09-17, plan `.claude/plans/three-carrier-blast-radius-v1.md`)

Arose from the −32 offset correction sweep (#1244) and the operator's challenge to it.
Expand Down
Loading
Loading