Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,43 @@
## 2026-09-18 — PR #1246 merged (`568965e9`): `EdgeBlock` is a `FacetCascade` on `main`, `Pred::Range` is in the mask-risc IR, and no manifest in this repo carries a `.0.0` pin

The two entries below dated 2026-09-17 (3) and (2) describe what is now on
`main`, not what is in PR: the `EdgeBlock` contract-inventory delta and
`D-MRX-7`. Read them as shipped.

- **New in the pin doctrine (operator, 2026-09-18):** *"never pin to x.00,
always float x.*"* / *"so no decimal .0.0"*. Eight exact-equals pins are
gone, one per DECLARATION: `lance` / `lance-linalg` / `lance-index` now
`11.*` and `lancedb` `0.38.*` in the workspace table (4), `lance` in
holograph (5), `lance-namespace` in `lance-graph-catalog` (6), and
`lance-namespace` + `lance-arrow` in `lance-graph` (7, 8).
`lance-namespace` is floated TWICE — it is declared in two manifests —
which is why an earlier wording that named it once read as seven. Resolution is UNCHANGED — arrow
58.4.0, datafusion 54.1.0, lancedb 0.38.0, lance family 11.0.0 — verified
byte-identical on throwaway lockfiles before and after, then compiled and
tested end-to-end by CI on the merge parent. `arrow` and `datafusion` were
already caret. **CLAUDE.md § Key Dependencies is updated in the same
arc**: its `=11.0.0` / `=0.38.0` lines were stale the moment the manifest
changed, and the superseded 2026-09-05 half-sentence is struck in place
rather than deleted.
- **New contract surface:** `FacetCascade::as_bytes_mut`, plus a
`target_endian = "little"` compile-time assert on `FacetCascade`. The
assert exists because `as_bytes()` is a raw reinterpret while `to_bytes()`
is an explicit LE encode of `facet_classid` — identical on LE, divergent
on BE, and the struct's in-memory image IS the canonical stored row image.
Failing at compile time beats corrupting a row at runtime. The shape that
needs no guard — `edges` byte-backed the way `NodeGuid` is, with the lanes
projected — is named and NOT built.
- **State consumers should know:** nothing about byte positions moved.
`NODE_ROW_STRIDE`, `node_rows_from_le_bytes` and
`ENVELOPE_LAYOUT_VERSION` are all unchanged; `EdgeBlock::default()` and
equality still compile.
- **Gated ranges cost two passes:** `OpHistogram` gained `ranges`, and a
gated `Pred::Range` charges `two_input` as well, because `exec.rs` runs
`mask_set_range` and then `mask_and_assign`. Every lane predicate has a
fused `*_to_mask_under` kernel and stays free under a gate; Range is the
sole exception until `mask_set_range_under` exists upstream. Arc:
`PR_ARC_INVENTORY.md` 2026-09-18 #1246.

## 2026-09-17 (3) — CONTRACT INVENTORY DELTA: `EdgeBlock` is now `pub type EdgeBlock = FacetCascade` (the V1 `12 + 4` struct is gone); `FacetCascade::as_bytes_mut` added

- **State consumers should know:** bytes 16..32 of `NodeRow` are the same
Expand Down
70 changes: 70 additions & 0 deletions .claude/board/PR_ARC_INVENTORY.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,73 @@
## 2026-09-18 — lance-graph PR #1246 (merged `568965e9`, branch `claude/great-pascal-k96kok`) — `EdgeBlock` becomes `FacetCascade`, `Pred::Range` joins the mask-risc IR, and every `.0.0` exact pin floats

- **Added:** `Pred::Range { lo: u32, hi: u32 }` + `ExecError::RangeOutOfBounds`
in `lance-graph-mask-risc` (D-MRX-7); `OpHistogram::ranges`;
`FacetCascade::as_bytes_mut`; a `target_endian = "little"` compile-time
assert on `FacetCascade` with a round-trip test proving `as_bytes()` and
`to_bytes()` agree on a non-zero `facet_classid`.
- **Retired:** the V1 `EdgeBlock { in_family: [u8; 12], out_family: [u8; 4] }`
struct. `EdgeBlock` is now `pub type EdgeBlock = FacetCascade` — bytes
16..32 are the same content-blind `4 + 12` facet as the key, and the 12+4
split no longer exists as a type. Byte positions, `NODE_ROW_STRIDE` and
`node_rows_from_le_bytes` are unchanged; no `ENVELOPE_LAYOUT_VERSION` bump.
Operator ruling `E-THE-SECOND-FACET-IS-NOT-AN-EDGE-BLOCK-1`: *"It's
forbidden for the edge block to even know it's an edge block."*
- **Pin doctrine changed (operator, 2026-09-18):** *"never pin to x.00,
always float x.*"* / *"so no decimal .0.0"*. Eight exact-equals pins
floated, one per DECLARATION: `lance` / `lance-linalg` / `lance-index` /
`lancedb` in the workspace table (4), `lance` in holograph (5),
`lance-namespace` in `lance-graph-catalog` (6), and `lance-namespace` +
`lance-arrow` in `lance-graph` (7, 8). `lance-namespace` is floated TWICE
because it is declared in two manifests — the earlier wording named it
once and so read as seven, which is what a reviewer counted. Verified a resolution NO-OP against throwaway
lockfiles before and after: arrow 58.4.0, datafusion 54.1.0, lancedb
0.38.0, the whole lance family 11.0.0, byte-identical. `arrow` and
`datafusion` were already caret and were not touched. CLAUDE.md's pin
block carried `=11.0.0` / `=0.38.0` until this arc's hygiene entry; the
superseded half-sentence is struck in place there, not deleted.
- **Locked:** a gated `Pred::Range` costs TWO mask passes, not one. Every
LANE predicate has a fused `*_to_mask_under` kernel, so gating one is
free; Range is the sole exception — `exec.rs`'s `run_pred`, in its
`(Pred::Range { .. }, Some(u))` arm, runs `mask_set_range` and THEN
`mask_and_assign`. Charged to `two_input` (the second pass
literally IS an `and`), which leaves the asymmetry visible in the
histogram instead of hidden behind a range-specific name. The fused
primitive that would close it is named in the doc comment as
`mask_set_range_under`; it does not exist upstream.
- **Deferred / named, not built:** `mask_set_range_under`; the planner-side
ordering witness quack's `prefix_u64` lowering waits on (the IR does not
know a lane is address-ordered, and `Filter` carries no ordering
evidence); the readers that still split at 12 on their own authority
(`ISS-EDGE-BLOCK-WAS-A-SECOND-TYPE-FOR-THE-SAME-FACET`); making `edges`
byte-backed the way `NodeGuid` is, which would retire the new
`target_endian` guard entirely.
- **Withdrawn before landing:** `EdgeCodecFlavor::Refs16`, `EdgeRefs`, the
`T8` facet shape, `ColumnDescriptor::class_id`, and a `le-contract.md` §3c
ruling. None reached `main` or the branch.
- **Review:** codex P2 on the endian exposure — real, and the fix is the
compile-time assert plus two stale `canonical_node.rs` SAFETY comments
that still justified themselves with "EdgeBlock is plain `[u8; _]`".
CodeRabbit raised three; one (the gated-range undercount) was correct and
is the Locked item above, and CodeRabbit confirmed the fix. Two did NOT
survive checking and were declined with reasons: `mask_set_range` costs
ONE mask pass, not three (every word of `out_words` is written exactly
once across disjoint segments), and `STATUS_BOARD.md` is ascending by
D-id, so the prepend rule — which names the append-only ledgers — does not
reach it. CodeRabbit withdrew the second and recorded a learning.
- **CI note:** `member-tests` was CANCELLED on both attempts of run
35330298662 with all 22 work steps green, dying in the
`Post Run Swatinem/rust-cache@v2` teardown. No test failed on either
attempt; `test`, `test-with-coverage`, Build and Style Check were green.
The one licensed re-run was spent on attempt 2; the operator merged during
attempt 2's teardown.
- **Confidence:** HIGH that the type change is byte-neutral (positions,
stride and the `from_le_bytes` path are untouched, and every in-tree field
site moved mechanically to the same bytes). HIGH that the pin change is a
resolution no-op (two throwaway lockfiles, byte-identical, plus a green
`Build` and a green `test` on `main`'s merge parent). MEDIUM on the
endian guard being the RIGHT shape rather than a correct stopgap — the
byte-backed `edges` above is the shape that needs no guard.

## 2026-09-17 — lance-graph PR #1245 (merged `83369cad`, branch `claude/great-pascal-k96kok`) — the facet's per-axis LCP goes BACK to the `shared6` byte chain, verbatim; #1241's masked readout is the test oracle now; three prefix-fold carriers named

- **Added:** `examples/facet_axis_lcp_probe.rs` — four-arm benchmark
Expand Down
2 changes: 1 addition & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ evaluates one; `execute` stays the consumer's call on a scratch it owns.
| D-MRX-4 | `ternlog_dispatch.rs` — generated 256-arm match, regenerate-and-diff gated | Shipped (#1226 `0b1ebaa`; PR3; CI `--check` line wired) | all 256 immediates equal the bit-serial reference |
| D-MRX-5 | the differential suite at 8 row counts × every op × both CI arms | Shipped (#1226 `0b1ebaa`; PR3; `x86-64-v3` in CI, `-v4` locally on 2026-09-14 — NEON/WASM/scalar unexercised) | anti-vacuity `survivors * 3 < n_rows` for the 8 non-`Ne` predicates, UNGATED, at `n_rows >= 3` (the two `Ne` shapes are selective by complement, and `n_rows ∈ {0, 1}` are execution-only — the bound is `0 < 0` there); the gate itself is pinned selective (`gated < ungated`) at `n_rows >= 64` |
| D-MRX-6 | `examples/count_probe.rs` — reference / interpreted / fused, 0 B per execute | Shipped (#1226 `0b1ebaa`; PR3; four arms agree on the COUNT — 4855 at 64k rows; `interpreted` and `fused` gated at 0 B; timings printed from one run, not pinned) | counting allocator reads 0 after warm-up; three arms agree |
| D-MRX-7 | `Pred::Range { lo, hi }` — the contiguous-range write (`ndarray::simd::mask_set_range`) as an IR op; `ExecError::RangeOutOfBounds`; gate-aware | **In PR** (branch `claude/great-pascal-k96kok`, `c3931a1a`). Closes the IR half of `ISS-MASK-RISC-HAD-NO-RANGE-OP`; quack's `prefix_u64` lowering still waits on lane-ordering evidence (planner side) | three differential tests at the 8 row counts: word-edge shapes incl. 63/64/65 ungated / under plane / under scratch (values AND scratch words); `== hi − lo` through the EXECUTOR (the oracle-counted first draft was vacuous — disable-run passed it); identical `RangeOutOfBounds` on both paths. Disable-run: off-by-one in the exec arm fails 2 of 3 |
| D-MRX-7 | `Pred::Range { lo, hi }` — the contiguous-range write (`ndarray::simd::mask_set_range`) as an IR op; `ExecError::RangeOutOfBounds`; gate-aware | Shipped (#1246 `568965e9`). Closes the IR half of `ISS-MASK-RISC-HAD-NO-RANGE-OP`; quack's `prefix_u64` lowering still waits on lane-ordering evidence (planner side) | three differential tests at the 8 row counts: word-edge shapes incl. 63/64/65 ungated / under plane / under scratch (values AND scratch words); `== hi − lo` through the EXECUTOR (the oracle-counted first draft was vacuous — disable-run passed it); identical `RangeOutOfBounds` on both paths. Disable-run: off-by-one in the exec arm fails 2 of 3. Cost accounting corrected post-review: a GATED range charges `ranges` AND `two_input` (`mask_set_range` then `mask_and_assign`), asserted two-sided against a gated lane predicate, which stays free |

## bbb-nars-lowering (D-id minted 2026-09-07 with the operator ruling)

Expand Down
72 changes: 72 additions & 0 deletions .claude/board/TECH_DEBT.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,75 @@
## TD-JC-CLIPPY-RED-ON-BASE-2 (2026-09-18) — the 1.98 pre-bump lint sweep was WORKSPACE-scoped, and `jc` is workspace-EXCLUDED

**`JC Substrate Proof` is RED on `main`** (run `35335429357`, head `568965e9`):
`clippy::chunks_exact_to_as_chunks` in `crates/jc/src/jirak.rs` and
`crates/jc/src/pearl.rs`, in each file's `deterministic_fingerprint`, both
spelled `chunks_exact_mut`, under the workflow's own `-D warnings`. Cited by
FUNCTION, not by line: the first draft of this entry wrote `jirak.rs:37` /
`pearl.rs:59` and the citation-decay gate failed it (one new decay, correctly
attributed to this entry) because the nearest backticked symbol to those
coordinates was the LINT NAME, which is nowhere near the code. The gate's
point, taken: a number is a coordinate in a moving frame.

**It is NOT the merge that surfaced it.** Both lines are present verbatim at
`83369cad` (`main` before #1246), and #1246 touched no `jc` file. Verified by
checking out the base and grepping the two paths, not inferred from the diff.

**Why the existing guards did not catch it, precisely:**

- `jc-proof.yml` DOES run on pull_request — but path-filtered to
`crates/jc/**`, `crates/lance-graph-contract/src/cam.rs` and the workflow
itself. #1246 changed `facet.rs` / `canonical_node.rs`, so it correctly
never ran there. The filter is not the defect.
- `git log -S` dates both sites to `5959463f` (#1170, 2026-09-04) — ONE DAY
BEFORE `TD-JC-CLIPPY-RED-ON-BASE-1` was closed as "jc clippy clean"
(`13fdd587`). They survived that sweep because the lint did not exist yet:
`chunks_exact_to_as_chunks` is new in 1.98.
- `rust-toolchain.toml`'s own bump log says the 1.97.1 → 1.98.1 leg is
"ONE lint … at ten sites across four crates", closed separately in #1194
and "verified clean on BOTH 1.97.1 and 1.98.1" — which is what licensed
the bump as a one-line channel edit. **That verification was
workspace-scoped, and `jc` is in `[workspace] exclude`.** A
workspace-wide `cargo clippy` cannot reach it — which is exactly the
reason `jc-proof.yml` carries its own `--manifest-path` clippy step in
the first place, as its comment says. The sweep and the gate knew about
the exclusion independently; the bump's measurement did not.

**The fix (2 lines, mechanical):** `fp.chunks_exact_mut(8)` →
`fp.as_chunks_mut::<8>().0`, which is what clippy suggests and what #1194
applied at its ten sites. NOT done here: it is code in a crate this PR does
not touch, and this PR is board hygiene.

**Wider candidate surface — GREP, NOT VERDICT.** A `chunks_exact(_mut)\(\d+\)`
census over all **32** workspace-excluded members finds **118** candidate
sites: thinking-engine 98 (mostly `examples/`), highheelbgz 4,
lance-graph-cognitive 4, onebrc-probe 3, reader-lm 3, bge-m3 2, jc 2,
quasicryth-research 1, weather-poc 1.

⊘ **The first version of this entry said 22 crates / 114 sites and missed
`onebrc-probe` and `weather-poc` entirely** (codex P2 on #1247, verified and
correct). Cause, which is the entry's own subject one level up: the census
parsed `[workspace].exclude` with a NON-GREEDY REGEX that stopped at the
first `]` and silently returned 22 of 32 members. An entry about a
workspace-scoped measurement missing workspace-excluded crates was itself
produced by a scope that silently truncated. Recounted with `tomllib` —
`len(exclude) == 32`, 118 sites across 9 crates — which reproduces codex's
numbers exactly. **Use a TOML parser for a TOML list; a regex over TOML is
the same class of error as a grep standing in for a read.**

**Only `jc` is RED**, because only `jc` has a clippy
gate; the rest are ungated, so none of the other 116 has been compiled
against the lint at all. `weather-poc` has its own workflow, but it runs
`cargo test` only — no clippy step — so its one site is ungated like the
rest. Whether they would fire is UNVERIFIED — the regex
matches a call shape, not a lint verdict, and `--all-targets` is what pulls
`examples/` in. Do not quote 118 as a defect count.

**Same shape as:** `TD-JC-CLIPPY-RED-ON-BASE-1` (resolved #1183) and
`TD-SIGKER-CLIPPY-RED-ON-BASE-1`. The recurring lesson is one line: **a
workspace-scoped measurement says nothing about a workspace-excluded
crate**, and this repo excludes 32 of them. A toolchain bump's "verified
clean" is only as wide as its manifest list.

## 2026-09-15 — `TierFloors::stack_early_exit` promises a settled reading; that is true only for non-negative intensity, and the doc does not say so

- **The premise, unstated.** `crates/perturbation-sim/src/rolling_floor.rs:220`
Expand Down
34 changes: 25 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,16 @@ workspace is local — prefer the local/fork source over the registry, always.
> finds them must not read the P0 above as a mandate to wire them in. Registry
> pins for this family are therefore CORRECT, not a policy violation to
> "fix". The version discipline that DOES apply: the family moves in exact
> lockstep (`=X.Y.Z` across every member, `lancedb` pinning its matching
> `lance`), so a bump is one deliberate, measured PR — never a drift.
> lockstep, so a bump is one deliberate, measured PR — never a drift.
> **⊘ The parenthetical that stood here — "(`=X.Y.Z` across every member,
> `lancedb` pinning its matching `lance`)" — is SUPERSEDED by the operator
> ruling 2026-09-18** (*"never pin to x.00, always float x.*"*): the family
> still moves in lockstep, but WE express that as a floating patch (`11.*`,
> `0.38.*`), never a `.0.0` exact pin. Lockstep is about the MAJOR moving
> together; it never required us to restate lancedb's own `=11.0.0`, and
> restating it can only narrow the graph. Full rule and the receipt:
> § Key Dependencies below — which is the ONE authoritative pin policy in
> this file.
- If a fork's coordinates (git URL, branch/tag, feature flag) are unknown,
**STOP and ask**. Do NOT fall back to crates.io as a convenience or to make a
build pass.
Expand Down Expand Up @@ -1208,8 +1216,16 @@ cd crates/lance-graph-python && maturin develop
## Key Dependencies

```toml
# THE PIN RULE (operator-ruled 2026-09-05). Exact-pin (`=X.Y.Z`) ONLY where an
# upstream crate itself demands exact-equals. Everywhere else the MAJOR, so the
# THE PIN RULE. ⊘ The 2026-09-05 half-sentence "Exact-pin (`=X.Y.Z`) ONLY
# where an upstream crate itself demands exact-equals" is SUPERSEDED by the
# operator ruling 2026-09-18: *"never pin to x.00, always float x.*"* / *"so no
# decimal .0.0"*. NOTHING in this workspace carries a `.0.0` exact pin any more;
Comment thread
AdaWorldAPI marked this conversation as resolved.
# the patch FLOATS (`11.*`, `0.38.*`) even where lancedb spells its own
# requirement `=11.0.0`. Floating is strictly SAFER, not looser: our own pin can
# only ever be equal to or NARROWER than what the family asks for, so an exact
# one buys nothing and deadlocks the moment lancedb's transitive requirement
# moves by a patch (the Cargo.toml:141-144 receipt below is exactly that
# failure). What survives verbatim: everywhere else the MAJOR, so the
# family can converge on the newest compatible patch. Nothing outside the
# lance / lancedb / arrow / datafusion family is pinned at all, and NO repo
# tracks a Cargo.lock (.gitignore; ISS-STALE-AUTHORITY-LOCKS, RESOLVED).
Expand All @@ -1231,7 +1247,7 @@ cd crates/lance-graph-python && maturin develop
# (Rust crate line is 0.33.0 -> 0.37.1 -> 0.38.0; 0.34/0.35/0.36 do not exist as
# Rust crates -- those numbers are the independently-versioned PyPI package.)
#
# The lance family moves in EXACT lockstep -- currently =11.0.0. Line of state:
# The lance family moves in EXACT lockstep -- currently the 11 line. Line of state:
# lance 7 (2026-06-14) -> 9 (the lance-9 sweep, b2b08b07 / PR #896 arc) -> 10
# (#1187, D-LNC-1) -> 11 (#1190, D-LNC-3, probe-gated on the D-LNC-2 fragment-id
# probe #1189). arrow/datafusion did NOT move with any of them.
Expand Down Expand Up @@ -1266,10 +1282,10 @@ datafusion = "54" # OUR direct pin, in every crate that DEPENDS on it
# 2026-08-18; crates.io releases still DF 53) — as its
# own deliberate PR if a consumer needs Delta.
# Probe: .claude/plans/lance9-datafusion54-upgrade-probe-v1.md
lance = "=11.0.0" # exact-pinned: lancedb 0.38.0 requires lance =11.0.0
lance-linalg = "=11.0.0"
lance-index = "=11.0.0"
lancedb = "=0.38.0" # the lance-11 pairing (`default-features = false` in
lance = "11.*" # lancedb 0.38 requires `=11.0.0`; we float the patch
lance-linalg = "11.*" # and resolve to whatever it demands (today 11.0.0)
lance-index = "11.*"
lancedb = "0.38.*" # the lance-11 pairing (`default-features = false` in
# the workspace table). NOTE: the Rust crate line is
# 0.33.0 -> 0.37.1 -> 0.38.0; 0.34/0.35/0.36 exist only
# as the independently-versioned PyPI package.
Expand Down
Loading