Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions .claude/board/EPIPHANIES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,93 @@
## 2026-09-18 — E-BYTES-ARE-STORED-INTEGERS-ARE-PROJECTED-1 — byte-agnosticism is the STORAGE superpower and little-endian is the COMPUTE superpower; the bug is always a stored projection

**Status:** OPERATOR-RULED (the framing is the operator's: *"byte is storage
superpower, LE is compute superpower"*) + SHIPPED at the one site that
violated it (`NodeRow::edges` is now the byte-backed `EdgeFacet([u8; 16])`).
**Confidence:** HIGH on the mechanics — measured census, 1356 contract tests,
every consumer unchanged, the new falsifier disable-verified red-then-green.
HIGH on the doctrine as a *reading of this tree*: two of the three sites
already obeyed it before it was written down.

### The line

**Bytes are stored. Integers are projected.**

- **Storage is byte-agnostic, and that is a superpower:** a byte array has no
endianness to get wrong, every bit pattern is valid, no niche, and the
in-memory image equals the wire image on every target for free.
- **Little-endian is a superpower too, but a COMPUTE one:** memory order and
arithmetic significance agree, so a prefix compare is `vpxor` + `tzcnt` with
nothing materialized. That is why the facet's byte-chain LCP measures
**1.72 ns** (`examples/facet_axis_lcp_probe.rs`, #1245) — the reinterpret IS
the speed.
- **They must not be mixed, and the failure has exactly one shape: storing a
projection.** The moment a typed, native-endian value becomes the stored
image, the storage lane inherits the compute lane's endianness — and the
only defence left is a target guard.

### The tree already agreed, at two sites out of three (measured)

| site | form | verdict |
|---|---|---|
| `NodeGuid([u8; 16])` | stores bytes, projects via `.facet()` | obeys — the exemplar |
| `AttentionFocusFacet { facet: FacetCascade, .. }` | holds the TYPED facet, is not `repr(C)`, no `SoaEnvelope`, and reaches bytes only through the explicit `to_bytes()` encode | obeys — compute, contained, costs nothing |
| `NodeRow::edges` | stored a `FacetCascade` inside a `repr(C, align(64))` row whose `as_le_bytes()` reinterprets `&[NodeRow] → &[u8]` for Lance | **violated** — a stored projection |

The third is why #1246 had to add
`const _: () = assert!(cfg!(target_endian = "little"))` to `facet::`. That
guard was a stopgap holding a seam shut, not a fix.

### What shipped

`EdgeFacet([u8; 16])` — `#[repr(C, align(16))]`, the exact mirror of
`NodeGuid` — with `as_bytes` / `as_bytes_mut` / `from_bytes` / `to_bytes` /
`facet()`, and `pub type EdgeBlock = EdgeFacet` so every call site compiles
unchanged. Consequence: **all three `NodeRow` fields are now byte arrays**
(`[u8;16] | [u8;16] | [u8;480]`), so the 512-byte row contains no
native-endian integer at all, and `as_le_bytes` is byte-identical across
targets by construction rather than by assertion. Both stale SAFETY comments
are corrected in place — one of them had said, in its own last sentence, that
`EdgeBlock` was "the one field that is not" a byte array.

### The correction this entry carries

#1246's arc entry and PR body both said byte-backing `edges` "would retire the
`target_endian` guard entirely." **That was wrong, and the code said so.**
`FacetCascade::as_bytes` / `ref_from_bytes` are still reinterprets, and they
are *supposed* to be — that reinterpret is the 1.72 ns hot path. So the guard
stays; what changed is its blast radius. It no longer protects a row at rest
(that dependency is gone); it protects the compute lens's own
`reinterpret == encode` identity, where a violation can mis-read a value in
flight and nothing more. The guard's comment is rewritten to say exactly that.

### Why the change was nearly free (the census, not a guess)

Every `EdgeBlock` site in the tree is `default()`, `as_bytes()` /
`as_bytes_mut()`, `from_bytes()`, equality, or a `Copy` — and every
struct-literal construction and every `.facet_classid` / `.tiers` read is on a
*projected* facet (`FacetCascade::from_bytes(&bytes).tiers[0]`,
`… .facet_classid == CLASSID`). **Not one is a field access on
`NodeRow::edges`.** #1246's migration had already moved them all to bytes. The
two real consumers (`symbiont::key_render`, `soa_graph`) read
`eb.as_bytes()[..12]` / `[12..]` and are untouched. So this change did not
impose the doctrine — it ratified what the code was already doing, and let the
type system say it.

### The reusable pattern, named

`byte-backed newtype + .facet() projection`. Storage type owns the bytes and
offers no integer; the lens type owns the integers and is obtained by an
explicit decode. If a type is `repr(C)` AND reachable from a stored image AND
contains a multi-byte integer, it is a stored projection — fix it by moving the
integer behind a projection, not by adding a target guard.

### Residue (unchanged by this entry)

The readers that still split those 16 bytes at 12 — the V1 `12 + 4` carving —
remain named in `ISS-EDGE-BLOCK-WAS-A-SECOND-TYPE-FOR-THE-SAME-FACET`. Byte
backing neither fixes nor worsens that; it is a *reading* of the bytes, and the
ClassView is still what should decide it.

## 2026-09-17 (18) — E-THE-SECOND-FACET-IS-NOT-AN-EDGE-BLOCK-1 — bytes 16..32 are just another content-blind facet cascade; giving them their own type was how the V1 `12 + 4` carving survived its own retirement

**Status:** OPERATOR-RULED (verbatim below) + SHIPPED (`pub type EdgeBlock =
Expand Down
37 changes: 37 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,40 @@
## 2026-09-18 (2) — CONTRACT INVENTORY DELTA: `EdgeFacet([u8; 16])` — `NodeRow::edges` is byte-backed, so no field of the 512-byte row is a native-endian integer any more

- **Added:** `lance_graph_contract::canonical_node::EdgeFacet` —
`#[repr(C, align(16))]` over `[u8; 16]`, the exact mirror of `NodeGuid`, with
`as_bytes` / `as_bytes_mut` / `from_bytes` / `to_bytes` / `facet()` and
`From`/`Into` against `FacetCascade`. **`pub type EdgeBlock = EdgeFacet`** —
the name is unchanged, the TYPE it points at changed.
- **Retyped:** `NodeRow::edges` was `FacetCascade` (since the 2026-09-17 alias);
it is now `EdgeFacet`. Byte positions, `NODE_ROW_STRIDE`,
`ENVELOPE_LAYOUT_VERSION` and `node_rows_from_le_bytes` are all unchanged.
- **State consumers should know:** nothing breaks. Every call site is
`default()`, `as_bytes()`, `as_bytes_mut()`, `from_bytes()`, equality or a
`Copy`, and all of those are preserved; the typed cascade is now reached by
**projection** — `row.edges.facet()` — the same way `NodeGuid::facet()` has
always worked. `weather-poc` (40 tests) and `lance-graph-planner` build and
pass unchanged.
- **Why:** `NodeRow` is `#[repr(C, align(64))]` and `as_le_bytes()` reinterprets
`&[NodeRow] → &[u8]` for Lance, so a typed `edges` made the compute type's
native-endian `u32` part of the **stored** image. Doctrine and full census:
`EPIPHANIES.md` `E-BYTES-ARE-STORED-INTEGERS-ARE-PROJECTED-1` — *bytes are
stored, integers are projected*.
- **The `target_endian` guard STAYS, narrowed.** #1246's arc entry said
byte-backing `edges` would retire it entirely; that was wrong.
`FacetCascade::as_bytes` is still a reinterpret **by design** — it is the
1.72 ns hot path — so the `reinterpret == encode` identity is still assumed.
What changed is the blast radius: a violation can now mis-read a value in
flight, never corrupt a row at rest. The guard's comment says so.
- **Corrected in place:** both `canonical_node.rs` SAFETY comments. One of them
ended by asserting `EdgeBlock` was "the one field that is not" a byte array —
true when written, false now, and struck rather than silently reworded.
- **Falsifier:** `edges_store_bytes_verbatim_and_project_the_integer_little_endian`
asserts both superpowers at once — the stored bytes are verbatim (endian-free)
AND `facet().facet_classid` decodes `0xDEAD_BEEF` little-endian. Disable-run:
byte-swapping the first four bytes in `from_bytes` fails it on "stored bytes
are verbatim"; restoring passes. 1356 contract tests green, clippy clean under
`-D warnings`.

## 2026-09-18 — PR #1246 merged (`568965e9`): `EdgeBlock` is a `FacetCascade` on `main`, `Pred::Range` is in the mask-risc IR, and no manifest in this repo carries a `.0.0` pin

The two entries below dated 2026-09-17 (3) and (2) describe what is now on
Expand Down
173 changes: 154 additions & 19 deletions crates/lance-graph-contract/src/canonical_node.rs
Original file line number Diff line number Diff line change
Expand Up @@ -685,7 +685,105 @@ impl core::fmt::Display for NodeGuid {
/// `ISS-EDGE-BLOCK-WAS-A-SECOND-TYPE-FOR-THE-SAME-FACET`. New code says
/// `FacetCascade` and reads through `as_bytes()` / the ClassView; migration
/// pointer per I-LEGACY-API-FEATURE-GATED.
pub type EdgeBlock = crate::facet::FacetCascade;
#[derive(Clone, Copy, PartialEq, Eq, Hash, Default)]
#[repr(C, align(16))]
pub struct EdgeFacet([u8; 16]);

impl EdgeFacet {
/// The 16 bytes, verbatim. This is a byte read, not a reinterpret of a
/// typed field, so it is identical on every target.
#[inline]
#[must_use]
pub const fn as_bytes(&self) -> &[u8; 16] {
&self.0
}

/// The 16 bytes, mutably — the only write path (`EdgeCodecFlavor` decides
/// what a byte MEANS; this type never does).
#[inline]
pub fn as_bytes_mut(&mut self) -> &mut [u8; 16] {
&mut self.0
}

/// Adopt 16 stored bytes. No decode: the bytes ARE the value.
#[inline]
#[must_use]
pub const fn from_bytes(b: &[u8; 16]) -> Self {
Self(*b)
}

/// The 16 bytes by value.
#[inline]
#[must_use]
pub const fn to_bytes(self) -> [u8; 16] {
self.0
}

/// **The projection.** Read these bytes through the typed cascade lens —
/// `facet_classid(4) | 6×(8:8)` — exactly as [`NodeGuid::facet`] does for
/// the key. The integer fields are DECODED here (`u32::from_le_bytes`),
/// they are not stored here.
#[inline]
#[must_use]
pub const fn facet(&self) -> crate::facet::FacetCascade {
crate::facet::FacetCascade::from_bytes(&self.0)
}
}

impl core::fmt::Debug for EdgeFacet {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
write!(f, "EdgeFacet({:02x?})", self.0)
}
}

impl From<crate::facet::FacetCascade> for EdgeFacet {
/// Encode a computed facet down into stored bytes — explicitly, via
/// [`FacetCascade::to_bytes`](crate::facet::FacetCascade::to_bytes).
#[inline]
fn from(f: crate::facet::FacetCascade) -> Self {
Self(f.to_bytes())
}
}

impl From<EdgeFacet> for crate::facet::FacetCascade {
#[inline]
fn from(e: EdgeFacet) -> Self {
e.facet()
}
}

/// The historical name for bytes 16..32. Kept as an alias so every existing
/// call site (`EdgeBlock::default()`, `as_bytes()`, `from_bytes()`, equality)
/// compiles unchanged; what changed is the TYPE it names.
///
/// # Migration — this alias is source-BREAKING for field access
///
/// `EdgeBlock` named [`FacetCascade`](crate::facet::FacetCascade) before this
/// change, so its `facet_classid` / `tiers` fields and every inherent
/// `FacetCascade` method were reachable straight through the alias. They are
/// not any more: [`EdgeFacet`] deliberately exposes **bytes only**, because a
/// typed integer inside [`NodeRow`] is a *stored projection* — the one shape
/// this failure ever takes.
///
/// The fix is to name the projection you were implicitly taking:
///
/// ```
/// # use lance_graph_contract::canonical_node::EdgeBlock;
/// # let edges = EdgeBlock::default();
/// // before: edges.facet_classid
/// let classid = edges.facet().facet_classid;
/// # let _ = classid;
/// ```
///
/// This breaks LOUDLY — it is a compile error at the call site, never a silent
/// reinterpretation of stored bytes — which is why no feature gate or
/// read-mode alias is warranted under `I-LEGACY-API-FEATURE-GATED` (that rule
/// governs a name whose *semantics* change silently, not one that stops
/// compiling). No consumer is known to be affected: every in-tree site is
/// `default()` / byte access / equality / `Copy`, and the one named
/// out-of-repo consumer of this module, `lance-graph-java`'s `lgj-abi`,
/// imports `canonical_node::EdgeCodecFlavor` and never the block type.
pub type EdgeBlock = EdgeFacet;
Comment thread
AdaWorldAPI marked this conversation as resolved.

/// Which edge-codec flavor a class uses to *read* its node's edge block.
///
Expand Down Expand Up @@ -1678,16 +1776,18 @@ impl<'a> SoaEnvelope for NodeRowPacket<'a> {
// every byte position is valid for reads (no padding past size_of,
// alignment of NodeRow (64) ⊇ alignment of u8 (1)).
//
// The NodeGuid and EdgeBlock fields hold their bytes in canon-LE
// order, so the resulting byte slice IS the envelope's LE packet — no
// translation needed at the boundary. NodeGuid earns that by being
// `[u8; 16]` outright. EdgeBlock does NOT: since the 2026-09-17 alias
// it is a `FacetCascade`, whose `facet_classid` is a native-endian
// `u32`, so this cast reproduces the LE packet only on a little-endian
// target. That is enforced, not assumed — `facet::` carries a
// crate-level `const _: () = assert!(cfg!(target_endian = "little"))`
// for exactly this path (codex P2 on #1246). Do not restate EdgeBlock
// as a byte array here; it is the one field that is not.
// The resulting byte slice IS the envelope's LE packet, with no
// translation at the boundary and no target dependency, because EVERY
// field of NodeRow is byte-backed: `NodeGuid([u8; 16])`,
// `EdgeFacet([u8; 16])`, `value: [u8; 480]`. There is no native-endian
// integer anywhere in the 512 bytes to reorder.
//
// ⊘ This comment previously said the opposite of its last sentence —
// that EdgeBlock was "the one field that is not" a byte array, being a
// `FacetCascade` with a native-endian `u32`, so the cast reproduced the
// LE packet on little-endian targets only. True from the 2026-09-17
// alias until `edges` was byte-backed; the typed cascade is now reached
// by PROJECTION (`row.edges.facet()`), never stored.
unsafe {
core::slice::from_raw_parts(
self.rows.as_ptr().cast::<u8>(),
Expand Down Expand Up @@ -1739,12 +1839,12 @@ pub fn node_rows_from_le_bytes(bytes: &[u8]) -> Option<&[NodeRow]> {
// (const-asserted above). We checked (1) bytes.len() is an exact multiple of
// the stride, so n rows span the whole slice with no trailing bytes, and (2)
// the pointer is aligned to align_of::<NodeRow>() (64). Every bit pattern in
// the 512 bytes is a valid NodeRow (NodeGuid is `[u8; 16]`; EdgeBlock is a
// `FacetCascade`, 16 B `repr(C, align(16))` whose fields are all plain
// integers, so it too has no niche; value is `[u8; 480]`) — nothing to
// invalidate, so the reinterpretation is sound. Soundness does not imply
// byte-identity across targets: see `as_le_bytes` above and the
// little-endian assert in `facet::` that this path also relies on. The returned slice borrows `bytes` for its lifetime (no copy).
// the 512 bytes is a valid NodeRow — all three fields are byte arrays
// (`NodeGuid([u8; 16])`, `EdgeFacet([u8; 16])`, `value: [u8; 480]`), so none
// has a niche — nothing to invalidate, and the reinterpretation is sound.
// Since no field is a native-endian integer, byte-identity across targets
// holds too (it did not while `edges` was a typed `FacetCascade`; see
// `as_le_bytes` above). The returned slice borrows `bytes` for its lifetime (no copy).
Some(unsafe { core::slice::from_raw_parts(bytes.as_ptr().cast::<NodeRow>(), n) })
}

Expand Down Expand Up @@ -2159,15 +2259,50 @@ mod tests {
core::mem::align_of::<EdgeBlock>(),
core::mem::align_of::<NodeGuid>()
);
let f: crate::facet::FacetCascade = e; // an alias, not a conversion
assert_eq!(f.as_bytes(), &[0u8; 16]);
let f: crate::facet::FacetCascade = e.facet(); // a PROJECTION, not an alias
assert_eq!(f.to_bytes(), [0u8; 16]);
let mut g = EdgeBlock::from_bytes(&[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]);
g.as_bytes_mut()[12] = 0;
assert_eq!(g.as_bytes()[11], 12);
assert_eq!(g.as_bytes()[12], 0);
assert_eq!(g.as_bytes()[13], 14);
}

/// **The two superpowers, in one test.** What is STORED is bytes, verbatim,
/// on any target; the integer is PROJECTED out of them, little-endian, by an
/// explicit decode.
///
/// This is the test the old `edges: FacetCascade` could not pass on a
/// big-endian target: `as_bytes()` was a reinterpret of a native-endian
/// `u32`, so the first assert would have seen a byte-swapped class id and
/// the stored row image would silently disagree with `to_bytes()`. Now the
/// first assert is a byte identity (nothing to swap) and the second is a
/// decode that names its own endianness — so BOTH hold everywhere, which is
/// why bytes 16..32 no longer depend on the target at all.
#[test]
fn edges_store_bytes_verbatim_and_project_the_integer_little_endian() {
let b: [u8; 16] = [
0xEF, 0xBE, 0xAD, 0xDE, // classid, LE on the wire
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
];
let e = EdgeFacet::from_bytes(&b);

// STORAGE: byte-agnostic. The stored image is the source bytes, and a
// round trip through the row field cannot reorder them.
assert_eq!(e.as_bytes(), &b, "stored bytes are verbatim");
assert_eq!(e.to_bytes(), b);

// COMPUTE: little-endian, and it says so. The integer exists only here.
assert_eq!(
e.facet().facet_classid,
0xDEAD_BEEF,
"the class id is DECODED from the stored bytes, never stored as a u32"
);

// And the projection is lossless back to the same stored bytes.
assert_eq!(EdgeFacet::from(e.facet()).as_bytes(), &b);
}

#[test]
fn edge_codec_flavor_default_is_coarse_only() {
// Zero-fallback default: the all-zero reading is the canon bootstrap.
Expand Down
Loading
Loading