fix(encoding): emit NIL control words when rep/def levels are non-empty but zero-width

[lichuang]

Summary

The structural full-zip encoder wrote one all-zero control-word byte per item for pages whose rep/def level buffers are non-empty but have a computed bit width of zero, while the decoder reads such a zero-bit layout as NIL (zero bytes per control word). The mismatch interleaves phantom bytes into the zipped value stream and corrupts every value that follows.

This adds a zero-width early return in build_control_word_iterator: when total_width == 0 (i.e. max_rep == 0 && max_def == 0), emit ControlWordIterator::Nilary regardless of which level buffers are present, so the writer agrees with the decoder's NIL interpretation.

Root cause

encode_full_zip (rust/lance-encoding/src/encodings/logical/primitive.rs) computes max_rep/max_def from the per-page sliced level buffers, not the schema. When a page's levels are all zero (e.g. a page holding only valid rows of a nullable column after page splitting), total_width == 0 and the layout metadata records bits_rep = 0, bits_def = 0. The decoder reconstructs ControlWordParser::new(0, 0) → NIL (bytes_per_word() == 0), but the writer selected its branch purely on buffer presence, producing a Unary8/Binary8 writer that emits one 0x00 byte per item (bytes_per_word() == 1). In the full-zip layout control words and values are interleaved per item, so the decoder's 0-byte step mis-aligns the value stream by one byte per item.

[lance-gatekeeper]

✅ Gate recommendation: approve.

The zero-width guard restores the full-zip writer/parser invariant: (bits_rep, bits_def) = (0, 0) now emits the NIL representation existing readers already consume, avoiding phantom bytes while preserving the stable 2.1 layout. Centralizing this in the control-word iterator is preferable to widening redundant all-valid definition levels or adding decoder ambiguity.