Skip to content

fix(index): stop appending a spurious byte in binary_quantization - #9029

Open
LuciferYang wants to merge 1 commit into
lance-format:mainfrom
LuciferYang:fix/bq-extra-byte
Open

fix(index): stop appending a spurious byte in binary_quantization#9029
LuciferYang wants to merge 1 commit into
lance-format:mainfrom
LuciferYang:fix/bq-extra-byte

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

Problem

binary_quantization appends the tail byte unconditionally, so an input whose length is a multiple of 8 produces floor(len / 8) + 1 bytes instead of ceil(len / 8). Through BinaryQuantization::transform that is one extra zero byte per row for any dimension that is a multiple of 8.

Fixes #9028.

What this changes

Compute the tail byte once and chain it only when chunks_exact(8) left a remainder. The closure mapped over the full chunks is untouched, so the loop the "auto vectorized" comment refers to is the same code as before.

BinaryQuantization has no callers in this repository, so no in-tree width changes. The old width also contradicted the crate's own sizing helper, rabit_binary_code_bytes, which is div_ceil(8). Deleting the type is the other option, since RabitQuantizer is what IVF_RQ uses, but it is public API and the behaviour is cheap to fix.

Test plan

test_binary_quantization_length_multiple_of_eight pins the helper at 16 values, and two tests pin the public entry point on both branches: 3 rows of dimension 8, and 2 rows of dimension 12 where the tail byte must still appear. Restoring the old implementation makes them fail with [85, 85, 0] against [85, 85] and 6 bytes against 3. The existing test_binary_quantization covers the remainder case at 11 values, so both branches stay asserted.

  • cargo test -p lance-index --lib vector::bq 233 passed
  • cargo clippy --all --tests --benches -- -D warnings clean
  • cargo fmt --all --check clean

binary_quantization chained one tail byte unconditionally, so an input whose length is a multiple of 8 produced floor(len/8)+1 bytes instead of ceil(len/8). Append the tail byte only when a tail exists.
@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer bug Something isn't working labels Sep 7, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The conditional tail emission fixes the boundary error at its root while preserving bit ordering and the performance-sensitive full-chunk loop.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: binary_quantization appends a spurious byte when the length is a multiple of 8

1 participant