Skip to content

fix: validate advanced encoder preconditions and bound HNSW prefetch look-ahead - #197

Merged
gouyt13 merged 2 commits into
mainfrom
fix/core-preconditions-hnsw-prefetch
Oct 31, 2026
Merged

gouyt13 merged 2 commits into
mainfrom
fix/core-preconditions-hnsw-prefetch

Conversation

@gouyt13

@gouyt13 gouyt13 commented Oct 31, 2026

Copy link
Copy Markdown
Member

Third follow-up to the audit (#195, #196): the core precondition items and the HNSW prefetch address derivation. Two commits.

fix(core): validate advanced encoder preconditions

  • code_factor_terms divides by dim - 1; quantize_full_single/quantize_split_single with dim == 1 returned a NaN error factor. The stable v1 API and the indexes already require more; the advanced encoders now throw std::invalid_argument.
  • rabitq_full_impl<T, TP> accepted a code element type too narrow for total_bits: uint8_t with nine bits truncated the sign bit (static_cast<TP>(1 << 8) is 0). It now requires total_bits in [1, 9] and <= numeric_limits<TP>::digits.
  • SplitSingleQuery validated bit widths but not padded_dim % 32 == 0; the AVX2/AVX-512 query transposition kernels load whole 32-byte blocks. Rejected now; the indexes always pass multiples of 32.
  • The five non-template helpers in utils/space.hpp were plain inline although the header is reached from the ISA-flagged HNSW search sources; they are static inline now per the implementation-linkage rule. check-isa-symbols.sh passes at -O3 and -O1.
  • Preconditions documented in quantizer.md; three regression tests.

fix(hnsw): derive prefetch addresses only from live link slots and pool entries

search_layer prefetched link slots 0 and 1 and search_base_layer_direct slots 1–4 regardless of the link count, and the base-layer search prefetched level0_links(search_pool.next_id()) after every insert even when the pool had no next entry — then next_id() reads the spare slot a full insert fills with the displaced entry, possibly with the checked bit set, so id * record_bytes lands outside the allocation. The prefetch itself cannot fault, but the index reads are stale or unwritten (uninitialized under parallel construction) and the pointer arithmetic is UB. The look-ahead is now bounded by the link count and the pool prefetch guarded by has_next().

Benchmark (prefetch is a hot-path change)

SIFT1M (1,000,000 × 128, 10,000 queries, k=10), HNSW M=16, ef_construction=200, 4 bits, L2; Intel Xeon Gold 6418H (96 cores), GCC 14.3 -O3 portable build (native tuning OFF, runtime dispatch → AVX-512); one index built by the baseline binary and loaded by both; runs alternated base/new/base/new, 3 rounds each, pinned with taskset. QPS medians (min–max):

threads ef baseline this PR Δ median recall@10
1 64 9568 (9321–10052) 9487 (9049–9872) −0.8% 0.9007 / 0.9007
1 128 6432 (6217–6581) 6416 (6318–6667) −0.3% 0.9179 / 0.9179
1 256 3910 (3819–4058) 3898 (3820–3977) −0.3% 0.9230 / 0.9230
16 128 84174 (84066–84731) 84168 (82007–85408) 0.0% 0.9179 / 0.9179
16 256 50656 (50036–50978) 50278 (49525–51128) −0.7% 0.9230 / 0.9230

Construction (32 threads, alternated): baseline 18.5 s, 18.1 s; this PR 18.5 s, 18.2 s. All deltas are inside the run-to-run spread; results and recall are identical because the change only affects prefetch hints.

Verification

  • check-format.sh, git diff --check, mkdocs build --strict.
  • GCC 14 portable build: ctest --no-tests=error 440/440 (three new tests); Clang 23 ASan/UBSan (with float-cast-overflow, float-divide-by-zero): 440/440, no diagnostics.
  • check-isa-symbols.sh at -O3 and -O1; installed-consumer build and tests against a fresh install prefix.
  • Focused clang-tidy on the HNSW search TUs, api.cpp, ivf.cpp, qg.cpp and the space_* SIMD TUs.
  • Reinstalled package: pytest tests/python 837 passed, 2 skipped.

Not verified: NEON (no hardware); MSVC.

code_factor_terms divides by dim - 1, so the advanced encoders returned a NaN error factor for dim == 1 (the stable API and the indexes already require more). rabitq_full_impl accepted a code element type too narrow for total_bits, truncating the sign bit for uint8_t at nine bits. SplitSingleQuery checked bit widths but not that padded_dim is a multiple of 32, although the AVX2 and AVX-512 transposition kernels load 32-byte blocks. All three now throw std::invalid_argument. The non-template helpers in utils/space.hpp gain internal linkage, as the implementation-linkage rule for headers reached from ISA-flagged sources requires.
…ol entries

search_layer prefetched the first two link slots and the base-layer search the first four regardless of the link count, and the base-layer search prefetched level0_links(search_pool.next_id()) after every insert even when the pool had no next entry, in which case next_id() reads the spare slot that a full insert fills with the displaced entry, possibly carrying the checked bit. Those reads are stale or unwritten IDs, and the resulting pointer arithmetic leaves the allocation. Bound the look-ahead by the link count and guard the pool prefetch with has_next().

SIFT1M (1M x 128, HNSW M=16, ef_construction=200, 4 bits), Xeon Gold 6418H, GCC 14 -O3 portable build, same saved index for both builds: single-thread QPS medians 9568/6432/3910 before and 9487/6416/3898 after at ef 64/128/256 (within the run-to-run spread), 16-thread 84174/50656 before and 84168/50278 after at ef 128/256; recall@10 identical (0.9007/0.9179/0.9230). Construction with 32 threads: 18.5 s and 18.1 s before, 18.5 s and 18.2 s after.
@github-actions github-actions Bot added the release:fixed Release category managed from the PR title label Oct 31, 2026
@gouyt13
gouyt13 merged commit 8d17b5d into main Oct 31, 2026
49 checks passed
@gouyt13
gouyt13 deleted the fix/core-preconditions-hnsw-prefetch branch October 31, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:fixed Release category managed from the PR title

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant