Skip to content

Use the HNSW seed for quantization rotation - #144

Merged
gouyt13 merged 2 commits into
mainfrom
fix/hnsw-rotation-seed
Oct 21, 2026
Merged

gouyt13 merged 2 commits into
mainfrom
fix/hnsw-rotation-seed

Conversation

@gouyt13

@gouyt13 gouyt13 commented Oct 21, 2026

Copy link
Copy Markdown
Member

Summary

HNSW's random_seed seeds the graph-level generator, but construction currently gives the FHT/Kac rotator a fresh random-device seed. Identical inputs and seeds can therefore produce different serialized indexes and search results even with one construction thread.

Pass the seed's low 32 bits to the rotator. Add C++ L2/IP and Python padded-dimension regressions that compare serialized bytes and search results from repeated serial builds, and check that another seed changes the index. Document the seed scope and the remaining scheduling/toolchain limits.

Newly constructed indexes now use a reproducible rotation. Existing files still restore their saved rotations; public signatures and the saved-index format are unchanged. Parallel insertion order remains scheduling-dependent.

Testing

Validated this branch independently of the NumPy alignment fix and IVF append optimization on Linux x86-64, using GCC 14.3, GoogleTest 1.14.0, Python 3.13, and RABITQ_ENABLE_NATIVE_OPTIMIZATION=OFF:

  • Full C++ suite, including historical persistence compatibility: 406 passed, 1 conditional OpenMP skip. The separate OpenMPBinding test passes.
  • Built and installed this branch's wheel into an isolated test prefix using the existing Python environment; verified the imported extension bytes against the wheel. Python suite excluding C++ example-binary tests: 700 passed, 2 platform-specific wheel skips.
  • The new Python seed regression fails against the retained unmodified-main extension and passes on this branch.
  • Full production/bindings clang-tidy 23, C++ formatting, Python format/lint, strict MkDocs build, and git diff --check pass.

No performance improvement is claimed: this changes seed plumbing, not the search kernels. Windows, macOS, and ARM64 execution remain for CI.

@gouyt13
gouyt13 merged commit 841ca5e into main Oct 21, 2026
43 checks passed
@gouyt13
gouyt13 deleted the fix/hnsw-rotation-seed branch October 21, 2026 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant