Skip to content

Align NumPy inputs before native processing - #143

Merged
gouyt13 merged 1 commit into
mainfrom
fix/numpy-input-alignment
Oct 21, 2026
Merged

gouyt13 merged 1 commit into
mainfrom
fix/numpy-input-alignment

Conversation

@gouyt13

@gouyt13 gouyt13 commented Oct 21, 2026

Copy link
Copy Markdown
Member

Summary

C-contiguous NumPy arrays can still start at an address that is not aligned for float32. A view with a one-byte offset currently reaches misaligned float loads during RaBitQKMeans training.

Copy unaligned two-dimensional inputs into aligned, owned arrays at the shared binding conversion boundary. Already-aligned compatible arrays retain their existing path, and input contents remain unchanged. Add read-only offset-1/2/3 regressions for both clustering classes and all three index wrappers, and document the copy behavior.

No public signature, saved-index format, or dependency changes.

Testing

Validated this branch independently of the HNSW seed fix and IVF append optimization on Linux x86-64, using GCC 14.3, GoogleTest 1.14.0, Python 3.13, and RABITQ_ENABLE_NATIVE_OPTIMIZATION=OFF:

  • Full C++ suite: 405 passed, 1 conditional OpenMP skip. The separate OpenMPBinding test passes.
  • Built and installed this branch's wheel into an isolated test prefix using the existing Python environment; verified the imported extension bytes against the wheel. Python suite excluding C++ example-binary tests: 714 passed, 2 platform-specific wheel skips.
  • The retained unmodified-main extension passes the aligned-input control and fails the offset-1 reproducer with load of misaligned address. This branch passes that reproducer and 15/15 alignment regressions with -fsanitize=alignment -fno-sanitize-recover=alignment on the bindings and included template code. This is a focused alignment check, not a whole-library ASan/UBSan run.
  • Full production/bindings clang-tidy 23, C++ formatting, Python format/lint, strict MkDocs build, and git diff --check pass.

This is a correctness fix; no throughput improvement is claimed. Windows, macOS, and ARM64 execution remain for CI.

@gouyt13
gouyt13 merged commit 68a20e2 into main Oct 21, 2026
39 checks passed
@gouyt13
gouyt13 deleted the fix/numpy-input-alignment branch October 21, 2026 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant