Repository navigation
fix: align diverged edge cases in FHT, HNSW, xy encoding and PiPNN - #185
Merged
Merged
Conversation
Rotators may be called in place, and the generic and NEON backends copy the input with memmove. The shared AVX2/AVX-512 template used memcpy, which is undefined for overlapping ranges. Use memmove and test overlapping calls on every available backend.
A default-constructed HierarchicalNSW has no rotator, so search and save dereferenced a null pointer. Reuse the initialization check from resize and throw std::logic_error before touching the rotator or opening the output file.
For a residual with zero norm, ex_bits_code_with_norm zeroed the magnitude codes and then inverted them, so quantize_xy_single wrote 2^ex_bits - 1 while the full, split and v1 encoders write 0. Return before the inversion. All factors are zero on this path, so estimates are unchanged.
QuantizedGraph and IVF accept up to kSearchBufferMaxPointCount points, but the PiPNN seed rejected a count equal to it. Use the same bound.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix four edge cases where duplicated code had diverged between backends or callers. Each commit has its own fix and regression test.
memmove, but the shared AVX2/AVX-512 template usedmemcpy, which is undefined for overlapping ranges. It now usesmemmove. The existing overlap test only ran on aarch64; a new test covers every available backend.HierarchicalNSW,searchandsavedereferenced a null rotator. Both now throwstd::logic_errorusing the initialization checkresizealready has.savethrows before opening the output file.quantize_xy_singlewrote magnitude codes2^ex_bits - 1, while the full, split and v1 encoders write 0. It now writes 0 as well. All factors are zero on this path, so distance estimates do not change; only these degenerate code bytes do.QuantizedGraphand IVF accept up tokSearchBufferMaxPointCountpoints, but the PiPNN seed rejected a count equal to it. It now uses the same bound.No public API, Python API or saved-index format changes.
Testing
memcpy-param-overlapinfht_rotate_avx512, a null-pointer SEGV inHierarchicalNSW::search, and xy code7vs. full code0. They pass after the fixes.RABITQ_ENABLE_NATIVE_OPTIMIZATION=OFF,--no-tests=error) on an AVX-512 Linux host:check-format.sh, fullcheck-tidy.sh build-tidyandgit diff --checkpassed.