Skip to content

fix: bound IVF initializers, QG degree sums, HNSW capacity errors and ISA helper linkage - #192

Merged
gouyt13 merged 5 commits into
mainfrom
fix/audit-defects
Oct 31, 2026
Merged

gouyt13 merged 5 commits into
mainfrom
fix/audit-defects

Conversation

@gouyt13

@gouyt13 gouyt13 commented Oct 31, 2026

Copy link
Copy Markdown
Member

Summary

Five fixes found while auditing the library. Each commit has its own fix and, where behavior changes, a regression test that fails before the fix.

  • QGBuilder degree average overflow. avg_degree accumulated uint32_t degrees into an unsigned int (initial value 0U), so the total wrapped above 2^32 − 1 (for example about 67 million points at degree 64). It now sums in size_t.
  • IVF initializer bounds. FlatInitializer::centroids_distances partially sorted to nprobe and memcpy'd nprobe entries into the caller's vector without checking either bound; HNSWInitializer also wrote without sizing the output. Both now clamp nprobe to the cluster count and resize the output, as FlatRaBitQInitializer already did. IVF always passed a clamped, presized vector, so index search is unchanged; direct callers of the public initializers could read or write out of range.
  • HNSW allocation failure and zero capacity. Out-of-memory in construct/add threw std::runtime_error while load/resize threw std::bad_alloc; all now throw std::bad_alloc. A zero capacity reached the allocator, which returns null for zero bytes; the constructor and resize(0) now reject it with std::invalid_argument. Null checks after huge_page_allocate, which throws instead of returning null, are removed.
  • Internal linkage for helpers shared by ISA builds. Headers compiled into generic, AVX2 and AVX-512 translation units defined external-linkage inline functions and templates. When not inlined, each object emits a weak copy and the linker keeps one for every caller, so a lower-ISA path could run code compiled for a higher ISA. docs/development.md already requires internal linkage here.
  • Comments and messages that contradicted the code in HNSW, IVF, FastScan, core, the k-means overflow message and an AVX-512 static_assert, plus two stale references in docs/development.md.

Python-visible changes: HnswIndex(max_elements=0, ...) now raises ValueError("HNSW max_elements must be greater than zero") instead of RuntimeError("Not enough memory"), and HNSW allocation failure during build/add raises MemoryError instead of RuntimeError. The k-means input-overflow message now says ptrdiff_t. No other API, Python keyword or saved-index format changes.

Testing

Linux x86-64 (Xeon Gold 6418H), GCC 14.3 and Clang 23, RABITQ_ENABLE_NATIVE_OPTIMIZATION=OFF.

  • CTest (Release, --no-tests=error): 427 passed, 1 skipped (the OpenMP placement test). Three new tests: QGConstructionTest.AverageDegreeDoesNotWrapAboveUint32Range, IvfInitializerTest.CentroidSearchSizesOutputAndClampsProbeCount, HnswResizeTest.RejectsZeroCapacity.
  • Before the fixes: the QG test returns 33038210 instead of 2^31, the IVF test ends in a segmentation fault, and zero capacity throws the old exception types.
  • Linkage: compiling every src/simd source at -O0 with its CMake ISA flags, weak rabitqlib symbols defined under more than one ISA flag set drop from 10 (GCC) and 11 (Clang) to 0. In Release, all 26 SIMD and FastScan objects keep identical .text sizes.
  • Static analysis: check-format.sh, full check-tidy.sh with clang-tidy 23 (bindings on), git diff --check.
  • Python (package rebuilt and installed): check-python.sh, pytest tests/python 795 passed, 18 skipped (the C++ clustering-example tests, which need a local example build, and wheel-only tests); tests/test_pyright.py and tests/test_mypy.py pass.
  • Not run locally: NEON (the NEON header change is linkage only), Windows/MSVC, and AVX2-only execution; CI covers them.

QGBuilder::avg_degree passed 0U to std::accumulate over uint32_t degrees, so the sum was an unsigned int and wrapped once the total degree exceeded 2^32 - 1, for example about 67 million points at degree 64. Start from size_t{0}. The new test sets 65 degrees of 2^31 and fails before the fix.
FlatInitializer::centroids_distances partially sorted to nprobe and copied nprobe entries into the caller's vector without checking either bound, so nprobe above the cluster count or an undersized output read and wrote out of range. HNSWInitializer also wrote into the output without sizing it. Both now clamp nprobe to the cluster count and resize the output, as FlatRaBitQInitializer already did, and the Initializer interface documents that contract. IVF always passed a clamped, presized vector, so index search is unchanged. The new test fails before the fix with a segmentation fault.
Out-of-memory in construct and add threw std::runtime_error, while load and resize threw std::bad_alloc, so Python saw RuntimeError or MemoryError for the same failure. Throw std::bad_alloc for the link-list allocations, and drop the null checks after huge_page_allocate, which throws instead of returning null.

A zero capacity reached the allocator, which returns null for zero bytes: the constructor reported "Not enough memory" and resize(0) on an empty index threw std::bad_alloc. Reject both with std::invalid_argument (Python ValueError) before allocating, and document the minimum. Also correct the entry-point promotion comment in add_point.
Headers compiled into generic, AVX2 and AVX-512 translation units defined external-linkage inline functions and templates (pack_4bit_block, load_excode_tail, bitops::load_word and countr_zero32, contribute_ip*, BoundedRescaleState's constructor). When the compiler does not inline them, each object emits a weak copy and the linker keeps one for every caller, so a lower-ISA path could run code compiled for a higher ISA. docs/development.md already requires internal linkage for these headers. Use anonymous namespaces or static inline. Release object code is unchanged; at -O0 the shared weak symbols drop from 10 (GCC) and 11 (Clang) to 0.
- docs/development.md: HNSW has kDefaultEfSearch, not default_ef_search_, and query search inserts removed points into search_pool, not candidate_set.
- HNSW: the quantized-construction, live-node and bounded-queue comments described old behavior or the wrong declaration.
- IVF: the save/load format comments omitted the initializer payload written first, and ex_bits_ is 0 with raw reranking.
- FastScan: kPos, the 1-bit code offsets and the HACC LUT layout comments were misleading or stale.
- Core: unpack_sign_bits accepts any 8-coordinate tail, not only 32.
- The k-means overflow message named size_t although the check uses ptrdiff_t, and an AVX-512 static_assert named a constant that does not exist.
@github-actions github-actions Bot added the release:fixed Release category managed from the PR title label Oct 31, 2026
@gouyt13
gouyt13 merged commit c0d559a into main Oct 31, 2026
49 checks passed
@gouyt13
gouyt13 deleted the fix/audit-defects branch October 31, 2026 06:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:fixed Release category managed from the PR title

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant