You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since 0.8.0 (#195), indexes reject non-finite and out-of-range coordinates with check_coordinates in include/rabitqlib/utils/input_checks.hpp. It reads the input twice
in serial: all_finite, then a range check. On 1M × 960 float32 vectors the check alone took
about 0.70 s, which accounts for most of the 0.6–0.7 s that IVF construction gained between
0.7.1 and 0.8.0 in our benchmark.
The range check !(std::fabs(v) <= bound) is already true for NaN and ±Inf, so one pass can
detect every bad value; the finite check is only needed to choose the error message.
Task
Make check_coordinates read the data once in the common (valid) case:
Count out-of-range values in one pass.
Only if that count is non-zero, call all_finite to choose between the two existing
messages.
Done when
Error types and messages are unchanged, including which one is raised when the input has
both NaN and large finite values (non-finite wins today).
Existing input tests pass: tests/python/test_input_boundaries.py and the C++ unit tests.
The PR reports the check's time before and after on a large array (for example 1M × 960
random float32), with the CPU, compiler and build type.
Notes
This header is included by ISA-specific translation units (src/fastscan/estimator_avx2.cpp
and src/fastscan/estimator_avx512.cpp). Besides ./scripts/check-format.sh and the C++ tests, run ./scripts/check-isa-symbols.sh on a
build with RABITQ_ENABLE_NATIVE_OPTIMIZATION=OFF (see verification by change type).
Parallelizing the scan is a possible follow-up; keep this PR to the single pass.
Hi! I'd like to work on this issue as my first contribution. I'll investigate the current implementation and submit a PR if nobody is already working on it.
Activity
nicklesternl commentedon Nov 3, 2026
Hi! I'd like to work on this issue as my first contribution. I'll investigate the current implementation and submit a PR if nobody is already working on it.