[gouyt13]

Summary

• Atomic index saves (fix).
  • HNSW, IVF and SymphonyQG save write to a temporary file beside the destination and rename it into place, so a failed save leaves an existing index unchanged and removes what it wrote. Previously a failure part-way through left a truncated file.
  • IVF renames its HNSW sidecar before the index file.
  • A replaced file keeps its permission bits, but not its owner or other hard links.
  • Symbolic links and other non-regular destinations are still written in place, as before.
  • HNSW serializes one loaded point count instead of the bytes of its atomic counter.
• IVF and QG builder (perf).
  • IVF::reserve refreshes only the grown cluster's view instead of rebuilding every view, which cost O(num_clusters) per call.
  • The QG builder's reverse-edge pass uses 4096 striped mutexes instead of one per point. Mutual exclusion is unchanged; memory no longer grows with the point count.
• Defaults and docs.
  • The HNSW construct default is now spelled as every available thread; 0 still auto-selects, so behavior is unchanged.
  • Docs state that one-thread add keeps links deterministic.
  • Rotator dimensions start initialized.

No public API or saved-format changes. The save behavior is documented under the shared persistence conventions in docs/docs/index/usage.md.

Known limitation: if the index-file rename fails after the IVF sidecar rename succeeded, a new sidecar can remain beside the old index. This needs both renames in one directory to succeed and fail in turn (for example, a permission change in between). It is documented; a journaled two-file update is out of scope here.

Behavior change to note: saving over an existing read-only index file now succeeds, because a rename needs write access to the directory, not the file. The replacement keeps the read-only bits.

Testing

Linux x86-64 (GCC 14.3, RABITQ_ENABLE_NATIVE_OPTIMIZATION=OFF):

• CTest (--no-tests=error): Release and ASan/UBSan, 424 passed each, 1 skipped (the placement test, which requires OpenMP binding). The first commit was also built and tested on its own.
• New C++ tests (file_replacement_test):
  • replacing the main file and its sidecar without leaving temporaries;
  • creating a missing destination;
  • a failed write keeping the old files and removing temporaries;
  • writing in place through non-regular destinations and symbolic links;
  • keeping the permissions of replaced files. This test failed before the permission copy: 0640 and 0600 became 0664 under umask 0002.
• Python: 810 passed, 2 skipped (wheel-only tests). This includes a new test that saves over existing index and sidecar files twice and checks that no extra files remain.
• Static checks: check-format.sh, check-python.sh, full check-tidy.sh build-tidy (clang-tidy 23), mkdocs build --strict and git diff --check pass.

Not run locally: Windows and macOS. On Windows the permission test is skipped, because only a read-only attribute exists there.