-
Install the package, build an IVF index, search it, and save it.
+
Install the package, build an IVF-RaBitQ index, search it, and save it.
@@ -60,10 +60,10 @@ actual recall and storage budgets differ.
## Built with RaBitQ-Library
-- [txtai](integrations/txtai.md) uses the Python IVF and HNSW indexes as an
+- [txtai](integrations/txtai.md) uses the Python IVF-RaBitQ and HNSW-RaBitQ indexes as an
embedding-search backend. See how to select the backend and its search settings.
- [zvec](integrations/zvec.md) uses core quantization in its IVF-RaBitQ and
- HNSW-RaBitQ implementations, with FastScan batch scoring in IVF. See the
+ HNSW-RaBitQ implementations, with FastScan batch scoring in IVF-RaBitQ. See the
integration walkthrough for storage, scoring, and platform boundaries.
These are direct library integrations, with versioned downstream source links.
diff --git a/docs/docs/index/hnsw.md b/docs/docs/index/hnsw.md
index 42cababb..c7d9b43e 100644
--- a/docs/docs/index/hnsw.md
+++ b/docs/docs/index/hnsw.md
@@ -1,9 +1,10 @@
-# HNSW RaBitQ
+# HNSW-RaBitQ
C++ callers include `rabitqlib/index/hnsw/hnsw.hpp` and link `rabitqlib::index`.
-[HNSW](https://arxiv.org/abs/1603.09320) is a popular graph-based index. Under
-comparable quantization settings, HNSW RaBitQ typically uses more memory
-than IVF RaBitQ because it stores graph structures in addition to quantized
+[HNSW](https://arxiv.org/abs/1603.09320) is a popular graph-based index.
+HNSW-RaBitQ combines HNSW graph traversal with RaBitQ quantization. Under
+comparable quantization settings, HNSW-RaBitQ typically uses more memory
+than IVF-RaBitQ because it stores graph structures in addition to quantized
vectors. The actual comparison depends on vector dimension, bit width, graph
degree, and index parameters.
@@ -12,7 +13,7 @@ degree, and index parameters.
### Python
Run the [Quick start](../quick_start.md#build-and-search-an-ivf-index) to prepare
-`data`, `queries` and `clustering`, then use those same arrays with HNSW:
+`data`, `queries` and `clustering`, then use those same arrays with HNSW-RaBitQ:
```python
from rabitqlib import HnswIndex
@@ -116,7 +117,7 @@ std::vector
>> HierarchicalNSW::search(const fl
- **queries**: Query vectors in the original input coordinates.
- **query_num**: The number of query vectors.
- **TOPK**: The number of nearest neighbors to search.
-- **efSearch**: The size of the candidate set for searching HNSW base layer.
+- **efSearch**: The size of the candidate set for searching HNSW-RaBitQ base layer.
- **thread_num**: Number of threads to use. Each query is processed by one thread.
The search window belongs to each call, so concurrent C++ searches can use
@@ -254,7 +255,7 @@ through 0.5.1 check every cluster ID on `load` and reject the file as invalid
instead of returning removed points. Releases before 0.3.7 do not check cluster
IDs, so do not open such a file with them.
-New HNSW indexes pad dimensions to multiples of 32 for all supported bit widths.
+New HNSW-RaBitQ indexes pad dimensions to multiples of 32 for all supported bit widths.
Loading preserves the saved padded dimension and rotation, so older indexes
using multiples of 64 remain readable. Older library versions cannot load new
files whose padded dimension is not a multiple of 64.
@@ -291,7 +292,7 @@ We first pre-process the query:
#### Upper Layers
-In the upper layers of HNSW, we compute the 1-bit estimated distance (using `BinData`) to quickly locate the entry point for the next layer.
+In the upper layers of HNSW-RaBitQ, we compute the 1-bit estimated distance (using `BinData`) to quickly locate the entry point for the next layer.
#### Base Layer
diff --git a/docs/docs/index/ivf.md b/docs/docs/index/ivf.md
index a545f286..64ad3c51 100644
--- a/docs/docs/index/ivf.md
+++ b/docs/docs/index/ivf.md
@@ -1,9 +1,9 @@
-# IVF RaBitQ
+# IVF-RaBitQ
C++ callers include `rabitqlib/index/ivf/ivf.hpp` and link `rabitqlib::index`.
[IVF](https://dl.acm.org/doi/10.1109/TPAMI.2010.57) is a classical
-clustering-based ANN method. This implementation stores RaBitQ codes and
+clustering-based ANN method. IVF-RaBitQ stores RaBitQ codes and
vector IDs, optionally retaining raw vectors for reranking, and uses
[FastScan](https://dl.acm.org/doi/abs/10.1145/3078971.3078992) to estimate a
batch of distances. Its actual memory, latency, and recall depend on the bit
@@ -13,7 +13,7 @@ width, number of clusters, and `nprobe`.
### Python
-The [complete Quick Start](../quick_start.md) is the runnable IVF example:
+The [complete Quick Start](../quick_start.md) is the runnable IVF-RaBitQ example:
train centroids, call `IvfIndex.build`, search, then save and reload the index.
The update examples below reuse its built `index`.
@@ -46,7 +46,7 @@ See [C++ integration](../cpp_integration.md) for build setup and
| Parameter | Contract |
| --- | --- |
| `dim` | Original input dimension; Python supports 64–65,536 and pads internally. |
-| `max_elements` | Python IVF build requires exactly this many rows; `add` grows storage automatically. C++ calls this constructor argument `num_points`. |
+| `max_elements` | Python IVF-RaBitQ build requires exactly this many rows; `add` grows storage automatically. C++ calls this constructor argument `num_points`. |
| `num_clusters` | Must match the centroid count; every cluster ID must be in `[0, num_clusters)`. |
| `nbits` / C++ `total_bits` | 1–9 total bits including the sign bit, or 32 for owned raw-vector reranking. |
| `metric` | Python `"l2"` or `"ip"`; defaults to `"l2"`. |
@@ -103,7 +103,7 @@ rabitqlib::load_vecs(cids_file, cids);
See the [complete indexing example](https://github.com/VectorDB-NTU/RaBitQ-Library/blob/main/sample/cpp/ivf_rabitq_indexing.cpp)
for argument parsing and the surrounding program.
-Then initialize an IVF object with the number of data points, vector
+Then initialize an IVF-RaBitQ object with the number of data points, vector
dimension, number of clusters, and total bits per dimension. The C++ index
accepts total quantized bit widths from 1 through 9, or `total_bits=32`
(Python `nbits=32`) to store raw float32 vectors for reranking.
@@ -148,7 +148,7 @@ auto routing = ivf.initializer_type();
routing is approximate: pruning can miss a true nearest centroid.
- `hnsw` searches a graph of float32 centroids approximately.
-All choices support L2 and inner product. Flat RaBitQ uses the IVF rotation and
+All choices support L2 and inner product. Flat RaBitQ uses the IVF-RaBitQ rotation and
padded domain, retains float32 centroids for refinement, and stores its mean and
codes in the main index file. Its code precision is independent of the index's
`nbits`. It trades additional code storage and an O(cluster count) query work
@@ -356,7 +356,7 @@ A removed point has its `f_add` value set to `+inf`. That value is the constant
term of the distance estimate, so the estimated distance and the lower bound of the
point are `+inf` for every bit width, metric, and SIMD backend. The point never
enters the result buffer and never triggers reranking. Any code that estimates
-IVF distances must keep this behavior and must never turn `+inf` into NaN.
+IVF-RaBitQ distances must keep this behavior and must never turn `+inf` into NaN.
Search may return fewer than `k` results once points are removed: the missing slots
hold `kPidMax` (`2**32 - 1` in Python) with an infinite distance, as when the probed
@@ -412,13 +412,13 @@ Extra-bit layouts retain full 64-coordinate blocks and store any final
### Data layout
-IVF stores the following data:
+IVF-RaBitQ stores the following data:
```c++
[batch data] // 1-bit code and factors
[ex_data] // code for remaining bits, or original float32 vectors
[ids] // PID of vectors (organized by clusters)
-[cluster_lst] // List of clusters' metadata in IVF
+[cluster_lst] // List of clusters' metadata in IVF-RaBitQ
```
In raw mode, `ex_data` holds `4 * num_points * dim` bytes in cluster order,
without padding or extra-bit factors. The one-bit filtering codes and factors
diff --git a/docs/docs/index/usage.md b/docs/docs/index/usage.md
index e4ec2c0e..0eb2f34e 100644
--- a/docs/docs/index/usage.md
+++ b/docs/docs/index/usage.md
@@ -1,6 +1,6 @@
# Index usage contracts
-Shared input, count, concurrency and persistence conventions for IVF, HNSW and
+Shared input, count, concurrency and persistence conventions for IVF-RaBitQ, HNSW-RaBitQ and
SymphonyQG. Start with the [Python quick start](../quick_start.md) for a complete example.
## Input requirements
@@ -22,10 +22,10 @@ the Python indexes do not expose a rotator choice.
## Index counts
The unified count properties below are introduced in **0.7.0**. In particular,
-HNSW's `num_points` now excludes removed records; see the
+HNSW-RaBitQ's `num_points` now excludes removed records; see the
[count migration guide](../modular_migration.md#unified-index-counts) when upgrading.
-IVF, HNSW, and SymphonyQG expose these read-only Python properties
+IVF-RaBitQ, HNSW-RaBitQ, and SymphonyQG expose these read-only Python properties
(C++ uses the same names with `()`):
| Property | Meaning |
@@ -37,15 +37,15 @@ IVF, HNSW, and SymphonyQG expose these read-only Python properties
`0 <= num_points <= num_stored_points <= max_elements`. Before building, both
point counts are zero; configured capacity may be nonzero.
-At build time, IVF requires exactly `max_elements` input rows. HNSW accepts
+At build time, IVF-RaBitQ requires exactly `max_elements` input rows. HNSW-RaBitQ accepts
1 through `max_elements` rows, so it can reserve capacity for later additions.
Removal decreases only `num_points`; slots and IDs remain occupied. Append IDs
start at the previous `num_stored_points`. Use this stored count for ID bounds,
and `num_points` for a live count. Counts survive save/load.
-IVF grows automatically and has no spare logical capacity after build or append.
-HNSW's `resize` changes only capacity. SymphonyQG has no incremental updates;
+IVF-RaBitQ grows automatically and has no spare logical capacity after build or append.
+HNSW-RaBitQ's `resize` changes only capacity. SymphonyQG has no incremental updates;
all three counts are equal after build.
## Estimate storage
@@ -55,28 +55,28 @@ memory or peak construction memory. Let `d` be the original dimension, `D` the
stored padded dimension, and `B` the total quantized bits per dimension. New
indexes use `D = 32 * ceil(d / 32)`; loaded indexes retain their saved padding.
-For IVF/HNSW, codes and factors occupy `V(B)` bytes per vector:
+For IVF-RaBitQ/HNSW-RaBitQ, codes and factors occupy `V(B)` bytes per vector:
- One bit: `V(1) = D / 8 + 12` (three float32 factors).
- Two through nine bits: `V(B) = D * B / 8 + 20` (three binary and two extra-code factors).
| Index / mode | Main bytes per stored point or capacity slot |
| --- | --- |
-| IVF quantized | Approximately `V(B) + 4`, including the point ID |
-| IVF raw (`nbits=32`) | Approximately `D / 8 + 12 + 4*d + 4`, including filtering codes, raw vector and ID |
-| HNSW | `V(B) + 8*M + 12` per **capacity slot** for base-layer records: up to `2*M` neighbor IDs, a link count, cluster ID and label |
+| IVF-RaBitQ quantized | Approximately `V(B) + 4`, including the point ID |
+| IVF-RaBitQ raw (`nbits=32`) | Approximately `D / 8 + 12 + 4*d + 4`, including filtering codes, raw vector and ID |
+| HNSW-RaBitQ | `V(B) + 8*M + 12` per **capacity slot** for base-layer records: up to `2*M` neighbor IDs, a link count, cluster ID and label |
| SymphonyQG raw | `4*d + R*(D/8 + 12)`, where `R = max_degree` |
| SymphonyQG quantized | `D*B/8 + 8 + R*(D/8 + 12)`, with `B` equal to 4 or 8 |
-IVF allocates one-bit codes and their three factors in full batches of 32
+IVF-RaBitQ allocates one-bit codes and their three factors in full batches of 32
**per cluster**. For cluster sizes `n_c`, that part costs
`sum(32 * ceil(n_c / 32)) * (D/8 + 12)` bytes. Extra codes/raw vectors and IDs
use the stored point count. Small clusters therefore increase padding overhead.
Centroids, routing structures and cluster metadata are additional.
-HNSW allocates base-layer records for `max_elements`, including unused capacity.
+HNSW-RaBitQ allocates base-layer records for `max_elements`, including unused capacity.
Upper-layer edges, label maps, locks, visited lists and other bookkeeping are
-additional. Removing points does not reclaim their storage in either IVF or HNSW.
+additional. Removing points does not reclaim their storage in either IVF-RaBitQ or HNSW-RaBitQ.
SymphonyQG stores a one-bit code and two factors **for each neighbor in each row**,
plus a four-byte neighbor ID. This accounts for `R*(D/8 + 12)` bytes per row,
@@ -84,7 +84,7 @@ even in quantized mode: reducing the vector's own code width does not shrink
these neighborhood copies.
For example, with `d=D=128`, four-bit storage, `M=16` and `R=32`, the terms above
-are about 88 bytes per IVF point before batch padding, 224 bytes per HNSW capacity
+are about 88 bytes per IVF-RaBitQ point before batch padding, 224 bytes per HNSW-RaBitQ capacity
slot, and 968 bytes per SymphonyQG row. These are layout calculations, not measured
RSS. All indexes also retain rotation state; construction, query scratch,
allocator overhead and update workspaces add to memory use. Measure process
diff --git a/docs/docs/integrations/txtai.md b/docs/docs/integrations/txtai.md
index 417dbf85..755e0297 100644
--- a/docs/docs/integrations/txtai.md
+++ b/docs/docs/integrations/txtai.md
@@ -1,7 +1,7 @@
# txtai: using the complete indexes
[txtai](https://github.com/neuml/txtai) exposes RaBitQ-Library through its
-`rabitq` ANN backend. Applications can select an IVF or HNSW index within txtai's
+`rabitq` ANN backend. Applications can select an IVF-RaBitQ or HNSW-RaBitQ index within txtai's
embedding-search workflow, while txtai handles the surrounding application API
and result conversion.
@@ -13,7 +13,7 @@ Consult the [current configuration reference][current-config] for later changes.
## Select the backend
In an existing txtai embeddings configuration, select `backend: rabitq` and add
-the corresponding settings. This fragment uses four-bit IVF:
+the corresponding settings. This fragment uses four-bit IVF-RaBitQ:
```yaml
backend: rabitq
@@ -28,10 +28,10 @@ adapter imports both `rabitqlib` and scikit-learn. The inspected
[package configuration][dependencies] declares `rabitqlib>=0.7.0` in its `ann` extra. The fragment above selects
the backend, rather than providing a complete model-and-data example.
-`clusters` controls the KMeans centroid count in both modes: IVF uses these
-centroids for partitioning, while HNSW uses them for residual quantization.
-`nprobe` controls how many IVF clusters are searched. For HNSW, select `mode: hnsw` and use `m`, `efconstruction`, and
-`efsearch`. Both modes accept 1–9 quantization bits; IVF also accepts `nbits: 32`
+`clusters` controls the KMeans centroid count in both modes: IVF-RaBitQ uses these
+centroids for partitioning, while HNSW-RaBitQ uses them for residual quantization.
+`nprobe` controls how many IVF-RaBitQ clusters are searched. For HNSW-RaBitQ, select `mode: hnsw` and use `m`, `efconstruction`, and
+`efsearch`. Both modes accept 1–9 quantization bits; IVF-RaBitQ also accepts `nbits: 32`
for original-vector reranking. Measure recall and latency on representative
queries when choosing precision and search parameters.
@@ -42,19 +42,19 @@ The [adapter source][adapter] shows the boundary between the projects:
| Stage | txtai's role | RaBitQ-Library's role |
| --- | --- | --- |
| Build | Converts embeddings to contiguous float32 and uses scikit-learn KMeans to produce centroids and assignments. | Builds `IvfIndex` or `HnswIndex` with the inner-product metric. |
-| Search | Selects `nprobe` or `ef`, drops missing-result IDs, and converts returned distances to scores with `1 - distance`. | Searches the native index and returns IDs and distances; quantized modes return estimates, while IVF raw mode uses original-vector reranking. |
+| Search | Selects `nprobe` or `ef`, drops missing-result IDs, and converts returned distances to scores with `1 - distance`. | Searches the native index and returns IDs and distances; quantized modes return estimates, while IVF-RaBitQ raw mode uses original-vector reranking. |
| Persistence | Calls the backend's save/load methods. | Stores and restores the native index state. |
-| Updates | Routes IVF append/delete requests to the backend. | Adds vectors or removes them from search. |
+| Updates | Routes IVF-RaBitQ append/delete requests to the backend. | Adds vectors or removes them from search. |
This is an example of integrating complete indexes: txtai calls the Python index
APIs without implementing RaBitQ encoding or graph traversal itself.
## Boundaries to keep in mind
-At the inspected revision, txtai enables append/delete only for IVF. Its HNSW
+At the inspected revision, txtai enables append/delete only for IVF-RaBitQ. Its HNSW-RaBitQ
adapter raises `NotImplementedError` for append/delete through the
-[ANN base class][base], even though RaBitQ-Library itself supports HNSW updates.
-IVF additions reuse the original clusters, and deletions retain their storage;
+[ANN base class][base], even though RaBitQ-Library itself supports HNSW-RaBitQ updates.
+IVF-RaBitQ additions reuse the original clusters, and deletions retain their storage;
txtai recommends rebuilding when new data differs enough to affect recall.
See its [versioned configuration documentation][config] for these limits.
@@ -67,7 +67,7 @@ production deployment scale or a performance gain, and this configuration has
not been exercised end to end in this documentation update.
To use the same indexes directly, follow the library's [Python quick start](../quick_start.md)
-and [HNSW guide](../index/hnsw.md). To integrate only quantization into your own
+and [HNSW-RaBitQ guide](../index/hnsw.md). To integrate only quantization into your own
index, see the [zvec walkthrough](zvec.md).
[snapshot]: https://github.com/neuml/txtai/tree/0ea7cb0d8234497e8ea9ed1a0f2f58492e00460b
diff --git a/docs/docs/integrations/zvec.md b/docs/docs/integrations/zvec.md
index e7199528..01bb10a7 100644
--- a/docs/docs/integrations/zvec.md
+++ b/docs/docs/integrations/zvec.md
@@ -13,8 +13,8 @@ This identifies the inspected source, not a released zvec package version.
| Index | How it uses RaBitQ-Library |
| --- | --- |
-| **IVF** | The [reformer][ivf-reformer] quantizes vectors in batches and prepares query lookup tables. The [search code][ivf-entity] uses FastScan estimates and extra-bit refinement for selected candidates. |
-| **HNSW** | The [reformer][hnsw-reformer] quantizes individual vectors. The [query algorithm][hnsw-query] uses core per-vector estimators during graph traversal; it does not use the IVF FastScan batch-scoring path. |
+| **IVF-RaBitQ** | The [reformer][ivf-reformer] quantizes vectors in batches and prepares query lookup tables. The [search code][ivf-entity] uses FastScan estimates and extra-bit refinement for selected candidates. |
+| **HNSW-RaBitQ** | The [reformer][hnsw-reformer] quantizes individual vectors. The [query algorithm][hnsw-query] uses core per-vector estimators during graph traversal; it does not use the IVF-RaBitQ FastScan batch-scoring path. |
The inspected reformers serialize centroids, padded dimensions, and rotator
state, then restore them when loading. Their query preprocessing uses the stored
@@ -31,7 +31,7 @@ targets and `SplitBatchQuery::reset`.
When `RABITQ_SUPPORTED` is enabled, the [dependency configuration][dependency]
links `rabitqlib::fastscan`, which also links core,
and disables RaBitQ's standalone indexes, Python bindings, OpenMP, and native
-tuning. The [IVF query preparation][ivf-reformer] passes lookup-table buffers owned by
+tuning. The [IVF-RaBitQ query preparation][ivf-reformer] passes lookup-table buffers owned by
its query state to `SplitBatchQuery::reset`. This permits reuse when that state
is reused; it does not imply that every search is allocation-free.
diff --git a/docs/docs/kernel_ip.md b/docs/docs/kernel_ip.md
index dd826e48..f1fe7dc3 100644
--- a/docs/docs/kernel_ip.md
+++ b/docs/docs/kernel_ip.md
@@ -55,7 +55,7 @@ additional error is typically negligible relative to the RaBitQ encoding error.
As the data-code bit width increases, the encoding error shrinks and LUT precision
becomes more significant. High-accuracy FastScan (HACC) uses 16-bit LUT entries
to reduce the error carried from the sign-code inner product into extra-bit
-refinement. IVF selects HACC by default for 4–9 total bits and standard FastScan
+refinement. IVF-RaBitQ selects HACC by default for 4–9 total bits and standard FastScan
for 1–3 bits or raw-vector reranking; see [precision selection](index/ivf.md#querying).
## The kernel for multi-bit codes
diff --git a/docs/docs/modular_migration.md b/docs/docs/modular_migration.md
index f9faaf31..e0843b10 100644
--- a/docs/docs/modular_migration.md
+++ b/docs/docs/modular_migration.md
@@ -22,7 +22,7 @@ Replace `rabitqlib::rabitqlib` with the smallest component needed:
| --- | --- | --- |
| `rabitqlib::core` | Encoding, single-vector estimation, rotation and packing | Numerical helpers and essential SIMD |
| `rabitqlib::fastscan` | Batch layouts, packing/encoding adapters and FastScan/HACC scoring | core |
-| `rabitqlib::index` | IVF, HNSW, SymphonyQG and clustering | fastscan and core |
+| `rabitqlib::index` | IVF-RaBitQ, HNSW-RaBitQ, SymphonyQG and clustering | fastscan and core |
| `rabitqlib::python` | Existing Python ANN extension; build-tree alias only | index |
The old implementation target `rabitq_headers` is also removed. C++ libraries remain
@@ -106,7 +106,7 @@ integer arguments; pass `METRIC_L2` or `METRIC_IP`.
### Index headers
-IVF, HNSW and SymphonyQG operations now compile into `rabitqlib::index`. Their
+IVF-RaBitQ, HNSW-RaBitQ and SymphonyQG operations now compile into `rabitqlib::index`. Their
main headers no longer import numerical implementations, search scratch, Eigen
or hnswlib. This implementation move preserves method signatures and file formats.
@@ -120,12 +120,14 @@ Include the types you use explicitly:
| Internal `SearchScratch` | `index/symqg/detail/search_scratch.hpp` |
Explicit includes of `core/rotator.hpp`, `core/encoder.hpp` and
-`index/ivf/initializer.hpp` still expose their associated types. HNSW backend
+`index/ivf/initializer.hpp` still expose their associated types. HNSW-RaBitQ backend
search templates are private in `src/index/hnsw_search_impl.hpp`.
-### Removed HNSW members
+
-HNSW no longer exposes `HierarchicalNSW::rawDataPtr_`, `ResultRecord`, `Candidate`,
+### Removed HNSW-RaBitQ members
+
+HNSW-RaBitQ no longer exposes `HierarchicalNSW::rawDataPtr_`, `ResultRecord`, `Candidate`,
or `BoundedKNN` as public interfaces. The construction pointer and search helpers
are private implementation details. This is a C++ source compatibility break:
code that accessed those members must stop doing so. Pass construction input through
@@ -145,7 +147,7 @@ Internal headers have explicit ownership:
| Core SIMD declarations | `core/detail/simd/` |
| FastScan SIMD declarations and LUT implementation | `fastscan/detail/` |
| FastScan batch-encoder adapter | `fastscan/detail/encoder.hpp` |
-| HNSW dispatch, search buffers, visited sets and shared scheduling | `index/detail/` |
+| HNSW-RaBitQ dispatch, search buffers, visited sets and shared scheduling | `index/detail/` |
| Non-template dispatcher implementation and private resolver | `src/simd/`, `src/fastscan/` and `src/index/` |
The old `simd/*` public paths, `fastscan/{simd,estimator_dispatch,lut}.hpp`,
@@ -155,8 +157,8 @@ operations; direct use of `detail/` binds a caller to that implementation revisi
Some detail headers must be installed because public templates include them. Installation
does not turn them into a stable extension API.
-HNSW and IVF share `index/detail/parallel_for.hpp`. Callers relying on HNSW
-to import the IVF initializer must include it explicitly. Scheduling behavior
+HNSW-RaBitQ and IVF-RaBitQ share `index/detail/parallel_for.hpp`. Callers relying on HNSW-RaBitQ
+to import the IVF-RaBitQ initializer must include it explicitly. Scheduling behavior
is unchanged; custom thread-pool and SIMD registration remain unsupported.
## Unified index counts
@@ -164,15 +166,15 @@ is unchanged; custom thread-pool and SIMD registration remain unsupported.
All three indexes now share the [count contract](index/usage.md#index-counts)
in C++ and Python.
-Compared with the 0.6.0 Python API, IVF adds `num_points` and `num_stored_points`,
-HNSW adds `num_stored_points`, and SymphonyQG adds `max_elements` and
+Compared with the 0.6.0 Python API, IVF-RaBitQ adds `num_points` and `num_stored_points`,
+HNSW-RaBitQ adds `num_stored_points`, and SymphonyQG adds `max_elements` and
`num_stored_points`.
-**HNSW `num_points` now excludes removed records.** Use `num_stored_points` for
+**HNSW-RaBitQ `num_points` now excludes removed records.** Use `num_stored_points` for
ID bounds, append offsets, and minimum resize capacity. Use `num_points` for a
live count, without subtracting a separate deletion counter.
-The IVF constructor keyword `max_elements` still specifies the required initial
+The IVF-RaBitQ constructor keyword `max_elements` still specifies the required initial
build size. SymphonyQG's C++ `num_vertices()` still returns the configured graph
size for builders. Existing index files need no conversion: loading recovers
counts from their removal markers.
diff --git a/docs/docs/quick_start.md b/docs/docs/quick_start.md
index c33ac4a9..5759a89e 100644
--- a/docs/docs/quick_start.md
+++ b/docs/docs/quick_start.md
@@ -1,6 +1,6 @@
# Quick start
-Build, search, save and reload a small IVF index with Python. This guide describes
+Build, search, save and reload a small IVF-RaBitQ index with Python. This guide describes
**0.7.0** and uses synthetic data, so no dataset download is required.
For C++, go directly to [C++ integration](cpp_integration.md).
@@ -18,7 +18,9 @@ Supply data and queries with shape `(count, dim)` in the original dimension.
Python indexes convert them to contiguous float32 and accept `64 <= dim <= 65536`;
they rotate and pad internally. See [input requirements](index/usage.md#input-requirements).
-## Build and search an IVF index
+
+
+## Build and search an IVF-RaBitQ index
The following complete example uses deterministic synthetic data and does not
require a dataset download:
@@ -36,7 +38,7 @@ clustering = RaBitQKMeans(
)
clustering.train(data)
-# IVF requires exactly max_elements rows when building.
+# IVF-RaBitQ requires exactly max_elements rows when building.
index = IvfIndex(
dim=64,
max_elements=len(data),
@@ -60,8 +62,8 @@ squared Euclidean distance, while `metric="ip"` estimates `1 - dot(data, query)`
Smaller values are better for both metrics. Quantized distances are estimates,
not exact distances to the original vectors.
-IVF and HNSW may return fewer than `k` valid neighbors, for example after removals
-or when the probed IVF clusters contain too few points. Unfilled slots contain
+IVF-RaBitQ and HNSW-RaBitQ may return fewer than `k` valid neighbors, for example after removals
+or when the probed IVF-RaBitQ clusters contain too few points. Unfilled slots contain
ID `2**32 - 1` and distance `+inf`. Filter them with
`valid = ids != np.iinfo(np.uint32).max` before using IDs to index your data.
@@ -118,15 +120,15 @@ with TemporaryDirectory() as directory:
np.testing.assert_allclose(loaded_distances, distances)
```
-The loaded index owns its data and restores the saved rotation. IVF indexes using
+The loaded index owns its data and restores the saved rotation. IVF-RaBitQ indexes using
HNSW centroid routing also need the accompanying `.hnsw` file; keep it with the
main index. This small example uses Flat routing and has no sidecar.
See [file compatibility](index/usage.md#index-file-compatibility) before downgrading.
## Next steps
-- [IVF](index/ivf.md): raw-vector reranking, precision, centroid routing and updates.
-- [HNSW](index/hnsw.md) and [SymphonyQG](index/qg.md): graph-index workflows and parameters.
+- [IVF-RaBitQ](index/ivf.md): raw-vector reranking, precision, centroid routing and updates.
+- [HNSW-RaBitQ](index/hnsw.md) and [SymphonyQG](index/qg.md): graph-index workflows and parameters.
- [Index usage contracts](index/usage.md): counts, threading, input lifetimes and file paths.
- [Clustering](clustering.md): choose flat RaBitQKMeans or graph-based QGKMeans.
- [C++ integration](cpp_integration.md): builds, installation and component selection.
diff --git a/docs/docs/rabitq/quantizer.md b/docs/docs/rabitq/quantizer.md
index 9843ab60..422b12ce 100644
--- a/docs/docs/rabitq/quantizer.md
+++ b/docs/docs/rabitq/quantizer.md
@@ -272,7 +272,7 @@ initialized `batch_data`, `centroid`, `dim`, and `batch_size`.
std::vector est_distance(batch_size); // store the estimated distances
- // QG scoring returns estimates only. Use the IVF batch format below
+ // QG scoring returns estimates only. Use the IVF-RaBitQ batch format below
// when error intervals are needed.
rabitqlib::qg_batch_estdist(
@@ -281,13 +281,15 @@ initialized `batch_data`, `centroid`, `dim`, and `batch_size`.
```
-### Format 4: Split single-vector scoring (HNSW)
+
+
+### Format 4: Split single-vector scoring (HNSW-RaBitQ)
#### Indexing
Store a vector as separate sign and extra-bit codes, with factors for each.
Score the sign code first, then refine selected candidates using the extra bits.
-HNSW uses this layout.
+HNSW-RaBitQ uses this layout.
```cpp
#include
@@ -407,12 +409,14 @@ Insert this before `return 0` in the indexing example, reusing its data and dime
);
```
-### Format 5: Split batch scoring (IVF)
+
+
+### Format 5: Split batch scoring (IVF-RaBitQ)
#### Indexing
Batch the sign codes for FastScan and store each vector's extra-bit codes
-separately. IVF uses this layout to filter a batch before refining candidates.
+separately. IVF-RaBitQ uses this layout to filter a batch before refining candidates.
```cpp
#include
@@ -491,7 +495,7 @@ Insert this before `return 0` in the indexing example, reusing its buffers and d
// `use_hacc = false` - each number in LUTs is quantized into 8 bits.
// `use_hacc = true` - each number in LUTs is quantized into 16 bits.
// SplitBatchQuery defaults to HACC; this example selects it explicitly.
- // IVF instead selects HACC for 4-9 total bits and standard FastScan for
+ // IVF-RaBitQ instead selects HACC for 4-9 total bits and standard FastScan for
// 1-3 total bits or raw-vector storage, unless the caller overrides it.
rabitqlib::SplitBatchQuery processed_query(
diff --git a/docs/docs/rabitq/reranking.md b/docs/docs/rabitq/reranking.md
index 0a5b3452..19f9c933 100644
--- a/docs/docs/rabitq/reranking.md
+++ b/docs/docs/rabitq/reranking.md
@@ -13,8 +13,8 @@ need refinement. It does not guarantee exact top-k membership or exact distances
for results that were never reranked; measure the recall/cost trade-off for the workload.
This is an application-level strategy, not a built-in disk-retrieval API.
-HNSW stores quantized vectors, so applications must retain raw vectors separately
-for exact reranking. [IVF raw mode](../index/ivf.md#index-construction)
+HNSW-RaBitQ stores quantized vectors, so applications must retain raw vectors separately
+for exact reranking. [IVF-RaBitQ raw mode](../index/ivf.md#index-construction)
(`nbits=32`) instead owns the original vectors and reranks them in memory.
## Algorithm description
diff --git a/docs/docs/rabitq/rotator.md b/docs/docs/rabitq/rotator.md
index 5bc005a6..83efdb97 100644
--- a/docs/docs/rabitq/rotator.md
+++ b/docs/docs/rabitq/rotator.md
@@ -16,7 +16,7 @@ The default FHT/Kac rotator supports `float` and accepts input dimensions from 6
for power-of-two blocks through `2^16`. Allocate the
output using `rotator->size()`, which can exceed the input dimension.
-RaBitQKMeans, QGKMeans, SymphonyQG, IVF, and HNSW also use multiples of 32
+RaBitQKMeans, QGKMeans, SymphonyQG, IVF-RaBitQ, and HNSW-RaBitQ also use multiples of 32
for all their supported bit widths.
Index loading preserves the padded domain used when the index was built.
Reducing the padded dimension changes the sampled rotation and quantization even
diff --git a/docs/docs/releases/0.6.0.md b/docs/docs/releases/0.6.0.md
index c8303887..aad226c4 100644
--- a/docs/docs/releases/0.6.0.md
+++ b/docs/docs/releases/0.6.0.md
@@ -1,6 +1,6 @@
# RaBitQ Library 0.6.0
-This release adds concurrent Python index searches, configurable IVF centroid
+This release adds concurrent Python index searches, configurable IVF-RaBitQ centroid
routing, and 32-coordinate padding. It upgrades Eigen to 5.0.1 and stops
propagating optimization flags to CMake consumers.
@@ -14,7 +14,7 @@ together. Updates require exclusive access; conflicts immediately raise
`RuntimeError("Index is busy: conflicting operation in progress")`.
Applications that relied on the GIL to serialize access must coordinate updates.
-Keep input arrays unchanged until each call returns. HNSW's Python `ef` remains
+Keep input arrays unchanged until each call returns. HNSW-RaBitQ's Python `ef` remains
optional; SymphonyQG's remains required. With a Python thread pool, consider
`num_threads=1` to avoid multiplying native worker pools. Concurrent saves need
distinct paths, including sidecars. See the
@@ -23,7 +23,9 @@ distinct paths, including sidecars. See the
C++ callers must synchronize mutations themselves. SymphonyQG adds
`search_batch_with_ef()` for per-call windows; existing search APIs remain available.
-### IVF routing and saved files
+
+
+### IVF-RaBitQ routing and saved files
Python's `initializer` accepts `"auto"`, `"flat"`, `"flat_rabitq"`, or `"hnsw"`;
C++ provides the corresponding `InitializerType`. New indexes using `auto` select:
@@ -38,20 +40,20 @@ Flat RaBitQ and HNSW routing are approximate, so rebuilding with `auto` can chan
recall and latency. To retain the previous routing policy, select Flat below
20,000 clusters and HNSW otherwise.
-New IVF saves use `RABQIDX1` version 2 and record the resolved initializer.
+New IVF-RaBitQ saves use `RABQIDX1` version 2 and record the resolved initializer.
**Older releases cannot read these files.** Current readers support legacy
quantized files, raw v1 files, and `RABQIDX1` v1 files, retaining historical routing
rules where needed. Keep the `.hnsw` sidecar with HNSW-routed indexes.
-See the [IVF format guide](../index/ivf.md).
+See the [IVF-RaBitQ format guide](../index/ivf.md).
### Padding and historical compatibility
New indexes and clustering pad dimensions to multiples of 32. Loading preserves
the saved padding and rotation, including historical 64-coordinate padding.
-Older HNSW and SymphonyQG readers that require multiples of 64 cannot load
+Older HNSW-RaBitQ and SymphonyQG readers that require multiples of 64 cannot load
new files whose stored padding is not divisible by 64.
-Tests cover frozen v0.5.2 IVF, HNSW, and raw/quantized SymphonyQG files under L2
+Tests cover frozen v0.5.2 IVF-RaBitQ, HNSW-RaBitQ, and raw/quantized SymphonyQG files under L2
and inner product, including queries, resave/load migration, and malformed files.
This coverage does not guarantee compatibility with every historical format.
Keep original files if you may need to downgrade. See the
@@ -75,7 +77,7 @@ this setting and runtime dispatch.
- Thread-count detection accounts for Linux CPU affinity and supported OpenMP placement settings.
Clustering reduces parallelism in short graph-building phases.
-- HNSW construction/insertion error handling and allocation-failure tests are improved.
+- HNSW-RaBitQ construction/insertion error handling and allocation-failure tests are improved.
- Linux packages include GNU OpenMP license and exception texts.
- Benchmark results and reproduction protocols are documented with their software versions.
diff --git a/docs/docs/releases/0.7.0.md b/docs/docs/releases/0.7.0.md
index 01181901..ae5f4190 100644
--- a/docs/docs/releases/0.7.0.md
+++ b/docs/docs/releases/0.7.0.md
@@ -27,8 +27,8 @@ Advanced templates and index APIs are not covered by this stability guarantee.
### Index counts in C++ and Python
-IVF, HNSW and SymphonyQG share `max_elements`, `num_stored_points` and `num_points`
-for capacity, stored records and live points respectively. **HNSW's `num_points`
+IVF-RaBitQ, HNSW-RaBitQ and SymphonyQG share `max_elements`, `num_stored_points` and `num_points`
+for capacity, stored records and live points respectively. **HNSW-RaBitQ's `num_points`
now excludes removed records.** Use `num_stored_points` for ID bounds, append
offsets and minimum resize capacity. See the
[count migration details](../modular_migration.md#unified-index-counts).
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index e222dbf1..f07ef7bd 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -27,8 +27,8 @@ nav:
- Inner product: kernel_ip.md
- Vector search:
- Shared contracts: index/usage.md
- - IVF RaBitQ: index/ivf.md
- - HNSW RaBitQ: index/hnsw.md
+ - IVF-RaBitQ: index/ivf.md
+ - HNSW-RaBitQ: index/hnsw.md
- SymphonyQG: index/qg.md
- Integrations:
- txtai: integrations/txtai.md
@@ -45,8 +45,8 @@ theme:
name: material
language: en
font: false
+ logo: assets/logo.png
icon:
- logo: material/vector-polygon
repo: fontawesome/brands/github
palette:
- media: "(prefers-color-scheme: light)"
diff --git a/sample/python/README.md b/sample/python/README.md
index ebd910e5..0112f0ac 100644
--- a/sample/python/README.md
+++ b/sample/python/README.md
@@ -6,7 +6,7 @@ with `rabitqlib>=0.7.0` and NumPy installed. When working from source,
so the examples and native extension use matching APIs. Clustering and indexing
both use RaBitQ Library; FAISS is needed only for `compare_with_faiss.py`.
-## IVF: cluster, then index
+## IVF-RaBitQ: cluster, then index
`kmeans_clustering.py` uses `--method qg` (the default) for QGKMeans graph
assignment or `--method rabitq` for RaBitQKMeans flat assignment. Both make exact
@@ -39,7 +39,7 @@ training vectors. Use queries and ground truth prepared for the same metric.
its own limit. See [threading](../../docs/docs/index/usage.md#threading-and-file-paths)
for affinity and OpenMP placement behavior.
-## HNSW
+## HNSW-RaBitQ
Use [RaBitQKMeans](../../docs/docs/clustering.md#rabitqkmeans) for this small-cluster
example, with the same clustering file format:
diff --git a/tests/README.md b/tests/README.md
index fb9dd877..8413b300 100644
--- a/tests/README.md
+++ b/tests/README.md
@@ -232,8 +232,8 @@ CI runs this workflow on Linux x86-64, Linux ARM64, macOS ARM64 and Windows x86-
Core and FastScan consumption must also work from a full installation; the standalone
`sample/cpp/core` and `sample/cpp/fastscan` projects test their respective targets. The original
-`tests/consumer` exercises `rabitqlib::index` and checks that HNSW headers compile with
-the IVF initializer header deliberately unavailable and index value types compile with
+`tests/consumer` exercises `rabitqlib::index` and checks that HNSW-RaBitQ headers compile with
+the IVF-RaBitQ initializer header deliberately unavailable and index value types compile with
Eigen deliberately unavailable. Numerical consumers forbid the removed `defines.hpp`.
All FastScan batch layout/encoding coverage lives under `unit/rabitqlib/fastscan/`;
index ID and nearest-neighbor helper coverage lives under `unit/rabitqlib/index/`.
@@ -281,7 +281,7 @@ Keep a Python construction/search/persistence case for every exported storage wi
both metrics, but do not cross every width with every original dimension and initializer.
`test_portable_indexes.py` separates supported storage from representative padded-domain
-boundaries. IVF mutation/batching/routing matrices use sign-only, extra-bit and raw-vector
+boundaries. IVF-RaBitQ mutation/batching/routing matrices use sign-only, extra-bit and raw-vector
storage; full width coverage remains in the corresponding C++ reference tests. The
SymphonyQG batch test checks both Python names on one built index per storage/metric pair,
including strided/empty queries and bitwise distance equality.
@@ -302,7 +302,7 @@ entry points in the same setting.
`construction_failure_test` isolates allocation replacement from the normal suite.
It checks OpenMP construction errors with one/four requested threads and, on Linux,
-wraps C allocations to sweep HNSW constructor failures and verify ownership. Run
+wraps C allocations to sweep HNSW-RaBitQ constructor failures and verify ownership. Run
these under ASan/UBSan as well as Release; do not disable allocator-mismatch checking.
`compact_codes_failure_test` covers cold scratch allocation and growth failures in
serial and OpenMP encoding, including the public FastScan batch entry points.
diff --git a/tests/fixtures/persistence/README.md b/tests/fixtures/persistence/README.md
index e4858802..ee820cf2 100644
--- a/tests/fixtures/persistence/README.md
+++ b/tests/fixtures/persistence/README.md
@@ -9,11 +9,11 @@ adding a new format or release.
The twelve cases cover L2 and inner product for:
-- IVF four-bit legacy files without a header, and raw-vector IVF v1 files;
-- HNSW four-bit files, including persisted removal of point 0;
+- IVF-RaBitQ four-bit legacy files without a header, and raw-vector IVF-RaBitQ v1 files;
+- HNSW-RaBitQ four-bit files, including persisted removal of point 0;
- SymphonyQG legacy raw files and versioned four/eight-bit files.
-IVF also persists removal of point 0. Each file contains 33 points in 65 original
+IVF-RaBitQ also persists removal of point 0. Each file contains 33 points in 65 original
dimensions and the historical **128-dimensional padded domain**. This differs
from current construction's 96 dimensions and detects accidental regeneration
of rotation or padding on load. The `.expected` files hold the point count,
@@ -24,11 +24,11 @@ Tests require identical result IDs and allow distance error of
`2e-4 * max(1, abs(reference))` in C++, or `rtol=atol=2e-4` in Python, to account
for compiler and SIMD reductions. The native formats contain `size_t`: these
fixtures target the supported little-endian, 64-bit platforms, not a portable
-interchange format for arbitrary architectures. Python also tests synthetic IVF
+interchange format for arbitrary architectures. Python also tests synthetic IVF-RaBitQ
v1 headers around the frozen historical payload, unknown versions, and invalid
padding. That header test is distinct from release-produced fixtures.
-The small IVF fixtures use Flat centroid routing. Current round-trip and failure
+The small IVF-RaBitQ fixtures use Flat centroid routing. Current round-trip and failure
tests exercise explicit HNSW routing and its sidecar; historical auto-selected
HNSW routing at 20,000 clusters is not represented by these small fixtures.
diff --git a/tests/fixtures/persistence/pre-ivf-per-cluster-storage/README.md b/tests/fixtures/persistence/pre-ivf-per-cluster-storage/README.md
index b270766b..3e7c742d 100644
--- a/tests/fixtures/persistence/pre-ivf-per-cluster-storage/README.md
+++ b/tests/fixtures/persistence/pre-ivf-per-cluster-storage/README.md
@@ -1,12 +1,12 @@
-# IVF fixtures from before per-cluster storage
+# IVF-RaBitQ fixtures from before per-cluster storage
These files were written by `main` at commit
-`64fc943e0e92fa7e18eec1477050b23342a976f8`, the last commit where IVF kept every
+`64fc943e0e92fa7e18eec1477050b23342a976f8`, the last commit where IVF-RaBitQ kept every
cluster in one shared buffer, using [generate.cpp](generate.cpp).
[manifest.json](manifest.json) records the producer and SHA-256 hashes.
Keep these files unchanged; tests never regenerate them with the current writer.
-The two cases cover four-bit and raw-vector (`nbits=32`) IVF for L2.
+The two cases cover four-bit and raw-vector (`nbits=32`) IVF-RaBitQ for L2.
Each index holds 130 points in 65 original dimensions (96 padded) across
four clusters of 0, 1, 32 and 97 points, so the file has an empty cluster, a
single-point cluster, a full FastScan batch and a partial tail batch. Point 0 is
diff --git a/tests/python/fixtures/README.md b/tests/python/fixtures/README.md
index 56972902..c7329683 100644
--- a/tests/python/fixtures/README.md
+++ b/tests/python/fixtures/README.md
@@ -1,4 +1,4 @@
-`ivf_legacy_4bit.index` was saved by the IVF implementation before raw-vector
+`ivf_legacy_4bit.index` was saved by the IVF-RaBitQ implementation before raw-vector
storage was introduced (legacy, unversioned x86-64 format). It contains three
vectors: `[0, 0, 0]`, `[1, 2, 3]`, and `[-1, -2, -3]`, each zero-padded to
64 dimensions, in one cluster with a zero centroid, four-bit L2 quantization,
@@ -13,13 +13,13 @@ zero centroid, and the same three vectors above with zeros in all remaining
coordinates. Their widths are 1 and 32 bits, with L2 distance and FHT/Kac rotation.
They verify that loading does not substitute the new 96-coordinate default.
-The IVF tests also query `q[i] = (i % 9 - 4) / 4` before and after migration.
+The IVF-RaBitQ tests also query `q[i] = (i % 9 - 4) / 4` before and after migration.
The expected IDs and distances were recorded using the pad64 implementation at
commit `a065785`, covering nonzero query rotation and quantized distance estimation
as well as raw-vector reranking.
-`hnsw_legacy_padding65_4bit.index` was written before HNSW switched to 32-coordinate
+`hnsw_legacy_padding65_4bit.index` was written before HNSW-RaBitQ switched to 32-coordinate
padding: dimension 65, padded dimension 128, 4 total bits, L2, capacity 8, M=2,
-ef_construction=8. Its three vectors and zero centroid match the dimension-65 IVF
+ef_construction=8. Its three vectors and zero centroid match the dimension-65 IVF-RaBitQ
fixtures above. A zero query returns IDs 0, 1, 2 with distances 0, 14, 14. Tests
check loading, preserving the old padded domain through save/load, and insertion.