Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -604,7 +604,7 @@ These conventions apply to new internal variables and compatible renaming work.
| --- | --- |
| Dimensions and counts | `dim`, `padded_dim`, `num_points`, `num_queries`, `num_clusters` |
| Index state | `max_elements_`, `num_stored_points_`, `num_points()` |
| Identity and outputs | `internal_id`, `node_id`, `label`, `cluster_id`, `result_ids`, `distances` |
| Identity and outputs | `internal_id`, `label`, `cluster_id`, `result_ids`, `distances` |
| Search | `candidate_set`, `search_pool`, `result_pool`, `ef_search`, `ef_construction`, `nprobe` |
| Storage | `*_offset_`; record/link byte sizes named after the stored object |
| Parallelism and encoding | `num_threads`, `num_workers`, `fast_quantization` |
Expand All @@ -630,7 +630,7 @@ counts from capacity only after construction completes.

**Point identity**

`internal_id` or `node_id` for graph slots; `label` for a caller-visible HNSW label;
`internal_id` for graph slots; `label` for a caller-visible HNSW label;
`cluster_id` for centroid routing. Never assume an external label equals an internal slot.

**Search outputs**
Expand Down
12 changes: 6 additions & 6 deletions docs/docs/index/ivf.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ For example, split SIFT into 4,096 clusters using squared L2 distance:
```

After files are prepared, load them into memory. The snippet below uses
`const char*` paths `data_file`, `centroids_file`, and `cids_file`. Include the
`const char*` paths `data_file`, `centroids_file`, and `cluster_ids_file`. Include the
Eigen aliases and I/O helpers explicitly; `ivf.hpp` does not import them.

```c++
Expand All @@ -89,15 +89,15 @@ Eigen aliases and I/O helpers explicitly; `ivf.hpp` does not import them.

using PID = rabitqlib::PID;
using data_type = rabitqlib::RowMajorArray<float>;
using gt_type = rabitqlib::RowMajorArray<PID>;
using cluster_ids_type = rabitqlib::RowMajorArray<PID>;

data_type data;
data_type centroids;
gt_type cids;
cluster_ids_type cluster_ids;

rabitqlib::load_vecs<float, data_type>(data_file, data);
rabitqlib::load_vecs<float, data_type>(centroids_file, centroids);
rabitqlib::load_vecs<PID, gt_type>(cids_file, cids);
rabitqlib::load_vecs<PID, cluster_ids_type>(cluster_ids_file, cluster_ids);
```

See the [complete indexing example](https://github.com/VectorDB-NTU/RaBitQ-Library/blob/main/sample/cpp/ivf_rabitq_indexing.cpp)
Expand Down Expand Up @@ -125,7 +125,7 @@ The optional final C++ constructor argument selects `InitializerType::Auto`,
`initializer="auto"`, `"flat"`, `"flat_rabitq"`, or `"hnsw"`:

```python
index = IvfIndex(dim, num_points, num_clusters, nbits=4,
index = IvfIndex(dim, max_elements, num_clusters, nbits=4,
initializer="flat_rabitq")
print(index.initializer) # resolved routing type
```
Expand Down Expand Up @@ -196,7 +196,7 @@ void IVF::construct(
For example:

```c++
ivf.construct(data.data(), centroids.data(), cids.data(), true);
ivf.construct(data.data(), centroids.data(), cluster_ids.data(), true);
```

## Querying
Expand Down
78 changes: 40 additions & 38 deletions docs/docs/rabitq/estimator.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ factors. For API usage, see [Stable core API](core_api.md) or the advanced
| $\mathbf{\bar o}$ | Unit-length quantized residual direction, expressed in the original padded domain. |
| $\mathbf{q}_r'$ | The query in the quantized coordinate system: $\mathbf{q}_r'=P^{-1}\mathbf{q}_r$. |
| $S_q$ | $S_q=\sum_{i=1}^{D} \mathbf{q}'_{r}[i].$ |
| $\Delta_x$ | Residual-norm scale defined below; distinct from the estimator factor `F_rescale`. |
| $\Delta_x$ | Residual-norm scale defined below; distinct from the estimator factor `f_rescale`. |
| $\mathbf{x}_u$ | Unsigned integer codes.|

## Quantization
Expand Down Expand Up @@ -91,16 +91,17 @@ $$
\end{align}
$$

Store the following factors (`F_` for data and `G_` for the query).
Store the following factors. Data factors use the `f_` accessors of the code layouts;
query factors use the `g_` accessors of the prepared query objects.

| Name (Type) of Variable | Description |
| ----------------- | ----------- |
| `F_add (float)`| $\|\mathbf{o}_r-\mathbf{c}\|^2+2\| \mathbf{o}_r-\mathbf{c}\| \frac{\left< \mathbf{\bar o}, \mathbf{c}\right>}{\left<\mathbf{\bar o},\mathbf{o} \right>}$ |
| `F_rescale (float)`| $-2\frac{\Delta_x}{\left<\mathbf{\bar o},\mathbf{o} \right>}$|
| `F_error (float)`| $\ 2 \|\mathbf{o}_r-\mathbf{c}\| \cdot \sqrt{\frac{1 - \left< \mathbf{\bar o},\mathbf{o}\right>^2}{\left< \mathbf{\bar o},\mathbf{o}\right>^2}} \frac{\epsilon_{0}}{\sqrt{D-1}}$|
| `G_add (float)`| $\|\mathbf{q}_r-\mathbf{c}\|^2$ |
| `G_kBxSumq (float)`| $c_BS_q$|
| `G_error (float)`| $\| \mathbf{q}_r-\mathbf{c}\|$|
| `f_add (float)`| $\|\mathbf{o}_r-\mathbf{c}\|^2+2\| \mathbf{o}_r-\mathbf{c}\| \frac{\left< \mathbf{\bar o}, \mathbf{c}\right>}{\left<\mathbf{\bar o},\mathbf{o} \right>}$ |
| `f_rescale (float)`| $-2\frac{\Delta_x}{\left<\mathbf{\bar o},\mathbf{o} \right>}$|
| `f_error (float)`| $\ 2 \|\mathbf{o}_r-\mathbf{c}\| \cdot \sqrt{\frac{1 - \left< \mathbf{\bar o},\mathbf{o}\right>^2}{\left< \mathbf{\bar o},\mathbf{o}\right>^2}} \frac{\epsilon_{0}}{\sqrt{D-1}}$|
| `g_add (float)`| $\|\mathbf{q}_r-\mathbf{c}\|^2$ |
| `g_kbxsumq (float)`| $c_BS_q$|
| `g_error (float)`| $\| \mathbf{q}_r-\mathbf{c}\|$|

### Estimator of inner product

Expand All @@ -126,17 +127,18 @@ $$
\end{align}
$$

Store the following factors (`F_` for data and `G_` for the query).
Store the following factors. Data factors use the `f_` accessors of the code layouts;
query factors use the `g_` accessors of the prepared query objects.

| Name (Type) of Variable | Description |
| ----------------- | ----------- |
| `F_add (float)`| $1-\left< \mathbf{o}_r-\mathbf{c},\mathbf{c}\right>+\| \mathbf{o}_r-\mathbf{c}\| \frac{\left< \mathbf{\bar o}, \mathbf{c}\right>}{\left<\mathbf{\bar o},\mathbf{o} \right>}$ |
| `F_rescale (float)`| $-\frac{\Delta_x}{\left<\mathbf{\bar o},\mathbf{o} \right>}$|
| `F_error (float)`| $\|\mathbf{o}_r-\mathbf{c}\| \cdot \sqrt{\frac{1 - \left< \mathbf{\bar o},\mathbf{o}\right>^2}{\left< \mathbf{\bar o},\mathbf{o}\right>^2}} \frac{\epsilon_{0}}{\sqrt{D-1}}$|
| `G_add (float)`| $-\left< \mathbf{q}_r,\mathbf{c}\right>$ |
| `G_kBxSumq (float)`| $c_BS_q$|
| `G_k1xSumq (float)`| $c_1S_q$|
| `G_error (float)`| $\| \mathbf{q}_r-\mathbf{c}\|$|
| `f_add (float)`| $1-\left< \mathbf{o}_r-\mathbf{c},\mathbf{c}\right>+\| \mathbf{o}_r-\mathbf{c}\| \frac{\left< \mathbf{\bar o}, \mathbf{c}\right>}{\left<\mathbf{\bar o},\mathbf{o} \right>}$ |
| `f_rescale (float)`| $-\frac{\Delta_x}{\left<\mathbf{\bar o},\mathbf{o} \right>}$|
| `f_error (float)`| $\|\mathbf{o}_r-\mathbf{c}\| \cdot \sqrt{\frac{1 - \left< \mathbf{\bar o},\mathbf{o}\right>^2}{\left< \mathbf{\bar o},\mathbf{o}\right>^2}} \frac{\epsilon_{0}}{\sqrt{D-1}}$|
| `g_add (float)`| $-\left< \mathbf{q}_r,\mathbf{c}\right>$ |
| `g_kbxsumq (float)`| $c_BS_q$|
| `g_k1xsumq (float)`| $c_1S_q$|
| `g_error (float)`| $\| \mathbf{q}_r-\mathbf{c}\|$|

## Implementation

Expand All @@ -150,34 +152,34 @@ when $B=1$.
```cpp

// Compute the estimated distance
// Note that G_add is dependent on the centroid.
float estimated_distance = F_add + G_add + F_rescale * (ip + G_kBxSumq);
// Note that g_add is dependent on the centroid.
float estimated_distance = f_add + g_add + f_rescale * (ip + g_kbxsumq);

// Compute the error bound
// Note that G_error is dependent on the centroid.
float error_bound = F_error * G_error;
// Compute the error bound
// Note that g_error is dependent on the centroid.
float error_bound = f_error * g_error;

// Compute the lower and upper bounds of the estimated distance
float lb_dist = estimated_distance - error_bound;
float ub_dist = estimated_distance + error_bound;
float lower_bound = estimated_distance - error_bound;
float upper_bound = estimated_distance + error_bound;
```

### Incremental distance estimation

Split codes separate the sign bit from the remaining $B-1$ extra bits.
Score the sign code first, then refine selected candidates with the extra bits.
Each part has its own factors. The binary layout stores `F_add`,
`F_rescale`, and `F_error`; the extended layout stores `F_add_ex` and
`F_rescale_ex`. For $B>1$, the logical codes satisfy
Each part has its own factors. The binary layout stores `f_add`,
`f_rescale`, and `f_error`; the extended layout stores `f_add_ex` and
`f_rescale_ex`. For $B>1$, the logical codes satisfy
$\mathbf{x}_u=2^{B-1}\mathbf{x}_{\mathrm{bin}}+\mathbf{x}_{\mathrm{ex}}$,
which gives the refined distance formula below.

The split-code implementation uses the binary error factor divided by
$2^{B-1}$ as its refined pruning margin. This is an approximation, not an
identity for the full-code error factor in the tables above: that factor depends
on the full-code direction's own $\langle\mathbf{\bar o},\mathbf{o}\rangle$.
The split layout does not store a separate `F_error_ex`.
The query factors are `G_add`, `G_error`, `G_kBxSumq` and `G_k1xSumq`.
The split layout does not store a separate `f_error_ex`.
The query factors are `g_add`, `g_error`, `g_kbxsumq` and `g_k1xsumq`.
Here `ip_bin` and `ip_ex` are the inner products of the sign and extra-bit
codes, respectively, with the rotated query.
These equations use exact code/query inner products. Binary filtering may instead
Expand All @@ -187,17 +189,17 @@ that additional error.

```cpp
// 1-bit dist
float estimated_distance = F_add + G_add + F_rescale * (ip_bin + G_k1xSumq);
float bound = F_error * G_error;
float ub_dist = estimated_distance + bound;
float lb_dist = estimated_distance - bound;
float estimated_distance = f_add + g_add + f_rescale * (ip_bin + g_k1xsumq);
float error_bound = f_error * g_error;
float upper_bound = estimated_distance + error_bound;
float lower_bound = estimated_distance - error_bound;

// boost to full-bit dist
float ex_scale = static_cast<float>(1 << (bits - 1));
float ex_est_dist =
F_add_ex + G_add +
F_rescale_ex * (ex_scale * ip_bin + ip_ex + G_kBxSumq);
float ex_bound = F_error * G_error / ex_scale;
float ex_ub_dist = ex_est_dist + ex_bound;
float ex_lb_dist = ex_est_dist - ex_bound;
float estimated_distance_ex =
f_add_ex + g_add +
f_rescale_ex * (ex_scale * ip_bin + ip_ex + g_kbxsumq);
float error_bound_ex = f_error * g_error / ex_scale;
float upper_bound_ex = estimated_distance_ex + error_bound_ex;
float lower_bound_ex = estimated_distance_ex - error_bound_ex;
```
16 changes: 8 additions & 8 deletions docs/docs/rabitq/quantizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ Insert the following before `return 0` in the indexing example above.

// See the Estimation guide for these query factors.
float c_1 = -static_cast<float>((1 << 1) - 1) / 2.F;
float k1xsumq = c_1 * std::accumulate(query.begin(), query.end(), 0.F);
float g_k1xsumq = c_1 * std::accumulate(query.begin(), query.end(), 0.F);
float g_add = rabitqlib::euclidean_sqr(query.data(), centroid.data(), dim);

float estimated_distance = rabitqlib::quant::full_est_dist(
Expand All @@ -206,7 +206,7 @@ Insert the following before `return 0` in the indexing example above.
f_add,
f_rescale,
g_add,
k1xsumq
g_k1xsumq
);
float gt_dist = rabitqlib::euclidean_sqr(query.data(), vector.data(), dim);

Expand Down Expand Up @@ -273,7 +273,7 @@ initialized `batch_data`, `centroid`, `dim`, and `batch_size`.

rabitqlib::BatchQuery<float> query_wrapper(query.data(), dim);

// G_add is the squared L2 distance between the query and the centroid.
// g_add is the squared L2 distance between the query and the centroid.
// This standalone example omits rotation, so both operands are raw vectors.
// In an index that applies rotation, rotate both with the same transformation.
query_wrapper.set_g_add(
Expand Down Expand Up @@ -400,8 +400,8 @@ Insert this before `return 0` in the indexing example, reusing its data and dime
auto ip_func = rabitqlib::select_excode_ipfunc(bits - 1);

// Compute more accurate distance based on full codes.
float est_dist_ex;
float low_dist_ex;
float estimated_distance_ex;
float lower_bound_ex;
float ip_x0_qr_ex;

rabitqlib::split_single_fulldist(
Expand All @@ -411,8 +411,8 @@ Insert this before `return 0` in the indexing example, reusing its data and dime
query_wrapper,
dim,
bits - 1,
est_dist_ex,
low_dist_ex,
estimated_distance_ex,
lower_bound_ex,
ip_x0_qr_ex,
distance_sqr,
distance_norm
Expand Down Expand Up @@ -512,7 +512,7 @@ Insert this before `return 0` in the indexing example, reusing its buffers and d
query.data(), dim, bits - 1, rabitqlib::METRIC_L2, true
);

// SplitBatchQuery expects the L2 norm and squares it internally for G_add.
// SplitBatchQuery expects the L2 norm and squares it internally for g_add.
// This example omits rotation, so query and centroid are both raw vectors.
float distance_norm =
std::sqrt(rabitqlib::euclidean_sqr(query.data(), centroid.data(), dim));
Expand Down
2 changes: 1 addition & 1 deletion include/rabitqlib/clustering/qgkmeans.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,7 @@ class QGAssigner {
hint,
query_sums.cached(i)
);
query_sums.store(i, scratch.batch_query.k1xsumq());
query_sums.store(i, scratch.batch_query.g_k1xsumq());
if (graph_->is_quantized()) {
// Keep objectives and returned distances relative to the original
// centroids; graph search scores are quantized estimates.
Expand Down
2 changes: 1 addition & 1 deletion include/rabitqlib/core/api.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ class QueryState {
size_t total_bits_ = 0;
MetricType metric_ = METRIC_L2;
float g_add_ = 0;
float k1xsumq_ = 0;
float g_k1xsumq_ = 0;
float (*extra_ip_)(const float*, const uint8_t*, size_t) = nullptr;

static float compute_checked(
Expand Down
8 changes: 4 additions & 4 deletions include/rabitqlib/core/encoder.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -295,10 +295,10 @@ inline void reconstruct_full_vec(
namespace detail {
template <typename T>
inline T full_est_dist_from_ip(
T ip, size_t bits, T f_add, T f_rescale, T g_add, T k1xsumq
T ip, size_t bits, T f_add, T f_rescale, T g_add, T g_k1xsumq
) {
return f_add + g_add +
(f_rescale * (ip + k1xsumq * static_cast<float>((1 << bits) - 1)));
(f_rescale * (ip + g_k1xsumq * static_cast<float>((1 << bits) - 1)));
}
} // namespace detail

Expand All @@ -312,10 +312,10 @@ inline T full_est_dist(
T f_add,
T f_rescale,
T g_add,
T k1xsumq
T g_k1xsumq
) {
return detail::full_est_dist_from_ip(
ip_func(query, quantized_vec, dim), bits, f_add, f_rescale, g_add, k1xsumq
ip_func(query, quantized_vec, dim), bits, f_add, f_rescale, g_add, g_k1xsumq
);
}
} // namespace rabitqlib::quant
16 changes: 8 additions & 8 deletions include/rabitqlib/core/estimator.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ inline float split_distance_boosting(
(cur_ex.f_rescale_ex() *
(static_cast<float>(1 << ex_bits) * ip_x0_qr +
ip_func(query_wrapper.rotated_query(), cur_ex.ex_code(), padded_dim) +
query_wrapper.kbxsumq()));
query_wrapper.g_kbxsumq()));

return ex_dist;
}
Expand Down Expand Up @@ -77,7 +77,7 @@ inline void split_single_fulldist_direct(
(cur_ex.f_rescale_ex() *
(static_cast<float>(1 << ex_bits) * ip_x0_qr +
ip_func(query_wrapper.rotated_query(), cur_ex.ex_code(), padded_dim) +
query_wrapper.kbxsumq()));
query_wrapper.g_kbxsumq()));

lower_bound = estimated_distance -
(cur_bin.f_error() * g_error / static_cast<float>(1 << ex_bits));
Expand Down Expand Up @@ -106,7 +106,7 @@ inline void split_single_fulldist(
(cur_ex.f_rescale_ex() *
(static_cast<float>(1 << ex_bits) * ip_x0_qr +
ip_func(query_wrapper.rotated_query(), cur_ex.ex_code(), padded_dim) +
query_wrapper.kbxsumq()));
query_wrapper.g_kbxsumq()));

lower_bound = estimated_distance -
(cur_bin.f_error() * g_error / static_cast<float>(1 << ex_bits));
Expand Down Expand Up @@ -134,7 +134,7 @@ inline void split_single_estdist(
);

estimated_distance = cur_bin.f_add() + g_add +
(cur_bin.f_rescale() * (ip_x0_qr + query_wrapper.k1xsumq()));
(cur_bin.f_rescale() * (ip_x0_qr + query_wrapper.g_k1xsumq()));

lower_bound = estimated_distance - (cur_bin.f_error() * g_error);
}
Expand Down Expand Up @@ -163,7 +163,7 @@ inline void split_single_estdist_direct(
);

estimated_distance = cur_bin.f_add() + g_add +
(cur_bin.f_rescale() * (ip_x0_qr + query_wrapper.k1xsumq()));
(cur_bin.f_rescale() * (ip_x0_qr + query_wrapper.g_k1xsumq()));

lower_bound = estimated_distance - (cur_bin.f_error() * g_error);
}
Expand All @@ -185,7 +185,7 @@ inline void xy_single_base_dist(
ip_base = base_ip_func(query_wrapper.rotated_query(), cur_base.base_code(), padded_dim);

estimated_distance = cur_base.f_add() + g_add +
(cur_base.f_rescale() * (ip_base + query_wrapper.kbase_sumq()));
(cur_base.f_rescale() * (ip_base + query_wrapper.g_kbase_sumq()));

lower_bound = estimated_distance - (cur_base.f_error() * g_error);
}
Expand All @@ -211,7 +211,7 @@ inline void xy_single_full_dist(
if (ex_bits == 0) {
estimated_distance =
cur_base.f_add() + g_add +
(cur_base.f_rescale() * (ip_base + query_wrapper.kbase_sumq()));
(cur_base.f_rescale() * (ip_base + query_wrapper.g_kbase_sumq()));
lower_bound = estimated_distance - (cur_base.f_error() * g_error);
return;
}
Expand All @@ -222,7 +222,7 @@ inline void xy_single_full_dist(
ex_ip_func(query_wrapper.rotated_query(), cur_ex.ex_code(), padded_dim);

estimated_distance = cur_ex.f_add_ex() + g_add +
(cur_ex.f_rescale_ex() * (ip + query_wrapper.kbxsumq()));
(cur_ex.f_rescale_ex() * (ip + query_wrapper.g_kbxsumq()));

lower_bound = estimated_distance -
(cur_base.f_error() * g_error / static_cast<float>(1 << ex_bits));
Expand Down
8 changes: 4 additions & 4 deletions include/rabitqlib/core/query.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,12 @@ class SplitSingleQuery {

[[nodiscard]] T vl() const { return vl_; }

[[nodiscard]] T k1xsumq() const { return G_k1xSumq_; }
[[nodiscard]] T g_k1xsumq() const { return G_k1xSumq_; }

// Equals k1xsumq() when base_bits == 1.
[[nodiscard]] T kbase_sumq() const { return G_kbaseSumq_; }
// Equals g_k1xsumq() when base_bits == 1.
[[nodiscard]] T g_kbase_sumq() const { return G_kbaseSumq_; }

[[nodiscard]] T kbxsumq() const { return G_kbxSumq_; }
[[nodiscard]] T g_kbxsumq() const { return G_kbxSumq_; }

[[nodiscard]] T g_add() const { return G_add_; }

Expand Down
Loading
Loading