Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,30 @@ reranking. QG uses `quantization_bits()` with 0 for raw storage and 4/8 for pack
vectors. Naming work does not change these values, allocation sizes, numeric
operations, serialized field order, or ownership.

## Numerical module variable names

Apply the same role-based naming to Core, FastScan, clustering and SIMD. Preserve
public and protected names, Python keywords, field order and numerical operations.

- Use `query_wrapper` for prepared query objects. Retain the existing `G_*`
private correction-field names where they identify mathematical coefficients.
- Distinguish a scalar `estimated_distance` or `lower_bound` from batch outputs
`estimated_distances` and `lower_bounds`. Keep `ip_x0_qr` where it identifies the
mathematical intermediate used by refinement.
- Use `lookup_table` for FastScan table pointers and `accumulated_values` for
integer accumulation outputs. Retain `lut` in compact local kernels where its
role is clear.
- Clustering's private state uses `dim_`, `num_points_`, `num_clusters_`,
`num_workers_` and `data_`. Keep public `d`, `k`, `num_threads` and graph parameter
names compatible. Retain the existing private `niter_` graph iteration name.
- FHT backend arguments use `rotated_data`, `padded_dim`, `trunc_dim` and
`scale_factor` consistently across declarations, dispatch and implementations.
A lower-level kernel's `dim` still denotes that kernel's own input shape.

Keep short mathematical symbols, loop indices and SIMD register names when their
meaning is local and clear. Naming cleanup must not alter formulas, arithmetic
order, packed layouts, ISA selection or runtime behavior.

## Static analysis

clang-tidy performs semantic checks and is kept separate from clang-format.
Expand Down
7 changes: 5 additions & 2 deletions docs/docs/index/hnsw.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,10 +282,13 @@ Each indexed element is stored in the following layout:
[edges]
[cluster ID] (its high bit marks a removed point, see Removing points)
[external label]
[BinData (1-bit * dim + factors)]
[ExData (ex-bits * dim + factors)]
[BinData (1-bit * padded_dim + factors)]
[ExData (ex-bits * padded_dim + factors)]
```

Here `padded_dim` is the stored rotated dimension, which may exceed the original
input `dim`. Loading preserves the index's saved padded dimension.

### Search algorithm

We first pre-process the query:
Expand Down
8 changes: 4 additions & 4 deletions docs/docs/index/ivf.md
Original file line number Diff line number Diff line change
Expand Up @@ -415,10 +415,10 @@ Extra-bit layouts retain full 64-coordinate blocks and store any final
IVF-RaBitQ stores the following data:

```c++
[batch data] // 1-bit code and factors
[ex_data] // code for remaining bits, or original float32 vectors
[ids] // PID of vectors (organized by clusters)
[cluster_lst] // List of clusters' metadata in IVF-RaBitQ
[batch data] // 1-bit code and factors
[ex_data] // code for remaining bits, or original float32 vectors
[ids] // PID of vectors (organized by clusters)
[cluster metadata] // List of clusters' metadata in IVF-RaBitQ
```
In raw mode, `ex_data` holds `4 * num_points * dim` bytes in cluster order,
without padding or extra-bit factors. The one-bit filtering codes and factors
Expand Down
12 changes: 6 additions & 6 deletions docs/docs/rabitq/estimator.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,15 +151,15 @@ when $B=1$.

// Compute the estimated distance
// Note that G_add is dependent on the centroid.
float est_dist = F_add + G_add + F_rescale * (ip + G_kBxSumq);
float estimated_distance = F_add + G_add + F_rescale * (ip + G_kBxSumq);

// Compute the error bound
// Note that G_error is dependent on the centroid.
float error_bound = F_error * G_error;

// Compute the lower and upper bounds of the estimated distance
float lb_dist = est_dist - error_bound;
float ub_dist = est_dist + error_bound;
float lb_dist = estimated_distance - error_bound;
float ub_dist = estimated_distance + error_bound;
```

### Incremental distance estimation
Expand Down Expand Up @@ -187,10 +187,10 @@ that additional error.

```cpp
// 1-bit dist
float est_dist = F_add + G_add + F_rescale * (ip_bin + G_k1xSumq);
float estimated_distance = F_add + G_add + F_rescale * (ip_bin + G_k1xSumq);
float bound = F_error * G_error;
float ub_dist = est_dist + bound;
float lb_dist = est_dist - bound;
float ub_dist = estimated_distance + bound;
float lb_dist = estimated_distance - bound;

// boost to full-bit dist
float ex_scale = static_cast<float>(1 << (bits - 1));
Expand Down
44 changes: 22 additions & 22 deletions docs/docs/rabitq/quantizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ Insert the following before `return 0` in the indexing example above.
float k1xsumq = c_1 * std::accumulate(query.begin(), query.end(), 0.F);
float g_add = rabitqlib::euclidean_sqr(query.data(), centroid.data(), dim);

float est_dist = rabitqlib::quant::full_est_dist(
float estimated_distance = rabitqlib::quant::full_est_dist(
code.data(),
query.data(),
rabitqlib::excode_ipimpl::ip_fxi,
Expand All @@ -200,7 +200,7 @@ Insert the following before `return 0` in the indexing example above.
);
float gt_dist = rabitqlib::euclidean_sqr(query.data(), vector.data(), dim);

std::cout << "Estimated distance: " << est_dist << '\n';
std::cout << "Estimated distance: " << estimated_distance << '\n';
std::cout << "GT distance: " << gt_dist << '\n';

```
Expand Down Expand Up @@ -261,22 +261,22 @@ initialized `batch_data`, `centroid`, `dim`, and `batch_size`.

std::vector<float> query(dim, 1.0F);

rabitqlib::BatchQuery<float> processed_query(query.data(), dim);
rabitqlib::BatchQuery<float> query_wrapper(query.data(), dim);

// G_add is the squared L2 distance between the query and the centroid.
// This standalone example omits rotation, so both operands are raw vectors.
// In an index that applies rotation, rotate both with the same transformation.
processed_query.set_g_add(
query_wrapper.set_g_add(
rabitqlib::euclidean_sqr(query.data(), centroid.data(), dim)
);

std::vector<float> est_distance(batch_size); // store the estimated distances
std::vector<float> estimated_distances(batch_size); // store the estimated distances

// QG scoring returns estimates only. Use the IVF-RaBitQ batch format below
// when error intervals are needed.

rabitqlib::qg_batch_estdist(
batch_data.data(), processed_query, dim, est_distance.data()
batch_data.data(), query_wrapper, dim, estimated_distances.data()
);

```
Expand Down Expand Up @@ -361,7 +361,7 @@ Insert this before `return 0` in the indexing example, reusing its data and dime
dim, rabitqlib::SplitSingleQuery<float>::kNumBits
);

rabitqlib::SplitSingleQuery<float> processed_query(
rabitqlib::SplitSingleQuery<float> query_wrapper(
query.data(), dim, bits - 1, query_config, rabitqlib::METRIC_L2
);

Expand All @@ -372,16 +372,16 @@ Insert this before `return 0` in the indexing example, reusing its data and dime

// Compute estimated distances based on binary codes
float ip_x0_qr;
float est_dist;
float low_dist;
float estimated_distance;
float lower_bound;

rabitqlib::split_single_estdist(
bin_data.data(),
processed_query,
query_wrapper,
dim,
ip_x0_qr,
est_dist,
low_dist,
estimated_distance,
lower_bound,
distance_sqr,
distance_norm
);
Expand All @@ -398,7 +398,7 @@ Insert this before `return 0` in the indexing example, reusing its data and dime
bin_data.data(),
ex_data.data(),
ip_func,
processed_query,
query_wrapper,
dim,
bits - 1,
est_dist_ex,
Expand Down Expand Up @@ -498,29 +498,29 @@ Insert this before `return 0` in the indexing example, reusing its buffers and d
// IVF-RaBitQ instead selects HACC for 4-9 total bits and standard FastScan for
// 1-3 total bits or raw-vector storage, unless the caller overrides it.

rabitqlib::SplitBatchQuery<float> processed_query(
rabitqlib::SplitBatchQuery<float> query_wrapper(
query.data(), dim, bits - 1, rabitqlib::METRIC_L2, true
);

// SplitBatchQuery expects the L2 norm and squares it internally for G_add.
// This example omits rotation, so query and centroid are both raw vectors.
float distance_norm =
std::sqrt(rabitqlib::euclidean_sqr(query.data(), centroid.data(), dim));
processed_query.set_g_add(distance_norm);
query_wrapper.set_g_add(distance_norm);

std::vector<float> est_distance(batch_size); // store the estimated distances
std::vector<float> low_distance(batch_size); // store the lower bound
std::vector<float> estimated_distances(batch_size); // store the estimated distances
std::vector<float> lower_bounds(batch_size); // store the lower bound
std::vector<float> ip_x0_qr(batch_size
); // store the intermediate result of inner product

// For estimates without error intervals, see the QG batch format above.

rabitqlib::split_batch_estdist(
batch_data.data(),
processed_query,
query_wrapper,
dim,
est_distance.data(),
low_distance.data(),
estimated_distances.data(),
lower_bounds.data(),
ip_x0_qr.data(),
true
);
Expand All @@ -530,10 +530,10 @@ Insert this before `return 0` in the indexing example, reusing its buffers and d

size_t i = 15;
// Replace this candidate's one-bit estimate with the refined distance.
est_distance[i] = rabitqlib::split_distance_boosting(
estimated_distances[i] = rabitqlib::split_distance_boosting(
ex_data.data() + (i * rabitqlib::ExDataMap<float>::data_bytes(dim, bits - 1)),
ip_func,
processed_query,
query_wrapper,
dim,
bits - 1,
ip_x0_qr[i]
Expand Down
44 changes: 22 additions & 22 deletions include/rabitqlib/clustering/detail/kmeans.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -81,12 +81,12 @@ inline void exact_assign(
: 64;
constexpr size_t kCentroidBlock = 256;
const size_t blocks = 1 + (n - 1) / point_block;
const auto threads = static_cast<uint32_t>(std::min<size_t>(num_threads, blocks));
const auto num_workers = static_cast<uint32_t>(std::min<size_t>(num_threads, blocks));
std::vector<double> centroid_norms(k);
std::vector<std::vector<float>> products(
threads, std::vector<float>(point_block * std::min(k, kCentroidBlock))
num_workers, std::vector<float>(point_block * std::min(k, kCentroidBlock))
);
std::vector<std::exception_ptr> errors(threads);
std::vector<std::exception_ptr> errors(num_workers);

const auto exact_distance = [d, spherical](const float* vector, const float* centroid) {
double distance = 0.0;
Expand All @@ -112,7 +112,7 @@ inline void exact_assign(
const double error_floor =
8.0 * static_cast<double>(d) * std::numeric_limits<float>::min() +
8.0 * std::numeric_limits<double>::epsilon();
#pragma omp parallel num_threads(threads)
#pragma omp parallel num_threads(num_workers)
{
// Set only this OpenMP task's ICV, not Eigen's process-global thread limit.
// This also prevents nested GEMM teams when the outer team has one worker.
Expand Down Expand Up @@ -244,8 +244,8 @@ class LloydKMeans : public Parameters {
std::string(Assignment::kName) + " input dimensions overflow size_t"
);
}
const uint32_t threads = effective_num_threads();
static_cast<void>(threads);
const uint32_t num_workers = effective_num_threads();
static_cast<void>(num_workers);
// Leave headroom for products of squared distances in graph pruning,
// as well as quantized reconstruction and rotated residuals.
const double max_coordinate = std::sqrt(
Expand All @@ -254,7 +254,7 @@ class LloydKMeans : public Parameters {
);
bool in_range = true;
bool finite = true;
#pragma omp parallel for num_threads(threads) reduction(&& : finite, in_range) \
#pragma omp parallel for num_threads(num_workers) reduction(&& : finite, in_range) \
schedule(static)
for (std::ptrdiff_t loop_index = 0; loop_index < static_cast<std::ptrdiff_t>(n * d);
++loop_index) {
Expand Down Expand Up @@ -289,7 +289,7 @@ class LloydKMeans : public Parameters {
x = input_copy.data();
}

Assignment assigner(d, k, x, n, threads, *this);
Assignment assigner(d, k, x, n, num_workers, *this);

centroids.resize(k * d);
assignments.resize(n);
Expand All @@ -304,7 +304,7 @@ class LloydKMeans : public Parameters {
std::vector<float> previous_centroids(centroids.size());
std::vector<size_t> cluster_sizes(k);
std::vector<double> sums(centroids.size());
const size_t update_workers = std::min(k, static_cast<size_t>(threads));
const size_t update_workers = std::min(k, static_cast<size_t>(num_workers));
std::vector<UpdatePartition> update_partitions(update_workers);
std::vector<size_t> update_worker_for_cluster(k);
for (size_t worker = 0; worker < update_workers; ++worker) {
Expand Down Expand Up @@ -360,7 +360,7 @@ class LloydKMeans : public Parameters {
d,
k,
spherical,
threads,
num_workers,
assignments.data(),
distances.data(),
assignments.data()
Expand Down Expand Up @@ -447,9 +447,9 @@ class LloydKMeans : public Parameters {
for (size_t point = 0; point < n; ++point) {
partitions[worker_for_cluster[assignments[point]]].points.push_back(point);
}
const uint32_t threads = effective_num_threads();
static_cast<void>(threads);
#pragma omp parallel for num_threads(threads) schedule(static)
const uint32_t num_workers = effective_num_threads();
static_cast<void>(num_workers);
#pragma omp parallel for num_threads(num_workers) schedule(static)
for (std::ptrdiff_t loop_worker = 0;
loop_worker < static_cast<std::ptrdiff_t>(partitions.size());
++loop_worker) {
Expand Down Expand Up @@ -485,8 +485,8 @@ class LloydKMeans : public Parameters {
std::vector<size_t>& cluster_sizes,
const std::vector<float>& previous_centroids
) {
const uint32_t threads = effective_num_threads();
static_cast<void>(threads);
const uint32_t num_workers = effective_num_threads();
static_cast<void>(num_workers);
// Means, normalization, and displacement share one worker team. Preserve
// the per-centroid arithmetic; empty rows are measured after refilling.
const auto cluster_shift = [&](size_t cluster) {
Expand All @@ -500,7 +500,7 @@ class LloydKMeans : public Parameters {
return result;
};
double shift = 0;
#pragma omp parallel for num_threads(threads) reduction(+ : shift)
#pragma omp parallel for num_threads(num_workers) reduction(+ : shift)
for (std::ptrdiff_t loop_index = 0; loop_index < static_cast<std::ptrdiff_t>(k);
++loop_index) {
const size_t cluster = static_cast<size_t>(loop_index);
Expand Down Expand Up @@ -548,8 +548,8 @@ class LloydKMeans : public Parameters {
};
if (nsplit == 1) {
// A single empty centroid needs only one eligible maximum per worker.
std::vector<size_t> best_points(threads, n);
#pragma omp parallel num_threads(threads)
std::vector<size_t> best_points(num_workers, n);
#pragma omp parallel num_threads(num_workers)
{
size_t best = n;
#pragma omp for schedule(static)
Expand Down Expand Up @@ -584,7 +584,7 @@ class LloydKMeans : public Parameters {
return {1, shift};
}

#pragma omp parallel for num_threads(threads) schedule(static)
#pragma omp parallel for num_threads(num_workers) schedule(static)
for (std::ptrdiff_t loop_index = 0; loop_index < static_cast<std::ptrdiff_t>(n);
++loop_index) {
const size_t point = static_cast<size_t>(loop_index);
Expand Down Expand Up @@ -624,9 +624,9 @@ class LloydKMeans : public Parameters {
}

void normalize_centroids() {
const uint32_t threads = effective_num_threads();
static_cast<void>(threads);
#pragma omp parallel for num_threads(threads)
const uint32_t num_workers = effective_num_threads();
static_cast<void>(num_workers);
#pragma omp parallel for num_threads(num_workers)
for (std::ptrdiff_t loop_index = 0; loop_index < static_cast<std::ptrdiff_t>(k);
++loop_index) {
const size_t cluster = static_cast<size_t>(loop_index);
Expand Down
Loading
Loading