Skip to content

fix the slow refinement Add() for sparse (post impr 1856) - #1902

Merged
sre-ci-robot merged 1 commit into
zilliztech:mainfrom
alexanderguzhva:fix_slow_refinement_add
Oct 20, 2026
Merged

sre-ci-robot merged 1 commit into
zilliztech:mainfrom
alexanderguzhva:fix_slow_refinement_add

Conversation

@alexanderguzhva

Copy link
Copy Markdown
Collaborator

Fixes the introduced problem in #1856, mentioned in the #1864 (comment) comment. It is all about slow incremental Add() calls.

The measured single-row Add averages dropped from 3.38 to 0.095 ms at 65K synthetic rows, and from 20.18 to 0.152 ms for the 65K-row MSMARCO prefix. The tradeoff is extra spare capacity in the growing seek vectors.

In the measured MSMARCO cases, this increased total reported index memory by 2.6% for single-row Adds and 7.4% for 32-row Adds. Posting IDs and values retain their existing representation; the extra memory is in refinement metadata capacity.

Generated using Codex and edited.

/kind improvement

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
@alexanderguzhva

Copy link
Copy Markdown
Collaborator Author

@chasingegg please take a look

@mergify mergify Bot added the dco-passed label Oct 19, 2026
@alexanderguzhva alexanderguzhva changed the title fix the slow refinement Add() for sparse fix the slow refinement Add() for sparse (post impr 1856) Oct 19, 2026
@mergify mergify Bot added the ci-passed label Oct 19, 2026

@chasingegg chasingegg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the fix!

/lgtm

@sre-ci-robot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alexanderguzhva, chasingegg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [alexanderguzhva,chasingegg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot
sre-ci-robot merged commit 54ba270 into zilliztech:main Oct 20, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants