feat: add CLI control for Milvus force-merge size and toggle by jamesgao-jpg · Pull Request #871 · zilliztech/VectorDBBench

jamesgao-jpg · 2026-09-15T07:47:00Z

What

Adds CLI control of the Milvus force-merge compaction in optimize(),
addressing #825 (the unconditional force merge is costly and
non-reproducible at 100M scale).

Two new options on every Milvus CLI command and on ZillizAutoIndex:

--force-merge-enabled/--no-force-merge-enabled — skip the force-merge
stage for a sooner ready-to-search state (segments are then not merged to
their fullest potential).
--force-merge-target-size-mb — request a bounded merged segment size
(e.g. 1024 MB); the effective cap depends on the Milvus server. Default
keeps the current unbounded single-segment behavior
(((1 << 63) - 1) // (1024**2) MB).

Both are also exposed in the web UI case configs for Milvus and Zilliz
Cloud (Load / Performance / FTS).

Behavior

_optimize() keeps flush → normal compaction → index-wait → refresh.
When force merge is disabled, only the force-merge stage is skipped.
_force_merge() resolves the target size from the case config; None
falls back to today's constant, so existing runs are byte-for-byte
unchanged.
GPU index types keep skipping force merge (unchanged).
force_merge_target_size_mb is validated as a positive integer or None.

Compatibility

Backward compatible: defaults (enabled=True, target=None) reproduce
current behavior; no result-schema or metric changes (optimize is
pre-search preparation, not a measured result).
Zilliz Cloud inherits the Milvus client and config base classes, so the
same knobs apply there too.

Testing

Level 1 mock-only unit tests: tests/test_milvus.py +
tests/test_milvus_zilliz_cli.py, 31 passed / 1 integration deselected,
including new tests for the configured target size, the disabled path,
config validation, and CLI flag mapping (Milvus / FTS / Zilliz Cloud).
No live-backend probe yet: whether a bounded target_size is honored as
a hard segment-size cap by the target Milvus server still needs a small
functional probe before benchmark readiness can be declared.

Limitations / open items

Functional probe for bounded target-size semantics on a live Milvus.
[WIP] — review welcome before finalizing defaults and docs.

Follow-up fix (`84beccd`)

`copy_fts_compatible_db_case_fields()` now also preserves
`force_merge_target_size_mb` when a non-FTS vector config is routed to an
FTS config class, so `--force-merge-target-size-mb` survives a
`--case-type FTSBm25Performance` invocation.
Added `test_cli_preserves_milvus_force_merge_options_when_routing_vector_config_to_fts`.

Expose the Milvus force-merge compaction during optimize as tunable CLI options, addressing zilliztech#825 (unconditional force merge is costly and non-reproducible at 100M scale): - --force-merge-enabled/--no-force-merge-enabled: skip the force-merge stage for a sooner ready-to-search state at the cost of segments not merged to their fullest potential. - --force-merge-target-size-mb: bound the merged segment size (e.g. 1024 for bounded, reproducible segments); defaults to the current unbounded single-segment behavior for backward compatibility. Applied to all Milvus index CLI commands and to Zilliz Cloud AutoIndex, and registered in the web UI case configs. GPU index types keep skipping force merge. Disabling force merge keeps flush, normal compaction, index-wait, and refresh-load. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>

sre-ci-robot · 2026-09-15T07:47:05Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jamesgao-jpg
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

copy_fts_compatible_db_case_fields() now also carries force_merge_target_size_mb when a non-FTS vector config is routed to an FTS config class, so --force-merge-target-size-mb survives a --case-type FTSBm25Performance invocation (force_merge_enabled was already whitelisted). Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>

All post-construction transformations of case configs now flow through finalize_db_case_config() in vectordb_bench/backend/db_case_config.py: - CLI-only FTS routing (previously select_cli_db_case_config in cli/cli.py) and the dataset-derived metric_type default (previously applied by mutating task.db_case_config in assembler.py) are composed in one resolver. - Both task entry points call it: cli.run() (with the CLI parameters dict) and the web UI generate_tasks() (without). - The assembler no longer mutates the config; consumers treat the finalized config as read-only. - select_cli_db_case_config remains as a thin backward-compatible wrapper for existing callers and tests. No behavior change: FTS-routing semantics, the metric_type default rule, and both entry-point results are preserved (covered by the new resolver tests plus the existing FTS-routing tests). Note: this refactor moves the FTS-compatible field whitelist (including force_merge_target_size_mb) into the resolver, so it subsumes the one-line whitelist fix carried by PR zilliztech#871's cli/cli.py. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>

jamesgao-jpg

Reviewed the effective diff at 84beccd. The force-merge toggle, target propagation, frontend wiring, GPU behavior, and FTS routing otherwise look correct. Focused Milvus/CLI/FTS tests passed (38 passed), and all four current CI jobs are green. Two findings remain below: regular Milvus CLI commands bypass the positive-size validator, and the public target-size wording overstates the backend's guarantees. No live Milvus/Zilliz functional probe was run.

jamesgao-jpg · 2026-09-16T03:03:38Z

-def _with_partition_key(db_case_config: BaseModel, parameters: dict) -> BaseModel:
-    return db_case_config.model_copy(update={"use_partition_key": _use_partition_key(parameters)})
+def _apply_milvus_case_defaults(db_case_config: BaseModel, parameters: dict) -> BaseModel:
+    return db_case_config.model_copy(


[P2] Validate the copied target size

model_copy(update=...) does not run Pydantic validators, so the validator in MilvusIndexConfig is bypassed on every regular Milvus command. VERIFIED: both --force-merge-target-size-mb 0 and -5 exit successfully and produce configs containing those values; PyMilvus rejects them only later in compact(), after a benchmark may already have loaded its dataset. Please use a CLI IntRange(min=1) or rebuild/revalidate the model, and add zero/negative CLI regressions.

jamesgao-jpg · 2026-09-16T03:03:38Z

+            help=(
+                "Target merged segment size in MB for the force-merge compaction during "
+                "optimize. Defaults to the current unbounded single-segment behavior; set "
+                "e.g. 1024 for bounded, reproducible segments. Must be a positive integer."


[P2] Describe this as a nominal target, not a reproducibility guarantee

The public wording is stronger than the backend contract. PyMilvus 2.6.16 describes its maximum-size sentinel as letting the server choose the segment size automatically, while the Milvus force-merge docs describe explicit values as approximate and safety-clamped by topology/memory. The PR also notes that no live probe established a hard cap. Since segment layout affects benchmark latency, please replace unbounded single-segment / bounded, reproducible in the CLI, UI, and PR description with server-selected automatic target / requested or nominal target, plus the backend constraints.

Addresses two review findings on the force-merge CLI feature: - Regular Milvus/Zilliz CLI commands applied the flag through model_copy(update=...) in the per-command defaults helper, which skips pydantic validation, so non-positive values bypassed the field validator. Add a click callback (_validate_positive_int_or_none) on the option that rejects non-positive integers at the CLI boundary; direct config construction stays guarded by the pydantic validator. - Soften the public wording: the server-side behavior of target_size is not probed, so describe it as a requested cap whose effective behavior depends on the Milvus server instead of claiming bounded, reproducible segments. Adds CLI rejection tests for 0 and negative values on Milvus and Zilliz AutoIndex commands. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>

jamesgao-jpg · 2026-09-16T07:25:38Z

Both findings addressed in 72b1219 (pushed).

1. CLI bypass of the positive-size validator — fixed.

Confirmed the mechanism from pydantic source: BaseModel.model_copy(update=...) documents "the data is not validated before creating the new model" (pydantic/main.py:405). Since _apply_milvus_case_defaults applies the flag via model_copy, --force-merge-target-size-mb 0 (or negative) reached the config without ever hitting the field_validator on the 17 index commands (the FTS command constructs directly and was already guarded).

Fix: a click callback _validate_positive_int_or_none on the option in both milvus/cli.py and zilliz_cloud/cli.py rejects non-positive values at the CLI boundary (click.BadParameter), while direct config construction remains guarded by the pydantic validator. Added test_milvus_autoindex_cli_rejects_non_positive_force_merge_target_size and test_zilliz_autoindex_cli_rejects_non_positive_force_merge_target_size (0 and -5 → non-zero exit + message).

2. Public wording overstates backend guarantees — fixed.

The CLI help and web-UI inputHelp now describe the value as a requested cap whose effective behavior depends on the Milvus server (no claim of guaranteed bounded/reproducible segments until the functional probe is run).

Verification: focused Milvus/CLI/FTS tests on the remote worktree at 72b1219 → 39 passed, 1 integration deselected. CI on the updated head will re-run; the live Milvus/Zilliz functional probe remains the open item before benchmark readiness (tracked in the PR description).

…ge-cli Resolve conflicts with PR zilliztech#876 (feat: add search level to Milvus AutoIndex): - vectordb_bench/backend/clients/milvus/config.py: combine imports — keep Field from zilliztech#876 and field_validator from zilliztech#871. - vectordb_bench/backend/clients/milvus/cli.py: MilvusAutoIndex now uses _apply_milvus_case_defaults(AutoIndexConfig(level=parameters["level"]), parameters) so both the search level and the force-merge defaults apply. - tests/test_milvus_zilliz_cli.py: keep both the force-merge tests and the search-level tests. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>

_validate_positive_int_or_none takes the required (ctx, param) callback arguments but only uses value; underscore-prefix them so the PR's lint check (ruff ARG001) passes. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>

…ge-cli Resolve conflicts with PR zilliztech#874 (feat: support configurable NQ for concurrent Milvus searches): - tests/test_milvus_zilliz_cli.py: keep the force-merge tests and the search-level tests alongside the new NQ test. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>

jamesgao-jpg changed the title ~~[WIP] feat: add CLI control for Milvus force-merge size and toggle~~ feat: add CLI control for Milvus force-merge size and toggle Sep 15, 2026

jamesgao-jpg mentioned this pull request Sep 15, 2026

[Question/Enhancement] Unconditional force-merge compaction in Milvus client _optimize() may be too costly and non-reproducible at 100M scale (LAION-100M) #825

Open

jamesgao-jpg mentioned this pull request Sep 15, 2026

[WIP] refactor: finalize db_case_config at a single choke point #872

Open

jamesgao-jpg commented Sep 16, 2026

View reviewed changes

jamesgao-jpg mentioned this pull request Sep 16, 2026

Tracking: ongoing refactors — case-config finalization and hardening #877

Open

4 tasks

jamesgao-jpg added 3 commits September 21, 2026 17:19

fix: silence ruff ARG001 for unused click callback args

e9522fd

_validate_positive_int_or_none takes the required (ctx, param) callback arguments but only uses value; underscore-prefix them so the PR's lint check (ruff ARG001) passes. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>

Navigation Menu

Repository navigation

feat: add CLI control for Milvus force-merge size and toggle - #871

feat: add CLI control for Milvus force-merge size and toggle#871
jamesgao-jpg wants to merge 6 commits into
zilliztech:mainfrom
jamesgao-jpg:feat/milvus-force-merge-cli

jamesgao-jpg commented Sep 15, 2026 •

edited

Loading

Uh oh!

sre-ci-robot commented Sep 15, 2026

Uh oh!

jamesgao-jpg left a comment

Uh oh!

jamesgao-jpg Sep 16, 2026

Uh oh!

jamesgao-jpg Sep 16, 2026

Uh oh!

jamesgao-jpg commented Sep 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jamesgao-jpg commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Behavior

Compatibility

Testing

Limitations / open items

Follow-up fix (84beccd)

Uh oh!

sre-ci-robot commented Sep 15, 2026

Uh oh!

jamesgao-jpg left a comment

Choose a reason for hiding this comment

Uh oh!

jamesgao-jpg Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

jamesgao-jpg Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

jamesgao-jpg commented Sep 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jamesgao-jpg commented Sep 15, 2026 •

edited

Loading

Follow-up fix (`84beccd`)