Skip to content

Bound the noncurrent and orphan-delete-marker listing cursors - #2696

Draft
delthas wants to merge 2 commits into
development/8.5from
improvement/ARSN-629/bound-lifecycle-listing-cursors
Draft

delthas wants to merge 2 commits into
development/8.5from
improvement/ARSN-629/bound-lifecycle-listing-cursors

Conversation

@delthas

@delthas delthas commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

DelimiterNonCurrent and DelimiterOrphanDeleteMarker stop themselves once filter() has seen maxScannedLifecycleListingEntries entries, but set no limit on the backend cursor. Everything the cursor returns past that point is read and thrown away. On a bucket whose pages return no results the budget is spent in full on every page, and behind mongos the shard cursor is opened with no batchSize and fills a 16MB batch each time: ~13,300 documents read per page against ~10,000 consumed.

Rebased onto development/8.5 now that #2685 (ARSN-620) has merged. A bound makes pages truncate on essentially every page of an orphan sweep, and truncation is only safe once the resume marker reliably advances, which is what ARSN-620 provides.

The change

_cursorLimit() in DelimiterVersions, returning maxScannedLifecycleListingEntries + 1, applied in genMDParamsV0() and to both param sets in genMDParamsV1() — the two ranges are served by separate cursors and either one can supply every entry the listing consumes, so each carries the full bound rather than a share of it. Placed in the base class so both listings inherit it and the reasoning lives in one place.

No v0 doubling, and this matters. filter() counts every entry the cursor delivers, masters and versions alike, in both formats. DelimiterCurrent multiplies by 2 on v0 only because its counter lives in addContents(), which v0 version keys never reach — copying that factor here would double the bound for nothing. Its comment currently claims the counter "maps directly to cursor documents regardless of bucket format", which the ×2 block below it contradicts; corrected as a drive-by so the next person doesn't copy it.

Getting the bound wrong is not a perf regression. A cursor that ends early is indistinguishable from the end of the keyspace, and DelimiterOrphanDeleteMarker.result() flushes its held candidate whenever the scan budget was not reached — so an under-sized bound reports a delete marker as orphaned while unseen versions sit behind the cursor, and lifecycle then deletes a marker that resurrects an old version.

On the expected benefit

~25% fewer documents read per page (13,342 → 10,001 at the default budget), not the 13x ARSN-629 currently claims in its summary. The only safe bound is the scan budget, not MaxKeys: nothing ties raw entries consumed to MaxKeys for these two listings — 10,000 single-version keys return zero results while burning the whole budget. The ticket's 13x was computed against MaxKeys + 1 = 1,001, which is not a bound these listings can honour. batchSize is the complementary lever and buys the same ~25% on this workload, since the dominant orphan-sweep pages consume the full budget; noted for a separate ticket. The ticket text still needs correcting.

Pre-existing bugs surfaced, not fixed here

Confirmed by running the full budget matrix with _cursorLimit() stashed — identical results, so neither is caused by this change. Re-verified after rebasing onto the merged ARSN-620; the matrix is unchanged. Both are documented in the spec where the sweep skips them:

  1. DelimiterNonCurrent cannot advance at budgets ≤ 4 on this fixture: resuming re-reads the marker version to recover the stale date, so the budget is consumed by skipped entries plus that re-read and NextVersionIdMarker never moves. The listing pages forever. (Same floor @nicolas2bert described in ARSN-620 Fix lifecycle listings stuck on PHD master keys #2685 when dropping the first-PHD fallback.)
  2. DelimiterOrphanDeleteMarker silently loses the last orphan when the keyspace ends on the exact entry the budget ends. filter() never sees the entry that would raise FILTER_END, so IsTruncated stays false while _isMaxScannedEntriesReached() is already true — result() then suppresses the held candidate and reports the listing complete. The orphan is dropped permanently, with no truncation to recover it. Same failure shape as the hazard this PR guards against, reached through the scan budget instead of the cursor. Worth its own ticket.

Notes for review

  • First commit is the prettier reformat of delimiterCurrent.ts, the only touched file still prettier-dirty at the base; ARSN-620 already formatted the others. The change commit is prettier-clean.
  • No version bump yet. ARSN-629's Jira fixVersion is 8.5.18 and the base is 8.5.17, so bert-e will want a bump commit — say the word and I will add one.
  • Unrelated: a stray .claude/worktrees/yarn-berry-poc git worktree is checked out inside the repo locally. It makes jest collect every suite twice and fail on duplicate TS declarations and mongo port collisions; test runs here excluded it.

Issue: ARSN-629

@codecov

codecov Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 74.71%. Comparing base (ecb7d87) to head (bcfba6b).

Files with missing lines Patch % Lines
lib/algos/list/delimiterCurrent.ts 83.33% 1 Missing ⚠️
Additional details and impacted files
@@                 Coverage Diff                 @@
##           development/8.5    #2696      +/-   ##
===================================================
+ Coverage            74.67%   74.71%   +0.03%     
===================================================
  Files                  227      227              
  Lines                18617    18629      +12     
  Branches              3847     3851       +4     
===================================================
+ Hits                 13903    13918      +15     
+ Misses                4709     4706       -3     
  Partials                 5        5              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nicolas2bert
nicolas2bert force-pushed the bugfix/ARSN-620/lc-phd branch 2 times, most recently from 5c216bb to c14bc52 Compare September 14, 2026 10:16
Base automatically changed from bugfix/ARSN-620/lc-phd to development/8.5 September 14, 2026 13:35
@bert-e

bert-e commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Hello delthas,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval ⭐
/bypass_build_status Bypass the build and test status ⭐
/bypass_commit_size Bypass the check on the size of the changeset TBA ⭐
/bypass_incompatible_branch Bypass the check on the source branch prefix ⭐
/bypass_jira_check Bypass the Jira issue check ⭐
/bypass_peer_approval Bypass the pull request peers' approval ⭐
/bypass_leader_approval Bypass the pull request leaders' approval ⭐
/bypass_source_branch_lineage Bypass the cross-branch contamination check ⭐
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request.
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@scality scality deleted a comment from bert-e Sep 14, 2026
DelimiterNonCurrent and DelimiterOrphanDeleteMarker stop themselves once
filter() has seen maxScannedLifecycleListingEntries entries, but set no
limit on the backend cursor. Everything the cursor returns past that
point is read and thrown away. On a bucket whose pages return no results
the budget is spent in full on every page, and behind mongos the shard
cursor is opened with no batchSize and fills a 16MB batch each time:
~13,300 documents read per page against ~10,000 consumed.

Bound the cursor at maxScannedLifecycleListingEntries + 1, in
DelimiterVersions so both listings inherit it. In v1 the bound goes on
BOTH param sets: the two ranges are served by separate cursors and
either one can supply every entry the listing consumes, so each carries
the full bound rather than a share of it.

The bound is +1 and carries no v0 doubling. filter() counts every entry
the cursor delivers, masters and versions alike, in both formats, and
returns FILTER_END on the entry after the budget is spent.
DelimiterCurrent multiplies by 2 on v0 only because its counter lives in
addContents(), which v0 version keys never reach; copying that factor
here would double the bound for nothing. Its comment is corrected to say
so, since it currently claims the opposite.

Getting the bound wrong is not a perf regression. A cursor that ends
early is indistinguishable from the end of the keyspace, and
DelimiterOrphanDeleteMarker.result() flushes its held candidate whenever
the scan budget was not reached -- so an under-sized bound reports a
delete marker as orphaned while unseen versions sit behind the cursor,
and lifecycle then deletes a marker that resurrects an old version.

Tests: unit coverage for the params on v0 and v1 and for the invariant
the bound rests on (the listing reaches FILTER_END within `limit`
entries), plus a functional spec paging real MongoDB listings to
exhaustion across a sweep of scan budgets. That directory had no
maxScannedLifecycleListingEntries coverage at all.

Expected effect is ~25% fewer documents read per page (13,342 -> 10,001
at the default budget), not the 13x the ticket first claimed: the only
safe bound is the scan budget, not MaxKeys.

Issue: ARSN-629
@delthas
delthas force-pushed the improvement/ARSN-629/bound-lifecycle-listing-cursors branch from c0cd144 to bcfba6b Compare September 21, 2026 12:33
@scality scality deleted a comment from bert-e Sep 21, 2026
@bert-e

bert-e commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@scality scality deleted a comment from bert-e Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants