Skip to content

ripple: drop the 50% limit and always run only the affected unit tests - #344134

Merged
Alexandru Dima (alexdima) merged 1 commit into
mainfrom
ripple/remove-max-share
Oct 25, 2026
Merged

Alexandru Dima (alexdima) merged 1 commit into
mainfrom
ripple/remove-max-share

Conversation

@alexdima

Copy link
Copy Markdown
Member

Follow-up to #342907.

What

ripple plan had a cost-based cutoff: when the selected unit tests were more than --max-share (default 0.5) of the unit test work, it reported running every unit test: the change affects N% of the unit test work (limit 50%) and set select: false, so macOS and Windows ran every unit test. This removes it, without a replacement threshold. A plan now either selects exactly the unit test files that a change can affect, however many, or runs every unit test for a dependency reason only (Selection::run_all): a runEverything file, a package or script of the test runners, a module that the runners use, or a module that does not parse. Linux still runs every unit test on every push.

  • build/ripple/src/main.rs: --max-share is gone from the usage, the option list and the parsing (passing it now fails with unknown option '--max-share'), and so are the max_share parameter of plan() and its reason. select is sel.run_all.is_none(). share stays in plan.json and in the reason of a selection, as information. rustfmt now packs OPTIONS onto two lines: "max-share" was its only element longer than rustfmt's threshold for short array elements (10 characters).
  • build/ripple/README.md: the CI paragraph and the plan command no longer mention the limit.

pr.yml never passed --max-share and needs no change: its Check ripple step reads .reason only for parse error (, and .select and .unit for a plan that selects nothing.

What the limit cost

On the replay of the 708 first-parent commits of main from Oct 13 to 20 (the one in the README), 18 commits (2.5%) went over the limit. They would have selected 69.6% of the unit test work on average, and ran all of it. Without the limit, the macOS and Windows unit test work per commit drops from 9.8% to 9.0% of a full run on average, about 0.4 VM minutes per push: the unit test steps of a full run took 49.4 VM minutes on macOS and Windows in the last run of #344104, which ran every unit test.

Open pull requests that read plan.json

Neither depends on the limit's reason text:

Validation

The 18 commits
Pull request Commit Share Replay: declTestFiles This branch main at a383e08
#341342 0695d91 76.2% 1,525 select: true, 1,525 files select: false (limit 50%)
#341406 50559e7 59.6% 1,083 select: true, 1,083 files select: false (limit 50%)
#341434 425e631 67.6% 1,383 select: true, 1,383 files select: false (limit 50%)
#341208 d548c2a 63.8% 1,188 select: true, 1,188 files select: false (limit 50%)
#341463 cf0ccf5 67.8% 1,398 select: true, 1,398 files select: false (limit 50%)
#341674 8734ece 67.8% 1,401 select: true, 1,401 files select: false (limit 50%)
#341640 54d466a 67.5% 1,396 select: true, 1,396 files select: false (limit 50%)
#334259 922ce6a 83.9% 1,820 select: true, 1,820 files select: false (limit 50%)
#341861 009603b 67.8% 1,414 select: true, 1,414 files select: false (limit 50%)
#340669 0b4e063 66.2% 1,276 select: true, 1,276 files select: false (limit 50%)
#341922 be4f9ba 67.5% 1,418 select: true, 1,418 files select: false (limit 50%)
#341965 c35fcfc 67.6% 1,420 select: true, 1,420 files select: false (limit 50%)
#342210 e630a74 67.5% 1,426 select: true, 1,426 files select: false (limit 50%)
#342485 91065a6 67.2% 1,428 select: true, 1,428 files select: false (limit 50%)
#342681 1155fd0 67.2% 1,428 select: true, 1,428 files select: false (limit 50%)
#342688 23794cf 81.4% 1,828 select: true, 1,828 files select: false (limit 50%)
#342941 d1ce11c 67.3% 1,437 select: true, 1,437 files select: false (limit 50%)
#342960 2c225c3 78.8% 1,745 select: true, 1,745 files select: false (limit 50%)
  • The 16 commits that run everything still do: select: false, with the replay's runAll reason (12 test job definitions, 2 test runners, 1 transpiler, 1 runtime version).
  • The CI path, for the commit of fix: path, URI, JSON edit utility bugs #334259 (83.9%), with --out-dir, GITHUB_OUTPUT and GITHUB_STEP_SUMMARY: select=true, a unit.txt with 1,820 files, and a job summary that lists them. The Check ripple expressions of pr.yml on that plan.json: no parse warning, selects-nothing=false.
  • ripple plan --config vscode.jsonc --base HEAD --max-share 0.5: Error: unknown option '--max-share'.
  • git grep -nE 'max-share|max_share|half of the unit test|limit 50|\(limit ' -- build/ripple .github test/unit build/lib: no matches.
  • node --experimental-strip-types build/hygiene.ts build/ripple/src/main.rs build/ripple/README.md: passed (it checks main.rs; README.md is not eligible). The pre-commit hook passed too.
  • Not run locally: the workflows. Since this changes build/ripple, its CI runs every unit test, and the Ripple check builds, tests and plans with this branch.

`ripple plan` ran every unit test when the selected unit tests were more
than `--max-share` (default 0.5) of the unit test work, so a change that
can affect 60% of the unit tests ran all of them on macOS and Windows.

Selection is now off only for a dependency reason (`Selection::run_all`):
a `runEverything` file, a package or script of the test runners, a module
that the runners use, or a module that does not parse. Otherwise the plan
selects the files the change can affect, however many. `--max-share` is
gone (passing it is an error), `select` is `sel.run_all.is_none()`, and
the share stays in plan.json and in the reason as information.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 25, 2026 13:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused removal matches the intended behavior, preserves existing plan contracts and dependency safeguards, and has no unresolved findings.

No open issues 🎉

What changed in this PR

Removes Ripple’s cost-based cutoff so macOS and Windows CI use affected unit test selections regardless of their share of total test work.

Changes:

  • Removes --max-share and the 50% fallback; dependency-based full-run safeguards remain.
  • Preserves share as informational output and leaves Linux full-test runs unchanged.
  • Updates CLI help and documentation to match the new behavior.
File Description
build/​ripple/​src/​main.rs Removes threshold parsing and makes selection depend only on run_all.
build/​ripple/​README.md Documents unrestricted selection and removes the obsolete option.

Review effort: Balanced

@alexdima
Alexandru Dima (alexdima) merged commit 03e4d0d into main Oct 25, 2026
62 checks passed
@alexdima
Alexandru Dima (alexdima) deleted the ripple/remove-max-share branch October 25, 2026 14:29
@vs-code-engineering vs-code-engineering Bot added this to the 1.144.0 milestone Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants