Skip to content

refactor(test): reduce cognitive complexity of the test runners - #1756

Open
marcossevilla wants to merge 9 commits into
mainfrom
refactor/cc-test-runner
Open

marcossevilla wants to merge 9 commits into
mainfrom
refactor/cc-test-runner

Conversation

@marcossevilla

@marcossevilla marcossevilla commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Status

READY

Description

Reduces the cognitive complexity of the test runners, and moves coverage handling into its own library along the way.

  • Test runner. TestCLIRunner.test hands each package to _testPackage. Test events go to _TestEventReporter, which switches on the event type and keeps the tallies and the list of failures. It now lives in its own test_event_reporter.dart part, so test_cli_runner.dart only orchestrates the run (719 → 415 lines).
  • Coverage library (lib/src/coverage/). CoverageOptions holds the settings of a run. A sealed CoverageReport per package owns coverage/lcov.info:
    • FlutterCoverageReport keeps the file flutter test writes.
    • DartCoverageReport converts the json coverage of dart test into lcov.
    • The runner picks the report once and no longer branches on the test type for coverage, including the --coverage argument.
    • CoverageMetrics, CoverageCollectionMode, MinCoverageNotMet and formatUncoveredLines move there too, so lib/src/cli depends on lib/src/coverage and not the other way around.
    • .gitignore re-includes lib/src/coverage/ and test/src/coverage/, which its coverage/ rule for reports also matched.
  • Commands. TestCommand and DartTestCommand share TestCLIRunner.validateTarget, and both run their tests through _runTests. The arguments they forward become private getters on their options classes.

Bug fix

With --collect-coverage-from all, the step that adds untested files to lcov.info didn't honor --exclude-coverage:

  • It parsed the value as a single glob, so the documented '**/*.g.dart **/*.freezed.dart' excluded nothing.
  • It matched globs against absolute paths, which package:glob resolves against the process's working directory instead of the package root. Recursive runs never excluded anything.

It now parses the value the same way CoverageMetrics does and matches paths relative to the package, like the lcov SF: entries. The threshold check was already correct, so only the contents of lcov.info change.

Other changes

  • The lcov.info existence assert for dart runs with collectCoverageFrom: all now runs before the untested-files step. The file already exists at that point, and asserts are off in release builds.
  • 4 command tests read argResults.rest inside verify(), which made mocktail verify that getter instead of the runner call. They now verify the runner.

Validation

Function Before After
TestCLIRunner.test 66 3
_testCommand 66 7
TestCommand.run 30 4
CoverageMetrics.fromLcovRecords 28 0
DartTestCommand.run 21 4
_enhanceLcovWithUntestedFiles (now _addUntestedFiles) 17 4
  • cognitive_complexity 0.2.4: compared with main, nothing increased and nothing breaks the 15 ceiling (net delta -118). The highest score in any touched file is 11.
  • undead 0.1.1 (closed-app): no new dead declarations or privatization candidates.
  • dedupe 0.1.0: duplicate lines between TestCommand and DartTestCommand drop from 784 to 755. The rest of that duplication predates this PR, and a shared base for the two commands would be a follow-up.
  • The public API (api_summary) is unchanged. Every changed line is covered by tests. New tests cover CoverageReport and the exclude-coverage fix, and the CoverageMetrics tests moved to test/src/coverage.

The commits are split so they can be stacked as separate PRs if needed.

Found by the full cognitive complexity scan in #1755, which tests VeryGoodOpenSource/very_good_workflows#520. This is part of a three-PR cleanup. Once all three merge, the full scan on #1755 passes:

Type of Change

  • ✨ New feature (non-breaking change which adds functionality)
  • 🛠️ Bug fix (non-breaking change which fixes an issue)
  • ❌ Breaking change (fix or feature that would cause existing functionality to change)
  • 🧹 Code refactor
  • ✅ Build configuration change
  • 📝 Documentation
  • 🗑️ Chore

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@marcossevilla

Copy link
Copy Markdown
Member Author

@vgvbot /review-pr

@unicoderbot

unicoderbot Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

I'll review the code and share my feedback shortly.

  • Preparing environment
    • Cloning repository
    • Setting up dev environment
  • Reviewing code
  • Posting review

Code review submitted — PR approved.

unicoderbot[bot]
unicoderbot Bot previously approved these changes Oct 1, 2026

@unicoderbot unicoderbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. This is a clean, careful refactor that significantly reduces cognitive complexity across the test runners while preserving behavior, log output, exit codes, and public APIs.

Verified correctness

  • CoverageMetrics.fromLcovRecords — the _add + _uncoveredLinesWith decomposition preserves per-file accumulation (via [...?uncoveredLines[file], ...newLines]) and still filters by exclude globs the same way.
  • _TestEventReporter switch — I confirmed TestEvent in very_good_test_runner 0.4.0 is abstract class TestEvent (not sealed), with subtypes StartTestEvent, AllSuitesTestEvent, DebugTestEvent, ExitTestEvent not matched in the new switch. In Dart, a non-exhaustive switch statement (not expression) over a non-sealed type simply falls through on no-match — matching the original if-chain's silent ignore of those types. ExitTestEvent continues to be handled explicitly in _testCommand.
  • _checkCoverage — the simplified if (uncoveredLines != null) is sound because uncoveredLines is assigned non-null only when options.showUncovered && coverageMetrics.uncoveredLines.isNotEmpty.
  • _reportDone — success = event.success ?? false; if (success) return; is equivalent to the original if (event.success != true) (continues on null/false, returns on true).
  • _writeLcov ordering change for Dart+all — assert runs before enhance now; _convertDartCoverageToLcov creates the file beforehand, so the invariant holds. The PR description already calls this out.
  • _enhanceLcovWithUntestedFiles — .map((r) => r.file).nonNulls.toSet() is equivalent to where((r) => r.file != null).map((r) => r.file!).toSet(); the _untestedFileRecord LF/LH/DA/SF format and the (pre-existing) prefix-based line filter ('part' still matching partition) are preserved verbatim.
  • _runTests / _validateTarget extractions in both commands are mechanical and preserve exit-code semantics.

Observations (no action required)

  • _runTests in DartTestCommand/TestCommand takes recursive even though it's only forwarded to TestCLIRunner.test — fine, keeps the extraction honest.
  • The near-identical _validateTarget duplication between TestCommand and DartTestCommand is already called out in the PR description as a follow-up.

No correctness, security, or lifecycle issues found. The reasoning that existing tests already reach every new branch is believable given the refactor is purely structural, but CI should confirm.

@unicoderbot

unicoderbot Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Heads up — I'm auto-updating this PR by merging the latest main into this branch. No action needed; I'll comment again only if this update hits a conflict or an unexpected error.

unicoderbot Bot and others added 6 commits October 5, 2026 09:40
Fix the grammar of the dart lcov conversion comment, document that
excludeFromCoverage holds space-separated globs, drop the unused async
from the test event listener, and skip computing uncovered lines for
lcov records without a source file.
TestCommand and DartTestCommand carried the same _validateTarget, only
differing in the project name of the message. Move it to
TestCLIRunner.validateTarget and align the run helpers on _runTests.

The rest-argument tests read argResults.rest inside verify(), which made
mocktail verify that getter instead of the runner call. They now verify
the runner with a local list, since validateTarget reads rest up front.
With collect-coverage-from all, the untested-files step parsed
--exclude-coverage as a single glob, so the documented
'**/*.g.dart **/*.freezed.dart' excluded nothing. It also matched the
absolute file path, which package:glob resolves against the process
working directory rather than the package root.

Parse the value the same way CoverageMetrics does and match the path
relative to the package, as the lcov SF entries are.
TestCLIRunner wrote, converted and checked coverage through a chain of
static helpers that each took the package root, the lcov path, the test
type and the coverage options, and branched on the test type to tell
dart and flutter runs apart.

Coverage now lives in its own library. CoverageOptions holds the
settings of a run, and a sealed CoverageReport per package owns the lcov
file: FlutterCoverageReport keeps what flutter test wrote, while
DartCoverageReport converts the json coverage of dart test. The runner
picks the report once and no longer branches on the test type for
coverage, including the argument that turns coverage collection on.

CoverageMetrics, CoverageCollectionMode, MinCoverageNotMet and
formatUncoveredLines move along with it, so lib/src/cli depends on
lib/src/coverage and not the other way around. The CoverageMetrics tests
move to test/src/coverage, next to new CoverageReport tests.
_TestEventReporter is a self-contained unit that turns test events into
terminal output. Give it its own part of the cli library so
test_cli_runner.dart only orchestrates the run.
--exclude-coverage has always accepted several space-separated globs,
and the untested-files step now honors them too. The test and dart test
help, the MCP test tool, the option dartdocs and the configuration docs
still described it as a single glob, unlike site/docs/commands/test.md.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant