Skip to content

ci: test results and coverage as artifacts, and the integration suite under 5.1 - #23

Merged
fadwen merged 1 commit into
feat/pipeline-input-and-viewsfrom
ci/upload-results-and-coverage
Oct 6, 2026
Merged

fadwen merged 1 commit into
feat/pipeline-input-and-viewsfrom
ci/upload-results-and-coverage

Conversation

@fadwen

@fadwen fadwen commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

The tests job computed coverage and gated on it but wrote no file, and no job kept its test results: a failure had to be read out of the log, and coverage could not be compared between runs. The Windows PowerShell 5.1 job ran the unit suites only, so a 5.1 trap in the harness showed on a device before it showed in CI. Every test job now writes its results as NUnit XML and the tests job its coverage as JaCoCo XML, uploaded as artifacts whether the job passes or fails, and the 5.1 job runs the integration suite as well.

Stacked on #22 (base branch); the diff is this change only. Merge #16 through #22, then this. The workflow runs on pull requests against main, so this change is exercised when the stack lands.

Changes

  • .github/workflows/quality-gates.yml. TestResult.OutputPath and CodeCoverage.OutputPath set before Invoke-Pester in the tests job, so a failed run still leaves both files; TestResult.OutputPath in the 5.1 and ARM64 jobs; an actions/upload-artifact@v7 step with if: always() after each. The 5.1 job's Run.Path gains ./Tests/Integration and writes the same step summary the ARM64 job does. The header comment says what each job covers.
  • .gitignore. testResults-*.xml, next to the two names already there.
  • Changelog. A Changed entry.

Verification

Notes

  • If the integration suite proves slow or flaky under 5.1 on the hosted runner, the fix is in the suite or the harness, not in dropping the job back to unit tests: a test that passes under pwsh and fails under 5.1 is the finding this job exists for.

@fadwen fadwen left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes on the lines whose reason the diff does not show.

# CoveragePercentTarget only reports; the gate below is what enforces it.
$cfg.CodeCoverage.CoveragePercentTarget = 80
# Written before the gates below run, so a failed run still leaves both files to upload
$cfg.CodeCoverage.OutputPath = 'coverage.xml'

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set on the configuration rather than written afterwards from $result, so the files exist the moment Invoke-Pester returns, before the two throws below can end the step; the upload runs with if: always() and finds them either way.

$cfg.Run.Path = './Tests/Unit', './Validation/Tests/Unit'
# The integration suite as well: it launches the 5.1 hosts the harness uses on a device,
# and the runner is elevated, so the SYSTEM-context tests run here as they do under pwsh
$cfg.Run.Path = './Tests/Unit', './Tests/Integration', './Validation/Tests/Unit'

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validation/Tests/Integration stays out: it needs a lab account and a Graph session the runner has not, and those files skip themselves; listing them would only add skipped rows.

if: always()
uses: actions/upload-artifact@v7
with:
name: tests-and-analysis

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three artifacts rather than one because the three jobs run in parallel on different runners and an artifact name may be uploaded once per workflow run.

@fadwen
fadwen force-pushed the ci/upload-results-and-coverage branch from eaeaaca to 34464d6 Compare October 6, 2026 09:00
… under 5.1

The tests job computed coverage and gated on it but wrote no file, and
no job kept its test results: a failure had to be read out of the log,
and coverage could not be compared between runs. The 5.1 job ran the
unit suites only, so a 5.1 trap in the harness, a stderr line turned
into a terminating error, a Group-Object order, showed on a device
before it showed in CI.

Every test job now writes its results as NUnit XML and the tests job its
coverage as JaCoCo XML, both uploaded as artifacts whether the job passes
or fails. The 5.1 job runs Tests/Integration as well: the runner is
elevated, so the SYSTEM-context tests run there as they do under pwsh.
The result files are ignored by git.
@fadwen
fadwen added this pull request to stack #24 October 6, 2026 16:17
@fadwen
fadwen merged commit 836f806 into main Oct 6, 2026
8 checks passed
@fadwen
fadwen deleted the ci/upload-results-and-coverage branch October 6, 2026 16:23
@fadwen fadwen mentioned this pull request Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant