Repository navigation
ci: test results and coverage as artifacts, and the integration suite under 5.1 - #23
Conversation
fadwen
left a comment
There was a problem hiding this comment.
Notes on the lines whose reason the diff does not show.
| # CoveragePercentTarget only reports; the gate below is what enforces it. | ||
| $cfg.CodeCoverage.CoveragePercentTarget = 80 | ||
| # Written before the gates below run, so a failed run still leaves both files to upload | ||
| $cfg.CodeCoverage.OutputPath = 'coverage.xml' |
There was a problem hiding this comment.
Set on the configuration rather than written afterwards from $result, so the files exist the moment Invoke-Pester returns, before the two throws below can end the step; the upload runs with if: always() and finds them either way.
| $cfg.Run.Path = './Tests/Unit', './Validation/Tests/Unit' | ||
| # The integration suite as well: it launches the 5.1 hosts the harness uses on a device, | ||
| # and the runner is elevated, so the SYSTEM-context tests run here as they do under pwsh | ||
| $cfg.Run.Path = './Tests/Unit', './Tests/Integration', './Validation/Tests/Unit' |
There was a problem hiding this comment.
Validation/Tests/Integration stays out: it needs a lab account and a Graph session the runner has not, and those files skip themselves; listing them would only add skipped rows.
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: tests-and-analysis |
There was a problem hiding this comment.
Three artifacts rather than one because the three jobs run in parallel on different runners and an artifact name may be uploaded once per workflow run.
eaeaaca to
34464d6
Compare
… under 5.1 The tests job computed coverage and gated on it but wrote no file, and no job kept its test results: a failure had to be read out of the log, and coverage could not be compared between runs. The 5.1 job ran the unit suites only, so a 5.1 trap in the harness, a stderr line turned into a terminating error, a Group-Object order, showed on a device before it showed in CI. Every test job now writes its results as NUnit XML and the tests job its coverage as JaCoCo XML, both uploaded as artifacts whether the job passes or fails. The 5.1 job runs Tests/Integration as well: the runner is elevated, so the SYSTEM-context tests run there as they do under pwsh. The result files are ignored by git.
Summary
The tests job computed coverage and gated on it but wrote no file, and no job kept its test results: a failure had to be read out of the log, and coverage could not be compared between runs. The Windows PowerShell 5.1 job ran the unit suites only, so a 5.1 trap in the harness showed on a device before it showed in CI. Every test job now writes its results as NUnit XML and the tests job its coverage as JaCoCo XML, uploaded as artifacts whether the job passes or fails, and the 5.1 job runs the integration suite as well.
Stacked on #22 (base branch); the diff is this change only. Merge #16 through #22, then this. The workflow runs on pull requests against
main, so this change is exercised when the stack lands.Changes
.github/workflows/quality-gates.yml.TestResult.OutputPathandCodeCoverage.OutputPathset beforeInvoke-Pesterin the tests job, so a failed run still leaves both files;TestResult.OutputPathin the 5.1 and ARM64 jobs; anactions/upload-artifact@v7step withif: always()after each. The 5.1 job'sRun.Pathgains./Tests/Integrationand writes the same step summary the ARM64 job does. The header comment says what each job covers..gitignore.testResults-*.xml, next to the two names already there.Verification
uses, three upload steps withif: always().actions/upload-artifactv7.0.1 is the current release, the same major as theactions/checkout@v7the workflow uses.tests-and-analysis(results and coverage),windows-powershell-5-1,windows-on-arm.Notes