Skip to content

Clarify GitHub Actions reporting setup - #10774

Open
Amaury Levé (Evangelink) wants to merge 5 commits into
mainfrom
dev/amauryleve/clarify-github-reporting
Open

Clarify GitHub Actions reporting setup#10774
Amaury Levé (Evangelink) wants to merge 5 commits into
mainfrom
dev/amauryleve/clarify-github-reporting

Conversation

@Evangelink

@Evangelink Amaury Levé (Evangelink) commented Aug 26, 2026

Copy link
Copy Markdown
Member

MSTest.Sdk's default ClassicEngine profile does not include Microsoft.Testing.Extensions.GitHubActionsReport, so passing --report-gh without opting in fails as an unknown option with little recovery guidance. This is particularly confusing alongside the third-party GitHubActionsTestLogger, which exposes the separate --report-github option.

This change:

  • adds a general recovery hint for unknown Microsoft.Testing.Platform options, pointing users to --help and explaining that extension options require a referenced and registered extension
  • documents the exact ClassicEngine opt-in property for --report-gh in both repository and NuGet-facing guidance
  • retains --report-gh to avoid a breaking rename or collision with GitHubActionsTestLogger's --report-github
  • preserves the existing default package graph and keeps both Azure DevOps and GitHub Actions reporting extensions opt-in

Validation:

  • targeted unknown-option unit test passed
  • targeted help acceptance test passed for net10.0, net8.0, and net462
  • repository package build succeeded

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 26, 2026 14:37
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Verdict: All clear

This PR adds a helpful hint message when unknown CLI options are detected, pointing users to --help and suggesting they check extension registration. The change is minimal, well-scoped, and correctly implemented.

Changes reviewed

Area Assessment
CommandLineOptionsValidator Hint appended after all unknown-option errors, before returning — correct placement
PlatformResources.resx New CommandLineUnknownOptionsHint resource with proper {Locked="'--help'"} scoping (includes surrounding quotes to avoid locking substrings in translatable words)
*.xlf files All 13 locale files updated with state="new" entries — consistent with UpdateXlf regeneration
PACKAGE.md / README.md Documentation for --report-gh extension enablement added
Unit test CommandLineHandlerTests updated to expect the new hint in the error message
Acceptance test HelpInfoTests wildcard pattern updated to include the hint line

Dimensions assessed

20 of 22 dimensions clean (N/A: Threading & Concurrency, IPC & Serialization). No findings to report.

Notable positives:

  • Localization follows repo conventions precisely — {Locked="'--help'"} includes the single quotes, avoiding the substring-locking pitfall called out in the repo guidelines.
  • No public API surface added.
  • Both unit and acceptance tests are updated to cover the new behavior.

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) August 26, 2026 14:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves guidance for unavailable Microsoft.Testing.Platform extension options, particularly GitHub Actions reporting.

Changes:

  • Adds a recovery hint for unknown options.
  • Tests the updated diagnostic output.
  • Documents enabling GitHub Actions reporting through MSTest.Sdk.
Show a summary per file
File Description
CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs Appends the recovery hint.
PlatformResources.resx Defines the localized hint.
PlatformResources.cs.xlf Adds Czech localization metadata.
PlatformResources.de.xlf Adds German localization metadata.
PlatformResources.es.xlf Adds Spanish localization metadata.
PlatformResources.fr.xlf Adds French localization metadata.
PlatformResources.it.xlf Adds Italian localization metadata.
PlatformResources.ja.xlf Adds Japanese localization metadata.
PlatformResources.ko.xlf Adds Korean localization metadata.
PlatformResources.pl.xlf Adds Polish localization metadata.
PlatformResources.pt-BR.xlf Adds Portuguese localization metadata.
PlatformResources.ru.xlf Adds Russian localization metadata.
PlatformResources.tr.xlf Adds Turkish localization metadata.
PlatformResources.zh-Hans.xlf Adds Simplified Chinese localization metadata.
PlatformResources.zh-Hant.xlf Adds Traditional Chinese localization metadata.
CommandLineHandlerTests.cs Updates the unit-test expectation.
HelpInfoTests.cs Updates the acceptance-test expectation.
MSTest.Sdk/README.md Documents ClassicEngine opt-in.
MSTest.Sdk/PACKAGE.md Adds NuGet-facing setup guidance.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 19/19 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/Package/MSTest.Sdk/PACKAGE.md Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 26, 2026 14:47
@github-actions

Copy link
Copy Markdown
Contributor

🧪 Expert test review — PR #10774

The only test-file changes in this PR are single-line additions of an expected output string (Run '--help' to see the options registered...) inside two already-existing test methods, mechanically updated to match the new CommandLineUnknownOptionsHint resource string added to production code:

  • CommandLineHandlerTests.ParseAndValidateAsync_UnknownOption_ReturnsFalse
  • HelpInfoTests.Help_WhenNoExtensionRegisteredAndUnknownOptionIsSpecified_OutputDefaultHelpContentAndUnknownOption

No new test methods were added and no test logic, assertions, or structure changed beyond adding this expected line to match production behavior. There is nothing substantive to grade or flag — the updates are correct and consistent with the new resource string being introduced.

This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 37.8 AIC · ⌖ 0.883 AIC · ⊞ 16.9K · [◷]( · )

@github-actions

Copy link
Copy Markdown
Contributor

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection results could not be parsed.

Review the workflow run logs for details.

🧵 Parallel-safety audit — PR #10774

Nothing audited here touches process-global state, shared filesystem paths, or [ResourceLock] / [DoNotParallelize] declarations. Nothing to flag for parallel-safety.

The only test-file changes are single-line additions to expected wildcard-match string literals within existing, unmodified test methods:

  • test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoTests.cs — adds one line to an expected --help/--info output pattern in Help_WhenNoExtensionRegisteredAndUnknownOptionIsSpecified_Outp....
  • test/UnitTests/Microsoft.Testing.Platform.UnitTests/CommandLine/CommandLineHandlerTests.cs — adds one line to an expected error-message string in ParseAndValidateAsync_UnknownOption_ReturnsFalse.

No lifecycle members ([TestInitialize], [ClassInitialize], constructors, Dispose), no assembly/class-level [ResourceLock]/[DoNotParallelize]/[Parallelize] declarations, and no .runsettings/testconfig.json/.csproj/.props/.targets parallelization-state files were changed by this PR.

Audited Microsoft.Testing.Platform.Acceptance.IntegrationTests and Microsoft.Testing.Platform.UnitTests at their existing (unchanged) parallelization scope.

Re-run with /parallel-audit.

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 35.8 AIC · ⌖ 1.43 AIC · ⊞ 24.8K · [◷]( · )

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 19/19 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink Amaury Levé (Evangelink) changed the title Clarify GitHub Actions reporting setup Include GitHub Actions reporting in the default SDK profile Aug 26, 2026
Copilot AI review requested due to automatic review settings August 26, 2026 15:03
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink Amaury Levé (Evangelink) changed the title Include GitHub Actions reporting in the default SDK profile Include CI reporting extensions in the default SDK profile Aug 26, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink Amaury Levé (Evangelink) changed the title Include CI reporting extensions in the default SDK profile Clarify GitHub Actions reporting setup Aug 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Note

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants