Show failure details in GitHub Actions step-summary collapsible sections

[Azat Mukhametshin (azat-msft)]

Fixes #10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception: System.InvalidOperationException

Location: src/Calc.cs:42

Expected: 42
Actual:   41

   at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests --help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

            GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

            if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

            int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

            string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

        string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

        int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
        int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
        int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

            GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

            if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

            int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

            if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

            return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:737

• An external summary writer does not acquire this private lock. After existing is read and the summary handle is closed, it can append while the temp file is being staged; this unconditional replacement then overwrites that append with the stale snapshot. Recheck the summary immediately before the swap and retry from a fresh read when it changed, as AppendWithLeadingNoticeCoreAsync does.

                    cancellationToken.ThrowIfCancellationRequested();
                    _fileSystem.ReplaceFile(tempPath, Path);

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• The length recheck still leaves a race before ReplaceFile: a co-writer that does not use this lock can append after GetSummaryLength() returns but before the swap, and that append is then lost with the old file. The code explicitly supports test frameworks that append without this lock, so hoisting the notice must avoid replacing the shared file unless the append/swap can be coordinated atomically with those writers; otherwise keep the notice in append-only content.

                if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

        var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

                    var budget = SummaryBudget.ForProject(currentLength);
                    bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
                    string markdown = condense
                        ? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
                        : BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

                if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

                if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[github-actions]

🧵 Parallel-safety audit — PR #10633

Nothing audited here touches process-global state, shared filesystem paths, or
[ResourceLock] / [DoNotParallelize] declarations. This PR only changes
production code under src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport
and src/Platform/SharedExtensionHelpers, plus docs and unshipped-API tracking
files — no files under test/ were modified, and no test methods, lifecycle
members, or assembly/class-level parallelization declarations were touched.
Nothing to flag for parallel-safety.

_{Re-run with /parallel-audit.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 41.3 AIC · ⌖ 0.933 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CiRunSummaryAggregationTests. * (logger factory wiring)	N/A	Updated constructor call correctly threads a mocked ILoggerFactory into the processor.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOrOffAsync	2/2 killed	Pins the exact on/off validator branch and message, not just the invalid verdict.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused positive-path check for the new option.	—
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end acceptance test asserts the exact collapsible markup, exception type and message text.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Verifies the opt-out flag suppresses details while keeping the compact failure line.	—
A (90–100)	mod HelpInfoAllExtensionsTests. * (help/info text)	N/A	Updated wildcard-match text stays alphabetically ordered and covers both --help and --info blocks.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Pins the exact collapsible fenced-block layout (details/summary/exception/location) as one composed string.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Confirms the opt-out flag suppresses exception/detail rendering.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the null-failure-details fallback path distinct from the disabled-flag path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ HtmlEncodesFailureNameInSummaryElement	2/2 killed	Regression-shaped test for HTML injection via generic type names in the summary element.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureMessageContainingCodeFence_ DoesNotBreakOutOfTheBlock	4/4 killed	Exercises fence-escaping with an embedded fence in the failure body, a real and specific edge case.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ TruncatesFailureList_ AndSaysSo	2/2 killed	Verifies both the count and the explicit truncation message for the 20-failure cap.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ DegradesOversizedFailuresToNamedLines_ KeepingEveryFailureListed	3/3 killed	Asserts partial expansion under a constrained budget while every failure stays named — a strong invariant check.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ ExhaustedBudget_ ListsFailuresAndSaysTheirDetailsWereOmitted	3/3 killed	Zero-budget edge case correctly distinguishes "no details" from "details omitted, said explicitly".	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ SharesTheBudgetAcrossFailures_ SoASmallerBudgetExpandsFewer	2/2 killed	Comparative assertion (generous vs. tight budget) that meaningfully constrains the sharing logic.	—
A (90–100)	new GitHubActionsSummaryReporterTests. Clip_ ManyShortRows_ IsTruncatedByRowCountEvenWhenUnderTheCharacterLimit	3/3 killed	Row-cap vs. char-cap distinction is explicitly guarded by a precondition assertion.	—
A (90–100)	new GitHubActionsSummaryReporterTests. Clip_ RowCountUnderLimit_ IsNotMarkedTruncated	2/2 killed	Straightforward negative-path check for the truncation marker.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetRemainingDetailsBudget_ EmptyFile_ ReturnsFullBudget	1/1 killed	Trivial boundary case, correctly isolated from the "existing content" variant below.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetRemainingDetailsBudget_ ExistingContent_ IsSubtracted	1/1 killed	Verifies the subtraction arithmetic precisely rather than just "budget decreased".	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetRemainingDetailsBudget_ FileAlreadyOverBudget_ ReturnsZero	1/1 killed	Clamps-at-zero boundary explicitly checked with rationale in the comment.	—
A (90–100)	new GitHubActionsSummaryReporterTests. Clip_ LoneCarriageReturns_ AreCountedAsRows	2/2 killed	Targets CR-only line endings specifically, a real cross-platform edge case for the row cap.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real non-ASCII (Japanese) text to prove the budget is byte- not char-denominated.	—
A (90–100)	new GitHubActionsSummaryReporterTests. TryGetFailureInfo_ ReadsTheExplanationAndException_ FromEveryFailingStateShape	6/6 killed	Table-style coverage of every state-property arm (Failed/Error/Timeout/Cancelled/Passed/null) via a local helper.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendRenderedStepSummarySectionAsync_ RendersAgainstTheLengthObservedUnderTheLock	2/2 killed	Verifies the length passed to the render callback matches the file's actual length under the lock.	—
A (90–100)	new GitHubActionsSummaryReporterTests. EffectiveStepSummaryLimit_ IsSlightlyBelowTheDocumentedLimit	2/2 killed	Encodes a real production incident's safety margin as an executable invariant.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ AreOrderedWithHeadroom	2/2 killed	Constant-ordering invariant test, cheap and directly tied to the degradation design.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Goes beyond the constants to prove the rendering behavior at the boundary condition.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetRemainingDetailsBudget_ FileOverGitHubLimit_ ReturnsZero	1/1 killed	Concise expression-bodied boundary test for the "already lost" case.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMinimalMarkdown_ IsSmallEnoughThatTheProjectedSizeGateIsMeaningful	2/2 killed	Correctly asserts the condensed form is small and independent of failure count/size.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildAggregateMarkdown_ ScalesWithModuleCount_ WithoutExceedingTheCap	3/3 killed	Data-driven across realistic module counts (40–5000) and asserts degradation actually engaged, not just size.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildAggregateMarkdown_ ManyModules_ StaysUnderTheLimitAndReportsOmittedModules	3/3 killed	Byte-accurate cap check plus assertion that the shortfall notice text is correct and non-duplicated.	—
A (90–100)	new GitHubActionsSummaryReporterTests. TruncationNotices_ KeepTheMarkerOnALineOfItsOwn	2/2 killed	Protects an exact line-format contract that an external grep-based validation workflow depends on.	—
A (90–100)	new GitHubActionsSummaryReporterTests. TruncationNotices_ ShareOneMarker_ SoASummaryCannotCarryTwoWarnings	3/3 killed	Exercises both writer modes sequentially and proves exactly one marker/end-marker survives.	—
A (90–100)	new GitHubActionsSummaryReporterTests. Clip_ LongValue_ TruncatesAndMarksIt	2/2 killed	Simple, correct truncation-marker check.	—
A (90–100)	new GitHubActionsSummaryReporterTests. Clip_ ShortOrEmptyValue_ IsReturnedAsIsOrNull	2/2 killed	Covers both the pass-through and null-for-blank branches in one focused test.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildAggregateMarkdown_ RendersFailureDetailsFromFragment	4/4 killed	Validates the aggregate path renders the same collapsible detail structure as the direct path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithRetryAsync_ RefusesTheWrite_ WhenItWouldCrossTheCap	2/2 killed	Guards a real safety property (never push a file over the GitHub cap) with a clear rationale comment.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ LeavesTheFileUntouched_ WhenTheResultWouldCrossTheCap	2/2 killed	Companion of the above for the upsert path; confirms no partial write occurred.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildAggregateMarkdown_ CondenseAllModules_ RendersOnlyVerdictLines	3/3 killed	Verifies the condensed-rendering path drops detail while keeping per-module verdicts.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildAggregateMarkdown_ CondenseAllModules_ StillStopsListing_ WhenEvenVerdictLinesWouldOverflow	2/2 killed	Extreme-scale edge case for the final degradation stage (unlisted modules).	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildAggregateMarkdown_ AccountsForWhatTheSharedFileAlreadyHolds	2/2 killed	Confirms the budget calculation subtracts pre-existing file content, not just the fresh render.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildAggregateMarkdown_ StaysUnderTheCap_ WhenTheContentIsNonAscii	2/2 killed	Non-ASCII byte-cap regression test paired with the earlier per-project variant.	—
A (90–100)	new GitHubActionsSummaryReporterTests. CreateModule_ ThenFragmentRoundTrip_ KeepsFailureDiagnostics_ AndOmitsThemFromSlowestTests	3/3 killed	Round-trip test across module creation and fragment serialization, verifying diagnostics survive correctly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. HasLeadingTruncationNotice_ IgnoresTheMarker_ WhenItAppearsInsideFailureDiagnostics	2/2 killed	Confusable-input test: marker text embedded in user data must not be mistaken for a real notice.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ StillHoistsTheNotice_ WhenAFailureMessageContainsTheMarker	3/3 killed	Explicitly documents why two marker occurrences (one real, one embedded) are the correct outcome.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ UpgradesAWeakerNotice_ WhenSectionsAreRemoved	3/3 killed	Correctly verifies notice-strength upgrade replaces rather than stacks.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ KeepsTheStrongerNotice_ WhenAWeakerOneFollows	3/3 killed	Symmetric case to the above, confirming a weaker notice never downgrades a stronger one.	—
A (90–100)	new GitHubActionsSummaryReporterTests. CountProjectSections_ IgnoresTheMarker_ WhenAFailureBodyPrintsItOnItsOwnLine	2/2 killed	Fence-aware scanning verified with a marker embedded verbatim in a fenced failure body.	—
A (90–100)	new GitHubActionsSummaryReporterTests. CountProjectSections_ CountsMarkersAfterAFenceCloses	2/2 killed	Confirms the scan correctly resumes counting after a fence closes.	—
A (90–100)	new GitHubActionsSummaryReporterTests. CountProjectSections_ HandlesLongerFences	2/2 killed	Nested-fence edge case (short fence inside a longer one) is correctly not treated as a close.	—
A (90–100)	new GitHubActionsSummaryReporterTests. TrimToRenderedFailures_ KeepsOnlyTheFailuresThatWillBeRendered	2/2 killed	Verifies the retained set stays bounded at every incremental step, not just the final state.	—
A (90–100)	new GitHubActionsSummaryReporterTests. TrimToRenderedFailures_ BreaksNameTiesByUid	2/2 killed	Targets a real tie-breaking correctness bug class (duplicate names across two independent sort paths).	—
A (90–100)	new GitHubActionsSummaryReporterTests. ApplyPendingFailure_ ReleasesTheSlotWhenARetryRecovers	3/3 killed	Exercises the retry-recovery slot-release path with a clear before/after assertion sequence.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildTruncationNotice_ SaysHowManyProjectsGotTheirResultsIn	1/1 killed	Straightforward message-content check.	—
A (90–100)	new GitHubActionsSummaryReporterTests. CountProjectSections_ CountsOnlyFullSections	2/2 killed	Boundary check between full and partial/incomplete section markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ HoistsNoticeToTheTop_ AndAppendsContentAfterIt	2/2 killed	Verifies both the hoist and the correct ordering of the appended content that follows.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ KeepsNoticeFirst_ WhenACoWriterAppendsAfterIt	2/2 killed	Concurrency-adjacent scenario (co-writer interleaving) covered without flaky timing dependence.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ PutsTheNoticeFirst_ AndNeverAddsASecondOne	2/2 killed	De-duplication invariant on repeated upserts, an easy regression to introduce silently.	—
A (90–100)	new GitHubActionsSummaryReporterTests. CountProjectSections_ IgnoresTheMarkerInsideTestOutput	2/2 killed	Confirms a test name that textually contains the marker does not inflate the section count.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	2/2 killed	Simulates a mid-write failure via a mocked throwing stream and proves atomicity (stage-then-swap).	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Injects a concrete interfering writer and asserts every foreign append survives the retry loop.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummary_ CountsOtherWritersSections_ ButNotItsOwn	3/3 killed	Distinguishes counting other writers' sections from this run's own re-inserted section on re-run.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ RefusesBeforeReadingAFileAlreadyOverTheBound	2/2 killed	Guards against unbounded memory allocation from a foreign oversized file, with an untouched-file check.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Uses a stream that throws on read to prove the fast-path never attempts to read an oversized file.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 136.4 AIC · ⌖ 1.83 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

    private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
        => builder.Append(BuildCondensedLine(
            module.AssemblyName,
            module.TargetFramework,
            module.TotalTests,
            module.PassedTests,
            module.FailedTests,
            module.SkippedTests,
            module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.