Preserve CTRF duplicate results and test attachments - #10769
Preserve CTRF duplicate results and test attachments#10769Amaury Levé (Evangelink) wants to merge 5 commits into
Conversation
Preserve duplicate-UID executions, emit test attachments, and avoid splitting Unicode surrogate pairs during truncation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Record the attachment projection in the internal API baseline and align the regression assertion with current analyzer rules. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Preserves duplicate-UID CTRF executions and adds test attachment serialization, addressing #10764.
Changes:
- Removes UID-based retry collapsing.
- Adds MIME-aware CTRF attachments.
- Expands unit and acceptance coverage.
Show a summary per file
| File | Description |
|---|---|
CapturedAttachment.cs |
Adds attachment DTO. |
CapturedTestResult.cs |
Stores captured attachments. |
TestResultCapture.cs |
Captures attachment metadata and MIME types. |
CtrfReportEngine.JsonTestWriter.cs |
Serializes CTRF attachments. |
CtrfReportEngine.JsonSerializer.cs |
Counts and emits every execution. |
CtrfReportEngine.TestCollapsing.cs |
Removes UID-based collapsing. |
InternalAPI.Unshipped.txt |
Tracks new internal APIs. |
CtrfReportEngineTests.cs |
Tests attachments and duplicate UIDs. |
CtrfReportTests.cs |
Adds end-to-end attachment and duplicate coverage. |
Review details
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 9/9 changed files
- Comments generated: 2
- Review effort level: Balanced
Correct the shared capture documentation and cover attachment path-resolution failures without dropping later valid artifacts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assert that MSTest retry attempts remain distinct CTRF results instead of expecting UID-based flaky inference. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Restore the required source BOM and cover every MIME mapping plus surrogate-safe truncation for attachment names, paths, and descriptions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review details
Suppressed comments (1)
src/Platform/Microsoft.Testing.Extensions.CtrfReport/CtrfReportEngine.JsonSerializer.cs:104
- Hard-coding
flakyto zero reintroduces #10292 for MSTest[Retry]. These updates carryRetryAttemptProperty(MSTestTestNodeConverter.cs:116-122), which was added specifically so CTRF can emitretryAttempts/flaky; its contract still documents that behavior (RetryAttemptProperty.cs:12-16). Stop UID-only inference for untagged duplicate UIDs, but retain retry collapsing for explicitly tagged attempt sequences. As written, a fail-then-pass retry is incorrectly reported as two unrelated tests rather than one flaky pass.
// MTP UIDs are not unique per execution, so they cannot establish retry
// relationships. Keep flaky at zero until explicit attempt correlation is
// available rather than dropping duplicate-UID results or inventing retries.
writer.WriteNumber("flaky", 0);
- Files reviewed: 11/11 changed files
- Comments generated: 0 new
- Review effort level: Balanced
| return attachments; | ||
| } | ||
|
|
||
| private static string GetAttachmentContentType(string path) |
There was a problem hiding this comment.
FYI, I used this list for our default mappings: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/MIME_types/Common_types
We're going in the opposite direction, since we are combining attachment name + media type into a filename, so we're mapping media types to file extensions, but it seems like you could use roughly the same list. We also do an on-Windows-only fallback to look things up in the registry under HKEY_CLASSES_ROOT.
Summary
FileArtifactPropertyvalues as schema-compliant CTRF attachments with MIME inference, fallback metadata, and descriptionsTesting
Microsoft.Testing.Extensions.UnitTests: 52 CTRF reporter tests passed onnet8.0Microsoft.Testing.Platform.Acceptance.IntegrationTests: 15 CTRF acceptance cases passed acrossnet462,net8.0, andnet10.0Fixes #10764