From a1452222e63ef1c1e897f6be734883e577dc9972 Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:20:50 +0200 Subject: [PATCH 01/13] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20refactor=20dotnet-ne?= =?UTF-8?q?w=20skills=20for=20native=20MTP=20test=20stack?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both scaffolds now define the coupled Codebelt 12.x and xunit.v3 4.x contract with native Microsoft.Testing.Platform selection, Codebelt Coverlet coverage, and executable TFM validation so generated solutions restore and report consistently. --- skills/dotnet-new-app-slnx/SKILL.md | 24 +++++++++++++++---- skills/dotnet-new-app-slnx/evals/evals.json | 9 +++++-- skills/dotnet-new-app-slnx/references/app.md | 7 ++++++ skills/dotnet-new-lib-slnx/SKILL.md | 21 +++++++++++++--- skills/dotnet-new-lib-slnx/evals/evals.json | 10 ++++++-- .../dotnet-new-lib-slnx/references/library.md | 10 +++++++- 6 files changed, 69 insertions(+), 12 deletions(-) diff --git a/skills/dotnet-new-app-slnx/SKILL.md b/skills/dotnet-new-app-slnx/SKILL.md index 91c0369..d881d77 100644 --- a/skills/dotnet-new-app-slnx/SKILL.md +++ b/skills/dotnet-new-app-slnx/SKILL.md @@ -42,6 +42,7 @@ The scaffold is incomplete unless it produces all required artifacts for the sel - `Directory.Build.props` - `Directory.Packages.props` - `testenvironments.json` +- root `global.json` selecting `Microsoft.Testing.Platform` - the shared governance/docs assets copied from `assets/shared/` If you cannot generate any required artifact from the documented templates and rules, halt and report the mismatch instead of improvising, omitting the file, or substituting a weaker fallback. @@ -89,7 +90,7 @@ Read `references/app.md` for the app-specific project structure, template file m ## Step 3: Resolve Dynamic Dependency Versions -Before writing `Directory.Packages.props`, resolve every `*_VERSION` placeholder in that file to the latest stable listed version for its matching package ID on NuGet.org. +Before writing `Directory.Packages.props`, resolve every `*_VERSION` placeholder in that file to the latest stable listed compatible version for its matching package ID on NuGet.org, respecting the xUnit v4 / Codebelt v12 lines below. When `pwsh` 7+ is available, prefer the deterministic helper in `/scripts/resolve-package-versions.ps1` over manual lookup. Run it as `pwsh -NoProfile -File "/scripts/resolve-package-versions.ps1" -TargetFramework `. By default it resolves placeholders from this skill's own `assets/shared/Directory.Packages.props`, so a normal scaffold run only needs `-TargetFramework`. Treat its JSON output as the source of truth for package placeholders. @@ -107,8 +108,8 @@ This includes shared and host-specific app packages such as: - `Codebelt.Extensions.Xunit.App` - `Microsoft.NET.Test.Sdk` - `MinVer` -- `coverlet.collector` -- `coverlet.msbuild` +- `Codebelt.Coverlet.MTP` +- `Microsoft.Testing.Extensions.HangDump` - `xunit.v3` - `xunit.v3.runner.console` - `xunit.runner.visualstudio` @@ -119,6 +120,16 @@ This includes shared and host-specific app packages such as: - `Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation` - `Microsoft.Extensions.Hosting` +### xUnit v4 / Codebelt v12 test stack + +Resolve `Codebelt.Extensions.Xunit.App` on the latest stable compatible **12.x** line and `xunit.v3` plus `xunit.v3.runner.console` on the latest stable compatible **4.x** line. xUnit v4 retains the `xunit.v3` package IDs; do not invent an `xunit.v4` ID. Resolve `xunit.runner.visualstudio` independently and check its compatibility. These major lines define the scaffold contract, not fixed patch-version pins. Inspect package assets and nuspec dependency ranges, then restore the combined package set for every selected test TFM; a version-index lookup alone does not prove compatibility. Stop on unresolved metadata or incompatible dependencies rather than reverting to Codebelt v11 or older xUnit. The verified `Codebelt.Extensions.Xunit.App` 12.0.1 meta-package has only net9.0/net10.0 dependency groups; a net8.0 application request requires an explicitly selected compatible consumer test runtime or a verified alternative host-package layout, not a silent downgrade or a claim that the app meta-package supports net8.0. + +Copy the shared root `global.json` with `test.runner = Microsoft.Testing.Platform`; `UseMicrosoftTestingPlatformRunner` alone does not select the SDK CLI. Use a generally supported, non-preview .NET SDK **10 or later** even when the application targets `net8.0` or `net9.0`. Verify the selected SDK with `dotnet --version` from the generated root. Keep an existing SDK pin and other `global.json` settings if generating into an existing repo; merge the runner setting and report incompatible SDK pins instead of silently replacing them. + +The shared test-only ItemGroup owns the versionless references to `Codebelt.Coverlet.MTP` and `Microsoft.Testing.Extensions.HangDump`; versions belong in `Directory.Packages.props`. Do not add duplicate references to test `.csproj` files. Replace legacy `coverlet.collector`, `coverlet.msbuild` and `coverlet.MTP`; do not install Microsoft's coverage engine alongside Codebelt Coverlet. The coverage assembly and registration hook remain `coverlet.MTP.dll` and `Coverlet.MTP.TestingPlatformBuilderHook`, not the NuGet package ID. + +Use Codebelt v12 `ManagedApplicationFixture` / `ManagedWebApplicationFixture` with entrypoint-owned deferred startup. Do not generate the removed `BlockingManagedApplicationFixture` or `BlockingManagedWebApplicationFixture` types. Generate a deterministic host startup/response or service-resolution functional test for each selected host, rather than leaving a zero-test project. + ## Step 4: Apply the Substitution Map When copying template files, replace these placeholders in file contents: @@ -165,7 +176,7 @@ Exception: update `testenvironments.json` with the derived `{UBUNTU_TESTRUNNER_T `testenvironments.json` is a required shared scaffold asset. Do **not** silently omit it. If you cannot generate it from the shared template plus `{UBUNTU_TESTRUNNER_TAG}`, halt and report the mismatch instead of skipping the file. -Exception: if the user selected multiple host types, rewrite the root `README.md` running section to list one `dotnet run --project ...` command per generated host project instead of leaving a single `{AppType}` placeholder example. +Exception: if the user selected multiple host types, rewrite the root `README.md` running section and `.github/CONTRIBUTING.md` test command examples to list one concrete command per generated host/test project instead of leaving a single `{AppType}` placeholder example. ### 2. Copy app `Directory.Build.props` Copy `assets/app/Directory.Build.props` to the project root, applying placeholder substitution. @@ -217,6 +228,11 @@ After generating, verify: - [ ] `Empty Web` uses the `Web` suffix, `Web API` uses `Api`, `MVC` uses `Mvc`, and `Web App / Razor` uses `WebApp` - [ ] MVC and Razor variants include their starter UI assets - [ ] Worker projects include `Worker.cs` +- [ ] `global.json` selects `Microsoft.Testing.Platform` and the selected SDK is generally supported, non-preview .NET 10 or later +- [ ] Codebelt test packages resolve to 12.x and the `xunit.v3` framework/console runner resolve to 4.x, with combined restore validated for every selected test TFM +- [ ] Each functional test project discovers and passes at least one actual host-behavior test +- [ ] Run `dotnet build -c Release`, then one `dotnet test --project --framework -c Release --results-directory -- --report-xunit-trx --coverlet --coverlet-output-format opencover` per project/TFM; verify nonempty TRX and OpenCover artifacts, coverage of the application's executed code, and `--hangdump` options in runner help +- [ ] Existing CI reusable-workflow/action refs support native MTP, OpenCover output and matching upload globs; preserve the caller pipeline rather than copying another repository's jobs or duplicating extension arguments supplied by the shared action If the scaffold is generated outside a git-initialized and tagged repository, expect MinVer to report a placeholder pre-release version such as `0.0.0-alpha.0` until the user initializes git and adds a version tag. Treat that as expected bootstrap state, not as a reason to remove MinVer or change the generated versioning setup. diff --git a/skills/dotnet-new-app-slnx/evals/evals.json b/skills/dotnet-new-app-slnx/evals/evals.json index e3f8a9b..66ddfa5 100644 --- a/skills/dotnet-new-app-slnx/evals/evals.json +++ b/skills/dotnet-new-app-slnx/evals/evals.json @@ -4,7 +4,7 @@ { "id": 1, "prompt": "Scaffold a new .NET Worker application solution named DemoApp with root namespace Acme, target framework net10.0, and the Minimal hosting pattern.", - "expected_output": "A worker-based scaffold with package versions resolved from NuGet, a generated Worker.cs file, and target-framework-aware test environment settings.", + "expected_output": "A worker-based scaffold with compatible Codebelt xUnit 12.x and xUnit 4.x packages resolved from NuGet, native MTP SDK selection, Codebelt Coverlet coverage, a generated Worker.cs file, and target-framework-aware test environment settings.", "expectations": [ "Recognizes the explicit Worker request and does not redundantly ask the user to restate app_host_types", "Collects target_framework in addition to solution name, namespace, host type, and hosting pattern", @@ -18,7 +18,12 @@ "Generates testenvironments.json instead of silently omitting the shared test environment asset", "Names the generated solution file DemoApp.slnx instead of lowercasing it to demoapp.slnx", "Still generates the solution file and functional test project even for a single-host Worker scaffold", - "Resolves worker package versions from NuGet instead of reusing stale remembered version numbers from prior runs" + "Resolves worker package versions from NuGet instead of reusing stale remembered version numbers from prior runs", + "Uses Codebelt.Extensions.Xunit.App 12.x with xunit.v3 and xunit.v3.runner.console 4.x, retaining the existing package IDs", + "Generates root global.json selecting Microsoft.Testing.Platform and verifies a supported non-preview .NET 10+ SDK independently of the target runtime", + "Uses centrally versioned Codebelt.Coverlet.MTP and Microsoft.Testing.Extensions.HangDump through the shared test-only ItemGroup without duplicate project references or legacy Coverlet packages", + "Generates and runs an actual worker host-behavior test using ManagedApplicationFixture rather than a removed blocking fixture or a zero-test project", + "Verifies nonzero discovery plus nonempty TRX and OpenCover output with native MTP arguments rather than VSTest logger/collector switches" ] }, { diff --git a/skills/dotnet-new-app-slnx/references/app.md b/skills/dotnet-new-app-slnx/references/app.md index af0b064..ec4ca9e 100644 --- a/skills/dotnet-new-app-slnx/references/app.md +++ b/skills/dotnet-new-app-slnx/references/app.md @@ -42,6 +42,7 @@ Do not cherry-pick only the files that feel essential. The shared scaffold contr - `CHANGELOG.md` - `Directory.Build.targets` - `Directory.Packages.props` +- `global.json` (native MTP runner opt-in) - `README.md` - `testenvironments.json` - `.bot/README.md` @@ -161,6 +162,12 @@ For framework-aligned ASP.NET packages, keep the selected target framework major - Example: a `net9.0` app should resolve these packages to the latest stable `9.x` version, not `10.x` - If the lookup step fails, stop and report the failure instead of guessing with stale package versions from prior runs +## Native MTP Testing + +Use the test-stack contract in `SKILL.md`: Codebelt xUnit 12.x, `xunit.v3` framework/console runner 4.x, live-resolved compatible `Codebelt.Coverlet.MTP` and HangDump, and root `global.json` selecting `Microsoft.Testing.Platform`. Use a supported non-preview .NET 10+ SDK independently of the app TFM. Keep coverage/diagnostic references in the shared test-only ItemGroup, not every test project. + +Generate a host-behavior functional test using managed v12 fixtures for each host. Build Release and run `dotnet test --project --framework -c Release --results-directory -- --report-xunit-trx --coverlet --coverlet-output-format opencover`. Verify nonzero tests, nonempty TRX/OpenCover output and hang-dump options in runner help. Preserve the shared CI caller: inspect the actual `jobs-dotnet-test` ref and action it resolves to for MTP support, coverage formats and upload paths; do not duplicate extension arguments already supplied by that action. + ## Test Environments Generate `testenvironments.json` from the selected target framework instead of keeping a hardcoded SDK patch tag. diff --git a/skills/dotnet-new-lib-slnx/SKILL.md b/skills/dotnet-new-lib-slnx/SKILL.md index 2fb3a82..d37fda7 100644 --- a/skills/dotnet-new-lib-slnx/SKILL.md +++ b/skills/dotnet-new-lib-slnx/SKILL.md @@ -52,7 +52,7 @@ Read `references/library.md` for the library-specific project structure, templat ## Step 3: Resolve Dynamic Dependency Versions -Before writing `Directory.Packages.props`, resolve every `*_VERSION` placeholder in that file to the latest stable listed version for its matching package ID on NuGet.org. +Before writing `Directory.Packages.props`, resolve every `*_VERSION` placeholder in that file to the latest stable listed compatible version for its matching package ID on NuGet.org, respecting the xUnit v4 / Codebelt v12 lines below. - Use the NuGet V3 service index at `https://api.nuget.org/v3/index.json` to discover the package metadata endpoints - Prefer registration metadata so you can ignore unlisted versions and prerelease builds @@ -66,6 +66,16 @@ This includes the benchmark-related packages: - `BenchmarkDotNet.Diagnostics.Windows` - `Codebelt.Extensions.BenchmarkDotNet.Console` +### xUnit v4 / Codebelt v12 test stack + +Resolve `Codebelt.Extensions.Xunit.App` on the latest stable compatible **12.x** line and `xunit.v3` plus `xunit.v3.runner.console` on the latest stable compatible **4.x** line. xUnit v4 retains the `xunit.v3` package IDs; do not invent an `xunit.v4` ID. Resolve `xunit.runner.visualstudio` independently and check its compatibility. These major lines define the scaffold contract, not fixed patch-version pins. Inspect package assets and nuspec dependency ranges, then restore the combined package set for every selected test TFM; a version-index lookup alone does not prove compatibility. Stop on unresolved metadata or incompatible dependencies rather than reverting to Codebelt v11 or older xUnit. + +Copy the shared root `global.json` with `test.runner = Microsoft.Testing.Platform`; `UseMicrosoftTestingPlatformRunner` alone does not select the SDK CLI. Use a generally supported, non-preview .NET SDK **10 or later** even when the library targets older runtimes. Verify `dotnet --version` from the generated root. Keep an existing SDK pin and other `global.json` settings if generating into an existing repo; merge the runner setting and report incompatible SDK pins instead of silently replacing them. + +The shared test-only ItemGroup owns the versionless references to live-resolved compatible `Codebelt.Coverlet.MTP` and `Microsoft.Testing.Extensions.HangDump`; versions belong in `Directory.Packages.props`. Do not add duplicate references to test `.csproj` files. Replace legacy `coverlet.collector`, `coverlet.msbuild` and `coverlet.MTP`; do not install Microsoft's coverage engine alongside Codebelt Coverlet. The coverage assembly and registration hook remain `coverlet.MTP.dll` and `Coverlet.MTP.TestingPlatformBuilderHook`, not the NuGet package ID. + +Use the Codebelt v12 `Test` base class with `ITestOutputHelper` from `Xunit`. Preserve the scaffold's `Codebelt.Extensions.Xunit.App` package choice and verify its resolved dependency groups against the selected test runtimes. If a selected runtime is incompatible, report that constraint rather than substituting packages or adding a separate report provider. For host tests use `ManagedApplicationFixture` / `ManagedWebApplicationFixture`; do not generate the removed blocking application fixtures. Generate at least one deterministic public-behavior test per library project. Keep non-executable source TFMs such as `netstandard2.0` out of test/benchmark execution: render the test and benchmark `TargetFrameworks` from selected executable TFMs while preserving the source matrix. If no executable TFM was selected, ask for a consumer test runtime rather than generating an unrunnable test project. + ## Step 4: Apply the Substitution Map When copying template files, replace these placeholders in file contents: @@ -127,7 +137,7 @@ Preserve the template's BOM policy by default. If the source template is UTF-8 w Exception: generate `testenvironments.json` instead of copying it verbatim. Always include the `WSL-Ubuntu` entry, then add one `Docker-Ubuntu` entry per selected target framework using the Docker image tag `codebeltnet/ubuntu-testrunner:{major}` where `{major}` comes from the TFM. -Exception: do not leave `Directory.Packages.props` with unresolved placeholder tokens. Resolve each package version placeholder to the latest stable listed NuGet.org version for that exact package ID before writing the file. +Exception: do not leave `Directory.Packages.props` with unresolved placeholder tokens. Resolve each package version placeholder to the latest stable listed compatible NuGet.org release for that exact package ID before writing the file, respecting the test-stack major lines in Step 3. Before finalizing the Docker entries, validate that each generated tag exists in the Docker Hub tags feed for `codebeltnet/ubuntu-testrunner`. Prefer the machine-readable tags API over manual inspection: @@ -179,7 +189,7 @@ After generating, verify: - [ ] Each packable project has a `.nuget/{ProjectName}/` folder with `PackageReleaseNotes.txt`, `icon.png` (placeholder), and `README.md` - [ ] `Directory.Packages.props` lists all `` packages used in the solution - [ ] `Directory.Packages.props` contains concrete version numbers with no unresolved `*_VERSION` placeholders -- [ ] Every `Directory.Packages.props` version was resolved from the latest stable listed NuGet.org package version at generation time +- [ ] Every `Directory.Packages.props` version was resolved from the latest stable listed compatible NuGet.org release at generation time, respecting Codebelt 12.x / xUnit 4.x test-stack lines - [ ] `tuning/{PROJECT_NAME}.Benchmarks/{PROJECT_NAME}.Benchmarks.csproj` references the main source project and relies on central package management - [ ] `tooling/{BENCHMARK_RUNNER_PROJECT_NAME}/Program.cs` contains one runtime job per selected executable TFM - [ ] `tooling/{BENCHMARK_RUNNER_PROJECT_NAME}/{BENCHMARK_RUNNER_PROJECT_NAME}.csproj` references the default tuning benchmark project and relies on central package management @@ -197,6 +207,11 @@ After generating, verify: - [ ] If any manifest entry was absent from the installed skill copy, `pwsh -NoProfile -File "/scripts/restore-missing-shared-assets.ps1"` was run (or files were fetched manually from the upstream raw URL) — not diagnosed iteratively - [ ] No manifest entries were silently skipped; if the restore script reported failures, generation was halted rather than continuing with incomplete shared assets - [ ] `.github/dependabot.yml` watches the repo root so central NuGet package management stays current after scaffolding +- [ ] Root `global.json` selects `Microsoft.Testing.Platform` and the selected SDK is generally supported, non-preview .NET 10 or later +- [ ] Combined test-package restore succeeds for every executable test TFM; source-only TFMs remain supported by source builds, not executable test projects +- [ ] Each library test project discovers and passes at least one public-behavior test using Codebelt v12 / xUnit v4 +- [ ] Run `dotnet build -c Release` (with `-p:SkipSignAssembly=true` only when the signing key is unavailable), then one `dotnet test --project --framework -c Release --results-directory -- --report-xunit-trx --coverlet --coverlet-output-format opencover` per project/TFM; verify nonempty TRX and OpenCover artifacts, coverage of the library's executed code, and `--hangdump` options in runner help +- [ ] Existing CI reusable-workflow/action refs support native MTP, OpenCover output and matching upload globs; preserve platform-specific test/coverage behavior, verify Windows-only targets on Windows, and do not duplicate reporting/coverage/hang-dump arguments supplied by the shared action Summarize what was generated and note any manual steps (e.g. registering with SonarCloud, populating `.docfx/images/` with logo/favicon). diff --git a/skills/dotnet-new-lib-slnx/evals/evals.json b/skills/dotnet-new-lib-slnx/evals/evals.json index f1b8079..c883ba0 100644 --- a/skills/dotnet-new-lib-slnx/evals/evals.json +++ b/skills/dotnet-new-lib-slnx/evals/evals.json @@ -4,7 +4,7 @@ { "id": 1, "prompt": "Scaffold a new .NET library solution named MyLibrary with root namespace Acme and the default target frameworks.", - "expected_output": "A single-project library scaffold with PROJECT_NAME-based paths, NuGet package metadata, DocFX files, and resolved package versions.", + "expected_output": "A single-project library scaffold with PROJECT_NAME-based paths, NuGet package metadata, DocFX files, compatible Codebelt xUnit 12.x / xUnit 4.x packages, native MTP SDK selection, and Codebelt Coverlet coverage.", "expectations": [ "Uses {PROJECT_NAME} as the default packable project name", "Offers all active non-preview LTS and STS target framework quick picks before free text", @@ -12,7 +12,13 @@ "Creates DocFX config using {DOCFX_TARGET_FRAMEWORK} rather than a hardcoded net10.0", "Locks DocFX metadata source to ../src (relative to .docfx/) so ONLY code in the src folder is digested for API documentation, excluding tools/, scripts/, and other root folders", "Keeps api/namespaces/**/*.md and api/types/**/*.md under build.overwrite and excludes api/namespaces/** and api/types/** from build.content so API overwrites are not emitted as standalone conceptual pages", - "Uses the current working directory as the scaffold root instead of creating a nested MyLibrary folder" + "Uses the current working directory as the scaffold root instead of creating a nested MyLibrary folder", + "Preserves Codebelt.Extensions.Xunit.App on the 12.x line with xunit.v3 and xunit.v3.runner.console 4.x, retaining the existing package IDs", + "Generates root global.json selecting Microsoft.Testing.Platform and verifies a supported non-preview .NET 10+ SDK independently of library target runtimes", + "Uses centrally versioned Codebelt.Coverlet.MTP and Microsoft.Testing.Extensions.HangDump through the shared test-only ItemGroup without duplicate project references or legacy coverage engines", + "Generates and executes a deterministic public-behavior test with the Codebelt Test base class instead of accepting zero discovery", + "Verifies nonempty TRX and OpenCover artifacts using native MTP arguments for each executable test TFM, keeping source-only frameworks out of executable tests", + "Uses xUnit's built-in --report-xunit-trx reporting without adding a separate report package or altering existing CI callers to request an alternate reporter" ] }, { diff --git a/skills/dotnet-new-lib-slnx/references/library.md b/skills/dotnet-new-lib-slnx/references/library.md index 7f4b464..53a7148 100644 --- a/skills/dotnet-new-lib-slnx/references/library.md +++ b/skills/dotnet-new-lib-slnx/references/library.md @@ -131,7 +131,7 @@ Always resolve `{LATEST_Nx}` from NuGet.org — never hardcode versions. Hard rule: -1. Resolve every `*_VERSION` placeholder to the latest stable listed version for that exact package ID on NuGet.org at generation time +1. Resolve every `*_VERSION` placeholder to the latest stable listed compatible release for that exact package ID on NuGet.org at generation time; keep Codebelt xUnit on 12.x and the `xunit.v3` framework/console runner on 4.x as required by `SKILL.md` 2. Exclude prerelease versions even if they are newer 3. Exclude unlisted versions when the metadata source exposes listing status 4. Resolve each package independently; never reuse one generic "latest" token across multiple packages @@ -158,6 +158,14 @@ The generated `.github/dependabot.yml` should watch the repo root (`directory: " --- +## Native MTP Testing + +Follow `SKILL.md` for the coupled Codebelt xUnit 12.x / xUnit 4.x package contract. The package IDs remain `xunit.v3` and `xunit.v3.runner.console`. Copy root `global.json` selecting `Microsoft.Testing.Platform`, use a supported non-preview .NET 10+ SDK independently of library TFMs, and retain the test-only shared references to `Codebelt.Coverlet.MTP` and `Microsoft.Testing.Extensions.HangDump` without duplicate project references or legacy coverage integrations. + +Generate at least one public-behavior test per library and use only executable TFMs for tests and benchmarks. Build Release, then run `dotnet test --project --framework -c Release --results-directory -- --report-xunit-trx --coverlet --coverlet-output-format opencover`; verify nonzero discovery, nonempty TRX/OpenCover output and runner help's hang-dump options. Inspect the actual shared CI workflow/action refs and report consumers before finalizing; preserve platform distinctions and avoid duplicating reporting/coverage/hang-dump arguments already provided by the shared action. Preserve the existing Codebelt app meta-package and report incompatible test-runtime choices rather than introducing compatibility workarounds. + +--- + ## Target Framework Default Default new libraries to a single target framework using the newest generally supported .NET LTS, but also offer every other generally supported non-preview .NET LTS and STS channel so the user can deliberately choose any actively supported track. From a8b9edfa20b7ff6daa675ff8c6d14cf321ab06d2 Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:23:43 +0200 Subject: [PATCH 02/13] =?UTF-8?q?=F0=9F=94=A7=20update=20dotnet-new=20temp?= =?UTF-8?q?lates=20for=20native=20MTP=20stack?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Scaffold output now references Codebelt Coverlet MTP and HangDump instead of legacy Coverlet packages, selects the native Microsoft.Testing.Platform runner through root global.json, and resolves test packages on the Codebelt 12.x and xunit.v3 4.x lines so generated solutions build with current test dependencies. --- .../assets/app/Directory.Build.props | 10 ++-------- .../assets/app/console/Program.minimal.cs | 2 +- .../assets/shared.manifest.json | 1 + .../shared/.github/copilot-instructions.md | 2 +- .../assets/shared/Directory.Packages.props | 4 ++-- .../assets/shared/global.json | 5 +++++ .../scripts/resolve-package-versions.ps1 | 15 ++++++++++++++- .../assets/library/Directory.Build.props | 16 +++++----------- .../assets/shared.manifest.json | 1 + .../shared/.github/copilot-instructions.md | 2 +- .../assets/shared/Directory.Packages.props | 4 ++-- .../assets/shared/global.json | 5 +++++ 12 files changed, 40 insertions(+), 27 deletions(-) create mode 100644 skills/dotnet-new-app-slnx/assets/shared/global.json create mode 100644 skills/dotnet-new-lib-slnx/assets/shared/global.json diff --git a/skills/dotnet-new-app-slnx/assets/app/Directory.Build.props b/skills/dotnet-new-app-slnx/assets/app/Directory.Build.props index de6f422..497b355 100644 --- a/skills/dotnet-new-app-slnx/assets/app/Directory.Build.props +++ b/skills/dotnet-new-app-slnx/assets/app/Directory.Build.props @@ -34,20 +34,14 @@ + all runtime; build; native; contentfiles; analyzers; buildtransitive - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - + diff --git a/skills/dotnet-new-app-slnx/assets/app/console/Program.minimal.cs b/skills/dotnet-new-app-slnx/assets/app/console/Program.minimal.cs index 477e17a..2e8026d 100644 --- a/skills/dotnet-new-app-slnx/assets/app/console/Program.minimal.cs +++ b/skills/dotnet-new-app-slnx/assets/app/console/Program.minimal.cs @@ -3,7 +3,7 @@ namespace {ROOT_NAMESPACE}.{AppType}; -public class Program : MinimalConsoleProgram +public class Program : MinimalConsoleProgram { static Task Main(string[] args) { diff --git a/skills/dotnet-new-app-slnx/assets/shared.manifest.json b/skills/dotnet-new-app-slnx/assets/shared.manifest.json index b1ad1df..7c37d46 100644 --- a/skills/dotnet-new-app-slnx/assets/shared.manifest.json +++ b/skills/dotnet-new-app-slnx/assets/shared.manifest.json @@ -19,6 +19,7 @@ "CHANGELOG.md", "Directory.Build.targets", "Directory.Packages.props", + "global.json", "README.md", "testenvironments.json" ] diff --git a/skills/dotnet-new-app-slnx/assets/shared/.github/copilot-instructions.md b/skills/dotnet-new-app-slnx/assets/shared/.github/copilot-instructions.md index 8e36e25..cb908d6 100644 --- a/skills/dotnet-new-app-slnx/assets/shared/.github/copilot-instructions.md +++ b/skills/dotnet-new-app-slnx/assets/shared/.github/copilot-instructions.md @@ -10,7 +10,7 @@ This document provides instructions for writing unit tests for a project/solutio **Always inherit from the `Test` base class** for all unit test classes. This ensures consistent setup, teardown, and output handling across all tests. -> Important: Do NOT add `using Xunit.Abstractions`. xUnit v3 no longer exposes that namespace; including it is incorrect and will cause compilation errors. Use the `Codebelt.Extensions.Xunit` Test base class and `using Xunit;` as shown in the examples below. If you need access to test output, rely on the Test base class (which accepts the appropriate output helper) rather than importing `Xunit.Abstractions`. +> Important: Do NOT add `using Xunit.Abstractions`. xUnit v4 (the `xunit.v3` 4.x package line) does not expose that namespace; including it is incorrect and will cause compilation errors. Use the `Codebelt.Extensions.Xunit` Test base class and `using Xunit;` as shown in the examples below. If you need access to test output, rely on the Test base class (which accepts the appropriate output helper) rather than importing `Xunit.Abstractions`. ```csharp using Codebelt.Extensions.Xunit; diff --git a/skills/dotnet-new-app-slnx/assets/shared/Directory.Packages.props b/skills/dotnet-new-app-slnx/assets/shared/Directory.Packages.props index bb0b218..1fb3414 100644 --- a/skills/dotnet-new-app-slnx/assets/shared/Directory.Packages.props +++ b/skills/dotnet-new-app-slnx/assets/shared/Directory.Packages.props @@ -6,8 +6,8 @@ - - + + diff --git a/skills/dotnet-new-app-slnx/assets/shared/global.json b/skills/dotnet-new-app-slnx/assets/shared/global.json new file mode 100644 index 0000000..3140116 --- /dev/null +++ b/skills/dotnet-new-app-slnx/assets/shared/global.json @@ -0,0 +1,5 @@ +{ + "test": { + "runner": "Microsoft.Testing.Platform" + } +} diff --git a/skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 b/skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 index 8baa28d..ce06e41 100644 --- a/skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 +++ b/skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 @@ -83,6 +83,13 @@ $frameworkAlignedPackages = @( 'Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation' ) +# These are authorized API lines, not frozen package-version pins. +$testStackMajors = @{ + 'Codebelt.Extensions.Xunit.App' = 12 + 'xunit.v3' = 4 + 'xunit.v3.runner.console' = 4 +} + [xml]$template = Get-Content -Path $TemplatePath -Raw $packageNodes = @($template.Project.ItemGroup.PackageVersion) if ($packageNodes.Count -eq 0) { @@ -108,7 +115,13 @@ foreach ($node in $packageNodes) { throw "No versions returned for $packageId from $indexUrl" } - $requiredMajor = if ($frameworkAlignedPackages -contains $packageId) { $targetMajor } else { -1 } + $requiredMajor = if ($testStackMajors.ContainsKey($packageId)) { + $testStackMajors[$packageId] + } elseif ($frameworkAlignedPackages -contains $packageId) { + $targetMajor + } else { + -1 + } $resolved = Select-LatestStableVersion -Versions $index.versions -RequiredMajor $requiredMajor $result[$placeholder] = [ordered]@{ diff --git a/skills/dotnet-new-lib-slnx/assets/library/Directory.Build.props b/skills/dotnet-new-lib-slnx/assets/library/Directory.Build.props index 5fd4fc4..fa676cf 100644 --- a/skills/dotnet-new-lib-slnx/assets/library/Directory.Build.props +++ b/skills/dotnet-new-lib-slnx/assets/library/Directory.Build.props @@ -67,20 +67,14 @@ + all runtime; build; native; contentfiles; analyzers; buildtransitive - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - + @@ -102,9 +96,9 @@ - - false - Exe + + false + Exe true diff --git a/skills/dotnet-new-lib-slnx/assets/shared.manifest.json b/skills/dotnet-new-lib-slnx/assets/shared.manifest.json index 08f8d17..bf68df3 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared.manifest.json +++ b/skills/dotnet-new-lib-slnx/assets/shared.manifest.json @@ -20,6 +20,7 @@ "CHANGELOG.md", "Directory.Build.targets", "Directory.Packages.props", + "global.json", "LICENSE", "README.md", "testenvironments.json" diff --git a/skills/dotnet-new-lib-slnx/assets/shared/.github/copilot-instructions.md b/skills/dotnet-new-lib-slnx/assets/shared/.github/copilot-instructions.md index 8b42991..a1e1ee0 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared/.github/copilot-instructions.md +++ b/skills/dotnet-new-lib-slnx/assets/shared/.github/copilot-instructions.md @@ -10,7 +10,7 @@ This document provides instructions for writing unit tests for a project/solutio **Always inherit from the `Test` base class** for all unit test classes. This ensures consistent setup, teardown, and output handling across all tests. -> Important: Do NOT add `using Xunit.Abstractions`. xUnit v3 no longer exposes that namespace; including it is incorrect and will cause compilation errors. Use the `Codebelt.Extensions.Xunit` Test base class and `using Xunit;` as shown in the examples below. If you need access to test output, rely on the Test base class (which accepts the appropriate output helper) rather than importing `Xunit.Abstractions`. +> Important: Do NOT add `using Xunit.Abstractions`. xUnit v4 (the `xunit.v3` 4.x package line) does not expose that namespace; including it is incorrect and will cause compilation errors. Use the `Codebelt.Extensions.Xunit` Test base class and `using Xunit;` as shown in the examples below. If you need access to test output, rely on the Test base class (which accepts the appropriate output helper) rather than importing `Xunit.Abstractions`. ```csharp using Codebelt.Extensions.Xunit; diff --git a/skills/dotnet-new-lib-slnx/assets/shared/Directory.Packages.props b/skills/dotnet-new-lib-slnx/assets/shared/Directory.Packages.props index c79cbf2..dccbe52 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared/Directory.Packages.props +++ b/skills/dotnet-new-lib-slnx/assets/shared/Directory.Packages.props @@ -7,8 +7,8 @@ - - + + diff --git a/skills/dotnet-new-lib-slnx/assets/shared/global.json b/skills/dotnet-new-lib-slnx/assets/shared/global.json new file mode 100644 index 0000000..3140116 --- /dev/null +++ b/skills/dotnet-new-lib-slnx/assets/shared/global.json @@ -0,0 +1,5 @@ +{ + "test": { + "runner": "Microsoft.Testing.Platform" + } +} From ce4ddd9d059d70731edc4ecf1a0a1ebe0216778e Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:27:38 +0200 Subject: [PATCH 03/13] =?UTF-8?q?=E2=9C=85=20update=20scaffold=20test=20co?= =?UTF-8?q?mmands=20to=20xUnit=20built-in=20reporter?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Generated contributing guides, agent instructions, and readmes now run tests with --report-xunit-trx plus Codebelt Coverlet and verify TRX and OpenCover output, matching the committed skill contracts without a separate TRX provider. --- .../assets/shared/.github/CONTRIBUTING.md | 4 +++- skills/dotnet-new-app-slnx/assets/shared/AGENTS.md | 8 ++++++-- skills/dotnet-new-app-slnx/assets/shared/README.md | 6 +++++- .../assets/shared/.github/CONTRIBUTING.md | 4 +++- skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md | 8 ++++++-- skills/dotnet-new-lib-slnx/assets/shared/README.md | 4 ++++ 6 files changed, 27 insertions(+), 7 deletions(-) diff --git a/skills/dotnet-new-app-slnx/assets/shared/.github/CONTRIBUTING.md b/skills/dotnet-new-app-slnx/assets/shared/.github/CONTRIBUTING.md index fcd05f4..1678544 100644 --- a/skills/dotnet-new-app-slnx/assets/shared/.github/CONTRIBUTING.md +++ b/skills/dotnet-new-app-slnx/assets/shared/.github/CONTRIBUTING.md @@ -14,9 +14,11 @@ Thank you for your interest in contributing! ```bash dotnet restore dotnet build -dotnet test +dotnet test --project test/{ROOT_NAMESPACE}.{AppType}.FunctionalTests/{ROOT_NAMESPACE}.{AppType}.FunctionalTests.csproj -c Release --results-directory TestResults -- --report-xunit-trx --coverlet --coverlet-output-format opencover ``` +Use a supported non-preview .NET 10+ SDK with the root `global.json` MTP runner selection. For multiple hosts, run the command once per functional test project. Verify tests were discovered and the TRX/OpenCover files are nonempty. + ## Code Standards - Use file-scoped namespaces diff --git a/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md b/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md index 5556f7a..d5b7ca1 100644 --- a/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md +++ b/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md @@ -13,7 +13,7 @@ This document provides guidance for AI agents working in this repository. - **Language version:** Always use the latest C# features (`LangVersion=latest`) - **Nullable:** Enable nullable reference types in all new code - **XML documentation:** All public APIs must have XML documentation comments -- **Testing:** Use xUnit v3 with Codebelt.Extensions.Xunit.App base classes +- **Testing:** Use xUnit v4 (`xunit.v3` 4.x packages) with Codebelt.Extensions.Xunit.App v12 base classes ## Markdown Prose Formatting @@ -30,7 +30,11 @@ Do not hard-wrap prose to a fixed column width. Keep paragraphs and Markdown lis - Test project names must end with `FunctionalTests` (e.g. `{ROOT_NAMESPACE}.Console.FunctionalTests`) - Tests are **functional tests** that exercise the running application as a whole - Test classes should inherit from the appropriate base class in `Codebelt.Extensions.Xunit` -- Use `Microsoft.Testing.Platform` as the test runner (`UseMicrosoftTestingPlatformRunner=true`) +- Use `Microsoft.Testing.Platform` as the test runner (root `global.json` selects `test.runner`; `UseMicrosoftTestingPlatformRunner=true` enables project integration) +- Use a supported non-preview .NET 10+ SDK even for older target runtimes +- Test-only shared references supply `Codebelt.Coverlet.MTP` and `Microsoft.Testing.Extensions.HangDump`; keep versions central and do not duplicate inherited references +- Run `dotnet test --project -c Release --results-directory -- --report-xunit-trx --coverlet --coverlet-output-format opencover`; verify nonzero discovery, TRX and OpenCover output +- Use managed application fixtures with deferred entrypoint-owned startup, not removed blocking application fixtures - All tests are executable (`OutputType=Exe`) ## Build & CI diff --git a/skills/dotnet-new-app-slnx/assets/shared/README.md b/skills/dotnet-new-app-slnx/assets/shared/README.md index bcab9bc..90b2a5b 100644 --- a/skills/dotnet-new-app-slnx/assets/shared/README.md +++ b/skills/dotnet-new-app-slnx/assets/shared/README.md @@ -11,7 +11,7 @@ ### Prerequisites -- .NET SDK (see `global.json` for version) +- A supported non-preview .NET 10+ SDK; `global.json` selects the native Microsoft.Testing.Platform runner even when the app targets an older runtime ### Running @@ -23,6 +23,10 @@ dotnet run --project src/{ROOT_NAMESPACE}.{AppType}/{ROOT_NAMESPACE}.{AppType}.c If you generated multiple host types, replace this section with one command per generated project under `src/` instead of leaving a single ambiguous `{AppType}` value. +## Testing + +Tests use xUnit v4 (`xunit.v3` 4.x), Codebelt xUnit v12, and `Codebelt.Coverlet.MTP`. Run each functional test project with `dotnet test --project -c Release --results-directory TestResults -- --report-xunit-trx --coverlet --coverlet-output-format opencover`. Verify nonzero discovery and nonempty TRX/OpenCover output. + ## Contributing Contributions are welcome! Please read [CONTRIBUTING.md](.github/CONTRIBUTING.md). diff --git a/skills/dotnet-new-lib-slnx/assets/shared/.github/CONTRIBUTING.md b/skills/dotnet-new-lib-slnx/assets/shared/.github/CONTRIBUTING.md index 033112e..cd3bd18 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared/.github/CONTRIBUTING.md +++ b/skills/dotnet-new-lib-slnx/assets/shared/.github/CONTRIBUTING.md @@ -16,9 +16,11 @@ git clone {REPOSITORY_URL} cd {REPO_SLUG} dotnet restore dotnet build -dotnet test +dotnet test --project test/{PROJECT_NAME}.Tests/{PROJECT_NAME}.Tests.csproj -c Release --results-directory TestResults -- --report-xunit-trx --coverlet --coverlet-output-format opencover ``` +Use a supported non-preview .NET 10+ SDK with the root `global.json` MTP runner selection. Run each test project and executable TFM separately when validating a multi-target solution; verify nonzero discovery and nonempty TRX/OpenCover files. + ## Code Standards - Use file-scoped namespaces diff --git a/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md b/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md index d5b2c02..52bc738 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md +++ b/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md @@ -15,7 +15,7 @@ This document provides guidance for AI agents working in this repository. - **Language version:** Always use the latest C# features (`LangVersion=latest`) - **Nullable:** Enable nullable reference types in all new code - **XML documentation:** All public APIs must have XML documentation comments -- **Testing:** Use xUnit v3 with Codebelt.Extensions.Xunit.App base classes +- **Testing:** Use xUnit v4 (`xunit.v3` 4.x packages) with Codebelt.Extensions.Xunit.App v12 base classes ## Markdown Prose Formatting @@ -36,7 +36,11 @@ Do not hard-wrap prose to a fixed column width. Keep paragraphs and Markdown lis - Test project names must end with `Tests` (e.g. `{PROJECT_NAME}.Tests`) - Test classes should inherit from the appropriate base class in `Codebelt.Extensions.Xunit` -- Use `Microsoft.Testing.Platform` as the test runner (`UseMicrosoftTestingPlatformRunner=true`) +- Use `Microsoft.Testing.Platform` as the test runner (root `global.json` selects `test.runner`; `UseMicrosoftTestingPlatformRunner=true` enables project integration) +- Use a supported non-preview .NET 10+ SDK even for older target runtimes +- Test-only shared references supply `Codebelt.Coverlet.MTP` and `Microsoft.Testing.Extensions.HangDump`; keep versions central and do not duplicate inherited references +- Run `dotnet test --project -c Release --results-directory -- --report-xunit-trx --coverlet --coverlet-output-format opencover`; verify nonzero discovery, TRX and OpenCover output +- Use managed application fixtures with deferred entrypoint-owned startup, not removed blocking application fixtures - All tests are executable (`OutputType=Exe`) ## Build & CI diff --git a/skills/dotnet-new-lib-slnx/assets/shared/README.md b/skills/dotnet-new-lib-slnx/assets/shared/README.md index 7f205ba..7bf1e72 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared/README.md +++ b/skills/dotnet-new-lib-slnx/assets/shared/README.md @@ -29,6 +29,10 @@ dotnet add package {PROJECT_NAME} Full documentation is available at [{PACKAGE_PROJECT_URL}]({PACKAGE_PROJECT_URL}). +## Testing + +Use a supported non-preview .NET 10+ SDK; root `global.json` selects Microsoft.Testing.Platform. Tests use xUnit v4 (`xunit.v3` 4.x), Codebelt xUnit v12, and `Codebelt.Coverlet.MTP`. Run `dotnet test --project test/{PROJECT_NAME}.Tests/{PROJECT_NAME}.Tests.csproj -c Release --results-directory TestResults -- --report-xunit-trx --coverlet --coverlet-output-format opencover`. Validate each executable TFM and verify nonzero discovery plus nonempty TRX/OpenCover files. + ## Contributing Contributions are welcome! Please read [CONTRIBUTING.md](.github/CONTRIBUTING.md). From 5cb11c22f26aa748993f738e3d94581a67ae3edb Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:27:53 +0200 Subject: [PATCH 04/13] =?UTF-8?q?=E2=9C=85=20add=20MTP=20scaffold=20regres?= =?UTF-8?q?sion=20validation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Validators enforce the MTP runner opt-in, package references, and xUnit reporting flags while the new regression script builds representative scaffolds and verifies discovery, TRX, coverage, and HangDump options with no separate TRX provider. --- scripts/tests/test-scaffold-mtp.ps1 | 245 +++++++++++++++++++++++++++ scripts/validate-skill-templates.ps1 | 52 +++++- 2 files changed, 291 insertions(+), 6 deletions(-) create mode 100644 scripts/tests/test-scaffold-mtp.ps1 diff --git a/scripts/tests/test-scaffold-mtp.ps1 b/scripts/tests/test-scaffold-mtp.ps1 new file mode 100644 index 0000000..c8311a2 --- /dev/null +++ b/scripts/tests/test-scaffold-mtp.ps1 @@ -0,0 +1,245 @@ +param() + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +$repoRoot = [System.IO.Path]::GetFullPath((Join-Path $PSScriptRoot '../..')) +$utf8 = [System.Text.UTF8Encoding]::new($false) +$workspace = Join-Path ([System.IO.Path]::GetTempPath()) ('agentic-scaffold-mtp-' + [Guid]::NewGuid().ToString('N')) +$completed = $false + +function Write-Text { + param([string]$Path, [string]$Content) + [void][System.IO.Directory]::CreateDirectory([System.IO.Path]::GetDirectoryName($Path)) + [System.IO.File]::WriteAllText($Path, $Content, $utf8) +} + +function Render-Template { + param([string]$Source, [string]$Destination, [hashtable]$Map) + $content = [System.IO.File]::ReadAllText((Join-Path $repoRoot $Source), $utf8) + foreach ($key in $Map.Keys) { $content = $content.Replace($key, [string]$Map[$key]) } + Write-Text -Path $Destination -Content $content +} + +function Invoke-DotNet { + param([string[]]$Arguments) + $output = @(& dotnet @Arguments 2>&1) + if ($LASTEXITCODE -ne 0) { + throw "dotnet $($Arguments -join ' ') failed (exit $LASTEXITCODE):`n$($output -join [Environment]::NewLine)" + } + return ($output -join [Environment]::NewLine) +} + +try { + [void][System.IO.Directory]::CreateDirectory($workspace) + # Test version-line selection with a newer future major, prereleases, and mismatched ASP.NET majors. + & { + function Invoke-RestMethod { + param([string]$Uri) + if ($Uri.EndsWith('/v3/index.json')) { + return @{ resources = @(@{ '@type' = 'PackageBaseAddress/3.0.0'; '@id' = 'https://fixture.invalid/' }) } + } + $versions = switch -Regex ($Uri) { + '/codebelt.extensions.xunit.app/' { @('11.2.1', '12.0.0', '12.0.1', '12.1.0-preview.1', '13.0.0'); break } + '/xunit.v3(?:.runner.console)?/' { @('3.2.2', '4.0.0', '4.0.1', '4.1.0-preview.1', '5.0.0'); break } + '/microsoft.aspnetcore.' { @('9.0.14', '10.0.12', '11.0.0'); break } + default { @('1.0.0', '1.0.1', '2.0.0-preview.1') } + } + return @{ versions = $versions } + } + $resolved = & (Join-Path $repoRoot 'skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1') -TargetFramework net10.0 | ConvertFrom-Json + foreach ($pair in @( + @('{CODEBELT_EXTENSIONS_XUNIT_APP_VERSION}', '12.0.1'), + @('{XUNIT_V3_VERSION}', '4.0.1'), + @('{XUNIT_V3_RUNNER_CONSOLE_VERSION}', '4.0.1'), + @('{MICROSOFT_ASPNETCORE_OPENAPI_VERSION}', '10.0.12') + )) { + if ($resolved.($pair[0]).version -ne $pair[1]) { throw "Resolver selected the wrong API line for $($pair[0])." } + } + Write-Host '[PASS] Resolver preserves authorized test-stack majors and excludes prereleases.' + } + + $sdkVersions = @(& dotnet --list-sdks) + if ($LASTEXITCODE -ne 0) { throw 'Unable to enumerate installed SDKs.' } + $sdk = @($sdkVersions | ForEach-Object { + if ($_ -match '^(\d+\.\d+\.\d+)\s' -and [version]$Matches[1] -ge [version]'10.0.100') { $Matches[1] } + } | Sort-Object { [version]$_ } -Descending | Select-Object -First 1) + if ($sdk.Count -ne 1) { throw 'A supported non-preview .NET 10+ SDK is required for scaffold MTP tests.' } + + # Fixed package versions are reproducible regression fixtures, not generated scaffold defaults. + $versions = @{ + '{CODEBELT_EXTENSIONS_XUNIT_APP_VERSION}' = '12.0.1' + '{XUNIT_V3_VERSION}' = '4.0.1' + '{XUNIT_V3_RUNNER_CONSOLE_VERSION}' = '4.0.1' + '{XUNIT_RUNNER_VISUALSTUDIO_VERSION}' = '4.0.0' + '{CODEBELT_COVERLET_MTP_VERSION}' = '10.1.0' + '{MICROSOFT_TESTING_EXTENSIONS_HANGDUMP_VERSION}' = '2.4.1' + '{MICROSOFT_NET_TEST_SDK_VERSION}' = '18.10.1' + '{MINVER_VERSION}' = '8.0.0' + '{CODEBELT_BOOTSTRAPPER_WEB_VERSION}' = '5.2.2' + '{CODEBELT_BOOTSTRAPPER_CONSOLE_VERSION}' = '5.2.2' + '{CODEBELT_BOOTSTRAPPER_WORKER_VERSION}' = '5.2.2' + '{MICROSOFT_ASPNETCORE_OPENAPI_VERSION}' = '10.0.12' + '{MICROSOFT_ASPNETCORE_MVC_RAZOR_RUNTIMECOMPILATION_VERSION}' = '10.0.12' + '{MICROSOFT_EXTENSIONS_HOSTING_VERSION}' = '10.0.12' + '{BENCHMARKDOTNET_VERSION}' = '0.15.8' + '{BENCHMARKDOTNET_DIAGNOSTICS_WINDOWS_VERSION}' = '0.15.8' + '{CODEBELT_EXTENSIONS_BENCHMARKDOTNET_CONSOLE_VERSION}' = '1.3.4' + } + # Exercise the app meta-package's net9/net10 runtime groups; do not add older-runtime workarounds. + $cases = @( + @{ Variant = 'app'; HostType = 'web'; AppType = 'Web'; Tfm = 'net10.0' }, + @{ Variant = 'app'; HostType = 'console'; AppType = 'Console'; Tfm = 'net9.0' }, + @{ Variant = 'app'; HostType = 'worker'; AppType = 'Worker'; Tfm = 'net10.0' }, + @{ Variant = 'lib'; Tfm = 'net10.0' }, + @{ Variant = 'lib'; Tfm = 'net9.0' } + ) + foreach ($case in $cases) { + $variant = $case.Variant + $assetVariant = if ($variant -eq 'app') { 'app' } else { 'library' } + $skillRoot = "skills/dotnet-new-$variant-slnx" + $projectName = if ($variant -eq 'app') { "Acme.$($case.AppType)" } else { 'Acme.Library' } + $testSuffix = if ($variant -eq 'app') { 'FunctionalTests' } else { 'Tests' } + $caseName = "$projectName-$($case.Tfm)" + $caseRoot = Join-Path $workspace $caseName + $map = $versions.Clone() + $map['{ROOT_NAMESPACE}'] = if ($variant -eq 'app') { 'Acme' } else { 'Acme.Library' } + $map['{PROJECT_NAME}'] = $projectName + $map['{SOLUTION_NAME}'] = 'MtpScaffold' + $map['{AppType}'] = if ($variant -eq 'app') { $case.AppType } else { '' } + $map['{TARGET_FRAMEWORK}'] = $case.Tfm + $map['{TARGET_FRAMEWORKS}'] = $case.Tfm + $map['{AUTHOR}'] = 'Scaffold Regression' + $map['{AUTHOR_EMAIL}'] = 'fixture@example.invalid' + $map['{COMPANY_OR_PERSON}'] = 'Scaffold Regression' + $map['{COPYRIGHT_YEAR}'] = '2026' + $map['{PACKAGE_PROJECT_URL}'] = 'https://example.invalid/library' + $map['{REPOSITORY_URL}'] = 'https://example.invalid/scaffold' + $map['{SNK_FILE}'] = 'fixture.snk' + + foreach ($file in @('Directory.Packages.props', 'Directory.Build.targets', 'global.json')) { + Render-Template -Source "$skillRoot/assets/shared/$file" -Destination (Join-Path $caseRoot $file) -Map $map + } + # Select a stable installed SDK only in this isolated regression workspace. + $globalPath = Join-Path $caseRoot 'global.json' + $global = [System.IO.File]::ReadAllText($globalPath) | ConvertFrom-Json + if ($global.test.runner -ne 'Microsoft.Testing.Platform') { throw "$caseName did not select MTP." } + $global | Add-Member -NotePropertyName sdk -NotePropertyValue @{ version = $sdk[0]; rollForward = 'latestPatch'; allowPrerelease = $false } + Write-Text -Path $globalPath -Content ($global | ConvertTo-Json -Depth 5) + Render-Template -Source "$skillRoot/assets/$assetVariant/Directory.Build.props" -Destination (Join-Path $caseRoot 'Directory.Build.props') -Map $map + $sourceProject = "src/$projectName/$projectName.csproj" + $testProject = "test/$projectName.$testSuffix/$projectName.$testSuffix.csproj" + $sourceTemplate = if ($variant -eq 'app') { "$($case.HostType).csproj" } else { 'source.csproj' } + Render-Template -Source "$skillRoot/assets/$assetVariant/$sourceTemplate" -Destination (Join-Path $caseRoot $sourceProject) -Map $map + Render-Template -Source "$skillRoot/assets/$assetVariant/test.csproj" -Destination (Join-Path $caseRoot $testProject) -Map $map + $sourceCode = if ($variant -eq 'app') { "$skillRoot/assets/app/$($case.HostType)/Program.minimal.cs" } else { '' } + if ($variant -eq 'app') { + Render-Template -Source $sourceCode -Destination (Join-Path $caseRoot "src/$projectName/Program.cs") -Map $map + if ($case.HostType -eq 'worker') { + Render-Template -Source "$skillRoot/assets/app/worker/Worker.cs" -Destination (Join-Path $caseRoot "src/$projectName/Worker.cs") -Map $map + } + } else { + Write-Text -Path (Join-Path $caseRoot "src/$projectName/PriceCalculator.cs") -Content @' +namespace Acme.Library; + +/// Computes a price including sales tax. +public static class PriceCalculator +{ + /// Applies the specified sales tax rate to a price. + /// The untaxed price. + /// The fractional tax rate. + /// The price including sales tax. + public static decimal AddTax(decimal price, decimal taxRate) + { + return price * (1 + taxRate); + } +} +'@ + } + $body = if ($variant -eq 'lib') { + 'Assert.Equal(120m, PriceCalculator.AddTax(100m, 0.20m));' + } elseif ($case.HostType -eq 'web') { + @' +using (var application = WebApplicationTestFactory.Create(hostFixture: new ManagedWebApplicationFixture())) + using (var client = application.Host.GetTestClient()) + { + Assert.Equal("Hello from Acme.Web.", await client.GetStringAsync("/")); + } +'@ + } else { + $assertion = if ($case.HostType -eq 'worker') { + 'Assert.Contains(application.Host.Services.GetServices(), service => service is Worker);' + } else { + 'Assert.NotNull(application.Host.Services.GetRequiredService());' + } + "using (var application = ApplicationTestFactory.Create(hostFixture: new ManagedApplicationFixture()))`n {`n $assertion`n }" + } + $returnType = if ($variant -eq 'app' -and $case.HostType -eq 'web') { 'async Task' } else { 'void' } + $imports = if ($variant -eq 'lib') { '' } else { + "using Codebelt.Extensions.Xunit.Hosting;`nusing Microsoft.Extensions.DependencyInjection;`nusing Microsoft.Extensions.Hosting;" + } + if ($variant -eq 'app' -and $case.HostType -eq 'web') { + $imports += "`nusing Codebelt.Extensions.Xunit.Hosting.AspNetCore;`nusing Microsoft.AspNetCore.TestHost;" + } + Write-Text -Path (Join-Path $caseRoot "test/$projectName.$testSuffix/BehaviorTest.cs") -Content @" +using System.Threading.Tasks; +using Codebelt.Extensions.Xunit; +using Xunit; +$imports + +namespace $projectName; + +public class BehaviorTest : Test +{ + public BehaviorTest(ITestOutputHelper output) : base(output) { } + + [Fact] + public $returnType PublicBehavior_ShouldWork_WithScaffoldTestStack() + { + $body + } +} +"@ + Write-Text -Path (Join-Path $caseRoot 'MtpScaffold.slnx') -Content "" + Push-Location $caseRoot + try { + [void](Invoke-DotNet -Arguments @('build', 'MtpScaffold.slnx', '-c', 'Release', '-p:SkipSignAssembly=true', '--nologo')) + $help = Invoke-DotNet -Arguments @('test', '--project', $testProject, '-c', 'Release', '--no-build', '--help') + $reportOption = '--report-xunit-trx' + foreach ($option in @($reportOption, '--coverlet', '--coverlet-output-format', '--hangdump', '--hangdump-timeout')) { + if (-not $help.Contains($option)) { throw "$caseName runner is missing $option." } + } + $run = Invoke-DotNet -Arguments @('test', '--project', $testProject, '--framework', $case.Tfm, '-c', 'Release', '--no-build', '--results-directory', 'TestResults', '--', $reportOption, '--coverlet', '--coverlet-output-format', 'opencover') + $trxFiles = @(Get-ChildItem -LiteralPath (Join-Path $caseRoot 'TestResults') -Filter '*.trx' -Recurse) + $coverageFiles = @(Get-ChildItem -LiteralPath (Join-Path $caseRoot 'TestResults') -Filter '*.xml' -Recurse | Where-Object { $_.Length -gt 0 }) + if ($trxFiles.Count -eq 0) { throw "$caseName emitted no TRX.`n$run" } + foreach ($file in $trxFiles) { + [xml]$trx = [System.IO.File]::ReadAllText($file.FullName) + $counts = $trx.TestRun.ResultSummary.Counters + if ([int]$counts.total -ne 1 -or [int]$counts.passed -ne 1 -or [int]$counts.failed -ne 0) { throw "$caseName did not discover and pass its behavior test.`n$run" } + } + $covered = $false + foreach ($file in $coverageFiles) { + [xml]$coverage = [System.IO.File]::ReadAllText($file.FullName) + if ($null -ne $coverage.SelectSingleNode("/CoverageSession/Modules/Module[ModuleName='$projectName']/Classes/Class/Methods/Method/SequencePoints/SequencePoint[number(@vc) > 0]")) { $covered = $true } + } + if (-not $covered) { throw "$caseName emitted no visited source sequence points in OpenCover.`n$run" } + $assets = [System.IO.File]::ReadAllText((Join-Path $caseRoot (Join-Path (Split-Path $testProject) 'obj/project.assets.json'))) + foreach ($id in @('Codebelt.Extensions.Xunit.App/12.0.1', 'xunit.v3/4.0.1', 'Codebelt.Coverlet.MTP/10.1.0', 'Microsoft.Testing.Extensions.HangDump/2.4.1')) { + if (-not $assets.Contains($id)) { throw "$caseName is missing restored dependency $id." } + } + $graph = $assets | ConvertFrom-Json + if (@($graph.libraries.PSObject.Properties.Name | Where-Object { $_ -like 'Microsoft.Testing.Extensions.TrxReport/*' }).Count -ne 0) { + throw "$caseName restored an out-of-scope TRX provider." + } + Write-Host "[PASS] ${caseName}: Release build, managed/public behavior, one passing test, xUnit TRX, visited OpenCover points, HangDump options and no separate TRX provider." + } finally { Pop-Location } + } + Write-Host '[PASS] Scaffold MTP regressions passed.' + $completed = $true +} finally { + if ($completed -and (Test-Path $workspace)) { + Remove-Item -LiteralPath $workspace -Recurse -Force + } elseif (Test-Path $workspace) { + Write-Host "Failed scaffold workspace retained for diagnosis: $workspace" + } +} diff --git a/scripts/validate-skill-templates.ps1 b/scripts/validate-skill-templates.ps1 index 5e7f313..61d8edf 100644 --- a/scripts/validate-skill-templates.ps1 +++ b/scripts/validate-skill-templates.ps1 @@ -645,14 +645,14 @@ function Get-AppPlaceholderMap { '{TARGET_FRAMEWORK}' = $TargetFramework '{AppType}' = $AppType '{UBUNTU_TESTRUNNER_TAG}' = $ubuntuTag - '{CODEBELT_EXTENSIONS_XUNIT_APP_VERSION}' = '11.0.7' + '{CODEBELT_EXTENSIONS_XUNIT_APP_VERSION}' = '12.0.1' '{MICROSOFT_NET_TEST_SDK_VERSION}' = '18.3.0' '{MINVER_VERSION}' = '7.0.0' - '{COVERLET_COLLECTOR_VERSION}' = '8.0.0' - '{COVERLET_MSBUILD_VERSION}' = '8.0.0' - '{XUNIT_V3_VERSION}' = '3.2.2' - '{XUNIT_V3_RUNNER_CONSOLE_VERSION}' = '3.2.2' - '{XUNIT_RUNNER_VISUALSTUDIO_VERSION}' = '3.1.5' + '{CODEBELT_COVERLET_MTP_VERSION}' = '10.1.0' + '{MICROSOFT_TESTING_EXTENSIONS_HANGDUMP_VERSION}' = '2.4.1' + '{XUNIT_V3_VERSION}' = '4.0.1' + '{XUNIT_V3_RUNNER_CONSOLE_VERSION}' = '4.0.1' + '{XUNIT_RUNNER_VISUALSTUDIO_VERSION}' = '4.0.0' '{CODEBELT_BOOTSTRAPPER_CONSOLE_VERSION}' = '5.0.5' '{CODEBELT_BOOTSTRAPPER_WEB_VERSION}' = '5.0.5' '{CODEBELT_BOOTSTRAPPER_WORKER_VERSION}' = '5.0.5' @@ -1077,6 +1077,46 @@ Add-ValidationResult -Results $results -Name 'App skill documents web-family App Assert-Contains -Name 'dotnet-new-app-slnx/SKILL.md' -Content $skill -Needle 'Worker.cs' } +Add-ValidationResult -Results $results -Name 'Both scaffolds select xUnit 4, Codebelt 12 and native MTP coverage together' -Action { + foreach ($variant in @('app', 'lib')) { + $skillRoot = "skills/dotnet-new-$variant-slnx" + $assetVariant = if ($variant -eq 'app') { 'app' } else { 'library' } + $props = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/assets/$assetVariant/Directory.Build.props" -GitRef $Ref + $packages = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/assets/shared/Directory.Packages.props" -GitRef $Ref + $global = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/assets/shared/global.json" -GitRef $Ref | ConvertFrom-Json + $manifest = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/assets/shared.manifest.json" -GitRef $Ref | ConvertFrom-Json + $skill = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/SKILL.md" -GitRef $Ref + if ($global.test.runner -ne 'Microsoft.Testing.Platform' -or $manifest.files -notcontains 'global.json') { + throw "$skillRoot must ship and inventory the SDK MTP runner opt-in." + } + $codebeltTestPackage = 'Codebelt.Extensions.Xunit.App' + foreach ($id in @('Codebelt.Coverlet.MTP', 'Microsoft.Testing.Extensions.HangDump', $codebeltTestPackage, 'xunit.v3', 'xunit.v3.runner.console')) { + Assert-Contains -Name $skillRoot -Content $props -Needle "true' + Assert-Contains -Name $skillRoot -Content $props -Needle 'MinVer' + } +} + +Add-ValidationResult -Results $results -Name 'Generated app and library scaffolds execute Codebelt v12 tests with native MTP artifacts' -Action { + $output = & pwsh -NoProfile -NonInteractive -File (Join-Path $repoRoot 'scripts/tests/test-scaffold-mtp.ps1') 2>&1 + if ($LASTEXITCODE -ne 0) { + throw ($output -join [Environment]::NewLine) + } +} + Add-ValidationResult -Results $results -Name 'App package template uses specific version placeholders' -Action { $packages = Get-FileText -RepoRoot $repoRoot -RelativePath 'skills/dotnet-new-app-slnx/assets/shared/Directory.Packages.props' -GitRef $Ref Assert-NotContains -Name 'app Directory.Packages.props' -Content $packages -Needle '{LATEST}' From e31f053a38206da453bc03352955014262beaeac Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:28:09 +0200 Subject: [PATCH 05/13] =?UTF-8?q?=F0=9F=93=9A=20document=20native=20MTP=20?= =?UTF-8?q?test=20stack=20in=20readme?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Root documentation surfaces the native MTP test stack with xUnit built-in TRX reporting and the supported SDK requirement so consumers understand the generated test setup before scaffolding. --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 2c37c24..86824ce 100644 --- a/README.md +++ b/README.md @@ -150,8 +150,8 @@ When repo-managed skills author Markdown, each prose paragraph and list item sta | [skill-creator-agnostic](skills/skill-creator-agnostic/SKILL.md) | **⚠️ Deprecated** — no longer maintained and retained only for backward compatibility until **1.0.0**. Do not use it for new skill-authoring work; use Anthropic `skill-creator` together with this repository's `AGENTS.md`. | | [markdown-illustrator](skills/markdown-illustrator/SKILL.md) | Reads a markdown file and answers directly in chat with one document-wide Visual Brief plus one compiled prompt. Infers a compact visual strategy by default, keeps follow-up questions near zero, and only branches when the user explicitly asks for added specificity. | | [git-repo-digest](skills/git-repo-digest/SKILL.md) | Turns any full repository URL into a deterministic digest workspace using the bundled .NET file-based runner `scripts/digest.cs`. Requires explicit `--repo-url`, resolves omitted output paths to `/.bot/digests` and passes that as `--output-root`, maps multiple positional URLs the same way for slash commands, bare pasted URLs, and natural-language requests by treating the first URL as the digest repo and every later URL as repeated `--external-repo-url`, always writes into `{output-root}/{repo-id}/{yyyyMMdd-HHmmssZ}`, accepts repeated curated public consumer repos, derives `{repo-id}`, fixes `result/`, performs shallow git clones, packs local tracked files with the bundled C# packer using `git ls-files`, separates XML evidence into `source.xml`, `tests.xml`, `projects.xml`, editorial `readmes.xml`, and scenario-only `external-usage.xml`, writes package and conceptual overview prompts under `prompts/`, emits public API summaries, engineering signals, evidence indexes, ordered XML chunks, referenced-package evidence maps for aggregate examples, and manifest-backed frontmatter hints, treats previous digest prose as contamination during fresh generation, then guides the agent to fully read the current phase's required evidence before writing package digests and a concept-led `result/Index.md` with YAML frontmatter containing Product-derived overview title metadata, validated documentation URLs resolved from PackageProjectUrl, documentation-host-filtered exact `.nuget//README.md` documentation links including emoji-prefixed Documentation headings and "More documentation..." blocks, DocFX `metadata[].dest` API paths, and source namespace page candidates from `src//**/*.cs`, target frameworks, package/library counts, external links, package-family links, and context glyphs, and validates authored result examples with `--validate-results` as a deterministic API-shape, Codebelt.Extensions.Xunit shape, PascalCase `MethodName_Scenario_ExpectedBehavior` test-method naming, Basic usage quality, and optimized NuGet-backed executable test gate with bounded parallelism. | -| [dotnet-new-lib-slnx](skills/dotnet-new-lib-slnx/SKILL.md) | Scaffold a new .NET NuGet library solution following codebeltnet engineering conventions. Dynamic defaults for TFM/repository metadata, latest-stable NuGet package resolution, tuning projects plus a tooling-based benchmark runner, TFM-aware test environments, strong-name signing, NuGet packaging, DocFX documentation, CI/CD pipeline, and code quality tooling. | -| [dotnet-new-app-slnx](skills/dotnet-new-app-slnx/SKILL.md) | Scaffold a new .NET standalone application solution following codebeltnet engineering conventions. Supports Console, Web, and Worker host families with Startup or Minimal hosting patterns; Web expands into Empty Web, Web API, MVC, or Web App / Razor, plus functional tests and a simplified CI pipeline. | +| [dotnet-new-lib-slnx](skills/dotnet-new-lib-slnx/SKILL.md) | Scaffold a new .NET NuGet library solution following codebeltnet engineering conventions. Dynamic defaults for TFM/repository metadata, latest-stable NuGet package resolution, tuning projects plus a tooling-based benchmark runner, TFM-aware test environments, xUnit v4 / Codebelt v12 tests with native MTP and Codebelt Coverlet coverage, strong-name signing, NuGet packaging, DocFX documentation, CI/CD pipeline, and code quality tooling. | +| [dotnet-new-app-slnx](skills/dotnet-new-app-slnx/SKILL.md) | Scaffold a new .NET standalone application solution following codebeltnet engineering conventions. Supports Console, Web, and Worker host families with Startup or Minimal hosting patterns; Web expands into Empty Web, Web API, MVC, or Web App / Razor, plus xUnit v4 / Codebelt v12 functional tests, native MTP with Codebelt Coverlet coverage, and a simplified CI pipeline. | | [trunk-first-repo](skills/trunk-first-repo/SKILL.md) | Initialize a git repository following [scaled trunk-based development](https://trunkbaseddevelopment.com/#scaled-trunk-based-development). Seeds an empty `main` branch, creates a versioned feature branch (`v0.1.0/init`), confirms configured remotes in its post-init summary, and supports a guarded later `push remote ` mode that checks the feature-branch/empty-main state before pushing `main` ahead of the first feature branch so content still reaches main only through peer-reviewed pull requests. | | [dotnet-strong-name-signing](skills/dotnet-strong-name-signing/SKILL.md) | Generate a strong name key (`.snk`) file for signing .NET assemblies using pure .NET cryptography — no Visual Studio Developer PowerShell or `sn.exe` required. Works in any terminal. Defaults to 1024-bit RSA (matching `sn.exe`), with 2048 and 4096 available as options. | | [git-remote-release](skills/git-remote-release/SKILL.md) | Generate GitHub release notes by summarizing all commits and pull requests between two Git tags or branches in a remote GitHub repository. Accepts a compare URL or separate owner/repo, previous ref, and current ref values; falls back to comparing the current branch against the upstream default branch when no input is provided. Produces a human-friendly `## What's Changed` section whose first summary line begins `This release `, follows it with curated dash bullets using bold lead-ins and actual explanatory prose, optionally inserts supported GitHub alert blocks, preserves exact verified `Sources:` entries with contributor-complete GitHub logins, keeps em-dash bans scoped to authored prose rather than verbatim source titles, and ends with the full changelog compare link. | @@ -571,6 +571,7 @@ Starting a new .NET solution "from scratch" usually means copying from your last - **Structured-input fallback stays consistent** — when a host does not render native form widgets, the scaffold skills now fall back to a deterministic one-field-at-a-time plain-text format instead of improvising the UX - **Explicit host prompts stay on rails** — if you already asked for `Console`, `Worker`, `Web API`, `MVC`, or `Razor`, the scaffold flow should preselect that host choice and move straight to the remaining fields instead of asking you to restate it - **Modern TFM choices** — the .NET scaffold skills compute active target framework quick-picks from the official .NET releases index, offering every supported non-preview LTS and STS channel plus an expanded multi-target preset where applicable +- **Native MTP test stack** — both scaffolds use the live-resolved compatible Codebelt xUnit 12.x and `xunit.v3` 4.x lines, ship root `global.json` selecting `Microsoft.Testing.Platform`, and require a supported non-preview .NET 10+ SDK even for older target runtimes; shared test references use `Codebelt.Coverlet.MTP` instead of legacy Coverlet integrations, include HangDump, and validate real test discovery plus TRX/OpenCover output through xUnit's built-in reporter, preserving the original `Codebelt.Extensions.Xunit.App` package choice without a separate TRX provider or .NET 8-specific workaround - **Latest stable dependencies** — `Directory.Packages.props` is generated from NuGet.org package metadata at scaffold time instead of carrying stale hardcoded NuGet package versions - **Central package management stays authoritative** — app scaffolds keep NuGet versions in `Directory.Packages.props` and do not “repair” restore issues by inlining versions into generated project files - **Deterministic package resolution beats memory** — the app scaffold now ships a NuGet resolver script so agents can fetch current per-package versions instead of guessing from stale remembered examples From 01337ab2c07c7c9ba62d0930cbab7429c5d25b83 Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:55:08 +0200 Subject: [PATCH 06/13] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20refactor=20dotnet-ne?= =?UTF-8?q?w=20skills=20to=20vendor-neutral=20agent=20guidance?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace the vendor-specific instruction file with root AGENTS.md as the single generated agent-instruction source while preserving the complete testing, coverage, benchmarking, XML documentation, and bot-workspace contract without summarization. --- skills/dotnet-new-app-slnx/SKILL.md | 5 +++-- skills/dotnet-new-app-slnx/evals/evals.json | 3 +++ skills/dotnet-new-app-slnx/references/app.md | 7 +++++-- skills/dotnet-new-lib-slnx/SKILL.md | 5 +++-- skills/dotnet-new-lib-slnx/evals/evals.json | 5 ++++- skills/dotnet-new-lib-slnx/references/library.md | 5 ++++- 6 files changed, 22 insertions(+), 8 deletions(-) diff --git a/skills/dotnet-new-app-slnx/SKILL.md b/skills/dotnet-new-app-slnx/SKILL.md index d881d77..9f39d81 100644 --- a/skills/dotnet-new-app-slnx/SKILL.md +++ b/skills/dotnet-new-app-slnx/SKILL.md @@ -18,6 +18,8 @@ description: > | **Raw base URL** | `https://raw.githubusercontent.com/codebeltnet/agentic/main/skills/dotnet-new-app-slnx/assets/shared` | | **Asset manifest** | `assets/shared.manifest.json` | +Generate root `AGENTS.md` as the single, vendor-neutral agent-instruction source. Preserve its project-specific coding, testing, coverage, benchmarking, XML documentation, and `.bot/` guidance; do not generate a separate vendor-specific instruction file. Preserve the complete governance contract, including examples, rationale, benefits, alternatives, and applicability conditions. Do not summarize it; merge only duplicates that lose no information. + This metadata is the single source of truth for restoring any file the installer may have dropped. Use it immediately — do not spend cycles confirming absence multiple ways first. Scaffold new .NET standalone application solutions following the codebeltnet engineering conventions — the same pattern used across [codebeltnet](https://github.com/codebeltnet). Produces a fully wired solution with CI pipeline, centralized build config, semantic versioning, code quality tooling, and proper folder structure. @@ -217,8 +219,7 @@ After generating, verify: - [ ] Root governance docs exist: `README.md`, `CHANGELOG.md`, `.github/CODE_OF_CONDUCT.md`, `.github/CONTRIBUTING.md` - [ ] Authored Markdown paragraphs and list items have no fixed-width hard wraps - [ ] `.editorconfig` is present with file-scoped namespace enforcement -- [ ] `AGENTS.md` references `.bot/` and coding guidelines -- [ ] `.github/copilot-instructions.md` has project-specific patterns +- [ ] Root `AGENTS.md` is the single generated agent-instruction source, covering `.bot/`, coding standards, test conventions, code coverage, benchmarking, and XML documentation - [ ] `.bot/` folder exists and is listed in `.gitignore` - [ ] `.bot/README.md` exists in the generated repo and came from the shared asset template - [ ] `testenvironments.json` uses the major-tag `codebeltnet/ubuntu-testrunner:{major}` convention for the selected target framework diff --git a/skills/dotnet-new-app-slnx/evals/evals.json b/skills/dotnet-new-app-slnx/evals/evals.json index 66ddfa5..20851eb 100644 --- a/skills/dotnet-new-app-slnx/evals/evals.json +++ b/skills/dotnet-new-app-slnx/evals/evals.json @@ -16,6 +16,9 @@ "Keeps package versions centralized in Directory.Packages.props instead of inlining Version attributes into generated project files", "Keeps target-framework selection centralized in the generated root Directory.Build.props instead of adding TargetFramework directly to generated project files", "Generates testenvironments.json instead of silently omitting the shared test environment asset", + "Generates root AGENTS.md as the only agent-instruction file, preserving test, coverage, benchmark, XML documentation, and bot-workspace guidance without vendor-specific frontmatter", + "Preserves all nine complete governance code examples plus namespace/project mappings, the full Public Facade Testing pattern, coverage rationale and alternatives, and benchmark/XML documentation applicability without summarizing them", + "Does not emit or recommend a separate vendor-specific instruction file; finalized bot-workspace guidance points only to root AGENTS.md", "Names the generated solution file DemoApp.slnx instead of lowercasing it to demoapp.slnx", "Still generates the solution file and functional test project even for a single-host Worker scaffold", "Resolves worker package versions from NuGet instead of reusing stale remembered version numbers from prior runs", diff --git a/skills/dotnet-new-app-slnx/references/app.md b/skills/dotnet-new-app-slnx/references/app.md index ec4ca9e..bb0aeb2 100644 --- a/skills/dotnet-new-app-slnx/references/app.md +++ b/skills/dotnet-new-app-slnx/references/app.md @@ -6,6 +6,7 @@ Slim guide for runnable applications. All file templates live in `assets/app/`. ``` . +├── AGENTS.md ├── src/ │ └── {ROOT_NAMESPACE}.{AppType}/ │ ├── {ROOT_NAMESPACE}.{AppType}.csproj @@ -48,9 +49,11 @@ Do not cherry-pick only the files that feel essential. The shared scaffold contr - `.bot/README.md` - `.github/CODE_OF_CONDUCT.md` - `.github/CONTRIBUTING.md` -- `.github/copilot-instructions.md` -- `.github/dependabot.yml` +- `.github/dependabot.yml` +- `.github/workflows/ci-pipeline.yml` +Root `AGENTS.md` is the single generated agent-instruction source. It contains coding standards, test conventions, coverage rules, optional benchmarking guidance, XML documentation conventions, and `.bot/` workspace guidance. + Treat missing files from this shared inventory as scaffold defects, not optional omissions. ## Testing Approach diff --git a/skills/dotnet-new-lib-slnx/SKILL.md b/skills/dotnet-new-lib-slnx/SKILL.md index d37fda7..a60ee19 100644 --- a/skills/dotnet-new-lib-slnx/SKILL.md +++ b/skills/dotnet-new-lib-slnx/SKILL.md @@ -18,6 +18,8 @@ description: > | **Raw base URL** | `https://raw.githubusercontent.com/codebeltnet/agentic/main/skills/dotnet-new-lib-slnx/assets/shared` | | **Asset manifest** | `assets/shared.manifest.json` | +Generate root `AGENTS.md` as the single, vendor-neutral agent-instruction source. Preserve its project-specific coding, testing, coverage, benchmarking, XML documentation, and `.bot/` guidance; do not generate a separate vendor-specific instruction file. Preserve the complete governance contract, including examples, rationale, benefits, alternatives, and applicability conditions. Do not summarize it; merge only duplicates that lose no information. + This metadata is the single source of truth for restoring any file the installer may have dropped. Use it immediately — do not spend cycles confirming absence multiple ways first. Scaffold new .NET NuGet library solutions following the codebeltnet engineering conventions — the same pattern used across [codebeltnet](https://github.com/codebeltnet). Produces a fully wired solution with multi-target framework support, strong-name signing, NuGet packaging, DocFX documentation, CI pipeline, centralized build config, semantic versioning, and code quality tooling. @@ -199,8 +201,7 @@ After generating, verify: - [ ] `.docfx/docfx.json` lists all source projects and has correct metadata - [ ] `.editorconfig` is present, sets `charset = utf-8`, and keeps file-scoped namespace enforcement - [ ] Generated text files do not contain common mojibake markers such as `—`, `–`, `â€`, or `�` -- [ ] `AGENTS.md` references `.bot/` and coding guidelines -- [ ] `.github/copilot-instructions.md` has project-specific patterns +- [ ] Root `AGENTS.md` is the single generated agent-instruction source, covering `.bot/`, coding standards, test conventions, code coverage, benchmarking, and XML documentation - [ ] `.bot/` folder exists and is listed in `.gitignore` - [ ] `.bot/README.md` exists in the generated repo and came from the shared asset template, not from a synthetic `.gitkeep` fallback - [ ] Every file listed in `assets/shared.manifest.json` exists in the generated repo at its declared relative path (this covers all dotfiles and dotfolders) diff --git a/skills/dotnet-new-lib-slnx/evals/evals.json b/skills/dotnet-new-lib-slnx/evals/evals.json index c883ba0..b74bf8e 100644 --- a/skills/dotnet-new-lib-slnx/evals/evals.json +++ b/skills/dotnet-new-lib-slnx/evals/evals.json @@ -38,7 +38,10 @@ "expectations": [ "Collects repository_url before package_project_url", "Uses COMPANY_OR_PERSON in DocFX metadata instead of a separate COMPANY placeholder", - "Copies assets/shared/.bot/README.md into the generated repo" + "Copies assets/shared/.bot/README.md into the generated repo", + "Generates root AGENTS.md as the only agent-instruction file, preserving test, coverage, benchmark, XML documentation, and bot-workspace guidance without vendor-specific frontmatter", + "Preserves all nine complete governance code examples plus namespace/project mappings, the full Public Facade Testing pattern, coverage rationale and alternatives, and benchmark/XML documentation applicability without summarizing them", + "Does not emit or recommend a separate vendor-specific instruction file; finalized bot-workspace guidance points only to root AGENTS.md" ] }, { diff --git a/skills/dotnet-new-lib-slnx/references/library.md b/skills/dotnet-new-lib-slnx/references/library.md index 53a7148..b92e7ba 100644 --- a/skills/dotnet-new-lib-slnx/references/library.md +++ b/skills/dotnet-new-lib-slnx/references/library.md @@ -2,12 +2,15 @@ Slim guide for scaffolding a NuGet library solution. All file templates live in `assets/library/`. +Copy every shared file listed in `assets/shared.manifest.json` to the generated root. Root `AGENTS.md` is the single generated agent-instruction source for coding standards, test conventions, coverage, benchmarking, XML documentation, and `.bot/` workspace guidance. + --- ## Folder Structure ``` . +├── AGENTS.md ├── .nuget/ │ └── {PROJECT_NAME}/ │ ├── PackageReleaseNotes.txt @@ -31,7 +34,7 @@ Slim guide for scaffolding a NuGet library solution. All file templates live in │ └── (benchmark reports and tuning output) ├── Directory.Build.props ├── {REPO_SLUG}.slnx -└── (shared skeleton files — see shared-files.md) +└── (shared skeleton files — see assets/shared.manifest.json) ``` The tree is shown **relative to the current working directory**. Generate these files directly in the folder the user is already in; do not create an extra solution-named wrapper folder unless they explicitly ask for one. From 9048ab41fda6c93259921b86420db41784e690ca Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:55:16 +0200 Subject: [PATCH 07/13] =?UTF-8?q?=F0=9F=94=A7=20update=20shared=20scaffold?= =?UTF-8?q?=20to=20root=20AGENTS.md=20contract?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expand the shared AGENTS.md governance contract, remove the Copilot-specific instruction file from the manifest and disk, and retarget bot-workspace guidance to root AGENTS.md. --- .../assets/shared.manifest.json | 1 - .../assets/shared/.bot/README.md | 2 +- .../shared/.github/copilot-instructions.md | 353 ---------------- .../assets/shared/AGENTS.md | 377 +++++++++++++++++- .../assets/shared.manifest.json | 1 - .../assets/shared/.bot/README.md | 2 +- .../shared/.github/copilot-instructions.md | 353 ---------------- .../assets/shared/AGENTS.md | 373 ++++++++++++++++- 8 files changed, 740 insertions(+), 722 deletions(-) delete mode 100644 skills/dotnet-new-app-slnx/assets/shared/.github/copilot-instructions.md delete mode 100644 skills/dotnet-new-lib-slnx/assets/shared/.github/copilot-instructions.md diff --git a/skills/dotnet-new-app-slnx/assets/shared.manifest.json b/skills/dotnet-new-app-slnx/assets/shared.manifest.json index 7c37d46..da2df3b 100644 --- a/skills/dotnet-new-app-slnx/assets/shared.manifest.json +++ b/skills/dotnet-new-app-slnx/assets/shared.manifest.json @@ -11,7 +11,6 @@ ".gitattributes", ".github/CODE_OF_CONDUCT.md", ".github/CONTRIBUTING.md", - ".github/copilot-instructions.md", ".github/dependabot.yml", ".github/workflows/ci-pipeline.yml", ".gitignore", diff --git a/skills/dotnet-new-app-slnx/assets/shared/.bot/README.md b/skills/dotnet-new-app-slnx/assets/shared/.bot/README.md index 2cfca89..397ae8c 100644 --- a/skills/dotnet-new-app-slnx/assets/shared/.bot/README.md +++ b/skills/dotnet-new-app-slnx/assets/shared/.bot/README.md @@ -7,4 +7,4 @@ This folder is reserved for local-only AI working material such as: - design alternatives - temporary agent state -Keep this folder out of source control. Move only finalized, non-confidential guidance into `AGENTS.md` or `.github/copilot-instructions.md`. +Keep this folder out of source control. Move only finalized, non-confidential guidance into root `AGENTS.md`. diff --git a/skills/dotnet-new-app-slnx/assets/shared/.github/copilot-instructions.md b/skills/dotnet-new-app-slnx/assets/shared/.github/copilot-instructions.md deleted file mode 100644 index cb908d6..0000000 --- a/skills/dotnet-new-app-slnx/assets/shared/.github/copilot-instructions.md +++ /dev/null @@ -1,353 +0,0 @@ ---- -description: 'Writing Unit Tests' -applyTo: "**/*.{cs,csproj}" ---- - -# Writing Unit Tests -This document provides instructions for writing unit tests for a project/solution. Please follow these guidelines to ensure consistency and maintainability. - -## 1. Base Class - -**Always inherit from the `Test` base class** for all unit test classes. This ensures consistent setup, teardown, and output handling across all tests. - -> Important: Do NOT add `using Xunit.Abstractions`. xUnit v4 (the `xunit.v3` 4.x package line) does not expose that namespace; including it is incorrect and will cause compilation errors. Use the `Codebelt.Extensions.Xunit` Test base class and `using Xunit;` as shown in the examples below. If you need access to test output, rely on the Test base class (which accepts the appropriate output helper) rather than importing `Xunit.Abstractions`. - -```csharp -using Codebelt.Extensions.Xunit; -using Xunit; - -namespace Your.Namespace -{ - public class YourTestClass : Test - { - public YourTestClass(ITestOutputHelper output) : base(output) - { - } - - // Your tests here - } -} -``` - -## 2. Test Method Attributes - -- Use `[Fact]` for standard unit tests. -- Use `[Theory]` with `[InlineData]` or other data sources for parameterized tests. - -## 3. Naming Conventions - -- **Test classes**: End with `Test` (e.g., `DateSpanTest`). -- **Test methods**: Use descriptive names that state the expected behavior (e.g., `ShouldReturnTrue_WhenConditionIsMet`). - -## 4. Assertions - -- Use `Assert` methods from xUnit for all assertions. -- Prefer explicit and expressive assertions (e.g., `Assert.Equal`, `Assert.NotNull`, `Assert.Contains`). - -## 5. File and Namespace Organization - -- Place test files in the appropriate test project and folder structure. -- Use namespaces that mirror the source code structure. The namespace of a test file MUST match the namespace of the System Under Test (SUT). Do NOT append ".Tests", ".Benchmarks" or similar suffixes to the namespace. Only the assembly/project name should indicate that the file is a test/benchmark. - - Example: If the SUT class is declared as: - ```csharp - namespace YourProject.Foo.Bar - { - public class Zoo { /* ... */ } - } - ``` -then the corresponding unit test class must use the exact same namespace: - ```csharp - namespace YourProject.Foo.Bar - { - public class ZooTest : Test { /* ... */ } - } - ``` - - Do NOT use: - ```csharp - namespace YourProject.Foo.Bar.Tests { /* ... */ } // ❌ - namespace YourProject.Foo.Bar.Benchmarks { /* ... */ } // ❌ - ``` - - The unit tests for the YourProject.Foo assembly live in the YourProject.Foo.Tests assembly. - - The functional tests for the YourProject.Foo assembly live in the YourProject.Foo.FunctionalTests assembly. - - Test class names end with Test and live in the same namespace as the class being tested, e.g., the unit tests for the Boo class that resides in the YourProject.Foo assembly would be named BooTest and placed in the YourProject.Foo namespace in the YourProject.Foo.Tests assembly. - - Modify the associated .csproj file to override the root namespace so the compiled namespace matches the SUT. Example: - ```xml - - YourProject.Foo - - ``` -- When generating test scaffolding automatically, resolve the SUT's namespace from the source file (or project/assembly metadata) and use that exact namespace in the test file header. - -- Notes: - - This rule ensures type discovery and XML doc links behave consistently and reduces confusion when reading tests. - - Keep folder structure aligned with the production code layout to make locating SUT <-> test pairs straightforward. - -## 6. Example Test - -```csharp -using System; -using Codebelt.Extensions.Xunit; -using Xunit; - -namespace YourProject -{ - public class SampleTest : Test - { - public SampleTest(ITestOutputHelper output) : base(output) - { - } - - [Fact] - public void ShouldReturnExpectedResult_WhenGivenValidInput() - { - var sut = new SampleClass(); - - var result = sut.Process("input"); - - Assert.NotNull(result); - Assert.Equal("expected", result.Value); - - TestOutput.WriteLine(result.ToString()); - } - - [Theory] - [InlineData(1, "one")] - [InlineData(2, "two")] - public void ShouldMapCorrectly_WhenGivenNumber(int input, string expected) - { - var sut = new SampleClass(); - - var result = sut.MapToString(input); - - Assert.Equal(expected, result); - } - } -} -``` - -## 7. Additional Guidelines - -- Keep tests focused and isolated. -- Do not rely on external systems except for xUnit itself and Codebelt.Extensions.Xunit (and derived from this). -- Ensure tests are deterministic and repeatable. - -## 8. Test Doubles - -- Preferred test doubles include dummies, fakes, stubs and spies if and when the design allows it. -- Under special circumstances, mock can be used (using Moq library). -- Before overriding methods, verify that the method is virtual or abstract; this rule also applies to mocks. -- Never mock IMarshaller; always use a new instance of JsonMarshaller. - -## 9. Avoid `InternalsVisibleTo` in Tests - -- **Do not** use `InternalsVisibleTo` to access internal types or members from test projects. -- Prefer **indirect testing via public APIs** that depend on the internal implementation (public facades, public extension methods, or other public entry points). - -### Preferred Pattern - -**Pattern name:** Public Facade Testing (also referred to as *Public API Proxy Testing*) - -**Description:** Internal classes and methods must be validated by exercising the public API that consumes them. Tests should assert observable behavior exposed by the public surface rather than targeting internal implementation details directly. - -### Example Mapping - -- **Internal helper:** `DelimitedString` (internal static class) -- **Public API:** `TestOutputHelperExtensions.WriteLines()` (public extension method) -- **Test strategy:** Write tests for `WriteLines()` and verify its public behavior. The internal call to `DelimitedString.Create()` is exercised implicitly. - -### Benefits - -- Avoids exposing internal types to test assemblies. -- Ensures tests reflect real-world usage patterns. -- Maintains strong encapsulation and a clean public API. -- Tests remain resilient to internal refactoring as long as public behavior is preserved. - -### When to Apply - -- Internal logic is fully exercised through existing public APIs. -- Public entry points provide sufficient coverage of internal code paths. -- The internal implementation exists solely as a helper or utility for public-facing functionality. - -## 10. ExcludeFromCodeCoverage Prohibition - -**Do not use `ExcludeFromCodeCoverage` attribute on any code.** This includes: - -- Test classes or test methods -- Production code -- Configuration code -- Any other code path - -### Rationale - -- Excluding code from coverage hides gaps and creates false confidence in test completeness. -- If a code path cannot or should not be tested, refactor the code to eliminate that path rather than hiding it from metrics. -- Every executable line should be covered by tests or be genuinely unreachable (dead code to be removed). - -### Alternative Approaches - -- **Untestable code paths**: Refactor to separate concerns and eliminate the untestable path. -- **External dependencies**: Use test doubles (fakes, stubs, spies) instead of excluding from coverage. -- **Configuration-only code**: Move to configuration files or extract into testable methods. -- **Generated or third-party code**: These should not be in the primary codebase; use NuGet packages or dedicated vendor folders if necessary. - ---- -description: 'Writing Performance Tests' -applyTo: "tuning/**, **/*Benchmark*.cs" ---- - -# Writing Performance Tests -This document provides guidance for writing performance tests (benchmarks) for a project/solution using BenchmarkDotNet. Follow these guidelines to keep benchmarks consistent, readable, and comparable. - -## 1. Naming and Placement - -- Place micro- and component-benchmarks under the `tuning/` folder or in projects named `*.Benchmarks`. -- Place benchmark files in the appropriate benchmark project and folder structure. -- Use namespaces that mirror the source code structure, e.g. do not suffix with `Benchmarks`. -Namespace rule: DO NOT append `.Benchmarks` to the namespace. Benchmarks must live in the same namespace as the production assembly. Example: if the production assembly uses `namespace YourProject.Security.Cryptography`, the benchmark file should also use: - ``` - namespace YourProject.Security.Cryptography - { - public class Sha512256Benchmark { /* ... */ } - } - ``` -The class name must end with `Benchmark`, but the namespace must match the assembly (no `.Benchmarks` suffix). - The benchmarks for the YourProject.Bar assembly live in the YourProject.Bar.Benchmarks assembly. - Benchmark class names end with Benchmark and live in the same namespace as the class being measured, e.g., the benchmarks for the Zoo class that resides in the YourProject.Bar assembly would be named ZooBenchmark and placed in the YourProject.Bar namespace in the YourProject.Bar.Benchmarks assembly. - Modify the associated .csproj file to override the root namespace, e.g., YourProject.Bar. - -## 2. Attributes and Configuration - -- Use `BenchmarkDotNet` attributes to express intent and collect relevant metrics: - - `[MemoryDiagnoser]` to capture memory allocations. - - `[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]` to group related benchmarks. - - `[Params]` for input sizes or variations to exercise multiple scenarios. - - `[GlobalSetup]` for one-time initialization that's not part of measured work. - - `[Benchmark]` on methods representing measured operations; consider `Baseline = true` and `Description` to improve report clarity. -- Keep benchmark configuration minimal and explicit; prefer in-class attributes over large shared configs unless re-used widely. - -## 3. Structure and Best Practices - -- Keep benchmarks focused: each `Benchmark` method should measure a single logical operation. -- Avoid doing expensive setup work inside a measured method; use `[GlobalSetup]`, `[IterationSetup]`, or cached fields instead. -- Use `Params` to cover micro, mid and macro input sizes (for example: small, medium, large) and verify performance trends across them. -- Use small, deterministic data sets and avoid external systems (network, disk, DB). If external systems are necessary, mark them clearly and do not include them in CI benchmark runs by default. -- Capture results that are meaningful: time, allocations, and if needed custom counters. Prefer `MemoryDiagnoser` and descriptive `Description` values. - -## 4. Naming Conventions for Methods - -- Method names should be descriptive and indicate the scenario, e.g., `Parse_Short`, `ComputeHash_Large`. -- When comparing implementations, mark one method with `Baseline = true` and use similar names so reports are easy to read. - -## 5. Example Benchmark - -```csharp -using BenchmarkDotNet.Attributes; -using BenchmarkDotNet.Configs; - -namespace YourProject -{ - [MemoryDiagnoser] - [GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] - public class SampleOperationBenchmark - { - [Params(8, 256, 4096)] - public int Count { get; set; } - - private byte[] _payload; - - [GlobalSetup] - public void Setup() - { - _payload = new byte[Count]; - } - - [Benchmark(Baseline = true, Description = "Operation - baseline")] - public int Operation_Baseline() => SampleOperation.Process(_payload); - - [Benchmark(Description = "Operation - optimized")] - public int Operation_Optimized() => SampleOperation.ProcessOptimized(_payload); - } -} -``` - -## 6. Reporting and CI - -- Benchmarks are primarily for local and tuning runs; be cautious about running heavy BenchmarkDotNet workloads in CI. Prefer targeted runs or harnesses for CI where appropriate. -- Keep benchmark projects isolated (e.g., `tuning/*.csproj`) so they don't affect package builds or production artifacts. - -## 7. Additional Guidelines - -- Keep benchmarks readable and well-documented; add comments explaining non-obvious choices. -- If a benchmark exposes regressions or optimizations, add a short note in the benchmark file referencing the relevant issue or PR. -- For any shared helpers for benchmarking, prefer small utility classes inside the `tuning` projects rather than cross-cutting changes to production code. - -For further examples, refer to the benchmark files under the `tuning/` folder. - ---- -description: 'Writing XML documentation' -applyTo: "**/*.cs" ---- - -# Writing XML Documentation -This document provides instructions for writing XML documentation. - -## 1. Documentation Style - -- Use the same documentation style as found throughout the codebase. -- Add XML doc comments to all public and protected classes, methods, properties, and constructors. -- Use `` for type and member descriptions. -- Use `` for method parameters. -- Use `` for return values. -- Use `` for generic type parameters. -- Use `` for additional context, especially default property values using ``. -- Use `` to reference related types and interfaces. -- Use `` inline to reference types, members, and parameters. -- Use `` to document thrown exceptions. - -## 2. Example - -```csharp -namespace YourProject -{ - /// - /// Provides utility methods for processing data. - /// - /// The type of the configured options. - /// - public class DataProcessor where TOptions : class, new() - { - /// - /// Initializes a new instance of the class. - /// - /// The which may be configured. - public DataProcessor(Action setup) - { - Options = setup != null ? Configure(setup) : new TOptions(); - } - - /// - /// Gets the configured options of this instance. - /// - /// The configured options of this instance. - public TOptions Options { get; } - - /// - /// Processes the specified and returns the result. - /// - /// The input data to process. - /// A containing the processed result. - /// - /// is null. - /// - public string Process(string input) - { - if (input == null) throw new ArgumentNullException(nameof(input)); - return input.ToUpperInvariant(); - } - } -} -``` - -## 3. Additional Guidelines - -- Keep descriptions concise but informative. -- Use consistent phrasing: "Gets or sets", "Initializes a new instance of", "Provides", "Represents". -- Reference parameter names with `` in descriptions. -- For options/settings classes, document default values in `` using a table format. diff --git a/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md b/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md index d5b7ca1..cedb277 100644 --- a/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md +++ b/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md @@ -1,6 +1,6 @@ # Agent Instructions for {SOLUTION_NAME} -This document provides guidance for AI agents working in this repository. +Root `AGENTS.md` is the canonical, vendor-neutral instruction contract for coding agents working in this repository. ## Project Overview @@ -12,8 +12,6 @@ This document provides guidance for AI agents working in this repository. - **Top-level statements:** Not allowed (enforced via `.editorconfig`) - **Language version:** Always use the latest C# features (`LangVersion=latest`) - **Nullable:** Enable nullable reference types in all new code -- **XML documentation:** All public APIs must have XML documentation comments -- **Testing:** Use xUnit v4 (`xunit.v3` 4.x packages) with Codebelt.Extensions.Xunit.App v12 base classes ## Markdown Prose Formatting @@ -23,12 +21,14 @@ Do not hard-wrap prose to a fixed column width. Keep paragraphs and Markdown lis - `src/` — Production source code - `test/` — Functional tests (project names end with `FunctionalTests`) -- `.github/` — CI/CD workflows, contributing guidelines, Copilot instructions +- `.github/` — CI/CD workflows and contributing guidelines ## Test Conventions -- Test project names must end with `FunctionalTests` (e.g. `{ROOT_NAMESPACE}.Console.FunctionalTests`) -- Tests are **functional tests** that exercise the running application as a whole +Use xUnit v4 (`xunit.v3` 4.x packages) with Codebelt.Extensions.Xunit.App v12 base classes. Functional host tests retain the appropriate managed application fixtures; the unit-test rules below apply to tests of individual classes. + +- Scaffolded test project names must end with `FunctionalTests` (e.g. `{ROOT_NAMESPACE}.Console.FunctionalTests`) +- Scaffolded tests are **functional tests** that exercise the running application as a whole - Test classes should inherit from the appropriate base class in `Codebelt.Extensions.Xunit` - Use `Microsoft.Testing.Platform` as the test runner (root `global.json` selects `test.runner`; `UseMicrosoftTestingPlatformRunner=true` enables project integration) - Use a supported non-preview .NET 10+ SDK even for older target runtimes @@ -37,6 +37,369 @@ Do not hard-wrap prose to a fixed column width. Keep paragraphs and Markdown lis - Use managed application fixtures with deferred entrypoint-owned startup, not removed blocking application fixtures - All tests are executable (`OutputType=Exe`) +## Unit Testing + +The following rules apply to unit-test source files and unit-test projects. The file and namespace organization rules also describe the relationship between unit-test and functional-test assemblies. + +This document provides instructions for writing unit tests for a project/solution. Please follow these guidelines to ensure consistency and maintainability. + +### Base Class + +**Always inherit from the `Test` base class** for all unit test classes. This ensures consistent setup, teardown, and output handling across all tests. + +> Important: Do NOT add `using Xunit.Abstractions`. xUnit v4 (the `xunit.v3` 4.x package line) does not expose that namespace; including it is incorrect and will cause compilation errors. Use the `Codebelt.Extensions.Xunit` Test base class and `using Xunit;` as shown in the examples below. If you need access to test output, rely on the Test base class (which accepts the appropriate output helper) rather than importing `Xunit.Abstractions`. + +```csharp +using Codebelt.Extensions.Xunit; +using Xunit; + +namespace Your.Namespace; + +public class YourTestClass : Test +{ + public YourTestClass(ITestOutputHelper output) : base(output) + { + } + + // Your tests here +} +``` + +### Test Method Attributes + +- Use `[Fact]` for standard unit tests. +- Use `[Theory]` with `[InlineData]` or other data sources for parameterized tests. + +### Test Naming Conventions + +- **Test classes**: End with `Test` (e.g., `DateSpanTest`). +- **Test methods**: Use descriptive names that state the expected behavior (e.g., `ShouldReturnTrue_WhenConditionIsMet`). + +### Assertions + +- Use `Assert` methods from xUnit for all assertions. +- Prefer explicit and expressive assertions (e.g., `Assert.Equal`, `Assert.NotNull`, `Assert.Contains`). + +### File and Namespace Organization + +- Place test files in the appropriate test project and folder structure. +- Use namespaces that mirror the source code structure. The namespace of a test file MUST match the namespace of the System Under Test (SUT). Do NOT append ".Tests", ".Benchmarks" or similar suffixes to the namespace. Only the assembly/project name should indicate that the file is a test/benchmark. + +Example: If the SUT class is declared as: + +```csharp +namespace YourProject.Foo.Bar; + +public class Zoo { /* ... */ } +``` + +then the corresponding unit test class must use the exact same namespace: + +```csharp +namespace YourProject.Foo.Bar; + +public class ZooTest : Test { /* ... */ } +``` + +Do NOT use: + +```csharp +namespace YourProject.Foo.Bar.Tests; /* ... */ // ❌ +namespace YourProject.Foo.Bar.Benchmarks; /* ... */ // ❌ +``` + +- The unit tests for the YourProject.Foo assembly live in the YourProject.Foo.Tests assembly. +- The functional tests for the YourProject.Foo assembly live in the YourProject.Foo.FunctionalTests assembly. +- Test class names end with Test and live in the same namespace as the class being tested, e.g., the unit tests for the Boo class that resides in the YourProject.Foo assembly would be named BooTest and placed in the YourProject.Foo namespace in the YourProject.Foo.Tests assembly. +- Modify the associated .csproj file to override the root namespace so the compiled namespace matches the SUT. Example: + +```xml + + YourProject.Foo + +``` + +- When generating test scaffolding automatically, resolve the SUT's namespace from the source file (or project/assembly metadata) and use that exact namespace in the test file header. + +- Notes: + - This rule ensures type discovery and XML doc links behave consistently and reduces confusion when reading tests. + - Keep folder structure aligned with the production code layout to make locating SUT <-> test pairs straightforward. + +Do not append `.FunctionalTests` to test namespaces either; the same SUT namespace equality rule applies. + +### Example Test + +```csharp +using System; +using Codebelt.Extensions.Xunit; +using Xunit; + +namespace YourProject; + +public class SampleTest : Test +{ + public SampleTest(ITestOutputHelper output) : base(output) + { + } + + [Fact] + public void ShouldReturnExpectedResult_WhenGivenValidInput() + { + var sut = new SampleClass(); + + var result = sut.Process("input"); + + Assert.NotNull(result); + Assert.Equal("expected", result.Value); + + TestOutput.WriteLine(result.ToString()); + } + + [Theory] + [InlineData(1, "one")] + [InlineData(2, "two")] + public void ShouldMapCorrectly_WhenGivenNumber(int input, string expected) + { + var sut = new SampleClass(); + + var result = sut.MapToString(input); + + Assert.Equal(expected, result); + } +} +``` + +### Additional Unit Test Guidelines + +- Keep tests focused and isolated. +- Do not rely on external systems except for xUnit itself and Codebelt.Extensions.Xunit (and derived from this). +- Ensure tests are deterministic and repeatable. + +### Test Doubles + +- Preferred test doubles include dummies, fakes, stubs and spies if and when the design allows it. +- Under special circumstances, mock can be used (using Moq library). +- Before overriding methods, verify that the method is virtual or abstract; this rule also applies to mocks. +- Never mock IMarshaller; always use a new instance of JsonMarshaller. + +### Public Facade Testing + +- **Do not** use `InternalsVisibleTo` to access internal types or members from test projects. +- Prefer **indirect testing via public APIs** that depend on the internal implementation (public facades, public extension methods, or other public entry points). + +#### Preferred Pattern + +**Pattern name:** Public Facade Testing (also referred to as *Public API Proxy Testing*) + +**Description:** Internal classes and methods must be validated by exercising the public API that consumes them. Tests should assert observable behavior exposed by the public surface rather than targeting internal implementation details directly. + +#### Example Mapping + +- **Internal helper:** `DelimitedString` (internal static class) +- **Public API:** `TestOutputHelperExtensions.WriteLines()` (public extension method) +- **Test strategy:** Write tests for `WriteLines()` and verify its public behavior. The internal call to `DelimitedString.Create()` is exercised implicitly. + +#### Benefits + +- Avoids exposing internal types to test assemblies. +- Ensures tests reflect real-world usage patterns. +- Maintains strong encapsulation and a clean public API. +- Tests remain resilient to internal refactoring as long as public behavior is preserved. + +#### When to Apply + +- Internal logic is fully exercised through existing public APIs. +- Public entry points provide sufficient coverage of internal code paths. +- The internal implementation exists solely as a helper or utility for public-facing functionality. + +## Code Coverage + +**Do not use `ExcludeFromCodeCoverage` attribute on any code.** This includes: + +- Test classes or test methods +- Production code +- Configuration code +- Any other code path + +### Coverage Rationale + +- Excluding code from coverage hides gaps and creates false confidence in test completeness. +- If a code path cannot or should not be tested, refactor the code to eliminate that path rather than hiding it from metrics. +- Every executable line should be covered by tests or be genuinely unreachable (dead code to be removed). + +### Coverage Alternatives + +- **Untestable code paths**: Refactor to separate concerns and eliminate the untestable path. +- **External dependencies**: Use test doubles (fakes, stubs, spies) instead of excluding from coverage. +- **Configuration-only code**: Move to configuration files or extract into testable methods. +- **Generated or third-party code**: These should not be in the primary codebase; use NuGet packages or dedicated vendor folders if necessary. + +Do not append .FunctionalTests to test namespaces either; the same SUT namespace equality rule applies. + +## Benchmarking + +Benchmarking is optional for application repositories. The following rules apply when benchmarks are added under `tuning/`, in `*.Benchmarks` projects, or in `*Benchmark*.cs` source files. + +This document provides guidance for writing performance tests (benchmarks) for a project/solution using BenchmarkDotNet. Follow these guidelines to keep benchmarks consistent, readable, and comparable. + +### Benchmark Naming and Placement + +- Place micro- and component-benchmarks under the `tuning/` folder or in projects named `*.Benchmarks`. +- Place benchmark files in the appropriate benchmark project and folder structure. +- Use namespaces that mirror the source code structure, e.g. do not suffix with `Benchmarks`. + +Namespace rule: DO NOT append `.Benchmarks` to the namespace. Benchmarks must live in the same namespace as the production assembly. Example: if the production assembly uses `namespace YourProject.Security.Cryptography`, the benchmark file should also use: + +``` +namespace YourProject.Security.Cryptography; + +public class Sha512256Benchmark { /* ... */ } +``` + +The class name must end with `Benchmark`, but the namespace must match the assembly (no `.Benchmarks` suffix). + +- The benchmarks for the YourProject.Bar assembly live in the YourProject.Bar.Benchmarks assembly. +- Benchmark class names end with Benchmark and live in the same namespace as the class being measured, e.g., the benchmarks for the Zoo class that resides in the YourProject.Bar assembly would be named ZooBenchmark and placed in the YourProject.Bar namespace in the YourProject.Bar.Benchmarks assembly. +- Modify the associated .csproj file to override the root namespace, e.g., `YourProject.Bar`. + +### Attributes and Configuration + +- Use `BenchmarkDotNet` attributes to express intent and collect relevant metrics: + - `[MemoryDiagnoser]` to capture memory allocations. + - `[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]` to group related benchmarks. + - `[Params]` for input sizes or variations to exercise multiple scenarios. + - `[GlobalSetup]` for one-time initialization that's not part of measured work. + - `[Benchmark]` on methods representing measured operations; consider `Baseline = true` and `Description` to improve report clarity. +- Keep benchmark configuration minimal and explicit; prefer in-class attributes over large shared configs unless re-used widely. + +### Structure and Best Practices + +- Keep benchmarks focused: each `Benchmark` method should measure a single logical operation. +- Avoid doing expensive setup work inside a measured method; use `[GlobalSetup]`, `[IterationSetup]`, or cached fields instead. +- Use `Params` to cover micro, mid and macro input sizes (for example: small, medium, large) and verify performance trends across them. +- Use small, deterministic data sets and avoid external systems (network, disk, DB). If external systems are necessary, mark them clearly and do not include them in CI benchmark runs by default. +- Capture results that are meaningful: time, allocations, and if needed custom counters. Prefer `MemoryDiagnoser` and descriptive `Description` values. + +### Benchmark Method Naming + +- Method names should be descriptive and indicate the scenario, e.g., `Parse_Short`, `ComputeHash_Large`. +- When comparing implementations, mark one method with `Baseline = true` and use similar names so reports are easy to read. + +### Example Benchmark + +```csharp +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Configs; + +namespace YourProject; + +[MemoryDiagnoser] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +public class SampleOperationBenchmark +{ + [Params(8, 256, 4096)] + public int Count { get; set; } + + private byte[] _payload; + + [GlobalSetup] + public void Setup() + { + _payload = new byte[Count]; + } + + [Benchmark(Baseline = true, Description = "Operation - baseline")] + public int Operation_Baseline() => SampleOperation.Process(_payload); + + [Benchmark(Description = "Operation - optimized")] + public int Operation_Optimized() => SampleOperation.ProcessOptimized(_payload); +} +``` + +### Reporting and CI + +- Benchmarks are primarily for local and tuning runs; be cautious about running heavy BenchmarkDotNet workloads in CI. Prefer targeted runs or harnesses for CI where appropriate. +- Keep benchmark projects isolated (e.g., `tuning/*.csproj`) so they don't affect package builds or production artifacts. + +### Additional Benchmark Guidelines + +- Keep benchmarks readable and well-documented; add comments explaining non-obvious choices. +- If a benchmark exposes regressions or optimizations, add a short note in the benchmark file referencing the relevant issue or PR. +- For any shared helpers for benchmarking, prefer small utility classes inside the `tuning` projects rather than cross-cutting changes to production code. + +For further examples, refer to the benchmark files under the `tuning/` folder. + +## XML Documentation + +The following rules apply to C# source files. All public APIs must have XML documentation comments, including protected members described below. + +This document provides instructions for writing XML documentation. + +### Documentation Style + +- Use the same documentation style as found throughout the codebase. +- Add XML doc comments to all public and protected classes, methods, properties, and constructors. +- Use `` for type and member descriptions. +- Use `` for method parameters. +- Use `` for return values. +- Use `` for generic type parameters. +- Use `` for additional context, especially default property values using ``. +- Use `` to reference related types and interfaces. +- Use `` inline to reference types, members, and parameters. +- Use `` to document thrown exceptions. + +### XML Documentation Example + +```csharp +namespace YourProject; + +/// +/// Provides utility methods for processing data. +/// +/// The type of the configured options. +/// +public class DataProcessor where TOptions : class, new() +{ + /// + /// Initializes a new instance of the class. + /// + /// The which may be configured. + public DataProcessor(Action setup) + { + Options = setup != null ? Configure(setup) : new TOptions(); + } + + /// + /// Gets the configured options of this instance. + /// + /// The configured options of this instance. + public TOptions Options { get; } + + /// + /// Processes the specified and returns the result. + /// + /// The input data to process. + /// A containing the processed result. + /// + /// is null. + /// + public string Process(string input) + { + if (input == null) throw new ArgumentNullException(nameof(input)); + return input.ToUpperInvariant(); + } +} +``` + +### Additional XML Documentation Guidelines + +- Keep descriptions concise but informative. +- Use consistent phrasing: "Gets or sets", "Initializes a new instance of", "Provides", "Represents". +- Reference parameter names with `` in descriptions. +- For options/settings classes, document default values in `` using a table format. + +Use `` for property-value descriptions, as illustrated in the example above. + ## Build & CI - Centralized package versions via `Directory.Packages.props` @@ -48,7 +411,7 @@ Do not hard-wrap prose to a fixed column width. Keep paragraphs and Markdown lis If a `.bot/` folder exists at the root, it contains **confidential, local-only** working material for AI agents — product requirement documents (PRDs), design proposals, agentic loop state, and brainstorming outputs. This folder is gitignored and never committed. -When starting creative or design work (new features, architecture decisions, PRD drafts), use the [brainstorming skill](https://skills.sh/obra/superpowers/brainstorming) and save outputs to `.bot/`. Only move finalized, non-confidential instructions into `AGENTS.md` or `.github/copilot-instructions.md`. +When starting creative or design work (new features, architecture decisions, PRD drafts), use the [brainstorming skill](https://skills.sh/obra/superpowers/brainstorming) and save outputs to `.bot/`. Only move finalized, non-confidential instructions into root `AGENTS.md`. ## Git Operations Safeguards diff --git a/skills/dotnet-new-lib-slnx/assets/shared.manifest.json b/skills/dotnet-new-lib-slnx/assets/shared.manifest.json index bf68df3..0836b9d 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared.manifest.json +++ b/skills/dotnet-new-lib-slnx/assets/shared.manifest.json @@ -12,7 +12,6 @@ ".github/CODE_OF_CONDUCT.md", ".github/codecov.yml", ".github/CONTRIBUTING.md", - ".github/copilot-instructions.md", ".github/dependabot.yml", ".github/workflows/ci-pipeline.yml", ".gitignore", diff --git a/skills/dotnet-new-lib-slnx/assets/shared/.bot/README.md b/skills/dotnet-new-lib-slnx/assets/shared/.bot/README.md index 2cfca89..397ae8c 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared/.bot/README.md +++ b/skills/dotnet-new-lib-slnx/assets/shared/.bot/README.md @@ -7,4 +7,4 @@ This folder is reserved for local-only AI working material such as: - design alternatives - temporary agent state -Keep this folder out of source control. Move only finalized, non-confidential guidance into `AGENTS.md` or `.github/copilot-instructions.md`. +Keep this folder out of source control. Move only finalized, non-confidential guidance into root `AGENTS.md`. diff --git a/skills/dotnet-new-lib-slnx/assets/shared/.github/copilot-instructions.md b/skills/dotnet-new-lib-slnx/assets/shared/.github/copilot-instructions.md deleted file mode 100644 index a1e1ee0..0000000 --- a/skills/dotnet-new-lib-slnx/assets/shared/.github/copilot-instructions.md +++ /dev/null @@ -1,353 +0,0 @@ ---- -description: 'Writing Unit Tests' -applyTo: "**/*.{cs,csproj}" ---- - -# Writing Unit Tests -This document provides instructions for writing unit tests for a project/solution. Please follow these guidelines to ensure consistency and maintainability. - -## 1. Base Class - -**Always inherit from the `Test` base class** for all unit test classes. This ensures consistent setup, teardown, and output handling across all tests. - -> Important: Do NOT add `using Xunit.Abstractions`. xUnit v4 (the `xunit.v3` 4.x package line) does not expose that namespace; including it is incorrect and will cause compilation errors. Use the `Codebelt.Extensions.Xunit` Test base class and `using Xunit;` as shown in the examples below. If you need access to test output, rely on the Test base class (which accepts the appropriate output helper) rather than importing `Xunit.Abstractions`. - -```csharp -using Codebelt.Extensions.Xunit; -using Xunit; - -namespace Your.Namespace -{ - public class YourTestClass : Test - { - public YourTestClass(ITestOutputHelper output) : base(output) - { - } - - // Your tests here - } -} -``` - -## 2. Test Method Attributes - -- Use `[Fact]` for standard unit tests. -- Use `[Theory]` with `[InlineData]` or other data sources for parameterized tests. - -## 3. Naming Conventions - -- **Test classes**: End with `Test` (e.g., `DateSpanTest`). -- **Test methods**: Use descriptive names that state the expected behavior (e.g., `ShouldReturnTrue_WhenConditionIsMet`). - -## 4. Assertions - -- Use `Assert` methods from xUnit for all assertions. -- Prefer explicit and expressive assertions (e.g., `Assert.Equal`, `Assert.NotNull`, `Assert.Contains`). - -## 5. File and Namespace Organization - -- Place test files in the appropriate test project and folder structure. -- Use namespaces that mirror the source code structure. The namespace of a test file MUST match the namespace of the System Under Test (SUT). Do NOT append ".Tests", ".Benchmarks" or similar suffixes to the namespace. Only the assembly/project name should indicate that the file is a test/benchmark. - - Example: If the SUT class is declared as: - ```csharp - namespace YourProject.Foo.Bar - { - public class Zoo { /* ... */ } - } - ``` -then the corresponding unit test class must use the exact same namespace: - ```csharp - namespace YourProject.Foo.Bar - { - public class ZooTest : Test { /* ... */ } - } - ``` - - Do NOT use: - ```csharp - namespace YourProject.Foo.Bar.Tests { /* ... */ } // ❌ - namespace YourProject.Foo.Bar.Benchmarks { /* ... */ } // ❌ - ``` - - The unit tests for the YourProject.Foo assembly live in the YourProject.Foo.Tests assembly. - - The functional tests for the YourProject.Foo assembly live in the YourProject.Foo.FunctionalTests assembly. - - Test class names end with Test and live in the same namespace as the class being tested, e.g., the unit tests for the Boo class that resides in the YourProject.Foo assembly would be named BooTest and placed in the YourProject.Foo namespace in the YourProject.Foo.Tests assembly. - - Modify the associated .csproj file to override the root namespace so the compiled namespace matches the SUT. Example: - ```xml - - YourProject.Foo - - ``` -- When generating test scaffolding automatically, resolve the SUT's namespace from the source file (or project/assembly metadata) and use that exact namespace in the test file header. - -- Notes: - - This rule ensures type discovery and XML doc links behave consistently and reduces confusion when reading tests. - - Keep folder structure aligned with the production code layout to make locating SUT <-> test pairs straightforward. - -## 6. Example Test - -```csharp -using System; -using Codebelt.Extensions.Xunit; -using Xunit; - -namespace YourProject -{ - public class SampleTest : Test - { - public SampleTest(ITestOutputHelper output) : base(output) - { - } - - [Fact] - public void ShouldReturnExpectedResult_WhenGivenValidInput() - { - var sut = new SampleClass(); - - var result = sut.Process("input"); - - Assert.NotNull(result); - Assert.Equal("expected", result.Value); - - TestOutput.WriteLine(result.ToString()); - } - - [Theory] - [InlineData(1, "one")] - [InlineData(2, "two")] - public void ShouldMapCorrectly_WhenGivenNumber(int input, string expected) - { - var sut = new SampleClass(); - - var result = sut.MapToString(input); - - Assert.Equal(expected, result); - } - } -} -``` - -## 7. Additional Guidelines - -- Keep tests focused and isolated. -- Do not rely on external systems except for xUnit itself and Codebelt.Extensions.Xunit (and derived from this). -- Ensure tests are deterministic and repeatable. - -## 8. Test Doubles - -- Preferred test doubles include dummies, fakes, stubs and spies if and when the design allows it. -- Under special circumstances, mock can be used (using Moq library). -- Before overriding methods, verify that the method is virtual or abstract; this rule also applies to mocks. -- Never mock IMarshaller; always use a new instance of JsonMarshaller. - -## 9. Avoid `InternalsVisibleTo` in Tests - -- **Do not** use `InternalsVisibleTo` to access internal types or members from test projects. -- Prefer **indirect testing via public APIs** that depend on the internal implementation (public facades, public extension methods, or other public entry points). - -### Preferred Pattern - -**Pattern name:** Public Facade Testing (also referred to as *Public API Proxy Testing*) - -**Description:** Internal classes and methods must be validated by exercising the public API that consumes them. Tests should assert observable behavior exposed by the public surface rather than targeting internal implementation details directly. - -### Example Mapping - -- **Internal helper:** `DelimitedString` (internal static class) -- **Public API:** `TestOutputHelperExtensions.WriteLines()` (public extension method) -- **Test strategy:** Write tests for `WriteLines()` and verify its public behavior. The internal call to `DelimitedString.Create()` is exercised implicitly. - -### Benefits - -- Avoids exposing internal types to test assemblies. -- Ensures tests reflect real-world usage patterns. -- Maintains strong encapsulation and a clean public API. -- Tests remain resilient to internal refactoring as long as public behavior is preserved. - -### When to Apply - -- Internal logic is fully exercised through existing public APIs. -- Public entry points provide sufficient coverage of internal code paths. -- The internal implementation exists solely as a helper or utility for public-facing functionality. - -## 10. ExcludeFromCodeCoverage Prohibition - -**Do not use `ExcludeFromCodeCoverage` attribute on any code.** This includes: - -- Test classes or test methods -- Production code -- Configuration code -- Any other code path - -### Rationale - -- Excluding code from coverage hides gaps and creates false confidence in test completeness. -- If a code path cannot or should not be tested, refactor the code to eliminate that path rather than hiding it from metrics. -- Every executable line should be covered by tests or be genuinely unreachable (dead code to be removed). - -### Alternative Approaches - -- **Untestable code paths**: Refactor to separate concerns and eliminate the untestable path. -- **External dependencies**: Use test doubles (fakes, stubs, spies) instead of excluding from coverage. -- **Configuration-only code**: Move to configuration files or extract into testable methods. -- **Generated or third-party code**: These should not be in the primary codebase; use NuGet packages or dedicated vendor folders if necessary. - ---- -description: 'Writing Performance Tests' -applyTo: "tooling/**, tuning/**, **/*Benchmark*.cs" ---- - -# Writing Performance Tests -This document provides guidance for writing performance tests (benchmarks) for a project/solution using BenchmarkDotNet. Follow these guidelines to keep benchmarks consistent, readable, and comparable. - -## 1. Naming and Placement - -- Place micro- and component-benchmarks under the `tuning/` folder or in projects named `*.Benchmarks`. -- Place benchmark files in the appropriate benchmark project and folder structure. -- Use namespaces that mirror the source code structure, e.g. do not suffix with `Benchmarks`. -Namespace rule: DO NOT append `.Benchmarks` to the namespace. Benchmarks must live in the same namespace as the production assembly. Example: if the production assembly uses `namespace YourProject.Security.Cryptography`, the benchmark file should also use: - ``` - namespace YourProject.Security.Cryptography - { - public class Sha512256Benchmark { /* ... */ } - } - ``` -The class name must end with `Benchmark`, but the namespace must match the assembly (no `.Benchmarks` suffix). - The benchmarks for the YourProject.Bar assembly live in the YourProject.Bar.Benchmarks assembly. - Benchmark class names end with Benchmark and live in the same namespace as the class being measured, e.g., the benchmarks for the Zoo class that resides in the YourProject.Bar assembly would be named ZooBenchmark and placed in the YourProject.Bar namespace in the YourProject.Bar.Benchmarks assembly. - Modify the associated .csproj file to override the root namespace, e.g., YourProject.Bar. - -## 2. Attributes and Configuration - -- Use `BenchmarkDotNet` attributes to express intent and collect relevant metrics: - - `[MemoryDiagnoser]` to capture memory allocations. - - `[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]` to group related benchmarks. - - `[Params]` for input sizes or variations to exercise multiple scenarios. - - `[GlobalSetup]` for one-time initialization that's not part of measured work. - - `[Benchmark]` on methods representing measured operations; consider `Baseline = true` and `Description` to improve report clarity. -- Keep benchmark configuration minimal and explicit; prefer in-class attributes over large shared configs unless re-used widely. - -## 3. Structure and Best Practices - -- Keep benchmarks focused: each `Benchmark` method should measure a single logical operation. -- Avoid doing expensive setup work inside a measured method; use `[GlobalSetup]`, `[IterationSetup]`, or cached fields instead. -- Use `Params` to cover micro, mid and macro input sizes (for example: small, medium, large) and verify performance trends across them. -- Use small, deterministic data sets and avoid external systems (network, disk, DB). If external systems are necessary, mark them clearly and do not include them in CI benchmark runs by default. -- Capture results that are meaningful: time, allocations, and if needed custom counters. Prefer `MemoryDiagnoser` and descriptive `Description` values. - -## 4. Naming Conventions for Methods - -- Method names should be descriptive and indicate the scenario, e.g., `Parse_Short`, `ComputeHash_Large`. -- When comparing implementations, mark one method with `Baseline = true` and use similar names so reports are easy to read. - -## 5. Example Benchmark - -```csharp -using BenchmarkDotNet.Attributes; -using BenchmarkDotNet.Configs; - -namespace YourProject -{ - [MemoryDiagnoser] - [GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] - public class SampleOperationBenchmark - { - [Params(8, 256, 4096)] - public int Count { get; set; } - - private byte[] _payload; - - [GlobalSetup] - public void Setup() - { - _payload = new byte[Count]; - } - - [Benchmark(Baseline = true, Description = "Operation - baseline")] - public int Operation_Baseline() => SampleOperation.Process(_payload); - - [Benchmark(Description = "Operation - optimized")] - public int Operation_Optimized() => SampleOperation.ProcessOptimized(_payload); - } -} -``` - -## 6. Reporting and CI - -- Benchmarks are primarily for local and tuning runs; be cautious about running heavy BenchmarkDotNet workloads in CI. Prefer targeted runs or harnesses for CI where appropriate. -- Keep benchmark projects isolated under `tuning/`, keep the runner isolated under `tooling/`, and write reports to `reports/`. - -## 7. Additional Guidelines - -- Keep benchmarks readable and well-documented; add comments explaining non-obvious choices. -- If a benchmark exposes regressions or optimizations, add a short note in the benchmark file referencing the relevant issue or PR. -- For any shared helpers for benchmarking, prefer small utility classes inside the benchmark projects under `tuning/` or inside the runner under `tooling/` rather than cross-cutting changes to production code. - -For further examples, refer to the benchmark projects under `tuning/`, the runner under `tooling/`, and the generated reports under `reports/`. - ---- -description: 'Writing XML documentation' -applyTo: "**/*.cs" ---- - -# Writing XML Documentation -This document provides instructions for writing XML documentation. - -## 1. Documentation Style - -- Use the same documentation style as found throughout the codebase. -- Add XML doc comments to all public and protected classes, methods, properties, and constructors. -- Use `` for type and member descriptions. -- Use `` for method parameters. -- Use `` for return values. -- Use `` for generic type parameters. -- Use `` for additional context, especially default property values using ``. -- Use `` to reference related types and interfaces. -- Use `` inline to reference types, members, and parameters. -- Use `` to document thrown exceptions. - -## 2. Example - -```csharp -namespace YourProject -{ - /// - /// Provides utility methods for processing data. - /// - /// The type of the configured options. - /// - public class DataProcessor where TOptions : class, new() - { - /// - /// Initializes a new instance of the class. - /// - /// The which may be configured. - public DataProcessor(Action setup) - { - Options = setup != null ? Configure(setup) : new TOptions(); - } - - /// - /// Gets the configured options of this instance. - /// - /// The configured options of this instance. - public TOptions Options { get; } - - /// - /// Processes the specified and returns the result. - /// - /// The input data to process. - /// A containing the processed result. - /// - /// is null. - /// - public string Process(string input) - { - if (input == null) throw new ArgumentNullException(nameof(input)); - return input.ToUpperInvariant(); - } - } -} -``` - -## 3. Additional Guidelines - -- Keep descriptions concise but informative. -- Use consistent phrasing: "Gets or sets", "Initializes a new instance of", "Provides", "Represents". -- Reference parameter names with `` in descriptions. -- For options/settings classes, document default values in `` using a table format. diff --git a/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md b/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md index 52bc738..35358b0 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md +++ b/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md @@ -1,6 +1,6 @@ # Agent Instructions for {SOLUTION_NAME} -This document provides guidance for AI agents working in this repository. +Root `AGENTS.md` is the canonical, vendor-neutral instruction contract for coding agents working in this repository. ## Project Overview @@ -14,8 +14,6 @@ This document provides guidance for AI agents working in this repository. - **Top-level statements:** Not allowed (enforced via `.editorconfig`) - **Language version:** Always use the latest C# features (`LangVersion=latest`) - **Nullable:** Enable nullable reference types in all new code -- **XML documentation:** All public APIs must have XML documentation comments -- **Testing:** Use xUnit v4 (`xunit.v3` 4.x packages) with Codebelt.Extensions.Xunit.App v12 base classes ## Markdown Prose Formatting @@ -30,10 +28,12 @@ Do not hard-wrap prose to a fixed column width. Keep paragraphs and Markdown lis - `reports/` — Benchmark reports and tuning output produced by tooling - `.nuget/` — Per-package NuGet metadata (icon, README, release notes) - `.docfx/` — DocFX documentation configuration -- `.github/` — CI/CD workflows, contributing guidelines, Copilot instructions +- `.github/` — CI/CD workflows and contributing guidelines ## Test Conventions +Use xUnit v4 (`xunit.v3` 4.x packages) with Codebelt.Extensions.Xunit.App v12 base classes. Functional host tests retain the appropriate managed application fixtures; the unit-test rules below apply to tests of individual classes. + - Test project names must end with `Tests` (e.g. `{PROJECT_NAME}.Tests`) - Test classes should inherit from the appropriate base class in `Codebelt.Extensions.Xunit` - Use `Microsoft.Testing.Platform` as the test runner (root `global.json` selects `test.runner`; `UseMicrosoftTestingPlatformRunner=true` enables project integration) @@ -43,6 +43,369 @@ Do not hard-wrap prose to a fixed column width. Keep paragraphs and Markdown lis - Use managed application fixtures with deferred entrypoint-owned startup, not removed blocking application fixtures - All tests are executable (`OutputType=Exe`) +## Unit Testing + +The following rules apply to unit-test source files and unit-test projects. The file and namespace organization rules also describe the relationship between unit-test and functional-test assemblies. + +This document provides instructions for writing unit tests for a project/solution. Please follow these guidelines to ensure consistency and maintainability. + +### Base Class + +**Always inherit from the `Test` base class** for all unit test classes. This ensures consistent setup, teardown, and output handling across all tests. + +> Important: Do NOT add `using Xunit.Abstractions`. xUnit v4 (the `xunit.v3` 4.x package line) does not expose that namespace; including it is incorrect and will cause compilation errors. Use the `Codebelt.Extensions.Xunit` Test base class and `using Xunit;` as shown in the examples below. If you need access to test output, rely on the Test base class (which accepts the appropriate output helper) rather than importing `Xunit.Abstractions`. + +```csharp +using Codebelt.Extensions.Xunit; +using Xunit; + +namespace Your.Namespace; + +public class YourTestClass : Test +{ + public YourTestClass(ITestOutputHelper output) : base(output) + { + } + + // Your tests here +} +``` + +### Test Method Attributes + +- Use `[Fact]` for standard unit tests. +- Use `[Theory]` with `[InlineData]` or other data sources for parameterized tests. + +### Test Naming Conventions + +- **Test classes**: End with `Test` (e.g., `DateSpanTest`). +- **Test methods**: Use descriptive names that state the expected behavior (e.g., `ShouldReturnTrue_WhenConditionIsMet`). + +### Assertions + +- Use `Assert` methods from xUnit for all assertions. +- Prefer explicit and expressive assertions (e.g., `Assert.Equal`, `Assert.NotNull`, `Assert.Contains`). + +### File and Namespace Organization + +- Place test files in the appropriate test project and folder structure. +- Use namespaces that mirror the source code structure. The namespace of a test file MUST match the namespace of the System Under Test (SUT). Do NOT append ".Tests", ".Benchmarks" or similar suffixes to the namespace. Only the assembly/project name should indicate that the file is a test/benchmark. + +Example: If the SUT class is declared as: + +```csharp +namespace YourProject.Foo.Bar; + +public class Zoo { /* ... */ } +``` + +then the corresponding unit test class must use the exact same namespace: + +```csharp +namespace YourProject.Foo.Bar; + +public class ZooTest : Test { /* ... */ } +``` + +Do NOT use: + +```csharp +namespace YourProject.Foo.Bar.Tests; /* ... */ // ❌ +namespace YourProject.Foo.Bar.Benchmarks; /* ... */ // ❌ +``` + +- The unit tests for the YourProject.Foo assembly live in the YourProject.Foo.Tests assembly. +- The functional tests for the YourProject.Foo assembly live in the YourProject.Foo.FunctionalTests assembly. +- Test class names end with Test and live in the same namespace as the class being tested, e.g., the unit tests for the Boo class that resides in the YourProject.Foo assembly would be named BooTest and placed in the YourProject.Foo namespace in the YourProject.Foo.Tests assembly. +- Modify the associated .csproj file to override the root namespace so the compiled namespace matches the SUT. Example: + +```xml + + YourProject.Foo + +``` + +- When generating test scaffolding automatically, resolve the SUT's namespace from the source file (or project/assembly metadata) and use that exact namespace in the test file header. + +- Notes: + - This rule ensures type discovery and XML doc links behave consistently and reduces confusion when reading tests. + - Keep folder structure aligned with the production code layout to make locating SUT <-> test pairs straightforward. + +Do not append `.FunctionalTests` to test namespaces either; the same SUT namespace equality rule applies. + +### Example Test + +```csharp +using System; +using Codebelt.Extensions.Xunit; +using Xunit; + +namespace YourProject; + +public class SampleTest : Test +{ + public SampleTest(ITestOutputHelper output) : base(output) + { + } + + [Fact] + public void ShouldReturnExpectedResult_WhenGivenValidInput() + { + var sut = new SampleClass(); + + var result = sut.Process("input"); + + Assert.NotNull(result); + Assert.Equal("expected", result.Value); + + TestOutput.WriteLine(result.ToString()); + } + + [Theory] + [InlineData(1, "one")] + [InlineData(2, "two")] + public void ShouldMapCorrectly_WhenGivenNumber(int input, string expected) + { + var sut = new SampleClass(); + + var result = sut.MapToString(input); + + Assert.Equal(expected, result); + } +} +``` + +### Additional Unit Test Guidelines + +- Keep tests focused and isolated. +- Do not rely on external systems except for xUnit itself and Codebelt.Extensions.Xunit (and derived from this). +- Ensure tests are deterministic and repeatable. + +### Test Doubles + +- Preferred test doubles include dummies, fakes, stubs and spies if and when the design allows it. +- Under special circumstances, mock can be used (using Moq library). +- Before overriding methods, verify that the method is virtual or abstract; this rule also applies to mocks. +- Never mock IMarshaller; always use a new instance of JsonMarshaller. + +### Public Facade Testing + +- **Do not** use `InternalsVisibleTo` to access internal types or members from test projects. +- Prefer **indirect testing via public APIs** that depend on the internal implementation (public facades, public extension methods, or other public entry points). + +#### Preferred Pattern + +**Pattern name:** Public Facade Testing (also referred to as *Public API Proxy Testing*) + +**Description:** Internal classes and methods must be validated by exercising the public API that consumes them. Tests should assert observable behavior exposed by the public surface rather than targeting internal implementation details directly. + +#### Example Mapping + +- **Internal helper:** `DelimitedString` (internal static class) +- **Public API:** `TestOutputHelperExtensions.WriteLines()` (public extension method) +- **Test strategy:** Write tests for `WriteLines()` and verify its public behavior. The internal call to `DelimitedString.Create()` is exercised implicitly. + +#### Benefits + +- Avoids exposing internal types to test assemblies. +- Ensures tests reflect real-world usage patterns. +- Maintains strong encapsulation and a clean public API. +- Tests remain resilient to internal refactoring as long as public behavior is preserved. + +#### When to Apply + +- Internal logic is fully exercised through existing public APIs. +- Public entry points provide sufficient coverage of internal code paths. +- The internal implementation exists solely as a helper or utility for public-facing functionality. + +## Code Coverage + +**Do not use `ExcludeFromCodeCoverage` attribute on any code.** This includes: + +- Test classes or test methods +- Production code +- Configuration code +- Any other code path + +### Coverage Rationale + +- Excluding code from coverage hides gaps and creates false confidence in test completeness. +- If a code path cannot or should not be tested, refactor the code to eliminate that path rather than hiding it from metrics. +- Every executable line should be covered by tests or be genuinely unreachable (dead code to be removed). + +### Coverage Alternatives + +- **Untestable code paths**: Refactor to separate concerns and eliminate the untestable path. +- **External dependencies**: Use test doubles (fakes, stubs, spies) instead of excluding from coverage. +- **Configuration-only code**: Move to configuration files or extract into testable methods. +- **Generated or third-party code**: These should not be in the primary codebase; use NuGet packages or dedicated vendor folders if necessary. + +Do not append .FunctionalTests to test namespaces either; the same SUT namespace equality rule applies. + +## Benchmarking + +The following rules apply to benchmark projects (`*.Benchmarks`), code under `tuning/` and `tooling/`, and `*Benchmark*.cs` source files. Namespace rules for benchmark types refer to the production code being measured; the executable benchmark runner host retains its own tooling-derived namespace. + +This document provides guidance for writing performance tests (benchmarks) for a project/solution using BenchmarkDotNet. Follow these guidelines to keep benchmarks consistent, readable, and comparable. + +### Benchmark Naming and Placement + +- Place micro- and component-benchmarks under the `tuning/` folder or in projects named `*.Benchmarks`. +- Place benchmark files in the appropriate benchmark project and folder structure. +- Use namespaces that mirror the source code structure, e.g. do not suffix with `Benchmarks`. + +Namespace rule: DO NOT append `.Benchmarks` to the namespace. Benchmarks must live in the same namespace as the production assembly. Example: if the production assembly uses `namespace YourProject.Security.Cryptography`, the benchmark file should also use: + +``` +namespace YourProject.Security.Cryptography; + +public class Sha512256Benchmark { /* ... */ } +``` + +The class name must end with `Benchmark`, but the namespace must match the assembly (no `.Benchmarks` suffix). + +- The benchmarks for the YourProject.Bar assembly live in the YourProject.Bar.Benchmarks assembly. +- Benchmark class names end with Benchmark and live in the same namespace as the class being measured, e.g., the benchmarks for the Zoo class that resides in the YourProject.Bar assembly would be named ZooBenchmark and placed in the YourProject.Bar namespace in the YourProject.Bar.Benchmarks assembly. +- Modify the associated .csproj file to override the root namespace, e.g., `YourProject.Bar`. + +### Attributes and Configuration + +- Use `BenchmarkDotNet` attributes to express intent and collect relevant metrics: + - `[MemoryDiagnoser]` to capture memory allocations. + - `[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]` to group related benchmarks. + - `[Params]` for input sizes or variations to exercise multiple scenarios. + - `[GlobalSetup]` for one-time initialization that's not part of measured work. + - `[Benchmark]` on methods representing measured operations; consider `Baseline = true` and `Description` to improve report clarity. +- Keep benchmark configuration minimal and explicit; prefer in-class attributes over large shared configs unless re-used widely. + +### Structure and Best Practices + +- Keep benchmarks focused: each `Benchmark` method should measure a single logical operation. +- Avoid doing expensive setup work inside a measured method; use `[GlobalSetup]`, `[IterationSetup]`, or cached fields instead. +- Use `Params` to cover micro, mid and macro input sizes (for example: small, medium, large) and verify performance trends across them. +- Use small, deterministic data sets and avoid external systems (network, disk, DB). If external systems are necessary, mark them clearly and do not include them in CI benchmark runs by default. +- Capture results that are meaningful: time, allocations, and if needed custom counters. Prefer `MemoryDiagnoser` and descriptive `Description` values. + +### Benchmark Method Naming + +- Method names should be descriptive and indicate the scenario, e.g., `Parse_Short`, `ComputeHash_Large`. +- When comparing implementations, mark one method with `Baseline = true` and use similar names so reports are easy to read. + +### Example Benchmark + +```csharp +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Configs; + +namespace YourProject; + +[MemoryDiagnoser] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +public class SampleOperationBenchmark +{ + [Params(8, 256, 4096)] + public int Count { get; set; } + + private byte[] _payload; + + [GlobalSetup] + public void Setup() + { + _payload = new byte[Count]; + } + + [Benchmark(Baseline = true, Description = "Operation - baseline")] + public int Operation_Baseline() => SampleOperation.Process(_payload); + + [Benchmark(Description = "Operation - optimized")] + public int Operation_Optimized() => SampleOperation.ProcessOptimized(_payload); +} +``` + +### Reporting and CI + +- Benchmarks are primarily for local and tuning runs; be cautious about running heavy BenchmarkDotNet workloads in CI. Prefer targeted runs or harnesses for CI where appropriate. +- Keep benchmark projects isolated under `tuning/`, keep the runner isolated under `tooling/`, and write reports to `reports/`. + +### Additional Benchmark Guidelines + +- Keep benchmarks readable and well-documented; add comments explaining non-obvious choices. +- If a benchmark exposes regressions or optimizations, add a short note in the benchmark file referencing the relevant issue or PR. +- For any shared helpers for benchmarking, prefer small utility classes inside the benchmark projects under `tuning/` or inside the runner under `tooling/` rather than cross-cutting changes to production code. + +For further examples, refer to the benchmark projects under `tuning/`, the runner under `tooling/`, and the generated reports under `reports/`. + +## XML Documentation + +The following rules apply to C# source files. All public APIs must have XML documentation comments, including protected members described below. + +This document provides instructions for writing XML documentation. + +### Documentation Style + +- Use the same documentation style as found throughout the codebase. +- Add XML doc comments to all public and protected classes, methods, properties, and constructors. +- Use `` for type and member descriptions. +- Use `` for method parameters. +- Use `` for return values. +- Use `` for generic type parameters. +- Use `` for additional context, especially default property values using ``. +- Use `` to reference related types and interfaces. +- Use `` inline to reference types, members, and parameters. +- Use `` to document thrown exceptions. + +### XML Documentation Example + +```csharp +namespace YourProject; + +/// +/// Provides utility methods for processing data. +/// +/// The type of the configured options. +/// +public class DataProcessor where TOptions : class, new() +{ + /// + /// Initializes a new instance of the class. + /// + /// The which may be configured. + public DataProcessor(Action setup) + { + Options = setup != null ? Configure(setup) : new TOptions(); + } + + /// + /// Gets the configured options of this instance. + /// + /// The configured options of this instance. + public TOptions Options { get; } + + /// + /// Processes the specified and returns the result. + /// + /// The input data to process. + /// A containing the processed result. + /// + /// is null. + /// + public string Process(string input) + { + if (input == null) throw new ArgumentNullException(nameof(input)); + return input.ToUpperInvariant(); + } +} +``` + +### Additional XML Documentation Guidelines + +- Keep descriptions concise but informative. +- Use consistent phrasing: "Gets or sets", "Initializes a new instance of", "Provides", "Represents". +- Reference parameter names with `` in descriptions. +- For options/settings classes, document default values in `` using a table format. + +Use `` for property-value descriptions, as illustrated in the example above. + ## Build & CI - Centralized package versions via `Directory.Packages.props` @@ -56,7 +419,7 @@ Do not hard-wrap prose to a fixed column width. Keep paragraphs and Markdown lis If a `.bot/` folder exists at the root, it contains **confidential, local-only** working material for AI agents — product requirement documents (PRDs), design proposals, agentic loop state, and brainstorming outputs. This folder is gitignored and never committed. -When starting creative or design work (new features, architecture decisions, PRD drafts), use the [brainstorming skill](https://skills.sh/obra/superpowers/brainstorming) and save outputs to `.bot/`. Only move finalized, non-confidential instructions into `AGENTS.md` or `.github/copilot-instructions.md`. +When starting creative or design work (new features, architecture decisions, PRD drafts), use the [brainstorming skill](https://skills.sh/obra/superpowers/brainstorming) and save outputs to `.bot/`. Only move finalized, non-confidential instructions into root `AGENTS.md`. ## Git Operations Safeguards From 506557a9830f87590ac886be69500adecd485664 Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:55:23 +0200 Subject: [PATCH 08/13] =?UTF-8?q?=E2=9C=85=20add=20vendor-neutral=20scaffo?= =?UTF-8?q?ld=20regression=20validation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Enforce the sole AGENTS.md inventory, forbid vendor-specific instruction semantics, and assert the nine complete governance examples plus coverage and benchmark applicability. --- scripts/tests/test-scaffold-mtp.ps1 | 17 +++- scripts/validate-skill-templates.ps1 | 140 ++++++++++++++++++++++++++- 2 files changed, 154 insertions(+), 3 deletions(-) diff --git a/scripts/tests/test-scaffold-mtp.ps1 b/scripts/tests/test-scaffold-mtp.ps1 index c8311a2..a12409d 100644 --- a/scripts/tests/test-scaffold-mtp.ps1 +++ b/scripts/tests/test-scaffold-mtp.ps1 @@ -116,9 +116,24 @@ try { $map['{REPOSITORY_URL}'] = 'https://example.invalid/scaffold' $map['{SNK_FILE}'] = 'fixture.snk' - foreach ($file in @('Directory.Packages.props', 'Directory.Build.targets', 'global.json')) { + $manifest = [System.IO.File]::ReadAllText((Join-Path $repoRoot "$skillRoot/assets/shared.manifest.json")) | ConvertFrom-Json + foreach ($file in $manifest.files) { Render-Template -Source "$skillRoot/assets/shared/$file" -Destination (Join-Path $caseRoot $file) -Map $map } + $agentsPath = Join-Path $caseRoot 'AGENTS.md' + if (-not (Test-Path -LiteralPath $agentsPath -PathType Leaf)) { throw "$caseName is missing root AGENTS.md." } + if (Get-ChildItem -LiteralPath $caseRoot -Recurse -File -Force -Filter 'copilot-instructions.md') { + throw "$caseName emitted a Copilot-specific instruction file." + } + $agents = [System.IO.File]::ReadAllText($agentsPath) + foreach ($rule in @('InternalsVisibleTo', 'ExcludeFromCodeCoverage', 'Xunit.Abstractions', 'BenchmarkDotNet', 'RootNamespace', '')) { + if (-not $agents.Contains($rule)) { throw "$caseName root AGENTS.md lost the $rule guidance." } + } + $bot = [System.IO.File]::ReadAllText((Join-Path $caseRoot '.bot/README.md')) + if (-not $bot.Contains('into root `AGENTS.md`.') -or $bot -match '(?i)Copilot') { + throw "$caseName .bot guidance must point only to root AGENTS.md." + } + Write-Host "[PASS] $caseName emits the complete shared inventory with vendor-neutral root AGENTS.md." # Select a stable installed SDK only in this isolated regression workspace. $globalPath = Join-Path $caseRoot 'global.json' $global = [System.IO.File]::ReadAllText($globalPath) | ConvertFrom-Json diff --git a/scripts/validate-skill-templates.ps1 b/scripts/validate-skill-templates.ps1 index 61d8edf..a091290 100644 --- a/scripts/validate-skill-templates.ps1 +++ b/scripts/validate-skill-templates.ps1 @@ -1194,7 +1194,7 @@ Add-ValidationResult -Results $results -Name 'App reference guide uses ROOT_NAME Assert-Contains -Name 'dotnet-new-app-slnx/references/app.md' -Content $guide -Needle 'Treat the files shown in this tree as required output, not aspirational examples.' Assert-Contains -Name 'dotnet-new-app-slnx/references/app.md' -Content $guide -Needle '## Required Shared Asset Inventory' Assert-Contains -Name 'dotnet-new-app-slnx/references/app.md' -Content $guide -Needle 'Do not cherry-pick only the files that feel essential.' - Assert-Contains -Name 'dotnet-new-app-slnx/references/app.md' -Content $guide -Needle '.github/copilot-instructions.md' + Assert-Contains -Name 'dotnet-new-app-slnx/references/app.md' -Content $guide -Needle 'Root `AGENTS.md` is the single generated agent-instruction source.' Assert-Contains -Name 'dotnet-new-app-slnx/references/app.md' -Content $guide -Needle 'Even when there is only one host type, still generate the `.slnx` file' Assert-Contains -Name 'dotnet-new-app-slnx/references/app.md' -Content $guide -Needle 'Directory.Packages.props` is the authoritative version source for app scaffolds.' Assert-Contains -Name 'dotnet-new-app-slnx/references/app.md' -Content $guide -Needle 'Do **not** duplicate `` inside the generated app or test `.csproj` files as a workaround.' @@ -1295,6 +1295,139 @@ Add-ValidationResult -Results $results -Name 'App dependabot and test environmen Assert-NotContains -Name 'app testenvironments' -Content $testEnvironments -Needle 'net8.0.418-9.0.311-10.0.103' } +Add-ValidationResult -Results $results -Name 'Scaffolds use root AGENTS.md as the sole agent-instruction contract' -Action { + foreach ($variant in @('app', 'lib')) { + $skillRoot = "skills/dotnet-new-$variant-slnx" + $manifest = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/assets/shared.manifest.json" -GitRef $Ref | ConvertFrom-Json + $sharedFiles = @(Get-RepoFileList -RepoRoot $repoRoot -RelativePath "$skillRoot/assets/shared" -GitRef $Ref | ForEach-Object { $_ -replace '\\', '/' }) + if (@($manifest.files | Where-Object { $_ -ceq 'AGENTS.md' }).Count -ne 1) { + throw "$skillRoot must inventory exactly one root AGENTS.md." + } + $inventoryDrift = @(Compare-Object -ReferenceObject @($manifest.files) -DifferenceObject $sharedFiles -CaseSensitive) + if ($inventoryDrift.Count -gt 0) { + throw "$skillRoot shared manifest and assets disagree: $($inventoryDrift | Out-String)" + } + foreach ($path in @($manifest.files) + $sharedFiles) { + if ($path -match '(?i)copilot-instructions\.md$') { + throw "$skillRoot must not emit a Copilot-specific instruction file: $path" + } + } + $agents = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/assets/shared/AGENTS.md" -GitRef $Ref + $renderedAgents = Apply-Replacements -Content $agents -Map @{ + '{SOLUTION_NAME}' = 'GovernanceFixture' + '{PROJECT_NAME}' = 'GovernanceFixture.Library' + '{ROOT_NAMESPACE}' = 'GovernanceFixture' + '{TARGET_FRAMEWORK}' = 'net10.0' + '{TARGET_FRAMEWORKS}' = 'net10.0' + } + # Generic XML cref syntax in the full documentation example is not a scaffold placeholder. + Assert-NoUnexpectedPlaceholders -Name "$skillRoot rendered AGENTS.md" -Content $renderedAgents -Allowed @('{TOptions}') + Assert-Match -Name "$skillRoot AGENTS.md" -Content $agents -Pattern '^# Agent Instructions' + if ($agents -match '(?im)^(?:---\s*$|applyTo\s*:|description\s*:)' -or $agents -match '(?i)Copilot') { + throw "$skillRoot AGENTS.md must be ordinary, vendor-neutral Markdown without instruction frontmatter." + } + $headings = @([regex]::Matches($agents, '(?m)^#{1,6} .+$') | ForEach-Object { $_.Value.Trim() }) + if (@($headings | Group-Object | Where-Object Count -gt 1).Count -gt 0) { + throw "$skillRoot AGENTS.md contains duplicate sections." + } + # Check small semantic anchors and prohibitions, rather than duplicating entire prose sections. + foreach ($needle in @( + 'xunit.v3', 'Codebelt.Extensions.Xunit.App v12', 'Codebelt.Extensions.Xunit', + 'ITestOutputHelper', 'base(output)', 'TestOutput.WriteLine', '[Fact]', '[Theory]', '[InlineData]', + 'ShouldReturnTrue_WhenConditionIsMet', 'Assert.Equal', 'System Under Test (SUT)', 'RootNamespace', + 'dummies, fakes, stubs and spies', 'Moq', 'virtual or abstract', 'JsonMarshaller', + 'public APIs', 'observable behavior', 'BenchmarkDotNet', '*.Benchmarks', '*Benchmark*.cs', + 'tuning/', '[MemoryDiagnoser]', 'BenchmarkLogicalGroupRule.ByCategory', '[Params]', + '[GlobalSetup]', '[IterationSetup]', 'Baseline = true', 'Description', 'allocations', + 'All public APIs must have XML documentation comments', 'public and protected classes', '', '', '', '', '', + '', '', '.*?)^```\s*$')) { + $code = (($block.Groups['code'].Value -split '\r?\n' | ForEach-Object { $_.Trim() } | Where-Object { $_.Length -gt 0 }) -join "`n") + [System.Convert]::ToHexString([System.Security.Cryptography.SHA256]::HashData($utf8NoBom.GetBytes($code))).ToLowerInvariant() + } + ) + foreach ($example in $exampleFingerprints.Keys) { + if ($actualFingerprints -notcontains $exampleFingerprints[$example]) { + throw "$skillRoot AGENTS.md lost or altered the complete '$example' example." + } + } + $detailContracts = [ordered]@{ + '### File and Namespace Organization' = @('YourProject.Foo.Tests assembly', 'YourProject.Foo.FunctionalTests assembly', 'BooTest', 'automatically', 'type discovery and XML doc links', 'SUT <-> test pairs') + '#### Preferred Pattern' = @('Public Facade Testing', 'Public API Proxy Testing', '**Description:**') + '#### Example Mapping' = @('**Internal helper:**', '**Public API:**', '**Test strategy:**', 'DelimitedString.Create()') + '#### Benefits' = @('Avoids exposing internal types', 'real-world usage patterns', 'strong encapsulation', 'resilient to internal refactoring') + '#### When to Apply' = @('fully exercised', 'sufficient coverage', 'solely as a helper') + '## Code Coverage' = @('Test classes or test methods', 'Production code', 'Configuration code', 'Any other code path') + '### Coverage Rationale' = @('false confidence', 'refactor the code', 'Every executable line') + '### Coverage Alternatives' = @('**Untestable code paths**', '**External dependencies**', '**Configuration-only code**', '**Generated or third-party code**', 'dedicated vendor folders') + '### Structure and Best Practices' = @('micro, mid and macro', 'network, disk, DB', 'do not include them in CI', 'custom counters') + '### Additional XML Documentation Guidelines' = @('Gets or sets', 'Initializes a new instance of', 'Provides', 'Represents', 'options/settings classes', 'table format') + } + foreach ($heading in $detailContracts.Keys) { + $section = [regex]::Match($agents, '(?ms)^' + [regex]::Escape($heading) + '\r?\n(?.*?)(?=^#{1,4} |\z)') + if (-not $section.Success) { throw "$skillRoot AGENTS.md is missing '$heading'." } + foreach ($detail in $detailContracts[$heading]) { + Assert-Contains -Name "$skillRoot AGENTS.md $heading" -Content $section.Groups['body'].Value -Needle $detail + } + } + if ($variant -eq 'app') { + Assert-Contains -Name "$skillRoot AGENTS.md" -Content $agents -Needle 'FunctionalTests' + Assert-Contains -Name "$skillRoot AGENTS.md" -Content $agents -Needle 'Benchmarking is optional' + } else { + foreach ($needle in @('tooling/', 'reports/', '.docfx/api/namespaces/')) { + Assert-Contains -Name "$skillRoot AGENTS.md" -Content $agents -Needle $needle + } + } + $skill = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/SKILL.md" -GitRef $Ref + Assert-Contains -Name "$skillRoot SKILL.md" -Content $skill -Needle 'single generated agent-instruction source' + $referenceName = if ($variant -eq 'app') { 'app' } else { 'library' } + $guide = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/references/$referenceName.md" -GitRef $Ref + Assert-Contains -Name "$skillRoot reference" -Content $guide -Needle 'single generated agent-instruction source' + if ($variant -eq 'app') { + foreach ($file in $manifest.files) { + Assert-Contains -Name "$skillRoot shared inventory reference" -Content $guide -Needle $file + } + } + $bot = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/assets/shared/.bot/README.md" -GitRef $Ref + Assert-Contains -Name "$skillRoot .bot README" -Content $bot -Needle 'into root `AGENTS.md`.' + foreach ($path in (Get-RepoFileList -RepoRoot $repoRoot -RelativePath $skillRoot -GitRef $Ref)) { + if ([System.IO.Path]::GetExtension($path) -notin @('.md', '.json', '.ps1')) { continue } + $content = Get-FileText -RepoRoot $repoRoot -RelativePath "$skillRoot/$path" -GitRef $Ref + if ($content -match '(?i)copilot-instructions\.md|Copilot instructions|^\s*applyTo\s*:') { + throw "$skillRoot/$path still contains vendor-specific scaffold instruction semantics." + } + } + } +} + Add-ValidationResult -Results $results -Name 'Shared .bot assets are tracked and not ignored away' -Action { $appIgnore = Get-FileText -RepoRoot $repoRoot -RelativePath 'skills/dotnet-new-app-slnx/assets/shared/.gitignore' -GitRef $Ref $appBot = Get-FileText -RepoRoot $repoRoot -RelativePath 'skills/dotnet-new-app-slnx/assets/shared/.bot/README.md' -GitRef $Ref @@ -3690,7 +3823,10 @@ Add-ValidationResult -Results $results -Name 'Rendered library templates leave n foreach ($file in $files) { $rendered = Apply-Replacements -Content (Get-FileText -RepoRoot $repoRoot -RelativePath $file -GitRef $Ref) -Map $map - Assert-NoUnexpectedPlaceholders -Name $file -Content $rendered + # The retained XML documentation example uses {TOptions} in generic cref syntax. + # It is literal example code, not a scaffold substitution; other templates/tokens stay strict. + $allowed = if ($file -eq 'skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md') { @('{TOptions}') } else { @() } + Assert-NoUnexpectedPlaceholders -Name $file -Content $rendered -Allowed $allowed } } From 6504387568d8bcd2406e59b5c4c389195334330d Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Sun, 4 Oct 2026 22:55:33 +0200 Subject: [PATCH 09/13] =?UTF-8?q?=F0=9F=93=9A=20document=20vendor-neutral?= =?UTF-8?q?=20agent=20guidance=20in=20readme?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Describe the vendor-neutral root AGENTS.md contract so generated repositories advertise a single agent-instruction source. --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 86824ce..6a63fc9 100644 --- a/README.md +++ b/README.md @@ -564,6 +564,7 @@ Starting a new .NET solution "from scratch" usually means copying from your last > [!NOTE] > These scaffolds are not speculative starter kits. They capture conventions already exercised across Codebelt repositories and turn them into a repeatable methodology for new solutions. +- **Vendor-neutral agent guidance** — generated repositories use root `AGENTS.md` as their single agent-instruction contract, preserving the complete coding, testing, coverage, benchmarking, XML documentation, and `.bot/` workspace contract, including code examples, rationale, alternatives, and applicability conditions - **Convention over configuration** — opinionated defaults that match real production setups - **Focused skills** — library and app concerns are fully separated, no variant confusion - **Lower cognitive load** — the library scaffold defaults the main project name from the solution name, pre-fills the repository URL from the repo root folder name, and lets the package website reuse that value unless you override it From 882afa88e958e4171c1ce15dd56b810fbb71b0a4 Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Mon, 5 Oct 2026 20:34:31 +0200 Subject: [PATCH 10/13] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20clarify=20provisiona?= =?UTF-8?q?l=20versions=20and=20executable=20TFMs=20in=20skills?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Version-index selections stay provisional until metadata inspection plus combined restore and build confirm compatibility, while executable TFMs keep source-only frameworks out of test and benchmark targets. --- skills/dotnet-new-app-slnx/SKILL.md | 2 +- skills/dotnet-new-app-slnx/evals/evals.json | 1 + skills/dotnet-new-app-slnx/references/app.md | 2 +- skills/dotnet-new-lib-slnx/SKILL.md | 1 + skills/dotnet-new-lib-slnx/evals/evals.json | 1 + skills/dotnet-new-lib-slnx/references/library.md | 2 ++ 6 files changed, 7 insertions(+), 2 deletions(-) diff --git a/skills/dotnet-new-app-slnx/SKILL.md b/skills/dotnet-new-app-slnx/SKILL.md index 9f39d81..9413f34 100644 --- a/skills/dotnet-new-app-slnx/SKILL.md +++ b/skills/dotnet-new-app-slnx/SKILL.md @@ -94,7 +94,7 @@ Read `references/app.md` for the app-specific project structure, template file m Before writing `Directory.Packages.props`, resolve every `*_VERSION` placeholder in that file to the latest stable listed compatible version for its matching package ID on NuGet.org, respecting the xUnit v4 / Codebelt v12 lines below. -When `pwsh` 7+ is available, prefer the deterministic helper in `/scripts/resolve-package-versions.ps1` over manual lookup. Run it as `pwsh -NoProfile -File "/scripts/resolve-package-versions.ps1" -TargetFramework `. By default it resolves placeholders from this skill's own `assets/shared/Directory.Packages.props`, so a normal scaffold run only needs `-TargetFramework`. Treat its JSON output as the source of truth for package placeholders. +When `pwsh` 7+ is available, prefer the deterministic helper in `/scripts/resolve-package-versions.ps1` over manual lookup. Run it as `pwsh -NoProfile -File "/scripts/resolve-package-versions.ps1" -TargetFramework `. By default it resolves placeholders from this skill's own `assets/shared/Directory.Packages.props`, so a normal scaffold run only needs `-TargetFramework`. Its JSON supplies candidate versions for package placeholders, with `compatibility_status = provisional` and the requested `target_framework` on every entry. The helper filters version lines but does not inspect dependency ranges or framework assets. Inspect those metadata and restore and build the combined scaffold for the selected framework before accepting the candidates as compatible; halt and report any incompatibility. - Use the NuGet V3 service index at `https://api.nuget.org/v3/index.json` to discover the package metadata endpoints - Prefer registration metadata so you can ignore unlisted versions and prerelease builds diff --git a/skills/dotnet-new-app-slnx/evals/evals.json b/skills/dotnet-new-app-slnx/evals/evals.json index 20851eb..531a69c 100644 --- a/skills/dotnet-new-app-slnx/evals/evals.json +++ b/skills/dotnet-new-app-slnx/evals/evals.json @@ -22,6 +22,7 @@ "Names the generated solution file DemoApp.slnx instead of lowercasing it to demoapp.slnx", "Still generates the solution file and functional test project even for a single-host Worker scaffold", "Resolves worker package versions from NuGet instead of reusing stale remembered version numbers from prior runs", + "Treats resolver entries marked compatibility_status = provisional as candidates and inspects dependency ranges and framework assets plus combined restore/build before claiming compatibility", "Uses Codebelt.Extensions.Xunit.App 12.x with xunit.v3 and xunit.v3.runner.console 4.x, retaining the existing package IDs", "Generates root global.json selecting Microsoft.Testing.Platform and verifies a supported non-preview .NET 10+ SDK independently of the target runtime", "Uses centrally versioned Codebelt.Coverlet.MTP and Microsoft.Testing.Extensions.HangDump through the shared test-only ItemGroup without duplicate project references or legacy Coverlet packages", diff --git a/skills/dotnet-new-app-slnx/references/app.md b/skills/dotnet-new-app-slnx/references/app.md index bb0aeb2..be3036b 100644 --- a/skills/dotnet-new-app-slnx/references/app.md +++ b/skills/dotnet-new-app-slnx/references/app.md @@ -156,7 +156,7 @@ Resolve each package-specific `*_VERSION` placeholder in `Directory.Packages.pro Keep target-framework selection centralized too: the generated root `Directory.Build.props` owns `{TARGET_FRAMEWORK}` for source and test projects. Do **not** duplicate `` inside the generated app or test `.csproj` files as a workaround. -When `pwsh` 7+ is available, prefer `pwsh -NoProfile -File "/scripts/resolve-package-versions.ps1" -TargetFramework {TARGET_FRAMEWORK}` to produce the package placeholder map for this skill. The script defaults to this skill's own `assets/shared/Directory.Packages.props`, so the normal path only needs `{TARGET_FRAMEWORK}`. Its output should drive the final substitutions instead of remembered version numbers. +When `pwsh` 7+ is available, prefer `pwsh -NoProfile -File "/scripts/resolve-package-versions.ps1" -TargetFramework {TARGET_FRAMEWORK}` to produce the package placeholder map for this skill. The script defaults to this skill's own `assets/shared/Directory.Packages.props`, so the normal path only needs `{TARGET_FRAMEWORK}`. Its output supplies candidate substitutions instead of remembered version numbers. Every entry is marked `compatibility_status = provisional`: version-line selection does not verify dependency ranges or target-framework support. Inspect package metadata, then restore and build the combined scaffold for the selected framework before accepting the versions as compatible; stop and report incompatibilities. For framework-aligned ASP.NET packages, keep the selected target framework major in mind when resolving the final version: diff --git a/skills/dotnet-new-lib-slnx/SKILL.md b/skills/dotnet-new-lib-slnx/SKILL.md index a60ee19..61b3850 100644 --- a/skills/dotnet-new-lib-slnx/SKILL.md +++ b/skills/dotnet-new-lib-slnx/SKILL.md @@ -96,6 +96,7 @@ When copying template files, replace these placeholders in file contents: | `{REPO_OWNER}` | GitHub org/user (from URL) | | `{REPO_SLUG}` | Repo name (last URL segment, lowercased) | | `{TARGET_FRAMEWORKS}` | Computed from the official .NET releases index; offer the newest generally supported LTS, every other supported LTS or STS single-target choice, or all generally supported non-preview channels for broader scope | +| `{EXECUTABLE_TARGET_FRAMEWORKS}` | Selected executable TFMs in selection order for test and benchmark projects, excluding source-only TFMs such as `netstandard*`; ask for a consumer test runtime if none remain, and validate the combined test package set for every runtime | | `{DOCFX_TARGET_FRAMEWORK}` | Highest selected generally supported non-preview TFM used for DocFX metadata generation | | `{BENCHMARK_RUNNER_PROJECT_NAME}` | Tooling project name for the benchmark host (default `benchmark-runner`) | | `{BENCHMARK_RUNNER_NAMESPACE}` | Benchmark runner namespace derived from the tooling project name, replacing invalid identifier characters such as `-` with `_` | diff --git a/skills/dotnet-new-lib-slnx/evals/evals.json b/skills/dotnet-new-lib-slnx/evals/evals.json index b74bf8e..01ff6a2 100644 --- a/skills/dotnet-new-lib-slnx/evals/evals.json +++ b/skills/dotnet-new-lib-slnx/evals/evals.json @@ -18,6 +18,7 @@ "Uses centrally versioned Codebelt.Coverlet.MTP and Microsoft.Testing.Extensions.HangDump through the shared test-only ItemGroup without duplicate project references or legacy coverage engines", "Generates and executes a deterministic public-behavior test with the Codebelt Test base class instead of accepting zero discovery", "Verifies nonempty TRX and OpenCover artifacts using native MTP arguments for each executable test TFM, keeping source-only frameworks out of executable tests", + "Renders EXECUTABLE_TARGET_FRAMEWORKS for test and benchmark projects while preserving TARGET_FRAMEWORKS as the complete source matrix", "Uses xUnit's built-in --report-xunit-trx reporting without adding a separate report package or altering existing CI callers to request an alternate reporter" ] }, diff --git a/skills/dotnet-new-lib-slnx/references/library.md b/skills/dotnet-new-lib-slnx/references/library.md index b92e7ba..e10fd10 100644 --- a/skills/dotnet-new-lib-slnx/references/library.md +++ b/skills/dotnet-new-lib-slnx/references/library.md @@ -165,6 +165,8 @@ The generated `.github/dependabot.yml` should watch the repo root (`directory: " Follow `SKILL.md` for the coupled Codebelt xUnit 12.x / xUnit 4.x package contract. The package IDs remain `xunit.v3` and `xunit.v3.runner.console`. Copy root `global.json` selecting `Microsoft.Testing.Platform`, use a supported non-preview .NET 10+ SDK independently of library TFMs, and retain the test-only shared references to `Codebelt.Coverlet.MTP` and `Microsoft.Testing.Extensions.HangDump` without duplicate project references or legacy coverage integrations. +In root `Directory.Build.props`, render `{TARGET_FRAMEWORKS}` as the complete source matrix and `{EXECUTABLE_TARGET_FRAMEWORKS}` as the selected executable TFMs in selection order for tests and benchmarks. Exclude source-only TFMs such as `netstandard*` from the executable list; if it is empty, ask for a consumer test runtime. Validate package compatibility for each executable TFM before accepting the scaffold. + Generate at least one public-behavior test per library and use only executable TFMs for tests and benchmarks. Build Release, then run `dotnet test --project --framework -c Release --results-directory -- --report-xunit-trx --coverlet --coverlet-output-format opencover`; verify nonzero discovery, nonempty TRX/OpenCover output and runner help's hang-dump options. Inspect the actual shared CI workflow/action refs and report consumers before finalizing; preserve platform distinctions and avoid duplicating reporting/coverage/hang-dump arguments already provided by the shared action. Preserve the existing Codebelt app meta-package and report incompatible test-runtime choices rather than introducing compatibility workarounds. --- From 8df9e2260932c8f1d7716c42a2297de0bc577f28 Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Mon, 5 Oct 2026 20:34:46 +0200 Subject: [PATCH 11/13] =?UTF-8?q?=F0=9F=8D=B1=20adjust=20scaffold=20templa?= =?UTF-8?q?tes=20for=20executable=20TFMs=20and=20resolver?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Test and benchmark templates render the executable TFM subset while the resolver marks version-index selections provisional, and shared agent guidance drops the duplicated functional-test namespace rule. --- skills/dotnet-new-app-slnx/assets/shared/AGENTS.md | 2 -- .../dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 | 3 +++ .../dotnet-new-lib-slnx/assets/library/Directory.Build.props | 4 ++-- skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md | 2 -- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md b/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md index cedb277..296c1cf 100644 --- a/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md +++ b/skills/dotnet-new-app-slnx/assets/shared/AGENTS.md @@ -234,8 +234,6 @@ public class SampleTest : Test - **Configuration-only code**: Move to configuration files or extract into testable methods. - **Generated or third-party code**: These should not be in the primary codebase; use NuGet packages or dedicated vendor folders if necessary. -Do not append .FunctionalTests to test namespaces either; the same SUT namespace equality rule applies. - ## Benchmarking Benchmarking is optional for application repositories. The following rules apply when benchmarks are added under `tuning/`, in `*.Benchmarks` projects, or in `*Benchmark*.cs` source files. diff --git a/skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 b/skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 index ce06e41..aaca8d9 100644 --- a/skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 +++ b/skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1 @@ -127,7 +127,10 @@ foreach ($node in $packageNodes) { $result[$placeholder] = [ordered]@{ package_id = $packageId version = $resolved + target_framework = $TargetFramework + compatibility_status = 'provisional' } } +Write-Warning 'Version-index selection is provisional. Inspect package assets and dependency ranges, then restore and build the combined scaffold for the selected framework before treating these versions as compatible.' $result | ConvertTo-Json -Depth 4 diff --git a/skills/dotnet-new-lib-slnx/assets/library/Directory.Build.props b/skills/dotnet-new-lib-slnx/assets/library/Directory.Build.props index fa676cf..df2ce73 100644 --- a/skills/dotnet-new-lib-slnx/assets/library/Directory.Build.props +++ b/skills/dotnet-new-lib-slnx/assets/library/Directory.Build.props @@ -51,7 +51,7 @@ - {TARGET_FRAMEWORKS} + {EXECUTABLE_TARGET_FRAMEWORKS} Exe false false @@ -79,7 +79,7 @@ - {TARGET_FRAMEWORKS} + {EXECUTABLE_TARGET_FRAMEWORKS} false false false diff --git a/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md b/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md index 35358b0..a1be490 100644 --- a/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md +++ b/skills/dotnet-new-lib-slnx/assets/shared/AGENTS.md @@ -240,8 +240,6 @@ public class SampleTest : Test - **Configuration-only code**: Move to configuration files or extract into testable methods. - **Generated or third-party code**: These should not be in the primary codebase; use NuGet packages or dedicated vendor folders if necessary. -Do not append .FunctionalTests to test namespaces either; the same SUT namespace equality rule applies. - ## Benchmarking The following rules apply to benchmark projects (`*.Benchmarks`), code under `tuning/` and `tooling/`, and `*Benchmark*.cs` source files. Namespace rules for benchmark types refer to the production code being measured; the executable benchmark runner host retains its own tooling-derived namespace. From bbfdee90ab356b59a1de320b1f004db23de1ff00 Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Mon, 5 Oct 2026 20:34:53 +0200 Subject: [PATCH 12/13] =?UTF-8?q?=E2=9C=85=20expand=20scaffold=20MTP=20reg?= =?UTF-8?q?ressions=20and=20guard=20ref=20execution?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Revision-aware validation skips working-tree scaffold execution while the MTP harness covers provisional status, net8 incompatibility, executable TFM rendering, and benchmark projects. --- scripts/test-validation-suites.ps1 | 26 +++++ scripts/tests/test-scaffold-mtp.ps1 | 142 ++++++++++++++++++++++----- scripts/validate-skill-templates.ps1 | 13 ++- 3 files changed, 153 insertions(+), 28 deletions(-) diff --git a/scripts/test-validation-suites.ps1 b/scripts/test-validation-suites.ps1 index 8f4a780..43f31d8 100644 --- a/scripts/test-validation-suites.ps1 +++ b/scripts/test-validation-suites.ps1 @@ -76,3 +76,29 @@ Write-Output 'CI suite coverage: PASS' } Write-Output 'Grouping reference layouts: PASS (6 cases)' } + +# A revision check must never register a regression that reads working-tree scaffolds. +& { + $tokens = $null + $parseErrors = $null + $validatorAst = [Management.Automation.Language.Parser]::ParseFile((Join-Path $repoRoot 'scripts/validate-skill-templates.ps1'), [ref]$tokens, [ref]$parseErrors) + $command = $validatorAst.Find({ param($node) $node -is [Management.Automation.Language.CommandAst] -and $node.GetCommandName() -eq 'Add-ValidationResult' -and $node.Extent.Text.Contains("-Name 'Generated app and library scaffolds execute Codebelt v12 tests with native MTP artifacts'") }, $true) + if ($null -eq $command) { throw 'Missing scaffold MTP validation entry.' } + $guard = $command.Parent + while ($null -ne $guard -and $guard -isnot [Management.Automation.Language.IfStatementAst]) { $guard = $guard.Parent } + if ($null -eq $guard) { throw 'Working-tree scaffold execution must be guarded in ref mode.' } + $registered = [Collections.Generic.List[string]]::new() + function Add-ValidationResult { + param($Results, $Name, $Action) + $registered.Add($Name) + } + $results = @() + $Suite = 'Templates' + foreach ($Ref in @('', ' ', 'HEAD', 'fixture-ref')) { + $registered.Clear() + & ([scriptblock]::Create($guard.Extent.Text)) + $expected = if ([string]::IsNullOrWhiteSpace($Ref)) { 1 } else { 0 } + if ($registered.Count -ne $expected) { throw "Scaffold MTP ref guard failed for '$Ref'." } + } + Write-Output 'Scaffold MTP ref scope: PASS (4 cases)' +} diff --git a/scripts/tests/test-scaffold-mtp.ps1 b/scripts/tests/test-scaffold-mtp.ps1 index a12409d..0f38662 100644 --- a/scripts/tests/test-scaffold-mtp.ps1 +++ b/scripts/tests/test-scaffold-mtp.ps1 @@ -29,6 +29,16 @@ function Invoke-DotNet { return ($output -join [Environment]::NewLine) } +function Assert-DotNetFailure { + param([string[]]$Arguments, [string]$ExpectedError) + $output = @(& dotnet @Arguments 2>&1) + $exitCode = $LASTEXITCODE + $diagnostics = $output -join [Environment]::NewLine + if ($exitCode -eq 0 -or $diagnostics -notmatch $ExpectedError) { + throw "dotnet $($Arguments -join ' ') must fail with '$ExpectedError' (exit $exitCode):`n$diagnostics" + } +} + try { [void][System.IO.Directory]::CreateDirectory($workspace) # Test version-line selection with a newer future major, prereleases, and mismatched ASP.NET majors. @@ -41,7 +51,7 @@ try { $versions = switch -Regex ($Uri) { '/codebelt.extensions.xunit.app/' { @('11.2.1', '12.0.0', '12.0.1', '12.1.0-preview.1', '13.0.0'); break } '/xunit.v3(?:.runner.console)?/' { @('3.2.2', '4.0.0', '4.0.1', '4.1.0-preview.1', '5.0.0'); break } - '/microsoft.aspnetcore.' { @('9.0.14', '10.0.12', '11.0.0'); break } + '/microsoft.aspnetcore.' { @('8.0.31', '9.0.14', '10.0.12', '11.0.0'); break } default { @('1.0.0', '1.0.1', '2.0.0-preview.1') } } return @{ versions = $versions } @@ -55,6 +65,16 @@ try { )) { if ($resolved.($pair[0]).version -ne $pair[1]) { throw "Resolver selected the wrong API line for $($pair[0])." } } + foreach ($entry in $resolved.PSObject.Properties.Value) { + if ($entry.compatibility_status -ne 'provisional' -or $entry.target_framework -ne 'net10.0') { + throw 'Version-index lookup must not claim verified compatibility.' + } + } + $olderRuntime = & (Join-Path $repoRoot 'skills/dotnet-new-app-slnx/scripts/resolve-package-versions.ps1') -TargetFramework net8.0 | ConvertFrom-Json + if ($olderRuntime.'{CODEBELT_EXTENSIONS_XUNIT_APP_VERSION}'.compatibility_status -ne 'provisional' -or + $olderRuntime.'{CODEBELT_EXTENSIONS_XUNIT_APP_VERSION}'.target_framework -ne 'net8.0') { + throw 'The resolver must preserve provisional status for an unsupported test runtime.' + } Write-Host '[PASS] Resolver preserves authorized test-stack majors and excludes prereleases.' } @@ -85,13 +105,17 @@ try { '{BENCHMARKDOTNET_DIAGNOSTICS_WINDOWS_VERSION}' = '0.15.8' '{CODEBELT_EXTENSIONS_BENCHMARKDOTNET_CONSOLE_VERSION}' = '1.3.4' } - # Exercise the app meta-package's net9/net10 runtime groups; do not add older-runtime workarounds. + # net8 is an incompatibility regression for the fixed v12 stack, not an older-stack workaround. + # Only libraries support multiple source TFMs, including source-only netstandard. $cases = @( @{ Variant = 'app'; HostType = 'web'; AppType = 'Web'; Tfm = 'net10.0' }, @{ Variant = 'app'; HostType = 'console'; AppType = 'Console'; Tfm = 'net9.0' }, @{ Variant = 'app'; HostType = 'worker'; AppType = 'Worker'; Tfm = 'net10.0' }, + @{ Variant = 'app'; HostType = 'console'; AppType = 'Console'; Tfm = 'net8.0' }, @{ Variant = 'lib'; Tfm = 'net10.0' }, - @{ Variant = 'lib'; Tfm = 'net9.0' } + @{ Variant = 'lib'; Tfm = 'net9.0' }, + @{ Variant = 'lib'; Tfm = 'net8.0' }, + @{ Variant = 'lib'; Tfm = 'netstandard2.0;net9.0;net10.0' } ) foreach ($case in $cases) { $variant = $case.Variant @@ -99,7 +123,8 @@ try { $skillRoot = "skills/dotnet-new-$variant-slnx" $projectName = if ($variant -eq 'app') { "Acme.$($case.AppType)" } else { 'Acme.Library' } $testSuffix = if ($variant -eq 'app') { 'FunctionalTests' } else { 'Tests' } - $caseName = "$projectName-$($case.Tfm)" + $runtimeTfms = @($case.Tfm.Split(';') | Where-Object { $_ -notlike 'netstandard*' }) + $caseName = "$projectName-$($case.Tfm.Replace(';', '-'))" $caseRoot = Join-Path $workspace $caseName $map = $versions.Clone() $map['{ROOT_NAMESPACE}'] = if ($variant -eq 'app') { 'Acme' } else { 'Acme.Library' } @@ -108,6 +133,13 @@ try { $map['{AppType}'] = if ($variant -eq 'app') { $case.AppType } else { '' } $map['{TARGET_FRAMEWORK}'] = $case.Tfm $map['{TARGET_FRAMEWORKS}'] = $case.Tfm + $map['{EXECUTABLE_TARGET_FRAMEWORKS}'] = $runtimeTfms -join ';' + $map['{BENCHMARK_RUNNER_TARGET_FRAMEWORK}'] = $runtimeTfms[-1] + $map['{BENCHMARK_RUNNER_NAMESPACE}'] = 'benchmark_runner' + $benchmarkRuntimes = @{ 'net8.0' = 'Core80'; 'net9.0' = 'Core90'; 'net10.0' = 'Core10_0' } + $map['{BENCHMARK_RUNTIME_JOBS}'] = ($runtimeTfms | ForEach-Object { + " .AddJob(slimJob.WithRuntime(CoreRuntime.$($benchmarkRuntimes[$_])))" + }) -join [Environment]::NewLine $map['{AUTHOR}'] = 'Scaffold Regression' $map['{AUTHOR_EMAIL}'] = 'fixture@example.invalid' $map['{COMPANY_OR_PERSON}'] = 'Scaffold Regression' @@ -129,6 +161,9 @@ try { foreach ($rule in @('InternalsVisibleTo', 'ExcludeFromCodeCoverage', 'Xunit.Abstractions', 'BenchmarkDotNet', 'RootNamespace', '')) { if (-not $agents.Contains($rule)) { throw "$caseName root AGENTS.md lost the $rule guidance." } } + if ([regex]::Matches($agents, 'Do not append `?\.FunctionalTests`? to test namespaces').Count -ne 1) { + throw "$caseName must keep the functional-test namespace rule exactly once." + } $bot = [System.IO.File]::ReadAllText((Join-Path $caseRoot '.bot/README.md')) if (-not $bot.Contains('into root `AGENTS.md`.') -or $bot -match '(?i)Copilot') { throw "$caseName .bot guidance must point only to root AGENTS.md." @@ -214,30 +249,89 @@ public class BehaviorTest : Test } } "@ - Write-Text -Path (Join-Path $caseRoot 'MtpScaffold.slnx') -Content "" + $solutionProjects = @($sourceProject, $testProject) + if ($variant -eq 'lib') { + $benchmarkProject = "tuning/$projectName.Benchmarks/$projectName.Benchmarks.csproj" + $runnerProject = 'tooling/benchmark-runner/benchmark-runner.csproj' + Render-Template -Source "$skillRoot/assets/library/benchmark.csproj" -Destination (Join-Path $caseRoot $benchmarkProject) -Map $map + Render-Template -Source "$skillRoot/assets/library/benchmark-runner.csproj" -Destination (Join-Path $caseRoot $runnerProject) -Map $map + Render-Template -Source "$skillRoot/assets/library/benchmark-program.cs" -Destination (Join-Path $caseRoot 'tooling/benchmark-runner/Program.cs') -Map $map + Write-Text -Path (Join-Path $caseRoot "tuning/$projectName.Benchmarks/PriceCalculatorBenchmark.cs") -Content @' +using BenchmarkDotNet.Attributes; + +namespace Acme.Library; + +public class PriceCalculatorBenchmark +{ + [Benchmark] + public decimal AddTax() => PriceCalculator.AddTax(100m, 0.20m); +} +'@ + $solutionProjects += @($benchmarkProject, $runnerProject) + } + $projectElements = ($solutionProjects | ForEach-Object { "" }) -join '' + Write-Text -Path (Join-Path $caseRoot 'MtpScaffold.slnx') -Content "$projectElements" Push-Location $caseRoot try { - [void](Invoke-DotNet -Arguments @('build', 'MtpScaffold.slnx', '-c', 'Release', '-p:SkipSignAssembly=true', '--nologo')) - $help = Invoke-DotNet -Arguments @('test', '--project', $testProject, '-c', 'Release', '--no-build', '--help') - $reportOption = '--report-xunit-trx' - foreach ($option in @($reportOption, '--coverlet', '--coverlet-output-format', '--hangdump', '--hangdump-timeout')) { - if (-not $help.Contains($option)) { throw "$caseName runner is missing $option." } + if ($variant -eq 'lib') { + foreach ($pair in @( + @($sourceProject, $case.Tfm), + @($testProject, ($runtimeTfms -join ';')), + @($benchmarkProject, ($runtimeTfms -join ';')) + )) { + $actual = Invoke-DotNet -Arguments @('msbuild', $pair[0], '-getProperty:TargetFrameworks', '--nologo') + if ($actual.Trim() -ne $pair[1]) { throw "$caseName rendered incorrect TargetFrameworks for $($pair[0]): $actual" } + } + $runnerTfm = Invoke-DotNet -Arguments @('msbuild', $runnerProject, '-getProperty:TargetFramework', '--nologo') + if ($runnerTfm.Trim() -ne $runtimeTfms[-1]) { throw "$caseName benchmark runner must target the highest executable TFM." } + Write-Host "[PASS] ${caseName}: source matrix retained; tests, benchmarks and runner use executable TFMs only." } - $run = Invoke-DotNet -Arguments @('test', '--project', $testProject, '--framework', $case.Tfm, '-c', 'Release', '--no-build', '--results-directory', 'TestResults', '--', $reportOption, '--coverlet', '--coverlet-output-format', 'opencover') - $trxFiles = @(Get-ChildItem -LiteralPath (Join-Path $caseRoot 'TestResults') -Filter '*.trx' -Recurse) - $coverageFiles = @(Get-ChildItem -LiteralPath (Join-Path $caseRoot 'TestResults') -Filter '*.xml' -Recurse | Where-Object { $_.Length -gt 0 }) - if ($trxFiles.Count -eq 0) { throw "$caseName emitted no TRX.`n$run" } - foreach ($file in $trxFiles) { - [xml]$trx = [System.IO.File]::ReadAllText($file.FullName) - $counts = $trx.TestRun.ResultSummary.Counters - if ([int]$counts.total -ne 1 -or [int]$counts.passed -ne 1 -or [int]$counts.failed -ne 0) { throw "$caseName did not discover and pass its behavior test.`n$run" } + if ($case.Tfm -eq 'net8.0') { + if ($variant -eq 'lib') { + [void](Invoke-DotNet -Arguments @('build', $benchmarkProject, '-c', 'Release', '-p:SkipSignAssembly=true', '--nologo')) + Assert-DotNetFailure -Arguments @('build', $runnerProject, '-c', 'Release', '-p:SkipSignAssembly=true', '--nologo') -ExpectedError 'error NU1202: Package Codebelt.Extensions.BenchmarkDotNet.Console' + } + $expectedError = if ($variant -eq 'app') { 'error NU1202: Package Codebelt.Bootstrapper.Console' } else { "error CS0246: The type or namespace name 'Codebelt'" } + Assert-DotNetFailure -Arguments @('build', $testProject, '-c', 'Release', '-p:SkipSignAssembly=true', '--nologo') -ExpectedError $expectedError + $testAssetsPath = Join-Path $caseRoot (Join-Path (Split-Path $testProject) 'obj/project.assets.json') + $testAssets = [System.IO.File]::ReadAllText($testAssetsPath) | ConvertFrom-Json + $nuspecPaths = @($testAssets.packageFolders.PSObject.Properties.Name | ForEach-Object { + Join-Path $_ 'codebelt.extensions.xunit.app/12.0.1/codebelt.extensions.xunit.app.nuspec' + } | Where-Object { Test-Path -LiteralPath $_ -PathType Leaf }) + if ($nuspecPaths.Count -ne 1) { throw "$caseName could not locate the restored Codebelt app meta-package metadata." } + [xml]$nuspec = [System.IO.File]::ReadAllText($nuspecPaths[0]) + $groups = @($nuspec.package.metadata.dependencies.group.targetFramework) + if ($groups -contains 'net8.0' -or $groups.Count -ne 2 -or $groups -notcontains 'net9.0' -or $groups -notcontains 'net10.0') { + throw "$caseName expected only net9/net10 dependency groups in the fixed Codebelt app meta-package." + } + Write-Host "[PASS] ${caseName}: incompatible net8 stack fails explicitly; Codebelt app meta-package has no net8 dependency group." + continue } - $covered = $false - foreach ($file in $coverageFiles) { - [xml]$coverage = [System.IO.File]::ReadAllText($file.FullName) - if ($null -ne $coverage.SelectSingleNode("/CoverageSession/Modules/Module[ModuleName='$projectName']/Classes/Class/Methods/Method/SequencePoints/SequencePoint[number(@vc) > 0]")) { $covered = $true } + [void](Invoke-DotNet -Arguments @('build', 'MtpScaffold.slnx', '-c', 'Release', '-p:SkipSignAssembly=true', '--nologo')) + foreach ($runtimeTfm in $runtimeTfms) { + $help = Invoke-DotNet -Arguments @('test', '--project', $testProject, '--framework', $runtimeTfm, '-c', 'Release', '--no-build', '--help') + $reportOption = '--report-xunit-trx' + foreach ($option in @($reportOption, '--coverlet', '--coverlet-output-format', '--hangdump', '--hangdump-timeout')) { + if (-not $help.Contains($option)) { throw "$caseName runner is missing $option." } + } + $resultsDirectory = "TestResults/$runtimeTfm" + $run = Invoke-DotNet -Arguments @('test', '--project', $testProject, '--framework', $runtimeTfm, '-c', 'Release', '--no-build', '--results-directory', $resultsDirectory, '--', $reportOption, '--coverlet', '--coverlet-output-format', 'opencover') + $trxFiles = @(Get-ChildItem -LiteralPath (Join-Path $caseRoot $resultsDirectory) -Filter '*.trx' -Recurse) + $coverageFiles = @(Get-ChildItem -LiteralPath (Join-Path $caseRoot $resultsDirectory) -Filter '*.xml' -Recurse | Where-Object { $_.Length -gt 0 }) + if ($trxFiles.Count -eq 0) { throw "$caseName emitted no TRX.`n$run" } + foreach ($file in $trxFiles) { + [xml]$trx = [System.IO.File]::ReadAllText($file.FullName) + $counts = $trx.TestRun.ResultSummary.Counters + if ([int]$counts.total -ne 1 -or [int]$counts.passed -ne 1 -or [int]$counts.failed -ne 0) { throw "$caseName did not discover and pass its behavior test.`n$run" } + } + $covered = $false + foreach ($file in $coverageFiles) { + [xml]$coverage = [System.IO.File]::ReadAllText($file.FullName) + if ($null -ne $coverage.SelectSingleNode("/CoverageSession/Modules/Module[ModuleName='$projectName']/Classes/Class/Methods/Method/SequencePoints/SequencePoint[number(@vc) > 0]")) { $covered = $true } + } + if (-not $covered) { throw "$caseName emitted no visited source sequence points in OpenCover.`n$run" } + Write-Host "[PASS] ${caseName}/${runtimeTfm}: passing behavior test, xUnit TRX, visited OpenCover points and HangDump options." } - if (-not $covered) { throw "$caseName emitted no visited source sequence points in OpenCover.`n$run" } $assets = [System.IO.File]::ReadAllText((Join-Path $caseRoot (Join-Path (Split-Path $testProject) 'obj/project.assets.json'))) foreach ($id in @('Codebelt.Extensions.Xunit.App/12.0.1', 'xunit.v3/4.0.1', 'Codebelt.Coverlet.MTP/10.1.0', 'Microsoft.Testing.Extensions.HangDump/2.4.1')) { if (-not $assets.Contains($id)) { throw "$caseName is missing restored dependency $id." } @@ -246,7 +340,7 @@ public class BehaviorTest : Test if (@($graph.libraries.PSObject.Properties.Name | Where-Object { $_ -like 'Microsoft.Testing.Extensions.TrxReport/*' }).Count -ne 0) { throw "$caseName restored an out-of-scope TRX provider." } - Write-Host "[PASS] ${caseName}: Release build, managed/public behavior, one passing test, xUnit TRX, visited OpenCover points, HangDump options and no separate TRX provider." + Write-Host "[PASS] ${caseName}: Release build with compatible dependencies and no separate TRX provider." } finally { Pop-Location } } Write-Host '[PASS] Scaffold MTP regressions passed.' diff --git a/scripts/validate-skill-templates.ps1 b/scripts/validate-skill-templates.ps1 index a091290..5bd0556 100644 --- a/scripts/validate-skill-templates.ps1 +++ b/scripts/validate-skill-templates.ps1 @@ -1110,11 +1110,15 @@ Add-ValidationResult -Results $results -Name 'Both scaffolds select xUnit 4, Cod } } -Add-ValidationResult -Results $results -Name 'Generated app and library scaffolds execute Codebelt v12 tests with native MTP artifacts' -Action { - $output = & pwsh -NoProfile -NonInteractive -File (Join-Path $repoRoot 'scripts/tests/test-scaffold-mtp.ps1') 2>&1 - if ($LASTEXITCODE -ne 0) { - throw ($output -join [Environment]::NewLine) +if ([string]::IsNullOrWhiteSpace($Ref)) { + Add-ValidationResult -Results $results -Name 'Generated app and library scaffolds execute Codebelt v12 tests with native MTP artifacts' -Action { + $output = & pwsh -NoProfile -NonInteractive -File (Join-Path $repoRoot 'scripts/tests/test-scaffold-mtp.ps1') 2>&1 + if ($LASTEXITCODE -ne 0) { + throw ($output -join [Environment]::NewLine) + } } +} elseif ($Suite -in @('All', 'Templates')) { + Write-Host '[SKIP] Scaffold MTP execution reads working-tree assets; ref validation uses revision-aware template checks only.' } Add-ValidationResult -Results $results -Name 'App package template uses specific version placeholders' -Action { @@ -3811,6 +3815,7 @@ Add-ValidationResult -Results $results -Name 'Rendered library templates leave n '{REPO_OWNER}' = 'acme' '{REPO_SLUG}' = 'mylibrary' '{TARGET_FRAMEWORKS}' = 'net10.0;net8.0' + '{EXECUTABLE_TARGET_FRAMEWORKS}' = 'net10.0;net8.0' '{DOCFX_TARGET_FRAMEWORK}' = 'net10.0' '{BENCHMARK_RUNNER_PROJECT_NAME}' = 'benchmark-runner' '{BENCHMARK_RUNNER_NAMESPACE}' = 'benchmark_runner' From f93883288819c4a7c92eaead07f10fbbbb181efe Mon Sep 17 00:00:00 2001 From: aicia-bot Date: Mon, 5 Oct 2026 20:35:00 +0200 Subject: [PATCH 13/13] =?UTF-8?q?=F0=9F=93=9A=20describe=20executable=20TF?= =?UTF-8?q?Ms=20and=20provisional=20versions=20in=20readme?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Keep the solution overview in sync with the scaffold behavior for source-only framework exclusion and provisional package compatibility. --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 6a63fc9..da2c52f 100644 --- a/README.md +++ b/README.md @@ -561,6 +561,8 @@ Starting a new .NET solution "from scratch" usually means copying from your last **dotnet-new-lib-slnx** and **dotnet-new-app-slnx** encode the full codebeltnet convention into repeatable scaffolds — from `Directory.Build.props` to CI pipelines to DocFX. Each skill is focused on its domain: libraries get multi-target frameworks, signing, and NuGet packaging; apps get host family selection, a conditional web-variant choice when needed, hosting patterns, and functional tests. +Library scaffolds keep the source framework matrix separate from executable test and benchmark targets, excluding source-only frameworks such as `netstandard2.0` from those projects. The app package resolver marks version selections as provisional until package metadata inspection and combined restore/build validation establish compatibility for the selected framework. + > [!NOTE] > These scaffolds are not speculative starter kits. They capture conventions already exercised across Codebelt repositories and turn them into a repeatable methodology for new solutions.