Skip to content

feat: tool titles and hints, default-free schemas, contract tests over tools/list - #50

Merged
Arthurvdv merged 4 commits into
mainfrom
feat/42-tool-contract
Oct 4, 2026
Merged

Arthurvdv merged 4 commits into
mainfrom
feat/42-tool-contract

Conversation

@Arthurvdv

@Arthurvdv Arthurvdv commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Summary

The published tools/list is the only contract most MCP clients see. This PR makes the five native tools publish a complete, clean contract and puts it under test.

  • Titles and hints. Every native tool now sets Title and all four hints explicitly (an omitted hint publishes nothing):

    Tool Title readOnly destructive idempotent openWorld
    analyze Analyze AL project true false true false
    list_rules List analyzer rules true false true false
    get_fixes Get code fixes true false true false
    apply_fix Apply code fix false false false false
    apply_fix_all Apply fix to all occurrences false false false false
  • Default-free input schemas. Microsoft.Extensions.AI emitted a JSON-schema default for every parameter with a C# default, including "default": null on every optional string. McpHost.NativeToolSchemaOptions strips the keyword from every schema node with a custom TransformSchemaNode. Parameter descriptions state the non-null defaults instead (apply_fix_all.dryRun gained "Default: false.").

  • Own registration loop. WithToolsFromAssembly() has no way to pass schema options, so McpHost now registers each [McpServerTool] method with McpServerTool.Create(method, null, new McpServerToolCreateOptions { Services = sp, SchemaCreateOptions = ... }). Services = sp keeps DI parameters out of the schema, as the SDK's own registration does.

  • McpHost.ConfigureServices is split out of RunAsync ([NoInlining] stays on RunAsync) and is the test seam.

  • Tools/ToolDescriptions.cs holds the titles and the two pinned sentences (AnalyzePreferOverAlCompile, VerifyAfterWrite). The attributes use these constants.

  • Contract tests (tests/ALCops.Mcp.Tests/Contracts/ToolContractTests.cs, 32 cases) build the real container with --no-proxy and serialize each tool with McpJsonUtilities.DefaultOptions, which gives the tools/list wire shape. They check:

    • exactly five tools and no custom list/call handler under --no-proxy, and the same five tools plus both handlers in the default mode;
    • no default keyword anywhere in a schema;
    • properties and required equal the method's non-service parameters (through IServiceProviderIsService) and match a literal list per tool;
    • every [McpServerTool] method is public static;
    • the titles and all four hints, both on the object model and on the wire;
    • the pinned sentences, as literals;
    • every non-null default stated in its parameter's description;
    • analyze requires no arguments.
  • The old AnalyzeToolTests.Schema_ExposesUserParameters_HidesServicesAndCancellationToken is removed. It built the tool without Services, so DI parameters leaked into its schema unnoticed. The new property/required test supersedes it.

  • Docs. src/ALCops.Mcp/README.md gets a native-tool table (Tool, Title, What it does, Writes files). AGENTS.md describes the registration loop, the annotation table, the schema transform, ToolDescriptions and the test seam.

Deviations from the issue

  • No WithToolsFromAssembly(..., McpServerToolCreateOptions) overload exists in ModelContextProtocol 0.9.0-preview.2. The only overload takes (Assembly?, JsonSerializerOptions?), and SchemaCreateOptions can only be passed through McpServerTool.Create(MethodInfo, target, options). That is why the tools are registered by an own loop. No DelegatingMcpServerTool wrapper is needed: the SDK copies the attribute's title and hints onto both Tool.Title and Tool.Annotations.
  • AIJsonSchemaTransformOptions.MoveDefaultKeywordToDescription was rejected. It appends (Default value: null) to every nullable optional parameter and repeats defaults the descriptions already state. A custom TransformSchemaNode removes the keyword instead.
  • CI runs tests on Ubuntu only, as the BC DevTools version matrix. "Passes on Windows and Ubuntu CI" therefore means: passes locally on Windows and on the Ubuntu CI matrix.
  • The README had no tool table, only a prose line. This PR adds a compact one.
  • No breaking prerelease pairing with One error envelope and a fixed error vocabulary for all native tools #41. Every push to main already publishes an alpha, and One error envelope and a fixed error vocabulary for all native tools #41 is out as 0.3.0-alpha.3. This change is not breaking for clients (it adds title and hints and removes a keyword), so it is feat: without !.
  • The proxied al_* tools are untouched, as the issue scopes.

Review notes

Carried as designed (raised in review, not changed):

  • destructiveHint: false on apply_fix / apply_fix_all: a code fix can rewrite or remove lines, but every write is guarded (stale check, atomic move, batch rollback) and touches only the lines the fix changes; prompting on every fix would defeat the autonomous-agent use case. This is the issue's table and a maintainer decision.
  • openWorldHint: false everywhere: the tools act only on the local AL project, and analyze reaches only the local almcp child process. This is the issue's table.
  • Every default keyword is stripped, not only null ones: the descriptions state the non-null defaults (enforced by a contract test), and one rule is simpler than two.
  • The own registration loop with lazy sp => factories: this is the same pattern as the SDK's WithTools<T>. Each tool is created when the container first resolves McpServerTool, with Services = sp.
  • The contract table appears in the README, AGENTS.md and the test data: each copy serves a different reader, and the tests fail on any drift from the attributes.
  • NativeToolMethods throws at startup for a tool method that is not public static. This is kept on purpose: the contract test catches it in CI first, and leaving the tool out of tools/list silently would be worse.
  • Tools are found by scanning the assembly, not from an explicit list. This is kept: the assembly is small, and the BC assembly resolver is registered before RunAsync.
  • TransformSchemaNode removes default from every JsonObject it is handed. No guard was added for a parameter literally named default: it would need @default in C#, and its name would only ever appear as a key of the properties map, which the SDK does not pass to the callback.
  • openWorldHint=false stays even though ALCops analyzers are provisioned from NuGet. That fetch is a hosted service that runs at startup whether or not any tool is called. It belongs to the server's startup, not to a tool call.

Round 1 (fixed)

  • Parameter names are now pinned as literals: InputSchema_PublishesThePinnedParameters gives the exact ordered properties and the required set per tool, next to the reflection-based DI-leak test.
  • [McpServerTool] methods that are not public static now fail loudly: McpHost.NativeToolMethods throws InvalidOperationException naming the method. EveryToolMethodInTheAssembly_IsPublicStatic pins the convention.
  • The fixture no longer depends on Metadata[0]: it uses Metadata.OfType<MethodInfo>().Single() with an explicit assertion message.

Round 2 (fixed)

  • Proxy-on registration is now tested. ToolsList_IsTheSameFiveNativeTools_WithProxyHandlers_UnderDefaultMode builds the container without --no-proxy. No host runs and AlMcpProxy is never resolved. The test asserts the same five tools, a non-null ListToolsHandler / CallToolHandler, and the five tools in ToolCollection.
  • ParameterDescriptions_StateEveryNonNullDefault is tighter: the default value must sit right next to the word "default" ((default 500), Default: false., 'project' (default, ...). Before, both only had to appear somewhere in the text.

Round 3 (fixed)

  • The default-wording regex gets a left word boundary ((?<!\w)) before the value in its first alternative. Before, "(default 1500)" matched the value 500.

Test plan

  • dotnet build --configuration Release: 0 warnings, 0 errors.
  • dotnet test --configuration Release: 376 passed, 0 failed, 0 skipped. That is 32 new contract cases and 1 superseded test removed.
  • Mutation checks, run locally and reverted:
    • Without the default removal, the 5 InputSchema_HasNoDefaultKeywordAnywhere cases fail.
    • Without Services = sp, the 5 property/required cases and Analyze_RequiresNoArguments fail.
  • End-to-end wire check: the built server ran with --no-proxy --alcops-analyzers off and was sent initialize, notifications/initialized and tools/list. The reply had five tools, each with title and all four hints, and contained zero "default" occurrences.
  • Ubuntu CI matrix (run 37200957815, all 7 BC DevTools versions green): ToolContractTests confirmed in the job logs for v18.0.41.39415 and v30.0.42.60748-beta.

Closes #42

🤖 Generated with Claude Code

Arthurvdv and others added 4 commits October 4, 2026 13:51
…r tools/list

- Annotations: every native tool sets Title plus all four hints explicitly
  (read-only tools idempotent; apply_fix/apply_fix_all non-destructive,
  non-idempotent; openWorld=false everywhere). Titles and the pinned
  description sentences live in Tools/ToolDescriptions.cs.
- Schemas: McpHost.NativeToolSchemaOptions strips the JSON-schema "default"
  keyword from every node via TransformSchemaNode ("default": null was
  published on every optional string). Parameter descriptions state the
  non-null defaults instead; apply_fix_all.dryRun gained "Default: false.".
- Registration: native tools are registered by McpHost's own loop with
  McpServerTool.Create(method, null, { Services = sp, SchemaCreateOptions }),
  replacing WithToolsFromAssembly(), which cannot take schema options.
- McpHost.ConfigureServices is split out of RunAsync as the test seam.
- Contract tests (Contracts/ToolContractTests.cs) build the real container
  under --no-proxy and check the tools/list wire shape: five tools and no
  custom handlers, no "default" anywhere, properties/required equal to the
  non-service parameters, titles and hints, pinned sentences, stated
  defaults, analyze needs no arguments. The old analyze schema test built
  the tool without Services (DI parameters leaked) and is removed.
- Docs: README native-tool table; AGENTS.md registration, annotation table,
  schema transform and test seam.

Deviations from the issue: there is no WithToolsFromAssembly overload taking
McpServerToolCreateOptions (own loop instead); MoveDefaultKeywordToDescription
was rejected because it appends "(Default value: null)"; CI tests run on
Ubuntu only; the README had no tool table to update, so one was added.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Pin the published parameters as literals: InputSchema_PublishesThePinnedParameters
  checks the exact ordered properties and the required set per tool; the
  reflection-based test cannot catch a renamed or dropped parameter.
- NativeToolMethods enumerates every [McpServerTool] method (any visibility,
  static or instance) and throws InvalidOperationException naming one that is
  not public static, instead of silently leaving it out of tools/list. A
  contract test pins the public-static convention.
- The contract fixture finds the tool's MethodInfo via
  Metadata.OfType<MethodInfo>() with an explicit assertion, instead of relying
  on Metadata[0].

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Test the default (proxy-on) registration: a second container built without
  --no-proxy (no host, AlMcpProxy never resolved) still publishes the same five
  native tools in ToolCollection and registers the list/call handlers.
- ParameterDescriptions_StateEveryNonNullDefault now requires the default
  value next to the word "default" ("(default 500)", "Default: false.",
  "'project' (default, ...") instead of both appearing anywhere in the text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- The default-wording check in ParameterDescriptions_StateEveryNonNullDefault
  now requires a left word boundary before the value in its first
  alternative, so "(default 1500)" no longer counts as stating 500.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Arthurvdv
Arthurvdv merged commit 7d3163a into main Oct 4, 2026
11 checks passed
@Arthurvdv
Arthurvdv deleted the feat/42-tool-contract branch October 4, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool annotations, schema defaults moved into descriptions, and contract tests over the published tools/list

1 participant