docs(findings): SEP-2640 v1 skill detection over skills/list in VS Code

[tobi-oye]

Follow-up to #116. That entry was written against the pre-v1 draft, before skills/list existed — as the note added in 59f8c14 records. This adds a second entry for a run of the same VS Code fork against the v1 extension surface, and leaves the original untouched.

Docs-only. Written to the findings template added in #112.

Motivation and Context

#116 verified that a mainstream host could discover skill:// resources at all. It could not verify the v1 loop, because v1 did not exist yet: no skills/list, no skills/get, no per-file digest manifest, and digest verification not yet a host MUST.

This run closes that gap against olaservo/skills-over-mcp-demo (@olaservo/ext-skills 0.13.0, SEP-2640 at 753b9f2).

The query it traces — "what does 5d10dh1 mean?" — was chosen deliberately. That deployment ships a roll_dice tool alongside the tabletop-dice skill, and a tool that executes dice notation cannot explain it. A correct answer therefore has to come from skill content, which makes the trace unambiguous:

06:48:27  skills/list                                     -> 3 entries, secret-menu correctly unlisted
10:03:34  resources/read skill://…/tabletop-dice/SKILL.md
10:03:38  resources/read skill://…/references/dice-notation.md

The second hop happens only because the SKILL.md body says "see references/dice-notation.md for the full grammar". The answer comes from that file's Keep/Drop table. Both reads were verified against the {uri, digest, size} manifest carried in the listing entry — a listing entry verifying content that had not been fetched yet is what makes detection and retrieval genuinely separable.

Screenshots

Two things the WG may want to act on

Nothing bounds how long a host may retain a listing. This host issued one skills/list and reused it for 27 hours across four connections. The server sends SEP-2549 ttlMs/cacheScope, but scopes them to 2026-07-28+ connections; VS Code negotiates 2025-11-25 and so receives no caching guidance at all. Since verification binds fetched bytes to digests from that cached manifest, a server redeploying between sessions has fresh content checked against a stale manifest — and at that point a legitimate update is indistinguishable from tampering. Re-listing on reconnect is what a server would expect, and nothing currently asks for it.

A server-side tool out-competes the skill it pairs with. On a control query ("roll 2d6+3"), the model called roll_dice and never loaded tabletop-dice, whose description overlaps the tool's almost verbatim — with three skills in context 28 seconds earlier and a BLOCKING REQUIREMENT to load skills first. The server's README states the tool "pairs with the skill without substituting for it"; in this host it substituted. This is a sharper mechanism than the decay recorded under Skill Reliability and Adherence: a matching tool schema beating a document, rather than a model losing the plot. Relevant to #37.

How Has This Been Tested?

Code - OSS Dev 1.133.0, source build, --log trace, against the live Space over Streamable HTTP. All claims in the entry come from the MCP output channel for that session; method totals and the exact read sequence are quoted in the Evidence and reproduction section.

The server side was checked independently with the demo's own conformance suite, which passes against the live deployment:

npx tsx src/smoke-http.ts https://olaservo-skills-over-mcp-demo.hf.space/mcp

The three defects under What didn't are host bugs in the fork, not server or spec problems, and the entry says so.

Limitations stated in the entry

skills/get was never exercised (this host has no path that yields an unlisted skill URI); "resources": "dynamic" and nested skills are untested because this server offers neither; per-turn model attribution is not recorded under copilot/auto.

Breaking Changes

None. Docs-only.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update