Skip to content

Studio MCP: Add the declared settings of core blocks to tool results - #5333

Merged
chubes4 merged 6 commits into
trunkfrom
studio-code/block-settings-in-mcp
Oct 23, 2026
Merged

Studio MCP: Add the declared settings of core blocks to tool results#5333
chubes4 merged 6 commits into
trunkfrom
studio-code/block-settings-in-mcp

Conversation

@chubes4

@chubes4 chubes4 commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Related issues

How AI was used in this PR

Claude (via the Pi coding agent in a Roadie session) wrote the change and the test and ran the end-to-end check below. I reviewed the diff and the results.

Proposed Changes

#5307 adds the declared settings of each core block a tool result shows (read from the site's own wp-includes/blocks/<block>/block.json). The notes are wired up in the Pi runtime's toToolDefinition, which the MCP server bypasses: it calls tool.rawHandler directly. So Claude Code, Codex, and the AI plugins never got them.

  • One module for result notes, cli/ai/result-notes.ts. createResultNotes() returns ( sitePath, text ) => notes: the facts read from that site about what the result shows, each stated once per tracker. Block settings are the only kind today; a new kind is added there and reaches every surface.
  • Both surfaces use it the same way. Each creates one tracker per conversation it can see and passes the site it works on:
  • The MCP call path moves into runtimes/mcp/served-tool.ts (runServedTool); pending work reports and error handling are unchanged.

Limits:

  • MCP clients read theme files with their own Read tool, which never passes through Studio. Notes only fire for Studio tools (wp_cli and the like), not for files the client reads itself.
  • Once per tracker means a note stated early can be lost if the context is compacted later: within a Studio Code turn (the tracker is created in createStudioAgentSession, which runAgentSessionTurn calls, so it is per turn, not per session), and within an MCP session, whose compaction Studio can't see.

Testing Instructions

  • npx vitest run apps/cli/ai/runtimes/mcp: served-tool.test.ts resolves a real site from a cli.json (no mocks): no notes for a call without a site or with an unknown one, the settings for the named site, then none on a repeat call in the same session.
  • End-to-end, on a throwaway site with an isolated config: built the CLI, ran studio mcp over stdio, and called wp_cli with post get 2 --field=post_content twice, then option get blogname. The first result had the page content plus: 
    core/paragraph settings, declared in <site>/wp-includes/blocks/paragraph/block.json: dropCap (boolean, default false), placeholder (string), direction ("ltr"|"rtl").
core/quote settings, declared in <site>/wp-includes/blocks/quote/block.json: textAlign (string).
    The repeat call and option get blogname had no notes.

Pre-merge Checklist

  • Have you checked for TypeScript, React or other console errors?

@chubes4
chubes4 marked this pull request as draft October 23, 2026 12:20
@chubes4
chubes4 marked this pull request as ready for review October 23, 2026 13:06
@chubes4
chubes4 merged commit a6a8cd1 into trunk Oct 23, 2026
13 checks passed
@chubes4
chubes4 deleted the studio-code/block-settings-in-mcp branch October 23, 2026 17:10
@chubes4 chubes4 mentioned this pull request Oct 23, 2026
Merged
1 task
chubes4 added a commit that referenced this pull request Oct 23, 2026
@chubes4
Studio Code: State saved Site Editor copies in tool results, when a c…
e62d922 
…all touches the theme file they replace (#5343)

## Related issues

- Follows #5301, addressing the review there: the saved Site Editor
copies were listed in every system prompt, though they matter only when
the agent touches a theme file they replace.
- Builds on #5333's shared result notes, so the notice reaches Studio
Code and `studio mcp` the same way.

## How AI was used in this PR

Claude (via the Pi coding agent in a Roadie session) wrote the change
and the test and ran the evals below. I reviewed the diff and the
results.

## Proposed Changes

When someone saves a template, template part or Global Styles in the
Site Editor, WordPress renders that saved copy instead of the theme
file. An agent that edits `parts/header.html` then changes nothing the
visitor sees. #5092 told the agent this in `edit-manage-site`; #5301
moved it into Active site facts, which listed every saved copy in every
prompt.

- The notice moves into tool results, stated when a call or its result
names a theme file a saved copy replaces, for example:
> WordPress renders the template part header from its copy saved in the
Site Editor (post 12), not from
wp-content/themes/twentytwentyfive/parts/header.html.
- `cli/ai/saved-copies.ts` reads the copies from the site's SQLite
database when a theme file is named, so copies saved earlier in the turn
count. Each copy is stated once per tracker.
- It is registered in `cli/ai/result-notes.ts` (#5333). The notes now
see the call's arguments with its result (a Write names the file in its
arguments), on both surfaces.
- Active site facts keep the WordPress version; saved copies leave them.

Limits:

- MCP clients read and write theme files with their own tools, which
never pass through Studio, so for them the notice fires only when a
Studio tool names the file.
- Like every result note, a notice stated early in a turn can be lost if
the context is compacted later in that turn.

## Testing Instructions

- `npx vitest run apps/cli/ai/tests/saved-copies.test.ts`: a real SQLite
database with a saved template part and Global Styles; notices for the
theme files they replace, none for other files, and each copy once; run
with `wp_` tables and with another prefix, as imported sites can have.
- Eval (`/tmp` override kit, 6 runs per case, Studio's default model): a
site whose header part and Global Styles were saved in the Site Editor;
the task edits the header, then the styles. Scored from the rendered
site.

  | Case | #5301 (facts in prompt) | This PR |
  |---|---|---|
  | Template part saved in the Site Editor | 6/6 | 6/6 |
  | Global Styles saved in the Site Editor | 4/4 | 6/6 |

The notice appeared in all 12 transcripts. An earlier run of this change
scored 5/6 on Global Styles; the miss was broken JSON the agent wrote to
a file and saved with `post update`, which the inline `--post_content`
JSON check (#5310) does not cover.

- Eval on site edits with no saved copies (8 cases from the routing
eval: contact form, carousel, newsletter, heading font, testimonials,
header search block, hero, spacing; each run stops at its first write).
Before #5301, `edit-manage-site` told the agent to check for saved
copies first; #5092's review asked what that costs.

| | Sessions that queried saved copies | Queries | Tool calls before the
first write |
  |---|---|---|---|
  | Before #5301 (Oct 22 runs) | 38/126 | 43 | 14.3 |
  | This PR | 0/32 | 0 | 14.1 |

The queries are gone, but calls before the first write are unchanged:
the agent spends them on other reading. Tokens (median 134k before, 136k
now) and wall time (median 38.6s on trunk, 36.4s now) are at parity too.

## Pre-merge Checklist

- [ ] Have you checked for TypeScript, React or other console errors?
@youknowriad youknowriad mentioned this pull request Oct 23, 2026
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@chubes4 chubes4

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@chubes4