Repository navigation
Studio Code: Refuse wp_cli --post_content JSON that does not parse - #5310
Merged
Merged
Conversation
1 task done
Collaborator
📊 Performance Test ResultsComparing f028cd9 vs trunk app-size
site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
chubes4
added a commit
that referenced
this pull request
Oct 23, 2026
…5312) ## Related issues - Pairs with #5310, which refuses `--post_content` JSON that doesn't parse. In the #5310 eval, both refusals were valid JSON that `wp_cli` had broken while splitting the command. This PR splits those commands correctly, so they save as written. It is independent of #5310 and can merge in either order. ## How AI was used Claude (via the Pi coding agent in a Roadie session) found the two quoting forms in eval transcripts, wrote the change and tests, and ran the checks below. I reviewed the diff and the results. ## Proposed Changes `wp_cli` now reads a quoted `--post_content` value the way a shell would, in the two forms agents used where it didn't before: - **Backslash-escaped quotes inside double quotes:** `--post_content="{\"styles\":…}"`. The value used to stop at the first `\"`, so `{\` was saved as the post content. Inside double quotes, a backslash now escapes `"`, `\`, `$` and backticks. Single-quoted values stay literal. - **Quotes around the whole argument:** `'--post_content={"styles":…}'`. The closing `'` used to become part of the content. A quote just before `--post_content=` now wraps the whole value. In both cases WP-CLI saved the broken content and printed `Success: Updated post 5.`. Global Styles JSON that doesn't parse is ignored by WordPress, so the background change never showed and the agent started debugging. **What else changes:** in 118 inline `--post_content` writes from 535 local agent transcripts, these two forms appear only in the two eval runs above. No double-quoted value contains any other backslash, so content written in the forms agents already use splits as before. ## Testing Instructions - `npx vitest run apps/cli/ai/tests/tools.test.ts` covers both forms with a trailing `--porcelain`. Both tests fail on trunk, and the existing quoting tests still pass. - **Replay on a real site:** the two commands from the #5310 eval, run unchanged through `wp_cli` on the `global-styles-db-override` site, now save the intended JSON (`post get 5 --field=post_content` returns it). The front end renders the new `#1e2a44` background, and the user's line height `1.7` is kept. - `npx vitest run apps/cli/ai apps/cli/lib/tests/wp-cli-post-content.test.ts` passes. Lint passes, and the typecheck shows only the existing errors in `commands/export.ts` and `commands/push.ts`. ## Pre-merge Checklist - [x] Have you checked for TypeScript, React or other console errors?
1 task done
1 task
chubes4
added a commit
that referenced
this pull request
Oct 23, 2026
…all touches the theme file they replace (#5343) ## Related issues - Follows #5301, addressing the review there: the saved Site Editor copies were listed in every system prompt, though they matter only when the agent touches a theme file they replace. - Builds on #5333's shared result notes, so the notice reaches Studio Code and `studio mcp` the same way. ## How AI was used in this PR Claude (via the Pi coding agent in a Roadie session) wrote the change and the test and ran the evals below. I reviewed the diff and the results. ## Proposed Changes When someone saves a template, template part or Global Styles in the Site Editor, WordPress renders that saved copy instead of the theme file. An agent that edits `parts/header.html` then changes nothing the visitor sees. #5092 told the agent this in `edit-manage-site`; #5301 moved it into Active site facts, which listed every saved copy in every prompt. - The notice moves into tool results, stated when a call or its result names a theme file a saved copy replaces, for example: > WordPress renders the template part header from its copy saved in the Site Editor (post 12), not from wp-content/themes/twentytwentyfive/parts/header.html. - `cli/ai/saved-copies.ts` reads the copies from the site's SQLite database when a theme file is named, so copies saved earlier in the turn count. Each copy is stated once per tracker. - It is registered in `cli/ai/result-notes.ts` (#5333). The notes now see the call's arguments with its result (a Write names the file in its arguments), on both surfaces. - Active site facts keep the WordPress version; saved copies leave them. Limits: - MCP clients read and write theme files with their own tools, which never pass through Studio, so for them the notice fires only when a Studio tool names the file. - Like every result note, a notice stated early in a turn can be lost if the context is compacted later in that turn. ## Testing Instructions - `npx vitest run apps/cli/ai/tests/saved-copies.test.ts`: a real SQLite database with a saved template part and Global Styles; notices for the theme files they replace, none for other files, and each copy once; run with `wp_` tables and with another prefix, as imported sites can have. - Eval (`/tmp` override kit, 6 runs per case, Studio's default model): a site whose header part and Global Styles were saved in the Site Editor; the task edits the header, then the styles. Scored from the rendered site. | Case | #5301 (facts in prompt) | This PR | |---|---|---| | Template part saved in the Site Editor | 6/6 | 6/6 | | Global Styles saved in the Site Editor | 4/4 | 6/6 | The notice appeared in all 12 transcripts. An earlier run of this change scored 5/6 on Global Styles; the miss was broken JSON the agent wrote to a file and saved with `post update`, which the inline `--post_content` JSON check (#5310) does not cover. - Eval on site edits with no saved copies (8 cases from the routing eval: contact form, carousel, newsletter, heading font, testimonials, header search block, hero, spacing; each run stops at its first write). Before #5301, `edit-manage-site` told the agent to check for saved copies first; #5092's review asked what that costs. | | Sessions that queried saved copies | Queries | Tool calls before the first write | |---|---|---|---| | Before #5301 (Oct 22 runs) | 38/126 | 43 | 14.3 | | This PR | 0/32 | 0 | 14.1 | The queries are gone, but calls before the first write are unchanged: the agent spends them on other reading. Tokens (median 134k before, 136k now) and wall time (median 38.6s on trunk, 36.4s now) are at parity too. ## Pre-merge Checklist - [ ] Have you checked for TypeScript, React or other console errors?
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
wp_cli post create|update --post_content=<file>from saving the file's path. This PR handles another way a post gets saved wrong while the command still reports success.How AI was used
Claude (via the Pi coding agent in a Roadie session) found the failure in eval transcripts, wrote the change and tests, and ran the eval below. I reviewed the diff and the results.
Proposed Changes
When
wp_cli post create|update --post_content=…content starts with{but doesn't parse as JSON, the tool saves nothing and returns the parse error:Block markup and HTML never start with
{, so content that does is JSON, such as a Global Styles post. When WordPress can't parse a Global Styles post, it ignores the whole post. WP-CLI still saves it and printsSuccess: Updated post 5., so the agent has no sign anything went wrong.That happened in an eval run of
global-styles-db-override. The agent's JSON was missing its last}. Over the next 45 tool calls it checked the CSS, cleared transients, read the theme JSON resolver's source, and hex-dumped the post before finding the syntax error. The run used 1.4M tokens; the other runs used about 90k.The check is in
getUnsupportedWpCliPostContentMessage, next to the existing$(cat …)refusal. It doesn't depend on post types or the database. It refuses instead of repairing because there's no single correct fix for broken JSON.Testing Instructions
npx vitest run apps/cli/lib/tests/wp-cli-post-content.test.tscovers the exact command from the failed run. It is refused with the parse error, and the same command with the brace restored is accepted.Replay on a real site: the failed run's command, sent unchanged through
wp_cli, was refused with the message above. The Global Styles post kept its previous valid content (#f6efe4, line height1.7).Eval:
global-styles-db-override, 6 runs on this branch. All 6 passed.wpin Bash, which this check doesn't cover. Its JSON was missing the last}, and it took the same debugging spiralIn both refused runs, the agent's JSON was valid but
wp_clisplit the quoting wrong. Run 0 wrapped the whole argument in single quotes, so a trailing'ended up in the content. Run 2 escaped the inner double quotes with backslashes, so the content was cut to{\. On trunk, both would have been saved and reported as successful.npx vitest run apps/cli/ai apps/cli/lib/tests/wp-cli-post-content.test.tspasses. Lint passes, and the typecheck shows only the existing errors incommands/export.tsandcommands/push.ts.Pre-merge Checklist