From 70011a32b1072521d2452de48a9b10342bf5ec9c Mon Sep 17 00:00:00 2001 From: cryptotavares Date: Sun, 1 Nov 2026 15:54:03 +0000 Subject: [PATCH 1/2] docs(testing): update visual-testing skill for client-mcp-core 0.13.0 --- CHANGELOG.md | 1 + .../references/cli-reference.md | 51 +++++++++++++------ .../references/error-recovery.md | 12 ++--- .../repos/metamask-extension.md | 46 +++++++++-------- 4 files changed, 68 insertions(+), 42 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ff5cb27a..22c21f98 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- Update the `testing/visual-testing` skill for `@metamask/client-mcp-core` 0.13.0: remove the knowledge store commands and the mandatory knowledge-reuse step, and document the global `--observe` and `--pretty` flags, one-line JSON output, observations without app state, `run-steps` `observe` (replaces `includeObservations`), timed-out calls being cancelled, and one element per ref. - Update the MetaMask Mobile performance overlay and animation references for React Native 0.86.x, Expo SDK 57, and Reanimated 4.x with `react-native-worklets`. From a worklet, call `scheduleOnRN` (`runOnJS` is a deprecated alias). Static-ESM tree-shaking stays off. ([#203](https://github.com/MetaMask/skills/pull/203)) - Require every revert yes/no to be one line that names the pull request or commit, the files, and exactly one yes/no question, whether or not security or privacy is involved. A shorter message is not that step. - Keep a new public method or other contract change on the brief and spec path when the objection is that another package should own the edit. Put that objection in the brief. diff --git a/domains/testing/skills/visual-testing/references/cli-reference.md b/domains/testing/skills/visual-testing/references/cli-reference.md index 31494bc2..3863b6aa 100644 --- a/domains/testing/skills/visual-testing/references/cli-reference.md +++ b/domains/testing/skills/visual-testing/references/cli-reference.md @@ -5,11 +5,12 @@ Complete command reference for the `mm` CLI. For core workflow and getting start ## Contents - [Syntax Rules](#syntax-rules) +- [Global Options](#global-options) - [Lifecycle](#lifecycle) - [Interaction](#interaction) - [Navigation & Tabs](#navigation--tabs) - [Context](#context) -- [State, Knowledge & Seeding](#state-knowledge--seeding) +- [State & Seeding](#state--seeding) - [Advanced](#advanced) ## Syntax Rules @@ -62,6 +63,26 @@ mm click --selector "#connectButton" # CSS selector (flag) | Type by testId | `mm type --testid X "text"` | `mm type --testId X "text"` | | Type by a11y ref | `mm type e2 "text"` | `mm type --ref e2 "text"` | +## Global Options + +These flags work with every command and can appear anywhere in the command line. + +| Flag | Description | +|---|---| +| `--project ` | Target a specific project directory. Overrides `MM_PROJECT` and git-based discovery | +| `--observe ` | Observation returned after the call. Default: `diff` for mutating commands and `run-steps`, `none` for reads, discovery and `cdp` | +| `--pretty` | Indent JSON output. Default: one line per result when stdout is not a terminal, indented on a terminal. `MM_PRETTY=1` / `0` sets the default | + +| `--observe` level | Response | +|---|---| +| `none` | No `observations`. Fastest. Refs are **not** refreshed: run `mm describe-screen` before you use refs again | +| `diff` | Only what changed since the previous observation | +| `full` | The whole compacted screen (`a11y` + `testIds`). Becomes the new diff baseline | + +Observations contain `testIds` and `a11y` only, no app state. Use `mm get-state` for state. + +Every tool, `launch` and `cleanup` response includes `meta.timing` (`queueMs`, `toolMs`, `observationMs`, `totalMs`). + ## Lifecycle | Command | Description | @@ -111,20 +132,18 @@ Tab roles: `extension`, `notification`, `dapp`, `other`. Two contexts: `e2e` (default — local Anvil, fixtures, seeding) and `prod` (no fixtures, no local chain). Cannot switch during active session — run `mm cleanup` first. -## State, Knowledge & Seeding - -| Command | Description | -| ----------------------------- | ----------------------------------------- | -| `mm get-state` | Get current extension state | -| `mm knowledge-search ` | Search steps across sessions | -| `mm knowledge-last` | Get last N step records from this session | -| `mm knowledge-sessions` | List recent sessions with metadata | -| `mm knowledge-summarize` | Generate session recipe | -| `mm run-steps ` | Execute multiple tools in sequence | -| `mm seed-contract ` | Deploy a test contract | -| `mm seed-contracts` | Deploy multiple test contracts | -| `mm get-contract-address` | Get deployed contract address | -| `mm list-contracts` | List all deployed contracts | +## State & Seeding + +| Command | Description | +| ------------------------- | ----------------------------------------------------------- | +| `mm get-state` | Get current extension state (the only source of app state) | +| `mm run-steps ` | Execute multiple tools in sequence | +| `mm seed-contract ` | Deploy a test contract | +| `mm seed-contracts` | Deploy multiple test contracts | +| `mm get-contract-address` | Get deployed contract address | +| `mm list-contracts` | List all deployed contracts | + +The knowledge store commands (`mm knowledge-search`, `mm knowledge-last`, `mm knowledge-sessions`, `mm knowledge-summarize`) and the `mm launch --goal` / `--flow-tags` flags were removed in `@metamask/client-mcp-core` 0.13.0. Session metadata is written to `test-artifacts//session.json`. ## Advanced @@ -136,5 +155,7 @@ Two contexts: `e2e` (default — local Anvil, fixtures, seeding) and `prod` (no | `mm mock-network requests [--limit ]` | Show recorded matched and missed mocked-origin requests | | `mm cdp [params-json] [--timeout ]` | Send raw Chrome DevTools Protocol command against active page| +`mm cdp` returns no observation by default. If the call changed page state, add `--observe full` or run `mm describe-screen`. + For mock network details, see [mock-network.md](mock-network.md). For CDP state manipulation, see [state-manipulation.md](state-manipulation.md). diff --git a/domains/testing/skills/visual-testing/references/error-recovery.md b/domains/testing/skills/visual-testing/references/error-recovery.md index 0919f698..78940926 100644 --- a/domains/testing/skills/visual-testing/references/error-recovery.md +++ b/domains/testing/skills/visual-testing/references/error-recovery.md @@ -14,13 +14,7 @@ - `home` → continue, but check for modals or blockers - `onboarding-*` → complete onboarding - `unknown` → take a screenshot and investigate -3. Query prior runs: - ```bash - mm knowledge-search "send" - mm knowledge-sessions - mm knowledge-last - ``` -4. Capture `mm screenshot --name "debug"` for diagnosis +3. Capture `mm screenshot --name "debug"` for diagnosis ## Error Codes @@ -65,3 +59,7 @@ | `MM_CONTEXT_SWITCH_BLOCKED` | Active session | `mm cleanup` before `mm set-context` | | Fixtures not available | Running in prod context | `mm set-context e2e` | | Stale a11yRefs after navigate | Refs not refreshed | `mm describe-screen` for fresh refs | +| Ref clicks the wrong element | Ref used after `--observe none` | `mm describe-screen`, or target by `--testid` | +| Strict mode violation on click | Target matches more than one element | Target by `--testid` or `--selector` | +| `Tool execution timed out` | Call cancelled after the request limit | `mm describe-screen` before retry; the action can have happened | +| `Error: unknown command 'knowledge-...'` | Knowledge store removed | No replacement. Discover with `mm describe-screen` | diff --git a/domains/testing/skills/visual-testing/repos/metamask-extension.md b/domains/testing/skills/visual-testing/repos/metamask-extension.md index 48115b84..ec5fd635 100644 --- a/domains/testing/skills/visual-testing/repos/metamask-extension.md +++ b/domains/testing/skills/visual-testing/repos/metamask-extension.md @@ -20,7 +20,12 @@ For architecture details, see `test/e2e/playwright/llm-workflow/README.md`. ## Gotchas -- `a11yRef`s (`e1`, `e2`, ...) are **ephemeral**. After `mm describe-screen`, `mm accessibility-snapshot`, or major navigation, re-describe before reusing refs. +- `a11yRef`s (`e1`, `e2`, ...) are **ephemeral**. Always take refs from the latest response (`mm describe-screen` or the `observations` of your last action). +- After a call with `--observe none`, refs are **not** refreshed. Target by `--testid`, or run `mm describe-screen` before you use refs again: a stale ref built from role and name can click a different element. +- Each ref resolves to exactly one element. If a click fails with a strict mode violation, target the element by `--testid` or `--selector`. +- Output is one line of JSON when stdout is not a terminal. Parse it as JSON. Add `--pretty` (or set `MM_PRETTY=1`) only when a human reads the output. +- Observations carry no app state. Use `mm get-state` for the screen, account, network, or balance. +- When a call times out (`Tool execution timed out`), it is cancelled. An action that already started can still have happened, so run `mm describe-screen` before you retry. - `mm type` uses Playwright `fill()` and **clears the field first**. - After confirm or reject in sidepanel mode, the page does **not** close. It stays open and navigates back to the home route. - `mm wait-for-notification` waits for the sidepanel confirmation route, not a legacy popup window. @@ -75,26 +80,26 @@ Available presets: `withMultipleAccounts`, `withERC20Tokens`, `withConnectedDapp Two contexts: **e2e** (default — local Anvil, fixtures, seeding) and **prod** (no fixtures, no local chain). Use `mm get-context` / `mm set-context` to switch. Cannot switch during active session. -### 2. Reuse Knowledge (Mandatory) +### 2. Describe Screen ```bash -mm knowledge-search "" -mm knowledge-sessions +mm describe-screen ``` -Reuse discovered sequences when they exist. If none exist, proceed with discovery and let this session record the new steps. +Returns current screen, active tab, visible testIds, and accessibility tree with refs. + +**Observation efficiency:** Mutating actions (`click`, `type`, `navigate`) and `mm run-steps` return compact diff-based observations after the first observation. Use these for quick next-step targeting. Reads, discovery commands and `mm cdp` return no observation. Call `mm describe-screen` when you need the full a11y tree, screenshots, or to reset the baseline. -### 3. Describe Screen +Use the global `--observe ` flag to change the observation of one call: ```bash -mm describe-screen +mm click --testid unlock-submit --observe none # chain known actions: skip the observation +mm navigate-home --observe full # after navigation: get the whole screen ``` -Returns current screen, active tab, visible testIds, and accessibility tree with refs. - -**Observation efficiency:** Mutating actions (`click`, `type`, `navigate`) return compact diff-based observations after the first mutation. Use these for quick next-step targeting. Call `mm describe-screen` when you need the full a11y tree, screenshots, or to reset the baseline. +Only mutating and batch calls wait for animations before they observe. To get a settled screen after an action, let the action observe (`--observe full`) instead of running a read afterwards. -### 4. Interact +### 3. Interact Use exactly **one targeting method** per call. Priority: `testId` (stable) > `a11yRef` (discovery) > `selector` (fallback). @@ -149,15 +154,15 @@ Phase-specific error codes on timeout: For the full error code table, see [references/error-recovery.md](references/error-recovery.md). -### 5. Verify-Fix Loop +### 4. Verify-Fix Loop After any interaction sequence: 1. `mm describe-screen` to verify expected state -2. If wrong: `mm screenshot --name "debug-"`, check `mm knowledge-search ""`, retry +2. If wrong: `mm screenshot --name "debug-"`, then retry 3. Continue only when expected state is confirmed -### 5b. Edge Case: Component Not Responding to State Changes +### 4b. Edge Case: Component Not Responding to State Changes If you change Redux state via CDP but the target component doesn't update: @@ -167,7 +172,7 @@ If you change Redux state via CDP but the target component doesn't update: 4. Disconnect the live data feed first (prevents overwrite), then push your test data. 5. Re-verify with `mm describe-screen`. -### 6. Confirmations (Dapp Flows) +### 5. Confirmations (Dapp Flows) ```bash mm navigate https://test-dapp.io # navigate to dapp @@ -181,7 +186,7 @@ mm switch-to-tab dapp # back to dapp Tab roles: `extension`, `notification`, `dapp`, `other`. -### 7. Navigate +### 6. Navigate ```bash mm navigate-home @@ -189,7 +194,7 @@ mm navigate-settings mm navigate https://test-dapp.io ``` -### 8. Screenshots +### 7. Screenshots ```bash mm screenshot --name "after-unlock" @@ -197,7 +202,7 @@ mm screenshot --name "after-unlock" Capture before and after meaningful state changes for visual validation evidence. -### 9. Cleanup (Always Required) +### 8. Cleanup (Always Required) ```bash mm cleanup # stop browser and services @@ -229,10 +234,11 @@ mm run-steps '{"steps":[ Details: - Prefer `testId`, `a11yRef`, or `selector` in args (not `ref`) - Use `within` in args to scope targeting within a parent element -- Add `batchTimeoutMs` for overall timeout +- Add `batchTimeoutMs` (1000–30000 ms) for an overall deadline. The whole call stops after 30 s, so use a lower value (for example 20000) to get partial results, and split longer flows into several batches +- Add a top-level `observe` (`none` / `diff` / `full`, default `diff`) to set the observation collected after the batch. `includeObservations` was removed; check `summary.ok` to react to failures - Tool aliases like `navigate_home` and `navigate-home` are supported -Pattern: discover with `mm describe-screen` → reuse from `mm knowledge-search` → batch with `mm run-steps` → re-verify with `mm describe-screen`. +Pattern: discover with `mm describe-screen` → batch known steps with `mm run-steps` → re-verify with `mm describe-screen`. ## Default Credentials From 64de147b0c323aec8414f3890eb06e7ba19a8768 Mon Sep 17 00:00:00 2001 From: cryptotavares Date: Mon, 2 Nov 2026 19:16:57 +0000 Subject: [PATCH 2/2] docs(testing): target client-mcp-core 0.13.1; use yarn build:test --- CHANGELOG.md | 2 +- .../testing/skills/visual-testing/references/cli-reference.md | 2 +- .../testing/skills/visual-testing/references/error-recovery.md | 2 +- .../testing/skills/visual-testing/repos/metamask-extension.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 22c21f98..57fb6a0b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed -- Update the `testing/visual-testing` skill for `@metamask/client-mcp-core` 0.13.0: remove the knowledge store commands and the mandatory knowledge-reuse step, and document the global `--observe` and `--pretty` flags, one-line JSON output, observations without app state, `run-steps` `observe` (replaces `includeObservations`), timed-out calls being cancelled, and one element per ref. +- Update the `testing/visual-testing` skill for `@metamask/client-mcp-core` 0.13.1: remove the knowledge store commands and the mandatory knowledge-reuse step, and document the global `--observe` and `--pretty` flags, one-line JSON output, observations without app state, `run-steps` `observe` (replaces `includeObservations`), timed-out calls being cancelled, and one element per ref. Replace the removed `yarn build:test:webpack` script with `yarn build:test`. - Update the MetaMask Mobile performance overlay and animation references for React Native 0.86.x, Expo SDK 57, and Reanimated 4.x with `react-native-worklets`. From a worklet, call `scheduleOnRN` (`runOnJS` is a deprecated alias). Static-ESM tree-shaking stays off. ([#203](https://github.com/MetaMask/skills/pull/203)) - Require every revert yes/no to be one line that names the pull request or commit, the files, and exactly one yes/no question, whether or not security or privacy is involved. A shorter message is not that step. - Keep a new public method or other contract change on the brief and spec path when the objection is that another package should own the edit. Put that objection in the brief. diff --git a/domains/testing/skills/visual-testing/references/cli-reference.md b/domains/testing/skills/visual-testing/references/cli-reference.md index 3863b6aa..7610137b 100644 --- a/domains/testing/skills/visual-testing/references/cli-reference.md +++ b/domains/testing/skills/visual-testing/references/cli-reference.md @@ -143,7 +143,7 @@ Two contexts: `e2e` (default — local Anvil, fixtures, seeding) and `prod` (no | `mm get-contract-address` | Get deployed contract address | | `mm list-contracts` | List all deployed contracts | -The knowledge store commands (`mm knowledge-search`, `mm knowledge-last`, `mm knowledge-sessions`, `mm knowledge-summarize`) and the `mm launch --goal` / `--flow-tags` flags were removed in `@metamask/client-mcp-core` 0.13.0. Session metadata is written to `test-artifacts//session.json`. +The knowledge store commands (`mm knowledge-search`, `mm knowledge-last`, `mm knowledge-sessions`, `mm knowledge-summarize`) and the `mm launch --goal` / `--flow-tags` flags were removed in `@metamask/client-mcp-core` 0.13.1 (the first published 0.13 version). Session metadata is written to `test-artifacts//session.json`. ## Advanced diff --git a/domains/testing/skills/visual-testing/references/error-recovery.md b/domains/testing/skills/visual-testing/references/error-recovery.md index 78940926..ce9850b9 100644 --- a/domains/testing/skills/visual-testing/references/error-recovery.md +++ b/domains/testing/skills/visual-testing/references/error-recovery.md @@ -47,7 +47,7 @@ | ------------------------------------------- | ---------------------------- | ------------------------------------------------------------- | | `MM_SESSION_ALREADY_RUNNING` | Previous session not cleaned | `mm cleanup` | | `MM_NO_ACTIVE_SESSION` | No browser running | `mm launch` | -| Extension not loading | Extension not built | `yarn build:test:webpack` then `mm launch` | +| Extension not loading | Extension not built | `yarn build:test` then `mm launch` | | `EADDRINUSE` port error | Orphan processes | Check `.mm-server` for ports, kill orphaned process | | `MM_TARGET_NOT_FOUND` | Element not visible | `mm describe-screen` to check state | | `MM_WAIT_TIMEOUT` | Slow environment | Increase `--timeout`, inspect screenshot | diff --git a/domains/testing/skills/visual-testing/repos/metamask-extension.md b/domains/testing/skills/visual-testing/repos/metamask-extension.md index ec5fd635..520e6c50 100644 --- a/domains/testing/skills/visual-testing/repos/metamask-extension.md +++ b/domains/testing/skills/visual-testing/repos/metamask-extension.md @@ -56,7 +56,7 @@ yarn mm # or: npx mm - `mm launch` validates the build and returns an actionable error if missing. ```bash -yarn install && yarn build:test:webpack +yarn install && yarn build:test # fresh worktree: run `yarn webpack:tsc` first ``` **Port conflicts** — if ports are stuck from a previous run, check `.mm-server` for active ports: