From cb92eea92dd44694b58afc847b8500d2d8cf552e Mon Sep 17 00:00:00 2001 From: cryptotavares Date: Sun, 1 Nov 2026 16:19:39 +0000 Subject: [PATCH 1/5] docs(testing): update mobile-visual-testing skill for client-mcp-core 0.13.0 --- CHANGELOG.md | 1 + .../references/cli-reference.md | 50 ++++++++++++------- .../references/error-recovery.md | 17 +++---- .../repos/metamask-mobile.md | 46 +++++++++++------ 4 files changed, 70 insertions(+), 44 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ff5cb27a..87cb7d3f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- Update the `testing/mobile-visual-testing` skill for `@metamask/client-mcp-core` 0.13.0 and `@metamask/device-mcp` 0.7.0: remove the knowledge store commands, the knowledge-reuse step and the `--goal` / `--flow-tags` launch flags, and document the global `--observe` and `--pretty` flags, one-line JSON output, observations without app state, the partial mobile `get-state`, `run-steps` `observe` (replaces `includeObservations`) and the 30 s `batchTimeoutMs` cap, timed-out calls being cancelled, calling `node_modules/.bin/mm` directly, and the Android `helper-first` snapshot default. - Update the MetaMask Mobile performance overlay and animation references for React Native 0.86.x, Expo SDK 57, and Reanimated 4.x with `react-native-worklets`. From a worklet, call `scheduleOnRN` (`runOnJS` is a deprecated alias). Static-ESM tree-shaking stays off. ([#203](https://github.com/MetaMask/skills/pull/203)) - Require every revert yes/no to be one line that names the pull request or commit, the files, and exactly one yes/no question, whether or not security or privacy is involved. A shorter message is not that step. - Keep a new public method or other contract change on the brief and spec path when the objection is that another package should own the edit. Put that objection in the brief. diff --git a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md index 678f5c76..c382ebfa 100644 --- a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md +++ b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md @@ -5,12 +5,12 @@ Command reference for the prod-only MetaMask Mobile consumer. The generic core C ## Contents - [Syntax Rules](#syntax-rules) +- [Global Options](#global-options) - [Lifecycle](#lifecycle) - [Destructive Launch Flags](#destructive-launch-flags) - [Installed State](#installed-state) - [Interaction](#interaction) - [Discovery](#discovery) -- [Knowledge Store](#knowledge-store) - [Batching](#batching) - [Hermes CDP](#hermes-cdp) - [Simulator Selection](#simulator-selection) @@ -61,6 +61,28 @@ Use exactly one targeting method per command: a positional accessibility referen - **Matching is fuzzy and case-insensitive.** idb matches accessibility label/identifier by substring, so `--testid Confirm` can match `Confirm Transaction`. Use exact, unique test IDs to disambiguate; there is no `--within` scoping fallback on mobile. - **`--testid` is lowercase.** `--testId` is not recognized as a flag and its value is treated as a positional target. +## Global Options + +These flags work with every command and can appear anywhere in the command line. + +| Flag | Description | +| --- | --- | +| `--project ` | Target a specific project directory. Overrides `MM_PROJECT` and git-based discovery | +| `--observe ` | Observation returned after the call. Default: `diff` for mutating commands and `run-steps`, `none` for reads, discovery and `cdp` | +| `--pretty` | Indent JSON output. Default: one line per result when stdout is not a terminal, indented on a terminal. `MM_PRETTY=1` / `0` sets the default | + +| `--observe` level | Response | +| --- | --- | +| `none` | No `observations`. Fastest. Refs are **not** refreshed: run `yarn mm describe-screen` before you use refs again | +| `diff` | Only what changed since the previous observation | +| `full` | The whole compacted screen (`a11y` + `testIds`). Becomes the new diff baseline | + +Observations contain `testIds` and `a11y` only, no app state. The mobile a11y tree is flat: layout-only nodes, the status bar, the navigation bar and the React Native dev overlay are filtered out. + +Every tool, `launch` and `cleanup` response includes `meta.timing` (`queueMs`, `toolMs`, `observationMs`, `totalMs`). + +Agents that run many commands can call `node_modules/.bin/mm ` directly. It takes the same arguments as `yarn mm ` and skips the Yarn startup (about 0.4–0.5 s per call). + ## Lifecycle | Command | Description | @@ -82,10 +104,10 @@ The locally installed core CLI parser exposes these launch flags that are meanin | `--device-id ` | Select an iOS Simulator by UDID | | `--app-bundle ` | Install a specific `.app` bundle before launching (for example `ios/build/MetaMask.app`). Required before any destructive flag | | `--metro-port ` | Attach to a running Metro bundler on the given port. Equivalent to `MM_METRO_PORT`; the flag wins when both are set | -| `--goal ` | Record the session goal in knowledge metadata | -| `--flow-tags ` | Record comma-separated flow tags | | `--force` | Replace an existing active session | +The `--goal` and `--flow-tags` launch flags were removed in `@metamask/client-mcp-core` 0.13.0. Session metadata is written to `test-artifacts//session.json`. + Launch timeout is 120 seconds. The project CLI does not build MetaMask. Install the intended app on the simulator before launching, or pass `--app-bundle ` to install a specific build. The destructive flags `--reinstall`, `--reset-app-data`, and `--allow-fox-code-mismatch` are also parsed but guarded — see [Destructive Launch Flags](#destructive-launch-flags). For command-line Metro attachment, use `--metro-port ` or the `MM_METRO_PORT` environment variable as documented below. @@ -143,23 +165,18 @@ All interaction commands accept on iOS: | Command | Description | | --- | --- | -| `yarn mm describe-screen` | Return app state, visible test IDs, accessibility tree, and prior knowledge | +| `yarn mm describe-screen` | Return the current screen, visible test IDs, and the accessibility tree with refs | | `yarn mm screenshot [--name ]` | Capture the current simulator screen | | `yarn mm accessibility-snapshot` | Return a trimmed accessibility tree (the shared `--root ` flag is ignored on iOS; the full tree is always returned) | | `yarn mm list-testids [--limit ]` | List visible test IDs | -| `yarn mm get-state` | Return the current app-state snapshot | +| `yarn mm get-state` | Return a partial app-state snapshot: `isLoaded`, `extensionId` (the bundle ID / package), `source: "stub"`, and `currentScreen` only when a known test ID identifies the screen. Wallet fields (unlock, account, network, balance) are omitted, not guessed | | `yarn mm get-context` | Report the static prod environment and available mobile capabilities | -Mutating tools return observations with fresh references. Later actions may return compact differences from the prior observation. Use `describe-screen` whenever a complete tree or refreshed baseline is needed. +Mutating tools (`click`, `type`, ...) and `run-steps` return observations with fresh references, as a compact diff from the prior observation by default. Reads, discovery commands and `cdp` return no observation. Use `describe-screen` whenever a complete tree or refreshed baseline is needed. -## Knowledge Store +Only mutating and batch calls wait for animations before they observe. To get a settled screen after an action, let the action observe (`--observe full`) instead of running a read afterwards. -| Command | Description | -| --- | --- | -| `yarn mm knowledge-search ` | Search recorded steps | -| `yarn mm knowledge-last` | Return recent steps from the current session | -| `yarn mm knowledge-sessions` | List recorded sessions and metadata | -| `yarn mm knowledge-summarize [--session ]` | Generate a session recipe | +The knowledge store commands (`yarn mm knowledge-search`, `knowledge-last`, `knowledge-sessions`, `knowledge-summarize`) were removed in `@metamask/client-mcp-core` 0.13.0. There is no replacement: discover with `yarn mm describe-screen`. ## Batching @@ -178,8 +195,8 @@ yarn mm run-steps '{"steps":[ | --- | --- | | `steps` | Required array of `{tool, args}` objects | | `stopOnError` | Stop after the first failure; defaults to `false` (remaining steps still run). Set `true` to abort on the first error | -| `includeObservations` | `'all'`, `'none'`, or `'failures'` | -| `batchTimeoutMs` | Overall deadline; remaining steps are skipped after expiry | +| `observe` | `none`, `diff` (default) or `full`: the observation collected after the batch. Replaces `includeObservations`, which was removed; check `summary.ok` to react to failures | +| `batchTimeoutMs` | Overall deadline, 1000–30000 ms; remaining steps are skipped after expiry. The whole call stops after 30 s, so use a lower value (for example 20000) to get partial results, and split longer flows into several batches | Each step is independently checked for mobile platform support. Do not put browser-only commands in a mobile batch. @@ -222,7 +239,7 @@ yarn mm launch --metro-port 8081 MM_METRO_PORT=8081 yarn mm launch ``` -On Node 20, add `NODE_OPTIONS="--experimental-websocket"` when launching. Run `yarn mm describe-screen` after runtime mutation to resynchronize observations. +On Node 20, add `NODE_OPTIONS="--experimental-websocket"` when launching. `yarn mm cdp` returns no observation by default. After runtime mutation, add `--observe full` or run `yarn mm describe-screen` to resynchronize observations. ## Simulator Selection @@ -259,7 +276,6 @@ This consumer always reports the static `prod` environment and provides no local | Command | Mobile behavior or alternative | | --- | --- | | `yarn mm set-context` | Environment switching is unavailable; the mobile consumer is always `prod` | -| `yarn mm get-state` | E2E fixture/state snapshot; use `yarn mm describe-screen` for live UI state | | `yarn mm seed-contract` | Contract seeding needs the e2e chain, which this consumer does not run | | `yarn mm seed-contracts` | Contract seeding needs the e2e chain, which this consumer does not run | | `yarn mm get-contract-address` | Depends on seeded e2e contracts that do not exist here | diff --git a/domains/testing/skills/mobile-visual-testing/references/error-recovery.md b/domains/testing/skills/mobile-visual-testing/references/error-recovery.md index 95c34f9d..abbf9d23 100644 --- a/domains/testing/skills/mobile-visual-testing/references/error-recovery.md +++ b/domains/testing/skills/mobile-visual-testing/references/error-recovery.md @@ -23,16 +23,8 @@ For in-app failures: 1. Run `yarn mm describe-screen`. 2. Identify the current screen and visible blockers. If the screen is unknown, capture a screenshot. -3. Query prior successful runs: - - ```bash - yarn mm knowledge-search "" - yarn mm knowledge-sessions - yarn mm knowledge-last - ``` - -4. Capture evidence with `yarn mm screenshot --name "debug"`. -5. Retry only after obtaining fresh accessibility references. +3. Capture evidence with `yarn mm screenshot --name "debug"`. +4. Retry only after obtaining fresh accessibility references. ## Error Codes @@ -51,7 +43,7 @@ Launch errors use the core `ErrorCode` set. `@metamask/client-mcp-core` only pre | `MM_GETTEXT_FAILED` | The target detached or does not expose text. Re-describe and re-target. | | `MM_GETTEXT_TIMEOUT` | Text retrieval exceeded the deadline. Retry with a fresh target or larger `--timeout`. | | `MM_PAGE_CLOSED` | The target closed during the action. This can be expected for transitions; inspect current state. | -| `MM_BATCH_TIMEOUT` | `run-steps` exceeded `batchTimeoutMs`. Reduce the batch or increase its deadline. | +| `MM_BATCH_TIMEOUT` | `run-steps` exceeded `batchTimeoutMs`. Split the batch; the deadline is capped at 30000 ms. | ### Session and Launch Errors @@ -97,6 +89,9 @@ Launch errors use the core `ErrorCode` set. `@metamask/client-mcp-core` only pre | `idb` not installed | Missing iOS Debug Bridge dependency (`MM_DEPENDENCIES_MISSING`) | Run `yarn mm:doctor`, then `brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb` | | Empty or failed screen snapshot | Splash screen, animation, app transition, or crash | Wait briefly, describe again, and capture a screenshot | | Stale accessibility references | Screen changed after the references were generated | Run `yarn mm describe-screen` and use fresh references | +| Ref taps the wrong element | Ref used after a call with `--observe none` | Run `yarn mm describe-screen`, or target by `--testid` | +| `Tool execution timed out` | Call cancelled after the request limit | Run `yarn mm describe-screen` before retrying; the action can have happened | +| `Error: unknown command 'knowledge-...'` | Knowledge store removed in `@metamask/client-mcp-core` 0.13.0 | No replacement. Discover with `yarn mm describe-screen` | | Interaction timeout | Animation, overlay, or stale/ambiguous target | Re-`describe-screen`, target by a unique test ID or fresh a11y ref (no `--within` on mobile), and increase `--timeout` only when appropriate | | `--testId` times out | Incorrect capitalization | Use `--testid` in lowercase | | Daemon address already in use | Stale daemon state (`MM_PORT_IN_USE`) | Run `yarn mm stop --force`, then `yarn mm launch` | diff --git a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md index 0ed659c1..01f593be 100644 --- a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md +++ b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md @@ -75,24 +75,24 @@ When a session is already active, `mm launch` rejects with `MM_SESSION_ALREADY_R When attaching to Metro (`--metro-port`), the workflow is **attach-only** — it never spawns Metro. If the app is already running and healthily attached to Metro (Hermes target found at `/json`), `mm launch` connects without relaunching. If the app is not healthily attached, it terminates and re-launches via the deep link. On a fresh-booted simulator, a one-time relaunch ensures the accessibility tree is valid. Release/prod builds have no Hermes inspector; Metro attach requires a dev build. -### 2. Reuse Knowledge +### 2. Observe Before Acting ```bash -yarn mm knowledge-search "" -yarn mm knowledge-sessions +yarn mm describe-screen ``` -Reuse a known successful sequence when available. Otherwise discover the flow and let the session record it. +Use fresh output after navigation. Accessibility refs (`e1`, `e2`, ...) are ephemeral: always take refs from the latest response (`describe-screen` or the `observations` of your last action). -### 3. Observe Before Acting +**Observation efficiency:** mutating actions (`click`, `type`, ...) and `run-steps` return a compact diff of what changed. Reads, discovery commands and `mm cdp` return no observation. Use the global `--observe ` flag to change the observation of one call: ```bash -yarn mm describe-screen +yarn mm click --testid log-in-button --observe none # chain known actions: skip the observation +yarn mm click e5 --observe full # after navigation: get the whole screen ``` -Use fresh output after navigation. Accessibility refs (`e1`, `e2`, ...) are ephemeral. +Only mutating and batch calls wait for animations before they observe. To get a settled screen after an action, let the action observe (`--observe full`) instead of running a read afterwards. -### 4. Interact +### 3. Interact Two targeting methods work on both iOS and Android: **test IDs** and **accessibility refs**. Use one per command; prefer a stable test ID, fall back to a fresh a11y ref from `describe-screen`. On Android, a test ID matches the element's `resource-id` / content-description / text (fuzzy, case-insensitive); on iOS it matches the accessibility identifier/label. @@ -110,7 +110,7 @@ yarn mm type e2 "text" The CLI flag is `--testid` (all lowercase); `--testId` is silently mis-parsed as a positional target and hits the wrong element. `mm type` clears the field before typing (iOS: idb `cmd+a` → delete → type; Android: the backend clears via adb before `input text`). -### 5. Verify and Capture Evidence +### 4. Verify and Capture Evidence After a mutating sequence: @@ -122,9 +122,9 @@ After a mutating sequence: yarn mm screenshot --name "after-action" ``` -If the state is wrong, capture a debug screenshot, search knowledge, and retry from fresh refs. +If the state is wrong, capture a debug screenshot and retry from fresh refs. -### 6. Cleanup +### 5. Cleanup ```bash yarn mm cleanup @@ -152,17 +152,20 @@ The Android device backend captures the hierarchy with `uiautomator dump`. But ` ERROR: could not get idle state. ``` -To handle this, `@metamask/device-mcp` **ships a small self-instrumenting snapshot-helper APK** (`node_modules/@metamask/device-mcp/dist/android/device-mcp-android-snapshot-helper-.apk`, package `io.metamask.devicemcp.snapshothelper`). The helper captures the hierarchy **without** waiting for idle and streams it back over `am instrument`. It is a `testOnly` APK installed on demand with `adb install -t`, reused across the session, and its signing certificate is cryptographically verified before use (fails **closed** on a signer mismatch rather than silently falling back). The helper is a **separate app** from MetaMask: it shares the `io.metamask` prefix but is a distinct package, and the backend resolves it by exact package name (`pm list packages`/`pm path` on the full `io.metamask.devicemcp.snapshothelper`), so it never collides with the wallet's `io.metamask` package or with `mm open-app`/`close-app`/`cdp --app-id`. +To handle this, `@metamask/device-mcp` (0.7.0 in MetaMask Mobile) **ships a small self-instrumenting snapshot-helper APK** (`node_modules/@metamask/device-mcp/dist/android/device-mcp-android-snapshot-helper-.apk`, package `io.metamask.devicemcp.snapshothelper`). The helper captures the hierarchy **without** waiting for idle and streams it back over `am instrument`. It is a `testOnly` APK installed on demand with `adb install -t`, reused across the session, and its signing certificate is cryptographically verified before use (fails **closed** on a signer mismatch rather than silently falling back). The helper is a **separate app** from MetaMask: it shares the `io.metamask` prefix but is a distinct package, and the backend resolves it by exact package name (`pm list packages`/`pm path` on the full `io.metamask.devicemcp.snapshothelper`), so it never collides with the wallet's `io.metamask` package or with `mm open-app`/`close-app`/`cdp --app-id`. Behavior is controlled by the `DEVICE_MCP_ADB_SNAPSHOT` env var on the device backend (`@metamask/device-mcp`, which the `mm` CLI drives on Android): -- **`auto`** (default) — fast `uiautomator dump` first (wins instantly on idle screens), then the bundled helper if the dump produced no hierarchy, then remaining dump retries as a last resort. -- **`instrument`** — use the bundled helper only. **Use this when snapshots repeatedly fail with `could not get idle state`** on a churny screen (e.g. loading balances, animated onboarding, a live dapp page). +- **`helper-first`** (default since `@metamask/device-mcp` 0.6.0) — the bundled helper first (fast on busy screens, and a fuller tree than the dump, including WebView content), then one `uiautomator dump` attempt (three if the helper never ran) for non-trust helper failures only. +- **`auto`** — fast `uiautomator dump` first, then the bundled helper, then remaining dump retries. Sticky per session: once the helper wins, later snapshots skip the quick dump. +- **`instrument`** — use the bundled helper only, with no dump fallback (fully fail-closed). - **`dump`** — stock `uiautomator dump` only; the helper APK is never installed. +The first helper snapshot of a session can take longer, because it can install the helper (its own timeout, up to 60 s). + Operational guidance when a snapshot fails to capture: -1. If you see `could not get idle state`, the screen is redrawing — retry; in `auto` the helper kicks in automatically. +1. If you see `could not get idle state` or `Capture did not stabilize within 500ms`, the screen is redrawing or in transition — retry; the backend already retries the helper before any dump fallback. 2. If it still fails, restart the session with `DEVICE_MCP_ADB_SNAPSHOT=instrument` in the environment to force the helper path. 3. If snapshots fail **closed with a trust/signer error**, a different app is squatting the helper's package name — uninstall it (`adb uninstall io.metamask.devicemcp.snapshothelper`) so the bundled, correctly-signed helper can install, or fall back to `DEVICE_MCP_ADB_SNAPSHOT=dump`. 4. As a last resort, use `mm screenshot` (always works) plus `mm tap-coordinates` from the image, or drive web content via the WebView CDP path. @@ -207,6 +210,11 @@ yarn mm run-steps '{"steps":[ The input must be an object containing `steps`, not a bare array. +- Add `batchTimeoutMs` (1000–30000 ms) for an overall deadline. The whole call stops after 30 s, so use a lower value (for example 20000) to get partial results, and split longer flows into several batches. +- Add a top-level `observe` (`none` / `diff` / `full`, default `diff`) to set the observation collected after the batch. `includeObservations` was removed; check `summary.ok` to react to failures. + +Pattern: discover with `yarn mm describe-screen` → batch known steps with `yarn mm run-steps` → re-verify with `yarn mm describe-screen`. + ## Mobile Limitations and Gotchas - **Only `--testid` and a11y refs target elements.** `--selector` throws (`CSS selectors are not supported on mobile`) and `--within` throws (`Scoped element search (within) is not supported on mobile`) at the driver on both iOS and Android, even though the shared CLI parses both flags. Do not use them. @@ -218,7 +226,13 @@ The input must be an object containing `steps`, not a bare array. - `navigate-home` and `navigate-settings` are not implemented; navigate through visible UI elements. - `mm cdp` requires Metro. - One Metro process per worktree is recommended. -- Mutating commands can return compact observations; request a full `describe-screen` whenever refs or state are uncertain. +- Mutating commands return compact diff observations; request a full `describe-screen` whenever refs or state are uncertain. +- **After a call with `--observe none`, refs are not refreshed.** Target by `--testid`, or run `describe-screen` before you use refs again: a stale ref can tap a different element. +- **Output is one line of JSON** when stdout is not a terminal. Parse it as JSON. Add `--pretty` (or set `MM_PRETTY=1`) only when a human reads the output. +- **Observations carry no app state**, and `mm get-state` on mobile only reports whether the app is loaded (`source: "stub"`) plus `currentScreen` when a known test ID identifies it. Read wallet state (account, network, balance) from the screen or through `mm cdp`. +- **Timed-out calls are cancelled.** On `Tool execution timed out`, an action that already started can still have happened, so run `describe-screen` before you retry. +- The knowledge store (`mm knowledge-*`) and the `--goal` / `--flow-tags` launch flags were removed in `@metamask/client-mcp-core` 0.13.0. +- Call `node_modules/.bin/mm` directly to skip the Yarn startup (about 0.4–0.5 s per call) when running many commands. - Never assume wallet credentials, balances, networks, or onboarding state. Inspect the installed app and obtain needed credentials from the user/environment. ## Error Recovery From c206c1fe3f20a221b94217d57fa1f4ec6254113a Mon Sep 17 00:00:00 2001 From: cryptotavares Date: Sun, 1 Nov 2026 17:06:22 +0000 Subject: [PATCH 2/5] docs(testing): pass --metro-port on each mm cdp call --- .../skills/mobile-visual-testing/references/cli-reference.md | 2 ++ .../skills/mobile-visual-testing/repos/metamask-mobile.md | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md index c382ebfa..daff8a29 100644 --- a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md +++ b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md @@ -239,6 +239,8 @@ yarn mm launch --metro-port 8081 MM_METRO_PORT=8081 yarn mm launch ``` +`yarn mm cdp` and `yarn mm hermes-targets` do not reuse the launch's `--metro-port`: they use the daemon's `HERMES_METRO_PORT` or 8081. When Metro runs on another port, pass `--metro-port ` on every `cdp` / `hermes-targets` call, or the call fails with `HERMES_CONNECTION_FAILED`. + On Node 20, add `NODE_OPTIONS="--experimental-websocket"` when launching. `yarn mm cdp` returns no observation by default. After runtime mutation, add `--observe full` or run `yarn mm describe-screen` to resynchronize observations. ## Simulator Selection diff --git a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md index 01f593be..eefbc59b 100644 --- a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md +++ b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md @@ -224,7 +224,7 @@ Pattern: discover with `yarn mm describe-screen` → batch known steps with `yar - No `mm build`; build/install separately. - No URL navigation, tab switching, browser notification pages, or browser clipboard APIs. - `navigate-home` and `navigate-settings` are not implemented; navigate through visible UI elements. -- `mm cdp` requires Metro. +- `mm cdp` requires Metro. It does not reuse the launch's `--metro-port`, so pass `--metro-port ` on each `cdp` / `hermes-targets` call when Metro is not on 8081. - One Metro process per worktree is recommended. - Mutating commands return compact diff observations; request a full `describe-screen` whenever refs or state are uncertain. - **After a call with `--observe none`, refs are not refreshed.** Target by `--testid`, or run `describe-screen` before you use refs again: a stale ref can tap a different element. From 54ef758f362b3ce64343832956a9b66cf1d42966 Mon Sep 17 00:00:00 2001 From: cryptotavares Date: Sun, 1 Nov 2026 22:10:25 +0000 Subject: [PATCH 3/5] docs(testing): get-state has no currentScreen on mobile; tap-coordinates for iOS home actions --- .../skills/mobile-visual-testing/references/cli-reference.md | 2 +- .../skills/mobile-visual-testing/repos/metamask-mobile.md | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md index daff8a29..1ac68eb6 100644 --- a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md +++ b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md @@ -169,7 +169,7 @@ All interaction commands accept on iOS: | `yarn mm screenshot [--name ]` | Capture the current simulator screen | | `yarn mm accessibility-snapshot` | Return a trimmed accessibility tree (the shared `--root ` flag is ignored on iOS; the full tree is always returned) | | `yarn mm list-testids [--limit ]` | List visible test IDs | -| `yarn mm get-state` | Return a partial app-state snapshot: `isLoaded`, `extensionId` (the bundle ID / package), `source: "stub"`, and `currentScreen` only when a known test ID identifies the screen. Wallet fields (unlock, account, network, balance) are omitted, not guessed | +| `yarn mm get-state` | Return a partial app-state snapshot: `isLoaded`, `extensionId` (the bundle ID / package) and `source: "stub"`. There is no `currentScreen`, and wallet fields (unlock, account, network, balance) are omitted, not guessed. Identify the screen from `describe-screen` test IDs (for example `login-password-input` = unlock, `wallet-screen` = wallet home) | | `yarn mm get-context` | Report the static prod environment and available mobile capabilities | Mutating tools (`click`, `type`, ...) and `run-steps` return observations with fresh references, as a compact diff from the prior observation by default. Reads, discovery commands and `cdp` return no observation. Use `describe-screen` whenever a complete tree or refreshed baseline is needed. diff --git a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md index eefbc59b..88a2c489 100644 --- a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md +++ b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md @@ -229,7 +229,8 @@ Pattern: discover with `yarn mm describe-screen` → batch known steps with `yar - Mutating commands return compact diff observations; request a full `describe-screen` whenever refs or state are uncertain. - **After a call with `--observe none`, refs are not refreshed.** Target by `--testid`, or run `describe-screen` before you use refs again: a stale ref can tap a different element. - **Output is one line of JSON** when stdout is not a terminal. Parse it as JSON. Add `--pretty` (or set `MM_PRETTY=1`) only when a human reads the output. -- **Observations carry no app state**, and `mm get-state` on mobile only reports whether the app is loaded (`source: "stub"`) plus `currentScreen` when a known test ID identifies it. Read wallet state (account, network, balance) from the screen or through `mm cdp`. +- **Observations carry no app state**, and `mm get-state` on mobile only reports whether the app is loaded (`source: "stub"`). Identify the screen from `describe-screen` test IDs (`login-password-input` = unlock, `wallet-screen` = wallet home). Read wallet state (account, network, balance) from the screen or through `mm cdp`. +- **iOS wallet home actions (Buy / Swap / Send / Receive) are not in the tree.** On iOS 18+ the snapshot reads the `axbridge` tree, where the balance hero and the action row are one unlabelled container, so `wallet-send-button` and the other action test IDs are missing and `click --testid wallet-send-button` times out. Take the button position from `mm screenshot` and use `mm tap-coordinates ` (points). The next screens (for example Send, with `send-navbar-back-button`) are in the tree again. - **Timed-out calls are cancelled.** On `Tool execution timed out`, an action that already started can still have happened, so run `describe-screen` before you retry. - The knowledge store (`mm knowledge-*`) and the `--goal` / `--flow-tags` launch flags were removed in `@metamask/client-mcp-core` 0.13.0. - Call `node_modules/.bin/mm` directly to skip the Yarn startup (about 0.4–0.5 s per call) when running many commands. From e11651d3af3a4f721f82c3fde68dde3569b46337 Mon Sep 17 00:00:00 2001 From: cryptotavares Date: Sun, 1 Nov 2026 22:30:32 +0000 Subject: [PATCH 4/5] docs(testing): keyboard and slow-navigation workarounds from live runs --- .../skills/mobile-visual-testing/repos/metamask-mobile.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md index 88a2c489..9c05a26f 100644 --- a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md +++ b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md @@ -230,6 +230,8 @@ Pattern: discover with `yarn mm describe-screen` → batch known steps with `yar - **After a call with `--observe none`, refs are not refreshed.** Target by `--testid`, or run `describe-screen` before you use refs again: a stale ref can tap a different element. - **Output is one line of JSON** when stdout is not a terminal. Parse it as JSON. Add `--pretty` (or set `MM_PRETTY=1`) only when a human reads the output. - **Observations carry no app state**, and `mm get-state` on mobile only reports whether the app is loaded (`source: "stub"`). Identify the screen from `describe-screen` test IDs (`login-password-input` = unlock, `wallet-screen` = wallet home). Read wallet state (account, network, balance) from the screen or through `mm cdp`. +- **The keyboard can cover the submit button.** After `mm type` into the unlock field, the iOS keyboard covers **Unlock**. On iOS, `mm dismiss-keyboard` fails (`idb ui key RETURN` is rejected by idb), so tap the keyboard's return key instead: `mm click --testid Done` (it also submits the unlock form). On Android, `mm press-button back` hides the keyboard, then `mm click --testid log-in-button`. +- **A navigation can finish after the observation.** The mobile observation retries for at most about 2 s, so after a tap that opens a slow screen (for example the first **Send** of a session) `observations` can still show the previous screen. Confirm with `mm wait-for --testid ` before the next step. - **iOS wallet home actions (Buy / Swap / Send / Receive) are not in the tree.** On iOS 18+ the snapshot reads the `axbridge` tree, where the balance hero and the action row are one unlabelled container, so `wallet-send-button` and the other action test IDs are missing and `click --testid wallet-send-button` times out. Take the button position from `mm screenshot` and use `mm tap-coordinates ` (points). The next screens (for example Send, with `send-navbar-back-button`) are in the tree again. - **Timed-out calls are cancelled.** On `Tool execution timed out`, an action that already started can still have happened, so run `describe-screen` before you retry. - The knowledge store (`mm knowledge-*`) and the `--goal` / `--flow-tags` launch flags were removed in `@metamask/client-mcp-core` 0.13.0. From 4850070da35200c40481ada8262137a77917b6c4 Mon Sep 17 00:00:00 2001 From: cryptotavares Date: Mon, 2 Nov 2026 15:02:43 +0000 Subject: [PATCH 5/5] docs(testing): --testid substring match and the Android account picker --- .../skills/mobile-visual-testing/repos/metamask-mobile.md | 1 + 1 file changed, 1 insertion(+) diff --git a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md index 9c05a26f..6b82791d 100644 --- a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md +++ b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md @@ -233,6 +233,7 @@ Pattern: discover with `yarn mm describe-screen` → batch known steps with `yar - **The keyboard can cover the submit button.** After `mm type` into the unlock field, the iOS keyboard covers **Unlock**. On iOS, `mm dismiss-keyboard` fails (`idb ui key RETURN` is rejected by idb), so tap the keyboard's return key instead: `mm click --testid Done` (it also submits the unlock form). On Android, `mm press-button back` hides the keyboard, then `mm click --testid log-in-button`. - **A navigation can finish after the observation.** The mobile observation retries for at most about 2 s, so after a tap that opens a slow screen (for example the first **Send** of a session) `observations` can still show the previous screen. Confirm with `mm wait-for --testid ` before the next step. - **iOS wallet home actions (Buy / Swap / Send / Receive) are not in the tree.** On iOS 18+ the snapshot reads the `axbridge` tree, where the balance hero and the action row are one unlabelled container, so `wallet-send-button` and the other action test IDs are missing and `click --testid wallet-send-button` times out. Take the button position from `mm screenshot` and use `mm tap-coordinates ` (points). The next screens (for example Send, with `send-navbar-back-button`) are in the tree again. +- **`--testid` matches a substring, not the exact ID.** The tap goes to the first element in tree order whose test ID *contains* the value, and an a11y ref resolves to the same test ID query. On the Android wallet home, `click --testid account-picker` (or its ref) taps `account-picker-avatar` inside `profile-button` and opens the profile screen ("Create profile"). If the screen after a tap is not the expected one, check `list-testids` for longer IDs that contain your value, and use `mm tap-coordinates` at the center of the element's `bounds` from `describe-screen` (Android: pixels; for `account-picker`, `384 136`). - **Timed-out calls are cancelled.** On `Tool execution timed out`, an action that already started can still have happened, so run `describe-screen` before you retry. - The knowledge store (`mm knowledge-*`) and the `--goal` / `--flow-tags` launch flags were removed in `@metamask/client-mcp-core` 0.13.0. - Call `node_modules/.bin/mm` directly to skip the Yarn startup (about 0.4–0.5 s per call) when running many commands.