Skip to content

docs(testing): update mobile-visual-testing skill for client-mcp-core 0.13.0 - #211

Draft
cryptotavares wants to merge 5 commits into
mainfrom
cryptotavares/mobile-visual-testing-client-mcp-core-0.13.0
Draft

cryptotavares wants to merge 5 commits into
mainfrom
cryptotavares/mobile-visual-testing-client-mcp-core-0.13.0

Conversation

@cryptotavares

@cryptotavares cryptotavares commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Description

Updates the Mobile testing/mobile-visual-testing skill for @metamask/client-mcp-core 0.13.0 (release PR #80) and @metamask/device-mcp 0.7.0. With 0.13.0, agents that follow the current skill fail on step 2: mm knowledge-search now returns Error: unknown command 'knowledge-search'.

This is the Mobile counterpart of #210 (Extension visual-testing skill).

Area Before After
Knowledge store "Reuse Knowledge" workflow step; mm knowledge-* in the CLI reference, error recovery and verify loop; --goal / --flow-tags launch flags Step, commands and flags removed; the CLI reference notes the removal and the test-artifacts/<sessionId>/session.json path
Observations "Mutating commands can return compact observations" Global --observe none|diff|full flag, per-category defaults (reads, discovery and cdp return none), stale refs after --observe none, only mutating/batch calls wait for animations, no app state in observations, flat mobile a11y tree without status/nav bars and dev overlay
get-state Listed both as available and as "E2E-context only, do not use" Documents the real mobile response: isLoaded, extensionId, source: "stub". No currentScreen and no wallet fields; identify the screen from test IDs
Live-run workarounds Not documented iOS: click --testid Done instead of dismiss-keyboard; tap-coordinates for wallet home Buy / Swap / Send / Receive (missing from the iOS 18+ axbridge tree). Android: press-button back to hide the keyboard. Both: wait-for after a slow navigation, because the observation can still show the previous screen
Output Not documented One-line JSON when not on a terminal; --pretty / MM_PRETTY; meta.timing on every response; node_modules/.bin/mm to skip Yarn startup
run-steps includeObservations; batchTimeoutMs without limits Top-level observe (replaces includeObservations); batchTimeoutMs range 1000–30000 and the 30 s request limit
Timeouts Not documented Timed-out calls are cancelled, but an action that already started can still have happened
Android snapshots auto described as the default helper-first is the default since device-mcp 0.6.0; auto / instrument / dump semantics and first-snapshot install time updated

Files:

  • domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md: workflow (knowledge step removed, observation guidance), batching, gotchas, Android snapshot modes
  • domains/testing/skills/mobile-visual-testing/references/cli-reference.md: new Global Options section; knowledge commands and launch flags removed; get-state, run-steps and cdp updated
  • domains/testing/skills/mobile-visual-testing/references/error-recovery.md: knowledge step removed; new failure rows
  • CHANGELOG.md: Unreleased entry

Companion Mobile PR: MetaMask/metamask-mobile#38594

Type of Change

  • New skill
  • Skill improvement/update
  • Bug fix
  • Documentation update
  • Other (please describe):

Checklist

  • I have read the CONTRIBUTING.md guidelines
  • My skill follows the SKILL_TEMPLATE.md format
  • I have tested this skill with an AI agent
  • My skill does not contain any secrets, private keys, or sensitive data
  • I have added appropriate documentation
  • My changes don't break existing skills

Testing

  • Behavior statements were checked against client-mcp-core release/0.13.0 (mm --help, MobilePlatformDriver.getAppState) and the installed @metamask/device-mcp 0.7.0 README.
  • Live sessions on the Mobile PR branch (core 0.13.0 via yalc, device-mcp 0.7.0, Metro attached) on an iOS Simulator (iPhone 17 Pro) and an Android emulator (API 36): launch, describe-screen, get-state ({"isLoaded":true,"extensionId":"...","source":"stub"}), type (diff observation), get-text (no observation), click --observe none|full, run-steps with observe and batchTimeoutMs, includeObservations and batchTimeoutMs > 30000 rejected, --pretty, cdp / hermes-targets with --metro-port, screenshot, list-testids, cleanup; knowledge-* rejected as unknown commands. Results are in chore(llm-workflow): upgrade to @metamask/client-mcp-core 0.13.0 and @metamask/device-mcp 0.7.0 metamask-mobile#38594.
  • Unlocked read-only flows on both devices (unlock, wallet home, account picker, open and close Send, cdp isUnlocked()); nothing was sent or signed.
  • Found while testing, all the same before 0.13.0, now documented in the skill:
    • mm cdp does not reuse the launch's --metro-port (it defaults to 8081). Pass --metro-port on each cdp / hermes-targets call.
    • Mobile get-state never returns currentScreen.
    • iOS 18+: the wallet home action buttons are missing from the axbridge tree.
    • iOS: mm dismiss-keyboard fails (idb ui key RETURN rejected).
    • --testid matches a substring (device-mcp fuzzyMatch): on Android wallet home, account-picker taps account-picker-avatar and opens the profile screen.
  • yarn lint passes and yarn test passes (73/73).
  • yarn audit:skills reports 1 error, in typescript/tsc-blindspots (description length). The same error is on main; this PR adds no new errors.

Additional Context

Keep this PR in draft until @metamask/client-mcp-core@0.13.0 is published and the Mobile PR merges. Agents in the Mobile repo on 0.9.0 still have the knowledge commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant