Skip to content

feat!: structured CLI errors with distinct exit codes #78

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
cryptotavares wants to merge 1 commit into from
Draft

feat!: structured CLI errors with distinct exit codes #78

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- **BREAKING:** `mm` failures are structured. A failed command prints one JSON line on stdout, `{"ok":false,"error":{"code","message","hint"?,"retryable"},"meta"?}`, instead of `Error: <message>` on stderr, and exits with a code per error class: 2 usage/validation, 3 no session or daemon down/lost, 4 target not found or not actionable, 5 timeout, 6 platform/device unavailable, 1 anything else (7 is reserved for a version mismatch). The daemon's `error.code` now reaches the caller. `exitCodeForError`, `isRetryableError`, `defaultHintForError`, `CliExitCodes` and `DaemonErrorCodes` are exported, and `ErrorCodes` adds `MM_BATCH_TIMEOUT`, the `MM_CLIPBOARD_*` codes and the CLI codes `MM_DAEMON_NOT_RUNNING`, `MM_DAEMON_START_FAILED`, `MM_DAEMON_ALREADY_RUNNING`, `MM_DAEMON_CONNECTION_LOST` and `MM_REQUEST_TIMEOUT`. Click and type timeouts say "state unknown". `launch` keeps any `MM_*` code thrown by a consumer, not only known codes.
- Mobile `click` never reports a tap that hit something else: device-mcp refuses a tap when another element covers the target, and the click fails with the new `MM_TARGET_OBSCURED` (`MM_TARGET_NOT_VISIBLE` when the target is off screen). `click --dismiss-keyboard` closes a covering keyboard and taps once more (`keyboardDismissed: true`). With an observation, a mobile click result has `screenChanged`. The hit-test needs a device-mcp release with MetaMask/device-mcp#51.
- Start the CLI faster: find the project root without running `git` and do not load zod. Load `@metamask/device-mcp` only when a mobile session uses it, so browser daemons start without it.
- **BREAKING:** `scroll_to_element` `direction` is now the content direction (`down` reveals content below; it was the finger direction) and defaults to `auto`. The result adds `swipes`, `inViewport` and `bounds`; a target that is found but not brought into view fails with the new `MM_TARGET_NOT_VISIBLE`, and one never found with `MM_TARGET_NOT_FOUND`. The viewport check needs a device-mcp release with MetaMask/device-mcp#50; with device-mcp 0.7.0 only `bounds` is returned.
Expand Down
54 changes: 50 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -744,10 +744,11 @@ them — for example by refusing `--reinstall` when no `--app-bundle` is supplie
or by requiring `--allow-fox-code-mismatch` before replacing an app whose
identity differs from the installed one.

When a consumer's `launch()` throws an error carrying a `code` that is a known
[`ErrorCode`](#error-classification), the `launch` tool preserves it instead of
collapsing it into `MM_LAUNCH_FAILED`, so precise device/prerequisite failures
reach the agent intact.
When a consumer's `launch()` throws an error carrying an `MM_*` `code` (a known
[`ErrorCode`](#error-classification) or a consumer-specific one), the `launch`
tool preserves it instead of collapsing it into `MM_LAUNCH_FAILED`, so precise
device/prerequisite failures reach the agent intact. The CLI maps a code it does
not know to exit 1.

#### Mobile device actions (iOS/Android only)

Expand Down Expand Up @@ -842,6 +843,51 @@ Tool errors are classified into specific error codes for structured handling:
| `MM_DEVICE_ACTION_FAILED` | A mobile device action (swipe, tap, app control, clipboard, etc.) failed |
| `MM_DEVICE_NOT_AVAILABLE` | A mobile-only tool was invoked without a mobile driver capability |
| `MM_TOOL_NOT_SUPPORTED_ON_PLATFORM` | Tool gated off the active platform (browser-only tool on mobile, or mobile-only tool on browser) |
| **CLI (set by `mm`, not a tool)** | |
| `MM_DAEMON_NOT_RUNNING` | No daemon, or the daemon refused the connection |
| `MM_DAEMON_START_FAILED` | The daemon did not become ready in time |
| `MM_DAEMON_ALREADY_RUNNING` | `mm serve` while a daemon is running |
| `MM_DAEMON_CONNECTION_LOST` | The connection failed after the request was sent; the action may have run |
| `MM_REQUEST_TIMEOUT` | The CLI stopped waiting for the daemon; the action state is unknown |

The daemon HTTP server also sets `VALIDATION_ERROR` (schema), `TOOL_NOT_FOUND`,
`SERVER_NOT_STARTED`, `TOOL_EXECUTION_FAILED` and `INTERNAL_ERROR`
(`DaemonErrorCodes`).

### CLI failure output and exit codes

A failed `mm` command prints one JSON line on **stdout** and exits non-zero.
stderr is for human logs only (on a terminal it also gets `Error: <message>`).

```json
{
"ok": false,
"error": {
"code": "MM_TARGET_NOT_FOUND",
"message": "...",
"hint": "Run `mm describe-screen` to get fresh refs.",
"retryable": false
},
"meta": { "totalMs": 512 }
}
```

`hint` is optional. `retryable` is true only when running the same command
again, unchanged, is safe (the call only reads, or nothing ran); it is false for
click, type and request timeouts because the action state is unknown. `meta` is
present when the daemon answered. `exitCodeForError()` in
`src/tools/types/errors.ts` is the one map from error code to exit code:

| Exit | Meaning | Codes (examples) |
| ---- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| 0 | ok | |
| 1 | anything else (internal error, tool failure, unknown code) | `MM_INTERNAL_ERROR`, `TOOL_EXECUTION_FAILED`, `MM_LAUNCH_FAILED`, `MM_SEED_FAILED` |
| 2 | usage or validation; nothing ran | `MM_INVALID_INPUT`, `MM_INVALID_CONFIG`, `VALIDATION_ERROR`, `TOOL_NOT_FOUND`, `MM_CDP_BLOCKED` |
| 3 | no session, daemon not running, daemon connection lost | `MM_NO_ACTIVE_SESSION`, `MM_PAGE_CLOSED`, `MM_DAEMON_NOT_RUNNING`, `MM_DAEMON_CONNECTION_LOST` |
| 4 | target not found or not actionable | `MM_TARGET_NOT_FOUND`, `MM_TARGET_NOT_VISIBLE`, `MM_TARGET_OBSCURED`, `MM_CLICK_FAILED`, `MM_TAB_NOT_FOUND` |
| 5 | timeout (click, type and request timeouts: action state unknown) | `MM_WAIT_TIMEOUT`, `MM_CLICK_TIMEOUT`, `MM_TYPE_TIMEOUT`, `MM_REQUEST_TIMEOUT`, `MM_BATCH_TIMEOUT` |
| 6 | platform, device, Metro or Hermes not available | `MM_DEVICE_ACTION_FAILED`, `MM_DEVICE_NOT_AVAILABLE`, `MM_HERMES_FAILED`, `MM_TOOL_NOT_SUPPORTED_ON_PLATFORM` |
| 7 | daemon version mismatch (reserved) | |

## Development

Expand Down
38 changes: 35 additions & 3 deletions SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Cleanup also disposes the session driver's owned resources after consumer app/se
- **Always `describe-screen` before interacting.** Refs like `e1`, `e2` are ephemeral — use the refs from the latest response (`describe-screen` or the `observations` of your last action).
- **Always `describe-screen` after interacting** — OR use inline `observations` from mutating tool responses. Mutating tools (click, type, navigate, etc.) return an `observations` object with fresh `testIds` and `a11y` refs. You can use these refs directly for the next interaction without calling `describe-screen`. Call `describe-screen` when you need the full tree or screenshots, and `get-state` when you need app state (observations carry none).
- **One target per command.** Specify exactly ONE of: a11y ref (`e5`), testId, or CSS selector.
- **Errors are structured.** Check the `error.code` field to decide recovery strategy (see Error Codes below).
- **Errors are structured.** A failed command prints one JSON line on stdout (`{"ok":false,"error":{"code","message","hint"?,"retryable"}}`) and exits non-zero. Branch on `error.code` or the exit code (see Error Codes below), not on stderr text. Never repeat a click or type that failed with exit 5 before you check the screen.

## Observation Behavior

Expand Down Expand Up @@ -685,7 +685,35 @@ Every interaction command (`click`, `type`, `get-text`, `wait-for`) needs a targ

## Error Codes

When a command fails, the response includes `error.code`. Use this to decide what to do:
When a command fails, `mm` prints one JSON line on **stdout** and exits non-zero:

```json
{
"ok": false,
"error": {
"code": "MM_TARGET_NOT_FOUND",
"message": "...",
"hint": "Run `mm describe-screen` to get fresh refs.",
"retryable": false
},
"meta": { "totalMs": 512 }
}
```

`hint` (optional) says what to do next. `retryable: true` means you can run the same command again unchanged. It is never true for a click, type or request timeout: the action state is unknown, so check the screen first.

| Exit | Meaning | Do |
| ---- | ------------------------------------------------------ | --------------------------------------------------------------------- |
| 0 | ok | |
| 1 | anything else (internal, tool failure) | Read `error.message` |
| 2 | usage or validation; nothing ran | Fix the command |
| 3 | no session, daemon not running, daemon connection lost | `mm launch`; after `MM_DAEMON_CONNECTION_LOST` check the screen first |
| 4 | target not found or not actionable | `mm describe-screen`, then re-target / scroll / dismiss the keyboard |
| 5 | timeout (click/type/request: action state unknown) | `mm describe-screen` before any retry |
| 6 | platform, device, Metro or Hermes not available | Check the device, Metro and the build |
| 7 | daemon version mismatch (reserved) | |

Use `error.code` to decide what to do:

| Code | Meaning | Recovery |
| ----------------------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
Expand All @@ -700,7 +728,7 @@ When a command fails, the response includes `error.code`. Use this to decide wha
| `MM_CLICK_FAILED` | Click failed after finding element | Element may be obscured; try waiting or scrolling |
| `MM_TARGET_OBSCURED` | Mobile: another element (keyboard, sheet, overlay) covers the target; nothing was tapped | Keyboard: `mm dismiss-keyboard` or `mm click ... --dismiss-keyboard`. Overlay: close it first |
| `MM_TARGET_NOT_VISIBLE` | Mobile: target found but outside the screen | `mm scroll-to-element <ref>`, then click |
| `MM_CLICK_TIMEOUT` | Click action timed out (element found, click hung) | Run `mm describe-screen` to verify if click completed; retry with `--timeout` or different approach |
| `MM_CLICK_TIMEOUT` | Click action timed out (element found, click hung); state unknown | Run `mm describe-screen` to check if the click ran before you retry |
| `MM_TYPE_FAILED` | Type failed after finding element | Element may not be an input; verify with describe-screen |
| `MM_TYPE_TIMEOUT` | Fill action timed out | Run `mm describe-screen` to verify state; retry with `--timeout` |
| `MM_GETTEXT_FAILED` | getText operational failure (non-timeout) | Element may be detached; run `mm describe-screen` and re-target |
Expand Down Expand Up @@ -732,6 +760,10 @@ When a command fails, the response includes `error.code`. Use this to decide wha
| `MM_DEVICE_ACTION_FAILED` | A mobile device action failed | Check the target/args; ensure the device/emulator is responsive |
| `MM_DEVICE_NOT_AVAILABLE` | Mobile-only tool invoked without a mobile driver | Launch a mobile (iOS/Android) session first |
| `MM_TOOL_NOT_SUPPORTED_ON_PLATFORM` | Tool gated off the active platform | Use the tool on its supported platform (browser vs mobile) |
| `MM_DAEMON_NOT_RUNNING` | No daemon, or it refused the connection | Run `mm launch` |
| `MM_DAEMON_CONNECTION_LOST` | Connection failed after the request was sent | The action may have run: `mm describe-screen` before you retry |
| `MM_REQUEST_TIMEOUT` | The CLI stopped waiting for the daemon | Action state unknown: `mm describe-screen` before you retry |
| `VALIDATION_ERROR` | Input failed the tool schema | Fix the arguments (exit 2) |

## Available Contracts (E2E only)

Expand Down
Loading
Loading