[cryptotavares]

Summary: Cancel timed-out requests cooperatively and fence ref-map commits against aborted or older requests. Capture mobile trees and test IDs together, and settle mobile observations after mutations.

Scope change (2026-10-27): the ref-stability layer (renumbering refs so unchanged elements keep their numbers) was removed in 6944868. Refs are positional again, as on main. See Why no ref stability.

Stack context: mm CLI v2, Phase 1, P1-08 + P1-09, combined in one PR. Base: #62 (cryptotavares/mmcli-p1-06-observe-level-no-state); followed by #65.

Problem

A queue timeout rejects the caller, but the task can keep running and overwrite another request's refs. An observation timeout also leaves collection running.

Mobile observations read test IDs and the tree from separate snapshots. A click can return the old screen or Unlock, busy.

Solution

Keep cancellation and commit fencing at separate boundaries:

1. Cooperative cancellation: RequestQueue passes an AbortSignal and aborts it before timeout rejection. Tools check supported checkpoints. Observations use a linked controller and their own deadline.
2. Commit fencing: the server owns a small RefStore (94 lines). Each started task gets a generation. Every ref write (sessionManager.setRefMap / clearRefMap from tools, and the observation) checks the generation and request signal, even when a tool ignores cancellation.

flowchart TD
    A[Queued task starts] --> B[Create signal and ref generation]
    B --> C[Run tool through guarded context]
    C --> D[Optional observation checks linked abort signal]
    C -->|ref write| F{Request current and not aborted?}
    D -->|ref write| F
    F -->|yes| G[Commit map]
    F -->|no| X[Drop write. Keep current refs.]
    T[Queue timeout aborts request] -.-> C
    T -.-> D
    T -.-> F

    classDef new fill:#fff3cd,stroke:#d39e00
    classDef stop fill:#f8d7da,stroke:#c82333
    class B,C,D,F,G,T new
    class X stop

Loading

Yellow boxes are new. Red boxes stop the ref write.

Tools write refs through context.sessionManager, which the daemon proxies: setRefMap and clearRefMap go through the request handle. Other members retain the real receiver, including #private fields. ISessionManager stays unchanged.

Optional captureScreen() derives both mobile outputs from one snapshot; drivers without it retain separate reads. After eligible mutations, retry busy screens and allow one unchanged-screen retry, compared against the last tree the agent received (lastScreen: the delivered observation, else the describe_screen / full accessibility_snapshot tree). Launch, cleanup, and non-batch tools that set refs skip settling.

What changes

Condition or boundary	Before	After
Queued task times out	Remaining work can continue and replace refs	Abort checkpoints stop cooperative work; the commit fence drops later ref writes
Observation reaches its own timeout	Collection continues after the response	Abort collection; the final checkpoint prevents its late ref write
Observation not delivered (observe none)	Still became the diff baseline	Not recorded; the next diff is against what the agent last saw
list_testids	Reads the tree and replaces refs despite returning no refs	Collect only test IDs; leave refs unchanged
Mobile discovery and observations	Separate snapshots can disagree or show an unsettled screen	Share a snapshot when supported; bound post-mutation settle retries

Queue timeout errors use HTTP 500 with TOOL_EXECUTION_FAILED and Tool execution timed out after ${effectiveTimeout}ms. MM_REQUEST_ABORTED is the internal checkpoint response. Cancellation does not undo an action already dispatched to a driver. Refresh with describe-screen before retrying.

Decisions for reviewers

Decision	Reason
Keep abort checkpoints and the commit fence	A driver operation can finish after cancellation. Ref ownership must remain safe independently. Synchronous abort-handler settlement can win the queue's timeout race.
No ref stability; refs stay positional	See Why no ref stability.
The observation's refMap always wins, also after a tool set refs	The observation is the last tree in the response, so every ref in it resolves.
lastScreen updates only after a delivered response	A timed-out request's tree is never the settle baseline.

Review guide

Read the files in this order:

#	File	What to check
1	src/server/request-queue.ts, src/types/http.ts	Abort-before-reject ordering, bounded settle wait, timeout clamp, and optional context additions.
2	src/server/ref-store.ts	Generation and abort checks; toolSetRefs.
3	src/server/create-server.ts	Proxy receiver binding (setRefMap, clearRefMap), observation cancellation checkpoints, screenAfterResponse / lastScreen, batch behavior, separate diff baseline.
4	src/tools/utils.ts, src/tools/interaction.ts, src/tools/batch.ts, src/tools/discovery-tools.ts	Checkpoints and list_testids tree-read removal.
5	src/platform/types.ts, src/platform/mobile-platform-driver.ts, src/server/screen-settle.ts	Optional capture boundary, shared snapshot, busy detection, and bounded retries.
6	Tests beside these modules; README.md, SKILL.md	Timeout isolation, matching exceptions, snapshot consistency, and caller recovery guidance.

Testing

At 6944868: yarn test 52 files / 1528 tests pass; coverage 97.03 / 93.95 / 95.21 / 97.19; yarn lint passes. Every branch above it in the stack (#65, #66, #67, #44, #68) was restacked and passes yarn test + yarn lint at its own tip.

Breaking changes

None.

• ToolContext.signal, IPlatformDriver.captureScreen and ScreenCapture are optional additions.
• RequestQueue.enqueue now passes the signal to its callback. Existing callbacks that take no argument keep working.

Consumer impact

Consumer	Change needed	Done on branch
Extension test/e2e/playwright/llm-workflow/	None. It does not use RequestQueue, and its setRefMap/getRefMap implementation is unchanged.	n/a
Mobile tests/llm-workflow/	None (same).	n/a

Why no ref stability

The first version of this PR also renumbered refs so an element kept its number across observations (stabilizeRefs, a session high-water mark, baseline checks, merging tool and observation refs: about 500 lines of code and 900 of tests). It was removed because the problem it solves is rare:

• Refs are deterministic per tree: the same screen gives the same refs, and an unchanged screen diffs as empty (device proof below).
• Since P1-05/P1-06, read commands collect no observation, so they no longer renumber anything.
• The only case it fixed is an element inserted above the one the agent holds. The response then carries the new refs (the diff lists the shifted nodes, or the full tree when every node shifted), and SKILL.md tells agents to use the refs from the latest response.
• No baseline defect (B1–B15) was a wrong-element click caused by renumbering.

If real runs show agents clicking stale refs, it can come back as its own ticket.

Proof

After the shrink (6944868, run on the restacked stack tip):

Check	Extension	iOS	Android
Timed-out click, then describe-screen (queueMs 0), then click by ref	pass	pass	pass
describe-screen twice: identical refs	17/17	67/67	242/242
Unchanged screen, 3 taps: diff	4,243 B, +0 −0	4,860 B, +0 −0	5,306 B, +0 −0
Navigate away and back, click by the latest ref	pass	pass	pass

Android navigation observations can still show the previous screen (the new screen mounts about 2 s later). That is the existing Android settle limit noted in #44, not changed here.

Earlier revision (with ref stability), kept for history:

Tested with yalc builds in the Extension (e2e build, headless) and Mobile (dev client on iOS sim and Android emulator). The first full run used an earlier revision (e804b95). The final commit got a targeted mobile recheck plus a smoke run on all three platforms. Details are in the proof file in the mm-cli-v2 workspace.

Check	Before (#62)	After
click --testid does-not-exist --timeout 500, then describe-screen, then click by ref	–	works on ext, iOS and Android (queueMs 0, intended element clicked)
Android unlock-click observation	login screen ("Unlock, busy") in 3/3	no login or busy in 4/4 (3 wallet screen, 1 tab bar only)
Unchanged-screen diff, iOS (wait-for wallet-screen)	7495 B	7518 B, diff +0 −0
Unchanged-screen diff, Android (click tab-bar-item-Wallet, p50 of 3)	7672 B, observationMs 1924	40–110 KB, observationMs 965–1001 (one snapshot instead of two)
Ext bench p50 (click / get-state / list-testids totalMs)	580 / 5 / 565	582 / 6 / 60 (list_testids no longer reads the tree)

The Android unchanged-screen diff gap in that revision is closed by #67 and #44 (see the table above).

[cryptotavares]

Rebased onto #62 (1bbfc70). Fixed CodeQL js/resource-exhaustion in 5131d82: queued task timeouts (from request bodies) are clamped to MAX_QUEUE_TIMEOUT_MS = 600 s. Core: 49 files / 1561 tests, lint 0.

[cryptotavares]

CodeQL js/resource-exhaustion on request-queue.ts is the same finding that is already dismissed on main (alert #32, "used in tests"); it re-appears because the lines moved. The timeout is now clamped to MAX_QUEUE_TIMEOUT_MS (600 s) and falls back to the default for non-positive values (7d0c565), and the schemas already cap every timeoutMs. Needs a maintainer to dismiss the re-opened alert.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[seaona]

2 small comments 🙏

[seaona]

batchTimeoutMs allows up to 300 s, but the queue budget only comes from timeoutMs, which run_steps doesn't have. So batches get the 30 s default, and the CLI also uses 30 s. Before, a batch that hit it kept running in the background. With this check, it's now cut off at the next step. Should the queue and CLI budgets follow batchTimeoutMs, or should its max be lowered to about 30 s?

[cryptotavares]

Thanks. We went with the lower max: batchTimeoutMs is now 1000-30000 (#71, fix!: cap run_steps batchTimeoutMs at 30 s), like the cdp timeoutMs. The schema text and docs say the deadline is checked between steps, so to get partial results it should be below the 30 s request limit (e.g. 20000). Longer flows can be split into several batches.

[seaona]

This check runs at the start of the click, before the queue timer could have fired, so it possibly almost never triggers. Inside run_steps, batch.ts already checks between steps. The slow part, waiting for the target, never sees the signal, so a timed-out click can still happen later.

Example, not sure if could happen in reality, but may be in slow devices?

0 s   mm click --testid confirm-button   → checked here: not aborted, starts waiting for the target
30 s  queue timeout → agent gets "timed out"; signal aborted, but nothing checks it
35 s  queue settle wait ends → agent's next command (describe-screen) starts
38 s  target found → the click happens anyway, while the agent thinks it didn't

Suggestion: could driver.click and driver.type take an optional signal and call signal?.throwIfAborted() after the target wait, right before the tap or click? Both drivers already split wait and action, so it's a one-line check per driver, and it stops a timed-out click from landing later.

// src/platform/types.ts
click(
  targetType: TargetType,
  targetValue: string,
  refMap: Map<string, string>,
  timeoutMs: number,
  within?: WithinScope,
  signal?: AbortSignal,
): Promise<ClickActionResult>;

and then each driver checks it right before the action:

// playwright-driver.ts, after waitForTarget(...)
signal?.throwIfAborted();
await locator.click({ timeout: remaining });

// mobile-platform-driver.ts, after waitForElement(...)
await this.#backend.waitForElement(query, timeoutMs);
signal?.throwIfAborted();
await withTimeout(this.#backend.tapElement(query), remaining, 'tapElement');

[cryptotavares]

Agreed, fixed in #71 (fix: do not click or type after the request timed out) as you suggested: IPlatformDriver.click/type take an optional signal, and both drivers call signal?.throwIfAborted() after the target wait, right before the action (mobile type also between the focus tap and typeText). The tools pass context.signal and return MM_REQUEST_ABORTED. Verified on the extension: a run_steps click whose target appears 1 s after the 30 s queue timeout clicked before, and does not click now. Limit (documented): an action that already started is not stopped. device-mcp's waitForElement/tapElement/typeText take no signal, and Playwright's actionability checks are part of locator.click. Cancelling inside device-mcp needs a device-mcp change, kept as a follow-up.