Skip to content

fix: exact element match first; add elements idb axbridge drops on iOS 18+ - #52

Open
cryptotavares wants to merge 3 commits into
mainfrom
cryptotavares/testid-exact-match-ios-fixes
Open

cryptotavares wants to merge 3 commits into
mainfrom
cryptotavares/testid-exact-match-ios-fixes

Conversation

@cryptotavares

@cryptotavares cryptotavares commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Two element-lookup fixes found while validating @metamask/client-mcp-core 0.13.0 on MetaMask Mobile (MetaMask/metamask-mobile#38594). Both made tap_element / wait_for miss or hit the wrong element on the MetaMask wallet home.

Problem Before After
Partial match wins over exact match (all platforms) findElement returned the first element, in tree order, whose field contained the query. On Android, { identifier: "account-picker" } matched the earlier account-picker-avatar (inside the profile button), so the tap opened the profile screen instead of the account list. An exact match anywhere in the tree wins. The first fuzzy match (case-insensitive, contains) is used only when there is no exact match.
axbridge drops elements (iOS 18+) Snapshots read only idb ui describe-all --api axbridge. On the wallet home it returns the balance and the Buy / Swap / Send / Receive row as one unlabelled view with no children. wallet-send-button was not found, and tap_element timed out. --api ax lists those buttons. When axbridge is the preferred backend, snapshot() also reads ax at the same time (best effort). Each ax element whose identifier is not in the axbridge tree is added as a leaf of the smallest element whose frame contains it. raw stays the axbridge output.
flowchart LR
  S["snapshot() on iOS 18+"] --> B["describe-all --api axbridge"]
  S --> A["describe-all --api ax<br/>(same time, best effort)"]
  B --> M["add ax elements whose<br/>identifier is missing"]
  A --> M
  M --> F["findElement:<br/>1. exact match<br/>2. else first fuzzy match"]
Loading

Decisions

Decision Why
Exact pass over the whole tree, then fuzzy Keeps every query that works today working (fuzzy fallback), and only changes the result when an exact match exists.
Exact means equal strings for every given field Case-insensitive equality would still pick Send vs send by tree order; equality is the clearest rule.
Read ax together with axbridge instead of switching to ax axbridge keeps the full tree (170 nodes vs 46 for ax on the wallet home) and the structure other code relies on. ax only fills gaps.
Only elements with an identifier are added They are what tap_element / wait_for target, and the identifier is how we detect "missing" without duplicates.
Supplement is best effort, no companion recovery The main axbridge read already handles recovery. A failed ax read leaves the snapshot as before.
No supplement when ax is preferred (iOS ≤ 17) or --api is unsupported Nothing to fill in.

Not in this PR: the iOS dismissKeyboard fix (idb ui key RETURN rejected) is already in #48.

Interaction with the open stack (#47 → #48 → #50 → #51)

Checked by merging this branch onto #51 (no git conflicts) and running the stack's proofs on iOS and Android:

  • One perf: one snapshot per tap; cache idb describe per connection #47 (P1-41) test needs a new count when the branches meet: reads idb describe once per connection for snapshots and taps expects 3 describe-all calls; with the ax supplement it is 6 (each snapshot = axbridge + ax, run at the same time). idb describe --json stays at 1. With that change: 752 passed.
  • fix: hit-test iOS taps; refuse off-screen Android taps #51 (P1-42) hit-test: the element at the center of an added button (for example main-action-button-content inside wallet-send-button) is inside the target frame, so click --testid wallet-send-button passes the hit-test and opens Send.
  • fix!: scrollToElement succeeds only when the element is in the viewport #50 (P1-36): scroll-to-element works on added elements (trending-token-row-item-…, 2 swipes, in viewport) and back to wallet-send-button.
  • Cost: iOS click p50 1579 → 1685 ms (+~100 ms, 15 runs each, back to back); describe-screen on the wallet home 11.4 → 17.9 kB (54 → 73 nodes), because the missing elements are now listed. Android: no change (only the match order differs).

Review guide

  1. src/utils/element.ts: findElement → findFirst(exact) then findFirst(fuzzy); matchesQuery gets an optional exact flag (default false, so existing callers are unchanged).
  2. src/backends/idb-backend.ts: snapshot() starts the ax read before the main read; #describeAll now takes the API order and returns which API succeeded; addMissingIdentifiedElements does the merge.
  3. Tests: src/utils/element.test.ts (exact over partial, fuzzy fallback, multi-field) and src/backends/idb-backend.test.ts (merge placement, no duplicates, ax failure, no supplement on iOS ≤ 17). One existing test now counts axbridge reads instead of matching the whole list, because the concurrent ax read is also a describe-all call.
Testing
Check 0.7.0 This PR
iOS 27 sim: wallet-send-button in snapshot ❌ missing ✅ frame (205, 242, 87×73)
iOS: mm click --testid wallet-send-button ❌ times out ✅ Send opens
iOS: mm click <ref> on Send ❌ no node ✅ Send opens
iOS: mm click --testid wallet-receive-button ❌ times out ✅ Receiving address opens
iOS: mm click --testid account-picker ✅ ✅
iOS: snapshot time on the wallet home (3 runs) 1323 / 988 / 1005 ms 1398 / 1051 / 1062 ms
Android API 36 emulator: mm click --testid account-picker ❌ opens "Create profile" ✅ Accounts list opens
Android: mm click <ref> on the account picker ❌ opens "Create profile" ✅ Accounts list opens
Android: mm click --testid profile-button ✅ ✅ still opens the profile
Android: mm click --testid wallet-send-button ✅ ✅

findElement returned the first element, in tree order, whose field
contained the query. On the MetaMask Android wallet home the query
{ identifier: 'account-picker' } matched the earlier
'account-picker-avatar' (inside the profile button), so taps opened the
profile screen instead of the account list.

findElement now returns an exact match anywhere in the tree first, and
falls back to the first fuzzy (case-insensitive, contains) match only
when there is none. matchesQuery takes an optional exact flag.
On iOS 18+ snapshots read idb --api axbridge. It can report a React
Native container as one unlabelled view without children: on the
MetaMask wallet home, the balance and the Buy/Swap/Send/Receive buttons,
so wallet-send-button could not be found or tapped. --api ax lists them.

When axbridge is the preferred backend, snapshot() now also reads ax,
concurrently and best effort, and adds each ax element whose identifier
is missing from the axbridge tree, as a leaf of the smallest element
whose frame contains it. raw stays the axbridge output.
cryptotavares added a commit that referenced this pull request Nov 2, 2026
Pins `.nvmrc` to Node.js `v24.22.1` instead of `lts/*`.

## Why

`lts/*` now resolves to Node 26 (26.13.0 became LTS on 2026-10-28). The
**Check workflows** job does not set a Node version, so it follows
`.nvmrc`, and on Node 26 its `yarn --immutable` step fails (3 retries),
which also fails **All jobs pass**:

```
node_modules/yargs/yargs:3
const {applyExtends, cjsPlatformShim, Parser, Yargs, processArgv} = require('./build/index.cjs')
ReferenceError: require is not defined in ES module scope, you can use import instead
Node.js v26.13.0
```

The allow-scripts install plugin runs `@lavamoat/allow-scripts` (3.4.3),
which does `require('yargs/yargs')`. In yargs 17.7.2 (locked) that is an
extension-less file in a `"type": "module"` package, which Node 26 no
longer loads with `require`.

| Run | Node | Check workflows |
| --- | --- | --- |
| #47, #48, #51 (Oct 29-30) | 24.22.1 | pass |
| #52 (Nov 2), and its re-run | 26.13.0 | fail |

Reproduced locally: `node
node_modules/@lavamoat/allow-scripts/src/cli.js --help` works on 24.22.1
and throws the same error on 26.13.0.

The build, lint and test jobs set their own matrix (20.x / 22.x / 24.x)
and are not affected. `engines` stays `^20 || ^22 || >=24`.

## Follow-up (not in this PR)

Node 26 support for the install tooling: `yargs` 17.7.3 points
`yargs/yargs` to a `.cjs` file and loads on Node 26 (checked locally),
so `yarn up -R yargs` (or `@lavamoat/allow-scripts` 5.x) would let
`.nvmrc` move to Node 26 later.
@cryptotavares
cryptotavares marked this pull request as ready for review November 2, 2026 17:57
@cryptotavares
cryptotavares requested a review from a team as a code owner November 2, 2026 17:57

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant