Skip to content

test(cloudxr): real-browser run_oob_connect() tests against MockCloudXR, with anticipatory reliability-branch coverage - #1148

Draft
gareth-morgan-nv wants to merge 6 commits into
gmorgan/MockADBfrom
gmorgan/oob-connect-tests
Draft

gareth-morgan-nv wants to merge 6 commits into
gmorgan/MockADBfrom
gmorgan/oob-connect-tests

Conversation

@gareth-morgan-nv

@gareth-morgan-nv gareth-morgan-nv commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Real-browser integration tests for run_oob_connect(), standardized on MockCloudXR (npm run build:app-mock) rather than the real @nvidia/cloudxr SDK: the real SDK genuinely tries to stream against whatever's on backend_port, which is unpredictable and gives a test nothing to assert against. MockCloudXR mounts the identical App.tsx/CloudXRComponent.tsx UI with a fully deterministic, externally controllable session (window.__mockCloudXRFail(message?, code?)), and never opens a socket of its own.
  • Always a static production build, never a dev-server (static_webxr_build()). Two real bugs made this necessary: webpack HMR duplicating the cloudxr-mock-alias.ts module instance (so the crash-trigger silently bound to an empty session), and <React.StrictMode> double-invoking CloudXR.createSession() in a dev-mode build. webpack.app-mock.js now builds in production mode. Also found and fixed: a stale webpack persistent filesystem cache silently serving an old bundle — cleared before every build now.
  • Three real-browser tests: the original success path, a new test proving the crash-trigger mechanism itself (real, deterministic, DOM-visible error banner on demand), and an anticipatory test written against gmorgan/oob-error-relaunch (cloudxr: auto-relaunch the teleop tab on a terminal client error (depends on #1138, #1145) #1146) before it's merged here — asserts on black-box observable recovery (a second real am start, a second real tab), so it fails cleanly today and should turn green once cloudxr: auto-relaunch the teleop tab on a terminal client error (depends on #1138, #1145) #1146 lands.
  • Also added test_build_teleop_url_forwards_reliability_config_from_env, the same "write it now, let it fail until merged" approach applied to gmorgan/oob-reliability-config-wiring (cloudxr: wire client reconnect/warm-up-timeout config through OOB launch #1145)'s config forwarding — no browser needed for that one.
  • Design/decisions captured in oob-real-browser-integration-test-plan.md (repo root, §8 has the final-state summary).
  • Still depends on gmorgan/MockADB (test(cloudxr): reusable fake-adb test double (FakeAdb/mock_adb) #1147) for FakeAdb/mock_adb() — base branch unchanged.
  • Manual-run only for now; not yet wired into CI.

Test plan

  • uv run pytest test_oob_teleop_adb.py test_oob_teleop_env.py test_oob_teleop_hub.py test_fake_adb.py test_wss_static_client.py -q — 141 passed, 2 expected anticipatory failures (config-wiring, auto-relaunch — both intentionally fail until their respective branches merge)
  • SKIP=check-copyright-year pre-commit run on all changed files — clean
  • npx eslint/npx prettier --check on changed TS files — clean

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Comment @coderabbitai help to get the list of available commands.

@gareth-morgan-nv
gareth-morgan-nv marked this pull request as draft September 29, 2026 14:43
@gareth-morgan-nv gareth-morgan-nv changed the title test(cloudxr): end-to-end tests for run_oob_connect() using mock_adb test(cloudxr): real-browser integration test for run_oob_connect() Sep 29, 2026
@gareth-morgan-nv gareth-morgan-nv changed the title test(cloudxr): real-browser integration test for run_oob_connect() test(cloudxr): real-browser run_oob_connect() tests against MockCloudXR, with anticipatory reliability-branch coverage Sep 29, 2026
@gareth-morgan-nv
gareth-morgan-nv force-pushed the gmorgan/oob-connect-tests branch from 861f8eb to 55922d6 Compare September 30, 2026 15:18
gareth-morgan-nv and others added 6 commits September 30, 2026 12:18
…p_tabs

Adds coverage for the "stale-tab" state from the web-client UI-state
reliability backlog on the host-orchestration side (oob_teleop_adb.py),
following up on the client-side coverage in #1136/#1137.

_discover_devtools_socket's candidate-matching and priority logic is
exercised via mocked _run_adb output. _close_stale_teleop_tabs is driven
against a real local HTTP server standing in for Chromium's CDP /json
endpoint (only the adb forward/remove calls either side are mocked, since
there's no real device here), so the oobEnable= tab-matching and
close-request logic itself is exercised for real rather than asserted via
mocked call arguments.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
…diness poll

Extends the fake-CDP pattern from the stale-tab tests to a WebSocket
responder (_CdpScript/_fake_cdp_ws), since _cdp_session_click_connect
speaks CDP entirely over one WebSocket rather than the HTTP /json endpoint
_close_stale_teleop_tabs uses. Covers the remaining two named states from
the web-client UI-state reliability backlog that are host-side
(oob_teleop_adb.py), completing the follow-up from #1136/#1137:

- certificate: both bypass layers - primary (Page.navigate re-navigate)
  and DOM fallback (details-button -> proceed-link click-through, forced
  by a chrome-error: interstitial URL, the same branch condition the real
  function checks).
- browser-launched-but-client-not-loaded (host-side half; the client-side
  half is in #1137): the #startButton readiness-poll state machine's
  ready path (multi-step initializing -> ready) and failed path (raises
  OobAdbError before ever dispatching a click).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
CodeRabbit review on #1138: server.shutdown() stops serve_forever() but
doesn't close the listening socket - only server_close() does. The port
is a hardcoded constant (_CDP_LOCAL_PORT, since _close_stale_teleop_tabs
hardcodes it internally) reused across multiple tests in this file, so
relying on garbage collection to eventually release it risks a delayed-
release port conflict between tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
Cover the real run_oob_connect() against a fake adb (mock_adb) and a fake
CDP tab-list server: successful launch through to monitor task creation,
plus the no-devtools-socket, am-start-failure, and no-matching-tab error
paths.

Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
…owser integration test

Drive the real run_oob_connect() against a real Chromium running the real
webxr client (real IWER-emulated WebXR, real CDP), a real wss.py proxy
(real OOBControlHub, no real CloudXR runtime behind it), and a
RealBrowserAdb whose am start/adb forward have real side effects instead
of canned output. Only adb itself stays faked, since no physical Android
device is available.

A minimal_mock_runtime() stand-in on backend_port is needed: without
something listening there, the real client's signaling handshake gets torn
down by wss.py's proxy_handler() right after connecting, which surfaces as
a real error banner and makes run_oob_connect() raise instead of returning
a monitor task. The stand-in speaks none of the real CloudXR/WebRTC
protocol - it only avoids that specific failure mode.

Design and inventory captured in oob-real-browser-integration-test-plan.md.
Manual-run only for now; not yet wired into CI.

Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
… builds, add crash/relaunch coverage

Switch every real-browser test from the real @nvidia/cloudxr SDK to
MockCloudXR (npm run build:app-mock): the real SDK genuinely tries to
stream against whatever's on backend_port, which is unpredictable and
gives a test nothing to assert against. MockCloudXR mounts the identical
App.tsx/CloudXRComponent.tsx UI with a fully deterministic, externally
controllable session, and never opens a socket of its own, so
minimal_mock_runtime() is gone entirely.

Switch from a dev-server to a static production build
(static_webxr_build()). Two real bugs made this necessary: webpack HMR
re-executes a module's top-level code on live-reload, creating a second,
disconnected instance of cloudxr-mock-alias.ts's activeSession that
window.__mockCloudXRFail() silently bound to instead of the real running
session; and <React.StrictMode> (present regardless of HMR) double-invokes
effects in a dev-mode build, calling CloudXR.createSession() twice for the
same reason. webpack.app-mock.js now builds in production mode. Along the
way, a stale webpack persistent filesystem cache (keyed only on the config
file, not source files) was found silently serving an old compiled bundle;
static_webxr_build() now clears it before every build.

window.__mockCloudXRFail() gains an optional `code` param: a code-less
failure is "recoverable" per isRecoverable(), and CloudXRComponent.tsx
auto-reconnects on those without ever showing the error banner, so
forcing a real terminal error needs a code in the non-retryable range.

Add two new real-browser tests: one proving the crash-trigger mechanism
itself (a real, deterministic, DOM-visible error banner on demand), and
one anticipatory test written against gmorgan/oob-error-relaunch (#1146)
before that branch has merged here - it asserts on black-box observable
recovery (a second real `am start`, a second real tab) rather than
importing any private function, so it fails cleanly on that assertion
today and should turn green once #1146 lands. Also add
test_build_teleop_url_forwards_reliability_config_from_env, the same
"write it now, let it fail until merged" idea applied to
gmorgan/oob-reliability-config-wiring (#1145)'s plain env-var/string
forwarding - no browser needed for that one.

Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
@gareth-morgan-nv
gareth-morgan-nv force-pushed the gmorgan/oob-connect-tests branch from 55922d6 to 3e68d67 Compare September 30, 2026 16:18

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants