Skip to content

Add native Decision commerce automation and shopping mode settings - #3230

Merged
Hillary Mutisya (hillary-mutisya) merged 2 commits into
mainfrom
dev/hillarym/commerceDecision
Oct 26, 2026
Merged

Hillary Mutisya (hillary-mutisya) merged 2 commits into
mainfrom
dev/hillarym/commerceDecision

Conversation

@hillary-mutisya

Copy link
Copy Markdown
Collaborator

Summary

Add an opt-in native Decision path for commerce UI automation, alongside the existing generative path. This extends the decision-model approach used for crosswords to two shopping operations: searching for a product and opening its product page.

User-visible behavior

  • Extension Options → Shopping Mode → Generative / Decision selects the implementation used by the normal product-search and product-page commands. Generative remains the default.
  • Shopping and Crossword modes are independent. Preferences support save/load, reset, and export; a saved shopping-mode change applies to the next command without restarting the agent.
  • Explicit decision search for ... and decision open product ... commands always use Decision.
  • Decision search selects the search input, submits its native form, and verifies the resulting search/no-results page. Decision product navigation selects the product link and verifies the destination product/variant.
  • Unavailable providers, ambiguity, insufficient evidence, and failed postconditions produce explicit failures: there is no automatic generative fallback or mutation retry.
  • Cart/checkout and other flow names are unchanged. Saved scripts are not overwritten, and mode routing preserves the saved command's site scope.

Implementation and review guide

  1. Grounded browser primitives: shared observation/action contracts and extension content-script/service-worker handling enumerate controls, bind opaque targets to browser-owned DOM nodes, and execute one-use actions. Selection/execution validate target identity; read-only page conditions classify captured evidence without requiring the dynamic DOM to remain unchanged during inference.
  2. Native decision orchestration: uiDecisionService.mts implements selectElement, evaluatePageCondition, and executeUiAction. Selection covers every eligible control in bounded batches (up to 32 controls / 24 KiB per request), runs those batches concurrently, waits for all to settle, and compares multiple winners globally.
  3. WebFlow integration: RPC, browser APIs, sandbox types, and sample scripts expose the new primitives. Shopping-mode routing selects the shipped Decision implementation for the two supported normal commands while retaining the existing saved implementation in Generative mode.
  4. Generative extraction and navigation: honor the caller's inline extraction schema while retaining the default discovery catalog when omitted. Page-load handling follows the current main document through navigation without waiting for unrelated iframe/ad resources; both variants accept the same optional navigation timeout.
  5. Options and documentation: add the Shopping selector beside the existing Crossword selector, validate authoritative stored settings, and document usage, scope, and limitations in the extension README.

The commerce samples explicitly use main-document evidence; they do not make claims about iframe content. The underlying API retains a strict all-frames default. This is a Chrome/Edge pilot for native search forms and product links, not general checkout or visual/canvas automation.

Validation

  • Integrated main through edf9e0e7e; the changes reapplied cleanly, with no unresolved conflicts. Updated dependencies using pnpm install --frozen-lockfile.
  • Full pnpm run build: passed.
  • 119 browser/native/crossword tests passed, including both shopping actions in both modes, next-command switching, explicit overrides, real native wire adapters with mocked HTTP, concurrent batches, and failure paths.
  • 91 extension tests passed, covering the actual options HTML, independent preferences, authoritative storage/RPC settings, DOM actions, and navigation.
  • 4 browser reasoning compatibility tests passed.
  • All PR gates passed against origin/main: repository-pinned formatting, lint, complexity, circular dependencies, and the debt/test-marker gate.

Live qualification performed during development

The latest paired Home Depot run completed all four flows and independent UI checks:

Scenario Generative handler time Decision handler time
Product search 12.973 s 6.604 s
Product navigation 7.880 s 4.966 s

The seven product-selection requests started within 16.7 ms; selection wall time was 1.467 s. Neither Decision flow made generative calls. These are observed single-run measurements on a dynamic live site, not benchmark guarantees. Recordings and detailed evidence remain outside Git; no videos, browser profiles, or local provider configuration are included in this PR.

Introduce grounded UI observation, native control and page-condition decisions, and one-use browser-bound execution. Run exhaustive control-selection batches concurrently and add verified product search/navigation WebFlows alongside the generative implementations.

Expose Shopping Mode beside the independent Crossword Mode, preserve generative defaults and saved flow scope, repair main-document navigation waits, and cover the new behavior with deterministic tests and usage documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@hillary-mutisya
Hillary Mutisya (hillary-mutisya) merged commit d21f371 into main Oct 26, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant