Skip to content

Run contract: bad --gl is usage, true last-page reason, real screenshot count, --out paths - #9

Merged
jehrr merged 1 commit into
mainfrom
fix/run-contract
Oct 22, 2026
Merged

jehrr merged 1 commit into
mainfrom
fix/run-contract

Conversation

@jehrr

@jehrr jehrr commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator

Problem

Five defects from the manual test of v0.2.0, each reproduced before fixing:

report # symptom cause
8 --gl USA → exit 3 after fresh-browser retries and proxy advice Play answers a country it does not serve with its own HTTP 400 page (1.7 KB, no store images), which was classified as blocked
6 screenshot_count = 1 on every app row the app page wraps screenshots one level deeper than a grid tile; the wrapper was counted
7 stop_reason: completed when the last page added nothing a post-loop line set completed whenever the page budget was met, overwriting no_new_products (also present in the reviews loop)
12 --out x.json → x.json.json, x.json.csv --out is a prefix and was used literally
13 --dump-html into a missing directory → navigation_failed, exit 5 the write raised mid-run, after the page had been fetched

Behaviour after

  • New page state bad_request (HTTP 400, or Play's 400 heading + no store images where Selenium has no status). The run stops with exit 2 naming --gl, no retries, no proxy advice. --gl is shape-checked (two letters) before any browser starts; no country list is kept, since ZZ is well-formed and Play does serve it. The Scraper API path returns 2 for the same page.
  • screenshot_count unwraps single-element wrappers: 9 for WhatsApp, 8 for Minecraft; 0 of 223 grid tiles in the captures change.
  • The post-loop override is removed from both loops. Every other stop_reason assignment sits before a break, so the line could only overwrite. Status and exit code are unchanged — only the reason was wrong.
  • A trailing .json / .csv / .meta.json is dropped from --out.
  • --out and --dump-html create missing directories (~ expanded once, so the directory created and the file written match); a path that cannot be created is exit 2 before a browser starts.

Also: the committable-captures check now uses git check-ignore --no-index. Without it, a tracked file is never reported, so the check only held until the fixture was committed. A control went silent once the control harness began committing its disposable copies, which is how this was found.

Evidence

  • Live, Playwright / Selenium / pyppeteer: --gl xx → exit 2, 0 block retries, 0 hints; --gl USA → exit 2 before launch; --category TOOLS --pages 3 --out new/deeper/tools.json --dump-html dumps/x/p → tools.json, tools.csv, tools.meta.json + 3 dumps, complete / no_new_products; --mode app --app-id com.whatsapp → screenshot_count 9.
  • Fake-driver runs through engine_core.main for each behaviour. Play's 400 page is committed verbatim as a fixture and is also a second control for the asset-count signal.
  • Controls: 8 of 8, each planted in a disposable copy against a green baseline and caught by its named check. PR A's and PR B's controls were re-run on the same harness with a baseline: 6/6 and 5/5.
  • Offline suite green with core requirements only and in each engine venv; credential scan clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VrpkXSoXyZMGV5nqF9JV6P

…ot count, --out paths

Five defects from a manual test of v0.2.0, each reproduced first.

- A `--gl` Play does not serve was exit 3. Play answers `gl=USA`, `xx` or
  `12` with its own HTTP 400 page (1.7 KB, no store images), which was read
  as a block: fresh-browser retries, exit 3, and advice to buy a proxy. It
  is now a `bad_request` page state — from the status, or from the page's
  own heading where Selenium has no status — and the run stops with exit 2
  naming the flag, without retrying. `--gl` is also checked for shape (two
  letters, upper-cased) before any browser starts; `ZZ` is well-formed and
  Play does serve it, so no list of countries is kept here. The Scraper API
  path returns 2 for the same page instead of falling through to "0
  products".
- `screenshot_count` was 1 on every app row: the app page wraps its
  screenshots one level deeper than a grid tile, and the wrapper was
  counted. 9 for WhatsApp and 8 for Minecraft in the captures; 0 of 223 grid
  tiles change.
- `stop_reason` was overwritten with `completed` whenever the page budget
  was met, including when the page that met it added nothing — the true
  reason is `no_new_products`. Every other assignment in both loops sits
  before a `break`, so the line could only overwrite; it is removed from the
  grid and the reviews loops. Status and exit code are unchanged.
- `--out x.json` wrote `x.json.json`; a trailing `.json`, `.csv` or
  `.meta.json` is now dropped from the prefix.
- `--out` and `--dump-html` into a missing directory: the dump failed
  mid-run and was reported as `navigation_failed`, exit 5, about a page that
  had been fetched. Missing directories are now created, `~` is expanded
  once so the directory made and the file written agree, and a path that
  cannot be created is exit 2 before a browser starts.

The committable-captures check now uses `git check-ignore --no-index`.
Without it a tracked file is never reported, so the check only held until
the fixture was committed — found by a control that went silent once the
control harness started committing its disposable copies.

Verified:
- live, Playwright, Selenium and pyppeteer: `--gl xx` exit 2 with no block
  retries; `--gl USA` exit 2 before launch; `--category TOOLS --pages 3
  --out new/deeper/tools.json --dump-html dumps/x/p` writes tools.json /
  .csv / .meta.json and three dumps, `complete`, `no_new_products`;
  `--mode app --app-id com.whatsapp` gives screenshot_count 9.
- fake-driver runs through `engine_core.main` for each behaviour; Play's
  400 page captured verbatim as a fixture and used as a second control for
  the asset-count signal.
- eight controls, each planted in a disposable copy against a green
  baseline and caught by its named check. The earlier PR's controls were
  re-run on the same harness with a baseline as well.
- offline suite green with core requirements and in each engine venv;
  credential scan clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VrpkXSoXyZMGV5nqF9JV6P
@jehrr
jehrr merged commit 8fdcb35 into main Oct 22, 2026
7 checks passed
@jehrr
jehrr deleted the fix/run-contract branch October 22, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant