Skip to content

test: main-suite hygiene — env-dependent failures skip cleanly, broken ones fixed (#1532) - #1533

Merged
aarontrowbridge merged 4 commits into
mainfrom
amico/1532-hygiene
Sep 25, 2026
Merged

aarontrowbridge merged 4 commits into
mainfrom
amico/1532-hygiene

Conversation

@aarontrowbridge

Copy link
Copy Markdown
Member

Closes #1532 — the 10 amico-run + 16 extension pre-existing suite failures on main @ 244e6a9, each diagnosed and made honest. No test deleted, no assertion weakened, no test marked broken.

Classification

# Test (suite) Verdict Diagnosis
1 cloud_verb › status › unreachable endpoint degrades to ONE clear line (amico-run) fixed (test) The assertion pinned node's fetch failed string; bun's fetch says Unable to connect. Is the computer able to access the url?. Assert the union of both runners' wording — the intent (ONE clear named connectivity failure) is unchanged.
2 cloud_verb › abort › preflights the channel (amico-run) fixed (test) Same runner-dependent wording as #1.
3–5 sota_papers › curlSotaFetch ×3 (amico-run) fixed (product code) execFileSync("curl", …) relied on default env inheritance; bun resolves unqualified executables from the spawn's env OPTION when given, else the process-START env snapshot — never live process.env mutations — so the tests' fake-curl PATH injection was invisible and the REAL arXiv fetch ran. Fix: pass env: process.env explicitly (byte-identical inheritance under node; live PATH honored under bun).
6 sota_papers › lens 404 round via the production transport (amico-run) fixed (product code) Same root cause as #3–5, via the default transport.
7 sota_codebase › loadRegistry bootstraps from the packaged seed (amico-run) fixed (product code) The #820 data-as-import contract is TEXT (esbuild .toml: "text", vitest tomlAsText plugin, src/sota_seed.d.ts); bun natively parses .toml imports into an object, so seedToml.endsWith crashed. Normalized back to text at the import boundary (smol-toml stringify; the validator re-parses it either way).
8 doctor › parseDoctorArgs usage error (amico-run) fixed (test) A regex nested inside toMatchObject substring-matches under vitest but is compared literally under bun. Message now asserted with toMatch — same strength, both runners.
9 spec_review › the no-real-model-calls guard (amico-run) skip (env) Requires the vitest setup file test/setup.ts (it pins $AMICO_CRITIC_BIN to an impossible path for the suite); bun test does not load vitest setup files. it.skipIf(!process.env.AMICO_CRITIC_BIN) — run pnpm test to exercise. The fail-closed behavior itself stays covered by the sibling opt-in case.
10 gh_cli › gh shim recursion guard (amico-run) fixed (test) + env skip The constructed PATH assumed dirname(process.execPath) holds node — true under the node runner, false under bun (bun's bin dir has no node; the bundle could not even spawn, exit −1). The node dir is now RESOLVED (realpath'd, so shim dirs that carry a real gh cannot defeat the guard). A host with no node binary at all skips with the requirement named.
11–22 amicode_service_{bootstrap_auth ×7, contract, engine_proxy, fleet_data_plane, wiring ×2} (extension) fixed (tests) The dev hosts run a LIVE amicode service whose ambient env exports AMICODE_SERVICE_AUTH=open (the #955 tunnel/LAN posture). Tests boot the service relying on the CREDENTIAL default and never isolated the env → open mode → every anonymous-401 pin returned 200. Save/clear/restore AMICODE_SERVICE_AUTH around those boots (the auth_mode file's env-carrier cases already used the idiom; the boot-log case was missing it — fixed to the same idiom).
23 amicode_service_auth_mode › authMode exposed for the wiring's log line (extension) fixed (test) Same leak; this case had forgotten the save/delete/restore its siblings use.
24 amicode_service_runner › a shelf without a built app dist fails with the named reason (extension) fixed (test) Pointed at the vendored engine binary, absent on hosts without the vendor fetch → the runner failed EARLIER with no engine binary — the wrong named reason. Now passes an existing stub bin so the DIST check is what fires; the fail-loud describe is truly "always on" as its header claims.
25 cli_gate › the freshly staged CLI passes every check (extension) skip guard tightened The guard checked bare bin/ existence; a PARTIAL staging (leftover dist/, no launchers) passed the guard and red-failed the gate. Now requires every declared bin's launcher + dist present — skip names the requirement (pnpm -r run build); with a real staging the test runs and passes.
26–27 terminal › OPENCODE_DB/OPENCODE_CONFIG_DIR injection ×2 (extension) fixed (tests) The terminal env is seeded from process.env; the dev host exports OPENCODE_DB, leaking into the no-injection pins. The describe now saves/clears/restores both vars so the settings, not the host env, are what's tested.
28 open_threads › returns null when bun:sqlite is unavailable (extension; bun test surface) skip (env) Requires the Node runtime; under bun bun:sqlite exists and the call would read the developer's REAL session DB. describe.skipIf(typeof Bun !== "undefined") — run pnpm test to exercise.

Verification (from package dirs only, fresh state)

packages/amico-run — bun test (the campaign's baseline surface, which reproduced the 10):

 1707 pass
 10 skip
 0 fail
Ran 1717 tests across 102 files.

(baseline on main @ 244e6a9: 1698 pass / 9 skip / 10 fail — the identical 10, byte-for-byte the #1528 record)

packages/amico-run — pnpm vitest run --passWithNoTests --exclude '**/slow/**':

Tests  1708 passed | 2 skipped (1710)

packages/extension — pnpm vitest run --passWithNoTests:

Tests  3680 passed | 61 skipped (3741)

(baseline: 16 failed | 3663 passed | 62 skipped (3741) — the identical 16, byte-for-byte the #1526 record; +1 pass/−1 skip is the cli_gate real-set test running for real after staging)

bun test test/open_threads.test.ts in packages/extension: 47 pass / 1 skip / 0 fail (the Node-runtime degradation case skips under bun with its requirement named; 48/48 under vitest).

bun typecheck in both packages: 0 errors.

Every skip carries its stated requirement in-code (endpoint/credential/binary/runtime + how to run it); skip guards are deterministic per environment.

… under every runtime

bun resolves unqualified executables from the spawn's env OPTION when
given, else the process-START env snapshot — never later process.env
mutations. The default inheritance therefore froze the PATH lookup at
startup under bun, breaking the fake-transport injection the B1 hermetic
tests rely on (they mutate PATH, not pass env). Passing env: process.env
explicitly is byte-identical inheritance under node and makes the live
PATH (and the tests' injection) effective under both runners.
…un's native toml parsing

The #820 data-as-import contract (src/sota_seed.d.ts, esbuild's .toml:
"text" loader, vitest's toml-as-text plugin) types resources/*.toml as
the file's text content. Bun natively PARSES .toml imports into an
object, so the seed reached validateWatchedRepoRegistry as an object and
loadRegistry's bootstrap crashed on seedToml.endsWith. Normalize back to
text at the import boundary (smol-toml stringify; the validator re-parses
it either way) so the seed is one canonical string under every runtime.
…ails under bun and vitest

- doctor.test: the usage-error message rides toMatch instead of a regex
  nested in toMatchObject (vitest substring-matches nested regexes, bun
  compares them literally); same assertion strength, both runners.
- gh_cli.test: the recursion guard RESOLVES the node binary's realpath'd
  dir instead of assuming dirname(process.execPath) holds node (under bun
  it is bun's bin dir — the bundle could not even spawn). A host without
  any node binary skips with the requirement named.
- spec_review.test: the A-11 suite-guard assertion requires the vitest
  setup file (test/setup.ts pins $AMICO_CRITIC_BIN); under runners that
  do not load it, skip with the requirement named — run pnpm test.
- cloud_verb.test: the unreachable-endpoint wording is the runtime's own
  (node: "fetch failed"; bun: "Unable to connect...") — assert the
  union, never one runner's string.
…ing/runtime guards — 0 fails

The dev hosts that hit these failures run a LIVE amicode service, whose
ambient env (AMICODE_SERVICE_AUTH=open, OPENCODE_DB, partial bin/
staging) leaked into tests that never isolated it:

- amicode_service_{wiring,bootstrap_auth,contract,engine_proxy,
  fleet_data_plane,auth_mode}: save/clear/restore AMICODE_SERVICE_AUTH
  around the boots that pin the CREDENTIAL default (the 401 discipline
  assertions). The auth_mode env-carrier cases already did this; the
  boot-log case was missing it — fixed to the same idiom.
- terminal.test: clear ambient OPENCODE_DB/OPENCODE_CONFIG_DIR around
  the injection describe so the no-injection assertions test the
  settings, not the host env.
- amicode_service_runner.test: the dist-reason fail-loud test pointed at
  the vendored engine binary, which is absent on most hosts — the runner
  then failed EARLIER with 'no engine binary', the wrong named reason.
  Point it at an existing stub bin so the DIST check is what fires; the
  test is truly always-on now, as its describe claims.
- cli_gate.test: the staged-bins guard now requires a COMPLETE staging
  (every declared bin's launcher + dist present), not a bare bin/ dir —
  a partial staging (leftover dist, no launchers) skips with the
  requirement named (pnpm -r run build) instead of red-failing.
- open_threads.test: the no-bun:sqlite degradation case requires the
  Node runtime; under bun the module exists and the call would read the
  developer's REAL session DB — skip with the requirement named (run
  pnpm test).
@coderabbitai

coderabbitai Bot commented Sep 24, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: ebbbbdd3-436f-4d0c-8854-bab5a20c15b0


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@aarontrowbridge
aarontrowbridge merged commit 442ff56 into main Sep 25, 2026
12 checks passed
@aarontrowbridge
aarontrowbridge deleted the amico/1532-hygiene branch September 25, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Main-suite hygiene: the 10 amico-run + 16 extension pre-existing failures skip cleanly or get fixed — no baseline-diff tax per campaign

1 participant