Skip to content

test(doctor): assert the hanging-backend property instead of a stopwatch - #82

Merged
AetherAI3 merged 2 commits into
mainfrom
fix/doctor-hanging-backend-flake
Aug 19, 2026
Merged

test(doctor): assert the hanging-backend property instead of a stopwatch#82
AetherAI3 merged 2 commits into
mainfrom
fix/doctor-hanging-backend-flake

Conversation

@AetherAI3

Copy link
Copy Markdown
Owner

Problem

test/diagnostics.test.ts"a hanging backend cannot stall the fast report" fails intermittently on loaded runners.

It injects clients that never resolve, sets timeoutMs: 50, and then asserts:

assert.ok(Date.now() - started < 500);

That measures the machine, not the behaviour. The 50ms timeout can fire exactly as designed and the assertion still fails because the runner was busy.

Observed:

where elapsed outcome
windows-latest CI 689ms ✖ fail
local, full-suite load 1818ms ✖ fail
local, in isolation 82ms ✔ pass

This is on main, so it can redden any branch that runs at the wrong moment. It has already done so once.

Contract

Assert the property the test exists to defend, not the clock: a probe fed by a hanging client must never come back claiming it verified anything.

agent.transport, auth.credential, agent.catalog and mcp.broker are named explicitly rather than filtered on an axis. A first attempt filtered on reachable !== "na" and failed on workspace.git — a local check that legitimately verifies in this fixture, because nothing about it touches the backend that is hanging.

The wall clock stays, but as a hang detector rather than a stopwatch. The failure this test guards against is an unbounded await that never returns at all; 30s separates that from a slow runner, and any finite margin would do. A busy machine is no longer a bug.

This is stronger than what it replaces, not weaker

Relaxing a timing bound looks like weakening a test, so here is the evidence it is not.

Mutation: make notChecked() return a verified axis — the exact false-green this suite exists to prevent.

result
mutated 5 pass / 3 fail — this test among them
restored 8 / 8

The old assertion passed that mutation untouched. It never looked at a single axis. So the previous test would let a false green through while failing on a busy runner — precisely backwards.

Tests

command result
npm run typecheck exit 0
npm test 922 pass / 0 fail

922 is the full suite on main at 41a7e261; this PR changes no production code and adds no test, so the count is unchanged by design.

Scope

One file, one test. No production code. git diff --stat: test/diagnostics.test.ts | 27 +++++++++-.

Why this is separate

It surfaced while working an unrelated lane, where it turned windows-latest red on a PR that had not touched anything near it. Fixing it there would have buried an independent main-line defect inside an unrelated change.

Recommend landing this first, ahead of the supercluster lanes (#72, #73, #74, #75, #77, #78) — every one of them carries this flake until it does.

The same fix is currently also committed on supercluster/a5-usage-continuity because that is the branch it was blocking; that copy should be dropped once this lands.

"a hanging backend cannot stall the fast report" injects clients that never
resolve, sets timeoutMs to 50, and then asserted:

    assert.ok(Date.now() - started < 500);

That measures the machine, not the behaviour. The 50ms timeout can fire exactly
as designed and the assertion still fails because the runner was busy. Observed
failing on windows-latest at 689ms, and locally at 1818ms under full-suite load
while passing in isolation at 82ms.

Replaced with the property the test exists to defend: a probe fed by a hanging
client must never come back claiming it verified anything. agent.transport,
auth.credential, agent.catalog and mcp.broker are named explicitly rather than
filtered on an axis, because local checks like workspace.git legitimately do
verify in this fixture — nothing about them touches the backend that is hanging.

The wall clock is still bounded, but as a hang detector rather than a stopwatch:
30s distinguishes "returned" from "awaited forever", which is the failure the
test was written to catch. A slow runner no longer registers as a bug.

Stronger than what it replaces, not weaker. Mutation-checked: making notChecked
return a verified axis fails this test along with two others (5 pass / 3 fail);
restoring gives 8 / 8. The old assertion passed that mutation untouched — it
never looked at a single axis, so a false green would have gone through while a
busy runner did not.

Gates at this commit:
  npm run typecheck   exit 0
  npm test            922 pass / 0 fail
@AetherAI3
AetherAI3 marked this pull request as ready for review August 19, 2026 12:37
@AetherAI3
AetherAI3 merged commit 4bb2322 into main Aug 19, 2026
5 checks passed
@AetherAI3
AetherAI3 deleted the fix/doctor-hanging-backend-flake branch August 19, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant