Skip to content

Say what the pre-pending marker test actually proves #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
g4lb merged 1 commit into from
Sep 15, 2026
Merged

Say what the pre-pending marker test actually proves #13

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 15 additions & 7 deletions test/cli/eval.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,10 +270,16 @@ describe('eval', () => {
// Every abort test above interrupts a run already under way. The sticky
// marker's whole point is the opposite case: `stop --force` left the
// marker behind after an earlier eval aborted, and it is STILL there,
// unconsumed, when the next eval starts — spec says that must be caught
// within one poll interval (~500ms), not after a full correctness gate.
// A repo whose test gate takes 8s if ever reached turns "checked late"
// into an unmissably slow test rather than a subtle wrong verdict.
// unconsumed, when the next eval starts.
//
// This covers that end to end — exit 2, ABORTED, no row, claim released —
// and NOT the entry checkpoint in evalOnce that makes it prompt. It cannot:
// without that checkpoint the poll still fires at ~500ms, and the abort
// listener Runner registers kills the gate subprocess mid-flight, so the
// same verdict arrives at roughly the same speed by an older route. The
// checkpoint itself is covered by the unit test in pipeline.test.js, which
// calls evalOnce with an already-aborted signal and genuinely fails
// without it. Keep both: this one guards the contract, that one the fix.
const dir = await makeBenchRepo()
await runCli(['init', '-C', dir])
await writeFiles(dir, {
Expand Down Expand Up @@ -305,9 +311,11 @@ describe('eval', () => {
const code = await new Promise((resolve) => child.on('close', resolve))
const elapsed = Date.now() - start

// Generous ceiling, not a tight bound: well under the 8s the test gate
// would cost if the marker were only noticed after claiming and gating,
// with headroom for windows-latest, the slowest runner in the matrix.
// Generous ceiling, not a tight bound. The 8s gate above is what makes it
// mean anything: an eval that sat through it would blow this, whichever
// route noticed the marker. Headroom is for windows-latest, the slowest
// runner in the matrix — raise the ceiling if it ever flakes there rather
// than trimming the gate, which is the part doing the work.
expect(elapsed).toBeLessThan(6000)
expect(code).toBe(2)
const payload = JSON.parse(out)
Expand Down