Skip to content

FIX: reduce CI test runtime and stabilise benchmark timeout coverage - #76

Merged
merefield merged 1 commit into
mainfrom
fix/ci-test-runtime
Oct 8, 2026
Merged

merefield merged 1 commit into
mainfrom
fix/ci-test-runtime

Conversation

@merefield

Copy link
Copy Markdown
Owner

Why

The default-branch macOS jobs are exhausting Go's default ten-minute package timeout while still doing useful rendering work. The UI package repeats its complete localised layout/click-surface suite sequentially for 17 locales under race and coverage instrumentation, leaving little headroom for runner variation. A separate benchmark recovery test can also expire its 20 ms wall-clock deadline during setup instead of the turn it intends to test.

This PR improves test scheduling and diagnostics without changing application behaviour or dropping locale checks, assertions, race detection or coverage instrumentation.

Changes

Bound and separate the rendering workload

  • Run at most two isolated locale test subprocesses concurrently. All 17 locale options still run the complete existing helper suite, including responsive layout, hover, click, approval and history checks.
  • Split core tests and the localisation matrix into separate invocations in both CI and release workflows. Each retains -race -cover and has an explicit ten-minute package timeout.
  • Bound each locale child with a three-minute test timeout (including stack diagnostics), an outer four-minute process deadline and a five-second pipe-drain limit. Worker deadlines begin only after acquiring a slot.
  • Set an explicit 25-minute Go job budget; cancel superseded ordinary CI runs without cancelling release workflows.

Reduce redundant work and remove flaky clock assumptions

  • Reuse the unchanged scheduler whole-screen render baseline across hover checks. Every hovered result is still rendered and checked for visual emphasis and unchanged whole-screen text, and all control extents remain checked.
  • Strip/split each approval-screen baseline once rather than repeating it for every option.
  • Run the benchmark timeout/interruption/continuation test inside testing/synctest: its existing timeout and delayed protocol events use deterministic virtual time rather than runner-dependent wall-clock scheduling. The assertions for interruption, measured failure, retained usage and successful continuation are unchanged.

Preserve evidence

  • Capture JSON test output and helper timings in seven-day artifacts for every platform, including failed jobs, using the maintained artifact action.
  • Document the split commands, retained coverage and timeout behaviour in README development guidance.

Validation

  • Core suite: go test -race -cover -timeout 10m -skip '^TestLocalisedScreens$' ./... passes locally (UI package 36.4 s).
  • Benchmark timeout test: 100 consecutive race-enabled runs pass.
  • Targeted scheduler rendered-surface race test passes; one local before/after run decreased from 3.8 s to 3.0 s. This is not a cross-platform timing guarantee.
  • go vet ./..., gofmt, git diff --check and actionlint on both workflows pass.
  • Complete 17-locale race/coverage matrix passes locally in 175.8 s with two workers. No locale or helper assertion was removed.

No production code, dependencies or embedded web assets change. Cross-platform runtime improvements will be verified by this PR's CI.

@merefield
merefield merged commit 7195770 into main Oct 8, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant