Skip to content

chore: overhaul the jobs test suite - #237

Merged
rustatian merged 2 commits into
masterfrom
chore/overhaul-test-suite
Aug 18, 2026
Merged

chore: overhaul the jobs test suite#237
rustatian merged 2 commits into
masterfrom
chore/overhaul-test-suite

Conversation

@rustatian

Copy link
Copy Markdown
Member

First of the nine jobs drivers, and the pattern the rest will follow. Memory needs no broker, so the whole suite runs locally.

26 fixed sleeps in the jobs file, including a flat 15s for the prefetch case and 25s for the protocol error — the suite spent most of its time waiting rather than testing. The helper now offers:

  • Start(t, cfg, plugins, opts...) with an rpc readiness probe, replacing the 3s sleep every test opened with
  • WaitLog(snippet, n) / RequireLogCount(snippet, n) — wait for the record a step produces, then assert the exact count, which also catches a driver that redelivers
  • WaitStats(pred) — poll the pipeline state instead of sleeping out a delayed job's delay

Jobs tests now run in about 20s.

Split per behaviour: boot, push and process, pause and resume, stats for delayed and drained pipelines, the prefetch limit, the protocol error, and the response handler error.

Three things I had to correct by reading the originals rather than assuming, all worth knowing for the other eight drivers:

  • pipeline was stopped and destroy signal received are written during container shutdown, not by jobs.Pause or jobs.Destroy — the pause record is pipeline was paused
  • pipeline was resumed only appears after an explicit jobs.Resume; pipelines listed under consume: log pipeline was started at boot
  • the v2.7 bad-response config pushes to two pipelines, so it expects two response handler error records, not one

CI: both workflow steps named a single test file (jobs_memory_test.go, kv_memory_test.go), so a split file simply would not run. They now select by test name over ./.... Dropped -failfast and added the coverage guard to both.

Not covered by this PR: kv_memory_test.go is untouched. It fails on my machine because its http worker needs PHP's ext-sockets, which I do not have — I verified it fails identically on a clean checkout, and memory's CI is green on it, so it is a local limitation rather than a regression.

Replace the per-test endure boilerplate with a Start helper that boots
the container, waits for the rpc listener and tears down through
t.Cleanup, plus WaitLog and WaitStats so a test waits for the record or
the pipeline state it cares about instead of sleeping.

The jobs file carried 26 fixed sleeps, including a flat 15s for the
prefetch case and 25s for the protocol error. Those are now waits on the
records they were standing in for, and the suite runs in about 20s.

Split the assertions per behaviour: boot, push and process, pause and
resume, stats for delayed and drained pipelines, the prefetch limit, the
protocol error and the response handler error. The stats case polls the
pipeline state rather than sleeping out the delay it pushed.
Both steps named a single test file, so the split would not run. Select
by test name instead, drop -failfast, and fail the codecov job when the
merged summary maps to no plugin source.
Copilot AI lite review requested due to automatic review settings August 17, 2026 20:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.86%. Comparing base (bdc068c) to head (0c92a32).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #237      +/-   ##
==========================================
- Coverage   76.25%   75.86%   -0.39%     
==========================================
  Files           6        6              
  Lines         518      518              
==========================================
- Hits          395      393       -2     
- Misses         91       93       +2     
  Partials       32       32              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rustatian rustatian self-assigned this Aug 18, 2026
@rustatian
rustatian merged commit 7c173d0 into master Aug 18, 2026
9 of 10 checks passed
@rustatian
rustatian deleted the chore/overhaul-test-suite branch August 18, 2026 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants