Skip to content

test(agent): make the cancelled-retry assertion match the property - #313

Merged
yusiwen merged 1 commit into
masterfrom
fix/cancel-retry-flake
Oct 31, 2026
Merged

yusiwen merged 1 commit into
masterfrom
fix/cancel-retry-flake

Conversation

@yusiwen

@yusiwen yusiwen commented Oct 31, 2026

Copy link
Copy Markdown
Owner

What changed

agent/retry_http_test.go, one test:

  • retries is now an int32 read with atomic, because the retry path and the test goroutine both
    touch it;
  • the assertion is retries <= 1 instead of retries == 0, with the reason in the comment: if the
    cancel lands after the retry decision but before the backoff's sleep observes it, OnRetry announces
    the attempt the sleep then abandons — and no second request follows, which is the property the test
    is named for. requests == 1 keeps its stricter check.

Why

This is the flake filed as #295, and it just failed again — master's post-merge run for 8d1ccb8
(Test repeated in one process) failed with OnRetry fired 1 times, want 0. #295's acceptance says to
decide which of the two readings is true and make the test say it; the measured ordering says the
announcement is allowed, so the assertion moves rather than the code. The alternative reading — forbid the
announcement — would mean adding a cancellation check before announcing, which does not remove the race
(it would move it one line earlier).

Verification

  • go test ./agent -run TestCancelledRunMakesNoFurtherRequest -count=50 → ok.
  • go test ./agent -count=3 (the gate that failed) → ok.
  • go test ./... -count=1 -race → ok.
  • git log --format='%h %G? %s' → G.

Honest scope

The change makes the test tolerant of the ordering that the code actually exhibits; it does not make the
outcome deterministic. A run that does retry would still fail requests == 1, which is the property the
test exists for.

Refs #295

TestCancelledRunMakesNoFurtherRequest failed on master's post-merge run of
8d1ccb8 with "OnRetry fired 1 times, want 0" — the second time this has happened
(issue #295, first seen on #294).

The assertion was stricter than the property the test is named for. If the
cancel lands after the retry decision but before the backoff's sleep observes it,
OnRetry announces the attempt the sleep then abandons — and no second request is
made, which is the fact that matters. The counter also crossed goroutines
untyped. It is now atomic and asserts at most one announcement, while the request
count keeps its stricter "exactly one" check.

Tested at -count=50 and with the whole package at -count=3 (the gate that
failed), plus the race detector across the repository.

Refs #295
@yusiwen yusiwen added the tests Test coverage and test infrastructure label Oct 31, 2026
@yusiwen
yusiwen merged commit 8a6b557 into master Oct 31, 2026
9 checks passed
@yusiwen
yusiwen deleted the fix/cancel-retry-flake branch October 31, 2026 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests Test coverage and test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant