Skip to content

test: fix racy mt_record_static (refresh raced recorder drops) - #155

Open
fcostaoliveira wants to merge 1 commit into
HdrHistogram:mainfrom
fcostaoliveira:fix/mt-record-static-race
Open

test: fix racy mt_record_static (refresh raced recorder drops)#155
fcostaoliveira wants to merge 1 commit into
HdrHistogram:mainfrom
fcostaoliveira:fix/mt-record-static-race

Conversation

@fcostaoliveira

Copy link
Copy Markdown
Contributor

What

tests/sync.rs::mt_record_static is flaky: it intermittently undercounts the total. On a 96-core machine it fails ~10 of 15 runs (observed totals 1.1–1.3M instead of 1.6M); it also reproduces on an 8-core laptop (~3/20). The shortfall is always a whole multiple of the per-thread count (100 000) — i.e. entire per-recorder histograms are missing.

Why

The test refreshes and reads h.len() before joining the recording threads:

barrier.wait();
h.refresh();
assert_eq!(h.len(), jhs.into_iter().map(|r| r.join().unwrap()).sum());

The Barrier only guarantees recording finished — not that the Recorders have been dropped. A live Recorder's samples become visible to refresh() only when it is dropped (or a later write observes the bumped phase), and a drop makes them visible to the next refresh(). Because h.len() (the assert's left operand) is evaluated before the .join() on the right, refresh() runs concurrently with the threads returning and dropping their recorders, so it can return before folding in the stragglers. No data is lost — a subsequent refresh() recovers it — so this is a test-ordering bug, not a library defect.

Fix

Join all recording threads first (dropping every Recorder), establishing happens-before, then refresh() and read:

barrier.wait();
let expected: u64 = jhs.into_iter().map(|r| r.join().unwrap()).sum();
h.refresh();
assert_eq!(h.len(), expected);

Deterministic afterwards: 25/25 locally (was ~3/20). Test-only change; no library code touched. This is the same anti-pattern #151 fixed in record_nodrop / recorder_drop_staged — this instance was missed.

🤖 Generated with Claude Code

mt_record_static refreshed and read h.len() before joining the recording
threads, so refresh() raced the Recorders being dropped. A dropped Recorder's
samples are only guaranteed visible to the *next* refresh(), so the single
racing refresh undercounts by whole per-recorder histograms (observed ~1.1-1.3M
instead of 1.6M, ~10/15 runs on a 96-core box). No data is lost -- a subsequent
refresh recovers it, so this is a test ordering bug, not a library defect.

Join all recording threads first (which drops every Recorder), then refresh and
read. Deterministic afterwards (25/25 locally). The HdrHistogram#151 cleanup fixed
record_nodrop and recorder_drop_staged but left this instance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant