test(skywalking): remove worker-0 dependency in backend timer assertion - #13902
Open
janiussyafiq wants to merge 1 commit into
Open
test(skywalking): remove worker-0 dependency in backend timer assertion#13902janiussyafiq wants to merge 1 commit into
janiussyafiq wants to merge 1 commit into
Conversation
AlinsRan
approved these changes
Sep 1, 2026
nic-6443
approved these changes
Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
t/plugin/skywalking2.tTEST 2 fails intermittently on CI, and since it can fail the flaky-rerun too, it turns wholebuildjobs red.Example: run 33146313320 on #13651, where it failed both the initial pass and the rerun with:
Cause: the test's patched
startBackendTimerlogsstart skywalking backend timeronly onngx.worker.id() == 0, so TEST 2 needs worker 0 to accept at least one of its 50 keepalive-off requests.The comment in the test assumed independent uniform accepts (
(3/4)^50, "vanishingly unlikely" to miss worker 0), but nginx accept distribution is skewed and correlated: measured locally over 60 runs, the per-worker split of the 50 requests was consistently about 17/16/14/3, with the starved worker receiving as few as 1 request and worker 0 sitting in the starved slot in roughly a quarter of runs.On a loaded 4-vCPU CI runner the starved worker can receive 0 requests, and when that worker is worker 0 the log line never appears.
Fix: replace the worker-id gate with an atomic
ngx.shared.DICT:add()first-starter guard, so whichever worker handles the first sampled request starts the timer and logs, exactly once.The assertion no longer depends on which worker the kernel picks; it only needs any request to be handled, which the block already asserts via the 50 expected 200s.
The 50-request loop is kept: with multiple workers each calling
startBackendTimeron their first request, the expected single log line now also proves the guard dedupes across workers.Verification (250 local runs):
The 1-request variant makes the failure condition (worker 0 gets no request) the common case, reproducing the CI failure on demand with the old guard and never with the new one.
Which issue(s) this PR fixes:
None filed; flake observed on #13651's CI.
Checklist