fix(queryeviction): fix flaky TestPrometheusMetrics_IncrementedCorrectly

[krunaljain]

What this PR does

Fixes the flaky TestPrometheusMetrics_IncrementedCorrectly test in pkg/util/queryeviction.

QueryEvictor.running() (pkg/util/queryeviction/evictor.go) calls victim.Cancel() — which closes
the test's evicted channel — before incrementing evictionsTotal:

for _, victim := range victims {
    metricValue := e.registry.metric(victim.Stats)
    victim.Cancel()                                            // closes `evicted` first

    level.Warn(e.logger).Log(...)

    e.evictionsTotal.WithLabelValues(string(breachedResource)).Inc()  // metric incremented after
}

The test synchronizes only on the eviction signal (waitEvicted), not on the metric write, so the
final assertion can observe the counter before the last increment lands:

--- FAIL: TestPrometheusMetrics_IncrementedCorrectly (0.03s)
    evictor_test.go:210:
        Error: Not equal: expected: 3  actual: 2

Verified the flake is real

Checked out the pre-fix version of evictor_test.go and stress-tested it directly (not just relying
on the issue report):

go test -count=500 -cpu=1,2,4 -run TestPrometheusMetrics_IncrementedCorrectly ./pkg/util/queryeviction/...

This reproduced the exact expected: 3, actual: 2 failure from the issue — 8 failures out of 500 runs.
With the fix applied, the same 500-run stress test (-cpu=1,2,4, plus a separate 100-run pass under
-race) passes cleanly every time.

The fix

Replace the immediate assertion with require.Eventually, polling for the metric to reach the
expected value instead of asserting right after the signal:

require.Eventually(t, func() bool {
    return promtest.ToFloat64(evictor.evictionsTotal.WithLabelValues(string(resource.CPU))) == float64(3)
}, 500*time.Millisecond, 5*time.Millisecond, "expected evictionsTotal to reach 3")

Test-only change; no production behavior is affected.

Which issue(s) this PR fixes

Fixes #7604

Checklist

• Tests updated (this PR is the test fix)
• CHANGELOG.md updated
• Documentation added — N/A, no user-facing behavior change
• docs/configuration/v1-guarantees.md updated — N/A, no new flags

AI-assistance disclosure

Per GENAI_POLICY.md: root-cause analysis of the race and the fix were drafted with Claude Code. I
reviewed the diff, understand the race (signal-before-metric ordering in running()), and independently
verified the flake reproduces pre-fix and is resolved post-fix using the stress-test runs described above.

[krunaljain]

Closing in favor of #7616, which was already open (opened 2026-06-11, ahead of this one) and fixes the root cause more thoroughly — it reorders the eviction metric increment/registry deregistration to happen before the cancel signal, rather than the test-side polling approach here, and also catches a related double-eviction accounting bug I hadn't spotted. Apologies for the duplicate.