Skip to content

fix: correct native histogram bucket merge on scale-down#2274

Open
arnabnandy7 wants to merge 1 commit into
prometheus:mainfrom
arnabnandy7:fix/native-histogram-bucket-resize
Open

fix: correct native histogram bucket merge on scale-down#2274
arnabnandy7 wants to merge 1 commit into
prometheus:mainfrom
arnabnandy7:fix/native-histogram-bucket-resize

Conversation

@arnabnandy7

Copy link
Copy Markdown
Contributor

Summary

Fix native histogram bucket remapping when the bucket width is doubled during schema scale-down.

The previous merge formula used:

(keys[i] + 1) / 2

That works for positive bucket indexes, but not for negative even indexes. Java integer division truncates toward zero, so negative even buckets were merged into the bucket immediately to the right of the correct target.

This matches the behaviour described in #2270 and aligns the merge formula with client_golang:

(keys[i] > 0 ? keys[i] + 1 : keys[i]) / 2

Why

When doubleBucketWidth() runs, counts from some negative native histogram buckets could be shifted into neighbouring buckets. That can make Prometheus interpret the reshuffled bucket counters as resets, causing spikes in queries such as:

histogram_count(rate(my_native_histogram[1m]))

Tests

Added a focused regression test for negative, zero, and positive bucket index remapping during doubleBucketWidth(Map).

Verified locally:

./mvnw test -pl prometheus-metrics-core -am \
  -Dtest=HistogramTest#testDoubleBucketWidthMergesNegativeIndexesCorrectly \
  -Dcoverage.skip=true \
  -Dcheckstyle.skip=true \
  -Dsurefire.failIfNoSpecifiedTests=false

Also verified:

./mvnw install -DskipTests -Dcoverage.skip=true

Fixes #2270

Signed-off-by: Arnab Nandy <arnab_nandy7@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug in resizing buckets of a native histogram

1 participant