feat: add PackedHistogram — memory-optimised sparse variant

[fcostaoliveira]

What

Adds PackedHistogram — a separate, opt-in histogram whose backing store grows with the number of populated buckets rather than counts_len. It reuses the dense bucket geometry exactly (through a Histogram<u8> oracle whose counts vector is emptied — the geometry lookups never index counts) and speaks the standard V2 / V2+DEFLATE format (byte-identical to V2Serializer), so it interoperates with every existing HdrHistogram reader. The dense Histogram<T> is untouched.

Why

Histogram::new_with_bounds eagerly allocates Vec<T> of length counts_len — ~184 KB per histogram at the default latency config (1, 3.6e9, 3) — regardless of how many buckets are ever used. For the many sparsely-populated histograms shape (per-endpoint / per-tenant / per-connection latency) that dominates the heap. Measured footprint (sparse backing, memory_size()):

Buckets populated / histo	Dense counts	Packed	Win
10	188,416 B	80 B	2355×
100	188,416 B	640 B	294×

Java has PackedHistogram; Rust had none.

Design

idx: Vec<u32> holds the populated flat counts indices (ascending); cnt: Vec<u8> holds one count per bucket at a uniform adaptive width (1/2/4/8 bytes) that widens on overflow. Record is a binary-search + insert; value_at_quantile is a width-specialized blocked prefix-sum over only the populated buckets, reusing the dense value_for / lowest_equivalent / highest_equivalent. All query paths are overflow-safe (saturating_*).

Only pub(crate) hooks were added to the dense code (no public-API growth): index_for/value_for, an oracle counts-clear, and re-exports of the existing varint / zig-zag helpers.

Correctness & testing

• Parity vs dense: 300 random trials, index-by-index counts, min/max/total, percentile sweeps — bit-for-bit.
• V2 interop: byte-identical to V2Serializer and V2DeflateSerializer; round-trips both directions; rejects a non-zero normalizingIndexOffset.
• Fuzzing: two randomized fuzzers — differential vs dense (4000 trials) and hostile-decode panic-safety (200k iterations).
• Coverage (cargo-llvm-cov): 98.7% lines / 100% of reachable lines (the 2 uncovered are documented-unreachable defensive branches); every function covered.
• Green under cargo test, cargo build --release, cargo fmt --check, and cargo clippy (no new warnings on packed.rs). The existing suite is unaffected.

Happy to adjust naming/layout to match how you'd want a new type to land.

---

Benchmark evidence — vs iopsystems/histogram v1.5.0 (the closest alternative)

Measured on three arches (AWS Intel Granite Rapids / AMD Zen 5 Turin / ARM Neoverse-V2), same bucket geometry both sides (21504 buckets), sparse latency-like workload (1605 populated):

Read (percentile) — ns/query, lower is better:

impl	Intel	AMD	ARM
PackedHistogram (this PR)	408	299	597
iop SparseHistogram	1238	936	1645
iop dense Histogram	10602	8840	28955
dense Histogram<u64>	380	316	512

• PackedHistogram reads 2.7–3.1× faster than iop's SparseHistogram, and it records live — iop's SparseHistogram is a read-only snapshot built from a dense histogram, so it doesn't help the many-sparse-recorders case this PR targets.
• Read parity with the dense Histogram (±7%; faster on Zen 5) — the blocked prefix-sum only pays for populated buckets.
• (Aside: iop's dense percentile() is 28–57× slower than HdrHistogram's — it does two full O(total_buckets) rescans plus a Vec/BTreeMap allocation per call; not this PR's concern, but it's why the sparse snapshot exists there.)

Memory (sparse workload): PackedHistogram 11.1 KB vs iop SparseHistogram 18.8 KB (1.7× smaller — adaptive 1–8 B counts vs iop's fixed 8 B) vs dense 168 KB (15× smaller). Results are bit-identical to the dense histogram (parity + differential fuzz enforce it).

[codecov]

Codecov Report

❌ Patch coverage is 96.76259% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.39%. Comparing base (ea926c4) to head (386b655).

Files with missing lines	Patch %	Lines
src/packed.rs	96.71%	18 Missing ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
src/lib.rs	89.65% <100.00%> (+0.35%)	⬆️
src/packed.rs	96.71% <96.71%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.