Add DI snapshot payload size pruning (RFC C6) - #6224
Draft
p-datadog wants to merge 3 commits into
Draft
Conversation
Contributor
|
Thank you for updating Change log entry section 👏 Visited at: 2026-08-31 21:51:34 UTC |
dd-octo-sts
Bot
added
the
debugger
|
BenchmarksBenchmark execution time: 2026-08-24 14:34:43 Comparing candidate commit 1db49c8 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 1 unstable metrics.
|
Add Datadog::DI::SnapshotPruner.prune, a Hash walker that fits an
oversized serialized snapshot under the 1 MB per-event cap by replacing
the largest top-level captured values (the {type:...} objects under
locals/arguments/throwable in the captures subtree) with
{"pruned":true} markers, preserving variable names and the structural
envelope. Measures each captured value by encoding it (JSON.dump is
deterministic, so a subtree's standalone encoding is byte-identical to
its span in the full dump), sorts by size descending, and replaces the
largest until the overage is reclaimed, then re-encodes once. Returns
nil when no captured values can be pruned or pruning cannot fit.
Replace the drop-if-too-big check in
DI::Transport::Input::Transport#send_input with prune-if-too-big:
an oversized snapshot is pruned (captured values replaced with
{"pruned":true}) and sent when it fits under
MAX_SERIALIZED_SNAPSHOT_SIZE, and dropped only when pruning cannot
bring it under the cap. Switch the cap check from encoded.length
(character count) to encoded.bytesize (the 1 MB network limit is a
byte limit; .length undercounted multibyte UTF-8). Increment a
snapshots_pruned_by_payload_size telemetry counter on each prune.
p-datadog
force-pushed
the
di-payload-size-prune
branch
from
August 24, 2026 14:07
51cbbfd to
1db49c8
Compare
Oversized snapshots are pruned (largest captured variables replaced with a pruned marker) and delivered partially, and dropped only when pruning cannot bring them under the 1 MB limit.
Contributor
Typing analysisNote: Ignored files are excluded from the next sections. Untyped methodsThis PR introduces 6 partially typed methods, and clears 31 partially typed methods. It increases the percentage of typed methods from 70.06% to 71.16% (+1.1%). Partially typed methods (+6-31)❌ Introduced:Untyped other declarationsThis PR clears 2 partially typed other declarations. It increases the percentage of typed other declarations from 85.33% to 85.68% (+0.35%). Partially typed other declarations (+0-2)✅ Cleared:If you believe a method or an attribute is rightfully untyped or partially typed, you can add |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DEBUG-6077
What does this PR do?
Caps each serialized DI snapshot at 1 MB. An oversized snapshot is pruned — its largest captured values are replaced with a
{"pruned":true}marker — and delivered partially. A snapshot is dropped when pruning cannot bring it under the cap. The cap check measures bytes, accounting for multibyte UTF-8 content. Customer documentation is updated to describe this behavior.Motivation:
A snapshot exceeding the size limit is dropped entirely, losing all captured data. Pruning preserves as much of the snapshot as possible within the payload limit, sending partial data.
Change log entry
Yes. Dynamic Instrumentation: a snapshot exceeding the 1 MB payload limit is now pruned and delivered partially, with its largest captured values replaced by a marker. A snapshot is dropped when pruning cannot bring it under the limit.
Additional Notes:
N/A
How to test the change?
Unit tests added