Skip to content

fix(config): accept streamed null as an empty configuration collection (backport #2762 to 1.7.x) - #2765

Merged
tobz merged 1 commit into
releases/1.7.xfrom
jszwedko/backport-2762-1.7.x
Oct 6, 2026
Merged

tobz merged 1 commit into
releases/1.7.xfrom
jszwedko/backport-2762-1.7.x

Conversation

@jszwedko

@jszwedko jszwedko commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

Backports #2762 to releases/1.7.x. The Datadog Agent streams null for an empty or cleared list or map, which ADP previously failed to handle; this treats null as an empty collection so cleared settings (e.g. additional_endpoints, histogram_aggregates) are applied instead of rejected. Fixes #2750 on the 1.7 release line.

Clean cherry-pick of 6e3f308 with no conflicts.

Test plan

🤖 Generated with Claude Code

#2762)

## Summary

The Datadog streams `null` for an empty or cleared list or map. We need to handle this as an empty collection.

Fixes #2750.

## Test plan

- [x] New reader unit tests in `list_de.rs` and `cast_de.rs`: `null` and empty inputs read as empty, null `replace_tags` elements read as empty maps (sequence and JSON-string forms)
- [x] New `null_collection_tests` in `datadog-agent-config`: every collection leaf in the generated model accepts `null`; `null` yields empty while an absent `histogram_aggregates` keeps its default
- [x] New `system.rs` tests: a startup snapshot with null collections translates, and a `null` update clears a previously non-empty `additional_endpoints`/`histogram_aggregates` (both fail without the fix)
- [x] New translator test: `replace_tags` rules missing `name` or `pattern` (including `[null]`) are rejected with the trace-agent's messages
- [ ] Follow-up once #2722 lands: remove the replay expectations this makes match

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: jesse.szwedko <jesse.szwedko@datadoghq.com>
(cherry picked from commit 6e3f308)
@jszwedko
jszwedko requested a review from a team as a code owner October 2, 2026 18:38
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-02T18:40:55.737591Z 349024f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-us1-prod datadog-datadog-us1-prod Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

JSON-encoded null strings still fail in both collection deserializers, so environment-originated empty arrays or maps can continue to block configuration application despite literal null now working.

Open Bits AI session

🤖 Bits Code Review · Commit 349024f · @DataDog review to ask questions

None => return Ok(Vec::new()),
Some(JsonArrayOrString::Array(values)) => values,
Some(JsonArrayOrString::String(value)) => {
serde_json::from_str::<Vec<Option<T>>>(&value).map_err(de::Error::custom)?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Accept encoded null arrays as empty

When a JSON-array setting arrives through the supported encoded-string path with value "null", it is deserialized directly as Vec<Option<T>> and rejected. Environment-originated empty arrays can therefore still prevent configuration application even though literal null arrays are now accepted.

Suggested change
serde_json::from_str::<Vec<Option<T>>>(&value).map_err(de::Error::custom)?
serde_json::from_str::<Option<Vec<Option<T>>>>(&value)
.map_err(de::Error::custom)?
.unwrap_or_default()

Was this helpful? React 👍 or 👎
🤖 Bits Code Review · @DataDog review to ask questions · Open Bits AI session

HashMap::deserialize(de::value::MapAccessDeserializer::new(map))
}

fn visit_unit<E: de::Error>(self) -> Result<Self::Value, E> {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Accept encoded null maps as empty

The new visitor methods handle literal null maps, but the existing encoded-string path still deserializes "null" directly into a HashMap and rejects it. A map setting such as an environment-originated additional_endpoints=null can still block startup instead of clearing the map.

Was this helpful? React 👍 or 👎
🤖 Bits Code Review · @DataDog review to ask questions · Open Bits AI session

@pr-commenter

pr-commenter Bot commented Oct 2, 2026

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: 98ec0b6 · Comparison: 349024f · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 38.27 MiB (baseline) vs 38.30 MiB (comparison)
Size Change: +28.95 KiB (+0.07%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
agent_data_plane_config_system::translators::datadog_translator +24.94 KiB 70
anon.a6926c6c21417749435f71dba1b992d8.976.llvm.13372907567088152362 +20.54 KiB 1
anon.a6926c6c21417749435f71dba1b992d8.976.llvm.1534094455157536209 -20.54 KiB 1
anon.61900d52399115b5efe8cc7341401cf5.2.llvm.6536046757030618740 -16.09 KiB 1
anon.ddcb7f9b52fa26bcbb78adf96efcb5f5.2.llvm.5509108223672863913 +16.09 KiB 1
anon.61900d52399115b5efe8cc7341401cf5.768.llvm.6536046757030618740 -14.62 KiB 1
anon.ddcb7f9b52fa26bcbb78adf96efcb5f5.768.llvm.5509108223672863913 +14.62 KiB 1
anon.53bb41fcd4d7cb0fd4749e93eb822af9.155.llvm.5834098439156321449 +14.40 KiB 1
anon.53bb41fcd4d7cb0fd4749e93eb822af9.155.llvm.2364525274383654594 -14.40 KiB 1
anon.61900d52399115b5efe8cc7341401cf5.766.llvm.6536046757030618740 -12.84 KiB 1
anon.ddcb7f9b52fa26bcbb78adf96efcb5f5.766.llvm.5509108223672863913 +12.83 KiB 1
anon.af6772eb050c605c9504dce13a278f5e.693.llvm.10692962610671248123 +11.14 KiB 1
anon.9e89adeaf08041411e2ad87207bd42dc.695.llvm.3460021552139762848 -11.13 KiB 1
datadog_agent_config::generated::witness -9.49 KiB 1
anon.9e89adeaf08041411e2ad87207bd42dc.4.llvm.3460021552139762848 -8.49 KiB 1
anon.af6772eb050c605c9504dce13a278f5e.4.llvm.10692962610671248123 +8.48 KiB 1
anon.61900d52399115b5efe8cc7341401cf5.704.llvm.6536046757030618740 -8.23 KiB 1
anon.ddcb7f9b52fa26bcbb78adf96efcb5f5.704.llvm.5509108223672863913 +8.23 KiB 1
anon.61900d52399115b5efe8cc7341401cf5.668.llvm.6536046757030618740 -7.67 KiB 1
anon.ddcb7f9b52fa26bcbb78adf96efcb5f5.668.llvm.5509108223672863913 +7.67 KiB 1
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +40.5Ki  [NEW] +40.4Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h6a0a0325e5a35be5
  [NEW] +34.9Ki  [NEW] +34.7Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h7f35d44d61060dd1
  [NEW] +31.4Ki  [NEW] +31.2Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::h701927d1fff6de84
  +0.4% +28.9Ki  +0.2% +11.9Ki    [17149 Others]
  [NEW] +28.7Ki  [NEW] +28.6Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::h697b0d9f49ed5ea2
  [NEW] +28.4Ki  [NEW] +28.2Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::connect::_{{closure}}::_{{closure}}::_{{closure}}::he8b12482fc68af93
  [NEW] +25.6Ki  [NEW] +25.4Ki    agent_data_plane::internal::remote_agent::run_remote_agent_registration_loop::_{{closure}}::h9a2c6a440827e953
  [NEW] +24.9Ki  [NEW] +24.8Ki    agent_data_plane::main::_{{closure}}::hb8386d92898a3832
  [NEW] +23.5Ki  [NEW] +23.3Ki    agent_data_plane::cli::debug::handle_debug_command::_{{closure}}::h76cc69fe35150fd4
  [NEW] +22.3Ki  [NEW] +22.2Ki    agent_data_plane::internal::env::ADPEnvironmentProvider::from_configuration::_{{closure}}::h0b8682d74d6328c7
  [NEW] +21.6Ki  [NEW] +21.4Ki    agent_data_plane::internal::env::workload::build_collector::_{{closure}}::h5e97d6ad03e0c55d
  [DEL] -21.6Ki  [DEL] -21.4Ki    agent_data_plane::internal::env::workload::build_collector::_{{closure}}::h3153eab666a7a9fa
  [DEL] -22.3Ki  [DEL] -22.2Ki    agent_data_plane::internal::env::ADPEnvironmentProvider::from_configuration::_{{closure}}::h179443bd5a69ff9a
  [DEL] -23.5Ki  [DEL] -23.3Ki    agent_data_plane::cli::debug::handle_debug_command::_{{closure}}::h0cd20d731b8ea9c5
  [DEL] -24.9Ki  [DEL] -24.8Ki    agent_data_plane::main::_{{closure}}::hb746ec0caf2d4edc
  [DEL] -25.6Ki  [DEL] -25.4Ki    agent_data_plane::internal::remote_agent::run_remote_agent_registration_loop::_{{closure}}::hc7cd90f82c9cf27e
  [DEL] -28.4Ki  [DEL] -28.2Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::connect::_{{closure}}::_{{closure}}::_{{closure}}::hee1adc6abddef53c
  [DEL] -28.7Ki  [DEL] -28.6Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::hb2a3129affe62bdb
  [DEL] -31.4Ki  [DEL] -31.2Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::hf3f67152355be866
  [DEL] -34.9Ki  [DEL] -34.7Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h48f5a75cbf949eb9
  [DEL] -40.5Ki  [DEL] -40.4Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h48199e6cde4750a2
  +0.1% +28.9Ki  +0.0% +11.9Ki    TOTAL

@pr-commenter

pr-commenter Bot commented Oct 2, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: 3855aff3-a4ca-4dc7-b0b5-da88f2e7a27c
Baseline: 98ec0b69 · Comparison: 349024f8 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.34 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ +0.14 metrics profiles logs
quality_gates_rss_idle memory ⚪ +0.14 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ +0.08 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ -0.15 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 232 MiB ≤ 250 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 51.4 MiB ≤ 60 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 91.3 MiB ≤ 100 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 388 MiB ≤ 420 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 33.2 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression. Experiments tagged (no analysis) show ⚠️ n/a: SMP ran them but produced no analysis, usually because a replicate failed and exhausted its retries. Check the SMP report for that experiment's replicate failures.

@tobz
tobz merged commit 488a6d9 into releases/1.7.x Oct 6, 2026
98 checks passed
@tobz
tobz deleted the jszwedko/backport-2762-1.7.x branch October 6, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants