perf: stop eager-building HTTPS_CLIENT before extension registration - #95
Open
RaphaelManke wants to merge 1 commit into
Open
perf: stop eager-building HTTPS_CLIENT before extension registration#95RaphaelManke wants to merge 1 commit into
RaphaelManke wants to merge 1 commit into
Conversation
route::init() force-built HTTPS_CLIENT via Lazy::force ahead of extension registration. Building it invokes hyper-rustls's with_native_roots(), which synchronously parses the entire OS trust store -- confirmed via debug logs during local profiling (162 certs on the profiling host). HTTPS_CLIENT is only used later, for outbound OTLP export. Registration gates when AWS lets the customer runtime start initializing, so this was pure CPU-bound work sitting in front of that gate for no benefit. Letting it stay lazy (first real use constructs it) removes it from the cold-start critical path entirely. Measured locally with perf-harness (native arm64, mock Extensions API): median spawn-to-registered time dropped ~1ms (7.93ms -> 6.89ms) on top of whatever the real trust-store parse costs on the actual Lambda runtime, which is likely larger than this dev machine's 162-cert bundle.
RaphaelManke
temporarily deployed
to
staging
September 10, 2026 14:07 — with
GitHub Actions
Inactive
RaphaelManke
temporarily deployed
to
staging
September 10, 2026 14:07 — with
GitHub Actions
Inactive
RaphaelManke
temporarily deployed
to
staging
September 10, 2026 14:07 — with
GitHub Actions
Inactive
RaphaelManke
temporarily deployed
to
staging
September 10, 2026 14:07 — with
GitHub Actions
Inactive
3 tasks
RaphaelManke
temporarily deployed
to
staging
September 10, 2026 14:12 — with
GitHub Actions
Inactive
RaphaelManke
temporarily deployed
to
staging
September 10, 2026 14:12 — with
GitHub Actions
Inactive
RaphaelManke
temporarily deployed
to
staging
September 10, 2026 14:12 — with
GitHub Actions
Inactive
RaphaelManke
temporarily deployed
to
staging
September 10, 2026 14:24 — with
GitHub Actions
Inactive
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
route::init()force-builtHTTPS_CLIENT(viaLazy::force) before the extension registers with the Lambda Extensions API. Building it invokeshyper-rustls's.with_native_roots(), which synchronously parses the entire OS trust store.Lazy::forcecall and let the client build lazily on first real use.Evidence
Caught via live profiling (native arm64 build against a local mock of the Lambda Extensions/Runtime API): debug logs showed
with_native_roots processed 162 valid and 0 invalid certslogged before theRegistered with accountIdline. After the fix, that log line no longer appears before registration.Measured with a small local perf harness (added separately, see the harness PR): median spawn-to-registered time dropped ~1ms (7.93ms -> 6.89ms) on this dev machine. The real win should be larger on the actual Lambda runtime, where the OS trust-store bundle is likely bigger than this Mac's 162-cert one.
Real-AWS cold-start verification
Confirmed against actual Lambda cold starts, not just the local harness:
pr-95-IntegrationTestsStack, account175948133317,eu-central-1) via the existing CI integration-test pipeline.v23, published from master the same day this branch was cut -- see release v23, layer ARNarn:aws:lambda:<region>:115813213817:layer:dash0-extension-node:23), temporarily attached to one of this PR's integration-test functions (pr-99-nodejs24-x-success-true-invocation-end-false-arm64) in place of this PR's own layer, then reverted afterward.update-function-configurationwith a unique description, wait for the update, 100ms bake), theninvokewithLogType=Tailand parseInit Durationfrom the LambdaREPORTlog line. n=15 cold starts per variant.update-function-configuration --environmentcall that overwrites rather than merges the variable map) had inadvertently clearedDASH0_TOKEN_SECRET_ARNon both functions used here, so neither arm of this comparison actually made a Secrets Manager call. That doesn't invalidate the result for this fix specifically --HTTPS_CLIENT's eager build has nothing to do with the token path, and both arms ran the identical (fast, no-op) "no token configured" branch ofresolve_token(), so the 33.1ms delta is still cleanly attributable to the fix under test here. It does mean this number says nothing about Secrets Manager behavior; see PR perf: resolve the Dash0 token lazily, off the extension's cold-start critical path #99 for that measurement, done after the environment was fixed.Test plan
cargo build --releasecargo test --release(325/325 passing)with_native_rootsno longer runs beforeRegistered with accountId