Skip to content

feat: Lambdas behind an Application Load Balancer get HTTP attributes on their spans - #102

Open
RaphaelManke wants to merge 1 commit into
masterfrom
RaphaelManke/AWS-ALB-http-attributes
Open

feat: Lambdas behind an Application Load Balancer get HTTP attributes on their spans#102
RaphaelManke wants to merge 1 commit into
masterfrom
RaphaelManke/AWS-ALB-http-attributes

Conversation

@RaphaelManke

Copy link
Copy Markdown
Contributor

TL;DR

  • What & why: Lambdas behind an Application Load Balancer now get HTTP attributes on their spans, where before they got none.
  • Implementation: The existing API Gateway extraction grew an ALB branch that reads the fields ALB actually sends.
  • Notable: ALB spans carry no http.route, because the event has no route in it to read.

Notable changes

Details
  • http.route and network.protocol.version are absent for ALB. The event carries no route template and no protocol field, so the extension omits them rather than guessing. It does not fall back to url.path as the route, since that value is high cardinality.
  • Under the existing DASH0_ENABLE_API_GATEWAY_SPAN_NAME, the extension names an ALB span after the bare method, such as GET. That is the HTTP semconv fallback when no route is available.
  • The four DASH0_API_GATEWAY_* variables now gate ALB capture too. Their names stay as they are so nothing breaks for current users, and the README notes it.
  • Internal renames follow the widened scope. ApiGatewayVersion becomes HttpEventKind, and the InvocationEntry fields lose their api_gateway_ prefix. No env var or attribute name changes.

Feature: HTTP attributes for ALB-triggered invocations

Details

What & Why:

  • The extension recognized ALB events only well enough to keep them out of the API Gateway path. They carry a requestContext and a top-level httpMethod, so without that guard the extension would have reported them as REST API v1. Nothing picked them back up afterwards, so an ALB-triggered invocation got no HTTP attributes and no response status code.
  • Extraction runs off the raw invoke event inside the extension, so this works on every runtime and also when auto-instrumentation is off.

Implementation:

  • Detection now reports which of the three shapes an event has. The ALB check runs first, since its top-level httpMethod would otherwise match the v1 branch.
  • The ALB branch omits attributes ALB has no source for. What it does have comes from the forwarding headers. url.scheme reads x-forwarded-proto, server.address and server.port read host with x-forwarded-port as the port fallback, and client.address takes the first x-forwarded-for hop.
  • That branch reads url.scheme rather than hardcoding https as the API Gateway branches do, because ALB listeners serve plain HTTP as well.
  • Target groups with lambda.multi_value_headers.enabled send multiValueHeaders and multiValueQueryStringParameters instead of their single-value counterparts, in both directions. Reading only headers returns nothing at all on those target groups, so the code resolves either shape. A header sent more than once becomes one comma-joined value, which keeps the attribute a string in both configurations.
  • Health check invocations arrive with a user-agent and nothing else, so each attribute is optional on its own rather than all-or-nothing.
flowchart LR
    E[invoke event] --> D{requestContext.elb?}
    D -->|yes| A[ALB]
    D -->|no| V{rawPath and<br/>requestContext.http?}
    V -->|yes| V2[API Gateway v2]
    V -->|no| V1{httpMethod?}
    V1 -->|yes| R[API Gateway v1]
    V1 -->|no| N[not HTTP, no attributes]
Loading

Verification:

  • 17 unit tests cover the ALB shapes, including both header configurations, the health check event, an IPv6 host header, and a multi-hop x-forwarded-for.
  • A temporary load balancer checked the event shapes against real traffic rather than the documentation alone. One default and one multi-value target group confirmed that:
    • requestContext holds only elb.targetGroupArn, with no route information anywhere in the event
    • the multi-value shape replaces the single-value fields instead of supplementing them
    • header names arrive lower-cased
    • ALB appends the client IP to an existing x-forwarded-for
    • query strings arrive without URL-decoding
    • a health check carries only user-agent
  • That load balancer is deleted.

Application Load Balancer events were detected only well enough to be
excluded from API Gateway classification, so ALB-triggered invocations got
no HTTP attributes at all. Extend the existing extraction to cover them.

ALB carries the same information under different field names, so
ApiGatewayVersion becomes HttpEventKind with an Alb variant and the
extraction branches on it. Attributes ALB has no source for are omitted
rather than guessed: there is no resource/routeKey, so no http.route, and
no protocol field, so no network.protocol.version.

Two ALB behaviours needed handling that API Gateway does not have:

- Target groups with lambda.multi_value_headers.enabled send
  multiValueHeaders and multiValueQueryStringParameters *instead of* their
  single-value counterparts, in both directions. Reading only `headers`
  yields nothing at all on those target groups.
- ALB listeners serve plain HTTP, so url.scheme follows x-forwarded-proto
  rather than being hardcoded to https.

Verified against a real ALB rather than the docs alone: a temporary load
balancer with both target group types confirmed the field shapes, header
lower-casing, x-forwarded-for chaining, un-decoded query strings, and the
headers-only health check event.

The DASH0_API_GATEWAY_* env vars now gate ALB capture too; their names are
kept for backwards compatibility.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant