Send instance name as x-api-key to the OTLP relay#316
Open
archandatta wants to merge 7 commits into
Open
Conversation
Adds an optional in-VM OTLP/HTTP export sink that converts browser telemetry events into OTLP log records and forwards them to a configured endpoint, alongside the existing S2 sink. Gated on OTLP_RELAY_ENDPOINT; no behavior change when unset. - Converter maps each telemetry envelope to an OTLP log record: event type to EventName (also mirrored to kernel.event.type for backends that drop it), structured JSON body, derived severity, and kernel.* attributes. Network and console fields are promoted to semantic-convention attributes. Screenshot and monitor categories are excluded from export. - Sink mirrors S2StorageWriter: an independent ring reader feeds records through the OTel log SDK (otlploghttp + batch processor), running independently of the S2 sink. Export failures surface via a logging exporter wrapper. - Config and startup wiring, plus a dev collector config for local testing.
Rename export env vars from OTLP_RELAY_* to BTEL_OTLP_* so they don't collide with the standard OTEL_* vars that configure the API server's own telemetry, and drop the metro-api-specific "relay" naming (dev points at a collector). Rename the platform-identity config fields to InstanceJWT / InstanceName / MetroName since they aren't OTLP-specific. Default export path is now the standard /v1/logs. Make sink startup non-fatal: an optional best-effort exporter failing to start must not take down the browser, so log and continue instead of os.Exit(1). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The OTel log SDK reports records dropped under sustained backpressure via its global logger at logr V(1). That logger is a no-op until wired, and even once wired it renders just below slog Info, so a default Info handler would still swallow it. Route the SDK global logger to a handler that admits that level when export is enabled, so drops surface instead of being silently discarded (which looked identical to a healthy idle sink). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
TestLoad's expected structs didn't account for the OTLPPath and OTLPServiceName defaults that Load applies, so it failed once those defaults were in place. Assert the applied defaults. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The batch processor caps records per export by count (200), not bytes. With the 1MB per-envelope publish ceiling, a full batch of large records could reach ~200MB and be rejected by the collector/relay, and the SDK drops rejected records without retry. Split each export in the loggingExporter into sub-requests under a byte budget (4MiB) so a batch of large records can't exceed the target's HTTP body limit; a single record (<=1MB) always fits in one sub-request. Adds a chunk-splitting unit test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The metro-api OTLP relay authenticates a VM by its instance name checked against active sessions (mirroring the capmonster/hcaptcha relays), so send INST_NAME as x-api-key instead of the instance JWT as a bearer token. Drop the now-unused KERNEL_INSTANCE_JWT exporter config (the env still exists for the envoy wrapper; the exporter just no longer reads it). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
archandatta
marked this pull request as ready for review
July 15, 2026 16:51
|
|
# Conflicts: # server/cmd/api/main.go # server/cmd/config/config.go
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ec5597d. Configure here.
| if config.InstanceJWT != "" { | ||
| headers["Authorization"] = "Bearer " + config.InstanceJWT | ||
| if config.InstanceName != "" { | ||
| headers["x-api-key"] = config.InstanceName |
There was a problem hiding this comment.
Stale OTLP auth after fork
High Severity
The new relay auth header is taken from config.InstanceName once at process start. In fork-identity-wait mode, kernel-images-api starts before identity apply and is intentionally not restarted, so x-api-key stays empty or wrong after the instance is bound and relay exports keep failing auth.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit ec5597d. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Aligns the in-VM OTLP exporter to the metro-api relay's auth contract. The relay authenticates a VM by its instance name checked against active sessions (the same model as the capmonster/hcaptcha relays), so the exporter now sends
INST_NAMEas thex-api-keyheader instead of the instance JWT as a bearer token. The now-unusedKERNEL_INSTANCE_JWTexporter config is dropped (the env still exists for the envoy wrapper — the exporter just no longer reads it).Companion to the metro-api relay PR (kernel/kernel#2763). Wire contract:
x-api-key: <instance_name>+ POST toBTEL_OTLP_PATH(set to/otlp-relay/v1/logsby provisioning when pointing a VM at the relay).Why
The relay decides which org's destination + credentials to use from the instance identity. Verifying an instance JWT signature would require plumbing
XDS_JWT_SECRETinto metro-api (which has no such secret today and verifies no instance-JWT signatures) and would make the relay the odd one out among sibling relays. Instance names are unguessable cuid2 pairs gated on an active session, sox-api-keyis the consistent, sufficient credential.Changes
server/cmd/api/main.go:Authorization: Bearer <InstanceJWT>→x-api-key: <InstanceName>.server/cmd/config/config.go: remove theInstanceJWTfield and itsLogValueredaction.Test plan
go build ./...andgo test ./cmd/config ./lib/eventsgreen.Notes
archand/kernel-1213/otel-export, Add in-VM OTLP telemetry export sink #308) since that isn't merged yet; retarget tomainonce Add in-VM OTLP telemetry export sink #308 lands.401/403/404from the relay as a normal steady state. A later change can quiet those specific statuses inloggingExporter(needs confirming the status is cleanly extractable from theotlploghttperror) so they don't inflate the failure counter/logs.Note
Medium Risk
Changes OTLP relay authentication; mis-deployed or mismatched relay/exporter pairs would break telemetry export until both sides are updated.
Overview
Aligns in-VM OTLP export with the metro-api relay auth model used by capmonster/hcaptcha relays: the VM is identified by instance name against active sessions, not by verifying an instance JWT.
When OTLP export is enabled, outbound headers change from
Authorization: Bearer <KERNEL_INSTANCE_JWT>tox-api-key: <INST_NAME>(only whenInstanceNameis set). Comments inmain.godocument that contract.Config cleanup: removes the
InstanceJWT/KERNEL_INSTANCE_JWTfield from serverConfigand its redactedLogValueentry.InstanceNameandMetroNameremain for relay auth and OTLP resource stamping. The env var can still exist on the VM for other components (e.g. envoy wrapper); the API server no longer reads it for OTLP.Reviewed by Cursor Bugbot for commit ec5597d. Bugbot is set up for automated code reviews on this repo. Configure here.