Skip to content

[Automated] Draft docs (kagent): feat(telemetry): Weaver live-check, registry catch-up, reference… - #601

Closed
github-actions[bot] wants to merge 1 commit into
mainfrom
pr-tracker-draft-kagent-3107
Closed

github-actions[bot] wants to merge 1 commit into
mainfrom
pr-tracker-draft-kagent-3107

Conversation

@github-actions

Copy link
Copy Markdown

kagent-dev/kagent#3107 — feat(telemetry): Weaver live-check, registry catch-up, reference collector

Docs versions: 1.x (first shipped in v1.0.0-alpha10)

What changes for users: Runtime telemetry now puts the top-level agent identity on resource attributes as gen_ai.main_agent.name and gen_ai.main_agent.id, while request spans keep invoked-agent identity in gen_ai.agent.*. Operators can enable OTLP metrics from agent runtimes and query gen_ai.invoke_agent.duration for invocation latency.

What the docs now say: The Tracing page now separates span attributes from resource attributes. The Metrics, OTel stack, and Lightweight OTel stack pages now enable runtime metrics, derive invocation duration from invoke_agent spans before sampling, and show the Prometheus query for invocation latency.

Not verified: No cluster; the steps are unrun.

How kagent-3107 was drafted, and what was not verified

Plan, and what changed it

  • Planned: Update the observability pages that describe runtime telemetry, collector setup, and metrics.
  • Learned: body.md and the code diff show both a resource-attribute rename and a new gen_ai.invoke_agent.duration metric; diff-truncated.md shows only tests were cut off.
  • Did: Updated the tracing, metrics, full OTel stack, and lightweight OTel stack pages; skipped the Helm reference target because the change needed prose and runnable stack configuration.

Why a documentation change is necessary

The pull request changes what readers see in their telemetry backends: the runtime identity kagent.runtime moved to the resource, and the main agent identity now uses gen_ai.main_agent.* on resources. The existing tracing page still described kagent.runtime as a request attribute. The pull request also adds the gen_ai.invoke_agent.duration metric and a reference collector connector that derives the metric from invoke_agent spans, but the stack guides did not enable runtime metrics or configure that connector.

What changed on disk

  • docs-site/content/kagent/1.x/observability/tracing.md — Clarified span attributes versus resource attributes for agent identity and runtime identity.
  • docs-site/content/kagent/1.x/observability/metrics.md — Added runtime OTLP metrics, the invocation-duration metric reference, and a Prometheus query.
  • docs-site/content/kagent/1.x/observability/otel-stack.md — Enabled runtime metrics and added the signaltometrics connector to derive invocation duration before sampling.
  • docs-site/content/kagent/1.x/observability/lightweight-otel-stack.md — Enabled runtime metrics and added the same signaltometrics connector to the lightweight collector.

How to verify this change

  1. Follow the OTel stack guide through Send kagent telemetry to the collector, including the collector signaltometrics connector and otel.metrics.enabled: true in the kagent Helm values.
  2. Create a new Session and send a request as shown in Send a request.
  3. In Prometheus, run histogram_quantile(0.95, sum by (le, gen_ai_agent_name) (rate(gen_ai_invoke_agent_duration_seconds_bucket[5m]))); the query shows invocation latency series for the agent that served the request.
  4. In Tempo or Jaeger, open the request trace for the agent service. Request spans show gen_ai.agent.* attributes, and the runtime resource shows gen_ai.main_agent.* and kagent.runtime.

What was not verified

  • The configuration was not applied to a cluster, so every command in this diff is unrun.
  • the code diff was truncated before go/pkg/tracing/invocation_test.go, go/pkg/tracing/runtime_test.go, and python/packages/kagent-core/tests/test_a2a_telemetry.py; no documentation claim depends on those cut-off files.
How the pages were chosen (triage report)

Draft a documentation change

kagent-dev/kagent#3107 — feat(telemetry): Weaver live-check, registry catch-up, reference collector

  • Product: kagent (upstream) → kagent-dev/website
  • Docs version: 1.x
  • Summary (the title; the body has no description): Weaver live-check, registry catch-up, reference collector
  • Not tested: no cluster was available, so every command in this diff is unrun
  • No rule covers 27 touched file(s). The candidate pages below come from the files that DID match a rule, so they may scope this narrower than the change is. Read these before deciding the change is only about the pages listed:
    • .gitignore
    • DEVELOPMENT.md
    • Makefile
    • docs/architecture/telemetry-contract.md
    • docs/architecture/telemetry.md
    • docs/env.md
    • ...and 21 more

Warnings

Draft branch: pr-tracker-draft-kagent-3107


Important

The test suite has not run on this pull request. It was opened by github-actions[bot], and GitHub does not start workflows for pull requests opened with the repository's own token. This repository's checks are held as action_required until somebody presses Approve and run on the Checks tab. Checks that come from GitHub Apps rather than Actions, such as DCO, run on their own.

Please approve the checks before reviewing the content: an absence of failures here means the tests have not run, not that they passed.

…registry catch-up, reference…

Signed-off-by: GitHub Action <action@github.com>
@Rachael-Graham

Copy link
Copy Markdown
Contributor

Closing this as superseded. The resource-attribute split, the runtime metrics section, the otel.metrics Helm row, and the invocation-latency query all landed on main in "Document kagent telemetry resource attributes and runtime metrics", which is why this branch now conflicts.

The one piece not on main is the signaltometrics connector, and it needs work before it ships: the copy omits the metric-type block, so the collector fails config validation and will not start. The upstream reference uses an exponential histogram, which the prometheus exporter renders as a native histogram with no _bucket series, so the documented histogram_quantile query would still return nothing. Tracking that as a separate issue.

@Rachael-Graham
Rachael-Graham deleted the pr-tracker-draft-kagent-3107 branch October 14, 2026 15:19
@Rachael-Graham

Copy link
Copy Markdown
Contributor

Closing this as superseded. The resource-attribute split, the runtime metrics section, the otel.metrics Helm row, and the invocation-latency query all landed on main in #592, which is why this branch now conflicts.

The one piece not already on main was the signaltometrics connector, and it did not work as written: the metric-type block was dropped, so the collector fails config validation with exactly one of the metrics must be defined, 0 found and never starts. Copying the upstream reference verbatim would not have fixed the guides either, because its exponential histogram reaches the prometheus exporter as a native histogram, which exposes only an le="+Inf" bucket in the text format these guides scrape, leaving histogram_quantile with nothing to interpolate.

Both stack guides now get the connector with an explicit bucket histogram instead, verified against collector 0.159.0. Thanks for flagging the gap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants