feat: add observability abstractions and wire to runtime by nborges-aws · Pull Request #2147 · aws/agentcore-cli

nborges-aws · 2026-08-31T15:04:52Z

Description

This PR adds reusable observability infrastructure. The setup is built to apply generally across our primitives, while allowing for resource-specific customization where necessary. This PR wires runtime to the infrastructure. Remaining primitives wiring will be released as a follow pending alignment on the abstractions added here.

Adds a reusable, logs-only ObservabilityClient as the shared API entry point
Adds a source resolver registry, with runtime-specific log-group resolution
Adds a primitive-agnostic CloudWatch source reader for search and live tail
Adds a shared observability handler factory and mounts logs under Runtime
Normalizes provider events into a generic LogRecord

Architecture

Runtime handler → ObservabilityClient → RuntimeSourceResolver → CloudWatchSourceReader → LogRecord

The resolver owns resource-to-log-group translation. The source reader owns CloudWatch mechanics without ever needing knowledge of our resource types. The client is responsible for the orchestration of these layers.

Commands

Tail logs:

agentcore runtime logs --id <runtime-id> --tail

Search logs:

agentcore runtime logs \
    --id <runtime-id> \
    --since 1h \
    --until now \
    --level error \
    --query '"timed out"' \
    --limit 100

--qualifier selects a non-default Runtime endpoint.

Type of Change

Testing

How have you tested the change?

bun run test (2322 pass, 0 fail)
I ran npm run test:unit and npm run test:integ
I ran npm run typecheck
I ran npm run lint
If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

Checklist

I have read the CONTRIBUTING document
I have added any necessary tests that prove my fix is effective or my feature works
I have updated the documentation accordingly
I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
My changes generate no new warnings
Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

agentcore-devx-automation · 2026-08-31T15:07:18Z

Claude Security Review: no high-confidence findings. (run)

codecov-commenter · 2026-08-31T15:08:44Z

Codecov Report

❌ Patch coverage is 98.55700% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.28%. Comparing base (27e3250) to head (c40047e).

Files with missing lines	Patch %	Lines
src/core/observability/insights.ts	91.80%	5 Missing ⚠️
src/handlers/observability/logs.ts	96.80%	3 Missing ⚠️
src/handlers/runtime/traces/get/index.tsx	96.22%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           refactor    #2147      +/-   ##
============================================
+ Coverage     97.25%   97.28%   +0.02%     
============================================
  Files           508      513       +5     
  Lines         33902    34090     +188     
============================================
+ Hits          32972    33164     +192     
+ Misses          930      926       -4

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

agentcore-devx-automation

AgentCore Harness Review

Verdict: Looks good

Nice, well-factored change. The ObservabilityClient / SourceReader split cleanly separates identity resolution from CloudWatch I/O and leaves an obvious extension point for future resource kinds. A few things I looked at carefully:

AWS SDK mocking: sourceReader.test.ts and client.test.ts mock at the SDK client boundary (send) rather than the pieces of internal state, which matches the guidance in the review criteria. No excessive mocking.
Pagination + limit: CloudWatchSourceReader.searchLogs correctly caps the per-page limit at query.limit - yielded, terminates when nextToken === requestToken, and short-circuits on limit <= 0. Edge cases (limit=1, single page, empty page with token) all look right.
Live Tail: tailLogs handles both the in-band SessionTimeoutException event and the thrown variant, reconnects only when timed out, and exits cleanly on abort. Legacy arn:...:* suffix stripping is guarded and covered.
Missing log group: nice consistent ResourceNotFoundError translation with actionable message in both search and tail paths (including the pre-flight DescribeLogGroups case for tail).
Handler wiring: input validation (--tail vs --since/--until, --limit outside search mode, --since > --until) all raise typed InputValidationErrors and are covered in tests. withUserCancellation propagates the abort into the SDK calls.
Telemetry: this codebase currently instruments telemetry at the top-level command run in src/index.ts rather than per-handler, so no per-feature instrumentation is missing here.

Non-blocking observations if you want to iterate later:

--tail is effectively a no-op flag when neither --since nor --until is passed (tailing is already the default in that case). Consider either making search the default with --tail required for streaming, or documenting the current behavior in the flag help. Either is fine, just be intentional.
ResourceFlagValues in handlers/observability/types.ts duplicates the existing FlagsOf in router/handler.tsx. Could reuse or export the router one to keep a single source of truth.
The log group naming convention /aws/bedrock-agentcore/runtimes/<id>-<qualifier> is hard-coded; if the service ever exposes this via an API, worth switching to that to avoid a lurking coupling.

Nothing here blocks merge. Ship it.

AlexanderRichey

There is some good stuff here but I think we can simplify a bit. I like the idea of creating a function that creates these handlers for us, but I think this can be done with a little less abstraction and more directly. It seems like what you want is something like:

const createLogsHandler = (client: ObservabilityClient, io: AppIO) => createHandler(...)

Then use this for Runtime and Harness:

export function createRuntimeHandler(...): Router {
    ...
    runtime.handler(createLogsHandler(obsC, io))
    ...
}

AlexanderRichey · 2026-09-01T02:33:08Z

+} from "./resolver";
+import type { LogSearchQuery, LogTailQuery, RawLogRecord, SourceReader } from "./sourceReader";
+
+export interface LogRecord {


Is this importable from a package somewhere? Is this what all log records look like in AC?

I looked and couldn't find any good source we could import this from. OpenTelemetry logRecord is the closest thing, but wouldn't support any cloudWatch metadata fields

AlexanderRichey · 2026-09-01T02:36:12Z

+  logs: readonly LogSource[];
+}
+
+export interface ObservabilitySourceResolver<R extends ObservableResourceRef> {


Do we really need all of these complex types?

I was able to simplify a decent chunk of this. The resolver registry is now just a record keyed by the resource type, and got rid of ResolvedResourceIdentity. The logical flow stays the same but we now lookup directly with resolvers[resuorce.kind]. So no mapped Extract type, or generic resolver parameters.

AlexanderRichey · 2026-09-01T02:40:24Z

+ * Builds reusable logs command behavior. Primitive routers contribute only
+ * identity flags and conversion to an ObservableResourceRef.
+ */
+export class ObservabilityHandlerFactory implements ObservabilityHandlerFactories {


Interesting. Did you consider writing a function that returns a handler function? That might be simpler.

Good call. I've refactored the handler as a static method

AlexanderRichey · 2026-09-01T02:41:27Z

+  const timestamp = Date.parse(trimmed);
+  if (!Number.isNaN(timestamp)) return timestamp;
+
+  throw new InputValidationError(


Any reason we're not using date-fns?

No reason except it isn't currently in the project. Using it would save us the ISO parsing, but the rest of the logic would still be needed

AlexanderRichey · 2026-09-01T02:41:48Z

+  toResource(flags: ResourceFlagValues<F>): Extract<ObservableResourceRef, { kind: K }>;
+}
+
+export interface ObservabilityHandlerFactories {


What's the upshot of an interface for this?

No reason to keep as interface. I've changed the handlers to come from a static function as suggested

jariy17

I would try to reduce the abstraction by getting rid of the factory. A static function like Alex suggested would reduce the complexity here. Otherwise, this is a very good abstraction that should expand to other AgentCore resources. Thanks for diving deep on this implementation.

jariy17 · 2026-08-31T20:35:48Z

@@ -0,0 +1,144 @@
+import { describe, expect, test } from "bun:test";


We should use Golden tests here. Runtime logs handler tests should cover this.

jariy17 · 2026-08-31T20:47:22Z

+} from "./resolver";
+import type { LogSearchQuery, LogTailQuery, RawLogRecord, SourceReader } from "./sourceReader";
+
+export interface LogRecord {


Since this is ONLY storing data, please declare it as a type.

jariy17 · 2026-08-31T20:57:28Z

+ * knowledge of Runtime or any other AgentCore resource type.
+ */
+export class CloudWatchSourceReader implements SourceReader {
+  constructor(private readonly clients: Pick<AwsClients, "logs">) {}


nit: Since we are only using the Logs Client, it should be logClient.

jariy17 · 2026-08-31T20:58:49Z

@@ -0,0 +1,189 @@
+import { describe, expect, test } from "bun:test";
+import type { LogRecord } from "../../core/observability";


Golden tests please.

Yep golden tests have been added

agentcore-devx-automation · 2026-09-01T20:28:14Z

Claude Security Review: no high-confidence findings. (run)

github-actions Bot added the size/xl PR size: XL label Aug 31, 2026

agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Aug 31, 2026

agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Aug 31, 2026

agentcore-devx-automation Bot reviewed Aug 31, 2026

View reviewed changes

agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Aug 31, 2026

AlexanderRichey requested changes Sep 1, 2026

View reviewed changes

jariy17 requested changes Sep 1, 2026

View reviewed changes

nborges-aws added 4 commits September 1, 2026 15:43

feat: add observability abstractions and wire to runtime

8083268

fix: simplify log handler and resolution classes

11c447f

chore: add observability logs golden fixture test

6f832ff

chore: use shared cloudWatch client in batch eval APIs

c40047e

nborges-aws force-pushed the logs-impl branch from b5e008a to c40047e Compare September 1, 2026 20:24

github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Sep 1, 2026

nborges-aws marked this pull request as ready for review September 1, 2026 20:24

agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026

agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026

		@@ -0,0 +1,144 @@
		import { describe, expect, test } from "bun:test";

		@@ -0,0 +1,189 @@
		import { describe, expect, test } from "bun:test";
		import type { LogRecord } from "../../core/observability";

Uh oh!

Conversation

nborges-aws commented Aug 31, 2026

Description

Architecture

Commands

Type of Change

Testing

Checklist

Uh oh!

agentcore-devx-automation Bot commented Aug 31, 2026

Uh oh!

codecov-commenter commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

agentcore-devx-automation Bot left a comment

Choose a reason for hiding this comment

AgentCore Harness Review

Uh oh!

AlexanderRichey left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jariy17 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agentcore-devx-automation Bot commented Sep 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov-commenter commented Aug 31, 2026 •

edited

Loading