Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/spec/trace/extend_opentelemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,7 @@ The collector path, set with `ACS_OTEL_ENDPOINT`, sends the same spans over OTLP

The two Guardian outputs fail independently. The first failed write or send disables that output for the rest of the process and prints one line on stderr. The other output keeps running, and the decision goes out unchanged, as the [failure isolation](#failure-isolation) rule below requires. The [Guardian README](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/packages/guardian/README.md#troubleshooting) lists the messages you may see.

The committed [sample trace](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl) holds both streams in one file: five spans from the Guardian, one of them a rejected request, and one from the converter. Every value in it is synthetic.
The committed [sample trace](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl) holds both streams in one file: nine spans from the Guardian, one of them a rejected request, and one from the converter. Every value in it is synthetic.

### What the labels don't prove

Expand Down
4 changes: 3 additions & 1 deletion reference-implementations/agt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,8 @@ To read the collector's file in step 7, run the same command with `.acs/otel-col

You should see two `gen_ai.tool.call` spans, one `allow` and one `deny`, and one `gen_ai.tool.result` span with `allow`. The spans carry the attributes [`trace/otel-mapping.json`](../../specification/v0.1.0/trace/otel-mapping.json) lists, and the [Guardian README](packages/guardian/README.md#export-traces) says which ones appear when.

You won't see the session and message spans (`acs.session`, `acs.message.user`, `acs.message.agent`, `acs.session.end`) in this run. The exporter can write them, but this Guardian doesn't evaluate those steps yet, so it doesn't span them, and the Claude Code shim doesn't send them. The [committed sample](samples/trace/acs-trace-sample.jsonl) shows all four.

**6. Capture the failures only the host sees.** With the Guardian stopped, send one more step. The shim can't reach the Guardian, so it proceeds under the posture the Guardian declared when the session started, which is `proceed` for the stock Guardian, and records that in `.acs/audit.jsonl`. The converter turns that record into a span in the same format.

```bash
Expand Down Expand Up @@ -312,7 +314,7 @@ Verified on macOS (arm64): natively with otelcol-contrib 0.162.0, and with the D

**Run it again.** Every output here except the converter's appends, so a second run on the same `.acs` doubles every count, and the converter refuses to run at all. Stop the Guardian and the collector first, with Ctrl-C or `docker stop acs-otelcol`. A process that is still running keeps writing into the deleted file, and its spans are lost without an error. Then delete it: `rm -rf .acs && mkdir -p .acs && chmod 700 .acs`.

**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability`, because the shim doesn't send it yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). They carry `acs.request_id_ref` on a result when the host supplied its call id. The committed sample drives the Guardian directly, so it carries `acs.capability`, and `acs.request_id_ref` on its results. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229).
**What this is, and isn't.** In a live run, the exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It can also write spans for `steps/sessionStart`, `steps/userMessage`, `steps/agentResponse` and `steps/sessionEnd`, which the committed sample shows, but a live run has none until the Guardian handles those steps ([issue #322](https://github.com/GenAI-Security-Project/agent-control-standard/issues/322)). It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability`, because the shim doesn't send it yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). They carry `acs.request_id_ref` on a result when the host supplied its call id. The committed sample drives the Guardian directly, so it carries `acs.capability`, and `acs.request_id_ref` on its results. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229).

The trace files, the audit log and the collector's port can be written by any process running as you, the observed agent included, and in a container by other containers on the same Docker network, so the output isn't tamper-evident. `service.name` (`acs-reference-guardian`, `acs-reference-audit-converter`) is a label each writer sets for itself, not authentication, and loopback isn't authentication either. These labels belong to the reference implementation, not to the ACS specification.

Expand Down
4 changes: 2 additions & 2 deletions reference-implementations/agt/packages/guardian/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,13 +122,13 @@ It prints how many spans reached the file and exits non-zero when that is fewer

### Sample trace

[`samples/trace/acs-trace-sample.jsonl`](../../samples/trace/acs-trace-sample.jsonl) is what this export looks like, so you can read it without running the Guardian. It holds five steps and one rejected request: an allowed tool call and its result, linked by `acs.request_id_ref`, a denied call carrying reasoning, reason codes, policy references and provenance, a forged request that reuses the allowed call's ids and fails the schema, recorded as a `guardian.rejected_request` span so the allowed call keeps the only span under its pair, a decision that finished after the timeout, and a step the host could not deliver, converted from its audit log. The decisions come from the stock policy, and the one late decision is delayed on purpose. Every value in it is synthetic.
[`samples/trace/acs-trace-sample.jsonl`](../../samples/trace/acs-trace-sample.jsonl) is what this export looks like, so you can read it without running the Guardian. It holds five steps and one rejected request: an allowed tool call and its result, linked by `acs.request_id_ref`, a denied call carrying reasoning, reason codes, policy references and provenance, a forged request that reuses the allowed call's ids and fails the schema, recorded as a `guardian.rejected_request` span so the allowed call keeps the only span under its pair, a decision that finished after the timeout, and a step the host could not deliver, converted from its audit log. The decisions come from the stock policy, and the one late decision is delayed on purpose. It also holds one span for each session and message step: `acs.session`, `acs.message.user` (with content types and provenance), `acs.message.agent` and `acs.session.end`. This Guardian does not dispatch those steps yet, so the script hands them to its exporter through `methodsEvaluated`, and each one carries a `guardian_error` failure rather than a decision. Every value in it is synthetic.

```bash
bun run sample:trace
```

That command regenerates it. `test/sample-trace.test.ts` checks the sample against `otel-mapping.json` and fails when the exporter's output no longer matches it, so a change to the exporter or the mapping comes with a regenerated sample. The converted span has no `gen_ai.tool.name`, because the audit log does not record the tool (issue #274). Like the exporter, the sample covers two hook methods and is not an ACS-Trace conformance claim.
That command regenerates it. `test/sample-trace.test.ts` checks the sample against `otel-mapping.json` and fails when the exporter's output no longer matches it, so a change to the exporter or the mapping comes with a regenerated sample. The converted span has no `gen_ai.tool.name`, because the audit log does not record the tool (issue #274). The sample is not an ACS-Trace conformance claim.

### Troubleshooting

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,15 @@ function policyColumn(references: unknown, field: string): string[] | undefined
return references.map((reference) => asString(asObject(reference)?.[field]) ?? "");
}

/** `content[].type` of a message payload, one entry per item in the wire's
* order and not deduplicated, so a reader can count the items of each type.
* An item without a string `type` adds nothing. Absent when no item has one. */
function contentTypes(content: unknown): string[] | undefined {
if (!Array.isArray(content)) return undefined;
const types = content.flatMap((item) => asString(asObject(item)?.type) ?? []);
return types.length === 0 ? undefined : types;
}

/** The parts of a request envelope the source tables read. Read off the
* parsed JSON rather than the validated envelope, because the span starts
* before validation runs and an envelope that fails the schema is still a
Expand Down Expand Up @@ -271,12 +280,18 @@ export type Source<Parts> = (parts: Parts) => AttributeValue | undefined;
* attribute `step_to_span` can name on a span this Guardian emits. The
* mapping decides WHICH of these a span carries; this table only says HOW
* each one is read. Paths follow the hook payload schemas
* (`hooks/tool-call-request.json`, `hooks/tool-call-result.json`) and
* `request-envelope.json`'s `AcsParams.metadata`.
* (`hooks/tool-call-request.json`, `hooks/tool-call-result.json`,
* `hooks/session-start.json`, `hooks/session-end.json`,
* `hooks/user-message.json`) and `request-envelope.json`'s `AcsParams`.
*/
export const STEP_ATTRIBUTE_SOURCES: Readonly<Record<string, Source<RequestParts>>> = {
"acs.session.id": (r) => asString(r.metadata.session_id),
"acs.agent.id": (r) => asString(r.metadata.agent_id),
"acs.tenant_id": (r) => asString(r.params.tenant_id),
"acs.user.id": (r) => asString(asObject(r.payload.user_identity)?.user_id),
"acs.policy_mode": (r) => asString(r.payload.policy_mode),
"acs.session.reason": (r) => asString(r.payload.reason),
"acs.content.types": (r) => contentTypes(r.payload.content),
"gen_ai.tool.name": (r) => asString(r.tool.name),
"acs.tool.provider": (r) => asString(r.tool.provider),
"acs.tool.version": (r) => asString(r.tool.version),
Expand Down Expand Up @@ -371,13 +386,17 @@ export const PROVENANCE_ATTRIBUTE_SOURCES: Record<string, (provenance: JsonObjec
* bytes' order because `JSON.parse` has already hoisted integer-like
* argument names ahead of the rest (envelope-log-sink.ts notes `{"0": ...}`
* is a real shape), so this is the order every reader of the envelope log
* sees too. The two never meet on one payload (the request gate carries
* `arguments`, the result gate `outputs`), and an argument or output
* without one contributes nothing. */
* sees too. The message hooks attach theirs to each `content` item, in
* array order, and `sessionStart` to its Intent as `parser_provenance`.
* Only one of these appears on any one payload (the request gate carries
* `arguments`, the result gate `outputs`, a message `content`), and a
* carrier without one contributes nothing. */
export function provenanceObjects(payload: JsonObject): JsonObject[] {
const carriers: unknown[] = [
...Object.values(asObject(payload.arguments) ?? {}),
...(Array.isArray(payload.outputs) ? payload.outputs : []),
...(Array.isArray(payload.content) ? payload.content : []),
{ provenance: asObject(payload.intent)?.parser_provenance },
];
const found: JsonObject[] = [];
for (const carrier of carriers) {
Expand Down
Loading
Loading