diff --git a/docs/spec/trace/extend_opentelemetry.md b/docs/spec/trace/extend_opentelemetry.md index 16c460a4..038f4678 100644 --- a/docs/spec/trace/extend_opentelemetry.md +++ b/docs/spec/trace/extend_opentelemetry.md @@ -238,7 +238,7 @@ The collector path, set with `ACS_OTEL_ENDPOINT`, sends the same spans over OTLP The two Guardian outputs fail independently. The first failed write or send disables that output for the rest of the process and prints one line on stderr. The other output keeps running, and the decision goes out unchanged, as the [failure isolation](#failure-isolation) rule below requires. The [Guardian README](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/packages/guardian/README.md#troubleshooting) lists the messages you may see. -The committed [sample trace](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl) holds both streams in one file: five spans from the Guardian, one of them a rejected request, and one from the converter. Every value in it is synthetic. +The committed [sample trace](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl) holds both streams in one file: nine spans from the Guardian, one of them a rejected request, and one from the converter. Every value in it is synthetic. ### What the labels don't prove diff --git a/reference-implementations/agt/README.md b/reference-implementations/agt/README.md index df8ae404..b95a9d13 100644 --- a/reference-implementations/agt/README.md +++ b/reference-implementations/agt/README.md @@ -262,6 +262,8 @@ To read the collector's file in step 7, run the same command with `.acs/otel-col You should see two `gen_ai.tool.call` spans, one `allow` and one `deny`, and one `gen_ai.tool.result` span with `allow`. The spans carry the attributes [`trace/otel-mapping.json`](../../specification/v0.1.0/trace/otel-mapping.json) lists, and the [Guardian README](packages/guardian/README.md#export-traces) says which ones appear when. +You won't see the session and message spans (`acs.session`, `acs.message.user`, `acs.message.agent`, `acs.session.end`) in this run. The exporter can write them, but this Guardian doesn't evaluate those steps yet, so it doesn't span them, and the Claude Code shim doesn't send them. The [committed sample](samples/trace/acs-trace-sample.jsonl) shows all four. + **6. Capture the failures only the host sees.** With the Guardian stopped, send one more step. The shim can't reach the Guardian, so it proceeds under the posture the Guardian declared when the session started, which is `proceed` for the stock Guardian, and records that in `.acs/audit.jsonl`. The converter turns that record into a span in the same format. ```bash @@ -312,7 +314,7 @@ Verified on macOS (arm64): natively with otelcol-contrib 0.162.0, and with the D **Run it again.** Every output here except the converter's appends, so a second run on the same `.acs` doubles every count, and the converter refuses to run at all. Stop the Guardian and the collector first, with Ctrl-C or `docker stop acs-otelcol`. A process that is still running keeps writing into the deleted file, and its spans are lost without an error. Then delete it: `rm -rf .acs && mkdir -p .acs && chmod 700 .acs`. -**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability`, because the shim doesn't send it yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). They carry `acs.request_id_ref` on a result when the host supplied its call id. The committed sample drives the Guardian directly, so it carries `acs.capability`, and `acs.request_id_ref` on its results. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229). +**What this is, and isn't.** In a live run, the exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It can also write spans for `steps/sessionStart`, `steps/userMessage`, `steps/agentResponse` and `steps/sessionEnd`, which the committed sample shows, but a live run has none until the Guardian handles those steps ([issue #322](https://github.com/GenAI-Security-Project/agent-control-standard/issues/322)). It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability`, because the shim doesn't send it yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). They carry `acs.request_id_ref` on a result when the host supplied its call id. The committed sample drives the Guardian directly, so it carries `acs.capability`, and `acs.request_id_ref` on its results. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229). The trace files, the audit log and the collector's port can be written by any process running as you, the observed agent included, and in a container by other containers on the same Docker network, so the output isn't tamper-evident. `service.name` (`acs-reference-guardian`, `acs-reference-audit-converter`) is a label each writer sets for itself, not authentication, and loopback isn't authentication either. These labels belong to the reference implementation, not to the ACS specification. diff --git a/reference-implementations/agt/packages/guardian/README.md b/reference-implementations/agt/packages/guardian/README.md index ec7c864d..34b3192f 100644 --- a/reference-implementations/agt/packages/guardian/README.md +++ b/reference-implementations/agt/packages/guardian/README.md @@ -122,13 +122,13 @@ It prints how many spans reached the file and exits non-zero when that is fewer ### Sample trace -[`samples/trace/acs-trace-sample.jsonl`](../../samples/trace/acs-trace-sample.jsonl) is what this export looks like, so you can read it without running the Guardian. It holds five steps and one rejected request: an allowed tool call and its result, linked by `acs.request_id_ref`, a denied call carrying reasoning, reason codes, policy references and provenance, a forged request that reuses the allowed call's ids and fails the schema, recorded as a `guardian.rejected_request` span so the allowed call keeps the only span under its pair, a decision that finished after the timeout, and a step the host could not deliver, converted from its audit log. The decisions come from the stock policy, and the one late decision is delayed on purpose. Every value in it is synthetic. +[`samples/trace/acs-trace-sample.jsonl`](../../samples/trace/acs-trace-sample.jsonl) is what this export looks like, so you can read it without running the Guardian. It holds five steps and one rejected request: an allowed tool call and its result, linked by `acs.request_id_ref`, a denied call carrying reasoning, reason codes, policy references and provenance, a forged request that reuses the allowed call's ids and fails the schema, recorded as a `guardian.rejected_request` span so the allowed call keeps the only span under its pair, a decision that finished after the timeout, and a step the host could not deliver, converted from its audit log. The decisions come from the stock policy, and the one late decision is delayed on purpose. It also holds one span for each session and message step: `acs.session`, `acs.message.user` (with content types and provenance), `acs.message.agent` and `acs.session.end`. This Guardian does not dispatch those steps yet, so the script hands them to its exporter through `methodsEvaluated`, and each one carries a `guardian_error` failure rather than a decision. Every value in it is synthetic. ```bash bun run sample:trace ``` -That command regenerates it. `test/sample-trace.test.ts` checks the sample against `otel-mapping.json` and fails when the exporter's output no longer matches it, so a change to the exporter or the mapping comes with a regenerated sample. The converted span has no `gen_ai.tool.name`, because the audit log does not record the tool (issue #274). Like the exporter, the sample covers two hook methods and is not an ACS-Trace conformance claim. +That command regenerates it. `test/sample-trace.test.ts` checks the sample against `otel-mapping.json` and fails when the exporter's output no longer matches it, so a change to the exporter or the mapping comes with a regenerated sample. The converted span has no `gen_ai.tool.name`, because the audit log does not record the tool (issue #274). The sample is not an ACS-Trace conformance claim. ### Troubleshooting diff --git a/reference-implementations/agt/packages/guardian/src/otel-mapping.ts b/reference-implementations/agt/packages/guardian/src/otel-mapping.ts index 0d9b900b..94792b29 100644 --- a/reference-implementations/agt/packages/guardian/src/otel-mapping.ts +++ b/reference-implementations/agt/packages/guardian/src/otel-mapping.ts @@ -239,6 +239,15 @@ function policyColumn(references: unknown, field: string): string[] | undefined return references.map((reference) => asString(asObject(reference)?.[field]) ?? ""); } +/** `content[].type` of a message payload, one entry per item in the wire's + * order and not deduplicated, so a reader can count the items of each type. + * An item without a string `type` adds nothing. Absent when no item has one. */ +function contentTypes(content: unknown): string[] | undefined { + if (!Array.isArray(content)) return undefined; + const types = content.flatMap((item) => asString(asObject(item)?.type) ?? []); + return types.length === 0 ? undefined : types; +} + /** The parts of a request envelope the source tables read. Read off the * parsed JSON rather than the validated envelope, because the span starts * before validation runs and an envelope that fails the schema is still a @@ -271,12 +280,18 @@ export type Source = (parts: Parts) => AttributeValue | undefined; * attribute `step_to_span` can name on a span this Guardian emits. The * mapping decides WHICH of these a span carries; this table only says HOW * each one is read. Paths follow the hook payload schemas - * (`hooks/tool-call-request.json`, `hooks/tool-call-result.json`) and - * `request-envelope.json`'s `AcsParams.metadata`. + * (`hooks/tool-call-request.json`, `hooks/tool-call-result.json`, + * `hooks/session-start.json`, `hooks/session-end.json`, + * `hooks/user-message.json`) and `request-envelope.json`'s `AcsParams`. */ export const STEP_ATTRIBUTE_SOURCES: Readonly>> = { "acs.session.id": (r) => asString(r.metadata.session_id), "acs.agent.id": (r) => asString(r.metadata.agent_id), + "acs.tenant_id": (r) => asString(r.params.tenant_id), + "acs.user.id": (r) => asString(asObject(r.payload.user_identity)?.user_id), + "acs.policy_mode": (r) => asString(r.payload.policy_mode), + "acs.session.reason": (r) => asString(r.payload.reason), + "acs.content.types": (r) => contentTypes(r.payload.content), "gen_ai.tool.name": (r) => asString(r.tool.name), "acs.tool.provider": (r) => asString(r.tool.provider), "acs.tool.version": (r) => asString(r.tool.version), @@ -371,13 +386,17 @@ export const PROVENANCE_ATTRIBUTE_SOURCES: Record { // or that attribute quietly stops appearing when the mapping changes -- // which is what #227's conditional_attributes would have done. it("has a source for every attribute the real mapping lists on an evaluated method, and for every decision attribute", () => { - for (const method of METHODS_EVALUATED) { + for (const method of [...METHODS_EVALUATED, ...SESSION_AND_MESSAGE_METHODS]) { const entry = MAPPING.stepToSpan[method]; if (entry === undefined) { continue; @@ -629,6 +632,200 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }); }); +// #322 slice 1: the session and message steps. This Guardian does not +// evaluate them yet, so these tests hand them to the exporter through +// methodsEvaluated, as the issue describes. +describe("createOtelTraceExporter -- the session and message steps", () => { + function sessionAndMessageExporter() { + return memoryExporter({ methodsEvaluated: [...METHODS_EVALUATED, ...SESSION_AND_MESSAGE_METHODS] }); + } + + /** makeEnvelope fixes `agent_id` and has no option for `tenant_id`, which + * sits on AcsParams beside `metadata`. */ + function stepEnvelope( + method: string, + payload: Record, + { sessionId, requestId, agentId, tenantId }: { sessionId?: string; requestId?: string; agentId?: string; tenantId?: string } = {}, + ): Record { + const envelope = makeEnvelope(method, payload, { sessionId, requestId }); + const params = envelope.params as Record; + return { + ...envelope, + params: { + ...params, + ...(tenantId === undefined ? {} : { tenant_id: tenantId }), + metadata: { ...(params.metadata as Record), ...(agentId === undefined ? {} : { agent_id: agentId }) }, + }, + }; + } + + /** The one span a single exchange produced, after checking that it is the + * only one and has the name the method maps to, so a test that looks for + * a missing attribute cannot pass because the span itself is missing. */ + async function onlySpan(method: string, envelope: Record, response: unknown = decisionResponse(1, { decision: "allow" })) { + const { exporter, spans } = sessionAndMessageExporter(); + exporter.end(exporter.start(envelope, method, TRUSTED), response); + await exporter.flush(); + expect(spans()).toHaveLength(1); + const [span] = spans(); + expect(span?.name).toBe(MAPPING.stepToSpan[method]!.span_name); + return span!; + } + + const cases: { method: string; span: string; envelope: (ids: { sessionId: string; requestId: string }) => Record; attributes: Record }[] = [ + { + method: "steps/sessionStart", + span: "acs.session", + envelope: (ids) => + stepEnvelope("steps/sessionStart", { user_identity: { user_id: "user-1" }, policy_mode: "strict" }, { ...ids, tenantId: "tenant-1" }), + attributes: { "acs.tenant_id": "tenant-1", "acs.user.id": "user-1", "acs.policy_mode": "strict" }, + }, + { + method: "steps/sessionEnd", + span: "acs.session.end", + envelope: (ids) => stepEnvelope("steps/sessionEnd", { reason: "completed" }, ids), + attributes: { "acs.session.reason": "completed" }, + }, + { + method: "steps/userMessage", + span: "acs.message.user", + envelope: (ids) => + stepEnvelope( + "steps/userMessage", + { content: [{ type: "text", value: "hi" }, { type: "image", value: "aGk=" }, { type: "text", value: "again" }] }, + ids, + ), + attributes: { "acs.content.types": ["text", "image", "text"] }, + }, + { + method: "steps/agentResponse", + span: "acs.message.agent", + envelope: (ids) => stepEnvelope("steps/agentResponse", { content: [{ type: "text", value: "done" }] }, { ...ids, agentId: "responder-7" }), + attributes: { "acs.agent.id": "responder-7" }, + }, + ]; + + for (const { method, span: spanName, envelope, attributes } of cases) { + it(`${method} becomes one ${spanName} span carrying every attribute the mapping lists and the wire supplied`, async () => { + const sessionId = crypto.randomUUID(); + const requestId = crypto.randomUUID(); + const span = await onlySpan(method, envelope({ sessionId, requestId })); + + expect(span.name).toBe(spanName); + expect(span.attributes["acs.session.id"]).toBe(sessionId); + expect(span.attributes["acs.request_id"]).toBe(requestId); + for (const [name, value] of Object.entries(attributes)) { + expect({ name, value: span.attributes[name] }).toEqual({ name, value }); + } + for (const name of MAPPING.stepToSpan[method]!.required_attributes) { + expect({ name, present: span.attributes[name] !== undefined }).toEqual({ name, present: true }); + } + }); + } + + it("leaves off the session span's optional attributes when the wire does not carry them", async () => { + const span = await onlySpan("steps/sessionStart", stepEnvelope("steps/sessionStart", {})); + + for (const name of ["acs.tenant_id", "acs.user.id", "acs.policy_mode"]) { + expect({ name, present: name in span.attributes }).toEqual({ name, present: false }); + } + }); + + // One entry per content item, in wire order, so a reader can count the + // images in a message. An item without a string type adds nothing, and a + // message that leaves no type at all gets no attribute. + const contentCases: { label: string; content: unknown; want: string[] | undefined }[] = [ + { label: "no content field", content: undefined, want: undefined }, + { label: "an empty content array", content: [], want: undefined }, + { label: "content that is not an array", content: "hi", want: undefined }, + { label: "only items without a string type", content: [{ value: "hi" }, { type: 7, value: "x" }], want: undefined }, + { label: "a mix of typed and untyped items", content: [{ type: "file", value: "a" }, { value: "b" }, { type: "audio", value: "c" }], want: ["file", "audio"] }, + ]; + + for (const { label, content, want } of contentCases) { + it(`sets acs.content.types for ${label} to ${JSON.stringify(want)}`, async () => { + const span = await onlySpan("steps/userMessage", stepEnvelope("steps/userMessage", content === undefined ? {} : { content })); + + expect(span.attributes["acs.content.types"]).toEqual(want); + }); + } + + // The provenance rule is the tool gates' (one column per attribute, one + // element per Provenance object, source_id padded with ""), read from + // where these payloads attach Provenance: each content item, and the + // session Intent's parser_provenance. + it("carries the provenance of each message content item that has one, in content order", async () => { + for (const method of ["steps/userMessage", "steps/agentResponse"]) { + const span = await onlySpan( + method, + stepEnvelope(method, { + content: [ + { type: "text", value: "a", provenance: provenance("user_input", { source_id: "user-1" }) }, + // No provenance: contributes no element to any column. + { type: "text", value: "b" }, + { type: "file", value: "c", provenance: provenance("retrieved") }, + ], + }), + ); + + expect({ method, origin: span.attributes["acs.provenance.origin"] }).toEqual({ method, origin: ["user_input", "retrieved"] }); + expect({ method, sourceId: span.attributes["acs.provenance.source_id"] }).toEqual({ method, sourceId: ["user-1", ""] }); + } + }); + + it("carries the session Intent's parser_provenance on the session span", async () => { + const span = await onlySpan( + "steps/sessionStart", + stepEnvelope("steps/sessionStart", { + intent: { raw: "summarize the report", parser_provenance: provenance("system", { source_id: "intent-parser" }) }, + }), + ); + + expect(span.attributes["acs.provenance.origin"]).toEqual(["system"]); + expect(span.attributes["acs.provenance.source_id"]).toEqual(["intent-parser"]); + }); + + it("carries no provenance attribute on a session or message span whose payload attaches none", async () => { + for (const [method, payload] of [ + ["steps/sessionStart", { intent: { raw: "summarize the report" } }], + ["steps/userMessage", { content: [{ type: "text", value: "hi" }] }], + ] as const) { + const span = await onlySpan(method, stepEnvelope(method, payload)); + + expect({ method, keys: Object.keys(span.attributes).filter((key) => key.startsWith("acs.provenance.")) }).toEqual({ method, keys: [] }); + } + }); + + for (const method of SESSION_AND_MESSAGE_METHODS) { + it(`records the decision on a ${method} span the same way it does on a tool call span`, async () => { + const span = await onlySpan( + method, + stepEnvelope(method, { reason: "completed", content: [{ type: "text", value: "hi" }] }), + decisionResponse(1, { decision: "allow", reason_codes: ["session_closed"] }), + ); + + expect(span.events.map((event) => event.name)).toEqual(["acs.decision"]); + expect(span.events[0]?.attributes?.["acs.decision"]).toBe("allow"); + expect(span.events[0]?.attributes?.["acs.reason_codes"]).toEqual(["session_closed"]); + }); + + it(`records a guardian_error failure on a ${method} span answered with a JSON-RPC error`, async () => { + const span = await onlySpan( + method, + makeEnvelope(method, { reason: "completed", content: [{ type: "text", value: "hi" }] }, { id: 4 }), + { jsonrpc: "2.0", id: 4, error: { code: -32020, message: "evaluation failed" } }, + ); + + expect(span.status.code).toBe(SpanStatusCode.ERROR); + expect(span.events.map((event) => event.name)).toEqual(["acs.decision.failure"]); + const failure = span.events[0]?.attributes; + expect(failure?.["acs.failure.class"]).toBe("guardian_error"); + expect(failure?.["acs.failure.reported_by"]).toBe("guardian"); + expect(failure?.["acs.failure.posture"]).toBe("fail_open"); + }); + } +}); + describe("Guardian trace export wiring", () => { it("emits one span per decided step, carrying the session and the decision event", async () => { const { exporter, spans } = memoryExporter(); diff --git a/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl b/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl index bb226510..daf24925 100644 --- a/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl +++ b/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl @@ -1,2 +1,2 @@ -{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"acs-reference-guardian"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"acs-reference-guardian"},"spans":[{"traceId":"7752aef7c63150fcc88f856974d22f5b","spanId":"7bd24ca6a5dd31f0","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792534253539454676","endTimeUnixNano":"1792534253649000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.capability","value":{"stringValue":"process.exec"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792534253649000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"180a78a6f4b17aa91f6a952195321255","spanId":"003c1193df7f552a","name":"gen_ai.tool.result","kind":1,"startTimeUnixNano":"1792534253650630391","endTimeUnixNano":"1792534253716000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b02"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.exit_status","value":{"stringValue":"success"}},{"key":"acs.request_id_ref","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792534253716000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"13717f291e2368066d2fa3f2434ed2f6","spanId":"2a7efedbb7ee3881","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792534253716796645","endTimeUnixNano":"1792534253780000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b03"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.provenance.origin","value":{"arrayValue":{"values":[{"stringValue":"user_input"},{"stringValue":"agent_generated"}]}}},{"key":"acs.provenance.source_id","value":{"arrayValue":{"values":[{"stringValue":"sample-user-1"},{"stringValue":""}]}}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"deny"}},{"key":"acs.reasoning","value":{"stringValue":"This command was blocked because it matches a destructive-shell-command pattern. Policy: destructive_shell_command_blocked, from AGT's stock bundle (agt_stock). Matched at offset 0."}},{"key":"acs.reason_codes","value":{"arrayValue":{"values":[{"stringValue":"destructive_shell_command_blocked"}]}}},{"key":"acs.policy.ids","value":{"arrayValue":{"values":[{"stringValue":"agt_stock"}]}}},{"key":"acs.policy.versions","value":{"arrayValue":{"values":[{"stringValue":""}]}}},{"key":"acs.policy.rule_ids","value":{"arrayValue":{"values":[{"stringValue":"destructive_shell_command_blocked"}]}}}],"name":"acs.decision","timeUnixNano":"1792534253780000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"2cadee9c2b0236f4cf64ecf54933d071","spanId":"d2666e080ce88014","name":"guardian.rejected_request","kind":1,"startTimeUnixNano":"1792534253779859236","endTimeUnixNano":"1792534253781000000","attributes":[{"key":"guardian.rejected.reason","value":{"stringValue":"envelope_invalid"}},{"key":"guardian.rejected.claimed_session_id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"guardian.rejected.claimed_request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}},{"key":"guardian.rejected.error_code","value":{"intValue":-32010}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":2,"message":"The Guardian could not vouch for this request, so it carries no ACS ids or events."},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"f92f9aa4700acfd946b59a2fe19e489e","spanId":"2e4aa2bc8b522e06","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792534253780950247","endTimeUnixNano":"1792534255851000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b05"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792534255851000000","droppedAttributesCount":0},{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"timeout"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.timeout_ms","value":{"intValue":1500}},{"key":"acs.failure.elapsed_ms","value":{"intValue":2070}},{"key":"acs.failure.payload_bytes","value":{"intValue":367}}],"name":"acs.decision.failure","timeUnixNano":"1792534255851000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257}]}]}]} -{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"acs-reference-audit-converter"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"acs-reference-audit-converter"},"spans":[{"traceId":"8ad7d2634fb9d194b1414a0033d90467","spanId":"8572a4b873e3e58e","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792534255853000000","endTimeUnixNano":"1792534255853000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b06"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"unreachable"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"observed_agent"}},{"key":"acs.failure.detail","value":{"stringValue":"The host could not reach the Guardian."}}],"name":"acs.decision.failure","timeUnixNano":"1792534255853000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The host could not reach the Guardian."},"links":[],"droppedLinksCount":0,"flags":257}]}]}]} +{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"acs-reference-guardian"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"acs-reference-guardian"},"spans":[{"traceId":"9e45991c02d7e7a2db0f68c74b3be36e","spanId":"3eee7872e289b0f2","name":"acs.session","kind":1,"startTimeUnixNano":"1793305800423341536","endTimeUnixNano":"1793305800463000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b07"}},{"key":"acs.tenant_id","value":{"stringValue":"sample-tenant"}},{"key":"acs.user.id","value":{"stringValue":"sample-user-1"}},{"key":"acs.policy_mode","value":{"stringValue":"moderate"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"guardian_error"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.error_code","value":{"intValue":-32011}},{"key":"acs.failure.elapsed_ms","value":{"intValue":39}},{"key":"acs.failure.payload_bytes","value":{"intValue":378}}],"name":"acs.decision.failure","timeUnixNano":"1793305800463000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The Guardian returned an error instead of a decision."},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"0f5008fe443366bcb3662866d141c7af","spanId":"de65bf444e25e305","name":"acs.message.user","kind":1,"startTimeUnixNano":"1793305800462860730","endTimeUnixNano":"1793305800463000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b08"}},{"key":"acs.content.types","value":{"arrayValue":{"values":[{"stringValue":"text"},{"stringValue":"file"}]}}},{"key":"acs.provenance.origin","value":{"arrayValue":{"values":[{"stringValue":"user_input"},{"stringValue":"user_input"}]}}},{"key":"acs.provenance.source_id","value":{"arrayValue":{"values":[{"stringValue":"sample-user-1"},{"stringValue":""}]}}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"guardian_error"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.error_code","value":{"intValue":-32011}},{"key":"acs.failure.elapsed_ms","value":{"intValue":0}},{"key":"acs.failure.payload_bytes","value":{"intValue":544}}],"name":"acs.decision.failure","timeUnixNano":"1793305800463000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The Guardian returned an error instead of a decision."},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"78e869fc2dba3a51b1c06677bb8793ae","spanId":"0c4f0a0c82c8f18f","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1793305800463413615","endTimeUnixNano":"1793305800781000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.capability","value":{"stringValue":"process.exec"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1793305800781000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"36ffe6cbdef4201b436d410a54d1606c","spanId":"44bfc07331839fb1","name":"gen_ai.tool.result","kind":1,"startTimeUnixNano":"1793305800780709349","endTimeUnixNano":"1793305801057000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b02"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.exit_status","value":{"stringValue":"success"}},{"key":"acs.request_id_ref","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1793305801057000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"830adfaf50f99b0834136ede6691546b","spanId":"8563bdca2cc6f9fb","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1793305801057881229","endTimeUnixNano":"1793305801107000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b03"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.provenance.origin","value":{"arrayValue":{"values":[{"stringValue":"user_input"},{"stringValue":"agent_generated"}]}}},{"key":"acs.provenance.source_id","value":{"arrayValue":{"values":[{"stringValue":"sample-user-1"},{"stringValue":""}]}}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"deny"}},{"key":"acs.reasoning","value":{"stringValue":"This command was blocked because it matches a destructive-shell-command pattern. Policy: destructive_shell_command_blocked, from AGT's stock bundle (agt_stock). Matched at offset 0."}},{"key":"acs.reason_codes","value":{"arrayValue":{"values":[{"stringValue":"destructive_shell_command_blocked"}]}}},{"key":"acs.policy.ids","value":{"arrayValue":{"values":[{"stringValue":"agt_stock"}]}}},{"key":"acs.policy.versions","value":{"arrayValue":{"values":[{"stringValue":""}]}}},{"key":"acs.policy.rule_ids","value":{"arrayValue":{"values":[{"stringValue":"destructive_shell_command_blocked"}]}}}],"name":"acs.decision","timeUnixNano":"1793305801107000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"5585caadb60f452c7a87e63a1b244f3f","spanId":"97d9e5ddde516975","name":"guardian.rejected_request","kind":1,"startTimeUnixNano":"1793305801106868040","endTimeUnixNano":"1793305801107000000","attributes":[{"key":"guardian.rejected.reason","value":{"stringValue":"envelope_invalid"}},{"key":"guardian.rejected.claimed_session_id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"guardian.rejected.claimed_request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}},{"key":"guardian.rejected.error_code","value":{"intValue":-32010}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":2,"message":"The Guardian could not vouch for this request, so it carries no ACS ids or events."},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"a8cee406c625c0e110e8c233baee66a8","spanId":"bfa139450458f43d","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1793305801108930717","endTimeUnixNano":"1793305803172000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b05"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1793305803172000000","droppedAttributesCount":0},{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"timeout"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.timeout_ms","value":{"intValue":1500}},{"key":"acs.failure.elapsed_ms","value":{"intValue":2063}},{"key":"acs.failure.payload_bytes","value":{"intValue":367}}],"name":"acs.decision.failure","timeUnixNano":"1793305803172000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"a9bf7728f27cd6265686268b180901c3","spanId":"90d016590b671eb4","name":"acs.message.agent","kind":1,"startTimeUnixNano":"1793305803171912256","endTimeUnixNano":"1793305803172000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b09"}},{"key":"acs.agent.id","value":{"stringValue":"sample-agent"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"guardian_error"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.error_code","value":{"intValue":-32011}},{"key":"acs.failure.elapsed_ms","value":{"intValue":0}},{"key":"acs.failure.payload_bytes","value":{"intValue":361}}],"name":"acs.decision.failure","timeUnixNano":"1793305803172000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The Guardian returned an error instead of a decision."},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"4dde151c7a193f7a6fe089823a08ab05","spanId":"2a6d20fbf8c18247","name":"acs.session.end","kind":1,"startTimeUnixNano":"1793305803172966644","endTimeUnixNano":"1793305803173000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b0a"}},{"key":"acs.session.reason","value":{"stringValue":"completed"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"guardian_error"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.error_code","value":{"intValue":-32011}},{"key":"acs.failure.elapsed_ms","value":{"intValue":0}},{"key":"acs.failure.payload_bytes","value":{"intValue":300}}],"name":"acs.decision.failure","timeUnixNano":"1793305803173000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The Guardian returned an error instead of a decision."},"links":[],"droppedLinksCount":0,"flags":257}]}]}]} +{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"acs-reference-audit-converter"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"acs-reference-audit-converter"},"spans":[{"traceId":"a28e022f7170b305d884999fe90b7d76","spanId":"8d185354c373c124","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1793305803174000000","endTimeUnixNano":"1793305803174000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b06"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"unreachable"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"observed_agent"}},{"key":"acs.failure.detail","value":{"stringValue":"The host could not reach the Guardian."}}],"name":"acs.decision.failure","timeUnixNano":"1793305803174000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The host could not reach the Guardian."},"links":[],"droppedLinksCount":0,"flags":257}]}]}]} diff --git a/reference-implementations/agt/scripts/make-sample-trace.ts b/reference-implementations/agt/scripts/make-sample-trace.ts index daf17591..9fdd8491 100644 --- a/reference-implementations/agt/scripts/make-sample-trace.ts +++ b/reference-implementations/agt/scripts/make-sample-trace.ts @@ -1,8 +1,11 @@ /** * Writes the committed sample trace (issue #266): what the reference * Guardian's OTLP export actually looks like, from a fixed set of synthetic - * requests through the two live hook methods, plus one host-side failure run - * through the audit converter. + * requests through the two live hook methods and the four session and + * message steps (#322), plus one host-side failure run through the audit + * converter. The Guardian does not dispatch the session and message steps + * yet, so this script hands them to its exporter through methodsEvaluated, + * and each of their spans carries a guardian_error failure, not a decision. * * bun run sample:trace * @@ -64,13 +67,29 @@ export const SCENARIOS: readonly Scenario[] = [ { name: "denied", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b03", decision: "deny" }, { name: "late", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b05", decision: "allow" }, { name: "unreachable", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b06" }, + // The session and message steps (#322 slice 1). This Guardian does not + // dispatch them yet and answers each with an error, so none has a decision. + { name: "session-start", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b07" }, + { name: "user-message", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b08" }, + { name: "agent-response", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b09" }, + { name: "session-end", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b0a" }, ]; +/** The session and message steps, which the sample's exporter spans although + * this Guardian does not evaluate them yet, so the sample shows their spans. */ +const SESSION_AND_MESSAGE_METHODS = ["steps/sessionStart", "steps/userMessage", "steps/agentResponse", "steps/sessionEnd"]; + function requestIdOf(name: string): string { return SCENARIOS.find((scenario) => scenario.name === name)!.requestId; } -function envelope(method: string, requestId: string, payload: Record, id: number | null): Record { +function envelope( + method: string, + requestId: string, + payload: Record, + id: number | null, + extraParams: Record = {}, +): Record { return { jsonrpc: "2.0", method, @@ -81,6 +100,7 @@ function envelope(method: string, requestId: string, payload: Record = {}): Record[] { return [ + envelope( + "steps/sessionStart", + requestIdOf("session-start"), + { user_identity: { user_id: "sample-user-1" }, policy_mode: "moderate" }, + 6, + { tenant_id: "sample-tenant" }, + ), + envelope( + "steps/userMessage", + requestIdOf("user-message"), + { + content: [ + { type: "text", value: "Tidy up this directory.", provenance: { provenance_id: "prov-sample-3", origin: "user_input", source_id: "sample-user-1" } }, + { type: "file", value: "notes.txt", provenance: { provenance_id: "prov-sample-4", origin: "user_input" } }, + ], + }, + 7, + ), envelope("steps/toolCallRequest", requestIdOf("allowed"), bash("ls -la", { capability: "process.exec" }), 1), envelope( "steps/toolCallResult", @@ -126,6 +164,8 @@ function requests(): Record[] { // No JSON-RPC id, so it fails the schema, and the allowed call's pair. envelope("steps/toolCallRequest", REJECTED.claimedRequestId, bash("pwd"), null), envelope("steps/toolCallRequest", requestIdOf("late"), bash(`echo ${SLOW_MARKER}`), 5), + envelope("steps/agentResponse", requestIdOf("agent-response"), { content: [{ type: "text", value: "Listed the directory; did not delete it." }] }, 8), + envelope("steps/sessionEnd", requestIdOf("session-end"), { reason: "completed" }, 9), ]; } @@ -166,7 +206,7 @@ export async function generateSampleTrace(outPath: string = SAMPLE_PATH): Promis }; const traceExporter = createOtelTraceExporter({ filePath: guardianTrace, - methodsEvaluated: buildServerHello().methods_evaluated, + methodsEvaluated: [...buildServerHello().methods_evaluated, ...SESSION_AND_MESSAGE_METHODS], timeoutConfig: { default_ms: TIMEOUT_MS }, posture: "fail_open", }); diff --git a/reference-implementations/agt/test/sample-trace.test.ts b/reference-implementations/agt/test/sample-trace.test.ts index 8ca2fda0..b5eb76ec 100644 --- a/reference-implementations/agt/test/sample-trace.test.ts +++ b/reference-implementations/agt/test/sample-trace.test.ts @@ -166,6 +166,32 @@ describe("the committed sample trace matches the mapping", () => { } }); + // The session and message steps (#322 slice 1). This Guardian does not + // dispatch them yet, so each carries a guardian_error failure rather than + // a decision, but its span and the attributes its request carried are real. + it("shows the session and message spans with the attributes their requests carried", () => { + const spanOf = (name: string) => { + const span = spanFor(SCENARIOS.find((scenario) => scenario.name === name)!.requestId); + return { name: span?.name, attributes: attributes(span) }; + }; + + const start = spanOf("session-start"); + expect(start.name).toBe("acs.session"); + expect(start.attributes).toMatchObject({ "acs.tenant_id": "sample-tenant", "acs.user.id": "sample-user-1", "acs.policy_mode": "moderate" }); + + const user = spanOf("user-message"); + expect(user.name).toBe("acs.message.user"); + expect(user.attributes).toMatchObject({ "acs.content.types": ["text", "file"], "acs.provenance.origin": ["user_input", "user_input"] }); + + const agent = spanOf("agent-response"); + expect(agent.name).toBe("acs.message.agent"); + expect(agent.attributes).toMatchObject({ "acs.agent.id": "sample-agent" }); + + const end = spanOf("session-end"); + expect(end.name).toBe("acs.session.end"); + expect(end.attributes).toMatchObject({ "acs.session.reason": "completed" }); + }); + it("links the result to its request through acs.request_id_ref", () => { const result = SCENARIOS.find((scenario) => scenario.requestIdRef !== undefined)!; expect(attributes(spanFor(result.requestId))["acs.request_id_ref"]).toBe(result.requestIdRef); @@ -201,8 +227,9 @@ describe("the committed sample trace matches the mapping", () => { } seen.add(`${failure["acs.failure.class"]}/${failure["acs.failure.reported_by"]}`); } - // The sample holds no step the Guardian answered with an error. - expect([...seen].sort()).toEqual(["timeout/guardian", "unreachable/observed_agent"]); + // The only steps the Guardian answers with an error are the session and + // message steps, which it does not dispatch yet. + expect([...seen].sort()).toEqual(["guardian_error/guardian", "timeout/guardian", "unreachable/observed_agent"]); }); it("holds one span per step, and keeps the forged request apart from the call whose ids it claimed", () => {