Repository navigation
Rlg/audit prompts snippets - #639
Conversation
Move the tool-attribute disclosure and trace-verification steps into reusable observability snippets, keeping the audit prompts page concise. Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Docs preview
Both are uploaded Worker versions and serve no production traffic. |
| 4. Open the most recent trace, and expand the `generate_content` span. The **Tags** section shows the three attributes described in [What a record holds](#what-a-record-holds). | ||
| {{% /tab %}} | ||
| {{< /tabs >}} | ||
|
|
There was a problem hiding this comment.
This is a pre-existing problem, but since the text is moving anyway: both tabs just said the span shows "the three attributes", and then this line says "If both attributes read {}". I think it means the two message attributes. Suggestion:
If
gen_ai.input.messagesandgen_ai.output.messagesread{}, the Session started without content capture.
Also, this line and the "previous section printed Current" reference tie the snippet to this one page. That's fine for a single-use override point. Just know that it isn't reusable elsewhere as written.
| ```text | ||
| { span.gen_ai.input.messages =~ ".*Audit check.*" } | ||
| ``` | ||
| 5. Open a trace, and select its `generate_content` span. The **Span Attributes** section shows the three attributes described in [What a record holds](#what-a-record-holds). |
There was a problem hiding this comment.
Small drift from a pure move. "the three attributes that [What a record holds] describes" became "the three attributes described in [What a record holds]" here and on line 29. The PR body says the built-page diff is whitespace only plus the tool paragraph split, so this change is unannounced. It's fine either way. Either revert it to keep the move clean, or mention it in the body so a reviewer comparing against production isn't surprised.
| @@ -0,0 +1,3 @@ | |||
| A tool's own arguments and reply are recorded separately, on the `execute_tool` span of each tool call, in the `gcp.vertex.agent.tool_call_args` and `gcp.vertex.agent.tool_response` attributes. `config.runtimeDefaults.telemetry.captureContent` does not govern those two attributes. The runtime records them whenever tracing is on. | |||
|
|
|||
| Treat the two tool attributes as part of the audit record, and as a disclosure to account for on an agent whose tools return sensitive data. | |||
There was a problem hiding this comment.
Nit: the split works, but the new second paragraph reads awkwardly on its own ("Treat the two tool attributes as part of the audit record, and as a disclosure to account for on an agent whose tools return sensitive data"). Maybe:
Treat the two tool attributes as part of the audit record. On an agent whose tools return sensitive data, account for them as a disclosure.
Separately, I'm not sure the split is needed. With the extras reuse rendering inline by default, a single-paragraph snippet at the top level emits bare text without <p>, and the extras CSS doesn't care. If you saw an actual rendering difference, ignore this.
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Move the tool-attributes paragraph and the Verify the setup steps of the Audit prompts page into snippets.
Two snippets are new under
snippets/observability/.audit-tool-attributes.mdholds the paragraph about thegcp.vertex.agent.tool_call_argsandtool_responseattributes, now split into two paragraphs so the snippet renders as a block.audit-verify-steps.mdholds the whole numbered list under Verify the setup, including the Grafana and Jaeger tabs.The page itself keeps its prose, its procedure, and the existing
tracing-backend.mdandclaude-logs-backend.mdcalls. Readers see the same text as before.Worth a close look: the
{{< tabs >}}block now renders from inside areusesnippet, the first snippet in this repo to do so. A built-page diff againstmainshows whitespace only, plus the tool paragraph split.Separately,
captureContent=trueno longer records content on thekagentruntime since kagent#3153. That bug is unrelated to this PR and is not filed yet.To review, compare the rendered
observability/audit-promptspage with production.