Skip to content

Rlg/audit prompts snippets - #639

Merged
Rachael-Graham merged 3 commits into
mainfrom
rlg/audit-prompts-snippets
Oct 20, 2026
Merged

Rachael-Graham merged 3 commits into
mainfrom
rlg/audit-prompts-snippets

Conversation

@Rachael-Graham

Copy link
Copy Markdown
Contributor

Move the tool-attributes paragraph and the Verify the setup steps of the Audit prompts page into snippets.

Two snippets are new under snippets/observability/. audit-tool-attributes.md holds the paragraph about the gcp.vertex.agent.tool_call_args and tool_response attributes, now split into two paragraphs so the snippet renders as a block. audit-verify-steps.md holds the whole numbered list under Verify the setup, including the Grafana and Jaeger tabs.

The page itself keeps its prose, its procedure, and the existing tracing-backend.md and claude-logs-backend.md calls. Readers see the same text as before.

Worth a close look: the {{< tabs >}} block now renders from inside a reuse snippet, the first snippet in this repo to do so. A built-page diff against main shows whitespace only, plus the tool paragraph split.

Separately, captureContent=true no longer records content on the kagent runtime since kagent#3153. That bug is unrelated to this PR and is not filed yet.

To review, compare the rendered observability/audit-prompts page with production.

Move the tool-attribute disclosure and trace-verification steps into reusable observability snippets, keeping the audit prompts page concise.

Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Docs preview

Link Points at
Branch preview The newest push to this branch. Updates in place.
Commit preview 307c2b3 only. Frozen.

Both are uploaded Worker versions and serve no production traffic.

4. Open the most recent trace, and expand the `generate_content` span. The **Tags** section shows the three attributes described in [What a record holds](#what-a-record-holds).
{{% /tab %}}
{{< /tabs >}}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pre-existing problem, but since the text is moving anyway: both tabs just said the span shows "the three attributes", and then this line says "If both attributes read {}". I think it means the two message attributes. Suggestion:

If gen_ai.input.messages and gen_ai.output.messages read {}, the Session started without content capture.

Also, this line and the "previous section printed Current" reference tie the snippet to this one page. That's fine for a single-use override point. Just know that it isn't reusable elsewhere as written.

```text
{ span.gen_ai.input.messages =~ ".*Audit check.*" }
```
5. Open a trace, and select its `generate_content` span. The **Span Attributes** section shows the three attributes described in [What a record holds](#what-a-record-holds).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small drift from a pure move. "the three attributes that [What a record holds] describes" became "the three attributes described in [What a record holds]" here and on line 29. The PR body says the built-page diff is whitespace only plus the tool paragraph split, so this change is unannounced. It's fine either way. Either revert it to keep the move clean, or mention it in the body so a reviewer comparing against production isn't surprised.

@@ -0,0 +1,3 @@
A tool's own arguments and reply are recorded separately, on the `execute_tool` span of each tool call, in the `gcp.vertex.agent.tool_call_args` and `gcp.vertex.agent.tool_response` attributes. `config.runtimeDefaults.telemetry.captureContent` does not govern those two attributes. The runtime records them whenever tracing is on.

Treat the two tool attributes as part of the audit record, and as a disclosure to account for on an agent whose tools return sensitive data.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the split works, but the new second paragraph reads awkwardly on its own ("Treat the two tool attributes as part of the audit record, and as a disclosure to account for on an agent whose tools return sensitive data"). Maybe:

Treat the two tool attributes as part of the audit record. On an agent whose tools return sensitive data, account for them as a disclosure.

Separately, I'm not sure the split is needed. With the extras reuse rendering inline by default, a single-paragraph snippet at the top level emits bare text without <p>, and the extras CSS doesn't care. If you saw an actual rendering difference, ignore this.

Signed-off-by: Rachael Graham <rachael.graham@solo.io>
@Rachael-Graham
Rachael-Graham merged commit 8b5faf1 into main Oct 20, 2026
4 checks passed
@Rachael-Graham
Rachael-Graham deleted the rlg/audit-prompts-snippets branch October 20, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants