Document Agent guardrails; fix stale AIAM pages - #432
Open
c1-squire-dev[bot] wants to merge 11 commits into
Open
Document Agent guardrails; fix stale AIAM pages#432c1-squire-dev[bot] wants to merge 11 commits into
c1-squire-dev[bot] wants to merge 11 commits into
Conversation
Ties together the existing controls that bound AI agents acting on the org's behalf (toolsets/access profiles, tool call hooks, kill switches, client lifecycle, step-up auth, the Agent policy step, and audit logging) into a single overview page, mirroring the structure of the Functions overview page. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
The first pass was purely conceptual. Restructure into a three-page set mirroring the Functions docs (overview / hands-on / reference): - agent-guardrails.mdx: tighten to an overview with a layer table (what each layer constrains, where it's configured, how it fails) - agent-guardrails-configure.mdx: six-step walkthrough with real UI paths, a worked built-in hook config, three TypeScript guardrail function examples, an observe-mode rollout pattern, and a kill-switch scope table - agent-guardrails-reference.mdx: hook payload/return contract, filter expression variables, built-in pattern options, audit statuses, constraints, lifecycle defaults, and troubleshooting All claims traced back to the existing mechanism pages; no new product behavior asserted. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
c1-squire-dev
Bot
changed the title
The previous version of these pages used "Agent guardrails" as an umbrella term for existing AIAM mechanisms (toolsets, hooks, kill switches, step-up auth). That name is already taken by a shipped feature with its own object model and admin page, so a reader who followed those pages and then opened AI > Guardrails found a UI that shared nothing with them but the name. Rewrite all three pages to document the actual feature: - agent-guardrails: guardrails as rule cascades over the three risk signals (private data / untrusted content / exfiltration), bindings, surfaces, and how rules relate to tool gates and hooks - agent-guardrails-configure: instantiate from a template, bind, author rules, attach gates and hooks, promote Observe -> Enforce, judge settings - agent-guardrails-reference: object model, rule and gate fields, CEL variables per surface, evaluation order, the built-in template catalog with the mode each ships in, constraints, troubleshooting Also correct the sibling pages these previously copied from, which were stale against the implementation: - tool-call-hooks: hooks live under AI > Guardrails, not C1 Gateway; add the pre-output event, the caller payload block, event-dependent filter variables, the six undocumented built-in patterns, the claw tool_source, corrected pattern defaults, and Managed by guardrails - tools-and-toolsets: classification is one five-value field, not Action/Risk axes; real toolset names and per-connector scope - enable-ai-access-management: empty allowed-client-types means all allowed; default classification is Sensitive; real client-lifecycle labels and defaults; per-server kill switch - ai-clients, aiam-overview: matching lifecycle and toolset fixes Verified against the implementation and by exercising the feature in a dev tenant, rather than against sibling docs. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
c1-squire-dev
Bot
changed the title
Templates/presets are out of scope for this doc set, so remove them rather than describing a surface readers may not have: - overview: drop the template concept row and template-sourced framing from the risk-signals section and the use cases - configure: replace the template-instantiation Step 1 with creating a guardrail directly, fold in the former from-scratch step, and renumber the remaining steps - reference: remove the built-in template catalog, the template object row and diagram edge, the Created from template provenance field, and the template troubleshooting entry Also confirmed these pages name no feature flags. Tenant availability is conveyed only by the standard "Activation required" note used across the AI access management pages. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
The object-model diagram and the CEL examples were wider than the rendered content column at common viewport widths, so they clipped instead of scrolling. - reference: flatten the object-model diagram (it no longer needs the binding arrow inline, now stated in prose) and wrap the CEL examples - configure: convert the CEL variable list from a code block to a table, since tables wrap and code blocks don't; wrap the example Widest code-block line is now 44 characters. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
The built-in patterns table packed each pattern's full default list into its "What it does" cell. Rendered in the docs content column that cell wrapped to a dozen lines and broke inline code chips mid-word. Keep the patterns table to one scannable line per pattern, and move configuration and defaults into their own table beside it. The replaces-not-extends warning is now stated once up front instead of repeated per row. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Correctness fixes: - reference: the Constraints table claimed gate filters see only two CEL variables, contradicting the nine-variable table two sections above - reference: rule Name doesn't appear in audit records — records identify a matched rule by its ID - reference: split ctx.tool_kind and ctx.caller_kind into their two actual vocabularies; they are not one shared list - reference: add the 32,768-byte cap on a rule's CEL condition - reference: Managed by can also be System (read-only, C1-curated) New: pre-output hooks were named but never explained, and the stage differs from the tool-call stages in four ways that each fail silently. Added a reference section plus a hooks-page subsection covering: - it is a separate cascade walk; a rule is only eligible if it uses Evaluate hooks AND selects a pre-output hook, so a Block rule never applies to outgoing responses and an ineligible rule falls through - only Block output and Link filter run there; custom function hooks are rejected at save time - ctx.untrusted_class (string) vs the rule's ctx.untrusted_content (ordered level) are the same score under two names and two types, and getting it wrong denies rather than errors - a Block output hook with no surfaces selected is inert Also drops ctx.agent_id from the documented rule-variable set per the doc owner; it remains in the tool gate filter set, which is separate. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
The pages said a guardrail set to "Any surface" can bind to either surface. That's true of the API contract but false in the product: both binding selectors filter on strict equality against their own surface, so an unset guardrail appears in neither and can't be bound from the UI at all. "Applies to" was also documented as a three-way choice. The picker offers only Agent and Gateway; "Any surface" is a display-only label for the unset value, retained for guardrails carried over from the older single-policy model. - overview: a guardrail targets one surface; create one per surface to cover both - configure: the agent selector matches Agent exactly - reference: two-way choice, plus a warning about the unbindable unset state and how to get out of it Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
The "Start from pattern" prefill helper and its six canned patterns are only present when the presets feature is enabled, so they don't belong in the general documentation — same reasoning as dropping templates. - configure: drop the prefill step; the filter is now entered directly - reference: drop the "Start from pattern" field row and the whole "Gate filter patterns" section Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Two gaps. Observe mode was documented without saying where to look, and what it did say was wrong: it pointed readers at AI > Conversations, which is the chat-assistant session history and a different data store. The guardrail evaluator never writes there. Replace it with the real path: decisions are written to the system log as guardrail_decision events, an Observe match carries would_have_fired, and reading them requires a configured system log exporter because there is no admin UI for these events. Documents the fields to filter on, that Observe records are informational severity (so a warnings-and-above filter hides them), and the 365-day default retention. Says plainly that nothing notifies you. Second: a rule can select several tool gates but only the first whose filter matches runs, across the union with unmanaged gates, ordered enabled-then-priority-then-age. That makes Priority load-bearing when filters overlap — two gates with different approvers won't both apply. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Review v3, items 2a and 2b. - A guardrail with no rules, or whose every rule is Disabled, matches nothing and is indistinguishable at call time from an unbound surface - A call that matches no rule writes no guardrail_decision event, so "traffic arrived and matched nothing" and "no traffic arrived" are identical silence in the system log. Added to the evaluation-order section and folded into the Observe troubleshooting entry, which previously gave the right advice without saying why both cases look the same Deliberately does not document a "system baseline rule": the proto comment claiming one is stale — "baseline" is only an audit label applied when no rule matched, not a rule object. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Contributor
Author
Review v3 addressed2a / 2b — done ( No "system baseline rule" documented, per the review — the proto comment claiming one is stale; 1 — gate filter patterns: staying out, by doc-owner decision. The review is factually right and I confirmed it independently: I raised the contradiction with the doc owner rather than restoring unilaterally, and the decision is to keep them out. The gate filter is documented as a CEL expression entered directly, which is correct and complete for authoring a gate; the prefill helper is just undocumented. Re-raise if that changes. Cross-page dependency — already resolved in this PR, not a follow-up. The review flags
That page also gained the pre-output event, event-dependent filter variables, the Still open for product, not docs:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed since the last review
The previous revision of this PR used "Agent guardrails" as an umbrella term for existing AIAM mechanisms (toolsets, hooks, kill switches, step-up auth). That name is already a shipped feature with its own object model (
Guardrail,GuardrailBinding,AgentGuardrailRule,ToolGate) and admin page at AI > Guardrails. A reader who followed those pages and then opened that page would have found a UI sharing nothing with them but the name.All three pages are rewritten to document the actual feature. The sibling pages the old version copied from are corrected too, since they were the source of most of its errors.
The pages
agent-guardrailsagent-guardrails-configureagent-guardrails-referenceScope note: guardrail templates/presets are intentionally not covered.
Sibling-page corrections
tool-call-hooks— hooks live under AI > Guardrails, not C1 Gateway (moved in an earlier PR); adds the pre-output event, thecallerpayload block (with the warning that it is never an authorization input), event-dependent filter variables, the six undocumented built-in patterns (11 exist, 5 were documented), theclawtool_source, corrected pattern defaults, and the Managed by guardrails toggletools-and-toolsets— classification is one five-value field, not "Action (Read/Write/Delete)" + "Risk (Sensitive/Dangerous)" axes; there is no "Delete" value; real toolset names (All approved tools/Read tools) and per-connector scopeenable-ai-access-management— empty allowed-client-types means all types allowed (was documented as personal-only); default classification is Sensitive, not Unclassified; real client-lifecycle labels and defaults; adds the per-server kill switchai-clients,aiam-overview— matching lifecycle and toolset fixesVerification method
The previous revision's stated method — tracing claims to sibling docs — was the root cause of its errors: it propagated their staleness and couldn't detect a collision with an undocumented feature. This revision verified against the implementation and by exercising the feature in a dev tenant.
Caught and fixed during verification:
Deliberately not documented, because they are declared but unimplemented:
Guardrail.default_outcomeat eval time, a system baseline rule on theGuardrailobject, and centrally-patched template distribution. Confirmed unimplemented rather than assumed.Test plan
IF ctx.private_data >= 2 && ctx.exfiltration >= 2→Block/Enforce), opened the rule form, checked the judge settings tab, and deleted the guardrail. All UI labels, helper text, and defaults in these pages are transcribed from that session.npx mintlify broken-links— no new broken links; anchors verified against real headings. The 10 reported are pre-existing in unrelated files.docs.jsonvalidates.For reviewers
Terminology. These pages call the risk axes "signals" and the cascade a "guardrail." If aigov uses different customer-facing words, tell me and I'll align.
Tenant availability is conveyed only by the standard "Activation required" note already used across the AI access management pages. The pages name no internal gating.
🤖 Generated with Claude Code