Repository navigation
Carry the Guardian's reasoning in additionalContext at the Claude Code request gate - #475
Conversation
…e request gate When several PreToolUse hooks deny one call, Claude Code shows the model the denial reason of only the hook that finishes last, by JSON or by exit 2, and drops the others. A project hook that answers after the shim therefore hides the Guardian's reasoning, though the call stays blocked (issue GenAI-Security-Project#109). An allow's permissionDecisionReason reaches only the debug log, so a fail-open proceed never explained itself to the model either. Claude Code keeps additionalContext from every hook. The hookmap now also writes reasoning there on allow, deny, defer and modify. Measured with the real shim and the reference Guardian under Claude Code 2.1.285 and 2.1.311, beside a project hook that denies by exit 2 and finishes last: before, the policy's reason never reached the model; after, it arrives as hook additional context. The result gate needs no change, because Claude Code passes every hook's block reason to the model. Refs GenAI-Security-Project#109 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Eli Sherman <esherman@credo.ai>
…t#109 comments A test now renders allow, deny, defer and modify from the shipped hookmap with a non-string reasoning and expects neither reason field, so dropping `type: string` from an additionalContext rule fails it. The README's sample shim output shows the new field. The hookmap comment points to the README for the measurements, and the modify comment no longer implies its permissionDecisionReason reaches the model. Refs GenAI-Security-Project#109 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Eli Sherman <esherman@credo.ai>
|
Thanks for this. It is queued rather than ignored. A lead has not reviewed this pull request yet. A lead will read it when they have time, and we can't promise a date. Nothing here is rejected. If a check below fails, please fix it. A pull request that waits a long time may be closed, and you can reopen it later. |
|
This pull request's description says it closes #109, which is on the project roadmap. When it merges, GitHub closes that issue and the roadmap counts it as delivered. Please confirm the change delivers the whole issue before merging. |
bar-capsule
left a comment
There was a problem hiding this comment.
Thanks, this fixes the deny case in #109. Your measurements match Claude Code's hooks docs (https://code.claude.com/docs/en/hooks): an allow's permissionDecisionReason goes only to the debug log, and Claude receives every hook's additionalContext. Please link that page in the new README section.
Three changes:
- Drop
additionalContextfromallow(inline comment). reference-implementations/agt/README.md:193still shows the deny output withoutadditionalContext.- Please change
Closes #109toRefs #109. This fixes the host side, but two points in the issue stay open: whether ACS should say what a host that can dropreasoningmeans for a conformance claim, and the fact that the model can't tell which hook a reason came from. I'll open a spec issue for those.
Review on GenAI-Security-Project#475: under on_decision_failure: proceed, an unreachable Guardian becomes an allow whose reasoning says the Guardian is unreachable and the step went through. As additionalContext, the model would read that before the step runs, and an injected agent could time its next calls by it. The spec points an ALLOW's reasoning at the user, and the audit log and trace export already record every allow. So allow no longer writes additionalContext; deny, defer and modify still do. The fail-open posture test now asserts the reasoning is not in additionalContext. The README links Claude Code's hooks documentation, and the tree README's sample deny output shows the new field. Refs GenAI-Security-Project#109 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Eli Sherman <esherman@credo.ai>
The host's audit log records a fail-open proceed; the Guardian's trace cannot, so the README and hookmap no longer claim both record every allow. A wire-shape comment no longer says an allow's reason reaches the transcript, the per-hook render test is no longer named "exactly as before", and the non-string reasoning matrix covers ask too. Refs GenAI-Security-Project#109 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Eli Sherman <esherman@credo.ai>
bar-capsule
left a comment
There was a problem hiding this comment.
Thanks, all addressed. Approving.
drk-ram
left a comment
There was a problem hiding this comment.
Reviewed at head 5576d2ed836292b28ce64b1413e4f5a2dc0f09b3 (based on current integration, bba1b84). The design is sound and the tests are good. I have one change to ask for, and it is a documentation fix.
Requested change: the root README.md still shows the old deny output.
README.md (JSON on line 63) runs the same demo as reference-implementations/agt/README.md:193 and hosts/claude-code/README.md:32. They show the same rm -rf / deny JSON and then say "That JSON is a real deny from a real Guardian, not a transcript." This PR updates the other two copies. Rock's earlier note about agt/README.md:193 was the same class of problem. The root copy is the third and was missed. After this change the shim emits "additionalContext":"<same string as the reason>" after permissionDecisionReason, so the root README's claim of a real deny from the shim is no longer byte-accurate. It is the front-door example, so please update it to match. git grep -n permissionDecisionReason -- '*.md' finds all three copies.
What I checked.
- Hookmap.
deny,deferandmodifygainhookSpecificOutput.additionalContext: { from: reasoning, type: string }.allowandaskdo not. I agree with both exclusions: a fail-open allow's reasoning says the Guardian is unreachable, and anaskreason goes to the person who answers. Thetype: stringguard on each new line means a non-stringreasoningdrops the field instead of emitting JSON Claude Code cannot read. The new non-stringit.eachpins that for all five decisions. Thedefercase renders asdenyand drops the field, so the table is consistent. - Tests.
wire-shape.test.ts,render-decision.test.tsand theposture.test.tsassertionadditionalContextisundefinedon a fail-open allow all line up with the hookmap. The posture assertion is the one that protects the security property the PR describes, and it is in the right place. - Stale comments. The old comments in the hookmap and
wire-shape.test.tssaid an allow's reason reaches "the transcript a human reads". That contradicts the measurements, and the PR replaces it with the debug-log-only statement. It also adds the "whyadditionalContext" block to the hookmap and the "When another hook answers the same call" section to the host README.
Verification (detached worktrees, Windows).
- Bun 1.3.11 (the version CI pins), frozen-lockfile install:
bun run typecheckpasses.bun test hosts/claude-code packages/host-adaptergives 435 pass, 5 fail. The same command onintegration(bba1b84) gives 430 pass and the same 5 fail, all in thecall_id: the real hookmaps…group. So the PR adds 5 passing cases and no new failure. - Bun 1.4.4: the cases that spawn the shim get empty stdout on Windows, and 53 cases fail. That is a runtime difference: run directly under 1.4.4 the shim exits 0 with nothing on stdout, and the author's description also notes 1.4.x-only failures. I did not diff it against
integrationon 1.4.4. I mention it only so nobody reads the 1.4.x result as a regression from this PR; the 1.3.11 comparison above is the like-for-like one. - I did not run
uv run pytestormkdocs build --strict. The diff touches no file those cover apart from READMEs.
Non-blocking. additionalContext is read by the model with more authority than a tool result. For modify and defer this PR starts putting the Guardian's reasoning there. For a Guardian whose reasoning quotes the matched input, that places attacker-controlled text in the higher-trust channel. This is the same string as permissionDecisionReason, and the README already tells integrators what is written. One sentence in the new README section saying that a Guardian's reasoning should not echo tool input, because it now reaches the model as context, would close it off. A spec issue is the natural home if you would rather not touch the README.
Once the root README.md example matches, I will approve.
What changed
At the request gate, the Claude Code shim now also writes the Guardian's
reasoningtohookSpecificOutput.additionalContextondeny,deferandmodify. With this change, the reasoning reaches the model even when another hook on the same call also denies.Which issue does this implement
Refs #109
meaning, so it needs no issue
Base branch
integration, because this touches the specification, schemas, a referenceimplementation, an adapter, tests, or CI
main, because every changed path is on the documentation lane allowlist inCONTRIBUTING.md
Type of change
Details
What Claude Code does. I measured this on Claude Code 2.1.285 and 2.1.311 with
claude -p, and read the session transcripts. The results agree with Claude Code's hooks documentation. See the comment on #109.PreToolUsehooks deny one call, the model gets the denial reason of only the hook that finishes last. The exit code (JSON deny or exit 2) does not change this. The other reasons are not kept.permissionDecisionReasonof anallowgoes only to Claude Code's debug log.additionalContextfrom every hook. The model reads it asPreToolUse:Bash hook additional context: ….PostToolUse, Claude Code gives the model every hook'sblockreason. The result gate needs no change.Why not exit 2. #109 suggested that a hook that exits 2 wins. In the measurements, finish order decided the result every time, so a change to the exit code does not fix it.
End-to-end check. The test ran the real shim and the reference Guardian, beside a project hook that denies by exit 2 and finishes 1.5 s later, on both Claude Code versions. With the
integrationhookmap, the policy reason did not reach the model. With this change, the policy reason reached the model as additional context.The change is three
additionalContextlines inclaude-code.hookmap.yaml. There is no production TypeScript change.allowdoes not get the field: underon_decision_failure: proceed, a fail-open allow's reasoning says the Guardian is unreachable, and the model must not read that before the step runs. The host's audit log still records a fail-open proceed.askdoes not get the field, because its reason is shown to the person who answers. When the shim is the only hook, the model reads a deny's reasoning twice. The README has a new section, "When another hook answers the same call".Not closed here. Two points in #109 stay open for a spec issue: what a host that can drop
reasoningmeans for a conformance claim, and that the model can't tell which hook a reason came from.Spec fit. §6.1 says
reasoning"serves both end-user display and audit/agent-internal consumption". The change does not alter any disposition, exit code, MODIFY rule or audit record.I tested this
uv run pytest -vpasses on my machine (1205 passed, 1 skipped)uv run mkdocs build --strictpasses on my machineAlso, in
reference-implementations/agt:bun run typecheckpasses.bun test: 1289 pass, 1 skip, 1 fail. The failure ishosts/claude-code/test/posture.test.tsand occurs only on Bun 1.4.x. It also fails onintegration, and CI uses Bun 1.3.11.wire-shape.test.tsandrender-decision.test.tsfirst, and saw them fail before the hookmap change.reasoningand expects no reason field. The test fails iftype: stringis removed from the new lines.additionalContext. It failed beforeallowdropped the field.Checklist
git commit -s(required by the DCO)Security
It changes what the model reads. Deny, defer and modify reasoning now reaches the model as additional context. The request gate keeps an allow's reasoning out of it, so a fail-open proceed does not tell the model that the Guardian is unreachable before the step runs. No disposition or enforcement changes.
Not changed here: the result gate's
allowalready wrote its reasoning toadditionalContextbefore this PR. So a fail-open proceed atPostToolUsestill tells the model, after the tool has run, that no decision arrived. That entry also carries warn-only policy messages, so separating the two needs a code change. I can open an issue for it.🤖 Generated with Claude Code