Skip to content

[FIX]: Evaluate attack verdicts over final traces - #150

Draft
spencrr wants to merge 6 commits into
microsoft:mainfrom
spencrr:dev/spencrr/trace-xpia-stopping
Draft

[FIX]: Evaluate attack verdicts over final traces#150
spencrr wants to merge 6 commits into
microsoft:mainfrom
spencrr:dev/spencrr/trace-xpia-stopping

Conversation

@spencrr

@spencrr spencrr commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Description

Moves XPIA verdict evaluation to the terminal trace and separates it from online stopping. stop_when accepts an explicit evaluator, None, or the default "auto". Auto mode reuses the verdict evaluator only for framework-owned conditions whose detected result is stable as turns are appended, such as cumulative tool and side-effect checks.

An identical stop/verdict evaluator is not called again when its latest result already covers the terminal trace. Final evaluation remains inside the active session and injection stack, observability downgrades are recorded without mutating response metadata, and cleanup failures discard otherwise successful verdict evidence and return ERROR.

Depends on #149. Because the branches live on a fork, this PR temporarily includes lower-layer diffs and targets main; those diffs disappear as dependencies merge.

Breaking changes

Behavioral change: unknown or stochastic verdict evaluators no longer run on every prefix by default. They evaluate once over the terminal trace and may run to max_turns unless an explicit stop_when is supplied. Auto-stopped attacks expose online evaluator feedback to adaptive drivers.

Checklist

  • pre-commit run --all-files passes
  • Tests added for automatic, explicit, and disabled stopping; exact call counts; classifier composition; cleanup ordering; zero/max turns; observability; metadata isolation; and summaries
  • Documentation updated

Validation: 116 cross-layer attack/probe/runner tests and 726 broad unit tests pass, with two known baseline cases deselected. Strict documentation build, all-files pre-commit, static checks, and privacy scan pass.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant