Skip to content

feat(adjudicate): delta-aware re-judging — judge on additive change, hold on pure subtraction (JEF-391)#207

Merged
thejefflarson merged 1 commit into
mainfrom
thejefflarson/jef-391-delta-aware-adjudication-judge-on-change-the-state-is
Jul 8, 2026
Merged

feat(adjudicate): delta-aware re-judging — judge on additive change, hold on pure subtraction (JEF-391)#207
thejefflarson merged 1 commit into
mainfrom
thejefflarson/jef-391-delta-aware-adjudication-judge-on-change-the-state-is

Conversation

@thejefflarson

Copy link
Copy Markdown
Owner

Closes JEF-391. Implements the ACCEPTED ADR-0023 — "the state is the context, the delta is the question."

What changed

Today build_judgment_prompt is a static full-state snapshot with no "what changed" markers, and the re-judge gate is purely "the whole-prompt fingerprint changed" — so every new reachable object (ephemeral pods cycling) makes the model re-derive the world, and a known peer flickering out re-judges even though nothing was ADDED.

Now:

  • Prompt (prompt.rs): the full-state snapshot is UNCHANGED and gains one section — Changes since the last decisive verdict: <<<additions>>> — the ADDITIONS since this entry's baseline (newly-reachable objectives, newly-running CVEs, newly-exposed secrets, new posture, newly-corroborated behaviors), (none) when nothing was added, fenced/sanitized like all other evidence. The instruction directs the model to judge the current state as a whole with particular attention to whether the NEW elements introduce exploitation evidence or complete an exploitable path.
  • Baseline (state/verdict_store.rs): on a DECISIVE verdict the entry's judged surface (a JudgedSurface — reachable-objective lines, running-CVE lines, secrets, posture, behaviors) is snapshotted with that verdict as the entry's bounded baseline. The delta is current_surface \ baseline.
  • Re-judge gate (adj_gate.rs, extracted from the orchestrator to hold mod.rs under the 1,000-line cap): re-judge on a non-empty ADDITIVE delta, or no baseline (first judgment), or an LRU miss; a purely SUBTRACTIVE change (a peer aged out, a pod vanished) HOLDS the prior decisive verdict — no fresh model call — since its surface only shrank and removal is de-escalated by the existing recency/reversion path (JEF-141), not a re-judge.

Correctness guards (security-relevant)

  • The full current state is ALWAYS in the prompt — the delta only directs attention, never replaces the state. A guard test proves a NEW running CVE on an already-reachable objective re-judges with BOTH the full state and the new element present.
  • Fail toward re-judging: additions are a per-category set-difference over the rendered lines, so a modified element reads as removed-old + added-new and re-judges. Only a provable pure removal is skipped. First judgment / a defensive !additive-without-baseline both re-judge.
  • Uncertain is never cached and never sets a baseline (JEF-234 backoff still gates its retries), so a failed call can never suppress a later re-judge.
  • The delta section is fence_list-sanitized (a >>> injection in an objective key is stripped — tested).

Resolving ADR-0023's open questions

  • Baseline location + projection: on VerdictEntry alongside the LRU (Option<VerdictBaseline>), projected from the SAME rendered prompt lines (no second source of truth), bounded by the entry's proven surface and pruned with the entry. In-memory only — after a restart the entry re-judges once (fail-safe) before its baseline is re-established; the LRU is still journal-re-seeded (JEF-301).
  • Delta gate vs fingerprint gate: the verdict-cache KEY is the hash of the FULL-STATE prompt and EXCLUDES the "Changes since…" section. The delta is delta-derived attention, not state; keying on it would make the same full state hash differently as its baseline shifts (extra churn, and a needless re-judge per entry across a restart). Excluding it keeps the JEF-390 LRU a true exact-state guard (restart-safe) and leaves the surface-delta gate as the sole additive re-judge driver. A test locks this: the same full state keys identically whether its delta is (none) or additive, while the model prompt still differs.
  • Fail-safe: any ambiguity re-judges; only a provably pure-subtractive delta against an existing decisive baseline holds.

Tests (all in this PR)

adj_gate_tests.rs (the layered gate), adjudicate/tests/delta.rs (the delta prompt + surface math + correctness guards + the cache-key resolution), and a verdict_store baseline round-trip. engine/tests.rs::an_uncertain_re_judge_keeps_showing_the_prior_decisive_verdict was updated to drive its re-judge with an ADDITIVE change (adding a CVE) rather than a subtractive one, since ADR-0023 deliberately no longer re-judges on pure subtraction.

Checks

cargo fmt, cargo clippy --all-targets (warnings clean), cargo nextest run794 passed, 2 skipped, including the file_size_guard cap check (all files < 1,000 lines).

Scope notes / risks

  • A subtractive HOLD serves the prior decisive verdict for the shrunk surface — this is ADR-0023's explicit design (removal can only reduce breach risk; de-escalation is the reversion path). It's the same class as the existing JEF-390 exact-state LRU hit, and actions remain additive/reversible/self-reverting.
  • The ADR-0023 doc itself lands on its own branch/PR (adr-0023-delta-aware-adjudication); this PR is the implementation off latest origin/main.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP

…hold on pure subtraction (JEF-391)

Implements ADR-0023 ("the state is the context, the delta is the question").
The adjudication prompt keeps the full-state snapshot unchanged and gains a
"Changes since the last decisive verdict" section listing the ADDITIONS since
this entry's baseline (newly-reachable objectives, newly-running CVEs, newly-
exposed secrets, new posture, newly-corroborated behaviors), fenced like all
other evidence. The re-judge gate now re-judges on a non-empty ADDITIVE delta
(or no baseline / an LRU miss), and HOLDS the prior decisive verdict on a purely
SUBTRACTIVE change — a peer aging out or a pod vanishing no longer re-judges,
which stops the ephemeral-churn ping-pong at its root.

- surface.rs: JudgedSurface projects the entry's judged evidence from the SAME
  rendered lines the prompt carries (no second source of truth); additions are a
  per-category set-difference, so a modified element reads as removed+added and
  re-judges (fail toward re-judging).
- verdict_store.rs: a bounded per-entry baseline (surface + decisive verdict),
  set only on a decisive verdict (Uncertain never baselines), pruned with the
  entry. In-memory only: after a restart the entry re-judges once before its
  baseline is re-established.
- adj_gate.rs (extracted to hold mod.rs under the 1,000-line cap): the layered
  gate — exact-fingerprint LRU hit / subtractive hold / breaker+backoff / judge.
- prompt.rs: the verdict-cache key is the FULL-STATE hash and EXCLUDES the delta
  section, so an identical full state always keys identically (the LRU stays a
  true exact-state guard, restart-safe with JEF-301) and the surface-delta gate
  is the sole additive re-judge driver — resolving ADR-0023's open question.

Correctness guards (security-relevant): the full current state is ALWAYS in the
prompt (the delta only directs attention); a new element that makes an already-
reachable objective exploitable re-judges with both the full state and the new
element present; the delta section is sanitized/fenced.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP
@thejefflarson thejefflarson merged commit c215758 into main Jul 8, 2026
5 checks passed
@thejefflarson thejefflarson deleted the thejefflarson/jef-391-delta-aware-adjudication-judge-on-change-the-state-is branch July 8, 2026 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant