fix(verdict-cache): bounded per-entry LRU so a return to a recent state HITS (JEF-390)#206
Merged
thejefflarson merged 1 commit intoJul 8, 2026
Conversation
…udged state HITS (JEF-390) The verdict cache was a single slot per entry, so a workload whose evidence oscillates between states A and B re-judged on every flip: judging B overwrote A's cached verdict, and flipping back to A missed. Widen the `cached` gate from one slot to a bounded per-entry LRU of (fingerprint -> decisive Verdict) — default 32 slots, env-configurable via PROTECTOR_VERDICT_CACHE_SLOTS (unset/unparseable/0 -> default; positive values floored at 2 so the cache can always retain one prior state). `cached_for` now hits on ANY fingerprint in the entry's LRU and promotes it to most-recently-used; `cache_decisive` inserts most-recently-used and evicts the least-recently-used past the cap. Dependency-light: an ordered Vec with move-to-front + truncate, no crate. Invariants preserved: only DECISIVE verdicts are ever inserted — an Uncertain is never cached and still arms the JEF-234 exponential backoff; an exact-fingerprint hit is byte-identical evidence (the fingerprint is the whole-prompt hash), so a served verdict is exactly as valid as when judged, just more retained — no new staleness. Journal-restore, display carry-forward, journaled dedup, recency, and the lock discipline are untouched. Tests (sibling verdict_store_tests.rs to stay under the 1,000-line cap): A->B->A serves A on the return (2 judgements, not 3); beyond the cap the LRU evicts the least-recently-used state and re-judges it (proving LRU, not FIFO); Uncertain is not cached and the entry backs off; the env knob defaults on invalid/zero and floors small values. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes JEF-390.
Problem
The verdict cache was single-slot per entry (
cached: Option<(String, Verdict)>). A workload whose evidence oscillates between states A and B re-judged on every flip: judging B overwrote A's cached verdict, and flipping back to A missed — hammering the slow CPU model on a state it had already decided.Change (
engine/src/engine/state/verdict_store.rs)(fingerprint → decisive Verdict)— a newVerdictLru(an orderedVecwith move-to-front + truncate; no crate).PROTECTOR_VERDICT_CACHE_SLOTS. Unset / unparseable /0→ default; a positive value is honoured but floored at 2 (MIN_VERDICT_CACHE_SLOTS) so the cache can always retain one prior state. The cap is resolved once at construction, not per insert.cached_forhits on any fingerprint in the entry's LRU and promotes it to most-recently-used;cache_decisiveinserts most-recently-used and evicts the least-recently-used past the cap.Invariants preserved (no regression)
cache_decisiveis the sole insert path and the engine calls it only in its decisive arm; anUncertainis never added and still arms the JEF-234EntryBackoff(verified by test).journaleddedup, recency (JEF-201), and the Mutex lock discipline are untouched. This ticket only widens thecachedgate from 1 slot to N.Tests
Added
engine/src/engine/state/verdict_store_tests.rs(sibling file to stay under the 1,000-line cap), all driven through a small harness that mirrors the engine judging loop:Uncertainis not cached and the entry backs off (JEF-234); a later decisive verdict clears the backoff and populates the LRU.0/negative, honours+trims positive values, floors1to the minimum.Checks
cargo fmt— cleancargo clippy --all-targets(warnings = errors) — cleancargo nextest run— 778 passed (4 new), file-size guard green🤖 Generated with Claude Code
https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP