Skip to content

Anthropic cache prefixes are rewritten during compaction and image overflow in long Codex turns #7307

Open

Description

@S0RYUASUKA

Client or integration

Codex App

Area

Provider adapter

Summary

Long Codex tasks routed through the Anthropic Messages adapter can repeatedly rewrite a large prompt cache even when most historical input is unchanged:

  • Routed compaction removes the tool catalog before requesting a summary. That changes the cached prefix independently of the later, necessary replacement of history with a summary.
  • Image normalization can demote historical images under aggregate pressure; the 413 recovery path rebuilds with tighter image tiers. Both change already-sent image bytes.
  • A separate, expected source of rewrites is five-minute TTL expiry while a main or child turn is still running tools. There is currently no opt-in running-turn renewal. This is not evidence of a prefix-rewrite bug by itself.

Expected: retain unchanged historical image encodings and the Anthropic system/tool prefix, prevent tools from executing during summaries, and compact old complete history units only when the actual request limit requires it. Optional renewal should protect verified running main/child turns and stop on completion or unknown state, without modifying Codex or its transcript.

Reproduction

  1. Route Codex Responses requests to Anthropic with short cache retention. Include client tools and a growing image history.
  2. Trigger routed compaction. Compare the ordinary request's tool catalog with the summary request: the current createResponsesEffects removes context.tools.
  3. Append enough valid images to exceed the aggregate image budget, or return an upstream 413. Compare historical image payloads before/after normalization or the biased retry.
  4. Separately, leave a turn running a long tool for more than five minutes. Its next natural request may legitimately miss expired cache. Do not conflate this with steps 2–3.

An offline regression against the current dev implementation reproduces step 2: the expected unchanged tool catalog becomes undefined. Public regression fixtures use generated PNGs, synthetic large rollout records, fake clocks and fake upstreams. No paid provider call is needed for these wire/lifecycle checks.

Version

Observed on installed 2.86.0. The compaction-prefix regression was also reproduced against dev at 8640c2f30edbd2fbb6947ed5c25e2793f5ae72a6 (2.89.0).

Operating system

Windows 11, build 26200

Provider and model

Anthropic Messages adapter / claude-opus-5-5

Logs or error output

Sanitized offline result from the unchanged dev compaction implementation:

compaction preserves client tool and system cache prefixes without changing tool choice
Expected: [ { name: "read", ... } ]
Received: undefined
0 pass, 1 fail

Low hit rates alone do not establish the cause. Some historical low-hit requests remain unattributed and are not claimed as fixed by this report.

Screenshots and supporting files

Related prior work: #4532 and closed, unmerged #4549 address stable image encoding. This report additionally covers compaction-prefix preservation, continued image-task recovery at real limits, and optional running-turn renewal. Private conversations, raw logs and actual user images are intentionally not attached.

Redacted configuration

{
  "cacheRetention": "short",
  "providers": {
    "anthropic": {
      "adapter": "anthropic",
      "baseUrl": "https://api.anthropic.com",
      "authMode": "key",
      "defaultModel": "claude-opus-5-5"
    }
  }
}

Authentication material is omitted. OAuth and custom provider names also need isolated ownership; they must not borrow a parent's thread as a child's own identity.

Checks

  • I searched existing issues and documentation.
  • I removed secrets, tokens, account details, request credentials, and personal data.

Activity

  1. added
    bugSomething isn't working
    providerProvider adapters, OpenAI-compat presets, upstream API quirks
    on Oct 18, 2026
  2. Ingwannu commented on Oct 18, 2026

    @Ingwannu
    Owner

    Tracking the proposed implementation in Draft #7308. At dev 8640c2f30edb, the prefix-rewrite paths and the ordinary short-cache expiry need separate acceptance criteria: preserving an unchanged prefix must not depend on enabling renewal. Please keep regression receipts for tool/system preservation and historical image-byte stability separate from the optional running-turn renewal receipts (ended/unknown/expired turns produce no renewal). The draft PR is not a shipped fix; this issue stays open while its runtime-ownership, image recovery and validation gates are reviewed.

  3. OctopusEngineering commented on Oct 24, 2026

    @OctopusEngineering

    Field data from a shared Mac mini running Codex Desktop through opencodex 2.84.0 (Anthropic Messages adapter, claude-opus-5-5, default short cache retention). Aggregate numbers only; no transcripts or images attached.

    Over 30 days we looked at ~59k Anthropic requests. About 40% of relative input cost (uncached 1, cache read 0.1, cache write 1.25) came from requests that re-wrote most of the conversation into the cache. Classifying those full rewrites:

    Cause Share
    Gap of more than 5 minutes since the previous request (TTL expiry) 57% (about half human idle time, half long tool or subagent waits inside a running turn)
    Earlier prefix changed 22%
    Right after compaction 17%
    First request of a new thread or subagent 4%

    Image history correlates strongly with the prefix-change bucket. The large-rewrite rate per request was 3.8% when the conversation held 20 or fewer images. It rose to 20–38% once it held more than 100 images. That matches the age-tier demotion and textification described above. Page-by-page visual QA sessions (hundreds of screenshots) were the worst case.

    We also simulated cacheRetention: "long" on the same traffic: about -1% net, because the higher write price cancels most of the avoided TTL rewrites. So for us the image-byte stability and compaction-prefix parts of this issue matter more than renewal. For reference, we have since lowered providers.anthropic.modelAutoCompactTokenLimits to 400000 locally, so threads stop growing toward 900k between compactions.

    Happy to rerun the measurement against a build with #7308 if that helps validate the image-stability gate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingproviderProvider adapters, OpenAI-compat presets, upstream API quirks

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions