Repository navigation
Conversation
 [Medium impact] Adds optional larger context window for OpenAI models. Fix the existing route-isolation issue before merging: an OAuth catalog limit can still reduce an API-key session’s context budget and cause premature compaction. Findings
Fix with agent prompt### Issue 1
crates/jcode-provider-openai-runtime/src/openai_provider_impl.rs:1184-1187
If an OAuth catalog has cached a limit for a model also used with an OpenAI API key, `context_window()` now applies that OAuth limit to the API-key route. The API-key catalog does not replace it. With a cached 872K limit, the API-key route budget fell from 1M to 872K in the reproduced case and triggered reactive compaction at 750K input tokens instead of continuing the conversation. Keep the limit associated with its credential route when calculating the budget.
### Issue 2
crates/jcode-base/src/config/default_file.rs:444-446
The larger-window opt-in leaves the default OpenAI native auto-compaction threshold at 200,000 tokens. With opt-in enabled, the generated request still asks OpenAI to compact at that threshold, so conversations can compact well before using an advertised 872K window. Users must adjust the separate compaction setting to avoid this non-blocking limitation.
### Issue 3
crates/jcode-base/src/provider/tests/catalog_subscription.rs:undefined-182
This disk-hydration fixture supplies an empty maximum-limit map. It cannot catch a future regression in saving and restoring a nonempty maximum when opt-in is enabled. Add a distinct maximum and assert the hydrated effective limit; this coverage gap is non-blocking.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
### Issue 4
crates/jcode-base/src/provider/models_catalog.rs:39-46
Changing `openai_use_max_context_window` in a running session leaves the old token budget in use. The flag is read only when the catalog loads from disk or is fetched again. Config reload updates custom-provider limits but does not reapply the OpenAI limits.
This is non-blocking, but users who disable the opt-in can keep sending larger requests until a catalog refresh or restart. Reapply the saved catalog limits when this setting changes.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.SummaryAdds an opt-in setting to budget OpenAI OAuth requests using the catalog’s
The previously reported OAuth/API-key limit crossover remains unresolved and must be fixed before merging. The separate 200K native-compaction threshold, missing nonempty-maximum disk-hydration coverage, and stale limits after changing the setting also remain outstanding; those three concerns are non-blocking. Reviews (23) · Last reviewed commit: "fix(openai): budget OAuth models from th..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| # Budget OpenAI OAuth models at the catalog's larger `max_context_window` | ||
| # (for example 872K) instead of the default `context_window` (272K). | ||
| # openai_use_max_context_window = true |
There was a problem hiding this comment.
The larger-window opt-in leaves the default OpenAI native auto-compaction threshold at 200,000 tokens. With opt-in enabled, the generated request still asks OpenAI to compact at that threshold, so conversations can compact well before using an advertised 872K window. Users must adjust the separate compaction setting to avoid this non-blocking limitation.
Knowledge Base Used: Sessions, context, and memory
Artifacts
Parent-revision offline request probe
- The authored Rust test installed the parent revision’s default catalog limit and inspected its generated Responses body, providing the executable before-side probe.
Parent-revision request-building test output
- The recorded Cargo command ran in the HEAD^ worktree and passed, showing a 272,000-token window and a 200,000-token threshold in the request body.
PR-revision offline request probe
- The authored Rust test installed the PR revision’s effective catalog limit and inspected its generated Responses body, providing the executable after-side probe.
PR-revision request-building test output with max-window opt-in
- The recorded Cargo command ran on HEAD with opt-in enabled and passed, showing an 872,000-token window but the same 200,000-token request threshold.
PR-revision request-building test output without opt-in
- The recorded Cargo command ran on HEAD with opt-in disabled and passed, showing that the 200,000-token request threshold also exists independently of the new opt-in.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/jcode-base/src/config/default_file.rs
Line: 373-375
Comment:
**Compaction remains at 200K**
The larger-window opt-in leaves the default OpenAI native auto-compaction threshold at 200,000 tokens. With opt-in enabled, the generated request still asks OpenAI to compact at that threshold, so conversations can compact well before using an advertised 872K window. Users must adjust the separate compaction setting to avoid this non-blocking limitation.
**Knowledge Base Used:** [Sessions, context, and memory](https://app.greptile.com/solo-systems/-/custom-context/knowledge-base/1jehuang/jcode/-/docs/sessions-context-and-memory.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| context_limits: [("openai-disk-only-model".to_string(), 424_242)] | ||
| .into_iter() | ||
| .collect(), | ||
| max_context_limits: HashMap::new(), |
There was a problem hiding this comment.
This disk-hydration fixture supplies an empty maximum-limit map. It cannot catch a future regression in saving and restoring a nonempty maximum when opt-in is enabled. Add a distinct maximum and assert the hydrated effective limit; this coverage gap is non-blocking.
Knowledge Base Used: Model provider integration
Artifacts
Original disk-hydration test with opt-in enabled
- Ran the unchanged targeted test with opt-in enabled; it passed while supplying no maximum value, leaving the claimed coverage gap.
Temporary nonempty-maximum test probe
- Captured the temporary test diff that saves 872000, checks the disk snapshot, and asserts the hydrated value; this shows exactly what was exercised.
Nonempty maximum persisted and hydrated with opt-in enabled
- Ran the probe with opt-in enabled; the saved-value assertion passed and hydration returned 872000, contradicting a current runtime-failure claim.
Nonempty maximum retained with opt-in disabled
- Ran the same probe with opt-in disabled; the saved-value assertion passed while hydration returned the default 424242, confirming opt-in selection.
Related catalog parsing and limit-selection tests
- Ran the four focused catalog tests offline; all passed, covering parsing and in-memory selection but not nonempty disk hydration.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/jcode-base/src/provider/tests/catalog_subscription.rs
Line: 168
Comment:
**Persisted maximum goes untested**
This disk-hydration fixture supplies an empty maximum-limit map. It cannot catch a future regression in saving and restoring a nonempty maximum when opt-in is enabled. Add a distinct maximum and assert the hydrated effective limit; this coverage gap is non-blocking.
**Knowledge Base Used:** [Model provider integration](https://app.greptile.com/solo-systems/-/custom-context/knowledge-base/1jehuang/jcode/-/docs/provider-integration.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Comments Outside DiffThese findings could not be posted inline.
|
|
Added a follow-up commit. A catalog cache written by an older build lacks the new field, and a recent one still counts as fresh, so the setting silently did nothing until the TTL expired. The persisted scope now carries a Note for reviewers: |
f73aa44 to
17474ca
Compare
|
Want your agent to iterate on Greptile's feedback? Try greploops. |
deceab5 to
88c2eb8
Compare
| // Resolve through jcode-base so the live OAuth catalog limits (including | ||
| // an opted-in max_context_window) are honored. The provider-core | ||
| // resolver alone has no cache and falls back to static family guesses. | ||
| jcode_base::provider::context_limit_for_model_with_provider(&model, Some(self.name())) |
There was a problem hiding this comment.
If an OAuth catalog has cached a limit for a model also used with an OpenAI API key, context_window() now applies that OAuth limit to the API-key route. The API-key catalog does not replace it. With a cached 872K limit, the API-key route budget fell from 1M to 872K in the reproduced case and triggered reactive compaction at 750K input tokens instead of continuing the conversation. Keep the limit associated with its credential route when calculating the budget.
Knowledge Base Used: Provider selection and runtime adapters
Artifacts
- The authored Rust test exercises the actual provider method, a local API-key catalog request, and the resulting compaction decision.
- The authored command runner temporarily selects each implementation, executes the same test, captures its output, and restores tracked source.
- The executed parent implementation returned a 1,000,000-token API-key window and did not compact at 750,000 tokens.
- The executed changed implementation returned an 872,000-token API-key window and did compact at 750,000 tokens.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/jcode-provider-openai-runtime/src/openai_provider_impl.rs
Line: 1235-1238
Comment:
**OAuth limit crosses routes**
If an OAuth catalog has cached a limit for a model also used with an OpenAI API key, `context_window()` now applies that OAuth limit to the API-key route. The API-key catalog does not replace it. With a cached 872K limit, the API-key route budget fell from 1M to 872K in the reproduced case and triggered reactive compaction at 750K input tokens instead of continuing the conversation. Keep the limit associated with its credential route when calculating the budget.
**Knowledge Base Used:** [Provider selection and runtime adapters](https://app.greptile.com/solo-systems/-/custom-context/knowledge-base/1jehuang/jcode/-/docs/provider-selection-and-runtime-adapters.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
04b9ad0 to
2350b25
Compare
| pub fn effective_context_limits(&self) -> HashMap<String, usize> { | ||
| effective_openai_context_limits( | ||
| &self.context_limits, | ||
| &self.max_context_limits, | ||
| crate::config::config() | ||
| .provider | ||
| .openai_use_max_context_window, | ||
| ) |
There was a problem hiding this comment.
Changing openai_use_max_context_window in a running session leaves the old token budget in use. The flag is read only when the catalog loads from disk or is fetched again. Config reload updates custom-provider limits but does not reapply the OpenAI limits.
This is non-blocking, but users who disable the opt-in can keep sending larger requests until a catalog refresh or restart. Reapply the saved catalog limits when this setting changes.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/jcode-base/src/provider/models_catalog.rs
Line: 39-46
Comment:
**Window setting stays stale**
Changing `openai_use_max_context_window` in a running session leaves the old token budget in use. The flag is read only when the catalog loads from disk or is fetched again. Config reload updates custom-provider limits but does not reapply the OpenAI limits.
This is non-blocking, but users who disable the opt-in can keep sending larger requests until a catalog refresh or restart. Reapply the saved catalog limits when this setting changes.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
6a960c7 to
a06d720
Compare
The ChatGPT/Codex model catalog advertises two windows per model: context_window (272K default) and max_context_window (872K for gpt-6-sol, gpt-6-luna, gpt-6-astra, gpt-5.6-sol/terra/luna and gpt-daybreak-blue-latest; 272K for gpt-5.5). Jcode only read the default, so every OAuth session budgeted and compacted at 272K. Record max_context_window separately in the parsed catalog and on-disk cache, and publish it as the context limit when provider.openai_use_max_context_window (or JCODE_OPENAI_USE_MAX_CONTEXT_WINDOW) is enabled. Default behavior is unchanged. Keeping the maximum separate means toggling the option takes effect on the next start without refetching the catalog. Refs 1jehuang#1323
… persisted A catalog cache written by an older build has no max_context_limits, and a recent one counts as fresh, so the opt-in would silently stay at 272K until the TTL expired. Version the persisted scope and hydrate older snapshots as stale (keeping their real observed_at) so a live refresh runs at once.
OpenAIProvider::context_window() called the provider-core resolver with an empty cache, so catalog limits (and the opted-in max_context_window) never reached the agent budget or compaction. GPT models fell to static family guesses instead: 1M for gpt-6*, 272K for gpt-5*, 200K default for gpt-daybreak-blue-latest. Resolve through jcode-base so the populated context-limit cache wins, matching how the OpenRouter runtime already reads it. Receipt: daemon log showed "Context limit cache: <gpt model> = 872k" while the provider ignored it. cargo check -p jcode-provider-openai-runtime passes. Refs 1jehuang#1323
a06d720 to
a796166
Compare
1 participant
Refs #1323. This adds the route metadata the triage comment asked for.
Problem
Jcode reads only
context_windowfrom the ChatGPT/Codex catalog (backend-api/codex/models). That catalog also advertises a largermax_context_windowper model. So OAuth sessions budget and compact at 272K even on routes that accept much more.Live catalog on 2026-09-25 (fields only, no account data):
So the OAuth ceiling is 872K, not the 1.05M the public API docs list for Luna. The backend enforces it as advertised. I sent needle-in-haystack requests to
/backend-api/codex/responsesover OAuth. gpt-5.6-sol, gpt-6-sol and gpt-daybreak-blue-latest each accepted 403,043 and 846,043 input tokens (reportedusage.input_tokens) and recalled the needle.Change
parse_openai_model_catalogrecordsmax_context_windowin a newmax_context_limitsmap. Non-positive or non-numeric values are ignored.#[serde(default)]for old cache files.provider.openai_use_max_context_window(envJCODE_OPENAI_USE_MAX_CONTEXT_WINDOW), off by default. When on, the advertised maximum replaces the default in the context-limit cache on live refresh and on disk hydration. Existing users see no change.It's opt-in because a bigger window means bigger, costlier requests against subscription limits, and later compaction. Happy to flip the default if you prefer.
Tests
Three new unit tests cover parsing, missing or invalid values, the opt-in merge, and a lower maximum. The
jcode-baselib suite gives the same 12 failures before and after this change (auth transfer, session, skill and harness-detection tests that fail in this environment on cleanmaster). The count goes from 1590 to 1593 passed.