Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs-site/src/content/docs/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -710,6 +710,22 @@ When routing Claude Code to the canonical ChatGPT Codex backend, OpenCodex remov

A combo can also advance after an intact HTTP 400 `invalid_request_error` that specifically rejects `user`, reports `unsupported_value` for `reasoning.effort`/`reasoning_effort`, or reports `param: input` with an exact model-scoped `does not support image inputs` rejection. A route that refuses the request's media with code `unsupported_input_modality`, such as image input sent to a CLI provider route without a verified multimodal contract, advances the same way. This is a mismatch for that request, not evidence that the target is unhealthy, so it records no cooldown. This compatibility recovery does not silently change `none` into a different effort or broaden this exception to arbitrary invalid requests. Policy refusals, cancellation and already-committed output remain non-replayable. A single-target request still returns an unresolved upstream rejection.

## Tool calls stay on one target

When a model asks your client to run a tool, the request that returns the tool result finishes the
same assistant turn. OpenCodex sends that request to the target that issued the tool call, as long
as the target is still enabled, still in the combo, and out of cooldown. This holds for every
strategy, including JEV, which skips its decision for that request. The next user message chooses a
target normally.

Tracking uses the client's thread or session id, so a client that sends neither gets a normal pick.
If the original target is unavailable or fails, the combo fails over as usual, and the new target
receives the earlier tool calls in a form it accepts. DeepSeek thinking models get a one-space
reasoning placeholder for a tool call that another model issued without reasoning. Gemini gets
Google's documented signature placeholder. Anthropic gets a plain tool call and result. Earlier
versions could send DeepSeek a bare tool call here and fail with `The reasoning_text in the
thinking mode must be passed back to the API`.

## Default reasoning effort

`defaultEffort` supplies a configured effort when the selected target has a known, nonempty supported ladder. With the default `defaultEffortMode: "fallback"`, an explicit caller effort keeps precedence. `defaultEffortMode: "force"` overrides a valid caller effort with the configured default; it requires a valid, non-null `defaultEffort` and can increase cost and latency. Force mode is an explicit operator choice through combo configuration or management.
Expand Down
3 changes: 3 additions & 0 deletions scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -873,6 +873,7 @@
"deepinfra-provider.test.ts": "providers",
"deepseek-inbound-wire.test.ts": "providers",
"deepseek-reasoning-replay-gaps.test.ts": "providers",
"deepseek-bare-tool-call-replay.test.ts": "providers",
"deepseek-reasoning-replay.test.ts": "providers",
"deepseek-responses-item-id-repair.test.ts": "providers",
"management-desktop-compatibility-routes.test.ts": "server", "management-desktop-compatibility-runtime-routes.test.ts": "server", "server-desktop-compatibility-startup.test.ts": "server", "management-desktop-compatibility-settings.test.ts": "server",
Expand Down Expand Up @@ -988,6 +989,7 @@
"google-models-listing.test.ts": "adapters/google",
"google-output-clamp.test.ts": "adapters/google",
"google-provider-metadata-roundtrip.test.ts": "adapters/google",
"google-ai-studio-foreign-call-signature.test.ts": "adapters/google",
"google-signature-history-roundtrip.test.ts": "adapters/google",
"google-strict-tool-validated-mode.test.ts": "adapters/google",
"google-structured-output.test.ts": "adapters/google",
Expand Down Expand Up @@ -1650,6 +1652,7 @@
"server-combo-failover-e2e.test.ts": "server",
"server-combo-held-response.test.ts": "server",
"server-combo-reasoning-replay-eligibility.test.ts": "server",
"server-combo-tool-loop-continuation.test.ts": "server",
"server-combo-zero-output-failover.test.ts": "server",
"server-google-antigravity-oauth-401-replay.test.ts": "server",
"server-antigravity-web-search-validation.test.ts": "server", "server-google-antigravity-oauth-429-budget.test.ts": "server",
Expand Down
6 changes: 6 additions & 0 deletions src/adapters/google.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1254,6 +1254,12 @@ export function createGoogleAdapter(provider: OcxProviderConfig): ProviderAdapte
const compiled = compileGoogleWireBody(body, toolSchemaProfile, toolSchemaPolicy);
reportToolSchemaLoss(compiled.toolSchemaLossReport);
restoreGoogleToolName = compiled.restoreToolName;
// A functionCall issued by another model (a combo hop, #7298) has no Gemini signature, and
// Gemini 3 refuses the turn without one. The same Gemini-only sentinel CCA and Vertex
// apply; a real signature carried by history always wins.
if (Array.isArray((compiled.body as { contents?: unknown[] }).contents)) {
applyAntigravityThoughtSignatureFallback(routedModelId, (compiled.body as { contents: unknown[] }).contents);
}
return { url, method: "POST", headers, body: JSON.stringify(compiled.body) };
},

Expand Down
69 changes: 68 additions & 1 deletion src/adapters/openai-responses/reasoning.ts
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,74 @@ export function sanitizeReasoningInputContent(
});

if (blankedPlaintext) opts?.onPlaintextReasoningBlanked?.();
return changed ? { ...raw, input } : body;
// Runs after the per-item pass so an item it emptied has already been backfilled, and a
// placeholder is only invented for a turn that carries no reasoning item at all.
const withTurnReasoning = opts?.requirePlaintextReasoning === true
? backfillBareToolCallTurnReasoning(input)
: input;
if (withTurnReasoning !== input) changed = true;
return changed ? { ...raw, input: withTurnReasoning } : body;
}

/** Assistant-authored history items: what DeepSeek merges into one assistant message. */
function isAssistantTurnItem(item: unknown): boolean {
if (!isPlainObject(item) || typeof item.type !== "string") return false;
if (item.type === "reasoning") return true;
if (item.type === "message") return item.role === "assistant";
return item.type.endsWith("_call");
}

function isClientToolCallItem(item: unknown): boolean {
return isPlainObject(item)
&& (item.type === "function_call" || item.type === "custom_tool_call" || item.type === "local_shell_call");
}

/**
* Give every assistant tool-call turn that has NO reasoning item a minimal plaintext one (#7298).
*
* A Combo can serve the turn that issued a tool call on one model and the continuation on
* another. When the issuing model was a GPT turn that produced no reasoning item (or the item
* was dropped upstream), the strict destination sees a bare `function_call` and DeepSeek's
* thinking mode refuses the whole request with `The reasoning_text in the thinking mode must be
* passed back to the API`. The per-item backfill above cannot help, because there is no item to
* backfill. This is the Responses-wire twin of the Chat adapter's `reasoning_content: " "`
* placeholder (#5421): the same single-space text, never fabricated reasoning, inserted at the
* head of the turn so it precedes the call exactly where a native turn's reasoning would be.
*
* A turn is a maximal run of assistant-authored items; any user message or tool output ends it.
* Turns that already carry a reasoning item, and turns with no client tool call, are untouched.
*/
export function backfillBareToolCallTurnReasoning(input: unknown[]): unknown[] {
const bareTurnStarts = new Set<number>();
let turnStart = -1;
let turnHasReasoning = false;
let turnHasToolCall = false;
const closeTurn = (): void => {
if (turnStart >= 0 && turnHasToolCall && !turnHasReasoning) bareTurnStarts.add(turnStart);
turnStart = -1;
turnHasReasoning = false;
turnHasToolCall = false;
};
for (let index = 0; index < input.length; index++) {
const item = input[index];
if (!isAssistantTurnItem(item)) {
closeTurn();
continue;
}
if (turnStart < 0) turnStart = index;
if ((item as Record<string, unknown>).type === "reasoning") turnHasReasoning = true;
if (isClientToolCallItem(item)) turnHasToolCall = true;
Comment on lines +221 to +222

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reorder reasoning that follows a tool call

When reconstructed, resumed, or synthetic history orders a reasoning item after its function_call, this marks the whole run as having reasoning and suppresses the placeholder, while leaving the call before its required reasoning_text. DeepSeek can therefore still reject the continuation with HTTP 400. This ordering is already documented as a supported input in tests/providers/deepseek-reasoning-replay-gaps.test.ts lines 157-161; track whether usable reasoning precedes the first client tool call, and move/backfill it at the turn head when it does not.

Useful? React with 👍 / 👎.

}
closeTurn();
if (bareTurnStarts.size === 0) return input;
const result: unknown[] = [];
for (let index = 0; index < input.length; index++) {
if (bareTurnStarts.has(index)) {
result.push({ type: "reasoning", summary: [], content: [{ type: "reasoning_text", text: " " }] });
}
result.push(input[index]);
}
return result;
}

/**
Expand Down
25 changes: 25 additions & 0 deletions src/server/responses/combo-session-recall.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,31 @@ export function recallComboForLane(
return entry.responseModel === model ? entry.comboId : undefined;
}

/**
* The concrete target that completed the lane's last turn for `comboId`, under the same
* freshness and ownership rules as `recallComboForLane`. A lane answered by another combo, or
* whose target left the combo or was disabled, recalls nothing.
*/
export function recallComboTargetForLane(
config: OcxConfig,
lane: string | undefined,
comboId: string,
): Pick<OcxComboTarget, "provider" | "model"> | undefined {
if (!lane || !comboId) return undefined;
const entry = recall.get(lane);
if (!entry || entry.comboId !== comboId) return undefined;
const combo = getCombo(config, entry.comboId);
const provider = config.providers[entry.target.provider];
if (Date.now() - entry.at >= RECALL_TTL_MS
|| !Object.hasOwn(config.providers, entry.target.provider)
|| !provider || provider.disabled === true
|| !combo?.targets.some(target => targetKey(target) === targetKey(entry.target))) {
deleteEntry(lane);
return undefined;
}
return { provider: entry.target.provider, model: entry.target.model };
}

/**
* Periodic expiry. Without it a lane that is never read again and never touched by a config
* reconciliation holds its entry for the life of the process: the existing TTL is only
Expand Down
70 changes: 70 additions & 0 deletions src/server/responses/combo-tool-loop-pin.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/**
* Combo target pinning for an in-flight tool loop (#7298).
*
* A combo picks a target per request. A tool loop is several requests that together form one
* assistant turn: the model emits a call, the client runs it, and the next request carries the
* result back. Re-picking between those requests hands the second half of a turn to a model that
* did not produce the first half. That is wasteful at best (a cold prompt cache, a different
* plan) and a hard failure at worst: DeepSeek's thinking mode refuses a tool continuation whose
* call carries no reasoning it can read, and a GPT turn never leaves one.
*
* So a continuation whose input ends in a tool output keeps the target that completed the lane's
* previous turn, as long as that target is still selectable right now: enabled, still in the
* combo, out of cooldown and passing every per-request eligibility filter the caller supplies.
* When it is not, normal selection (and normal failover) runs, and the strict serializers make
* the foreign history acceptable instead: see `backfillBareToolCallTurnReasoning` and the Chat
* adapter's reasoning placeholder.
*
* A new user message is never pinned; re-deciding per user turn is what a combo is for.
*/
import { pickComboTarget, targetKey, type ComboPick } from "../../combos";
import type { NormalizedComboTarget } from "../../combos/types";
import type { OcxConfig } from "../../types";
import { recallComboTargetForLane } from "./combo-session-recall";

export const COMBO_TOOL_LOOP_PIN_REASON = "combo-tool-loop-pin";

function isToolOutputItem(item: unknown): boolean {
if (!item || typeof item !== "object" || Array.isArray(item)) return false;
const type = (item as { type?: unknown }).type;
return typeof type === "string" && type.endsWith("_output");
}

/**
* Whether this request resumes an assistant turn by returning tool results: its Responses
* `input` (or Chat `messages`) ends with a tool output.
*/
export function isToolLoopContinuation(body: unknown): boolean {
if (!body || typeof body !== "object" || Array.isArray(body)) return false;
const { input, messages } = body as { input?: unknown; messages?: unknown };
if (Array.isArray(input)) return input.length > 0 && isToolOutputItem(input[input.length - 1]);
if (Array.isArray(messages) && messages.length > 0) {
const last = messages[messages.length - 1];
return !!last && typeof last === "object" && (last as { role?: unknown }).role === "tool";
}
return false;
}

/**
* The pick that keeps a tool loop on its previous target, or undefined to select normally.
* Uses a non-mutating preview so a pinned request never disturbs round-robin rotation state.
*/
export function toolLoopPinnedPick(args: {
config: OcxConfig;
comboId: string;
lane: string | undefined;
body: unknown;
eligible: (target: NormalizedComboTarget) => boolean;
now?: number;
}): ComboPick | undefined {
if (!isToolLoopContinuation(args.body)) return undefined;
const previous = recallComboTargetForLane(args.config, args.lane, args.comboId);
if (!previous) return undefined;
const previousKey = targetKey(previous);
const pick = pickComboTarget(args.config, args.comboId, {
eligible: target => targetKey(target) === previousKey && args.eligible(target),
now: args.now,
preview: true,
});
return pick ? { ...pick, attempted: [previousKey] } : undefined;
}
24 changes: 20 additions & 4 deletions src/server/responses/core-combo.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ import type { CodexAuthContext } from "../../codex/auth-context";
import type { ResponsesTerminalStatus } from "../../bridge";
import { beginRequestAttempt, sealRequestAttemptIdentity, finishRequestAttempt } from "../request-log";
import { rememberComboForLane } from "./combo-session-recall";
import { COMBO_TOOL_LOOP_PIN_REASON, toolLoopPinnedPick } from "./combo-tool-loop-pin";
import { runTurnAdapterSseResponses } from "./core-lifetime";
import { normalizePersistedJevDecision } from "../../usage/jev-stats";
import {
Expand Down Expand Up @@ -589,7 +590,19 @@ export async function executeComboResponses(
waitForCooldownMs: combo.waitForCooldownMs,
abortSignal: options.abortSignal,
});
let pick = await pickWithWait({
// An in-flight tool loop stays on the target that issued its calls while that target is still
// selectable (#7298). Override routes never record a lane target, so they never pin either.
const toolLoopPin = options.compactionRoutingOverride || options.mainHardLockFallbackOverride
? undefined
: toolLoopPinnedPick({
config,
comboId,
lane: sessionLaneIdFromRequest(req.headers),
body,
eligible: target => (meteredAllowed || !target.metered) && targetEligible(target),
now: initialNow,
});
let pick = toolLoopPin ?? await pickWithWait({
eligible: targetEligible,
now: initialNow,
});
Expand Down Expand Up @@ -625,7 +638,7 @@ export async function executeComboResponses(
: comboUnavailable(comboId, undefined, comboExclusionReasons(config, comboId, targetEligible));
}
let jevDecision: JevDecision | undefined;
if (combo.strategy === "jev") {
if (combo.strategy === "jev" && !toolLoopPin) {
const choices = eligibleJevComboChoices(config, comboId, targetEligible, Date.now());
const first = choices[0];
if (!first) {
Expand Down Expand Up @@ -715,8 +728,11 @@ export async function executeComboResponses(
// One immutable combo selection trace, before any child dispatch; child
// adoption below must never replace it with a concrete child route trace.
const decision = comboRouteDecisionTrace(config, comboId, pick, requestedModel);
logCtx.routeDecision = options.memoryModelPhase
? { ...decision, selected: { ...decision.selected, reason: memoryModelRouteReason(options.memoryModelPhase) } }
const selectedReason = options.memoryModelPhase
? memoryModelRouteReason(options.memoryModelPhase)
: toolLoopPin ? COMBO_TOOL_LOOP_PIN_REASON : undefined;
logCtx.routeDecision = selectedReason
? { ...decision, selected: { ...decision.selected, reason: selectedReason } }
: decision;

const originalReasoning = body && typeof body === "object" && !Array.isArray(body)
Expand Down
4 changes: 4 additions & 0 deletions structure/providers/google.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ Vertex prefers Codex's opaque `prompt_cache_key` for session identity and falls
first-user-message derivation for clients that omit it; only the fixed hash is retained.
Both streaming and non-streaming responses feed the store; request compilation happens before replay
so matching uses the provider-visible tool name.
A first `functionCall` that still has no real signature after replay receives the Gemini-only
`skip_thought_signature_validator` sentinel through `applyAntigravityThoughtSignatureFallback`, on
Antigravity, Vertex and AI Studio alike. AI Studio gained it for combo hops that hand Gemini a call
another model issued (#7298); non-Gemini model ids never receive it.

`src/adapters/openai-responses/passthrough.ts` removes item-level `extra_content` through
`stripProviderInputItemMetadata` in `src/adapters/openai-responses/request-strips.ts` only for
Expand Down
Loading
Loading