Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -302,7 +302,7 @@ Only fields named by `supported` may be written; null is not false, and an inert
policy is not active pooling. Threshold zero and `enabled:false` are independent.
Use `--enabled on|off`, `--threshold 0..100`, `--strategy`, `--sticky 1..100`, and
`--quota-window` only where supported. OpenAI does not support pool enabled/window
writes; `reset-first` is OpenAI-only, `least-loaded` Kiro-only, and Anthropic windows
writes; `reset-first` is for OpenAI and generic OAuth pools (never Anthropic), `least-loaded` Kiro-only, and Anthropic windows
are `five-hour`, `weekly`, `max-utilization`. Omitted fields stay unchanged.
Read DTO fields such as routes, per-account concurrency or Anthropic `nativeMessages`
do not imply a setter in this command; use only the listed CLI options. A save that
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1149,7 +1149,7 @@ second account.
| --- | --- | --- | --- |
| `oauthAccountFailover.enabled?` | `boolean` | presence-driven | Global override for the **pre-dispatch account preference** only. `false` stops a healthy request being steered toward the account with more known headroom. It does **not** disable 429 rotation. |
| `providers.<name>.oauthAccountFailover.enabled?` | `boolean` | inherits | Per-provider override for the same preference; beats the global setting in either direction. `false` declines the preference for this provider even when the global setting is `true`, and `true` opts this provider in even when the global setting is `false`. Reactive 429 rotation is unaffected either way. |
| `providers.<name>.oauthAccountFailover.strategy?` | `"quota" \| "round-robin" \| "fill-first" \| "least-loaded"` (Kiro only) | — | Kiro's `least-loaded` picks the eligible account with the fewest in-flight requests, using account order for ties. It requires `pool.kernel` and proactive preference; other strategies retain their existing rules. |
| `providers.<name>.oauthAccountFailover.strategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first" \| "least-loaded"` (Kiro only) | — | Kiro's `least-loaded` picks the eligible account with the fewest in-flight requests, using account order for ties. It requires `pool.kernel` and proactive preference; other strategies retain their existing rules. `reset-first` also requires `pool.kernel`: among eligible accounts that are not exhausted, the first attempt (while the proactive preference is on) and a 429 rotation both prefer the account whose quota resets soonest (the weekly window when one is reported, otherwise the 5-hour window), with more headroom breaking ties. Exhausted accounts go last. Without quota evidence, or when a passive provider has evidence for only some accounts, it falls back to headroom order. The Anthropic pool does not accept it. |
| `providers.kiro.oauthAccountFailover.maxConcurrentPerAccount?` | integer 1–100 | unlimited | A process-local per-account queue holds a slot through response completion or cancellation. A full selected account waits at most 250 ms, then returns 503 `account_capacity` with `Retry-After: 1`. The cap does not move the request to another account. |
| `providers.<name>.oauthAccountFailover.autoSwitchThreshold?` | `number` | `80` | 0–100 usage percent at which `fill-first` advances off the active account (#695). Set with `ocx account auto-switch <provider> threshold <n>`. Read only under `pool.kernel` with `strategy: "fill-first"`; an account with no measured usage counts as under the threshold. |
| `providers.<name>.oauthAccountFailover.stickyLimit?` | `number` | `1` | Successful dispatches retained on one `round-robin` selection, 1–100 (#695). Read only under `pool.kernel` with `strategy: "round-robin"`. |
Expand Down
2 changes: 1 addition & 1 deletion skills/ocx/references/01_surface_accounts.md
Original file line number Diff line number Diff line change
Expand Up @@ -858,7 +858,7 @@ State-changing: yes.
|---|---|---|
| `--enabled` | string | Stored pool preference; unsupported for Codex. Not equivalent to threshold0. |
| `--threshold` | number | Auto-switch threshold0–100. |
| `--strategy` | string | Use a strategy supported by this provider kind; reset-first is Codex, least-loaded is Kiro. |
| `--strategy` | string | Use a strategy supported by this provider kind; reset-first is Codex or generic OAuth, least-loaded is Kiro. |
| `--sticky` | number | Sticky limit1–100. |
| `--quota-window` | string | Anthropic only: five-hour, weekly or max-utilization. |
| `--json` | boolean | Emit the validated task result as one JSON document. |
Expand Down
2 changes: 1 addition & 1 deletion skills/ocx/references/03_recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -697,7 +697,7 @@ The target's `supported` list decides which fields can be written. Preserve null
`enabledEffective`, and optional `inert`: null is not false, stored policy is not
proof of an available account, and threshold zero is not the pool enable switch.
OpenAI has no pool `--enabled` or `--quota-window` writer. `reset-first` strategy
is OpenAI-only; `least-loaded` is Kiro-only. Anthropic pool windows are
is for OpenAI and generic OAuth pools, never Anthropic; `least-loaded` is Kiro-only. Anthropic pool windows are
`five-hour`, `weekly`, `max-utilization`. For a requested supported policy edit:

```bash
Expand Down
3 changes: 2 additions & 1 deletion src/cli/account-policy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,8 @@ async function pool(args: string[], json: boolean, deps: RuntimeApiDeps): Promis
if (limit !== undefined) patch.autoSwitchThreshold = integer(limit, 0);
if (sticky !== undefined) patch.stickyLimit = integer(sticky, 1);
if (strategy !== undefined) {
if (!["quota", "round-robin", "fill-first", ...(provider === "openai" ? ["reset-first"] : []),
// reset-first is a Codex and generic-pool strategy; the Anthropic pool does not run it.
if (!["quota", "round-robin", "fill-first", ...(isAnthropicInstanceId(provider) ? [] : ["reset-first"]),
...(provider === "kiro" ? ["least-loaded"] : [])].includes(strategy)) usage();
patch.strategy = strategy;
}
Expand Down
2 changes: 1 addition & 1 deletion src/cli/capabilities-accounts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ export const ACCOUNT_CAPABILITIES: readonly Capability[] = [
usage: "ocx account pool <provider> [--enabled <on|off>] [--threshold <0-100>] [--strategy <name>] [--sticky <1-100>] [--quota-window <five-hour|weekly|max-utilization>] [--json]",
summary: "Read or edit the selected runtime's supported pool policy fields.",
routes: [{"method":"GET","path":"/api/pool/settings"},{"method":"PUT","path":"/api/pool/settings"}],
flags: [{"name":"--enabled","value":"string","summary":"Stored pool preference; unsupported for Codex. Not equivalent to threshold0."},{"name":"--threshold","value":"number","summary":"Auto-switch threshold0–100."},{"name":"--strategy","value":"string","summary":"Use a strategy supported by this provider kind; reset-first is Codex, least-loaded is Kiro."},{"name":"--sticky","value":"number","summary":"Sticky limit1–100."},{"name":"--quota-window","value":"string","summary":"Anthropic only: five-hour, weekly or max-utilization."},{"name":"--json","value":"boolean","summary":"Emit the validated task result as one JSON document."}],
flags: [{"name":"--enabled","value":"string","summary":"Stored pool preference; unsupported for Codex. Not equivalent to threshold0."},{"name":"--threshold","value":"number","summary":"Auto-switch threshold0–100."},{"name":"--strategy","value":"string","summary":"Use a strategy supported by this provider kind; reset-first is Codex or generic OAuth, least-loaded is Kiro."},{"name":"--sticky","value":"number","summary":"Sticky limit1–100."},{"name":"--quota-window","value":"string","summary":"Anthropic only: five-hour, weekly or max-utilization."},{"name":"--json","value":"boolean","summary":"Emit the validated task result as one JSON document."}],
mutates: true,
json: "envelope",
details: ["Reads the target supported-field list before a narrow PUT. Null/unsupported, stored/effective and generic inert policy stay distinct; disabling does not promise that reactive429 rotation stops.","No local classification/defaults are round-tripped into the live target; supported-field reads are not CAS."],
Expand Down
67 changes: 67 additions & 0 deletions src/oauth/account-quota-rank.ts
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,73 @@ export function isAccountQuotaExhausted(
return headroom !== null && headroom <= 0;
}

/**
* Earliest future reset timestamp (in ms) for the relevant quota window, or null if unknown.
* Prefers weekly reset when multiple windows exist so expiring allowance is consumed first.
*/
export function accountResetTimestamp(
provider: string,
accountId: string,
requestedModelId?: string | null,
now = Date.now(),
): number | null {
const quota = getCachedProviderAccountQuota(provider, accountId);
if (!quota) return null;
const family = classifyModelFamilyForQuota(provider, requestedModelId);
const windows = (quota.customWindows ?? []).filter(w => !family || windowMatchesFamily(w.label, family));
const weekly = windows.find(w => /week/i.test(w.label));
let rawReset = weekly?.resetAt ?? quota.weeklyResetAt;
if (rawReset === undefined && provider !== "google-antigravity" && provider !== "anthropic") {
// Providers without a weekly cadence: prefer an explicitly 5h-labeled window before
// falling back to first-window order.
const fiveHour = windows.find(w => /5h|five/i.test(w.label)) ?? windows[0];
rawReset = fiveHour?.resetAt ?? quota.fiveHourResetAt;
Comment on lines +158 to +163

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Include monthly rollover evidence in reset-first ranking

When reset-first is enabled for Kiro—or for monthly-only xAI/Cursor quota responses—the strategy silently degenerates to headroom ordering: those quota readers store the known rollover in monthlyResetAt (and Kiro also exposes it through kiroAccountEvidence), but this selector only considers weekly, five-hour, and custom-window timestamps. Consequently every healthy Kiro candidate receives an infinite reset rank in both proactive selection and 429 rotation, so the advertised strategy never prefers the account that actually resets first. Include valid monthly/account-bound reset evidence among the candidate deadlines.

Useful? React with 👍 / 👎.

} else if (rawReset === undefined) {
rawReset = windows[0]?.resetAt ?? quota.fiveHourResetAt;
}
if (typeof rawReset !== "number" || !Number.isFinite(rawReset)) return null;
const ms = rawReset < 1e11 ? rawReset * 1000 : rawReset;
return ms > now ? ms : null;
}

/**
* Rank candidates by soonest reset timestamp first among healthy accounts.
* Accounts whose quota will reset earliest get priority so allowances do not expire unused.
*
* When a passive provider has incomplete quota evidence (some ring members with no
* cached record), reset timestamps are unreliable: an unknown account converts to
* Infinity and always loses to a known near-exhausted one. Fall back to headroom
* order instead of inventing a reset ordering.
*/
export function rankAccountsByResetFirst(
provider: string,
ring: readonly string[],
requestedModelId?: string | null,
now = Date.now(),
accounts?: ReadonlyMap<string, ProviderAccount>,
): string[] {
if (ring.length < 2) return [...ring];
const ranked = rankAccountsByHeadroom(provider, ring, requestedModelId, accounts);
const healthy = ranked.filter(id => !isAccountQuotaExhausted(provider, id, requestedModelId, accounts?.get(id)));
if (healthy.length < 2) return ranked;
if (
hasPassiveAccountQuota(provider)
&& !ring.every(id => headroomOf(provider, id, requestedModelId, accounts?.get(id)) !== null)
) return ranked;

const withReset = healthy.map((id, index) => ({
id,
resetAt: accountResetTimestamp(provider, id, requestedModelId, now) ?? Number.POSITIVE_INFINITY,
headroom: headroomOf(provider, id, requestedModelId, accounts?.get(id)) ?? 0,
index,
}));

withReset.sort((a, b) => (a.resetAt - b.resetAt) || (b.headroom - a.headroom) || (a.index - b.index));
const sortedHealthy = withReset.map(r => r.id);
const exhausted = ranked.filter(id => isAccountQuotaExhausted(provider, id, requestedModelId, accounts?.get(id)));
return [...sortedHealthy, ...exhausted];
}

/**
* Order candidates best-first.
*
Expand Down
23 changes: 21 additions & 2 deletions src/oauth/generic-account-failover.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import {
hasHeadroomEvidence,
isAccountQuotaExhausted,
rankAccountsByHeadroom,
rankAccountsByResetFirst,
classifyModelFamilyForQuota,
type QuotaModelFamily,
} from "./account-quota-rank";
Expand Down Expand Up @@ -351,7 +352,7 @@ export function rotateAntigravityAccountOnAuthRefusal(
}

/** Generic pool strategies the kernel can actually run. `quota` IS the pre-kernel path. */
type ActiveGenericStrategy = "round-robin" | "fill-first" | "least-loaded";
type ActiveGenericStrategy = "round-robin" | "fill-first" | "reset-first" | "least-loaded";

/** Matches the Codex and Anthropic pools; the DTO still reports `null` for "not stored". */
const DEFAULT_GENERIC_AUTO_SWITCH_THRESHOLD = 80;
Expand All @@ -365,10 +366,13 @@ const DEFAULT_GENERIC_AUTO_SWITCH_THRESHOLD = 80;
* branch instead of a four-way one.
*/
function activeGenericStrategy(config: OcxConfig, providerName: string): ActiveGenericStrategy | null {
// All pool strategies run behind the kernel flag; `quota` is the pre-kernel path.
// Letting `reset-first` bypass it would change 429 selection for pools that never
// opted into kernel-gated strategies.
if (config.pool?.kernel !== true) return null;
const raw = config.providers?.[providerName]?.oauthAccountFailover?.strategy;
if (providerName === "kiro" && raw === "least-loaded") return raw;
return raw === "round-robin" || raw === "fill-first" ? raw : null;
return raw === "round-robin" || raw === "fill-first" || raw === "reset-first" ? raw : null;
}

function genericStickyLimit(config: OcxConfig, providerName: string): number {
Expand Down Expand Up @@ -574,6 +578,10 @@ export function rotateGenericOAuthAccountOnRefusal(
}
return null;
}
if (strategy === "reset-first") {
return rankAccountsByResetFirst(providerName, candidates, requestedModelId, now,
new Map(set.accounts.map(account => [account.id, account])))[0] ?? null;
}
// With no quota evidence this returns the ring untouched, so providers without
// per-account quota keep exactly the traversal they have today.
return rankAccountsByHeadroom(providerName, candidates, requestedModelId,
Expand Down Expand Up @@ -697,6 +705,17 @@ export function preferredInitialAccount(
const picked = pickFillFirstGenericAccount(config, providerName, active, now, requestedModelId, accountRows);
return picked && picked !== active ? picked : null;
}
if (strategy === "reset-first") {
// Same candidate set as round-robin: paused, reauth-needed and cooled accounts are out,
// and an account whose model listing omits the requested model loses to one that has it.
// Reset ordering needs quota evidence; without it the ordinary active-account path runs.
const family = classifyModelFamilyForQuota(providerName, requestedModelId);
const eligibleNow = preferKiroModelSupport(providerName,
eligibleFailoverAccounts(providerName, now, family), requestedModelId, config);
if (eligibleNow.length === 0 || !hasHeadroomEvidence(providerName, eligibleNow, requestedModelId, accountRows)) return null;
const best = rankAccountsByResetFirst(providerName, eligibleNow, requestedModelId, now, accountRows)[0] ?? null;
return best && best !== active ? best : null;
}

const activeRow = selected.accounts.find(account => account.id === active);
if (activeRow && activeRow.paused !== true && activeRow.needsReauth !== true
Expand Down
6 changes: 4 additions & 2 deletions src/oauth/pool-settings-capability.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import { readAnthropicModelRoutes } from "./anthropic-model-routes";
*/
export type PoolSettingsKind = "codex" | "anthropic" | "generic";

export const GENERIC_POOL_STRATEGIES = ["quota", "round-robin", "fill-first", "least-loaded"] as const;
export const GENERIC_POOL_STRATEGIES = ["quota", "round-robin", "fill-first", "reset-first", "least-loaded"] as const;
export type GenericPoolStrategy = typeof GENERIC_POOL_STRATEGIES[number];

export function poolSettingsCapability(name: string, provider: OcxProviderConfig | undefined): PoolSettingsKind | null {
Expand All @@ -33,7 +33,9 @@ export function poolSettingsCapability(name: string, provider: OcxProviderConfig
}

export function parseGenericPoolStrategy(value: unknown, providerName?: string): GenericPoolStrategy | null {
// Keep the common three names in the shared parser; the fourth is Kiro-specific.
// reset-first is shared with the Codex pool, so it is named here rather than added to the
// three-name shared parser that the Anthropic kind also uses. least-loaded is Kiro-specific.
if (value === "reset-first") return value;
if (value === "least-loaded") return providerName === "kiro" ? value : null;
return parseAccountPoolStrategy(value);
}
Expand Down
15 changes: 10 additions & 5 deletions src/server/management/oauth-account-routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -597,10 +597,15 @@ export async function handleOauthAccountRoutes(ctx: ManagementContext): Promise<
// sticky limit is refused identically whichever pool is addressed.
let strategy: string | undefined;
if (fields.strategy !== undefined) {
const parsed = kind === "codex" ? parseCodexAccountPoolStrategy(fields.strategy) : parseGenericPoolStrategy(fields.strategy, provider);
if (parsed === null) return jsonResponse({ error: kind === "codex"
? "strategy must be one of: quota, round-robin, fill-first, reset-first"
: `strategy must be one of: quota, round-robin, fill-first${provider === "kiro" ? ", least-loaded" : ""}` }, 400);
// reset-first is a Codex and generic strategy. The Anthropic runtime implements only
// quota, round-robin and fill-first, so that kind keeps refusing it rather than storing
// a strategy it would silently ignore.
const parsed = kind === "codex" ? parseCodexAccountPoolStrategy(fields.strategy)
: kind === "generic" ? parseGenericPoolStrategy(fields.strategy, provider)
: parseAccountPoolStrategy(fields.strategy);
if (parsed === null) return jsonResponse({ error: kind === "anthropic"
? "strategy must be one of: quota, round-robin, fill-first"
: `strategy must be one of: quota, round-robin, fill-first, reset-first${kind === "generic" && provider === "kiro" ? ", least-loaded" : ""}` }, 400);
strategy = parsed;
}
let stickyLimit: number | undefined;
Expand Down Expand Up @@ -754,7 +759,7 @@ export async function handleOauthAccountRoutes(ctx: ManagementContext): Promise<
if (body.strategy === null) delete next.strategy;
else {
const parsed = parseGenericPoolStrategy(body.strategy, provider);
if (parsed === null) return jsonResponse({ error: `strategy must be one of: quota, round-robin, fill-first${provider === "kiro" ? ", least-loaded" : ""}` }, 400);
if (parsed === null) return jsonResponse({ error: `strategy must be one of: quota, round-robin, fill-first, reset-first${provider === "kiro" ? ", least-loaded" : ""}` }, 400);
next.strategy = parsed;
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/types/provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -726,7 +726,7 @@ export interface OcxProviderConfig {
* contract. Consumed by the selector only while `pool.kernel` is on; with the flag off
* it is still merely persisted, so omitted and set behave the same.
*/
strategy?: "quota" | "round-robin" | "fill-first" | "least-loaded";
strategy?: "quota" | "round-robin" | "fill-first" | "reset-first" | "least-loaded";
/**
* 0-100 usage percent at which fill-first advances off the active account (#695).
* Read only under `pool.kernel` with `strategy: "fill-first"`; 80 when unset, matching
Expand Down
11 changes: 11 additions & 0 deletions structure/transports/inventory.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,17 @@ the same routed model during account rotation, without bypassing their send-budg
admission or account-snapshot pairing. The forwarding contract is covered in
`tests/oauth/oauth-account-quota-rank.test.ts`; the core facade remains orchestration-only.

Generic OAuth pools accept `reset-first` under `pool.kernel` (`activeGenericStrategy` in
`src/oauth/generic-account-failover.ts`). `rankAccountsByResetFirst` orders non-exhausted
accounts by the soonest future reset of the requested family's weekly window, falling back to
the 5-hour window, then by headroom; exhausted accounts go last. A passive provider with
evidence for only part of the ring keeps headroom order. The first-attempt pick uses the same
eligible set as round-robin, including per-account model listings, and needs headroom evidence.
Kiro passes account-bound quota evidence through both selection paths, preserving headroom
ordering and explicit exhaustion when reset times are unknown. The Anthropic kind refuses
the strategy. Coverage: `tests/oauth/oauth-account-quota-rank.test.ts`
and `tests/server/account-pool-management-api.test.ts`.

## SOCKS5 dispatch boundary

`src/lib/socks5-handshake.ts` shares RFC1928/RFC1929 framing between the fetch tunnel and
Expand Down
Loading
Loading