Repository navigation
Revive the aimlapi aggregator build on current upstream - #1
Merged
Merged
Conversation
…eaders The AI/ML API provider pointed at https://api.aimlapi.com/v2, which does not exist on the public API (all /v2 routes 404); chat completions therefore failed. Point the provider at /v1, and send X-AIMLAPI-Source / X-AIMLAPI-Partner-ID so provider-side attribution works. Extend the dispatch test to cover both.
…vider Sort featured providers (AIML API) to the top of the builder integration provider list, and show the provider as 'aimlapi.com' in the credentials UI.
…ld arg
Lets a hosted deployment bake its real public origin into the client bundle
(the browser API client calls ${NEXT_PUBLIC_FRONTEND_BASE_URL}/api/proxy).
Defaults to localhost; overridden via compose build.args on the dev server.
… origin
getServerAuthToken fetched ${BETTER_AUTH_URL}/api/auth/token — the Next server
calling itself via the PUBLIC url. Behind a reverse proxy the container cannot
reach its own public origin (hairpin), so every SSR token fetch hung to the 10s
timeout and login/signup failed. Target the loopback instead (AUTH_SELF_BASE_URL
override, default http://localhost:3000).
…integration surfaces - Display name -> 'aimlapi.com' in the builder integration list, the Connect-a- service modal, the credential field, and the model picker provider label. - New provider description (text/image/video/audio/embeddings, one key). - Green 'Recommended' badge + pin aimlapi.com first in both the builder list (backend featured order) and the Connect-a-service modal (sort).
Backend: self-contained api/features/aimlapi module (config + service + router)
implementing the RFC 8628 device-authorization 'agent-auth' flow — POST
/api/aimlapi/authorize/{start,poll}; device code kept server-side, X-AIMLAPI-Source
(agent/autogpt) + X-AIMLAPI-Partner-ID on every call, source on the consent URL.
Frontend: 'Get API key' button in the aimlapi.com Connect-a-service form (opens
the consent tab, polls, auto-fills the key, green success / red error), prefilled
Name 'My aimlapi.com key'.
…API_INFERENCE_URL) Lets a deployment point inference at staging (api-staging.aimlapi.com/v1) without a code change; defaults to production.
…design - window.open must not use noopener when we need the handle to redirect the tab; with noopener it returns null, leaving a blank about:blank page - move the 'Have a key? Paste it here.' hint to a full-width label row (right-aligned) and drop the divider bars around 'or'
- provider descriptions in the connect list row and detail header wrap to two lines (line-clamp-2) instead of a single truncated line that ran off to the right; the list row grows (min-h) to fit the second line - drop the extra 'Have a key? Paste it here.' hint on the AIMLAPI key field - put the key input, 'or' and the button on one horizontal line and give the button the same rounding (rounded-xl) and height as the input
Fetch the AIMLAPI chat catalog from GET /v1/models at import (with a committed snapshot fallback) and inject every non-duplicate model as an aiml_api LlmModel member, so a single AIMLAPI key serves ~260 models (OpenAI, Anthropic, DeepSeek, Qwen, GLM, Kimi, …) across all LLM blocks. - new backend/blocks/aiml_models.py loader + data/aiml_models_snapshot.json - llm.py: post-hoc member injection, skipping ids that already resolve to a native member so existing graphs are never rerouted; metadata + pricing + hottest set built for injected models - block_cost_config.py: MODEL_COST/TOKEN_COST derived from catalog pricing - frontend: aimlapi provider icon, is_hottest flag, and a "Hottest" section pinned to the top of the model picker
This reverts commit 5947232.
…atalog"" This reverts commit 12dca57.
Surface only AIMLAPI-served models in the LLM Model picker and default new blocks to an AIMLAPI model, so every model in a block runs on the single aimlapi.com key instead of each provider's own key. - llm.py: DEFAULT_LLM_MODEL now resolves to a flagship AIMLAPI model (with a native fallback so a missing catalog can't break import) - LlmModelField: filter the picker to provider == "aiml_api"
- catalog loader dedupes API alias twins by (developer, display name), so a model exposed under several id spellings (e.g. gemini-3.6-flash vs gemini-3-6-flash, the 3 GLM 5.2 ids) appears once - model picker: replace the flat "Hottest" list + single "Recommended" item with a single "Recommended" group (star) pinned first, holding all hottest models and reading like the creator groups below it
Extract the AIMLAPI device-grant flow into a shared useAimlapiGetApiKey hook and reuse it in the builder's "Add new API key" modal (APIKeyCredentialsModal), so the aimlapi.com provider gets the same "Get API key" button next to the key field as the settings Connect-service form. Widen the button 1.5x in both forms.
Stagehand blocks only run OpenAI/Anthropic models (StagehandRecommendedLlmModel) but their model credential used the shared AICredentials, which advertises every LLM provider — surfacing Stagehand under each provider's integration (incl. aimlapi.com) despite it not being usable there. Narrow the field to Anthropic/OpenAI so Stagehand no longer appears under those integrations.
… providers" This reverts commit 19cd696.
Add POST /api/aimlapi/validate-key: sends an empty /chat/completions request (auth is checked before the body, so a bad key returns 401 while a valid key returns 400 — no completion generated, zero cost). Both credential forms (the settings Connect form and the in-builder API-key modal) verify a manually entered aimlapi.com key on submit and show an error instead of saving an invalid key. Fail open on network/backend errors so a real key is never blocked.
Drop the backend validate-key endpoint. The balance endpoint is CORS-enabled, so the browser verifies an entered key directly (401 = invalid). Base URL comes from NEXT_PUBLIC_AIMLAPI_API_URL (prod default) so a staging deploy checks against staging.
Port the AIMLAPI catalog onto upstream's catalog-as-code registry. Upstream moved model facts out of `blocks/llm.py` into `data/llm_registry/` (catalog.py holds the models, registry.py the in-process view, llm_models.py the enum + projections). Our dynamic catalog lived in the file that got emptied, so this is a port, not a merge resolution: * `blocks/aiml_models.py` -> `data/llm_registry/aiml_catalog.py` (with its snapshot). It is catalog code and now sits next to the catalog; keeping it under `blocks/` would have made `llm_models` import `backend.blocks`, whose package init pulls `_base` — an import cycle. * `catalog.py` keeps its hand-authored payload as `_build_static_catalog()` and `_build_catalog()` appends the loaded AIMLAPI models. Injected entries never shadow a slug the static catalog declares. * `llm_models.py` registers a matching enum member per injected slug — `_build_model_metadata()` joins catalog to enum BY SLUG, so a catalog entry without a member is invisible. Slugs that already resolve (directly or via `_missing_`'s provider-prefix strip) are skipped so an aggregator entry never reroutes an existing graph through AIMLAPI. * `MODEL_COST` now comes from upstream's catalog-derived path; the injected entries carry `cost.run_credits`, so our old manual loop is gone. `TOKEN_COST` still bills from `AIML_TOKEN_PRICING`. * `DEFAULT_LLM_MODEL` still prefers an AIMLAPI model — this is the aggregator build, a fresh install must work with the one key the user has. Upstream's catalog recommendation stays the fallback, and `llm_models_test` now asserts that contract instead of the native default. Dropped: our SSR auth-token loopback fix. Upstream fixed the same bug independently and better (`BETTER_AUTH_INTERNAL_URL` + a bounded hop). Verified: 51 llm_registry tests + 84 providers tests pass; the live catalog injects 275 AIMLAPI models (302 loaded, 27 shadowed by native slugs) and the default resolves to openai/gpt-5.6-terra-pro via aiml_api. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
HEADERS.md requires X-AIMLAPI-Source + X-AIMLAPI-Partner-ID on EVERY aimlapi.com request — "inference/API calls, catalog, checkout, and auth alike, not just login/sign-up". The catalog loader called GET /v1/models bare, so the one request every boot makes was the one request that went unattributed. Reuses `attribution_headers()` from the feature config rather than restating the pair, so the id cannot drift between the two call sites. The import is function-local: that module is stdlib-only, and keeping it out of the module header avoids `data/` taking a load-time dependency on `api/`. Verified live: the fetch now carries both headers and still returns the full 302-model catalog. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last aimlapi.com call that went out unattributed. HEADERS.md wants the pair on every request; this one is issued from the browser, so it cannot inherit the backend client's default headers and has to carry them itself. Custom headers turn this from a simple request into a preflighted one, so this is only safe because the API allows them: OPTIONS on /v1/billing/balance returns access-control-allow-headers "authorization,x-aimlapi-source,x-aimlapi-partner-id" and reflects the caller's origin. Verified against production, and the GET still answers 401 for a bad key — the status this function keys off is unchanged. The partner id also needs a Dockerfile ARG: Next inlines NEXT_PUBLIC_* at build time, so without it the documented env override could never reach the bundle. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
This PR targets the Automatically setting the base branch to |
github-actions
Bot
added
platform/blocks
cla: pending
Lookoff-AIMLAPI
pushed a commit
that referenced
this pull request
Sep 11, 2026
…Significant-Gravitas#14244) Every turn a Copilot agent spawns now inherits a permission slip that can only narrow — which tools it may use, how deep the chain may go, whether its context is tainted — and every tree of spawned turns is capped on how many agents it may run and metered against a per-tree spend ceiling, so a delegation cannot hand a child more authority than its parent held or run away on cost. ### Why / What / How **Honest read first, because it is more useful to a reviewer than a capability claim.** This PR ships a *substrate*, not a demonstrated capability win. Two things it enables are structurally unavailable to a single agent: **width made grantable** (N clean child contexts whose spend is bounded per tree rather than by the concurrency cap) and **quarantine** (a child pinned read-only can be pointed at untrusted input with no action surface to steer). **Quality-at-width is unproven** — whether N narrow children plus a merge beat one careful agent on correctness is untested, and T5's evidence is about process discipline, not correctness. Treat the capability case as the next experiment, not as shipped value. **Why.** Spawned copilot turns (`run_sub_session`, `delegate_to_expert`, `handoff_to_expert`) currently inherit their parent's authority *unchanged* — `permissions=get_current_permissions()` is copied verbatim, so a child holds exactly what its parent holds with no way to give it less. Separately, the 3-hop chain bound is computed by walking `delegated_by_session_id`, which same-identity sub-sessions never write, so isolates do not count toward it. Neither is exploitable today (see **Reachability** below), but together they mean there is no mechanism by which a spawn can be *narrowed*, and no bound on a tree's total spend or size. **What.** Every copilot turn now carries a `TurnEnvelope` (tree id, depth, tool ceiling, taint bit, deadline), derived at one chokepoint from the *spawning turn's* envelope by operations that can only narrow. Every tree gets a Redis ledger bounding node count and metered spend. **How.** `dispatch_turn` is the single seam every turn passes through — HTTP chat, scheduler, `AutoPilotBlock`, and all three spawn tools — so the envelope is derived and admitted there and nowhere else. Enforcement is in `BaseTool.execute`, so a tool outside the envelope is *refused* on both engines rather than merely hidden from the schema. ### What a tree can spend, per plan A tree — one root turn plus every turn it spawns — stops at whichever comes first: a per-tree spend ceiling sized from the user's plan, a cap of 8 turns including the root, or 3 hops of depth. The ceiling is `min(remaining daily/weekly budget, max(½ × the plan's daily allowance, $0.50), $10.00)`, resolved once when the first spawn opens the tree (`copilot/tree.py:380-418`), so it scales with the plan instead of being a flat number. The figures below are the code defaults, and production serves them unchanged: both LaunchDarkly flags that can override them — `copilot-cost-limits` and `copilot-tier-multipliers` — are **off** in production and serve `{}`, which both parsers read as unset and fall through to `ChatConfig` (`copilot/rate_limit.py:201-231` and `233-257`). The one thing that could move them is a `CHAT_DAILY_COST_LIMIT_MICRODOLLARS` override on the backend deployment, which I could not read. | Plan | Daily allowance | Weekly | Per-tree ceiling | Per-turn cap | |---|---:|---:|---:|---:| | NO_TIER | $0.00 | $0.00 | $0.00 | — | | BASIC | $1.00 | $5.00 | $0.50 | $1.00 | | PRO | $5.00 | $25.00 | $2.50 | $5.00 | | MAX | $20.00 | $100.00 | $10.00 | $10.00 | | BUSINESS | $60.00 | $300.00 | $10.00 | $10.00 | | ENTERPRISE | $60.00 | $300.00 | $10.00 | $10.00 | Allowances are the `ChatConfig` base ($1.00/day, $5.00/week — `copilot/config.py:361-370`) times the tier multiplier (`copilot/rate_limit.py:108-122`). Ceilings are what `resolve_root_ceiling_microdollars` returns for a user who has spent nothing yet; they shrink through the day as spend accrues, because `remaining` is the first term of the `min`. The per-turn cap is `min($10.00, remaining daily/weekly)` floored at $0.50 (`copilot/sdk/service.py:256-297`) and applies on the SDK path; a Codex-transport turn gets no USD cap, since it spends the connected account's own allowance. **At the edges.** Reaching the ceiling mid-task does not fail the user's request — the spawn tool hands the calling agent "This task has spent its budget; report what you have instead of starting more work", and that agent finishes with what it already has (`copilot/tree.py:324-328`, `tools/run_sub_session.py:541`). The node cap reads the same way and says outright that 8 is a lifetime budget for the request rather than a concurrency limit, so waiting will not free one. NO_TIER resolves to a $0 ceiling, so a user with no subscription gets no delegation; in production they never reach a turn at all, because `enable-platform-payment` serves `true` on every rule and on the fallthrough there, and `enforce_payment_paywall` returns 402 first. The ceiling is metered rather than reserved — `spent` advances only when a turn *ends*, so spawns admitted in the same round all read the same figure and are all let through; the worst case is the 7 non-root nodes admitted together, each running to its own per-turn cap, which is $7.00 on BASIC and $70.00 on MAX and above before the tree stops admitting. Two numbers here are decisions this PR does not settle: the $0.50 ceiling floor, which funds about one median turn and arguably wants to be one median *delegation* nearer $3.00, and `tree_max_nodes = 8`, which is what bounds the overshoot above. ### Changes 🏗️ **New** — `backend/copilot/tree.py`: - `TurnEnvelope` + `derive_child_envelope()` (pure): `depth+1 ≤ 3`, `tools = spawner ∩ (request or ALL − denied)`, `deadline = min(...)`, `tainted = spawner OR born`. No field a request supplies can widen. - `TreeLedger`: one Redis hash per tree, opened by the **first spawn** (roots never touch Redis, so HTTP chat / scheduler / `AutoPilotBlock` paths are unchanged). Admits on node cap + metered spend ceiling; charges from the existing cost-recording path. It **meters, it does not reserve** — `max_budget_usd` is per-query, floors up to $0.50, and does not exist on the Codex transport, so a reservation could never be enforced. **Two-tier child defaults, and why the asymmetry is principled rather than incidental:** | | denied | reasoning | |---|---|---| | **Every child** (`DESCENT_DENIED_TOOLS`) | `connect_integration`, `setup_agent_webhook_trigger`, `update_preset`, `delete_*`, staffing tools | Binds the account or is irreversible. Not part of any delegate's normal job, so withholding breaks nothing that works. | | **A delegate additionally KEEPS** | `post_to_chat_platform`, `schedule_followup`, `run_mcp_tool`, `store_skill`, `memory_store`, `add_understanding` | A delegate runs under **its own** identity, memory namespace and budget. These are its job. The scheduled-standup post (the `chat_platform` headline use case) and MCP delegations keep working. | | **An isolate additionally loses** (`ISOLATE_DENIED_TOOLS`) | the six above | An isolate shares the **spawner's** identity and namespace, so its `memory_store` is read back by the parent next turn as the parent's own memory, and its post goes out under the parent's name. The restriction is tied to the actual reason for it, not applied uniformly because uniform is tidier. | **There is no escape hatch, and that is deliberate.** No spawn tool takes a grant parameter, because under absolute denial one cannot exist: the derivation subtracts `denied` last (`tools = (ceiling & requested) - denied`), so a granted tool that was denied would be stripped and one that was not denied is already in the default set. An isolate that genuinely needs `memory_store` is a human-approval case, not a self-service one. `_CHAR_BUDGET` stays at 61,000; the registry measures 60,280 against it. **Other enforcement:** - `BaseTool.execute` refuses a tool outside the turn's envelope — both engines, all registry tools. - Spawns and resumes never queue into an in-flight turn (that path would run a child's prompt inside *another* session's turn under *that* turn's permissions). - `run_sub_session` writes `delegated_by_session_id`; **resume** now requires the creating session, which removes a self- and sibling-resume deadlock. Reads keep their existing scope-equality rule — narrowing those too would break a same-scope sub reporting progress and a handed-off expert reading its own task, for no authority gain (a same-scope reader is the same identity under the same user). - Isolates count toward the depth bound and are `may_spawn=True` (matching today's ability to nest, now bounded). **Docs** — `docs/platform/contributing/agent-collaboration-architecture.md` (filed beside the other contributing docs and added to `SUMMARY.md`; review suggested `docs/content/` per AGENTS.md, but that path is stale — the file AGENTS.md names there lives in `docs/platform/`): the design, both adversarial roasts recorded with what they changed and what I rejected, and §2.6 establishing that the typed **brief/report is the general mechanism** and `consult_teammate` (Significant-Gravitas#14207) is **edge type #1**. That framing is what stops a third payload shape being invented next month. ### Reachability of the two grounding facts (traced, not assumed) - **Permissions equal on descent:** for a child to *gain*, a restricted parent would have to spawn a less restricted child. Interactive/expert sessions run with `permissions=None`; the only non-`None` filter comes from `AutoPilotBlock`, and it is forwarded, so children get **equal**, never more. The one real gap is the `_parent` block-filter `PrivateAttr` being dropped on the queue — within one user's own account, requiring the author to have set a block filter, gaining no cross-user reach. Defence-in-depth weakening, not a vulnerability. This PR closes the tool dimension; `block_filters` is a named follow-up. - **Depth bound skipping isolates:** exceeding depth buys only more hops, i.e. more spend — already capped by the per-user daily/weekly cost limits and the 15-in-flight cap, with loops caught by the existing `seen` walk. No capability appears at depth 4 that was absent at depth 1. Churn-guardrail gap, not a vulnerability. Closed here anyway. ### Deploy notes - **In-flight sub-sessions keep working.** Resuming a sub now requires the session that opened it, and that provenance was never written before this PR. Rather than strand every sub that exists at rollout, a sub with no provenance falls back to the scope + origin rules it was created under. The group is closed (nothing new joins it) and ages out with the 6h turn lifetime. - **Behaviour change:** continuing a delegation to a teammate who is still working (`delegate_to_expert` with `delegated_session_id`) used to queue into their running turn. It now returns a refusal telling the caller to wait or start fresh — a queued prompt would otherwise execute under that turn's envelope rather than its own. `AutoPilotBlock` keeps the queueing behaviour. - **Behaviour change:** sub-sessions now record delegation provenance, so a sub hop counts toward the delegation chain bound like any other hop, and a sub delegating back to its own parent expert is correctly refused as a loop. - **Config:** four new fields, all with defaults, no `.env` change required: `tree_ceiling_fraction_of_daily` (0.5), `tree_ceiling_floor_microdollars` ($0.50), `tree_ceiling_microdollars` ($10.00, the absolute cap) and `tree_max_nodes` (8). What they produce per plan is in the table above. ### Not yet reachable from any tool Stated so the shipped surface is not read as wider than it is: `tainted` is propagated and tested but has no consumer until the auto-mode gate lands; `SpawnRequest.max_seconds`, `SpawnRequest.tools` (the exact-set pin) and `may_spawn=False` have no caller, so deadlines never fire, no leaf is created, and the read-only quarantine shape is expressible but not selectable. Tracked in the module docstring. ### Known limit worth a reviewer's attention **1. `run_agent` resets authority, depth and budget — not just outward reach.** `run_agent` is not in `DESCENT_DENIED_TOOLS`, so a narrowed child keeps it. It dispatches through `add_graph_execution` into the **graph-executor process**, where the envelope contextvar does not exist; if that graph contains an `AutoPilotBlock`, the turn it starts is minted as a **root**: `tools=None`, `depth=0`, `tainted=False`, and a fresh tree with its own ceiling and node cap. So the child's tool narrowing, its place under the depth bound, and its share of the tree's budget are all reset. Nothing here regresses against base (spawned turns previously had no envelope at all), and **Significant-Gravitas#14265 closes it** by carrying tree identity through `ExecutionContext` into the executor process, the same way `expert_id` already rides it for billing. Adding `run_agent` to `DESCENT_DENIED_TOOLS` is the one-line alternative if that stack does not land; it costs every delegate the ability to run a graph, which is why it is not the default choice. **2. A descent-denied child is not an outward-safe child.** The default narrowing removes the named outward tools but leaves `run_agent`, and `run_block`/`run_agent` reach ~78 outward blocks plus arbitrary HTTP via `SendWebRequestBlock`, none flagged `is_sensitive_action`. So the quarantine guarantee holds **only** for a child pinned to an explicit read-only tool set, never for one that merely inherited the default. Stated as a rule in the design doc rather than left for a reader to assume. **3. The spend ceiling is metered, not reserved, so it does not bind under parallel fan-out.** `spent` advances only when a turn *ends*, so children dispatched in one round all read the same figure and are all admitted. The honest bound is `(max_nodes − 1) × the per-turn cap` — held finite by the **node** cap, not by the ceiling — and it is widest in exactly the wide-fan-out shape the design doc advertises, which is what the ledger docstring and the design doc state too. Reserving at admit and reconciling at charge is the fix, and it needs a per-query spend figure the Codex transport does not expose. ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] `tree_test.py` — monotonicity of every envelope field, refusal on every overflow (depth, nodes, spend, expired deadline), concurrent admits against a fake Redis, fail-closed for children / open for roots when Redis is down - [x] **Enforcement-seam tests** — `BaseTool.execute` refuses a denied tool and never reaches `_execute` (mutation-checked: deleting the enforcement block fails it with "the denied tool's body executed anyway"); all three spawn tools pass `allow_queue=False` and `run_sub_session` passes `shares_memory=True`; `charge_turn` is reached with the right microdollar amount; a `source_platform` session promotes a clean envelope to tainted - [x] **Real pytest, 1855 passed / 21 skipped** across `copilot/`, `copilot/tools/`, `copilot/executor/` on the final head - [x] `scripts/tree_fanout_demo.py` — 30 spawns under one root: 7 admitted at `max_nodes=8`, leaves cannot spawn or act outward, depth refuses hop 4 through an isolate→delegate→isolate chain, a child never regains a tool its spawner lacks, a tree past its ceiling admits nothing - [x] Executed 94 synchronous test bodies across `sub_session_test` / `permissions_test` / `context_test` directly (0 failed, 9 skipped needing fixtures) - [x] Imported every touched test module (catches stale imports after the rename/merge) - [x] `pyright` clean on all 12 changed files - [x] Merged current `dev` and re-verified all of the above against the merged base #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) Four new `ChatConfig` fields, all with defaults and no `.env` change required, listed under **Config** in **Deploy notes**. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
hugoaimlapi
pushed a commit
that referenced
this pull request
Oct 13, 2026
…ignificant-Gravitas#15412) ### Why / What / How **Why.** Right after "Subscribe to Pro" on a cancelled trial (the Significant-Gravitas#15382 path), the plan card read "Renews on Oct 18", the old trial end, until a reload. The real date was Nov 11. Found by the patrol's hosted dogfood on dev (ISSUE-005). Someone who has just paid sees a second charge coming in a week. **Root cause.** The status endpoint reads the active subscription through `_get_active_subscription_cached` (`backend/data/credit.py`), a 15 s cache held **in each process's memory**. The conversion cleared only the worker that served the purchase. The frontend then: 1. refetches status just before buying (`isStillOnTrial()`), which warms worker B with the trial; 2. sends the purchase, which converts on worker A and clears A only; 3. refetches status, which lands on B and is served the trial's `current_period_end`. Nothing refetches again until a reload. No webhook invalidated the cache at all. The frontend's react-query cache doesn't cause this (no optimistic update; its refetch only received stale data), so there is no frontend change. **How.** `backend/data/subscription_lookup_cache.py` (new) owns the cache, all in **async** Redis. - **Keys:** per customer, `…:{cus}:version` (a random token, 24 h TTL) and `…:{cus}:lookup` (JSON of the 4 display fields plus the token it was read under, 15 s TTL). The hash tag puts both on one cluster slot, so one `MGET` reads them and one script can check and write them. - **Read:** a hit needs the entry's token to equal the current token. - **A missing token is never a version.** A reader that finds none establishes a random one (`SET NX`; if it loses the race it reads the winner's) *before* asking Stripe, and before it picks its in-flight key. So a read after Redis lost its data never joins a lookup started before the loss. - The answer is written by a Lua script **only if the token is still the one it was read under**. So a slow read that started before a change can't write after it: not after a bump, and not after Redis lost its data. It also can't overwrite a fresh entry. - **Invalidation:** writes a fresh token, so every worker's next read misses. - **Coalescing:** within a process, concurrent lookups for one customer share one Stripe call. The shared call is keyed by the token the caller read, so a read after a change never joins one started before it. It re-reads the entry before asking Stripe, so concurrent first reads still make one call. - **Ownership:** a lookup stores its answer (Redis entry or fallback memo) only while it still owns its in-flight slot. Invalidating in this process drops the slot, so an answer read before the change goes to the callers that were already waiting and is never kept. - **Failed invalidation:** if an invalidation can't store the new token, a write already on its way lands under the old one. So this process marks the customer until any such entry has expired (entry TTL + 1 s). Meanwhile it reads that customer as if Redis were out and writes nothing for it. A later successful invalidation clears the mark. A live mark is never evicted. Expired marks are purged first, and if the map is still full of live marks, every customer is treated as unconfirmed until the latest mark would expire. The fallback memo may drop its oldest answer, which only costs a Stripe call. In-flight lookups aren't bounded and leave only when done or invalidated. - **Redis is only an optimisation; billing never waits on it:** - Every Redis operation, connecting included, runs under `asyncio.wait_for(…, 0.3 s)`. The shared client's own connect retries can take minutes; the deadline cuts them off. - After any failure or timeout, this process skips Redis for reads for 5 s. - Meanwhile each process keeps its answers for 5 s, which bounds Stripe traffic to one lookup per customer per worker per 5 s. Another worker can show the old state for up to 5 s after a change made while Redis is out; this process drops its own copy when it makes the change. - **A standalone (non-cluster) Redis gets only this fallback, never caching.** docker-compose and single-container both run Redis in cluster mode. - Invalidation never raises, finishes within the deadline, and always comes **after** the state change it follows. - Invalidations don't skip during the backoff. Skipping would let a Redis blip bring back the stale date after a purchase. The cost while Redis hangs is at most 0.3 s per invalidation; the trial reconciliation sweep pays it once per trial subscription. - **What's cached:** only `status`, period start/end and price id. A pickled `stripe.Subscription` carries its API key (verified locally), and the old `@cached` path pickled. The sync `@cached` wrapper is no longer used on this path. It made synchronous Redis calls inside the event loop on a client with no read timeout. ### Changes 🏗️ **Cache reads (display only):** - `_get_active_subscription_cached` calls `read_active_subscription(customer_id, lambda: _get_active_subscription(customer_id))`. - Its callers read the cached fields: `get_proration_credit_cents` (status, period start/end; 0 when either is missing), `get_user_billing_cycle` (price id) and `get_active_subscription_period_end` (period end). - These feed `GET /credits/subscription`, and the `POST` response through `get_subscription_status`. - Mutation flows (modify, release, pending change, rate-limit tier) still call the uncached `_get_active_subscription`. **Invalidations (each after the change it follows):** - `sync_subscription_from_stripe`: in a `finally` after `_sync_subscription_tier_from_stripe`, which includes trial reconciliation that can cancel a trial in Stripe. It runs even when the save fails. This covers: - every `customer.subscription.*` and `trial_will_end` webhook, and `subscription_schedule.*`; - checkout completion and the reconcile sweeps; - the notification service's trial syncs (another process; reached through Redis); - trial cancel / resume / convert. - `_save_conversion`: the paid tier is saved first (as before, a failed save is logged and the request still succeeds), and the invalidation follows in a `finally`. - `_cancel_customer_subscriptions`: in a `finally` whenever it touched a subscription. The loop moved, unchanged, into `_cancel_listed_subscriptions`. - `modify_stripe_subscription_for_tier`: in its existing `finally`, after the Stripe change and the DB flip. - `release_pending_subscription_schedule`: in its existing `finally`. - `handle_subscription_payment_success` (`invoice.payment_succeeded` / `invoice_payment.paid`): for subscription invoices, since a paid invoice can start a new period. Nothing after it changes Stripe, so the order doesn't matter there. It no longer waits for the `customer.subscription.updated` that usually follows. Unchanged: `get_pending_subscription_change` (already Redis-shared, deleted by `user_id`) and the price-amount cache. **Tests** - `backend/conftest.py`: an autouse fixture gives every test its own cache key prefix and a reset backoff, because Redis entries outlive a test and are shared across parallel runs. - `backend/data/subscription_lookup_cache_test.py`, against real Redis. "Worker B" is a separately loaded copy of `backend.data.credit` *and* of the cache module, so its process-local state is its own. One fake Stripe sits at the SDK boundary. - **Shared hit:** one worker's lookup serves both workers. The Stripe lookup count stays 1 and the entry is read back from Redis. - **Purchase on A → read on B:** paid period end; then A reuses it with no new Stripe call. - **Webhook on A → read on B:** fresh. - **Read in flight during the purchase:** its old answer is not served, and doesn't overwrite the entry A refreshed (2 Stripe lookups in all). - **Read before Redis lost its data:** trial read → purchase → keys deleted → slow answer arrives. Nothing is written and the next reads are fresh (Codex r2 P2-1). - **60 concurrent misses on one worker → 1 Stripe lookup** (60 on the previous head). - **Read after Redis lost its data** does not join an older lookup (Codex r3 #2; on a69609e it waited for the old lookup and timed out). - **Answer read before a change is not kept during an outage** (CodeRabbit r4243795891 / Codex r3 #1; on a69609e it served the trial period from the memo). - **A write racing a failed invalidation** (Codex r4): the write is held after the ownership check; the change plus a failed token SET follow; a fresh read returns paid; the write is released and lands under the old token; after the backoff this worker still serves paid. Red on 174a082. - **A live mark survives 2,100 other failed invalidations**, and **a renewed mark outlives expired ones** (Codex r5). Both red on 916699b. - **Seeded interleaving test** (`subscription_lookup_cache_interleaving_test.py`; 300 seeds, about 1.3 s, fake clock, no sleeps). - Invariant: no read that starts after this worker's change and invalidation returns the old subscription. - It interleaves reads, changes, Redis outages, failed SETs, replies lost after the write applied, lost data, slow Stripe answers and slow script writes (dropped past the client deadline), with memo and mark bounds shrunk to 1. - Mutation check: disabling memo ownership, the unconfirmed-read guard, local forgetting or the no-evict rule each makes it fail. - It doesn't model another worker's changes; the targeted token-loss test covers that. - The fake Stripe holds one *lookup*, not one task: on Python 3.11, `asyncio.wait_for` inside `stripe_call` runs each SDK call in a task of its own. That was the 3.11-only `None` on a69609e. Verified on 3.11.15 and 3.12. - **Order:** the tier is saved, then invalidated, even when saving fails. - **Outage**, for each of connect hangs / command hangs / refused / standalone Redis: - 60 concurrent plus 100 sequential lookups are correct, take under 1.5 s in total, and make **1 Stripe lookup** (160 on the previous head; Codex r2 P2-2); - the conversion completes in under 2 s and saves the paid plan; - the next read shows the new period. - A `redis_up` fixture fails the healthy-path tests when Redis is unusable. Every wait is bounded, and fake Stripe answers take 20 ms so that "concurrent" lookups really overlap. - `subscription_trial_conversion_test.py`: the real conversion's call order is `modify, sync, invalidate, unlock`, also when the sync fails. - Updated `credit_subscription_test.py` and `credit_trial_subscription_test.py` to the model and the async invalidation. User-ID scoping (`data/*.py`): no new user-facing query. The cache key is the Stripe customer id, which each caller resolves from the authenticated `user_id` via `get_user_by_id`, as before. Invalidations take the customer id from our own user row or from a Stripe-signed payload. **Playwright.** `test.fixme` for billing journey J5: "trial → cancel → Subscribe to Pro → the plan card shows today + 1 period without reload". It can't buy in CI until stripe-mock lands (W13), so it belongs in J5 once W13 is in. ### Agents and large language models used Claude Code — Claude Opus 5.5 ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [ ] I have tested my changes according to the test plan: - [x] The cross-worker tests **fail on origin/dev** (`assert 1792354142 == 1794427742`: the trial end against the paid period end) and pass here. - [x] With dev changed to just `shared_cache=True` plus a delete, the webhook and in-flight cases still fail, which is why the token design is used. - [x] With Redis stopped, the healthy-path cache tests error (`redis_up`), and the outage and ordering tests pass. - [x] Billing suites pass locally against a real Redis cluster: credit subscription, trial, conversion, checkout, webhook, routes, rate-limit and cache tests. The DB-backed ones run in CI. - [x] `!deploy` preview at `723f26c`, with a fresh `@previews.agpt.co` user, a normal Chrome UA and test card `4242…`: onboarding paywall → **Start 7-day trial** (Checkout; tier TRIAL, ends Oct 18) → **Cancel trial** ("Cancellation pending") → **Subscribe to Pro** → confirm. - **Without a reload**, 13 s after confirming, the plan card reads **"Pro · Active · $50.00 / month · Renews on Nov 11, 2026"**. The API agrees: `current_period_end` = 2026-11-11T23:28:15Z, tier PRO. Invoices: $0.00 (trial) and $50.00 paid. - Network order matched the bug's: `GET /credits/subscription` (the pre-purchase check), `POST /credits/subscription`, then `GET /credits/subscription` (the refetch). - Screenshots are in the patrol evidence folder (`reports/w23-preview-evidence/pr15412-after-subscribe-pro-no-reload.png`). GitHub's API can't attach images, so they aren't attached here. - How many API workers the preview runs isn't visible from outside, so this proves the user-facing flow end to end; the cross-worker property itself is proven by the CI test above. #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes (existing Redis; docker-compose and single-container both run Redis in cluster mode, and a non-cluster or missing Redis falls through to Stripe) - [x] `docker-compose.yml` is updated or already compatible with my changes Fixes the patrol's dogfood ISSUE-005 (W23). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (Claude Code) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings our branded aggregator build back onto current upstream and closes the remaining attribution gaps. The branch had drifted 95 commits behind and no longer applied.
Port to upstream's catalog-as-code registry. Upstream moved model facts out of
blocks/llm.pyintodata/llm_registry/, emptying the exact file our dynamic AIMLAPI catalog lived in — so this is a port, not a merge resolution:blocks/aiml_models.py→data/llm_registry/aiml_catalog.py. It is catalog code and now sits next to the catalog; leaving it underblocks/would makellm_modelsimportbackend.blocks, whose package init pulls_base— an import cycle.catalog.pykeeps its hand-authored payload as_build_static_catalog();_build_catalog()appends the loaded models. Injected entries never shadow a slug the static catalog declares.llm_models.pyregisters a matching enum member per injected slug —_build_model_metadata()joins catalog to enum by slug, so a catalog entry without a member is invisible.LLMModelmixes instr, so members are built withstr.__new__._missing_strips the provider prefix, soanthropic/claude-sonnet-5lands on the native member. 34 of 302 loaded models are shadowed this way — deliberate, so an aggregator entry never reroutes an existing graph.MODEL_COSTnow comes from upstream's catalog-derived path (injected entries carrycost.run_credits);TOKEN_COSTstill bills fromAIML_TOKEN_PRICING.DEFAULT_LLM_MODELstill prefers an AIMLAPI model — this is the aggregator build, a fresh install must work with the one key the user has. Upstream's catalog recommendation stays the fallback.Fix the base URL.
aiml_apidispatched athttps://api.aimlapi.com/v2, which 404s — that host only serves/v2for billing/usage/keys, never chat. Verified:/v1/chat/completions→ 401,/v2/chat/completions→ 404.Close the attribution gaps.
X-AIMLAPI-Source+X-AIMLAPI-Partner-IDnow go on every aimlapi.com request, not just inference: the boot-time catalog fetch (GET /v1/models) and the browser-side key check (GET /v1/billing/balance) were both going out bare. The browser one is only safe because the API preflights it —OPTIONSreturnsaccess-control-allow-headers: authorization,x-aimlapi-source,x-aimlapi-partner-id.NEXT_PUBLIC_*is inlined at build, so the partner id also needed a DockerfileARG.Dropped: our SSR auth-token loopback fix — upstream fixed the same bug independently and better (
BETTER_AUTH_INTERNAL_URL+ a bounded hop).Verification
llm_registrytests + 84providerstests pass.openai/gpt-5.6-terra-proviaaiml_api.tscerror count is identical to clean upstream/dev (1704) — zero introduced. All are pre-existing, from the ungenerated Orval client.