Add OpenRouter sample: prompt batch with cached retries by DABH · Pull Request #520 · temporalio/samples-typescript

DABH · 2026-09-11T06:17:39Z

Adds an openrouter sample: a Workflow that fans one Activity out per prompt through OpenRouter's Auto Router and returns answer, model, and reported cost per prompt.

The Activity uses the openai client pointed at OpenRouter with maxRetries: 0, classifies 4xx as non-retryable and 408/429/5xx as retryable, passes Retry-After through as nextRetryDelay, heartbeats, checks for OpenRouter's 200-with-error body, and sends X-OpenRouter-Cache: true so a retried identical request is served from OpenRouter's cache at $0. A --fail-once flag on the client demonstrates that; verified live with a HIT at $0 on attempt 2.

Tests use a fake fetch and a fake Activity; no API key needed. Registered in the CI project list and the sample list. This is the TypeScript port of temporalio/samples-python#366 (prompt_batch); the budget_gate scenario stays Python-only for now.

Note: this also regenerates pnpm-lock.yaml with pnpm 10. The lockfile on main still has a pnpm 9 packageExtensionsChecksum, and CI installs pnpm 10, which refuses a frozen install with it; main has not had a green CI run since September 1 for that reason.

Merge after temporalio/samples-python#366: the README links to the Python openrouter/prompt_batch and budget_gate samples on main.

Call OpenRouter from an Activity with the openai client pointed at OpenRouter, client retries off so Temporal owns every attempt, error classification with Retry-After as the next retry delay, heartbeats, and OpenRouter response caching so a retried identical request is billed at zero. The Workflow fans a prompt batch out under bounded concurrency and reports skipped prompts instead of failing the batch.

main's lockfile carries a pnpm 9 checksum; CI installs pnpm 10, which expects sha256 and refuses a frozen install. Regenerated with pnpm 10.34 so CI can run.

Copilot

Copilot review overview

🟡 Changes recommended

Workflow cancellation and concurrency edge cases are mishandled, and Worker connection configuration can diverge from the starter.

Review effort: Balanced
Findings: 1 · 2 · 1

Open (4)

Do not swallow Workflow cancellation as a skipped prompt · New
Use configured Temporal connection options in the Worker · New
Validate and cap maxConcurrency before allocating runners · New
Include API key setup in advertised startup commands · New

What changed in this PR

Adds an OpenRouter sample demonstrating Temporal-managed retries, cached retry requests, and bounded prompt batching.

Changes:

Implements OpenRouter Activities, Workflow, Worker, and client.
Adds fake-fetch and Workflow integration tests.
Registers the sample in documentation, CI, and workspace metadata.

File	Description
`README.md`	Lists the OpenRouter sample.
`pnpm-lock.yaml`	Adds resolved sample dependencies.
`openrouter/tsconfig.json`	Configures TypeScript compilation.
`openrouter/src/workflows.ts`	Implements bounded prompt batching.
`openrouter/src/worker.ts`	Runs the Worker and Activities.
`openrouter/src/shared.ts`	Defines shared inputs and results.
`openrouter/src/mocha/workflows.test.ts`	Tests batch result handling.
`openrouter/src/mocha/activities.test.ts`	Tests requests, errors, and caching.
`openrouter/src/client.ts`	Starts and displays batch results.
`openrouter/src/activities.ts`	Calls OpenRouter with Temporal retries.
`openrouter/README.md`	Documents setup and behavior.
`openrouter/package.json`	Defines dependencies and scripts.
`openrouter/.prettierrc`	Configures formatting.
`openrouter/.prettierignore`	Excludes generated output.
`openrouter/.post-create`	Provides startup instructions.
`openrouter/.nvmrc`	Selects Node.js 22.
`openrouter/.npmrc`	Disables package-lock generation.
`openrouter/.gitignore`	Excludes generated files.
`openrouter/.eslintrc.js`	Configures linting.
`openrouter/.eslintignore`	Excludes generated files from linting.
`openrouter/.env.example`	Documents environment variables.
`.scripts/list-of-samples.json`	Registers the sample.
`.github/workflows/ci.yml`	Adds the sample to CI.

Files not reviewed (1)

pnpm-lock.yaml: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…ry-After dates, reported cost - Rethrow Workflow cancellation instead of recording it as a skipped prompt. - Reject a non-positive maxConcurrency and cap runners at the prompt count. - Worker loads the same env-config connection options as the client. - Parse the HTTP-date form of Retry-After as well as delta-seconds. - A response without usage.cost reports an unknown cost (null), not zero, and the batch total is named reportedCostUsd with its scope documented. - .post-create mentions OPENROUTER_API_KEY; excluded from the shared-file copy.

brianstrauch

Requesting changes for the three issues below: configured namespaces are ignored, in-flight Activity requests do not react to cancellation, and the README promises per-attempt cost records that are not persisted in Workflow history.

Validation: build, lint, and all 10 tests passed. Additional probes with fake OpenRouter responses reproduced the namespace and cancellation issues and verified the retry-history behavior.

…ding - Client and Worker pass the namespace from the loaded connection config. - The OpenRouter request carries the Activity's cancellation signal; an abort caused by cancellation surfaces as CancelledFailure. Test added. - README describes what Event History records versus Worker logs.

…lation coverage - A 402 whose error.metadata.limit_source is openrouter_in_flight_budget is transient per OpenRouter's docs; retry it after Retry-After instead of treating it as out of credits. - openai's APIError.error is already the inner error object, so the message path never matched and every failure message carried the status twice. - The client parses its flags explicitly, rejects unknown ones, and accepts --max-concurrency; .env.example removed since nothing read it. - Empty or non-positive Retry-After values are ignored. - Any error while cancellation is pending surfaces as CancelledFailure. - Cancellation test waits for the request to start instead of sleeping, and a Workflow-level cancellation test was added.

- --model and --max-concurrency require a value; --max-concurrency must be a positive integer, checked client-side instead of silently defaulting. - Only ActivityFailure is recorded as a skipped prompt; anything else propagates. - Non-numeric error codes fall back to 500 instead of becoming a bogus type. - Tests for a plain 4xx via the APIError path and for connection errors propagating unchanged. - README: 408 listed as retryable, two-prompt output matches its command, Other options moved out of the --fail-once explanation.

brianstrauch

Reviewed at 244a83a. Build, lint, and all 19 tests passed. One non-blocking cost-reporting suggestion is noted inline.

reportedCostUsd is a subtotal of known costs; unknownCostCount makes that visible in the result and the client output instead of printing $0 for a batch with unknown charges.

DABH requested review from a team as code owners September 11, 2026 06:17

This was referenced Sep 11, 2026

Add OpenRouter integration page (Python) temporalio/documentation#5307

Open

Add OpenRouter samples: prompt batch and budget gate temporalio/samples-python#366

Open

DABH added 3 commits September 30, 2026 13:25

Log each OpenRouter call's attempt, cost, and cache status

5c8c2d1

Treat 402 and 403 key-limit errors as one out-of-credits type

bdab719

DABH force-pushed the dabh/openrouter branch from 8814f48 to bdab719 Compare September 30, 2026 18:42

DABH requested a review from a team as a code owner September 30, 2026 18:42

Relock with pnpm 10

01e2f84

main's lockfile carries a pnpm 9 checksum; CI installs pnpm 10, which expects sha256 and refuses a frozen install. Regenerated with pnpm 10.34 so CI can run.

DABH requested a balanced review from Copilot October 1, 2026 06:09

Copilot started reviewing on behalf of DABH October 1, 2026 06:10 View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

Comment thread openrouter/src/workflows.ts

Comment thread openrouter/src/worker.ts Outdated

Comment thread openrouter/src/workflows.ts Outdated

Comment thread openrouter/.post-create

brianstrauch requested changes Oct 1, 2026

View reviewed changes

Comment thread openrouter/src/client.ts Outdated

Comment thread openrouter/src/activities.ts

Comment thread openrouter/README.md Outdated

DABH requested a review from brianstrauch October 1, 2026 18:12

DABH added 4 commits October 1, 2026 13:22

Treat a provider error on the choice as an error, not an answer

fe3e9b6

Cap Retry-After at five minutes; retry a response with no choices

244a83a

brianstrauch approved these changes Oct 1, 2026

View reviewed changes

Comment thread openrouter/src/workflows.ts

DABH added 2 commits October 4, 2026 14:55

Report how many prompts came back without a cost

16cdcf1

reportedCostUsd is a subtotal of known costs; unknownCostCount makes that visible in the result and the client output instead of printing $0 for a batch with unknown charges.

README: separate the unknown-cost sentence

f6cbd0f

chris-olszewski approved these changes Oct 5, 2026

View reviewed changes

Navigation Menu

Repository navigation

Add OpenRouter sample: prompt batch with cached retries - #520

Add OpenRouter sample: prompt batch with cached retries#520
DABH wants to merge 12 commits into
mainfrom
dabh/openrouter

DABH commented Sep 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brianstrauch left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brianstrauch left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

DABH commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brianstrauch left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brianstrauch left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

DABH commented Sep 11, 2026 •

edited

Loading