A production-grade LangGraph agent that gives confident, specific interior design advice. Built to demonstrate the LaunchDarkly AI iteration loop — AI Configs for runtime-managed prompts and models, progressive release, online evals, and observability.
🌀 Temporal version: this project has been "Temporalized" — the same agent re-expressed as a durable, retryable, human-in-the-loop Temporal workflow. See TEMPORAL.md for the before/after architecture and runbook, and docs/TALK.md for the presentation walkthrough.
This repo is the live demo for "When Your API Call Takes Five Minutes: An Operations Playbook for Long-Running LLM Endpoints" at API World + CloudX + AI TechWorld 2026 (API operations track).
📑 Slides: Google Slides
REST was designed for sub-second, stateless, deterministic calls. LLM endpoints break every one of those assumptions: a single request can take minutes, retries don't mean what they used to, models and prompts change faster than deploys, and workers crash mid-request. The talk argues that a long-running LLM endpoint isn't an API anymore — it's a workflow with an HTTP face — and uses this design assistant to show what operating one takes: endpoint shapes, durable execution (the worker gets killed on stage and the user never notices), runtime config control, and the observability that ties them together.
Talk materials live in docs/:
- docs/cfp-api-world-2026.md — accepted abstract
- docs/talk-api-world-2026.md — talk draft and narrative arc
- docs/talk-script-api-world-2026.txt — speaker script
- docs/talk-assets/ — code screenshots used in the slides
Users ask Decora, a senior interior design advisor, about colors, layouts, and trends. The agent routes each question to one of three specialist tools, synthesizes a short opinionated response, and returns it alongside rich metadata for observability.
Example questions:
- "What paint color works with dark oak floors?" →
style_advisor - "I have a 12x14 living room with a $2000 budget" →
room_planner - "Is terrazzo still trending?" →
trend_spotter - "How do I make a small bathroom feel bigger?" →
room_planner - "Hello!" → direct response, no tool call
This repo ships two implementations of the same agent that run side-by-side:
- Before — LangGraph (
/api/chat, the/UI): an in-process state machine. Described below. - After — Temporal (
/api/temporal/*, the/temporalUI): the same domain logic re-expressed as a durable, retryable, human-in-the-loop workflow. See TEMPORAL.md for the full Temporal architecture, the worker/workflow/activity breakdown, and the runbook.
START
↓
input_guard (length / PII / empty checks — deterministic)
↓
agent (Claude with bound tools — picks a tool or responds directly)
↓
execute_tools (ToolNode runs the selected tool, which makes its own specialist LLM call)
↓
error_handler (bounded retry up to max_retries, then graceful fallback)
↓
agent (loops back to synthesize the tool result)
↓
response_formatter (builds metadata sidecar: routed_to, tool_calls_made, tokens, latency)
↓
END
Each node is a checkpoint boundary, so a failure in execute_tools resumes from there on retry, not from the start.
client → Temporal Service (:7233, persists Event History)
↓ task queue "decor-agent"
Worker (app/worker.py) hosts:
• DecorAgentWorkflow — deterministic orchestration (signals, query, durable wait, fan-out)
• activities — all LLM / side-effecting work, run in a ThreadPoolExecutor
The workflow is pure/deterministic; every side effect is an activity. Human approval is a durable wait_condition driven by approve/reject/tweak_budget signals, and a snapshot query powers the live status UI. Full detail in TEMPORAL.md.
decor-agent/
├── app/
│ ├── config.py # Pydantic-settings singleton
│ ├── logging.py # structlog (JSON prod / console dev)
│ ├── state.py # AgentState + metadata merge reducer
│ ├── prompts.py # Four structured system prompts
│ ├── flags.py # LaunchDarkly AI Configs integration
│ ├── llm.py # Claude client factory (shared by both versions)
│ │
│ │ # --- before: LangGraph ---
│ ├── graph.py # Graph definition + run_agent()
│ ├── nodes/
│ │ ├── input_guard.py
│ │ ├── agent.py
│ │ ├── error_handler.py
│ │ └── response_formatter.py
│ ├── tools/
│ │ ├── style_advisor.py
│ │ ├── room_planner.py
│ │ └── trend_spotter.py
│ │
│ │ # --- after: Temporal ---
│ ├── workflow.py # DecorAgentWorkflow — deterministic orchestration, signals, query
│ ├── activities.py # @activity.defn functions — all LLM / side-effecting work
│ └── worker.py # Worker process: registers workflow + activities (task queue "decor-agent")
│
├── server.py # FastAPI — /api/chat (before), /api/temporal/* (after), static /web
├── test_agent.py # LangGraph end-to-end suite
├── test_workflow.py # Temporal workflow tests (time-skipping, mocked activities)
├── generate_traffic.py # Load generator
├── web/
│ ├── index.html / app.js # "before" chat UI (/)
│ └── temporal.html / temporal.js # "after" durable UI (/temporal)
├── TEMPORAL.md # before/after architecture + Temporal runbook
├── docs/
│ ├── TALK.md # presentation talking points
│ └── *.png # README assets
├── requirements.txt
└── .env.example
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env # then edit to add ANTHROPIC_API_KEY (and ANTHROPIC_WORKSPACE_ID if your key is identity-linked)
python server.py # starts on http://localhost:8000Open http://localhost:8000/docs for the interactive Swagger UI, or hit the API directly:
curl -X POST http://localhost:8000/api/chat \
-H 'Content-Type: application/json' \
-d '{"message": "What color goes with walnut floors?"}'The Temporal "after" needs two more processes (the dev server and a worker). Full runbook — including the crash/durability demo — is in TEMPORAL.md. Short version:
temporal server start-dev # terminal 1 — dev server + Web UI (:8233)
python -m app.worker # terminal 2 — hosts workflow + activities
python server.py # terminal 3 — API + /temporal UILOG_LEVEL=WARNING python test_agent.py # LangGraph (before)
python -m pytest test_workflow.py -v # Temporal workflow (after) — no API key / server needed| Variable | Default | Purpose |
|---|---|---|
ANTHROPIC_API_KEY |
required | Claude API key |
ANTHROPIC_WORKSPACE_ID |
"" |
Required if your API key is identity-linked (the API returns anthropic-workspace-id is required). Find it in the Anthropic Console under Settings → Workspaces; starts with wrkspc_. Sent as the anthropic-workspace-id header on every request. |
ENCRYPTION_KEY |
"" |
Base64-encoded 32-byte key. When set, Temporal payloads are encrypted at rest with AES-256-GCM (see TEMPORAL.md) |
LD_SDK_KEY |
"" |
LaunchDarkly server SDK key (used by flags.py AI Configs) |
LOG_LEVEL |
INFO |
structlog level |
ENVIRONMENT |
development |
Switches log format between console and JSON |
- Input validation at two layers — Pydantic on the HTTP boundary,
input_guardinside the graph - Bounded retries —
max_retries=2, then a graceful fallback message - Structured logs on every step:
input_guard.pass,agent.invoke,tool.invoke/success/error,error_handler.retry/exhausted,http.request - Metadata sidecar on every response:
routed_to,tool_calls_made, token usage, per-node latency, error counts — ready to feed evals and analytics - Errors never leak to the client; full tracebacks go to logs only
- Request IDs honored from
x-request-idheader or generated per request
Python 3.12+ · LangGraph · LangChain · Anthropic Claude · Temporal (Python SDK) · FastAPI · Pydantic · structlog · LaunchDarkly (server SDK + AI SDK)
The Temporal version makes a few deliberate design choices (sync activities + thread-pool executor for the blocking LLM calls; no heartbeat_timeout/handler-drain, with reasons) — these are documented in the TEMPORAL.md "Sync activities" and "Design decisions & trade-offs" sections.