Rate-limit and abuse-protect public MCP reads and LLM endpoints #11

Activity

1. harrymove-ctrl

   added this to the Sui Overflow milestone on Jun 16, 2026
2. harrymove-ctrl

   added

   P0Demo-blocking: required for a working Sui Overflow demoDemo-blocking: required for a working Sui Overflow demo

   featureUser- or agent-facing capabilityUser- or agent-facing capability

   platformBackend platform plumbing: Worker, D1, queues, secrets, meteringBackend platform plumbing: Worker, D1, queues, secrets, metering

   on Jun 16, 2026
3. harrymove-ctrl commented on Jun 19, 2026

   

   harrymove-ctrl

   on Jun 19, 2026

   OwnerAuthor

   More actions

   Implementation Plan

   Approach
   Add a binding-agnostic rateLimit(env, key, { limit, windowSec }) helper backed by a D1 fixed-window counter that mirrors consumeDemoQuota (apps/api/src/worker.ts:4296) and returns { allowed, remaining, resetAt }. Enforce it from a single chokepoint — the router routeWorkerRequest (worker.ts:575) — wrapping each protected route so the handler bodies stay untouched. Use two bucket classes: anon keyed by hashed clientIp (worker.ts:4313) and authed keyed by the validated read-token hash / MemWal ownerId, each with its own env-var limit (authed > anon). Key design decision: go D1-first rather than the native Cloudflare Rate Limiting binding — the native binding only returns { success } (no remaining/resetAt), bakes its limit into config (can't be driven by a RATE_LIMIT_*_PER_MIN var), and isn't exercisable in vitest/Miniflare — all three are hard acceptance criteria. Keep the helper signature binding-shaped so a CF ratelimit binding can be slotted in later for the hottest unauth path.

   Prerequisites & dependencies

   • No blocking issues. Per-IP limiting ships independently (issue states "None blocking"). Per-token/account buckets get stronger once real accounts land, but the MemWal-delegate ownerId (readHostedRunAuth, worker.ts:1461) and ctxm_ read tokens already give us a usable authed key today.
   • No new bindings required for the D1 path (reuses CONTEXTMEM_DB). If the team later wants the CF binding accelerator, it needs an unsafe.bindings entry of type: "ratelimit" in both top-level and env.staging of cloudflare/wrangler.jsonc — document but do not depend on it.
   • New env vars (all optional, safe defaults) added to WorkerEnv (worker.ts:32-55) and cloudflare/wrangler.jsonc vars (top-level + env.staging).
   • Deploy reminder (per repo memory): Worker is not shipped by CI — after merge run wrangler deploy --config apps/api/cloudflare/wrangler.jsonc and --env staging, and apply migration 0007 to both D1 dbs.

   Step-by-step

   1. DB migration. Create apps/api/migrations/0007_rate_limits.sql (next number after 0006):

      CREATE TABLE IF NOT EXISTS contextmem_rate_limits (
        bucket_key TEXT PRIMARY KEY,        -- `${windowSec}:${windowStart}:${endpoint}:${idHash}`
        endpoint TEXT NOT NULL,
        window_start INTEGER NOT NULL,      -- epoch seconds, floored to window
        count INTEGER NOT NULL DEFAULT 0,
        updated_at TEXT NOT NULL
      );
      CREATE INDEX IF NOT EXISTS idx_rate_limits_window ON contextmem_rate_limits(window_start);

      Mirror contextmem_demo_limits shape (migrations/0004_growth_loops.sql:12).

   2. Core helper. In worker.ts near consumeDemoQuota (~4296) add:

      • async function rateLimit(env, key, { limit, windowSec }): Promise<{ allowed; remaining; resetAt }> — compute now = Math.floor(Date.now()/1000), windowStart = now - (now % windowSec), resetAt = (windowStart + windowSec) * 1000, bucketKey = ${windowSec}:${windowStart}:${key}. Atomic increment-and-read via the upsert pattern at worker.ts:4304 plus RETURNING count:

        INSERT INTO contextmem_rate_limits (bucket_key, endpoint, window_start, count, updated_at)
        VALUES (?, ?, ?, 1, ?)
        ON CONFLICT(bucket_key) DO UPDATE SET count = count + 1, updated_at = excluded.updated_at
        RETURNING count

        allowed = count <= limit; remaining = Math.max(0, limit - count). Counting rejected hits too is intentional (keeps hammered keys blocked).
      • function rateLimitConfig(env) — parse RATE_LIMIT_* vars to numbers with defaults; honor RATE_LIMIT_ENABLED !== "false".
      • async function rateLimitKey(request, endpoint, opts) — resolve identity: if readHostedRunAuth(request) (worker.ts:1461) present → acct:${ownerId} (authed); else if a validated read token is known → tok:${sha256Hex(token)} (authed); else ip:${await sha256Hex(clientIp(request))} (anon). Return { key: ${endpoint}:${idPart}, authed }. Reuse sha256Hex (worker.ts:4507).
      • async function enforceRateLimit(request, env, endpoint, { authedLimit, anonLimit, windowSec }) → returns the rateLimit result + authed flag.
      • function rateLimited429(rl, isMcp) — for MCP return mcpJsonError(message, 429) (worker.ts:5281); else build a Response directly (do not throw statusError, so we control headers) with body shaped like statusError(..., 429, "RATE_LIMITED") → { error: { code: "RATE_LIMITED", message, hint }, message }, status 429, headers Retry-After: ceil((resetAt-now)/1000), X-RateLimit-Remaining: 0, X-RateLimit-Reset: resetAt, wrapped in cors().
      • function withRateLimitHeaders(res, rl) — clone response, set X-RateLimit-Remaining/X-RateLimit-Reset.

   3. Wire enforcement into the router (routeWorkerRequest, worker.ts:575), each as a one-liner guard before the existing handler call:

      • /api/memwal/recall (worker.ts:631), /api/memwal/chat (worker.ts:634): const rl = await enforceRateLimit(request, env, "chat"|"recall", ...); if (!rl.allowed) return rateLimited429(rl, false); return withRateLimitHeaders(await memwalChat(request, env), rl);
      • aiQueryRun in the hostedRunRoute block (worker.ts:652): same wrapper, endpoint "ai_query". (Demo runs key by anon IP since ownerId is demo:<ip>; authed runs key by ownerId.)
      • /api/directory (worker.ts:687): wrap listDirectoryNamespaces (anon-heavy, the scrape target), endpoint "directory".
      • Public MCP reads, MCP branch (worker.ts:772-795): enforce after authorizeNamespace succeeds (so a valid read token yields the authed bucket) but before createMcpHandler. Skip the GET browser-landing path (worker.ts:774-776). On block return rateLimited429(rl, true). Endpoint "mcp".
      • Leave createDemoExtraction/consumeDemoQuota as-is (separate durable quota; do not double-enforce).

   4. Expose headers to browsers. In cors() (worker.ts:5303) extend access-control-expose-headers from "mcp-session-id" to also include x-ratelimit-remaining, x-ratelimit-reset, retry-after, else fetch clients can't read them.

   5. Env + config. Add to WorkerEnv (worker.ts:32): RATE_LIMIT_ENABLED?, RATE_LIMIT_WINDOW_SEC?, RATE_LIMIT_MCP_ANON_PER_MIN?/_AUTHED_, RATE_LIMIT_CHAT_PER_MIN?/_AUTHED_, RATE_LIMIT_RECALL_PER_MIN?/_AUTHED_, RATE_LIMIT_AI_QUERY_PER_MIN?/_AUTHED_, RATE_LIMIT_DIRECTORY_PER_MIN? (all string since wrangler vars are strings). Add the same keys with conservative defaults to cloudflare/wrangler.jsonc vars (:6) and env.staging.vars (:66) so staging can override. Suggested defaults: chat/ai_query 10/min anon, 60/min authed; recall 20/120; mcp 30/300; directory 30 anon.

   6. Cron cleanup (optional, cheap). In processDueSchedules (cron */30, worker.ts:540) add a DELETE FROM contextmem_rate_limits WHERE window_start < ? (older than ~1h) so the table doesn't grow unbounded.

   7. Docs. Document the new vars + behavior in .env.example (root) and README.md; note the optional CF ratelimit binding stub. Tick the relevant line in docs/launch-checklist.md (D5 abuse-protection precondition for going public).

   New / changed files

   • apps/api/migrations/0007_rate_limits.sql (new)
   • apps/api/src/worker.ts (helpers + router guards + cors/WorkerEnv edits)
   • apps/api/cloudflare/wrangler.jsonc (vars top-level + env.staging)
   • apps/api/cloudflare/wrangler.example.jsonc (mirror the new vars)
   • apps/api/src/worker.test.ts (new tests + MemoryD1 mock support)
   • .env.example, README.md, docs/launch-checklist.md (docs)

   Test plan

   • Test-infra prerequisite (gotcha): MemoryD1Database (worker.test.ts:845) only answers enumerated queries; unknown queries return null/empty, so the rate-limit upsert would never count. Add a contextmem_rate_limits map + handle the INSERT ... ON CONFLICT ... RETURNING count in MemoryD1Statement.first() (worker.test.ts:878) returning { count }, incrementing per bucket_key.
   • Unit (vitest, worker.test.ts), using vi.useFakeTimers()/vi.setSystemTime to pin the window:
     1. Under limit passes — N < limit requests to /api/memwal/chat return non-429 and a decreasing X-RateLimit-Remaining.
     2. Over limit 429s — the (limit+1)th anon request returns 429 with Retry-After header and body code RATE_LIMITED.
     3. Authed bucket > anon — set RATE_LIMIT_CHAT_PER_MIN low; exhaust as anon (no headers) → 429; same count with x-memwal-account-id+x-memwal-authorization delegate headers still passes (uses _AUTHED_ limit).
     4. Per-key not global — exhaust IP A (cf-connecting-ip: 1.1.1.1) → 429; IP B (2.2.2.2) still 200. Repeat for two distinct read tokens on /mcp.
     5. Window reset — advance system time past windowSec; a blocked key passes again.
     6. MCP shape — over-limit /mcp returns JSON-RPC 429 (mcpJsonError) not the REST body.
   • Manual / staging — for i in $(seq 1 40); do curl -s -o /dev/null -w "%{http_code} " https://contextmem-backend-staging.petlofi.workers.dev/api/memwal/chat -d '{"message":"hi"}' -H 'content-type: application/json'; done shows 200s then 429s; inspect X-RateLimit-Remaining counting down and Retry-After on the 429. Confirm SSRF guard isPrivateIpv4 (worker.ts:4288) and /api/demo/extractions 1/day quota still behave (no regression).

   Risks & gotchas

   • MemoryD1 mock must be extended or every rate-limit test silently passes (count always 0). Highest-leverage gotcha.
   • CORS expose-headers — without step 4, browser clients see the 429 but can't read X-RateLimit-*.
   • D1 write on every public read adds a write + latency to hot paths; window is short and counts are tiny, but note it. The CF binding is the future escape hatch for the hottest unauth path.
   • Fixed-window burst — up to 2× limit across a boundary; acceptable for abuse protection (note sliding-window as a later upgrade).
   • IP trust — clientIp prefers cf-connecting-ip (trustworthy behind CF); the x-forwarded-for fallback is spoofable but only hit off-CF (local/tests).
   • MCP client tolerance — return HTTP 429 + Retry-After; mcp-remote may not surface the body, so status + header carry the signal.
   • Don't throw statusError for 429 — the central catch in handleWorkerRequest (worker.ts:565) rebuilds the body via json() and would drop Retry-After/X-RateLimit-*; build the Response directly.
   • Don't double-count — enforce only at the router; keep consumeDemoQuota independent (durable daily quota vs ephemeral window).
   • Migration is manual on this repo (Worker not auto-deployed) — apply 0007 to prod + staging D1.

   Acceptance-criteria mapping

   • rateLimit(env, key, {limit, windowSec}) → {allowed, remaining, resetAt} via D1 sliding/fixed window mirroring consumeDemoQuota → Step 2 (D1 chosen over CF binding for testability + remaining/resetAt; rationale in Approach).
   • Distinct anon (per-IP) vs authed (per read-token/account) buckets, higher authed limits → Steps 2 (rateLimitKey) + 5 (separate _AUTHED_ vars); test fix(ci): force bun + pin wrangler in Deploy workflow #3.
   • Enforced on /mcp, /api/memwal/chat, aiQueryRun, /api/memwal/recall, /api/directory → Step 3 (all five wired).
   • 429 with Retry-After + typed RATE_LIMITED code; X-RateLimit-Remaining/-Reset on success → Step 2 (rateLimited429/withRateLimitHeaders) + Step 4 (expose headers); tests Walrus storage + Memory via Tatum, plus CI deploy #1, Memory knowledge graph: 3D neural constellation + grounded entity panel #2, Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6.
   • Env-configurable via wrangler.jsonc vars with safe defaults, staging override → Step 5.
   • Tests: under-limit pass, over-limit 429, authed>anon, per-key not global → Test plan unit Walrus storage + Memory via Tatum, plus CI deploy #1–docs: add product roadmap (Harbor storage + Talus on-chain direction) #4 (+ Spike: run @mysten/seal encrypt+decrypt inside a Cloudflare Worker #5/Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6).
   • Implementation notes: keep separate from usage ledger, don't regress SSRF guard → Step 1 (own table) + Steps 3/Risks (isPrivateIpv4 untouched); optional rate_limited ledger event deferred (no ledger exists yet — issue's usage-ledger item is separate).

   Effort
   M — ~2 person-days. ~0.5d helper + router wiring + config, ~0.5d MemoryD1 mock + tests, ~0.5d docs + staging deploy/verify, ~0.5d buffer for tuning defaults and the MCP 429 path.

   ---

   Implementation plan generated for execution (planning phase). Part of the roadmap (#4).
4. roblambert9 commented on Jun 21, 2026

   

   roblambert9

   on Jun 21, 2026

   More actions

   Hi, I read issue #11 on contextMeM. You wrote that the token-less public MCP reads and LLM endpoints "are completely uncapped: an open cost/abuse vector that blocks flipping the repo public," and that you want per-IP and per-token limits so a single client can't run up unbounded Workers AI / OpenRouter cost.

   Rate limiting stops the bleeding. The adjacent question is whether you want those same per-token reads to eventually meter and bill per call instead of just 429ing, since you already have the read-token and account buckets you describe.

   I run a small, fixed-scope thing for exactly that gap: a $99 MCP Tool Metering Readiness Audit. It is async, takes me about one to two days, and for your worker.ts shape it gives you: where a per-call meter and a tamper-evident SHA-256 receipt would sit relative to your existing consumeDemoQuota and ledger split, a pricing shape for the /mcp reads and the chat/recall endpoints, a runnable PAPER_MODE demo wired to your endpoint shape so no real money moves, and a copy-pasteable go-live plan. It is built on an MIT SDK I maintain (nano-empire-tollbooth).

   Full honesty so there is no confusion: the SDK has earned $0 so far and runs in paper mode. I am selling readiness, not revenue, and I am not claiming live payments or traction. The only thing I can prove today is a reproducible dogfood run where an external agent hits a 402, pays a 1 cent toll through a mock verifier, escrow locks then releases, and 33 tests pass. That proves the protocol mechanics, not real money changing hands.

   Repo: github.com/roblambert9/nano-empire-tollbooth · PyPI: pypi.org/project/nano-empire-tollbooth

   If it's not useful, no worries, I won't follow up.