Harden the abandoned-session sweep against starvation, mis-reported discards, and filename collisions

[jodeleeuw]

Summary

Six review findings against the streaming-ingest staging tier's abandoned-session sweep (docs/streaming-ingest-design.md), each addressed here:

1. Candidate starvation. scheduled-staging-sweep.ts read one page of the oldest open-session candidates per run and never advanced past it, so a page's worth of long-lived or zombie sessions at the head of the queue (ordered by expiresAt) could block recovery of everything abandoned behind them for up to the 24-hour TTL. listOldestOpenSessions (staging.ts) now takes an optional startAfter(expiresAt, sessionId) cursor, and the sweep pages through candidates until MAX_SESSIONS_PER_RUN sessions are actually recovered or discarded, the table is exhausted, or a hard MAX_PAGES_PER_RUN (20) ceiling is hit. Pages fetched are recorded as pages in SweepStats and systemStatus/staging.

2. Discard outcome accuracy. discardSession swallowed its own RTDB error and returned void, so the sweep reported "recovered" regardless of whether the staging node was actually removed — if the discard failed after a queue entry was written, the next run would reassemble and re-queue the same session, landing a duplicate partial if the first entry had already completed. discardSession now returns a boolean (still never throws); the sweep only counts/logs "recovered" or "discarded" when the removal actually succeeded, otherwise counting it as an error. recoverSession also checks for an existing uploadQueue document with the same deduplication key (queue-upload.ts's new queueDocIdFor helper) that is already "completed", and skips the re-queue if so.

3. Concurrency limiter. The live-sessions reconciliation pass ran an unbounded Promise.all over up to 500 getSessionMeta calls. Replaced with mapWithConcurrency (new functions/src/concurrency-limit.ts), a small local worker-pool helper with no new dependency, at a concurrency of 20.

4. Partial filename collisions. partialFilenameFor used only the client-supplied filename, so two abandoned sessions named e.g. data.csv produced the identical experimentID:filename deduplication key and the second recovery silently overwrote the first's payload in Cloud Storage. A client-supplied name now always gets an 8-hex-char suffix (a sha256 of the session id); the id-only fallback for sessions with no filename is unchanged, since it's already unique. Updated the filename description in pages/docs/experiments/sending-data.js and docs/streaming-ingest-design.md.

5. Salvage instead of discard on data-content refusals. api-data.ts discarded the staged copy on every refusal branch, including INVALID_DATA and both duplicate-filename refusals (the collision cache's "duplicate" verdict and the provider's own NAME_CONFLICT dual-run backstop) — refusals about this submission, not about the experiment's willingness to accept data at all. Those three branches now leave staging alone; the sweep's second door (finalized/active re-check) still covers a since-closed experiment. The four experiment-state gates (finalized, inactive, session-cap, unknown experiment) are unchanged and still discard.

6. Unreachable database. Confirmed (by reading the code, and with a cheap regression test) that a missing or unreachable RTDB is caught at every call site in the sweep and only ever surfaces as a recorded systemStatus/staging error, never an uncaught throw.

Deliberately untouched

assembleSession (staging-assembly.ts), api-session-start.ts, and database.rules.json are untouched on purpose — a sibling PR is concurrently changing them, and this scoping is what lets the two merge cleanly.

Tests added

• functions/src/__tests__/concurrency-limit.test.js (new, pure, no emulator): mapWithConcurrency ordering, concurrency ceiling, exhaustiveness, rejection propagation, edge cases.
• functions/src/__tests__/staging-assembly.test.js: partialFilenameFor cases updated for the new hash suffix, plus a same-filename/different-session collision case.
• functions/src/__tests__/staging-emulator.test.js: paging past a wall of live sessions to recover an abandoned one behind them; two same-filename sessions surviving as distinct queue entries; a discard failure not being counted as recovered; no re-queue of a session whose recovery already completed before a discard failure; the sweep surviving an unreachable staging database; and INVALID_DATA leaving the staged session in place (the existing finalized-still-discards test serves as the control). Existing p07/p09 filename assertions updated for the new hash suffix.

Test plan

• cd functions && npm run build clean
• npm run lint (repo root) clean
• Emulator run in the worktree green: staging-emulator, staging-assembly, concurrency-limit, session-id-validation-emulator, data-emulator, collision-integration-emulator (98 tests)

🤖 Generated with Claude Code

https://claude.ai/code/session_01Q8xf16ov88M3KojPVfQwJT