Skip to content

Stop the live-sessions reconciler deleting genuinely open sessions - #227

Merged
jodeleeuw merged 1 commit into
testfrom
fix-live-sessions-reconcile
Sep 14, 2026
Merged

jodeleeuw merged 1 commit into
testfrom
fix-live-sessions-reconcile

Conversation

@jodeleeuw

Copy link
Copy Markdown
Member

The bug

reconcileLiveSessions() (functions/src/live-sessions.ts) decided which
liveSessions mirror documents to delete by comparing two independently
truncated sets: existing, the first MAX_RECONCILE (500) liveSessions
docs by document id (a SHA-256 hash, effectively random but stable order),
and wanted, the first MAX_RECONCILE open sessions in the caller's RTDB
read order. Those orderings are unrelated, so with more than 500 concurrent
sessions a mirror document whose session was genuinely still open, but
ranked 501+ in that run's RTDB listing, was absent from wanted, passed the
startedAt > readAt - 60s creation-race guard, and got deleted. It wasn't in
the scoped set either, so it was never recreated: a real in-progress
participant vanished from the dashboard for the rest of their session, and
mirrorFixed would go non-zero every run.

The fix

Never delete on a run whose open-session read was itself capped -- a capped
read can prove a session is open, but never prove one is absent, since an
absence might just mean it ranked outside the cut.

Two pure functions now carry the logic, both unit-tested without an
emulator:

  • readWasTruncated(entriesCount, cap = MAX_RECONCILE) -- true once the
    entries this run received hit the cap. The caller already always hands in
    at most MAX_RECONCILE entries (it slices its own RTDB read before
    fetching each session's meta), so entries.length >= MAX_RECONCILE is a
    sufficient, self-contained signal from inside this module alone. It can't
    tell that case apart from "there happen to be exactly cap open
    sessions" -- treating both as truncated is the conservative side of that
    ambiguity.
  • idsToDelete(existing, wanted, readAt, truncated) -- the extracted
    deletion decision. Returns nothing at all when truncated is true;
    otherwise applies the pre-existing 60-second creation-race guard exactly
    as before.

No signature change to reconcileLiveSessions, and no change to
scheduled-staging-sweep.ts's call site -- the fix is entirely internal to
live-sessions.ts. Creation and update behavior is unchanged; the
creation-side cap (a session ranking outside MAX_RECONCILE gets no mirror
row until it ranks inside one on a later run) is inherent to bounding
per-run RTDB reads, not a soundness bug like deletion's was, and is now
documented on MAX_RECONCILE itself alongside the deletion fix.

Reviewed components/dashboard/LiveSessionsPanel.js: its copy already makes
no completeness claim ("participants who have started and not yet
finished", plus its own "N more... not shown" note for the unrelated
25-row display cap), so no change was needed there.

Tests

functions/src/__tests__/live-sessions.test.js (pure, no emulator):

  • readWasTruncated: false under the cap, true at and past it (with an
    injectable cap parameter so the test doesn't need to build fixtures at
    production scale), and the production default (MAX_RECONCILE).
  • idsToDelete:
    • a mirror doc for a session that is genuinely gone is deleted once the
      read was not truncated
    • a session present in the wanted set is never deleted
    • a doc created after the read is protected regardless of the wanted set
    • the actual bug: MAX_RECONCILE + 1 mirror docs, all for genuinely open
      sessions, with wanted covering only the first MAX_RECONCILE
      (standing in for a read that couldn't fit the rest) and truncated = true -- asserts zero deletions.

Verified in this session's worktree: cd functions && npm run build and
root npm run lint are clean, and the coordinator's emulator run reports
staging-emulator, live-sessions, and session-id-validation-emulator
all green (58 tests).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Q8xf16ov88M3KojPVfQwJT

reconcileLiveSessions() compared two independently truncated reads:
the first MAX_RECONCILE liveSessions docs by document id (a content
hash, effectively random order) against the first MAX_RECONCILE open
RTDB sessions in the caller's read order. With more than 500 sessions
open at once, a mirror document for a session that simply ranked
outside the second cut looked identical to one whose session had
ended, and got deleted -- erasing a genuine in-progress participant
from the dashboard for the rest of their session, with mirrorFixed
going non-zero every run.

Fixed by never deleting on a run whose open-session read was itself
capped: readWasTruncated(entries.length) treats entries.length hitting
the cap as proof the read may be incomplete (indistinguishable, from
inside this module, from there happening to be exactly that many open
sessions -- the conservative side of that ambiguity is to skip
deletion, not to delete). idsToDelete() is the extracted, pure
decision -- given the existing docs, the wanted ids, and whether the
read was truncated -- so the fix is a couple of unit-testable functions
rather than a change to the caller's signature. Creation and update
keep working exactly as before, including their own inherent cap
(documented on MAX_RECONCILE): a session ranking outside the cap gets
no mirror row until it ranks inside one on a later run, which is a
bounded-read tradeoff, not a soundness bug like deletion's was.

Reviewed LiveSessionsPanel.js and confirmed its copy already avoids
any completeness claim ("participants who have started and not yet
finished", plus its own "N more... not shown" note for the display-side
cap) -- no change needed there.

functions/src/__tests__/live-sessions.test.js: unit tests for
readWasTruncated and idsToDelete, including the actual bug -- more
open sessions than MAX_RECONCILE, all with mirror docs, none deleted
-- and the case it must still catch: a mirror doc for a session that
is genuinely gone is deleted once the read was not truncated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q8xf16ov88M3KojPVfQwJT
@jodeleeuw
jodeleeuw merged commit 6e03b62 into test Sep 14, 2026
1 check passed
@jodeleeuw
jodeleeuw deleted the fix-live-sessions-reconcile branch September 14, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant