Skip to content

fix(task-execution): Supersede stale mailbox Turns on new input - #2562

Merged
dcramer merged 1 commit into
mainfrom
fix/supersede-stale-mailbox-turns
Nov 2, 2026
Merged

dcramer merged 1 commit into
mainfrom
fix/supersede-stale-mailbox-turns

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

What failed

JUNIOR-9M: getActiveMailboxTurnId throws Conversation … has multiple active mailbox Turns (for example local:web:67232d03… and slack:C0B595QDZLL:1793638594.088319). Every resume attempt of the Conversation throws, retries run out (conversation.work.retry.exhausted), and the Conversation stays stuck until a person posts again.

Root cause

Generic conversation work can stop a Turn without closing its Turn cursor. Examples: the heartbeat clears an expired lease after a worker is lost (deploy rollout, freeze), lost-lease recovery runs out of retries, or a final-attempt Run failure in conversation-turn.ts that marks the Turn failed in the Conversation but leaves the checkpoint running/paused. The next web or event message starts a new Turn next to it. New input only superseded auth-paused Turns, so the Conversation now has two open mailbox Turns. When the new Turn yields, its resume fails.

Fix

When new mailbox input starts a Turn, abandon every other open web or event Turn, not only auth-paused ones. The OAuth prompt cleanup still runs only when an auth-paused Turn is superseded. Slack Turns are not mailbox Turns and stay as before.

Why a Turn that waits to resume is not superseded

New mailbox input starts a Turn only when no yielded Turn waits:

  • Every path that leaves a Turn to resume sets execution paused: yield (requestAnotherSlice), expired-lease recovery (clearExpiredConversationLease), a counted retry (recordConversationRetry), and lost-lease recovery in the worker.
  • selectAttemptMessages returns no web or event input while execution is paused. A steered web message is not an attempt interrupt while paused. Deferred input appended while paused keeps paused.
  • The lost-worker retry pause from fix(task-execution): Resume a turn once after its worker stops checking in #2298 runs only in runNextPausedTurn (Slack and dispatch workers). Mailbox Turns do not use it.
  • A Slack mention can interrupt a paused Conversation. It does not reach this path: resolveMailboxTurnWork returns no work for Slack input, and the Slack worker resumes the paused Turn and steers the mention into it.

So a different open mailbox Turn at this point is stale. conversation-turn-work.test.ts › "resumes a paused dashboard Turn before a deferred event" protects the selection rule: it fails when the paused check in selectAttemptMessages is removed.

Review evidence

An end-to-end run through processConversationQueueMessage, the real conversation-turn worker and recoverConversationWork reproduces JUNIOR-9M: a web Turn yields, five workers take the lease and are lost, and the heartbeat sets the Conversation to failed with the cursor still paused. On main, the next message yields and every resume fails with "multiple active mailbox Turns". On this branch, the stale Turn is abandoned and the next message yields, resumes and replies.

That test is not in this PR: scripts/check-test-architecture.mjs does not allow tests in packages/junior/tests to run the agent or compose the worker. Its home is evals/integration/conversation/lifecycle.eval.ts, with a fixture step that loses workers. That needs new model recordings.

Other checks:

  • A late save from a lost worker of the abandoned Turn does not reopen it after the new Turn commits history: the save fails with AgentHistoryBranchError and the cursor stays abandoned.
  • No terminal lifecycle event is written for the superseded Turn. Do not add one here: the new Turn already wrote turn_started, and the dashboard reads the latest lifecycle event to show that Junior is responding.

Remaining risk

via @dcramer.

--

View Session [Sentry]

Conversation work can stop after repeated failed attempts (for example a
worker lost at deploy rollout) without closing the Turn cursor. The next
web or event Turn then started next to it, and every resume failed with
"multiple active mailbox Turns" until retries ran out.

New mailbox input now abandons every other open web or event Turn, not
only auth-paused ones.

Fixes JUNIOR-9M

Co-Authored-By: BYK <126780+BYK@users.noreply.github.com>
@vercel

vercel Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
junior-docs Ready Ready Preview Nov 2, 2026 8:03pm UTC

Request Review

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

3 screenshot changes — 3 changed · 0 added · 0 removed

Review screenshots in Frameshift

Conversation Child Agent Drawer · Desktop
Conversation Child Agent Drawer · Desktop
Changed
Conversation Child Agent Drawer · Mobile
Conversation Child Agent Drawer · Mobile
Changed
Settings Mcp · Desktop
Settings Mcp · Desktop
Changed

@BYK
BYK requested a review from dcramer November 2, 2026 21:43
@BYK
BYK marked this pull request as ready for review November 2, 2026 21:43
@github-actions github-actions Bot added the risk: medium PR risk score: medium label Nov 2, 2026
@dcramer
dcramer merged commit aaf51a5 into main Nov 2, 2026
48 checks passed
@dcramer
dcramer deleted the fix/supersede-stale-mailbox-turns branch November 2, 2026 22:49

This branch was successfully deployed

1 active deployment
Preview – junior-docs — 33efda75 Deployed Nov 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: medium PR risk score: medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants