fix(chat): don't tell the model a completed answer was cut short - #909
Merged
Conversation
A completed turn could leave an interrupted-turn marker behind, and the NEXT
turn then fed the model a false account of the conversation.
HOW THE MARKER GETS THERE. The client's Stop writes `lastTurnInterrupted`
immediately (app-api, source=client_signal), but the server only observes the
armed cancel on the lease heartbeat — and that loop sleeps
LEASE_HEARTBEAT_SECONDS (10s) BEFORE its first check:
while True:
await asyncio.sleep(LEASE_HEARTBEAT_SECONDS)
cancel_requested = await renew_session_lease(lease)
So a turn that finishes inside that window races the first tick and wins. The
stream completes normally, `session_manager.cancelled` is never flipped, the
cooperative-stop arm never runs, and the marker survives describing a turn
that was never cut short.
WHY THAT MATTERS. The next turn pops the marker and prepends
`_build_interruption_note("user_stopped")`, which tells the model that "the
user deliberately stopped your previous response before it finished (the last
assistant message above is the partial that was delivered)" and to "not resume
or repeat it". Every clause is false when the answer was complete, and the
note demonstrably steers the following answer. A reload also shows the
"response interrupted" affordance against a complete message.
Verified in dev 2026-09-02: Stop at 2.6s on a 10.4s turn; full answer
persisted; the follow-up turn logged "Cleared interrupted_turn ...
(reason=user_stopped)", i.e. the note fired against a complete reply.
THE FIX is the same reconciliation already used for pending attachments: a
turn that reaches the end of the success path produced a complete answer, so
it was not interrupted, whatever the client signalled. The genuine
interruption arms sit in `except` blocks and re-set the marker after
persisting their partial, so a real interruption is untouched.
Deliberately NOT fixed by retuning the heartbeat: a turn shorter than one tick
is unstoppable however the ticks are spaced, and check-then-sleep just moves
the first check to t=0, before any Stop can have been pressed. The marker has
to be reconciled against what actually happened.
Pre-existing bug, unrelated to the 5f34d2b0 outage work — found while
validating those fixes against dev.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-existing bug, unrelated to the
5f34d2b0outage work. Found while validating those fixes against dev.Stop has a ~10-second dead zone
The client's Stop writes
lastTurnInterruptedimmediately (app-api,source=client_signal). The server only observes the armed cancel on the lease heartbeat — and that loop sleeps before it checks:So the earliest a Stop can be seen is 10s into a turn. Anything shorter races the first tick and wins: the stream completes normally,
session_manager.cancelledis never flipped, the cooperative-stop arm never runs, and the marker survives describing a turn that was never cut short.The in-loop check in
stream_coordinatoris fine — it runs on every event. The flag just never gets set in time.Why the leftover marker matters
The next turn pops it and prepends
_build_interruption_note("user_stopped"):Every clause is false when the answer was complete, and the note measurably steers the following answer. A reload also shows the "response interrupted" affordance against a complete message.
Verified in dev, 2026-09-02: Stop at 2.6s on a 10.4s turn → full answer persisted → the follow-up turn logged
Cleared interrupted_turn … (reason=user_stopped), i.e. the note fired against a complete reply.The fix
Same reconciliation already used for pending attachments: a turn that reaches the end of the success path produced a complete answer, so it was not interrupted — whatever the client signalled. The genuine interruption arms live in
exceptblocks and re-set the marker after persisting their partial, so a real interruption is untouched.Deliberately not fixed by retuning the heartbeat. A turn shorter than one tick is unstoppable however the ticks are spaced, and check-then-sleep just moves the first check to t=0, before any Stop could have been pressed. The marker has to be reconciled against what actually happened.
Scope note — what this does NOT fix
Stop still does not stop a sub-10s turn; the model runs to completion and is billed in full. The cost impact is small (turns too short to stop are short because they're cheap), so I've left the cadence alone rather than adding a DynamoDB write every couple of seconds to every turn. Worth a separate decision if you want Stop to be genuinely immediate.
Testing
3 new tests: a completed turn clears the marker, an interrupted turn keeps it, and a clear failure never breaks the stream. Confirmed the first fails without the fix.
Full backend suite: 7014 passed, 6 skipped.
🤖 Generated with Claude Code