Skip to content

fix(chat): don't tell the model a completed answer was cut short - #909

Merged
philmerrell merged 1 commit into
developfrom
fix/false-interrupted-marker
Sep 2, 2026
Merged

fix(chat): don't tell the model a completed answer was cut short#909
philmerrell merged 1 commit into
developfrom
fix/false-interrupted-marker

Conversation

@philmerrell

Copy link
Copy Markdown
Contributor

Pre-existing bug, unrelated to the 5f34d2b0 outage work. Found while validating those fixes against dev.

Stop has a ~10-second dead zone

The client's Stop writes lastTurnInterrupted immediately (app-api, source=client_signal). The server only observes the armed cancel on the lease heartbeat — and that loop sleeps before it checks:

while True:
    await asyncio.sleep(LEASE_HEARTBEAT_SECONDS)   # 10s, first
    cancel_requested = await renew_session_lease(lease)

So the earliest a Stop can be seen is 10s into a turn. Anything shorter races the first tick and wins: the stream completes normally, session_manager.cancelled is never flipped, the cooperative-stop arm never runs, and the marker survives describing a turn that was never cut short.

The in-loop check in stream_coordinator is fine — it runs on every event. The flag just never gets set in time.

Why the leftover marker matters

The next turn pops it and prepends _build_interruption_note("user_stopped"):

"The user deliberately stopped your previous response before it finished (the last assistant message above is the partial that was delivered). Treat that as meaningful feedback — do not resume or repeat it."

Every clause is false when the answer was complete, and the note measurably steers the following answer. A reload also shows the "response interrupted" affordance against a complete message.

Verified in dev, 2026-09-02: Stop at 2.6s on a 10.4s turn → full answer persisted → the follow-up turn logged Cleared interrupted_turn … (reason=user_stopped), i.e. the note fired against a complete reply.

The fix

Same reconciliation already used for pending attachments: a turn that reaches the end of the success path produced a complete answer, so it was not interrupted — whatever the client signalled. The genuine interruption arms live in except blocks and re-set the marker after persisting their partial, so a real interruption is untouched.

Deliberately not fixed by retuning the heartbeat. A turn shorter than one tick is unstoppable however the ticks are spaced, and check-then-sleep just moves the first check to t=0, before any Stop could have been pressed. The marker has to be reconciled against what actually happened.

Scope note — what this does NOT fix

Stop still does not stop a sub-10s turn; the model runs to completion and is billed in full. The cost impact is small (turns too short to stop are short because they're cheap), so I've left the cadence alone rather than adding a DynamoDB write every couple of seconds to every turn. Worth a separate decision if you want Stop to be genuinely immediate.

Testing

3 new tests: a completed turn clears the marker, an interrupted turn keeps it, and a clear failure never breaks the stream. Confirmed the first fails without the fix.

Full backend suite: 7014 passed, 6 skipped.

🤖 Generated with Claude Code

A completed turn could leave an interrupted-turn marker behind, and the NEXT
turn then fed the model a false account of the conversation.

HOW THE MARKER GETS THERE. The client's Stop writes `lastTurnInterrupted`
immediately (app-api, source=client_signal), but the server only observes the
armed cancel on the lease heartbeat — and that loop sleeps
LEASE_HEARTBEAT_SECONDS (10s) BEFORE its first check:

    while True:
        await asyncio.sleep(LEASE_HEARTBEAT_SECONDS)
        cancel_requested = await renew_session_lease(lease)

So a turn that finishes inside that window races the first tick and wins. The
stream completes normally, `session_manager.cancelled` is never flipped, the
cooperative-stop arm never runs, and the marker survives describing a turn
that was never cut short.

WHY THAT MATTERS. The next turn pops the marker and prepends
`_build_interruption_note("user_stopped")`, which tells the model that "the
user deliberately stopped your previous response before it finished (the last
assistant message above is the partial that was delivered)" and to "not resume
or repeat it". Every clause is false when the answer was complete, and the
note demonstrably steers the following answer. A reload also shows the
"response interrupted" affordance against a complete message.

Verified in dev 2026-09-02: Stop at 2.6s on a 10.4s turn; full answer
persisted; the follow-up turn logged "Cleared interrupted_turn ...
(reason=user_stopped)", i.e. the note fired against a complete reply.

THE FIX is the same reconciliation already used for pending attachments: a
turn that reaches the end of the success path produced a complete answer, so
it was not interrupted, whatever the client signalled. The genuine
interruption arms sit in `except` blocks and re-set the marker after
persisting their partial, so a real interruption is untouched.

Deliberately NOT fixed by retuning the heartbeat: a turn shorter than one tick
is unstoppable however the ticks are spaced, and check-then-sleep just moves
the first check to t=0, before any Stop can have been pressed. The marker has
to be reconciled against what actually happened.

Pre-existing bug, unrelated to the 5f34d2b0 outage work — found while
validating those fixes against dev.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@philmerrell
philmerrell merged commit 748bb7e into develop Sep 2, 2026
4 checks passed
@philmerrell
philmerrell deleted the fix/false-interrupted-marker branch September 2, 2026 17:02
@philmerrell philmerrell mentioned this pull request Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant