Skip to content

feat(agent): compact the run's own messages, and count what it costs - #311

Merged
yusiwen merged 1 commit into
masterfrom
feat/run-compaction
Oct 28, 2026
Merged

yusiwen merged 1 commit into
masterfrom
feat/run-compaction

Conversation

@yusiwen

@yusiwen yusiwen commented Oct 28, 2026

Copy link
Copy Markdown
Owner

What changed

  • agent/compaction.go (new): compactRunMessages(ctx, messages, step) compacts the loop's own traffic
    at a step boundary once it crosses CompressionThreshold. Head (system prompt, history, this run's
    prompt) and the last eight messages stay verbatim; the rest becomes
    [COMPACTED RUN — REFERENCE ONLY]. The cut snaps forward to an assistant message, so a tool_calls
    message never loses its results. A failed summary logs and leaves the messages alone.
  • agent/compression.go: summarizeText is now the single summary call site and it goes through
    recordCall, so the summary's tokens and cost reach UsageTotal, the budgets and the status bar —
    they were invisible before. describeMessages is shared by both compaction paths.
  • agent/agent.go: the loop calls the compaction hook before the provider call.
  • Pin persistence: chatMessage.Pinned → persistSession → historyFromSession; the TUI mirrors the
    agent's pins into the transcript by role and content (newest first), because the transcript also holds
    system and banner messages. PinnedTurnTexts is the view it mirrors from.
  • Docs: CODEBASE.md (the two new functions, the account path), roadmap B5 (S1 and S2 landed, S3 and S4
    open), test count 1270 → 1277.

Why

Compaction ran only at the start of a run, on a.History. The messages the loop actually sends — every
step's assistant message and every tool result — were never compressed, so a long round grew until the
provider refused. A continuation round (D7) is one Run, which makes that the routine case rather than
the edge one.

The accounting gap is the same class of defect: the summary was a real provider call whose spend no
budget could see. Fixing it once, at the single call site, is cheaper than remembering it at each caller.

Verification

  • go test ./... -count=1 → 16 packages ok; go vet and gofmt -l clean.
  • Mutation check: the loop hook disabled → TestRunCompactsItsOwnMessages fails with
    no provider request carried a compacted run across 12 calls; restored, it passes.
  • New tests: 5. Compaction summarizes the middle and keeps head and tail (and the summarizer's usage
    lands in UsageTotal); the cut never separates a tool call from its results; a short run is untouched
    with no provider call; compressHistory now counts its own call; and a full run through Run carries
    the compaction marker into a later request.
  • TUI: /pin marks the transcript, persistSession writes the flag, historyFromSession restores it.
  • git log --format='%h %G? %s' → G.

Honest scope

  • The compaction threshold is the history threshold (compression_threshold, 500k by default): a run
    has to grow a long way before it compacts. That is deliberate for now — a lower run-level bound is a
    tuning decision with no measurement behind it.
  • Path-targeted unload is not here. /drop <path> cannot act on the run's messages: they live inside
    Run, and the TUI refuses commands while a run is active. S2's acceptance sentence assumed a command
    channel that does not exist; the issue is corrected and the work is S4 (a pre-run rule compaction
    honours when it walks tool calls by path, or a tool the model calls).
  • Pin persistence matches by role and content, so two turns with byte-identical text pin the newest one.
    Acceptable for a command a person types; noted rather than hidden.

Follow-ups

Refs #306, #168

B5's S2. compressHistory ran only at the start of a run, on a.History; the
loop's own messages — every assistant step and every tool result — grew inside a
run and were never compressed, so a long round grew until the provider refused.
That is the failure D7's continuation rounds make routine.

`compactRunMessages` compacts that stream at a step boundary once it crosses the
compression threshold: the head (system prompt, history, this run's prompt) and
the last eight messages stay verbatim, the rest becomes one
`[COMPACTED RUN — REFERENCE ONLY]` message. The cut snaps forward to an
assistant message, because each round of the loop starts with one and a
tool_calls message separated from its results is rejected by the provider. A
failed summary is a log line: the messages stay as they were and the run
continues.

The summarizer call is now accounted. It used to bypass `recordCall`, so its
tokens and cost were invisible to UsageTotal, the token budgets and the status
bar — a pre-existing gap that compaction would have multiplied. One helper,
`summarizeText`, is the only place compression talks to a provider, so the
spend is counted in exactly one place.

Pins now survive a resume: `chatMessage.Pinned` travels through persistSession
into the session file, and historyFromSession restores it into the history the
compressor reads. The TUI mirrors the agent's pins into the transcript by
matching role and content, newest first, because the transcript also holds
system and banner messages and the two lists are not index-aligned.

Path-targeted unload is split out as S4 and recorded in the issue: `/drop
<path>` cannot act on the run's messages, because those live inside Run and the
TUI refuses commands while a run is active. The acceptance sentence that assumed
a command channel is corrected there rather than quietly dropped.

Verified: go test ./... -count=1, 16 packages ok; go vet and gofmt clean; the
loop hook mutation-checked (disabling it fails TestRunCompactsItsOwnMessages
with "no provider request carried a compacted run across 12 calls").

Refs #306, #168
@yusiwen yusiwen added the feature A new capability rather than an improvement to existing behaviour label Oct 28, 2026
@yusiwen
yusiwen merged commit 65d35d4 into master Oct 28, 2026
9 checks passed
@yusiwen
yusiwen deleted the feat/run-compaction branch October 28, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature A new capability rather than an improvement to existing behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant