Skip to content

feat(agent): truncate tool output to a token budget, with a tail and a summary - #305

Merged
yusiwen merged 1 commit into
masterfrom
feat/truncate-token-budget
Oct 28, 2026
Merged

yusiwen merged 1 commit into
masterfrom
feat/truncate-token-budget

Conversation

@yusiwen

@yusiwen yusiwen commented Oct 28, 2026

Copy link
Copy Markdown
Owner

What changed

  • agent/truncate.go: the preview is now bounded by a token budget (TruncMaxTokens, 4000) beside
    the line and byte caps, keeps the tail (TruncTailLines, 20), and reports what it dropped:
    ... 18,431 of 20,000 lines omitted (4 mention an error, 9 a warning) .... The counts come from a
    regexp scan of the dropped lines. A single line larger than every bound is still clipped into a
    readable preview rather than producing an empty one.
  • config: truncation.max_tokens and truncation.tail_lines, in DefaultConfig and in
    config.example.json; SetTruncationConfig takes the two new values and keeps its rule that zero or
    negative values never lower a bound.
  • agent/truncate_test.go: rewritten around the new behaviour — the budget, the tail, the summary
    counts, the untouched small output, the enormous first line, and two pins (the estimator's inverse, and
    package defaults == config defaults).
  • Docs: CODEBASE.md's truncation section, the example config's note, roadmap B6 → Delivered — [agent] Truncate tool output to a token budget, keeping head, tail and a summary #304
    (16 delivered, 22 not started, 40 total).

Why

The old preview measured bytes the model does not pay for and kept the head. On a failing command that is
exactly backwards: the evidence is at the end, and the head can be 50k tokens of build log. The token
budget is the bound that matters; the tail is where the failure is; the counts are what let a reader
decide whether to open the saved file at all.

The summary is deliberately not a model call: the output was already too large to keep, and paying a
second call to describe it would move the cost rather than remove it.

Verification

  • go test ./... -count=1 → 16 packages ok; go vet and gofmt -l clean.
  • Mutation check: the head budget replaced with the fixed byte cap → TestTruncateRespectsTheTokenBudget
    fails; restored, it passes.
  • New tests: 9 in agent/truncate_test.go, including the control (an output inside every bound is
    returned byte for byte, no file written) and TestBytesPerTokenMatchesTheEstimator.
  • git log --format='%h %G? %s' → G.

Honest scope

  • The summary counts lines, not causes: it says "4 mention an error", not "the build failed at step 3".
    That is the deliberate limit of a deterministic scan.
  • The budget covers head and tail; the message and the file pointer add a small amount on top, which the
    test allows for explicitly.
  • No per-tool budgets: one budget applies to every tool result.

Follow-ups

Refs #304, #168

…a summary

The preview was bounded by lines and bytes, and it was the head only. Two
things were wrong with that, and both are the roadmap's B6:

- a byte cap measures the wrong thing once text is dense: 200 KiB of it is ~50k
  tokens, a whole context window spent on one tool result. The preview now has a
  token budget (4000 by default) beside the two caps, measured with the same
  estimator the compression threshold uses.
- the kept part was the beginning. A failing command's evidence — the panic, the
  FAIL, the last warning — is at the end. The preview now keeps the last
  `tail_lines` (20) as well, and the budget covers the whole preview: the tail is
  taken first, the head gets what is left.

What fell between them is counted rather than merely mentioned: `... 18,431 of
20,000 lines omitted (4 mention an error, 9 a warning) ...`. The counts are a
regexp scan over the dropped lines — cheap, deterministic, explainable — because
a model call to summarise output that was already too large would cost another
turn.

Config: `truncation.max_tokens` and `truncation.tail_lines`, defaults in
`DefaultConfig`, and a test pins the package defaults to the config defaults.
`SetTruncationConfig` keeps its rule that zero or negative values never lower a
bound.

Verified: go test ./... -count=1, 16 packages ok; go vet and gofmt clean; the
budget mutation-checked (a fixed byte budget in place of the token one fails
TestTruncateRespectsTheTokenBudget, restored it passes).

Refs #304, #168
@yusiwen yusiwen added the enhancement New feature or request label Oct 28, 2026
@yusiwen
yusiwen merged commit ccec4e1 into master Oct 28, 2026
9 checks passed
@yusiwen
yusiwen deleted the feat/truncate-token-budget branch October 28, 2026 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant