Repository navigation
feat(agent): truncate tool output to a token budget, with a tail and a summary - #305
Merged
Merged
Conversation
…a summary The preview was bounded by lines and bytes, and it was the head only. Two things were wrong with that, and both are the roadmap's B6: - a byte cap measures the wrong thing once text is dense: 200 KiB of it is ~50k tokens, a whole context window spent on one tool result. The preview now has a token budget (4000 by default) beside the two caps, measured with the same estimator the compression threshold uses. - the kept part was the beginning. A failing command's evidence — the panic, the FAIL, the last warning — is at the end. The preview now keeps the last `tail_lines` (20) as well, and the budget covers the whole preview: the tail is taken first, the head gets what is left. What fell between them is counted rather than merely mentioned: `... 18,431 of 20,000 lines omitted (4 mention an error, 9 a warning) ...`. The counts are a regexp scan over the dropped lines — cheap, deterministic, explainable — because a model call to summarise output that was already too large would cost another turn. Config: `truncation.max_tokens` and `truncation.tail_lines`, defaults in `DefaultConfig`, and a test pins the package defaults to the config defaults. `SetTruncationConfig` keeps its rule that zero or negative values never lower a bound. Verified: go test ./... -count=1, 16 packages ok; go vet and gofmt clean; the budget mutation-checked (a fixed byte budget in place of the token one fails TestTruncateRespectsTheTokenBudget, restored it passes). Refs #304, #168
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
agent/truncate.go: the preview is now bounded by a token budget (TruncMaxTokens, 4000) besidethe line and byte caps, keeps the tail (
TruncTailLines, 20), and reports what it dropped:... 18,431 of 20,000 lines omitted (4 mention an error, 9 a warning) .... The counts come from aregexp scan of the dropped lines. A single line larger than every bound is still clipped into a
readable preview rather than producing an empty one.
config:truncation.max_tokensandtruncation.tail_lines, inDefaultConfigand inconfig.example.json;SetTruncationConfigtakes the two new values and keeps its rule that zero ornegative values never lower a bound.
agent/truncate_test.go: rewritten around the new behaviour — the budget, the tail, the summarycounts, the untouched small output, the enormous first line, and two pins (the estimator's inverse, and
package defaults == config defaults).
CODEBASE.md's truncation section, the example config's note, roadmap B6 → Delivered — [agent] Truncate tool output to a token budget, keeping head, tail and a summary #304(16 delivered, 22 not started, 40 total).
Why
The old preview measured bytes the model does not pay for and kept the head. On a failing command that is
exactly backwards: the evidence is at the end, and the head can be 50k tokens of build log. The token
budget is the bound that matters; the tail is where the failure is; the counts are what let a reader
decide whether to open the saved file at all.
The summary is deliberately not a model call: the output was already too large to keep, and paying a
second call to describe it would move the cost rather than remove it.
Verification
go test ./... -count=1→ 16 packagesok;go vetandgofmt -lclean.TestTruncateRespectsTheTokenBudgetfails; restored, it passes.
agent/truncate_test.go, including the control (an output inside every bound isreturned byte for byte, no file written) and
TestBytesPerTokenMatchesTheEstimator.git log --format='%h %G? %s'→G.Honest scope
That is the deliberate limit of a deterministic scan.
test allows for explicitly.
Follow-ups
[memory] Align the Hindsight adapter with a real deployment #302 owns the Hindsight adapter's contract.
Refs #304, #168