Skip to content

[agent] Truncate tool output to a token budget, keeping head, tail and a summary #304

Closed

Description

@yusiwen

Context

Roadmap B6 (anchor #168): tool output is truncated by a line cap and a byte cap, not by what the model
pays for, and a truncated result is a head-only slice of raw text.

agent/truncate.go splits on lines and keeps the first max_lines / max_bytes; the rest is written to
a file and the model is told where it went. Two consequences:

  • The preview's size is measured in bytes the model does not pay for directly: 200 KiB of dense text is
    ~50k tokens, which is a whole context window spent on one tool result.
  • The kept part is the head. A failing command's evidence — the panic, the FAIL, the last warning
    — is at the end, and a head-only preview drops exactly the lines a reader needs.

What to build

  • A token budget beside the line and byte caps (truncation.max_tokens, default 4000, measured with
    the same estimator the compression threshold uses, EstimateTokens). The budget covers the whole
    preview: the tail is taken first, the head gets what is left.
  • A tail (truncation.tail_lines, default 20): the last lines of the output, so a failure survives
    truncation.
  • A summary of what was omitted, not just a byte count: how many lines were dropped, and how many of
    them mention an error or a warning. That is a deterministic scan, not a model call — the preview must
    stay cheap and reproducible.
  • The saved file, its 0600 mode and the [TRUNCATED] pointer stay exactly as they are.

Acceptance

  • An output past max_tokens produces a preview whose EstimateTokens is within the budget (the tail
    included), and the last line of the output is present in it.
  • The omitted-lines line reports the real count and the error/warning counts of the dropped lines.
  • An output inside every bound is returned unchanged, byte for byte, with no file written.
  • A single line larger than the byte cap still yields a preview that starts with that line's first
    characters (never an empty preview).
  • SetTruncationConfig keeps its rule that zero or negative values do not lower a bound.
  • The config defaults and the package defaults are asserted equal.
  • go test ./... -count=1, -race, go vet, gofmt, staticcheck clean.

Out of scope

Summarising with a model (the preview must be free and deterministic), and per-tool budgets: this is one
budget for every tool result.

Activity

  1. yusiwen commented on Oct 28, 2026

    @yusiwen
    OwnerAuthor

    Delivered as #305, merged as ccec4e1.

    What the preview does now, against this issue's acceptance list:

    • bounded by truncation.max_tokens (default 4000) as well as lines and bytes, with the budget covering
      the whole preview — the tail is taken first, the head gets the rest;
    • keeps tail_lines (default 20) of the end, so the panic / FAIL / last warning survives;
    • reports ... N of M lines omitted (E mention an error, W a warning) ..., counted by a regexp scan of
      the dropped lines, not a model call;
    • an output inside every bound is returned byte for byte with no file written (the control test);
    • an enormous first line is still clipped into a readable preview instead of an empty one;
    • SetTruncationConfig still never lowers a bound on a zero or negative value, and a test pins the
      package defaults to DefaultConfig.

    One implementation note worth recording: the line cap had to be re-applied inside the new head helper —
    the first version bounded the head by bytes and tokens only, and the summary test caught it by producing
    no omitted lines at all.

    Roadmap B6 is marked Delivered — #304.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions