Skip to content

Audit fixes P0-P3: correctness, honest tools, e2e rig, thinking dials - #42

Merged
mike-diff merged 5 commits into
mainfrom
fix/repo-audit-p0-p3
Aug 14, 2026
Merged

Audit fixes P0-P3: correctness, honest tools, e2e rig, thinking dials#42
mike-diff merged 5 commits into
mainfrom
fix/repo-audit-p0-p3

Conversation

@mike-diff

Copy link
Copy Markdown
Owner

What

A full audit pass over the repo, every finding verified empirically first
(the real binary, headless, against a scripted openai+anthropic endpoint
with wire captures), then fixed in four waves. Unit tests were written
before each fix and proven by mutation: every breaker was applied and its
test confirmed red before being reverted.

The four commits

Commit Wave Contents
fix(p0) correctness write preserves file mode (a 0755 script kept its exec bit; a 0600 file was widened to 0644 on every overwrite). Anthropic adapter never serializes an empty assistant turn as content:null (a 400 on every later call in that session). max_tokens becomes a dial (profile max_tokens + max_output_tokens tuning) and a 400 naming the model's real output cap self-heals to it: 16000 -> 400 -> retry at 8192 verified on the wire, so sub-16k-output models work out of the box. A failed turn keeps exchanges whose tool results already changed disk (a completed write used to vanish from history while the file stayed). An unparseable judge verdict gets one JSON-only repair retry, then defaults to not-done instead of stopping the drive; transport failures still stop.
feat(p1) honest tools read gains offset/limit (1-based lines) and an over-cap read ends with [N of M lines shown; re-read with offset N+1] instead of a generic truncation: the model can finally reach the end of a big file. search names files over the 1MB guard it did not search (a silent skip let the model confidently report the wrong file set). loc counts mainstream extensions (java, rb, cpp, tsx, ...) and discloses what it did not count. The gate mod executable re-resolves per mutating call, so a boundary installed mid-session binds immediately. mcp_timeout_secs dial replaces the fixed 60s call ceiling.
feat(p2) infra Startup session scans (lastBrain, latestSession) read each file's header only, cut at the turns key: startup cost is O(header bytes) across every saved session, not O(all transcripts). A standing end-to-end rig (SESH_E2E=1, wired into CI) builds the binary and drives it headless against a scripted endpoint with isolated HOMEs, pinning all the regressions above plus the gate-mod path and the anthropic wire shapes.
feat(p3) features thinking_budget (anthropic extended thinking; max_tokens is raised to clear the budget) and reasoning_effort (openai hint; a max_tokens cap then travels as max_completion_tokens) as profile fields, both default-off. The adapter captures streamed thinking blocks with signatures onto the assistant turn and passes them back on the final assistant message of the next request: the API rejects tool-use continuations under thinking without that round-trip.

Validation

  • Red-first tests; every fix breaker-verified by mutation (applied, test
    failed, reverted). Two of my own assertions and one pre-existing pin
    (which had codified the silent search skip) were corrected in the process.
  • User-level runs per wave: sesh -p -yes against the scripted mock with
    wire dumps (9/9 original finding scenarios pass on the final tree).
  • The e2e rig itself was proven source-sensitive: reverting a fix at the
    source makes the rig red.
  • gofmt -l . clean, go vet ./... clean, go test ./... green; CI now
    also runs the e2e rig on every push and PR.

Behavior notes

  • search and loc outputs can grow one disclosure line each; the old
    silent behavior was the bug.
  • A failed or cancelled turn now leaves completed tool work in history (with
    an adjusted notice); only an unconsumed opening rolls back.
  • New profile dials: max_tokens, thinking_budget, reasoning_effort;
    new tuning dials: max_output_tokens, mcp_timeout_secs (documented in
    the scaffolded tuning.json.example and ~/.sesh README).

Left open

  • Live cost/success bench for thinking_budget against a real thinking
    model before recommending it beyond default-off (wire behavior is pinned
    by unit + e2e tests).

- read: offset (1-based line) and limit page past the 28k-char cap; an
  over-cap unpaged read ends with [N of M lines shown; re-read with
  offset N+1] instead of the core's generic truncation, and a paged read
  footers its exact window (lines X-Y of Z)
- search: files over the 1MB guard are named in the output instead of
  silently shrinking the result set, on every shape including no-matches
- loc: counts the mainstream code extensions (java, rb, cpp, tsx, ...)
  and discloses what it did not count, by extension
- gate: the gate mod executable is re-resolved on every mutating call, so
  a boundary installed or fixed mid-session binds at the next mutation
- mcp: the per-call ceiling is the mcp_timeout_secs dial (default 60s),
  for both the stdio handshake and the http transport
- sessions: the startup scans (lastBrain, latestSession) read each file's
  header only, cut at the turns key, instead of parsing every transcript
  ever saved; startup cost is O(header bytes), not O(all transcripts),
  and a torn file that fails a full parse still lists (fallback covers a
  header beyond the 64KB window). -list keeps its full-load counts
- e2e: harness/e2e_test.go (opt-in SESH_E2E=1, wired into CI) builds the
  binary and drives it headless against a scripted openai+anthropic
  endpoint with isolated HOMEs: write mode preservation, work surviving
  failed drive iterations, judge prose retry and fallback, read paging,
  oversize search disclosure, the empty-reply anthropic resume, the
  max_tokens profile wire value and its self-heal, and gate-mod denial.
  These are the regressions the verification rig actually caught
- profile fields: thinking_budget (anthropic extended thinking; thinking
  bills as output, max_tokens is raised to clear the budget) and
  reasoning_effort (openai-protocol hint; when set, a max_tokens cap
  travels as max_completion_tokens, which reasoning models require)
- the anthropic adapter captures streamed thinking blocks with their
  signatures (and redacted blobs) onto the assistant turn, and passes
  them back on the final assistant message of the next request: the API
  rejects tool-use continuations under thinking without that round-trip.
  Display stays streaming-only; persistence exists so the wire contract
  holds across a session and its resumption
- buildProvider takes the profile's dials as one value; resolveSpec
  carries them; both defaults keep today's wire shape exactly
- sessionMeta carries only the fields the startup scans read (id, cwd,
  continued_by, updated); title/provider/protocol/url/model were parsed
  but never consumed
- the beyond-64KB-header fallback test now pins the fallback itself: a
  session whose header exceeds the meta window must stay visible to
  brain adoption via the full-load path (verified by breaker: removing
  the fallback fails the test; the previous shape could not)
- challenged and kept: brainDials (replaces an 8-parameter
  buildProvider; every field consumed), the five new dials (each maps
  to the repo's assumptions-live-in-files doctrine and every default
  preserves the prior wire, pinned by tests), the e2e mock harness
  (one file, the SESH_RIG opt-in pattern, is the PR's user-level
  validation)
@mike-diff
mike-diff merged commit 1649ae5 into main Aug 14, 2026
2 checks passed
@mike-diff
mike-diff deleted the fix/repo-audit-p0-p3 branch August 14, 2026 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant