Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions CODEBASE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# TinyCode — CODEBASE Map

> AI coding agent in pure Go. Single binary, Bubble Tea TUI, ReAct agent loop, 25 built-in tools + MCP, LSP diagnostics, session persistence. 1285 test functions + 10 fuzz targets, race-detector clean.
> AI coding agent in pure Go. Single binary, Bubble Tea TUI, ReAct agent loop, 25 built-in tools + MCP, LSP diagnostics, session persistence. 1291 test functions + 10 fuzz targets, race-detector clean.

## Quick Reference

Expand Down Expand Up @@ -264,6 +264,7 @@ provider is implemented — reaching IMDS from an agent process is a separate se

### `context.go` — Turn-level Context Control

- **`(*Agent) KeepPath(p)` / `UnkeepPath(p)` / `keepsPath(args)`** — a kept path is one whose tool reads must survive the compaction of a run: `compactRunMessages` keeps any round whose `tool_calls` arguments name it and summarizes the rest. Set before a run; the list travels with the session (`session.Session.KeptPaths`) so a resume keeps the rule.
- **`(*Agent) PinLast(n)` / `UnpinLast(n)` / `DropLast(n)` / `PinnedTurnTexts()` / `PinnedTurnCount()`** — a *turn* is a user
message plus everything up to the next one, the grouping `compressHistory` works in. Pins ride on
`types.Message.Pinned` and are honoured by the compressor; `/drop` removes turns from the live history
Expand Down Expand Up @@ -873,7 +874,7 @@ User Input (textarea / CLI arg)

## Testing

- **1285 test functions + 10 fuzz targets** across the repository — what `go test ./... -count=1` runs for the main module, plus the evaluation corpora under `eval/` (nested modules the command still walks); each count below is listed with the command that produces it, and no count is carried in a copy that nothing checks
- **1291 test functions + 10 fuzz targets** across the repository — what `go test ./... -count=1` runs for the main module, plus the evaluation corpora under `eval/` (nested modules the command still walks); each count below is listed with the command that produces it, and no count is carried in a copy that nothing checks
- `make fuzz` (`FUZZTIME=30s`) explores every fuzz target; `go test` already runs their seed corpora, so CI exercises them on every push
- `make test-browser` (`BROWSER_TEST=1`, `-run TestBrowserSmoke`) renders a JavaScript page through both browser paths against a loopback server and asserts the request carried the proxy's `Via` header, which proves the filtering proxy was used; `TestBrowserSmokeRefusesABlockedSubresource` then serves a page whose own authority is exempted from the loopback rule while the policy stays enforced, and asserts a subresource pointing at a *second, live* loopback service is refused (the script's `onerror` marker in the DOM) and that the service was never reached — a refusal is proven, not only that the proxy was used (issue #8). Both skip without a browser, so `make test` never launches one
- `make test-tui-visual` (`TUI_SHOT=1`) is the only place the TUI is looked at rather than asserted on: it renders the committed frame scenarios to PNGs through headless Chromium, starts `bin/tinycode` on a real 80x24 PTY (asserting the stream carried SGR styling and that the binary quits on the documented double Ctrl+C), runs it once more on a size-less PTY, and replays the live stream into a screen buffer that is screenshotted as `tinycode-pty-welcome-80x24.png`. Both tests skip without the variable, so `make test` needs neither a browser, nor a binary, nor a terminal device; PNGs land in `TUI_SHOT_DIR` (default `/tmp`)
Expand Down Expand Up @@ -919,7 +920,7 @@ Counts in this document are produced by these commands at the commit they descri

```bash
find . -name '*_test.go' -not -path './tuiprobe/*' -print0 \
| xargs -0 grep -h '^func Test' | wc -l # 1285 test functions (main module)
| xargs -0 grep -h '^func Test' | wc -l # 1291 test functions (main module)
grep -rn '^func Fuzz' --include=*_test.go . | wc -l # 10 fuzz targets
ls tui/testdata/golden/frames/*.txt | wc -l # 29 plain-text frames
ls tui/testdata/golden/ansi/*.ansi | wc -l # 1 raw ANSI frame
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Custom **CellGrid** frame-buffer renders markdown directly in the terminal — n
- **GitHub Actions**: Two workflows — main.yml (build + lint + test on push/PR) and release.yml (cross-compile + GitHub Releases on tags v*)
- **main.yml jobs**: `ci` (build, gofmt gate, vet, tests, `-race`, repeated run), `lsp` (real gopls), `browser` (real Chromium), `tui-visual` (frame PNGs + the built binary on a PTY), `cross` (linux/amd64, linux/arm64, darwin/arm64 build + vet) and `staticcheck`
- **Makefile improvements**: test target preserves exit code with pass/fail message; releases target cross-compiles all platforms + .tar.gz archives
- **1285 test functions + 10 fuzz targets** across the repository — the main module plus the evaluation corpora, with `tuiprobe/` a nested module the command below excludes — counted with `find . -name '*_test.go' -not -path './tuiprobe/*' -print0 | xargs -0 grep -h '^func Test' | wc -l` and `grep -rn '^func Fuzz' --include=*_test.go . | wc -l`; the command is part of the record, and `CODEBASE.md` → Testing lists every count with its measurement
- **1291 test functions + 10 fuzz targets** across the repository — the main module plus the evaluation corpora, with `tuiprobe/` a nested module the command below excludes — counted with `find . -name '*_test.go' -not -path './tuiprobe/*' -print0 | xargs -0 grep -h '^func Test' | wc -l` and `grep -rn '^func Fuzz' --include=*_test.go . | wc -l`; the command is part of the record, and `CODEBASE.md` → Testing lists every count with its measurement
- **Annotations**: the jobs carry one `ubuntu-latest` migration notice each, plus the `setup-chrome@v1` Node 20 warning on `browser` and `tui-visual`; the gate is no *new* annotations, not zero

### Running the checks locally
Expand Down
4 changes: 4 additions & 0 deletions agent/agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ type Agent struct {
// MemoryRecallLimit is how many facts the recall step injects; zero means
// DefaultMemoryRecallLimit. It comes from memory.recall_limit.
MemoryRecallLimit int
// KeptPaths name files whose tool reads survive the compaction of a run
// (roadmap B5, S4). Set before a run; the TUI persists the list with the
// session so a resume keeps the rule.
KeptPaths []string
// Session persistence
SessionStore interface {
Append(msg types.Message) error
Expand Down
45 changes: 43 additions & 2 deletions agent/compaction.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,18 @@ import (
"github.com/yusiwen/tinycode/types"
)

// roundKeepsPath reports whether any tool call in a round names a kept path.
func (a *Agent) roundKeepsPath(round []types.Message) bool {
for _, msg := range round {
for _, call := range msg.ToolCalls {
if a.keepsPath(call.Arguments) {
return true
}
}
}
return false
}

// compactRunMessages compresses the *run's own* traffic — the assistant and tool
// messages between the prompt and now — when it crosses the compression
// threshold (roadmap B5, S2).
Expand Down Expand Up @@ -53,13 +65,41 @@ func (a *Agent) compactRunMessages(ctx context.Context, messages []types.Message
return messages, false
}

// A round whose tool call names a kept path is context the user asked to
// keep: it is not summarized, and it is put back after the marker and before
// the recent tail (roadmap B5, S4).
var (
text strings.Builder
kept []types.Message
)
for i := 0; i < cut; {
end := i + 1
if rest[i].Role == types.RoleAssistant {
for end < cut && rest[end].Role == types.RoleTool {
end++
}
}
round := rest[i:end]
if round[0].Role == types.RoleAssistant && a.roundKeepsPath(round) {
kept = append(kept, round...)
} else {
text.WriteString(describeMessages(round))
}
i = end
}
if strings.TrimSpace(text.String()) == "" {
// Everything in the compactable part is kept: a summary would be of
// nothing, so leave the run alone.
return messages, false
}

const instruction = `Summarize what this run has done so far. Focus on:
- The tools that were called and what they established
- Files read or modified, and why
- What is still open

Run so far:`
summary, err := a.summarizeText(ctx, instruction, describeMessages(rest[:cut]), 1024, step)
summary, err := a.summarizeText(ctx, instruction, text.String(), 1024, step)
if err != nil {
// A failed compaction costs the run one step of context, not the run:
// the messages stay as they were and the loop continues.
Expand All @@ -70,12 +110,13 @@ Run so far:`
return messages, false
}

out := make([]types.Message, 0, len(messages)-cut+1)
out := make([]types.Message, 0, len(messages)-cut+1+len(kept))
out = append(out, messages[:prompt+1]...)
out = append(out, types.Message{
Role: types.RoleSystem,
Content: fmt.Sprintf("[COMPACTED RUN — REFERENCE ONLY]\n%s\n[END COMPACTED RUN]", summary),
})
out = append(out, kept...)
out = append(out, rest[cut:]...)
tlog.Info("agent.loop", "compacted run messages", "summarized", cut, "kept", len(rest)-cut, "step", step)
return out, true
Expand Down
74 changes: 74 additions & 0 deletions agent/compaction_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -190,3 +190,77 @@ func TestRunCompactsItsOwnMessages(t *testing.T) {
t.Fatalf("no provider request carried a compacted run across %d calls", len(provider.requests))
}
}

// keptRunMessages builds a run whose second round reads a file the test keeps,
// followed by filler rounds and a recent tail.
func keptRunMessages() []types.Message {
messages := []types.Message{
{Role: types.RoleSystem, Content: "system prompt"},
{Role: types.RoleUser, Content: "do the thing"},
}
round := func(id, path, output string) []types.Message {
return []types.Message{
{Role: types.RoleAssistant, ToolCalls: []types.ToolCall{{
ID: id, Name: "read_file", Arguments: `{"path":"` + path + `"}`,
}}},
{Role: types.RoleTool, ToolCallID: id, Name: "read_file", Content: output},
}
}
messages = append(messages, round("1", "internal/filler.go", "filler one")...)
messages = append(messages, round("2", "internal/kept.go", "KEPT CONTENT")...)
for i := 0; i < 4; i++ {
messages = append(messages, round(string(rune('3'+i)), "internal/filler.go", "filler round")...)
}
return messages
}

// TestCompactRunMessagesKeepsPathsTheUserKept is S4's acceptance: a read of a
// kept path survives compaction verbatim and the summary does not absorb it,
// while a read of any other path is summarized away as before.
func TestCompactRunMessagesKeepsPathsTheUserKept(t *testing.T) {
provider := &recordingProvider{content: "RUN SUMMARY"}
ag := New(provider)
ag.ContextLength = 1000
ag.CompressionThreshold = 1
if !ag.KeepPath("internal/kept.go") {
t.Fatal("the path was not kept")
}

in := keptRunMessages()
out, changed := ag.compactRunMessages(context.Background(), in, 2)
if !changed {
t.Fatal("the run was not compacted")
}
if !containsMessage(out, "KEPT CONTENT") {
t.Fatal("the kept read was summarized away")
}
if containsMessage(out, "filler one") {
t.Fatal("an unkept read survived the summary")
}
if !containsMessage(out, "RUN SUMMARY") {
t.Fatal("the summary is missing")
}
if prompt := provider.requests[0].Messages[1].Content; strings.Contains(prompt, "KEPT CONTENT") {
t.Fatalf("the summarizer was given the kept read:\n%s", prompt)
}
}

// TestCompactRunMessagesDoesNothingWhenEveryRoundIsKept: if every read is kept,
// a summary would be of nothing, so the run is left alone and no call is paid.
func TestCompactRunMessagesDoesNothingWhenEveryRoundIsKept(t *testing.T) {
provider := &recordingProvider{content: "RUN SUMMARY"}
ag := New(provider)
ag.ContextLength = 1000
ag.CompressionThreshold = 1
ag.KeepPath("internal/filler.go")
ag.KeepPath("internal/kept.go")

in := keptRunMessages()
out, changed := ag.compactRunMessages(context.Background(), in, 2)
if changed {
t.Fatalf("a fully kept run was compacted: %d -> %d messages", len(in), len(out))
}
if len(provider.requests) != 0 {
t.Fatalf("the summarizer was called %d times for nothing", len(provider.requests))
}
}
58 changes: 57 additions & 1 deletion agent/context.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,62 @@
package agent

import "github.com/yusiwen/tinycode/types"
import (
"path/filepath"
"strings"

"github.com/yusiwen/tinycode/types"
)

// Path-level control over compaction (roadmap B5, S4).
//
// A run's messages hold the tool results, so "unload this file" is a decision
// about those. `/drop <path>` cannot make it — the messages live inside Run and
// the TUI refuses commands while a run is active — so the decision is made
// before the run instead: a kept path's reads survive compaction verbatim, and
// every other read is summarized away as usual.

// KeepPath adds a path whose tool reads must survive compaction. The path is
// cleaned and stored once; an empty path is ignored.
func (a *Agent) KeepPath(path string) (added bool) {
path = filepath.Clean(strings.TrimSpace(path))
if path == "" || path == "." {
return false
}
for _, kept := range a.KeptPaths {
if kept == path {
return false
}
}
a.KeptPaths = append(a.KeptPaths, path)
return true
}

// UnkeepPath removes a path, reporting whether it was there.
func (a *Agent) UnkeepPath(path string) (removed bool) {
path = filepath.Clean(strings.TrimSpace(path))
for i, kept := range a.KeptPaths {
if kept != path {
continue
}
a.KeptPaths = append(a.KeptPaths[:i], a.KeptPaths[i+1:]...)
return true
}
return false
}

// keepsPath reports whether a tool call's arguments name a kept path. Arguments
// are the signal because that is where a read names the file it reads.
func (a *Agent) keepsPath(arguments string) bool {
if arguments == "" {
return false
}
for _, kept := range a.KeptPaths {
if kept != "" && strings.Contains(arguments, kept) {
return true
}
}
return false
}

// Turn-level control over the live history (roadmap B5).
//
Expand Down
24 changes: 24 additions & 0 deletions agent/context_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -194,3 +194,27 @@ func TestPinnedTurnTexts(t *testing.T) {
t.Fatalf("PinnedTurnCount = %d, want 1", got)
}
}

// TestKeepPathCleansAndDeduplicates pins the list's shape: cleaned, stored once,
// and empty values ignored.
func TestKeepPathCleansAndDeduplicates(t *testing.T) {
ag := New(nil)
if !ag.KeepPath(" internal/store.go ") {
t.Fatal("a path was not kept")
}
if ag.KeepPath("internal/store.go") {
t.Fatal("the same path was kept twice")
}
if ag.KeepPath(" ") {
t.Fatal("an empty path was kept")
}
if len(ag.KeptPaths) != 1 || ag.KeptPaths[0] != "internal/store.go" {
t.Fatalf("kept paths = %v", ag.KeptPaths)
}
if !ag.UnkeepPath("internal/store.go") {
t.Fatal("the path was not dropped")
}
if ag.UnkeepPath("internal/store.go") {
t.Fatal("dropping a path twice reported success")
}
}
15 changes: 11 additions & 4 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ after it (see *Provenance and method*).

| State | Entries | Note |
| :--- | ---: | :--- |
| Delivered | 16 | the entry's **Picked up** line names the issue that landed it |
| Delivered | 17 | the entry's **Picked up** line names the issue that landed it |
| Refuted | 1 | E5 — the capability exists, recorded so it is not re-filed |
| In progress | 2 | F1 — three increments landed, the next one is not opened; B5 — S1 landed, S2 and S3 open |
| In progress | 1 | F1 — three increments landed, the next one is not opened |
| Not started | 21 | verified gaps, no issue opened |
| **Total** | **40** | `grep -Ec '^\| [A-H][0-9] \| .* \| (Delivered —\|Refuted\|In progress\|Not started)' docs/roadmap.md` |

Expand All @@ -43,7 +43,7 @@ B4 (repository index), F1, then the larger D and E items.
| B2 | No local memory backend | Delivered — #293 |
| B3 | No embeddings, vector search or rerank | Delivered — #297 |
| B4 | No repository index or repo map | Not started — spiked 2026-10-28, no-go (#308) |
| B5 | Context management has a single strategy | In progress — #306 |
| B5 | Context management has a single strategy | Delivered — #306 |
| B6 | Truncation is fixed, not budget-aware | Delivered — #304 |
| C1 | No checkpoint and no undo | Delivered — #234 |
| C2 | No automatic change preview and commit loop, and no remote operations | Not started |
Expand Down Expand Up @@ -395,7 +395,14 @@ current tree. D7 is the exception: it was recorded later, so its references are
own messages — the part D7 depends on — the summarizer's spend now counted, and pins carried through
the session file), S3 (a compression preview that calls no model — it reads the same partition
`compressHistory` executes), S4 (a path-targeted unload, split out of S2 because the command channel it
assumed does not exist). **S1–S3 landed** (2026-10-28, 2026-10-31); S4 is open.
assumed does not exist). **S1–S4 landed** (2026-10-28 … 2026-10-31); the entry is delivered.
- **Delivered (2026-10-31)**: the single strategy is now four surfaces — turn-level `/pin`, `/unpin` and
`/drop` (pins survive compression and a resume), compaction of the run's own messages at a step
boundary, `/compress preview` (it reads the same plan the compression executes, and makes no model
call), and `/keep <path>`, which keeps a file's tool reads verbatim through a compaction. The
summarizer's spend, which used to be invisible to the budgets, is counted. The one thing not built is a
*mid-run* command to unload a file: the run owns its messages and the TUI refuses commands while it
runs, so the rule is set before the run instead — recorded in #306's S4.
- **Gap**: compression is automatic and all-or-nothing. No `/drop`, no `/pin`, no unloading by
message or by file, and no preview of what a compression would discard.
- **Evidence**: one strategy, `compressHistory` (`agent/compression.go:59`) — head, LLM-summarized
Expand Down
5 changes: 5 additions & 0 deletions session/session.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ type Session struct {
// Permission paths allowed for this session (Allow session)
AllowedPaths []string `json:"allowed_paths,omitempty"`

// KeptPaths are files whose tool reads must survive compaction of a run
// (roadmap B5, S4). They live with the session because the rule is the
// user's, and a resume has to keep it.
KeptPaths []string `json:"kept_paths,omitempty"`

// Containment records what enforced file boundaries when this session ran,
// as reported by the tool package at creation ("kernel (…)",
// "userspace (…)"). It is recorded once because it is a property of the
Expand Down
39 changes: 39 additions & 0 deletions tui/context_command.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ func (m *TuiModel) handleContextCommand(cmd, name string) (tea.Model, tea.Cmd) {
m.ShowStatus("Cannot " + strings.TrimPrefix(name, "/") + ": a run is in progress")
return m, nil
}
if name == "/keep" || name == "/unkeep" {
m.handleKeep(cmd, name)
return m, nil
}

n, err := contextTurns(cmd, name)
if err != nil {
m.appendSystem(err.Error())
Expand Down Expand Up @@ -106,3 +111,37 @@ func (m *TuiModel) syncPinsToTranscript() {
}
}
}

// handleKeep manages the paths whose tool reads survive compaction (S4 of #306).
// The rule is set before a run — that is why it is a command and not something a
// mid-run model call decides — and it is persisted with the session.
func (m *TuiModel) handleKeep(cmd, name string) {
arg := strings.TrimSpace(strings.TrimPrefix(cmd, name))
if name == "/keep" && arg == "" {
if len(m.agent.KeptPaths) == 0 {
m.appendSystem("no kept paths; /keep <path> keeps a file's reads in context across a compaction")
return
}
m.appendSystem("kept paths (their reads survive compaction):\n " + strings.Join(m.agent.KeptPaths, "\n "))
return
}
if arg == "" {
m.appendSystem("usage: /unkeep <path>")
return
}
if name == "/keep" {
if !m.agent.KeepPath(arg) {
m.appendSystem("already kept: " + arg)
return
}
m.persistSession()
m.appendSystem("kept " + arg + ": a tool call that names it is kept verbatim when the run is compacted")
return
}
if !m.agent.UnkeepPath(arg) {
m.appendSystem("not kept: " + arg)
return
}
m.persistSession()
m.appendSystem("dropped " + arg + " from the kept paths")
}
Loading
Loading