diff --git a/CODEBASE.md b/CODEBASE.md index 501fd1c..dd69849 100644 --- a/CODEBASE.md +++ b/CODEBASE.md @@ -1,6 +1,6 @@ # TinyCode — CODEBASE Map -> AI coding agent in pure Go. Single binary, Bubble Tea TUI, ReAct agent loop, 25 built-in tools + MCP, LSP diagnostics, session persistence. 1224 test functions + 10 fuzz targets, race-detector clean. +> AI coding agent in pure Go. Single binary, Bubble Tea TUI, ReAct agent loop, 25 built-in tools + MCP, LSP diagnostics, session persistence. 1228 test functions + 10 fuzz targets, race-detector clean. ## Quick Reference @@ -849,7 +849,7 @@ User Input (textarea / CLI arg) ## Testing -- **1224 test functions + 10 fuzz targets** in the main module — what `go test ./... -count=1` runs, since `tuiprobe/` is a nested module; each count below is listed with the command that produces it, and no count is carried in a copy that nothing checks +- **1228 test functions + 10 fuzz targets** in the main module — what `go test ./... -count=1` runs, since `tuiprobe/` is a nested module; each count below is listed with the command that produces it, and no count is carried in a copy that nothing checks - `make fuzz` (`FUZZTIME=30s`) explores every fuzz target; `go test` already runs their seed corpora, so CI exercises them on every push - `make test-browser` (`BROWSER_TEST=1`, `-run TestBrowserSmoke`) renders a JavaScript page through both browser paths against a loopback server and asserts the request carried the proxy's `Via` header, which proves the filtering proxy was used; `TestBrowserSmokeRefusesABlockedSubresource` then serves a page whose own authority is exempted from the loopback rule while the policy stays enforced, and asserts a subresource pointing at a *second, live* loopback service is refused (the script's `onerror` marker in the DOM) and that the service was never reached — a refusal is proven, not only that the proxy was used (issue #8). Both skip without a browser, so `make test` never launches one - `make test-tui-visual` (`TUI_SHOT=1`) is the only place the TUI is looked at rather than asserted on: it renders the committed frame scenarios to PNGs through headless Chromium, starts `bin/tinycode` on a real 80x24 PTY (asserting the stream carried SGR styling and that the binary quits on the documented double Ctrl+C), runs it once more on a size-less PTY, and replays the live stream into a screen buffer that is screenshotted as `tinycode-pty-welcome-80x24.png`. Both tests skip without the variable, so `make test` needs neither a browser, nor a binary, nor a terminal device; PNGs land in `TUI_SHOT_DIR` (default `/tmp`) @@ -895,7 +895,7 @@ Counts in this document are produced by these commands at the commit they descri ```bash find . -name '*_test.go' -not -path './tuiprobe/*' -print0 \ - | xargs -0 grep -h '^func Test' | wc -l # 1224 test functions (main module) + | xargs -0 grep -h '^func Test' | wc -l # 1228 test functions (main module) grep -rn '^func Fuzz' --include=*_test.go . | wc -l # 10 fuzz targets ls tui/testdata/golden/frames/*.txt | wc -l # 29 plain-text frames ls tui/testdata/golden/ansi/*.ansi | wc -l # 1 raw ANSI frame diff --git a/Makefile b/Makefile index 5fc3892..77d56eb 100644 --- a/Makefile +++ b/Makefile @@ -46,6 +46,22 @@ test-race: test-repeat: go test ./... -count=3 +# The memory recall benchmark (docs/memory-bench.md) measured against a real +# embedding endpoint — Ollama's /api/embed by default, or any OpenAI-compatible +# /embeddings with MEMORY_BENCH_PROVIDER=openai. Without a URL and a model the +# test skips with the reason, like every other gate, so an ordinary `make test` +# never needs a model. +# +# make memory-bench MEMORY_BENCH_URL=http://127.0.0.1:11434 MEMORY_BENCH_MODEL=nomic-embed-text +# +# -v on purpose: the per-group numbers and the per-miss lines are the output. +memory-bench: + TINYCODE_MEMORY_BENCH_URL=$(MEMORY_BENCH_URL) \ + TINYCODE_MEMORY_BENCH_MODEL=$(MEMORY_BENCH_MODEL) \ + TINYCODE_MEMORY_BENCH_PROVIDER=$(MEMORY_BENCH_PROVIDER) \ + TINYCODE_MEMORY_BENCH_API_KEY=$(MEMORY_BENCH_API_KEY) \ + go test -count=1 -v -run TestMemoryBenchRealEndpoint ./agent/ + # Language server used by the gated LSP integration tests. Keep this in step # with the flake's pkgs.gopls (0.23.0). GOPLS_VERSION ?= v0.23.0 diff --git a/README.md b/README.md index cf2a27a..2c5a8f5 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ Custom **CellGrid** frame-buffer renders markdown directly in the terminal — n - **A measurement, before a framework**: `TINYCODE_EVAL=1 go run ./eval/runner` runs six small tasks through three agent configurations, grades each attempt by the task's own `go test`, and reports what the trajectories say — solved tasks, noise, tokens per solved task. Gated and never in CI, because it spends money. See [docs/eval.md](docs/eval.md). - **Every run leaves a trajectory**: one JSONL file per run — steps, per-call durations and TTFT, tokens, cost, and each tool call with a digest of its arguments, its result size and how it ended — written to `~/.tinycode/trajectories` by default, read back with `--trajectories` and `--trajectory `. Metadata only unless you ask for text, bounded at 200 runs / 64 MiB, and not writable by a tool. See [docs/trajectories.md](docs/trajectories.md). - **A goal that continues itself**: `/goal ` stores one objective in the session — phase, round budget, blocker — and when the session goes idle the agent starts another round toward it, up to 256 rounds, with the objective, the round number and the evidence rule in the prompt. A round may complete or block the goal but never rewrite it; Ctrl+C pauses instead of retrying; a resumed or forked session comes back disarmed until you say `/goal resume` (or just "continue"). The build agent is told to create one for work that will not fit a single turn; `/goal` is the explicit path. See [docs/goals.md](docs/goals.md). -- **Memory that outlives the session**: recall is on by default and reads a local store — one file per project under `~/.tinycode/memory`, deliberately outside the project root and refused to the tool layer, because a fact the agent can rewrite is not one. Nothing is stored unless you ask: `/remember