[yusiwen]

What changed

• agent/testdata/memory-bench.json (new): 24 fixed facts and 20 queries in three groups — 10 that
  share a distinctive word with their fact, 6 English paraphrases that share none, 4 Chinese queries
  against English facts.
• agent/memory_bench_test.go (new): the harness. Ranks at depth 10, reports hit@1/3/5 and MRR per
  group, and asserts what can be asserted deterministically — corpus hygiene (a paraphrase query must
  share no token with its target, or the lexical ranker could answer it), the local ranker's top-3
  floor, the metric arithmetic, and the counts the page quotes.
• TestMemoryBenchRealEndpoint: gated on TINYCODE_MEMORY_BENCH_URL / _MODEL; embeds the corpus
  against a real endpoint, refuses to report if any fact failed to embed, and prints the same table for
  the semantic and lexical rankers.
• make memory-bench with MEMORY_BENCH_* pass-through, documented in the target's comment.
• docs/memory-bench.md (new): the corpus, the columns, the deterministic baseline, how to run, how to
  read (compare within a model; the cross-lingual column is a model property), and the explicit note
  that no semantic number is recorded yet.
• Links: README's memory bullet and roadmap B3's Delivered note point at the instrument.
• Test count 1224 → 1228 in the two places that state it.

Why

B3's PR said, in its own honest-scope section, that recall quality was unmeasured. "Semantic beats
lexical" is a claim; the next decision in this area — consolidation, quantisation, a cross-encoder, a
different model — should be made against a number. The benchmark is also the reason the claim stays
honest: the page carries the lexical baseline (deterministic, reproduced by CI) and an empty cell where
a model's numbers would go, rather than a sentence that implies one.

Verification

• go test ./... -count=1 and -race: 16 packages ok; go vet and gofmt -l clean.
• Baseline printed by the harness (deterministic, so exact):
  MEMBENCH ranker=lexical kind=lexical n=10 hit1=1.00 hit3=1.00 hit5=1.00 mrr=1.00,
  kind=paraphrase n=6 hit1=0.00 …, kind=crosslingual n=4 hit1=0.00 ….
• Mutation check: the paraphrase query how do I cut a version changed to how do I cut a release version → TestMemoryBenchCorpusIsWellFormed fails with shares [release]; restored, it passes.
• make memory-bench with no endpoint → --- SKIP: TestMemoryBenchRealEndpoint and exit 0, so an
  ordinary run never needs a model.
• git log --format='%h %G? %s' → G.

Honest scope

• The semantic column is not measured. No embedding endpoint was configured here; the doc says so
  in the section where the numbers would go.
• The corpus is one person's idea of what a repository's memory looks like, 24 facts deep. It is a
  regression instrument, not a model evaluation.
• The gated run needs a human with a model; CI only proves the harness, the hygiene rule and the
  lexical floor.

Follow-ups

• Run it against a real model and record the numbers (the doc has the command and the table shape).
• [memory] Embedding-backed recall, with a rerank stage and a lexical floor #297 S2 stays open: consolidation, quantisation, and the file-level consumer (B4).
• [test] TestCancelledRunMakesNoFurtherRequest fails under the repeated run #295 remains open and did not reproduce.

Refs #297, #168