EMR: optional local embeddings for semantic recall

[warheart1984-ctrl]

Replaces #9, which GitHub closed when #8's branch was deleted on merge. Same branch and content, plus a commit restoring app/emr.py's CRLF line endings (the original commit had rewritten it as LF, so its diff showed every line). Against main: 8 files, +351/−5.

What

• app/emr_embed.py, an optional semantic signal for EMR query alignment. A small local model embeds the query and each memory, and Q = max(Q_lex, Q_sem).
  • Model: fastembed with BAAI/bge-small-en-v1.5. It runs as ONNX on the CPU with no torch, takes 65 MB on disk, and embeds about 13 ms per sentence on a 4-core box.
  • Scaling: the cosine maps onto Q's scale between SIM_FLOOR = 0.65 (no signal) and SIM_FULL = 0.85 (full alignment).
  • Caching: memory vectors are cached by content hash and model in JARVIS_EMR_EMBED_CACHE, so each memory is embedded once.
  • Auditing: ActivationBreakdown gains Q_lex and Q_sem, so any recall can be audited.
• Off by default. Turn it on with pip install ".[embed]" and JARVIS_EMR_EMBEDDINGS=1. If the model can't load (missing extra, offline first run, bad name), EMR stays lexical and recall doesn't fail.
• python -m app.emr_bench --embeddings scores the benchmark with embeddings on.

Results (same 130-case benchmark as #8)

	Before #8	#8 (lexical)	This PR
Top-5 recall, answerable	0.32	0.88	0.95
Top-1	0.25	0.67	0.74
MRR	0.28	0.77	0.82
Natural questions	0.29	0.96	1.00
Paraphrases	0.05	0.45	0.70
Memory over 1 month old	0.00	0.87	0.97
Unrelated question answered	0.40	0.00	0.00
Near-miss returned memories	0.60	1.00	1.00

Calibration. I picked the floor and ceiling on the odd-numbered half of the cases and checked them on the even half. The dev-best setting (0.60–0.85) dropped on the held-out half: paraphrases fell from 1.00 to 0.60, with only 10 paraphrases per half. 0.65–0.85 was nearly tied on dev and held up on held-out (0.94 top-5, 0.76 top-1), so that's what ships. Disclosure: I looked at the held-out half for four configurations, so it's not perfectly clean.

Near-misses, measured end to end. Embeddings don't separate near-misses: "sister's birthday" scores 0.79 against "brother's birthday", higher than a true paraphrase. So I measured what matters, the answer. All 10 near-miss questions went through recall plus groq/gpt-oss-20b via llm-gateway, with the recalled memories as context. 9 of 10 answered correctly ("not on record"). The one miss divided the Jarvis tenant's daily budget by 24 for "spend per hour on OpenAI", ignoring "OpenAI". The paraphrases tried (truck, coffee, pet) and the original failing question ("Where does llm-gateway run, and on which port?" now returns 127.0.0.1:8080 with a citation) were all correct. The retrieval-level near-miss number overstates the harm: returning "Jon has a dog named Bruno" for "what's the cat's name?" is the context that lets the model say there's no cat on record.

Tests

• Fake-model tests: tests/test_emr_embed.py uses a fake model, so there's no download. It covers: off by default, an unloadable model falls back to lexical, the cosine mapping, a paraphrase with no shared words being recalled, Q as the max of the two signals, and the vector cache (per content hash, and invalidated by a model change).
• Embedding benchmark floors: tests/test_emr_bench.py holds the numbers above as a regression floor, and skips without the extra.
• Locally, merged onto current main (AMUL LLM: llm-gateway backend, API keys, and HTTP routes #7 + EMR: labelled benchmark, lexical retrieval, and an age-free evidence gate #8): 254 passed with the extra installed (3.12). On Python 3.11 with .[dev] only (as CI runs it), 252 passed and 2 skipped.
• Isolation: conftest keeps embeddings off and the vector cache in tmp for every test.

For the reviewer

• First-run download. The model downloads from Hugging Face on first use, into JARVIS_EMR_EMBED_MODEL_DIR (default data/emr-embed-model/, now git-ignored). After that it runs offline. The Docker image doesn't install the extra.
• Synthetic benchmark. As in EMR: labelled benchmark, lexical retrieval, and an age-free evidence gate #8, the benchmark is hand-written. A labelled slice of a real ledger is the next check worth doing.

🤖 Generated with Claude Code

[greptile-apps]

Your trial has ended. Reactivate Greptile to resume code reviews.