roadmap.md

Roadmap

Every phase of the work, one row each, with its status. Phases 0 to 5 were the first roadmap, which reached the basic memory layer on 2026-10-12. This file changes when a phase opens or closes, and not in between. The detail of the open phase lives in plan.md. The design and its reasons live in brief.md.

Rules. One phase is open at a time. A phase closes when every issue it opened is closed, or is listed under Deferred below with a reason. The next phase opens only then. Each phase is one pull request in each repository that it touches, and ends with the verification layer that proves it.

The first roadmap: the basic memory layer

Phase	Delivers	Status
0. Documents and measurements	The public contracts and guarantees files, CONTRIBUTING, the Makefile with the first documentation check, the local container setup, and measurements M1 to M6.	closed 2026-10-05, PR #2
1. The storage contract, offline	The record, the Store protocol, the in-memory driver, the conformance suite, the frozen surfaces, the ledger test, and CI.	closed 2026-10-06, PR #5
2. The Hotdata driver	HotdataStore with four tables, the serialized writer, the three-stage retrieval query, the sweeper, the integration leg against the local stack in CI, and the oracle test.	closed 2026-10-08, PR #7
3. The memory contract	Memory with remember, recall, candidates, supersede, forget, profile, and capture, the rendered blocks pinned by tests, and the skill file with its command check.	closed 2026-10-08, PR #9
4. The first consumer	A document loader in hotmemory. A post-mortem loader and a recall at investigation start in the consumer's own repository, proven by a replay case. Two pull requests, one in each repository.	closed 2026-10-10, PR #11 and the consumer's pull request
5. A public benchmark	A LongMemEval-S runner measuring retrieval recall at k first and end-to-end accuracy second, with the numbers in docs/benchmarks.md.	closed 2026-10-12, PR #14

The second workstream: memory accuracy

Agreed on 2026-10-13. plan.md holds the tasks, the acceptance criteria, and the stop conditions of each phase. The library stays deterministic: no code in src/hotmemory calls a model. Latency is not a goal, and its work waits under Deferred.

Phase	Delivers	Status
6. The accuracy loop	A second sample of 54 questions, and the memory arm configured from the command line. The diagnosis split for each question type, and the baselines of both samples.	closed 2026-10-14, PR #18
7. The extractor	A benchmark extractor that keeps what the assistant said, with sources, measured against the 19 misses where no stored fact held the answer. No library change.	open, issue #19
8. Episodes behind facts	capture stores its text as an episode that its facts cite. recall can add the cited or ranked episodes to its block, within one budget.	proposed
9. The ranking	An ablation of the fused ranking, and a default ranking chosen by the benchmark.	proposed
10. Supersede in capture	A fact can name the current record that it replaces, and remember closes that record. The extractor decides, and the library applies the rule.	proposed
11. Search over history	recall(as_of=T) returns the revision valid at T, with the as-of filter in SQL, and a date window from the caller.	proposed
12. The full benchmark	The final configuration on all 500 LongMemEval-S questions, and a LoCoMo smoke test.	proposed
13. Release on PyPI	The release flow of the sibling repository hotdata-langchain, and the first release.	proposed

Deferred

An entry here names the issue or the phase it came from, and the reason it is not being taken up now.

• Adapters, planned as phase 6 of the first roadmap: a LangGraph BaseStore adapter in hotdata-langchain, then an MCP server. They wait until the memory contract scores well and is on PyPI.
• Latency, from phase 5: native DML for writes, the filtered vector index search, a conditional write for more than one writer, text loaded only for ranked rows, and the time of each cloud request. The second workstream measures accuracy first.
• Model calls, from the discussion of 2026-10-13: a model-backed extractor in an extra, and decision calls with a probability, for example to decide a supersede or to route a recall. The library stays deterministic in the second workstream.
• The ideas of the hindsight study of 2026-10-09 that the second workstream does not take, for later planning:
  • A chunk-hash delta on re-capture.
  • "No time" kept distinct from "omitted".
  • Word and number checks on entity names.
  • A forget that cascades to derived records.
  • A parquet export.

Closed phases

A closed phase keeps its row above with the date it closed and the pull request.

• Phase 0 closed on 2026-10-05 with PR #2, which closed issue #1. Its measurements are in docs/guarantees.md. It changed the plan for later phases in three ways. The local development target is a three-container stack (Postgres, RustFS, and the engine), not a bare container. Fused retrieval needs plain vector indexes and an embedder from the caller, because a provider-backed index excludes every other index. The integration leg can run against the local stack in CI.
• Phase 1 closed on 2026-10-06 with PR #5, which closed issue #4. It changed the plan for later phases in three ways. The record refuses more than the brief said: an empty label, a label with /, a key with / or @, and empty content. Deduplication skips a current revision past forget_after, so a forgotten fact can come back. Two questions moved to phase 3: how supersede sets fields of the old record, and how a fact gains a source when its content does not change.
• Phase 2 closed on 2026-10-08 with PR #7, which closed issue #6. It changed the plan for later phases in four ways. The engine allows one vector index per table, so the cue vectors have their own table. The engine refuses an index on an empty table, so provision builds over a seed row. A filtered search through a vector index misses rows loaded after the build, so the vector rankings filter after their fetch. The defect is reported to the engine team, and it reproduces in production. One process writes to a database until the engine has a conditional write.
• Phase 3 closed on 2026-10-08 with PR #9, which closed issue #8. It changed the plan for later phases in four ways. The vector rankings of HotdataStore filter first and rank by a scan, so the vector indexes serve no query until the engine fixes the filtered index search. At 100,000 rows of 64 dimensions, a search in a 1 percent scope took 97 ms locally. put gained close_previous, and a put of the same content with a new source writes a merged revision. recall reads current revisions only, so a search over history is still open. A count in the block of profile is a lower bound when a sub-namespace fills the window of list.
• Phase 4 closed on 2026-10-10 with PR #11 and the consumer's pull request, which closed issue #10. The consumer loaded 12 post-mortems, and counted 129 episodes and 256 facts with SQL. A second load wrote nothing. It changed the plan for later phases in four ways. The pass rule of the replay case judges the investigation loop from its trace, because the report step drops the date of a rejected hypothesis. A consumer that renders its own block passes budget and k to recall, because the default budget of 2,000 characters cut the list first. A single-part upload in the Hotdata SDK does not retry on a closed connection, so a long load can fail and must be run again. It is reported to the SDK repository. A document that gets fewer chunks keeps its old last episodes current.
• Phase 5 closed on 2026-10-12 with PR #14, which closed issue #13. On LongMemEval-S, an evidence session was in the top 5 for 98.3 percent of the 419 scored questions, with the vector ranking. On a sample of 54 questions, the reader scored 39 with every session, 45 with the top 10 sessions, and 19 with the block of recall. It changed the plan for later phases in four ways. The extractor is the main loss of the memory contract: in 19 of its 35 misses, no stored fact held the answer. On this data the vector ranking scored higher than the fused ranking, and the reason was not measured. With every question in one table, the fused ranking scored higher at the top 5 and slightly lower at the top 10. A BM25 ranking inside the scope can wait. In the cloud, a fused search took 998 ms at p50 and a vector search 361 ms.
• Phase 6 closed on 2026-10-14 with PR #18, which closed issue #16. A replay of sample A from the caches reproduced 19 of 54 and each cause of each miss, at no model cost. On sample B, the reader scored 43 with every session, 48 with the top 10 sessions, and 27 with the block of recall. It changed the plan for later phases in four ways. The memory arm differs by 14.8 points between the two samples with the same configuration, so a change counts only if both samples move the same way. The extractor is the main loss on both samples: no stored fact held the answer in 19 of 35 misses on sample A and 16 of 27 on sample B. When the block held a needed fact, it held every needed fact in only 2 of 25 misses. A fact ranked below 30 caused 2 misses on sample A and none on sample B, so the ranking has little to gain on these samples. The runner gained --replay and --workers, and a replay of the memory arm of sample A took 519 s with 4 workers, against 2,264 s with one process.