feat(normalizr): Keep results a committed render read

.changeset/memo-commit-pins.md

@@ -0,0 +1,9 @@
+---
+'@data-client/normalizr': patch
+'@data-client/core': patch
+'@data-client/react': patch
+---
+
+Keep results that a committed render read while a transition retries
+
+`DataProvider` now tells the memo which store version committed, so results a committed render read stay cached through any number of pending-transition retries and optimistic updates. Custom `Controller` memos can implement the new optional `commit(entities, settled)` method; ones without it keep working.

examples/benchmark/README.md

@@ -40,6 +40,8 @@ Both arguments are optional:
   - `denormalizeLong withCache update first|last row` - steady state: each op builds a new `entities` with one project replaced, then reads the cached list; the changed row is first or last in the list
   - `denormalizeLong Scalar` (cold) and `denormalizeLong Scalar withCache update one cell` - the 700-row `Scalar` list in a new `MemoCache`, and read again after each op replaces one `Scalar` cell
   - `denormalizeLong two Scalars, 25% Stocks missing` (and `withCache`) - a `Holding` list with two `Scalar` lenses (one through its `Stock`); two stores missing different Stocks are read in turn, so list nodes keep more than one `argsKey` branch
+  - `... withCache committed` - the `denormalizeLong`, `Values`, `Scalar` and `All` withCache cases on a `MemoCache` that has `commit()`ed the `entities` it reads, so reads take the committed path
+  - `denormalizeShort 500 endpoints withCache` (and `then commit`) - each op reads 500 small cached endpoints with a new `entities`, then commits it; the difference is the cost of `commit()`
 - `spread` - degenerate-case writes where spread cost scales with store size (single-entity `setResponse` into 1k/10k/100k entity stores, 10k cached endpoint keys, collection push onto 10k items, invalidateAll/expireAll). One case (`setOneEntity in 10k entity store`) is tracked over time in CI via `.github/workflows/benchmark-spread.yml`; the rest are manual-only.
 - `micro` - isolated microbenchmarks for testing specific operations
 - `old-normalizr` - runs equivalent benchmarks using the normalizr package
@@ -124,9 +126,10 @@ yarn start:memo-tradeoffs [filter]
 
 Measures where caching fewer versions could cost reads or memory: one list
 read against several store snapshots in turn (hit rate and time per read),
-heap the memo keeps beyond the store after a run of updates, and one list read
-under many `argsKey` values (hit rate and heap held). Local-only — not tracked
-in CI.
+heap the memo keeps beyond the store after a run of updates, an optimistic
+streak (the settled store committed, then 1, 3 or 8 unsettled commits, then
+the settled store read again: hit rate and heap held), and one list read under
+many `argsKey` values (hit rate and heap held). Local-only — not tracked in CI.
 
 ### Profiling
 

examples/benchmark/memo-tradeoffs.js

@@ -6,6 +6,9 @@
  *   a different snapshot. Reports how many reads hit and time per read.
  * - retained memory: heap the memo holds beyond the store itself, after a run
  *   of updates. Entity churn (every row replaced) is the worst case.
+ * - optimistic streak: the settled store is committed, then M optimistic
+ *   updates each render and commit unsettled, then the store settles back.
+ *   Reports how often the settled result is still cached, and heap held.
  * - argsKey buckets: one list read under N different `argsKey` values (a
  *   portfolio app switching between N portfolios), all cached, and the heap
  *   the memo holds for them.
@@ -200,6 +203,26 @@ const memoryCases = [
       return store;
     },
   },
+  {
+    // a trade tape: old trades leave the store, so nothing should keep their
+    // cached denormalized rows alive
+    name: 'retained: 100-row trade tape, 3 new trades per flush x 600',
+    run(memo) {
+      let store;
+      let trades = [];
+      for (let flush = 0; flush < 600; flush++) {
+        const fresh = Array.from({ length: 3 }, (_, j) => ({
+          id: flush * 3 + j,
+          name: `trade ${flush * 3 + j}`,
+          value: flush,
+        }));
+        trades = [...fresh, ...trades].slice(0, 100);
+        store = normalize([Row], trades);
+        memo.denormalize([Row], store.result, store.entities);
+      }
+      return store;
+    },
+  },
   ...[1000, 10000].map(n => ({
     name: `retained: ${n}-row list refetched (every row replaced) x 20`,
     run(memo) {
@@ -226,6 +249,46 @@ for (const { name, run } of memoryCases) {
   });
 }
 
+// --- optimistic streak ---------------------------------------------------------
+
+const STREAKS = 20;
+for (const flipBack of [false, true]) {
+  for (const m of [1, 3, 8]) {
+    const name = `optimistic streak: ${ROWS}-row list, ${m} unsettled commits${flipBack ? ' and a flip back' : ''}, then the settled store x ${STREAKS}`;
+    if (!want(name)) continue;
+    let hits = 0;
+    const bytes = retained(memo => {
+      const read = state =>
+        memo.denormalize(StockSchema, base.result, state.entities, stockArgs)
+          .data;
+      const settled = read(base);
+      memo.commit(base.entities, true);
+      for (let streak = 0; streak < STREAKS; streak++) {
+        // each optimistic update renders and commits a store with one more
+        // row changed; then they all settle and the store is `base` again
+        const states = snapshots(m + 1).slice(1);
+        // a pending transition rendered between the first two commits; reading
+        // it back moves the flip-back pin off `base`, leaving only the settled pin
+        const pending = flipBack && snapshots(2)[1];
+        if (pending) read(pending);
+        states.forEach((state, i) => {
+          read(state);
+          memo.commit(state.entities, false);
+          if (pending && i === 0) read(pending);
+        });
+        if (read(base) === settled) hits++;
+        memo.commit(base.entities, true);
+      }
+      return base;
+    });
+    rows.push({
+      scenario: name,
+      'hit rate': `${((hits / STREAKS) * 100).toFixed(0)}%`,
+      'memo retains': `${(bytes / 1024 / 1024).toFixed(2)} MB`,
+    });
+  }
+}
+
 // --- argsKey buckets ------------------------------------------------------------
 
 for (const n of [1, 10, 100, 1000]) {

examples/benchmark/normalizr.js

@@ -26,6 +26,7 @@ import {
   buildStockData,
   buildStockScalarUpdate,
   PortfolioScalar,
+  BuildTypeDescription,
   HoldingSchema,
   buildHoldingData,
 } from './schemas.js';
@@ -122,6 +123,16 @@ const withMissingStocks = offset => {
 };
 const holdingStores = [withMissingStocks(0), withMissingStocks(2)];
 
+const BuildTypeSchema = { buildType: BuildTypeDescription };
+
+/** A MemoCache that has read `input` with `ents`, then committed `ents`. */
+function committedMemo(schema, input, ents, args) {
+  const memo = new MemoCache();
+  memo.denormalize(schema, input, ents, args);
+  memo.commit(ents);
+  return memo;
+}
+
 export default function addNormlizrSuite(suite, filter) {
   const memo = new MemoCache();
   // prime the cache
@@ -372,6 +383,78 @@ export default function addNormlizrSuite(suite, filter) {
       [],
     );
   });
+  // committed: each MemoCache has committed the `entities` it reads, so reads
+  // take the path a committed render's re-reads take
+  const memoCommitted = committedMemo(ProjectSchema, result, entities, []);
+  add('denormalizeLong withCache committed', () => {
+    return memoCommitted.denormalize(ProjectSchema, result, entities, []);
+  });
+  const memoValuesCommitted = committedMemo(
+    ProjectSchemaValues,
+    valuesState.result,
+    valuesState.entities,
+    [],
+  );
+  add('denormalizeLong Values withCache committed', () => {
+    return memoValuesCommitted.denormalize(
+      ProjectSchemaValues,
+      valuesState.result,
+      valuesState.entities,
+      [],
+    );
+  });
+  const memoStockCommitted = committedMemo(
+    StockSchema,
+    stockState.result,
+    stockState.entities,
+    stockArgs,
+  );
+  add('denormalizeLong Scalar withCache committed', () => {
+    return memoStockCommitted.denormalize(
+      StockSchema,
+      stockState.result,
+      stockState.entities,
+      stockArgs,
+    );
+  });
+  const memoAllCommitted = committedMemo(
+    AllProjects,
+    queryState.result,
+    queryState.entities,
+    [],
+  );
+  add('denormalizeLong All withCache committed', () => {
+    return memoAllCommitted.denormalize(
+      AllProjects,
+      queryState.result,
+      queryState.entities,
+      [],
+    );
+  });
+  // commit() cost: each op reads 500 small endpoints with a new `entities`,
+  // then (in the second case) commits it
+  const buildTypeInputs = Object.keys(entities.BuildTypeDescription)
+    .slice(0, 500)
+    .map(id => ({ buildType: id }));
+  const memoReads = new MemoCache();
+  const memoCommits = new MemoCache();
+  for (const input of buildTypeInputs) {
+    memoReads.denormalize(BuildTypeSchema, input, entities, []);
+    memoCommits.denormalize(BuildTypeSchema, input, entities, []);
+  }
+  add('denormalizeShort 500 endpoints withCache', () => {
+    const next = { ...entities };
+    for (let i = 0; i < 500; ++i) {
+      memoReads.denormalize(BuildTypeSchema, buildTypeInputs[i], next, []);
+    }
+  });
+  add('denormalizeShort 500 endpoints withCache then commit', () => {
+    const next = { ...entities };
+    for (let i = 0; i < 500; ++i) {
+      memoCommits.denormalize(BuildTypeSchema, buildTypeInputs[i], next, []);
+    }
+    memoCommits.commit(next);
+  });
   add('denormalizeLongAndShort withEntityCacheOnly', () => {
     memo.endpoints = new WeakDependencyMap();
     memo.denormalize(ProjectSchema, result, entities);

packages/core/src/controller/Controller.ts

@@ -49,12 +49,18 @@ import type { ActionTypes, State } from '../types.js';
 export type GenericDispatch = (value: any) => Promise<void>;
 export type DataClientDispatch = (value: ActionTypes) => Promise<void>;
 
+/** Optional: lets the memo keep results from committed stores (see `MemoCache.commit`). */
+interface MemoCommit {
+  commit?(entities: object, settled?: boolean): void;
+}
+
 export interface ControllerConstructorProps<
   D extends GenericDispatch = DataClientDispatch,
 > {
   dispatch?: D;
   getState?: () => State<unknown>;
-  memo?: Pick<MemoCache, 'denormalize' | 'query' | 'buildQueryKey'>;
+  memo?: Pick<MemoCache, 'denormalize' | 'query' | 'buildQueryKey'> &
+    MemoCommit;
   gcPolicy?: GCInterface;
 }
 
@@ -98,7 +104,8 @@ export default class Controller<
   declare readonly memo: Pick<
     MemoCache,
     'denormalize' | 'query' | 'buildQueryKey'
-  >;
+  > &
+    MemoCommit;
 
   /**
    * Handles garbage collection

packages/normalizr/AGENTS.md

@@ -19,7 +19,7 @@ Every change to `src/memo/`, or to a schema's `denormalize`, must keep these. Te
 2. **Complete dependencies.** Every entity and `argsKey` a computation reads is recorded, including deps replayed from a nested hit, and including entities that are missing, `null` or `INVALID`.
 3. **Deterministic.** The same reads lead to the same further reads and an equal result. All args-dependence goes through a pure, referentially stable `argsKey` fn.
 4. **Complete journey.** The paths returned with a hit cover every entity the value depends on; subscriptions and GC count refs from them. Callers never mutate it.
-5. **Weak, bounded retention.** A value lives only while its first dependency is reachable, with at most `MAX_ENTRIES` values per first dependency and `argsKey` results.
+5. **Weak, bounded retention.** A value lives only while its first dependency is reachable, with at most the two most recently used values, plus three pinned ones, per first dependency and `argsKey` results.
 6. **Stable identity.** The same reads return the same object while it is cached.
 7. **One `WeakMap` level.** Never nest a `WeakMap` per dependency. Hermes resolves `WeakMap` entries one nesting level per stop-the-world marking pass, and each pass rescans every entry in the app, so N nested levels cost N passes (multi-second freezes). `yarn workspace example-benchmark start:ephemerons` measures it.
 
@@ -32,7 +32,9 @@ Accepted limitations (intentional; don't "fix"):
 
 - A render that applies a pending store update rebuilds every entity that update touches (`useReducer` has no eager state). Those reads are one-offs.
 - Within one committed store, a reader re-reads at most one version unchanged: renders read the committed `optimisticState`, managers and callbacks read `controller.getState()`. Every commit starts a new version. Rolling back an optimistic update returns to the older real entities.
-- So `WeakDependencyMap` keeps the most recently used entries plus the one the store last flipped back to. Bound: `useSuspense` serves urgent re-renders of an unchanged store from its own `useMemo`, so during a retrying transition a newly committed version can be recomputed once.
+- So `WeakDependencyMap` keeps the two most recently used entries, and never evicts three more: the one the store last flipped back to, the one a committed render last read, and the one a committed render of a store with no optimistic updates last read. Each optimistic update rebuilds the entities it touches on every store change, so once the last of any number in flight settles, that last one is the only earlier result the store can return to.
+- Commits come from `MemoCache.commit(entities, settled)`, which `DataStore` calls from an effect (effects never run for discarded renders). Reads are buffered per `entities` (one per node) until it commits; reads with the latest committed `entities` are kept right away, so late mounts are covered. Recency alone can't find the committed result: `useSuspense` serves urgent re-renders of an unchanged store from its own `useMemo`, so those reads never reach the memo.
+- Pins go on the result's own entry: in the endpoints tier, or in the top-level entity's map when the input is a pk (single-entity endpoints, entity and Collection queries). Only those `get`/`set` calls pass `track`, and `MemoCache` reads and clears the record right after. Nested entity maps never track: they live as long as the entity cache, so a recorded entry nobody clears would never be freed.
 - `yarn workspace example-benchmark start:memo-tradeoffs` measures hit rates and retained memory for these choices.
 
 ## Schema-tree traversal pattern