diff --git a/.changeset/memo-shallow-weakmaps.md b/.changeset/memo-shallow-weakmaps.md new file mode 100644 index 000000000000..798d4525040c --- /dev/null +++ b/.changeset/memo-shallow-weakmaps.md @@ -0,0 +1,10 @@ +--- +'@data-client/normalizr': patch +'@data-client/core': patch +'@data-client/react': patch +'@data-client/vue': patch +--- + +Fix multi-second freezes on React Native (Hermes) when large results read streaming entities + +On Hermes, an app that kept reading a large result while its entities updated could freeze for 0.5–3s at a time, and the freezes grew the longer it ran. An example is `useSuspense()` on a few thousand entities that join live prices. Each freeze was a garbage collection pause caused by memoization. Cached reads no longer slow down Hermes's garbage collector, and they are faster on every platform. diff --git a/examples/benchmark/README.md b/examples/benchmark/README.md index 91bedfca566f..d10b1067183d 100644 --- a/examples/benchmark/README.md +++ b/examples/benchmark/README.md @@ -37,6 +37,9 @@ Both arguments are optional: - `core` - benchmarks entire operations using [Controller](https://dataclient.io/docs/api/Controller) - `normalizr` - benchmarks just normalize/denormalize - `denormalizeLong nested 25% missing|null|INVALID` (and `withCache`) - the project list with every 4th nested `BuildTypeDescription` missing from the store, `null`, or `INVALID`; those are still tracked as memo dependencies + - `denormalizeLong withCache update first|last row` - steady state: each op builds a new `entities` with one project replaced, then reads the cached list; the changed row is first or last in the list + - `denormalizeLong Scalar` (cold) and `denormalizeLong Scalar withCache update one cell` - the 700-row `Scalar` list in a new `MemoCache`, and read again after each op replaces one `Scalar` cell + - `denormalizeLong two Scalars, 25% Stocks missing` (and `withCache`) - a `Holding` list with two `Scalar` lenses (one through its `Stock`); two stores missing different Stocks are read in turn, so list nodes keep more than one `argsKey` branch - `spread` - degenerate-case writes where spread cost scales with store size (single-entity `setResponse` into 1k/10k/100k entity stores, 10k cached endpoint keys, collection push onto 10k items, invalidateAll/expireAll). One case (`setOneEntity in 10k entity store`) is tracked over time in CI via `.github/workflows/benchmark-spread.yml`; the rest are manual-only. - `micro` - isolated microbenchmarks for testing specific operations - `old-normalizr` - runs equivalent benchmarks using the normalizr package @@ -82,6 +85,49 @@ counts (minor/major/other) and total GC pause during the loop, and retained heap after a full GC (should be ~0; nonzero indicates actual retention). Local-only — not tracked in CI. +### WeakMap (ephemeron) pressure + +Memoization keys cached values on entities through `WeakMap`s. How much a +collection costs for those depends on the engine, so measure the worst case +with: + +```bash +yarn start:ephemerons [filter] +``` + +Each scenario takes a V8 heap snapshot and replays Hermes's marking loop on +it. It reports the live `WeakMap` entries, how many marking passes Hermes +would need, and their product, which is the number of entry visits in one +Hermes old-generation pause. It also reports V8's own full-GC time. The +scenarios mirror a live market screen: a 3,736-symbol endpoint whose symbols +join streaming tickers, and a 100-row trade tape that prepends prints. A +50-level Department/Building chain whose deepest Building changes every tick +measures passes against nesting depth. +Local-only — not tracked in CI. + +How engines resolve `WeakMap` entries during a major collection: + +| Engine | Algorithm | Where it runs | Cost for N nested entries | +| --- | --- | --- | --- | +| Hermes (React Native) | Repeats a scan of every `WeakMap` entry slot in the heap until a pass marks nothing new (`HadesGC::markWeakMapEntrySlots`). No key → value index. | Stop-the-world, on the JS thread, at the end of every old-gen collection | passes ≈ N + 1, each visiting every slot: quadratic | +| V8 (Node, Chrome) | The same fixpoint for at most `--ephemeron-fixpoint-iterations` (default 10) passes, then a linear algorithm over a key → values map (`MarkTransitiveClosureLinear` in `mark-compact.cc`) | Major GC; most marking is concurrent | linear | + +So a structure that V8 collects in milliseconds can stall Hermes for seconds: +one pass per level of nesting, and every pass rescans every `WeakMap` entry in +the app, not only the memo's own. Keep `WeakMap` nesting shallow. + +### Memo cache worst cases + +```bash +yarn start:memo-tradeoffs [filter] +``` + +Measures where caching fewer versions could cost reads or memory: one list +read against several store snapshots in turn (hit rate and time per read), +heap the memo keeps beyond the store after a run of updates, and one list read +under many `argsKey` values (hit rate and heap held). Local-only — not tracked +in CI. + ### Profiling For opt/deopt investigation: diff --git a/examples/benchmark/ephemerons.js b/examples/benchmark/ephemerons.js new file mode 100644 index 000000000000..e82b0fae978a --- /dev/null +++ b/examples/benchmark/ephemerons.js @@ -0,0 +1,341 @@ +/** + * WeakMap (ephemeron) pressure from memoized denormalization. + * + * Hermes resolves WeakMap entries in the stop-the-world end of every + * old-generation collection (`HadesGC::markWeakMapEntrySlots`): it rescans + * every WeakMap entry slot in the heap, repeating until a pass marks nothing + * new. A value reachable only through N nested WeakMap entries needs N + * passes, so one collection costs passes x slots entry visits, on the JS + * thread. + * + * V8 (Node, Chrome) runs the same fixpoint for at most + * `--ephemeron-fixpoint-iterations` (10) passes, then switches to a linear + * key -> values algorithm, so its pause stays small. To get the Hermes cost + * on any engine, each scenario takes a V8 heap snapshot and replays Hermes's + * marking loop on it. + * + * Each scenario mirrors a live screen: a large endpoint whose entities join + * frequently written entities, or a short list that streams new rows. + * + * Local-only; not part of CI benchmark tracking. Run with: + * yarn workspace example-benchmark start:ephemerons [filter] + */ +import { readFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import vm from 'node:vm'; +import v8 from 'v8'; + +v8.setFlagsFromString('--expose_gc'); +const gc = vm.runInNewContext('gc'); + +// Count what the library creates. Installed before the bundle loads so its +// `new WeakMap()` calls see it. +const NativeWeakMap = WeakMap; +let created = 0; +let entriesSet = 0; +class CountingWeakMap extends NativeWeakMap { + constructor(entries) { + super(entries); + created++; + } + + set(key, value) { + entriesSet++; + return super.set(key, value); + } +} +globalThis.WeakMap = CountingWeakMap; + +const { Entity, normalize, MemoCache } = await import('./dist/index.js'); +// after the bundle, so a static import doesn't load it before CountingWeakMap +const { Department, buildBidirectionalChain } = await import('./schemas.js'); + +const EPHEMERON = + /part of key \(.*? @(\d+)\) -> value \(.*? @(\d+)\) pair in WeakMap \(table @(\d+)\)/; + +/** + * Replays Hermes's old-gen marking on a V8 heap snapshot of the live heap. + * + * Marks everything strongly reachable from the roots, then, like + * `markWeakMapEntrySlots`, scans every WeakMap entry and marks values whose + * map and key are marked, draining strong references after each pass, until + * a pass marks nothing. Returns the entry count, passes, and entry visits. + */ +function replayHermesMarking() { + const snap = JSON.parse(readSnapshot()); + const { node_fields, edge_fields, edge_types } = snap.snapshot.meta; + const NF = node_fields.length; + const EF = edge_fields.length; + const { nodes, edges, strings } = snap; + const nodeCount = nodes.length / NF; + const edgeCountField = node_fields.indexOf('edge_count'); + const idField = node_fields.indexOf('id'); + const toField = edge_fields.indexOf('to_node'); + const nameField = edge_fields.indexOf('name_or_index'); + const types = edge_types[0]; + const weak = types.indexOf('weak'); + const named = new Set( + ['context', 'property', 'internal', 'shortcut'].map(t => types.indexOf(t)), + ); + + const firstEdge = new Uint32Array(nodeCount + 1); + const indexById = new Map(); + for (let i = 0, e = 0; i < nodeCount; i++) { + firstEdge[i] = e; + e += nodes[i * NF + edgeCountField] * EF; + indexById.set(nodes[i * NF + idField], i); + } + firstEdge[nodeCount] = edges.length; + + // Ephemeron pairs show up as labelled edges; they are not strong. + const ephemeronEdge = new Uint8Array(edges.length / EF); + const entries = new Map(); + for (let e = 0; e < edges.length; e += EF) { + const type = edges[e]; + if (!named.has(type)) continue; + const m = EPHEMERON.exec(strings[edges[e + nameField]]); + if (!m) continue; + ephemeronEdge[e / EF] = 1; + entries.set(`${m[3]}:${m[1]}`, [ + indexById.get(+m[3]), + indexById.get(+m[1]), + indexById.get(+m[2]), + ]); + } + const slots = [...entries.values()].filter(s => + s.every(i => i !== undefined), + ); + + const marked = new Uint8Array(nodeCount); + const stack = []; + const drain = () => { + while (stack.length) { + const n = stack.pop(); + for (let e = firstEdge[n]; e < firstEdge[n + 1]; e += EF) { + if (edges[e] === weak || ephemeronEdge[e / EF]) continue; + const to = edges[e + toField] / NF; + if (!marked[to]) { + marked[to] = 1; + stack.push(to); + } + } + } + }; + marked[0] = 1; + stack.push(0); + drain(); + let passes = 0; + let newlyMarked; + do { + passes++; + newlyMarked = false; + for (const [table, key, value] of slots) { + if (marked[table] && marked[key] && !marked[value]) { + marked[value] = 1; + stack.push(value); + newlyMarked = true; + } + } + drain(); + } while (newlyMarked); + return { slots: slots.length, passes, visits: passes * slots.length }; +} + +function readSnapshot() { + const file = v8.writeHeapSnapshot( + join(tmpdir(), `ephemerons-${process.pid}.heapsnapshot`), + ); + try { + return readFileSync(file, 'utf8'); + } finally { + rmSync(file); + } +} + +// --- schemas shaped like native-binance's market data ---------------------- + +class Ticker extends Entity { + symbol = ''; + last = 0; + eventTime = 0; + pk() { + return this.symbol; + } + + static key = 'Ticker'; +} + +class MarketSymbol extends Entity { + symbol = ''; + baseAsset = ''; + quoteAsset = ''; + ticker = undefined; + pk() { + return this.symbol; + } + + static key = 'Symbol'; + static schema = { ticker: Ticker }; + static process(input) { + return { ...input, ticker: input.symbol }; + } +} + +class PlainSymbol extends Entity { + symbol = ''; + baseAsset = ''; + quoteAsset = ''; + pk() { + return this.symbol; + } + + static key = 'PlainSymbol'; +} + +class Trade extends Entity { + a = 0; + p = 0; + pk() { + return `${this.a}`; + } + + static key = 'Trade'; +} + +const SYMBOLS = 3736; +const TICKERS_PER_TICK = 500; +const TICKS = 60; +const TAPE = 100; +const PRINTS_PER_FLUSH = 3; +const FLUSHES = 600; + +const symbolRows = Array.from({ length: SYMBOLS }, (_, i) => ({ + symbol: `S${i}USDT`, + baseAsset: `S${i}`, + quoteAsset: 'USDT', +})); +const tickerRows = (tick, count) => + Array.from({ length: count }, (_, i) => ({ + symbol: `S${(tick * 7919 + i * 31) % SYMBOLS}USDT`, + last: tick + i, + eventTime: tick, + })); + +/** One reader of a 3,736-symbol endpoint while tickers stream (one write per tick). */ +function exchangeInfo(symbolSchema, streaming) { + const memo = new MemoCache(); + const schema = { symbols: [symbolSchema] }; + let store = normalize(schema, { symbols: symbolRows }); + const input = store.result; + store = normalize([Ticker], tickerRows(0, SYMBOLS), [], store); + for (let tick = 1; tick <= TICKS; tick++) { + if (streaming) { + store = normalize( + [Ticker], + tickerRows(tick, TICKERS_PER_TICK), + [], + store, + ); + } + memo.denormalize(schema, input, store.entities); + } + // The store keeps the endpoint's normalized result alive, and the memo is keyed by it. + return { memo, store, input }; +} + +/** A 100-row tape that prepends prints on each flush. */ +function tradeTape() { + const memo = new MemoCache(); + let store = normalize([Trade], []); + let rows = []; + let next = 0; + for (let flush = 0; flush < FLUSHES; flush++) { + const prints = []; + for (let i = 0; i < PRINTS_PER_FLUSH; i++) + prints.unshift({ a: next, p: next++ }); + rows = [...prints, ...rows].slice(0, TAPE); + store = normalize([Trade], rows, [], store); + memo.denormalize([Trade], store.result, store.entities); + } + return { memo, store }; +} + +const CHAIN = 50; + +/** A Department -> Building -> Department chain `CHAIN` Departments deep whose + * deepest Building is replaced on every tick, so every level recomputes. */ +function deepChain() { + const memo = new MemoCache(); + const { entities: first, result } = buildBidirectionalChain(CHAIN); + let entities = first; + const leaf = `bldg-${CHAIN - 1}`; + for (let tick = 1; tick <= TICKS; tick++) { + entities = { + ...entities, + Building: { + ...entities.Building, + [leaf]: { ...entities.Building[leaf], name: `Building ${tick}` }, + }, + }; + memo.denormalize(Department, result, entities); + } + return { memo, entities, result }; +} + +const scenarios = [ + { + name: `exchangeInfo: ${SYMBOLS} symbols join tickers, ${TICKERS_PER_TICK} tickers x ${TICKS} ticks`, + fn: () => exchangeInfo(MarketSymbol, true), + }, + { + name: `exchangeInfo: ${SYMBOLS} symbols, no join, ${TICKS} reads`, + fn: () => exchangeInfo(PlainSymbol, false), + }, + { + name: `trade tape: ${TAPE} rows, ${PRINTS_PER_FLUSH} prints x ${FLUSHES} flushes`, + fn: tradeTape, + }, + { + name: `deep chain: ${CHAIN} Departments deep, deepest Building replaced x ${TICKS} ticks`, + fn: deepChain, + }, +].filter(s => !process.argv[2] || s.name.includes(process.argv[2])); + +const rows = []; +const baseline = replayHermesMarking(); +rows.push({ + scenario: 'baseline (Node and the bundle, nothing cached)', + 'WeakMaps made': created, + 'entries set': entriesSet, + 'live entries': baseline.slots, + passes: baseline.passes, + 'Hermes visits/GC': baseline.visits, +}); +for (const { name, fn } of scenarios) { + gc(); + created = 0; + entriesSet = 0; + const start = performance.now(); + globalThis.kept = fn(); + const ms = performance.now() - start; + // One full collection with the run's garbage still in the heap. + const gcStart = performance.now(); + gc(); + const gcMs = performance.now() - gcStart; + const { slots, passes, visits } = replayHermesMarking(); + rows.push({ + scenario: name, + 'WeakMaps made': created, + 'entries set': entriesSet, + 'live entries': slots, + passes, + // Live slots only. Hermes also scans slots whose keys died since its + // last old-gen collection, up to passes x 'entries set'. + 'Hermes visits/GC': visits, + time: `${ms.toFixed(0)} ms`, + 'V8 full GC': `${gcMs.toFixed(1)} ms`, + }); + globalThis.kept = undefined; +} +console.table(rows); diff --git a/examples/benchmark/memo-tradeoffs.js b/examples/benchmark/memo-tradeoffs.js new file mode 100644 index 000000000000..75f47193bb5a --- /dev/null +++ b/examples/benchmark/memo-tradeoffs.js @@ -0,0 +1,276 @@ +/** + * Worst cases for the memo cache: where caching less could cost reads or memory. + * + * - store flips: one result read against K store snapshots in turn, as when + * overlapping optimistic updates or several pending transitions each render + * a different snapshot. Reports how many reads hit and time per read. + * - retained memory: heap the memo holds beyond the store itself, after a run + * of updates. Entity churn (every row replaced) is the worst case. + * - argsKey buckets: one list read under N different `argsKey` values (a + * portfolio app switching between N portfolios), all cached, and the heap + * the memo holds for them. + * + * Local-only; not part of CI benchmark tracking. Run with: + * yarn workspace example-benchmark start:memo-tradeoffs [filter] + */ +import vm from 'node:vm'; +import v8 from 'v8'; + +import { Entity, normalize, initialState, MemoCache } from './dist/index.js'; +import { StockSchema, buildStockData } from './schemas.js'; + +v8.setFlagsFromString('--expose_gc'); +const gc = vm.runInNewContext('gc'); +const filter = process.argv[2]; + +function fullGC() { + gc(); + gc(); +} +const heapUsed = () => v8.getHeapStatistics().used_heap_size; + +/** µs per call, after warmup, over at least `ms` of wall time. */ +function timeit(fn, ms = 400) { + for (let i = 0; i < 20; i++) fn(i); + let n = 0; + const start = performance.now(); + let now = start; + while (now - start < ms) { + for (let j = 0; j < 20; j++) fn(n++); + now = performance.now(); + } + return ((now - start) * 1000) / n; +} + +const rows = []; +const want = name => !filter || name.includes(filter); + +// --- store flips ------------------------------------------------------------- + +const ROWS = 700; +const stockArgs = [{ portfolio: 'A' }]; +const meta = (i = 0) => ({ + fetchedAt: 1 + i, + date: 1 + i, + expiresAt: Infinity, +}); +const base = normalize( + StockSchema, + buildStockData(ROWS), + stockArgs, + initialState, + meta(), +); + +/** Snapshot i changes one row's price, like one more optimistic update. */ +function snapshots(k) { + const out = [base]; + for (let i = 1; i < k; i++) { + const prev = out[i - 1]; + const id = `s-${i % ROWS}`; + out.push({ + ...prev, + entities: { + ...prev.entities, + Stock: { + ...prev.entities.Stock, + [id]: { ...prev.entities.Stock[id], name: `rename ${i}` }, + }, + }, + }); + } + return out; +} + +for (const k of [1, 2, 3, 4, 8]) { + const name = `store flips: ${ROWS}-row list read against ${k} snapshots in turn`; + if (!want(name)) continue; + const states = snapshots(k); + const memo = new MemoCache(); + const last = new Array(k); + let hits = 0; + let reads = 0; + const read = i => { + const s = i % k; + const { data } = memo.denormalize( + StockSchema, + base.result, + states[s].entities, + stockArgs, + ); + if (data === last[s]) hits++; + last[s] = data; + reads++; + }; + // fill every snapshot once so later reads can hit + for (let i = 0; i < k; i++) read(i); + hits = 0; + reads = 0; + const us = timeit(read); + rows.push({ + scenario: name, + 'µs/read': us.toFixed(1), + 'hit rate': `${((hits / reads) * 100).toFixed(0)}%`, + }); +} + +// --- retained memory ---------------------------------------------------------- + +class Ticker extends Entity { + symbol = ''; + last = 0; + pk() { + return this.symbol; + } + + static key = 'Ticker'; +} +class MarketSymbol extends Entity { + symbol = ''; + baseAsset = ''; + quoteAsset = ''; + ticker = undefined; + pk() { + return this.symbol; + } + + static key = 'Symbol'; + static schema = { ticker: Ticker }; + static process(input) { + return { ...input, ticker: input.symbol }; + } +} +class Row extends Entity { + id = 0; + name = ''; + value = 0; + pk() { + return `${this.id}`; + } + + static key = 'Row'; +} + +const SYMBOLS = 3736; + +/** + * Heap held by the memo: used heap with memo and store alive, minus used heap + * with only the store alive. `run(memo)` returns the latest store. + */ +function retained(run) { + fullGC(); + let store; + // the memo is only reachable inside this call + const withMemo = (() => { + const memo = new MemoCache(); + store = run(memo); + fullGC(); + return heapUsed(); + })(); + fullGC(); + const storeOnly = heapUsed(); + void store; + return withMemo - storeOnly; +} + +const symbolRows = Array.from({ length: SYMBOLS }, (_, i) => ({ + symbol: `S${i}USDT`, + baseAsset: `S${i}`, + quoteAsset: 'USDT', +})); +const tickerRows = (tick, count) => + Array.from({ length: count }, (_, i) => ({ + symbol: `S${(tick * 7919 + i * 31) % SYMBOLS}USDT`, + last: tick + i, + })); + +const memoryCases = [ + { + name: `retained: ${SYMBOLS} symbols join tickers, 500 tickers x 60 ticks`, + run(memo) { + const schema = { symbols: [MarketSymbol] }; + let store = normalize(schema, { symbols: symbolRows }); + const input = store.result; + store = normalize([Ticker], tickerRows(0, SYMBOLS), [], store); + for (let tick = 1; tick <= 60; tick++) { + store = normalize([Ticker], tickerRows(tick, 500), [], store); + memo.denormalize(schema, input, store.entities); + } + store.input = input; + return store; + }, + }, + ...[1000, 10000].map(n => ({ + name: `retained: ${n}-row list refetched (every row replaced) x 20`, + run(memo) { + let store; + for (let t = 0; t < 20; t++) { + const data = Array.from({ length: n }, (_, id) => ({ + id, + name: `row ${id}`, + value: t, + })); + store = normalize([Row], data, [], store ?? undefined); + memo.denormalize([Row], store.result, store.entities); + } + return store; + }, + })), +]; +for (const { name, run } of memoryCases) { + if (!want(name)) continue; + const bytes = retained(run); + rows.push({ + scenario: name, + 'memo retains': `${(bytes / 1024 / 1024).toFixed(2)} MB`, + }); +} + +// --- argsKey buckets ------------------------------------------------------------ + +for (const n of [1, 10, 100, 1000]) { + const name = `argsKey buckets: ${ROWS}-row Scalar list read under ${n} portfolios in turn`; + if (!want(name)) continue; + const portfolios = Array.from({ length: n }, (_, i) => [ + { portfolio: `p${i}` }, + ]); + let store = base; + for (const args of portfolios) { + store = normalize(StockSchema, buildStockData(ROWS), args, store, meta()); + } + // heap held once every portfolio is cached + const bytes = retained(memo => { + for (const args of portfolios) { + memo.denormalize(StockSchema, store.result, store.entities, args); + } + return store; + }); + const memo = new MemoCache(); + const last = new Array(n); + let hits = 0; + let reads = 0; + const read = i => { + const p = i % n; + const { data } = memo.denormalize( + StockSchema, + store.result, + store.entities, + portfolios[p], + ); + if (data === last[p]) hits++; + last[p] = data; + reads++; + }; + for (let i = 0; i < n; i++) read(i); + hits = 0; + reads = 0; + const us = timeit(read, 800); + rows.push({ + scenario: name, + 'µs/read': us.toFixed(1), + 'hit rate': `${((hits / reads) * 100).toFixed(0)}%`, + 'memo retains': `${(bytes / 1024 / 1024).toFixed(2)} MB`, + }); +} + +console.table(rows); diff --git a/examples/benchmark/normalizr.js b/examples/benchmark/normalizr.js index e283969a27c4..40c638456933 100644 --- a/examples/benchmark/normalizr.js +++ b/examples/benchmark/normalizr.js @@ -25,6 +25,9 @@ import { StockScalarValuesSchema, buildStockData, buildStockScalarUpdate, + PortfolioScalar, + HoldingSchema, + buildHoldingData, } from './schemas.js'; import userData from './user.json' with { type: 'json' }; @@ -91,6 +94,34 @@ const stockUpdatedState = normalize( globalThis.__scalarChurn = () => denormalize(Stock, 's-0', stockState.entities, stockArgs); +/** A new `entities` with one row of `key` replaced by a new object. */ +function replaceRow(entities, key, pk) { + const table = entities[key]; + return { ...entities, [key]: { ...table, [pk]: { ...table[pk] } } }; +} +const projectKey = ProjectWithBuildTypesDescription.key; +const firstProject = result.project[0]; +const lastProject = result.project[result.project.length - 1]; +const stockCell = `${Stock.key}|s-350|${stockArgs[0].portfolio}`; + +// Holdings join Stocks (portfolio lens) and have their own currency lens. Each +// store is missing a different quarter of the Stocks, so the list reaches the +// two lenses in a different order in each. +const holdingArgs = [{ portfolio: 'portfolioA', currency: 'EUR' }]; +const holdingState = normalize( + HoldingSchema, + buildHoldingData(700), + holdingArgs, + initialState, + actionMeta, +); +const withMissingStocks = offset => { + const stocks = { ...holdingState.entities[Stock.key] }; + for (let i = offset; i < 700; i += 4) delete stocks[`s-${i}`]; + return { ...holdingState.entities, [Stock.key]: stocks }; +}; +const holdingStores = [withMissingStocks(0), withMissingStocks(2)]; + export default function addNormlizrSuite(suite, filter) { const memo = new MemoCache(); // prime the cache @@ -159,6 +190,22 @@ export default function addNormlizrSuite(suite, filter) { valuesState.entities, ); }); + add('denormalizeLong Scalar', () => { + return new MemoCache().denormalize( + StockSchema, + stockState.result, + stockState.entities, + stockArgs, + ); + }); + add('denormalizeLong two Scalars, 25% Stocks missing', () => { + return new MemoCache().denormalize( + HoldingSchema, + holdingState.result, + holdingStores[0], + holdingArgs, + ); + }); add('denormalizeLong donotcache', () => { return denormalize(ProjectSchema, result, entities); }); @@ -225,6 +272,23 @@ export default function addNormlizrSuite(suite, filter) { }, }, ); + // steady state: each op is a store update that replaces one project, then + // a read of the cached list + for (const [row, pk] of [ + ['first', firstProject], + ['last', lastProject], + ]) { + const memoUpdate = new MemoCache(); + memoUpdate.denormalize(ProjectSchema, result, entities, []); + add(`denormalizeLong withCache update ${row} row`, () => { + return memoUpdate.denormalize( + ProjectSchema, + result, + replaceRow(entities, projectKey, pk), + [], + ); + }); + } for (const [hole, holeEntities] of nestedHoles) { // one MemoCache each, so the variants don't evict each other's result const memoHoles = new MemoCache(); @@ -257,6 +321,41 @@ export default function addNormlizrSuite(suite, filter) { stockArgs, ); }); + const memoScalarCell = new MemoCache(); + memoScalarCell.denormalize( + StockSchema, + stockState.result, + stockState.entities, + stockArgs, + ); + add('denormalizeLong Scalar withCache update one cell', () => { + return memoScalarCell.denormalize( + StockSchema, + stockState.result, + replaceRow(stockState.entities, PortfolioScalar.key, stockCell), + stockArgs, + ); + }); + // two stores whose lists reach the two lenses in different orders, read in + // turn, so list nodes keep more than one argsKey branch + const memoHolding = new MemoCache(); + let holdingRead = 0; + for (const store of holdingStores) { + memoHolding.denormalize( + HoldingSchema, + holdingState.result, + store, + holdingArgs, + ); + } + add('denormalizeLong two Scalars, 25% Stocks missing withCache', () => { + return memoHolding.denormalize( + HoldingSchema, + holdingState.result, + holdingStores[holdingRead++ & 1], + holdingArgs, + ); + }); add('denormalizeLong All withCache', () => { return memo.denormalize( AllProjects, @@ -299,6 +398,8 @@ export default function addNormlizrSuite(suite, filter) { printStatus(ProjectWithBuildTypesDescription.prototype.pk); printStatus(Entity.merge); printStatus(Entity.validate); + printStatus(WeakDependencyMap.prototype.get); + printStatus(WeakDependencyMap.prototype.set); } }); } diff --git a/examples/benchmark/package.json b/examples/benchmark/package.json index b18f26c56b20..6f8a7f11e4c8 100644 --- a/examples/benchmark/package.json +++ b/examples/benchmark/package.json @@ -14,6 +14,8 @@ "build": "webpack --mode=production --target=node --env readable", "start": "NODE_ENV=production node --allow-natives-syntax ./index.js", "start:memory": "NODE_ENV=production node ./memory.js", + "start:ephemerons": "NODE_ENV=production node ./ephemerons.js", + "start:memo-tradeoffs": "NODE_ENV=production node --max-old-space-size=8192 ./memo-tradeoffs.js", "start:trace": "yarn run start --trace_opt --trace_deopt", "start:deopt": "NODE_ENV=production npx dexnode --out v8.log --redirect-code-traces-to=/tmp/codetrace --allow-natives-syntax ./index.js" }, diff --git a/examples/benchmark/schemas.js b/examples/benchmark/schemas.js index 2ae156083d81..2a19b54a4fc0 100644 --- a/examples/benchmark/schemas.js +++ b/examples/benchmark/schemas.js @@ -179,6 +179,43 @@ export function buildStockScalarUpdate(count = 700) { return { stock }; } +/** A position in a Stock, with its own Scalar on a different lens. A row whose + * Stock is missing skips the portfolio `argsKey`, so rows reach the two lenses + * in different orders. */ +export class Holding extends Entity { + id = ''; + stock = undefined; + fx = 0; + + static key = 'Holding'; + pk() { + return this.id; + } +} +export const CurrencyScalar = new Scalar({ + lens: args => args[0]?.currency, + key: 'currency', + entity: Holding, +}); +Holding.schema = { + stock: Stock, + fx: CurrencyScalar, +}; +export const HoldingSchema = { + holding: [Holding], +}; + +export function buildHoldingData(count = 700) { + const { stock } = buildStockData(count); + return { + holding: stock.map((s, i) => ({ + id: `h-${i}`, + stock: s, + fx: 1 + (i % 50) / 100, + })), + }; +} + // Degenerate bidirectional chain for #3822 stack overflow testing export class Department extends Entity { id = ''; diff --git a/packages/normalizr/AGENTS.md b/packages/normalizr/AGENTS.md index 1bbeacc8e74b..ab7fef03ed76 100644 --- a/packages/normalizr/AGENTS.md +++ b/packages/normalizr/AGENTS.md @@ -5,7 +5,7 @@ Non-obvious constraints when changing `src/memo/`, `src/denormalize/`, or `src/i ## Correctness constraints - **Referential equality is a contract.** Successive `MemoCache.denormalize` calls with identical inputs must return the same object reference. `useSuspense`/`useCache`/`useQuery` skip re-renders on `===`, and many tests assert `toBe`. Probe-and-discard caching schemes (write to a throwaway cache, "real" cache repopulates next call) silently break this. -- **`WeakDependencyMap` entity links are identity-keyed.** Entity deps live on a `WeakMap` keyed by the entity object; `argsKey` deps live on a parallel `Map` keyed by `fn(args)` strings branching off a stable function ref (see `WeakDependencyMap.ts` `next` / `nextStr`). Cache hits on the entity segment depend on every ref in the recorded chain being `===` to the live walk's refs — anything that wraps/proxies refs on read defeats memoization. Variants that introduce new caches (buckets, scopes) must share the same `getEntity` (the `DenormGetEntity` from `IMemoPolicy.getEntities(entities)`) so chain refs stay coherent. +- **`WeakDependencyMap` entries are identity-keyed.** The first dependency is a `WeakMap` key; the rest are stored on the entry and compared with `===` on `get`; `argsKey` results branch on a `Map` keyed by `fn(args)`, one branch per `argsKey` fn (a missing entity can change which fn comes first). Anything that wraps/proxies refs on read defeats memoization. Variants that introduce new caches (buckets, scopes) must share the same `getEntity` (the `DenormGetEntity` from `IMemoPolicy.getEntities(entities)`) so recorded refs stay coherent. - **Both MemoCache tiers must stay coherent** — `endpoints` (top-level) and `_getCache` (per-entity) walk against the same store. Any new caching dimension applies to both, or top-level hits return stale per-entity values. - **Cache lookup precedes traversal.** Pick `WeakDependencyMap` → look up → on miss, traverse and write. You can't decide which map to use *based on* what the traversal observes. Discover-then-lookup designs need either a separate pre-scan or a `setRaw`-style promotion primitive (doesn't exist yet). - **Storage shape is API.** `state.entities[key][pk]` is observed by snapshot tests, SSR payloads, and devtools. Shape changes are breaking even when normalize/denormalize round-trip correctly. @@ -19,14 +19,22 @@ Every change to `src/memo/`, or to a schema's `denormalize`, must keep these. Te 2. **Complete dependencies.** Every entity and `argsKey` a computation reads is recorded, including deps replayed from a nested hit, and including entities that are missing, `null` or `INVALID`. 3. **Deterministic.** The same reads lead to the same further reads and an equal result. All args-dependence goes through a pure, referentially stable `argsKey` fn. 4. **Complete journey.** The paths returned with a hit cover every entity the value depends on; subscriptions and GC count refs from them. Callers never mutate it. -5. **Weak retention.** A cached value lives only while what it is keyed on is reachable. +5. **Weak, bounded retention.** A value lives only while its first dependency is reachable, with at most `MAX_ENTRIES` values per first dependency and `argsKey` results. 6. **Stable identity.** The same reads return the same object while it is cached. +7. **One `WeakMap` level.** Never nest a `WeakMap` per dependency. Hermes resolves `WeakMap` entries one nesting level per stop-the-world marking pass, and each pass rescans every entry in the app, so N nested levels cost N passes (multi-second freezes). `yarn workspace example-benchmark start:ephemerons` measures it. Accepted limitations (intentional; don't "fix"): - The results cache is keyed by the input object, not the schema. Two schemas only see the same result object if two endpoints share a cache key, which is already a bug in the endpoints. - Entities past `MAX_ENTITY_DEPTH` (`src/denormalize/unvisit.ts`) are read but not tracked, so changes that deep don't invalidate ancestors. +### Which versions to keep (React concurrency) + +- A render that applies a pending store update rebuilds every entity that update touches (`useReducer` has no eager state). Those reads are one-offs. +- Within one committed store, a reader re-reads at most one version unchanged: renders read the committed `optimisticState`, managers and callbacks read `controller.getState()`. Every commit starts a new version. Rolling back an optimistic update returns to the older real entities. +- So `WeakDependencyMap` keeps the most recently used entries plus the one the store last flipped back to. Bound: `useSuspense` serves urgent re-renders of an unchanged store from its own `useMemo`, so during a retrying transition a newly committed version can be recomputed once. +- `yarn workspace example-benchmark start:memo-tradeoffs` measures hit rates and retained memory for these choices. + ## Schema-tree traversal pattern When walking the schema *definition* (not data), the canonical recursion is used in many places. Keep it consistent: @@ -72,7 +80,7 @@ Contract: - Reading `delegate.args` directly does **not** contribute to memoization — only `argsKey` adds a dep. - Storage shape must let `denormalize` recover the right cell from the bucket key (typically by encoding it into `pk` at normalize time). `Scalar` is the reference implementation. -Under the hood: `argsKey` pushes `{path: fn, entity: undefined}` onto the current `GlobalCache` frame; `WeakDependencyMap` branches function-typed paths via a lazy `Map` alongside the usual `WeakMap`. See `src/memo/WeakDependencyMap.ts` (`KeyFn`, `hasStringDeps`) and `src/memo/globalCache.ts` (`argsKey`, `_hasArgsKey`) for the fast-path gating. +Under the hood: `argsKey` pushes `{path: fn, entity: undefined}` onto the current `GlobalCache` frame; `WeakDependencyMap` branches function-typed paths via a lazy `Map` under the root's `WeakMap` entry. See `src/memo/WeakDependencyMap.ts` (`KeyFn`, `hasStringDeps`) and `src/memo/globalCache.ts` (`argsKey`, `_hasArgsKey`) for the fast-path gating. Schemas without args-dependence need no opt-in. When no `argsKey` fires in a frame, `GlobalCache.paths`/`getResults` take the entity-only fast path (gated on `_hasArgsKey` and `WeakDependencyMap.hasStringDeps`). diff --git a/packages/normalizr/src/__tests__/WeakDependencyMap.test.ts b/packages/normalizr/src/__tests__/WeakDependencyMap.test.ts index 966edb61e337..3d11eb35359e 100644 --- a/packages/normalizr/src/__tests__/WeakDependencyMap.test.ts +++ b/packages/normalizr/src/__tests__/WeakDependencyMap.test.ts @@ -92,6 +92,66 @@ describe('WeakDependencyMap', () => { expect(wem.get({}, getEntity)[0]).toBeUndefined(); }); + it('hits and misses on a long dependency list', () => { + const items = Array.from({ length: 5000 }, (_, i) => ({ i })); + const big = Object.fromEntries(items.map((v, i) => [`${i}`, v])); + const deps = [ + depA, + ...items.map((entity, i) => ({ path: { key: 'D', pk: `${i}` }, entity })), + ]; + const wem = new WeakDependencyMap(); + wem.set(deps, 'all'); + + expect(wem.get(a, MemoPolicy.getEntities({ ...state, D: big }))[0]).toBe( + 'all', + ); + const changed = { ...big, '4999': { i: 4999 } }; + expect( + wem.get(a, MemoPolicy.getEntities({ ...state, D: changed }))[0], + ).toBeUndefined(); + }); + + it('keeps three entries, never evicting the one the store flipped back to', () => { + const wem = new WeakDependencyMap(); + const [b2, b3, b4, b5] = [[2], [3], [4], [5]]; + const withB = (entity: object) => + MemoPolicy.getEntities({ ...state, B: { '2': entity } }); + const setB = (entity: object, value: string) => + wem.set([depA, { ...depB, entity }], value); + setB(b, 'b'); + setB(b2, 'b2'); + setB(b3, 'b3'); + // full: the least recently used goes, except the first entry, kept until a flip + setB(b4, 'b4'); + expect(wem.get(a, withB(b2))[0]).toBeUndefined(); + expect(wem.get(a, withB(b))[0]).toBe('b'); + + // reading b3 back after newer entries makes it the kept one instead; + // most recently used now: b3, b, b4 + expect(wem.get(a, withB(b3))[0]).toBe('b3'); + setB(b5, 'b5'); + expect(wem.get(a, withB(b4))[0]).toBeUndefined(); + setB(b4, 'b4 again'); + // most recently used: b4, b5, b3 (kept), b; b goes + expect(wem.get(a, withB(b))[0]).toBeUndefined(); + expect(wem.get(a, withB(b3))[0]).toBe('b3'); + }); + + it('keeps a re-read value while one-off values come and go', () => { + // a pending transition: React re-reads the committed store between + // retries that each build new entities + const wem = new WeakDependencyMap(); + const withB = (entity: object) => + MemoPolicy.getEntities({ ...state, B: { '2': entity } }); + wem.set([depA, depB], 'committed'); + for (let i = 0; i < 5; i++) { + const retry = [i]; + expect(wem.get(a, withB(retry))[0]).toBeUndefined(); + wem.set([depA, { ...depB, entity: retry }], `retry ${i}`); + expect(wem.get(a, withB(b))[0]).toBe('committed'); + } + }); + it.each<[string, any]>([ ['null', null], ['INVALID', INVALID], @@ -212,3 +272,244 @@ describe('WeakDependencyMap', () => { }); }); }); + +describe('WeakDependencyMap eviction', () => { + const root = { root: true }; + const rootDep = { path: { key: 'A', pk: '1' }, entity: root }; + const dep = (entity: object) => ({ path: { key: 'B', pk: '1' }, entity }); + const withB = (entity: object) => + MemoPolicy.getEntities({ A: { '1': root }, B: { '1': entity } }); + + it('keeps the committed version after the store moves on to a new one (two epochs)', () => { + const wem = new WeakDependencyMap(); + const S0 = { v: 'S0' }; + wem.set([rootDep, dep(S0)], 'S0'); + // epoch 1: pending transition; retries build fresh entities, urgent + // renders re-read S0. This is the pattern the existing tests cover. + for (let i = 0; i < 2; i++) { + const x = { v: `x${i}` }; + expect(wem.get(root, withB(x))[0]).toBeUndefined(); + wem.set([rootDep, dep(x)], `x${i}`); + expect(wem.get(root, withB(S0))[0]).toBe('S0'); + } + // the transition commits: its final render's entities ARE the new store + const S1 = { v: 'S1' }; + wem.set([rootDep, dep(S1)], 'S1'); + expect(wem.get(root, withB(S1))[0]).toBe('S1'); + // epoch 2: another pending transition builds a fresh entity... + wem.set([rootDep, dep({ v: 'y0' })], 'y0'); + // ...and an urgent render re-reads the committed store. S1 is alive and + // was read moments ago; the old chain would hit. S0 is dead but pinned. + expect(wem.get(root, withB(S1))[0]).toBe('S1'); + }); + + it('a stale entry does not crowd out a version that keeps being read', () => { + const wem = new WeakDependencyMap(); + const dead = { v: 'dead' }; + const S = { v: 'S' }; + wem.set([rootDep, dep(dead)], 'dead'); + wem.set([rootDep, dep(S)], 'S'); + // alternate one-offs with re-reads of S many times; S should earn its keep + let misses = 0; + for (let i = 0; i < 10; i++) { + wem.set([rootDep, dep({ v: `x${i}` })], `x${i}`); + if (wem.get(root, withB(S))[0] === undefined) { + misses++; + wem.set([rootDep, dep(S)], 'S'); + } + } + expect(misses).toBe(0); + }); + + it('a stable version first read after two one-offs is retained', () => { + // e.g. a list whose first render is inside a pending transition that + // suspended once (render, ping, render) before any urgent render + const wem = new WeakDependencyMap(); + const S = { v: 'S' }; + wem.set([rootDep, dep({ v: 'x0' })], 'x0'); + wem.set([rootDep, dep({ v: 'x1' })], 'x1'); + wem.set([rootDep, dep(S)], 'S'); + wem.set([rootDep, dep({ v: 'x2' })], 'x2'); + expect(wem.get(root, withB(S))[0]).toBe('S'); + }); + + describe('argsKey branches', () => { + // a missing entity skips its nested argsKey, so which fn comes first can + // change between store snapshots of the same schema and args + const portfolio = (args: readonly any[]) => args[0]?.portfolio; + const currency = (args: readonly any[]) => args[0]?.currency; + const region = (args: readonly any[]) => args[0]?.region; + const args = [{ portfolio: 'P', currency: 'USD', region: 'EU' }]; + const c = (entity: object | undefined) => ({ + path: { key: 'C', pk: '2' }, + entity, + }); + const fn = (path: (args: readonly any[]) => string | undefined) => ({ + path, + entity: undefined, + }); + const withC = (entity: object | undefined) => + MemoPolicy.getEntities({ A: { '1': root }, C: { '2': entity as any } }); + const loaded = { c: 1 }; + const changed = { c: 2 }; + function setup() { + const wem = new WeakDependencyMap(); + wem.set([rootDep, c(undefined)], 'plain', args); + wem.set([rootDep, c(undefined), fn(currency)], 'C missing', args); + wem.set( + [rootDep, c(loaded), fn(portfolio), fn(currency)], + 'C loaded', + args, + ); + // a second branch below the root: the lookup has to come back to it + wem.set([rootDep, c(changed), fn(portfolio), fn(region)], 'region', args); + return wem; + } + + it('finds the entry for each snapshot as they alternate', () => { + const wem = setup(); + for (let i = 0; i < 2; i++) { + // the first branch's child doesn't match, so the other branch is tried + expect(wem.get(root, withC(loaded), args)[0]).toBe('C loaded'); + // root entries are checked before any branch + expect(wem.get(root, withC(undefined), args)[0]).toBe('plain'); + // two branches deep: currency's child misses, then region's is tried + expect(wem.get(root, withC(changed), args)[0]).toBe('region'); + } + }); + + it('never returns an entry whose own argsKey results differ', () => { + const wem = setup(); + expect( + wem.get(root, withC(loaded), [{ ...args[0], currency: 'EUR' }])[0], + ).toBeUndefined(); + expect( + wem.get(root, withC(changed), [{ ...args[0], region: 'US' }])[0], + ).toBeUndefined(); + expect(wem.get(root, withC({ c: 3 }), args)[0]).toBeUndefined(); + }); + + it('keeps rows under the same fn apart by args', () => { + // a Scalar lens: every row records the same argsKey fn + const wem = new WeakDependencyMap(); + const row2 = (entity: object) => ({ + path: { key: 'D', pk: '2' }, + entity, + }); + const withRows = (c2: object, d2: object) => + MemoPolicy.getEntities({ + A: { '1': root }, + C: { '2': c2 }, + D: { '2': d2 }, + }); + const r1 = { r: 1 }; + const r2 = { r: 2 }; + for (const p of ['A', 'B']) { + wem.set([rootDep, fn(portfolio), c(r1), fn(portfolio), row2(r2)], p, [ + { portfolio: p }, + ]); + } + for (let i = 0; i < 2; i++) { + for (const p of ['A', 'B']) { + expect(wem.get(root, withRows(r1, r2), [{ portfolio: p }])[0]).toBe( + p, + ); + } + } + expect( + wem.get(root, withRows(r1, { r: 3 }), [{ portfolio: 'A' }])[0], + ).toBeUndefined(); + }); + + it('treats a selector that throws on unreached args as a miss', () => { + // the Scalar inside C reads args[0].portfolio; with C missing, traversal + // never calls it, so a lookup must not fail on it either + const wem = new WeakDependencyMap(); + const strict = (a: readonly any[]) => a[0].portfolio; + wem.set([rootDep, c(loaded), fn(strict)], 'with C', [{ portfolio: 'P' }]); + expect(() => wem.get(root, withC(undefined), [])).not.toThrow(); + expect(wem.get(root, withC(undefined), [])[0]).toBeUndefined(); + expect(wem.get(root, withC(loaded), [{ portfolio: 'P' }])[0]).toBe( + 'with C', + ); + }); + + it('bounds the routes a lookup searches', () => { + // each entry reaches a different argsKey first, as missing entities + // can make happen; lookups must not slow down as routes pile up + const wem = new WeakDependencyMap(); + let calls = 0; + const fns = Array.from({ length: 40 }, (_, i) => (a: readonly any[]) => { + calls++; + return `${i}${a[0]?.x}`; + }); + const getEntity = MemoPolicy.getEntities({ A: { '1': root } }); + fns.forEach((path, i) => + wem.set([rootDep, fn(path)], `v${i}`, [{ x: 1 }]), + ); + calls = 0; + // every route is valid for these args, so any of the kept ones may hit + expect(wem.get(root, getEntity, [{ x: 1 }])[0]).toBeDefined(); + expect(calls).toBeLessThanOrEqual(10); + // a long chain of one argsKey after another is not a fork + wem.set([rootDep, ...fns.map(fn)], 'chain', [{ x: 1 }]); + expect(wem.get(root, getEntity, [{ x: 2 }])[0]).toBeUndefined(); + }); + + it('evicts inside an argsKey child as at the root', () => { + const wem = new WeakDependencyMap(); + const [b2, b3, b4, b5] = [[2], [3], [4], [5]]; + const b = [1]; + const one = [{ portfolio: 'P' }]; + const setB = (entity: object, value: string) => + wem.set([rootDep, fn(portfolio), dep(entity)], value, one); + const get = (entity: object) => wem.get(root, withB(entity), one)[0]; + setB(b, 'b'); + setB(b2, 'b2'); + setB(b3, 'b3'); + setB(b4, 'b4'); + expect(get(b2)).toBeUndefined(); + expect(get(b)).toBe('b'); + expect(get(b3)).toBe('b3'); + setB(b5, 'b5'); + expect(get(b4)).toBeUndefined(); + setB(b4, 'b4 again'); + expect(get(b)).toBeUndefined(); + expect(get(b3)).toBe('b3'); + }); + }); + + it('looks up values behind many nested argsKey deps', () => { + // a list whose rows each record an argsKey nests one node per row + const wem = new WeakDependencyMap(); + const fns = Array.from({ length: 20000 }, (_, i) => () => `${i}`); + const getEntity = MemoPolicy.getEntities({ A: { '1': root } }); + wem.set( + [rootDep, ...fns.map(path => ({ path, entity: undefined }))], + 'deep', + ); + expect(wem.get(root, getEntity)[0]).toBe('deep'); + }); + + it('a shorter dependency list at the same root is served when the longer one mismatches', () => { + // each entry is checked against its own dependencies, so the older one + // still serves reads that never touch C + const wem = new WeakDependencyMap(); + const b = { b: 1 }; + const c = { c: 1 }; + const depC = (entity: object | undefined) => ({ + path: { key: 'C', pk: '1' }, + entity, + }); + wem.set([rootDep, dep(b)], 'short'); + wem.set([rootDep, dep(b), depC(c)], 'long'); + const both = MemoPolicy.getEntities({ + A: { '1': root }, + B: { '1': b }, + C: { '1': c }, + }); + const noC = MemoPolicy.getEntities({ A: { '1': root }, B: { '1': b } }); + expect(wem.get(root, both)[0]).toBe('long'); + expect(wem.get(root, noC)[0]).toBe('short'); + }); +}); diff --git a/packages/normalizr/src/memo/WeakDependencyMap.ts b/packages/normalizr/src/memo/WeakDependencyMap.ts index 156675843524..7ed2d32d1034 100644 --- a/packages/normalizr/src/memo/WeakDependencyMap.ts +++ b/packages/normalizr/src/memo/WeakDependencyMap.ts @@ -34,21 +34,29 @@ export function depKey(entity: unknown): object { * * Useful as a memoization cache for flat/normalized stores. * - * Object dependencies are weakly referenced (via `WeakMap`), allowing - * automatic garbage collection when the dependency is no longer used. - * String-keyed dependencies (used by `argsKey`) sit on a `Map` keyed by the - * value returned from `path(args)`, branching on a stable function reference. + * Only the first dependency is a `WeakMap` key, so a cached value is + * collected with it. The rest are stored on the entry and compared by + * identity on `get`. Keeping one level of `WeakMap` matters: garbage + * collectors resolve nested `WeakMap`s one level per marking pass, and + * Hermes rescans every `WeakMap` entry in the heap on each pass, so a chain + * as long as the dependency list stalls every old-generation collection. + * + * String-keyed dependencies (used by `argsKey`) branch on a `Map` keyed by + * the value `path(args)` returns, so each argument set keeps its own entries + * and a lookup only checks its own. A node keeps one branch per `argsKey` + * function: which one a computation reaches first can depend on the + * entities before it (a missing entity skips its nested `argsKey`). + * + * Invariants every change must keep: "Memo invariants" in + * `packages/normalizr/AGENTS.md`. */ export default class WeakDependencyMap< Path, K extends object = object, V = any, > { - private readonly next = new WeakMap>(); - private nextPath: Path | KeyFn | undefined = undefined; - /** Sticky: true once any function-typed (`argsKey`) dep has been stored. - * Lets `get` pick the entity-only fast path when no schema in this map - * uses `argsKey` — avoids a polymorphic `typeof` branch per walk step. */ + private readonly next = new WeakMap>(); + /** Sticky: true once any function-typed (`argsKey`) dep has been stored. */ private hasStr = false; get( @@ -56,38 +64,11 @@ export default class WeakDependencyMap< getDependency: GetDependency, args: readonly any[] = [], ) { - let curLink = this.next.get(depKey(entity) as K); - if (!curLink) return EMPTY; - if (this.hasStr) return this._getMixed(curLink, getDependency, args); - while (curLink.nextPath) { - const nextDependency = depKey(getDependency(curLink.nextPath as Path)); - const nextLink = curLink.next.get(nextDependency as any); - if (!nextLink) return EMPTY; - curLink = nextLink; - } - // curLink exists, but has no path - so must have a value - return [curLink.value, curLink.journey] as readonly [V, Path[]]; - } - - /** Slow path: dep chain may interleave entity and `argsKey`-style deps. */ - private _getMixed( - curLink: Link, - getDependency: GetDependency, - args: readonly any[], - ) { - while (curLink.nextPath) { - let nextLink: Link | undefined; - if (typeof curLink.nextPath === 'function') { - const keyValue = (curLink.nextPath as KeyFn)(args) ?? UNDEF_KEY; - nextLink = curLink.nextStr?.get(keyValue); - } else { - const nextDependency = depKey(getDependency(curLink.nextPath as Path)); - nextLink = curLink.next.get(nextDependency as any); - } - if (!nextLink) return EMPTY; - curLink = nextLink; - } - return [curLink.value, curLink.journey] as readonly [V, Path[]]; + const root = this.next.get(depKey(entity) as K); + const node = root && find(root, getDependency, args); + if (!node) return EMPTY; + const entry = node.entries[0]; + return [entry.value, entry.journey] as readonly [V, Path[]]; } set( @@ -103,34 +84,68 @@ export default class WeakDependencyMap< journey?: Path[], ) { if (dependencies.length < 1) throw new KeySize(); - let curLink: Link = this as any; - for (const dep of dependencies) { - let nextLink: Link | undefined; + const root = depKey(dependencies[0].entity) as K; + let node = this.next.get(root); + let top = node; + const paths = new Array(dependencies.length - 1); + const keys = new Array(dependencies.length - 1); + for (let i = 1; i < dependencies.length; i++) { + const dep = dependencies[i]; + paths[i - 1] = dep.path; if (typeof dep.path === 'function') { this.hasStr = true; - if (!curLink.nextStr) curLink.nextStr = new Map(); const k = (dep.path as KeyFn)(args) ?? UNDEF_KEY; - nextLink = curLink.nextStr.get(k); - if (!nextLink) { - nextLink = new Link(); - curLink.nextStr.set(k, nextLink); + keys[i - 1] = k; + if (!node) { + node = top = newNode(undefined); + this.next.set(root, node); } - } else { - const key = depKey(dep.entity) as K; - nextLink = curLink.next.get(key); - if (!nextLink) { - nextLink = new Link(); - curLink.next.set(key, nextLink); + if (!node.branches) node.branches = []; + let branch = node.branches.find(b => b.fn === dep.path); + if (!branch) { + // Missing entities change which argsKey comes next, so the routes + // under one root can multiply, and a lookup searches all of them. + // Past MAX_FORKS, start this root over instead. + if (node.branches.length && ++top!.forks > MAX_FORKS) { + this.next.delete(root); + return this.set(dependencies, value, args, journey); + } + branch = { fn: dep.path as KeyFn, children: new Map() }; + node.branches.push(branch); + } + let child = branch.children.get(k); + if (!child) { + child = newNode(undefined); + branch.children.set(k, child); } + node = child; + } else { + keys[i - 1] = depKey(dep.entity); } - curLink.nextPath = dep.path as any; - curLink = nextLink; } - // in case there used to be more - curLink.nextPath = undefined; - curLink.value = value; - // we could recompute this on get, but it would have a cost and we optimize for `get` - curLink.journey = journey ?? (dependencies.map(d => d.path) as Path[]); + const entry: Entry = { + paths, + keys, + value, + // we could recompute this on get, but it would have a cost and we optimize for `get` + journey: journey ?? (dependencies.map(d => d.path) as Path[]), + }; + if (!node) { + this.next.set(root, newNode(entry)); + return; + } + // A store can flip back to a version it already showed: React renders the + // committed store between retries of a pending transition, which build new + // entities each time, and rolling back an optimistic update returns to the + // entities from before it. So beyond the newest entries, keep the one the + // store last flipped back to (or the first one, until a flip happens). + const { entries } = node; + if (!node.kept) node.kept = entry; + entries.unshift(entry); + if (entries.length > MAX_ENTRIES) { + const last = entries.length - 1; + entries.splice(entries[last] === node.kept ? last - 1 : last, 1); + } } /** True once any `argsKey`-style dep has been written. Consumers can use @@ -140,6 +155,72 @@ export default class WeakDependencyMap< } } +/** The node at or below `node` with an entry that matches the store, if any; + * that entry is moved to the front of its entries. A loop, not recursion: + * a list can record one `argsKey` per row, so the nodes nest that deep. */ +function find( + node: Node | undefined, + getDependency: GetDependency, + args: readonly any[], +): Node | undefined { + // other branches to try, only allocated when a node has more than one + let pending: Node[] | undefined; + while (node) { + const { entries, branches } = node; + for (let i = 0; i < entries.length; i++) { + const entry = entries[i]; + if (matches(entry, getDependency)) { + if (i > 0) { + // read back after something newer was cached: the store flipped + // back to this version, so it is the one to keep + node.kept = entry; + entries.splice(i, 1); + entries.unshift(entry); + } + return node; + } + } + let next: Node | undefined; + if (branches) { + for (let b = 0; b < branches.length; b++) { + const { fn, children } = branches[b]; + const key = select(fn, args); + const child = key === undefined ? undefined : children.get(key); + if (!child) continue; + if (!next) next = child; + else (pending || (pending = [])).push(child); + } + } + node = next ?? pending?.pop(); + } +} + +/** A branch's `argsKey` result. A branch is tried before the entities that + * led to it are checked, so its selector can see args its own schema path + * would never reach with (say, a nested entity that is now missing): treat a + * throw as no match, since traversal wouldn't have called it. */ +function select(fn: KeyFn, args: readonly any[]): string | undefined { + try { + return fn(args) ?? UNDEF_KEY; + } catch { + return undefined; + } +} + +function matches( + entry: Entry, + getDependency: GetDependency, +): boolean { + const { paths, keys } = entry; + for (let i = 0; i < paths.length; i++) { + const path = paths[i]; + // the route to this entry already picked every argsKey result + if (typeof path === 'function') continue; + if (depKey(getDependency(path as Path)) !== keys[i]) return false; + } + return true; +} + export type GetDependency = ( lookup: Path, ) => K | undefined; @@ -151,14 +232,45 @@ export interface Dep { const EMPTY = [undefined, undefined] as const; -/** Link in a chain */ -class Link { - next = new WeakMap>(); - /** Lazily allocated branch for string-keyed (`argsKey`) deps. */ - nextStr: Map> | undefined = undefined; - nextPath: Path | KeyFn | undefined = undefined; - value: V | undefined = undefined; - journey: Path[] = []; +/** Entries kept per (first dependency, `argsKey` results). */ +const MAX_ENTRIES = 3; + +/** Nodes under one first dependency that may gain a second `argsKey` branch. */ +const MAX_FORKS = 8; + +/** Entries for one set of `argsKey` results, most recently used first, and the + * branches on the next one. */ +interface Node { + entries: Entry[]; + /** Never evicted: the entry most recently read back after a newer one. */ + kept: Entry | undefined; + /** One per next `argsKey` dependency; its result picks the child. */ + branches: Branch[] | undefined; + /** On a first dependency's node: extra branches made anywhere below it. */ + forks: number; +} + +interface Branch { + readonly fn: KeyFn; + readonly children: Map>; +} + +function newNode(entry: Entry | undefined): Node { + return { + entries: entry ? [entry] : [], + kept: entry, + branches: undefined, + forks: 0, + }; +} + +/** One cached value: the dependencies after the first, and what they resolved to. */ +interface Entry { + readonly paths: (Path | KeyFn)[]; + /** `depKey()` of the entity for entity paths; `argsKey` result for function paths. */ + readonly keys: unknown[]; + readonly value: V; + readonly journey: Path[]; } class KeySize extends Error { diff --git a/packages/react/src/__tests__/concurrent-memo.web.tsx b/packages/react/src/__tests__/concurrent-memo.web.tsx index 9f6caf4c9c90..a8497f7f21cf 100644 --- a/packages/react/src/__tests__/concurrent-memo.web.tsx +++ b/packages/react/src/__tests__/concurrent-memo.web.tsx @@ -338,7 +338,11 @@ describeConcurrent.each([1, 2])( await finishTransition('Y'); trace('second transition'); expect(flipBacks(reads)).toBeGreaterThan(0); - expect(avoidableMisses(reads)).toBe(0); + // The committed rows changed when the first write committed, and typing + // re-renders are served by useSuspense's useMemo, so the memo never + // sees them re-read before the transition's retries push them out. + // That costs one recompute, after which they are kept. + expect(avoidableMisses(reads)).toBeLessThanOrEqual(1); }); it('reads outside render during optimistic updates, across a resolved mutation', async () => { @@ -389,7 +393,11 @@ describeConcurrent.each([1, 2])( await finishTransition('X'); trace('typing after two committed writes'); expect(flipBacks(reads)).toBeGreaterThan(0); - expect(avoidableMisses(reads)).toBe(0); + // The committed rows changed when the first write committed, and typing + // re-renders are served by useSuspense's useMemo, so the memo never + // sees them re-read before the transition's retries push them out. + // That costs one recompute, after which they are kept. + expect(avoidableMisses(reads)).toBeLessThanOrEqual(1); }); }, ); diff --git a/website/blog/2026-10-03-v0.19-batch-set.md b/website/blog/2026-10-03-v0.19-batch-set.md index e49cb2373724..ad77c3cf72a6 100644 --- a/website/blog/2026-10-03-v0.19-batch-set.md +++ b/website/blog/2026-10-03-v0.19-batch-set.md @@ -56,6 +56,7 @@ Coding agents also get a Claude Code plugin, Vue skills and llms.txt; see - Polling keeps the schema of endpoints given [pollFrequency](/rest/api/RestEndpoint#pollfrequency) with `.extend()`, so a polled [useLive()](/docs/api/useLive) list updates right away from mutations and optimistic updates instead of waiting for the next poll ([#4437](https://github.com/reactive/data-client/pull/4437)) - [DataProvider](/docs/api/DataProvider) only warns that it doesn't update during SSR when something fetches while the server renders, instead of on every server render, so pages that load data after mount (like Expo Router's static web output) stay quiet ([#4449](https://github.com/reactive/data-client/pull/4449)) - Server rendering with React 19 no longer logs "Uncaught suspense" when nothing suspended; React renders and discards fallbacks of large boundaries, and the warning now waits for a fallback that actually shows ([#4450](https://github.com/reactive/data-client/pull/4450)) +- React Native apps that keep reading large results while entities stream no longer freeze for seconds at a time; memoization had made each Hermes garbage collection rescan its `WeakMap`s once per cached dependency, and cached reads are now faster on every platform ([#4459](https://github.com/reactive/data-client/pull/4459)) - A cached result no longer keeps showing a nested entity as missing after that entity was `INVALID` or `null` and becomes valid again ([#4460](https://github.com/reactive/data-client/pull/4460)) - Vue `createDataClient()` is deprecated, since it's internal to [DataClientPlugin](/vue/api/DataClientPlugin); install the plugin with `app.use(DataClientPlugin, options)` instead of calling it directly ([#4193](https://github.com/reactive/data-client/pull/4193)) - Vue `renderDataCompose()`'s `waitForNextUpdate()` is deprecated, since it gives up silently after 1 second and lets a test pass while still suspended; `await result` instead, or `await allSettled()` after a change, which now also waits for fetches a prop change starts ([unit testing composables](/vue/guides/unit-testing-composables#renderdatacompose-api), [#4221](https://github.com/reactive/data-client/pull/4221))