Skip to content

Add indexed topic curation, Decision-assisted admission, and frozen replay - #3233

Merged
Hillary Mutisya (hillary-mutisya) merged 8 commits into
mainfrom
dev/hillarym/memory28-shared-integration
Oct 27, 2026
Merged

Hillary Mutisya (hillary-mutisya) merged 8 commits into
mainfrom
dev/hillarym/memory28-shared-integration

Conversation

@hillary-mutisya

@hillary-mutisya Hillary Mutisya (hillary-mutisya) commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds indexed topic curation, an opt-in Decision-assisted curation engine, and reproducible evaluation from saved inputs. Both engines use the same generator API, evidence/ownership indexes, request inspection, validation, and replay utilities.

Functionality

Shared curation and model infrastructure

  • Index source incidence, exact/canonical evidence ownership, and lexical concept nominations instead of repeatedly scanning the corpus. Concept replacements update only affected postings; native target identifiers retain frozen transaction ordering.
  • Reject stale evidence indexes and validate complete original membership, citations, pending assignments, cluster coverage receipts, and maintenance signatures. Keep canonical member support separate from the citations selected for a generated summary.
  • Replace positional generator construction with a named options object selecting raw, generative, or decision mode. Migrate service hosts, tools, and tests; reject malformed modes and incompatible engine settings.
  • Expose transport-free inspection of the complete serialized chat request, including effective framing and schema. Propagate child cancellation, call kinds, and ordinals through completion adapters, and emit privacy-filtered timing/usage receipts.

Generative curation

  • Add optional packing against the complete request's token/context constraints, including separate prior-concept input/output allocation.
  • Add bounded two-worker speculation with canonical commit order. Rebuild packet dependencies before reuse; abort, drain, and rerun stale proposals rather than publishing stale decisions.
  • Preserve full original-member source context when validating cross-batch concept evidence.

Decision-assisted curation

  • Ground salience and membership decisions in original source evidence, roles, and chronology; reconcile candidate relationships and generate concept metadata through the common completion path.
  • Add fixed-definition member admission, localized uncertainty review, retryable pending assignments, and prioritized old-member maintenance.
  • Support bounded native request waves, configurable packet sizing/concurrency/lookahead, and freshness-checked speculative transactions. Bind native execution to model/provider/configuration identity.
  • Wire engine configuration, pending/coverage/maintenance metadata, and operation receipts through the durable service, strict client protocol, and benchmark bridge.

Generative curation remains the service default, with legacy packing, one worker, and lazy summaries. Token-aware packing, generative speculation, and Decision curation are explicit options.

Frozen replay and evaluation

  • Run curation from captured build inputs or retained view-history commits without repeating ingestion.
  • Provide provider-free capture and exact recorded-response playback, bounded live recording, atomic progress/resume, CPU parity diagnostics, and disposable view/index storage measurements.
  • Bind replay to input, implementation, selection, and policy identity. Native tapes also bind ordered requests, model/wire identity, admission checks, and duplicate request occurrences.
  • Resume partial or completed node-subset checkpoints while retaining and validating complete source incidence; reject changed selections and corrupt evidence outside the subset.
  • Add Decision study/microtrial/sweep tools and benchmark operation receipts. LongMemEval continuation can copy a terminated interrupted run into a new isolated store, reuse completed imports, and apply an explicitly increased readiness allowance with provenance checks.

Review guide

Area Entry points
Shared indexes and validation topicLookup.ts, topicSnapshot.ts, topicCuration.ts
Generator API and host selection topicGeneration.ts, viewHostOptions.ts, fileMemoryService.ts
Generative packing and scheduling topicCuration.ts, topicCurationScheduler.ts
Native admission and maintenance topicDecision.ts, topicDecisionCuration.ts, topicDecisionClusters.ts, topicClusterMaintenance.ts
Model inspection and receipts openai.ts, decisionModels.ts, searchTiming.ts
Replay and evaluation contracts topic-curation-replay.mjs, topic-decision-replay.mjs, replay-tape.mjs, longmemeval-live.mjs

Validation

  • All nine TypeScript CI matrix jobs and Linux/Windows Azure smoke tests passed; shell packaging, .NET, CodeQL, formatting, and policy checks also passed.
  • Full workspace build: pnpm run build.
  • Complete offline memory-service suite: 1,264 tests across 91 suites, run serially with the existing CI heap setting.
  • Complete client suite: 461 tests across 30 suites.
  • pnpm run test:memory-quality: 169 tests.
  • Related client protocol, model inspection, timing, and observer contracts: 154 tests.
  • Pinned formatting and lint, complexity, circular-dependency, and test-debt ratchets.
  • Saved 2,909-node input: evidence, ownership-validation, and nomination parity; historical full snapshot validation; two saved 96-node serial runs reproduce exact curation.
  • Fresh 96-node live curation: all nodes accounted for in three calls; exact offline replay and zero-call checkpoint restart.

Preserve default packing and serial commits while adding complete-request inspection, opt-in prior allocation and freshness-checked speculation. Keep exact evidence validation and durable checkpoints; document measured limits.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Integrate native feature feede39 and shared infrastructure d082947 on the current memory28 tree without replaying squashed history. Use one validated named-mode generator API and a delta-only shared lexical index with transaction-owned native positions. Preserve generative defaults and add both-engine replay, protocol and incremental replacement regressions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Validate named generator modes, migrate dynamic callers, remove shared ordinal ownership, cover constant-delta catalog visits and large replay identity patches, and make retry timing tests independent of body stream I/O. Correct the frozen workspace dependency link and document the canonical contracts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Validate complete source incidence before selecting replay nodes, then validate checkpoint membership against the selected subset. Cover partial and completed restart, selection mismatch, and evidence outside the selected nodes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exclude unrelated automatic maintenance from history and projection read counts using the existing async operation context. Exercise concurrent unrelated reads while retaining exact one-projection and zero-full-history assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pin retry jitter to its maximum and verify the exact Retry-After plus jitter boundary. Restore timers and mocks after every test so a timed-out fake-clock test cannot stall subsequent credential timeout and cancellation tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@hillary-mutisya
Hillary Mutisya (hillary-mutisya) merged commit 38566f0 into main Oct 27, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant