From 490b5a0ce0d98d9c24aee097429b10b3db8b5b6c Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Wed, 14 Oct 2026 12:08:23 -0400 Subject: [PATCH 01/11] fix(dump): dump the live omp session through a confirmed recall_dump tool omp sets CLAUDECODE=1 as well as OMPCODE=1, and session discovery only knew Claude, OpenCode and Pi, so recall dump from omp imported the newest Claude transcript. Session files cannot identify omp's live branch or profile, so the omp extension now registers a recall_dump tool that sends its live getBranch() text and session id to dist/index.js dump --stdin (new contract 1 input) after the user confirms in the omp UI. Inside omp, file discovery returns nothing and recall dump points at the tool. --- agent-skills/do-recall-dump/SKILL.md | 2 + docs/OMP_INTEGRATION.md | 8 +- docs/cli-reference.md | 2 +- hosts/omp/recall.ts | 169 ++++++++++++++---- .../skills/do-recall-dump/SKILL.md | 2 + .../recall/skills/do-recall-dump/SKILL.md | 2 + src/commands/capture.ts | 2 +- src/commands/dump.ts | 63 ++++++- src/hosts/session-sources.ts | 3 + src/index.ts | 4 +- tests/api.test.ts | 3 +- tests/commands/dump-portable.test.ts | 52 +++++- tests/helpers/env.ts | 12 ++ tests/hosts/native-hosts.test.ts | 11 +- tests/omp-integration.test.ts | 13 ++ 15 files changed, 307 insertions(+), 41 deletions(-) create mode 100644 tests/helpers/env.ts diff --git a/agent-skills/do-recall-dump/SKILL.md b/agent-skills/do-recall-dump/SKILL.md index 249f7608..d6527a79 100644 --- a/agent-skills/do-recall-dump/SKILL.md +++ b/agent-skills/do-recall-dump/SKILL.md @@ -8,6 +8,8 @@ Dump the current conversation session into the Recall SQLite database, making al Run this at the end of every session or when you want to persist the current conversation mid-session. +**In omp:** call the `recall_dump` tool with the title instead of running `recall dump`; omp asks the user to confirm before anything is written. Inside omp the shell command refuses, because transcript files on disk belong to other conversations. + ## Usage ```bash diff --git a/docs/OMP_INTEGRATION.md b/docs/OMP_INTEGRATION.md index a7502d3d..0f79ef1b 100644 --- a/docs/OMP_INTEGRATION.md +++ b/docs/OMP_INTEGRATION.md @@ -36,9 +36,13 @@ This is turn-completion capture, not crash recovery. An interrupted turn before Capture sends only user and assistant text, at most 25 MiB per payload, and bounds its child process to 30 seconds. Tool output, images, and other non-message entries are not sent and do not count toward the limit. Oversize, malformed, cancelled, or failed captures warn without blocking the agent or publishing a partial branch. +## Explicit dump: `recall_dump` tool + +The extension also registers a `recall_dump` tool (title, optional project, tags, `continues`, `skip_fabric`). It reads the live branch and session id from omp, so it dumps this conversation on any profile or branch, and sends it to the package-local `dist/index.js dump --stdin`. The tool asks you to confirm in the omp UI before writing anything and refuses in headless sessions. `recall dump` run from a shell inside omp (`OMPCODE=1`) refuses and points to the tool: transcript files on disk belong to other conversations. + ## Skills and MCP remain separate -`recall install` still links the nine canonical `do-recall-*` skills into `~/.omp/agent/skills/` when omp is detected. It does not activate this extension. The native package does not duplicate those skills, register MCP, or inject session-start memory. Existing manual MCP configuration remains untouched. +`recall install` still links the nine canonical `do-recall-*` skills into `~/.omp/agent/skills/` when omp is detected. It does not activate this extension. The native package does not duplicate those skills, register MCP, or inject session-start memory; `recall_dump` is its only tool. Existing manual MCP configuration remains untouched. ## Verify and remove @@ -56,7 +60,7 @@ This verifies the packed local-link integration in headless text mode. It does n Restart omp after uninstalling. Saved Recall memory remains. `recall uninstall` removes installer-owned skills, not the separately managed omp plugin; uninstall the plugin through omp before removing Recall's runtime. -If capture warns, check `bun` is available to omp, run `bun dist/index.js init` from the linked checkout, and check `RECALL_DB_PATH` permissions. A branch whose user and assistant text alone exceeds 25 MiB needs a new session; restarting alone does not shrink it. +If capture warns, check `bun` is available to omp, run `bun dist/index.js init` from the linked checkout, and check `RECALL_DB_PATH` permissions. An omp process started before the extension was refreshed keeps the old code until it restarts: older versions measured the whole raw branch and warned `payload exceeds 25MiB` on long tool-heavy sessions. Only a branch whose user and assistant text alone exceeds 25 MiB needs a new session. ## Native documentation diff --git a/docs/cli-reference.md b/docs/cli-reference.md index efe24b82..4ef11f2e 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -78,7 +78,7 @@ Prove a write with `recall search "" -t messages`. recall dump "Session Title" # Import session + capture LoA entry ``` -Combines `recall import` and Curated LoA capture in one step. Run this at the end of every session to persist the conversation and run the `fabric` Extractor. The title should describe what was accomplished in the session. `--skip-fabric` writes a basic summary instead. +Combines `recall import` and Curated LoA capture in one step. Run this at the end of every session to persist the conversation and run the `fabric` Extractor. The title should describe what was accomplished in the session. `--skip-fabric` writes a basic summary instead. `--stdin` reads the session as contract 1 JSON (`harness`, `session_id`, optional `cwd`/`project`, `messages` of user/assistant `role`/`content`/`timestamp`) for host extensions that hold the live transcript; inside omp, use the [`recall_dump` tool](OMP_INTEGRATION.md#explicit-dump-recall_dump-tool) instead of the shell command. ### Library of Alexandria (LoA) diff --git a/hosts/omp/recall.ts b/hosts/omp/recall.ts index a2d64ae5..4ae8ebcc 100644 --- a/hosts/omp/recall.ts +++ b/hosts/omp/recall.ts @@ -4,6 +4,7 @@ import { spawn } from 'node:child_process'; import { join } from 'node:path'; import { fileURLToPath } from 'node:url'; +import type { z } from 'zod'; const PACKAGE_ROOT = fileURLToPath(new URL('../..', import.meta.url)); const CLI_PATH = join(PACKAGE_ROOT, 'dist', 'index.js'); @@ -18,26 +19,56 @@ export interface OmpSessionStopEvent { export interface OmpExtensionContext { cwd: string; hasUI?: boolean; - ui?: { notify: (message: string, type?: 'info' | 'warning' | 'error') => void }; + ui?: { + notify: (message: string, type?: 'info' | 'warning' | 'error') => void; + confirm?: (title: string, message: string) => Promise; + }; sessionManager: { getSessionId: () => string; getBranch: (fromId?: string) => unknown[]; }; } +export interface OmpToolResult { + content: { type: 'text'; text: string }[]; +} + export interface OmpExtensionAPI { on( event: 'session_stop', handler: (event: OmpSessionStopEvent, ctx: OmpExtensionContext) => unknown, ): void; + registerTool?(tool: { + name: string; + label: string; + description: string; + parameters: unknown; + execute( + toolCallId: string, + params: RecallDumpParams, + signal: AbortSignal | undefined, + onUpdate: unknown, + ctx: OmpExtensionContext, + ): Promise; + }): void; + zod?: typeof z; logger?: { warn: (message: string) => void }; } +export interface RecallDumpParams { + title: string; + project?: string; + tags?: string; + continues?: number; + skip_fabric?: boolean; +} + export type RunBoundedChild = ( file: string, args: readonly string[], stdin: string, signal: AbortSignal, + onOutput?: (chunk: string) => void, ) => Promise; function warn(ctx: OmpExtensionContext, pi: OmpExtensionAPI | undefined, message: string): void { @@ -61,20 +92,23 @@ function warn(ctx: OmpExtensionContext, pi: OmpExtensionAPI | undefined, message } } -/** Spawn a child, write stdin, honor abort (SIGTERM then SIGKILL). */ +/** Spawn a child, write stdin, honor abort (SIGTERM then SIGKILL). `onOutput` opts into stdout+stderr. */ export function runBoundedChild( file: string, args: readonly string[], stdin: string, signal: AbortSignal, + onOutput?: (chunk: string) => void, ): Promise { const { promise, resolve, reject } = Promise.withResolvers(); let settled = false; let killTimer: ReturnType | undefined; - const child = spawn(file, [...args], { - stdio: ['pipe', 'ignore', 'ignore'], - }); + const output = onOutput ? 'pipe' : 'ignore'; + const child = spawn(file, [...args], { stdio: ['pipe', output, output] }); + for (const stream of [child.stdout, child.stderr]) { + stream?.setEncoding('utf8').on('data', (chunk: string) => onOutput?.(chunk)); + } const onAbort = () => { try { @@ -142,12 +176,14 @@ function ompMessageText(content: unknown): string { return parts.join('\n'); } -/** Ordered user/assistant text. Malformed branches throw so capture is not partial. */ -function ompBranchText(entries: unknown): string { +export interface OmpMessage { role: 'user' | 'assistant'; content: string; timestamp?: string } + +/** Ordered user/assistant messages. Malformed branches throw so capture/dump is not partial. */ +export function ompBranchMessages(entries: unknown): OmpMessage[] { if (!Array.isArray(entries)) { throw new Error('omp lifecycle capture requires an entries array'); } - const parts: string[] = []; + const messages: OmpMessage[] = []; const seenIds = new Set(); for (const entry of entries) { if (!entry || typeof entry !== 'object' || Array.isArray(entry)) { @@ -178,9 +214,13 @@ function ompBranchText(entries: unknown): string { } const content = ompMessageText(message.content); if (!content.trim()) continue; - parts.push(content); + messages.push({ + role: message.role, + content, + timestamp: typeof row.timestamp === 'string' && row.timestamp ? row.timestamp : undefined, + }); } - return parts.join('\n\n'); + return messages; } export async function captureOmpSessionStop( @@ -202,7 +242,7 @@ export async function captureOmpSessionStop( return; } - const text = ompBranchText(entries); + const text = ompBranchMessages(entries).map(message => message.content).join('\n\n'); if (!text.trim()) return; const payload = JSON.stringify({ @@ -218,30 +258,99 @@ export async function captureOmpSessionStop( return; } - const timeout = new AbortController(); - const timer = setTimeout(() => timeout.abort(), CHILD_TIMEOUT_MS); - timer.unref(); - const signal = event.signal - ? AbortSignal.any([event.signal, timeout.signal]) - : timeout.signal; - - try { - const code = await runChild('bun', [CLI_PATH, 'capture'], payload, signal); - if (code === 0) return; - warn( - ctx, - pi, - signal.aborted ? 'Recall omp capture cancelled' : 'Recall omp capture failed', - ); - } finally { - clearTimeout(timer); - timeout.abort(); - } + const { code, aborted } = await runCli(runChild, ['capture'], payload, event.signal, CHILD_TIMEOUT_MS); + if (code === 0) return; + warn(ctx, pi, aborted ? 'Recall omp capture cancelled' : 'Recall omp capture failed'); } catch { warn(ctx, pi, 'Recall omp capture failed'); } } +/** Run the package-local CLI under the caller's signal plus a hard timeout. */ +async function runCli( + runChild: RunBoundedChild, + args: readonly string[], + payload: string, + outer: AbortSignal | undefined, + timeoutMs: number, + onOutput?: (chunk: string) => void, +): Promise<{ code: number | null; aborted: boolean }> { + const timeout = new AbortController(); + const timer = setTimeout(() => timeout.abort(), timeoutMs); + timer.unref(); + const signal = outer ? AbortSignal.any([outer, timeout.signal]) : timeout.signal; + try { + const code = await runChild('bun', [CLI_PATH, ...args], payload, signal, onOutput); + return { code, aborted: signal.aborted }; + } finally { + clearTimeout(timer); + timeout.abort(); + } +} + +/** The fabric Extractor can take minutes on a long session. */ +const DUMP_TIMEOUT_MS = 5 * 60_000; + +/** + * `recall_dump` tool: the live branch and session id come from omp itself, so the dump + * matches this conversation on every profile and branch. `recall dump` without this + * input refuses inside omp rather than guessing from files. Requires user confirmation. + */ +export async function dumpOmpSession( + params: RecallDumpParams, + ctx: OmpExtensionContext, + signal?: AbortSignal, + runChild: RunBoundedChild = runBoundedChild, +): Promise { + const reply = (text: string): OmpToolResult => ({ content: [{ type: 'text', text }] }); + try { + // Dumping is always the user's call (agent-skills/AGENTS.md): the model can request + // it, but nothing is written without a yes in the omp UI. Headless sessions refuse. + if (!ctx.hasUI || !ctx.ui?.confirm) return reply('Recall dump needs the interactive omp UI to confirm; nothing was saved'); + const approved = await ctx.ui.confirm('Save this conversation to Recall?', `recall dump "${params.title}"`); + if (!approved) return reply('Recall dump declined; nothing was saved'); + const sessionId = ctx.sessionManager.getSessionId(); + if (!sessionId) return reply('Recall dump failed: omp did not report a session id'); + const messages = ompBranchMessages(ctx.sessionManager.getBranch()); + if (messages.length === 0) return reply('Recall dump skipped: no user or assistant text in this session yet'); + const payload = JSON.stringify({ contract: 1, harness: 'omp', session_id: sessionId, cwd: ctx.cwd, messages }); + if (Buffer.byteLength(payload, 'utf8') > MAX_OMP_STDIN_BYTES) { + return reply('Recall dump skipped: conversation text exceeds 25MiB'); + } + + const args = ['dump', params.title, '--stdin']; + if (params.project) args.push('-p', params.project); + if (params.tags) args.push('-t', params.tags); + if (params.continues !== undefined) args.push('-c', String(params.continues)); + if (params.skip_fabric) args.push('--skip-fabric'); + + let output = ''; + const { code, aborted } = await runCli(runChild, args, payload, signal, DUMP_TIMEOUT_MS, chunk => { + output += chunk; + }); + if (code === 0) return reply(output.trim() || 'Recall dump saved'); + if (aborted) return reply('Recall dump cancelled'); + return reply(`Recall dump failed (exit ${code})${output.trim() ? `\n${output.trim()}` : ''}`); + } catch (error) { + return reply(`Recall dump failed: ${error instanceof Error ? error.message : String(error)}`); + } +} + export default function recallOmpExtension(pi: OmpExtensionAPI): void { pi.on('session_stop', (event, ctx) => captureOmpSessionStop(event, ctx, pi)); + const z = pi.zod; + if (!pi.registerTool || !z) return; + pi.registerTool({ + name: 'recall_dump', + label: 'Recall dump', + description: 'Save this omp conversation (its live branch) to Recall and write a Curated LoA entry. Use this for do-recall-dump inside omp.', + parameters: z.object({ + title: z.string().min(1).describe('Descriptive title for this session'), + project: z.string().optional().describe('Project name tag'), + tags: z.string().optional().describe('Comma-separated tags'), + continues: z.number().int().positive().optional().describe('Previous LoA entry id to chain from'), + skip_fabric: z.boolean().optional().describe('Write a basic summary instead of running the fabric Extractor'), + }), + execute: (_toolCallId, params, signal, _onUpdate, ctx) => dumpOmpSession(params, ctx, signal), + }); } diff --git a/hosts/plugins/recall-claude/skills/do-recall-dump/SKILL.md b/hosts/plugins/recall-claude/skills/do-recall-dump/SKILL.md index 249f7608..d6527a79 100644 --- a/hosts/plugins/recall-claude/skills/do-recall-dump/SKILL.md +++ b/hosts/plugins/recall-claude/skills/do-recall-dump/SKILL.md @@ -8,6 +8,8 @@ Dump the current conversation session into the Recall SQLite database, making al Run this at the end of every session or when you want to persist the current conversation mid-session. +**In omp:** call the `recall_dump` tool with the title instead of running `recall dump`; omp asks the user to confirm before anything is written. Inside omp the shell command refuses, because transcript files on disk belong to other conversations. + ## Usage ```bash diff --git a/hosts/plugins/recall/skills/do-recall-dump/SKILL.md b/hosts/plugins/recall/skills/do-recall-dump/SKILL.md index 3d843bf4..4d1b86f1 100644 --- a/hosts/plugins/recall/skills/do-recall-dump/SKILL.md +++ b/hosts/plugins/recall/skills/do-recall-dump/SKILL.md @@ -15,6 +15,8 @@ Dump the current conversation session into the Recall SQLite database, making al Run this at the end of every session or when you want to persist the current conversation mid-session. +**In omp:** call the `recall_dump` tool with the title instead of running `recall dump`; omp asks the user to confirm before anything is written. Inside omp the shell command refuses, because transcript files on disk belong to other conversations. + ## Usage ```bash diff --git a/src/commands/capture.ts b/src/commands/capture.ts index 690e1ae9..342e61d0 100644 --- a/src/commands/capture.ts +++ b/src/commands/capture.ts @@ -58,7 +58,7 @@ function usingFlagForm(options: CaptureCliOptions): boolean { ].some(value => value !== undefined); } -async function readStdin(maxBytes = MAX_CAPTURE_BYTES): Promise { +export async function readStdin(maxBytes = MAX_CAPTURE_BYTES): Promise { const chunks: Buffer[] = []; let bytes = 0; for await (const chunk of process.stdin) { diff --git a/src/commands/dump.ts b/src/commands/dump.ts index 1bf33f1d..43aa87d9 100644 --- a/src/commands/dump.ts +++ b/src/commands/dump.ts @@ -18,6 +18,8 @@ import { embed, embeddingToBlob, checkEmbeddingService } from '../lib/embeddings import { formatMessagesForExtraction, generateBasicSummary, runFabricExtract, ExtractorConfigError } from '../lib/extraction.js'; import { discoverCurrentSession } from '../hosts/session-sources.js'; import type { ParsedSession, SessionSource } from '../hosts/session-source.js'; +import { HARNESS_ID, readStdin } from './capture.js'; +import { basename } from 'path'; export type { ParsedSession, SessionSource } from '../hosts/session-source.js'; export { parseMarkdownDrop } from '../hosts/markdown-session-source.js'; @@ -28,6 +30,8 @@ interface DumpOptions { tags?: string; limit?: number; skipFabric?: boolean; + /** Read a host-supplied session (contract 1 JSON) from stdin instead of discovering one. */ + stdin?: boolean; /** Test/internal seam; normal CLI and MCP behavior still attempts LoA embedding. */ skipEmbed?: boolean; /** Test/internal seam; production uses runFabricExtract. */ @@ -44,6 +48,51 @@ interface DumpMessageRow { project: string | null; } +/** + * Host-supplied session (contract 1) for hosts whose live transcript only their own + * extension can read, e.g. omp's `recall_dump` tool. Malformed input is rejected whole. + */ +export function parseDumpSessionJson(raw: string): ParsedSession { + let value: unknown; + try { + value = JSON.parse(raw); + } catch { + throw new Error('dump --stdin expects contract 1 JSON'); + } + const payload = value as Record | null; + if (!payload || typeof payload !== 'object' || Array.isArray(payload) || payload.contract !== 1) { + throw new Error('dump --stdin expects contract 1 JSON'); + } + const { harness, session_id: sessionId, messages, cwd, project } = payload; + if (typeof harness !== 'string' || !HARNESS_ID.test(harness)) throw new Error('invalid harness'); + if (typeof sessionId !== 'string' || !sessionId || sessionId.length > 512 || /[\u0000-\u001f\u007f]/.test(sessionId)) { + throw new Error('invalid session_id'); + } + if (!Array.isArray(messages)) throw new Error('messages must be an array'); + const sessionProject = typeof project === 'string' && project + ? project + : typeof cwd === 'string' && cwd ? basename(cwd) : harness; + return { + source: harness, + sessionId, + project: sessionProject, + filePath: '', + messages: messages.map(message => { + const row = message as Record | null; + if (!row || (row.role !== 'user' && row.role !== 'assistant') || typeof row.content !== 'string' || !row.content.trim()) { + throw new Error('messages must be user/assistant rows with text content'); + } + return { + session_id: sessionId, + timestamp: typeof row.timestamp === 'string' && row.timestamp ? row.timestamp : new Date().toISOString(), + role: row.role, + content: row.content, + project: sessionProject, + }; + }), + }; +} + // ============ Internal Helpers ============ async function autoEmbedLoaEntry(id: number, title: string, fabricExtract: string): Promise { @@ -197,7 +246,10 @@ export async function coreDump(title: string, options: DumpOptions & { session?: const session = options.session || discoverCurrentSession(); if (!session) { - return { success: false, sessionId: '', messageCount: 0, source: 'mcp', error: 'No session input or supported host session files found' }; + const error = process.env.OMPCODE === '1' + ? 'Inside omp, use the recall_dump tool: the omp extension supplies the live session' + : 'No session input or supported host session files found'; + return { success: false, sessionId: '', messageCount: 0, source: 'mcp', error }; } if (options.project) { @@ -369,7 +421,14 @@ export async function runDump(title: string, options: DumpOptions): Promise', 'Comma-separated tags') .option('-n, --limit ', 'Max messages to process') .option('--skip-fabric', 'Skip the fabric Extractor and write a basic summary') + .option('--stdin', 'Read the session as contract 1 JSON from stdin (host extensions, e.g. omp recall_dump)') .action(async (title, options) => { await runDump(title, { project: options.project, continues: options.continues ? parseInt(options.continues, 10) : undefined, tags: options.tags, limit: options.limit ? parseInt(options.limit, 10) : undefined, - skipFabric: options.skipFabric + skipFabric: options.skipFabric, + stdin: options.stdin, }); closeDb(); }); diff --git a/tests/api.test.ts b/tests/api.test.ts index 5f7e4f9a..b8a82934 100644 --- a/tests/api.test.ts +++ b/tests/api.test.ts @@ -17,6 +17,7 @@ import { registeredSessionSources, runStart, } from '../src/api'; +import { withEnv } from './helpers/env'; const REPO = join(import.meta.dir, '..'); @@ -94,7 +95,7 @@ describe('recall-memory/api harness seams', () => { }; disposers.push(registerSessionSource({ id: 'acme', discover: () => session })); expect(registeredSessionSources().map(adapter => adapter.id)).toEqual(['acme']); - expect(discoverCurrentSession(registeredSessionSources())?.sessionId).toBe('ses-acme'); + expect(withEnv('OMPCODE', undefined, () => discoverCurrentSession(registeredSessionSources()))?.sessionId).toBe('ses-acme'); const dumpSource = readFileSync(join(REPO, 'src', 'hosts', 'session-sources.ts'), 'utf-8'); expect(dumpSource).toContain('registeredSessionSources()'); }); diff --git a/tests/commands/dump-portable.test.ts b/tests/commands/dump-portable.test.ts index 6759dbe9..94d79231 100644 --- a/tests/commands/dump-portable.test.ts +++ b/tests/commands/dump-portable.test.ts @@ -1,5 +1,7 @@ import { afterAll, beforeAll, describe, expect, test } from 'bun:test'; -import { coreDump } from '../../src/commands/dump'; +import { coreDump, parseDumpSessionJson } from '../../src/commands/dump'; +import { dumpOmpSession } from '../../hosts/omp/recall'; +import { withEnv } from '../helpers/env'; import { getDb } from '../../src/db/connection'; import { createLoaEntry, getLoaMessages } from '../../src/lib/memory'; import { ingestHostTranscript } from '../../src/lib/host-ingest'; @@ -335,4 +337,52 @@ describe('portable explicit session dump', () => { ).all(sessionId) as Array<{ fabric_extract: string }>; expect(loa).toEqual([]); }); + + test('omp recall_dump payload stores the live branch under omp, not a discovered transcript', async () => { + let stdin = ''; + let args: readonly string[] = []; + const reply = await dumpOmpSession({ title: 'omp live branch', skip_fabric: true }, { + cwd: '/work/Recall', + hasUI: true, + ui: { notify: () => {}, confirm: async () => true }, + sessionManager: { + getSessionId: () => 'omp-live-session', + getBranch: () => [ + { type: 'message', id: 'u1', timestamp: '2026-10-14T12:00:00.000Z', message: { role: 'user', content: 'dump this omp question' } }, + { type: 'message', id: 't1', message: { role: 'toolResult', content: [{ type: 'text', text: 'tool noise' }] } }, + { type: 'message', id: 'a1', message: { role: 'assistant', content: [{ type: 'text', text: 'omp answer' }] } }, + ], + }, + }, undefined, async (_file, childArgs, childStdin, _signal, onOutput) => { + args = childArgs; + stdin = childStdin; + onOutput?.('✓ LoA #7 captured'); + return 0; + }); + expect(args.slice(1)).toEqual(['dump', 'omp live branch', '--stdin', '--skip-fabric']); + expect(reply.content[0].text).toBe('✓ LoA #7 captured'); + + const result = await coreDump('omp live branch', { skipFabric: true, skipEmbed: true, session: parseDumpSessionJson(stdin) }); + expect(result).toMatchObject({ success: true, source: 'omp', sessionId: 'omp-live-session', messageCount: 2 }); + const rows = getDb().prepare('SELECT role, content, project FROM messages WHERE session_id = ? ORDER BY id') + .all('omp-live-session'); + expect(rows).toEqual([ + { role: 'user', content: 'dump this omp question', project: 'Recall' }, + { role: 'assistant', content: 'omp answer', project: 'Recall' }, + ]); + }); + + test('inside omp, dump without a supplied session refuses and points at recall_dump', async () => { + const result = await withEnv('OMPCODE', '1', () => coreDump('no omp input', { skipFabric: true, skipEmbed: true })); + expect(result.success).toBe(false); + expect(result.error).toContain('recall_dump'); + }); + + test('dump --stdin rejects malformed sessions whole', () => { + const base = { contract: 1, harness: 'omp', session_id: 's' }; + expect(() => parseDumpSessionJson('not json')).toThrow('contract 1'); + expect(() => parseDumpSessionJson(JSON.stringify({ ...base, contract: 2, messages: [] }))).toThrow('contract 1'); + expect(() => parseDumpSessionJson(JSON.stringify({ ...base, harness: 'Bad Id', messages: [] }))).toThrow('harness'); + expect(() => parseDumpSessionJson(JSON.stringify({ ...base, messages: [{ role: 'system', content: 'x' }] }))).toThrow('user/assistant'); + }); }); diff --git a/tests/helpers/env.ts b/tests/helpers/env.ts new file mode 100644 index 00000000..66373d32 --- /dev/null +++ b/tests/helpers/env.ts @@ -0,0 +1,12 @@ +/** Run `fn` with env `name` set to `value` (undefined = unset), restoring it afterwards. */ +export function withEnv(name: string, value: string | undefined, fn: () => T): T { + const saved = process.env[name]; + if (value === undefined) delete process.env[name]; + else process.env[name] = value; + try { + return fn(); + } finally { + if (saved === undefined) delete process.env[name]; + else process.env[name] = saved; + } +} diff --git a/tests/hosts/native-hosts.test.ts b/tests/hosts/native-hosts.test.ts index 19614e9e..3a5f5df8 100644 --- a/tests/hosts/native-hosts.test.ts +++ b/tests/hosts/native-hosts.test.ts @@ -7,6 +7,7 @@ import { claudePaths } from '../../src/hosts/claude'; import { extractClaudeProjectFromPath } from '../../src/hosts/claude-session-source'; import { parseMarkdownDrop } from '../../src/hosts/markdown-session-source'; import { discoverCurrentSession } from '../../src/hosts/session-sources'; +import { withEnv } from '../helpers/env'; describe('native host boundaries', () => { test('each verified host owns its MCP config shape', () => { @@ -23,7 +24,7 @@ describe('native host boundaries', () => { test('session discovery is adapter ordered and stops at the first supported transcript', () => { const calls: string[] = []; - const session = discoverCurrentSession([ + const session = withEnv('OMPCODE', undefined, () => discoverCurrentSession([ { id: 'claude', discover: () => { calls.push('claude'); return null; } }, { id: 'codex', @@ -39,7 +40,7 @@ describe('native host boundaries', () => { }, }, { id: 'pi', discover: () => { calls.push('pi'); return null; } }, - ]); + ])); expect(session?.source).toBe('codex'); expect(calls).toEqual(['claude', 'codex']); }); @@ -64,4 +65,10 @@ describe('native host boundaries', () => { rmSync(root, { recursive: true, force: true }); } }); + + test('inside omp, file discovery fails closed instead of returning another host session', () => { + const foreign = { id: 'claude', discover: () => ({ source: 'claude', sessionId: 'foreign', project: 'x', filePath: 'x.jsonl', messages: [] }) }; + expect(withEnv('OMPCODE', '1', () => discoverCurrentSession([foreign]))).toBeNull(); + expect(withEnv('OMPCODE', undefined, () => discoverCurrentSession([foreign]))?.sessionId).toBe('foreign'); + }); }); diff --git a/tests/omp-integration.test.ts b/tests/omp-integration.test.ts index 213ccb82..4f091ee0 100644 --- a/tests/omp-integration.test.ts +++ b/tests/omp-integration.test.ts @@ -1,6 +1,7 @@ import { describe, expect, test } from 'bun:test'; import { captureOmpSessionStop, + dumpOmpSession, MAX_OMP_STDIN_BYTES, runBoundedChild, type OmpExtensionContext, @@ -205,4 +206,16 @@ describe('omp native extension', () => { ); expect(messages).toEqual(['Recall omp capture cancelled']); }); + + test('recall_dump writes nothing unless the user confirms in the UI', async () => { + let spawned = 0; + const child = async () => { spawned++; return 0; }; + const declined = await dumpOmpSession({ title: 't' }, uiCtx(() => {}, { + ui: { notify: () => {}, confirm: async () => false }, + }), undefined, child); + const headless = await dumpOmpSession({ title: 't' }, uiCtx(() => {}, { hasUI: false }), undefined, child); + expect(spawned).toBe(0); + expect(declined.content[0].text).toContain('declined'); + expect(headless.content[0].text).toContain('nothing was saved'); + }); }); From c2aeb3405a8dd3282873e04290005d3bf41b473d Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Wed, 14 Oct 2026 14:53:50 -0400 Subject: [PATCH 02/11] no-mistakes(review): Fix dump scrubbing, project attribution, and behavioral coverage --- src/commands/dump.ts | 17 +++++++-- tests/api.test.ts | 2 - tests/commands/dump-portable.test.ts | 56 +++++++++++++++++++++++++++- 3 files changed, 68 insertions(+), 7 deletions(-) diff --git a/src/commands/dump.ts b/src/commands/dump.ts index 43aa87d9..d047f874 100644 --- a/src/commands/dump.ts +++ b/src/commands/dump.ts @@ -19,6 +19,8 @@ import { formatMessagesForExtraction, generateBasicSummary, runFabricExtract, Ex import { discoverCurrentSession } from '../hosts/session-sources.js'; import type { ParsedSession, SessionSource } from '../hosts/session-source.js'; import { HARNESS_ID, readStdin } from './capture.js'; +import { detectProject } from '../lib/project.js'; +import { scrub } from '../lib/write-safety.js'; import { basename } from 'path'; export type { ParsedSession, SessionSource } from '../hosts/session-source.js'; @@ -71,7 +73,7 @@ export function parseDumpSessionJson(raw: string): ParsedSession { if (!Array.isArray(messages)) throw new Error('messages must be an array'); const sessionProject = typeof project === 'string' && project ? project - : typeof cwd === 'string' && cwd ? basename(cwd) : harness; + : typeof cwd === 'string' && cwd ? detectProject(cwd) || basename(cwd) : harness; return { source: harness, sessionId, @@ -243,15 +245,23 @@ export async function coreDump(title: string, options: DumpOptions & { session?: source: string; error?: string; }> { - const session = options.session || discoverCurrentSession(); + const inputSession = options.session || discoverCurrentSession(); - if (!session) { + if (!inputSession) { const error = process.env.OMPCODE === '1' ? 'Inside omp, use the recall_dump tool: the omp extension supplies the live session' : 'No session input or supported host session files found'; return { success: false, sessionId: '', messageCount: 0, source: 'mcp', error }; } + const session = { + ...inputSession, + messages: inputSession.messages.flatMap(message => { + const content = scrub(message.content).text; + return content.trim() ? [{ ...message, content }] : []; + }), + }; + if (options.project) { for (const msg of session.messages) { msg.project = options.project; @@ -303,7 +313,6 @@ export async function coreDump(title: string, options: DumpOptions & { session?: }); } - // Raw conversation capture is verbatim (ADR-0001). const messagesToImport = existingSnapshot ? [] : session.messages; const importedCount = addMessagesBatch( messagesToImport.map(message => ({ ...message, provenance: 'verbatim' as const })) diff --git a/tests/api.test.ts b/tests/api.test.ts index b8a82934..b38f1a27 100644 --- a/tests/api.test.ts +++ b/tests/api.test.ts @@ -96,8 +96,6 @@ describe('recall-memory/api harness seams', () => { disposers.push(registerSessionSource({ id: 'acme', discover: () => session })); expect(registeredSessionSources().map(adapter => adapter.id)).toEqual(['acme']); expect(withEnv('OMPCODE', undefined, () => discoverCurrentSession(registeredSessionSources()))?.sessionId).toBe('ses-acme'); - const dumpSource = readFileSync(join(REPO, 'src', 'hosts', 'session-sources.ts'), 'utf-8'); - expect(dumpSource).toContain('registeredSessionSources()'); }); test('drop seam is the MEMORY/-sessions filesystem convention', () => { diff --git a/tests/commands/dump-portable.test.ts b/tests/commands/dump-portable.test.ts index 94d79231..6a549142 100644 --- a/tests/commands/dump-portable.test.ts +++ b/tests/commands/dump-portable.test.ts @@ -3,7 +3,7 @@ import { coreDump, parseDumpSessionJson } from '../../src/commands/dump'; import { dumpOmpSession } from '../../hosts/omp/recall'; import { withEnv } from '../helpers/env'; import { getDb } from '../../src/db/connection'; -import { createLoaEntry, getLoaMessages } from '../../src/lib/memory'; +import { createLoaEntry, getLoaMessages, search } from '../../src/lib/memory'; import { ingestHostTranscript } from '../../src/lib/host-ingest'; import { ExtractorConfigError } from '../../src/lib/extraction'; import { setupTestDb, teardownTestDb } from '../helpers/setup'; @@ -51,6 +51,60 @@ describe('portable explicit session dump', () => { expect(session).toEqual({ source: 'codex', project: 'recall-test' }); }); + test('scrubs dump messages before storage, summaries, extraction, and snapshot matching', async () => { + const sessionId = 'omp-scrubbed-dump'; + const secret = 'abcdefgh' + '12345678'; + ingestHostTranscript({ + source: 'omp', + sessionId, + project: 'recall-test', + messages: [{ role: 'user', content: 'Automatic capture stays separate.', sourcePosition: 0 }], + }); + const session = parseDumpSessionJson(JSON.stringify({ + contract: 1, + harness: 'omp', + session_id: sessionId, + project: 'recall-test', + messages: [ + { role: 'user', content: `api_key=${secret}` }, + { role: 'assistant', content: ' \u200b\u202e ' }, + { role: 'assistant', content: 'Safe answer.' }, + ], + })); + const options = { session, skipFabric: true, skipEmbed: true }; + const first = await coreDump('Scrubbed omp dump', options); + expect(first).toMatchObject({ success: true, source: 'omp', messageCount: 2 }); + const db = getDb(); + expect(db.prepare('SELECT content FROM published_messages WHERE session_id = ? ORDER BY id').all(sessionId)) + .toEqual([ + { content: 'Automatic capture stays separate.' }, + { content: 'api_key=[REDACTED:generic-assignment]' }, + { content: 'Safe answer.' }, + ]); + expect(search(secret, { table: 'messages' })).toEqual([]); + const loa = db.prepare('SELECT fabric_extract, message_count FROM loa_entries WHERE id = ?') + .get(first.loaId!) as { fabric_extract: string; message_count: number }; + expect(loa.message_count).toBe(2); + expect(loa.fabric_extract).toContain('api_key=[REDACTED:generic-assignment]'); + expect(loa.fabric_extract).toContain('Safe answer.'); + expect(loa.fabric_extract).not.toContain(secret); + let extractionInput = ''; + const second = await coreDump('Scrubbed omp dump', { + ...options, + skipFabric: false, + extract: content => { + extractionInput = content; + return 'Curated safe extraction.'; + }, + }); + expect(second).toMatchObject({ success: true, messageCount: 0, loaId: first.loaId }); + expect(extractionInput).toContain('api_key=[REDACTED:generic-assignment]'); + expect(extractionInput).toContain('Safe answer.'); + expect(extractionInput).not.toContain(secret); + expect(extractionInput).not.toContain('\u200b'); + expect(session.messages[0].content).toBe(`api_key=${secret}`); + }); + test('re-import preserves automatic lifecycle capture for the same session', async () => { const sessionId = 'codex-lifecycle-reimport'; const lifecycleContent = 'Preserve this lifecycle message across explicit re-imports.'; From a39eb31186c910434ff093cf2f1f3489ed67e634 Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Wed, 14 Oct 2026 15:07:25 -0400 Subject: [PATCH 03/11] no-mistakes(review): Propagate omp dump cancellation through confirmation and dispatch --- hosts/omp/recall.ts | 14 ++++-- tests/omp-integration.test.ts | 94 +++++++++++++++++++++++++++++++++++ 2 files changed, 104 insertions(+), 4 deletions(-) diff --git a/hosts/omp/recall.ts b/hosts/omp/recall.ts index 4ae8ebcc..368b9d19 100644 --- a/hosts/omp/recall.ts +++ b/hosts/omp/recall.ts @@ -21,7 +21,7 @@ export interface OmpExtensionContext { hasUI?: boolean; ui?: { notify: (message: string, type?: 'info' | 'warning' | 'error') => void; - confirm?: (title: string, message: string) => Promise; + confirm?: (title: string, message: string, options?: { signal?: AbortSignal }) => Promise; }; sessionManager: { getSessionId: () => string; @@ -100,6 +100,8 @@ export function runBoundedChild( signal: AbortSignal, onOutput?: (chunk: string) => void, ): Promise { + if (signal.aborted) return Promise.resolve(null); + const { promise, resolve, reject } = Promise.withResolvers(); let settled = false; let killTimer: ReturnType | undefined; @@ -142,8 +144,7 @@ export function runBoundedChild( }); child.stdin?.end(stdin, 'utf8'); - if (signal.aborted) onAbort(); - else signal.addEventListener('abort', onAbort, { once: true }); + signal.addEventListener('abort', onAbort, { once: true }); return promise; } @@ -275,6 +276,8 @@ async function runCli( timeoutMs: number, onOutput?: (chunk: string) => void, ): Promise<{ code: number | null; aborted: boolean }> { + if (outer?.aborted) return { code: null, aborted: true }; + const timeout = new AbortController(); const timer = setTimeout(() => timeout.abort(), timeoutMs); timer.unref(); @@ -304,10 +307,12 @@ export async function dumpOmpSession( ): Promise { const reply = (text: string): OmpToolResult => ({ content: [{ type: 'text', text }] }); try { + if (signal?.aborted) return reply('Recall dump cancelled'); // Dumping is always the user's call (agent-skills/AGENTS.md): the model can request // it, but nothing is written without a yes in the omp UI. Headless sessions refuse. if (!ctx.hasUI || !ctx.ui?.confirm) return reply('Recall dump needs the interactive omp UI to confirm; nothing was saved'); - const approved = await ctx.ui.confirm('Save this conversation to Recall?', `recall dump "${params.title}"`); + const approved = await ctx.ui.confirm('Save this conversation to Recall?', `recall dump "${params.title}"`, { signal }); + if (signal?.aborted) return reply('Recall dump cancelled'); if (!approved) return reply('Recall dump declined; nothing was saved'); const sessionId = ctx.sessionManager.getSessionId(); if (!sessionId) return reply('Recall dump failed: omp did not report a session id'); @@ -332,6 +337,7 @@ export async function dumpOmpSession( if (aborted) return reply('Recall dump cancelled'); return reply(`Recall dump failed (exit ${code})${output.trim() ? `\n${output.trim()}` : ''}`); } catch (error) { + if (signal?.aborted) return reply('Recall dump cancelled'); return reply(`Recall dump failed: ${error instanceof Error ? error.message : String(error)}`); } } diff --git a/tests/omp-integration.test.ts b/tests/omp-integration.test.ts index 4f091ee0..3b9701dd 100644 --- a/tests/omp-integration.test.ts +++ b/tests/omp-integration.test.ts @@ -218,4 +218,98 @@ describe('omp native extension', () => { expect(declined.content[0].text).toContain('declined'); expect(headless.content[0].text).toContain('nothing was saved'); }); + + test('already cancelled dump skips confirmation and child dispatch', async () => { + let confirmed = false; + let spawned = false; + const result = await dumpOmpSession({ title: 't' }, uiCtx(() => {}, { + ui: { notify: () => {}, confirm: async () => { confirmed = true; return true; } }, + }), AbortSignal.abort(), async () => { spawned = true; return 0; }); + expect(confirmed).toBe(false); + expect(spawned).toBe(false); + expect(result.content[0].text).toBe('Recall dump cancelled'); + }); + + test('abort dismisses pending confirmation and finishes the dump without spawning', async () => { + const ac = new AbortController(); + const confirmation = Promise.withResolvers(); + let dialogOpen = false; + let spawned = false; + const pending = dumpOmpSession({ title: 't' }, uiCtx(() => {}, { + ui: { + notify: () => {}, + confirm: (_title, _message, options) => { + dialogOpen = true; + options?.signal?.addEventListener('abort', () => { + dialogOpen = false; + confirmation.resolve(false); + }, { once: true }); + return confirmation.promise; + }, + }, + }), ac.signal, async () => { spawned = true; return 0; }); + try { + expect(dialogOpen).toBe(true); + ac.abort(); + expect(dialogOpen).toBe(false); + const result = await pending; + expect(spawned).toBe(false); + expect(result.content[0].text).toBe('Recall dump cancelled'); + } finally { + ac.abort(); + confirmation.resolve(false); + await pending; + } + }); + + test.each(['approve', 'reject'] as const)('cancellation takes precedence when confirmation settles with %s', async outcome => { + const ac = new AbortController(); + let spawned = false; + const result = await dumpOmpSession({ title: 't' }, uiCtx(() => {}, { + ui: { + notify: () => {}, + confirm: async () => { + ac.abort(); + if (outcome === 'reject') throw ac.signal.reason; + return true; + }, + }, + }), ac.signal, async () => { spawned = true; return 0; }); + expect(spawned).toBe(false); + expect(result.content[0].text).toBe('Recall dump cancelled'); + }); + + test.each(['capture', 'dump'] as const)('%s skips child dispatch when branch reading aborts the signal', async action => { + const ac = new AbortController(); + const messages: string[] = []; + let spawned = false; + const ctx = uiCtx(message => messages.push(message), { + ui: { notify: message => messages.push(message), confirm: async () => true }, + sessionManager: { + getSessionId: () => 'sess-cancelled', + getBranch: () => { + ac.abort(); + return [{ type: 'message', id: 'u1', message: { role: 'user', content: 'hi' } }]; + }, + }, + }); + const child = async () => { spawned = true; return 0; }; + if (action === 'capture') { + await captureOmpSessionStop({ signal: ac.signal }, ctx, undefined, child); + expect(messages).toEqual(['Recall omp capture cancelled']); + } else { + const result = await dumpOmpSession({ title: 't' }, ctx, ac.signal, child); + expect(result.content[0].text).toBe('Recall dump cancelled'); + } + expect(spawned).toBe(false); + }); + + test('already cancelled bounded child never attempts to execute a missing command', async () => { + await expect(runBoundedChild( + `${import.meta.dir}/missing-omp-child`, + [], + '', + AbortSignal.abort(), + )).resolves.toBeNull(); + }); }); From cfe675a2d015814a20f383795ed781148077ce7a Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Wed, 14 Oct 2026 15:33:56 -0400 Subject: [PATCH 04/11] no-mistakes(document): Clarify dump contracts and consolidate credential guidance --- agent-skills/do-recall-dump/SKILL.md | 2 +- docs/OMP_INTEGRATION.md | 2 ++ docs/agent-skills.md | 2 +- docs/api.md | 2 +- docs/cli-reference.md | 8 +++++++- docs/mcp-tools.md | 2 ++ .../plugins/recall-claude/skills/do-recall-dump/SKILL.md | 2 +- hosts/plugins/recall/skills/do-recall-dump/SKILL.md | 2 +- 8 files changed, 16 insertions(+), 6 deletions(-) diff --git a/agent-skills/do-recall-dump/SKILL.md b/agent-skills/do-recall-dump/SKILL.md index d6527a79..514713fb 100644 --- a/agent-skills/do-recall-dump/SKILL.md +++ b/agent-skills/do-recall-dump/SKILL.md @@ -8,7 +8,7 @@ Dump the current conversation session into the Recall SQLite database, making al Run this at the end of every session or when you want to persist the current conversation mid-session. -**In omp:** call the `recall_dump` tool with the title instead of running `recall dump`; omp asks the user to confirm before anything is written. Inside omp the shell command refuses, because transcript files on disk belong to other conversations. +**In omp:** call `recall_dump` with the title. See the [omp dump contract](https://github.com/edheltzel/Recall/blob/main/docs/OMP_INTEGRATION.md#explicit-dump-recall_dump-tool) for confirmation and session-source rules. ## Usage diff --git a/docs/OMP_INTEGRATION.md b/docs/OMP_INTEGRATION.md index 0f79ef1b..14d73032 100644 --- a/docs/OMP_INTEGRATION.md +++ b/docs/OMP_INTEGRATION.md @@ -40,6 +40,8 @@ Capture sends only user and assistant text, at most 25 MiB per payload, and boun The extension also registers a `recall_dump` tool (title, optional project, tags, `continues`, `skip_fabric`). It reads the live branch and session id from omp, so it dumps this conversation on any profile or branch, and sends it to the package-local `dist/index.js dump --stdin`. The tool asks you to confirm in the omp UI before writing anything and refuses in headless sessions. `recall dump` run from a shell inside omp (`OMPCODE=1`) refuses and points to the tool: transcript files on disk belong to other conversations. +Dump payloads are limited to 25 MiB, and the CLI child has a five-minute timeout. `skip_fabric` defaults to false; set it to true for a basic summary. Shared message scrubbing and default project attribution follow the [Session Dump contract](cli-reference.md#session-dump). + ## Skills and MCP remain separate `recall install` still links the nine canonical `do-recall-*` skills into `~/.omp/agent/skills/` when omp is detected. It does not activate this extension. The native package does not duplicate those skills, register MCP, or inject session-start memory; `recall_dump` is its only tool. Existing manual MCP configuration remains untouched. diff --git a/docs/agent-skills.md b/docs/agent-skills.md index 08ae3049..fa974372 100644 --- a/docs/agent-skills.md +++ b/docs/agent-skills.md @@ -97,4 +97,4 @@ Skill names use the canonical `do-recall-*` namespace. Claude and Codex load the The Codex plugin adapters are generated with `bun run build:codex-plugin` and packaged with the native plugin bundle documented in [Codex Integration](CODEX_INTEGRATION.md). Claude copies are generated with `bun run build:claude-plugin`; see [Claude Integration](CLAUDE_INTEGRATION.md). -If installer-linked skills are missing after an update, re-run `./packaging/install.sh` to relink them. These links are omp's supported skill path; its native extension handles capture only. Claude, Codex, and Pi prefer their native attachments. +If installer-linked skills are missing after an update, re-run `./packaging/install.sh` to relink them. See [omp Integration](OMP_INTEGRATION.md#skills-and-mcp-remain-separate) for its skill and extension ownership. Claude, Codex, and Pi prefer their native attachments. diff --git a/docs/api.md b/docs/api.md index 9f9345a7..a00f725c 100644 --- a/docs/api.md +++ b/docs/api.md @@ -34,7 +34,7 @@ bun -e 'import { describeHarnessSeams } from "./src/api.ts"; console.log(JSON.st |------|-------------------|---------------| | **start** | `recall start` renders L0/L1 via `hooks/lib/session-start-context.ts`. `--format cursor` wraps `{ additional_context }`. | `registerStartFormat(id, wrap)` — in-process, for `runStart({ format: id })`. The public CLI still accepts only `markdown` \| `cursor`. | | **drop** | Write markdown into `MEMORY/-sessions/`. Batch extract scans `*-sessions` generically. | None. The directory name **is** the extension point. `parseMarkdownDrop` / `markdownDropDirName` are the helpers. | -| **capture** | Cursor: `catalogCursorSessions()` (catalog only, no SQLite insert). Dump: `discoverCurrentSession()`. Ambient harness text: `recall capture` ([Capture adapter](CAPTURE_ADAPTER.md)). Hidden `host-hook` is deprecated for ambient writes; Codex `SessionStart` still uses it for injection only. | `registerSessionSource(adapter)` — in-process, consulted by `discoverCurrentSession()`. Cannot replace builtin `SESSION_SOURCES` ids. | +| **capture** | Cursor: `catalogCursorSessions()` (catalog only, no SQLite insert). Dump: `discoverCurrentSession()` ([omp restriction](OMP_INTEGRATION.md#explicit-dump-recall_dump-tool)). Ambient harness text: `recall capture` ([Capture adapter](CAPTURE_ADAPTER.md)). Hidden `host-hook` is deprecated for ambient writes; Codex `SessionStart` still uses it for injection only. | `registerSessionSource(adapter)` — in-process, consulted by `discoverCurrentSession()`. Cannot replace builtin `SESSION_SOURCES` ids. | | **inject** | Cursor snippets under `templates/cursor/` call unqualified `recall start --format cursor`. Claude/Codex call the same assembler through their own hooks. | `mergeCursorHooksJson` is the Cursor hooks.json helper. Do not add a marketplace plugin. | MCP (`recall-mcp`, server name `recall-memory`) and Agent Skills (`agent-skills/`) are already cross-host. Point at [MCP Tools](mcp-tools.md) and [Agent Skills](agent-skills.md); do not fork those catalogs. diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 4ef11f2e..a7afd78f 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -78,7 +78,13 @@ Prove a write with `recall search "" -t messages`. recall dump "Session Title" # Import session + capture LoA entry ``` -Combines `recall import` and Curated LoA capture in one step. Run this at the end of every session to persist the conversation and run the `fabric` Extractor. The title should describe what was accomplished in the session. `--skip-fabric` writes a basic summary instead. `--stdin` reads the session as contract 1 JSON (`harness`, `session_id`, optional `cwd`/`project`, `messages` of user/assistant `role`/`content`/`timestamp`) for host extensions that hold the live transcript; inside omp, use the [`recall_dump` tool](OMP_INTEGRATION.md#explicit-dump-recall_dump-tool) instead of the shell command. +Combines `recall import` and Curated LoA capture in one step. Run this at the end of every session to persist the conversation and run the `fabric` Extractor. The title should describe what was accomplished in the session. `--skip-fabric` writes a basic summary instead. + +Without `--stdin`, Recall discovers native transcripts in Claude Code, OpenCode, then Pi order, followed by registered session sources. For omp's live-session route and discovery restriction, see the [`recall_dump` contract](OMP_INTEGRATION.md#explicit-dump-recall_dump-tool). + +`--stdin` reads up to 25 MiB of contract 1 JSON for host extensions that hold the live transcript: `contract: 1`, `harness`, `session_id`, optional `cwd`/`project`, and `messages` with user/assistant `role`, nonblank text `content`, and optional `timestamp`. Malformed input is rejected before writes. `-p` overrides the payload's project; otherwise Recall uses its `project`, then the origin repository name or directory basename from `cwd`, then `harness` if `cwd` is absent. + +Both CLI dumps and MCP `memory_dump` scrub each message's content with the shared write-safety guard before snapshot matching, storage, and summary or extraction. Known credential patterns become visible `[REDACTED:]` markers, invisible/bidi controls are removed, and messages left blank are dropped. This does not detect every secret; follow the [credential-handling rule](hosts/FOR_CLAUDE.md#core-rules) before dumping sensitive conversations. ### Library of Alexandria (LoA) diff --git a/docs/mcp-tools.md b/docs/mcp-tools.md index a660571d..b63f0f45 100644 --- a/docs/mcp-tools.md +++ b/docs/mcp-tools.md @@ -188,6 +188,8 @@ memory_dump({ `memory_dump` is an explicit operation, not lifecycle auto-capture. MCP has no portable event contract for session stop, session start, or pre-compaction. +Message scrubbing follows the shared [Session Dump contract](cli-reference.md#session-dump). Inside omp, use the native [`recall_dump` tool](OMP_INTEGRATION.md#explicit-dump-recall_dump-tool) for the live branch. + When `session_id` already belongs to lifecycle capture, a dump from the same source is supplemental: Recall preserves lifecycle-owned rows and their automatic summary. A conflicting source is rejected rather than taking over the session. --- diff --git a/hosts/plugins/recall-claude/skills/do-recall-dump/SKILL.md b/hosts/plugins/recall-claude/skills/do-recall-dump/SKILL.md index d6527a79..514713fb 100644 --- a/hosts/plugins/recall-claude/skills/do-recall-dump/SKILL.md +++ b/hosts/plugins/recall-claude/skills/do-recall-dump/SKILL.md @@ -8,7 +8,7 @@ Dump the current conversation session into the Recall SQLite database, making al Run this at the end of every session or when you want to persist the current conversation mid-session. -**In omp:** call the `recall_dump` tool with the title instead of running `recall dump`; omp asks the user to confirm before anything is written. Inside omp the shell command refuses, because transcript files on disk belong to other conversations. +**In omp:** call `recall_dump` with the title. See the [omp dump contract](https://github.com/edheltzel/Recall/blob/main/docs/OMP_INTEGRATION.md#explicit-dump-recall_dump-tool) for confirmation and session-source rules. ## Usage diff --git a/hosts/plugins/recall/skills/do-recall-dump/SKILL.md b/hosts/plugins/recall/skills/do-recall-dump/SKILL.md index 4d1b86f1..849f5102 100644 --- a/hosts/plugins/recall/skills/do-recall-dump/SKILL.md +++ b/hosts/plugins/recall/skills/do-recall-dump/SKILL.md @@ -15,7 +15,7 @@ Dump the current conversation session into the Recall SQLite database, making al Run this at the end of every session or when you want to persist the current conversation mid-session. -**In omp:** call the `recall_dump` tool with the title instead of running `recall dump`; omp asks the user to confirm before anything is written. Inside omp the shell command refuses, because transcript files on disk belong to other conversations. +**In omp:** call `recall_dump` with the title. See the [omp dump contract](https://github.com/edheltzel/Recall/blob/main/docs/OMP_INTEGRATION.md#explicit-dump-recall_dump-tool) for confirmation and session-source rules. ## Usage From 9760aa57fdae3733c16c3b0e87b2de5c53b99a67 Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Wed, 14 Oct 2026 20:17:10 -0400 Subject: [PATCH 05/11] fix(dump): reject empty sessions before clearing an existing dump An empty supplied session (dump --stdin with no messages, or text that scrubs to nothing) reached coreDump, which cleared a non-lifecycle session's messages before finding nothing to import. --- src/commands/dump.ts | 5 +++++ tests/commands/dump-portable.test.ts | 14 ++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/src/commands/dump.ts b/src/commands/dump.ts index d047f874..7072daa5 100644 --- a/src/commands/dump.ts +++ b/src/commands/dump.ts @@ -268,6 +268,11 @@ export async function coreDump(title: string, options: DumpOptions & { session?: } } + // Checked before any write: replacing a session with nothing would clear its messages. + if (session.messages.length === 0) { + return { success: false, sessionId: session.sessionId, messageCount: 0, source: session.source, error: 'Session has no user or assistant text to dump' }; + } + const replacingSession = sessionExists(session.sessionId); const lifecycleSource = replacingSession ? lifecycleSessionSource(session.sessionId) diff --git a/tests/commands/dump-portable.test.ts b/tests/commands/dump-portable.test.ts index 6a549142..35afe545 100644 --- a/tests/commands/dump-portable.test.ts +++ b/tests/commands/dump-portable.test.ts @@ -439,4 +439,18 @@ describe('portable explicit session dump', () => { expect(() => parseDumpSessionJson(JSON.stringify({ ...base, harness: 'Bad Id', messages: [] }))).toThrow('harness'); expect(() => parseDumpSessionJson(JSON.stringify({ ...base, messages: [{ role: 'system', content: 'x' }] }))).toThrow('user/assistant'); }); + + test('an empty supplied session is rejected without clearing the existing dump', async () => { + const sessionId = 'empty-replacement'; + const message = { session_id: sessionId, timestamp: '2026-10-14T00:00:00.000Z', role: 'user' as const, content: 'keep me', project: 'p' }; + const session = { source: 'codex', sessionId, project: 'p', filePath: '', messages: [message] }; + const first = await coreDump('first', { skipFabric: true, skipEmbed: true, session }); + expect(first.success).toBe(true); + + const empty = await coreDump('empty', { skipFabric: true, skipEmbed: true, session: parseDumpSessionJson(JSON.stringify({ contract: 1, harness: 'codex', session_id: sessionId, messages: [] })) }); + expect(empty.success).toBe(false); + const rows = getDb().prepare('SELECT content FROM messages WHERE session_id = ?').all(sessionId); + expect(rows).toEqual([{ content: 'keep me' }]); + expect(getDb().prepare('SELECT COUNT(*) AS n FROM loa_entries WHERE id = ?').get(first.loaId!)).toEqual({ n: 1 }); + }); }); From 22722c8a5c7c41e0c45900a109352e45004cd159 Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Thu, 15 Oct 2026 16:15:14 -0400 Subject: [PATCH 06/11] fix(dump): pass omp titles after -- and scrub legacy snapshots before matching --- hosts/omp/recall.ts | 4 ++- src/commands/dump.ts | 23 ++++++++++++---- tests/commands/dump-portable.test.ts | 41 ++++++++++++++++++++++++++-- 3 files changed, 58 insertions(+), 10 deletions(-) diff --git a/hosts/omp/recall.ts b/hosts/omp/recall.ts index 368b9d19..b6e2ff72 100644 --- a/hosts/omp/recall.ts +++ b/hosts/omp/recall.ts @@ -323,11 +323,13 @@ export async function dumpOmpSession( return reply('Recall dump skipped: conversation text exceeds 25MiB'); } - const args = ['dump', params.title, '--stdin']; + const args = ['dump', '--stdin']; if (params.project) args.push('-p', params.project); if (params.tags) args.push('-t', params.tags); if (params.continues !== undefined) args.push('-c', String(params.continues)); if (params.skip_fabric) args.push('--skip-fabric'); + // `--` keeps a title like `--help` from being parsed as an option. + args.push('--', params.title); let output = ''; const { code, aborted } = await runCli(runChild, args, payload, signal, DUMP_TIMEOUT_MS, chunk => { diff --git a/src/commands/dump.ts b/src/commands/dump.ts index 7072daa5..9a71de11 100644 --- a/src/commands/dump.ts +++ b/src/commands/dump.ts @@ -195,8 +195,10 @@ function explicitSnapshotKey(message: Pick ({ row, content: scrub(row.content).text })); const expectedKeys = session.messages.map(message => explicitSnapshotKey({ role: message.role, @@ -217,14 +220,22 @@ function findExplicitSnapshot(session: ParsedSession): DumpMessageRow[] | undefi for (let start = rows.length - expectedKeys.length; start >= 0; start--) { const snapshot = rows.slice(start, start + expectedKeys.length); - const contiguous = snapshot.every((message, index) => - index === 0 || message.id === snapshot[index - 1].id + 1 + const contiguous = snapshot.every(({ row }, index) => + index === 0 || row.id === snapshot[index - 1].row.id + 1 ); if ( contiguous && - snapshot.every((message, index) => explicitSnapshotKey(message) === expectedKeys[index]) + snapshot.every(({ row, content }, index) => explicitSnapshotKey({ ...row, content }) === expectedKeys[index]) ) { - return snapshot; + const stale = snapshot.filter(({ row, content }) => row.content !== content); + if (stale.length > 0) { + const rewrite = db.prepare('UPDATE messages SET content = ? WHERE id = ?'); + db.transaction(() => { + for (const { row, content } of stale) rewrite.run(content, row.id); + deleteRecordEmbeddingsByIdsInTransaction(db, 'messages', stale.map(({ row }) => row.id)); + })(); + } + return snapshot.map(({ row, content }) => ({ ...row, content })); } } diff --git a/tests/commands/dump-portable.test.ts b/tests/commands/dump-portable.test.ts index 35afe545..2ebab6b3 100644 --- a/tests/commands/dump-portable.test.ts +++ b/tests/commands/dump-portable.test.ts @@ -3,7 +3,7 @@ import { coreDump, parseDumpSessionJson } from '../../src/commands/dump'; import { dumpOmpSession } from '../../hosts/omp/recall'; import { withEnv } from '../helpers/env'; import { getDb } from '../../src/db/connection'; -import { createLoaEntry, getLoaMessages, search } from '../../src/lib/memory'; +import { addMessagesBatch, createLoaEntry, getLoaMessages, search } from '../../src/lib/memory'; import { ingestHostTranscript } from '../../src/lib/host-ingest'; import { ExtractorConfigError } from '../../src/lib/extraction'; import { setupTestDb, teardownTestDb } from '../helpers/setup'; @@ -105,6 +105,41 @@ describe('portable explicit session dump', () => { expect(session.messages[0].content).toBe(`api_key=${secret}`); }); + test('re-dump matches and scrubs an explicit snapshot stored before scrubbing', async () => { + const sessionId = 'omp-unscrubbed-snapshot'; + const secret = 'abcdefgh' + '87654321'; + ingestHostTranscript({ + source: 'omp', + sessionId, + project: 'recall-test', + messages: [{ role: 'user', content: 'Automatic capture.', sourcePosition: 0 }], + }); + const timestamp = '2026-10-14T00:00:00.000Z'; + addMessagesBatch([ + { session_id: sessionId, timestamp, role: 'user', content: `api_key=${secret}`, project: 'recall-test', provenance: 'verbatim' }, + { session_id: sessionId, timestamp, role: 'assistant', content: 'Old answer.', project: 'recall-test', provenance: 'verbatim' }, + ]); + const session = parseDumpSessionJson(JSON.stringify({ + contract: 1, + harness: 'omp', + session_id: sessionId, + project: 'recall-test', + messages: [ + { role: 'user', content: `api_key=${secret}` }, + { role: 'assistant', content: 'Old answer.' }, + ], + })); + const result = await coreDump('Legacy snapshot', { session, skipFabric: true, skipEmbed: true }); + expect(result).toMatchObject({ success: true, messageCount: 0 }); + expect(getDb().prepare('SELECT content FROM published_messages WHERE session_id = ? ORDER BY id').all(sessionId)) + .toEqual([ + { content: 'Automatic capture.' }, + { content: 'api_key=[REDACTED:generic-assignment]' }, + { content: 'Old answer.' }, + ]); + expect(search(secret, { table: 'messages' })).toEqual([]); + }); + test('re-import preserves automatic lifecycle capture for the same session', async () => { const sessionId = 'codex-lifecycle-reimport'; const lifecycleContent = 'Preserve this lifecycle message across explicit re-imports.'; @@ -395,7 +430,7 @@ describe('portable explicit session dump', () => { test('omp recall_dump payload stores the live branch under omp, not a discovered transcript', async () => { let stdin = ''; let args: readonly string[] = []; - const reply = await dumpOmpSession({ title: 'omp live branch', skip_fabric: true }, { + const reply = await dumpOmpSession({ title: '--help', skip_fabric: true }, { cwd: '/work/Recall', hasUI: true, ui: { notify: () => {}, confirm: async () => true }, @@ -413,7 +448,7 @@ describe('portable explicit session dump', () => { onOutput?.('✓ LoA #7 captured'); return 0; }); - expect(args.slice(1)).toEqual(['dump', 'omp live branch', '--stdin', '--skip-fabric']); + expect(args.slice(1)).toEqual(['dump', '--stdin', '--skip-fabric', '--', '--help']); expect(reply.content[0].text).toBe('✓ LoA #7 captured'); const result = await coreDump('omp live branch', { skipFabric: true, skipEmbed: true, session: parseDumpSessionJson(stdin) }); From ecffe96555f4adac428c2763a058a863135afb0f Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Thu, 15 Oct 2026 16:34:03 -0400 Subject: [PATCH 07/11] fix(dump): match legacy snapshots whose rows scrub to blank without breaking id contiguity --- src/commands/dump.ts | 21 ++++++++++++++------- tests/commands/dump-portable.test.ts | 20 ++++++++++++-------- 2 files changed, 26 insertions(+), 15 deletions(-) diff --git a/src/commands/dump.ts b/src/commands/dump.ts index 9a71de11..cb741a06 100644 --- a/src/commands/dump.ts +++ b/src/commands/dump.ts @@ -195,7 +195,11 @@ function explicitSnapshotKey(message: Pick ({ row, content: scrub(row.content).text })); + .map((row, index) => ({ row, index, content: scrub(row.content).text })); + const kept = rows.filter(({ content }) => content.trim()); const expectedKeys = session.messages.map(message => explicitSnapshotKey({ role: message.role, @@ -217,17 +222,19 @@ function findExplicitSnapshot(session: ParsedSession): DumpMessageRow[] | undefi project: message.project ?? session.project ?? null, }) ); + if (expectedKeys.length === 0) return undefined; - for (let start = rows.length - expectedKeys.length; start >= 0; start--) { - const snapshot = rows.slice(start, start + expectedKeys.length); - const contiguous = snapshot.every(({ row }, index) => - index === 0 || row.id === snapshot[index - 1].row.id + 1 + for (let start = kept.length - expectedKeys.length; start >= 0; start--) { + const snapshot = kept.slice(start, start + expectedKeys.length); + const span = rows.slice(snapshot[0].index, snapshot[snapshot.length - 1].index + 1); + const contiguous = span.every(({ row }, index) => + index === 0 || row.id === span[index - 1].row.id + 1 ); if ( contiguous && snapshot.every(({ row, content }, index) => explicitSnapshotKey({ ...row, content }) === expectedKeys[index]) ) { - const stale = snapshot.filter(({ row, content }) => row.content !== content); + const stale = span.filter(({ row, content }) => row.content !== content); if (stale.length > 0) { const rewrite = db.prepare('UPDATE messages SET content = ? WHERE id = ?'); db.transaction(() => { diff --git a/tests/commands/dump-portable.test.ts b/tests/commands/dump-portable.test.ts index 2ebab6b3..e1de92b8 100644 --- a/tests/commands/dump-portable.test.ts +++ b/tests/commands/dump-portable.test.ts @@ -117,6 +117,7 @@ describe('portable explicit session dump', () => { const timestamp = '2026-10-14T00:00:00.000Z'; addMessagesBatch([ { session_id: sessionId, timestamp, role: 'user', content: `api_key=${secret}`, project: 'recall-test', provenance: 'verbatim' }, + { session_id: sessionId, timestamp, role: 'assistant', content: ' \u200b\u202e ', project: 'recall-test', provenance: 'verbatim' }, { session_id: sessionId, timestamp, role: 'assistant', content: 'Old answer.', project: 'recall-test', provenance: 'verbatim' }, ]); const session = parseDumpSessionJson(JSON.stringify({ @@ -126,17 +127,20 @@ describe('portable explicit session dump', () => { project: 'recall-test', messages: [ { role: 'user', content: `api_key=${secret}` }, + { role: 'assistant', content: ' \u200b\u202e ' }, { role: 'assistant', content: 'Old answer.' }, ], })); - const result = await coreDump('Legacy snapshot', { session, skipFabric: true, skipEmbed: true }); - expect(result).toMatchObject({ success: true, messageCount: 0 }); - expect(getDb().prepare('SELECT content FROM published_messages WHERE session_id = ? ORDER BY id').all(sessionId)) - .toEqual([ - { content: 'Automatic capture.' }, - { content: 'api_key=[REDACTED:generic-assignment]' }, - { content: 'Old answer.' }, - ]); + const options = { session, skipFabric: true, skipEmbed: true }; + expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0 }); + expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0 }); + expect(getDb() + .prepare(`SELECT content FROM published_messages WHERE session_id = ? AND trim(content) != '' ORDER BY id`) + .all(sessionId)).toEqual([ + { content: 'Automatic capture.' }, + { content: 'api_key=[REDACTED:generic-assignment]' }, + { content: 'Old answer.' }, + ]); expect(search(secret, { table: 'messages' })).toEqual([]); }); From 347ba1a6f64b98b8addac0e9b64f1216077a26c5 Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Thu, 15 Oct 2026 17:25:43 -0400 Subject: [PATCH 08/11] fix(dump): keep blank legacy rows in the matched span so the existing LoA is reused and rescrubbed --- src/commands/dump.ts | 5 +++-- tests/commands/dump-portable.test.ts | 23 +++++++++++++++++++++-- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/src/commands/dump.ts b/src/commands/dump.ts index cb741a06..cc7a911d 100644 --- a/src/commands/dump.ts +++ b/src/commands/dump.ts @@ -198,7 +198,8 @@ function explicitSnapshotKey(message: Pick row.id)); })(); } - return snapshot.map(({ row, content }) => ({ ...row, content })); + return span.map(({ row, content }) => ({ ...row, content })); } } diff --git a/tests/commands/dump-portable.test.ts b/tests/commands/dump-portable.test.ts index e1de92b8..70344640 100644 --- a/tests/commands/dump-portable.test.ts +++ b/tests/commands/dump-portable.test.ts @@ -120,6 +120,21 @@ describe('portable explicit session dump', () => { { session_id: sessionId, timestamp, role: 'assistant', content: ' \u200b\u202e ', project: 'recall-test', provenance: 'verbatim' }, { session_id: sessionId, timestamp, role: 'assistant', content: 'Old answer.', project: 'recall-test', provenance: 'verbatim' }, ]); + const legacyIds = (getDb() + .prepare(`SELECT id FROM messages WHERE session_id = ? AND content NOT LIKE 'Automatic%' ORDER BY id`) + .all(sessionId) as Array<{ id: number }>).map(({ id }) => id); + const legacyLoa = createLoaEntry({ + title: 'Legacy snapshot', + description: 'Explicit memory dump.', + fabric_extract: `Summary mentioning api_key=${secret}`, + message_range_start: legacyIds[0], + message_range_end: legacyIds[2], + session_id: sessionId, + project: 'recall-test', + message_count: 3, + source_ids: JSON.stringify(legacyIds.map(id => ({ table: 'messages', id }))), + provenance: 'extracted', + }); const session = parseDumpSessionJson(JSON.stringify({ contract: 1, harness: 'omp', @@ -132,8 +147,12 @@ describe('portable explicit session dump', () => { ], })); const options = { session, skipFabric: true, skipEmbed: true }; - expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0 }); - expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0 }); + expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0, loaId: legacyLoa }); + expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0, loaId: legacyLoa }); + const loas = getDb().prepare('SELECT fabric_extract FROM loa_entries WHERE session_id = ?') + .all(sessionId) as Array<{ fabric_extract: string }>; + expect(loas).toHaveLength(1); + expect(loas[0].fabric_extract).not.toContain(secret); expect(getDb() .prepare(`SELECT content FROM published_messages WHERE session_id = ? AND trim(content) != '' ORDER BY id`) .all(sessionId)).toEqual([ From 6a159098a80b9e3c7b1ff21379518971f3f3fc2b Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Thu, 15 Oct 2026 17:36:32 -0400 Subject: [PATCH 09/11] fix(dump): include leading and trailing blank legacy rows in the matched snapshot --- src/commands/dump.ts | 8 +++++++- tests/commands/dump-portable.test.ts | 26 ++++++++++++++------------ 2 files changed, 21 insertions(+), 13 deletions(-) diff --git a/src/commands/dump.ts b/src/commands/dump.ts index cc7a911d..dbc5c4ed 100644 --- a/src/commands/dump.ts +++ b/src/commands/dump.ts @@ -227,7 +227,13 @@ function findExplicitSnapshot(session: ParsedSession): DumpMessageRow[] | undefi for (let start = kept.length - expectedKeys.length; start >= 0; start--) { const snapshot = kept.slice(start, start + expectedKeys.length); - const span = rows.slice(snapshot[0].index, snapshot[snapshot.length - 1].index + 1); + // Leading/trailing blank rows of the same batch belong to this snapshot too. + // ponytail: greedy, so a directly adjacent legacy dump that starts blank loses that row to us. + let first = snapshot[0].index; + let last = snapshot[snapshot.length - 1].index; + while (first > 0 && !rows[first - 1].content.trim() && rows[first - 1].row.id + 1 === rows[first].row.id) first--; + while (last < rows.length - 1 && !rows[last + 1].content.trim() && rows[last + 1].row.id === rows[last].row.id + 1) last++; + const span = rows.slice(first, last + 1); const contiguous = span.every(({ row }, index) => index === 0 || row.id === span[index - 1].row.id + 1 ); diff --git a/tests/commands/dump-portable.test.ts b/tests/commands/dump-portable.test.ts index 70344640..e25f00e4 100644 --- a/tests/commands/dump-portable.test.ts +++ b/tests/commands/dump-portable.test.ts @@ -115,11 +115,17 @@ describe('portable explicit session dump', () => { messages: [{ role: 'user', content: 'Automatic capture.', sourcePosition: 0 }], }); const timestamp = '2026-10-14T00:00:00.000Z'; - addMessagesBatch([ - { session_id: sessionId, timestamp, role: 'user', content: `api_key=${secret}`, project: 'recall-test', provenance: 'verbatim' }, - { session_id: sessionId, timestamp, role: 'assistant', content: ' \u200b\u202e ', project: 'recall-test', provenance: 'verbatim' }, - { session_id: sessionId, timestamp, role: 'assistant', content: 'Old answer.', project: 'recall-test', provenance: 'verbatim' }, - ]); + // Blank-after-scrub rows at the start, middle, and end of the legacy batch. + const legacy = [ + { role: 'user' as const, content: '\u200b' }, + { role: 'user' as const, content: `api_key=${secret}` }, + { role: 'assistant' as const, content: ' \u200b\u202e ' }, + { role: 'assistant' as const, content: 'Old answer.' }, + { role: 'assistant' as const, content: '\u202e' }, + ]; + addMessagesBatch(legacy.map(message => ({ + ...message, session_id: sessionId, timestamp, project: 'recall-test', provenance: 'verbatim' as const, + }))); const legacyIds = (getDb() .prepare(`SELECT id FROM messages WHERE session_id = ? AND content NOT LIKE 'Automatic%' ORDER BY id`) .all(sessionId) as Array<{ id: number }>).map(({ id }) => id); @@ -128,10 +134,10 @@ describe('portable explicit session dump', () => { description: 'Explicit memory dump.', fabric_extract: `Summary mentioning api_key=${secret}`, message_range_start: legacyIds[0], - message_range_end: legacyIds[2], + message_range_end: legacyIds[legacyIds.length - 1], session_id: sessionId, project: 'recall-test', - message_count: 3, + message_count: legacyIds.length, source_ids: JSON.stringify(legacyIds.map(id => ({ table: 'messages', id }))), provenance: 'extracted', }); @@ -140,11 +146,7 @@ describe('portable explicit session dump', () => { harness: 'omp', session_id: sessionId, project: 'recall-test', - messages: [ - { role: 'user', content: `api_key=${secret}` }, - { role: 'assistant', content: ' \u200b\u202e ' }, - { role: 'assistant', content: 'Old answer.' }, - ], + messages: legacy, })); const options = { session, skipFabric: true, skipEmbed: true }; expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0, loaId: legacyLoa }); From de794613374aaa6d6f9a26baa029e460ef15a98f Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Thu, 15 Oct 2026 17:50:31 -0400 Subject: [PATCH 10/11] fix(dump): match legacy LoAs by non-blank source ids instead of expanding over blank rows --- src/commands/dump.ts | 64 +++++++++++++++++----------- tests/commands/dump-portable.test.ts | 56 ++++++++++++++---------- 2 files changed, 73 insertions(+), 47 deletions(-) diff --git a/src/commands/dump.ts b/src/commands/dump.ts index dbc5c4ed..452881ec 100644 --- a/src/commands/dump.ts +++ b/src/commands/dump.ts @@ -195,13 +195,15 @@ function explicitSnapshotKey(message: Pick; +} + +/** Old dumps stored unscrubbed text: compare scrubbed, then rewrite the matched span in place. */ +function findExplicitSnapshot(session: ParsedSession): ExplicitSnapshot | undefined { const db = getDb(); const rows = (db .prepare(` @@ -216,6 +218,7 @@ function findExplicitSnapshot(session: ParsedSession): DumpMessageRow[] | undefi .all(session.sessionId) as DumpMessageRow[]) .map((row, index) => ({ row, index, content: scrub(row.content).text })); const kept = rows.filter(({ content }) => content.trim()); + const blankIds = new Set(rows.filter(({ content }) => !content.trim()).map(({ row }) => row.id)); const expectedKeys = session.messages.map(message => explicitSnapshotKey({ role: message.role, @@ -227,13 +230,8 @@ function findExplicitSnapshot(session: ParsedSession): DumpMessageRow[] | undefi for (let start = kept.length - expectedKeys.length; start >= 0; start--) { const snapshot = kept.slice(start, start + expectedKeys.length); - // Leading/trailing blank rows of the same batch belong to this snapshot too. - // ponytail: greedy, so a directly adjacent legacy dump that starts blank loses that row to us. - let first = snapshot[0].index; - let last = snapshot[snapshot.length - 1].index; - while (first > 0 && !rows[first - 1].content.trim() && rows[first - 1].row.id + 1 === rows[first].row.id) first--; - while (last < rows.length - 1 && !rows[last + 1].content.trim() && rows[last + 1].row.id === rows[last].row.id + 1) last++; - const span = rows.slice(first, last + 1); + // Blank rows between matched rows stay on disk, so ids stay contiguous across re-dumps. + const span = rows.slice(snapshot[0].index, snapshot[snapshot.length - 1].index + 1); const contiguous = span.every(({ row }, index) => index === 0 || row.id === span[index - 1].row.id + 1 ); @@ -241,7 +239,10 @@ function findExplicitSnapshot(session: ParsedSession): DumpMessageRow[] | undefi contiguous && snapshot.every(({ row, content }, index) => explicitSnapshotKey({ ...row, content }) === expectedKeys[index]) ) { - const stale = span.filter(({ row, content }) => row.content !== content); + // Matched span plus every row that scrubs to blank (hidden bidi/zero-width only, owner irrelevant). + const stale = rows.filter(({ row, content, index }) => row.content !== content && ( + !content.trim() || (index >= snapshot[0].index && index <= snapshot[snapshot.length - 1].index) + )); if (stale.length > 0) { const rewrite = db.prepare('UPDATE messages SET content = ? WHERE id = ?'); db.transaction(() => { @@ -249,7 +250,7 @@ function findExplicitSnapshot(session: ParsedSession): DumpMessageRow[] | undefi deleteRecordEmbeddingsByIdsInTransaction(db, 'messages', stale.map(({ row }) => row.id)); })(); } - return span.map(({ row, content }) => ({ ...row, content })); + return { messages: snapshot.map(({ row, content }) => ({ ...row, content })), blankIds }; } } @@ -349,8 +350,9 @@ export async function coreDump(title: string, options: DumpOptions & { session?: ); const db = getDb(); + const snapshot = lifecycleOwned ? existingSnapshot ?? findExplicitSnapshot(session) : undefined; const snapshotMessages = lifecycleOwned - ? (existingSnapshot ?? findExplicitSnapshot(session) ?? []) + ? snapshot?.messages ?? [] : db.prepare(` SELECT id, content, role, timestamp, project FROM published_messages @@ -393,13 +395,26 @@ export async function coreDump(title: string, options: DumpOptions & { session?: const sourceIds = JSON.stringify( importedMessages.map(message => ({ table: 'messages', id: message.id })) ); + // Legacy LoAs may also list rows that now scrub to blank; ignore those when matching identity. + const importedKey = importedMessages.map(message => message.id).join(','); const existingLoa = lifecycleOwned - ? db.prepare(` - SELECT id, title, fabric_extract FROM loa_entries - WHERE session_id = ? AND description = ? AND source_ids = ? - ORDER BY id DESC LIMIT 1 - `).get(session.sessionId, EXPLICIT_DUMP_DESCRIPTION, sourceIds) as - { id: number; title: string; fabric_extract: string } | undefined + ? (db.prepare(` + SELECT l.id, l.title, l.fabric_extract, ( + SELECT group_concat(json_extract(s.value, '$.id')) + FROM json_each(CASE WHEN json_valid(l.source_ids) THEN + CASE WHEN json_type(l.source_ids) = 'array' THEN l.source_ids ELSE '[]' END + ELSE '[]' END) s + WHERE CASE WHEN s.type = 'object' THEN json_extract(s.value, '$.table') END = 'messages' + ) AS message_ids + FROM loa_entries l + WHERE l.session_id = ? AND l.description = ? + ORDER BY l.id DESC + `).all(session.sessionId, EXPLICIT_DUMP_DESCRIPTION) as + Array<{ id: number; title: string; fabric_extract: string; message_ids: string | null }>) + .find(loa => (loa.message_ids ?? '') + .split(',') + .filter(id => id && !snapshot?.blankIds.has(Number(id))) + .join(',') === importedKey) : undefined; let loaId: number; if (existingLoa) { @@ -407,7 +422,7 @@ export async function coreDump(title: string, options: DumpOptions & { session?: db.prepare(` UPDATE loa_entries SET title = ?, fabric_extract = ?, message_range_start = ?, message_range_end = ?, - snapshot_max_message_id = ?, project = ?, tags = ?, message_count = ?, + snapshot_max_message_id = ?, project = ?, tags = ?, message_count = ?, source_ids = ?, created_at = CURRENT_TIMESTAMP WHERE id = ? `).run( @@ -419,6 +434,7 @@ export async function coreDump(title: string, options: DumpOptions & { session?: options.project || session.project, options.tags ?? null, importedMessages.length, + sourceIds, existingLoa.id ); if (existingLoa.title !== title || existingLoa.fabric_extract !== fabricExtract) { diff --git a/tests/commands/dump-portable.test.ts b/tests/commands/dump-portable.test.ts index e25f00e4..a2581cf6 100644 --- a/tests/commands/dump-portable.test.ts +++ b/tests/commands/dump-portable.test.ts @@ -115,7 +115,32 @@ describe('portable explicit session dump', () => { messages: [{ role: 'user', content: 'Automatic capture.', sourcePosition: 0 }], }); const timestamp = '2026-10-14T00:00:00.000Z'; - // Blank-after-scrub rows at the start, middle, and end of the legacy batch. + const seedLegacyDump = (messages: Array<{ role: 'user' | 'assistant'; content: string }>, extract: string) => { + const maxRow = getDb().prepare('SELECT COALESCE(MAX(id), 0) AS id FROM messages').get() as { id: number }; + const before = maxRow.id; + addMessagesBatch(messages.map(message => ({ + ...message, session_id: sessionId, timestamp, project: 'recall-test', provenance: 'verbatim' as const, + }))); + const ids = (getDb().prepare('SELECT id FROM messages WHERE session_id = ? AND id > ? ORDER BY id') + .all(sessionId, before) as Array<{ id: number }>).map(({ id }) => id); + return createLoaEntry({ + title: 'Legacy snapshot', + description: 'Explicit memory dump.', + fabric_extract: extract, + message_range_start: ids[0], + message_range_end: ids[ids.length - 1], + session_id: sessionId, + project: 'recall-test', + message_count: ids.length, + source_ids: JSON.stringify(ids.map(id => ({ table: 'messages', id }))), + provenance: 'extracted', + }); + }; + // An adjacent older dump that ends blank, then the target with blanks at start, middle, and end. + const earlierLoa = seedLegacyDump( + [{ role: 'user', content: 'Earlier question.' }, { role: 'assistant', content: '\u200b' }], + 'Earlier summary.', + ); const legacy = [ { role: 'user' as const, content: '\u200b' }, { role: 'user' as const, content: `api_key=${secret}` }, @@ -123,24 +148,7 @@ describe('portable explicit session dump', () => { { role: 'assistant' as const, content: 'Old answer.' }, { role: 'assistant' as const, content: '\u202e' }, ]; - addMessagesBatch(legacy.map(message => ({ - ...message, session_id: sessionId, timestamp, project: 'recall-test', provenance: 'verbatim' as const, - }))); - const legacyIds = (getDb() - .prepare(`SELECT id FROM messages WHERE session_id = ? AND content NOT LIKE 'Automatic%' ORDER BY id`) - .all(sessionId) as Array<{ id: number }>).map(({ id }) => id); - const legacyLoa = createLoaEntry({ - title: 'Legacy snapshot', - description: 'Explicit memory dump.', - fabric_extract: `Summary mentioning api_key=${secret}`, - message_range_start: legacyIds[0], - message_range_end: legacyIds[legacyIds.length - 1], - session_id: sessionId, - project: 'recall-test', - message_count: legacyIds.length, - source_ids: JSON.stringify(legacyIds.map(id => ({ table: 'messages', id }))), - provenance: 'extracted', - }); + const legacyLoa = seedLegacyDump(legacy, `Summary mentioning api_key=${secret}`); const session = parseDumpSessionJson(JSON.stringify({ contract: 1, harness: 'omp', @@ -151,14 +159,16 @@ describe('portable explicit session dump', () => { const options = { session, skipFabric: true, skipEmbed: true }; expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0, loaId: legacyLoa }); expect(await coreDump('Legacy snapshot', options)).toMatchObject({ success: true, messageCount: 0, loaId: legacyLoa }); - const loas = getDb().prepare('SELECT fabric_extract FROM loa_entries WHERE session_id = ?') - .all(sessionId) as Array<{ fabric_extract: string }>; - expect(loas).toHaveLength(1); - expect(loas[0].fabric_extract).not.toContain(secret); + expect(getDb().prepare('SELECT id, fabric_extract, message_count FROM loa_entries WHERE session_id = ? ORDER BY id') + .all(sessionId)).toEqual([ + { id: earlierLoa, fabric_extract: 'Earlier summary.', message_count: 2 }, + { id: legacyLoa, fabric_extract: expect.not.stringContaining(secret), message_count: 2 }, + ]); expect(getDb() .prepare(`SELECT content FROM published_messages WHERE session_id = ? AND trim(content) != '' ORDER BY id`) .all(sessionId)).toEqual([ { content: 'Automatic capture.' }, + { content: 'Earlier question.' }, { content: 'api_key=[REDACTED:generic-assignment]' }, { content: 'Old answer.' }, ]); From f9c55c22831956090dc81dbbd09a3baac8901faf Mon Sep 17 00:00:00 2001 From: Atlas-Key <296298943+Atlas-Key@users.noreply.github.com> Date: Fri, 16 Oct 2026 13:27:16 -0400 Subject: [PATCH 11/11] no-mistakes(document): Clarify omp dump limits and discovery guidance --- docs/OMP_INTEGRATION.md | 4 ++-- docs/cli-reference.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/OMP_INTEGRATION.md b/docs/OMP_INTEGRATION.md index 14d73032..244da75f 100644 --- a/docs/OMP_INTEGRATION.md +++ b/docs/OMP_INTEGRATION.md @@ -38,7 +38,7 @@ Capture sends only user and assistant text, at most 25 MiB per payload, and boun ## Explicit dump: `recall_dump` tool -The extension also registers a `recall_dump` tool (title, optional project, tags, `continues`, `skip_fabric`). It reads the live branch and session id from omp, so it dumps this conversation on any profile or branch, and sends it to the package-local `dist/index.js dump --stdin`. The tool asks you to confirm in the omp UI before writing anything and refuses in headless sessions. `recall dump` run from a shell inside omp (`OMPCODE=1`) refuses and points to the tool: transcript files on disk belong to other conversations. +The extension also registers a `recall_dump` tool (title, optional project, tags, `continues`, `skip_fabric`). It reads the live branch and session id from omp, so it dumps this conversation on any profile or branch, and sends it to the package-local `dist/index.js dump --stdin`. The tool asks you to confirm in the omp UI before writing anything and refuses in headless sessions. Without `--stdin`, `recall dump` run from a shell inside omp (`OMPCODE=1`) refuses and points to the tool: native transcript discovery cannot identify the live omp conversation. Dump payloads are limited to 25 MiB, and the CLI child has a five-minute timeout. `skip_fabric` defaults to false; set it to true for a basic summary. Shared message scrubbing and default project attribution follow the [Session Dump contract](cli-reference.md#session-dump). @@ -62,7 +62,7 @@ This verifies the packed local-link integration in headless text mode. It does n Restart omp after uninstalling. Saved Recall memory remains. `recall uninstall` removes installer-owned skills, not the separately managed omp plugin; uninstall the plugin through omp before removing Recall's runtime. -If capture warns, check `bun` is available to omp, run `bun dist/index.js init` from the linked checkout, and check `RECALL_DB_PATH` permissions. An omp process started before the extension was refreshed keeps the old code until it restarts: older versions measured the whole raw branch and warned `payload exceeds 25MiB` on long tool-heavy sessions. Only a branch whose user and assistant text alone exceeds 25 MiB needs a new session. +If capture warns, check `bun` is available to omp, run `bun dist/index.js init` from the linked checkout, and check `RECALL_DB_PATH` permissions. An omp process started before the extension was refreshed keeps the old code until it restarts: older versions measured the whole raw branch and warned `payload exceeds 25MiB` on long tool-heavy sessions. If the filtered user/assistant payload still exceeds the capture limit, start a new session. The limit counts serialized JSON bytes, including escaping and metadata. ## Native documentation diff --git a/docs/cli-reference.md b/docs/cli-reference.md index a7afd78f..36eedae9 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -84,7 +84,7 @@ Without `--stdin`, Recall discovers native transcripts in Claude Code, OpenCode, `--stdin` reads up to 25 MiB of contract 1 JSON for host extensions that hold the live transcript: `contract: 1`, `harness`, `session_id`, optional `cwd`/`project`, and `messages` with user/assistant `role`, nonblank text `content`, and optional `timestamp`. Malformed input is rejected before writes. `-p` overrides the payload's project; otherwise Recall uses its `project`, then the origin repository name or directory basename from `cwd`, then `harness` if `cwd` is absent. -Both CLI dumps and MCP `memory_dump` scrub each message's content with the shared write-safety guard before snapshot matching, storage, and summary or extraction. Known credential patterns become visible `[REDACTED:]` markers, invisible/bidi controls are removed, and messages left blank are dropped. This does not detect every secret; follow the [credential-handling rule](hosts/FOR_CLAUDE.md#core-rules) before dumping sensitive conversations. +Both CLI dumps and MCP `memory_dump` scrub each message's content with the shared write-safety guard before snapshot matching, storage, and summary or extraction. Known credential patterns become visible `[REDACTED:]` markers, invisible/bidi controls are removed, and incoming messages left blank are dropped. This does not detect every secret; follow the [credential-handling rule](mcp-tools.md#memory_add) before dumping sensitive conversations. ### Library of Alexandria (LoA)