diff --git a/ts/packages/copilot-plugin/README.md b/ts/packages/copilot-plugin/README.md index ecf3c4c785..7045cf3528 100644 --- a/ts/packages/copilot-plugin/README.md +++ b/ts/packages/copilot-plugin/README.md @@ -21,9 +21,9 @@ User prompt Explicit @typeagent run commands and optional structured action MCP tools remain available. They are not automatic recording or execution fallbacks. -Registered alongside routing (calls are disabled in bypass mode): +Registered alongside routing (calls are disabled in bypass mode by default): typeagent-workspace MCP tools - typeagent-macros MCP tools -> checked program execution + typeagent-macros MCP tools -> checked program execution (explicit bypass opt-in below) ``` The hook output fields `handled`, `responseContent`, and `handledBy` are supported in current Copilot CLI behavior, allowing the hook to skip the agentic loop entirely when TypeAgent handles a request. For local runtime debugging against the runtime repo, use `pnpm copilot:dev`. @@ -652,7 +652,8 @@ No owner setting is required: the default and only enabled owner is `plugin`. `plugin` and `off`. Close and restart the native session after changing it. An obsolete explicit `typeagent` value is an actionable error, not a different catalog selection. `off` disables recipes, including recording; `bypass` disables -recipe matching/execution and MCP calls, but permits explicitly armed recordings. +recipe matching and MCP calls by default, but permits explicitly armed recordings. +The macro-tool opt-in permits discovery and execution without prompt matching. Unmatched ordinary prompts go to native Copilot, not the TypeAgent planner. Explicit TypeAgent commands/tools remain optional backend operations; they are not required by the local recipe path. @@ -811,8 +812,10 @@ set a workspace learning preference, then arm the next interaction: To record Copilot-native or third-party MCP tools without recipe routing, use `/typeagent-mode bypass` instead. The selected task stays with Copilot, including -the success receipt required for learning. TypeAgent workspace and macro MCP -calls remain disabled in bypass; use the extension recording/status commands. +the success receipt required for learning. TypeAgent workspace MCP calls remain +disabled, and macro MCP calls are disabled by default; use the extension +recording/status commands. The macro-tool opt-in described below is not required +for recording. For the MCP-mode example, send this as the next ordinary message, rather than using the natural-language file-reading shortcut: @@ -944,7 +947,7 @@ job data or create a manual draft to force a retry. ### Review and run a prepared program Use the lifecycle MCP tools in direct, MCP or dev mode. They call the selected -owner; bypass disables them. After learning produces a candidate: +owner; bypass disables them by default. After learning produces a candidate: 1. Inspect its returned candidate and version, then call `validate_macro`. 2. In Prepare mode, review it and call `approve_macro`. Drafts cannot run. @@ -956,6 +959,41 @@ owner; bypass disables them. After learning produces a candidate: `processCommand` does not create or approve macros. The lifecycle tools have no `preference` parameter, manual step-draft creation or adapted-candidate submission. +### Explicit macro tools while Copilot handles prompts + +To let Copilot discover and run existing approved macros without enabling prompt +routing, opt in before starting Copilot: + +```powershell +$env:TYPEAGENT_MODE = 'bypass' +$env:TYPEAGENT_BYPASS_MACRO_TOOLS = 'true' +pnpm copilot --experimental +``` + +Alternatively, set `"bypassMacroTools": true` alongside `"mode": "bypass"` in +the plugin config. The environment variable overrides the saved setting and +accepts `true`/`false`, `1`/`0`, or `on`/`off`; invalid values are rejected. +The default is `false`. Restart Copilot after changing environment variables; +config changes are read on subsequent calls. + +Only `list_macros`, `search_macros`, `inspect_macro`, `get_macro_requirements`, +`run_macro`, and `get_macro_run` are permitted by this opt-in. Validation, +approval, disablement, and deletion remain blocked in bypass; use direct, MCP, +or dev mode for those lifecycle operations. `@typeagent status` and +`/typeagent-status` report `Macro tools: discovery and execution only` when +the bypass opt-in is active. + +Ordinary prompts still go to Copilot without local recipe matching or automatic +TypeAgent delegation. Workspace and other TypeAgent MCP tools remain subject to +their existing mode gates. This setting does not change learning preferences, +enable a disabled recipe owner, or grant tool/backend permissions. + +Execution uses the same local owner and current approved immutable program +version as other modes. Unsupported tools, backend denials, failed or cancelled +runs remain failures; there is no planner fallback or automatic retry. This does +not add replay support for arbitrary native Copilot tools or make a captured +trace into a runnable macro. Approval and backend authorization are separate. + ### Route approved macros from natural language The plugin-local matcher uses the current approved program's generated grammar @@ -977,7 +1015,8 @@ disable and deletion refresh schemas and reconcile stored routes. ### Recording controls and recovery -Program authoring/execution is unconditional. Only these recording boundaries +Program authoring/execution has no legacy feature flag. Explicit macro access in +bypass is separately opt-in as described above. These recording boundaries remain configurable (enabled by default): | Environment variable | Boundary | @@ -1092,8 +1131,9 @@ that mode selection requires natural-language dispatch. Ordinary prompts use local checked-program matching followed by Copilot on a miss, in `direct`, `mcp` and `dev`. None of these modes selects a backend macro -owner or author. `bypass` disables recipe matching/execution and MCP calls, -while still allowing explicitly armed recordings. +owner or author. `bypass` disables recipe matching and MCP calls by default, +while still allowing explicitly armed recordings. The explicit macro-tool opt-in +above permits discovery and execution without prompt matching. The settings remain for explicit TypeAgent commands and MCP availability. `@typeagent run ` intentionally delegates to the TypeAgent dispatcher; @@ -1153,20 +1193,22 @@ The plugin stores config at `%USERPROFILE%\.typeagent-copilot\config.json` (Wind ```json { "mode": "direct", - "mcpRouting": "delegate" + "mcpRouting": "delegate", + "bypassMacroTools": false } ``` **Environment variable overrides** (take precedence over config file): -| Variable | Default | Description | -| --------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -| `TYPEAGENT_MODE` | `direct` | `direct`, `mcp`, `dev`, or `bypass` | -| `TYPEAGENT_HOST` | `localhost` | TypeAgent server host | -| `TYPEAGENT_PORT` | `8999` | TypeAgent server port | -| `TYPEAGENT_CONVERSATION_ID` | Session-selected server default | Optional existing public conversation ID for all routes; overrides config `conversationId` and automatic selection | -| `TYPEAGENT_PLUGIN_DATA` | `~/.typeagent-copilot` | Config directory | -| `TYPEAGENT_WORKSPACE_ROOTS` | Copilot process working directory | Approved roots for workspace MCP tools, separated by the platform path delimiter | +| Variable | Default | Description | +| ------------------------------ | --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| `TYPEAGENT_MODE` | `direct` | `direct`, `mcp`, `dev`, or `bypass` | +| `TYPEAGENT_BYPASS_MACRO_TOOLS` | `false` | Opt in to explicit macro discovery/execution in bypass; overrides config `bypassMacroTools` without changing routing or permissions | +| `TYPEAGENT_HOST` | `localhost` | TypeAgent server host | +| `TYPEAGENT_PORT` | `8999` | TypeAgent server port | +| `TYPEAGENT_CONVERSATION_ID` | Session-selected server default | Optional existing public conversation ID for all routes; overrides config `conversationId` and automatic selection | +| `TYPEAGENT_PLUGIN_DATA` | `~/.typeagent-copilot` | Config directory | +| `TYPEAGENT_WORKSPACE_ROOTS` | Copilot process working directory | Approved roots for workspace MCP tools, separated by the platform path delimiter | --- diff --git a/ts/packages/copilot-plugin/src/extension/commands.ts b/ts/packages/copilot-plugin/src/extension/commands.ts index 39fc622189..304aa8d3f6 100644 --- a/ts/packages/copilot-plugin/src/extension/commands.ts +++ b/ts/packages/copilot-plugin/src/extension/commands.ts @@ -2,7 +2,12 @@ // Licensed under the MIT License. import type { CommandDefinition } from "@github/copilot-sdk"; -import { getConfigPath, getModeLabel } from "../shared/plugin-config.js"; +import { + getConfigPath, + getMacroToolAvailability, + getMode, + getModeLabel, +} from "../shared/plugin-config.js"; import { getModeDescription, handleModeSetting, @@ -70,7 +75,10 @@ function statusText(): string { "TypeAgent configuration", `Mode: ${getModeLabel()}`, `Routing: ${getModeDescription()}`, - "Ordinary prompts: local recipes, then Copilot", + getMode() === "bypass" + ? "Ordinary prompts: Copilot (no local recipe matching)" + : "Ordinary prompts: local recipes, then Copilot", + `Macro tools: ${getMacroToolAvailability()}`, `Server: ws://${host}:${port}`, `Config: ${getConfigPath()}`, "Mode settings are shared by sessions using this config.", diff --git a/ts/packages/copilot-plugin/src/hooks/hook-router.ts b/ts/packages/copilot-plugin/src/hooks/hook-router.ts index 69e6fac85a..59109d9293 100644 --- a/ts/packages/copilot-plugin/src/hooks/hook-router.ts +++ b/ts/packages/copilot-plugin/src/hooks/hook-router.ts @@ -39,6 +39,7 @@ import { getConfigPath, getMode, getModeLabel, + getMacroToolAvailability, getRecipeOwner, type Mode, type RecipeOwner, @@ -193,8 +194,11 @@ function handleStatusCommand( "", `- Mode: **${getModeLabel()}**`, `- Routing: ${getModeDescription()}`, - "- Ordinary prompts: local recipes, then Copilot", - `- Macro workspace tools: **${mode === "bypass" ? "disabled" : "available"}**`, + mode === "bypass" + ? "- Ordinary prompts: Copilot (no local recipe matching)" + : "- Ordinary prompts: local recipes, then Copilot", + `- Workspace tools: **${mode === "bypass" ? "disabled" : "available"}**`, + `- Macro tools: **${getMacroToolAvailability()}**`, `- Server: ws://${host}:${port}`, `- Config: ${configPath}`, "- Mode settings are shared by sessions using this config.", diff --git a/ts/packages/copilot-plugin/src/mcp/macroServer.ts b/ts/packages/copilot-plugin/src/mcp/macroServer.ts index 1cdf00073c..4b05a31893 100644 --- a/ts/packages/copilot-plugin/src/mcp/macroServer.ts +++ b/ts/packages/copilot-plugin/src/mcp/macroServer.ts @@ -6,7 +6,11 @@ import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js" import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; import { randomUUID } from "node:crypto"; import { z } from "zod"; -import { getMode, type Mode } from "../shared/plugin-config.js"; +import { + getBypassMacroToolsEnabled, + getMode, + type Mode, +} from "../shared/plugin-config.js"; import { connectToMacroOwner, type MacroOwnerConnection, @@ -15,11 +19,13 @@ import { export interface MacroServerDependencies { connect: () => Promise; getMode: () => Mode; + getBypassMacroToolsEnabled?: () => boolean; } const defaultDependencies: MacroServerDependencies = { connect: connectToMacroOwner, getMode, + getBypassMacroToolsEnabled, }; function result(value: unknown): CallToolResult { @@ -57,30 +63,35 @@ export class MacroCatalogAdapter { state?: "draft" | "approved" | "disabled" | undefined; limit?: number | undefined; }) { - return this.call((connection) => - connection.listMacros({ - ...(request.state ? { state: request.state } : {}), - ...(request.limit !== undefined - ? { limit: request.limit } - : {}), - }), + return this.call( + (connection) => + connection.listMacros({ + ...(request.state ? { state: request.state } : {}), + ...(request.limit !== undefined + ? { limit: request.limit } + : {}), + }), + true, ); } searchMacros(request: { query: string; limit?: number | undefined }) { - return this.call((connection) => - connection.searchMacros({ - query: request.query, - ...(request.limit !== undefined - ? { limit: request.limit } - : {}), - }), + return this.call( + (connection) => + connection.searchMacros({ + query: request.query, + ...(request.limit !== undefined + ? { limit: request.limit } + : {}), + }), + true, ); } inspectMacro(request: { macroId: string; version?: number | undefined }) { - return this.call((connection) => - connection.inspectMacro(this.macroRef(request)), + return this.call( + (connection) => connection.inspectMacro(this.macroRef(request)), + true, ); } @@ -88,8 +99,10 @@ export class MacroCatalogAdapter { macroId: string; version?: number | undefined; }) { - return this.call((connection) => - connection.getMacroRequirements(this.macroRef(request)), + return this.call( + (connection) => + connection.getMacroRequirements(this.macroRef(request)), + true, ); } @@ -117,7 +130,10 @@ export class MacroCatalogAdapter { } getMacroRun(request: { runId: string }) { - return this.call((connection) => connection.getMacroRun(request.runId)); + return this.call( + (connection) => connection.getMacroRun(request.runId), + true, + ); } async runMacro( @@ -130,16 +146,14 @@ export class MacroCatalogAdapter { }, signal?: AbortSignal, ): Promise { - if (this.dependencies.getMode() === "bypass") { - return errorResult( - "TypeAgent macro tools are disabled in bypass mode.", - ); - } - const runId = randomUUID(); let connection: MacroOwnerConnection | undefined; let removeAbort: (() => void) | undefined; try { + this.assertAccess(true); + signal?.throwIfAborted(); + const runId = randomUUID(); connection = await this.dependencies.connect(); + signal?.throwIfAborted(); const activeConnection = connection; if (signal) { const cancel = () => { @@ -188,14 +202,11 @@ export class MacroCatalogAdapter { private async call( operation: (connection: MacroOwnerConnection) => Promise, + allowInBypass = false, ): Promise { - if (this.dependencies.getMode() === "bypass") { - return errorResult( - "TypeAgent macro tools are disabled in bypass mode.", - ); - } let connection: MacroOwnerConnection | undefined; try { + this.assertAccess(allowInBypass); connection = await this.dependencies.connect(); return result(await operation(connection)); } catch (error) { @@ -204,6 +215,20 @@ export class MacroCatalogAdapter { await connection?.close(); } } + + private assertAccess(allowInBypass: boolean): void { + if (this.dependencies.getMode() !== "bypass") return; + if ( + allowInBypass && + this.dependencies.getBypassMacroToolsEnabled?.() === true + ) + return; + throw new Error( + allowInBypass + ? "TypeAgent macro tools are disabled in bypass mode. Opt in with bypassMacroTools or TYPEAGENT_BYPASS_MACRO_TOOLS." + : "TypeAgent macro management tools are disabled in bypass mode.", + ); + } } const macroRefSchema = { diff --git a/ts/packages/copilot-plugin/src/shared/plugin-config.ts b/ts/packages/copilot-plugin/src/shared/plugin-config.ts index b4ab39302c..f7d169f160 100644 --- a/ts/packages/copilot-plugin/src/shared/plugin-config.ts +++ b/ts/packages/copilot-plugin/src/shared/plugin-config.ts @@ -18,6 +18,7 @@ export interface PluginConfig { /** Catalog revisions to expose to this Copilot session. */ selectedSkills?: SkillSelection[]; recipeOwner?: RecipeOwner; + bypassMacroTools?: boolean; recipeReadOnlyMcpTools?: { mcpServerName: string; toolName: string }[]; recipeInferenceCliPath?: string; [key: string]: unknown; @@ -163,6 +164,38 @@ export function getModeLabel(): string { return mode === "mcp" ? `mcp (${getMcpRouting()})` : mode; } +export function getBypassMacroToolsEnabled(): boolean { + const environment = process.env.TYPEAGENT_BYPASS_MACRO_TOOLS; + if (environment !== undefined) { + switch (environment.trim().toLowerCase()) { + case "true": + case "1": + case "on": + return true; + case "false": + case "0": + case "off": + return false; + default: + throw new Error( + "TYPEAGENT_BYPASS_MACRO_TOOLS must be true/false, 1/0, or on/off.", + ); + } + } + const configured = readConfig()?.bypassMacroTools; + if (configured === undefined) return false; + if (typeof configured !== "boolean") + throw new Error("bypassMacroTools must be a boolean."); + return configured; +} + +export function getMacroToolAvailability(): string { + if (getMode() !== "bypass") return "available"; + return getBypassMacroToolsEnabled() + ? "discovery and execution only" + : "disabled"; +} + export function getSelectedSkills(): SkillSelection[] { const environment = process.env.TYPEAGENT_SELECTED_SKILLS; const value = diff --git a/ts/packages/copilot-plugin/test/bypassMacroTools.spec.ts b/ts/packages/copilot-plugin/test/bypassMacroTools.spec.ts new file mode 100644 index 0000000000..755abebecd --- /dev/null +++ b/ts/packages/copilot-plugin/test/bypassMacroTools.spec.ts @@ -0,0 +1,182 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { jest } from "@jest/globals"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + getBypassMacroToolsEnabled, + getConfigPath, + getMacroToolAvailability, + writeConfig, +} from "../src/shared/plugin-config.js"; +import { createExtensionCommands } from "../src/extension/commands.js"; +import { handleSlashCommand, routePrompt } from "../src/hooks/hook-router.js"; +import { MacroCatalogAdapter } from "../src/mcp/macroServer.js"; + +describe("explicit macro tools in bypass mode", () => { + let directory: string; + const names = [ + "TYPEAGENT_PLUGIN_DATA", + "TYPEAGENT_MODE", + "TYPEAGENT_BYPASS_MACRO_TOOLS", + "COPILOT_AGENT_SESSION_ID", + "TYPEAGENT_RECIPE_OWNER", + ] as const; + const original = Object.fromEntries( + names.map((name) => [name, process.env[name]]), + ); + beforeEach(() => { + directory = mkdtempSync(join(tmpdir(), "bypass-macro-tools-")); + for (const name of names) delete process.env[name]; + process.env.TYPEAGENT_PLUGIN_DATA = directory; + }); + afterEach(() => { + rmSync(directory, { recursive: true, force: true }); + for (const name of names) { + if (original[name] === undefined) delete process.env[name]; + else process.env[name] = original[name]; + } + }); + + it("defaults to disabled without a file or explicit setting", () => { + expect(getBypassMacroToolsEnabled()).toBe(false); + writeConfig({ mode: "bypass" }); + expect(getBypassMacroToolsEnabled()).toBe(false); + expect(getMacroToolAvailability()).toBe("disabled"); + }); + + it.each([false, true])( + "reads the explicit %s config setting", + (enabled) => { + writeConfig({ mode: "bypass", bypassMacroTools: enabled }); + expect(getBypassMacroToolsEnabled()).toBe(enabled); + }, + ); + + it.each([ + ["true", true], + ["1", true], + ["on", true], + [" ON ", true], + ["false", false], + ["0", false], + ["off", false], + ] as const)( + "environment %s overrides the saved setting", + (value, enabled) => { + writeConfig({ mode: "bypass", bypassMacroTools: !enabled }); + process.env.TYPEAGENT_BYPASS_MACRO_TOOLS = value; + expect(getBypassMacroToolsEnabled()).toBe(enabled); + }, + ); + + it.each(["", "enabled", "truthy", "2"])( + "rejects invalid environment value %p", + (value) => { + writeConfig({ mode: "bypass", bypassMacroTools: true }); + process.env.TYPEAGENT_BYPASS_MACRO_TOOLS = value; + expect(() => getBypassMacroToolsEnabled()).toThrow( + "TYPEAGENT_BYPASS_MACRO_TOOLS must be", + ); + }, + ); + + it.each([null, "true", 1, [], {}])( + "rejects invalid configured value %p", + (value) => { + writeFileSync( + getConfigPath(), + JSON.stringify({ mode: "bypass", bypassMacroTools: value }), + ); + expect(() => getBypassMacroToolsEnabled()).toThrow( + "bypassMacroTools must be a boolean", + ); + }, + ); + + it("rejects malformed configuration rather than enabling macro tools", () => { + writeFileSync(getConfigPath(), "{broken"); + expect(() => getBypassMacroToolsEnabled()).toThrow( + "Unable to read plugin configuration", + ); + }); + + it.each(["direct", "mcp", "dev"] as const)( + "does not change macro availability in %s mode", + (mode) => { + writeConfig({ mode, bypassMacroTools: false }); + expect(getMacroToolAvailability()).toBe("available"); + }, + ); + + it.each([false, true])( + "reports matching hook and extension status with opt-in %s", + async (enabled) => { + writeConfig({ mode: "bypass", bypassMacroTools: enabled }); + const expected = enabled + ? "discovery and execution only" + : "disabled"; + const hook = await handleSlashCommand({ + prompt: "@typeagent status", + sessionId: "status-session", + timestamp: 1, + cwd: directory, + }); + const messages: string[] = []; + const command = createExtensionCommands(async (message) => { + messages.push(message); + }).find(({ name }) => name === "typeagent-status"); + await command?.handler({ args: "" } as never); + expect(hook?.responseContent).toContain( + `Macro tools: **${expected}**`, + ); + expect(hook?.responseContent).toContain( + "Workspace tools: **disabled**", + ); + expect(messages[0]).toContain(`Macro tools: ${expected}`); + expect(hook?.responseContent).toContain( + "Ordinary prompts: Copilot (no local recipe matching)", + ); + expect(messages[0]).toContain( + "Ordinary prompts: Copilot (no local recipe matching)", + ); + }, + ); + + it("uses the saved opt-in in the production adapter and still requires a native session", async () => { + const adapter = new MacroCatalogAdapter(); + writeConfig({ mode: "bypass", bypassMacroTools: false }); + expect((await adapter.listMacros({})).content[0]).toMatchObject({ + text: expect.stringContaining("disabled in bypass"), + }); + writeConfig({ mode: "bypass", bypassMacroTools: true }); + expect((await adapter.listMacros({})).content[0]).toMatchObject({ + text: "Local recipes require the actual native Copilot session.", + }); + }); + + it("leaves ordinary bypass prompts with Copilot even when macro tools are enabled", async () => { + writeConfig({ mode: "bypass", bypassMacroTools: true }); + const localRecipes = jest.fn(async () => ({})); + await expect( + routePrompt( + { + prompt: "Read the latest package metadata", + sessionId: "prompt-session", + timestamp: 1, + cwd: directory, + }, + "bypass", + new AbortController().signal, + { + recipeOwner: () => "plugin", + claimRecording: jest.fn(async () => false), + localRecipes, + }, + ), + ).resolves.toEqual({}); + expect(localRecipes).not.toHaveBeenCalled(); + }); +}); diff --git a/ts/packages/copilot-plugin/test/localRecipeService.spec.ts b/ts/packages/copilot-plugin/test/localRecipeService.spec.ts index 54afcd0e4f..4e93b22781 100644 --- a/ts/packages/copilot-plugin/test/localRecipeService.spec.ts +++ b/ts/packages/copilot-plugin/test/localRecipeService.spec.ts @@ -7,6 +7,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { createLocalRecipeService } from "../src/extension/local-recipe-service.js"; import { connectToMacroOwner } from "../src/shared/macro-owner.js"; +import { MacroCatalogAdapter } from "../src/mcp/macroServer.js"; const originalData = process.env.TYPEAGENT_PLUGIN_DATA; const originalOwner = process.env.TYPEAGENT_RECIPE_OWNER; @@ -62,6 +63,26 @@ test("canonical macro commands and ordinary misses need no TypeAgent or model", await expect(service.connection.listMacros()).resolves.toEqual([]); expect(forbidden).not.toHaveBeenCalled(); }); +test("opted-in bypass macro calls still use the real local owner's execution checks", async () => { + const adapter = new MacroCatalogAdapter({ + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => true, + connect: () => connectToMacroOwner(directory, "native-session"), + }); + expect((await adapter.listMacros({})).isError).toBeUndefined(); + const response = await adapter.runMacro({ macroId: "missing-macro" }); + expect(response.isError).toBe(true); + expect(response.content[0]).not.toMatchObject({ + text: expect.stringContaining("disabled in bypass"), + }); + expect(forbidden).not.toHaveBeenCalled(); + process.env.TYPEAGENT_RECIPE_OWNER = "off"; + expect((await adapter.listMacros({})).isError).toBe(true); + expect((await adapter.runMacro({ macroId: "missing-macro" })).isError).toBe( + true, + ); + expect(forbidden).not.toHaveBeenCalled(); +}); test("MCP resolves its actual native session workspace independently of process cwd", async () => { expect(process.cwd()).not.toBe(directory); const client = await connectToMacroOwner(undefined, "native-session"); diff --git a/ts/packages/copilot-plugin/test/macroBypassIntegration.spec.ts b/ts/packages/copilot-plugin/test/macroBypassIntegration.spec.ts new file mode 100644 index 0000000000..367587a1d4 --- /dev/null +++ b/ts/packages/copilot-plugin/test/macroBypassIntegration.spec.ts @@ -0,0 +1,474 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"; +import { + induceMacroFromTrace, + MacroManager, + type RunMacroResponse, +} from "@typeagent/copilot-macros"; +import { compileRecipeProgram } from "@typeagent/copilot-macros/recipeCompiler"; +import { + recipeDigest, + sealRecipeArtifact, + type RecipeCapability, +} from "@typeagent/copilot-macros/recipeCore"; +import { RecipeExecutor } from "@typeagent/copilot-macros/recipeExecutor"; +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { createLocalRecipeService } from "../src/extension/local-recipe-service.js"; +import { captureNativeRecipeBinding } from "../src/extension/recipe-native-host.js"; +import { createNativeTypeAgentRecipeHost } from "../src/extension/recipe-typeagent-host.js"; +import { + localRecipeDirectory, + localRecipeScope, +} from "../src/shared/recipe-local-paths.js"; +import { writeConfig } from "../src/shared/plugin-config.js"; +import { nativeFixture } from "./recipeNativeFixtures.js"; + +async function saveProgram( + manager: MacroManager, + cwd: string, + capability: RecipeCapability, +) { + const trace = { + schemaVersion: 1 as const, + sessionId: "fixture-capture", + cwd, + prompt: "read alpha.txt", + response: "fixture-read", + startedAt: "2026-10-23T00:00:00Z", + completedAt: "2026-10-23T00:00:01Z", + toolCalls: [ + { + toolCallId: "read", + name: "read", + mcpServerName: "fixture", + arguments: { path: "alpha.txt" }, + result: "fixture-read", + status: "completed" as const, + }, + ], + }; + const macro = await induceMacroFromTrace( + "procedure:bypass-integration", + trace, + "bypass-read", + "Read file", + "Read current file contents", + trace.completedAt, + { + inspectTool: async () => ({ + toolName: "read", + mcpServerName: "fixture", + schemaFingerprint: capability.schemaDigest, + readOnly: true, + inputSchema: capability.inputSchema, + }), + callTool: async () => { + throw new Error("The step replayer must not run."); + }, + }, + ); + const source = `async function execute(api, params) { + if (params === null || typeof params !== "object" || Array.isArray(params)) + throw new Error("Expected file inputs."); + return await api.tools.call(${JSON.stringify(capability.id)}, { path: params.step_1_path }); + }`; + macro.executionKind = "program"; + macro.program = sealRecipeArtifact({ + schemaVersion: 1, + kind: "program", + recipeId: macro.macroId, + name: macro.name, + description: macro.description, + execution: "symbolic", + authorization: "live_host", + source: { + language: "typescript", + text: source, + digest: recipeDigest(source), + }, + executable: compileRecipeProgram(source), + inputSchema: { + type: "object", + properties: { step_1_path: { type: "string" } }, + required: ["step_1_path"], + additionalProperties: false, + }, + outputSchema: { type: "string" }, + output: { + kind: "value", + requirements: ["Return the current read result"], + identity: "current_run", + }, + capabilities: [capability], + semanticFunctions: [], + applicability: { assumptions: [], unsupported: [] }, + generalization: [], + specification: { + id: "bypass-read-spec", + revision: "1", + intent: "Read file contents", + outputRequirements: ["Return the current read result"], + rubric: "Exact", + counterexamples: ["Delete a file"], + }, + matching: { + grammar: [ + " = read $(step_1_path:string) -> { step_1_path };", + ], + examples: [ + { + request: trace.prompt, + input: { step_1_path: "alpha.txt" }, + evidenceIds: ["fixture-evidence"], + }, + ], + negatives: ["delete file"], + collisionEvidenceIds: [], + }, + provenance: { + captureId: macro.sourceTraceId, + taskId: "fixture-task", + tokenId: "fixture-token", + evidenceIds: ["fixture-evidence"], + legacySource: { macroId: macro.macroId, version: macro.version }, + }, + fixtures: [ + { + id: "fixture", + origin: "synthetic", + result: "passed", + specificationRevision: "1", + noLiveEffects: true, + }, + ], + createdAt: trace.completedAt, + }); + const draft = await manager.saveDraft(macro); + const approved = await manager.approveMacro(draft); + return { + draft: { macroId: draft.macroId, version: draft.version }, + approved: { macroId: approved.macroId, version: approved.version }, + }; +} + +describe("bypass macros over the production stdio MCP server", () => { + const originalData = process.env.TYPEAGENT_PLUGIN_DATA; + const originalOwner = process.env.TYPEAGENT_RECIPE_OWNER; + let directory: string; + let fixture: ReturnType; + let executor: RecipeExecutor; + let service: Awaited>; + let client: Client | undefined; + let refs: Awaited>; + + beforeEach(async () => { + directory = await mkdtemp(join(tmpdir(), "bypass-macro-integration-")); + process.env.TYPEAGENT_PLUGIN_DATA = directory; + delete process.env.TYPEAGENT_RECIPE_OWNER; + writeConfig({ mode: "bypass" }); + fixture = nativeFixture("bypass-program-session"); + fixture.metadata.workingDirectory = directory; + const scope = localRecipeScope(directory); + const binding = await captureNativeRecipeBinding( + fixture.session, + scope.profileId, + ); + const bridge = createNativeTypeAgentRecipeHost( + fixture.session, + binding, + { + invokeSemantic: async () => { + throw new Error( + "A symbolic program must not call a model.", + ); + }, + }, + { + readOnlyMcpTools: [ + { mcpServerName: "fixture", toolName: "read" }, + ], + }, + ); + const inspected = await bridge.inspectTool({ + mcpServerName: "fixture", + toolName: "read", + }); + if (inspected.status !== "available") + throw new Error(inspected.message); + const manager = new MacroManager( + localRecipeDirectory(directory), + undefined, + scope.profileId, + ); + try { + refs = await saveProgram(manager, directory, inspected.capability); + } finally { + await manager.close(); + } + executor = new RecipeExecutor(); + const noLearning = async (): Promise => { + throw new Error("Executing a fixture must not start learning."); + }; + service = await createLocalRecipeService({ + cwd: directory, + sessionId: fixture.session.sessionId, + report: (message) => { + throw new Error(message); + }, + replayHost: { + inspectTool: async () => { + throw new Error("Execution must use the program host."); + }, + callTool: async () => { + throw new Error("Execution must not replay steps."); + }, + }, + learning: () => ({ + withAuthorQuery: () => { + throw new Error("Execution must not start authorship."); + }, + extract: noLearning, + build: noLearning, + generateGrammar: noLearning, + validateGrammar: noLearning, + buildProgram: noLearning, + }), + program: { + execute: (artifact, input, context) => + executor.execute( + artifact, + input, + { + runId: context.runId, + signal: context.signal, + binding, + host: bridge.host, + }, + { + timeoutMs: context.timeoutMs, + onCheckpoint: context.onCheckpoint, + }, + ), + }, + }); + }); + + afterEach(async () => { + await client?.close(); + client = undefined; + await service?.close(); + await executor?.close(); + if (originalData === undefined) + delete process.env.TYPEAGENT_PLUGIN_DATA; + else process.env.TYPEAGENT_PLUGIN_DATA = originalData; + if (originalOwner === undefined) + delete process.env.TYPEAGENT_RECIPE_OWNER; + else process.env.TYPEAGENT_RECIPE_OWNER = originalOwner; + await rm(directory, { recursive: true, force: true }); + }); + + async function connect(flag?: string) { + client = new Client({ name: "bypass-program-test", version: "1" }); + await client.connect( + new StdioClientTransport({ + command: process.execPath, + args: [ + fileURLToPath(new URL("../mcp/server.js", import.meta.url)), + "--macros", + ], + cwd: directory, + env: { + ...(process.env.SystemRoot + ? { SystemRoot: process.env.SystemRoot } + : {}), + TYPEAGENT_PLUGIN_DATA: directory, + TYPEAGENT_MODE: "bypass", + TYPEAGENT_RECIPE_PROFILE_ID: + localRecipeScope(directory).profileId, + COPILOT_AGENT_SESSION_ID: fixture.session.sessionId, + ...(flag === undefined + ? {} + : { TYPEAGENT_BYPASS_MACRO_TOOLS: flag }), + }, + stderr: "pipe", + }), + ); + } + + async function call(name: string, args: Record = {}) { + if (!client) throw new Error("MCP client is not connected."); + const response = await client.callTool({ name, arguments: args }); + if ( + !Array.isArray(response.content) || + response.content[0]?.type !== "text" || + typeof response.content[0].text !== "string" + ) + throw new Error("Expected the macro server's text envelope."); + const text: string = response.content[0].text; + return { isError: response.isError === true, text }; + } + + it("discovers and executes an approved program, then reads its durable result", async () => { + await connect("true"); + const search = await call("search_macros", { query: "Read file" }); + expect(search.isError).toBe(false); + expect(search.text).toContain(refs.approved.macroId); + const requirements = await call("get_macro_requirements", { + macroId: refs.approved.macroId, + }); + expect(requirements.isError).toBe(false); + expect(requirements.text).toContain("step_1_path"); + expect(fixture.execute).not.toHaveBeenCalled(); + const response = await call("run_macro", { + macroId: refs.approved.macroId, + inputs: { step_1_path: "beta.txt" }, + }); + expect(response.isError).toBe(false); + const result = JSON.parse(response.text) as RunMacroResponse; + expect(result).toMatchObject({ + status: "completed", + run: { + ...refs.approved, + result: "fixture-read", + program: { status: "completed", fallback: "never" }, + }, + }); + expect(fixture.execute).toHaveBeenCalledTimes(1); + expect(fixture.execute).toHaveBeenCalledWith( + expect.objectContaining({ + name: "mcp_fixture_read", + arguments: { path: "beta.txt" }, + }), + ); + const persisted = await call("get_macro_run", { + runId: result.run.runId, + }); + expect(persisted.isError).toBe(false); + expect(JSON.parse(persisted.text)).toMatchObject({ + runId: result.run.runId, + status: "completed", + result: "fixture-read", + }); + }); + + it.each([undefined, "false", "invalid"])( + "blocks execution with opt-in %p before native dispatch", + async (flag) => { + await connect(flag); + expect( + ( + await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: "beta.txt" }, + }) + ).isError, + ).toBe(true); + expect(fixture.execute).not.toHaveBeenCalled(); + }, + ); + + it("honors the saved opt-in without an environment override", async () => { + writeConfig({ mode: "bypass", bypassMacroTools: true }); + await connect(); + const response = await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: "beta.txt" }, + }); + expect(response.isError).toBe(false); + expect(JSON.parse(response.text).status).toBe("completed"); + expect(fixture.execute).toHaveBeenCalledTimes(1); + }); + + it("blocks lifecycle operations even with the opt-in", async () => { + await connect("true"); + for (const name of [ + "validate_macro", + "approve_macro", + "disable_macro", + "delete_macro", + ]) { + const response = await call(name, { + macroId: refs.approved.macroId, + }); + expect(response.isError).toBe(true); + expect(response.text).toContain("management"); + } + expect( + await service.connection.inspectMacro(refs.approved), + ).toMatchObject({ state: "approved" }); + expect(fixture.execute).not.toHaveBeenCalled(); + }); + + it("rejects draft and disabled/stale versions without native dispatch", async () => { + await connect("true"); + const inputs = { step_1_path: "beta.txt" }; + expect( + (await call("run_macro", { ...refs.draft, inputs })).isError, + ).toBe(true); + await service.connection.disableMacro({ + macroId: refs.approved.macroId, + }); + expect( + (await call("run_macro", { ...refs.approved, inputs })).isError, + ).toBe(true); + expect( + ( + await call("run_macro", { + macroId: refs.approved.macroId, + inputs, + }) + ).isError, + ).toBe(true); + expect(fixture.execute).not.toHaveBeenCalled(); + }); + + it("retains a backend permission denial without retrying", async () => { + fixture.execute.mockResolvedValue({ + resultType: "denied", + textResultForLlm: "Fixture backend permission denied", + }); + await connect("true"); + const response = await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: "beta.txt" }, + }); + expect(response.isError).toBe(false); + expect(JSON.parse(response.text)).toMatchObject({ + status: "failed", + run: { + status: "failed", + error: { code: "denied" }, + program: { status: "denied", fallback: "never" }, + }, + }); + expect(fixture.execute).toHaveBeenCalledTimes(1); + }); + + it("rejects invalid inputs and a disabled owner before native dispatch", async () => { + await connect("true"); + expect( + ( + await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: 42 }, + }) + ).isError, + ).toBe(true); + process.env.TYPEAGENT_RECIPE_OWNER = "off"; + expect( + ( + await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: "beta.txt" }, + }) + ).isError, + ).toBe(true); + expect(fixture.execute).not.toHaveBeenCalled(); + }); +}); diff --git a/ts/packages/copilot-plugin/test/macroServer.spec.ts b/ts/packages/copilot-plugin/test/macroServer.spec.ts index 9e22443091..1128537477 100644 --- a/ts/packages/copilot-plugin/test/macroServer.spec.ts +++ b/ts/packages/copilot-plugin/test/macroServer.spec.ts @@ -2,6 +2,7 @@ // Licensed under the MIT License. import { jest } from "@jest/globals"; +import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; import type { MacroOwnerConnection } from "../src/shared/macro-owner.js"; import type { RunMacroRequest, @@ -17,7 +18,143 @@ function connection(overrides: Partial = {}) { } as MacroOwnerConnection; } +const operations: { + name: string; + allowInBypass: boolean; + invoke: (adapter: MacroCatalogAdapter) => Promise; +}[] = [ + { + name: "list", + allowInBypass: true, + invoke: (adapter) => adapter.listMacros({}), + }, + { + name: "search", + allowInBypass: true, + invoke: (adapter) => adapter.searchMacros({ query: "read" }), + }, + { + name: "inspect", + allowInBypass: true, + invoke: (adapter) => adapter.inspectMacro({ macroId: "macro-1" }), + }, + { + name: "requirements", + allowInBypass: true, + invoke: (adapter) => + adapter.getMacroRequirements({ macroId: "macro-1" }), + }, + { + name: "run", + allowInBypass: true, + invoke: (adapter) => adapter.runMacro({ macroId: "macro-1" }), + }, + { + name: "run status", + allowInBypass: true, + invoke: (adapter) => adapter.getMacroRun({ runId: "run-1" }), + }, + { + name: "validate", + allowInBypass: false, + invoke: (adapter) => adapter.validateMacro({ macroId: "macro-1" }), + }, + { + name: "approve", + allowInBypass: false, + invoke: (adapter) => adapter.approveMacro({ macroId: "macro-1" }), + }, + { + name: "disable", + allowInBypass: false, + invoke: (adapter) => adapter.disableMacro({ macroId: "macro-1" }), + }, + { + name: "delete", + allowInBypass: false, + invoke: (adapter) => adapter.deleteMacro({ macroId: "macro-1" }), + }, +]; + describe("macro MCP catalog adapter", () => { + it.each(operations)( + "rejects $name in bypass without opt-in before connecting", + async ({ invoke }) => { + const connect = jest.fn(async () => connection()); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => "bypass", + }); + expect((await invoke(adapter)).isError).toBe(true); + expect(connect).not.toHaveBeenCalled(); + }, + ); + + it.each(operations)( + "limits opted-in bypass access for $name", + async ({ invoke, allowInBypass }) => { + const connect = jest.fn(async (): Promise => { + throw new Error("Owner reached"); + }); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => true, + }); + const response = await invoke(adapter); + expect(response.isError).toBe(true); + expect(connect).toHaveBeenCalledTimes(allowInBypass ? 1 : 0); + expect(response.content[0]).toMatchObject({ + text: expect.stringContaining( + allowInBypass ? "Owner reached" : "management", + ), + }); + }, + ); + + it.each(["direct", "mcp", "dev"] as const)( + "preserves every macro operation in %s mode", + async (mode) => { + const connect = jest.fn(async (): Promise => { + throw new Error("Owner reached"); + }); + const optIn = jest.fn(() => false); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => mode, + getBypassMacroToolsEnabled: optIn, + }); + for (const { invoke } of operations) { + const response = await invoke(adapter); + expect(response.content[0]).toMatchObject({ + text: "Owner reached", + }); + } + expect(connect).toHaveBeenCalledTimes(operations.length); + expect(optIn).not.toHaveBeenCalled(); + }, + ); + + it("fails closed when reading an invalid opt-in", async () => { + const connect = jest.fn(async () => connection()); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => { + throw new Error("Invalid opt-in"); + }, + }); + for (const invoke of [ + () => adapter.listMacros({}), + () => adapter.runMacro({ macroId: "macro-1" }), + ]) { + expect((await invoke()).content[0]).toMatchObject({ + text: "Invalid opt-in", + }); + } + expect(connect).not.toHaveBeenCalled(); + }); + it("returns catalog data and closes its local owner connection", async () => { const client = connection({ listMacros: jest.fn(async () => [ @@ -66,38 +203,50 @@ describe("macro MCP catalog adapter", () => { expect(connect).not.toHaveBeenCalled(); }); - it("runs a macro with a preallocated run ID", async () => { - const runMacro = jest.fn(async (request: RunMacroRequest) => ({ - status: "completed" as const, - run: { - runId: request.runId, - macroId: request.macroId, - version: 2, + it.each(["dev", "bypass"] as const)( + "runs a macro with a preallocated run ID in %s", + async (mode) => { + const runMacro = jest.fn(async (request: RunMacroRequest) => ({ status: "completed" as const, - executionClass: "replayable" as const, - inputs: {}, - steps: [], - startedAt: "2026-08-14T10:00:00.000Z", - completedAt: "2026-08-14T10:00:01.000Z", - }, - })); - const client = connection({ runMacro }); - const adapter = new MacroCatalogAdapter({ - connect: jest.fn(async () => client), - getMode: () => "dev", - }); - - const response = await adapter.runMacro({ macroId: "macro-1" }); + run: { + runId: request.runId, + macroId: request.macroId, + version: 2, + status: "completed" as const, + executionClass: "replayable" as const, + inputs: {}, + steps: [], + startedAt: "2026-08-14T10:00:00.000Z", + completedAt: "2026-08-14T10:00:01.000Z", + }, + })); + const client = connection({ runMacro }); + const adapter = new MacroCatalogAdapter({ + connect: jest.fn(async () => client), + getMode: () => mode, + getBypassMacroToolsEnabled: () => true, + }); - expect(response.isError).toBeUndefined(); - expect(runMacro).toHaveBeenCalledWith( - expect.objectContaining({ - runId: expect.any(String), + const response = await adapter.runMacro({ macroId: "macro-1", - }), - ); - expect(client.close).toHaveBeenCalledTimes(1); - }); + version: 2, + inputs: { name: "alpha" }, + timeoutMs: 5000, + }); + + expect(response.isError).toBeUndefined(); + expect(runMacro).toHaveBeenCalledWith( + expect.objectContaining({ + runId: expect.any(String), + macroId: "macro-1", + version: 2, + inputs: { name: "alpha" }, + timeoutMs: 5000, + }), + ); + expect(client.close).toHaveBeenCalledTimes(1); + }, + ); it("returns a sanitized macro run record", async () => { const getMacroRun = jest.fn(async (runId: string) => ({ @@ -128,54 +277,110 @@ describe("macro MCP catalog adapter", () => { expect(client.close).toHaveBeenCalledTimes(1); }); - it("cancels the preallocated run when the MCP call is aborted", async () => { - let resolveRun: ((response: RunMacroResponse) => void) | undefined; - let notifyStarted: (() => void) | undefined; - const started = new Promise((resolve) => { - notifyStarted = resolve; - }); - const runMacro = jest.fn( - async (request: RunMacroRequest): Promise => { - notifyStarted?.(); - return new Promise((resolve) => { - resolveRun = resolve; - }); - }, - ); - const cancelMacroRun = jest.fn(async (runId: string) => { - resolveRun?.({ - status: "cancelled", - run: { - runId, - macroId: "macro-1", - version: 2, - status: "cancelled", - executionClass: "replayable", - inputs: {}, - steps: [], - startedAt: "2026-08-14T10:00:00.000Z", - completedAt: "2026-08-14T10:00:01.000Z", - error: { code: "cancelled", message: "Cancelled" }, + it.each(["dev", "bypass"] as const)( + "cancels the preallocated run when the %s MCP call is aborted", + async (mode) => { + let resolveRun: ((response: RunMacroResponse) => void) | undefined; + let notifyStarted: (() => void) | undefined; + const started = new Promise((resolve) => { + notifyStarted = resolve; + }); + const runMacro = jest.fn( + async (request: RunMacroRequest): Promise => { + notifyStarted?.(); + return new Promise((resolve) => { + resolveRun = resolve; + }); }, + ); + const cancelMacroRun = jest.fn(async (runId: string) => { + resolveRun?.({ + status: "cancelled", + run: { + runId, + macroId: "macro-1", + version: 2, + status: "cancelled", + executionClass: "replayable", + inputs: {}, + steps: [], + startedAt: "2026-08-14T10:00:00.000Z", + completedAt: "2026-08-14T10:00:01.000Z", + error: { code: "cancelled", message: "Cancelled" }, + }, + }); + }); + const client = connection({ runMacro, cancelMacroRun }); + const adapter = new MacroCatalogAdapter({ + connect: jest.fn(async () => client), + getMode: () => mode, + getBypassMacroToolsEnabled: () => true, + }); + const controller = new AbortController(); + + const response = adapter.runMacro( + { macroId: "macro-1" }, + controller.signal, + ); + await started; + controller.abort(); + + expect((await response).isError).toBeUndefined(); + const requestedRunId = runMacro.mock.calls[0][0].runId; + expect(cancelMacroRun).toHaveBeenCalledWith(requestedRunId); + expect(client.close).toHaveBeenCalledTimes(1); + }, + ); + + it.each(["before connection", "during connection"])( + "does not dispatch an already aborted run %s", + async (when) => { + const controller = new AbortController(); + const runMacro = jest.fn(); + const client = connection({ runMacro }); + const connect = jest.fn(async () => { + controller.abort(); + return client; }); + if (when === "before connection") controller.abort(); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => true, + }); + const response = await adapter.runMacro( + { macroId: "macro-1" }, + controller.signal, + ); + expect(response.isError).toBe(true); + expect(connect).toHaveBeenCalledTimes( + when === "before connection" ? 0 : 1, + ); + expect(client.close).toHaveBeenCalledTimes( + when === "before connection" ? 0 : 1, + ); + expect(runMacro).not.toHaveBeenCalled(); + }, + ); + + it("returns backend rejection once without a retry or fallback", async () => { + const runMacro = jest.fn(async (): Promise => { + throw new Error("Backend denied this macro"); }); - const client = connection({ runMacro, cancelMacroRun }); + const client = connection({ runMacro }); + const connect = jest.fn(async () => client); const adapter = new MacroCatalogAdapter({ - connect: jest.fn(async () => client), - getMode: () => "dev", + connect, + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => true, + }); + const response = await adapter.runMacro({ macroId: "macro-1" }); + expect(response).toMatchObject({ + isError: true, + content: [{ type: "text", text: "Backend denied this macro" }], }); - const controller = new AbortController(); - - const response = adapter.runMacro( - { macroId: "macro-1" }, - controller.signal, - ); - await started; - controller.abort(); - - expect((await response).isError).toBeUndefined(); - const requestedRunId = runMacro.mock.calls[0][0].runId; - expect(cancelMacroRun).toHaveBeenCalledWith(requestedRunId); + expect(connect).toHaveBeenCalledTimes(1); + expect(runMacro).toHaveBeenCalledTimes(1); expect(client.close).toHaveBeenCalledTimes(1); }); }); diff --git a/ts/packages/memory/service/test/timeline.spec.ts b/ts/packages/memory/service/test/timeline.spec.ts index a5e155e333..f4dc5675d7 100644 --- a/ts/packages/memory/service/test/timeline.spec.ts +++ b/ts/packages/memory/service/test/timeline.spec.ts @@ -87,6 +87,8 @@ describe("timeline configured service pipeline", () => { const open = () => new FileMemoryService(root, { viewDrafts: true, + maintenanceMode: "manual", + discoveryPolicy: "off", indexFactory: (_id, directory) => new FakeSemanticCorpusIndex(directory), }); diff --git a/ts/packages/memory/service/test/viewBuilds.spec.ts b/ts/packages/memory/service/test/viewBuilds.spec.ts index 6d02e61ea3..02b460ec87 100644 --- a/ts/packages/memory/service/test/viewBuilds.spec.ts +++ b/ts/packages/memory/service/test/viewBuilds.spec.ts @@ -372,6 +372,7 @@ describe("durable draft builds and explicit edit merge", () => { const open = () => new FileMemoryService(root, { informationalViews: false, + discoveryPolicy: "off", indexFactory: (_id, directory) => new FakeSemanticCorpusIndex(directory), viewSynthesisAdapter: { diff --git a/ts/packages/memory/service/test/viewEditorialTemporal.spec.ts b/ts/packages/memory/service/test/viewEditorialTemporal.spec.ts index 015659fb8c..b3915a6b61 100644 --- a/ts/packages/memory/service/test/viewEditorialTemporal.spec.ts +++ b/ts/packages/memory/service/test/viewEditorialTemporal.spec.ts @@ -166,17 +166,22 @@ class EditorialIndex extends FakeSemanticCorpusIndex { } async function settled(service: FileMemoryService, corpusId: string) { + const maintenance = await service.drainMaintenance(); + if ( + maintenance.reason || + maintenance.pending.length || + maintenance.running.length + ) + throw new Error( + `Offline editorial maintenance did not drain: ${JSON.stringify(maintenance)}`, + ); for (let attempt = 0; attempt < 1000; attempt++) { const snapshot = await service.listViews(corpusId); const running = (await service.listViewBuilds(corpusId)).some( (job) => job.state === "running", ); - if ( - !running && - !snapshot.wikiFamily?.pending && - !snapshot.projectFamily?.pending - ) - return snapshot; + // Family pending also includes conflicts awaiting a human decision. + if (!running) return snapshot; await new Promise((resolve) => setTimeout(resolve, 20)); } @@ -345,6 +350,7 @@ test("actual P0 revisions automate correction/withdrawal around human edits and const open = () => new FileMemoryService(root, { sourceSynopses: false, + discoveryPolicy: "off", indexFactory: (_id, directory) => new EditorialIndex(directory), eventIndexFactory: (_id, directory) => new FakeSemanticCorpusIndex(directory), diff --git a/ts/packages/memory/service/test/wikiConsolidationTemporal.spec.ts b/ts/packages/memory/service/test/wikiConsolidationTemporal.spec.ts index bd543ac524..9a1e1e5a19 100644 --- a/ts/packages/memory/service/test/wikiConsolidationTemporal.spec.ts +++ b/ts/packages/memory/service/test/wikiConsolidationTemporal.spec.ts @@ -25,6 +25,7 @@ test("actual P0 C4 clarification consolidates generated reviews losslessly, old const open = () => new FileMemoryService(root, { sourceSynopses: false, + discoveryPolicy: "off", indexFactory: (_id, directory) => new ConsolidationIndex(directory), viewSynthesisAdapter: adapter, }); @@ -173,6 +174,7 @@ test("actual P0 C5 scope split keeps an old navigation article and independent I const { adapter } = offlineConsolidationAdapter(); const service = new FileMemoryService(root, { sourceSynopses: false, + discoveryPolicy: "off", indexFactory: (_id, directory) => new ConsolidationIndex(directory), viewSynthesisAdapter: adapter, }); @@ -236,6 +238,7 @@ test("actual P0 C7 authenticated title/annotation and historical human edits pro const { adapter, calls } = offlineConsolidationAdapter(); const service = new FileMemoryService(root, { sourceSynopses: false, + discoveryPolicy: "off", indexFactory: (_id, directory) => new ConsolidationIndex(directory), viewSynthesisAdapter: adapter, });