From e1b9ae135b0c657663f2b2b35f1c1ea6973575cc Mon Sep 17 00:00:00 2001 From: George Ng Date: Fri, 23 Oct 2026 17:56:37 -0700 Subject: [PATCH 1/2] Allow opt-in Copilot macro tools in bypass mode Keep prompt routing and macro management disabled while allowing explicit discovery and checked execution through the existing local owner. Preserve backend authorization, reject invalid opt-ins, and stop aborted calls before dispatch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0040b8ff-2701-4fb7-821f-760e05fe6b4b --- ts/packages/copilot-plugin/README.md | 67 +++- .../copilot-plugin/src/extension/commands.ts | 12 +- .../copilot-plugin/src/hooks/hook-router.ts | 8 +- .../copilot-plugin/src/mcp/macroServer.ts | 87 +++-- .../src/shared/plugin-config.ts | 33 ++ .../test/bypassMacroTools.spec.ts | 182 +++++++++ .../test/localRecipeService.spec.ts | 21 ++ .../copilot-plugin/test/macroServer.spec.ts | 349 ++++++++++++++---- 8 files changed, 638 insertions(+), 121 deletions(-) create mode 100644 ts/packages/copilot-plugin/test/bypassMacroTools.spec.ts diff --git a/ts/packages/copilot-plugin/README.md b/ts/packages/copilot-plugin/README.md index 159286fe70..3ed1ac6b32 100644 --- a/ts/packages/copilot-plugin/README.md +++ b/ts/packages/copilot-plugin/README.md @@ -21,9 +21,9 @@ User prompt Explicit @typeagent run commands and optional structured action MCP tools remain available. They are not automatic recording or execution fallbacks. -Registered alongside routing (calls are disabled in bypass mode): +Registered alongside routing (calls are disabled in bypass mode by default): typeagent-workspace MCP tools - typeagent-macros MCP tools -> checked program execution + typeagent-macros MCP tools -> checked program execution (explicit bypass opt-in below) ``` The hook output fields `handled`, `responseContent`, and `handledBy` are supported in current Copilot CLI behavior, allowing the hook to skip the agentic loop entirely when TypeAgent handles a request. For local runtime debugging against the runtime repo, use `pnpm copilot:dev`. @@ -933,7 +933,7 @@ job data or create a manual draft to force a retry. ### Review and run a prepared program Use the lifecycle MCP tools in direct, MCP or dev mode. They call the selected -owner; bypass disables them. After learning produces a candidate: +owner; bypass disables them by default. After learning produces a candidate: 1. Inspect its returned candidate and version, then call `validate_macro`. 2. In Prepare mode, review it and call `approve_macro`. Drafts cannot run. @@ -945,6 +945,41 @@ owner; bypass disables them. After learning produces a candidate: `processCommand` does not create or approve macros. The lifecycle tools have no `preference` parameter, manual step-draft creation or adapted-candidate submission. +### Explicit macro tools while Copilot handles prompts + +To let Copilot discover and run existing approved macros without enabling prompt +routing, opt in before starting Copilot: + +```powershell +$env:TYPEAGENT_MODE = 'bypass' +$env:TYPEAGENT_BYPASS_MACRO_TOOLS = 'true' +pnpm copilot --experimental +``` + +Alternatively, set `"bypassMacroTools": true` alongside `"mode": "bypass"` in +the plugin config. The environment variable overrides the saved setting and +accepts `true`/`false`, `1`/`0`, or `on`/`off`; invalid values are rejected. +The default is `false`. Restart Copilot after changing environment variables; +config changes are read on subsequent calls. + +Only `list_macros`, `search_macros`, `inspect_macro`, `get_macro_requirements`, +`run_macro`, and `get_macro_run` are permitted by this opt-in. Validation, +approval, disablement, and deletion remain blocked in bypass; use direct, MCP, +or dev mode for those lifecycle operations. `@typeagent status` and +`/typeagent-status` report `Macro tools: discovery and execution only` when +the bypass opt-in is active. + +Ordinary prompts still go to Copilot without local recipe matching or automatic +TypeAgent delegation. Workspace and other TypeAgent MCP tools remain subject to +their existing mode gates. This setting does not change learning preferences, +enable a disabled recipe owner, or grant tool/backend permissions. + +Execution uses the same local owner and current approved immutable program +version as other modes. Unsupported tools, backend denials, failed or cancelled +runs remain failures; there is no planner fallback or automatic retry. This does +not add replay support for arbitrary native Copilot tools or make a captured +trace into a runnable macro. Approval and backend authorization are separate. + ### Route approved macros from natural language The plugin-local matcher uses the current approved program's generated grammar @@ -966,7 +1001,8 @@ disable and deletion refresh schemas and reconcile stored routes. ### Recording controls and recovery -Program authoring/execution is unconditional. Only these recording boundaries +Program authoring/execution has no legacy feature flag. Explicit macro access in +bypass is separately opt-in as described above. These recording boundaries remain configurable (enabled by default): | Environment variable | Boundary | @@ -1081,7 +1117,8 @@ that mode selection requires natural-language dispatch. Ordinary prompts use local checked-program matching followed by Copilot on a miss, in `direct`, `mcp` and `dev`. None of these modes selects a backend macro -owner or author. `bypass` disables recipe hook/MCP routing. +owner or author. `bypass` disables recipe hook/MCP routing by default; the explicit +macro-tool opt-in above permits discovery and execution without prompt matching. The settings remain for explicit TypeAgent commands and MCP availability. `@typeagent run ` intentionally delegates to the TypeAgent dispatcher; @@ -1141,20 +1178,22 @@ The plugin stores config at `%USERPROFILE%\.typeagent-copilot\config.json` (Wind ```json { "mode": "direct", - "mcpRouting": "delegate" + "mcpRouting": "delegate", + "bypassMacroTools": false } ``` **Environment variable overrides** (take precedence over config file): -| Variable | Default | Description | -| --------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -| `TYPEAGENT_MODE` | `direct` | `direct`, `mcp`, `dev`, or `bypass` | -| `TYPEAGENT_HOST` | `localhost` | TypeAgent server host | -| `TYPEAGENT_PORT` | `8999` | TypeAgent server port | -| `TYPEAGENT_CONVERSATION_ID` | Session-selected server default | Optional existing public conversation ID for all routes; overrides config `conversationId` and automatic selection | -| `TYPEAGENT_PLUGIN_DATA` | `~/.typeagent-copilot` | Config directory | -| `TYPEAGENT_WORKSPACE_ROOTS` | Copilot process working directory | Approved roots for workspace MCP tools, separated by the platform path delimiter | +| Variable | Default | Description | +| ------------------------------ | --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| `TYPEAGENT_MODE` | `direct` | `direct`, `mcp`, `dev`, or `bypass` | +| `TYPEAGENT_BYPASS_MACRO_TOOLS` | `false` | Opt in to explicit macro discovery/execution in bypass; overrides config `bypassMacroTools` without changing routing or permissions | +| `TYPEAGENT_HOST` | `localhost` | TypeAgent server host | +| `TYPEAGENT_PORT` | `8999` | TypeAgent server port | +| `TYPEAGENT_CONVERSATION_ID` | Session-selected server default | Optional existing public conversation ID for all routes; overrides config `conversationId` and automatic selection | +| `TYPEAGENT_PLUGIN_DATA` | `~/.typeagent-copilot` | Config directory | +| `TYPEAGENT_WORKSPACE_ROOTS` | Copilot process working directory | Approved roots for workspace MCP tools, separated by the platform path delimiter | --- diff --git a/ts/packages/copilot-plugin/src/extension/commands.ts b/ts/packages/copilot-plugin/src/extension/commands.ts index 39fc622189..304aa8d3f6 100644 --- a/ts/packages/copilot-plugin/src/extension/commands.ts +++ b/ts/packages/copilot-plugin/src/extension/commands.ts @@ -2,7 +2,12 @@ // Licensed under the MIT License. import type { CommandDefinition } from "@github/copilot-sdk"; -import { getConfigPath, getModeLabel } from "../shared/plugin-config.js"; +import { + getConfigPath, + getMacroToolAvailability, + getMode, + getModeLabel, +} from "../shared/plugin-config.js"; import { getModeDescription, handleModeSetting, @@ -70,7 +75,10 @@ function statusText(): string { "TypeAgent configuration", `Mode: ${getModeLabel()}`, `Routing: ${getModeDescription()}`, - "Ordinary prompts: local recipes, then Copilot", + getMode() === "bypass" + ? "Ordinary prompts: Copilot (no local recipe matching)" + : "Ordinary prompts: local recipes, then Copilot", + `Macro tools: ${getMacroToolAvailability()}`, `Server: ws://${host}:${port}`, `Config: ${getConfigPath()}`, "Mode settings are shared by sessions using this config.", diff --git a/ts/packages/copilot-plugin/src/hooks/hook-router.ts b/ts/packages/copilot-plugin/src/hooks/hook-router.ts index 186832e3fe..b18895852d 100644 --- a/ts/packages/copilot-plugin/src/hooks/hook-router.ts +++ b/ts/packages/copilot-plugin/src/hooks/hook-router.ts @@ -39,6 +39,7 @@ import { getConfigPath, getMode, getModeLabel, + getMacroToolAvailability, getRecipeOwner, type Mode, type RecipeOwner, @@ -192,8 +193,11 @@ function handleStatusCommand( "", `- Mode: **${getModeLabel()}**`, `- Routing: ${getModeDescription()}`, - "- Ordinary prompts: local recipes, then Copilot", - `- Macro workspace tools: **${mode === "bypass" ? "disabled" : "available"}**`, + mode === "bypass" + ? "- Ordinary prompts: Copilot (no local recipe matching)" + : "- Ordinary prompts: local recipes, then Copilot", + `- Workspace tools: **${mode === "bypass" ? "disabled" : "available"}**`, + `- Macro tools: **${getMacroToolAvailability()}**`, `- Server: ws://${host}:${port}`, `- Config: ${configPath}`, "- Mode settings are shared by sessions using this config.", diff --git a/ts/packages/copilot-plugin/src/mcp/macroServer.ts b/ts/packages/copilot-plugin/src/mcp/macroServer.ts index 1cdf00073c..4b05a31893 100644 --- a/ts/packages/copilot-plugin/src/mcp/macroServer.ts +++ b/ts/packages/copilot-plugin/src/mcp/macroServer.ts @@ -6,7 +6,11 @@ import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js" import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; import { randomUUID } from "node:crypto"; import { z } from "zod"; -import { getMode, type Mode } from "../shared/plugin-config.js"; +import { + getBypassMacroToolsEnabled, + getMode, + type Mode, +} from "../shared/plugin-config.js"; import { connectToMacroOwner, type MacroOwnerConnection, @@ -15,11 +19,13 @@ import { export interface MacroServerDependencies { connect: () => Promise; getMode: () => Mode; + getBypassMacroToolsEnabled?: () => boolean; } const defaultDependencies: MacroServerDependencies = { connect: connectToMacroOwner, getMode, + getBypassMacroToolsEnabled, }; function result(value: unknown): CallToolResult { @@ -57,30 +63,35 @@ export class MacroCatalogAdapter { state?: "draft" | "approved" | "disabled" | undefined; limit?: number | undefined; }) { - return this.call((connection) => - connection.listMacros({ - ...(request.state ? { state: request.state } : {}), - ...(request.limit !== undefined - ? { limit: request.limit } - : {}), - }), + return this.call( + (connection) => + connection.listMacros({ + ...(request.state ? { state: request.state } : {}), + ...(request.limit !== undefined + ? { limit: request.limit } + : {}), + }), + true, ); } searchMacros(request: { query: string; limit?: number | undefined }) { - return this.call((connection) => - connection.searchMacros({ - query: request.query, - ...(request.limit !== undefined - ? { limit: request.limit } - : {}), - }), + return this.call( + (connection) => + connection.searchMacros({ + query: request.query, + ...(request.limit !== undefined + ? { limit: request.limit } + : {}), + }), + true, ); } inspectMacro(request: { macroId: string; version?: number | undefined }) { - return this.call((connection) => - connection.inspectMacro(this.macroRef(request)), + return this.call( + (connection) => connection.inspectMacro(this.macroRef(request)), + true, ); } @@ -88,8 +99,10 @@ export class MacroCatalogAdapter { macroId: string; version?: number | undefined; }) { - return this.call((connection) => - connection.getMacroRequirements(this.macroRef(request)), + return this.call( + (connection) => + connection.getMacroRequirements(this.macroRef(request)), + true, ); } @@ -117,7 +130,10 @@ export class MacroCatalogAdapter { } getMacroRun(request: { runId: string }) { - return this.call((connection) => connection.getMacroRun(request.runId)); + return this.call( + (connection) => connection.getMacroRun(request.runId), + true, + ); } async runMacro( @@ -130,16 +146,14 @@ export class MacroCatalogAdapter { }, signal?: AbortSignal, ): Promise { - if (this.dependencies.getMode() === "bypass") { - return errorResult( - "TypeAgent macro tools are disabled in bypass mode.", - ); - } - const runId = randomUUID(); let connection: MacroOwnerConnection | undefined; let removeAbort: (() => void) | undefined; try { + this.assertAccess(true); + signal?.throwIfAborted(); + const runId = randomUUID(); connection = await this.dependencies.connect(); + signal?.throwIfAborted(); const activeConnection = connection; if (signal) { const cancel = () => { @@ -188,14 +202,11 @@ export class MacroCatalogAdapter { private async call( operation: (connection: MacroOwnerConnection) => Promise, + allowInBypass = false, ): Promise { - if (this.dependencies.getMode() === "bypass") { - return errorResult( - "TypeAgent macro tools are disabled in bypass mode.", - ); - } let connection: MacroOwnerConnection | undefined; try { + this.assertAccess(allowInBypass); connection = await this.dependencies.connect(); return result(await operation(connection)); } catch (error) { @@ -204,6 +215,20 @@ export class MacroCatalogAdapter { await connection?.close(); } } + + private assertAccess(allowInBypass: boolean): void { + if (this.dependencies.getMode() !== "bypass") return; + if ( + allowInBypass && + this.dependencies.getBypassMacroToolsEnabled?.() === true + ) + return; + throw new Error( + allowInBypass + ? "TypeAgent macro tools are disabled in bypass mode. Opt in with bypassMacroTools or TYPEAGENT_BYPASS_MACRO_TOOLS." + : "TypeAgent macro management tools are disabled in bypass mode.", + ); + } } const macroRefSchema = { diff --git a/ts/packages/copilot-plugin/src/shared/plugin-config.ts b/ts/packages/copilot-plugin/src/shared/plugin-config.ts index b4ab39302c..f7d169f160 100644 --- a/ts/packages/copilot-plugin/src/shared/plugin-config.ts +++ b/ts/packages/copilot-plugin/src/shared/plugin-config.ts @@ -18,6 +18,7 @@ export interface PluginConfig { /** Catalog revisions to expose to this Copilot session. */ selectedSkills?: SkillSelection[]; recipeOwner?: RecipeOwner; + bypassMacroTools?: boolean; recipeReadOnlyMcpTools?: { mcpServerName: string; toolName: string }[]; recipeInferenceCliPath?: string; [key: string]: unknown; @@ -163,6 +164,38 @@ export function getModeLabel(): string { return mode === "mcp" ? `mcp (${getMcpRouting()})` : mode; } +export function getBypassMacroToolsEnabled(): boolean { + const environment = process.env.TYPEAGENT_BYPASS_MACRO_TOOLS; + if (environment !== undefined) { + switch (environment.trim().toLowerCase()) { + case "true": + case "1": + case "on": + return true; + case "false": + case "0": + case "off": + return false; + default: + throw new Error( + "TYPEAGENT_BYPASS_MACRO_TOOLS must be true/false, 1/0, or on/off.", + ); + } + } + const configured = readConfig()?.bypassMacroTools; + if (configured === undefined) return false; + if (typeof configured !== "boolean") + throw new Error("bypassMacroTools must be a boolean."); + return configured; +} + +export function getMacroToolAvailability(): string { + if (getMode() !== "bypass") return "available"; + return getBypassMacroToolsEnabled() + ? "discovery and execution only" + : "disabled"; +} + export function getSelectedSkills(): SkillSelection[] { const environment = process.env.TYPEAGENT_SELECTED_SKILLS; const value = diff --git a/ts/packages/copilot-plugin/test/bypassMacroTools.spec.ts b/ts/packages/copilot-plugin/test/bypassMacroTools.spec.ts new file mode 100644 index 0000000000..755abebecd --- /dev/null +++ b/ts/packages/copilot-plugin/test/bypassMacroTools.spec.ts @@ -0,0 +1,182 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { jest } from "@jest/globals"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + getBypassMacroToolsEnabled, + getConfigPath, + getMacroToolAvailability, + writeConfig, +} from "../src/shared/plugin-config.js"; +import { createExtensionCommands } from "../src/extension/commands.js"; +import { handleSlashCommand, routePrompt } from "../src/hooks/hook-router.js"; +import { MacroCatalogAdapter } from "../src/mcp/macroServer.js"; + +describe("explicit macro tools in bypass mode", () => { + let directory: string; + const names = [ + "TYPEAGENT_PLUGIN_DATA", + "TYPEAGENT_MODE", + "TYPEAGENT_BYPASS_MACRO_TOOLS", + "COPILOT_AGENT_SESSION_ID", + "TYPEAGENT_RECIPE_OWNER", + ] as const; + const original = Object.fromEntries( + names.map((name) => [name, process.env[name]]), + ); + beforeEach(() => { + directory = mkdtempSync(join(tmpdir(), "bypass-macro-tools-")); + for (const name of names) delete process.env[name]; + process.env.TYPEAGENT_PLUGIN_DATA = directory; + }); + afterEach(() => { + rmSync(directory, { recursive: true, force: true }); + for (const name of names) { + if (original[name] === undefined) delete process.env[name]; + else process.env[name] = original[name]; + } + }); + + it("defaults to disabled without a file or explicit setting", () => { + expect(getBypassMacroToolsEnabled()).toBe(false); + writeConfig({ mode: "bypass" }); + expect(getBypassMacroToolsEnabled()).toBe(false); + expect(getMacroToolAvailability()).toBe("disabled"); + }); + + it.each([false, true])( + "reads the explicit %s config setting", + (enabled) => { + writeConfig({ mode: "bypass", bypassMacroTools: enabled }); + expect(getBypassMacroToolsEnabled()).toBe(enabled); + }, + ); + + it.each([ + ["true", true], + ["1", true], + ["on", true], + [" ON ", true], + ["false", false], + ["0", false], + ["off", false], + ] as const)( + "environment %s overrides the saved setting", + (value, enabled) => { + writeConfig({ mode: "bypass", bypassMacroTools: !enabled }); + process.env.TYPEAGENT_BYPASS_MACRO_TOOLS = value; + expect(getBypassMacroToolsEnabled()).toBe(enabled); + }, + ); + + it.each(["", "enabled", "truthy", "2"])( + "rejects invalid environment value %p", + (value) => { + writeConfig({ mode: "bypass", bypassMacroTools: true }); + process.env.TYPEAGENT_BYPASS_MACRO_TOOLS = value; + expect(() => getBypassMacroToolsEnabled()).toThrow( + "TYPEAGENT_BYPASS_MACRO_TOOLS must be", + ); + }, + ); + + it.each([null, "true", 1, [], {}])( + "rejects invalid configured value %p", + (value) => { + writeFileSync( + getConfigPath(), + JSON.stringify({ mode: "bypass", bypassMacroTools: value }), + ); + expect(() => getBypassMacroToolsEnabled()).toThrow( + "bypassMacroTools must be a boolean", + ); + }, + ); + + it("rejects malformed configuration rather than enabling macro tools", () => { + writeFileSync(getConfigPath(), "{broken"); + expect(() => getBypassMacroToolsEnabled()).toThrow( + "Unable to read plugin configuration", + ); + }); + + it.each(["direct", "mcp", "dev"] as const)( + "does not change macro availability in %s mode", + (mode) => { + writeConfig({ mode, bypassMacroTools: false }); + expect(getMacroToolAvailability()).toBe("available"); + }, + ); + + it.each([false, true])( + "reports matching hook and extension status with opt-in %s", + async (enabled) => { + writeConfig({ mode: "bypass", bypassMacroTools: enabled }); + const expected = enabled + ? "discovery and execution only" + : "disabled"; + const hook = await handleSlashCommand({ + prompt: "@typeagent status", + sessionId: "status-session", + timestamp: 1, + cwd: directory, + }); + const messages: string[] = []; + const command = createExtensionCommands(async (message) => { + messages.push(message); + }).find(({ name }) => name === "typeagent-status"); + await command?.handler({ args: "" } as never); + expect(hook?.responseContent).toContain( + `Macro tools: **${expected}**`, + ); + expect(hook?.responseContent).toContain( + "Workspace tools: **disabled**", + ); + expect(messages[0]).toContain(`Macro tools: ${expected}`); + expect(hook?.responseContent).toContain( + "Ordinary prompts: Copilot (no local recipe matching)", + ); + expect(messages[0]).toContain( + "Ordinary prompts: Copilot (no local recipe matching)", + ); + }, + ); + + it("uses the saved opt-in in the production adapter and still requires a native session", async () => { + const adapter = new MacroCatalogAdapter(); + writeConfig({ mode: "bypass", bypassMacroTools: false }); + expect((await adapter.listMacros({})).content[0]).toMatchObject({ + text: expect.stringContaining("disabled in bypass"), + }); + writeConfig({ mode: "bypass", bypassMacroTools: true }); + expect((await adapter.listMacros({})).content[0]).toMatchObject({ + text: "Local recipes require the actual native Copilot session.", + }); + }); + + it("leaves ordinary bypass prompts with Copilot even when macro tools are enabled", async () => { + writeConfig({ mode: "bypass", bypassMacroTools: true }); + const localRecipes = jest.fn(async () => ({})); + await expect( + routePrompt( + { + prompt: "Read the latest package metadata", + sessionId: "prompt-session", + timestamp: 1, + cwd: directory, + }, + "bypass", + new AbortController().signal, + { + recipeOwner: () => "plugin", + claimRecording: jest.fn(async () => false), + localRecipes, + }, + ), + ).resolves.toEqual({}); + expect(localRecipes).not.toHaveBeenCalled(); + }); +}); diff --git a/ts/packages/copilot-plugin/test/localRecipeService.spec.ts b/ts/packages/copilot-plugin/test/localRecipeService.spec.ts index 54afcd0e4f..4e93b22781 100644 --- a/ts/packages/copilot-plugin/test/localRecipeService.spec.ts +++ b/ts/packages/copilot-plugin/test/localRecipeService.spec.ts @@ -7,6 +7,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { createLocalRecipeService } from "../src/extension/local-recipe-service.js"; import { connectToMacroOwner } from "../src/shared/macro-owner.js"; +import { MacroCatalogAdapter } from "../src/mcp/macroServer.js"; const originalData = process.env.TYPEAGENT_PLUGIN_DATA; const originalOwner = process.env.TYPEAGENT_RECIPE_OWNER; @@ -62,6 +63,26 @@ test("canonical macro commands and ordinary misses need no TypeAgent or model", await expect(service.connection.listMacros()).resolves.toEqual([]); expect(forbidden).not.toHaveBeenCalled(); }); +test("opted-in bypass macro calls still use the real local owner's execution checks", async () => { + const adapter = new MacroCatalogAdapter({ + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => true, + connect: () => connectToMacroOwner(directory, "native-session"), + }); + expect((await adapter.listMacros({})).isError).toBeUndefined(); + const response = await adapter.runMacro({ macroId: "missing-macro" }); + expect(response.isError).toBe(true); + expect(response.content[0]).not.toMatchObject({ + text: expect.stringContaining("disabled in bypass"), + }); + expect(forbidden).not.toHaveBeenCalled(); + process.env.TYPEAGENT_RECIPE_OWNER = "off"; + expect((await adapter.listMacros({})).isError).toBe(true); + expect((await adapter.runMacro({ macroId: "missing-macro" })).isError).toBe( + true, + ); + expect(forbidden).not.toHaveBeenCalled(); +}); test("MCP resolves its actual native session workspace independently of process cwd", async () => { expect(process.cwd()).not.toBe(directory); const client = await connectToMacroOwner(undefined, "native-session"); diff --git a/ts/packages/copilot-plugin/test/macroServer.spec.ts b/ts/packages/copilot-plugin/test/macroServer.spec.ts index 9e22443091..1128537477 100644 --- a/ts/packages/copilot-plugin/test/macroServer.spec.ts +++ b/ts/packages/copilot-plugin/test/macroServer.spec.ts @@ -2,6 +2,7 @@ // Licensed under the MIT License. import { jest } from "@jest/globals"; +import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; import type { MacroOwnerConnection } from "../src/shared/macro-owner.js"; import type { RunMacroRequest, @@ -17,7 +18,143 @@ function connection(overrides: Partial = {}) { } as MacroOwnerConnection; } +const operations: { + name: string; + allowInBypass: boolean; + invoke: (adapter: MacroCatalogAdapter) => Promise; +}[] = [ + { + name: "list", + allowInBypass: true, + invoke: (adapter) => adapter.listMacros({}), + }, + { + name: "search", + allowInBypass: true, + invoke: (adapter) => adapter.searchMacros({ query: "read" }), + }, + { + name: "inspect", + allowInBypass: true, + invoke: (adapter) => adapter.inspectMacro({ macroId: "macro-1" }), + }, + { + name: "requirements", + allowInBypass: true, + invoke: (adapter) => + adapter.getMacroRequirements({ macroId: "macro-1" }), + }, + { + name: "run", + allowInBypass: true, + invoke: (adapter) => adapter.runMacro({ macroId: "macro-1" }), + }, + { + name: "run status", + allowInBypass: true, + invoke: (adapter) => adapter.getMacroRun({ runId: "run-1" }), + }, + { + name: "validate", + allowInBypass: false, + invoke: (adapter) => adapter.validateMacro({ macroId: "macro-1" }), + }, + { + name: "approve", + allowInBypass: false, + invoke: (adapter) => adapter.approveMacro({ macroId: "macro-1" }), + }, + { + name: "disable", + allowInBypass: false, + invoke: (adapter) => adapter.disableMacro({ macroId: "macro-1" }), + }, + { + name: "delete", + allowInBypass: false, + invoke: (adapter) => adapter.deleteMacro({ macroId: "macro-1" }), + }, +]; + describe("macro MCP catalog adapter", () => { + it.each(operations)( + "rejects $name in bypass without opt-in before connecting", + async ({ invoke }) => { + const connect = jest.fn(async () => connection()); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => "bypass", + }); + expect((await invoke(adapter)).isError).toBe(true); + expect(connect).not.toHaveBeenCalled(); + }, + ); + + it.each(operations)( + "limits opted-in bypass access for $name", + async ({ invoke, allowInBypass }) => { + const connect = jest.fn(async (): Promise => { + throw new Error("Owner reached"); + }); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => true, + }); + const response = await invoke(adapter); + expect(response.isError).toBe(true); + expect(connect).toHaveBeenCalledTimes(allowInBypass ? 1 : 0); + expect(response.content[0]).toMatchObject({ + text: expect.stringContaining( + allowInBypass ? "Owner reached" : "management", + ), + }); + }, + ); + + it.each(["direct", "mcp", "dev"] as const)( + "preserves every macro operation in %s mode", + async (mode) => { + const connect = jest.fn(async (): Promise => { + throw new Error("Owner reached"); + }); + const optIn = jest.fn(() => false); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => mode, + getBypassMacroToolsEnabled: optIn, + }); + for (const { invoke } of operations) { + const response = await invoke(adapter); + expect(response.content[0]).toMatchObject({ + text: "Owner reached", + }); + } + expect(connect).toHaveBeenCalledTimes(operations.length); + expect(optIn).not.toHaveBeenCalled(); + }, + ); + + it("fails closed when reading an invalid opt-in", async () => { + const connect = jest.fn(async () => connection()); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => { + throw new Error("Invalid opt-in"); + }, + }); + for (const invoke of [ + () => adapter.listMacros({}), + () => adapter.runMacro({ macroId: "macro-1" }), + ]) { + expect((await invoke()).content[0]).toMatchObject({ + text: "Invalid opt-in", + }); + } + expect(connect).not.toHaveBeenCalled(); + }); + it("returns catalog data and closes its local owner connection", async () => { const client = connection({ listMacros: jest.fn(async () => [ @@ -66,38 +203,50 @@ describe("macro MCP catalog adapter", () => { expect(connect).not.toHaveBeenCalled(); }); - it("runs a macro with a preallocated run ID", async () => { - const runMacro = jest.fn(async (request: RunMacroRequest) => ({ - status: "completed" as const, - run: { - runId: request.runId, - macroId: request.macroId, - version: 2, + it.each(["dev", "bypass"] as const)( + "runs a macro with a preallocated run ID in %s", + async (mode) => { + const runMacro = jest.fn(async (request: RunMacroRequest) => ({ status: "completed" as const, - executionClass: "replayable" as const, - inputs: {}, - steps: [], - startedAt: "2026-08-14T10:00:00.000Z", - completedAt: "2026-08-14T10:00:01.000Z", - }, - })); - const client = connection({ runMacro }); - const adapter = new MacroCatalogAdapter({ - connect: jest.fn(async () => client), - getMode: () => "dev", - }); - - const response = await adapter.runMacro({ macroId: "macro-1" }); + run: { + runId: request.runId, + macroId: request.macroId, + version: 2, + status: "completed" as const, + executionClass: "replayable" as const, + inputs: {}, + steps: [], + startedAt: "2026-08-14T10:00:00.000Z", + completedAt: "2026-08-14T10:00:01.000Z", + }, + })); + const client = connection({ runMacro }); + const adapter = new MacroCatalogAdapter({ + connect: jest.fn(async () => client), + getMode: () => mode, + getBypassMacroToolsEnabled: () => true, + }); - expect(response.isError).toBeUndefined(); - expect(runMacro).toHaveBeenCalledWith( - expect.objectContaining({ - runId: expect.any(String), + const response = await adapter.runMacro({ macroId: "macro-1", - }), - ); - expect(client.close).toHaveBeenCalledTimes(1); - }); + version: 2, + inputs: { name: "alpha" }, + timeoutMs: 5000, + }); + + expect(response.isError).toBeUndefined(); + expect(runMacro).toHaveBeenCalledWith( + expect.objectContaining({ + runId: expect.any(String), + macroId: "macro-1", + version: 2, + inputs: { name: "alpha" }, + timeoutMs: 5000, + }), + ); + expect(client.close).toHaveBeenCalledTimes(1); + }, + ); it("returns a sanitized macro run record", async () => { const getMacroRun = jest.fn(async (runId: string) => ({ @@ -128,54 +277,110 @@ describe("macro MCP catalog adapter", () => { expect(client.close).toHaveBeenCalledTimes(1); }); - it("cancels the preallocated run when the MCP call is aborted", async () => { - let resolveRun: ((response: RunMacroResponse) => void) | undefined; - let notifyStarted: (() => void) | undefined; - const started = new Promise((resolve) => { - notifyStarted = resolve; - }); - const runMacro = jest.fn( - async (request: RunMacroRequest): Promise => { - notifyStarted?.(); - return new Promise((resolve) => { - resolveRun = resolve; - }); - }, - ); - const cancelMacroRun = jest.fn(async (runId: string) => { - resolveRun?.({ - status: "cancelled", - run: { - runId, - macroId: "macro-1", - version: 2, - status: "cancelled", - executionClass: "replayable", - inputs: {}, - steps: [], - startedAt: "2026-08-14T10:00:00.000Z", - completedAt: "2026-08-14T10:00:01.000Z", - error: { code: "cancelled", message: "Cancelled" }, + it.each(["dev", "bypass"] as const)( + "cancels the preallocated run when the %s MCP call is aborted", + async (mode) => { + let resolveRun: ((response: RunMacroResponse) => void) | undefined; + let notifyStarted: (() => void) | undefined; + const started = new Promise((resolve) => { + notifyStarted = resolve; + }); + const runMacro = jest.fn( + async (request: RunMacroRequest): Promise => { + notifyStarted?.(); + return new Promise((resolve) => { + resolveRun = resolve; + }); }, + ); + const cancelMacroRun = jest.fn(async (runId: string) => { + resolveRun?.({ + status: "cancelled", + run: { + runId, + macroId: "macro-1", + version: 2, + status: "cancelled", + executionClass: "replayable", + inputs: {}, + steps: [], + startedAt: "2026-08-14T10:00:00.000Z", + completedAt: "2026-08-14T10:00:01.000Z", + error: { code: "cancelled", message: "Cancelled" }, + }, + }); + }); + const client = connection({ runMacro, cancelMacroRun }); + const adapter = new MacroCatalogAdapter({ + connect: jest.fn(async () => client), + getMode: () => mode, + getBypassMacroToolsEnabled: () => true, + }); + const controller = new AbortController(); + + const response = adapter.runMacro( + { macroId: "macro-1" }, + controller.signal, + ); + await started; + controller.abort(); + + expect((await response).isError).toBeUndefined(); + const requestedRunId = runMacro.mock.calls[0][0].runId; + expect(cancelMacroRun).toHaveBeenCalledWith(requestedRunId); + expect(client.close).toHaveBeenCalledTimes(1); + }, + ); + + it.each(["before connection", "during connection"])( + "does not dispatch an already aborted run %s", + async (when) => { + const controller = new AbortController(); + const runMacro = jest.fn(); + const client = connection({ runMacro }); + const connect = jest.fn(async () => { + controller.abort(); + return client; }); + if (when === "before connection") controller.abort(); + const adapter = new MacroCatalogAdapter({ + connect, + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => true, + }); + const response = await adapter.runMacro( + { macroId: "macro-1" }, + controller.signal, + ); + expect(response.isError).toBe(true); + expect(connect).toHaveBeenCalledTimes( + when === "before connection" ? 0 : 1, + ); + expect(client.close).toHaveBeenCalledTimes( + when === "before connection" ? 0 : 1, + ); + expect(runMacro).not.toHaveBeenCalled(); + }, + ); + + it("returns backend rejection once without a retry or fallback", async () => { + const runMacro = jest.fn(async (): Promise => { + throw new Error("Backend denied this macro"); }); - const client = connection({ runMacro, cancelMacroRun }); + const client = connection({ runMacro }); + const connect = jest.fn(async () => client); const adapter = new MacroCatalogAdapter({ - connect: jest.fn(async () => client), - getMode: () => "dev", + connect, + getMode: () => "bypass", + getBypassMacroToolsEnabled: () => true, + }); + const response = await adapter.runMacro({ macroId: "macro-1" }); + expect(response).toMatchObject({ + isError: true, + content: [{ type: "text", text: "Backend denied this macro" }], }); - const controller = new AbortController(); - - const response = adapter.runMacro( - { macroId: "macro-1" }, - controller.signal, - ); - await started; - controller.abort(); - - expect((await response).isError).toBeUndefined(); - const requestedRunId = runMacro.mock.calls[0][0].runId; - expect(cancelMacroRun).toHaveBeenCalledWith(requestedRunId); + expect(connect).toHaveBeenCalledTimes(1); + expect(runMacro).toHaveBeenCalledTimes(1); expect(client.close).toHaveBeenCalledTimes(1); }); }); From 90cfa6a7eee5c9d013ee5ad33479ece0ed6777ce Mon Sep 17 00:00:00 2001 From: George Ng Date: Fri, 23 Oct 2026 18:32:58 -0700 Subject: [PATCH 2/2] Exercise bypass macros through production MCP transport Add approved-program integration coverage using the actual bundled stdio server, local owner, compiled executor, and native host with an isolated SDK fixture. Verify opt-in precedence, durable output, version checks, lifecycle rejection, owner shutdown, and backend denial without retry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0040b8ff-2701-4fb7-821f-760e05fe6b4b --- .../test/macroBypassIntegration.spec.ts | 474 ++++++++++++++++++ 1 file changed, 474 insertions(+) create mode 100644 ts/packages/copilot-plugin/test/macroBypassIntegration.spec.ts diff --git a/ts/packages/copilot-plugin/test/macroBypassIntegration.spec.ts b/ts/packages/copilot-plugin/test/macroBypassIntegration.spec.ts new file mode 100644 index 0000000000..367587a1d4 --- /dev/null +++ b/ts/packages/copilot-plugin/test/macroBypassIntegration.spec.ts @@ -0,0 +1,474 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"; +import { + induceMacroFromTrace, + MacroManager, + type RunMacroResponse, +} from "@typeagent/copilot-macros"; +import { compileRecipeProgram } from "@typeagent/copilot-macros/recipeCompiler"; +import { + recipeDigest, + sealRecipeArtifact, + type RecipeCapability, +} from "@typeagent/copilot-macros/recipeCore"; +import { RecipeExecutor } from "@typeagent/copilot-macros/recipeExecutor"; +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { createLocalRecipeService } from "../src/extension/local-recipe-service.js"; +import { captureNativeRecipeBinding } from "../src/extension/recipe-native-host.js"; +import { createNativeTypeAgentRecipeHost } from "../src/extension/recipe-typeagent-host.js"; +import { + localRecipeDirectory, + localRecipeScope, +} from "../src/shared/recipe-local-paths.js"; +import { writeConfig } from "../src/shared/plugin-config.js"; +import { nativeFixture } from "./recipeNativeFixtures.js"; + +async function saveProgram( + manager: MacroManager, + cwd: string, + capability: RecipeCapability, +) { + const trace = { + schemaVersion: 1 as const, + sessionId: "fixture-capture", + cwd, + prompt: "read alpha.txt", + response: "fixture-read", + startedAt: "2026-10-23T00:00:00Z", + completedAt: "2026-10-23T00:00:01Z", + toolCalls: [ + { + toolCallId: "read", + name: "read", + mcpServerName: "fixture", + arguments: { path: "alpha.txt" }, + result: "fixture-read", + status: "completed" as const, + }, + ], + }; + const macro = await induceMacroFromTrace( + "procedure:bypass-integration", + trace, + "bypass-read", + "Read file", + "Read current file contents", + trace.completedAt, + { + inspectTool: async () => ({ + toolName: "read", + mcpServerName: "fixture", + schemaFingerprint: capability.schemaDigest, + readOnly: true, + inputSchema: capability.inputSchema, + }), + callTool: async () => { + throw new Error("The step replayer must not run."); + }, + }, + ); + const source = `async function execute(api, params) { + if (params === null || typeof params !== "object" || Array.isArray(params)) + throw new Error("Expected file inputs."); + return await api.tools.call(${JSON.stringify(capability.id)}, { path: params.step_1_path }); + }`; + macro.executionKind = "program"; + macro.program = sealRecipeArtifact({ + schemaVersion: 1, + kind: "program", + recipeId: macro.macroId, + name: macro.name, + description: macro.description, + execution: "symbolic", + authorization: "live_host", + source: { + language: "typescript", + text: source, + digest: recipeDigest(source), + }, + executable: compileRecipeProgram(source), + inputSchema: { + type: "object", + properties: { step_1_path: { type: "string" } }, + required: ["step_1_path"], + additionalProperties: false, + }, + outputSchema: { type: "string" }, + output: { + kind: "value", + requirements: ["Return the current read result"], + identity: "current_run", + }, + capabilities: [capability], + semanticFunctions: [], + applicability: { assumptions: [], unsupported: [] }, + generalization: [], + specification: { + id: "bypass-read-spec", + revision: "1", + intent: "Read file contents", + outputRequirements: ["Return the current read result"], + rubric: "Exact", + counterexamples: ["Delete a file"], + }, + matching: { + grammar: [ + " = read $(step_1_path:string) -> { step_1_path };", + ], + examples: [ + { + request: trace.prompt, + input: { step_1_path: "alpha.txt" }, + evidenceIds: ["fixture-evidence"], + }, + ], + negatives: ["delete file"], + collisionEvidenceIds: [], + }, + provenance: { + captureId: macro.sourceTraceId, + taskId: "fixture-task", + tokenId: "fixture-token", + evidenceIds: ["fixture-evidence"], + legacySource: { macroId: macro.macroId, version: macro.version }, + }, + fixtures: [ + { + id: "fixture", + origin: "synthetic", + result: "passed", + specificationRevision: "1", + noLiveEffects: true, + }, + ], + createdAt: trace.completedAt, + }); + const draft = await manager.saveDraft(macro); + const approved = await manager.approveMacro(draft); + return { + draft: { macroId: draft.macroId, version: draft.version }, + approved: { macroId: approved.macroId, version: approved.version }, + }; +} + +describe("bypass macros over the production stdio MCP server", () => { + const originalData = process.env.TYPEAGENT_PLUGIN_DATA; + const originalOwner = process.env.TYPEAGENT_RECIPE_OWNER; + let directory: string; + let fixture: ReturnType; + let executor: RecipeExecutor; + let service: Awaited>; + let client: Client | undefined; + let refs: Awaited>; + + beforeEach(async () => { + directory = await mkdtemp(join(tmpdir(), "bypass-macro-integration-")); + process.env.TYPEAGENT_PLUGIN_DATA = directory; + delete process.env.TYPEAGENT_RECIPE_OWNER; + writeConfig({ mode: "bypass" }); + fixture = nativeFixture("bypass-program-session"); + fixture.metadata.workingDirectory = directory; + const scope = localRecipeScope(directory); + const binding = await captureNativeRecipeBinding( + fixture.session, + scope.profileId, + ); + const bridge = createNativeTypeAgentRecipeHost( + fixture.session, + binding, + { + invokeSemantic: async () => { + throw new Error( + "A symbolic program must not call a model.", + ); + }, + }, + { + readOnlyMcpTools: [ + { mcpServerName: "fixture", toolName: "read" }, + ], + }, + ); + const inspected = await bridge.inspectTool({ + mcpServerName: "fixture", + toolName: "read", + }); + if (inspected.status !== "available") + throw new Error(inspected.message); + const manager = new MacroManager( + localRecipeDirectory(directory), + undefined, + scope.profileId, + ); + try { + refs = await saveProgram(manager, directory, inspected.capability); + } finally { + await manager.close(); + } + executor = new RecipeExecutor(); + const noLearning = async (): Promise => { + throw new Error("Executing a fixture must not start learning."); + }; + service = await createLocalRecipeService({ + cwd: directory, + sessionId: fixture.session.sessionId, + report: (message) => { + throw new Error(message); + }, + replayHost: { + inspectTool: async () => { + throw new Error("Execution must use the program host."); + }, + callTool: async () => { + throw new Error("Execution must not replay steps."); + }, + }, + learning: () => ({ + withAuthorQuery: () => { + throw new Error("Execution must not start authorship."); + }, + extract: noLearning, + build: noLearning, + generateGrammar: noLearning, + validateGrammar: noLearning, + buildProgram: noLearning, + }), + program: { + execute: (artifact, input, context) => + executor.execute( + artifact, + input, + { + runId: context.runId, + signal: context.signal, + binding, + host: bridge.host, + }, + { + timeoutMs: context.timeoutMs, + onCheckpoint: context.onCheckpoint, + }, + ), + }, + }); + }); + + afterEach(async () => { + await client?.close(); + client = undefined; + await service?.close(); + await executor?.close(); + if (originalData === undefined) + delete process.env.TYPEAGENT_PLUGIN_DATA; + else process.env.TYPEAGENT_PLUGIN_DATA = originalData; + if (originalOwner === undefined) + delete process.env.TYPEAGENT_RECIPE_OWNER; + else process.env.TYPEAGENT_RECIPE_OWNER = originalOwner; + await rm(directory, { recursive: true, force: true }); + }); + + async function connect(flag?: string) { + client = new Client({ name: "bypass-program-test", version: "1" }); + await client.connect( + new StdioClientTransport({ + command: process.execPath, + args: [ + fileURLToPath(new URL("../mcp/server.js", import.meta.url)), + "--macros", + ], + cwd: directory, + env: { + ...(process.env.SystemRoot + ? { SystemRoot: process.env.SystemRoot } + : {}), + TYPEAGENT_PLUGIN_DATA: directory, + TYPEAGENT_MODE: "bypass", + TYPEAGENT_RECIPE_PROFILE_ID: + localRecipeScope(directory).profileId, + COPILOT_AGENT_SESSION_ID: fixture.session.sessionId, + ...(flag === undefined + ? {} + : { TYPEAGENT_BYPASS_MACRO_TOOLS: flag }), + }, + stderr: "pipe", + }), + ); + } + + async function call(name: string, args: Record = {}) { + if (!client) throw new Error("MCP client is not connected."); + const response = await client.callTool({ name, arguments: args }); + if ( + !Array.isArray(response.content) || + response.content[0]?.type !== "text" || + typeof response.content[0].text !== "string" + ) + throw new Error("Expected the macro server's text envelope."); + const text: string = response.content[0].text; + return { isError: response.isError === true, text }; + } + + it("discovers and executes an approved program, then reads its durable result", async () => { + await connect("true"); + const search = await call("search_macros", { query: "Read file" }); + expect(search.isError).toBe(false); + expect(search.text).toContain(refs.approved.macroId); + const requirements = await call("get_macro_requirements", { + macroId: refs.approved.macroId, + }); + expect(requirements.isError).toBe(false); + expect(requirements.text).toContain("step_1_path"); + expect(fixture.execute).not.toHaveBeenCalled(); + const response = await call("run_macro", { + macroId: refs.approved.macroId, + inputs: { step_1_path: "beta.txt" }, + }); + expect(response.isError).toBe(false); + const result = JSON.parse(response.text) as RunMacroResponse; + expect(result).toMatchObject({ + status: "completed", + run: { + ...refs.approved, + result: "fixture-read", + program: { status: "completed", fallback: "never" }, + }, + }); + expect(fixture.execute).toHaveBeenCalledTimes(1); + expect(fixture.execute).toHaveBeenCalledWith( + expect.objectContaining({ + name: "mcp_fixture_read", + arguments: { path: "beta.txt" }, + }), + ); + const persisted = await call("get_macro_run", { + runId: result.run.runId, + }); + expect(persisted.isError).toBe(false); + expect(JSON.parse(persisted.text)).toMatchObject({ + runId: result.run.runId, + status: "completed", + result: "fixture-read", + }); + }); + + it.each([undefined, "false", "invalid"])( + "blocks execution with opt-in %p before native dispatch", + async (flag) => { + await connect(flag); + expect( + ( + await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: "beta.txt" }, + }) + ).isError, + ).toBe(true); + expect(fixture.execute).not.toHaveBeenCalled(); + }, + ); + + it("honors the saved opt-in without an environment override", async () => { + writeConfig({ mode: "bypass", bypassMacroTools: true }); + await connect(); + const response = await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: "beta.txt" }, + }); + expect(response.isError).toBe(false); + expect(JSON.parse(response.text).status).toBe("completed"); + expect(fixture.execute).toHaveBeenCalledTimes(1); + }); + + it("blocks lifecycle operations even with the opt-in", async () => { + await connect("true"); + for (const name of [ + "validate_macro", + "approve_macro", + "disable_macro", + "delete_macro", + ]) { + const response = await call(name, { + macroId: refs.approved.macroId, + }); + expect(response.isError).toBe(true); + expect(response.text).toContain("management"); + } + expect( + await service.connection.inspectMacro(refs.approved), + ).toMatchObject({ state: "approved" }); + expect(fixture.execute).not.toHaveBeenCalled(); + }); + + it("rejects draft and disabled/stale versions without native dispatch", async () => { + await connect("true"); + const inputs = { step_1_path: "beta.txt" }; + expect( + (await call("run_macro", { ...refs.draft, inputs })).isError, + ).toBe(true); + await service.connection.disableMacro({ + macroId: refs.approved.macroId, + }); + expect( + (await call("run_macro", { ...refs.approved, inputs })).isError, + ).toBe(true); + expect( + ( + await call("run_macro", { + macroId: refs.approved.macroId, + inputs, + }) + ).isError, + ).toBe(true); + expect(fixture.execute).not.toHaveBeenCalled(); + }); + + it("retains a backend permission denial without retrying", async () => { + fixture.execute.mockResolvedValue({ + resultType: "denied", + textResultForLlm: "Fixture backend permission denied", + }); + await connect("true"); + const response = await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: "beta.txt" }, + }); + expect(response.isError).toBe(false); + expect(JSON.parse(response.text)).toMatchObject({ + status: "failed", + run: { + status: "failed", + error: { code: "denied" }, + program: { status: "denied", fallback: "never" }, + }, + }); + expect(fixture.execute).toHaveBeenCalledTimes(1); + }); + + it("rejects invalid inputs and a disabled owner before native dispatch", async () => { + await connect("true"); + expect( + ( + await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: 42 }, + }) + ).isError, + ).toBe(true); + process.env.TYPEAGENT_RECIPE_OWNER = "off"; + expect( + ( + await call("run_macro", { + ...refs.approved, + inputs: { step_1_path: "beta.txt" }, + }) + ).isError, + ).toBe(true); + expect(fixture.execute).not.toHaveBeenCalled(); + }); +});