Remove temporary tool preview feature flag by gandhipratik203 · Pull Request #88 · contextforge-org/contextforge-web-ui

gandhipratik203 · 2026-08-27T21:49:22Z

Summary

This removes the temporary VITE_ENABLE_TOOL_PREVIEW flag and makes the Tools details drawer always render the Try it / Definition tab layout.

Changes:

remove the tool-preview feature helper
always open Tools details on Try it
remove test env stubs and the obsolete false-flag test
remove Playwright, Dockerfile, and docker-compose.e2e flag wiring
remove the now-empty ImportMetaEnv block

Context

Closes Remove temporary tool preview feature flag after backend preview endpoint lands IBM/mcp-context-forge#6322.
Refs [UI-REWRITE]: Add preview experience to ToolDetailsPanel IBM/mcp-context-forge#5630.
Depends on [FEATURE][API]: Add tool preview endpoint for safe invocation dry-run IBM/mcp-context-forge#5629.
Builds on merged PR Add gated tool live invocation and snippets #79, where live invoke also rode the same Try-it feature flag.

Blocker

This PR is intentionally draft-only until the backend preview endpoint lands.

Removing the flag flips the production/default image path from Try-it hidden to Try-it visible. Since /rpc exists but /tools/preview/{name} does not until IBM/mcp-context-forge#5629 lands, the post-removal production state would expose gated live invoke beside a Preview button that 404s.

The current docker e2e gateway image is ghcr.io/ibm/mcp-context-forge:v1.0.8, which predates the preview endpoint.

Un-draft Gate

[FEATURE][API]: Add tool preview endpoint for safe invocation dry-run IBM/mcp-context-forge#5629 is merged.
docker-compose.e2e.yml is bumped to the first ghcr.io/ibm/mcp-context-forge release containing POST /tools/preview/{name}.
npm run e2e:docker passes against that image.
Manual verification confirms both Preview and Live invoke from the default Tools Try-it tab.

Tests

git grep -n "VITE_ENABLE_TOOL_PREVIEW\|isToolPreviewEnabled" -- . - no tracked matches
npm run lint
npm run format:check
./node_modules/.bin/tsc -b
npm run test
npm run build
PLAYWRIGHT_SKIP_WEBSERVER=1 PLAYWRIGHT_BASE_URL=http://127.0.0.1:5173 npm run e2e -- e2e/tools.spec.ts -g "previews a tool|live invokes|confirms destructive|cancellation|tools.execute|servers.use|federated|denied passthrough"
git diff --check

Not Run

npm run e2e:docker - blocked until the pinned gateway image contains [FEATURE][API]: Add tool preview endpoint for safe invocation dry-run IBM/mcp-context-forge#5629.

Explanatory Diagrams

Try-it request flow

User
 |
 | clicks Preview / Live invoke
 v
React UI
 |
 | browser request
 v
BFF server
 |
 | checks session
 | adds security/auth
 | forwards request
 v
ContextForge Gateway
 |
 | previews or runs the tool
 v
Tool
 |
 | returns result
 v
ContextForge Gateway
 |
 v
BFF server
 |
 v
React UI
 |
 v
User sees result

Manual Verification

Mock-backed happy-path verification was run for the default Tools Try-it drawer. This verifies frontend wiring only. A separate compatibility check against IBM/mcp-context-forge#6443 is documented below; npm run e2e:docker remains blocked until a released gateway image contains IBM/mcp-context-forge#5629.

Manual verification steps

Setup

git checkout feat/6322-remove-tool-preview-flag
npm ci                 # if node_modules is missing
npm run generate       # if src/generated/ is missing

Save the mock script from the next collapsible at the repo root as tool-try-it-happy-path-manual.mjs.

Two terminals:

# terminal A - dev server, no VITE_ENABLE_TOOL_PREVIEW needed
npm run dev

# terminal B - opens the mocked browser
node tool-try-it-happy-path-manual.mjs

Terminal B opens a Chrome for Testing window with /auth/session, /api/rbac/my/permissions, /api/tools, /api/gateways, /api/tools/preview/customer_lookup, and /api/rpc mocked. Ctrl-C in terminal B to close. Do everything in that window, in the tab it opens.

Steps

1. Open More options for demo-tools -> View details.
Expect: the details drawer opens with Try it selected by default. This should work without setting VITE_ENABLE_TOOL_PREVIEW.

2. Fill customer_id with acme-001.
Expect: the required argument is accepted by the schema form.

3. Leave include_orders checked if it defaults on, or toggle it on.

4. Click Add header. Enter X-Tenant-Id as the header name and demo-team as the value.

5. Inspect the snippet tabs.
Expect: curl, JSON-RPC, Python, and TypeScript tabs render with an MCP 2025-11-25 badge. Snippets target $MCPGATEWAY_URL/rpc, not browser-only /api/rpc, and do not include server_id.

6. Click Preview.
Expect: Preview 200, Tool result, Resolved arguments, and Raw preview response. Terminal B should log /api/tools/preview/customer_lookup with the filled arguments and x-tenant-id: demo-team.

7. Click Live invoke.
Expect: Live invoke 200, Tool result, the text Live result: Acme Corp is active., and structured output containing customer_id, customer_name, status, include_orders, and tenant_header.

8. Look at terminal B.
Expect: /api/rpc request body has method: "tools/call", params.name: "customer_lookup", the filled arguments, and no server_id. The interesting headers log includes x-tenant-id: demo-team.

Teardown

Ctrl-C both terminals. If :5173 is stuck:

lsof -ti:5173 | xargs kill

Mock script (tool-try-it-happy-path-manual.mjs)

Save at the repo root. Requires @playwright/test, already a dev dependency; run npx playwright install chromium if the browser is missing.

// Manual UI testing for contextforge-web-ui#88 - default Tools Try-it happy path.
//
//   npm run dev                               # terminal A, Vite on :5173
//   node tool-try-it-happy-path-manual.mjs   # terminal B
//
// Ctrl-C in terminal B to close the headed browser.
//
// This mocks the backend endpoints needed by /app/tools, including
// /api/tools/preview/customer_lookup and /api/rpc. It verifies frontend wiring
// only: the Try-it tab is visible without VITE_ENABLE_TOOL_PREVIEW, Preview
// uses the dry-run endpoint, Live invoke uses the JSON-RPC path, passthrough
// headers are forwarded, and direct tool invoke does not include server_id.

import { chromium } from "@playwright/test";

const BASE = process.env.BASE_URL ?? "http://localhost:5173";
const HEADED = !process.env.HEADLESS;

const USER = {
  email: "test@example.com",
  full_name: "Test User",
  is_admin: true,
  is_active: true,
  auth_provider: "local",
  email_verified: true,
  password_change_required: false,
};

const TOOL = {
  id: "tool-customer-lookup",
  name: "customer_lookup",
  originalName: "customer_lookup",
  description: "Look up a customer profile from a mocked MCP tool.",
  originalDescription: "Look up a customer profile from a mocked MCP tool.",
  title: "Customer lookup",
  displayName: "Customer lookup",
  gatewayId: "gw-demo-tools",
  gatewaySlug: "demo-tools",
  customName: "customer_lookup",
  customNameSlug: "customer_lookup",
  enabled: true,
  reachable: true,
  deprecated: false,
  executionCount: 12,
  successRate: 99,
  avgResponseTime: 86,
  tags: [{ id: "demo", label: "demo" }],
  integrationType: "mcp",
  requestType: "http",
  url: "https://demo.example/mcp",
  headers: {},
  inputSchema: {
    type: "object",
    required: ["customer_id"],
    properties: {
      customer_id: { type: "string", description: "Customer ID" },
      include_orders: { type: "boolean", description: "Include recent order summary" },
    },
  },
  outputSchema: { type: "object" },
  annotations: { readOnlyHint: true },
  jsonpathFilter: null,
  auth: null,
  visibility: "team",
  createdAt: "2026-04-10T10:00:00Z",
  updatedAt: "2026-08-20T09:30:00Z",
};

const GATEWAY_RESPONSE = {
  gateways: [
    {
      id: "gw-demo-tools",
      name: "demo-tools",
      url: "https://demo.example/mcp",
      description: "Mocked MCP gateway for the Tools Try-it happy-path demo",
    },
  ],
  nextCursor: null,
};

function json(body, status = 200) {
  return {
    status,
    contentType: "application/json",
    body: JSON.stringify(body),
  };
}

function fallbackApiBody(pathname) {
  if (pathname.startsWith("/api/resources")) return [];
  if (pathname.startsWith("/api/prompts")) return [];
  if (pathname.startsWith("/api/servers")) return [];
  if (pathname.startsWith("/api/gateways")) return { gateways: [], nextCursor: null };
  if (pathname.startsWith("/api/tools")) return [];
  return {};
}

function interestingHeaders(headers) {
  return Object.fromEntries(
    Object.entries(headers).filter(([name]) =>
      ["x-csrf-token", "x-tenant-id", "x-api-key", "authorization"].includes(name.toLowerCase()),
    ),
  );
}

function toolResult(text, args, tenantHeader) {
  return {
    content: [{ type: "text", text, mimeType: "text/plain" }],
    structured_output: {
      customer_id: args.customer_id ?? null,
      customer_name: "Acme Corp",
      status: "active",
      include_orders: Boolean(args.include_orders),
      tenant_header: tenantHeader ?? null,
    },
  };
}

const browser = await chromium.launch({ headless: !HEADED });
const context = await browser.newContext({ viewport: { width: 1512, height: 950 } });
const page = await context.newPage();

page.on("console", (message) => {
  if (["error", "warning"].includes(message.type())) {
    console.log(`browser ${message.type()}: ${message.text()}`);
  }
});
page.on("pageerror", (error) => {
  console.log(`browser pageerror: ${error.message}`);
});

// Register broad API fallbacks first. Playwright evaluates the newest matching
// route first, so endpoint-specific mocks below must be registered after this.
await page.route("**/*", (route) => {
  const pathname = new URL(route.request().url()).pathname;
  if (pathname.startsWith("/api/")) return route.fulfill(json(fallbackApiBody(pathname)));
  return route.fallback();
});

await page.route("**/auth/session", (route) =>
  route.fulfill(
    json({
      authenticated: true,
      user: USER,
      csrfToken: "mock-csrf-token",
    }),
  ),
);

await page.route("**/api/rbac/my/permissions**", (route) => route.fulfill(json(["*"])));
await page.route("**/api/tools?*", (route) => route.fulfill(json([TOOL])));
await page.route("**/api/gateways?*", (route) => route.fulfill(json(GATEWAY_RESPONSE)));

await page.route("**/api/tools/preview/customer_lookup", async (route) => {
  const request = route.request();
  const body = request.postDataJSON();
  const args = body?.arguments ?? {};
  const headers = request.headers();

  console.log("\npreview request body:");
  console.log(JSON.stringify(body, null, 2));
  console.log("preview interesting headers:");
  console.log(JSON.stringify(interestingHeaders(headers), null, 2));

  return route.fulfill(
    json({
      target: { kind: "local" },
      resolved_arguments: args,
      annotations: { readOnlyHint: true },
      pre_hooks_run: [],
      warnings: [],
      content: [
        {
          type: "text",
          text: "Preview validated the customer lookup arguments without executing the tool.",
          mimeType: "text/plain",
        },
      ],
      structured_output: {
        validated: true,
        customer_id: args.customer_id ?? null,
        include_orders: Boolean(args.include_orders),
      },
    }),
  );
});

await page.route("**/api/rpc", async (route) => {
  const request = route.request();
  const body = request.postDataJSON();
  const args = body?.params?.arguments ?? {};
  const headers = request.headers();

  console.log("\n/api/rpc request body:");
  console.log(JSON.stringify(body, null, 2));
  console.log("/api/rpc interesting headers:");
  console.log(JSON.stringify(interestingHeaders(headers), null, 2));
  console.log(`/api/rpc server_id present: ${Object.hasOwn(body?.params ?? {}, "server_id")}`);

  if (body?.method === "notifications/cancelled") {
    return route.fulfill(json({ jsonrpc: "2.0", id: body.id, result: {} }));
  }

  return route.fulfill(
    json({
      jsonrpc: "2.0",
      id: body.id,
      result: toolResult(
        "Live result: Acme Corp is active. Last order was placed on 2026-08-20.",
        args,
        headers["x-tenant-id"],
      ),
    }),
  );
});

await page.addInitScript(() => {
  sessionStorage.setItem("mcpgateway_token", "placeholder-token");
});

await page.goto(`${BASE}/app/tools`, { waitUntil: "networkidle" });

const cardCount = await page.getByRole("button", { name: "More options for demo-tools" }).count();
console.log(`tools card: ${cardCount ? "ok" : "MISSING"}`);

if (!HEADED) {
  await browser.close();
} else {
  console.log(`
Browser open. Try:

  1. Open "More options for demo-tools" -> "View details".
     Expect the details drawer to open with "Try it" selected by default.
     This should work without setting VITE_ENABLE_TOOL_PREVIEW.

  2. Fill customer_id="acme-001".
     Leave include_orders checked if it defaults on, or toggle it on.

  3. Add header X-Tenant-Id=demo-team.

  4. Inspect the snippet tabs.
     Expect curl, JSON-RPC, Python, TypeScript, and MCP 2025-11-25.
     Snippets should target "$MCPGATEWAY_URL/rpc", not "/api/rpc".
     Snippets should not include "server_id".

  5. Click "Preview".
     Expect "Preview 200", "Tool result", "Resolved arguments", and
     "Raw preview response".
     Terminal should log /api/tools/preview/customer_lookup with the args and
     X-Tenant-Id header.

  6. Click "Live invoke".
     Expect "Live invoke 200", "Tool result", and the text:
     "Live result: Acme Corp is active."
     Terminal should log /api/rpc with method "tools/call", params.name
     "customer_lookup", the filled args, no server_id, and X-Tenant-Id.

Ctrl-C to close.
`);
  await new Promise(() => {});
}

Manual verification results

Mock-backed happy-path run completed against this PR branch.

#	Check	Expected	Result
1	Default Try-it tab	Details drawer opens with Try it selected without setting `VITE_ENABLE_TOOL_PREVIEW`	Pass
2	Schema arguments	`customer_id=acme-001` and `include_orders=true` are accepted and rendered in resolved arguments	Pass
3	Snippets	Snippet tabs render, target `$MCPGATEWAY_URL/rpc`, show MCP 2025-11-25, and omit `server_id`	Pass
4	Preview action	Mocked `/api/tools/preview/customer_lookup` returns Preview 200 with tool result, resolved arguments, and raw preview response	Pass
5	Live invoke action	Mocked `/api/rpc` returns Live invoke 200 with text result and structured output	Pass
6	Direct invoke payload	Terminal log confirms `method: "tools/call"`, `params.name: "customer_lookup"`, filled args, forwarded `x-tenant-id`, and no `server_id`	Pass

Observed UI output included Preview 200, Resolved arguments, Raw preview response, Live invoke 200, the text Live result: Acme Corp is active., and structured output for Acme Corp.

Scope of this verification: all backend responses are mocked. This covers frontend wiring only: default Try-it visibility after flag removal, schema argument form, snippets, preview request construction, live JSON-RPC request construction, passthrough headers, and result rendering. It does not verify a released gateway image; that remains part of the un-draft gate.

## Backend PR #6443 Compatibility Verification

This integration test was run against IBM/mcp-context-forge#6443, the backend PR completing IBM/mcp-context-forge#5629. It verifies UI PR #88 against a real gateway backend plus a deterministic local REST target. Full npm run e2e:docker is still blocked until a released ghcr.io/ibm/mcp-context-forge image contains that backend work.

Integration test steps

Setup

Use sibling checkouts:

# UI repo
cd /Users/pratik/Desktop/work/new_mcf/contextforge-web-ui
git checkout feat/6322-remove-tool-preview-flag
npm ci                 # if node_modules is missing

# backend repo, in a separate terminal if you want to inspect it
cd /Users/pratik/Desktop/work/new_mcf/mcp-context-forge
git fetch origin 5629-tool-preview-endpoint-invoke-tool-endpoints
git checkout 5629-tool-preview-endpoint-invoke-tool-endpoints

Save the script from the next collapsible at the UI repo root as tool-try-it-real-backend-test.mjs.

Run it from the UI repo:

cd /Users/pratik/Desktop/work/new_mcf/contextforge-web-ui
node tool-try-it-real-backend-test.mjs

The script reuses healthy services if they are already running. Otherwise it starts:

backend PR #6443 with JWT_SECRET_KEY=compat-test-jwt-secret-1234567890 AUTH_ENCRYPTION_SECRET=compat-test-auth-secret-1234567890 PASSWORD_CHANGE_ENFORCEMENT_ENABLED=false make dev
a local deterministic GitHub issue lookup REST target on 127.0.0.1:9010
the UI BFF on 127.0.0.1:3000
one unique REST tool named github_issue_lookup_test_*

Browser flow

1. Open http://127.0.0.1:3000/app/tools.

2. Log in with:

admin@example.com
changeme

3. Open the REST tools card and select the generated github_issue_lookup_test_* tool printed by the script.

4. Confirm the drawer opens on Try it by default.

5. Fill the schema-generated arguments:

owner = IBM
repo = mcp-context-forge
issue_number = 5630

6. Click Preview.
Expect: Preview 200, Resolved arguments, and Raw preview response. The resolved arguments should include owner, repo, and issue_number.

7. Click Live invoke.
Expect: Live invoke 200 and a tool result containing issue #5630 details: title, state, labels, updated date, and summary.

Teardown

Keep the script process running during the manual test. Press Ctrl-C in that terminal when finished. It stops the services it started itself.

Integration test script (tool-try-it-real-backend-test.mjs)

Save at the UI repo root. The script assumes the backend checkout exists at ../mcp-context-forge; override with BACKEND_DIR=/path/to/mcp-context-forge if needed.

#!/usr/bin/env node
// Integration test setup for contextforge-web-ui#88 against backend PR #6443.
//
// Usage:
//   node tool-try-it-real-backend-test.mjs
//
// What it does:
//   - reuses or starts backend make dev on :8000
//   - reuses or starts a local GitHub issue lookup REST endpoint on :9010
//   - reuses or starts the UI BFF on :3000
//   - seeds one unique REST tool
//   - verifies preview and live invoke through backend and BFF
//   - prints the short browser test flow
//
// Leave this process running during the manual test if it starts any services.

import { createHmac, randomBytes, randomUUID } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir } from "node:fs/promises";
import { createServer } from "node:http";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { spawn } from "node:child_process";

const UI_DIR = path.dirname(fileURLToPath(import.meta.url));
const BACKEND_DIR = process.env.BACKEND_DIR ?? path.resolve(UI_DIR, "../mcp-context-forge");
const BACKEND_URL = process.env.BACKEND_URL ?? "http://127.0.0.1:8000";
const BFF_URL = process.env.BFF_URL ?? "http://127.0.0.1:3000";
const ECHO_HOST = "127.0.0.1";
const ECHO_PORT = Number(process.env.ECHO_PORT ?? "9010");
const ECHO_URL = `http://${ECHO_HOST}:${ECHO_PORT}`;
const ISSUE_OWNER = "IBM";
const ISSUE_REPO = "mcp-context-forge";
const ISSUE_NUMBER = 5630;
const JWT_SECRET = process.env.JWT_SECRET_KEY ?? "compat-test-jwt-secret-1234567890";
const AUTH_SECRET = process.env.AUTH_ENCRYPTION_SECRET ?? "compat-test-auth-secret-1234567890";
const ADMIN_EMAIL = process.env.ADMIN_EMAIL ?? "admin@example.com";
const ADMIN_PASSWORD = process.env.ADMIN_PASSWORD ?? "changeme";
const RUN_UI_BUILD = process.env.TEST_SKIP_UI_BUILD !== "1";
const VERBOSE = process.env.TEST_VERBOSE === "1";

const startedChildren = [];
let echoServer;

function printSetupOverview() {
  console.log(`
ContextForge Try-it test setup

Architecture:
  Browser UI -> BFF (${BFF_URL}) -> Gateway (${BACKEND_URL}) -> GitHub issue lookup tool (${ECHO_URL}/github/issues)

What this script prepares:
  1. Gateway backend from PR #6443, started with make dev if needed.
  2. Local REST GitHub issue lookup target on :${ECHO_PORT}; it returns deterministic test issue data.
  3. UI BFF from PR #88 on ${BFF_URL}; the browser calls this, not the gateway directly.
  4. One unique REST tool with an input schema for owner, repo, and issue_number.
  5. Smoke checks for Preview and Live invoke through both backend and BFF.

What this verifies:
  The schema draws the Try-it form. Preview validates through the backend without executing
  the tool. Live invoke executes through /rpc and reaches the GitHub issue lookup endpoint.
`);
}

function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

function base64UrlJson(value) {
  return Buffer.from(JSON.stringify(value)).toString("base64url");
}

function createJwt() {
  const now = Math.floor(Date.now() / 1000);
  const header = { alg: "HS256", typ: "JWT" };
  const payload = {
    username: ADMIN_EMAIL,
    iat: now,
    iss: "mcpgateway",
    aud: "mcpgateway-api",
    jti: randomUUID(),
    env: "development",
    sub: ADMIN_EMAIL,
    user: {
      email: ADMIN_EMAIL,
      full_name: "CLI User",
      is_admin: true,
      auth_provider: "cli",
    },
    teams: null,
    exp: now + 3600,
  };
  const data = `${base64UrlJson(header)}.${base64UrlJson(payload)}`;
  const signature = createHmac("sha256", JWT_SECRET).update(data).digest("base64url");
  return `${data}.${signature}`;
}

async function fetchJson(url, options = {}) {
  const response = await fetch(url, {
    ...options,
    signal: AbortSignal.timeout(options.timeoutMs ?? 15000),
  });
  const text = await response.text();
  let body = null;
  if (text) {
    try {
      body = JSON.parse(text);
    } catch {
      body = text;
    }
  }
  if (!response.ok) {
    throw new Error(`${options.method ?? "GET"} ${url} -> ${response.status}: ${text}`);
  }
  return { response, body };
}

async function isHealthy(url) {
  try {
    const response = await fetch(url, { signal: AbortSignal.timeout(1500) });
    return response.ok;
  } catch {
    return false;
  }
}

async function waitFor(label, url, timeoutMs = 120000) {
  const start = Date.now();
  while (Date.now() - start < timeoutMs) {
    if (await isHealthy(url)) return;
    await sleep(1000);
  }
  throw new Error(`${label} did not become ready at ${url}`);
}

async function runLogged(label, command, args, cwd, env = {}) {
  await mkdir("/tmp/contextforge-test", { recursive: true });
  const logPath = `/tmp/contextforge-test/${label}.log`;
  const log = createWriteStream(logPath, { flags: "a" });
  if (VERBOSE) console.log(`Running ${label}; log: ${logPath}`);

  return new Promise((resolve, reject) => {
    const child = spawn(command, args, {
      cwd,
      env: { ...process.env, ...env },
      stdio: ["ignore", "pipe", "pipe"],
    });
    child.stdout.pipe(log);
    child.stderr.pipe(log);
    child.on("error", reject);
    child.on("exit", (code) => {
      log.end();
      if (code === 0) resolve();
      else reject(new Error(`${label} exited with code ${code}; see ${logPath}`));
    });
  });
}

async function startLogged(label, command, args, cwd, env = {}) {
  await mkdir("/tmp/contextforge-test", { recursive: true });
  const logPath = `/tmp/contextforge-test/${label}.log`;
  const log = createWriteStream(logPath, { flags: "a" });
  const child = spawn(command, args, {
    cwd,
    detached: true,
    env: { ...process.env, ...env },
    stdio: ["ignore", "pipe", "pipe"],
  });

  child.stdout.pipe(log);
  child.stderr.pipe(log);
  startedChildren.push({ label, child, logPath });
  if (VERBOSE) console.log(`Started ${label} pid=${child.pid}; log: ${logPath}`);
  return child;
}

async function ensureBackend() {
  if (await isHealthy(`${BACKEND_URL}/health`)) {
    if (VERBOSE) console.log(`Backend already healthy at ${BACKEND_URL}`);
    return;
  }

  await startLogged("backend-make-dev", "make", ["dev"], BACKEND_DIR, {
    JWT_SECRET_KEY: JWT_SECRET,
    AUTH_ENCRYPTION_SECRET: AUTH_SECRET,
    PASSWORD_CHANGE_ENFORCEMENT_ENABLED: "false",
  });
  await waitFor("backend", `${BACKEND_URL}/health`);
}

async function ensureEchoServer() {
  if (await isTestIssueServerHealthy()) {
    if (VERBOSE) console.log(`GitHub issue lookup server already healthy at ${ECHO_URL}`);
    return;
  }

  echoServer = createServer((request, response) => {
    let body = "";
    request.on("data", (chunk) => {
      body += chunk;
    });
    request.on("end", () => {
      response.setHeader("Content-Type", "application/json");
      const parsedBody = body ? JSON.parse(body) : {};
      const args = parsedBody?.arguments ?? parsedBody ?? {};

      if (request.url === "/probe") {
        response.end(JSON.stringify({ ok: true, kind: "github-issue-test" }));
        return;
      }

      response.end(JSON.stringify(buildIssueLookupResponse(args)));
    });
  });

  await new Promise((resolve) => echoServer.listen(ECHO_PORT, ECHO_HOST, resolve));
  if (VERBOSE) console.log(`Started echo server at ${ECHO_URL}`);
}

async function isTestIssueServerHealthy() {
  try {
    const { body } = await fetchJson(`${ECHO_URL}/probe`, { timeoutMs: 1500 });
    return body?.kind === "github-issue-test";
  } catch {
    return false;
  }
}

function buildIssueLookupResponse(args) {
  return {
    owner: args.owner ?? ISSUE_OWNER,
    repo: args.repo ?? ISSUE_REPO,
    issue_number: Number(args.issue_number ?? ISSUE_NUMBER),
    title: "Add a Try-it experience for tools",
    state: "open",
    labels: ["tools", "ui", "mcp"],
    updated_at: "2026-08-31T18:20:00Z",
    summary:
      "Adds a UI flow to preview tool arguments, run live tool calls, and inspect responses from the browser.",
  };
}

function testArguments() {
  return {
    owner: ISSUE_OWNER,
    repo: ISSUE_REPO,
    issue_number: ISSUE_NUMBER,
  };
}

async function ensureBff() {
  if (await isHealthy(`${BFF_URL}/healthz`)) {
    if (VERBOSE) console.log(`BFF already healthy at ${BFF_URL}`);
    return;
  }

  if (RUN_UI_BUILD) {
    await runLogged("ui-build", "npm", ["run", "build"], UI_DIR);
  }

  await startLogged("ui-bff", "npm", ["run", "dev"], path.join(UI_DIR, "server"), {
    NODE_ENV: "development",
    CONTEXTFORGE_URL: BACKEND_URL,
    COOKIE_SECURE: "false",
    PORT: new URL(BFF_URL).port || "3000",
    LOG_LEVEL: "error",
    REDIS_URL: "memory://",
  });
  await waitFor("BFF", `${BFF_URL}/healthz`);
}

async function seedTool() {
  const suffix = `${Date.now().toString().slice(-6)}_${randomBytes(2).toString("hex")}`;
  const rawName = `github_issue_lookup_test_${suffix}`;
  const displayName = `GitHub issue lookup test ${suffix}`;
  const token = createJwt();

  const payload = {
    tool: {
      name: rawName,
      displayName,
      title: displayName,
      url: `${ECHO_URL}/github/issues`,
      description: "Look up a GitHub issue from a local deterministic test endpoint",
      integration_type: "REST",
      request_type: "POST",
      inputSchema: {
        type: "object",
        properties: {
          owner: { type: "string", title: "Owner" },
          repo: { type: "string", title: "Repository" },
          issue_number: { type: "integer", title: "Issue number" },
        },
        required: ["owner", "repo", "issue_number"],
      },
      annotations: { title: displayName, readOnlyHint: true },
      visibility: "public",
    },
  };

  const { body } = await fetchJson(`${BACKEND_URL}/tools`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${token}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(payload),
  });

  if (VERBOSE) console.log(`Seeded tool: ${body.name}`);
  return body;
}

async function verifyBackend(toolName) {
  const token = createJwt();
  const preview = await fetchJson(`${BACKEND_URL}/tools/preview/${toolName}`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${token}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      arguments: testArguments(),
    }),
  });

  if (preview.body?.resolvedArguments?.issue_number !== ISSUE_NUMBER) {
    throw new Error("Backend preview did not return resolvedArguments");
  }

  const live = await fetchJson(`${BACKEND_URL}/rpc`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${token}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      jsonrpc: "2.0",
      id: "test-live-backend-1",
      method: "tools/call",
      params: {
        name: toolName,
        arguments: testArguments(),
      },
    }),
  });

  const text = live.body?.result?.content?.[0]?.text ?? "";
  if (!text.includes('"issue_number": 5630')) {
    throw new Error("Backend live invoke did not return the GitHub issue lookup result");
  }

  if (VERBOSE) console.log("Backend preview/live checks passed");
}

function getSetCookies(response) {
  if (typeof response.headers.getSetCookie === "function") {
    return response.headers.getSetCookie();
  }
  const combined = response.headers.get("set-cookie");
  return combined ? combined.split(/,(?=[^;]+?=)/) : [];
}

async function verifyBff(toolName) {
  const login = await fetchJson(`${BFF_URL}/auth/login`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ email: ADMIN_EMAIL, password: ADMIN_PASSWORD }),
  });
  const csrfToken = login.body?.csrfToken;
  const cookieHeader = getSetCookies(login.response)
    .map((cookie) => cookie.split(";")[0])
    .join("; ");

  if (!csrfToken || !cookieHeader) {
    throw new Error("BFF login did not return a session cookie and CSRF token");
  }

  const commonHeaders = {
    Cookie: cookieHeader,
    "X-CSRF-Token": csrfToken,
    "Content-Type": "application/json",
  };

  const preview = await fetchJson(`${BFF_URL}/api/tools/preview/${toolName}`, {
    method: "POST",
    headers: commonHeaders,
    body: JSON.stringify({
      arguments: testArguments(),
    }),
  });
  if (preview.body?.resolvedArguments?.issue_number !== ISSUE_NUMBER) {
    throw new Error("BFF preview proxy did not return resolvedArguments");
  }

  const live = await fetchJson(`${BFF_URL}/api/rpc`, {
    method: "POST",
    headers: commonHeaders,
    body: JSON.stringify({
      jsonrpc: "2.0",
      id: "test-live-bff-1",
      method: "tools/call",
      params: {
        name: toolName,
        arguments: testArguments(),
      },
    }),
  });

  const text = live.body?.result?.content?.[0]?.text ?? "";
  if (!text.includes('"issue_number": 5630')) {
    throw new Error("BFF live invoke did not return the GitHub issue lookup result");
  }

  if (VERBOSE) console.log("BFF preview/live checks passed");
}

function printTestSteps(tool) {
  console.log(`
Integration test setup is ready.

Open:
  ${BFF_URL}/app/tools

Login:
  ${ADMIN_EMAIL}
  ${ADMIN_PASSWORD}

Manual test flow:
  1. Open the "REST tools" card.
  2. Select "${tool.name}".
  3. Show that "Try it" opens by default.
  4. Enter owner = ${ISSUE_OWNER}.
  5. Enter repo = ${ISSUE_REPO}.
  6. Enter issue_number = ${ISSUE_NUMBER}.
  7. Click "Preview".
     Expect "Preview 200" and "Resolved arguments".
  8. Click "Live invoke".
     Expect "Live invoke 200" and a result containing the GitHub issue title and labels.

Backend command used when started by this script:
  JWT_SECRET_KEY=${JWT_SECRET} AUTH_ENCRYPTION_SECRET=${AUTH_SECRET} PASSWORD_CHANGE_ENFORCEMENT_ENABLED=false make dev
`);
}

async function cleanup() {
  if (echoServer) {
    await new Promise((resolve) => echoServer.close(resolve));
  }
  for (const { label, child, logPath } of startedChildren.reverse()) {
    if (VERBOSE) console.log(`Stopping ${label}; log: ${logPath}`);
    try {
      process.kill(-child.pid, "SIGINT");
    } catch {
      child.kill("SIGINT");
    }
  }
}

async function holdIfNeeded() {
  if (!echoServer && startedChildren.length === 0) return;
  console.log(
    "Leave this process running during the manual test. Press Ctrl-C to stop services started by the script.",
  );
  await new Promise(() => {});
}

process.on("SIGINT", async () => {
  await cleanup();
  process.exit(130);
});
process.on("SIGTERM", async () => {
  await cleanup();
  process.exit(143);
});

try {
  printSetupOverview();
  await ensureBackend();
  await ensureEchoServer();
  await ensureBff();
  const tool = await seedTool();
  await verifyBackend(tool.name);
  await verifyBff(tool.name);
  printTestSteps(tool);
  await holdIfNeeded();
} catch (error) {
  console.error(error instanceof Error ? error.message : error);
  await cleanup();
  process.exit(1);
}

Integration test results

Compatibility run completed against:

UI: contextforge-web-ui PR Remove temporary tool preview feature flag #88, feat/6322-remove-tool-preview-flag
Backend: IBM/mcp-context-forge PR #6443, 5629-tool-preview-endpoint-invoke-tool-endpoints
Browser entrypoint: UI BFF at http://127.0.0.1:3000/app/tools
Tool target: deterministic local GitHub issue lookup REST endpoint at http://127.0.0.1:9010/github/issues

#	Check	Expected	Result
1	Backend preview endpoint	`POST /tools/preview/{name}` returns preview data with `resolvedArguments`	Pass
2	Backend live invoke	`POST /rpc` with `tools/call` executes the REST tool and returns MCP content blocks	Pass
3	BFF preview proxy	`POST /api/tools/preview/{name}` forwards through the session/CSRF proxy	Pass
4	BFF live proxy	`POST /api/rpc` forwards JSON-RPC live invoke through the BFF	Pass
5	Default Try-it tab	Tools drawer opens with Try it selected without `VITE_ENABLE_TOOL_PREVIEW`	Pass
6	camelCase preview payload	UI renders backend `resolvedArguments` as Resolved arguments	Pass
7	Live result rendering	UI renders Live invoke 200 and the MCP `content` result	Pass

Observed live result content:

{
  "content": [
    {
      "type": "text",
      "text": "{\n  \"owner\": \"IBM\",\n  \"repo\": \"mcp-context-forge\",\n  \"issue_number\": 5630,\n  \"title\": \"Add a Try-it experience for tools\",\n  \"state\": \"open\",\n  \"labels\": [\n    \"tools\",\n    \"ui\",\n    \"mcp\"\n  ],\n  \"updated_at\": \"2026-08-31T18:20:00Z\",\n  \"summary\": \"Adds a UI flow to preview tool arguments, run live tool calls, and inspect responses from the browser.\"\n}"
    }
  ],
  "isError": false
}

Observed UI output included Preview 200, Resolved arguments, Raw preview response, Live invoke 200, and a rendered MCP content block containing IBM/mcp-context-forge issue #5630 data.

Scope of this verification: this is a real gateway/BFF compatibility test against backend PR #6443, but the final tool target is a deterministic local REST endpoint instead of the public GitHub API. It validates the UI-to-BFF-to-gateway contract for Preview and Live invoke; released-image docker e2e remains part of the un-draft gate.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

gandhipratik203 added 2 commits August 27, 2026 22:48

Remove temporary tool preview flag

60bbd9d

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

Handle camelCase preview resolved arguments

3726e36

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

Navigation Menu

Uh oh!

Remove temporary tool preview feature flag - #88

Remove temporary tool preview feature flag#88
gandhipratik203 wants to merge 2 commits into
mainfrom
feat/6322-remove-tool-preview-flag

gandhipratik203 commented Aug 27, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

gandhipratik203 commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

Blocker

Un-draft Gate

Tests

Not Run

Explanatory Diagrams

Manual Verification

Setup

Steps

Teardown

Setup

Browser flow

Teardown

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

gandhipratik203 commented Aug 27, 2026 •

edited

Loading