Add Kitesurf support to Browser Tools

[cjol]

This PR adds connection-scoped Kitesurf support to Agents Browser Tools and uses it in the ai-chat example. Fixes #2095.

Why

• Browser Tools previously assumed Browser Run always acquires a reconnectable Chromium session. Kitesurf instead acquires the browser directly through /v1/devtools/browser?browser=kitesurf, where the WebSocket itself owns the browser lifetime.
• Treating Kitesurf's analytics session identifier as reconnectable would make pause, reuse, cleanup, and Live View behavior incorrect.
• Kitesurf Quick Actions exist over the public REST API, but the Worker binding's quickAction() RPC cannot currently carry an engine selector. Passing browser in its options is rejected as part of the action body.
• This PR therefore adds the verified CDP path while rejecting unsupported durable lifecycle features. A possible connection-pinned reuse mode remains a separate design direction because it could not survive socket loss or Durable Object restart.

Public API Surface

Symbol	Kind	Notes
ConnectBrowserOptions.browser	Additive option	Selects the "kitesurf" engine for direct CDP acquisition.
BrowserConnectorSessionOptions.browser	Additive option	Selects Kitesurf for createBrowserTools() and createBrowserRuntime().

Selecting Kitesurf defaults Quick Actions off because the binding cannot select that engine. Callers can explicitly request Quick Actions to create a mixed toolset where CDP uses Kitesurf and Quick Actions use Chromium.

Architectural Changes

Chromium
acquire session -> persist session id -> connect/reconnect WebSocket -> delete session

Kitesurf
open /v1/devtools/browser?browser=kitesurf WebSocket -> use browser -> close connection

• Kitesurf is exposed as one-shot and connection-scoped. The connector does not expose protocol discovery, Live View, shared session management, recording, keep-alive, or pause/resume for it.
• A durable execution marker prevents a paused or lost Kitesurf connection from silently continuing in a fresh browser.
• The design notes distinguish this current policy from a future best-effort mode that could pin an open WebSocket across codemode calls without promising durable reconnection.

Code Changes

• browser-run.ts acquires Kitesurf directly over WebSocket and rejects incompatible Browser Run options before making a request.
• BrowserConnector selects the Kitesurf acquisition path, narrows its model-facing tools and instructions, and fails explicitly when a connection can no longer resume.
• browser_execute preserves canonical screenshot results for AI SDK UIs while toModelOutput sends the model only a compact attachment summary.
• The Kitesurf-specific tool description documents the raw CDP call shape, attachment handles, bounded readiness polling, and screenshot return contract without asking models to search for CDP commands through codemode.search().
• examples/ai-chat demonstrates natural-language Kitesurf navigation and inline screenshot rendering with a remote Browser Run binding.
• Browser documentation records the supported surface, current Quick Action limitation, and connection-pinned reuse boundary.

[changeset-bot]

🦋 Changeset detected

Latest commit: cc89773

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
agents	Minor
@cloudflare/agent-think	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2098

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2098

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2098

hono-agents

npm i https://pkg.pr.new/hono-agents@2098

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2098

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2098

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2098

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2098

commit: cc89773

[devin-ai-integration]

Pushed cc89773 for the two Devin Review findings (on top of c29c3c7):

• keepAliveMs: 0 + Kitesurf: rather than rejecting it (your allows explicitly disabled Kitesurf session options test says explicitly-disabled options are fine), connectBrowser now never forwards keep_alive on the Kitesurf path, so keep_alive=0 can't reach the service. Non-zero values still throw. Test extended to cover keepAliveMs: 0.
• examples/ai-chat: formatToolOutput no longer keys redaction off the strict browser_screenshot predicate — any tool result with a large data string is redacted in the transcript, while <img> rendering stays strict.

[devin-ai-integration]

Devin Review found 2 new potential issues.

View 3 additional findings in Devin Review.

[devin-ai-integration]

🔴 Screenshots re-enter the chat history at full size on the next message

The chat history is rebuilt for the model without telling it how each browser result should be summarized (convertToModelMessages(this.messages) at examples/ai-chat/src/server.ts:105), so a captured screenshot is resent in full on the following turn instead of the short summary.

Impact: After a screenshot, the very next user message ships the entire image as text to a text-only model, blowing up context and cost and likely failing the request.

Why the compact summary is bypassed on later turns

createBrowserRuntime attaches toModelOutput to browser_execute (packages/agents/src/browser/ai.ts:392-393) so the model sees only "Screenshot captured successfully …". That hook is applied by streamText for results produced during the current run, and by convertToModelMessages only when the tool set is passed in options — see the in-repo convention at packages/think/src/think.ts:5327-5330 (convertToModelMessages(truncated, { tools, ignoreIncompleteToolCalls: true })).

The example calls convertToModelMessages(this.messages) with no tools, so persisted tool parts are converted with their raw output. Because transformBrowserResult deliberately leaves canonical screenshot results untruncated (packages/agents/src/browser/ai.ts:250-257), the persisted part contains the complete base64 payload. pruneMessages({ toolCalls: "before-last-2-messages" }) keeps the most recent assistant turn's tool results, so the screenshot is included verbatim in the next request.

Prompt for agents

In examples/ai-chat/src/server.ts, onChatMessage builds model messages with `convertToModelMessages(this.messages)` and no tool set. The browser_execute tool created by createBrowserTools defines `toModelOutput` (packages/agents/src/browser/ai.ts) so that a canonical screenshot result is replaced by a short text summary, while the raw base64 stays in the persisted UI message for inline rendering. Because convertToModelMessages only applies toModelOutput when the tools are passed in its options (see packages/think/src/think.ts for the in-repo convention), every follow-up turn re-serializes the full base64 screenshot into the model request. Fix by constructing the full tool set once and passing it to convertToModelMessages (e.g. `convertToModelMessages(this.messages, { tools, ignoreIncompleteToolCalls: true })`) so persisted browser results are summarized consistently.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Browser screenshots are sent to the model unbounded through the TanStack wrapper

Screenshot results are now exempted from the size cap that used to bound every browser result (transformResult: transformBrowserResult at packages/agents/src/browser/ai.ts:378), so callers that have no way to substitute a short summary hand the whole image to the model.

Impact: A TanStack AI agent that takes a screenshot pushes up to a megabyte of image text into the model request, wasting context and potentially failing the call.

Only the AI SDK path has the compensating summary hook

transformBrowserResult (packages/agents/src/browser/ai.ts:250-257) returns canonical { type: "browser_screenshot", mediaType, data } results untouched instead of running them through truncateResult, relying on the AI SDK-only toModelOutput hook added at packages/agents/src/browser/ai.ts:390-394 to keep the model context small.

packages/agents/src/browser/tanstack-ai.ts:44-69 builds its ServerTool from createBrowserRuntime(...).tools.browser_execute but only forwards description, inputSchema and execute; the returned ProxyToolOutput (including result) goes straight back to the model. The connector hint injected for both paths (packages/agents/src/browser/ai.ts:383-387) explicitly instructs the model to return that screenshot shape, so this path is reachable in practice. Before this change truncateResult capped the payload.

Prompt for agents

transformBrowserResult in packages/agents/src/browser/ai.ts now bypasses truncateResult for canonical browser_screenshot results, on the assumption that the tool's toModelOutput hook will give the model a compact summary. That hook only exists on the AI SDK tool object. packages/agents/src/browser/tanstack-ai.ts wraps the same runtime but forwards only description/inputSchema/execute, so a TanStack consumer returns the untruncated base64 result directly to the model. Consider applying the same summarization in the TanStack wrapper (reuse browserExecuteModelOutput / a shared exported helper on the returned ProxyToolOutput), or making the screenshot passthrough opt-in for consumers that can render the image.

---

Was this helpful? React with 👍 or 👎 to provide feedback.