Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ services:
- CODEAPI_BRIDGE_DYNAMIC_WORKERS=${CODEAPI_BRIDGE_DYNAMIC_WORKERS:-true}
- CODEAPI_BRIDGE_MAX_WORKSPACE_LEASE_SLOTS=${CODEAPI_BRIDGE_MAX_WORKSPACE_LEASE_SLOTS:-1}
- CODEAPI_BRIDGE_LANE_GIT=${CODEAPI_BRIDGE_LANE_GIT:-false}
- CODEAPI_BRIDGE_IMPORT_FILE_MAX_BYTES=${CODEAPI_BRIDGE_IMPORT_FILE_MAX_BYTES:-104857600}
- CODEAPI_BRIDGE_WORKER_ID=${CODEAPI_BRIDGE_WORKER_ID:-}
- CODEAPI_BRIDGE_RECOVERY_SERVER_ID=${CODEAPI_BRIDGE_RECOVERY_SERVER_ID:-}
- CODEAPI_BRIDGE_ENROLLMENT_TTL_SECONDS=${CODEAPI_BRIDGE_ENROLLMENT_TTL_SECONDS:-0}
Expand Down
65 changes: 65 additions & 0 deletions docs/adr/0014-workspace-file-import.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# 0014: Import chat uploads into attached workspaces

Status: Accepted

## Context

A LibreChat chat can run on a personal machine through the remote bridge. Files
the user uploads in that chat stay in Code API storage: only programmatic Bash
stages them, per run, in a private scratch directory. `write_file` accepts
1 MiB of UTF-8 text, so binaries such as archives, images, and PDFs cannot reach
the workspace at all.

## Decision

Add one additive workspace operation, `import_file`, under protocol version 1.

- **Request.** `{ path, source: { storage_session_id, id }, bytes, sha256,
overwrite? }` plus the usual workspace selectors. `sha256` is lowercase hex.
- **Ownership.** Code API, not the caller, decides whether `source` may be read.
It applies the `GET /download/:session_id/:fileId` rule for a user-kind
object: the live `session:<id>` key must name the caller's storage key, and an
upload record must exist for that key, session, and object.
- **Grant.** Code API asks the egress gateway for a grant whose only input is
that object, with no upload capacity, two reads, and an expiry of the
request's admission plus execution budget plus 30 seconds. The relayed
request carries the grant and opaque handles in place of the raw ids. A
caller-supplied `egressGrant` is rejected. Synchronous requests revoke the
grant when they finish; durable requests let it expire.
- **Worker.** The worker streams the object from
`LIBRECHAT_CODE_FILE_RELAY_UPSTREAM`, the route programmatic inputs already
use, and holds no storage credential. It applies `write_file`'s path rules,
refuses `.git` segments and their filesystem aliases, refuses symlinked
parents, creates missing parents as 0755, enforces `bytes` while streaming
into a same-directory temporary file, verifies size and SHA-256, syncs, and
installs the file atomically with mode 0644. The size and hash are checked on
the synced temporary file itself. A create links exclusively; an overwrite
rechecks the inspected inode immediately before its rename, as `write_file`
does.
- **Idempotency.** Identical existing bytes return `unchanged: true` and
`bytesWritten: 0` without a transfer. Different bytes return `EDIT_CONFLICT`
unless `overwrite: true`.
- **Limits.** Code API (`CODEAPI_BRIDGE_IMPORT_FILE_MAX_BYTES`) and the worker
(`LIBRECHAT_CODE_IMPORT_FILE_MAX_BYTES`) each enforce a per-file ceiling,
100 MiB by default, as `WRITE_LIMIT_EXCEEDED`. The worker also requires the
file plus 64 MiB to fit in the target filesystem's free space.
- **Policy.** `import_file` is a file write. It shares the mutation guard with
`write_file`, is never advertised for a read-only root, and a read-only root
refuses it with `WRITE_DISABLED`.
- **Negotiation.** Code API lists `import_file` in
`supportedWorkspaceToolOperations` only when `EGRESS_GATEWAY_URL` is set. A
worker advertises it by default whenever writes are enabled, a relay
upstream is configured, and the Code API listed it; operators opt out with
`--no-workspace-imports` or `LIBRECHAT_CODE_ALLOW_WORKSPACE_IMPORTS=false`.
LibreChat sends it only when advertised. LibreChat releases that predate
LibreChat-AI/LibreChat#17376 reject a worker status listing an unknown
operation, so LibreChat is updated first and the opt-out is the escape
hatch for workers serving an older one.

## Consequences

Uploads land in the workspace without a model round trip, and older Code APIs,
workers, and LibreChat releases keep their current behavior. Deployments
without an egress gateway do not offer the operation. Parent directories are
created only beneath a descriptor-held root, as for `write_file`; a legacy
pathname-only root reports a missing parent instead.
5 changes: 5 additions & 0 deletions docs/remote-bridge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,11 @@ whitespace-tolerant matching and multi-location replacement (see
text, require an existing in-workspace parent directory, reject symlinks, and
commit atomically. The worker capability is an enforcement boundary; LibreChat
should still route every mutation through its configurable tool-approval hooks.
Writable workers with a file relay upstream also offer `import_file` by
default (opt out with `LIBRECHAT_CODE_ALLOW_WORKSPACE_IMPORTS=false`), which
copies a chat upload the caller owns into the workspace through a one-object
egress grant (ADR 0014). It is a file write: a worker without writes enabled
never advertises it and refuses it with `WRITE_DISABLED`.

The workspace root can be an existing project, a Git repository, or an empty
directory; Git is not required. This boundary keeps that directory local to the
Expand Down
21 changes: 21 additions & 0 deletions packages/code/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,27 @@ with `--allow-workspace-writes` or
`overwrite: false` to require an atomic create that returns `EDIT_CONFLICT` if
the target already exists. Code API dispatches that mode only after the worker
and server negotiate `create` in `writeFileModes`.

`import_file` copies one stored chat upload into a writable workspace as a
binary-safe file. It is on by default whenever the worker can perform it:
writes are enabled and `LIBRECHAT_CODE_FILE_RELAY_UPSTREAM` names the
deployment's egress gateway. It is advertised only to a Code API that lists it
in `supportedWorkspaceToolOperations` (one that can mint egress grants). Opt out
with `--no-workspace-imports` or `LIBRECHAT_CODE_ALLOW_WORKSPACE_IMPORTS=false`.
Update LibreChat before workers: LibreChat releases that predate
LibreChat-AI/LibreChat#17376 reject a worker status that lists an operation
they do not know, so a worker serving one of those must opt out. Code API checks that the caller owns the object
and relays a short-lived grant scoped to that object alone; the worker holds no
storage credential. The worker applies `write_file`'s path rules, refuses any
`.git` segment (including case, Unicode-ignorable, trailing-dot and `GIT~1`
aliases) and symlinked parents, creates missing parents as 0755 under a held
root, streams into a same-directory temporary file with `bytes` as a hard cap,
verifies size and SHA-256, syncs, and installs the file atomically with mode
0644. Identical existing bytes return `unchanged: true` without a transfer;
different bytes return `EDIT_CONFLICT` unless `overwrite: true`. A failure
removes the temporary file and any parents it created. The per-file ceiling is
`LIBRECHAT_CODE_IMPORT_FILE_MAX_BYTES` (`--import-file-max-bytes`, default
100 MiB), and the filesystem must keep 64 MiB free after the file lands.
`edit_file` accepts either the legacy `oldText`/`newText` pair or an ordered
`edits` array; every exact replacement is validated before the updated file is
installed as one atomic mutation. Code API dispatches the batch form only after
Expand Down
19 changes: 19 additions & 0 deletions packages/code/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ import {
} from './storage.js';
import { BridgeWorker } from './worker.js';
import { LocalWorkspaceTools, SandboxWorkspaceTools } from './workspace.js';
import { createWorkspaceFileImporter } from './workspace-import.js';
import {
DockerRuntimeSupervisor,
EndpointRuntimeSupervisor,
Expand Down Expand Up @@ -78,6 +79,7 @@ import type { RuntimeSupervisor } from './runtime.js';
import type { GitHubCredentialProvider } from './github.js';
import type { WorkspaceToolExecutor } from './workspace.js';
import {
BRIDGE_WORKSPACE_IMPORT_FILE_DEFAULT_MAX_BYTES,
BRIDGE_WORKSPACE_NAME_MAX_LENGTH,
BridgeProtocolError,
isValidBridgeWorkerCapabilities,
Expand Down Expand Up @@ -914,10 +916,25 @@ async function run(
]),
)
: undefined;
// A chat upload reaches the workspace only through the same relay upstream
// programmatic inputs use; without one the worker never advertises import_file.
const workspaceImporter = createWorkspaceFileImporter({
optOut: args.includes('--no-workspace-imports'),
enabled: process.env.LIBRECHAT_CODE_ALLOW_WORKSPACE_IMPORTS,
writable: allowWorkspaceWrites,
upstreamUrl: fileRelayUpstream,
maxBytes: positiveInteger(
'LIBRECHAT_CODE_IMPORT_FILE_MAX_BYTES',
option(args, '--import-file-max-bytes') ??
process.env.LIBRECHAT_CODE_IMPORT_FILE_MAX_BYTES,
BRIDGE_WORKSPACE_IMPORT_FILE_DEFAULT_MAX_BYTES,
),
});
const localWorkspaceTools = workerDirectory
? await LocalWorkspaceTools.create({
workspaces: roots,
repositoryInstructions: args.includes('--repository-instructions'),
importer: workspaceImporter,
})
: undefined;
let workspaceTools: WorkspaceToolExecutor | undefined = localWorkspaceTools;
Expand Down Expand Up @@ -1298,6 +1315,7 @@ async function run(
},
repositoryInstructions: args.includes('--repository-instructions'),
writable: root.writable ?? false,
importer: workspaceImporter,
},
]),
),
Expand Down Expand Up @@ -1333,6 +1351,7 @@ async function run(
command: nativeOptionsForWorkspace(root.id),
repositoryInstructions: args.includes('--repository-instructions'),
writable: root.writable ?? false,
importer: workspaceImporter,
},
]),
),
Expand Down
4 changes: 3 additions & 1 deletion packages/code/src/linked-worktrees.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import type {
WorkspaceToolRequest,
WorkspaceToolResult,
} from './protocol.js';
import type { WorkspaceToolExecutor } from './workspace.js';
import type { WorkspaceFileImporter, WorkspaceToolExecutor } from './workspace.js';

/** Linked worktrees are only admitted from this directory beneath a checkout. */
export const LINKED_WORKTREE_DIRECTORY = '.worktrees';
Expand Down Expand Up @@ -60,6 +60,7 @@ export interface LinkedWorktreeSource {
command?: NativeProcessSandboxOptions;
repositoryInstructions: boolean;
writable: boolean;
importer?: WorkspaceFileImporter;
}

export interface VerifiedLinkedWorktree {
Expand Down Expand Up @@ -324,6 +325,7 @@ export class LinkedWorktreeWorkspaceTools implements WorkspaceToolExecutor, Link
fingerprint,
value: LocalWorkspaceTools.create({
repositoryInstructions: source.repositoryInstructions,
importer: source.importer,
workspaces: [
{ id: internalId, identity: lane.identity, root: lane.root, writable: source.writable },
],
Expand Down
21 changes: 9 additions & 12 deletions packages/code/src/native-programmatic.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,11 @@ import {
isSafePortableRelativePath,
isSupportedBridgeArtifactName,
} from './protocol.js';
import { validateFileRelayUpstream } from './relay.js';
import {
FILE_RELAY_GRANT_HEADER,
fileRelayObjectUrl,
validateFileRelayUpstream,
} from './relay.js';
import { WorkspaceToolError } from './workspace.js';

import type {
Expand All @@ -25,7 +29,6 @@ import type {
} from './protocol.js';
import type { NativeSrtWorkspaceCommandSandbox } from './native-sandbox.js';

const EGRESS_GRANT_HEADER = 'X-CodeAPI-Egress-Grant';
const EXECUTION_MAIN_FILE = 'main.sh';
const EXECUTION_HISTORY_FILE = '_ptc_history.json';
const EXECUTION_CONTROL_FILE = '_ptc_pending_result.json';
Expand Down Expand Up @@ -327,12 +330,9 @@ export class NativeWorkspaceProgrammaticExecutor {
const timer = setTimeout(() => controller.abort(), transferTimeoutMs);
try {
const response = await this.fetchImpl(
new URL(
`sessions/${encodeURIComponent(file.storage_session_id)}/objects/${encodeURIComponent(file.id)}`,
`${this.upstream.toString().replace(/\/+$/, '')}/`,
),
fileRelayObjectUrl(this.upstream, file.storage_session_id, file.id),
{
headers: { [EGRESS_GRANT_HEADER]: grant },
headers: { [FILE_RELAY_GRANT_HEADER]: grant },
redirect: 'error',
signal: controller.signal,
},
Expand Down Expand Up @@ -704,14 +704,11 @@ export class NativeWorkspaceProgrammaticExecutor {
let response: Response;
try {
response = await this.fetchImpl(
new URL(
`sessions/${encodeURIComponent(outputSessionId!)}/objects/${id}`,
`${this.upstream.toString().replace(/\/+$/, '')}/`,
),
fileRelayObjectUrl(this.upstream, outputSessionId!, id),
{
method: 'PUT',
headers: {
[EGRESS_GRANT_HEADER]: grant!,
[FILE_RELAY_GRANT_HEADER]: grant!,
'Content-Type': bridgeArtifactMediaType(name),
'Content-Length': String(bytes.byteLength),
'X-Original-Filename': encodeURIComponent(name),
Expand Down
Loading
Loading