diff --git a/AGENTS.md b/AGENTS.md index 7d39cfd..2b2a0e3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -121,7 +121,7 @@ ReAct cycle: observe → think → act → repeat. ### Extension capabilities (odek-extension/v1, v1.24.0) - **MCP per-server limits** — `timeout_seconds` (30s/3600s cap), `max_response_bytes` (10 MiB/64 MiB ceiling), `max_result_chars` (200k/1M cap, structured truncation notice), `artifact_roots` (empty ⇒ refs rejected). Resolved per client; approval keys hash all four fields. - **Artifact references** — MCP tools return `odek.tool-result/v1` envelopes with `file://` refs instead of bulk content; validated fail-closed in `internal/artifact`; model sees metadata only. -- **Runtime events** — `odek.event/v1` via `Config.EventHandler` (non-blocking, panic-isolated) and `run --events-jsonl`. Types: run_started, iteration_completed, tool_call_*, session_saved, context_trimmed, budget_exceeded, run_completed/run_failed. +- **Runtime events** — `odek.event/v1` via `Config.EventHandler` (non-blocking, panic-isolated) and `run --events-jsonl`. Types: run_started, iteration_completed, tool_call_*, session_saved, context_trimmed, budget_exceeded, run_completed/run_failed, plan_created, plan_updated, subagent_denied, subagent_spawned, subagent_completed. - **External session refs** — `Session.ExternalRefs` + `--external-ref` on run/continue; validated, deduped, never dereferenced. - **Execution budgets** — `limits` config section + `--max-runtime/--max-tool-calls/--max-input-tokens/--max-output-tokens/--max-cost-usd` on `run`; typed `budget.Error` → CLI exit code 4; session persisted before return. Per-model prices via `limits.model_prices` with flat-pair fallback; cost enforcement only when cap + prices configured. `odek init --global` scaffolds the section (zeros = off). `GET /api/limits` on serve exposes limits + effective prices for cost rendering. @@ -129,7 +129,7 @@ ReAct cycle: observe → think → act → repeat. All built-in tools with zero subprocess forks: batch_read, batch_patch, parallel_shell, http_batch, math_eval, diff, count_lines, multi_grep, json_query, tree, checksum, sort, head_tail, base64, tr, word_count, transcribe, browser, read_file, write_file, search_files, patch, shell, delegate_tasks, session_search. ### Terminal Rendering (`internal/render/`) -Vertical space compression — `Start()` is a no-op; blank lines removed from Iteration/FinalAnswer/Summary. Raw-mode cursor uses `\r\n` for cross-platform compatibility. +Vertical space compression is baked into the render paths; blank lines removed from Iteration/FinalAnswer/Summary. Raw-mode cursor uses `\r\n` for cross-platform compatibility. ### Identity System prompt priority: `--system` flag > `~/.odek/IDENTITY.md` > compiled-in defaultSystem. Explicit prompts and IDENTITY.md are capped at 256 KiB and scanned with `danger.ScanInjection` (failure → compiled-in default). Project `AGENTS.md` ignored if >256 KiB. The compiled-in default carries the execution-provenance rules (justification from the principal; read what you execute; failed reads never become executions; deferred-execution confirmation; tool metadata is not directives) and is itself scanner-clean — pinned by `TestDefaultSystem_PassesOwnInjectionScan` so a copy into `IDENTITY.md` is never rejected. Operator identity surfaces (`--system`, `ODEK_SYSTEM`, config `system`, `IDENTITY.md`) carry identity only — name, mission, persona. `buildSystemPrompt` force-composes the invariant pillar on top of every accepted identity via `composeSystem` (idempotent — an identity already carrying the pillar is kept whole), so no operator surface can drop the security rules; the compiled default is `defaultIdentity` + pillar, pinned by `cmd/odek/system_pillar_test.go`. Sub-agents compose the same invariant pillar into `subagentSystem` (shared `securityPillar` const: Safety, Execution provenance, IPI) plus role amendments — a child has no principal channel, so confirmation rules become skip-and-report, justification scope is the declared task, and deferred execution requires the task to name the mechanism. Parity is pinned by `cmd/odek/subagent_pillar_test.go`; the operator-writable parent surface (`--system`, `IDENTITY.md`) never propagates to children. @@ -143,7 +143,7 @@ Layered prompt-injection / approval-fatigue defenses. The full per-mitigation li - **Danger classifier** (`internal/danger/classifier.go`) — bypass-resistant normalization ($IFS, command substitution, wrappers, backslashes, basenames); covers awk/sed/editor escapes, pipe-fed xargs composition, root-level mutation targets, git data-loss verbs, `gh` as network egress, `git -c`/config code exec, find/rsync destructive flags, env dumps, shell operand/redirect path classification (writes to shell rc files, ~/.ssh, ~/.odek escalate to system_write). Trust anchors under `~/.odek` are write-protected from generic file tools. The read ledger is fingerprinted (`WasReadFresh`: post-read mutation re-fires the unread-script gate), and unread-script approvals carry a pre-exec injection-scan enrichment incl. single-layer base64/hex decode (`cmd/odek/unreadscan.go`, scan never populates the ledger). - **Approval friction** — TTY/WS/Telegram approvers engage friction after 3 same-class approvals in 60s (type `approve`, pause, trust shortcut hidden); `destructive`/`blocked`/`unknown` never get trust shortcuts. TTY prompts are process-wide serialized. - **Sub-agent caps** — `delegate_tasks` carries trust_level + max_risk enforced via the sub-agent's DangerousConfig; MCP tools withheld from untrusted sub-agents; API keys handed off via unlinked-tempfile FD, never env. -- **MCP hardening** — subprocess env sanitization (secret-pattern stripping), tool-name/description/inputSchema validation + injection scans, per-tool approval for project servers (keys hash command/args/env + all four limit fields), per-server limits with absolute ceilings, artifact-ref fail-closed validation. +- **MCP hardening** — subprocess env sanitization (secret-pattern stripping), tool-name/description/inputSchema validation + injection scans, per-tool approval for every server (keys hash command/args/env + schema hash + description text + all four limit fields), per-server limits with absolute ceilings, artifact-ref fail-closed validation. - **Config trust split** — `./odek.json` is untrusted: sensitive sections (base_url, api_key, system, dangerous, memory, telegram, web_search, embedding, sessions, skills.dirs) ignored with warnings; sandbox knobs gated behind explicit operator approval (incl. implicit `Dockerfile.odek` builds, content-hash keyed); project limits may only lower global budgets, project prices rejected outright. Global config/secrets permission-checked; config files size-capped. - **Serve / network surface** — per-instance CSRF token on `/ws` and all `/api/*`, loopback Host checks, local-origin requirement for mutations, per-session auth tokens + rate limiting, clickjacking headers, WS message-size caps. SSRF dial guard (DNS-rebinding-safe, internal-IP refusal, proxy refusal) on browser/http_batch/web_search. - **Budgets, events, refs (v1.24.0)** — budget clamp merge (see above); event stream carries SHA-256 arg hashes + sizes only (never raw args), redact applied, JSONL sink 0600/no-symlink/fsync-per-event, drop-on-full dispatch; external refs validated and never dereferenced. diff --git a/README.md b/README.md index 6b72968..1d3ca10 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ odek is not a framework. It's a **runtime** — the smallest possible surface ar | Dependencies | **5.** 2× 21no.de, 3× golang.org/x | 200+ packages | | Binary size | ~11 MB static | 50-200 MB with venv | | Startup | **Instant** | 2-10s (Python imports) | -| Sandbox | `--sandbox` flag | Requires manual Docker setup | +| Sandbox | **Default-on** Docker sandbox (`--no-sandbox` to opt out) | Requires manual Docker setup | | Tool interface | One interface, one method | Class hierarchies + decorators | --- @@ -35,10 +35,10 @@ odek is not a framework. It's a **runtime** — the smallest possible surface ar ## Strategic Features ### 🔒 Sandboxed Execution -Every session can run in an isolated Docker container: no network, no host mounts beyond the working directory, zero capabilities, destroyed on exit. `odek serve` enables the sandbox **by default**; `odek run` keeps it opt-in but warns when running unsandboxed. `--ctx` files are auto-injected into the container at `/workspace/`. Full security model in [docs/SANDBOXING.md](docs/SANDBOXING.md). +Every session can run in an isolated Docker container: no network, no host mounts beyond the working directory, zero capabilities, destroyed on exit. Sandboxing is **on by default** for `odek run`, `odek repl`, and `odek serve`; opt out with `--no-sandbox` / `ODEK_NO_SANDBOX=1` (unsandboxed runs warn loudly, and `ODEK_REQUIRE_SANDBOX=1` makes them fatal). `--ctx` files are auto-injected into the container at `/workspace/`. Full security model in [docs/SANDBOXING.md](docs/SANDBOXING.md). ### 🛡️ Prompt-Injection-Aware -External content the agent ingests (`browser`, `read_file`, `shell`, `search_files`, `multi_grep`, `transcribe`, `vision`, `web_search`, `session_search`, MCP tools) is wrapped in per-call nonce'd `` boundaries so the model can distinguish data from instructions. Redirect hops are re-classified (`browser`/`http_batch`), MCP tool descriptions are scanned for injection at registration, and the MCP error channel is wrapped too. The danger classifier resists 8 known shell-evasion tricks (`$()`, backticks, `$IFS`, `command`/`exec`, `\rm`, basenamed absolute paths). Approvers engage friction mode after 3 same-class approvals in 60 s. Memory episodes from tainted sessions are stored but never auto-replayed. Imported and project skills track provenance — untrusted ones stay excluded from trigger matching until explicit `odek skill promote --force`. `odek audit ` surfaces every ingest + per-turn divergence heuristic. Full threat model in [docs/SECURITY.md](docs/SECURITY.md). +External content the agent ingests (`browser`, `read_file`, `shell`, `search_files`, `multi_grep`, `transcribe`, `vision`, `web_search`, `session_search`, MCP tools) is wrapped in per-call nonce'd `` boundaries so the model can distinguish data from instructions. Redirect hops are re-classified (`browser`/`http_batch`), MCP tool descriptions are scanned for injection at registration, and the MCP error channel is wrapped too. The danger classifier resists common shell-evasion tricks (`$()`/backtick substitution, `$IFS`, brace expansion, `command`/`env` wrappers, `\rm`, basenamed absolute paths, and more). Approvers engage friction mode after 3 same-class approvals in 60 s. Memory episodes from tainted sessions are stored but never auto-replayed. Imported and project skills track provenance — untrusted ones stay excluded from trigger matching until explicit `odek skill promote --force`. `odek audit ` surfaces every ingest + per-turn divergence heuristic. Full threat model in [docs/SECURITY.md](docs/SECURITY.md). ### 🧩 Sub-Agent Delegation Parallel OS-process sub-agents via `delegate_tasks`. True isolation — each sub-agent is a fresh `odek subagent` process with its own config, tools, and termination timeout. Up to 8 concurrent workers. Operator-defined **capability profiles** (top-level `profiles` config) override a sub-agent's permissions by name and fail closed on unknown names — a curated starter set of 21 task profiles ships in [`profiles.template.json`](profiles.template.json). See [docs/SUBAGENTS.md](docs/SUBAGENTS.md) and [docs/SECURITY.md](docs/SECURITY.md). diff --git a/docs/CHEATSHEET.md b/docs/CHEATSHEET.md index fdd8023..fedcd25 100644 --- a/docs/CHEATSHEET.md +++ b/docs/CHEATSHEET.md @@ -241,7 +241,7 @@ via background consolidation (`consolidate_on_end`, default true). ### Memory Tool -Single `memory` tool, 6 actions: `add`, `replace`, `remove`, `consolidate`, `read`, `search`. Targets: `user` or `env`. Facts are frozen in system prompt at session start — live writes appear next session. +Single `memory` tool, 16 actions: `add`, `replace`, `remove`, `consolidate`, `read`, `search`, `view`, `stats`, `add_atom`, `search_atoms`, `forget_atom`, `pin_atom`, `list_quarantine`, `list_pending_review`, `confirm_pending_review`, `reject_pending_review`. Targets: `user` or `env`. Facts are frozen in system prompt at session start — live writes appear next session. ## Subagents @@ -421,4 +421,6 @@ odek mcp # stdio transport | `session_search` | Browse, search, and recall past sessions (semantic vector search) | | `clarify` | Ask the user for clarification | | `send_message` | Send text/photo/document to Telegram | -| `skill_load/list/save/patch/delete` | Skill CRUD | +| `skill_load`, `skill_list` | Read the loaded skill / list available skills | +| `list_subagent_profiles` | Discover operator-defined sub-agent capability profiles (+ built-in default) | +| `artifact_read` | Read a sub-agent result artifact by id | diff --git a/docs/CLI.md b/docs/CLI.md index f25eae4..ce0f268 100644 --- a/docs/CLI.md +++ b/docs/CLI.md @@ -7,7 +7,7 @@ | `odek run [flags] ` | Execute a task with the agent loop (single-shot by default) | | `odek run --session [flags] ` | Execute and save conversation as a multi-turn session | | `odek continue [--id ] [--external-ref ] ` | Continue the most recent session (or by `--id`). Sessions persist per completed step: Ctrl-C/SIGTERM resumes from the last step; SIGKILL may lose the in-flight step | -| `odek repl [flags]` | Interactive REPL mode (persistent multi-turn session). Flags: `--id`, `--model`, `--thinking`, `--thinking-budget`, `--sandbox`, `--sandbox-*`, `--prompt-caching`, `--stream`, `--compaction`, `--planning` / `--no-planning`, `--interaction-mode`. Unknown flags are silently ignored — in particular `--tool` / `--no-tool` are **not** supported in repl (use `odek run`, `serve`, or the `tools` config instead). | +| `odek repl [flags]` | Interactive REPL mode (persistent multi-turn session). Flags: `--id`, `--model`, `--thinking`, `--thinking-budget`, `--sandbox`, `--sandbox-*`, `--prompt-caching`, `--stream`, `--compaction`, `--planning` / `--no-planning`, `--interaction-mode`. Unrecognized flags are rejected with an error — in particular `--tool` / `--no-tool` are **not** supported in repl (use `odek run`, `serve`, or the `tools` config instead). | | `odek session list` | List sessions | | `odek session show [id]` | Show session details (default: latest) | | `odek session delete ` | Delete a session | @@ -25,8 +25,8 @@ | `odek memory extended [args]` | Extended-memory operations: delete/promote/pin atoms, list or confirm/reject pending-review atoms, quarantine listing, manual compaction, store stats, consolidate, and proactive-nudge management | | `odek audit ` | Print the prompt-injection audit log for a session (JSON) | | `odek audit --list` | List sessions with non-zero ingest counts and divergence flags | -| `odek serve [--addr ] [--open] [--no-sandbox] [--trusted-proxies ] [--log-file ]` | Web UI server (default `127.0.0.1:8080`). Sandbox is on by default; pass `--no-sandbox` to disable. Flags: `--tool` / `--no-tool` (repeatable), `--prompt-caching`, `--compaction` / `--no-compaction`, `--planning` / `--no-planning`, `--stream` / `--no-stream`, `--log-file` (durable run/turn log, default `~/.odek/serve.log`). Binding to a non-loopback address prints a loud warning because anyone with the token can drive the agent. `--trusted-proxies` honours `X-Forwarded-For`/`X-Real-Ip` only from those addresses. | -|| `odek subagent --goal [flags]` | Run a focused sub-task; outputs JSON on stdout. Spawned by `delegate_tasks` tool. Flags: `--goal`, `--task `, `--context`, `--timeout` (≤1800s), `--max-iter` (≤100), `--profile `, `--parent-session `, `--quiet`, `--stream`. | +| `odek serve [--addr ] [--open] [--no-sandbox] [--trusted-proxies ] [--log-file ]` | Web UI server (default `127.0.0.1:8080`). Sandbox is on by default; pass `--no-sandbox` to disable. Flags: `--tool` / `--no-tool` (repeatable), `--prompt-caching`, `--compaction`, `--planning` / `--no-planning`, `--stream` / `--no-stream`, `--log-file` (durable run/turn log, default `~/.odek/serve.log`). Binding to a non-loopback address prints a loud warning because anyone with the token can drive the agent. `--trusted-proxies` honours `X-Forwarded-For`/`X-Real-Ip` only from those addresses. | +| `odek subagent --goal [flags]` | Run a focused sub-task; outputs JSON on stdout. Spawned by `delegate_tasks` tool. Flags: `--goal`, `--task `, `--context`, `--timeout` (≤1800s), `--max-iter` (≤100), `--profile `, `--parent-session `, `--quiet`, `--stream`. | | `odek init [--global|--local] [--force]` | Create a config file template (scope-aware: full schema globally, project-safe fields locally) | | `odek mcp [--sandbox]` | Start MCP server (expose tools to Claude Code) or connect to external MCP servers (via `mcp_servers` config) | | `odek telegram` | Start the Telegram bot (long-polling). Hosts the embedded scheduler unless `schedules.enabled=false` | @@ -40,7 +40,7 @@ Unknown flags are a **hard error** — they are never folded into the task text | Flag | Type | Default | Description | |------|------|---------|-------------| -| `--model ` | string | `deepseek-chat` | LLM model — profiles auto-set thinking/timeout (see [Providers](PROVIDERS.md)). Consider using `deepseek-v4-flash` for faster/cheaper tasks. | +| `--model ` | string | `deepseek-v4-flash` | LLM model — profiles auto-set thinking/timeout (see [Providers](PROVIDERS.md)). | | `--base-url ` | string | `https://api.deepseek.com/v1` | OpenAI-compatible API endpoint | | `--max-iter ` | int | `90` | Max think→act cycles | | `--thinking ` | string | profile default | Reasoning depth: `enabled`/`disabled`/`low`/`medium`/`high`. Requires a model that supports extended thinking. | @@ -303,7 +303,7 @@ name: docker-build description: Build and optimize Docker images version: 1.0.0 author: odek -`odek: +odek: trigger: topic: docker container image action: build optimize diff --git a/docs/CONFIG.md b/docs/CONFIG.md index c77265e..5e0358c 100644 --- a/docs/CONFIG.md +++ b/docs/CONFIG.md @@ -129,7 +129,7 @@ This lets you keep secrets out of config files entirely: ## Environment variables -Every config knob has a `ODEK_*` counterpart: +Most config knobs have a `ODEK_*` counterpart: | Variable | Maps to | Type | |----------|---------|------| @@ -146,7 +146,6 @@ Every config knob has a `ODEK_*` counterpart: | `ODEK_PROMPT_CACHING` | `prompt_caching` | bool | | `ODEK_STREAM` | `stream` | bool | | `ODEK_COMPACTION` | `compaction` | bool | -| `ODEK_TOOL_PROGRESS` | `tool_progress` | string (all\|new\|verbose\|off) | | `ODEK_SANDBOX_IMAGE` | `--sandbox-image` | string | | `ODEK_SANDBOX_NETWORK` | `--sandbox-network` | string | | `ODEK_SANDBOX_READONLY` | `--sandbox-readonly` | bool | diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index 5e09980..a62553a 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -58,7 +58,7 @@ internal/ sandbox_test.go Sandbox tests (BuildRunArgs, ResolveImage, InjectFiles) danger/ classifier.go Command/URL classification for security gating - classifier_test.go Risk classification, 8 classes, config overrides + classifier_test.go Risk classification, 11 classes, config overrides approver.go Approver interface + TTYApprover (CLI /dev/tty) memory/ memory.go MemoryManager orchestrator (facts, buffer, episodes) @@ -131,6 +131,20 @@ docs/ Documentation SECURITY.md Prompt injection, security model SANDBOXING.md Sandbox configuration MCP.md MCP server over stdio (Model Context Protocol) + EXTENSIONS.md odek-extension/v1: limits, artifacts, events, refs, budgets + EXTENDED_MEMORY.md Atom store, quarantine, consolidation, nudges + MEMORY.md Facts, buffer, episodes, promotion + PLANNING.md Plan tool + protected plan message + TELEGRAM.md Telegram bot integration + SCHEDULES.md Native cron scheduler + STREAMING.md Token/reasoning streaming + CACHING.md Prompt caching + MAINTENANCE.md Storage janitor + odek cleanup + TOOL_SELECTION.md tools.enabled/disabled reference + CHEATSHEET.md Quick reference + REDACTION_HARDENING.md Secret redaction patterns + DAILY-WORKER.md Daily-worker usage guide + DOCKER_COMPOSE_USER_GUIDE.md docker/ compose stack guide DEVELOPMENT.md This file ``` @@ -184,7 +198,7 @@ CI (`.github/workflows/test.yml`) runs the unit suite under `-race` on every pus | `internal/ws` | WebSocket constant verification | | `internal/resource` | @-reference parsing, file resolution, session resolution, security | | `internal/render` | Terminal output, no-color mode, nil safety, tool call/result rendering | -| `internal/danger` | Command classification across 9 risk classes (incl. fail-closed `unknown`), config overrides, allow/denylist, classifier-bypass attempts, approver friction | +| `internal/danger` | Command classification across 11 risk classes (incl. fail-closed `unknown`), config overrides, allow/denylist, classifier-bypass attempts, approver friction | | `internal/memory` | Facts CRUD, buffer ring, episodes, merge detector (go-vector), ReplaceEntry/AppendEntry, memory tool, security scan, LLM ranking, episode provenance | | `internal/skills` | Loading, triggers, import, agent tools (skill_load/skill_list), ValidateSkillName, isPrivateHost | | `internal/telegram` | Bot client, long-polling, command handlers, session management, plan CRUD, voice/photo download, health server, retry/backoff | diff --git a/docs/DOCKER_COMPOSE_USER_GUIDE.md b/docs/DOCKER_COMPOSE_USER_GUIDE.md index dc79d23..c135a02 100644 --- a/docs/DOCKER_COMPOSE_USER_GUIDE.md +++ b/docs/DOCKER_COMPOSE_USER_GUIDE.md @@ -25,7 +25,11 @@ two permission profiles: local Ollama endpoint, etc.). Odek reads it from `ODEK_API_KEY` (with legacy fallbacks `DEEPSEEK_API_KEY` → `OPENAI_API_KEY`). -All files below live in the **repository root** (next to `go.mod`). Create them as shown. +All files below live in the **`docker/` directory** — not the repository root. The compose +file builds with `context: ..` (repo root) and `dockerfile: docker/Dockerfile`, and all +`docker compose` commands are meant to run from `docker/` so the relative paths and `.env` +resolve. Everything ships with the repository — the only file you create yourself is +`.env` (copied from `.env.example`). --- @@ -35,65 +39,69 @@ After following this guide you will have added: ``` odek/ -├── Dockerfile # builds the odek binary -├── docker-compose.yml # restricted + godmode services -├── .env # your API key + model settings (gitignored) -├── config.restricted.json # Restricted permission policy -├── config.godmode.json # Godmode (YOLO) permission policy -├── workspace/ # the directory the agent works in (mounted into the container) -└── .odek/ # Telegram bot state: sessions, skills, lock (mounted in) +├── go.mod # repo root — the compose build context (`context: ..`) +└── docker/ # run all `docker compose` commands from here + ├── Dockerfile # 4-stage image build (see §3) + ├── Dockerfile.embeddings # llama.cpp embeddings sidecar (bundled GGUF) + ├── docker-compose.yml # restricted + godmode + telegram profiles, sidecars + ├── .env.example # template — copy to `.env` + ├── .env # your API key + model settings (you create this) + ├── config.restricted.json # Restricted permission policy + ├── config.godmode.json # Godmode (YOLO) permission policy + ├── searxng/settings.yml # SearXNG sidecar settings + ├── piguard/ # PIGuard sidecar (model download script, models dir) + ├── workspace/ # the directory the agent works in (mounted into the container) + └── .odek/ # Telegram bot state: sessions, skills, lock (created on first run) ``` -> Add `.env`, `workspace/`, and `.odek/` to your `.gitignore` so you never commit secrets -> or scratch files. +> `.env`, `workspace/`, and `.odek/` are already ignored by the repository's root `.gitignore` +> (`docker/.env`, `docker/workspace/*`, `docker/.odek/*`), so secrets and scratch files are +> never committed. --- ## 3. The Dockerfile -A multi‑stage build: compile the static binary with the Go toolchain, then ship it on a -small runtime image that already has a shell and common tooling for the agent to use. +A four‑stage build: compile the static binary with the Go toolchain, build the whisper.cpp +CLI and fetch its multilingual `small` model, build the llama‑mtmd‑cli vision runner and +fetch the MiniCPM‑V model, then assemble a Debian (bookworm‑slim) runtime image with the +agent's tooling. The full file lives at `docker/Dockerfile`; the annotated skeleton below +shows what each stage does. ```dockerfile # syntax=docker/dockerfile:1 +# Build context MUST be the repository root (compose sets +# build: { context: .., dockerfile: docker/Dockerfile }). # ---- build stage ---- FROM golang:1.25-alpine AS build -WORKDIR /src +# go mod download → CGO_ENABLED=0 go build -ldflags "-s -w" → /out/odek +# (fully static, so it runs unchanged on the Debian runtime stage) -# Cache modules first -COPY go.mod go.sum ./ -RUN go mod download +# ---- whisper stage ---- +FROM debian:bookworm-slim AS whisper +# Builds whisper.cpp's CLI from a pinned release (WHISPER_VERSION) and fetches +# the multilingual `small` GGML model (WHISPER_MODEL, default small) into a +# fixed image path so the `transcribe` tool works with zero setup. -# Build the static binary (mirrors the Makefile `build` target) -COPY . . -RUN CGO_ENABLED=0 go build -ldflags "-s -w" -o /out/odek ./cmd/odek +# ---- minicpm-v stage ---- +FROM debian:bookworm-slim AS minicpm +# Builds llama-mtmd-cli from source (LLAMA_VERSION) and fetches the +# MiniCPM-V GGUF + vision projector (MINICPM_QUANT, default Q4_K_M) so the +# `vision` tool works with zero setup. # ---- runtime stage ---- -FROM alpine:latest -# Tooling the agent commonly needs inside the sandbox container. -# Trim or extend this list to taste. -RUN apk add --no-cache ca-certificates git bash coreutils curl jq - -# Run as a non-root user — defense in depth even inside the container. -# Pre-create ~/.odek owned by the user so it's writable for config, sessions, -# and the Telegram lock (whether backed by an image dir or a mounted folder). -RUN adduser -D -u 1000 odek \ - && mkdir -p /home/odek/.odek /workspace \ - && chown -R odek:odek /home/odek/.odek /workspace - -COPY --from=build /out/odek /usr/local/bin/odek - -# Docker does NOT set $HOME from USER, but Odek resolves ~/.odek via $HOME. -# Set it explicitly so config.json, sessions, and the Telegram lock land in -# /home/odek/.odek (where the volume and config bind mounts are). -ENV HOME=/home/odek -USER odek -WORKDIR /workspace - -ENTRYPOINT ["odek"] +FROM debian:bookworm-slim +# Agent tooling via apt: ca-certificates git bash coreutils curl jq ffmpeg +# libstdc++6, plus gh (official apt repo), python3 + venv, Go (official +# tarball, GO_VERSION) and Bun. Copies in the odek binary and the whisper + +# minicpm artifacts. Runs as non-root user `odek` (uid 1000, HOME=/home/odek), +# WORKDIR /workspace, ENTRYPOINT ["odek"]. ``` +> Model sizes and build args (WHISPER_MODEL, MINICPM_QUANT, GO_VERSION, …) are documented +> inline in `docker/Dockerfile`. + > **Why no `--sandbox` inside the container?** Odek's `--sandbox` mode launches *nested* > Docker containers for each command, which would require mounting the Docker socket > (Docker‑in‑Docker) — a much larger attack surface. Running Odek directly in this @@ -163,24 +171,29 @@ These JSON files are mounted to `/home/odek/.odek/config.json` inside the contai ### 5a. Restricted policy — `config.restricted.json` -Commands are risk‑classified; destructive and unrecognised ones are denied, the rest -prompt for approval. Crucially, `non_interactive` is set to **`deny`** so that if the -agent runs in a container *without* an attached terminal or Web UI, anything that would -prompt is blocked rather than silently allowed. +Commands are risk‑classified. Safe reads and local writes run without approval — as does +network egress, which the LLM API and sidecars need. Installs, code execution, system +writes, and persistence attempts prompt for approval; `unknown` and `destructive` are +denied outright. For headless runs, `non_interactive` is **`read_only`**: read‑only +inspection proceeds without a human channel, and anything that would prompt is denied. ```json { "sandbox": false, "dangerous": { - "action": "prompt", - "non_interactive": "deny", + "non_interactive": "read_only", "classes": { - "destructive": "deny", - "system_write": "prompt", - "network_egress": "prompt", - "code_execution": "prompt", + "safe": "allow", + "local_write": "allow", "install": "prompt", - "local_write": "allow" + "network_egress": "allow", + "code_execution": "prompt", + "persistence": "prompt", + "unread_exec": "prompt", + "system_write": "prompt", + "unknown": "deny", + "destructive": "deny", + "blocked": "deny" }, "allowlist": [], "denylist": ["rm -rf /"] @@ -194,7 +207,7 @@ prompt is blocked rather than silently allowed. | --- | --- | | `sandbox` | `false` runs commands directly in this container (the Compose setup already *is* the sandbox). `true` would nest a second Docker sandbox — not what you want here. | | `action` | **Global default** action for any class **not** listed under `classes`. `"prompt"` here, `"allow"` = godmode, `"deny"` = lockdown. ⚠️ This overrides the *built‑in* per‑class defaults (see the gotcha below). | -| `non_interactive` | What to do with a **prompt**‑level command when there is no human channel (no TTY, no Web UI). `"deny"` blocks it; `"allow"` runs it. Always set this to `"deny"` for unattended/automated containers. | +| `non_interactive` | What to do with a **prompt**‑level command when there is no human channel (no TTY, no Web UI). `"deny"` blocks it; `"allow"` runs it; `"read_only"` (the shipped Restricted value) lets read‑only/inspection commands proceed and denies the rest. | | `classes` | Per‑class action overrides. The most specific setting — it wins over `action` and the built‑in defaults. Only list the classes you want to pin. | | `allowlist` | Commands that always run, **exact string match**, no classification. Highest priority of all. Use for a handful of trusted exact commands (e.g. `"npm run deploy"`). | | `denylist` | Commands that are always denied, **prefix match** after trimming. Beats classification and even godmode — but **not** the allowlist. | @@ -203,19 +216,20 @@ prompt is blocked rather than silently allowed. | Class | Examples | Built‑in default | This profile | | --- | --- | --- | --- | -| `safe` | `ls`, `cat`, `grep`, `git status` | allow | prompt¹ | +| `safe` | `ls`, `cat`, `grep`, `git status` | allow | allow | | `local_write` | write files in the working dir | allow | allow | | `install` | `npm install`, `pip install`, `apk add` | prompt | prompt | -| `network_egress` | `curl`, `wget`, `ssh`, DNS lookups | prompt | prompt | +| `network_egress` | `curl`, `wget`, `ssh`, DNS lookups | prompt | allow | | `code_execution` | `curl … \| sh`, `bash -c`, `python -c`, `go run` | prompt | prompt | | `system_write` | `sudo`, writes to `/etc`, reads of `~/.ssh` | prompt | prompt | -| `unknown` | any command whose program name Odek does **not** recognise | deny | prompt¹ → denied unattended | +| `unknown` | any command whose program name Odek does **not** recognise | deny | deny | | `destructive` | `rm -rf /`, `dd … of=/dev/sda`, `mkfs` | deny | **deny** | | `blocked` | fork bombs, fully‑specified `dd` to a block device | **always deny** | **always deny** (cannot be overridden) | -> ¹ `safe` and `unknown` are not listed under `classes`, so the global -> `action: "prompt"` applies to them — see the gotcha below. With a human channel -> they prompt; unattended (`non_interactive: "deny"`) they are denied. +> The shipped Restricted file pins the classes explicitly: `safe`, `local_write`, and +> `network_egress` are allowed; `install`, `code_execution`, `persistence`, `unread_exec`, +> and `system_write` prompt; `unknown`, `destructive`, and `blocked` are denied. See the +> gotcha below before adding a global `action`. Odek **fails closed**: the `unknown` class catches any command whose verb isn't in the built‑in safe/dangerous tables, so a novel or obfuscated command can't slip through as @@ -229,19 +243,12 @@ or relax the class with `"unknown": "prompt"`. 3. Otherwise classify it, then: explicit **`classes`** entry → `blocked` is **always deny** → global **`action`** (if set) → built‑in class default. 4. If the result is **prompt** and there's no human channel, **`non_interactive`** decides. -> **Gotcha — `action` overrides *every* unlisted class.** Because `action: "prompt"` is -> set, any class you don't list under `classes` resolves to *prompt*, including `safe`. -> So with this profile as written, even `ls` prompts (and is denied unattended). Two ways -> to get the usual "safe commands just run" behavior: -> -> - add `"safe": "allow"` to `classes` (keep `action: "prompt"` as the catch‑all for -> everything else, including `unknown`), **or** -> - **omit `action` entirely** and only override the classes you care about — then unlisted -> classes keep their built‑in defaults (safe/local_write allow; destructive/blocked/unknown -> deny; system_write/network_egress/code_execution/install prompt). -> -> The second form is the better default if you want `unknown` to stay deny‑by‑default -> rather than prompt. +> **Gotcha — `action` overrides *every* unlisted class.** The shipped Restricted file +> does **not** set `action`; it pins the classes it cares about explicitly (see the table +> above), so any class it omits keeps its built‑in default. If you add a global `action`, +> it becomes the default for every class you don't list — e.g. `action: "prompt"` would +> make even `ls` prompt (and be denied unattended). Either keep pinning classes +> explicitly, or pick `action` with that catch‑all behavior in mind. > Approvals require a human channel: the **Web UI** (`odek serve`, modal approval over > WebSocket) or an **interactive terminal** (`odek repl` with `docker compose run -it`). @@ -289,8 +296,11 @@ configurable. ## 6. The Compose file -Two services share the same image but mount a different policy file. Compose -**profiles** keep them from starting together — you opt into one at a time. +Four odek services share the same image but mount a different policy file (the Telegram +pair also mounts a writable `./.odek` state folder). Compose **profiles** keep them from +starting together — you opt into one at a time. Every odek service also co‑starts three +sidecars — `searxng`, `llama-embeddings`, and `piguard-gateway` — via `depends_on`; the +excerpt below is abridged. ```yaml # docker-compose.yml @@ -299,7 +309,9 @@ services: # ── Restricted (default) — interactive Web UI with approval prompts ── odek-restricted: profiles: ["restricted"] - build: . + build: + context: .. # repo root + dockerfile: docker/Dockerfile image: odek:local env_file: .env command: ["serve", "--addr", "0.0.0.0:8080", "--no-sandbox"] @@ -309,11 +321,14 @@ services: - ./workspace:/workspace - ./config.restricted.json:/home/odek/.odek/config.json:ro restart: "no" + depends_on: [searxng, llama-embeddings, piguard-gateway] # ── Godmode (all permissions) — non-interactive, disposable container ── odek-godmode: profiles: ["godmode"] - build: . + build: + context: .. # repo root + dockerfile: docker/Dockerfile image: odek:local env_file: .env # No published ports (no inbound needed). Outbound networking stays on — @@ -323,6 +338,7 @@ services: - ./workspace:/workspace - ./config.godmode.json:/home/odek/.odek/config.json:ro restart: "no" + depends_on: [searxng, llama-embeddings, piguard-gateway] ``` Notes: @@ -361,8 +377,9 @@ Then: WebSocket token, e.g. `http://127.0.0.1:8080/?token=...`. Open that exact URL in your browser (plain `http://127.0.0.1:8080` no longer receives the token). 2. Type a task, e.g. *"List the files in this directory and summarize the README."* -3. When the agent attempts a higher‑risk command (network, install, code execution), an - **approval modal** appears showing the command and its risk class. Approve or deny. +3. When the agent attempts a higher‑risk command (an install, code execution, system + write, or persistence attempt), an **approval modal** appears showing the command and + its risk class. Approve or deny. 4. Destructive commands are rejected automatically — you'll see the denial in the stream. Stop with `Ctrl‑C`, then `docker compose --profile restricted down`. @@ -384,9 +401,10 @@ docker compose run --rm -it \ > argument here. > One‑shot `odek run ""` works too, but it is non‑interactive: with the Restricted -> policy above, `prompt`‑class commands are **denied** (`non_interactive: "deny"`) and -> destructive ones are always denied. Use this for tasks that only need safe / local‑write -> operations, or add specific commands to the policy's `allowlist`. +> policy above, `non_interactive: "read_only"` lets read‑only/inspection commands proceed +> and denies everything that would prompt (`unknown` and `destructive` are denied +> regardless). Use this for tasks that only need safe / local‑write operations, or add +> specific commands to the policy's `allowlist`. --- @@ -493,7 +511,7 @@ global `action` → built‑in defaults. The `blocked` class is always denied re | Symptom | Likely cause / fix | | --- | --- | | `odek serve` exits complaining about sandbox / Docker | You omitted `--no-sandbox`. Odek tried to start nested sandbox containers. Add `--no-sandbox` to the `command`. | -| Agent says "operation denied by configuration" for normal commands | You're running non‑interactively under the Restricted policy (`non_interactive: "deny"`). Use the Web UI / `repl -it`, or add the command to `allowlist`. | +| Agent says "operation denied by configuration" for normal commands | You're running non‑interactively under the Restricted policy (`non_interactive: "read_only"` — only read‑only commands proceed). Use the Web UI / `repl -it`, or add the command to `allowlist`. | | Approval modal never appears; risky commands just run | The Godmode policy is mounted, or `action` is `allow`. Check `/home/odek/.odek/config.json` inside the container. | | "no API key" / auth errors | `.env` not loaded or key invalid. Confirm `env_file: .env` is set and `ODEK_API_KEY` is correct. | | Config changes ignored | The file is mounted read‑only at startup; recreate the container (`docker compose ... up` again) after editing the JSON. | @@ -538,40 +556,49 @@ ODEK_TELEGRAM_SESSION_TTL_HOURS=24 # optional ### 13c. Compose services -Add these to `docker-compose.yml`. State (per‑chat sessions, the daily‑budget counter, and -the singleton lock) lives in a local **`./.odek` folder** — an external host folder, just -like `./workspace` — so it survives restarts and is easy to inspect. No `ports` are needed. +The compose file ships these two services (shown abridged). State (per‑chat sessions, the +daily‑budget counter, and the singleton lock) lives in a local **`./.odek` folder** — an +external host folder, just like `./workspace` — so it survives restarts and is easy to +inspect. No `ports` are needed. ```yaml # ── Telegram bot — Restricted (approvals via inline keyboards) ── odek-telegram-restricted: profiles: ["telegram-restricted"] - build: . + build: + context: .. # repo root + dockerfile: docker/Dockerfile image: odek:local env_file: .env command: ["telegram"] + init: true # reaps agent child processes + forwards SIGTERM for clean shutdown volumes: - ./workspace:/workspace - ./.odek:/home/odek/.odek - ./config.restricted.json:/home/odek/.odek/config.json:ro restart: unless-stopped + depends_on: [searxng, llama-embeddings, piguard-gateway] # ── Telegram bot — Godmode (no prompts; disposable container) ── odek-telegram-godmode: profiles: ["telegram-godmode"] - build: . + build: + context: .. # repo root + dockerfile: docker/Dockerfile image: odek:local env_file: .env command: ["telegram"] + init: true volumes: - ./workspace:/workspace - ./.odek:/home/odek/.odek - ./config.godmode.json:/home/odek/.odek/config.json:ro restart: unless-stopped + depends_on: [searxng, llama-embeddings, piguard-gateway] ``` -Create the folder first (so the container's non‑root user can write to it) and gitignore -its contents: +Create the folder first (so the container's non‑root user can write to it) — the repo's +root `.gitignore` already ignores its contents (`docker/.odek/*`): ```bash mkdir -p .odek && chmod 777 .odek && touch .odek/.gitkeep diff --git a/docs/EXTENSIONS.md b/docs/EXTENSIONS.md index 57d9f36..476bbe6 100644 --- a/docs/EXTENSIONS.md +++ b/docs/EXTENSIONS.md @@ -162,7 +162,8 @@ odek can emit a structured runtime event stream: **one JSON object per line - `type` is one of: `run_started`, `iteration_completed`, `tool_call_started`, `tool_call_completed`, `tool_call_failed`, `session_saved`, `context_trimmed`, `budget_exceeded`, `run_completed`, - `run_failed`, `plan_created`, `plan_updated`, `subagent_denied`. + `run_failed`, `plan_created`, `plan_updated`, `subagent_denied`, + `subagent_spawned`, `subagent_completed`. - `run_id` is a random 128-bit hex identifier generated per agent run and stamped on every event of that run. `session_id` appears once the session is known; earlier events omit it. `iteration` is the 1-based loop diff --git a/docs/MAINTENANCE.md b/docs/MAINTENANCE.md index 910c2b3..8f3b284 100644 --- a/docs/MAINTENANCE.md +++ b/docs/MAINTENANCE.md @@ -73,7 +73,8 @@ The `[maintenance]` section (all keys optional — defaults shown): "sessions_max_age_days": 30, "audit_max_age_days": 14, "log_max_mb": 50, - "plans_max_age_days": 30 + "plans_max_age_days": 30, + "artifacts_max_age_hours": 24 } } ``` @@ -86,6 +87,7 @@ The `[maintenance]` section (all keys optional — defaults shown): | `audit_max_age_days` | `14` | Delete prompt-injection audit records older than this | | `log_max_mb` | `50` | Rotate logs larger than this | | `plans_max_age_days` | `30` | Delete plans older than this | +| `artifacts_max_age_hours` | `24` | Sweep orphaned sub-agent artifacts older than this (`0` keeps them forever; live removal still happens on session delete) | The maintenance config is **operator-only**: like `base_url`, `api_key`, and the `dangerous` section, it is honored from `~/.odek/config.json` (and process diff --git a/docs/MCP.md b/docs/MCP.md index 8546a49..54ad7b9 100644 --- a/docs/MCP.md +++ b/docs/MCP.md @@ -45,7 +45,9 @@ Default exposure (no `tools` config, no SearXNG): | `read_file`, `write_file`, `search_files`, `batch_read`, `glob`, `file_info` | file access | | `browser`, `http_batch`, `web_search`* | web access (*only when SearXNG configured) | | `multi_grep`, `diff`, `tree`, `count_lines`, `head_tail`, `word_count`, `checksum`, `sort`, `base64`, `tr`, `json_query`, `math_eval` | inspection & transforms | -| `session_search`, `transcribe`, `vision`, `plan` | sessions, media, planning | +| `session_search`, `transcribe`, `vision` | sessions & media | +| `plan`* | planning (*only when planning is enabled) | +| `skill_load`, `skill_list`, `artifact_read`, `list_subagent_profiles` | skills, artifacts, sub-agent profiles | ### Sandbox @@ -193,16 +195,18 @@ Approval methods: ODEK_APPROVE_MCP=1 odek run "task" ``` 3. **Persisted approvals** — approvals are stored in - `~/.odek/mcp_approvals.json` (0600) keyed by project directory + server name + `~/.odek/mcp_approvals.json` (0600) keyed by project directory + server + name + command + args + sorted `env` map hash + the canonical-JSON SHA-256 + of the input schema + the full tool description text + the + odek-extension/v1 limit fields (`timeout_seconds`, `max_response_bytes`, + `max_result_chars`, `artifact_roots`). If any of these change, you are + prompted again — editing `artifact_roots` widens the set of files a server + may hand to the agent, so it can never silently reuse an old approval. + **Upgrade note:** because the odek-extension/v1 limit fields joined the + hash, approvals persisted by an older odek re-prompt **once** after + upgrading, then stick. 4. **`auto_approve: true`** in the global config — pre-trust a server and skip both prompts; see [Auto-approving trusted servers](#auto-approving-trusted-servers) - + command + args + sorted `env` map hash + the odek-extension/v1 limit - fields (`timeout_seconds`, `max_response_bytes`, `max_result_chars`, - `artifact_roots`). If any of these change, you are prompted again — editing - `artifact_roots` widens the set of files a server may hand to the agent, so - it can never silently reuse an old approval. **Upgrade note:** because the - odek-extension/v1 limit fields joined the hash, approvals persisted by an - older odek re-prompt **once** after upgrading, then stick. If approval is required and cannot be obtained, odek aborts before spawning any MCP server. @@ -256,8 +260,12 @@ Tool approval uses the same methods as server approval: name + tool name + sorted `env` map hash. If a tool is renamed, a new tool appears, or the server's `env` changes, it must be approved again. -Tools from global servers (`~/.odek/config.json`) are operator-trusted and do -not require per-tool approval. +Per-tool approval is required for **every** server — global servers are not +exempt. A globally-configured server can be pre-trusted with `auto_approve: true` +(global config only; the field is stripped from project config with a warning), +or per-tool prompts can be satisfied via `ODEK_APPROVE_MCP=1` or a persisted +approval. Without one of these, tool registration fails closed in +non-interactive runs. ### How it works @@ -330,8 +338,9 @@ odek: connected MCP server "playwright" (5 tools) odek: connected MCP server "fetch" (1 tool) ``` -Errors during discovery are reported at startup — the server is skipped and -odek continues with the remaining servers. +Errors during discovery are fatal at startup: odek reports the error, shuts +down any servers already started, and aborts rather than running with a +partial tool set. ### Config reference diff --git a/docs/PLANNING.md b/docs/PLANNING.md index 8610b23..dff6226 100644 --- a/docs/PLANNING.md +++ b/docs/PLANNING.md @@ -425,7 +425,7 @@ the cadence is cheap. |---------|-----------| | Approval fatigue | None. Explicit Safe class in `classifyToolCall` — plan calls never surface in approval prompts or the batch gate. State changes are confined to engine memory. Pinned end-to-end by `TestReport_PlanToolClassifiedSafe`. | | Untrusted-content boundary | Step-line bodies derive from task/tool content and are re-injected as system context every iteration. Wrapped via the engine's `SetUntrustedWrapper` with source `"plan"`, matching the compaction-digest precedent; header stays outside the wrapper so recognition survives. Audit ingest recorder records the injection where active. | -| Forgery via tool output | A hostile tool result containing a literal `[Current plan:` line cannot become *the* plan message: recognition requires role `system`, and only `refreshPlanMessage` writes that role/content pair. Rendered plan text inside a tool result stays inside the nonce'd tool-result delimiters. `TestIsPlanMessage` pins rejection of tool/assistant roles and mid-text mentions. | +| Forgery via tool output | A hostile tool result containing a literal `[Current plan:` line cannot become *the* plan message: recognition requires role `system`, and only `refreshPlanMessage` writes that role/content pair. Rendered plan text inside a tool result stays inside the nonce'd tool-result delimiters. Rejection of tool/assistant roles and mid-text mentions is enforced in `refreshPlanMessage`; the classification pin is `TestClassifyToolCall_PlanSafe`. | | Secret leakage | Plan titles/notes can echo secrets from task context. Sessions redact every message at save time (`internal/redact`) — the plan message is covered because it *is* a message riding the normal session save. | | Resume parsing | Strict and total: bad header, over-cap steps, unknown status token, count mismatch, duplicate ids, omission marker in any position (overflowed plans are not resumable), unterminated wrapper, content after the wrapper close tag — any deviation drops the whole plan (and removes the failed message from the history) instead of approximating. 19 rejection cases pinned by `TestPlan_ParseStrictRejections`. | | Config trust split | See project clamp rules above: project config cannot raise caps or flip a globally-disabled feature on. The tool reads resolved values only. | @@ -446,8 +446,8 @@ the cadence is cheap. overflow (`TestPlan_OverflowDropsDoneFirst`, `TestPlan_RenderRespectsCapAlways`). - Strict parser: 19 rejection cases (`TestPlan_ParseStrictRejections`); untrusted-wrapper unwrapping (`TestPlan_ParseUnwrapsUntrustedBody`). -- Recognition/forgery pins (`TestIsPlanMessage`); classification unit pin - (`TestClassifyToolCall_PlanSafe`). +- Recognition/forgery enforced in `refreshPlanMessage` (role/content checks); + classification unit pin (`TestClassifyToolCall_PlanSafe`). Engine integration — additions to `internal/loop/loop_test.go`: `TestEngine_Run_PlanLifecycle` (scripted fake-LLM run: create → work → diff --git a/docs/PROVIDERS.md b/docs/PROVIDERS.md index a64422d..a036465 100644 --- a/docs/PROVIDERS.md +++ b/docs/PROVIDERS.md @@ -50,7 +50,7 @@ odek ships with built-in **model profiles** that automatically apply sensible de | Model | Family | Default Thinking | Timeout | Max Context | Best For | |-------|--------|-----------------|---------|-------------|----------| -| `deepseek-chat` | DeepSeek (legacy) | (provider default) | 120s | 128K | General | +| `deepseek-chat` | DeepSeek (generic prefix match) | (provider default) | 120s | 128K | General — matched by the `deepseek-` prefix, not a dedicated profile | | `deepseek-v4-flash` | DeepSeek v4 Flash | — (faster/cheaper) | 90s | 128K | Quick tasks, coding | | `deepseek-v4-pro` | DeepSeek v4 Pro | `enabled` | 180s | **1M** | Deep reasoning | | `glm-5.3` | GLM 5.3 (Z.ai) | (always on — forced) | 300s | **1M** | Agentic coding | @@ -60,7 +60,7 @@ odek ships with built-in **model profiles** that automatically apply sensible de | `kimi-…` (e.g. `kimi-for-coding`) | Kimi | (provider default) | 300s | 256K | Agentic coding | | `k3` | Kimi | (provider default) | 300s | **1M** | Agentic coding | | `k3-256k` | Kimi | (provider default) | 300s | 256K | Agentic coding | -| *(any other)* | Generic | (profile default) | 120s | (no limit) | Custom models | +| *(any other)* | — (no profile) | (provider defaults; no profile overrides apply) | 120s | — | Custom models | ### How profiles work diff --git a/docs/SANDBOXING.md b/docs/SANDBOXING.md index 8087f1e..d26f20d 100644 --- a/docs/SANDBOXING.md +++ b/docs/SANDBOXING.md @@ -1,6 +1,6 @@ # Sandboxing -odek runs agent shell commands inside an **isolated Docker container** when `--sandbox` is active. This document covers all configuration options, the `Dockerfile.odek` build system, security guarantees, and best practices. +odek runs agent shell commands inside an **isolated Docker container** — sandboxing is **on by default** for `odek run`, `odek repl`, and `odek serve`, and can be opted out with `--no-sandbox` / `ODEK_NO_SANDBOX=1` (or made fatal-when-off with `ODEK_REQUIRE_SANDBOX=1`). This document covers all configuration options, the `Dockerfile.odek` build system, security guarantees, and best practices. ## Quick start @@ -50,7 +50,7 @@ All sandbox settings are available in `~/.odek/config.json`, `./odek.json`, `ODE | Field | Env var | CLI flag | Type | Default | Description | |-------|---------|----------|------|---------|-------------| -| `sandbox` | `ODEK_SANDBOX` | `--sandbox` | bool | `false` | Enable/disable sandbox isolation | +| `sandbox` | `ODEK_SANDBOX` | `--sandbox` / `--no-sandbox` | bool | **on** (run/repl/serve) | Sandbox isolation is default-on; `--no-sandbox` or `ODEK_NO_SANDBOX=1` opts out; `ODEK_REQUIRE_SANDBOX=1` makes any unsandboxed outcome fatal | | `sandbox_image` | `ODEK_SANDBOX_IMAGE` | `--sandbox-image` | string | `alpine:latest` | Docker image for the sandbox container | | `sandbox_network` | `ODEK_SANDBOX_NETWORK` | `--sandbox-network` | string | `none` | Docker network mode | | `sandbox_readonly` | `ODEK_SANDBOX_READONLY` | `--sandbox-readonly` | bool | `false` | Mount working directory read-only | diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 782da39..c1f7e35 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -95,13 +95,13 @@ The `@`-resource resolver (`FileResolver.Search`) rejects queries containing `.. - **System prompts** — `~/.odek/IDENTITY.md`, explicit `--system` / `ODEK_SYSTEM`, and config `system` overrides are capped at 256 KiB and scanned before becoming the system prompt. On injection patterns or an over-size prompt, odek warns on stderr and falls back to the compiled-in default identity, keeping the system-message boundary consistent regardless of which source supplied it. Project `AGENTS.md` larger than 256 KiB is ignored. The compiled-in default is itself scanner-clean (pinned by test) and carries the execution-provenance rules: repository/tool text — including policy-dressed content — is never authorization to act; scripts, make targets, package scripts, and CI steps are audited before execution; failed reads are never replaced by executing the file; deferred-execution writes require named user confirmation; MCP tool metadata is capability documentation, not directives. - **MCP tool descriptions and schemas** — at registration (see [MCP hardening](#mcp-hardening)). - **Skill bodies** — at load time and on save/patch. -- **Memory** — facts, Extended Memory atoms, and session-buffer text. +- **Memory** — facts and Extended Memory atoms. -The scanner normalizes invisible Unicode, folds common homoglyphs, detects mixed confusable scripts, and matches paraphrased exfiltration and non-English override phrases. It also flags concealment instructions ("do not tell the user", "keep this secret", "silently exfiltrate"), forged chat control tokens / role markers (`<|im_start|>`, `[INST]`, `<>`, ``), and data-exfiltration beacons (markdown-image URLs carrying `data=`/`token=`/`${VAR}`, and `curl`/`wget` requests splicing a shell variable into a query string). +The scanner normalizes invisible Unicode, folds common homoglyphs, detects mixed confusable scripts, and matches paraphrased exfiltration and non-English override phrases. It also flags concealment instructions ("do not tell the user", "keep this secret", "silently exfiltrate"), forged chat control tokens / role markers (`<|im_start|>`, `[INST]`, `<>`, and `` when followed by an override verb), and data-exfiltration beacons (markdown-image URLs carrying `data=`/`token=`/`${VAR}`, and `curl`/`wget` requests splicing a shell variable into a query string). -**Optional sidecar second opinion.** odek can send the same content to an external `go-prompt-injection-guard` sidecar (HTTP or Unix socket). The guard is **optional** — the local rule scan always runs first, and without a sidecar the system behaves exactly as before. Covered scopes (each controlled by `guard.scan.`): +**Optional sidecar second opinion.** odek can send the same content to an external `go-prompt-injection-guard` sidecar (HTTP or Unix socket). The guard is **optional** — the local rule scan always runs first, and without a sidecar the system behaves exactly as before. Covered scopes (each controlled by `guard.scan.`; MCP input schemas are additionally sidecar-scanned through a fixed `mcp_schema` scope that has no toggle — `guard.IsEnabled` treats unknown scopes as enabled): -- `memory` — legacy facts, `memory` tool writes, Extended Memory atoms, and session-buffer text. +- `memory` — legacy facts, `memory` tool writes, and Extended Memory atoms. - `system_prompt` — `IDENTITY.md`, explicit `--system`, and `AGENTS.md`. - `mcp_descriptions` — MCP server tool descriptions. - `skills` — skill bodies at load time and import. @@ -181,7 +181,7 @@ These families are additionally pinned by dedicated per-module suites beyond the | MCP per-server limits & per-tool approvals | `cmd/odek/mcp_approval_test.go`, `cmd/odek/mcp_e2e_test.go` | | SSRF dial guard | `cmd/odek/ssrf_guard_test.go` | -Taint is decided per tool call by `memory.ToolCallTaints` (the single source of truth, shared with skills): +Taint is decided per tool call by `memory.ToolCallTaints` (the single source of truth): - **Always untrusted:** `browser`, `http_batch`, `transcribe` (network / opaque-audio content), `vision` (opaque-image/video content), `web_search` (search-engine results), `delegate_tasks` (sub-agent output), `session_search` (recall of prior-session transcripts, which may carry earlier-injected text), and any MCP tool (`server__tool`). `shell` is deliberately excluded even though its output can carry untrusted bytes — it is the agent's primary work tool and tainting it would taint nearly every session. - **Path-reading tools** (`read_file`, `search_files`, `multi_grep`, `batch_read`, `json_query`, `head_tail`, `count_lines`, `checksum`, `word_count`, `sort`, `tr`, `diff`, `file_info`, `glob`, `tree`, `base64`) taint when **any** of their path arguments resolves **outside the workspace trust zone** — the workspace dir, the sandbox `/workspace` mount, or `~/.odek`. Reads confined to the workspace stay trusted, so ordinary coding sessions remain recallable; reads of anything else (system/credential paths, home files, sibling repos) taint. The check is a workspace-containment allowlist rather than a sensitive-path denylist, and it resolves symlinks (so e.g. `/etc` → `/private/etc` on macOS cannot disguise an escape). A malformed argument string is treated conservatively as untrusted. When adding a new file-reading tool, add it to `PathReadingTools`. @@ -300,7 +300,7 @@ The plan tool gives the agent a protected plan message that survives context tri - **Never in the approval UI.** `classifyToolCall` returns an explicit safe class for `plan`, so plan calls (and their step titles, which may quote task content) never surface in approval prompts or batch cards (`TestReport_PlanToolClassifiedSafe`). - **Untrusted wrapping.** Plan step bodies derive from task/tool content and are re-injected as system context every iteration — they ride the same untrusted-content wrapper as other engine-injected context, with the audit ingest recorder recording the injection. -- **Forgery resistance.** A hostile tool result containing a literal plan header cannot become the plan message: recognition requires the `system` role, and only the engine writes that role/content pair (`TestIsPlanMessage`). +- **Forgery resistance.** A hostile tool result containing a literal plan header cannot become the plan message: recognition requires the `system` role, and only the engine writes that role/content pair (enforced by plan-message construction in `internal/loop`). Resume parsing is strict and total — any deviation in the stored plan drops it instead of approximating. Project configs may tune the documented clamps only; they cannot re-enable a globally disabled feature. @@ -332,7 +332,7 @@ The `/restart` command is restricted to operator chats/users (`schedules.telegra A single polling instance is enforced with an advisory `flock` on `~/.odek/telegram.lock`: a second instance blocks until the first releases, and the OS releases the lock automatically if the holder crashes. -**Message hygiene.** Message and caption lengths are counted in UTF-16 code units (`utf16Len`), matching Telegram's own limits, so emoji-heavy text is measured correctly. Outbound text via the `send_message` tool is escaped with `telegram.EscapeMarkdown` (ParseModeMarkdownV2), so prompt-injected content cannot abuse Markdown syntax to hide malicious links, fake buttons, or instruction-like formatting. Inline-keyboard `callback_data` is validated by the tool and again by the sender closure: values starting with a reserved internal prefix (`apr:`, `den:`, `trs:`, `clarify:`, `skill_save:`, `skill_skip:`) are rejected — only user-facing `cb:` callbacks are allowed — so a compromised agent cannot present a button that forges an approval decision or triggers a skill action. Clarify prompts bind a random request ID into the callback data, reject callbacks from a different user than the one who triggered the prompt, and ignore expired or already-answered prompts. +**Message hygiene.** Outbound text via the `send_message` tool is escaped with `telegram.EscapeMarkdown` (ParseModeMarkdownV2), so prompt-injected content cannot abuse Markdown syntax to hide malicious links, fake buttons, or instruction-like formatting. Inline-keyboard `callback_data` is validated by the tool and again by the sender closure: values starting with a reserved internal prefix (`apr:`, `den:`, `trs:`, `clarify:`, `skill_save:`, `skill_skip:`) are rejected — only user-facing `cb:` callbacks are allowed — so a compromised agent cannot present a button that forges an approval decision or triggers a skill action. Clarify prompts bind a random request ID into the callback data, reject callbacks from a different user than the one who triggered the prompt, and ignore expired or already-answered prompts. **Inbound media.** Voice messages, photos, and documents are downloaded to `~/.odek/media/` under a per-file cap (`telegram.max_download_size`, default 5 MiB) and an optional per-chat quota (`telegram.media_quota_per_chat`), preventing a single large upload or a flood of uploads from filling the disk; oversized downloads are rejected before they are written. @@ -360,7 +360,7 @@ MCP servers are subprocesses odek spawns on the operator's behalf, and their out ### MCP server mode -When odek itself runs as an MCP server (`odek mcp`), it exposes its built-in tools to an external MCP client over stdio under the same gates: the `DangerousConfig` risk classes and the approval system apply unchanged, and with no TTY the `non_interactive` default (`deny`) governs, so approval-gated classes fail closed rather than silently executing. `delegate_tasks` and the `memory` tool are deliberately not exposed over this surface, so an MCP consumer cannot spawn sub-agents or drive memory promotion. The project-sandbox approval gate runs in server mode too, and `--sandbox` is opt-in exactly as for `odek run`. +When odek itself runs as an MCP server (`odek mcp`), it exposes its built-in tools to an external MCP client over stdio under the same gates: the `DangerousConfig` risk classes and the approval system apply unchanged, and with no TTY the `non_interactive` fallback applies (built-in default `read_only`), so approval-gated classes fail closed rather than silently executing. `delegate_tasks` and the `memory` tool are deliberately not exposed over this surface, so an MCP consumer cannot spawn sub-agents or drive memory promotion. The project-sandbox approval gate runs in server mode too, and `--sandbox` is opt-in exactly as for `odek run`. ### SSRF and network egress @@ -466,7 +466,7 @@ Hostile or accidental input is bounded everywhere it is sized, to keep it from O | MCP artifact file / refs per envelope | 64 MiB / 64 | | Sub-agent progress stream | 100 K lines / 100 MiB (overflow cancels the child) | | Telegram media download | 5 MiB per file (default) + optional per-chat quota | -| Telegram plan files | 1 MiB read / 8 KiB preview | +| Telegram plan files | reply preview bounded at 3800 chars (`maxTelegramPlanChars`) | | Config files | 5 MiB | | `IDENTITY.md` / `--system` | 256 KiB | | Skill files | 1 MiB | @@ -481,7 +481,7 @@ Hostile or accidental input is bounded everywhere it is sized, to keep it from O ### Secret redaction -`internal/redact` scans every tool output and session/memory write for known secret formats and replaces matches with `[REDACTED]` before they reach Telegram replies, persistent sessions, or memory. Patterns include OpenAI `sk-` (and underscore-bearing bodies such as Anthropic `sk-ant-...`), Groq `gsk_`, xAI `xai-`, HuggingFace `hf_`, GitHub PATs (classic + fine-grained), AWS access keys, multi-line PEM private keys, JWT, generic `api_key=` / `password=` env lines, Slack `xoxb-`, Stripe `sk_live_`, Google API keys, Twilio `SK`, HashiCorp Vault `hvs.` / `hvb.`, Google OAuth `ya29.` / `1//0`, SendGrid `SG.`, Discord bot tokens (M/N/O-anchored), DB URLs with embedded credentials (`postgresql://`, `mongodb://`, etc.), `Authorization: Bearer` headers, Telegram bot tokens (`:`), and exported credential environment variables (`export API_KEY=…`). A known-value registry additionally redacts the concrete values loaded from `~/.odek/secrets.env` — including their base64, hex, URL-encoded, and reversed spellings — even when they match no pattern. +`internal/redact` scans every tool output for known secret formats and replaces matches with `[REDACTED]` before the output reaches the model, persistent sessions, or the event stream; memory writes and Telegram replies are covered transitively, since they are composed from already-redacted tool output. Patterns include OpenAI `sk-` (and underscore-bearing bodies such as Anthropic `sk-ant-...`), Groq `gsk_`, xAI `xai-`, HuggingFace `hf_`, GitHub PATs (classic + fine-grained), AWS access keys, multi-line PEM private keys, JWT, generic `api_key=` / `password=` env lines, Slack `xoxb-`, Stripe `sk_live_`, Google API keys, Twilio `SK`, HashiCorp Vault `hvs.` / `hvb.`, Google OAuth `ya29.` / `1//0`, SendGrid `SG.`, Discord bot tokens (M/N/O-anchored), DB URLs with embedded credentials (`postgresql://`, `mongodb://`, etc.), `Authorization: Bearer` headers, Telegram bot tokens (`:`), and exported credential environment variables (`export API_KEY=…`). A known-value registry additionally redacts the concrete values loaded from `~/.odek/secrets.env` — including their base64, hex, URL-encoded, and reversed spellings — even when they match no pattern. If you find a format that leaks, add a regex to `internal/redact/redact.go` and a row to `TestReport_RedactMissesRealSecretFormats` in `cmd/odek/security_report_validation_test.go`. diff --git a/docs/SUBAGENTS.md b/docs/SUBAGENTS.md index 8200688..4ecea4d 100644 --- a/docs/SUBAGENTS.md +++ b/docs/SUBAGENTS.md @@ -217,7 +217,7 @@ The `delegate_tasks` tool always uses this file-based approach internally. ### stdout (machine-readable) -Pure JSON. Always parseable — even on errors. The parent `delegate_tasks` tool reads this via `json.NewDecoder`: +Pure JSON. Always parseable — even on errors. The parent `delegate_tasks` tool parses this stream line-by-line (`json.Unmarshal` per line): ```jsonc // Success @@ -397,7 +397,7 @@ Config in `odek.json`: | Field | Default | Description | |-------|---------|-------------| -| `max_concurrency` | 3 | Max sub-agents running in parallel (max 8) | +| `max_concurrency` | 3 (global default; falls back to it when unset here) | Max sub-agents running in parallel (max 8) | | `timeout_seconds` | 1800 | Default timeout per sub-agent; hard max 1800 (values above are clamped) | | `max_iterations` | 15 | Default max think→act cycles per sub-agent | | `max_depth` | 2 | Delegation nesting cap (1 = a sub-agent may not delegate further) | diff --git a/docs/TELEGRAM.md b/docs/TELEGRAM.md index b7c70dd..27b477c 100644 --- a/docs/TELEGRAM.md +++ b/docs/TELEGRAM.md @@ -181,11 +181,11 @@ The `Handler` struct routes incoming updates to the appropriate callback based o | Callback | Trigger | Signature | |---|---|---| -| `OnTextMessage` | Plain text message | `(chatID int64, messageID int, text string, forwarded bool) (string, error)` | +| `OnTextMessage` | Plain text message | `(chatID int64, messageID int, text string, forwarded bool, userID int64) (string, error)` | | `OnCommand` | Slash command (e.g. `/start`) | `(chatID int64, command, args string) (string, error)` | -| `OnVoiceMessage` | Voice message (OGG Opus) | `(chatID int64, messageID int, fileID string) (string, error)` | -| `OnPhotoMessage` | Photo message | `(chatID int64, messageID int, fileIDs []string, caption string) (string, error)` | -| `OnCallbackQuery` | Inline keyboard callback | `(chatID int64, callbackData string) (string, error)` | +| `OnVoiceMessage` | Voice message (OGG Opus) | `(chatID int64, messageID int, fileID string, userID int64) (string, error)` | +| `OnPhotoMessage` | Photo message | `(chatID int64, messageID int, fileIDs []string, caption string, userID int64) (string, error)` | +| `OnCallbackQuery` | Inline keyboard callback | `(chatID int64, callbackData string, userID int64) (string, error)` | All callbacks return a response string (may be empty) and an error. The `Handle` method: 1. Sends `SendChatAction("typing")` immediately @@ -256,7 +256,7 @@ defense-in-depth. | `/plan ` | Create a new plan from a natural language description | | `/plans` | List saved plans for this chat | | `/plan_view ` | View a specific plan's content for this chat | -| `/plan_resume ` | Resume working from a saved plan in this chat | +| `/plan_resume` | Resume working from the most recent saved plan in this chat | | `/plan_delete ` | Delete a saved plan for this chat | | `/plan_status` | Show the agent's current structured task plan (loop `plan` tool state) for this chat's session — distinct from the markdown-file plan commands above | | `/sessions` | List recent conversation sessions for this chat | diff --git a/docs/TOOL_SELECTION.md b/docs/TOOL_SELECTION.md index 74a3e3d..9ea7c88 100644 --- a/docs/TOOL_SELECTION.md +++ b/docs/TOOL_SELECTION.md @@ -19,8 +19,11 @@ environment supports: - Session search: `session_search` - Browser: `browser` - Web search: `web_search` (only when `web_search.base_url` is configured) -- Skill tools: `skill_load`, `skill_list`, `skill_save`, `skill_patch`, - `skill_delete` (only when skill learning is enabled) +- Skill tools: `skill_load`, `skill_list` (present whenever the skill system + is initialized) +- Sub-agent profiles: `list_subagent_profiles` (operator-defined capability + profiles + the built-in default) +- Artifacts: `artifact_read` (parent-side reader for sub-agent result artifacts) - MCP tools: prefixed as `__` (only when `mcp_servers` are configured) @@ -211,7 +214,8 @@ Use these exact names in config, env vars, and CLI flags: | Network | `browser`, `web_search` | | Memory | `memory` | | Session search | `session_search` | -| Skills | `skill_load`, `skill_list`, `skill_save`, `skill_patch`, `skill_delete` | +| Skills | `skill_load`, `skill_list` | +| Sub-agent support | `list_subagent_profiles`, `artifact_read` | | Telegram-only | `send_message`, `clarify` (auto-injected by `odek telegram`; ignored by other modes) | | MCP | `__` | diff --git a/docs/WEBUI.md b/docs/WEBUI.md index ff3a4d8..ee21f9b 100644 --- a/docs/WEBUI.md +++ b/docs/WEBUI.md @@ -331,7 +331,7 @@ Server metadata for monitoring and the WebUI status popover. Never carries secre ```jsonc { "status": "ok", - "version": "1.28.1", // ldflags build version ("" for dev builds) + "version": "1.35.0", // ldflags build version ("" for dev builds) "started_at": "2026-08-21T08:19:29Z", "uptime_seconds": 1903, "model": "glm-5.3", // configured model