Run hindsight on Docker Desktop (Windows) after the WSL2 migration

.env.example

@@ -81,6 +81,25 @@ KALI_GPU_RENDER_NODE=/dev/dri/renderD128
 KALI_GPU_RENDER_GID=990
 KALI_GPU_VIDEO_GID=44
 
+# ==============================================================================
+# HINDSIGHT (optional — agent memory server, claude-code LLM provider)
+# ==============================================================================
+# Host ports for the API and the Control Plane UI (loopback-bound by default).
+HINDSIGHT_API_PORT=8888
+HINDSIGHT_UI_PORT=9999
+
+# UID/GID that must match your host user (not the fleet's usual PUID/PGID):
+# the container mounts your live ~/.claude credentials and needs matching
+# ownership to read them, and to write the embedded pg0 data directory if it
+# isn't a named volume. Check with: id -u / id -g
+HOST_UID=1000
+HOST_GID=1000
+
+# Claude CLI version installed at ~/.local/share/claude/versions/ on the host,
+# mounted in to override the older CLI bundled in the image. Check installed
+# versions with: ls ~/.local/share/claude/versions/
+CLAUDE_CLI_VERSION=2.1.277
+
 # ==============================================================================
 # LOCAL STORAGE VOLUMES
 # ==============================================================================
@@ -93,3 +112,9 @@ MEDIA_MOVIES=./media/Movies
 
 # Host target path for qBittorrent downloads
 DOWNLOADS_DIR=./downloads
+
+# Directory holding a LINUX claude CLI at versions/<CLAUDE_CLI_VERSION>, mounted
+# into the container. On Windows the host CLI is a .exe and cannot be reused:
+# copy the ELF from a Linux/WSL install, e.g.
+#   wsl cp ~/.local/share/claude/versions/2.1.277 <this dir>/versions/
+CLAUDE_CLI_DIR=./claude-cli

.gitignore

@@ -15,3 +15,7 @@ kali-desktop/opt/
 .DS_Store
 Thumbs.db
 *.log
+
+# Linux claude CLI copied for the hindsight container (large binary)
+hindsight/claude-cli/
+hindsight/claude.json

AGENTS.md

@@ -1,6 +1,6 @@
-# AI Coding Agent Playbook: Resilient Home-Lab Compose Stack 🤖
+# Home-Lab Compose Stack Playbook: Modular Architecture & Hardening Standards 🤖
 
-Welcome, AI Agent! This document provides the architectural blueprint, structural patterns, and strict resiliency rules governing this repository. Read this playbook thoroughly before modifying or adding services to maintain our production-grade home-lab environment.
+Welcome! This document provides the architectural blueprint, structural patterns, and strict resiliency rules governing this repository. Read this playbook thoroughly before modifying or adding services to maintain our production-grade home-lab environment.
 
 ---
 
@@ -9,7 +9,7 @@ Welcome, AI Agent! This document provides the architectural blueprint, structura
 This stack is organized as a **highly modular, decentralized multi-container deployment** utilizing the modern Docker Compose `include` directive. Instead of a single monolithic compose file, each service is fully self-contained within its own directory.
 
 ```
-/home/jond/.docker/
+<repo root>/                # C:/AIonUI/Work/docker on the current machine
 ├── docker-compose.yml       # Root orchestrator (includes active services)
 ├── .env.example             # Global environment variables blueprint
 ├── .gitignore               # Excludes sensitive data (.env, .env.local, config folders)
@@ -34,7 +34,7 @@ This stack is organized as a **highly modular, decentralized multi-container dep
     └── docker-compose.yml   # Headless Chromium automation engine
 ```
 
-The root `/home/jond/.docker/docker-compose.yml` serves as the entry point, selectively importing active service configurations:
+The root `docker-compose.yml` serves as the entry point, selectively importing active service configurations:
 ```yaml
 include:
   # - path: ./changedetection/docker-compose.yml
@@ -169,37 +169,65 @@ Create a dedicated subdirectory for the service (e.g., `./my-service`).
 
 ### Step 2: Write the `docker-compose.yml`
 Create `./my-service/docker-compose.yml`. Make sure to:
-- Use standard images (prefer verified, official, or reputable publishers like LinuxServer).
+- Use standard images (prefer verified, official, or reputable publishers like LinuxServer, Docker Official, or trusted open-source maintainers).
 - Define ports using loopback `127.0.0.1` and customizable env variables (e.g., `${MY_SERVICE_PORT:-9000}`).
 - Set up the **4-layered env_file block** pointing to `../.env`, `../.env.local`, `.env`, and `.env.local`.
-- Apply **Resource limits**, **Log rotation**, **Healthchecks**, and **init: true** (if running subprocesses).
+- Apply **Resource limits**, **Log rotation**, **Healthchecks**, **Capability hardening**, and **init: true** (if running subprocesses).
 - Use local folder mappings for config storage (e.g., `./config:/config`).
+- Never hardcode secrets; all sensitive values must come from environment variables.
 
 ### Step 3: Update Environments
-- Add the default configuration key value pairs to `/home/jond/.docker/.env.example`.
-- If the user has a local `.env`, append the new service variables to it with helpful comments.
+- Add default configuration key-value pairs to the root `.env.example` with clear comments.
+- Document all new environment variables with their purpose, defaults, and any required values.
+- Keep all host-specific paths (storage, credentials) parameterized in `.env` — never hardcode them.
 
 ### Step 4: Register in Orchestrator
-Add the service directory relative path to the root `/home/jond/.docker/docker-compose.yml` under `include:`.
+Add the service directory relative path to the root `docker-compose.yml` under `include:`. Comment it out if it should be optional.
 
 ---
 
 ## 🚦 Verification Command Playbook
 
-Before marking any task as complete, run these commands to verify syntax, config values, and deployment validity:
+Before marking any service deployment as complete, run these commands to verify syntax, configuration, and runtime health:
 
 ```bash
 # 1. Verify compose syntax and structural formatting
 docker compose config
 
-# 2. Start the services (dry-run/daemon mode)
+# 2. Start the services in daemon mode
 docker compose up -d
 
-# 3. Check health and lifecycle status
+# 3. Check health and lifecycle status of all containers
 docker compose ps
 
-# 4. View logs of specific containers to ensure error-free startups
+# 4. View logs to ensure clean startups (follow mode for real-time)
 docker compose logs -f <service-name>
+
+# 5. Inspect resource usage and container stats
+docker stats
+
+# 6. Verify healthcheck status
+docker ps --format "table {{.Names}}\t{{.Status}}"
 ```
 
-Happy hacking, AI Agent! Keep the stack secure, resilient, and blazing fast. 🚀
+---
+
+## ⚠️ Common Pitfalls
+
+**Hardening mistakes:**
+- `cap_drop: ALL` without `cap_add` for root→user privilege-drop services breaks entrypoints.
+- Forgetting `init: true` on containers with subprocess spawning (Node, browser engines, scrapers) leads to zombie processes.
+- Omitting `security_opt: [no-new-privileges:true]` allows containers to re-escalate if a process is exploited.
+
+**Resource & stability issues:**
+- Missing memory limits cause OOM kills and crash-loops; always set `deploy.resources.limits.memory`.
+- Unbounded log files fill the host disk; always use `logging.options.max-size` and `max-file`.
+- No healthchecks hide crashing services; `docker compose ps` alone won't catch them without `-a`.
+- Unbuffered volumes (bind mounts to non-existent host paths) are created as root, causing permission errors in rootless containers.
+
+**Environment & secrets:**
+- Hardcoding secrets in compose files exposes them in version control; use env variables + `.gitignore`.
+- Not documenting env vars in `.env.example` makes onboarding impossible.
+- Mixing host-specific paths in compose creates non-portable configs; parameterize them.
+
+Keep the stack secure, resilient, and maintainable. 🚀

docker-compose.yml

@@ -4,6 +4,8 @@ include:
   #  - path: ./browser/docker-compose.yml
   #  - path: ./metamcp/docker-compose.yml
   #  - path: ./kali-desktop/docker-compose.yml
-  - path: ./searxng/docker-compose.yml
-  - path: ./jellyfin/docker-compose.yml
-  - path: ./qbittorrent/docker-compose.yml
+  #  - path: ./hindsight/docker-compose.yml
+#  - path: ./searxng/docker-compose.yml
+#  - path: ./jellyfin/docker-compose.yml
+#  - path: ./qbittorrent/docker-compose.yml
+  - path: ./hindsight/docker-compose.yml

hindsight/MODELS.md

@@ -0,0 +1,41 @@
+## Divid to Conquere
+### reflect and consolidation require the most reasoning.
+They must resolve contradictions, deduplicate overlapping facts, and synthesize complex answers while following your bank's mission and directives.
+- Use DeepSeek V4 Pro for these.
+
+### retain requires minimal reasoning.
+It simply parses raw text into structured facts.
+- Use DeepSeek V4 Flash for this to keep ingestion fast and cheap.
+
+Set DeepSeek V4 Flash as your global default. Then override the model specifically for the heavier operations using these environment variables.
+
+### DeepSeek Version
+```yaml
+HINDSIGHT_API_LLM_PROVIDER: deepseek
+HINDSIGHT_API_LLM_MODEL: deepseek-v4-flash
+HINDSIGHT_API_REFLECT_LLM_MODEL: deepseek-v4-pro
+HINDSIGHT_API_CONSOLIDATION_LLM_MODEL: deepseek-v4-pro
+```
+
+### Claude Code Version
+```yaml
+HINDSIGHT_API_LLM_PROVIDER: claude-code
+HINDSIGHT_API_LLM_MODEL: claude-haiku-4-5
+HINDSIGHT_API_RETAIN_LLM_MODEL: claude-haiku-4-5
+HINDSIGHT_API_REFLECT_LLM_MODEL: claude-sonnet-5
+HINDSIGHT_API_CONSOLIDATION_LLM_MODEL: claude-sonnet-5
+```
+
+### Mix Version
+```yaml
+# Global & Retain (Uses your Claude subscription)
+HINDSIGHT_API_LLM_PROVIDER: claude-code
+HINDSIGHT_API_LLM_MODEL: claude-haiku-4-5
+HINDSIGHT_API_RETAIN_LLM_PROVIDER: claude-code
+HINDSIGHT_API_RETAIN_LLM_MODEL: claude-haiku-4-5
+
+# Reflect & Mental Model Refresh (Routes to a cheaper API)
+HINDSIGHT_API_REFLECT_LLM_PROVIDER: deepseek
+HINDSIGHT_API_REFLECT_LLM_API_KEY: sk-your-deepseek-api-key
+HINDSIGHT_API_REFLECT_LLM_MODEL: deepseek-v4-flash
+```

hindsight/README.md

@@ -0,0 +1,113 @@
+## Hindsight with Claude Code (Claude Pro/Max subscription)
+
+Run Hindsight inside Docker using the `claude-code` LLM provider, backed by
+your host machine's Claude Pro or Max subscription credentials.
+
+The standalone Hindsight Docker image ships `claude-agent-sdk` but does **not**
+bundle the host `claude` CLI binary or any Claude credentials. This Compose
+file bind-mounts the host's CLI install and credentials into the container so
+the `claude-code` provider works without an API key.
+
+## When to use this
+
+- You have an active Claude Pro or Max subscription and want to use it for
+Hindsight without paying separate Anthropic API costs.
+- You want a one-command `docker compose up` instead of a long `docker run`
+invocation with many flags.
+- You are running on **Linux/amd64** — macOS Docker Desktop and Windows host
+paths differ and are not yet covered (please open an issue if you'd like to
+contribute a verified recipe for either).
+
+> **Personal-use only.**  Anthropic's
+Agent SDK documentation
+states that third-party developers should not offer claude.ai login or rate
+limits for their products. Hindsight does **not** perform any login on your
+behalf — it uses credentials you've already authenticated via
+`claude auth login`. In January 2026, Anthropic
+enforced restrictions
+against tools that spoofed the Claude Code client identity; Hindsight uses
+the official Claude Agent SDK instead.
+> 
+> 
+> Do not deploy this configuration to shared environments or production. For
+> that, use the `anthropic` provider with an API key from the
+> Anthropic Console. Usage counts against
+> your Claude Pro/Max subscription limits.
+> 
+
+## Prerequisites
+
+- Host has `claude` CLI installed (e.g., `npm install -g @anthropics/claude-code`)
+and `claude auth login` has been run successfully.
+- `~/.claude.json` and `~/.claude/.credentials.json` exist on the host.
+- Host `claude` CLI version is **2.1.128 or newer** — the version bundled with
+`claude-agent-sdk` 0.5.x has a protocol incompatibility in containers, so
+the recipe overrides it with the host binary.
+
+## Quick start
+
+```
+# Set your host UID/GID (defaults to 1000:1000 if unset)
+export HOST_UID=$(id -u)
+export HOST_GID=$(id -g)
+docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
+```
+
+- API: http://localhost:8888
+- Control Plane: http://localhost:9999
+
+## Post-setup (one-time)
+
+After the container starts for the first time, run these commands to fix
+permissions and symlink the host `claude` binary into `$PATH`:
+
+```
+# Make ~/.claude writable by your UID (the CLI writes session/project state)
+docker exec --user 0:0 hindsight-claude-code chown $(id -u):$(id -g) /home/hindsight/.claude
+docker exec --user 0:0 hindsight-claude-code chmod 755 /home/hindsight/.claude
+# Symlink the host claude binary into PATH
+docker exec --user 0:0 hindsight-claude-code \
+  ln -sf /home/hindsight/.local/share/claude/versions/2.1.128 /usr/local/bin/claude
+```
+
+If you set `CLAUDE_CLI_VERSION` to a version other than `2.1.128`, update the
+symlink path accordingly.
+
+## Notes on the bind-mount surface (every flag is load-bearing)
+
+- **Host `claude` binary required** — the image ships only `claude-agent-sdk`,
+not the CLI itself.
+- **SDK bundled-binary override** — the override of
+`claude_agent_sdk/_bundled/claude` works around a protocol issue in the
+bundled v2.1.121 binary inside containers. Once `claude-agent-sdk` ships
+with v2.1.128+ this override can be dropped. Set `CLAUDE_CLI_VERSION` to
+match your installed version.
+- **Single-file credential mounts** — credentials are mounted as individual
+`:ro` files rather than a whole-directory `:ro` mount of `~/.claude`,
+because the CLI writes session/project state at runtime and a read-only
+directory mount silently breaks it.
+- **`-user` / `user:`** — the `user: ${HOST_UID}:${HOST_GID}` pattern
+requires `chmod 755 /home/hindsight`, which is built into the image since
+v0.6.0 (see #1481).
+- **`~/.hindsight-docker` data directory** — the pg0 data bind mount must be
+writable by your host UID (see
+#1483).
+- **Verified** on `linux/amd64` against `ghcr.io/vectorize-io/hindsight:latest`
+v0.5.6+.
+
+## Using a different Claude CLI version
+
+If your host has a `claude` version other than 2.1.128, set
+`CLAUDE_CLI_VERSION` before starting:
+
+```
+export CLAUDE_CLI_VERSION=2.2.0
+docker compose -f docker/docker-compose/claude-code/docker-compose.yaml up -d
+```
+
+Then update the post-setup symlink to match:
+
+```
+docker exec --user 0:0 hindsight-claude-code \
+  ln -sf /home/hindsight/.local/share/claude/versions/2.2.0 /usr/local/bin/claude
+```

hindsight/TEMPLATES.md

@@ -0,0 +1,22 @@
+## Research Assistant — deploy that one, and only that one.
+Why it, over the alternatives. Its own reflect_mission reads:
+- "monitors sources, writes briefings and digests, and maintains a second brain."
+
+### That's your notes/ideas/blog bank verbatim.
+It ships three mental models
+1. Interests & Focus
+2. Knowledge Base
+3. Sources & Reliability
+- plus a "learn from what's ignored" directive that uses what you skip to filter future curation.
+- Dispositions are skepticism 4 / empathy 2: a thinking partner that pushes back, not a warm helper.
+
+## Personal Assistant
+is the near-miss — but it's routines, schedule, people, commitments. That's task management.
+
+## For vibecoding
+don't deploy a template at all. The Coding Agent template is for a bank you drive yourself. The coding-agents plugin does this automatically: one bank per repo (coding-agent::{gitProject}), and with manageBankConfig: true it seeds the missions, the five retain strategies and the knowledge label group itself. Hand-importing on top just duplicates that. 
+
+## Self-Hosted
+The coding side is a separate install: `npx @vectorize-io/hindsight-coding-agents install claude-code --server self-hosted --api-url http://localhost:8888`
+
+The `--server` flag matters. Default is Hindsight Cloud, a bare install prompts for it, and re-running never re-asks — so a wrong answer there quietly ships your prompts off-machine and sticks.