Skip to content

feat(docker): per-worktree Docker isolation with worktrunk hooks - #202

Draft
matrixise wants to merge 5 commits into
masterfrom
feat/docker-branch-isolation
Draft

matrixise wants to merge 5 commits into
masterfrom
feat/docker-branch-isolation

Conversation

@matrixise

@matrixise matrixise commented Feb 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Give every git worktree its own Docker Compose environment by relying on worktrunk (wt) lifecycle hooks instead of home-grown branch detection. Several branches can run task run / task tests side by side without sharing a database, an image or a port.

Nothing changes for contributors who do not use worktrunk: without a .env, every value falls back to the previous defaults (port 8000, Postgres on 5432, image python.ie/website-dev).

How it works

.config/wt.toml declares the hooks (each one commented):

Hook Effect
pre-start Writes a .env in the new worktree: COMPOSE_PROJECT_NAME=pythonie-<branch>, DOCKER_IMAGE=python.ie/website-dev:<branch>, WEB_PORT and PGPORT derived with hash_port
post-start uv sync --all-groups in the background (host .venv)
post-remove Destructive, scoped to the worktree's compose project: removes its containers (including running docker compose run one-offs), network, volumes (Postgres and Redis data) and its branch image
[list] url Shows each worktree's dev server URL in wt list
  • docker-compose.yml reads these variables with the old values as defaults and gains build: . so the image tag comes from one place.
  • Taskfile.yaml loads .env (so psql / task database:pull use the worktree's PGPORT), builds through docker compose build, and no longer sets DOCKER_IMAGE (it would override .env).
  • .env is gitignored.
  • Docs: new "Parallel Worktrees with worktrunk" section in DEVELOPMENT.md, linked from the README.

Removed compared to the first version of this PR

  • Git branch detection, env:write, branch:* tasks, GIT_BRANCH plumbing, DOCKER-BRANCHES.md, .env.example
  • Per-branch SQLite in settings/dev.py: the database file already lives inside each worktree
  • Redis 7 upgrade and healthchecks: unrelated to isolation, better in a separate PR

Kept: removal of the minio / mc services. Nothing in the dev or test settings uses them, and their fixed host ports (9000/9001) would collide between worktrees.

Verification

  • Two worktrees running in parallel: distinct compose projects, networks, Postgres volumes, images and host ports; the web server answers 200 on both after migrations
  • task tests run concurrently in both containers: OK
  • wt remove: volumes removed, but a running docker compose run container survived the first version of the hook. Fixed in the last commit and validated by running the rendered script; a full wt remove cycle with the fixed hook is still to be redone.
  • uv run ruff format --check pythonie and uv run ruff check pythonie: OK

Open questions

  • WAGTAILADMIN_BASE_URL is still hard-coded to localhost:8000 in settings/dev.py; should it follow WEB_PORT?
  • toast.yml still uses python.ie/website-dev (the latest tag), not the per-branch image.
  • The branch needs a rebase on the latest master (docs conflicts).

@matrixise
matrixise marked this pull request as draft February 4, 2026 11:16
@matrixise

Copy link
Copy Markdown
Contributor Author

I've converted this PR to draft status because I realize I've committed too many separate concerns together, which doesn't follow good PR practices.

I will split this work into separate PRs:

  1. First PR: Docker Compose infrastructure updates

    • Upgrade Redis from 6.2 to 7-alpine
    • Add healthchecks for PostgreSQL and Redis
    • Remove unused minio/mc services
    • Other docker-compose.yml improvements
  2. Second PR: Branch isolation feature

    • Automatic environment isolation by git branch
    • Branch-specific Docker images, databases, and volumes
    • New task branch:* commands
    • Documentation (DOCKER-BRANCHES.md, README updates)

This separation will make the changes easier to review and allow for incremental adoption.

I'll close this PR once the split PRs are ready.

matrixise and others added 5 commits September 25, 2026 07:58
Implement automatic environment isolation based on git branch names,
allowing developers to work on multiple branches simultaneously without
conflicts between databases, containers, or Docker volumes.

Key changes:
- Auto-detect git branch and normalize name (replace special chars)
- Tag Docker images by branch (python.ie/website:{branch})
- Isolate PostgreSQL databases per branch (pythonie_{branch})
- Create separate Docker volumes per branch
- Add configurable ports for parallel development (WEB_PORT, PG_PORT)
- Upgrade Redis from 6.2 to 7-alpine with persistence
- Add healthchecks for PostgreSQL and Redis
- Remove unused minio and mc services

New Taskfile commands:
- task branch:info - Show current branch environment
- task branch:volumes - List all Docker volumes by branch
- task branch:clean - Remove volumes for current branch
- task env:write - Generate .env file for docker-compose

Documentation:
- Add DOCKER-BRANCHES.md with usage examples and troubleshooting
- Add .env.example template for custom port overrides
- Update .gitignore to exclude generated .env file

This enables developers to switch branches or run multiple instances
in parallel without losing data or experiencing port conflicts.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Extend branch isolation to SQLite databases for local development
and add comprehensive documentation for the branch isolation feature.

Changes to SQLite isolation:
- Modify dev.py to use branch-specific SQLite files (db-{branch}.sqlite3)
- Add *.sqlite3 to .gitignore to exclude database files
- Remove debug print statement from dev.py
- Each branch now has isolated SQLite data without requiring worktrees

Documentation improvements:
- Add complete "Docker Branch Isolation" section to README.md
- Document all branch:* commands (info, verify, volumes, clean)
- Add examples for multi-branch development and branch switching
- Add troubleshooting section for database configuration issues
- Update DOCKER-BRANCHES.md to mention SQLite isolation
- Add note in local setup about GIT_BRANCH export

New Taskfile command:
- task branch:verify - Comprehensive verification of branch configuration
  Shows git branch, expected database, .env file, and PostgreSQL databases

This ensures complete data isolation between branches whether using
Docker (PostgreSQL) or local development (SQLite), preventing data
loss when switching branches.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace the home-grown git branch detection with worktrunk lifecycle
hooks declared in .config/wt.toml:

- pre-start writes a .env with COMPOSE_PROJECT_NAME, a per-branch
  DOCKER_IMAGE tag and hash_port-derived WEB_PORT / PGPORT
- post-start runs `uv sync --all-groups` for the host virtualenv
- post-remove tears down the worktree's compose project (containers,
  network, volumes) and its image

docker-compose.yml reads these variables with the previous values as
defaults, so nothing changes without worktrunk. The Taskfile loads .env
(for PGPORT on the host) and builds through compose so the image tag
comes from a single place.

Drop the branch:* tasks, env:write, GIT_BRANCH plumbing, per-branch
SQLite (the file already lives inside each worktree), the Redis 7 and
healthcheck changes (unrelated), and the unused minio/mc services whose
fixed host ports would collide between worktrees.
Add a "Parallel Worktrees with worktrunk" section to DEVELOPMENT.md, link it from the README quick start, and drop the removed minio service from the docs.
docker compose down skips running containers started with docker compose run (such as an open task run), which kept the network and the branch image in use after wt remove.
@matrixise
matrixise force-pushed the feat/docker-branch-isolation branch from ebee7af to ef5808c Compare October 19, 2026 05:53
@matrixise matrixise changed the title feat: Docker branch isolation for parallel development feat(docker): per-worktree Docker isolation with worktrunk hooks Oct 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant