engined

---

One local engine broker for the whole workstation.

engined is a systemd --user daemon that owns every inference engine on a box and serves them behind a single OpenAI-shaped door on loopback. Consumers send a model string; engined picks the engine, starts its container on demand, keeps one model resident per role, and stops it when idle.

Requirements

• git, Bun 1.4, Docker, and rsync.
• The shipped engine images are optimised for AMD Strix Halo (Ryzen AI Max, gfx1151): ROCm 7.2 for the PyTorch engines, RADV Vulkan for llama.cpp and whisper.cpp. Other hardware works by pointing an engine at your own image: one spec_dir = "..." line on its [[engine]] in config.toml, naming a spec whose image suits your GPU or CPU. See docs/configuration.md and docs/engines.md.

Quick start

git clone https://github.com/Rethunk-Tech/engined.git
cd engined
bash scripts/install.sh
mkdir -p ~/.config/engined
cp config.example.toml ~/.config/engined/config.toml
$EDITOR ~/.config/engined/config.toml
systemctl --user restart engined.service

Edit the config before the first start: a route naming a model file that is not on disk refuses to parse, and the example names models you may not have. Engine images are built locally: an engine whose image is missing reports the docker build line to run. Then curl -s localhost:29200/engined/v1/engines. Runbook: HUMANS.md.

Highlights

• One door for five modalities — chat, embeddings, speech, transcription, images.
• Start on demand, stop when idle — leases counted; idle 35B returns ~27 GiB.
• One resident model per role — occupancy is engined's job.
• Chains with provenance — fallback lists record which engine answered.
• Agentic CLIs as an engine kind — claude, opencode, and cursor, writing disabled: claude and cursor by their own flags, opencode by a bwrap sandbox, because it has no flag that would do it.
• Fill-in-the-middle — POST /openai/v1/completions over llama's own /infill, for a route that opts in.
• Vision bridge — a text-only chat route can take an image attachment by captioning it through a vision route first.
• Per-day usage totals — GET /engined/v1/usage sums provenance without keeping request content.
• Failures name their own fix — literal docker build or secret-tool store.

Documentation

Document	Contents
HUMANS.md	Install, configure, verify, operate, uninstall
AGENTS.md	Test tiers, config/spec split, invariants
docs/http-api.md	Routes, model spellings, chains
docs/configuration.md	Schema, key set, args precedence
docs/engines.md	Per-engine images and occupancy
docs/tuning.md	Speculative-decoding measurements
docs/security-model.md	Origin check, agentic boundary
docs/design.md	Ownership line, why 29200
CHANGELOG.md	Release notes
SECURITY.md	Reporting a vulnerability
THIRD_PARTY_NOTICES.md	Upstream projects, models, and their licences

License

MIT, see LICENSE. Third-party projects engined builds on, fetches, or borrows from are credited in THIRD_PARTY_NOTICES.md.