Turn an ordinary Linux machine into a private AI workstation that runs entirely on your own hardware.
No subscription, no API key, no data leaving the building. You run the models, on your computer, for the cost of the electricity.
Companion files for the PatterOS · Local AI Budget Build video series, from PatterTech.
1 token per second is infinitely more than 0. Everything after that is just profit. If your hardware is older or smaller than you would like, start anyway.
flowchart LR
A["Your browser<br/>or phone"] --> B["Odysseus<br/>workspace<br/>:7000"]
B --> C["llama.cpp<br/>engine<br/>:8020"]
C --> D[("~/models<br/>your models<br/>on your disk")]
E["Other apps<br/>Cursor, scripts"] -.optional.-> C
Two pieces, both started automatically when your computer boots, and both private to your machine unless you deliberately choose otherwise:
- A model server that runs the AI, and speaks the same API most AI software already understands.
- A web workspace for chatting, writing, notes, documents and agents.
Probably. This is the short version:
| What you have | Will it work? | What to expect |
|---|---|---|
| NVIDIA card | Yes, tested by us | The fastest option. CUDA. |
| AMD card | Yes, tested by us | Uses Vulkan, no vendor toolkit needed |
| Intel Arc / Intel iGPU | Untested by us | The installer still takes the Vulkan path, same as AMD. If it fails, --cpu runs on the processor, and an issue report is welcome |
| Built-in graphics only | Yes | Smaller models, slower. Intel built-in graphics is untested; see the Intel row |
| No graphics card at all | Yes, tested by us | Runs on the processor. Slower, but it genuinely works |
You also need: Linux Mint 22 or Ubuntu 24.04, an internet connection, and about 20 GB of free disk space.
More memory on your graphics card means bigger models. That is the one spec that really decides things. A model either fits in memory or it does not.
| Graphics memory | Models that fit comfortably |
|---|---|
| None (processor only) | Gemma 4 E2B, 2.7 GB |
| 8 GB | E2B and E4B fully; 12B partly, and PatterOS sets that up for you |
| 16 GB | Up to Gemma 4 12B, 6.8 GB |
| 24 GB | Everything, including Gemma 4 31B, 17.3 GB |
Not built a machine yet? Part 1 below has a full parts list from about £484.
Three ways in. Pick whichever suits you.
git clone https://github.com/Daniel-Parke/PatterOS.git
cd PatterOS
# Read it first. Never run a script from the internet, ours included,
# without looking inside. It is heavily commented in plain English.
less scripts/install_local_ai.sh
sudo bash scripts/install_local_ai.shIt asks what you want, checks your hardware, and tells you what it is about to do before it does it. Everything is optional and everything can be undone.
Useful options
| Option | What it does |
|---|---|
--full |
Also download the bigger models. Needs a 24 GB card and the disk space |
--no-models |
Set everything up, download nothing. Add models later |
--no-odysseus |
Model server only, no web workspace |
--cpu |
Ignore the graphics card and use the processor |
--skip-drivers |
Do not touch graphics drivers at all |
-y |
Do not ask anything. Assumes you have read the script |
--help |
The full list, including environment settings |
Prefer not to run someone else's script on a new machine? Sensible instinct. The manual setup guide does exactly the same job one command at a time, with every step explained.
The Local AI Handbook covers the ideas behind all of it. What a model actually is, how the pieces fit together, and what local AI can realistically do. No background assumed.
| What | Where |
|---|---|
| Web workspace | http://localhost:7000 |
| Model server | http://localhost:8020/v1 |
| Your models | ~/models |
| First-time login | sudo cat /var/lib/patteros/odysseus-first-login.txt |
# What models do I have?
curl http://localhost:8020/v1/models
# Added a new one? Tell the server to look again.
curl 'http://localhost:8020/v1/models?reload=1'Reaching it from another computer or your phone is a deliberate extra step, and Steps 10 and 11 of the guide walk through doing it safely.
# See exactly what would be removed, without removing anything
sudo bash scripts/uninstall_local_ai.sh --dry-run
# Put things back
sudo bash scripts/uninstall_local_ai.shIt keeps your downloaded models by default, because they are slow to fetch
again. --all removes everything.
| Part | What it covers | Files here |
|---|---|---|
| 1. Building the rig | Choosing parts, the full bill of materials, and putting it together. From about £484 | Parts list and build guide (Word) |
| 2. Setting it up | Fresh Linux install to working AI workstation | Installer · uninstaller · manual guide |
| 3. Comparing models | The same hard research task across nine model and quantisation combinations, unedited | Results and method |
All of it is optional, and none of it is ours. These are separate open-source projects with their own licences:
| Project | Licence | What it does here |
|---|---|---|
| llama.cpp | MIT | Runs the models. Built from source, pinned to a tested version |
| Odysseus | AGPL-3.0 | The web workspace. Installed and run unmodified, pinned to a commit |
| LACT | MIT | Optional. Graphics card power and fan tuning |
| Gemma 4 | Apache 2.0 | The models, downloaded from Hugging Face |
A word about Odysseus. It includes an AI agent that can run commands and read files. PatterOS keeps it on your machine only. Do not expose it to the internet.
Bug reports, corrections and code are all welcome, and no question is too basic. See CONTRIBUTING.md to get started, or SECURITY.md if you have found something that should be reported privately.
What changed and why: CHANGELOG.md.
Code and documentation here are under the Apache License 2.0. See also NOTICE.
The names Patter, PatterOS, PatterStage and PatterTech, and the
visual identity in branding/, are not covered by that licence.
See TRADEMARK.md. Forking? REBRANDING.md is a
short checklist.
Your hardware. Your data. Your control.