Run a Local LLM on a Mini PC: Self-Host Your Own ChatGPT
You don’t need a $2,000 GPU to run a usable local AI. A ~$400 mini PC with an integrated GPU and enough RAM will run 7B–8B models fast enough for daily use — and every token stays on your hardware. Here’s the exact setup I run.
What you need
- A mini PC with a modern APU (I use a Ryzen 7 5700G) and at least 32GB RAM (I run 64GB so I can hold a model and still run other services).
- ~20GB of disk for a couple of models.
- Linux (this guide assumes Ubuntu/Debian; Proxmox VM or bare metal both work).
No discrete GPU. On an APU the model runs on CPU + shared memory, which is why RAM matters more than anything else here.
Step 1 — install Ollama
Ollama is the laziest way to pull and serve models. One line:
curl -fsSL https://ollama.com/install.sh | sh
Pull a model and talk to it:
ollama run llama3.1:8b
The first run downloads ~4.7GB. After that it’s instant to start.
Step 2 — pick a model that actually fits
RAM is the constraint. A rough rule for a CPU-only box:
| Model size | RAM needed | Feels like |
|---|---|---|
3B (e.g. llama3.2:3b) |
~4GB | Fast, fine for summaries/classification |
7–8B (llama3.1:8b, qwen2.5:7b) |
~8GB | The sweet spot — general chat, coding help |
| 14B | ~16GB | Noticeably smarter, slower on CPU |
On the 5700G I get ~10–14 tokens/sec on an 8B model — faster than you read. Above 14B on CPU it drops below reading speed and stops being fun.
Step 3 — add a real chat UI
The terminal is fine for testing, but you want a proper interface. Open WebUI gives you a ChatGPT-style UI, pointed at your local Ollama. Run it in Docker:
docker run -d -p 3000:8080 \
--add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
--name open-webui --restart always \
ghcr.io/open-webui/open-webui:main
Open http://<your-mini-pc-ip>:3000, create the first account (it becomes admin),
and it auto-detects your Ollama models. Done — a private ChatGPT on hardware you own.
Keep it on your LAN. There’s no reason to expose this to the public internet — if you want it on your phone away from home, reach it over a VPN like Tailscale, not a port-forward.
Is it actually usable?
For chat, drafting, summarizing, classification, and coding help: yes, an 8B model on this box is genuinely useful every day. For frontier-level reasoning it won’t match GPT-class cloud models — but it’s free, private, and always available, and that trade is worth it for most of what you actually do.
That’s the whole build — and it runs alongside n8n on the same box, no extra hardware.
Gear used in this build
* Affiliate links — I earn a small commission at no cost to you.It's gear I use and would genuinely recommend. As an Amazon Associate I earn from qualifying purchases. See the full disclosure.
New self-hosted AI & homelab shorts, daily.
Subscribe on YouTubeRelated guides
Run Local AI on Old NVIDIA/AMD GPUs with Ollama
Dust off that graphics card and run Llama, Gemma, Qwen locally with Ollama. Pick a model that fits your VRAM, skip the hype.
Run AI Models on PS5 with llama.cpp GGUF
Turn your PlayStation 5 into a local AI inference engine using llama.cpp's native GGUF support. Run Llama, Qwen, and other quantized models directly on console hardware for private, offline inference.
Run MiniMax H3 in ComfyUI: Local Audio & 2K Video
Generate audio and 2K video together on your own hardware using MiniMax H3 in ComfyUI. No API costs, full control, offline workflows.