Run a Local LLM on a Mini PC: Self-Host Your Own ChatGPT

You don’t need a $2,000 GPU to run a usable local AI. A ~$400 mini PC with an integrated GPU and enough RAM will run 7B–8B models fast enough for daily use — and every token stays on your hardware. Here’s the exact setup I run.

What you need

A mini PC with a modern APU (I use a Ryzen 7 5700G) and at least 32GB RAM (I run 64GB so I can hold a model and still run other services).
~20GB of disk for a couple of models.
Linux (this guide assumes Ubuntu/Debian; Proxmox VM or bare metal both work).

No discrete GPU. On an APU the model runs on CPU + shared memory, which is why RAM matters more than anything else here.

Step 1 — install Ollama

Ollama is the laziest way to pull and serve models. One line:

curl -fsSL https://ollama.com/install.sh | sh

Pull a model and talk to it:

ollama run llama3.1:8b

The first run downloads ~4.7GB. After that it’s instant to start.

Step 2 — pick a model that actually fits

RAM is the constraint. A rough rule for a CPU-only box:

Model size	RAM needed	Feels like
3B (e.g. `llama3.2:3b`)	~4GB	Fast, fine for summaries/classification
7–8B (`llama3.1:8b`, `qwen2.5:7b`)	~8GB	The sweet spot — general chat, coding help
14B	~16GB	Noticeably smarter, slower on CPU

On the 5700G I get ~10–14 tokens/sec on an 8B model — faster than you read. Above 14B on CPU it drops below reading speed and stops being fun.

Step 3 — add a real chat UI

The terminal is fine for testing, but you want a proper interface. Open WebUI gives you a ChatGPT-style UI, pointed at your local Ollama. Run it in Docker:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main

Open http://<your-mini-pc-ip>:3000, create the first account (it becomes admin), and it auto-detects your Ollama models. Done — a private ChatGPT on hardware you own.

Keep it on your LAN. There’s no reason to expose this to the public internet — if you want it on your phone away from home, reach it over a VPN like Tailscale, not a port-forward.

Is it actually usable?

For chat, drafting, summarizing, classification, and coding help: yes, an 8B model on this box is genuinely useful every day. For frontier-level reasoning it won’t match GPT-class cloud models — but it’s free, private, and always available, and that trade is worth it for most of what you actually do.

The full build (including running this alongside n8n on the same box) is in the video.