Run Open WebUI for Private AI Chat on Your Hardware

Open WebUI is a self-hosted AI platform that runs entirely on your hardware. It connects to local models via Ollama or any OpenAI-compatible API, giving you a private, offline-first AI companion with RAG, voice, video, and persistent memory. No data leaves your network.

Prerequisites

You need Python 3.11 or later, Docker, or both. If using Ollama for local models, have that running first on the same machine or accessible on your LAN.

Installation via Python pip

1. Verify Python version:

python3 --version

Ensure you see 3.11 or higher.

2. Install Open WebUI:

pip install open-webui

3. Start the server:

open-webui serve

The service listens on http://localhost:8080. Open that URL in your browser and create an admin account on first login.

Installation via Docker

Docker is cleaner for isolation and persistence. Use one of these commands depending on your setup.

If Ollama runs on the same machine:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

This maps port 3000 on your host to 8080 inside the container and lets the container reach Ollama via host.docker.internal.

If Ollama runs on a different server:

docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=https://ollama-server.local:11434 -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Replace https://ollama-server.local:11434 with your actual Ollama server URL and port.

For CUDA acceleration (Nvidia GPU):

If you have an Nvidia GPU and CUDA installed, use the :cuda tagged image:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always --gpus all ghcr.io/open-webui/open-webui:cuda

Ensure the Nvidia CUDA container toolkit is installed on your system first.

First Login and Setup

1. Access the UI: Navigate to http://localhost:8080 (or http://your-homelab-ip:3000 if running Docker).

2. Create admin account: On first visit, sign up with a username and password. This becomes your admin user.

3. Connect a model provider:

   • If Ollama is running, go to Admin Panel → Settings → Models. Open WebUI auto-discovers Ollama models.
   • To use an external API (LMStudio, Mistral, GroqCloud, etc.), add the API URL under Settings → Models and enter your API key.

4. Pull a model (if using Ollama): In the chat interface, select a model from the dropdown. If it’s not listed, you may need to pull it in Ollama first:

ollama pull mistral

Enable Voice and Video (Optional)

Open WebUI supports hands-free voice and video calls. Configure Speech-to-Text and Text-to-Speech providers in Admin Panel → Settings → Audio:

• STT options: Local Whisper (runs on your hardware), or cloud providers like OpenAI, Deepgram, Azure.
• TTS options: Azure, ElevenLabs, OpenAI, Transformers (local), or WebAPI.

For full privacy, use local Whisper and Transformers.

Set Up RAG (Retrieval Augmented Generation)

RAG lets the AI read your documents and search the web.

1. Upload documents: In chat, click the # button and upload PDFs, text files, or paste URLs. Open WebUI extracts content automatically.

2. Configure vector database (optional): By default, Open WebUI uses an embedded database. For production, go to Admin Panel → Settings → Database and select ChromaDB, Qdrant, PGVector, or another option from the list.

3. Web search (optional): Enable web search under Settings → Web Search and choose a provider (SearXNG, Brave, DuckDuckGo, etc.) to inject live results into conversations.

Multi-Model Conversations

Open WebUI lets you chat with multiple models in one conversation. In the chat header, enable Multi-Model Mode and select two or more models. They will respond in parallel, letting you compare answers side by side.

Create Custom Agents

Wrap any base model with custom instructions, tools, and knowledge:

1. Go to Admin Panel → Models.
2. Click Create Agent and name it (e.g., “Code Helper”, “Writing Coach”).
3. Set a System Prompt with custom instructions.
4. Optionally attach tools (web search, code execution, external APIs) and knowledge documents.
5. Set per-user or per-group access control.
6. Save and use it like any other model in chat.

Persistent Storage and Backups

• Python pip: Data is stored in ~/.local/share/open-webui/ by default.
• Docker: Data is in the open-webui volume. Backup it:

docker run --rm -v open-webui:/data -v $(pwd):/backup alpine tar czf /backup/open-webui-backup.tar.gz -C /data .

To restore:

docker run --rm -v open-webui:/data -v $(pwd):/backup alpine tar xzf /backup/open-webui-backup.tar.gz -C /data

Security: Keep It Private

Critical: Open WebUI should never be exposed to the public internet. Always:

• Run it on your LAN only, or behind a VPN.
• Use a firewall to block external access to port 3000 (Docker) or 8080 (pip).
• If you must expose it remotely, put it behind a reverse proxy (nginx, Caddy) with authentication and HTTPS.
• Use strong admin passwords.
• Enable RBAC in Admin Panel → Access Control to limit user permissions.

Troubleshooting

Models not showing: Ensure Ollama is running and reachable. Check OLLAMA_BASE_URL in Docker environment variables.

Port already in use: Change the host port in Docker (-p 4000:8080 instead of -p 3000:8080) or use a different port for pip (open-webui serve --port 9000).

Database errors: If using Docker, verify the open-webui volume exists and is writable:

docker volume ls | grep open-webui

Slow responses: Check if your model is running (ollama ps) and if your hardware has enough RAM. Smaller models (7B parameters) run faster on modest hardware.

Is It Worth It?

Yes, if you value privacy and control. Open WebUI is the most feature-complete self-hosted AI UI available—RAG, voice, video, multi-model, agents, all offline. The setup is straightforward on a homelab. The trade-off: you manage the infrastructure and models, and response times depend on your hardware. For a serious home AI companion or team tool, it’s solid. For casual use, a cloud service is faster. But for a homelab where you already run Ollama or control your own LLM infrastructure, Open WebUI is the obvious choice.

Sources

• https://github.com/open-webui/open-webui
• https://docs.openwebui.com