Run Private AI Chat Apps with Ollama

Ollama lets you run large language models locally on real hardware—no subscriptions, no sending your conversations to the cloud. You pull open models like Gemma 4, run them on your machine, and wire them up to web interfaces or desktop apps so you can actually talk to them. This guide walks you through getting Ollama running and connecting it to a conversational UI.

Install Ollama

Ollama runs on macOS, Windows, Linux, and Docker. Pick your platform:

macOS and Linux

1. Run the install script:

curl -fsSL https://ollama.com/install.sh | sh

2. Verify installation:

ollama --version

Windows

1. Run the PowerShell installer:

irm https://ollama.com/install.ps1 | iex

2. Or download the installer manually from https://ollama.com/download/OllamaSetup.exe and run it.

Docker

If you’re running Ollama in a container:

docker run --rm -it ollama/ollama

For GPU acceleration on Docker, add --gpus all if you have NVIDIA drivers.

Pull and Run a Model

1. Start Ollama in the background (it listens on localhost:11434 by default):

ollama serve

2. In another terminal, pull a model. Gemma 4 is a good starting point:

ollama pull gemma4

3. Chat with it directly:

ollama run gemma4

Type your questions and press Enter. Type /bye to exit.

Connect a Web Chat Interface

The command-line is fine for testing, but you’ll want a proper chat UI. Open WebUI is the most popular choice—it’s self-hosted, extensible, and works great with Ollama.

Option 1: Open WebUI with Docker

If you have Docker running:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway \
  --name open-webui \
  ghcr.io/open-webui/open-webui:latest

Then visit http://localhost:3000 in your browser. On first load, create an admin account. Open WebUI will auto-detect Ollama at http://host.docker.internal:11434.

Option 2: Open WebUI from Source

If you prefer to run it natively:

1. Clone the repo:

git clone https://github.com/open-webui/open-webui.git
cd open-webui

2. Install dependencies and start (requires Node.js and Python):

bash install.sh

3. Open http://localhost:3000.

Option 3: Other Chat Interfaces

The README lists many alternatives. A few lightweight ones:

• Hollama: minimal web interface, very fast
• Chatbox: desktop app for macOS, Windows, Linux
• Lobe Chat: modern chat framework with plugin support
• Bionic GPT: on-premise AI platform with knowledge base features

All of them talk to Ollama via the REST API on port 11434.

Use the REST API Directly

You don’t need a UI at all. Ollama exposes a REST API:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'

Python

pip install ollama

from ollama import chat

response = chat(model='gemma4', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response.message.content)

JavaScript

npm i ollama

import ollama from "ollama";

const response = await ollama.chat({
  model: "gemma4",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);

Integrate with Coding Tools

Ollama works as a backend for code editors and IDEs. Popular integrations include Continue (VS Code, JetBrains), Cline (VS Code), twinny (VS Code, Vim), and Copilot for Obsidian. Each one points to localhost:11434 and uses Ollama as a local Copilot replacement.

Use Ollama as a Personal AI Assistant

OpenClaw turns Ollama into a chatbot that works across WhatsApp, Telegram, Slack, and Discord:

ollama launch openclaw

This is useful if you want to query your local model from your phone or team chat without exposing anything to the internet.

Keep It Private

Important: Ollama listens on localhost:11434 by default, which means it’s only accessible from your machine. If you want to use it from other devices on your LAN, you can bind it to your local IP, but never expose it directly to the public internet. If you need remote access, always run it behind a VPN or reverse proxy with authentication.

To bind to your LAN IP (e.g., 192.168.1.100):

OLLAMA_HOST=192.168.1.100:11434 ollama serve

Then access it from another device at http://192.168.1.100:11434/api/chat. But keep your LAN segmented and never forward this port publicly.

Is It Worth It?

Yes, if you want to own your AI. Ollama is fast, lightweight, and runs on modest hardware. You get models that are good enough for most tasks—summarization, Q&A, coding assistance, brainstorming—without paying per API call or worrying about data leaving your network. The ecosystem of community chat UIs and integrations is mature and active. The trade-off is that local models are slower than cloud APIs and generally smaller than GPT-4, but for privacy and cost, it’s a solid play.

Sources

• https://github.com/ollama/ollama
• https://ollama.com/install.sh
• https://ollama.com/download/OllamaSetup.exe
• https://ollama.com/download/Ollama.dmg
• https://docs.ollama.com/quickstart
• https://docs.ollama.com/api
• https://docs.ollama.com/cli
• https://github.com/ollama/ollama-python
• https://github.com/ollama/ollama-js
• https://github.com/open-webui/open-webui
• https://docs.ollama.com/integrations/openclaw