Run Bonsai 27B Locally with Ollama

Bonsai 27B is a capable open-source model that runs on modest hardware. With Ollama, you can pull it, run it locally, and integrate it into applications—all without sending data to the cloud. This guide walks you through installation and getting Bonsai 27B working on your machine.

Install Ollama

Ollama runs on macOS, Windows, Linux, and Docker. Pick your platform:

macOS:

1. Download and install from the official installer:

curl -fsSL https://ollama.com/install.sh | sh

Or download manually from https://ollama.com/download/Ollama.dmg

Windows:

1. Run the PowerShell installer:

irm https://ollama.com/install.ps1 | iex

Or download manually from https://ollama.com/download/OllamaSetup.exe

Linux:

1. Run the install script:

curl -fsSL https://ollama.com/install.sh | sh

For manual installation, check the official Linux docs.

Docker:

1. Use the official image ollama/ollama from Docker Hub if you prefer containerized deployment.

Pull and Run Bonsai 27B

Once Ollama is installed, you can pull and run Bonsai 27B. The model library at ollama.com/library lists available models.

1. Pull the Bonsai 27B model:

ollama pull bonsai

Ollama will download the model weights to your machine. This is a one-time operation; subsequent runs use the cached model.

2. Start an interactive chat session:

ollama run bonsai

You’ll get a prompt where you can type questions and receive responses from Bonsai 27B running locally.

Use Bonsai via REST API

Ollama exposes a REST API on http://localhost:11434 by default. This lets you integrate Bonsai into applications.

1. Query Bonsai 27B with curl:

curl http://localhost:11434/api/chat -d '{
  "model": "bonsai",
  "messages": [{
    "role": "user",
    "content": "Explain how transformers work"
  }],
  "stream": false
}'

The response includes the model’s answer in JSON format.

Integrate with Python

For Python applications, install the Ollama client library:

1. Install the package:

pip install ollama

2. Write a simple script:

from ollama import chat

response = chat(model='bonsai', messages=[
  {
    'role': 'user',
    'content': 'What are the benefits of local inference?',
  },
])
print(response.message.content)

Run it and Bonsai will respond without leaving your network.

Integrate with JavaScript

For Node.js and browser environments:

1. Install the package:

npm i ollama

2. Write a simple script:

import ollama from "ollama";

const response = await ollama.chat({
  model: "bonsai",
  messages: [{ role: "user", content: "Explain quantum computing" }],
});
console.log(response.message.content);

Keep It on the LAN

By default, Ollama listens on localhost:11434, which is safe. If you need to access it from other machines on your network, bind it explicitly:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

Important: Only do this on a trusted private network. Never expose Ollama to the public internet without authentication. Use a firewall rule or VPN to restrict access to your LAN.

Verify It’s Running

Check that Ollama is responding:

curl http://localhost:11434/api/tags

You’ll see a JSON list of models you’ve pulled, including Bonsai 27B.

Is It Worth It?

Yes. Bonsai 27B on Ollama gives you a capable model with zero cloud costs, zero latency waiting for external APIs, and complete privacy. Your data stays on your hardware. The trade-off is that you need enough VRAM or system RAM to run it—Bonsai 27B typically needs 16–32 GB depending on quantization. If you’ve got the hardware, this is a solid foundation for local AI work.

Sources

• https://github.com/ollama/ollama
• https://ollama.ai
• https://docs.ollama.com