OpenClaw Setup Guide: Run OpenClaw with a Local LLM

What is OpenClaw?

OpenClaw is an open-source AI agent you can run on your own hardware. It connects to a language model and executes multi-step tasks such as writing messages, automating workflows, or running commands. When you pair OpenClaw with a local LLM, your prompts and outputs stay on your machine.

Install OpenClaw

OpenClaw can run in Docker or via Node.js. Docker is usually the best default because it isolates dependencies and simplifies upgrades.

# Docker (recommended)
docker pull openclaw/openclaw
docker run -d --name openclaw \
  -v openclaw-data:/app/data \
  openclaw/openclaw

# Or via npm
npx openclaw

Set Up a Local LLM Backend (Ollama, llama.cpp, or LM Studio)

OpenClaw needs a local model endpoint. The fastest path is Ollama, but llama.cpp server and LM Studio also work because they expose OpenAI-compatible APIs. Install one backend, pull a model, and keep the API URL ready for OpenClaw configuration.

# 1. Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# 2. Pull a recommended model
ollama pull llama3.1:8b       # 5.5 GB VRAM — fast, general purpose
ollama pull qwen2.5-coder:7b  # 5.5 GB VRAM — best for coding tasks
ollama pull deepseek-r1:7b    # 5.5 GB VRAM — strong reasoning

# 3. Start the Ollama server
ollama serve

Connect OpenClaw to Your Local LLM API

Point OpenClaw to your local OpenAI-compatible endpoint and set a model that fits your hardware. If responses are slow, choose a smaller quantization or a lower-parameter model.

# In your OpenClaw config (config.yaml or .env)
llm:
  name: local-ollama
  type: openai-compatible
  base_url: http://localhost:11434/v1
  model: llama3.1:8b
  timeout_ms: 60000

# For llama.cpp server:
# base_url: http://localhost:8080/v1

# For LM Studio:
# base_url: http://localhost:1234/v1

Best Local Models for OpenClaw by VRAM Tier

Agent quality depends heavily on instruction-following and tool-use reliability. Start with a model tier that matches your VRAM budget:

- Budget (4-6 GB VRAM): Phi-4 Mini, Llama 3.1 8B, DeepSeek R1 7B - Mid-range (8-16 GB VRAM): Llama 4 Scout 17B, Mistral 7B (Q8), Qwen 2.5 Coder 7B - High-end (20+ GB VRAM): QwQ 32B, Qwen 2.5 Coder 32B, DeepSeek R1 32B

OpenClaw Hardware Requirements

OpenClaw itself is lightweight (< 500 MB RAM), but model inference determines real performance. Most users should target at least 5-6 GB VRAM for small 7-8B models and 12+ GB VRAM for more capable setups. For a responsive agent workflow, aim for at least 10 tokens/sec and validate model fit on the OpenClaw hardware checker page.