docker-compose.yml
Requires NVIDIA Container Toolkit on the host. The deploy.resources block requests one GPU. Persist models in a named volume.
yaml
services:
ollama:
image: ollama/ollama:latest
container_name: ollama
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
restart: unless-stopped
volumes:
ollama_data:Run and pull models
Use docker compose (v2) on Linux. On Windows Docker Desktop, enable WSL2 backend and GPU support in settings first.
bash
docker compose up -d
docker exec -it ollama ollama pull llama3.1:8b
docker exec -it ollama ollama run llama3.1:8b
# API test
curl http://localhost:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"Hello","stream":false}'Related guides
BeginnerDocker5 min read
Docker Compose LLM Stack: Ollama + Open WebUI
A production-ready Docker Compose stack that gives you a local ChatGPT experience with one command.
IntermediateServer / VPS12 min read
Multi-Model API Server on RTX 4090 with vLLM
Serve multiple AWQ-quantized models with vLLM's continuous batching for production-grade throughput.
IntermediateEdge / Local10 min read
WSL2 + Ollama GPU Passthrough on Windows
Run Ollama with NVIDIA GPU acceleration inside WSL2 — the most reliable Windows path for local LLMs.
Deployment guides are educational. Each model is subject to its own license — read the official Hugging Face model card before downloading or deploying.