IntermediateServer / VPS 7 min read

CPU Inference: OpenBLAS Tuning for llama.cpp

Maximize tokens/sec on a CPU-only VPS with thread count and BLAS backend tuning.

Verified stack

CPU-only VPS · llama.cpp OpenBLAS · -t physical cores · 8B Q4_K_M

Last verified 2026-07-22

CPUllama.cppOpenBLASVPS

Thread count

Set -t to physical core count (not hyperthreads). Use -tb 1 for single-batch interactive use.

bash

./build/bin/llama-server \
  -m ./models/Llama-3.1-8B-Q4_K_M.gguf \
  -t 8 -tb 1 -c 4096 \
  --host 0.0.0.0 --port 8080

Expected performance

A 8-core VPS with OpenBLAS achieves ~8–15 tok/s on 8B Q4_K_M. Usable for personal API, not production throughput.

text

Hetzner CX32 (8 vCPU, 32GB): ~12 tok/s
AWS c7i.2xlarge (8 vCPU): ~15 tok/s

Related guides

BeginnerServer / VPS8 min read

Run Llama 3.1 8B on a €20/month VPS

A complete guide to running a private LLM API on a budget Linux VPS using llama.cpp server mode.

IntermediateServer / VPS11 min read

Nginx Reverse Proxy for Local LLM APIs

Put Ollama or llama.cpp behind Nginx with TLS, rate limiting, and a stable /v1 endpoint for your apps.

IntermediateServer / VPS12 min read

Multi-Model API Server on RTX 4090 with vLLM

Serve multiple AWQ-quantized models with vLLM's continuous batching for production-grade throughput.

Deployment guides are educational. Each model is subject to its own license — read the official Hugging Face model card before downloading or deploying.