BeginnerServer / VPS 4 min read

Run Llama 3.1 8B on a €20/month VPS

A complete guide to running a private LLM API on a budget Linux VPS using llama.cpp server mode.

Written against

Ubuntu 22.04/24.04 VPS · llama.cpp CPU build (GGML_BLAS optional) · GGUF Q4_K_M · llama-server

Not re-run since it was last edited — treat the commands as a starting point, not a tested recipe.

llama.cppVPSLinuxGGUFAPI

What you need first

A Linux VPS with at least 16 GB RAM — 32 GB gives more comfortable headroom for the OS and a longer context window. CPU-only inference on a small model is genuinely usable for personal use, which is the honest case for this whole guide: it is not a production API, it is a private assistant that costs less than a coffee subscription.

bash
# Tested on Ubuntu 22.04/24.04 LTS. Typical cost: ~€15-25/month
# for an 8-core, 16-32GB instance (Hetzner CX32-class or similar).
free -h
nproc --all

Build llama.cpp

A plain CPU build works out of the box. OpenBLAS is worth adding only if you plan to process long documents in large batches — llama.cpp’s own documentation states BLAS acceleration helps prompt processing above batch size 32 and does not change generation speed, so skip it for a pure chat API.

bash
sudo apt update && sudo apt install -y build-essential cmake git
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release -j$(nproc)

Download the model and start the server

Q4_K_M is the right default — 4.6 GB of weights, comfortable in 16 GB of RAM with room for the OS and a real context window. Set the thread count to your physical core count, bind to loopback, and put an API key in front of it if this box has any public exposure at all.

bash
pip install huggingface_hub
huggingface-cli download bartowski/Meta-Llama-3.1-8B-Instruct-GGUF \
  --include "Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf" --local-dir ./models

./build/bin/llama-server \
  -m ./models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  --host 127.0.0.1 --port 8080 \
  -c 8192 -t $(nproc) \
  --api-key "your-secret-key"

Check it actually worked

A successful start is not the same as a usably fast one on a VPS — confirm both. The binary is `llama-server`; older tutorials referencing a bare `server` binary predate the CLI rename.

bash
curl -H "Authorization: Bearer your-secret-key" \
  http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hi"}]}'

# Watch resident memory while it runs:
ps aux | grep llama-server

What the numbers should look like

Llama 3.1 8B at Q4_K_M is 4.6 GB of weights. Generation speed is bandwidth divided by that figure, and system RAM bandwidth on a shared VPS varies far more than GPU VRAM bandwidth does — it depends on what the host actually allocates you, not just the advertised vCPU count — so this guide will not quote a specific tok/s figure nobody here measured on your specific provider. Memory is the number worth watching: 4.6 GB of weights plus the OS should sit comfortably inside 16 GB, leaving room for a context window past the 8K used above.

When it does not work

Out of memory on a 16 GB box: check what else is running — a database or a web server sharing the same VPS competes for the same RAM the model needs. Generation is slower than you expected: system RAM bandwidth on budget VPS tiers is often the real bottleneck, and no amount of thread tuning fixes a shared, oversubscribed memory bus — a bigger instance tier is the actual fix, not a flag. The server binds but nothing outside the box can reach it: that is correct if you bound to `127.0.0.1`, which you should have — put a reverse proxy with TLS in front rather than binding to `0.0.0.0` directly. And if you skipped `--api-key` on a box with any public exposure, that is the first thing to fix, not the last.

Common questions

Is a €20/month VPS actually usable for running an 8B model?

For personal use, yes. Llama 3.1 8B at Q4_K_M needs 4.6 GB of weights, comfortable on a 16GB instance. Generation speed depends on the provider’s system RAM bandwidth, which this index has not measured across providers — treat it as usable for a private assistant, not as a production API with guaranteed throughput.

Should I build llama.cpp with OpenBLAS on a VPS?

Only if you process long documents in large batches. llama.cpp’s documentation states BLAS acceleration helps prompt processing above batch size 32 and has no effect on generation speed — for a chat API, a plain CPU build is enough.

How do I secure a llama.cpp server on a public VPS?

Bind the server to `127.0.0.1`, not `0.0.0.0`, always pass `--api-key`, and put a reverse proxy with TLS in front for anything reachable from the internet — llama.cpp’s built-in server has no rate limiting or user management of its own.

What this guide uses

Hardware
16 GB RAM (CPU)16GB
Models
Llama 3.1 8B Instruct8B
Format
GGUF
Read next
WSL2 + Ollama GPU Passthrough on WindowsCPU Inference: OpenBLAS Tuning for llama.cppAMD GPU + llama.cpp via ROCm (Quick Start)

Next steps

See everything that fits 💻 16 GB RAM (CPU)Reverse lookup — 16GB at 4096 context, ranked by quality

Models covered in this guide

Did this actually run?

Copying a command is not the same as it working, so this is the only place the site asks. Nothing is collected beyond the answer itself.

Related guides

IntermediateServer / VPS3 min read

CPU Inference: OpenBLAS Tuning for llama.cpp

Maximize tokens/sec on a CPU-only VPS with thread count and BLAS backend tuning.

IntermediateServer / VPS5 min read

Nginx Reverse Proxy for Local LLM APIs

Put Ollama or llama.cpp behind Nginx with TLS, rate limiting, and a stable /v1 endpoint for your apps.

IntermediateServer / VPS3 min read

Multi-Model API Server on RTX 4090 with vLLM

Serve multiple AWQ-quantized models with vLLM's continuous batching for production-grade throughput.

Deployment guides are educational. Each model is subject to its own license — read the official Hugging Face model card before downloading or deploying.