AdvancedEdge / Local 12 min read

AMD GPU + llama.cpp via ROCm (Quick Start)

Run GGUF models on Radeon RX 7900 / 6800 series with llama.cpp HIP backend — what works and what does not.

Verified stack

Linux · ROCm 6.x · llama.cpp GGML_HIP=ON · RX 7900/6800 class

Last verified 2026-07-22

AMDROCmllama.cppHIPLinux

Supported hardware

ROCm support on consumer AMD GPUs is Linux-only and varies by architecture. RDNA3 (7900 XTX) and RDNA2 (6800 XT) have the best community reports. Windows ROCm for llama.cpp remains experimental.

text

Good: RX 7900 XTX, RX 6800 XT, RX 6900 XT (Linux)
Patchy: RX 6700 XT, older Polaris
Not supported: integrated Radeon graphics

Build llama.cpp with HIP

Install ROCm 6.x, then build with GGML_HIP=ON. Set HSA_OVERRIDE_GFX_VERSION if your GPU is not officially listed.

bash

# Ubuntu 22.04 example
sudo apt install rocm-dev

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_HIP=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
cmake --build build -j$(nproc)

./build/bin/llama-server -m ./model.gguf -ngl 99 -c 4096 --host 0.0.0.0

Related guides

AdvancedEdge / Local11 min read

Running 70B on Dual RTX 3090 with llama.cpp

Tensor-split across two 24GB cards to run Llama 3.1 70B or Qwen2.5 72B at Q4.

AdvancedEdge / Local12 min read

Quantize Your Own Model to GGUF

Use llama.cpp's quantize tool to convert any HF model to GGUF Q4_K_M for local inference.

BeginnerEdge / Local9 min read

llama.cpp on Windows with CUDA

Build llama.cpp with NVIDIA GPU support on Windows 11 — the path of least resistance for PC gamers.

Deployment guides are educational. Each model is subject to its own license — read the official Hugging Face model card before downloading or deploying.