Local LLM error lookup
Paste the line that stopped you. Each entry quotes the message from the project's own source code, says what it means, and lists fixes that name real flags and commands.
19 errors
Out of memory
- llama.cppCUDA error: out of memory (llama.cpp)
CUDA error: out of memoryllama.cpp asked the card for memory and none was left: the weights, the KV cache for your context and the compute buffers add up to more than the free VRAM. - llama.cppunable to allocate CUDA0 buffer (llama.cpp)
unable to allocate CUDA0 bufferllama.cpp could not reserve room for the model's weights on that device. The device name follows your backend: CUDA0, ROCm0, Vulkan0 or Metal. - llama.cppfailed to allocate buffer for kv cache
failed to allocate buffer for kv cacheThe weights loaded, but there was no room left for the KV cache at the context you asked for. It is usually followed by "failed to initialize the context". - llama.cppMetal: failed to allocate buffer on a Mac
ggml_metal_buffer_init: error: failed to allocate buffer, size = <n> MiBOn a Mac, Metal would not give llama.cpp a GPU buffer that large. macOS lets the GPU use only part of unified memory, by default about three quarters. - vLLMCUDA out of memory. Tried to allocate (vLLM)
CUDA out of memory. Tried to allocate <n> GiB. GPU 0 has a total capacity of <n> GiB of which <n> MiB is free.PyTorch, which vLLM runs on, could not allocate the next tensor. In vLLM it usually means the model plus its KV cache does not fit at your settings. - vLLMvLLM: free memory less than GPU memory utilization
Free memory on device <dev> (<free>/<total> GiB) on startup is less than desired GPU memory utilization (<u>, <n> GiB). Decrease GPU memory utilization or reduce GPU memory used by other processes.vLLM reserves a fixed share of the card at startup (--gpu-memory-utilization, 0.92 in current releases unless you set it). Something else already uses enough of the card that this share is not free.
The model file
- llama.cppllama.cpp: unknown model architecture
llama_model_load: error loading model: unknown model architecture: '<arch>'Your llama.cpp build does not know this model family yet. New architectures reach llama.cpp first, then Ollama and LM Studio when they update. - llama.cppllama.cpp: wrong number of tensors
done_getting_tensors: wrong number of tensors; expected <n>, got <m>The file holds more or fewer tensors than this llama.cpp expects for its architecture — the file and the program disagree about the model's layout. A sibling message reads "tensor … has wrong shape". - llama.cppinvalid magic characters, expected 'GGUF'
gguf_init_from_reader: invalid magic characters: '<xxxx>', expected 'GGUF'Every GGUF file starts with the bytes "GGUF" and this one does not. It is usually a web page, a Git LFS pointer or a safetensors file named .gguf.
The GPU is not being used
- llama.cppggml_cuda_init: failed to initialize CUDA
ggml_cuda_init: failed to initialize CUDA: <reason>A CUDA build of llama.cpp started but could not talk to any NVIDIA GPU, so it falls back to the CPU. The text after the colon is CUDA's own reason. - DockerDocker: could not select device driver with [[gpu]]
could not select device driver "" with capabilities: [[gpu]]Docker was asked for a GPU (--gpus all) but has no GPU driver registered. On NVIDIA it comes from the NVIDIA Container Toolkit, missing or unconfigured. - OllamaOllama shows 100% CPU instead of the GPU
100% CPUollama ps shows where a model is loaded. "100% CPU" means none of it is on the GPU; a split such as "48%/52% CPU/GPU" means it did not fit. - PyTorchCUDA capability sm_120 is not compatible with PyTorch
<GPU> with CUDA capability sm_<nn> is not compatible with the current PyTorch installation.The installed PyTorch was not built for this GPU generation — the usual case is an RTX 50-series card (sm_120) with a PyTorch built for an older CUDA. vLLM and Transformers then fail or fall back.
AMD and ROCm
- ROCmrocBLAS error: Cannot read TensileLibrary.dat
rocBLAS error: Cannot read <path>/TensileLibrary.dat: No such file or directory for GPU arch : gfx<nnnn>rocBLAS ships kernels only for the GPU targets ROCm supports. Your card's gfx target, at the end of the line, is not one of them. - ROCmHSA_OVERRIDE_GFX_VERSION: ROCm on unlisted Radeons
HSA_OVERRIDE_GFX_VERSIONAn environment variable that makes ROCm treat your Radeon as a nearby supported model, so ROCm builds of Ollama and llama.cpp can use a card ROCm does not list. Linux only.
Ollama
- OllamaOllama: model may be incompatible with your version
this model may be incompatible with your version of Ollama. If you previously pulled this model, try updating it by running `ollama pull <model>`Ollama's built-in llama.cpp could not load the file: the model is newer than your Ollama, or your copy was pulled long ago and since republished. - OllamaOllama: model requires a newer Ollama version
model requires ollama version <x.y.z> or newer (current version is v<a.b.c>)The model's manifest names a minimum Ollama version and yours is older. The download is fine; Ollama refuses before loading because it cannot run it. - OllamaOllama: model not found, try pulling it first
model "<name>" not found, try pulling it firstThe API was asked for a model this machine does not have. Ollama's API never downloads on its own; the name must match a pulled model, tag included. - OllamaOllama: pull model manifest: file does not exist
pull model manifest: file does not existThe registry has no model under that name and tag. It is almost always a typo or a tag that was never published, not a network problem.
Not listed? An error that is not here was not found in source code we could read, so it is not quoted. Write to zjy@quantized.uk with the full log line.