Docker: could not select device driver with [[gpu]]

The message

could not select device driver "" with capabilities: [[gpu]]

What it means

Docker was asked for a GPU (--gpus all) but has no GPU driver registered. On NVIDIA it comes from the NVIDIA Container Toolkit, missing or unconfigured.

Why it happens

• The NVIDIA Container Toolkit is not installed on the host.
• It is installed, but Docker was not configured to use it or not restarted afterwards.
• The card is AMD: --gpus is NVIDIA-only.

How to fix it

1. Install the NVIDIA Container Toolkit, then sudo nvidia-ctk runtime configure --runtime=docker and sudo systemctl restart docker.
2. Check with docker run --rm --gpus all ubuntu nvidia-smi.
3. On AMD, drop --gpus and pass the devices instead: --device /dev/kfd --device /dev/dri, with a ROCm image.

Guides that cover this

• Docker: Ollama with NVIDIA or AMD GPU Passthrough
• Docker Compose LLM Stack: Ollama + Open WebUI

Related errors

• ggml_cuda_init: failed to initialize CUDA