CUDA capability sm_120 is not compatible with PyTorch

The message

<GPU> with CUDA capability sm_<nn> is not compatible with the current PyTorch installation.

Parts in <angle brackets> change from run to run.

What it means

The installed PyTorch was not built for this GPU generation — the usual case is an RTX 50-series card (sm_120) with a PyTorch built for an older CUDA. vLLM and Transformers then fail or fall back.

Why it happens

• A PyTorch wheel built for CUDA older than 12.8; Blackwell cards need 12.8 or newer.

How to fix it

1. Install vLLM with uv pip install vllm --torch-backend=auto, which picks the PyTorch build that matches your driver.
2. For PyTorch alone, use the install selector on pytorch.org and choose a CUDA 12.8+ build.

Guides that cover this

• RTX 5090: What 32GB Actually Buys for Local LLMs
• vLLM + AWQ in Production: Tuning Guide