Prerequisites
You need Windows 11 (or Win10 21H2+), an NVIDIA GPU, and the latest Windows NVIDIA driver. WSL2 CUDA does not need a separate Linux driver install.
powershell
# In PowerShell (Admin)
wsl --install
wsl --update
# Verify GPU visible inside WSL
wsl nvidia-smiInstall Ollama in WSL
Install inside Ubuntu WSL, not Windows native, for best GPU compatibility. Access the API from Windows at localhost:11434.
bash
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5:7b
ollama run qwen2.5:7b
# From Windows browser or PowerShell:
# curl http://localhost:11434/api/tagsRelated guides
BeginnerEdge / Local6 min read
Ollama on Windows (Native, No WSL)
Install the Windows Ollama app for the simplest path — GPU works on NVIDIA; AMD is CPU-only for now.
BeginnerEdge / Local9 min read
llama.cpp on Windows with CUDA
Build llama.cpp with NVIDIA GPU support on Windows 11 — the path of least resistance for PC gamers.
IntermediateEdge / Local9 min read
Run GPT-OSS 20B (and 120B) locally without re-quantizing
GPT-OSS ships natively in MXFP4, so the usual "download the Q4_K_M" habit makes it bigger and worse. Sizing, the right flags, and how MoE expert-offload puts the 120B on a 24GB card.
Deployment guides are educational. Each model is subject to its own license — read the official Hugging Face model card before downloading or deploying.