What you need first
Windows 11 and a recent NVIDIA driver. That is genuinely the list — Ollama’s Windows installer ships the GPU runtime it needs, so there is no CUDA Toolkit to install and no build step. If you already run WSL for other work you can keep it; this guide is for the case where you would rather not.
# Download and run the installer from ollama.com, then:
ollama --version
nvidia-smiPull a model and run it
Start with an 8B at Q4_K_M — Llama 3.1 8B needs 5.6 GB at 4K context on this index’s numbers, so it fits an 8GB card with room to spare. Ollama picks the GPU by itself; there is no flag to turn it on.
ollama pull llama3.1:8b
ollama run llama3.1:8bWhere the models actually go
On Windows the blobs land under your user profile, which is usually the C: drive, and a handful of models will fill a small SSD without ever telling you why. Move the store before you download rather than after: `OLLAMA_MODELS` is read at service start, so set it as a user environment variable and restart Ollama from the tray.
# Default: %USERPROFILE%\.ollama\models
setx OLLAMA_MODELS "D:\ollama\models"
# Then quit Ollama from the system tray and start it again.Check it actually ran on the GPU
Ollama will quietly place part of a model on the CPU when it does not fit, and the only symptom is slowness. `ollama ps` reports what it decided, per loaded model — the PROCESSOR column is the answer, and anything other than 100% GPU on a model that should fit is worth chasing.
ollama ps
# NAME SIZE PROCESSOR UNTIL
# llama3.1:8b 6.1 GB 100% GPU 4 minutes from now
nvidia-smi --query-gpu=memory.used,utilization.gpu --format=csv -l 1Using it from other programs
The daemon serves an HTTP API on port 11434 whether or not you ever open a terminal, and that is what editor plugins and desktop clients talk to. By default it listens on localhost only. `OLLAMA_HOST` changes that — and exposing it to your network means anyone on that network can use your GPU, so it deserves a deliberate decision rather than a copied command.
curl http://localhost:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"hello","stream":false}'
# Keep a model resident for an hour instead of the 5-minute default:
curl http://localhost:11434/api/generate -d '{"model":"llama3.1:8b","keep_alive":"1h"}'When it does not work
`ollama ps` shows a CPU share on a model that should fit: something else is holding VRAM — a browser with hardware acceleration is the usual culprit — or System Memory Fallback is on in the NVIDIA control panel, which spills VRAM into system RAM instead of failing and turns "does not fit" into "inexplicably slow". The C: drive fills up: that is the model store, and `OLLAMA_MODELS` has to be set before the download, not after. A plugin cannot reach the API: the daemon is bound to localhost by default, which is correct — change it deliberately or point the plugin at `127.0.0.1:11434`. And a model that ran yesterday will not load today usually means another model is still resident; they unload after five minutes idle, or immediately with `ollama stop`.
Common questions
Do I need WSL to run Ollama on Windows?
No. The native Windows installer ships the GPU runtime it needs and uses an NVIDIA card directly — no CUDA Toolkit, no build step. WSL remains a reasonable choice if you want a Linux environment for other reasons, but it adds a layer rather than removing one.
How do I move Ollama models off the C: drive?
Set `OLLAMA_MODELS` to the path you want and restart the Ollama service from the system tray — it is read at service start, so a running daemon will not pick it up. Do this before downloading rather than after: the default store is under your user profile, and a handful of models will fill a small system drive without explaining itself.
How do I know whether Ollama is using my GPU?
Run `ollama ps` while a model is loaded. The PROCESSOR column reads `100% GPU` when the whole model is on the card, and names a CPU share otherwise. If it reports a CPU share on a model that should fit, check what else is holding VRAM and whether System Memory Fallback is enabled in the NVIDIA control panel.
What this guide uses
- Format
- GGUF
Next steps
Models covered in this guide
Did this actually run?
Copying a command is not the same as it working, so this is the only place the site asks. Nothing is collected beyond the answer itself.
Related guides
WSL2 + Ollama GPU Passthrough on Windows
Run Ollama with NVIDIA GPU acceleration inside WSL2 — the most reliable Windows path for local LLMs.
llama.cpp on Windows with CUDA
Build llama.cpp with NVIDIA GPU support on Windows 11 — the path of least resistance for PC gamers.
8GB GPU Starter Guide: 3060 / 4060 / 3070
The most common local LLM hardware tier — which models, quants, and context lengths actually fit in 8GB VRAM.