What LLM Can I Run Locally? Check Your GPU
Select your GPU to find local models, quantization levels and context lengths within its memory budget. Compare options before downloading weights.
Pick your GPU and see what models you can run
Common cards:
What will you run?
Text inference: excludes vision encoders. Models must support the selected context length.
4K4,096 tokens
KV cache type
f16 is the default in llama.cpp and Ollama.
1
14816
Select hardware...
Frequently asked questions
Which LLM can I run locally?
Select your exact GPU and context length. Start with results that leave memory headroom, then compare quantization levels on the model page.
How do you choose an LLM you can run locally?
Start with your task, such as chat or coding. For long documents, check the fit again at the context you need. For images, select image input.