12GB VRAM
RTX 3080 Ti — what LLMs can it run?
45 of 79 indexed models fit comfortably in 12GB at 4K context, each at the highest-quality quant that still leaves headroom.
14B · 16
| Model | Quant | Est. VRAM | Headroom | tok/s |
|---|---|---|---|---|
| DeepSeek-Coder-V2-Lite Instruct16B | AWQ INT4 | 9.13 GB | +2.9 GB | 192 |
| DeepSeek-V2-Lite Chat16B | AWQ INT4 | 9.13 GB | +2.9 GB | 188 |
| StarCoder2 15B15B | Q4_K_M | 10.19 GB | +1.8 GB | 92 |
| Qwen3 14B Instruct14B | EXL2 4.65bpw | 9.66 GB | +2.3 GB | 125 |
| Qwen2.5 14B Instruct14B | EXL2 4.65bpw | 9.75 GB | +2.3 GB | 138 |
| DeepSeek-R1-Distill-Qwen-14B14B | EXL2 4.65bpw | 9.75 GB | +2.3 GB | 128 |
| Phi-4 14B14B | Q4_K_M | 10.17 GB | +1.8 GB | 88 |
| Phi-3 Medium 14B Instruct14B | Q4_K_M | 9.73 GB | +2.3 GB | 102 |
| Mistral Nemo 12B Instruct12B | Q4_K_M | 8.42 GB | +3.6 GB | 112 |
| Gemma 3 12B IT12B | Q4_K_M | 9.28 GB | +2.7 GB | 105 |
| Stable LM 2 12B Chat12B | Q4_K_M | 8.35 GB | +3.7 GB | 108 |
| Jamba 1.5 Mini12B | Q4_K_M | 8.15 GB | +3.9 GB | 95 |
| Llama 3.2 11B Vision Instruct11B | Q4_K_M | 7.66 GB | +4.3 GB | 88 |
| Solar 10.7B Instruct11B | Q4_K_M | 7.6 GB | +4.4 GB | 125 |
| Falcon 3 10B Instruct10B | Q4_K_M | 7.28 GB | +4.7 GB | 118 |
| Gemma 2 9B Instruct9B | Q4_K_M | 7.3 GB | +4.7 GB | 132 |
7B · 19
| Model | Quant | Est. VRAM | Headroom | tok/s |
|---|---|---|---|---|
| GLM-4-9B-Chat9B | Q8_0 | 10.16 GB | +1.8 GB | 105 |
| Qwen3-VL 8B Instruct8B | Q8_0 | 10.39 GB | +1.6 GB | 108 |
| Qwen2-VL 7B Instruct7B | Q4_K_M | 5.49 GB | +6.5 GB | 72 |
| Granite 3.1 8B Instruct8B | Q4_K_M | 5.88 GB | +6.1 GB | 142 |
| Qwen3 8B Instruct8B | Q6_K | 7.64 GB | +4.4 GB | 122 |
| Llama 3.1 8B Instruct8B | Q8_0 | 9.47 GB | +2.5 GB | 118 |
| Nous Hermes 3 Llama 3.1 8B8B | EXL2 4.65bpw | 5.43 GB | +6.6 GB | 232 |
| Aya 23 8B8B | Q4_K_M | 5.64 GB | +6.4 GB | 145 |
| OpenChat 3.6 8B8B | EXL2 4.65bpw | 5.43 GB | +6.6 GB | 228 |
| DeepSeek-R1-Distill-Llama-8B8B | Q5_K_M | 6.51 GB | +5.5 GB | 128 |
| InternLM2 7B Chat7B | Q4_K_M | 5.45 GB | +6.6 GB | 148 |
| Qwen2.5 7B Instruct7B | Q6_K | 6.77 GB | +5.2 GB | 132 |
| Qwen2.5-Coder 7B Instruct7B | EXL2 4.65bpw | 4.87 GB | +7.1 GB | 248 |
| WizardLM-2 7B7B | Q4_K_M | 5.07 GB | +6.9 GB | 152 |
| DeepSeek-R1-Distill-Qwen-7B7B | EXL2 4.65bpw | 4.87 GB | +7.1 GB | 210 |
| OLMo 2 7B Instruct7B | Q8_0 | 10.3 GB | +1.7 GB | 125 |
| Mistral 7B Instruct v0.37B | Q6_K | 6.75 GB | +5.3 GB | 135 |
| Zephyr 7B Beta7B | Q6_K | 6.75 GB | +5.3 GB | 132 |
| Gemma 3 4B IT4B | Q8_0 | 5.36 GB | +6.6 GB | 145 |
≤3B · 10
| Model | Quant | Est. VRAM | Headroom | tok/s |
|---|---|---|---|---|
| Qwen3 4B Instruct4B | Q6_K | 4.05 GB | +8 GB | 145 |
| Phi-4 Mini Instruct3.8B | Q8_0 | 4.68 GB | +7.3 GB | 262 |
| Phi-3.5 Mini Instruct3.8B | Q8_0 | 5.89 GB | +6.1 GB | 255 |
| Llama 3.2 3B Instruct3B | Q8_0 | 4.05 GB | +8 GB | 285 |
| Qwen2.5 3B Instruct3B | Q8_0 | 3.67 GB | +8.3 GB | 290 |
| Gemma 2 2B Instruct2B | Q8_0 | 3.34 GB | +8.7 GB | 320 |
| Qwen3 1.7B Instruct1.7B | Q8_0 | 2.39 GB | +9.6 GB | 240 |
| Qwen2.5 1.5B Instruct1.5B | Q8_0 | 1.83 GB | +10.2 GB | 410 |
| Llama 3.2 1B Instruct1B | Q8_0 | 1.51 GB | +10.5 GB | 450 |
| Qwen2.5 0.5B Instruct0.5B | Q8_0 | 0.6 GB | +11.4 GB | 540 |
How this list is built
Each row is the lowest-perplexity-loss quant of that model whose estimated total — weights plus KV cache at 4K context plus activation buffer — uses at most 88% of the card. That is the calculator's "green" threshold, so every row here has real headroom rather than only just fitting. Raise the context length and the list shortens; the calculator lets you check any combination directly.
Stepping up
A RTX 4080 Super (16GB) fits 6 more of the indexed models than this card. RTX 4080 Super →
45 of 79 indexed models fit comfortably in 12GB at 4K context, each at the highest-quality quant that still leaves headroom.