Local LLM GPU Requirements by Model Size

Compare five GPUs across model sizes at 4K context. Each populated cell links to a model and its quantization details.

Memory estimates at 4,096 tokens for one text sequence.

Each cell shows one catalog model with headroom, its quantization and its estimated memory. A dash means no qualifying example was found in that size class.

GPU≤3B7B14B32B70B+
RTX 4060 Ti 8G
Qwen3 4B Instruct
GGUF · Q6_K · 4.0 GiB
GLM-4-9B-Chat
GGUF · Q4_K_M · 5.9 GiB
Falcon 3 10B Instruct
GPTQ · INT4 · 6.1 GiB
——
RTX 3060 12G
Qwen3 4B Instruct
GGUF · Q6_K · 4.0 GiB
GLM-4-9B-Chat
GGUF · Q8_0 · 10.2 GiB
DeepSeek-Coder-V2-Lite Instruct
GGUF · Q4_K_M · 10.1 GiB
Hy-Memory-30B-A3B
GGUF · Q8_0-expsIQ2_XXS · 9.9 GiB
—
RTX 4060 Ti 16G
Qwen3 4B Instruct
GGUF · Q6_K · 4.0 GiB
GLM-4-9B-Chat
GGUF · Q8_0 · 10.2 GiB
DeepSeek-Coder-V2-Lite Instruct
GGUF · Q4_K_M · 10.1 GiB
Xing4.0 29B-A4B
GGUF · Q2_K · 13.6 GiB
—
RTX 4090
Qwen3 4B Instruct
GGUF · Q6_K · 4.0 GiB
GLM-4-9B-Chat
GGUF · Q8_0 · 10.2 GiB
DeepSeek-Coder-V2-Lite Instruct
GGUF · Q8_0 · 17.6 GiB
Recursive Flint 40B-A2.5B
GGUF · Q3_K_M · 20.6 GiB
Kolibri-1 78B-A3.5B
GGUF · IQ1_M · 20.9 GiB
RTX 5090
Qwen3 4B Instruct
GGUF · Q6_K · 4.0 GiB
GLM-4-9B-Chat
GGUF · Q8_0 · 10.2 GiB
DeepSeek-Coder-V2-Lite Instruct
GGUF · Q8_0 · 17.6 GiB
Recursive Flint 40B-A2.5B
GGUF · Q4_K_M · 26.7 GiB
Kolibri-1 78B-A3.5B
GGUF · IQ2_XS · 25.6 GiB