Best Local LLMs for 8–32 GB GPUs
Compare up to three catalog picks per GPU memory tier for chat, coding and multimodal tasks, with quantization levels and estimated memory at 4K context.
Memory estimates at 4,096 tokens for one text sequence.
For each task, these NVIDIA tiers favor a larger model that leaves memory headroom. A model selected for two tasks appears once. Estimates assume one text sequence. Image inputs also need encoder memory.
8 GB
Chat / Code
Falcon 3 10B InstructGPTQ · INT4 · 6.1 GiBMultimodal
Qwen3.5 9BGGUF · Q4_K_M · 6.6 GiB
12 GB
Chat
Hy-Memory-30B-A3BGGUF · Q8_0-expsIQ2_XXS · 9.9 GiBCode
DeepSeek-V2-Lite ChatGGUF · Q4_K_M · 10.1 GiBMultimodal
Ternary Bonsai 2 27BGGUF · Q2_0 · 8.2 GiB
16 GB
Chat
Xing4.0 29B-A4BGGUF · Q2_K · 13.6 GiBCode / Multimodal
Arete 27BGGUF · Q3_K_M · 14.0 GiB
24 GB
Chat
Kolibri-1 78B-A3.5BGGUF · IQ1_M · 20.9 GiBCode
Ripple-65B-PreviewGGUF · IQ1_M · 20.0 GiBMultimodal
Apex 35B V1.0GGUF · Q3_K_M · 18.7 GiB
32 GB
Chat
Kolibri-1 78B-A3.5BGGUF · IQ2_XS · 25.6 GiBCode / Multimodal
Nemotron 3.5 Bolt 74B-A9BGGUF · IQ2_XXS · 23.1 GiB