DeepSeek-R1-Distill-Llama-8B

DeepSeek

R1 reasoning distilled into Llama 3.1 8B. Best chain-of-thought for 8–12GB cards; huge community GGUF support.

⬇ 9.2K HF downloads♥ 53 likesbartowski/DeepSeek-R1-Distill-Llama-8B-GGUF· stats from 7/27/2026

Consumer GPUMac / Apple SiliconCPU / VPS

131K

Max Context

Quant Variants

GGUF Q5_K_M

Best Quality

98.8%

Accuracy Retained

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = site benchmarks · Estimated = formula · Community = public reports

Format	Level	BPW	VRAM	PPL Loss	Speed	Source	Actions
GGUF	Q4_K_M	4.85	5.6 GB	2.6%	145 tok/s	Community	Calc HF
GGUF	Q5_K_M	5.68	6.5 GB	1.2%	128 tok/s	Estimated	Calc HF
EXL2	4.65bpw	4.65	5.3 GB	1.9%	210 tok/s	Estimated	Calc HF
AWQ	INT4	4	4.9 GB	3.5%	188 tok/s	Estimated	Calc HF

DeepSeek

R1 reasoning in a 7B footprint. Best value for 8–12GB VRAM CoT experiments.

DeepSeek

MoE general model (~2.4B active). Long context and strong multilingual chat.

DeepSeek

R1 reasoning distilled into 14B. Huge community interest; excellent chain-of-thought.

DeepSeek

R1 distilled to 32B. Near-frontier reasoning on a single 24GB card (Q3/Q4).