Llama 3.1 70B Instruct

70B

Meta Llama 3.1

Meta's frontier 70B model. Requires 40GB+ VRAM; dual 3090 or M2 Ultra.

⬇ 15.6K HF downloads♥ 71 likesbartowski/Meta-Llama-3.1-70B-Instruct-GGUF· stats from 6/24/2026

Pro GPUMac / Apple Silicon

131K

Max Context

Quant Variants

GGUF Q5_K_M

Best Quality

98.8%

Accuracy Retained

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Format	Level	BPW	VRAM	PPL Loss	Speed	Actions
GGUF	Q4_K_M	4.85	43.5 GB	2.8%	38 tok/s	Calc HF
GGUF	Q5_K_M	5.68	49.8 GB	1.2%	32 tok/s	Calc HF
AWQ	INT4	4	38.2 GB	3.9%	55 tok/s	Calc HF
EXL2	3.5bpw	3.5	33.4 GB	5.2%	62 tok/s	Calc HF