Gemma 2 9B Instruct

Google Gemma 2

Google's compact Gemma 2 with sliding window attention. Punches above 9B.

⬇ 25.7K HF downloads♥ 231 likesbartowski/gemma-2-9b-it-GGUF· stats from 6/24/2026

Consumer GPUMac / Apple Silicon

Max Context

Quant Variants

GGUF Q8_0

Best Quality

99.8%

Accuracy Retained

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Format	Level	BPW	VRAM	PPL Loss	Speed	Actions
GGUF	Q4_K_M	4.85	6.5 GB	3.3%	132 tok/s	Calc HF
GGUF	Q8_0	8.5	10.2 GB	0.2%	108 tok/s	Calc HF
AWQ	INT4	4	5.8 GB	4.6%	188 tok/s	Calc HF