Jamba 1.5 Mini

12B

AI21 Labs

Hybrid SSM-Transformer with 256K context. Efficient long-document QA on 16GB.

Consumer GPUMac / Apple Silicon

262K

Max Context

2

Quant Variants

GGUF Q4_K_M

Best Quality

96.6%

Accuracy Retained

Calculate VRAMHugging FaceCompare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = site benchmarks · Estimated = formula · Community = public reports

FormatLevelBPWVRAMPPL LossSpeedSourceActions
GGUFQ4_K_M4.858.5 GB3.4%95 tok/sEstimated
CalcHF
AWQINT447.5 GB4.8%125 tok/sEstimated
CalcHF

Similar models

Compare with Qwen2.5 14B
14B

Qwen2.5 14B Instruct

Alibaba Qwen2.5

Consumer GPUMac / Apple Silicon
9.2 GBmin VRAM·98.6%accuracy

The sweet spot between performance and resource usage. 16GB VRAM with Q4.

12B

Mistral Nemo 12B Instruct

Mistral AI

Consumer GPUMac / Apple Silicon
7.8 GBmin VRAM·99.1%accuracy

Mistral + NVIDIA collaboration. 128K context, excellent multilingual support.

9B

Gemma 2 9B Instruct

Google Gemma 2

Consumer GPUMac / Apple Silicon
5.8 GBmin VRAM·99.8%accuracy

Google's compact Gemma 2 with sliding window attention. Punches above 9B.

11B

Solar 10.7B Instruct

Upstage

Consumer GPUMac / Apple Silicon
6.5 GBmin VRAM·97.0%accuracy

Depth-upscaled 10.7B punching above weight. Strong on reasoning benchmarks.