DBRX Instruct

132B

Databricks

MoE flagship (~36B active). Needs multi-GPU; strong code and reasoning at scale.

Pro GPU

33K

Max Context

Quant Variants

GGUF Q4_K_M

Best Quality

97.5%

Accuracy Retained

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = site benchmarks · Estimated = formula · Community = public reports

Format	Level	BPW	VRAM	PPL Loss	Speed	Source	Actions
GGUF	Q4_K_M	4.85	78.5 GB	2.5%	15 tok/s	Estimated	Calc HF
GGUF	Q3_K_M	3.87	63.2 GB	5.8%	18 tok/s	Estimated	Calc HF

Alibaba Qwen3

Qwen3 flagship MoE (22B active / 235B total). Q4_K_M ~142GB; rivals DeepSeek-R1 class models.

DeepSeek

DeepSeek-V3 frontier MoE (~37B active / 671B total). MLA + FP8; multi-node GPU cluster required at Q4.

DeepSeek

DeepSeek-R1 reasoning model built on V3 MoE. Chain-of-thought at frontier scale — use distill variants for local GPUs.

Mistral AI

Mistral 3 flagship MoE (41B active / 675B total) with vision encoder. FP8 on 8×H200; GGUF quant for research clusters only.