Skip to content

Can RTX 4090 run DeepSeek-R1-Distill-Qwen-14B?

Comfortable: EXL2 4.65bpw needs 9.8 GB at 4K, batch 1; headroom 14.3 GB.

Text input, FP16 KV cache, one card. Unified-memory capacity uses the GPU guide’s usable limit; negative headroom may require host offload. Speed source settings may differ from this 4K estimate.

Measured runs

FrameworkQuanttok/sVRAM used (GB)
ExLlamaV2EXL2 4.65bpw1289.8

Quant memory at 4K

QuantTotal GBVerdictWeight sizing
GGUF Q4_K_M10.1ComfortableEstimated weights
EXL2 4.65bpw9.8ComfortableEstimated weights
AWQ INT48.5ComfortableEstimated weights

Context capacity · EXL2 4.65bpw

ContextTotal GBVerdict
4K9.8Comfortable
16K12.2Comfortable
32K15.5Comfortable
128K35.3Does not fit

Generation speed

Measured: 128 tok/s, EXL2 4.65bpw, ExLlamaV2

EXL2 4.65bpw: 128 tok/s · Measured

Other model / GPU pairs