Skip to content

Can RTX 4090 run Qwen3 32B Instruct?

Comfortable: AWQ INT4 needs 18.2 GB at 4K, batch 1; headroom 5.8 GB.

Text input, FP16 KV cache, one card. Unified-memory capacity uses the GPU guide’s usable limit; negative headroom may require host offload. Speed source settings may differ from this 4K estimate.

Measured runs

FrameworkQuanttok/sVRAM used (GB)
llama.cppGGUF Q4_K_M4222.5

Quant memory at 4K

QuantTotal GBVerdictWeight sizing
GGUF Q3_K_M17.7ComfortableEstimated weights
GGUF Q4_K_M21.9TightEstimated weights
EXL2 3.5bpw16.1ComfortableEstimated weights
AWQ INT418.2ComfortableEstimated weights

Context capacity · AWQ INT4

ContextTotal GBVerdict
4K18.2Comfortable
16K21.5Tight
32K25.9Does not fit

Generation speed

Measured: 42 tok/s, GGUF Q4_K_M, llama.cpp

AWQ INT4: 65 tok/s · Bandwidth ceiling (not measured)

Other model / GPU pairs