Seed-OSS 36B Instruct

36B

ByteDance Seed

Dense 36B with a native 512K context. Q4 weights are ~22GB, which lands on a 24GB card or a 2×16GB split — but the context is the real cost: 512K of KV cache is roughly 128GB on its own, so budget context first and weights second.

⬇ 42.2K HF downloads♥ 505 likesByteDance-Seed/Seed-OSS-36B-Instruct· stats from 8/14/2026
Consumer GPUPro GPUMac / Apple Silicon

524K

Max Context

4

Quant Variants

GGUF Q5_K_M

Best Quality

98.7%

Accuracy Retained

Calculate VRAMHugging FaceCompare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = site benchmarks · Estimated = formula · Community = public reports

FormatLevelBPWVRAMPPL LossSpeedSourceActions
GGUFQ4_K_M4.8521.8 GB2.7%42 tok/sCommunity
CalcHF
GGUFQ3_K_M3.8717.4 GB5.6%48 tok/sEstimated
CalcHF
GGUFQ5_K_M5.6825.6 GB1.3%37 tok/sEstimated
CalcHF
AWQINT4419.5 GB3.8%55 tok/sEstimated
CalcHF

Similar models

Compare with Qwen2.5 32B
32B

Qwen2.5 32B Instruct

Alibaba Qwen2.5

Consumer GPUPro GPU
16.4 GBmin VRAM·97.3%accuracy

Near-GPT-4 reasoning on a 24GB VRAM card (Q4_K_S). Groundbreaking value.

32B

DeepSeek-R1-Distill-Qwen-32B

DeepSeek

Consumer GPUPro GPU
16.8 GBmin VRAM·97.4%accuracy

R1 distilled to 32B. Near-frontier reasoning on a single 24GB card (Q3/Q4).

32B

Qwen3 32B Instruct

Alibaba Qwen3

Consumer GPUPro GPU
16.8 GBmin VRAM·97.5%accuracy

Qwen3 dense 32B — successor to Qwen2.5-32B with stronger reasoning and thinking mode.

30B-A3B

Qwen3 30B-A3B Instruct

Alibaba Qwen3

Consumer GPUMac / Apple Silicon
17.5 GBmin VRAM·98.9%accuracy

Qwen3 MoE with only 3B active params. Q4 ~19GB file; outperforms QwQ-32B on 16GB cards.

How to actually run this

Deployment guides for this model and this class of hardware.