Kimi Linear 48B-A3B Instruct
48B-A3BMoonshot Kimi Linear
Moonshot's hybrid linear-attention model: 48B total, 3B active, a 1M-token window. Only 7 of its 27 layers keep a growing cache, and those store MLA's compressed vectors, so even the full million tokens needs about 8 GB of cache — the weights, not the context, decide what it runs on. At Q4 a Mac with 48 GB or more runs it comfortably; a 32 GB card holds it only with no room to spare. Sizes below use the calculator's generic rates: no GGUF file size could be checked, and quality loss per level has not been published.
1049K
Max Context
2
Quant Variants
GGUF Q8_0
Best Quality
—
Accuracy Retained
Quantization Variants
Per-quant VRAM, quality loss, and inference speed on RTX 4090
Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports
Similar models
Compare with Llama 3.1Llama 3.1 70B Instruct
Meta Llama 3.1
Meta's frontier 70B model. Requires 40GB+ VRAM; dual 3090 or M2 Ultra.
Llama 3.3 70B Instruct
Meta Llama 3.3
Latest Meta 70B with improved multilingual. Drop-in upgrade from Llama 3.1 70B.
Jamba 1.5 Mini
AI21 Labs
AI21's hybrid Mamba-Transformer MoE: 52B total, about 12B active, 256K context. Only 4 of its 32 layers keep a KV cache, so long context is cheap — but all 52B of weights still have to be resident, about 31 GB at Q4. That is a 32 GB card or a large Mac, not the 16 GB card this entry used to claim. Sizes use the calculator's generic rates; no speed has been measured here.
GPT-OSS 120B
OpenAI GPT-OSS
The big GPT-OSS (117B total / 5.1B active). Native MXFP4 checkpoint is ~61GB — fits one 80GB card or a 128GB unified-memory Mac. Partial offload on 24GB consumer cards is slow but works.
Running Kimi Linear 48B-A3B Instruct locally
At Q4_K_M and 4K of context, Kimi Linear 48B-A3B Instruct needs about 30.4 GB — 27.6 GB of weights, 0.03 GB of KV cache and a 2.8 GB activation buffer. The smallest card in this index that clears that comfortably is the A100 40G at 40 GB, and 17 of the 59 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.
What longer context costs
Going from 4K to 32K adds about 0.22 GB, taking the total to 30.7 GB. That is gentler than the parameter count suggests: only 7 of 27 layers keep a KV cache that grows with context, so the cache scales at a fraction of the usual rate. The model's native window is 1M; holding all of it at Q4_K_M would need about 39 GB.
Which build to download
This index only tracks Kimi Linear 48B-A3B Instruct in GGUF, across 2 levels (Q4_K_M, Q8_0). No per-level perplexity sweep has been published for this model, so the quality column is empty rather than estimated — within one model, more bits per weight is the only ordering the data supports. No throughput has been measured for this model on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 and CUDA only.
Common questions
- How much VRAM does Kimi Linear 48B-A3B Instruct need?
- About 30.4 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 30.7 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
- Will Kimi Linear 48B-A3B Instruct run on a 40GB GPU?
- Yes — at Q4_K_M and 4K context it needs about 30.4 GB, which leaves 9.6 GB spare on a A100 40G. That is the smallest card in this index that clears it comfortably; 17 of 59 do.
- Which quantization of Kimi Linear 48B-A3B Instruct should I use?
- No published quality comparison exists for this model, so pick by footprint: Q4_K_M is the level most people run, and a higher bits-per-weight level is more faithful. All 2 levels in the index are Q4_K_M, Q8_0. Note this is a mixture-of-experts model — all parameters must be resident even though only a fraction are active per token, so the memory cost follows the total, not the active count.
Where this model fits
Sized at Q4_K_M with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.
- Runs comfortably on
- A100 40G40GB · +9.6Mac M4 Pro 48G48GB · +5.6Mac M3 Max 48G48GB · +5.6A40 48G48GB · +17.6L40S 48G48GB · +17.6
- Tight but possible
- RTX 509032GBInstinct MI100 32G32GB
- Ships in
- GGUF
- Try it yourself
- Size it in the calculatorGenerate the run command
How to actually run this
Deployment guides for this model and this class of hardware.
This page's figures change when the model or the runtime does.Last updated 2026-09-30 RSS → /feed.xml