Kimi Linear 48B-A3B Instruct
48B-A3BMoonshot Kimi Linear
Moonshot's hybrid linear-attention model: 49.1B parameters in total (the name rounds it to 48B), about 3B active, a 1M-token window. Only 7 of its 27 layers keep a growing cache, and those store MLA's compressed vectors, so even the full million tokens needs about 8 GB of cache — the weights, not the context, decide what it runs on. At Q4 a Mac with 48 GB or more runs it comfortably; a 32 GB card holds it only with no room to spare. Sizes below use the calculator's generic rates: no GGUF file size could be checked, and quality loss per level has not been published.
1049K
Max Context
2
Quant Variants
GGUF Q8_0
Best Quality
—
Perplexity rise (best build)
Quantization Variants
Per-quant VRAM, quality loss, and inference speed on RTX 4090
Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports
| Format | Level | BPW | VRAM | PPL Loss | Speed | Source | Actions |
|---|---|---|---|---|---|---|---|
| GGUF | Q4_K_M | 4.85 | 31.1 GB | — | — | Estimated | |
| GGUF | Q8_0 | 8.5 | 54.6 GB | — | — | Estimated |
Which file each size is for
A file size is not a memory figure: the VRAM column is the calculator's total at 4K context — weights plus KV cache plus a 10% activation buffer. Units differ on purpose: file sizes are decimal GB, as Hugging Face lists them; the VRAM column is GiB (1024³ bytes) with 2% added to the weights, so 7.12 GB of file counts as about 6.8 GiB there. A vision projector (mmproj), where the repo ships one, is a separate download and is in neither number.
- GGUF Q4_K_MSized at the calculator's generic rate for this level; no file was checked.
- GGUF Q8_0Sized at the calculator's generic rate for this level; no file was checked.
Related community runs
These runs describe their own settings. Their speeds do not replace this estimate or prove your configuration fits.
Our reviewed link index has no run for this model yet. You can browse community reports or share your own test below.
Browse more runs on llamaPerf ↗Similar models
Compare with Llama 3.1Llama 3.1 70B Instruct
Meta Llama 3.1
Meta's frontier 70B model. Requires 40GB+ VRAM; dual 3090 or M2 Ultra.
Llama 3.3 70B Instruct
Meta Llama 3.3
Latest Meta 70B with improved multilingual. Drop-in upgrade from Llama 3.1 70B.
Jamba 1.5 Mini
AI21 Labs
AI21's hybrid Mamba-Transformer MoE: 52B total, about 12B active, 256K context. Only 4 of its 32 layers keep a KV cache, so long context is cheap — but all 52B of weights still have to be resident, about 31 GB at Q4. That is a 32 GB card or a large Mac, not the 16 GB card this entry used to claim. Sizes use the calculator's generic rates; no speed has been measured here.
GPT-OSS 120B
OpenAI GPT-OSS
The big GPT-OSS (117B total / 5.1B active). Native MXFP4 checkpoint is ~61GB — fits one 80GB card or a 128GB unified-memory Mac. Partial offload on 24GB consumer cards is slow but works.
Running Kimi Linear 48B-A3B Instruct locally
At Q4_K_M and 4K of context, Kimi Linear 48B-A3B Instruct needs about 31.1 GB — 28.3 GB of weights, 0.03 GB of KV cache and a 2.8 GB activation buffer. The smallest card in this index that clears that comfortably is the A100 40G at 40 GB, and 20 of the 78 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.
What longer context costs
Going from 4K to 32K adds about 0.22 GB, taking the total to 31.4 GB. That is gentler than the parameter count suggests: only 7 of 27 layers keep a KV cache that grows with context, so the cache scales at a fraction of the usual rate. The model's native window is 1M; holding all of it at Q4_K_M would need about 40 GB.
Which build to download
This index only tracks Kimi Linear 48B-A3B Instruct in GGUF, across 2 levels (Q4_K_M, Q8_0). No per-level perplexity sweep has been published for this model, so the quality column is empty rather than estimated — within one model, more bits per weight is the only ordering the data supports. No throughput has been measured for this model on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.
Common questions
- How much VRAM does Kimi Linear 48B-A3B Instruct need?
- About 31.1 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 31.4 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
- Will Kimi Linear 48B-A3B Instruct run on a 40GB GPU?
- Yes — at Q4_K_M and 4K context it needs about 31.1 GB, which leaves 8.9 GB spare on a A100 40G. That is the smallest card in this index that clears it comfortably; 20 of 78 do.
- Which quantization of Kimi Linear 48B-A3B Instruct should I use?
- No published quality comparison exists for this model, so pick by footprint: Q4_K_M is the level most people run, and a higher bits-per-weight level is more faithful. All 2 levels in the index are Q4_K_M, Q8_0. Note this is a mixture-of-experts model — all parameters must be resident even though only a fraction are active per token, so the memory cost follows the total, not the active count.
Where this model fits
Sized at Q4_K_M with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.
- Runs comfortably on
- A100 40G40GB · +8.9Mac M4 Pro 48G48GB · +4.9Mac M3 Max 48G48GB · +4.9A40 48G48GB · +16.9L40S 48G48GB · +16.9
- Tight but possible
- RTX 509032GBArc Pro B70 32G32GBRadeon AI PRO R9700 32G32GB
- Ships in
- GGUF
- Try it yourself
- Size it in the calculatorGenerate the run command
How to actually run this
Deployment guides for this model and this class of hardware.
This page's figures change when the model or the runtime does.Last updated 2026-11-15 RSS → /feed.xml