Does this include the operating system and display overhead?

No. It estimates what the model needs. On a card that is also driving a desktop, subtract roughly 0.5–1.5 GB before comparing — which is part of why the green threshold stops at 88% rather than 100%.

Why does the same quant level give different sizes for different models?

Because bits-per-weight is an average over a mixed scheme. `Q4_K_M` keeps some tensors at higher precision, and how many depends on the architecture — an MoE model with most of its parameters in experts quantizes differently from a dense one. Where a model in the index ships that level, the calculator uses its measured figure instead of the generic one.

Can I run a model that shows amber?

Often yes, with less context or a smaller batch — both shrink the KV cache directly. Lower the context slider and watch the total fall. If it is still amber at short context, the weights alone are the problem and you need a smaller quant level.

What does reverse mode do differently?

Forward mode answers "will this model fit on my card". Reverse mode starts from the card and lists every model × quant pair in the index that fits, sorted by quality, speed or footprint. Both size from the same per-model figures, so the two views agree.

Are these numbers measured or estimated?

The VRAM figure is computed, not measured. The per-quant `vramGB` and speed values in the model index carry a confidence marker — measured, estimated or community — and the benchmarks page documents the hardware and framework versions behind the measured ones.

⚡ VRAM Calc

VRAM / RAM Calculator

Precise memory estimate for any model × quant × context combination

Pick a model and see if it fits your hardware

Model

Quantization Format

Context Length (tokens)4K

Batch Size1

14816

Select a model and format above to see memory estimate

How the estimate is calculated

Three terms are added. Model weights are params × bpw ÷ 8, converted to GiB, plus 2% for embedding and norm tensors that most quantizers keep at higher precision. KV cache is 2 (K and V) × layers × kv_heads × head_dim × context × batch × 2 bytes, the last factor being fp16 cache. Activation buffer is a flat 10% of the first two, covering the transient tensors an inference runtime allocates per forward pass. When you pick an indexed model the calculator uses that model's own measured bits-per-weight rather than the generic per-level table, because the two can differ sharply — GPT-OSS ships mostly-MXFP4 weights, so its Q8_0 is 5.10 bpw, not 8.5.

Why context length matters more than you expect

Weights are fixed once you pick a quant level; the KV cache is not. It grows linearly with context and with batch size, and on long-context models it can overtake the weights entirely. The lever that decides how steep that growth is is grouped-query attention: kv_heads is often far smaller than the number of attention heads. Llama 3.1 8B has 32 attention heads but only 8 KV heads, so its cache is a quarter of what multi-head attention would cost. Two models of identical size can therefore have very different memory curves, which is why this calculator reads layers, kvHeads and headDim from each model's real config rather than assuming a shape.

How close is it to reality

For Llama 3.1 8B at Q4_K_M the estimate is 4.62 GB against a 4.58 GiB file from bartowski — close enough to plan a card purchase around. Treat it as an estimate all the same. Runtimes differ in how much they reserve: llama.cpp's compute buffer scales with batch and context, vLLM pre-allocates a fixed fraction of the card (--gpu-memory-utilization, 0.85 by default in our generated commands), and CUDA itself takes a few hundred MB of context before any weights load. The verdict colours build that in — green means the estimate uses at most 88% of the card, amber up to 105%, so a green result already has headroom.

Common questions

Does this include the operating system and display overhead?: No. It estimates what the model needs. On a card that is also driving a desktop, subtract roughly 0.5–1.5 GB before comparing — which is part of why the green threshold stops at 88% rather than 100%.
Why does the same quant level give different sizes for different models?: Because bits-per-weight is an average over a mixed scheme. Q4_K_M keeps some tensors at higher precision, and how many depends on the architecture — an MoE model with most of its parameters in experts quantizes differently from a dense one. Where a model in the index ships that level, the calculator uses its measured figure instead of the generic one.
Can I run a model that shows amber?: Often yes, with less context or a smaller batch — both shrink the KV cache directly. Lower the context slider and watch the total fall. If it is still amber at short context, the weights alone are the problem and you need a smaller quant level.
What does reverse mode do differently?: Forward mode answers "will this model fit on my card". Reverse mode starts from the card and lists every model × quant pair in the index that fits, sorted by quality, speed or footprint. Both size from the same per-model figures, so the two views agree.
Are these numbers measured or estimated?: The VRAM figure is computed, not measured. The per-quant vramGB and speed values in the model index carry a confidence marker — measured, estimated or community — and the benchmarks page documents the hardware and framework versions behind the measured ones.