Why does the smaller model sometimes need more VRAM?

Its KV cache. A model with more layers or more KV heads carries a heavier cache per token, and at long context that can outweigh a difference in parameter count. Drop the context slider and watch the ordering change.

Are the speed numbers comparable across frameworks?

Roughly, and only at batch 1. The benchmarks page lists the exact framework versions and test parameters; throughput rankings change substantially once you batch requests, which is where vLLM pulls ahead.

Can I share a comparison?

Yes — both models and the context length are kept in the URL, so copying the address bar shares exactly what you are looking at, in the language you are reading it in.

⚖️ Model Compare

Model A vs B Comparison

Side-by-side VRAM, speed, quality, and GPU fit — pick the right model for your hardware.

Model A

Model B

Context length

Select two models above to compare

What the columns mean

Every row is labelled with how its number was arrived at. Estimated rows are recomputed by the same function the calculator uses, at the context length you set — two models can swap places as context grows if their KV cache shapes differ. Published rows are fixed figures from the release and do not move with the controls. Spec rows are properties of the model. Speed is tokens per second on an RTX 4090 at batch 1. Quality loss is perplexity increase against the unquantized weights, so lower is better, and the absolute value is only comparable within one model.

Comparing across families is harder than it looks

Perplexity is measured on a fixed corpus, and models trained on different data start from different baselines — a 2% loss on one model and a 2% loss on another do not represent the same amount of degradation. Use the figure to choose a quant level within a model, and use the model index and your own evaluation to choose between models.

Common questions

Why does the smaller model sometimes need more VRAM?: Its KV cache. A model with more layers or more KV heads carries a heavier cache per token, and at long context that can outweigh a difference in parameter count. Drop the context slider and watch the ordering change.
Are the speed numbers comparable across frameworks?: Roughly, and only at batch 1. The benchmarks page lists the exact framework versions and test parameters; throughput rankings change substantially once you batch requests, which is where vLLM pulls ahead.
Can I share a comparison?: Yes — both models and the context length are kept in the URL, so copying the address bar shares exactly what you are looking at, in the language you are reading it in.