Tools

Estimate VRAM for any quantized LLM — or any model on Hugging Face — with a verdict for your card or mixed cards, the speed ceiling, the wait before the first token, how many people it can serve, and the memory to fine-tune it.

Generate runnable llama.cpp, Ollama, vLLM and ExLlamaV2 commands for any indexed model, including Docker and Compose variants.

Answer three questions about your hardware and priorities to get a quantization format, runtime and quant level that actually run together.

Put two quantized models side by side: parameters, context, VRAM at your chosen context length, speed and quality loss.

Stream one sentence at a card's measured or scaled token rate, side by side with another card. Bandwidth ceilings are labelled and are not played back as a speed.

Tokens per second for the models that fit the most common cards: a calibrated estimate from each card's own measured run, and the upper bound its memory bandwidth sets.

Which quant level to download for each recommended model on the most common cards: every level sized at 4K context, and the highest-quality one that fits comfortably.

The longest context each recommended model keeps comfortably on the most common cards, with the default fp16 KV cache and with the cache quantized to q8_0 or q4_0.

The messages llama.cpp, Ollama, vLLM, ROCm and Docker print when a model will not load or will not use the GPU — each quoted from the project's own source, with what it means and what fixes it.

One page per card — 78 of them — listing which of the 125 indexed models fit it comfortably at 4K context, at what quant level, with how much headroom.

4 head-to-head pages — GGUF vs AWQ, GGUF vs EXL2 and the rest — on hardware support, runtime and the models this index ships in both formats.