Qwen3-VL 8B Instruct
8BAlibaba Qwen3-VL
Current-generation vision-language model that still fits a single 8–12GB card at Q4 (~5.9GB). The realistic multimodal option for people without a 24GB GPU — note the vision encoder adds VRAM the KV-cache math below does not model.
262K
Max Context
4
Quant Variants
GGUF Q8_0
Best Quality
+0.3%
Perplexity rise (best build)
Quantization Variants
Per-quant VRAM, quality loss, and inference speed on RTX 4090
Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports
| Format | Level | BPW | VRAM | PPL Loss | Speed | Source | Actions |
|---|---|---|---|---|---|---|---|
| GGUF | Q4_K_M | 4.85 | 6.2 GB | 3.0% | 140 tok/s | Community | |
| GGUF | Q5_K_M | 5.68 | 7.2 GB | 1.5% | 126 tok/s | Estimated | |
| GGUF | Q8_0 | 8.5 | 10.4 GB | 0.3% | 108 tok/s | Estimated | |
| AWQ | INT4 | 4 | 5.2 GB | 4.2% | 165 tok/s | Estimated |
Which file each size is for
A file size is not a memory figure: the VRAM column is the calculator's total at 4K context — weights plus KV cache plus a 10% activation buffer. Units differ on purpose: file sizes are decimal GB, as Hugging Face lists them; the VRAM column is GiB (1024³ bytes) with 2% added to the weights, so 7.12 GB of file counts as about 6.8 GiB there. A vision projector (mmproj), where the repo ships one, is a separate download and is in neither number.
- GGUF Q4_K_MSized at the calculator's generic rate for this level; no file was checked.
- GGUF Q5_K_MSized at the calculator's generic rate for this level; no file was checked.
- GGUF Q8_0Sized at the calculator's generic rate for this level; no file was checked.
- AWQ INT4Sized at the calculator's generic rate for this level; no file was checked.
Related community runs
These runs describe their own settings. Their speeds do not replace this estimate or prove your configuration fits.
Our reviewed link index has no run for this model yet. You can browse community reports or share your own test below.
Browse more runs on llamaPerf ↗Similar models
Compare with Qwen3-VL 30B-A3BQwen3-VL 30B-A3B Instruct
Alibaba Qwen3-VL
Multimodal MoE with only ~3B active parameters, so it stays responsive on Apple unified memory and survives CPU offload far better than a dense 30B. At Q4_K_M it needs about 19.9 GB at 4K, comfortable on a 24 GB card; Q3_K_M brings that to 15.1 GB. The vision encoder is a separate 1.1 GB file.
Qwen3.5 9B
Alibaba Qwen3.5
The newest small Qwen and a common pick for 8 to 16 GB cards. Only 8 of its 32 layers keep a growing KV cache, so it can use a long context on modest hardware. At Q4_K_M it needs about 6.6 GB at 4K and 7.0 GB at 16K, comfortable on an 8 GB card; at 32K (7.5 GB) it is tight there and comfortable from 10 GB up. Its full 256K window needs about 15.2 GB. Q8_0 needs about 10.4 GB, comfortable from 12 GB up. Reasoning is off by default on the small Qwen3.5 sizes, and it reads images through a separate vision encoder. Sizes come from the published GGUF file sizes, and no per-level quality loss has been published.
MiMo-V2.6-Distill-Qwen-9B
Xiaomi MiMo
Xiaomi's MiMo-V2.6 distilled into the Qwen3.5 9B architecture, released in September, with image input through a separate vision encoder. Only 8 of its 32 layers keep a cache that grows with context. At Q4_K_M it needs about 6.3 GB at 4K and 6.7 GB at 16K, comfortable on an 8 GB card; at 32K (7.2 GB) it is tight on an 8 GB card and comfortable from 10 GB up. It needs about 10.5 GB at 128K, comfortable on a 12 GB card. Q8_0 needs about 10.1 GB at 4K, comfortable on a 12 GB card. The vision encoder adds about 0.9 GB. Sizes come from bartowski's GGUF files, and no per-level quality loss has been published.
Ministral 3 14B Instruct
Mistral Ministral 3
The larger Ministral 3 instruction model under Apache 2.0, with text and image input. Its language backbone has 40 full-attention layers, 8 KV heads of 128 dimensions and a 256K window. Parameters count the language model reconstructed from the official config; its separate vision encoder is recorded as an additional download, not folded into the language weights. Long-context and image use need more memory than the GGUF file alone. No per-level perplexity loss has been published.
Running Qwen3-VL 8B Instruct locally
At Q4_K_M and 4K of context, Qwen3-VL 8B Instruct needs about 6.2 GB — 5.1 GB of weights, 0.56 GB of KV cache and a 0.6 GB activation buffer. The smallest cards in this index that clear that comfortably are 8 GB ones — 21 of them, from the RTX 4060 up — and 98 of the 105 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.
What longer context costs
Going from 4K to 32K adds about 3.94 GB, taking the total to 10.5 GB. Weights do not move with context — only the KV cache does, and it grows linearly, so this is the number to watch when planning for long documents. The model's native window is 256K; holding all of it at Q4_K_M would need about 45 GB.
Which build to download
This index tracks 2 formats for it — GGUF, AWQ — across 4 levels. Q8_0 carries the lowest published perplexity loss at 0.3%. The fastest level listed is AWQ INT4 at about 165 tok/s on an RTX 4090, batch 1 — an estimate, not a run on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.
Common questions
- How much VRAM does Qwen3-VL 8B Instruct need?
- About 6.2 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 10.5 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
- Will Qwen3-VL 8B Instruct run on a 8GB GPU?
- Yes — at Q4_K_M and 4K context it needs about 6.2 GB, which leaves 1.8 GB spare on a RTX 4060, or on any of the other 8 GB cards here (21 in all). That is the smallest size in this index that clears it comfortably; 98 of 105 do.
- Which quantization of Qwen3-VL 8B Instruct should I use?
- Q8_0 has the lowest published quality loss (0.3%), and Q4_K_M is the level most people run. All 4 levels in the index are Q4_K_M, Q5_K_M, Q8_0, AWQ INT4.
Where this model fits
Sized at Q4_K_M with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.
- Runs comfortably on
- RTX 40608GB · +1.8RTX 4060 Ti 8G8GB · +1.8Radeon RX 9060 XT 8G8GB · +1.8RTX 5060 Ti 8G8GB · +1.8RTX 50608GB · +1.8
- Tight but possible
- RTX 4050 Laptop GPU6GBRTX 3060 Laptop GPU6GBRTX 3050 Laptop GPU 6G6GB
- Compared here
- GGUF vs AWQ
- Try it yourself
- Size it in the calculatorGenerate the run command
Cards for this model
How to actually run this
Deployment guides for this model and this class of hardware.
This page's figures change when the model or the runtime does.Last updated 2026-11-22 RSS → /feed.xml