Skip to content

Qwen3-VL 8B Instruct

8B

Alibaba Qwen3-VL

Current-generation vision-language model that still fits a single 8–12GB card at Q4 (~5.9GB). The realistic multimodal option for people without a 24GB GPU — note the vision encoder adds VRAM the KV-cache math below does not model.

⬇ 6.1M HF downloads♥ 1274 likesQwen/Qwen3-VL-8B-Instruct· stats from 11/22/2026
Consumer GPUMac / Apple SiliconCPU / VPS

262K

Max Context

4

Quant Variants

GGUF Q8_0

Best Quality

+0.3%

Perplexity rise (best build)

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports

FormatLevelBPWVRAMPPL LossSpeedSourceActions
GGUFQ4_K_M4.856.2 GB3.0%140 tok/sCommunity
CalcHF
GGUFQ5_K_M5.687.2 GB1.5%126 tok/sEstimated
CalcHF
GGUFQ8_08.510.4 GB0.3%108 tok/sEstimated
CalcHF
AWQINT445.2 GB4.2%165 tok/sEstimated
CalcHF

Which file each size is for

A file size is not a memory figure: the VRAM column is the calculator's total at 4K context — weights plus KV cache plus a 10% activation buffer. Units differ on purpose: file sizes are decimal GB, as Hugging Face lists them; the VRAM column is GiB (1024³ bytes) with 2% added to the weights, so 7.12 GB of file counts as about 6.8 GiB there. A vision projector (mmproj), where the repo ships one, is a separate download and is in neither number.

  • GGUF Q4_K_MSized at the calculator's generic rate for this level; no file was checked.
  • GGUF Q5_K_MSized at the calculator's generic rate for this level; no file was checked.
  • GGUF Q8_0Sized at the calculator's generic rate for this level; no file was checked.
  • AWQ INT4Sized at the calculator's generic rate for this level; no file was checked.

Related community runs

These runs describe their own settings. Their speeds do not replace this estimate or prove your configuration fits.

Our reviewed link index has no run for this model yet. You can browse community reports or share your own test below.

Browse more runs on llamaPerf ↗
30B-A3B

Qwen3-VL 30B-A3B Instruct

Alibaba Qwen3-VL

Consumer GPUPro GPU
Q4_K_M19.9 GB+2.6%perplexity rise

Multimodal MoE with only ~3B active parameters, so it stays responsive on Apple unified memory and survives CPU offload far better than a dense 30B. At Q4_K_M it needs about 19.9 GB at 4K, comfortable on a 24 GB card; Q3_K_M brings that to 15.1 GB. The vision encoder is a separate 1.1 GB file.

9B

Qwen3.5 9B

Alibaba Qwen3.5

Consumer GPUMac / Apple Silicon
Q4_K_M6.6 GB—perplexity rise

The newest small Qwen and a common pick for 8 to 16 GB cards. Only 8 of its 32 layers keep a growing KV cache, so it can use a long context on modest hardware. At Q4_K_M it needs about 6.6 GB at 4K and 7.0 GB at 16K, comfortable on an 8 GB card; at 32K (7.5 GB) it is tight there and comfortable from 10 GB up. Its full 256K window needs about 15.2 GB. Q8_0 needs about 10.4 GB, comfortable from 12 GB up. Reasoning is off by default on the small Qwen3.5 sizes, and it reads images through a separate vision encoder. Sizes come from the published GGUF file sizes, and no per-level quality loss has been published.

9B

MiMo-V2.6-Distill-Qwen-9B

Xiaomi MiMo

Consumer GPUMac / Apple Silicon
Q4_K_M6.3 GB—perplexity rise

Xiaomi's MiMo-V2.6 distilled into the Qwen3.5 9B architecture, released in September, with image input through a separate vision encoder. Only 8 of its 32 layers keep a cache that grows with context. At Q4_K_M it needs about 6.3 GB at 4K and 6.7 GB at 16K, comfortable on an 8 GB card; at 32K (7.2 GB) it is tight on an 8 GB card and comfortable from 10 GB up. It needs about 10.5 GB at 128K, comfortable on a 12 GB card. Q8_0 needs about 10.1 GB at 4K, comfortable on a 12 GB card. The vision encoder adds about 0.9 GB. Sizes come from bartowski's GGUF files, and no per-level quality loss has been published.

14B

Ministral 3 14B Instruct

Mistral Ministral 3

Consumer GPUMac / Apple Silicon
Q4_K_M9.3 GB—perplexity rise

The larger Ministral 3 instruction model under Apache 2.0, with text and image input. Its language backbone has 40 full-attention layers, 8 KV heads of 128 dimensions and a 256K window. Parameters count the language model reconstructed from the official config; its separate vision encoder is recorded as an additional download, not folded into the language weights. Long-context and image use need more memory than the GGUF file alone. No per-level perplexity loss has been published.

Running Qwen3-VL 8B Instruct locally

At Q4_K_M and 4K of context, Qwen3-VL 8B Instruct needs about 6.2 GB — 5.1 GB of weights, 0.56 GB of KV cache and a 0.6 GB activation buffer. The smallest cards in this index that clear that comfortably are 8 GB ones — 21 of them, from the RTX 4060 up — and 98 of the 105 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.

What longer context costs

Going from 4K to 32K adds about 3.94 GB, taking the total to 10.5 GB. Weights do not move with context — only the KV cache does, and it grows linearly, so this is the number to watch when planning for long documents. The model's native window is 256K; holding all of it at Q4_K_M would need about 45 GB.

Which build to download

This index tracks 2 formats for it — GGUF, AWQ — across 4 levels. Q8_0 carries the lowest published perplexity loss at 0.3%. The fastest level listed is AWQ INT4 at about 165 tok/s on an RTX 4090, batch 1 — an estimate, not a run on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.

Common questions

How much VRAM does Qwen3-VL 8B Instruct need?
About 6.2 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 10.5 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
Will Qwen3-VL 8B Instruct run on a 8GB GPU?
Yes — at Q4_K_M and 4K context it needs about 6.2 GB, which leaves 1.8 GB spare on a RTX 4060, or on any of the other 8 GB cards here (21 in all). That is the smallest size in this index that clears it comfortably; 98 of 105 do.
Which quantization of Qwen3-VL 8B Instruct should I use?
Q8_0 has the lowest published quality loss (0.3%), and Q4_K_M is the level most people run. All 4 levels in the index are Q4_K_M, Q5_K_M, Q8_0, AWQ INT4.

Where this model fits

Sized at Q4_K_M with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.

Ships in
GGUFAWQ
Compared here
GGUF vs AWQ

Cards for this model

    This page's figures change when the model or the runtime does.Last updated 2026-11-22 RSS → /feed.xml