Ling-3.1-flash
560B-A25BinclusionAI Ling
inclusionAI's Ling-3.1-flash (Ant Group), released in October under MIT: a 560B mixture-of-experts reasoning model with 25B parameters active per token. It is aimed at agents, coding and long office tasks. Only 9 of its 72 layers keep a growing cache, and those store MLA's compressed vectors, so even its full 256K window adds only about 3 GB: the weights decide where it runs. At Q4_K_M it needs about 371 GB at 4K context, more than any single GPU in this index. The 512 GB Mac Studio gives its GPU about 384 GB, which holds it only tightly on this site's rule; on servers the vendor's reference setup is eight H200s or four GB300s running FP8. Q8_0 needs about 628 GB. The vendor quotes a 1M-token window, but the released config sets 262,144 positions with no scaling, so that is the limit used here. llama.cpp runs it as `bailingmoe3` from build b11571. Sizes come from a published GGUF file, whose maker reports KL divergence rather than perplexity, so no quality-loss figure is shown.
262K
Max Context
2
Quant Variants
GGUF Q8_0
Best Quality
—
Accuracy Retained
Quantization Variants
Per-quant VRAM, quality loss, and inference speed on RTX 4090
Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports
Similar models
Compare with DBRX InstructDBRX Instruct
Databricks
MoE flagship (~36B active). Needs multi-GPU; strong code and reasoning at scale.
Qwen3 235B-A22B Instruct
Alibaba Qwen3
Qwen3 flagship MoE (22B active / 235B total). Q4_K_M ~142GB; rivals DeepSeek-R1 class models.
DeepSeek-V3
DeepSeek
DeepSeek-V3 frontier MoE (~37B active / 671B total). MLA + FP8; multi-node GPU cluster required at Q4. MLA cache: a GGUF converted by current llama.cpp (attn_k_b / attn_v_b) holds about 2 GB at 32K, which the estimates here assume; an older conversion with a single attn_kv_b tensor caches full keys and values — about 150 GB at 32K.
DeepSeek-R1
DeepSeek
DeepSeek-R1 reasoning model built on V3 MoE. Chain-of-thought at frontier scale — use distill variants for local GPUs. MLA cache: a GGUF converted by current llama.cpp (attn_k_b / attn_v_b) holds about 2 GB at 32K, which the estimates here assume; an older conversion with a single attn_kv_b tensor caches full keys and values — about 150 GB at 32K.
Running Ling-3.1-flash locally
At Q4_K_M and 4K of context, Ling-3.1-flash needs about 370.6 GB — 336.8 GB of weights, 0.04 GB of KV cache and a 33.7 GB activation buffer. No card in this index clears that comfortably at Q4_K_M, so at that level it is a multi-GPU or CPU-offload proposition. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.
What longer context costs
Going from 4K to 32K adds about 0.28 GB, taking the total to 370.9 GB. That is gentler than the parameter count suggests: only 9 of 72 layers keep a KV cache that grows with context, so the cache scales at a fraction of the usual rate. The model's native window is 256K; holding all of it at Q4_K_M would need about 373 GB.
Which build to download
This index only tracks Ling-3.1-flash in GGUF, across 2 levels (Q4_K_M, Q8_0). No per-level perplexity sweep has been published for this model, so the quality column is empty rather than estimated — within one model, more bits per weight is the only ordering the data supports. No throughput has been measured for this model on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.
Common questions
- How much VRAM does Ling-3.1-flash need?
- About 370.6 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 370.9 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
- Can Ling-3.1-flash run on a single GPU?
- Not comfortably on any single card in this index at Q4_K_M. It needs about 370.6 GB, which means splitting across GPUs or offloading layers to system RAM.
- Which quantization of Ling-3.1-flash should I use?
- No published quality comparison exists for this model, so pick by footprint: Q4_K_M is the level most people run, and a higher bits-per-weight level is more faithful. All 2 levels in the index are Q4_K_M, Q8_0. Note this is a mixture-of-experts model — all parameters must be resident even though only a fraction are active per token, so the memory cost follows the total, not the active count.
Where this model fits
Sized at Q4_K_M with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.
- Tight but possible
- Mac M5 Ultra 512G512GB
- Ships in
- GGUF
- Try it yourself
- Size it in the calculatorGenerate the run command
How to actually run this
Deployment guides for this model and this class of hardware.
This page's figures change when the model or the runtime does.Last updated 2026-10-18 RSS → /feed.xml