Nemotron 3.5 Bolt E4B
E4BNVIDIA Nemotron
NVIDIA's Nemotron 3.5 Bolt E4B is a small vision model built on Google's Gemma 4 E4B. It has the same 42-layer shape, about 4.7B parameters doing the work plus a 2.8B per-layer embedding table, with image input and a 64K window. Only 4 of its 42 layers keep a cache that grows with context, so long context is cheap. The one plain Q4_K_M file (LM Studio's, 6.2 GB) is much larger than the level name suggests, because the embedding table is not stored at 4 bits. It needs about 6.6 GB at 4K, comfortable on an 8 GB card, and 7.6 GB at 64K, tight on an 8 GB card. Q8_0 (an 8.0 GB file) needs about 8.5 GB at 4K and wants a 12 GB card. On a discrete card llama.cpp keeps the embedding table in system RAM, so real VRAM use is lower than shown. The vision encoder is a separate 0.6–1.0 GB file published by unsloth, and a small DFlash draft model for speculative decoding ships beside it. NVIDIA's benchmark claims are its own, and no per-level quality loss has been published.
66K
Max Context
2
Quant Variants
GGUF Q8_0
Best Quality
—
Perplexity rise (best build)
Quantization Variants
Per-quant VRAM, quality loss, and inference speed on RTX 4090
Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports
| Format | Level | BPW | VRAM | PPL Loss | Speed | Source | Actions |
|---|---|---|---|---|---|---|---|
| GGUF | Q4_K_M | 6.57 | 6.6 GB | — | — | Estimated | |
| GGUF | Q8_0 | 8.54 | 8.5 GB | — | — | Estimated |
Which file each size is for
A file size is not a memory figure: the VRAM column is the calculator's total at 4K context — weights plus KV cache plus a 10% activation buffer. Units differ on purpose: file sizes are decimal GB, as Hugging Face lists them; the VRAM column is GiB (1024³ bytes) with 2% added to the weights, so 7.12 GB of file counts as about 6.8 GiB there. A vision projector (mmproj), where the repo ships one, is a separate download and is in neither number.
- GGUF Q4_K_Mlmstudio-community/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF · NVIDIA-Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf · 6.18 GB (6,180,446,208 bytes) · listing read 2026-10-24
- GGUF Q8_0lmstudio-community/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF · NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf · 8.03 GB (8,031,243,872 bytes) · listing read 2026-10-24
Similar models
Compare with Nemotron 3.5Nemotron 3.5 Bolt 74B-A9B
NVIDIA Nemotron
NVIDIA's Nemotron 3.5 Bolt is a 74B mixture-of-experts model with about 9B parameters active per token, built to fit a 24 GB card. It is compressed from Nemotron 3.5 Super VL and ships in one build, NVIDIA's own native 2-bit file of 22.0 GB. Only 8 of its 88 layers keep a growing cache, so context costs little: about 23.1 GB at 4K, 24.1 GB at 128K and 25.2 GB at its full 256K window. That is the 24 GB NVIDIA designed for, but over this site's comfortable line, so a 24 GB card reads as tight; a 32 GB card or a 48 GB Mac runs it comfortably. The image encoder is a separate 1.7 GB file. NVIDIA's benchmark claims are its own, and no quality loss for the 2-bit build has been published.
Nemotron 3.5 Super VL 120B-A12B
NVIDIA Nemotron
NVIDIA's Nemotron 3.5 Super VL is a 120B mixture-of-experts vision-language model with about 12B parameters active per token. It is built for coding, tool use and document work, with a context window of up to 1M tokens. Only 8 of its 88 layers keep a growing cache, so even the full window is cheap: at Q4_0 (ggml-org's 71.8 GB file) it needs about 75 GB at 4K and 84 GB at 1M. That runs comfortably on a 128 GB Mac or a DGX Spark 128G, and tightly on one 80 GB card. Q8_0 (128 GB) wants a 256 GB Mac. The vision encoder is a separate 1–1.7 GB file. Sizes come from the published GGUF files; NVIDIA's benchmark claims are its own and no per-level quality loss has been published.
Nemotron 3.5 Lightning 30B-A3B
NVIDIA Nemotron
The small member of NVIDIA's Nemotron 3.5 family: a 30B text-only mixture-of-experts model with about 3B parameters active per token. It generates at roughly the speed of a 3B model. Only 6 of its 52 layers keep a growing cache, which makes long context almost free: at Q4_0 (ggml-org's 18.9 GB file) it needs about 19.8 GB at 4K, 20.6 GB at 128K and 21.4 GB at its full 256K window. That is comfortable on a 24 GB card up to 128K and tight only at the full window; a 16 GB card cannot hold it. Q8_0 (about 35 GB) wants a 48 GB card or a 64 GB Mac. An optional 1.2 GB draft file for speculative decoding ships separately. Sizes come from the published GGUF files; NVIDIA's benchmark claims are its own and no per-level quality loss has been published.
Gemma 3 4B IT
Google Gemma 3
Google Gemma 3 multimodal 4B. 128K context; strong vision + text on 8GB cards.
Running Nemotron 3.5 Bolt E4B locally
At Q4_K_M and 4K of context, Nemotron 3.5 Bolt E4B needs about 6.6 GB — 5.9 GB of weights, 0.10 GB of KV cache and a 0.6 GB activation buffer. The smallest cards in this index that clear that comfortably are 8 GB ones — 9 of them, from the RTX 4060 up — and 76 of the 78 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.
What longer context costs
Going from 4K to 32K adds about 0.44 GB, taking the total to 7.0 GB. That is gentler than the parameter count suggests: only 4 of 42 layers keep a KV cache that grows with context, and 20 more cap theirs at a sliding window, so the cache scales at a fraction of the usual rate. The model's native window is 64K; holding all of it at Q4_K_M would need about 8 GB.
Which build to download
This index only tracks Nemotron 3.5 Bolt E4B in GGUF, across 2 levels (Q4_K_M, Q8_0). No per-level perplexity sweep has been published for this model, so the quality column is empty rather than estimated — within one model, more bits per weight is the only ordering the data supports. No throughput has been measured for this model on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.
Common questions
- How much VRAM does Nemotron 3.5 Bolt E4B need?
- About 6.6 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 7.0 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
- Will Nemotron 3.5 Bolt E4B run on a 8GB GPU?
- Yes — at Q4_K_M and 4K context it needs about 6.6 GB, which leaves 1.4 GB spare on a RTX 4060, or on any of the other 8 GB cards here (9 in all). That is the smallest size in this index that clears it comfortably; 76 of 78 do.
- Which quantization of Nemotron 3.5 Bolt E4B should I use?
- No published quality comparison exists for this model, so pick by footprint: Q4_K_M is the level most people run, and a higher bits-per-weight level is more faithful. All 2 levels in the index are Q4_K_M, Q8_0.
Where this model fits
Sized at Q4_K_M with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.
- Runs comfortably on
- RTX 40608GB · +1.4RTX 4060 Ti 8G8GB · +1.4Radeon RX 9060 XT 8G8GB · +1.4RTX 5060 Ti 8G8GB · +1.4RTX 50608GB · +1.4
- Just misses
- Mac M1 8G+0.6GB overMac M3 8G+0.6GB over
- Ships in
- GGUF
- Try it yourself
- Size it in the calculatorGenerate the run command
How to actually run this
Deployment guides for this model and this class of hardware.
This page's figures change when the model or the runtime does.Last updated 2026-10-24 RSS → /feed.xml