IntermediateEdge / Local 9 min read

DeepSeek-R1 Distill 14B: EXL2 vs GGUF

Head-to-head on RTX 4090 — when to pick turboderp EXL2 over bartowski GGUF.

Verified stack

RTX 4090 · R1-Distill-Qwen-14B · EXL2 4.65bpw vs GGUF Q4_K_M · ExLlamaV2 / llama.cpp

Last verified 2026-07-22

DeepSeek-R1EXL2GGUFExLlamaV2

Speed vs simplicity

EXL2 via ExLlamaV2 delivers ~35% faster inference than GGUF via llama.cpp on the same 14B model. GGUF wins on setup simplicity and Ollama compatibility.

text

EXL2 4.65bpw → ~128 tok/s (ExLlamaV2, RTX 4090)
GGUF Q4_K_M  → ~95 tok/s (llama.cpp, RTX 4090)

Download EXL2

Grab the turboderp EXL2 quant from Hugging Face. Use TabbyAPI or ExLlamaV2 server for OpenAI-compatible API.

bash

huggingface-cli download turboderp/DeepSeek-R1-Distill-Qwen-14B-exl2 \
  --include "*4.65bpw*" --local-dir ./models/r1-14b-exl2

Related guides

IntermediateEdge / Local10 min read

ExLlamaV2 on RTX 4090: Full Setup Guide

Install ExLlamaV2, load an EXL2 quant, and serve an OpenAI-compatible API in under 10 minutes.

BeginnerEdge / Local7 min read

What Can You Run on RTX 4060 Ti 16G?

A practical guide to picking the right model and quant level for NVIDIA's best budget 16GB card.

IntermediateEdge / Local8 min read

Qwen2.5-Coder 32B on a Single RTX 4090

The best open coding model that fits in 24GB — quant selection and tuning tips.

Deployment guides are educational. Each model is subject to its own license — read the official Hugging Face model card before downloading or deploying.