NVIDIA GPU

NVIDIA GeForce RTX 4090

The NVIDIA GeForce RTX 4090 has 24GB of VRAM and 1010 GB/s of memory bandwidth. Its sweet spot tops out around Qwen3-30B-A3B — the largest model that loads cleanly at a quality quant with llama.cpp at 8K context.

VRAM: 24 GB
Memory bandwidth: 1010 GB/s
TDP: 450 W
MSRP: $1,599
Backends: CUDA · Vulkan

Specs as of 2026-07-08 · source

Affiliate link — we may earn a commission. Verdicts are computed before any link is attached.

What an NVIDIA GeForce RTX 4090 can run

Every model we track, graded at its best clean-fit quant. Click any verdict for the full breakdown.

Model	Best-quant verdict · llama.cpp · 8K context
Qwen3-8B	Fits · A+ · Q8_0interactive
gemma-4-26B-A4B-it	Fits · B · UD-Q4_K_Minteractivebeta
Qwen2.5-7B-Instruct	Fits · A+ · Q8_0interactive
Qwen2.5-1.5B-Instruct	Fits · A+ · Q8_0interactive
Llama-3.2-1B-Instruct	Fits · A+ · Q8_0interactive
Llama-3.1-8B-Instruct	Fits · A+ · Q8_0interactive
DeepSeek-R1	Won't fitbeta
Qwen3.5-9B	Fits · A+ · Q8_0interactivebeta
gpt-oss-20b	Fits · A+ · MXFP4interactivebeta
Qwen3.6-35B-A3B	Tight · C · MXFP4_MOEinteractivebeta
Qwen2.5-3B-Instruct	Fits · A+ · Q8_0interactive
Qwen3-32B	Tight · C · Q4_K_Minteractive
Qwen3.6-27B	Fits · B · Q4_K_Minteractivebeta
Qwen2.5-0.5B-Instruct	Fits · A+ · Q8_0interactive
Mistral-7B-Instruct-v0.3	Fits · A+ · Q8_0interactive
gpt-oss-120b	Tight · C · MXFP4interactive · offloadbeta
Qwen3-14B	Fits · B · Q8_0interactive
Qwen3-30B-A3B	Fits · B · Q4_K_Minteractivebeta
Qwen2.5-32B-Instruct	Tight · C · Q4_K_Minteractive
DeepSeek-V4-Flash	Won't fitbeta
Qwen2.5-14B-Instruct	Fits · B · Q8_0interactive
Llama-3.2-3B-Instruct	Fits · A+ · Q8_0interactive
gemma-3-4b-it	Fits · A+ · Q8_0interactivebeta
gemma-3-12b-it	Fits · A+ · Q8_0interactivebeta
Llama-3.1-70B-Instruct	Tight · C · Q4_K_Musable · offloadbeta
Phi-3.5-mini-instruct	Fits · A+ · Q8_0interactive
Phi-4	Fits · B · Q8_0interactive
DeepSeek-R1-Distill-Qwen-32B	Tight · C · Q4_K_Minteractive
gemma-3-27b-it	Tight · C · Q4_K_Minteractivebeta
Llama-3.3-70B-Instruct	Tight · C · Q4_K_Musable · offloadbeta
Qwen2.5-72B-Instruct	Tight · C · Q4_K_Mpainful · offloadbeta
gemma-2-2b-it	Fits · A+ · Q8_0interactivebeta
DeepSeek-R1-Distill-Qwen-14B	Fits · B · Q8_0interactive
DeepSeek-R1-Distill-Llama-8B	Fits · A+ · Q8_0interactive
DeepSeek-R1-Distill-Qwen-7B	Fits · A+ · Q8_0interactive
gemma-2-9b-it	Fits · A+ · Q8_0interactivebeta
Mistral-Nemo-Instruct-2407	Fits · A+ · Q8_0interactive
QwQ-32B	Tight · C · Q4_K_Minteractive
Mistral-Small-24B-Instruct-2501	Fits · B · Q5_K_Minteractive
gemma-2-27b-it	Fits · B · Q4_K_Minteractivebeta

Verdicts computed from published specs and measured GGUF sizes — how the math works.