TokenAssemble
Check a machineStack BuilderWorkflowsCompareMethodology

Browse stack

NVIDIA GPU

NVIDIA GeForce RTX 5090

The NVIDIA GeForce RTX 5090 has 32GB of VRAM and 1790 GB/s of memory bandwidth. Its sweet spot tops out around Qwen3.6-35B-A3B — the largest model that loads cleanly at a quality quant with llama.cpp at 8K context.

VRAM
32 GB
Memory bandwidth
1790 GB/s
TDP
575 W
MSRP
$1,999
Backends
CUDA · Vulkan

Also good for gaming — 4K-class (estimated from FP32 compute, not a game benchmark)

Specs as of 2026-07-08 · source

Check price on Amazon

NVIDIA GeForce RTX 5090 for local AI — common questions

How much VRAM does the NVIDIA GeForce RTX 5090 have?
The NVIDIA GeForce RTX 5090 has 32GB of VRAM with 1790 GB/s of memory bandwidth. VRAM sets which models fit; bandwidth sets how fast they feel once they do.
Is the NVIDIA GeForce RTX 5090 good for running AI models locally?
Yes. 32GB is enough headroom for the large open models most people want to run locally. The largest model it loads cleanly is Qwen3.6-35B-A3B, at a quality quant with llama.cpp at 8K context.
What runtimes work with the NVIDIA GeForce RTX 5090?
It runs on CUDA · Vulkan, which covers Ollama, llama.cpp and LM Studio. Runtime choice affects speed and features, not whether a model fits.

Affiliate link — we may earn a commission. Verdicts are computed before any link is attached.

What an NVIDIA GeForce RTX 5090 can run

Every model we track, graded at its best clean-fit quant. Click any verdict for the full breakdown.

ModelBest-quant verdict · llama.cpp · 8K context
Qwen3-8BFits · A · Q8_0interactive
gemma-4-26B-A4B-itFits · B · UD-Q5_K_Minteractivebeta
Qwen2.5-7B-InstructFits · A · Q8_0interactive
Qwen2.5-1.5B-InstructFits · A · Q8_0interactive
Llama-3.2-1B-InstructFits · A · Q8_0interactive
Llama-3.1-8B-InstructFits · A · Q8_0interactive
DeepSeek-R1Won't fitbeta
Qwen3.5-9BFits · A · Q8_0interactivebeta
gpt-oss-20bFits · A · MXFP4interactivebeta
Qwen3.6-35B-A3BFits · B · UD-Q5_K_XLinteractivebeta
Qwen2.5-3B-InstructFits · A · Q8_0interactive
Qwen3-32BFits · B · Q5_K_Minteractive
Qwen3.6-27BFits · B · Q6_Kinteractivebeta
Qwen2.5-0.5B-InstructFits · A · Q8_0interactive
Mistral-7B-Instruct-v0.3Fits · A · Q8_0interactive
gpt-oss-120bTight · C · MXFP4interactive · offloadbeta
Qwen3-14BFits · A · Q8_0interactive
Qwen3-30B-A3BFits · B · Q6_Kinteractivebeta
Qwen2.5-32B-InstructFits · B · Q5_K_Minteractive
DeepSeek-V4-FlashWon't fitbeta
Qwen2.5-14B-InstructFits · A · Q8_0interactive
Llama-3.2-3B-InstructFits · A · Q8_0interactive
gemma-3-4b-itFits · A · Q8_0interactivebeta
gemma-3-12b-itFits · A · Q8_0interactivebeta
Llama-3.1-70B-InstructTight · C · Q4_K_Musable · offloadbeta
Phi-3.5-mini-instructFits · A · Q8_0interactive
Phi-4Fits · A · Q8_0interactive
DeepSeek-R1-Distill-Qwen-32BFits · B · Q5_K_Minteractive
gemma-3-27b-itFits · B · Q6_Kinteractivebeta
Llama-3.3-70B-InstructTight · C · Q4_K_Musable · offloadbeta
Qwen2.5-72B-InstructTight · C · Q4_K_Musable · offloadbeta
gemma-2-2b-itFits · A · Q8_0interactivebeta
DeepSeek-R1-Distill-Qwen-14BFits · A · Q8_0interactive
DeepSeek-R1-Distill-Llama-8BFits · A · Q8_0interactive
DeepSeek-R1-Distill-Qwen-7BFits · A · Q8_0interactive
gemma-2-9b-itFits · A · Q8_0interactivebeta
Mistral-Nemo-Instruct-2407Fits · A · Q8_0interactive
QwQ-32BFits · B · Q5_K_Minteractive
Mistral-Small-24B-Instruct-2501Fits · B · Q8_0interactive
gemma-2-27b-itFits · B · Q6_Kinteractivebeta

Compare NVIDIA GeForce RTX 5090

Have a specific model in mind? Check it against the NVIDIA GeForce RTX 5090 at your own quant and context length.

Check with the NVIDIA GeForce RTX 5090 preselected →

Verdicts computed from published specs and measured GGUF sizes — how the math works.