Browse · 36 tracked
GPUs for local LLMs
We track 36 GPUs for local LLM inference — 26 NVIDIA, 8 AMD, and 2 Intel — with sourced VRAM, bandwidth, and price, each graded against every model we cover. “Runs up to” is the largest model that loads cleanly (grade B or better) at a quality quant with llama.cpp at 8K context — see how the math works.
NVIDIA
| GPU | VRAM | Bandwidth | MSRP | Runs up to |
|---|---|---|---|---|
| NVIDIA RTX 6000 Ada Generation | 48 GB | 960 GB/s | $6,799 | Qwen3.6-35B-A3B |
| NVIDIA RTX A6000 | 48 GB | 768 GB/s | $4,649 | Qwen3.6-35B-A3B |
| NVIDIA GeForce RTX 5090 | 32 GB | 1790 GB/s | $1,999 | Qwen3.6-35B-A3B |
| NVIDIA RTX 5000 Ada Generation | 32 GB | 576 GB/s | $4,000 | Qwen3.6-35B-A3B |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | 1010 GB/s | $1,999 | Qwen3-30B-A3B |
| NVIDIA GeForce RTX 4090 | 24 GB | 1010 GB/s | $1,599 | Qwen3-30B-A3B |
| NVIDIA GeForce RTX 3090 | 24 GB | 936.2 GB/s | $1,499 | Qwen3-30B-A3B |
| NVIDIA GeForce RTX 5080 | 16 GB | 960 GB/s | $999 | gpt-oss-20b |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | 896 GB/s | $749 | gpt-oss-20b |
| NVIDIA GeForce RTX 4080 SUPER | 16 GB | 736.3 GB/s | $999 | gpt-oss-20b |
| NVIDIA GeForce RTX 4080 | 16 GB | 716.8 GB/s | $1,199 | gpt-oss-20b |
| NVIDIA GeForce RTX 4070 Ti SUPER | 16 GB | 672.3 GB/s | $799 | gpt-oss-20b |
| NVIDIA GeForce RTX 5060 Ti 16 GB | 16 GB | 448 GB/s | $429 | gpt-oss-20b |
| NVIDIA GeForce RTX 4060 Ti 16 GB | 16 GB | 288 GB/s | $499 | gpt-oss-20b |
| NVIDIA GeForce RTX 3080 12 GB | 12 GB | 912.4 GB/s | $799 | Mistral-Nemo-Instruct-2407 |
| NVIDIA GeForce RTX 3080 Ti | 12 GB | 912.4 GB/s | $1,199 | Mistral-Nemo-Instruct-2407 |
| NVIDIA GeForce RTX 5070 | 12 GB | 672 GB/s | $549 | Mistral-Nemo-Instruct-2407 |
| NVIDIA GeForce RTX 4070 | 12 GB | 504.2 GB/s | $599 | Mistral-Nemo-Instruct-2407 |
| NVIDIA GeForce RTX 4070 SUPER | 12 GB | 504.2 GB/s | $599 | Mistral-Nemo-Instruct-2407 |
| NVIDIA GeForce RTX 4070 Ti | 12 GB | 504.2 GB/s | $799 | Mistral-Nemo-Instruct-2407 |
| NVIDIA GeForce RTX 3060 12 GB | 12 GB | 360 GB/s | $329 | Mistral-Nemo-Instruct-2407 |
| NVIDIA GeForce RTX 3080 10 GB | 10 GB | 760.3 GB/s | $699 | Qwen3.5-9B |
| NVIDIA GeForce RTX 3070 Ti | 8 GB | 608.3 GB/s | $599 | Qwen3-8B |
| NVIDIA GeForce RTX 3070 | 8 GB | 448 GB/s | $499 | Qwen3-8B |
| NVIDIA GeForce RTX 4060 Ti 8 GB | 8 GB | 288 GB/s | $399 | Qwen3-8B |
| NVIDIA GeForce RTX 4060 | 8 GB | 272 GB/s | $299 | Qwen3-8B |
AMDbeta tiers
| GPU | VRAM | Bandwidth | MSRP | Runs up to |
|---|---|---|---|---|
| AMD Radeon PRO W7900 | 48 GB | 864 GB/s | $3,999 | Qwen3.6-35B-A3B |
| AMD Radeon RX 7900 XTX | 24 GB | 960 GB/s | $999 | Qwen3-30B-A3B |
| AMD Radeon RX 7900 XT | 20 GB | 800 GB/s | $899 | Mistral-Small-24B-Instruct-2501 |
| AMD Radeon RX 9070 | 16 GB | 640 GB/s | $549 | gpt-oss-20b |
| AMD Radeon RX 9070 XT | 16 GB | 640 GB/s | $599 | gpt-oss-20b |
| AMD Radeon RX 7800 XT | 16 GB | 624 GB/s | $499 | gpt-oss-20b |
| AMD Radeon RX 6800 | 16 GB | 512 GB/s | $579 | gpt-oss-20b |
| AMD Radeon RX 6900 XT | 16 GB | 512 GB/s | $999 | gpt-oss-20b |
Intelbeta tiers
| GPU | VRAM | Bandwidth | MSRP | Runs up to |
|---|---|---|---|---|
| Intel Arc A770 16 GB | 16 GB | 560 GB/s | $349 | gpt-oss-20b |
| Intel Arc B580 | 12 GB | 456 GB/s | $249 | Mistral-Nemo-Instruct-2407 |
AMD and Intel fit verdicts are physics and hold; their speed tiers are beta until real measurements calibrate them. Specs stamped and sourced on each card's page.