Best GPU for Local LLMs (2026): Measured tok/s on RTX 5090, 4090, 3090
Measured decode tok/s for RTX 5090, 4090 and 3090 across 8B to 32B models, plus what fits in 24GB vs 32GB. Every number is from a signed benchmark run.
MeasuredOn Qwen3.8-27B Q4_K_M (short-chat workload), the RTX 5090 measured 71.3 tok/s (r_exg-o5xt410, llama.cpp build 12158, 4K context) vs 43.8 on the RTX 4090 (r_cfp_53ic9g0, llama.cpp build 12158, 4K context), about 63% faster. On the same llama.cpp build 12158 (e8df20d), the RTX 4090 was about 12% faster than the RTX 3090 (39.0 tok/s, r_ijjtui41c8o).
On [Qwen3.8-27B](/m/qwen3-8-27b) Q4_K_M at 4K context (llama.cpp build 12158 (e8df20d), chat-short), llm-speed measured the RTX 5090 at 71.27 tok/s ([r_exg-o5xt410](/r/r_exg-o5xt410)), the fastest single consumer card we've tested, and its 32 GB fits more. On the 24 GB cards at the same 4K context and llama.cpp build, llm-speed measured 43.83 tok/s on an RTX 4090 (r_cfp_53ic9g0) and 38.95 on an RTX 3090 (r_ijjtui41c8o), so the 5090 decoded about 1.63× as fast as the 4090 and 1.83× as fast as the 3090 (calculated, one run per card). These same-build figures replace the earlier mixed-build 4K figures (75.19 tok/s on the 5090 at build 11669; 43.38 and 38.61 at build 11803): a llama.cpp build change, not a regression. A community-submitted run, not measured by llm-speed, reports 84.51 tok/s on a 4090 with MTP speculative decoding on (r_7j8rpk9mwhg; context not recorded).
Choose the model file, context and number of simultaneous requests first. Then check runtime support and measure that workload before buying another card.
Start with the workload you need to run. A model loading successfully does not establish useful response time or answer quality. The examples below show reported configurations and source settings; use them to find relevant measurements, then validate your intended model, quantization and workload before buying.
Three checks before buying or adding a GPU
1. Can the runtime use this exact card?
Check the GPU, operating system, driver and runtime together in the Ollama hardware support list. A supported vendor name does not establish support for every card or mixed-card setup. Check motherboard slots, physical clearance, cooling and power requirements against the manufacturers' specifications too.
2. Does the full workload fit?
Budget for the exact weight file, context cache, runtime allocations and other resident models. Two simultaneous long conversations need a different memory budget from one short chat. Ollama documents how parallel requests increase context allocation. Start with the memory estimator, then measure your actual concurrency; its estimate is not a guarantee.
3. Is a second card solving the right problem?
Two cards do not behave like one card merely because their capacities add up. Ollama prefers one GPU when the model fits, otherwise it can split across available GPUs. llama.cpp exposes device and split controls. Validate support and placement on each device; more capacity does not establish a speed increase. System RAM plus VRAM is also a different configuration from GPU-only inference. In Ollama, inspect ollama ps and follow the placement and latency checklist.
Before spending, run representative prompts at your intended context and concurrency. Record first-token wait, decode speed, memory use and whether the answers meet your task. Our Qwen3.8 and Gemma 4 RTX 5090 study supplies specific speed measurements; it does not validate a different GPU pair, long-context concurrency or coding quality.
Sources checked September 22, 2026. The examples below retain reported host capacities, including nonstandard configurations. They do not prove GPU-only residency, current prices or which purchase has the best value. Token counts describe measured prompts and responses, not the maximum configured context. Missing fields remain explicit.
Short answer
The RTX 5090 (32GB) is the fastest consumer card on llm-speed: llm-speed measured 71.27 tok/s on dense Qwen3.8-27B Q4_K_M at 4K context (r_exg-o5xt410), and a community-submitted run reports 318.4 tok/s on gpt-oss-20b (run), both llama.cpp on the chat-short workload. For a 24GB RTX 4090 or RTX 3090, llm-speed measured 43.83 and 38.95 tok/s on the same Qwen3.8-27B Q4_K_M at the same 4K context (r_cfp_53ic9g0, r_ijjtui41c8o), so at 4K context the 5090 decoded about 1.63× as fast as the 4090 and 1.83× as fast as the 3090 (calculated; all three runs on the same llama.cpp build 12158 (e8df20d), one run per card). Pick by the largest model you need to keep fully on the GPU, then by speed.
The tables below are not a controlled comparison: hosts, llama.cpp builds and dates differ across rows, and some rows use Ollama 0.31.1. Runs from a host whose RTX 4090 reports 48GB of GPU memory (a retail 4090 has 24GB) are left out. All rows are single-request decode speeds. See methodology.
Measured speeds by model size
Each row says whether llm-speed measured it (@llmspeed) or it is community-submitted.
8B to 9B dense
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run | Source |
|---|---|---|---|---|---|---|
| RTX 5090 (32GB) | Llama-3.1-8B-Instruct quant not reported | llama.cpp chat-short | 232.2tok/s | 33.8ms | r_kfrkg-vn376 2026-07-01 | Community-submitted |
| RTX 3090 (24GB) | llama3.1 Q4_K_M | ollama0.31.1 chat-short · TTFT includes a cold model load | 136.2tok/s | 49,960ms | r_9hurqggbshk 2026-07-02 | Community-submitted |
14B to 20B (dense and MoE)
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run | Source |
|---|---|---|---|---|---|---|
| RTX 5090 (32GB) | gpt-oss-20b quant not reported | llama.cpp chat-short | 318.4tok/s | 173ms | r_b9ul-vxh9sc 2026-07-01 | Community-submitted |
| RTX 3090 (24GB) | coder-v2 Q4_0 | ollama0.31.1 chat-short | 189.5tok/s | 735ms | r_o2-1w665rtq 2026-07-02 | Community-submitted |
| RTX 3090 (24GB) | qwen2.5-coder Q4_K_M | ollama0.31.1 chat-short | 69.21tok/s | 326ms | r_6ahy-dq0f_0 2026-07-02 | Community-submitted |
27B to 32B dense
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run | Source |
|---|---|---|---|---|---|---|
| RTX 5090 (32GB) | Qwen3.8-27B Q4_K_M | llama.cppb12158 chat-short · Qwen3.8-27B, llama.cpp build 12158 (e8df20d), 4K context | 71.27tok/s | 105ms | r_exg-o5xt410 2026-11-16 | Measured by llm-speed@llmspeed |
| RTX 4090 (24GB) | Qwen3.8-27B Q4_K_M | llama.cppb12158 chat-short · Qwen3.8-27B on an RTX 4090, llama.cpp build 12158 (e8df20d), 4K context | 43.83tok/s | 228ms | r_cfp_53ic9g0 2026-11-16 | Measured by llm-speed@llmspeed |
| RTX 3090 (24GB) | Qwen3.8-27B Q4_K_M | llama.cppb12158 chat-short · Qwen3.8-27B on an RTX 3090, llama.cpp build 12158 (e8df20d), 4K context | 38.95tok/s | 209ms | r_ijjtui41c8o 2026-11-16 | Measured by llm-speed@llmspeed |
| RTX 5090 (32GB) | Qwen3.8-27B Q4_K_M | 9725a31 chat-short · 1 of 3 repeats (65.43 to 65.73), older build 9725a31 | 65.66tok/s | 118ms | r_dz5a3lzxhrj 2026-09-05 | Measured by llm-speed@llmspeed |
| RTX 5090 (32GB) | Qwen3.6-27B-Q4_K_M.gguf quant not reported | llama.cpp chat-short · single run | 73.96tok/s | 190ms | r_f7ulllqu4vk 2026-07-01 | Community-submitted |
| RTX 5090 (32GB) | Qwen2.5-Coder-32B-Instruct quant not reported | llama.cpp chat-short | 71.91tok/s | 71.0ms | r_nkbs6d3-d21 2026-07-01 | Community-submitted |
| RTX 4090 (24GB) | gemma3 Q4_K_M | ollama0.31.1 chat-short | 46.95tok/s | 817ms | r_x23y_sg24pm 2026-07-02 | Community-submitted |
30B-class MoE
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run | Source |
|---|---|---|---|---|---|---|
| RTX 5090 (32GB) | Qwen3-Coder-30B-A3B-Instruct quant not reported | llama.cpp chat-short | 259.9tok/s | 218ms | r_c7qyvvmmsv1 2026-07-01 | Community-submitted |
| RTX 4090 (24GB) | qwen3-coder Q4_K_M | ollama0.31.1 chat-short | 145.2tok/s | 562ms | r_wjq32z47vlp 2026-07-02 | Community-submitted |
24GB vs 32GB: what the extra memory buys
In llm-speed's RTX 5090 study, Qwen3.8-27B Q4_K_M (an 18.97 GB file) ran with all 65 layers on the GPU and the runtime reported 18.62 GiB allocated for model, context and compute buffers, with a 16,384-token context configured. On 24GB cards, llm-speed ran the same model with every layer on the GPU at 4K, 8K and 16K context on llama.cpp build 11803, and on each card the three contexts decoded within 0.3 tok/s of one another (chat-short). On the later 4K controls (llama.cpp build 12158), llm-speed measured 43.83 tok/s on an RTX 4090 (run) and 38.95 on an RTX 3090 (run). Other community-submitted runs on a 24GB RTX 4090 report Gemma 3 27B Q4_K_M at 47.0 tok/s (run) and the 30B MoE qwen3-coder at 145.2 tok/s (run), both via Ollama. Card memory sizes are manufacturer specs. Use the memory-fit estimator for your exact file and context.
Is a used RTX 3090 still worth it?
For models up to about 16B, the community-submitted 3090 rows are fast: 189.5 tok/s on deepseek-coder-v2 (15.7B, Q4_0) and 136.2 tok/s on llama3.1 8B Q4_K_M. The 14.8B qwen2.5-coder ran at 69.2 tok/s. These come from one host on Ollama 0.31.1, so treat them as examples rather than a ranking. Same-backend runs of the same models on a 3090, 4090 and 5090 are the comparison we still need.
What we have not measured yet
Any RTX 5080, RTX 5070 Ti or RX 7900 XTX run. If you own one, run the suite and submit: your row will appear here after the next build.
FAQ
What is the fastest GPU for local LLMs?
Of the GPUs on llm-speed, the RTX 5090: llm-speed measured 71.27 tok/s on Qwen3.8-27B Q4_K_M at 4K context (43.83 on a 4090 at the same context), and community-submitted runs report 318.4 tok/s on gpt-oss-20b and 259.9 tok/s on Qwen3-Coder-30B-A3B (llama.cpp, chat-short).
How many tok/s does an RTX 5090 get on a 27B model?
Measured by llm-speed: 71.27 tok/s for Qwen3.8-27B Q4_K_M (r_exg-o5xt410, llama.cpp build 12158, chat-short, 4K context) and a median of 65.66 tok/s on another RTX 5090 with an older build (three repeats, llama.cpp 9725a31). A community-submitted run reports 74.0 tok/s for Qwen3.6-27B Q4_K_M.
Is 24GB of VRAM enough for a 27B model?
Yes at 4-bit. On llama.cpp build 11803, llm-speed ran Qwen3.8-27B Q4_K_M with every layer on the GPU of a 24GB RTX 4090 and a 24GB RTX 3090 at 4K, 8K and 16K context (chat-short, within 0.3 tok/s across contexts on each card); on the later 4K controls (llama.cpp build 12158) it measured 43.83 tok/s on the 4090 and 38.95 tok/s on the 3090.
Which machine do I need to run a local coding agent?
For a local coding agent on Qwen3.8-27B Q4_K_M, Measured by llm-speed (@llmspeed), llama.cpp build 12158 (e8df20d), one chat, November 2026, at a filled ~16K prompt: 68.0 tok/s on an RTX 5090 (r_zhzepwtpurt), 41.7 tok/s on an RTX 4090 (r_5h1eygtqqnj), and 37.1 tok/s on an RTX 3090 (r_507_5ghuydq). Prefill at that same point was 4061.4, 2983.8 and 1411.3 tok/s. At 4K short-chat, the same cards measured 71.3, 43.8 and 39.0 tok/s (r_exg-o5xt410, r_cfp_53ic9g0, r_ijjtui41c8o). Those figures and the setup are on /m/qwen3-8-27b. A 24GB 4090 or 3090 runs that 27B file; the 32GB 5090 is the fastest of the three on this set. Our only M5 Max 27B data is one exploratory MLX study outside our standard suite, so we don't quote a Mac tok/s in this comparison. That study (/blog/qwen3-8-27b-m5-max-48gb) had uncontrolled background load and a wide spread.
Related: RTX 5090 vs RTX 4090 · Qwen3.8-27B · Apple Silicon vs NVIDIA · cheatsheet
Submitted benchmarks
| Reported host | Model and settings | Decode | Source |
|---|---|---|---|
| L40S (45GB) | Qwen3.8-27B-ctx32k Q4_K_M · llama.cpp0.7.0-dev (build 11803, commit a37cc4610) Prompt / output tokens: 154 / 256 TTFT: 272.77 ms | 34.27tok/s | r_jcyn1luqf6w Submitted 2026-10-29 |
| RTX 3090 (24GB) | Qwen3.8-27B Q4_K_M · llama.cpp0.9.0-dev (build 12158, commit e8df20d23) Prompt / output tokens: 114 / 256 TTFT: 209.12 ms | 38.95tok/s | r_ijjtui41c8o Submitted 2026-11-16 |
| RTX 3090 (24GB) + AMD EPYC 7763 64-Core Processor (128c) + 1008GB | Flint-40B-A2.5B-UD-cpumoe6-ctx8k Q4_K_XL · llama.cpp0.7.0-dev (build 11803, commit a37cc4610) Prompt / output tokens: 98 / 256 TTFT: 488.857 ms | 16.84tok/s | r_klxxsjjp2m- Submitted 2026-10-22 |
| RTX 4090 (24GB) | Qwen3.8-27B Q4_K_M · llama.cpp0.9.0-dev (build 12158, commit e8df20d23) Prompt / output tokens: 114 / 256 TTFT: 227.59 ms | 43.83tok/s | r_cfp_53ic9g0 Submitted 2026-11-16 |
| RTX 4090 (24GB) + AMD EPYC 7452 32-Core Processor (32c) + 504GB | Flint-40B-A2.5B-UD-cpumoe6-ctx8k Q4_K_XL · llama.cpp0.7.0-dev (build 11803, commit a37cc4610) Prompt / output tokens: 98 / 256 TTFT: 356.144 ms | 39.82tok/s | r_wap61ewkpzm Submitted 2026-10-22 |
| RTX 5090 (32GB) | Qwen3.8-27B Q4_K_M · llama.cpp0.9.0-dev (build 12158, commit e8df20d23) Prompt / output tokens: 114 / 256 TTFT: 105.384 ms | 71.27tok/s | r_exg-o5xt410 Submitted 2026-11-16 |
| RTX 5880 (48GB) | llama3.2 Q8_0 · ollama0.34.2 Prompt / output tokens: 125 / 256 TTFT: 2,820.586 ms | 466.9tok/s | r_b948ehgi8g4 Submitted 2026-09-19 |
Check it for your setup
- Predict your exact setup: estimate whether Qwen3.8-27B fits on RTX 5090 at your quant and context, next to measured speeds where we have them.
- Request this pair: ask for a measured Qwen3.8-27B run on RTX 5090. Requests are ranked by votes.
- Full run table (56 runs): every Qwen3.8-27B result we have, with links to each source run.
Measured speeds for this pair: Can an RTX 5090 run Qwen3.8-27B?
See also: All hardware · All models · Methodology