Skip to content
llm-speed

Best GPU for Local LLMs (2026): Measured tok/s on RTX 5090, 4090, 3090

Measured decode tok/s for RTX 5090, 4090 and 3090 across 8B to 32B models, plus what fits in 24GB vs 32GB. Every number is from a signed benchmark run.

MeasuredOn Qwen3.8-27B Q4_K_M (short-chat workload), the RTX 5090 measured 75.2 tok/s (r_qfrb2zmyk0k, llama.cpp build 11669, 4K context per server log, not in the run record) vs 43.4 on the RTX 4090 (r_7cap38a021-, llama.cpp build 11803, 4K context), about 73% faster. The two runs used different llama.cpp builds (11669 and 11803) and different GGUF publishers (lmstudio-community vs ggml-org). On the same llama.cpp build 11803, the RTX 4090 was about 12% faster than the RTX 3090 (38.6 tok/s, r_mmtzel_i367).

Verdict (measured runs)

On [Qwen3.8-27B](/m/qwen3-8-27b) Q4_K_M at 4K context (llama.cpp, chat-short), llm-speed measured the RTX 5090 at 75.19 tok/s ([r_qfrb2zmyk0k](/r/r_qfrb2zmyk0k)), the fastest single consumer card we've tested, and its 32 GB fits more. On the 24 GB cards at the same 4K context, llm-speed measured 43.38 tok/s on an RTX 4090 (r_7cap38a021-) and 38.61 on an RTX 3090 (r_mmtzel_i367), both on llama.cpp build 11803, so the 5090 decoded about 1.73× as fast as the 4090 and 1.95× as fast as the 3090 (calculated). The 5090 run used an older llama.cpp build and a different GGUF publisher, so treat the 5090 gap as approximate. A community-submitted run, not measured by llm-speed, reports 84.51 tok/s on a 4090 with MTP speculative decoding on (r_7j8rpk9mwhg; context not recorded).

Recommendation

Choose the model file, context and number of simultaneous requests first. Then check runtime support and measure that workload before buying another card.

Start with the workload you need to run. A model loading successfully does not establish useful response time or answer quality. The examples below show reported configurations and source settings; use them to find relevant measurements, then validate your intended model, quantization and workload before buying.

Three checks before buying or adding a GPU

1. Can the runtime use this exact card?

Check the GPU, operating system, driver and runtime together in the Ollama hardware support list. A supported vendor name does not establish support for every card or mixed-card setup. Check motherboard slots, physical clearance, cooling and power requirements against the manufacturers' specifications too.

2. Does the full workload fit?

Budget for the exact weight file, context cache, runtime allocations and other resident models. Two simultaneous long conversations need a different memory budget from one short chat. Ollama documents how parallel requests increase context allocation. Start with the memory estimator, then measure your actual concurrency; its estimate is not a guarantee.

3. Is a second card solving the right problem?

Two cards do not behave like one card merely because their capacities add up. Ollama prefers one GPU when the model fits, otherwise it can split across available GPUs. llama.cpp exposes device and split controls. Validate support and placement on each device; more capacity does not establish a speed increase. System RAM plus VRAM is also a different configuration from GPU-only inference. In Ollama, inspect ollama ps and follow the placement and latency checklist.

Before spending, run representative prompts at your intended context and concurrency. Record first-token wait, decode speed, memory use and whether the answers meet your task. Our Qwen3.8 and Gemma 4 RTX 5090 study supplies specific speed measurements; it does not validate a different GPU pair, long-context concurrency or coding quality.

Sources checked September 22, 2026. The examples below retain reported host capacities, including nonstandard configurations. They do not prove GPU-only residency, current prices or which purchase has the best value. Token counts describe measured prompts and responses, not the maximum configured context. Missing fields remain explicit.

Short answer

The RTX 5090 (32GB) is the fastest consumer card on llm-speed: llm-speed measured 75.19 tok/s on dense Qwen3.8-27B Q4_K_M at 4K context (r_qfrb2zmyk0k), and a community-submitted run reports 318.4 tok/s on gpt-oss-20b (run), both llama.cpp on the chat-short workload. For a 24GB RTX 4090 or RTX 3090, llm-speed measured 43.38 and 38.61 tok/s on the same Qwen3.8-27B Q4_K_M at the same 4K context (r_7cap38a021-, r_mmtzel_i367; llama.cpp build 11803, same settings on both cards), so at 4K context the 5090 decoded about 1.73× as fast as the 4090 and 1.95× as fast as the 3090 (calculated; the 5090 run used an older llama.cpp build and a different GGUF publisher, so treat those ratios as approximate). Pick by the largest model you need to keep fully on the GPU, then by speed.

Not a controlled comparison: hosts, llama.cpp builds and dates differ, and some rows below use Ollama 0.31.1. Runs from a host whose RTX 4090 reports 48GB of GPU memory (a retail 4090 has 24GB) are left out. All rows are single-request decode speeds. See methodology.

Measured speeds by model size

Each row says whether llm-speed measured it (@llmspeed) or it is community-submitted.

8B to 9B dense

Reported hardwareModel · quantBackend · workloadDecode (measured)TTFT (measured)RunSource
RTX 5090 (32GB)Llama-3.1-8B-Instruct
quant not reported
llama.cpp
chat-short
232.2tok/s33.8msr_kfrkg-vn376
2026-07-01
Community-submitted
RTX 3090 (24GB)llama3.1
Q4_K_M
ollama0.31.1
chat-short · TTFT includes a cold model load
136.2tok/s49,960msr_9hurqggbshk
2026-07-02
Community-submitted

14B to 20B (dense and MoE)

Reported hardwareModel · quantBackend · workloadDecode (measured)TTFT (measured)RunSource
RTX 5090 (32GB)gpt-oss-20b
quant not reported
llama.cpp
chat-short
318.4tok/s173msr_b9ul-vxh9sc
2026-07-01
Community-submitted
RTX 3090 (24GB)coder-v2
Q4_0
ollama0.31.1
chat-short
189.5tok/s735msr_o2-1w665rtq
2026-07-02
Community-submitted
RTX 3090 (24GB)qwen2.5-coder
Q4_K_M
ollama0.31.1
chat-short
69.21tok/s326msr_6ahy-dq0f_0
2026-07-02
Community-submitted

27B to 32B dense

Reported hardwareModel · quantBackend · workloadDecode (measured)TTFT (measured)RunSource
RTX 5090 (32GB)Qwen3.8-27B
Q4_K_M
llama.cppb11669
chat-short · Qwen3.8-27B, llama.cpp 620d2225c, 4K context
75.19tok/s273msr_qfrb2zmyk0k
2026-10-15
Measured by llm-speed@llmspeed
RTX 4090 (24GB)Qwen3.8-27B-ctx4k
Q4_K_M
llama.cppb11803
chat-short · Qwen3.8-27B on an RTX 4090, llama.cpp build 11803, 4K context
43.38tok/s344msr_7cap38a021-
2026-10-22
Measured by llm-speed@llmspeed
RTX 3090 (24GB)Qwen3.8-27B-ctx4k
Q4_K_M
llama.cppb11803
chat-short · Qwen3.8-27B on an RTX 3090, llama.cpp build 11803, 4K context
38.61tok/s319msr_mmtzel_i367
2026-10-22
Measured by llm-speed@llmspeed
RTX 5090 (32GB)Qwen3.8-27B
Q4_K_M
9725a31
chat-short · 1 of 3 repeats (65.43 to 65.73), older build 9725a31
65.66tok/s118msr_dz5a3lzxhrj
2026-09-05
Measured by llm-speed@llmspeed
RTX 5090 (32GB)Qwen3.6-27B-Q4_K_M.gguf
quant not reported
llama.cpp
chat-short · single run
73.96tok/s190msr_f7ulllqu4vk
2026-07-01
Community-submitted
RTX 5090 (32GB)Qwen2.5-Coder-32B-Instruct
quant not reported
llama.cpp
chat-short
71.91tok/s71.0msr_nkbs6d3-d21
2026-07-01
Community-submitted
RTX 4090 (24GB)gemma3
Q4_K_M
ollama0.31.1
chat-short
46.95tok/s817msr_x23y_sg24pm
2026-07-02
Community-submitted

30B-class MoE

Reported hardwareModel · quantBackend · workloadDecode (measured)TTFT (measured)RunSource
RTX 5090 (32GB)Qwen3-Coder-30B-A3B-Instruct
quant not reported
llama.cpp
chat-short
259.9tok/s218msr_c7qyvvmmsv1
2026-07-01
Community-submitted
RTX 4090 (24GB)qwen3-coder
Q4_K_M
ollama0.31.1
chat-short
145.2tok/s562msr_wjq32z47vlp
2026-07-02
Community-submitted

24GB vs 32GB: what the extra memory buys

In llm-speed's RTX 5090 study, Qwen3.8-27B Q4_K_M (an 18.97 GB file) ran with all 65 layers on the GPU and the runtime reported 18.62 GiB allocated for model, context and compute buffers, with a 16,384-token context configured. On 24GB cards, llm-speed ran the same model with every layer on the GPU at 4K, 8K and 16K context: 43.38 tok/s on an RTX 4090 (run) and 38.61 on an RTX 3090 (run) at 4K, llama.cpp chat-short; the 8K and 16K runs decoded within 0.3 tok/s of that. Other community-submitted runs on a 24GB RTX 4090 report Gemma 3 27B Q4_K_M at 47.0 tok/s (run) and the 30B MoE qwen3-coder at 145.2 tok/s (run), both via Ollama. Card memory sizes are manufacturer specs. Use the memory-fit estimator for your exact file and context.

Is a used RTX 3090 still worth it?

For models up to about 16B, the community-submitted 3090 rows are fast: 189.5 tok/s on deepseek-coder-v2 (15.7B, Q4_0) and 136.2 tok/s on llama3.1 8B Q4_K_M. The 14.8B qwen2.5-coder ran at 69.2 tok/s. These come from one host on Ollama 0.31.1, so treat them as examples rather than a ranking. Same-backend runs of the same models on a 3090, 4090 and 5090 are the comparison we still need.

What we have not measured yet

The RTX 5090 on the same llama.cpp build as llm-speed's 4090 and 3090 runs (build 11803); and any RTX 5080, RTX 5070 Ti or RX 7900 XTX run. If you own one, run the suite and submit: your row will appear here after the next build.

FAQ

What is the fastest GPU for local LLMs?

Of the GPUs on llm-speed, the RTX 5090: llm-speed measured 75.19 tok/s on Qwen3.8-27B Q4_K_M at 4K context (43.38 on a 4090 at the same context), and community-submitted runs report 318.4 tok/s on gpt-oss-20b and 259.9 tok/s on Qwen3-Coder-30B-A3B (llama.cpp, chat-short).

How many tok/s does an RTX 5090 get on a 27B model?

Measured by llm-speed: 75.19 tok/s for Qwen3.8-27B Q4_K_M (r_qfrb2zmyk0k, llama.cpp 0.7.0-dev, chat-short, 4K context) and a median of 65.66 tok/s on another RTX 5090 with an older build (three repeats, llama.cpp 9725a31). A community-submitted run reports 74.0 tok/s for Qwen3.6-27B Q4_K_M.

Is 24GB of VRAM enough for a 27B model?

Yes at 4-bit. llm-speed measured Qwen3.8-27B Q4_K_M with every layer on the GPU at 43.38 tok/s on a 24GB RTX 4090 and 38.61 tok/s on a 24GB RTX 3090 (llama.cpp build 11803, chat-short) at 4K context; the 8K and 16K runs were within 0.3 tok/s.

Related: RTX 5090 vs RTX 4090 · Qwen3.8-27B · Apple Silicon vs NVIDIA · cheatsheet

Submitted benchmarks

Reported hostModel and settingsDecodeSource
L40S (45GB)
Qwen3.8-27B-ctx32k
Prompt / output tokens: 154 / 256
TTFT: 272.77 ms
34.27tok/sr_jcyn1luqf6w
Submitted 2026-10-29
RTX 3090 (24GB)
hf.co/unsloth/Qwen3.8-27B-GGUF
Q4_K_M · ollama0.40.14
Prompt / output tokens: 154 / 256
TTFT: 882.417 ms
41.37tok/sr_sf9x2u7872j
Submitted 2026-11-04
RTX 3090 (24GB) + AMD EPYC 7763 64-Core Processor (128c) + 1008GB
Flint-40B-A2.5B-UD-cpumoe6-ctx8k
Prompt / output tokens: 98 / 256
TTFT: 488.857 ms
16.84tok/sr_klxxsjjp2m-
Submitted 2026-10-22
RTX 4090 (24GB)
Qwen3.8-27B
Prompt / output tokens: 154 / 256
TTFT: 442.673 ms
41.81tok/sr_z6--z4sihzt
Submitted 2026-11-01
RTX 4090 (24GB) + AMD EPYC 7452 32-Core Processor (32c) + 504GB
Flint-40B-A2.5B-UD-cpumoe6-ctx8k
Prompt / output tokens: 98 / 256
TTFT: 356.144 ms
39.82tok/sr_wap61ewkpzm
Submitted 2026-10-22
RTX 5090 (32GB)
Ripple-65B-Preview-AD
Prompt / output tokens: 108 / 256
TTFT: 125.242 ms
56.25tok/sr__2bhvnz5xs9
Submitted 2026-11-12
RTX 5880 (48GB)
llama3.2
Q8_0 · ollama0.34.2
Prompt / output tokens: 125 / 256
TTFT: 2,820.586 ms
466.9tok/sr_b948ehgi8g4
Submitted 2026-09-19

Check it for your setup

  • Predict your exact setup: estimate whether Qwen3.8-27B fits on RTX 5090 at your quant and context, next to measured speeds where we have them.
  • Request this pair: ask for a measured Qwen3.8-27B run on RTX 5090. Requests are ranked by votes.
  • Full run table (37 runs): every Qwen3.8-27B result we have, with links to each source run.

Measured speeds for this pair: Can an RTX 5090 run Qwen3.8-27B?

See also: All hardware · All models · Methodology