RTX 4090 LLM benchmark

Community folklore on RTX 4090

51 unverified claims extracted from Reddit/HN comments. Lower trust than signed runs above; every row links to the source.

• communityconfidence 75%

  60.00tok/s — Llama 3.1 70b on RTX 4090 via ollama IQ2_XS

  our signed data: RTX 4090 · Llama 3.1 70b

  “Llama 3.1 70b at 60 tok/s on RTX 4090 (IQ2_XS) Setup GPU: 1 x RTX 4090 (24 GB VRAM) CPU: Xeon® E5-2695 v3 (16 cores) RAM: 64 GB RAM Running PyTorch 2.2.0 + CUDA 1”

  source: Reddit · u/grey-seagull · 2024-09-20

• communityconfidence 75%

  92.40tok/s — Qwen3-8B on RTX 4090 via llama.cpp Q8_0

  our signed data: RTX 4090 · Qwen3-8B

  “tok/s` generation at `128` output tokens - `Q8_0`: about `9975 tok/s` prompt processing at `512` tokens, `9955 tok/s` at `1024`, and about `92.4 tok/s` generation at `128` output tokens Hardware / runtime for those numbers: - `RTX 4090` - `Ryzen 9 7900X` - `llama.cpp` build com…”

  source: Reddit · u/RiverRatt · 2026-03-23

• communityconfidence 75%

  548.9tok/s — Llama-3.1-70B on RTX 4090 via vllm FP8

  our signed data: RTX 4090 · Llama-3.1-70B

  “neuralmagic\_Meta-Llama-3.1-70B-Instruct-FP8-dynamic Avg generation throughput: \~29-30 tokens/s Avg prompt throughput: 548.9 tokens/s (4 GPUs, 4090, power limited to 325) (8x, 8x, 4x , 4x) 5950x taichi x570 vllm backend i didn't do the specific prompt to get the va”

  source: Reddit · u/I_can_see_threw_time · 2024-08-01

• communityconfidence 75%

  30.00tok/s — Llama-3.1-70B on RTX 4090 via vllm FP8

  our signed data: RTX 4090 · Llama-3.1-70B

  “neuralmagic\_Meta-Llama-3.1-70B-Instruct-FP8-dynamic Avg generation throughput: \~29-30 tokens/s Avg prompt throughput: 548.9 tokens/s (4 GPUs, 4090, power limited to 325) (8x, 8x, 4x , 4x) 5950x taichi x570 vllm backend i di”

  source: Reddit · u/I_can_see_threw_time · 2024-08-01

• communityconfidence 70%

  140.0tok/s — Qwen3-Coder on RTX 4090 via llama.cpp

  our signed data: RTX 4090 · Qwen3-Coder

  “bled, KV cache at Q8) on my 4090. This is fully on the GPU, no offloading to CPU. Depending on context length I'm getting anywhere from 100-140 tokens/sec. If you wanted more context you'd have to offload some layers to CPU and it takes a massive hit (my recent post has some benc…”

  source: Reddit · u/ConversationNice3225 · 2025-08-05

• communityconfidence 70%

  665.2tok/s — Qwen3-Coder-Next on RTX 4090 via llama.cpp

  our signed data: RTX 4090 · Qwen3-Coder-Next

  “with 256k context, llama.cpp autoparser branch, windows 11. prompt eval time = 50498.86 ms / 33594 tokens ( 1.50 ms per token, 665.24 tokens per second) eval time = 2282.98 ms / 82 tokens ( 27.84 ms per token, 35.92 tokens per second) t…”

  source: Reddit · u/gtrak · 2026-02-18

• communityconfidence 70%

  37.00tok/s — Qwen3 Coder Next on RTX 4090 via llama.cpp

  our signed data: RTX 4090 · Qwen3 Coder Next

  “tches I used: `--jinja --temp 1.0 --top-p 0.95 --min-p 0.01 --top-k 40 -fa on --fit on -c 131072 --no-mmap` With 32K context only it does 37 tok/s. EDIT: the llamacpp version is b7936 (teh first with the Qwen3 Coder Next fix) and the GGUF version is the updated one from unslot…”

  source: Reddit · u/tmvr · 2026-02-06

• communityconfidence 70%

  35.00tok/s — Qwen3 Coder Next on RTX 4090 via llama.cpp

  our signed data: RTX 4090 · Qwen3 Coder Next

  “26 tok/s seems low for that hardware. I get 35 tok/s with Q4\_K\_XL (slightly larger than Q4\_K\_S) using a 4090 and DDR5-4800 RAM. and that's with 128K context and non-quantized KV. These are”

  source: Reddit · u/tmvr · 2026-02-06

• communityconfidence 65%

  10.00tok/s — DeepSeek V3 on RTX 4090 FP8

  our signed data: RTX 4090 · DeepSeek V3

  “g fast. Expert weights on CPU (4-bit GGUF): All the huge MoE banks live in system RAM and load as needed. End result: I’m seeing about \~10 tokens/sec with a 32K context window—pretty smooth for local tinkering. KTransformers made it so easy with its Docker image. It handles …”

  source: Reddit · u/texasdude11 · 2025-04-27

• communityconfidence 60%

  12.00tok/s — qwen2.5 on RTX 4090 via ollama

  our signed data: RTX 4090 · qwen2.5

  “a CPU (which is what happens with the 4090), but it's still too slow to be worth a $4k purchase. You probably read at a speed greater than 12t/s; thus anything less than that will become fairly uncomfortable and you might feel ripped off. Unless you are absolutely sure about th…”

  source: Reddit · u/darth_chewbacca · 2024-11-14

See all 51 claims for RTX 4090 →

Which models can an RTX 4090 run?

Fastest decode speed for each model on this hardware, from llm-speed's own runs and community-submitted runs. Each page lists every quant and runtime and labels which runs llm-speed measured.

• gpt-oss-20b · 242.6 tok/s
• Qwen3-Coder 30B-A3B · 232.5 tok/s
• LFM2.5 24B-A2B · 231.9 tok/s
• Sage-A1.5 · 216.8 tok/s at 128K context
• Flint 40B-A2.5B · 179.4 tok/s at 8K context
• glm-4.7-flash 29.9B · 164.9 tok/s
• Qwen3.6 35B-A3B · 160.6 tok/s
• Gemma 4 12B IT QAT · 97.7 tok/s at 8K context
• Muse-Glimmer-30B · 53.7 tok/s at 8K context
• Qwen3.8 27B · 44.2 tok/s at 8K context
• Qwen3 32B · 43.5 tok/s

Models measured on RTX 4090

• 3.1-8B-Instruct benchmarks
• Qwen3-8B benchmarks
• R1-0528-Qwen3-8B benchmarks
• Qwen3.8-27B-ctx8k benchmarks
• Muse-Glimmer-30B-ctx128k-kvq8 benchmarks
• Muse-Glimmer-30B-ctx96k-kvq8 benchmarks
• Muse-Glimmer-30B-ctx64k-kvq8 benchmarks
• Muse-Glimmer-30B-ctx48k-kvq8 benchmarks
• Muse-Glimmer-30B-ctx32k-kvq8 benchmarks
• Muse-Glimmer-30B-ctx8k-kvq8 benchmarks
• Muse-Glimmer-30B-ctx128k benchmarks
• Muse-Glimmer-30B-ctx96k benchmarks
• Muse-Glimmer-30B-ctx64k benchmarks
• Muse-Glimmer-30B-ctx48k benchmarks
• Muse-Glimmer-30B-ctx32k benchmarks
• Muse-Glimmer-30B-ctx8k benchmarks
• Sage-A1.5-ctx128k benchmarks
• Sage-A1.5-ctx96k benchmarks
• Sage-A1.5-ctx64k benchmarks
• Sage-A1.5-ctx48k benchmarks
• Sage-A1.5-ctx32k benchmarks
• Sage-A1.5-ctx16k benchmarks
• Sage-A1.5-ctx8k benchmarks
• Qwen3.8-27B-ctx16k benchmarks
• Qwen3.8-27B-ctx4k benchmarks
• Flint-40B-A2.5B-NVFP4 benchmarks
• Flint-40B-A2.5B-UD-cpumoe6-ctx8k benchmarks
• Flint-40B-A2.5B-UD-ctx16k benchmarks
• Flint-40B-A2.5B-UD-ctx8k benchmarks
• glm-4.7-flash benchmarks
• gemma-4-12b-it-qat benchmarks
• Qwen3.6-35B-A3B benchmarks
• Nemotron-3.5-Bolt-74B-A9B-24GB-ctx16k benchmarks
• Nemotron-3.5-Bolt-74B-A9B-24GB-ctx8k benchmarks
• Nemotron-3.5-Bolt-74B-A9B-24GB benchmarks
• LFM2.5-24B-A2B benchmarks
• Qwen3.6-27B benchmarks
• qwen2.5-coder benchmarks
• llama3.1 benchmarks
• Qwen3-32B benchmarks
• Qwen2.5-Coder-14B-Instruct benchmarks
• Qwen3-Coder-30B-A3B-Instruct benchmarks
• gpt-oss-20b benchmarks
• Qwen3.8-27B-mtp-on benchmarks
• Qwen3.8-27B benchmarks
• qwen3.6 benchmarks
• gemma3 benchmarks
• r1 benchmarks
• qwen3-coder benchmarks
• gpt-oss benchmarks

Compare RTX 4090 head-to-head

• RTX 5090 vs RTX 4090
• RTX 4090 vs RTX 3090
• RTX 4090 vs M4 Max