RTX 3090 (24GB) LLM benchmark
The fastest LLM measured on the RTX 3090 (24GB) is coder-v2 at 189.5 decode tok/s via ollama (signed run). Across 16 reproducible runs on 3 models, this page lists decode tok/s, prefill, and TTFT for each, every number linking to the run it came from.
coder-v2
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_0 | 189.5tok/s | 159.2tok/s | 735ms | r_o2-1w665rtq |
| chat-long | ollama@0.31.1 | Q4_0 | 77.56tok/s | 3,907.5tok/s | 838ms | r_o2-1w665rtq |
| concurrent-decode | ollama@0.31.1 | Q4_0 | 155.6tok/s | no data | no data | r_o2-1w665rtq |
| agent-trace | ollama@0.31.1 | Q4_0 | 101.2tok/s | 6,933.1tok/s | 352ms | r_o2-1w665rtq |
qwen2.5-coder
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | 69.21tok/s | 402.0tok/s | 326ms | r_6ahy-dq0f_0 |
| chat-long | ollama@0.31.1 | Q4_K_M | 65.83tok/s | 2,070.6tok/s | 1,530ms | r_6ahy-dq0f_0 |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 67.64tok/s | no data | no data | r_6ahy-dq0f_0 |
| agent-trace | ollama@0.31.1 | Q4_K_M | 63.63tok/s | 3,518.8tok/s | 483ms | r_6ahy-dq0f_0 |
| chat-short | ollama@0.31.1 | Q4_K_M | 39.86tok/s | 2.83tok/s | 46,337ms | r_pb0jpnbujji |
| chat-long | ollama@0.31.1 | Q4_K_M | 135.0tok/s | 3,319.4tok/s | 954ms | r_pb0jpnbujji |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 139.2tok/s | no data | no data | r_pb0jpnbujji |
| agent-trace | ollama@0.31.1 | Q4_K_M | 134.2tok/s | 4,111.1tok/s | 487ms | r_pb0jpnbujji |
llama3.1
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | 136.2tok/s | 2.20tok/s | 49,960ms | r_9hurqggbshk |
| chat-long | ollama@0.31.1 | Q4_K_M | 127.5tok/s | 3,004.3tok/s | 1,044ms | r_9hurqggbshk |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 133.9tok/s | no data | no data | r_9hurqggbshk |
| agent-trace | ollama@0.31.1 | Q4_K_M | 127.5tok/s | 3,475.9tok/s | 527ms | r_9hurqggbshk |
Community folklore on RTX 3090 (24GB)
401 unverified claims extracted from Reddit/HN comments. Lower trust than signed runs above; every row links to the source.
- communityconfidence 85%
10.00tok/s — qwen-2.5-32B on RTX 3090 via ollama Q8
our signed data: RTX 3090 · qwen-2.5-32B
“a beast! 28 tok/sec at 32K context is more than usable for a lot of coding situations. * The P40s continue to surprise. A single P40 can do 10 tok/sec, which is perfectly usable. * 3xP40 fits 120K context at Q8 comfortably. * performance doesn't scale with more P40s. Using `-sm r…”
- communityconfidence 85%
28.00tok/s — qwen-2.5-32B on RTX 3090 via ollama Q8
our signed data: RTX 3090 · qwen-2.5-32B
“ase of qwen-2.5-32B today. I bench marked the Q4 and Q8 quants on my local rig (3xP40, 1x3090). Some observations: * the 3090 is a beast! 28 tok/sec at 32K context is more than usable for a lot of coding situations. * The P40s continue to surprise. A single P40 can do 10 tok/se…”
- communityconfidence 85%
10.00tok/s — qwen-2.5-32B on RTX 3090 via ollama Q8
our signed data: RTX 3090 · qwen-2.5-32B
“a beast! 28 tok/sec at 32K context is more than usable for a lot of coding situations. * The P40s continue to surprise. A single P40 can do 10 tok/sec, which is perfectly usable. * 3xP40 fits 120K context at Q8 comfortably. * performance doesn't scale with more P40s. Using `-sm r…”
- communityconfidence 85%
28.00tok/s — qwen-2.5-32B on RTX 3090 via ollama Q8
our signed data: RTX 3090 · qwen-2.5-32B
“ase of qwen-2.5-32B today. I bench marked the Q4 and Q8 quants on my local rig (3xP40, 1x3090). Some observations: * the 3090 is a beast! 28 tok/sec at 32K context is more than usable for a lot of coding situations. * The P40s continue to surprise. A single P40 can do 10 tok/se…”
- communityconfidence 85%
65.00tok/s — Qwen3-Coder-Next on RTX 3090 via vllm FP8
our signed data: RTX 3090 · Qwen3-Coder-Next
“Next-FP8-Dynamic quant with vLLM v0.16.0 over 4 RTX 3090s with a 200k context window. I get around 71tps at low context, dropping to around 65tps at the 100k context point. Something to note, I'm seeing pretty bad issues with vLLM nightly, affecting this model especially, that c…”
- communityconfidence 85%
71.00tok/s — Qwen3-Coder-Next on RTX 3090 via vllm FP8
our signed data: RTX 3090 · Qwen3-Coder-Next
“sted. I'm running Unsloth's Qwen3-Coder-Next-FP8-Dynamic quant with vLLM v0.16.0 over 4 RTX 3090s with a 200k context window. I get around 71tps at low context, dropping to around 65tps at the 100k context point. Something to note, I'm seeing pretty bad issues with vLLM nightly…”
- communityconfidence 85%
65.00tok/s — Qwen3-Coder-Next on RTX 3090 via vllm FP8
our signed data: RTX 3090 · Qwen3-Coder-Next
“Next-FP8-Dynamic quant with vLLM v0.16.0 over 4 RTX 3090s with a 200k context window. I get around 71tps at low context, dropping to around 65tps at the 100k context point. Something to note, I'm seeing pretty bad issues with vLLM nightly, affecting this model especially, that c…”
- communityconfidence 85%
71.00tok/s — Qwen3-Coder-Next on RTX 3090 via vllm FP8
our signed data: RTX 3090 · Qwen3-Coder-Next
“sted. I'm running Unsloth's Qwen3-Coder-Next-FP8-Dynamic quant with vLLM v0.16.0 over 4 RTX 3090s with a 200k context window. I get around 71tps at low context, dropping to around 65tps at the 100k context point. Something to note, I'm seeing pretty bad issues with vLLM nightly…”
- communityconfidence 85%
65.00tok/s — Qwen3-Coder-Next on RTX 3090 via vllm FP8
our signed data: RTX 3090 · Qwen3-Coder-Next
“Next-FP8-Dynamic quant with vLLM v0.16.0 over 4 RTX 3090s with a 200k context window. I get around 71tps at low context, dropping to around 65tps at the 100k context point. Something to note, I'm seeing pretty bad issues with vLLM nightly, affecting this model especially, that c…”
- communityconfidence 85%
71.00tok/s — Qwen3-Coder-Next on RTX 3090 via vllm FP8
our signed data: RTX 3090 · Qwen3-Coder-Next
“sted. I'm running Unsloth's Qwen3-Coder-Next-FP8-Dynamic quant with vLLM v0.16.0 over 4 RTX 3090s with a 200k context window. I get around 71tps at low context, dropping to around 65tps at the 100k context point. Something to note, I'm seeing pretty bad issues with vLLM nightly…”
Models measured on RTX 3090 (24GB)
Common questions about RTX 3090 (24GB)
Direct Q&A drawn from the runs above: fastest LLM, supported model classes, backend rankings, quantization guidance.