RTX 3090 (24GB) LLM benchmark
The fastest LLM measured on the RTX 3090 (24GB) is coder-v2 at 189.5 decode tok/s via ollama (signed run). Across 16 reproducible runs on 3 models, this page lists decode tok/s, prefill, and TTFT for each, every number linking to the run it came from.
coder-v2
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_0 | 189.5tok/s | 159.2tok/s | 735ms | r_o2-1w665rtq |
| chat-long | ollama@0.31.1 | Q4_0 | 77.56tok/s | 3,907.5tok/s | 838ms | r_o2-1w665rtq |
| concurrent-decode | ollama@0.31.1 | Q4_0 | 155.6tok/s | no data | no data | r_o2-1w665rtq |
| agent-trace | ollama@0.31.1 | Q4_0 | 101.2tok/s | 6,933.1tok/s | 352ms | r_o2-1w665rtq |
qwen2.5-coder
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | 69.21tok/s | 402.0tok/s | 326ms | r_6ahy-dq0f_0 |
| chat-long | ollama@0.31.1 | Q4_K_M | 65.83tok/s | 2,070.6tok/s | 1,530ms | r_6ahy-dq0f_0 |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 67.64tok/s | no data | no data | r_6ahy-dq0f_0 |
| agent-trace | ollama@0.31.1 | Q4_K_M | 63.63tok/s | 3,518.8tok/s | 483ms | r_6ahy-dq0f_0 |
| chat-short | ollama@0.31.1 | Q4_K_M | 39.86tok/s | 2.83tok/s | 46,337ms | r_pb0jpnbujji |
| chat-long | ollama@0.31.1 | Q4_K_M | 135.0tok/s | 3,319.4tok/s | 954ms | r_pb0jpnbujji |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 139.2tok/s | no data | no data | r_pb0jpnbujji |
| agent-trace | ollama@0.31.1 | Q4_K_M | 134.2tok/s | 4,111.1tok/s | 487ms | r_pb0jpnbujji |
llama3.1
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | 136.2tok/s | 2.20tok/s | 49,960ms | r_9hurqggbshk |
| chat-long | ollama@0.31.1 | Q4_K_M | 127.5tok/s | 3,004.3tok/s | 1,044ms | r_9hurqggbshk |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 133.9tok/s | no data | no data | r_9hurqggbshk |
| agent-trace | ollama@0.31.1 | Q4_K_M | 127.5tok/s | 3,475.9tok/s | 527ms | r_9hurqggbshk |
Models measured on RTX 3090 (24GB)
Common questions about RTX 3090 (24GB)
Direct Q&A drawn from the runs above: fastest LLM, supported model classes, backend rankings, quantization guidance.