RTX 4090 (24GB) LLM benchmark
The fastest LLM measured on the RTX 4090 (24GB) is gemma3 at 195.0 decode tok/s via ollama (signed run). Across 40 reproducible runs on 5 models, this page lists decode tok/s, prefill, and TTFT for each, every number linking to the run it came from.
glm-4.7-flash
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_o636l3cc-rr |
| chat-long | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_o636l3cc-rr |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_o636l3cc-rr |
| agent-trace | ollama@0.31.1 | Q4_K_M | 129.9tok/s | 2,491.1tok/s | 1,262ms | r_o636l3cc-rr |
gemma3
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | 46.95tok/s | 144.5tok/s | 817ms | r_x23y_sg24pm |
| chat-long | ollama@0.31.1 | Q4_K_M | 45.25tok/s | 1,594.1tok/s | 1,998ms | r_x23y_sg24pm |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 46.17tok/s | no data | no data | r_x23y_sg24pm |
| agent-trace | ollama@0.31.1 | Q4_K_M | 45.33tok/s | 2,556.8tok/s | 956ms | r_x23y_sg24pm |
| chat-short | ollama@0.31.1 | Q4_K_M | 92.65tok/s | 171.4tok/s | 689ms | r_3kmrc135e0e |
| chat-long | ollama@0.31.1 | Q4_K_M | 87.60tok/s | 2,206.4tok/s | 1,444ms | r_3kmrc135e0e |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 90.43tok/s | no data | no data | r_3kmrc135e0e |
| agent-trace | ollama@0.31.1 | Q4_K_M | 88.53tok/s | 3,123.9tok/s | 815ms | r_3kmrc135e0e |
| chat-short | ollama@0.31.1 | Q4_K_M | 195.0tok/s | 167.2tok/s | 706ms | r_dlanfbgym0h |
| chat-long | ollama@0.31.1 | Q4_K_M | 187.6tok/s | 3,254.1tok/s | 979ms | r_dlanfbgym0h |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 193.1tok/s | no data | no data | r_dlanfbgym0h |
| agent-trace | ollama@0.31.1 | Q4_K_M | 190.0tok/s | 3,543.4tok/s | 715ms | r_dlanfbgym0h |
r1
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_zrjj-pj93q8 |
| chat-long | ollama@0.31.1 | Q4_K_M | 81.09tok/s | 240.4tok/s | 13,070ms | r_zrjj-pj93q8 |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_zrjj-pj93q8 |
| agent-trace | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_zrjj-pj93q8 |
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_fg77v2hhohb |
| chat-long | ollama@0.31.1 | Q4_K_M | 131.9tok/s | 686.3tok/s | 4,578ms | r_fg77v2hhohb |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_fg77v2hhohb |
| agent-trace | ollama@0.31.1 | Q4_K_M | 133.8tok/s | 8,852.4tok/s | 366ms | r_fg77v2hhohb |
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_l2ck67ls40i |
| chat-long | ollama@0.31.1 | Q4_K_M | 3.76tok/s | 14.28tok/s | 219,986ms | r_l2ck67ls40i |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_l2ck67ls40i |
| agent-trace | ollama@0.31.1 | Q4_K_M | 3.79tok/s | 3,515.1tok/s | 971ms | r_l2ck67ls40i |
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_2r1p3w9ps7c |
| chat-long | ollama@0.31.1 | Q4_K_M | 131.0tok/s | 599.9tok/s | 5,238ms | r_2r1p3w9ps7c |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_2r1p3w9ps7c |
| agent-trace | ollama@0.31.1 | Q4_K_M | 132.4tok/s | 9,612.8tok/s | 337ms | r_2r1p3w9ps7c |
qwen3-coder
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | 145.2tok/s | 195.6tok/s | 562ms | r_wjq32z47vlp |
| chat-long | ollama@0.31.1 | Q4_K_M | 165.5tok/s | 3,527.2tok/s | 892ms | r_wjq32z47vlp |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 179.9tok/s | no data | no data | r_wjq32z47vlp |
| agent-trace | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_wjq32z47vlp |
gpt-oss
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | MXFP4 | no data | no data | no data | r_iu2sfa9ykvw |
| chat-long | ollama@0.31.1 | MXFP4 | 141.7tok/s | 631.5tok/s | 5,048ms | r_iu2sfa9ykvw |
| concurrent-decode | ollama@0.31.1 | MXFP4 | no data | no data | no data | r_iu2sfa9ykvw |
| agent-trace | ollama@0.31.1 | MXFP4 | no data | no data | no data | r_iu2sfa9ykvw |
Models measured on RTX 4090 (24GB)
Common questions about RTX 4090 (24GB)
Direct Q&A drawn from the runs above: fastest LLM, supported model classes, backend rankings, quantization guidance.