Skip to content
llm-speed

Free tool · source benchmarks

Look up measured tokens per second

Find measured tokens per second for a model and GPU or Mac. Compare separate model sizes, workloads and reported settings before estimating what your setup might deliver. Every displayed rate links to its source.

Compare the reported settings in each result.

Qwen3.8-27B on RTX 5090

33 measured workload rows from 25 source runs, separated into 33 reported-settings groups. Choose the size, workload and settings closest to yours. Each range describes its included rows; it is not a forecast or confidence interval.

Size not reported · long-context-decay · Q4_K_M67.96tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 1 · filled context: 15,858 prompt tokens · output: 256 tokens.

Observed range: 67.96tok/s–67.96tok/s.

Size not reported · chat-short · Q4_K_M71.27tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 1 · context: 114 tokens · input/output: 114/256 tokens.

Observed range: 71.27tok/s–71.27tok/s.

Size not reported · concurrent-decode · Q4_K_M18.64tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 4 · context: 32241 tokens · input/output: 32241/256 tokens.

Observed range: 18.64tok/s–18.64tok/s.

Size not reported · concurrent-decode · Q4_K_M35.81tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 2 · context: 32241 tokens · input/output: 32241/256 tokens.

Observed range: 35.81tok/s–35.81tok/s.

Size not reported · concurrent-decode · Q4_K_M64.96tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 1 · context: 32241 tokens · input/output: 32241/256 tokens.

Observed range: 64.96tok/s–64.96tok/s.

Size not reported · concurrent-decode · Q4_K_M17.48tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 8 · context: 15857 tokens · input/output: 15857/256 tokens.

Observed range: 17.48tok/s–17.48tok/s.

Size not reported · concurrent-decode · Q4_K_M25.79tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 4 · context: 15857 tokens · input/output: 15857/256 tokens.

Observed range: 25.79tok/s–25.79tok/s.

Size not reported · concurrent-decode · Q4_K_M44.20tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 2 · context: 15857 tokens · input/output: 15857/256 tokens.

Observed range: 44.20tok/s–44.20tok/s.

Size not reported · concurrent-decode · Q4_K_M67.82tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 1 · context: 15857 tokens · input/output: 15857/256 tokens.

Observed range: 67.82tok/s–67.82tok/s.

Size not reported · concurrent-decode · Q4_K_M17.25tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 32 · context: 991 tokens · input/output: 991/256 tokens.

Observed range: 17.25tok/s–17.25tok/s.

Size not reported · concurrent-decode · Q4_K_M30.31tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 16 · context: 991 tokens · input/output: 991/256 tokens.

Observed range: 30.31tok/s–30.31tok/s.

Size not reported · concurrent-decode · Q4_K_M46.13tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 8 · context: 991 tokens · input/output: 991/256 tokens.

Observed range: 46.13tok/s–46.13tok/s.

Size not reported · concurrent-decode · Q4_K_M45.29tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 4 · context: 991 tokens · input/output: 991/256 tokens.

Observed range: 45.29tok/s–45.29tok/s.

Size not reported · concurrent-decode · Q4_K_M58.18tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 2 · context: 991 tokens · input/output: 991/256 tokens.

Observed range: 58.18tok/s–58.18tok/s.

Size not reported · concurrent-decode · Q4_K_M71.02tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.23

Batch: 1 · context: 991 tokens · input/output: 991/256 tokens.

Observed range: 71.02tok/s–71.02tok/s.

Size not reported · chat-short · Q4_K_M70.28tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.18

Batch: 1 · context: 154 tokens · input/output: 154/256 tokens.

Observed range: 70.28tok/s–70.28tok/s.

Size not reported · chat-long · Q4_K_M70.15tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.18

Batch: 1 · context: 3224 tokens · input/output: 3224/1024 tokens.

Observed range: 70.15tok/s–70.15tok/s.

Size not reported · chat-short · Q4_K_M68.83tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.18

Batch: 1 · context: 154 tokens · input/output: 154/256 tokens.

Observed range: 68.83tok/s–68.83tok/s.

Size not reported · chat-long · Q4_K_M70.09tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12158, commit e8df20d23) · suite suite-v1 · client 0.0.18

Batch: 1 · context: 3224 tokens · input/output: 3224/872 tokens.

Observed range: 70.09tok/s–70.09tok/s.

Size not reported · long-context-decay · Q4_K_M71.55tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12157, commit fecdc81ee) · suite suite-v1 · client 0.0.18

Batch: 1 · filled context: 11,827 prompt tokens · output: 73 tokens.

Observed range: 71.55tok/s–71.55tok/s.

Size not reported · long-context-decay · UD-Q4_K_XL76.94tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12157, commit fecdc81ee) · suite suite-v1 · client 0.0.18

Batch: 1 · filled context: 11,827 prompt tokens · output: 125 tokens.

Observed range: 76.94tok/s–76.94tok/s.

Size not reported · chat-short · UD-Q4_K_XL79.54tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12157, commit fecdc81ee) · suite suite-v1 · client 0.0.18

Batch: 1 · context: 114 tokens · input/output: 114/256 tokens.

Observed range: 79.54tok/s–79.54tok/s.

Size not reported · chat-short · Q4_K_M74.18tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.9.0-dev (build 12157, commit fecdc81ee) · suite suite-v1 · client 0.0.18

Batch: 1 · context: 114 tokens · input/output: 114/256 tokens.

Observed range: 74.18tok/s–74.18tok/s.

Size not reported · chat-short · Q4_K_M75.19tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.7.0-dev (build 11669, commit 620d2225c) · suite suite-v1 · client 0.0.7

Batch: 1 · context: 154 tokens · input/output: 154/256 tokens.

Observed range: 75.19tok/s–75.19tok/s.

Size not reported · chat-long · Q4_K_M74.57tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.7.0-dev (build 11669, commit 620d2225c) · suite suite-v1 · client 0.0.7

Batch: 1 · context: 3224 tokens · input/output: 3224/872 tokens.

Observed range: 74.57tok/s–74.57tok/s.

Size not reported · concurrent-decode · Q4_K_M75.40tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.7.0-dev (build 11669, commit 620d2225c) · suite suite-v1 · client 0.0.7

Batch: 1 · context: 1024 tokens · input/output: 1024/256 tokens.

Observed range: 75.40tok/s–75.40tok/s.

Size not reported · agent-trace · Q4_K_M73.52tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp0.7.0-dev (build 11669, commit 620d2225c) · suite suite-v1 · client 0.0.7

Batch: 1 · context: 3496 tokens · input/output: 3496/840 tokens.

Observed range: 73.52tok/s–73.52tok/s.

Size not reported · chat-short · Q4_K_M65.66tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp commit 9725a31 · suite suite-v1 · client 0.0.5

Batch: 1 · context: 114 tokens · input/output: 114/256 tokens.

Observed range: 65.66tok/s–65.66tok/s.

Size not reported · chat-long · Q4_K_M65.10tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp commit 9725a31 · suite suite-v1 · client 0.0.5

Batch: 1 · context: 3184 tokens · input/output: 3184/874 tokens.

Observed range: 65.10tok/s–65.10tok/s.

Size not reported · chat-short · Q4_K_M65.73tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp commit 9725a31 · suite suite-v1 · client 0.0.5

Batch: 1 · context: 114 tokens · input/output: 114/256 tokens.

Observed range: 65.73tok/s–65.73tok/s.

Size not reported · chat-long · Q4_K_M65.09tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp commit 9725a31 · suite suite-v1 · client 0.0.5

Batch: 1 · context: 3184 tokens · input/output: 3184/874 tokens.

Observed range: 65.09tok/s–65.09tok/s.

Size not reported · chat-short · Q4_K_M65.43tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp commit 9725a31 · suite suite-v1 · client 0.0.5

Batch: 1 · context: 114 tokens · input/output: 114/256 tokens.

Observed range: 65.43tok/s–65.43tok/s.

Size not reported · chat-long · Q4_K_M65.10tok/s observed median · 1 row / 1 run

RTX 5090 (32GB)

llama.cpp commit 9725a31 · suite suite-v1 · client 0.0.5

Batch: 1 · context: 3184 tokens · input/output: 3184/874 tokens.

Observed range: 65.10tok/s–65.10tok/s.

Hardware benchmarks · Model benchmarks

How to estimate LLM speed from benchmarks

Start with the same model size, quantization, hardware configuration and runtime. Then match the workload, batch size and actual token lengths. A short chat result does not establish speed for a long prompt or several simultaneous users.

Tokens per second and waiting time answer different questions

Decode tok/s measures token generation speed. First-token latency captures the wait before generation starts. Keep both alongside input and output lengths when comparing a chat or coding workflow. Tokenizers differ, so equal token rates do not guarantee equal useful work.

What the groups mean

This lookup includes successful positive decode measurements from matching model and hardware families in up to the latest 500 listed submissions. Groups preserve reported model size, quantization, backend and version, hardware summary, suite/client versions, workload, batch, context and input/output lengths. Missing configuration fields keep separate source runs apart.

Matching reported fields does not prove identical prompts, model files, runtime flags or hardware. The median and minimum/maximum describe the included rows; they do not predict an unmeasured setup. Open the source run and use the methodology to assess comparability.

For a hardware purchase, check the memory estimator and tested model files as well. A fast submitted run alone does not establish that your chosen file and context will fit.

Full benchmark table · Contribute a measurement · Download benchmark data