Skip to content
llm-speed
Leaderboard/models/qwen3-8-27b-mtp-on

Qwen3.8-27B-mtp-on

4 workload results across 1 hardware configuration.

Fastest local config

87.0 decode tok/s

on RTX 4090 (24GB) + AMD EPYC 7642 48-Core Processor (48c) + 252GB via llama.cpp (Q4_K_M). see full run

Local runs (4 workload results)

Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.

RTX 4090 (24GB) + AMD EPYC 7642 48-Core Processor (48c) + 252GBRTX 4090 (24GB) + AMD EPYC 7642 48-Core Processor (48c) + 252GB

WorkloadBackendQuantdecode tok/sprefill tok/sTTFTRun
chat-shortllama.cpp@0.6.0-dev (build 1, commit df18bee)Q4_K_M84.51tok/s299.3tok/s515msr_7j8rpk9mwhg
chat-longllama.cpp@0.6.0-dev (build 1, commit df18bee)Q4_K_M73.10tok/s1,585.8tok/s2,033msr_7j8rpk9mwhg
concurrent-decodellama.cpp@0.6.0-dev (build 1, commit df18bee)Q4_K_M87.04tok/sno datano datar_7j8rpk9mwhg
agent-tracellama.cpp@0.6.0-dev (build 1, commit df18bee)Q4_K_M77.60tok/s2,912.1tok/s787msr_7j8rpk9mwhg

Qwen3.8-27B-mtp-on on hardware