Qwen3-8B-4bit
4 workload results across 1 hardware configuration.
Fastest local config
120.5 decode tok/s
on M3 Ultra (60-core GPU) + 96GB unified via mlx. see full run
Local runs (4 runs)
Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.
M3 Ultra (60-core GPU) + 96GB unified
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | mlx@0.31.3 | - | 120.5tok/s | 446.1tok/s | 247ms | r_b1awwhkoo2v |
| chat-long | mlx@0.31.3 | - | 103.3tok/s | 1,108.8tok/s | 2,838ms | r_b1awwhkoo2v |
| concurrent-decode | mlx@0.31.3 | - | 114.3tok/s | no data | no data | r_b1awwhkoo2v |
| agent-trace | mlx@0.31.3 | - | 109.4tok/s | 1,109.1tok/s | 1,885ms | r_b1awwhkoo2v |