Leaderboard/model/qwen2-5-7b-instruct-4bit

Qwen2.5-7B-Instruct-4bit

Name: Qwen2.5-7B-Instruct-4bit — community LLM benchmarks
Creator: llm-speed
License: https://www.apache.org/licenses/LICENSE-2.0
Keywords: Qwen2.5-7B-Instruct-4bit, LLM benchmark, tokens per second, decode tok/s, prefill, TTFT

1 workload result across 1 hardware configuration.

Fastest known config

30.5 decode tok/s

on M3 Pro (18-core GPU) + 36GB unified via mlx (4bit) — see full run

M3 Pro (18-core GPU) + 36GB unified

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	mlx@0.31.3	4bit	30.52tok/s	161.9tok/s	809ms	r_llzv_g-ymaf

Qwen2.5-7B-Instruct-4bit on hardware

M3 Pro (18-core GPU) LLM benchmarks

M3 Pro (18-core GPU) + 36GB unifiedM3 Pro (18-core GPU) + 36GB unified

Qwen2.5-7B-Instruct-4bit on hardware

M3 Pro (18-core GPU) + 36GB unified