Leaderboard/models/mlx-community-qwen2-5-coder-7b-instruct-4bit

Qwen2.5-Coder-7B-Instruct-4bit

Name: Qwen2.5-Coder-7B-Instruct-4bit — community LLM benchmarks
Creator: llm-speed
License: https://www.apache.org/licenses/LICENSE-2.0
Keywords: Qwen2.5-Coder-7B-Instruct-4bit, LLM benchmark, tokens per second, decode tok/s, prefill, TTFT

1 workload result across 1 hardware configuration.

Fastest local config

138.6 decode tok/s

on M3 Ultra (60-core GPU) + 96GB unified via mlx — see full run

Local runs (1 run)

Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.

M3 Ultra (60-core GPU) + 96GB unified

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	mlx@0.31.3	—	138.6tok/s	539.6tok/s	243ms	r_uoehjq0nvc0

Qwen2.5-Coder-7B-Instruct-4bit on hardware

M3 Ultra (60-core GPU) LLM benchmarks

M3 Ultra (60-core GPU) + 96GB unifiedM3 Ultra (60-core GPU) + 96GB unified

Qwen2.5-Coder-7B-Instruct-4bit on hardware

M3 Ultra (60-core GPU) + 96GB unified