Leaderboard/models/deepseek-r1-0528-qwen3-8b

DeepSeek-R1-0528-Qwen3-8B-MLX-4bit

Name: DeepSeek-R1-0528-Qwen3-8B-MLX-4bit: community LLM benchmarks
Creator: llm-speed
License: https://creativecommons.org/licenses/by/4.0/
Keywords: DeepSeek-R1-0528-Qwen3-8B-MLX-4bit, LLM benchmark, tokens per second, decode tok/s, prefill, TTFT

5 workload results across 2 hardware configurations.

Fastest config measured by llm-speed (@llmspeed)

155.0 decode tok/s

on RTX 4090 (24GB) via llama.cpp (Q4_K_M). see full run

Signed runs (5 workload results)

1 of 2 runs were measured by llm-speed (@llmspeed); the other one is community-submitted (signed on the submitter's own machine, not measured by llm-speed, and not used for llm-speed verdicts).

RTX 4090 (24GB)measured by llm-speed

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run	Source
chat-short	llama.cpp@0.7.0-dev (build 11803, commit a37cc4610)	Q4_K_M	155.0tok/s	2,106.7tok/s	49.8ms	r_ll6l5npoik8	Measured by llm-speed@llmspeed

M3 Ultra (60-core GPU) + 96GB unified

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run	Source
chat-short	mlx@0.31.3	4BIT	116.8tok/s	424.5tok/s	247ms	r_yy_6jfx70jq	Community-submitted
chat-long	mlx@0.31.3	4BIT	100.5tok/s	1,104.0tok/s	2,846ms	r_yy_6jfx70jq	Community-submitted
concurrent-decode	mlx@0.31.3	4BIT	110.6tok/s	no data	no data	r_yy_6jfx70jq	Community-submitted
agent-trace	mlx@0.31.3	4BIT	107.2tok/s	1,108.9tok/s	1,884ms	r_yy_6jfx70jq	Community-submitted

Can DeepSeek-R1-0528-Qwen3-8B-MLX-4bit run on your hardware?

Fastest decode speed on each GPU or Mac, from llm-speed's own runs and community-submitted runs. Each page lists every quant and runtime and labels which runs llm-speed measured.

M3 Ultra · 116.8 tok/s