Leaderboard/model/llama-3-1-8b-instruct-4bit

Llama-3.1-8B-Instruct-4bit

Name: Llama-3.1-8B-Instruct-4bit — community LLM benchmarks
Creator: llm-speed
License: https://www.apache.org/licenses/LICENSE-2.0
Keywords: Llama-3.1-8B-Instruct-4bit, LLM benchmark, tokens per second, decode tok/s, prefill, TTFT

1 workload result across 1 hardware configuration.

Fastest known config

29.2 decode tok/s

on M3 Pro (18-core GPU) + 36GB unified via mlx — see full run

M3 Pro (18-core GPU) + 36GB unified

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	mlx@0.31.3	—	29.20tok/s	203.3tok/s	669ms	r_h0-use1ypnb

Llama-3.1-8B-Instruct-4bit on hardware

M3 Pro (18-core GPU) LLM benchmarks

M3 Pro (18-core GPU) + 36GB unifiedM3 Pro (18-core GPU) + 36GB unified

Llama-3.1-8B-Instruct-4bit on hardware

M3 Pro (18-core GPU) + 36GB unified