Leaderboard/model/llama-3-3-70b-instruct-4bit

llama-3.3-70b-Instruct-4bit

Name: llama-3.3-70b-Instruct-4bit — community LLM benchmarks
Creator: llm-speed
License: https://www.apache.org/licenses/LICENSE-2.0
Keywords: llama-3.3-70b-Instruct-4bit, LLM benchmark, tokens per second, decode tok/s, prefill, TTFT

1 workload result across 1 hardware configuration.

Fastest known config

16.8 decode tok/s

on M3 Ultra (60-core GPU) + 96GB unified via mlx — see full run

M3 Ultra (60-core GPU) + 96GB unified

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	mlx@0.31.3	—	16.78tok/s	25.09tok/s	5,420ms	r_sx3a4y9n-m4

llama-3.3-70b-Instruct-4bit on hardware

M3 Ultra (60-core GPU) LLM benchmarks

M3 Ultra (60-core GPU) + 96GB unifiedM3 Ultra (60-core GPU) + 96GB unified

llama-3.3-70b-Instruct-4bit on hardware

M3 Ultra (60-core GPU) + 96GB unified