3.1-8B-Instruct
1 workload result across 1 hardware configuration.
Fastest config measured by llm-speed (@llmspeed)
160.8 decode tok/s
on RTX 4090 (24GB) via llama.cpp (Q4_K_M). see full run
Measured runs (1 workload result)
This run was measured by llm-speed (@llmspeed).
RTX 4090 (24GB)measured by llm-speed
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run | Source |
|---|---|---|---|---|---|---|---|
| chat-short | llama.cpp@0.7.0-dev (build 11803, commit a37cc4610) | Q4_K_M | 160.8tok/s | 4,283.1tok/s | 31.5ms | r_32ipo1hbbwl | Measured by llm-speed@llmspeed |
Can 3.1-8B-Instruct run on your hardware?
Fastest decode speed on each GPU or Mac, from llm-speed's own runs and community-submitted runs. Each page lists every quant and runtime and labels which runs llm-speed measured.
- RTX 4090 · 160.8 tok/s
3.1-8B-Instruct on hardware
Check it for your setup
- Predict your exact setup: estimate whether 3.1-8B-Instruct fits your GPU or Mac at your quant and context, next to measured speeds where we have them.
- Request this pair: ask for a measured 3.1-8B-Instruct run on hardware we have not measured. Requests are ranked by votes.
- Full run table (1 run): every 3.1-8B-Instruct result we have, with links to each source run.