MI300X LLM benchmark
No MI300X LLM benchmarks yet. Be the first to submit a signed, reproducible tok/s run from your own MI300X.
Will it run local LLMs?
No signed MI300X run yet, but its 192GB of VRAM tells you what fits. At 4-bit it comfortably runs models up to about 200B parameters with room for context, and up to roughly 317B with tighter quantization. Check a specific model with the VRAM-fit tool.
No MI300X benchmarks yet.
Run on YOUR hardware to populate this page: pipx install llm-speed && llm-speed bench
$ pipx install llm-speed && llm-speed bench
Community folklore on MI300X
2 unverified claims extracted from Reddit/HN comments. Lower trust than signed runs above; every row links to the source.
- communityconfidence 55%
353.0tok/s — llama-2-70b on MI300X FP16
our signed data: MI300X · llama-2-70b
“85c28e456902a874152d145e2ea3f4d28f84ebf8 After some calculations: MI300X: * Price: $15,000 (or 1.5 million alpaca tokens) * Performance: 353 tokens/s/GPU (FP16) * Memory: 192GB HBM3 (that's a lot of context for your LLM to chew on) vs H100 * Bandwidth: 5.2 TB/s (faster than yo…”
- communityconfidence 45%
370.0tok/s — on MI300X FP16
our signed data: MI300X
“on) vs H100 * Bandwidth: 5.2 TB/s (faster than your desk llama can spit) H100: * Price: $28,000 (approximately one kidney) * Performance: 370 tokens/s/GPU (FP16), but it doesn't fit into one. * Memory: 80GB (MI300X has almost 2.5 times more VRAM!!) Key Points: 1. H100 is \~4.…”
Common questions about MI300X
Direct Q&A drawn from the runs above: fastest LLM, supported model classes, backend rankings, quantization guidance.