llm-speed
LLM speed benchmarks: community-verified tok/s for every model + hardware combo
The benchmark suite for local + hosted LLM inference.
Run it. See your tok/s. Compare your rig.
$ pipx install llm-speed && llm-speed benchRuns in about a minute. Auto-detects your hardware and backends. Open source.
See a signed 287 tok/s run