Measurement wanted · request DR-003 · opened 2026-10-31
Qwen3.8-27B with MTP on RTX 5090: not measured yet
We don't have a Qwen3.8-27B with MTP speed run on RTX 5090 yet, so there is no tokens-per-second number here. People asked for it, and this page is where the result will appear. Anyone with that hardware can run it with the free llm-speed CLI, and we'll post it as a community run.
MTP (multi-token prediction) runs need a llama-server build with MTP support started with it switched on; name the model or alias with "mtp" so the run is labelled, and record the flags with --llama-args.
What's being requested
- Model
- Qwen3.8-27B with MTP · unsloth/Qwen3.8-27B-GGUF
- Hardware
- RTX 5090
- Also requested
- RTX 4090
Runs we already have
- r_9yhg3wvngab · RTX 4090 (24GB) · 75.3 tok/s decode (chat-short) · measured by llm-speed (@llmspeed)
- r_7j8rpk9mwhg · RTX 4090 (24GB) · 84.5 tok/s decode (chat-short) · community run
Run it yourself
Start llama-server with the model (flags below are a starting point; adjust offload to your machine), then point llm-speed at it. Add --llama-args='...' with the flags you used so the run records them.
pipx install llm-speed # or: pipx upgrade llm-speed
# any recent llama.cpp release
llama-server -m /path/to/MODEL.gguf --port 8080 -ngl 999
# then, in a second terminal:
LLAMA_CPP_SERVER_URL=http://127.0.0.1:8080 \
llm-speed bench --backend llama.cpp \
--model /path/to/MODEL.gguf \
--workload chat-shortThe upload is signed and public. How contributing works · what the CLI sends.
Get an alert
Email alerts for this page are switching on soon. Until then, follow @llmspeed on X, where new runs are posted.