Qwen3.8-27B on a 24GB GPU at 64K filled context: not measured yet

Runs we already have

• r_z6--z4sihzt · RTX 4090 (24GB) · 42.7 tok/s decode (chat-long) · measured by llm-speed (@llmspeed)
• r_do9h933nvme · RTX 4090 (24GB) · 43.5 tok/s decode (concurrent-decode) · measured by llm-speed (@llmspeed)
• r_x0ugw32ix4y · RTX 4090 (24GB) · 39.0 tok/s decode (long-context-decay) · measured by llm-speed (@llmspeed)
• r_f0etksoa6-y · RTX 4090 (24GB) · 41.2 tok/s decode (long-context-decay) · measured by llm-speed (@llmspeed)
• r_qfrb2zmyk0k · RTX 5090 (32GB) · 75.4 tok/s decode (concurrent-decode) · measured by llm-speed (@llmspeed)
• r_ptct8o01o5l · RTX 3090 (24GB) · 37.6 tok/s decode (chat-short) · community run
• r_11kn1epxewh · RTX 4090 (24GB) · 43.1 tok/s decode (chat-short) · community run
• r_dz5a3lzxhrj · RTX 5090 (32GB) · 65.7 tok/s decode (chat-short) · measured by llm-speed (@llmspeed)
• r_c3ps9wjygi9 · RTX 5090 (32GB) · 65.7 tok/s decode (chat-short) · measured by llm-speed (@llmspeed)
• r_7mod38qsldj · RTX 5090 (32GB) · 65.4 tok/s decode (chat-short) · measured by llm-speed (@llmspeed)

Run it yourself

Start llama-server with the model (flags below are a starting point; adjust offload to your machine), then point llm-speed at it. Add --llama-args='...' with the flags you used so the run records them.

pipx install llm-speed   # or: pipx upgrade llm-speed
# any recent llama.cpp release
llama-server -m /path/to/MODEL.gguf --port 8080 -ngl 999 -c 69632
# then, in a second terminal:
LLAMA_CPP_SERVER_URL=http://127.0.0.1:8080 \
  llm-speed bench --backend llama.cpp \
  --model /path/to/MODEL.gguf \
  --workload long-context-decay --llama-args='-c 69632'

The upload is signed and public. How contributing works · what the CLI sends.