Why do Qwen3.8-27B tok/s headlines differ from one chat? A measured single-chat baseline
Published 2026-11-12
Measured single-chat baseline
| Card | Context | Decode tok/s | Prefill tok/s | TTFT | llama.cpp build | Run |
|---|---|---|---|---|---|---|
| RTX 5090 (32GB) | 4K | 75.2 | 563.4 | 273.4 ms | b11669 (620d2225c) | r_qfrb2zmyk0k |
| RTX 3090 (24GB) | 4K | 38.61 | 482.3 | 319.3 ms | b11803 (a37cc4610) | r_mmtzel_i367 |
| RTX 3090 (24GB) | 8K | 38.83 | 479.1 | 321.5 ms | b11803 (a37cc4610) | r_w40_vv7f77w |
| RTX 3090 (24GB) | 16K | 38.57 | 482.7 | 319.0 ms | b11803 (a37cc4610) | r_gscy1f7ey_k |
| RTX 4090 (24GB) | 4K | 43.38 | 448.3 | 343.6 ms | b11803 (a37cc4610) | r_7cap38a021- |
The 75.2 and 38.61 figures are not a same-build comparison. The RTX 5090 yardstick is llama.cpp b11669, received 15 Oct 2026, 154-token prompt, 256-token output, batch size 1. The RTX 3090 and RTX 4090 4K rows are llama.cpp b11803, received 22 Oct 2026, same prompt length and batch size. There is no published Qwen3.8-27B chat-short control on the RTX 5090 at b11803. An earlier RTX 5090 set, same model and quant, chat-short, llama.cpp commit 9725a31 (5 Sep 2026), is a median 65.66 tok/s from three repeats: r_c3ps9wjygi9 (65.73), r_dz5a3lzxhrj (65.66), r_7mod38qsldj (65.43). Each used a 114-token prompt. That only shows that baselines move with the build. Do not average 65.66 with 75.2.
On the RTX 3090, same build b11803, still one chat, chat-short decode stayed within 0.3 tok/s from 4K to 16K: 38.61 (r_mmtzel_i367), 38.83 (r_w40_vv7f77w), 38.57 (r_gscy1f7ey_k). A second 8K chat-short run on the same card and build is 38.67 (r_7250vqsziqj).
The RTX 4090 4K companion above is 43.38 tok/s (raw 43.3845, r_7cap38a021-), prefill 448.3 tok/s, TTFT 343.6 ms, b11803, chat-short. It is not a 5090 comparison.
Same run, still batch size 1, on the 5090 yardstick r_qfrb2zmyk0k at 4K: chat-short 75.2, chat-long 74.6 (raw 74.5688, prefill 2,499.5 tok/s, TTFT 1,289.9 ms, 3,224-token prompt), agent-trace 73.5 (raw 73.5219). A longer prompt mainly changes prefill and TTFT, not decode, in this stock setup. The flattened concurrent-decode row on that run is 75.4 tok/s at batch size 1 (1,024-token prompt).
What the yardstick is
Stock llama.cpp, one chat at a time, no draft model, Q4_K_M, a short prompt, and a 4K window. These suite-v1 chat-short rows are batch size 1, and the model label has no MTP or draft variant. That is the same single-chat, no-drafter yardstick as /can-run/qwen3-8-27b/rtx-5090 and the live Q4_K_M 4K comparison on /m/qwen3-8-27b. The 4K run pages are r_qfrb2zmyk0k and r_mmtzel_i367. This is a baseline, not a claim that nobody can go faster. Thinking is not included in the 273.4 ms TTFT.
Reported headlines
Each block below is labelled reported and linked. The source's own words name the variables. Some claims are described by their authors as one stream or one generation. Others are described as a combined total. llm-speed has not repeated these setups, so these blocks do not confirm or reject the figures. None of them belong in the measured table.
Reported
YASAKANI, 1,260 tok/s on one RTX 5090. Unverified developer claim.
yasakani_dev, 10 Nov 2026, 21:02 SGT announced a free release for 11/13(金)22時. The post's own footnote: one RTX 5090, Qwen3.8-27B, single generation (単一生成). 1,260 tok/s is long JSONL generation, thinking off, under the author's comparison conditions against eight engines in seven tasks. Variables the source names: the YASAKANI engine, one 5090, Qwen3.8-27B, a single generation, long JSONL, thinking off, and a Windows + RTX 5090 description. llm-speed has not verified this figure. Our measurement of this setup: not yet.
Reported
TensorFold / M3TiS on one RTX 3090: 208 tok/s code, 147 tok/s chat, 45 without a draft model.
The model card says "Measured on one RTX 3090 (2026-10-13)" by the author, not by llm-speed: median of 3, single user, 262K window, 4-bit context store, 256-token replies. Greedy: code (write a Fibonacci function) 208, chat (explain GPUs, no thinking) 147. "Plain decoding, no draft model (for reference)": 45. Draft model: z-lab/Qwen3.8-27B-DFlash2. Weights: Vontra/Qwen3.8-27B-MLX-4bit. The card describes M3TiS as a TensorFold native (Zig) server build with 3090-specific changes. Variables the source names: TensorFold native / M3TiS, a DFlash2 draft model, code vs chat, a 4-bit KV cache, a 262K context, and 4-bit MLX weights rather than the GGUF Q4_K_M in the measured table. Our measurement of this setup: not yet.
Reported
ashxhart, ~700 tok/s on a single RTX 5090, Qwen3.8-27B (TensorFold / NVFP4).
This is the lead reported example of a headline the author later described as a combined total across parallel coding lanes. The original demo post is deleted. Do not treat that URL as a live post: ashxhart, 11 Nov 2026, 07:09 SGT. Ash later said the deletion was accidental: "I accidentally deleted it thinking it was as another post I did earlier 😭😭😭😭😭😭" (12 Nov 2026, 03:38, Asia/Singapore). Shortly after the post went up, @lasse_ai asked "Is that single stream?" and Ash replied "YEAH HAHAHAH". That is a jokey reply, not a confirmation that ~700 was one stream.
Later the same day (11 Nov 2026, 19:22, Asia/Singapore), @DevSalim asked whether he was branching from a single prompt and then running multiple streams. Ash replied, in his own words: "Yeah dude, 8 parallel coding lanes on the one 5090. ~700tps combined for that task, it'll vary with what you're running." That is the clearest remaining statement about what the ~700 measured.
Next to the measured 75.2 at 4K (r_qfrb2zmyk0k): Ash describes ~700 tok/s as combined across 8 parallel coding lanes, which works out to roughly 88 tok/s per lane (our division of his number, not his figure). This isn't measured by us. Per-lane speed with 8 lanes running at once and single-chat speed are not the same measure: per-stream speed usually drops as lanes are added. The RTX 5090 run also recorded 8 requests at once (about 42.5 tok/s decode per request; the 111.3 tok/s combined figure counts total output over wall time, including prompt processing; 1K-token prompt, stock llama.cpp Q4_K_M), a different setup from Ash's TensorFold/NVFP4 lanes. A same-setup 8-lane run is planned. The setups are also not like-for-like. His remaining replies name TensorFold, NVFP4, 256K context, and Windows (reply). The yardstick is stock llama.cpp Q4_K_M, chat-short, 4K. Our measurement of this setup: not yet.
Later replies in other threads, quoted so they are not used to overwrite the "~700tps combined" / "8 parallel coding lanes" wording, and not used to re-assert single-stream for the deleted demo: "I am doing this on a single stream" (contrasting a 64-request batch on a 3090), "stream parallelism", "300tps decode on prose with 4 lanes", and "I already have it working over 16 lanes 😄" Remaining author-stated variables: one RTX 5090, Qwen3.8-27B, a TensorFold custom kernel on Windows, NVFP4, 256K context, "8 parallel coding lanes," and "~700tps combined for that task." The workload in that reply is coding lanes. The later "4 lanes" post is prose.
Reported, different model or hardware
TensorFold on an M5 Pro Mac mini, Qwen3.8-27B 4-bit plus DFlash2: a Reddit post reports 40–60 tok/s. Different machine, still a draft-model setup. r/LocalLLaMA thread.
Basalt vs llama.cpp on Qwen3.8-Flash-Next (not 27B), one RTX 5090, Q2_0, reported by witcheer: 259 vs 89 tok/s without drafting, 500 vs 98 with each engine's own drafting. witcheer, 10 Nov 2026, 22:00 SGT. This is not a 27B result.
Why the numbers differ
One chat vs many chats
A headline can be the total across several requests running at once, or the speed of one request. These are different quantities. The yardstick is one chat (batch size 1). Some headlines really are totals. Others are not. For the ~700 tok/s TensorFold demo, Ash's later wording in the same thread is "8 parallel coding lanes on the one 5090. ~700tps combined for that task." That is a combined-lane figure, and it is the lead example of this variable. Next to the measured 75.2 at 4K, that works out to roughly 88 tok/s per lane (our division of his number, not his figure; not measured by us). Per-lane speed with 8 lanes running at once and single-chat speed are not the same measure: per-stream speed usually drops as lanes are added. The RTX 5090 run also recorded 8 requests at once (about 42.5 tok/s decode per request; the 111.3 tok/s combined figure counts total output over wall time, including prompt processing; 1K-token prompt, stock llama.cpp Q4_K_M), a different setup from Ash's TensorFold/NVFP4 lanes. A same-setup 8-lane run is planned. The earlier "YEAH HAHAHAH" is not a single-stream confirmation. YASAKANI's footnote still describes 1,260 tok/s as a single generation (an unverified developer claim). A large figure is not automatically a sum across parallel requests.
On r_qfrb2zmyk0k, at a 1,024-token prompt, the public extras list a concurrency curve. Batch size 1 is 75.4 tok/s per request, the flattened concurrent-decode row. At 4, 8 and 16 requests, per-request decode is 40.2, 42.5 and 43.0 tok/s. The combined figures are 117.2, 111.3 and 112.8 tok/s. Each combined figure counts total output over wall time, including prompt processing. The 8-request point is about 42.5 tok/s decode per request; the 111.3 tok/s combined figure counts total output over wall time, including prompt processing. That curve is not a like-for-like test of Ash's demo.
Draft or speculative models
A smaller model, or extra heads on the same model, guesses upcoming tokens and the main model checks them. When guesses are accepted, decode can rise a lot. Measured by llm-speed (@llmspeed), one form of this is MTP inside llama.cpp, not TensorFold or YASAKANI. Same card, same build, Qwen3.8-27B Q4_K_M, RTX 4090 (24GB), llama.cpp 0.8.0-dev build 12017, commit 5047748f1, 31 Oct 2026, chat-short, batch size 1.
| Setting | Decode tok/s | Prefill tok/s | TTFT | Run |
|---|---|---|---|---|
| MTP off | 43.11 (raw 43.1054) | 440.4 | 349.7 ms | r_do9h933nvme |
| MTP on | 75.26 (raw 75.2617) | 274.6 | 560.7 ms | r_9yhg3wvngab |
The M3TiS card reports 45 tok/s without its DFlash2 draft and 208 / 147 with it. That block is reported, on a different engine and different weights. An older MTP pair, r_7j8rpk9mwhg and r_11kn1epxewh, is labelled community in the live dataset and is not used as an llm-speed measurement.
Workload
Code, chat, and long JSON are not the same job. YASAKANI's 1,260 is reported as long JSONL, thinking off. M3TiS reports 208 on a code prompt and 147 on a chat prompt, same machine. The yardstick is the suite chat-short prompt: 154 tokens in, 256 out.
Quant, context length, and KV-cache settings
The baseline is Q4_K_M GGUF at a 4K window. M3TiS reports 4-bit MLX weights, a 262K window, and a 4-bit context store. Changing any of these can move speed and what fits. On the RTX 3090, raising the configured window from 4K to 16K barely moved chat-short decode (38.61 to 38.57, r_mmtzel_i367 and r_gscy1f7ey_k). That does not say what a 262K 4-bit cache does in another engine. See decode tok/s, prefill tok/s, TTFT, KV cache, and context length.
Prompt speed vs decode speed
Decode is how fast new tokens appear after the first one. Prefill (prompt tok/s) is how fast the prompt is read. TTFT is the wait for the first token. On r_qfrb2zmyk0k at 4K those are 75.2 decode, 563.4 prefill, and 273.4 ms TTFT. A headline that quotes prefill, or a tokens-per-second figure that mixes the two, will not match a decode number. Decode vs prefill tok/s explains the split.
What is not measured yet
We have not measured TensorFold or YASAKANI. The figures above stay reported.
llama.cpp build 12157 (fecdc81ee) already has an RTX 5090 chat-short run at 74.18 tok/s (r_tqajf0g23ca, 114-token prompt), labelled Measured by llm-speed (@llmspeed), on /m/qwen3-8-27b. The planned move of this post's main comparison to that 74.18 figure plus 3090/4090 reruns on build 12157 is superseded: the /m/qwen3-8-27b card now uses the same-build llama.cpp build 12158 (e8df20d) 4K controls (71.3 / 43.8 / 39.0 tok/s on the RTX 5090 / 4090 / 3090), a build change, not a regression. This post's comparison stays on 75.2, with the published 38.61 and 43.38 figures and the build footnote. The live 75.2 (different llama.cpp build (11669 vs 12157 or 12158) and prompt (154 vs 114 tokens)) must not be read against those later runs as if they were the same build.
| Engine | Hardware | Model / quant | Decode tok/s | Prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| TensorFold (planned) | M3 Ultra | Qwen3.8-27B MLX-4bit, same file as the baseline | — | — | — | pending Security/QA |
| mlx-lm baseline (planned) | M3 Ultra | Qwen3.8-27B MLX-4bit | — | — | — | pending Security/QA |
The M3TiS 3090 card figures (208 / 147 / 45) stay in the reported block only.
YASAKANI, after 13 Nov 2026, and only once Security has signed off on how it will be installed and run: one chat and, if the method allows, the engine's advertised workload, recorded exactly. Until that run exists, 1,260 tok/s stays an unverified developer claim.
- No published RTX 5090 Qwen3.8-27B chat-short control on the same b11803 build as the 3090/4090 4K pair. The /m/qwen3-8-27b card now uses the same-build llama.cpp build 12158 (e8df20d) 4K set (71.3 / 43.8 / 39.0 tok/s) instead. This post's comparison stays on 75.2, with the published 38.61 and 43.38 figures and the build footnote. A 5090 run at 74.18 tok/s on build 12157 (r_tqajf0g23ca) remains on that model page as a different-build point, not this post's main comparison.
- The concurrency curve above is already in the public extras. A same-setup 8-lane run is planned. The yardstick remains one chat.
- No 32K or 64K Qwen3.8-27B rows on the RTX 3090 are in the published dataset. A longer-context control has been discussed. It is not on the site yet.
- Exact flags for some reported setups are still incomplete. YASAKANI's full config is not in the post cited above. Ash's remaining replies name TensorFold, NVFP4, 256K, Windows, and "8 parallel coding lanes" / "~700tps combined." They do not give a full flag set, and the original demo post and its video are gone.
Methodology and run links
A run page is one signed upload. Open it for the raw decode, prefill, TTFT, prompt length, batch size, and llama.cpp build. How the suite is defined: methodology. The current download is /data (llm-speed-runs-2026-11-06.csv). Decode, prefill, and TTFT: decode vs prefill and the glossary.
More of the same measured rows: Qwen3.8-27B, RTX 5090 vs RTX 4090, can an RTX 5090 run it, can an RTX 3090 run it, RTX 5090, RTX 3090, RTX 4090, VRAM fit, and why a local LLM is slow (a troubleshooting list, not this comparison).
FAQ
How fast is Qwen3.8-27B on an RTX 5090 in one chat?
Measured by llm-speed (@llmspeed): 75.2 tok/s decode for Qwen3.8-27B Q4_K_M, stock llama.cpp b11669, one chat, no draft model, short prompt, 4K context (https://llm-speed.com/r/r_qfrb2zmyk0k).
How fast is it on an RTX 3090?
Measured by llm-speed (@llmspeed): 38.61 tok/s decode at the same model, quant, short prompt and 4K context, stock llama.cpp b11803, one chat, no draft model (https://llm-speed.com/r/r_mmtzel_i367). That is a different llama.cpp build from the 5090 run.
Is 1,260 tok/s on one RTX 5090 real?
It is an unverified developer claim from YASAKANI, not measured by us, and it does not belong in a measured table. The author's footnote describes it as one RTX 5090, Qwen3.8-27B, a single generation, on long JSONL with thinking off (https://x.com/yasakani_dev/status/2120139913891115414). We have not run YASAKANI yet.
Is 700 tok/s on a 5090 just many chats added together?
We have not measured that setup. ashxhart's original ~700 tok/s demo post is deleted (he said that was accidental: https://x.com/ashxhart/status/2120602106365202695). In a later reply in the same thread he wrote: "8 parallel coding lanes on the one 5090. ~700tps combined for that task" (https://x.com/ashxhart/status/2120477340840911289). An earlier jokey "YEAH HAHAHAH" to "Is that single stream?" is not a confirmation. A deleted-video overlay reading comes from third-party transcriptions, not from Ash's remaining posts, and stays unverified. Next to the measured 75.2 at 4K (https://llm-speed.com/r/r_qfrb2zmyk0k): Ash describes ~700 tok/s as combined across 8 parallel coding lanes, which works out to roughly 88 tok/s per lane (our division of his number, not his figure). This isn't measured by us. Per-lane speed with 8 lanes running at once and single-chat speed are not the same measure: per-stream speed usually drops as lanes are added. The RTX 5090 run also recorded 8 requests at once (about 42.5 tok/s decode per request; the 111.3 tok/s combined figure counts total output over wall time, including prompt processing; 1K-token prompt, stock llama.cpp Q4_K_M), a different setup from Ash's TensorFold/NVFP4 lanes. A same-setup 8-lane run is planned.
Why does TensorFold show 208 tok/s on a 3090 when your 3090 number is 38.61?
The 208 / 147 figures are reported on the M3TiS TensorFold 3090 card for code and chat with a DFlash2 draft model, versus 45 without the draft, on 4-bit MLX weights and a 4-bit 262K cache (https://huggingface.co/CanadaApollo6/Qwen3.8-27B-M3TiS-3090). Measured by llm-speed (@llmspeed), 38.61 tok/s is stock llama.cpp, Q4_K_M GGUF, no draft model, chat-short, 4K (https://llm-speed.com/r/r_mmtzel_i367). They are not the same setup.
Does a draft or speculative model actually change speed?
Measured by llm-speed (@llmspeed): MTP in llama.cpp on an RTX 4090, Qwen3.8-27B Q4_K_M, went from 43.11 to 75.26 tok/s decode (1.75×), while prefill slowed and TTFT rose (https://llm-speed.com/r/r_do9h933nvme, https://llm-speed.com/r/r_9yhg3wvngab). That is one form of speculative decoding, not a TensorFold or YASAKANI run.
Are you saying the big numbers are fake?
No. We are saying they often measure a different job. This page is a single-chat, stock llama.cpp, no-draft baseline, and other engines' numbers stay labelled as reported until the same setup is run.