30B MoE vs dense: measured local speed on RTX 5090, M3 Ultra, M4 Max
Published 2026-10-13
Short answer: on the same RTX 5090 with llama.cpp, the mixture-of-experts Qwen3.6-35B-A3B decoded at 224.0 tok/s, about 3x the dense Qwen3.6-27B at 74.0 tok/s (both Q4_K_M, chat-short, measured). The same pattern holds on Apple Silicon: on an M3 Ultra, Qwen3-30B-A3B ran at 112.5 tok/s against 34.4 tok/s for dense Qwen3-32B (MLX 0.31.3, 4-bit). Every number below is a measured, signed run.
Measured MoE and dense rows
RTX 5090, llama.cpp, chat-short:
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run |
|---|---|---|---|---|---|
| RTX 5090 (32GB) | Qwen3-Coder-30B-A3B-Instruct quant not reported | llama.cpp chat-short · MoE | 259.9tok/s | 218ms | r_c7qyvvmmsv1 2026-07-01 |
| RTX 5090 (32GB) | Qwen3.6-35B-A3B-Q4_K_M.gguf quant not reported | llama.cpp chat-short · MoE | 224.0tok/s | 170ms | r_g1aw3mizd32 2026-07-01 |
| RTX 5090 (32GB) | Qwen3.6-27B-Q4_K_M.gguf quant not reported | llama.cpp chat-short · dense | 73.96tok/s | 190ms | r_f7ulllqu4vk 2026-07-01 |
| RTX 5090 (32GB) | Qwen3.8-27B Q4_K_M | llama.cpp 1 (9725a31) chat-short · dense; 1 of 3 repeats | 65.66tok/s | 118ms | r_dz5a3lzxhrj 2026-09-05 |
M3 Ultra (96GB), MLX 0.31.3, chat-short:
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run |
|---|---|---|---|---|---|
| M3 Ultra (60-core GPU) | Qwen3-30B-A3B-Instruct-2507-4bit quant not reported | mlx 0.31.3 chat-short · MoE | 112.5tok/s | 356ms | r_o6wbmfegmde 2026-07-12 |
| M3 Ultra (60-core GPU) | Qwen3-Coder-30B-A3B-Instruct-4bit quant not reported | mlx 0.31.3 chat-short · MoE | 112.2tok/s | 539ms | r_fpsca03u2o_ 2026-04-28 |
| M3 Ultra (60-core GPU) | Qwen3-Next-80B-A3B-Instruct-MLX-4bit 4bit | mlx 0.31.3 chat-short · MoE, 80B total | 80.34tok/s | 4,493ms | r_1pl79r50ofy 2026-05-12 |
| M3 Ultra (60-core GPU) | Qwen3-32B-4bit quant not reported | mlx 0.31.3 chat-short · dense | 34.41tok/s | 1,156ms | r_anmmc80-aoq 2026-04-28 |
M4 Max and RTX 4090 (Ollama), chat-short:
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run |
|---|---|---|---|---|---|
| M4 Max (40-core GPU) | qwen3-coder-bench-32k Q4_K_M | ollama 0.30.11 chat-short · MoE (qwen3-coder, 32K context tag) | 113.3tok/s | 1,308ms | r_roktphpc--8 2026-06-27 |
| RTX 4090 (24GB) | qwen3-coder Q4_K_M | ollama 0.31.1 chat-short · MoE | 145.2tok/s | 562ms | r_wjq32z47vlp 2026-07-02 |
Rows within a table share a rig and backend but not always a date or quant file; rows across tables use different backends. Read them as measured examples, not a controlled hardware ranking.
Why a bigger MoE decodes faster
Single-request decode is mostly limited by how many bytes of weights the GPU must read per token. A dense 27B model reads all of its weights for every token. A 30B-A3B or 35B-A3B MoE routes each token through a few experts, so only about 3B parameters are active per token. That is why Qwen3-Next-80B-A3B still decoded at 80.3 tok/s on the M3 Ultra, faster than the dense 32B. This is the mechanism, not a measurement: the measured rows above are the evidence.
The catch is memory. All experts must still be loaded, so a 35B MoE needs memory for 35B parameters even though it computes like a 3B one. Check the fit for your card or Mac with the memory-fit estimator.
The Mac angle
MoE models suit high-memory Macs: unified memory holds the full set of experts, and the small active set keeps decode fast. On the M3 Ultra, every MoE row above beat the dense 32B by more than 2x. First-token waits on the Mac were longer than on the RTX 5090 in these runs, especially for the 80B model.
What we have not measured yet
- MoE chat-short on the RTX 4090 and RTX 3090 with llama.cpp, for a same-backend GPU table.
- MoE on a 32GB or 36GB Mac (fit and speed).
- Gemma 4 26B A4B on any hardware.
Have one of these? Run the suite and submit.
FAQ
Is MoE faster than dense locally? In our measured same-rig pairs, yes: 224.0 vs 74.0 tok/s on an RTX 5090 (Qwen3.6-35B-A3B vs Qwen3.6-27B) and 112.5 vs 34.4 tok/s on an M3 Ultra (Qwen3-30B-A3B vs Qwen3-32B).
How fast is Qwen3-Coder-30B on a Mac? Measured 112.2 tok/s on an M3 Ultra (MLX) and 113.3 tok/s on an M4 Max (Ollama, qwen3-coder Q4_K_M). Different backends, so not a head-to-head.
Does a 30B MoE fit on a 24GB GPU? Our RTX 4090 (24GB) ran qwen3-coder (30.5B, Q4_K_M) through Ollama at 145.2 tok/s in chat-short. Check the run page for its configuration.
Related: fastest rig for Qwen3-Coder-30B · fastest small-MoE coder on a Mac · Qwen3-Coder-30B-A3B · fastest local coding models