Can an RTX 3090 run Llama-3.1-8B-Instruct?
llm-speed has not measured Llama-3.1-8B-Instruct on RTX 3090 itself yet, so there is no llm-speed verdict. 1 community-submitted run exists (fastest 136.2 tok/s at Q4_K_M on ollama); community-submitted, not measured by llm-speed, so it gives no verdict.
1 signed run on llm-speed. Single-request decode speed in tokens per second; the short-chat workload unless noted. This run is community-submitted (signed on the submitter's own machine, not measured by llm-speed, and not used for llm-speed verdicts). Verdicts and headline numbers use only llm-speed's own runs and fit tests.
Measured tok/s by quant and runtime
| Quant | Runtime | Build | Decode tok/s (best run) | Workload | Runs | Source |
|---|---|---|---|---|---|---|
| Q4_K_M | ollama | 0.31.1 | 136.2 | chat-short | 1 | Community-submitted |
llm-speed has not measured this pair itself, so there is no llm-speed verdict or fastest quant. The rows above are community-submitted, not measured by llm-speed.
VRAM: not recorded for these runs. The VRAM calculator gives an estimate (predicted, not measured).
Every measured row
| Run | Date (UTC) | Workload | Quant | Runtime / build | Prompt tokens | Output tokens | Decode tok/s | Prefill tok/s | TTFT ms | VRAM GiB | Source |
|---|---|---|---|---|---|---|---|---|---|---|---|
| r_9hurqggbshk | 2026-07-02 | chat-short | Q4_K_M | ollama · 0.31.1 | 110 | 256 | 136.2 | 2.2 | 49960 | – | Community-submitted |
| r_9hurqggbshk | 2026-07-02 | chat-long | Q4_K_M | ollama · 0.31.1 | 3137 | 390 | 127.5 | 3004.3 | 1044 | – | Community-submitted |
| r_9hurqggbshk | 2026-07-02 | agent-trace | Q4_K_M | ollama · 0.31.1 | 3210 | 840 | 127.5 | 3475.9 | 527 | – | Community-submitted |
| r_9hurqggbshk | 2026-07-02 | concurrent-decode | Q4_K_M | ollama · 0.31.1 | 1024 | 256 | 133.9 | – | – | – | Community-submitted |
Questions
Can an RTX 3090 run Llama-3.1-8B-Instruct?
llm-speed has not measured Llama-3.1-8B-Instruct on RTX 3090 itself yet, so there is no llm-speed verdict. 1 community-submitted run exists (fastest 136.2 tok/s at Q4_K_M on ollama); community-submitted, not measured by llm-speed, so it gives no verdict.
What did community runs measure for Llama-3.1-8B-Instruct on an RTX 3090?
1 community-submitted run (community-submitted, not measured by llm-speed): Q4_K_M on ollama: 136.2 tok/s. They are listed for reference and give no llm-speed verdict.