Can an RTX 4090 run Sage-A1.5?
Yes: an RTX 4090 runs Sage-A1.5 at 216.8 tok/s at Q4_K_M, measured on llama.cpp with 128K context in October 2026.
7 signed runs on llm-speed. Single-request decode speed in tokens per second; the short-chat workload unless noted. All runs were measured by llm-speed (@llmspeed). Runs measured by llm-speed are credited to @llmspeed. TTFT includes reasoning tokens (time to the first token, not to the first answer); chat-short prefill is not shown because its short prompt makes it unrepresentative, so see chat-long for prefill.
Measured tok/s by quant and runtime
| Quant | Runtime | Build | Decode tok/s (best run) | Workload | Runs | Source |
|---|---|---|---|---|---|---|
| Q4_K_M | llama.cpp | commit a37cc4610 | 216.8 | chat-short | 7 | Measured by llm-speed@llmspeed |
Fastest quant that fits: Q4_K_M on llama.cpp, 216.8 tok/s (measured by llm-speed; it fits because it ran on this hardware). Quants without a run are not listed.
Context: the official Q4_K_M loaded fully on the GPU at every context from 8K to 128K (23,448 MiB peak at 128K of the card's 24,564 MiB); 192K and 256K ran out of memory at load while allocating the KV cache. Speeds are short prompts at each configured context, not a filled 128K window. Text only: the 899 MB vision projector was not loaded.
VRAM used: 20.4 GiB at Q4_K_M (r_4x-4g-xhhgk) peak, 8K context, of 24.0 GiB (nvidia-smi peak).
Fit sweep: what loaded and what ran out of memory
Each row is one attempt to load the model on llama.cpp at a quant and context, all layers on the GPU unless the GPU layers column or the result says otherwise. Out-of-memory and failed attempts are kept. Speeds for these configs are in the signed runs above.
| Quant | Context | GPU layers | Result | Decode tok/s | Prefill tok/s | Peak VRAM GiB | Date (UTC) |
|---|---|---|---|---|---|---|---|
| Q4_K_M | 8K | all | ran (speed in the signed run) | – | – | 20.4 | 2026-10-22 |
| Q4_K_M | 16K | all | ran (speed in the signed run) | – | – | 20.6 | 2026-10-22 |
| Q4_K_M | 32K | all | ran (speed in the signed run) | – | – | 20.9 | 2026-10-22 |
| Q4_K_M | 48K | all | ran (speed in the signed run) | – | – | 21.3 | 2026-10-22 |
| Q4_K_M | 64K | all | ran (speed in the signed run) | – | – | 21.6 | 2026-10-22 |
| Q4_K_M | 96K | all | ran (speed in the signed run) | – | – | 22.2 | 2026-10-22 |
| Q4_K_M | 128K | all | ran (speed in the signed run) | – | – | 22.9 | 2026-10-22 |
| Q4_K_M | 192K | all | out of memory (context) | – | – | – | 2026-10-22 |
| Q4_K_M | 256K | all | out of memory (context) | – | – | – | 2026-10-22 |
Every measured row
| Run | Date (UTC) | Workload | Context | Quant | Runtime / build | Prompt tokens | Output tokens | Decode tok/s | Prefill tok/s | TTFT incl. reasoning tokens, ms | VRAM GiB | Source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| r_-i4k-ieitrk | 2026-10-22 | chat-short | 128K | Q4_K_M | llama.cpp · commit a37cc4610 | 112 | 256 | 216.8 | – | 109 | 22.9 peak | Measured by llm-speed@llmspeed |
| r_9uktwjj22y- | 2026-10-22 | chat-short | 32K | Q4_K_M | llama.cpp · commit a37cc4610 | 112 | 256 | 216.8 | – | 109 | 20.9 peak | Measured by llm-speed@llmspeed |
| r_vusd3gw5tpg | 2026-10-22 | chat-short | 64K | Q4_K_M | llama.cpp · commit a37cc4610 | 112 | 256 | 216.6 | – | 107 | 21.6 peak | Measured by llm-speed@llmspeed |
| r_anosq1xr6tc | 2026-10-22 | chat-short | 16K | Q4_K_M | llama.cpp · commit a37cc4610 | 112 | 256 | 216.6 | – | 113 | 20.6 peak | Measured by llm-speed@llmspeed |
| r_4x-4g-xhhgk | 2026-10-22 | chat-short | 8K | Q4_K_M | llama.cpp · commit a37cc4610 | 112 | 256 | 216.5 | – | 120 | 20.4 peak | Measured by llm-speed@llmspeed |
| r_izwoxrsouc4 | 2026-10-22 | chat-short | 48K | Q4_K_M | llama.cpp · commit a37cc4610 | 112 | 256 | 216.3 | – | 113 | 21.3 peak | Measured by llm-speed@llmspeed |
| r_5nh2jxjbzur | 2026-10-22 | chat-short | 96K | Q4_K_M | llama.cpp · commit a37cc4610 | 112 | 256 | 215.3 | – | 110 | 22.2 peak | Measured by llm-speed@llmspeed |
| r_4x-4g-xhhgk | 2026-10-22 | chat-long | 8K | Q4_K_M | llama.cpp · commit a37cc4610 | 3182 | 1024 | 213.1 | 6342.5 | 502 | 20.4 peak | Measured by llm-speed@llmspeed |
| r_izwoxrsouc4 | 2026-10-22 | chat-long | 48K | Q4_K_M | llama.cpp · commit a37cc4610 | 3182 | 1024 | 212.9 | 6267.7 | 508 | 21.3 peak | Measured by llm-speed@llmspeed |
| r_vusd3gw5tpg | 2026-10-22 | chat-long | 64K | Q4_K_M | llama.cpp · commit a37cc4610 | 3182 | 1024 | 212.4 | 6336.7 | 502 | 21.6 peak | Measured by llm-speed@llmspeed |
| r_-i4k-ieitrk | 2026-10-22 | chat-long | 128K | Q4_K_M | llama.cpp · commit a37cc4610 | 3182 | 1024 | 212.0 | 6312.9 | 504 | 22.9 peak | Measured by llm-speed@llmspeed |
| r_5nh2jxjbzur | 2026-10-22 | chat-long | 96K | Q4_K_M | llama.cpp · commit a37cc4610 | 3182 | 1024 | 211.9 | 6300.5 | 505 | 22.2 peak | Measured by llm-speed@llmspeed |
| r_9uktwjj22y- | 2026-10-22 | chat-long | 32K | Q4_K_M | llama.cpp · commit a37cc4610 | 3182 | 1024 | 211.9 | 6341.3 | 502 | 20.9 peak | Measured by llm-speed@llmspeed |
| r_anosq1xr6tc | 2026-10-22 | chat-long | 16K | Q4_K_M | llama.cpp · commit a37cc4610 | 3182 | 1024 | 211.6 | 6290.8 | 506 | 20.6 peak | Measured by llm-speed@llmspeed |
| r_vusd3gw5tpg | 2026-10-22 | agent-trace | 64K | Q4_K_M | llama.cpp · commit a37cc4610 | 3454 | 840 | 210.4 | 10178.8 | 201 | 21.6 peak | Measured by llm-speed@llmspeed |
| r_5nh2jxjbzur | 2026-10-22 | agent-trace | 96K | Q4_K_M | llama.cpp · commit a37cc4610 | 3454 | 840 | 209.3 | 10163.9 | 201 | 22.2 peak | Measured by llm-speed@llmspeed |
| r_izwoxrsouc4 | 2026-10-22 | agent-trace | 48K | Q4_K_M | llama.cpp · commit a37cc4610 | 3454 | 840 | 209.2 | 10163.1 | 201 | 21.3 peak | Measured by llm-speed@llmspeed |
| r_-i4k-ieitrk | 2026-10-22 | agent-trace | 128K | Q4_K_M | llama.cpp · commit a37cc4610 | 3454 | 840 | 208.0 | 10121.2 | 201 | 22.9 peak | Measured by llm-speed@llmspeed |
| r_9uktwjj22y- | 2026-10-22 | agent-trace | 32K | Q4_K_M | llama.cpp · commit a37cc4610 | 3454 | 840 | 207.9 | 10174.9 | 201 | 20.9 peak | Measured by llm-speed@llmspeed |
| r_anosq1xr6tc | 2026-10-22 | agent-trace | 16K | Q4_K_M | llama.cpp · commit a37cc4610 | 3454 | 840 | 207.9 | 10169.8 | 203 | 20.6 peak | Measured by llm-speed@llmspeed |
| r_4x-4g-xhhgk | 2026-10-22 | agent-trace | 8K | Q4_K_M | llama.cpp · commit a37cc4610 | 3454 | 840 | 207.5 | 10214.9 | 200 | 20.4 peak | Measured by llm-speed@llmspeed |
| r_izwoxrsouc4 | 2026-10-22 | concurrent-decode | 48K | Q4_K_M | llama.cpp · commit a37cc4610 | 1024 | 256 | 217.2 | – | – | 21.3 peak | Measured by llm-speed@llmspeed |
| r_4x-4g-xhhgk | 2026-10-22 | concurrent-decode | 8K | Q4_K_M | llama.cpp · commit a37cc4610 | 1024 | 256 | 216.9 | – | – | 20.4 peak | Measured by llm-speed@llmspeed |
| r_5nh2jxjbzur | 2026-10-22 | concurrent-decode | 96K | Q4_K_M | llama.cpp · commit a37cc4610 | 1024 | 256 | 216.6 | – | – | 22.2 peak | Measured by llm-speed@llmspeed |
| r_vusd3gw5tpg | 2026-10-22 | concurrent-decode | 64K | Q4_K_M | llama.cpp · commit a37cc4610 | 1024 | 256 | 216.5 | – | – | 21.6 peak | Measured by llm-speed@llmspeed |
| r_-i4k-ieitrk | 2026-10-22 | concurrent-decode | 128K | Q4_K_M | llama.cpp · commit a37cc4610 | 1024 | 256 | 216.4 | – | – | 22.9 peak | Measured by llm-speed@llmspeed |
| r_anosq1xr6tc | 2026-10-22 | concurrent-decode | 16K | Q4_K_M | llama.cpp · commit a37cc4610 | 1024 | 256 | 215.5 | – | – | 20.6 peak | Measured by llm-speed@llmspeed |
| r_9uktwjj22y- | 2026-10-22 | concurrent-decode | 32K | Q4_K_M | llama.cpp · commit a37cc4610 | 1024 | 256 | 215.4 | – | – | 20.9 peak | Measured by llm-speed@llmspeed |
Questions
Can an RTX 4090 run Sage-A1.5?
Yes: an RTX 4090 runs Sage-A1.5 at 216.8 tok/s at Q4_K_M, measured on llama.cpp with 128K context in October 2026.
How many tokens per second does Sage-A1.5 get on an RTX 4090?
Decode speed measured by llm-speed, by quantization and runtime: Q4_K_M on llama.cpp: 216.8 tok/s. Based on 7 runs measured by llm-speed; single-request chat-short unless noted.
What is the fastest quant of Sage-A1.5 that fits on an RTX 4090?
Q4_K_M on llama.cpp was the fastest measured setting at 216.8 tok/s; it fit because it ran. Other quants are listed only where a run measured them.
How much VRAM does Sage-A1.5 use on an RTX 4090?
nvidia-smi recorded a peak of 22.9 GiB of 24.0 GiB at Q4_K_M with 128K context (run r_-i4k-ieitrk).