Can an RTX 4090 run Muse-Glimmer-30B?

Yes: an RTX 4090 runs Muse-Glimmer-30B at 53.7 tok/s at UD-Q4_K_XL, measured on llama.cpp with 8K context in October 2026; UD-Q4_K_XL loaded fully on the GPU at up to 128K context.

12 signed runs (6 f16 KV, 6 q8_0 KV) on llm-speed. Single-request decode speed in tokens per second; the short-chat workload unless noted. All runs were measured by llm-speed (@llmspeed). Runs measured by llm-speed are credited to @llmspeed. TTFT includes reasoning tokens (time to the first token, not to the first answer); chat-short prefill is not shown because its short prompt makes it unrepresentative, so see chat-long for prefill.

Measured tok/s by quant and runtime

Quant	Runtime	Build	Decode tok/s (best run)	Workload	Runs	Source
UD-Q4_K_XL	llama.cpp	commit a37cc4610	53.7	chat-short	6	Measured by llm-speed@llmspeed
UD-Q4_K_XLq8_0 KV cache (-ctk q8_0 -ctv q8_0) (counted separately)	llama.cpp	commit a37cc4610	52.7	chat-short	6	Measured by llm-speed@llmspeed

Fastest quant that fits: UD-Q4_K_XL on llama.cpp, 53.7 tok/s (measured by llm-speed; it fits because it ran on this hardware). Quants without a run are not listed.

Context: unsloth's UD-Q4_K_XL GGUF (15.9 GB) loaded fully on the GPU at every context from 8K to the model's 128K maximum, with either KV cache type: 17,182 MiB peak at 128K with the default f16 KV cache and 16,272 MiB with a q8_0 KV cache, of the card's 24,564 MiB. Speeds are short prompts at each configured context, not a filled 128K window. q8_0 KV cache: those runs passed -ctk q8_0 -ctv q8_0 to llama-server, and the CLI records that setting as declared rather than reading it back from the server; their 192 to 910 MiB lower peaks are the independent sign it took effect. q8_0 decoded about 2% slower here (52.7 vs 53.7 tok/s). The fit table lists the f16 loads. Text only: the vision projector was not loaded.

VRAM used: 15.1 GiB at UD-Q4_K_XL (r_21z5pn8262x) peak, 8K context, of 24.0 GiB (nvidia-smi peak).

Fit sweep: what loaded and what ran out of memory

Each row is one attempt to load the model on llama.cpp at a quant and context, all layers on the GPU unless the GPU layers column or the result says otherwise. Out-of-memory and failed attempts are kept. Speeds for these configs are in the signed runs above.

Quant	Context	GPU layers	Result	Decode tok/s	Prefill tok/s	Peak VRAM GiB	Date (UTC)
UD-Q4_K_XL	8K	all	ran (speed in the signed run)	–	–	15.1	2026-10-22
UD-Q4_K_XL	32K	all	ran (speed in the signed run)	–	–	15.5	2026-10-22
UD-Q4_K_XL	48K	all	ran (speed in the signed run)	–	–	15.7	2026-10-22
UD-Q4_K_XL	64K	all	ran (speed in the signed run)	–	–	15.9	2026-10-22
UD-Q4_K_XL	96K	all	ran (speed in the signed run)	–	–	16.3	2026-10-22
UD-Q4_K_XL	128K	all	ran (speed in the signed run)	–	–	16.8	2026-10-22

Every measured row

Run	Date (UTC)	Workload	Context	Quant	Runtime / build	Prompt tokens	Output tokens	Decode tok/s	Prefill tok/s	TTFT incl. reasoning tokens, ms	VRAM GiB	Source
r_21z5pn8262x	2026-10-22	chat-short	8K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	53.7	–	235	15.1 peak	Measured by llm-speed@llmspeed
r_sz089qf84ge	2026-10-22	chat-short	32K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	53.7	–	235	15.5 peak	Measured by llm-speed@llmspeed
r_jn2059subhm	2026-10-22	chat-short	128K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	53.7	–	236	16.8 peak	Measured by llm-speed@llmspeed
r_j8qmkq2kd8l	2026-10-22	chat-short	48K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	53.7	–	234	15.7 peak	Measured by llm-speed@llmspeed
r_xva_h_3wgzn	2026-10-22	chat-short	96K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	53.7	–	235	16.3 peak	Measured by llm-speed@llmspeed
r_d6vbcmoirgr	2026-10-22	chat-short	64K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	53.6	–	227	15.9 peak	Measured by llm-speed@llmspeed
r_j_abawmxlle	2026-10-22	chat-shortq8_0 KV cache (-ctk q8_0 -ctv q8_0)	96K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	52.7	–	248	15.6 peak	Measured by llm-speed@llmspeed
r_d1mp5onfwe8	2026-10-22	chat-shortq8_0 KV cache (-ctk q8_0 -ctv q8_0)	64K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	52.7	–	225	15.4 peak	Measured by llm-speed@llmspeed
r_8dtxula7h35	2026-10-22	chat-shortq8_0 KV cache (-ctk q8_0 -ctv q8_0)	128K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	52.7	–	251	15.9 peak	Measured by llm-speed@llmspeed
r_g5vn9b96z6e	2026-10-22	chat-shortq8_0 KV cache (-ctk q8_0 -ctv q8_0)	32K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	52.7	–	249	15.1 peak	Measured by llm-speed@llmspeed
r_7srd_wfvix-	2026-10-22	chat-shortq8_0 KV cache (-ctk q8_0 -ctv q8_0)	48K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	52.7	–	251	15.3 peak	Measured by llm-speed@llmspeed
r_sbcga8o1-af	2026-10-22	chat-shortq8_0 KV cache (-ctk q8_0 -ctv q8_0)	8K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	154	256	52.7	–	249	14.9 peak	Measured by llm-speed@llmspeed
r_xva_h_3wgzn	2026-10-22	chat-long	96K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.8	2682.8	1170	16.3 peak	Measured by llm-speed@llmspeed
r_jn2059subhm	2026-10-22	chat-long	128K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.8	2667.3	1177	16.8 peak	Measured by llm-speed@llmspeed
r_sz089qf84ge	2026-10-22	chat-long	32K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.5	2685.3	1169	15.5 peak	Measured by llm-speed@llmspeed
r_21z5pn8262x	2026-10-22	chat-long	8K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.5	2664.2	1178	15.1 peak	Measured by llm-speed@llmspeed
r_j8qmkq2kd8l	2026-10-22	chat-long	48K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.5	2685.3	1169	15.7 peak	Measured by llm-speed@llmspeed
r_d6vbcmoirgr	2026-10-22	chat-long	64K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.4	2685.4	1169	15.9 peak	Measured by llm-speed@llmspeed
r_g5vn9b96z6e	2026-10-22	chat-longq8_0 KV cache (-ctk q8_0 -ctv q8_0)	32K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.1	2726.8	1151	15.1 peak	Measured by llm-speed@llmspeed
r_8dtxula7h35	2026-10-22	chat-longq8_0 KV cache (-ctk q8_0 -ctv q8_0)	128K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.1	2739.3	1146	15.9 peak	Measured by llm-speed@llmspeed
r_d1mp5onfwe8	2026-10-22	chat-longq8_0 KV cache (-ctk q8_0 -ctv q8_0)	64K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.1	2772.1	1132	15.4 peak	Measured by llm-speed@llmspeed
r_j_abawmxlle	2026-10-22	chat-longq8_0 KV cache (-ctk q8_0 -ctv q8_0)	96K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.1	2792.5	1124	15.6 peak	Measured by llm-speed@llmspeed
r_7srd_wfvix-	2026-10-22	chat-longq8_0 KV cache (-ctk q8_0 -ctv q8_0)	48K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	51.1	2743.7	1144	15.3 peak	Measured by llm-speed@llmspeed
r_sbcga8o1-af	2026-10-22	chat-longq8_0 KV cache (-ctk q8_0 -ctv q8_0)	8K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3139	1024	50.7	2830.0	1109	14.9 peak	Measured by llm-speed@llmspeed
r_7srd_wfvix-	2026-10-22	agent-traceq8_0 KV cache (-ctk q8_0 -ctv q8_0)	48K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	51.0	5986.9	358	15.3 peak	Measured by llm-speed@llmspeed
r_sz089qf84ge	2026-10-22	agent-trace	32K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3362	840	50.8	5323.3	365	15.5 peak	Measured by llm-speed@llmspeed
r_21z5pn8262x	2026-10-22	agent-trace	8K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	50.7	5340.7	376	15.1 peak	Measured by llm-speed@llmspeed
r_d6vbcmoirgr	2026-10-22	agent-trace	64K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	50.6	5323.1	364	15.9 peak	Measured by llm-speed@llmspeed
r_j8qmkq2kd8l	2026-10-22	agent-trace	48K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	50.3	5370.0	364	15.7 peak	Measured by llm-speed@llmspeed
r_8dtxula7h35	2026-10-22	agent-traceq8_0 KV cache (-ctk q8_0 -ctv q8_0)	128K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	50.1	5439.6	360	15.9 peak	Measured by llm-speed@llmspeed
r_g5vn9b96z6e	2026-10-22	agent-traceq8_0 KV cache (-ctk q8_0 -ctv q8_0)	32K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	49.6	5505.1	366	15.1 peak	Measured by llm-speed@llmspeed
r_j_abawmxlle	2026-10-22	agent-traceq8_0 KV cache (-ctk q8_0 -ctv q8_0)	96K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	49.5	5506.3	366	15.6 peak	Measured by llm-speed@llmspeed
r_xva_h_3wgzn	2026-10-22	agent-trace	96K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	49.5	5371.1	365	16.3 peak	Measured by llm-speed@llmspeed
r_sbcga8o1-af	2026-10-22	agent-traceq8_0 KV cache (-ctk q8_0 -ctv q8_0)	8K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	49.0	5508.7	368	14.9 peak	Measured by llm-speed@llmspeed
r_jn2059subhm	2026-10-22	agent-trace	128K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	48.9	5362.0	363	16.8 peak	Measured by llm-speed@llmspeed
r_d1mp5onfwe8	2026-10-22	agent-traceq8_0 KV cache (-ctk q8_0 -ctv q8_0)	64K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	3262	840	48.6	5518.1	342	15.4 peak	Measured by llm-speed@llmspeed
r_21z5pn8262x	2026-10-22	concurrent-decode	8K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	53.2	–	–	15.1 peak	Measured by llm-speed@llmspeed
r_jn2059subhm	2026-10-22	concurrent-decode	128K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	53.2	–	–	16.8 peak	Measured by llm-speed@llmspeed
r_sz089qf84ge	2026-10-22	concurrent-decode	32K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	53.1	–	–	15.5 peak	Measured by llm-speed@llmspeed
r_j8qmkq2kd8l	2026-10-22	concurrent-decode	48K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	53.1	–	–	15.7 peak	Measured by llm-speed@llmspeed
r_d6vbcmoirgr	2026-10-22	concurrent-decode	64K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	53.1	–	–	15.9 peak	Measured by llm-speed@llmspeed
r_xva_h_3wgzn	2026-10-22	concurrent-decode	96K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	53.1	–	–	16.3 peak	Measured by llm-speed@llmspeed
r_g5vn9b96z6e	2026-10-22	concurrent-decodeq8_0 KV cache (-ctk q8_0 -ctv q8_0)	32K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	52.3	–	–	15.1 peak	Measured by llm-speed@llmspeed
r_d1mp5onfwe8	2026-10-22	concurrent-decodeq8_0 KV cache (-ctk q8_0 -ctv q8_0)	64K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	52.3	–	–	15.4 peak	Measured by llm-speed@llmspeed
r_j_abawmxlle	2026-10-22	concurrent-decodeq8_0 KV cache (-ctk q8_0 -ctv q8_0)	96K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	52.3	–	–	15.6 peak	Measured by llm-speed@llmspeed
r_7srd_wfvix-	2026-10-22	concurrent-decodeq8_0 KV cache (-ctk q8_0 -ctv q8_0)	48K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	52.3	–	–	15.3 peak	Measured by llm-speed@llmspeed
r_sbcga8o1-af	2026-10-22	concurrent-decodeq8_0 KV cache (-ctk q8_0 -ctv q8_0)	8K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	52.3	–	–	14.9 peak	Measured by llm-speed@llmspeed
r_8dtxula7h35	2026-10-22	concurrent-decodeq8_0 KV cache (-ctk q8_0 -ctv q8_0)	128K	UD-Q4_K_XL	llama.cpp · commit a37cc4610	1024	256	52.3	–	–	15.9 peak	Measured by llm-speed@llmspeed

Questions