Best GPU for local LLMs under $2,000: measured RTX 3090 vs RTX 4090

Same model on both cards (measured by llm-speed)

llm-speed measured both runs below itself on October 22, 2026: the same CLI, llama.cpp build 11803 (commit a37cc4610), the same Qwen3.8-27B Q4_K_M file, 8K context and every layer on the GPU. One run per card; the comparisons are calculated from those runs.

4090 vs 3090 (calculated from llm-speed's runs): about 12% faster decode on Qwen3.8-27B. Across all four workloads of the same runs, the RTX 3090 measured 38.83 / 38.35 / 38.54 / 37.88 tok/s and the RTX 4090 43.41 / 42.95 / 43.33 / 42.18 (chat-short, chat-long, concurrent-decode, agent-trace), so the gap holds at 11–12%. Runs at 4K and 16K context gave the same decode speed within 0.3 tok/s.

Prefill and first token: on the long chat prompt, prefill was 1,237.5 tok/s on the 3090 vs 2,008.7 on the 4090, and time to first token was 2,605 ms vs 1,605 ms. The 4090's lead is larger when prompts are long.

Older Ollama runs from a host whose RTX 4090 reports 48GB of GPU memory are left out of this comparison: a standard RTX 4090 has 24GB, and we can't confirm what that card is.

What else runs on a 24GB RTX 4090 (community-submitted runs)

Qwen3-32B Q4_K_M ran at 43.47 tok/s, Qwen2.5-Coder-14B at 89.87, Qwen3-Coder-30B-A3B at 232.5 and gpt-oss-20b at 242.6. MoE models with few active parameters decoded 5× or more faster than the dense 27–32B models on the same card.

The free speedup is software, not a pricier card

With MTP (multi-token prediction, a form of speculative decoding) on, a community-submitted run on the same RTX 4090 host decoded Qwen3.8-27B at 84.51 tok/s vs 43.06 with it off on that host (r_7j8rpk9mwhg vs r_11kn1epxewh, both community-submitted; calculated ≈1.96×). That is faster than llm-speed's RTX 5090 without MTP (75.19 at 4K context, r_qfrb2zmyk0k; the two MTP runs don't record their context). The two use different builds and hosts, so it's a reference point, not a controlled test. MTP also costs prefill speed and first-token time; see the MTP post.

How to choose

• NVIDIA launch prices (launch MSRP, not today's price; check current listings yourself): RTX 3090 $1,499 (NVIDIA newsroom, 2020-09-01); RTX 4090 $1,599 (nvidia.com, available 2022-10-12).
• Pick the 3090 if 24 GB is what you need and you can buy one well below a 4090. Measured by llm-speed, it gives up about 12% decode on a 27B model.
• Pick the 4090 for faster prefill on long prompts and about 12% more decode on Qwen3.8-27B.
• Neither fits a 70B at Q4 in 24 GB. For scale, Qwen3.8-27B used 18.62 GiB in llm-speed's 5090 run. Check your model and context with the memory-fit estimator.

Not measured, or not in this tier

• RTX 5090: excluded. NVIDIA's launch price was $1,999 (NVIDIA newsroom, Jan 2025), but we have no dated, sourced current price under $2,000. See RTX 5090 vs RTX 4090.
• RTX 5080 16GB, RX 7900 XTX, Arc B580, Radeon AI PRO R9700, dual RTX 3090: not measured.

FAQ

What is the cheapest GPU for running local LLMs well?

Of the cards on llm-speed, the RTX 3090 (24GB) is the lower-cost option. llm-speed measured Qwen3.8-27B Q4_K_M at 38.83 tok/s on it, vs 43.41 on an RTX 4090 (llama.cpp, same build and settings). We haven't measured cheaper 12–16GB cards yet.

Is the RTX 4090 worth it over a 3090 for LLMs?

In llm-speed's runs on Qwen3.8-27B, the 4090 was about 12% faster at decode and about 1.6× faster at long-prompt prefill, with the same 24 GB. Whether that's worth it depends on the price gap you see.

Can a 24GB card run a 27B or 32B model?

Yes, at 4-bit. llm-speed measured Qwen3.8-27B Q4_K_M with every layer on the GPU at 43.41 tok/s on an RTX 4090 and 38.83 on an RTX 3090 at 8K context; the 4K and 16K runs were within 0.3 tok/s. A community-submitted run (not measured by llm-speed) reports Qwen3-32B Q4_K_M at 43.47 tok/s on a 4090.

Can a 24GB card run a 70B model?

Not fully on the GPU at Q4. Use the memory-fit estimator for your model and context.

How do I make a 3090 or 4090 faster without buying a new card?

Turn on MTP or speculative decoding if your model supports it. In community-submitted runs on an RTX 4090, Qwen3.8-27B went from 43.06 to 84.51 tok/s (chat-short), at the cost of slower prefill and a later first token.

Why isn't the RTX 5090 here?

We couldn't source a dated current price under $2,000. Its results are on RTX 5090 vs RTX 4090.

Related: RTX 3090 · RTX 4090 · RTX 4090 vs RTX 3090 · Qwen3.8-27B · RTX 3090 value pick · best GPU 2026 · cheatsheet · methodology