Can RTX 3090 24GB run Qwen3.8 27B?
Measured results for Qwen3.8 27B (27.78B dense) on RTX 3090 24GB, plus what fits and the commands to run it.
Short answer
Yes. Qwen3.8 27B Q4_K_M fits fully on RTX 3090 24GB at 16k context, using 17.7 of 23.0 GB usable, and generates about 42.9 tokens/s in our own run. As the context fills it slows down: 40.6 tok/s with 16k tokens already in context (95% of the empty-context speed) and 35.5 tok/s at 64k (83%), measured. For UD-Q5_K_M, our bandwidth-only estimate was ~26 tok/s against a measured median of ~38 (31% too pessimistic).
Measured generation speed by quantization
| Quant | Generation tok/s | Range | Prompt tok/s | Runs | Origin |
|---|---|---|---|---|---|
| Q4 class (median) | 42.9 | 42.9–42.9 | 1507 | 1 | includes our runs |
| Q6 class (median) | 33.9 | 33.9–33.9 | 1360 | 1 | includes our runs |
| Q4_K_M (our run, depth 0) | 42.9 | 3 repeats | 1507 | 3 | own measurement |
| Q6_K (our run, depth 0) | 33.9 | 3 repeats | 1360 | 3 | own measurement |
Generation = tg128 (128 generated tokens, short context). Public rows are medians of community llama.cpp results (sources); our own runs use llama-bench b11658 on a rented GPU.
Speed vs context depth measured
Our own llama-bench runs on GeForce RTX 3090 (flash attention on, 3 repeats): generation and prompt speed with the given number of tokens already in the context.
| Quant | Context depth | Generation tok/s | vs depth 0 | Prompt tok/s |
|---|---|---|---|---|
| Q4_K_M | 0 | 42.9 | 100% | 1507 |
| Q4_K_M | 4k | 42.4 | 99% | 1438 |
| Q4_K_M | 16k | 40.6 | 95% | 1234 |
| Q4_K_M | 32k | 38.7 | 90% | 1058 |
| Q4_K_M | 64k | 35.5 | 83% | 820 |
| Q6_K | 0 | 33.9 | 100% | 1360 |
| Q6_K | 4k | 33.5 | 99% | 1299 |
| Q6_K | 16k | 32.5 | 96% | 1139 |
| Q6_K | 32k | 31.2 | 92% | 984 |
Alternatives
Less memory that still fits Qwen3.8 27B
Smaller memory usually means a cheaper card or Mac (we do not list prices).
- Mac mini M6 32GB: UD-Q4_K_M, 22 GB usable, ~6 tok/s estimated
- MacBook Air M5 32GB: UD-Q4_K_M, 22 GB usable, ~6 tok/s estimated
Faster for Qwen3.8 27B
- RTX 5090 32GB: Q8_0, 31 GB usable, ~53 tok/s measured
- RTX PRO 6000 96GB: Q8_0, 95 GB usable, ~50 tok/s scaled from measured
Other models on RTX 3090 24GB, similar size
- Qwen3.6 27B: Q5_K_M, fits fully, ~36 tok/s (scaled from measured)
- Gemma 4 26B-A4B: UD-Q5_K_M, fits fully, ~112 tok/s (scaled from measured)
- Qwen3-Coder 30B-A3B: Q4_K_M, fits fully, ~206 tok/s (measured)
- GLM-4.7-Flash (hardware page): Q5_K_M, fits fully, ~192 tok/s (scaled from measured)
Memory and context limits
UD-Q5_K_M weights are 19.8 GB; the 16k f16 KV cache adds 1.23 GB. Largest context that fits in 23.0 GB usable:
| KV cache type | Max context |
|---|---|
| f16 | 32k |
| q8_0 | 64k |
| q4_0 | 128k |
Native limit 256k, 1M with YaRN. Full KV cache table for Qwen3.8 27B.
Copy-paste commands
The advisor's pick for this hardware: UD-Q5_K_M, 16k context.
llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_M -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama serve ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_M
FAQ
How fast is Qwen3.8 27B on RTX 3090 24GB?
About 42.9 tokens/s for generation (Q4_K_M on llama.cpp CUDA, 128-token generation), 1507 tokens/s for prompt processing. Speed drops as the context fills; see the depth table.
How much memory does Qwen3.8 27B need on RTX 3090 24GB?
The advisor's pick, UD-Q5_K_M, is 19.8 GB of weights plus 1.23 GB of KV cache at 16k context (21.0 GB), against 23.0 GB usable. The measured Q4_K_M run needs 17.7 GB.
Is the Qwen3.8 27B speed on RTX 3090 24GB measured or estimated?
Measured: our own llama-bench run on a rented GeForce RTX 3090, including runs with 4k to 64k tokens already in context.
What can I use instead of RTX 3090 24GB for Qwen3.8 27B?
With less memory, Mac mini M6 32GB (~6 tok/s), MacBook Air M5 32GB (~6 tok/s) also fit it fully. Faster: RTX 5090 32GB (~53 tok/s), RTX PRO 6000 96GB (~50 tok/s).
What is the maximum context for Qwen3.8 27B on RTX 3090 24GB?
With an f16 KV cache about 32k tokens, 64k with q8_0 and 128k with q4_0, bounded by the model's own limit.
Related
All models on RTX 3090 24GB · Qwen3.8 27B on all hardware · VRAM tier · all measured pages · leaderboard
Compare: RTX 3090 24GB vs RTX 4090 24GB · RTX 3090 24GB vs RTX 5090 32GB · RTX 3090 24GB vs AMD RX 7900 XTX 24GB