Can RTX 5090 32GB run Qwen3.6 27B?
Measured results for Qwen3.6 27B (27.78B dense) on RTX 5090 32GB, plus what fits and the commands to run it.
Short answer
Yes. Qwen3.6 27B Q8_0 fits fully on RTX 5090 32GB at 16k context, using 29.8 of 31.0 GB usable, and generates about 80.7 tokens/s. For Q8_0, our bandwidth-only estimate was ~34 tok/s against a measured median of ~49 (29% too pessimistic).
Measured generation speed by quantization
| Quant | Generation tok/s | Range | Prompt tok/s | Runs | Origin |
|---|---|---|---|---|---|
| Q4 class (median) | 80.7 | 77.1–84.3 | 4620 | 2 | public runs |
| Q6 class (median) | 61.9 | 61.9–61.9 | 3222 | 1 | public runs |
Generation = tg128 (128 generated tokens, short context). Public rows are medians of community llama.cpp results (sources); our own runs use llama-bench on a rented GPU.
Memory and context limits
Q8_0 weights are 28.6 GB; the 16k f16 KV cache adds 1.23 GB. Largest context that fits in 31.0 GB usable:
| KV cache type | Max context |
|---|---|
| f16 | 32k |
| q8_0 | 32k |
| q4_0 | 64k |
Native limit 256k, 1.01M with YaRN. Full KV cache table for Qwen3.6 27B.
Copy-paste commands
The advisor's pick for this hardware: Q8_0, 16k context.
llama-server -hf unsloth/Qwen3.6-27B-GGUF:Q8_0 -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/Qwen3.6-27B-GGUF:Q8_0
FAQ
How fast is Qwen3.6 27B on RTX 5090 32GB?
About 80.7 tokens/s for generation (Q4 class, median of 2 public runs). Speed drops as the context fills; see the depth table on this hardware's other pages.
How much memory does Qwen3.6 27B need on RTX 5090 32GB?
The advisor's pick, Q8_0, is 28.6 GB of weights plus 1.23 GB of KV cache at 16k context (29.8 GB), against 31.0 GB usable.
What is the maximum context for Qwen3.6 27B on RTX 5090 32GB?
With an f16 KV cache about 32k tokens, 32k with q8_0 and 64k with q4_0, bounded by the model's own limit.
Related
All models on RTX 5090 32GB · Qwen3.6 27B on all hardware · VRAM tier · all measured pages · leaderboard
Compare: RTX 4090 24GB vs RTX 5090 32GB · RTX 3090 24GB vs RTX 5090 32GB · 2x RTX 3090 (48GB total) vs RTX 5090 32GB