Can RTX 5090 32GB run Qwen3.6 35B-A3B?
Measured results for Qwen3.6 35B-A3B (35.95B MoE, 3B active) on RTX 5090 32GB, plus what fits and the commands to run it.
Short answer
Yes. Qwen3.6 35B-A3B Q4_K_M fits fully on RTX 5090 32GB at 16k context, using 22.5 of 31.0 GB usable, and generates about 256.9 tokens/s in our own run. As the context fills it slows down: 247.2 tok/s with 16k tokens already in context (96% of the empty-context speed), measured. For UD-Q6_K, our bandwidth-only estimate was ~403 tok/s against a measured median of ~260 (55% too optimistic).
Measured generation speed by quantization
| Quant | Generation tok/s | Range | Prompt tok/s | Runs | Origin |
|---|---|---|---|---|---|
| Q4 class (median) | 263.9 | 256.9–271 | 9421 | 2 | includes our runs |
| Q6 class (median) | 259.7 | 259.7–259.7 | 8322 | 1 | public runs |
| Q4_K_M (our run, depth 0) | 256.9 | 3 repeats | 9602 | 3 | own measurement |
Generation = tg128 (128 generated tokens, short context). Public rows are medians of community llama.cpp results (sources); our own runs use llama-bench b11658 on a rented GPU.
Speed vs context depth measured
Our own llama-bench runs on GeForce RTX 5090 (flash attention on, 3 repeats): generation and prompt speed with the given number of tokens already in the context.
| Quant | Context depth | Generation tok/s | vs depth 0 | Prompt tok/s |
|---|---|---|---|---|
| Q4_K_M | 0 | 256.9 | 100% | 9602 |
| Q4_K_M | 4k | 243.8 | 95% | 9249 |
| Q4_K_M | 16k | 247.2 | 96% | 8408 |
Alternatives
Less memory that still fits Qwen3.6 35B-A3B
Smaller memory usually means a cheaper card or Mac (we do not list prices).
- AMD RX 7900 XTX 24GB: UD-Q4_K_M, 23 GB usable, ~249 tok/s estimated
- RTX 4090 24GB: UD-Q4_K_M, 23 GB usable, ~190 tok/s measured
- RTX 3090 24GB: UD-Q4_K_M, 23 GB usable, ~159 tok/s measured
Other models on RTX 5090 32GB, similar size
- Ornith 1.5 35B-A3B (hardware page): Q6_K, fits fully, ~216 tok/s (scaled from measured)
- Laguna XS 2.1 (hardware page): Q6_K, fits fully, ~343 tok/s (scaled from measured)
- DeepSeek R1 Distill Qwen 32B (hardware page): Q5_K_M, fits fully, ~246 tok/s (scaled from measured)
- Nemotron 3 Nano 30B-A3B (hardware page): Q5_K_M, fits fully, ~342 tok/s (scaled from measured)
Memory and context limits
UD-Q6_K weights are 29.3 GB; the 16k f16 KV cache adds 0.40 GB. Largest context that fits in 31.0 GB usable:
| KV cache type | Max context |
|---|---|
| f16 | 64k |
| q8_0 | 128k |
| q4_0 | 256k |
Native limit 256k, 1.01M with YaRN. Full KV cache table for Qwen3.6 35B-A3B.
Copy-paste commands
The advisor's pick for this hardware: UD-Q6_K, 16k context.
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q6_K -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama serve ollama run hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q6_K
FAQ
How fast is Qwen3.6 35B-A3B on RTX 5090 32GB?
About 256.9 tokens/s for generation (Q4_K_M on llama.cpp CUDA, 128-token generation), 9602 tokens/s for prompt processing. Speed drops as the context fills; see the depth table.
How much memory does Qwen3.6 35B-A3B need on RTX 5090 32GB?
The advisor's pick, UD-Q6_K, is 29.3 GB of weights plus 0.40 GB of KV cache at 16k context (29.7 GB), against 31.0 GB usable. The measured Q4_K_M run needs 22.5 GB.
Is the Qwen3.6 35B-A3B speed on RTX 5090 32GB measured or estimated?
Measured: our own llama-bench run on a rented GeForce RTX 5090, including runs with 4k to 16k tokens already in context.
What can I use instead of RTX 5090 32GB for Qwen3.6 35B-A3B?
With less memory, AMD RX 7900 XTX 24GB (~249 tok/s), RTX 4090 24GB (~190 tok/s), RTX 3090 24GB (~159 tok/s) also fit it fully.
What is the maximum context for Qwen3.6 35B-A3B on RTX 5090 32GB?
With an f16 KV cache about 64k tokens, 128k with q8_0 and 256k with q4_0, bounded by the model's own limit.
Related
All models on RTX 5090 32GB · Qwen3.6 35B-A3B on all hardware · VRAM tier · all measured pages · leaderboard
Compare: RTX 4090 24GB vs RTX 5090 32GB · RTX 3090 24GB vs RTX 5090 32GB · 2x RTX 3090 (48GB total) vs RTX 5090 32GB