Can RTX 5090 32GB run Flint 40B-A2.5B?
Estimated speed and memory fit for Flint 40B-A2.5B (39.79B MoE, 2.5B active) on RTX 5090 32GB. We have no measured run for this pair yet, so the speed is an estimate; submit yours.
Short answer
Yes. Flint 40B-A2.5B UD-Q5_K_XL fits fully on RTX 5090 32GB at 16k context, using 30.4 of 31.0 GB usable, and generates about ~391 tokens/s (estimated, not measured). For UD-Q5_K_XL, our bandwidth-only estimate was ~524 tok/s against a measured median of ~391 (34% too optimistic).
Measured generation speed by quantization
No measured rows for Flint 40B-A2.5B on RTX 5090 32GB yet. The ~391 tok/s above is a bandwidth estimate (1792 GB/s × 0.71 efficiency). Have this machine? Submit a llama-bench result and this page will switch to measured data.
Generation = tg128 (128 generated tokens, short context). Public rows are medians of community llama.cpp results (sources); our own runs use llama-bench on a rented GPU.
Alternatives
Other models on RTX 5090 32GB, similar size
- Qwen3.6 35B-A3B: UD-Q6_K, fits fully, ~260 tok/s (measured)
- Ornith 1.5 35B-A3B (hardware page): Q6_K, fits fully, ~216 tok/s (scaled from measured)
- Laguna XS 2.1 (hardware page): Q6_K, fits fully, ~343 tok/s (scaled from measured)
- Nanbeige4.3 33B: Q5_K_M, fits fully, ~64 tok/s (scaled from measured)
Memory and context limits
UD-Q5_K_XL weights are 29.9 GB; the 16k f16 KV cache adds 0.51 GB. Largest context that fits in 31.0 GB usable:
| KV cache type | Max context |
|---|---|
| f16 | 32k |
| q8_0 | 64k |
| q4_0 | 128k |
Native limit 256k. Full KV cache table for Flint 40B-A2.5B.
Copy-paste commands
The advisor's pick for this hardware: UD-Q5_K_XL, 16k context.
llama-server -hf unsloth/flint-GGUF:UD-Q5_K_XL -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama serve ollama run hf.co/unsloth/flint-GGUF:UD-Q5_K_XL
FAQ
How fast is Flint 40B-A2.5B on RTX 5090 32GB?
About 390.9 tokens/s for generation (UD-Q5_K_XL, estimated from 1792 GB/s memory bandwidth). Speed drops as the context fills; see the depth table on this hardware's other pages.
How much memory does Flint 40B-A2.5B need on RTX 5090 32GB?
The advisor's pick, UD-Q5_K_XL, is 29.9 GB of weights plus 0.51 GB of KV cache at 16k context (30.4 GB), against 31.0 GB usable.
Is the Flint 40B-A2.5B speed on RTX 5090 32GB measured or estimated?
Estimated. No measured llama.cpp run for this pair is on file yet, so the figure comes from memory bandwidth and our calibration against measured hardware.
What can I use instead of RTX 5090 32GB for Flint 40B-A2.5B?
Nothing with less memory in our list fits it fully.
What is the maximum context for Flint 40B-A2.5B on RTX 5090 32GB?
With an f16 KV cache about 32k tokens, 64k with q8_0 and 128k with q4_0, bounded by the model's own limit.
Related
All models on RTX 5090 32GB · Flint 40B-A2.5B on all hardware · VRAM tier · all measured pages · leaderboard
Compare: RTX 4090 24GB vs RTX 5090 32GB · RTX 3090 24GB vs RTX 5090 32GB · 2x RTX 3090 (48GB total) vs RTX 5090 32GB