Flint 40B-A2.5B: VRAM requirements & speed
Flint 40B-A2.5B (39.79B MoE, 2.5B active, apache-2.0). Native context 256k. Smallest GGUF: 25.05 GB.
Memory needed by quantization and context
GGUF file size plus f16 KV cache (and fixed overhead of ~1 GB per GPU not shown). Source: unsloth/flint-GGUF.
| Quant | File | +8k ctx | +32k ctx | +128k ctx | +256k ctx |
|---|---|---|---|---|---|
| Q4_K_M UD-Q4_K_XL | 25.05 GB | 25.3 GB | 26.0 GB | 28.8 GB | 32.6 GB |
| Q5_K_M UD-Q5_K_XL | 29.92 GB | 30.2 GB | 30.9 GB | 33.7 GB | 37.5 GB |
| Q8_0 | 42.3 GB | 42.6 GB | 43.3 GB | 46.1 GB | 49.9 GB |
KV cache size by context length
Computed from the model's attention layout (sliding-window layers capped at their window).
| Context | f16 | q8_0 | q4_0 |
|---|---|---|---|
| 8k | 0.27 GB | 0.14 GB | 0.08 GB |
| 32k | 0.98 GB | 0.52 GB | 0.27 GB |
| 128k | 3.79 GB | 2.02 GB | 1.07 GB |
| 256k | 7.55 GB | 4.01 GB | 2.12 GB |
Which hardware runs Flint 40B-A2.5B? · 16k context
| Hardware | Quant | Fit | Memory | Tok/s | Speed source |
|---|---|---|---|---|---|
| RTX 5090 32GB | UD-Q5_K_XL | fits fully | 30.4 GB | 391 | scaled from measured |
| Mac Studio M5 Ultra 256GB | Q8_0 | fits fully | 42.8 GB | 224 | estimated |
| Mac Studio M5 Ultra 512GB (ships late Oct 2026) | Q8_0 | fits fully | 42.8 GB | 224 | estimated |
| RTX PRO 6000 96GB | Q8_0 | fits fully | 42.8 GB | 215 | scaled from measured |
| 2x RTX 3090 (48GB total) | Q8_0 | fits fully | 42.8 GB | 191 | estimated |
| RTX 4090 24GB | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 190 | estimated |
| RTX 3090 24GB | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 185 | estimated |
| AMD RX 7900 XTX 24GB | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 178 | estimated |
| MacBook Pro 14/16 (2024) M4 Max 128GB | Q8_0 | fits fully | 42.8 GB | 122 | scaled from measured |
| Mac Studio (2025) M3 Ultra 96GB | Q8_0 | fits fully | 42.8 GB | 96 | estimated |
| MacBook Pro 14/16 M5 Pro 48GB | UD-Q5_K_XL | fits fully | 30.4 GB | 89 | estimated |
| Mac Studio M5 Max 128GB | Q8_0 | fits fully | 42.8 GB | 86 | scaled from measured |
| MacBook Pro 14/16 M5 Max 128GB | Q8_0 | fits fully | 42.8 GB | 86 | scaled from measured |
| RTX 5080 16GB | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 74 | estimated |
| RTX 5070 Ti 16GB | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 73 | estimated |
| NVIDIA RTX Spark N1X 128GB | Q8_0 | fits fully | 42.8 GB | 71 | estimated |
| RTX 5060 Ti 16GB | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 68 | estimated |
| AMD Ryzen AI Max+ PRO 495 (Ryzen AI Halo) 192GB | Q8_0 | fits fully | 42.8 GB | 65 | estimated |
| NVIDIA DGX Spark 128GB | Q8_0 | fits fully | 42.8 GB | 64 | scaled from measured |
| 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM) | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 64 | estimated |
| RTX 5070 12GB | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 54 | estimated |
| RTX 4070 12GB | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 53 | estimated |
| AMD Ryzen AI Max+ 395 (Strix Halo) 128GB | Q8_0 | fits fully | 42.8 GB | 53 | scaled from measured |
| Mac mini (2024) M4 Pro 64GB | Q8_0 | fits fully | 42.8 GB | 47 | estimated |
| 8GB VRAM class (RTX 4060 8GB, 32GB RAM) | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 42 | estimated |
| RTX 3060 12GB (32GB RAM) | UD-Q4_K_XL | MoE offload to CPU | 25.6 GB | 31 | estimated |
| Mac mini / iMac M4 16GB | - | does not fit | - | - | |
| Mac mini M6 32GB | - | does not fit | - | - | |
| MacBook Air M5 32GB | - | does not fit | - | - |
Discrete GPUs are modelled with 32 GB of system RAM available for offload. Does not fit even with offload: Mac mini / iMac M4 16GB, Mac mini M6 32GB, MacBook Air M5 32GB.
MLX conversions for Apple Silicon
No MLX conversion yet: we found no verified mlx-community 4-bit or 8-bit repo for Flint 40B-A2.5B. Use the GGUF files with llama.cpp or ollama on a Mac. How we check.
Links
Model card · GGUF files · Open in the advisor · last verified 2026-10-26
Can it run? Flint 40B-A2.5B on: RTX 5090 32GB · MacBook Pro 14/16 M5 Max 128GB (all pairs)
FAQ
How much VRAM does Flint 40B-A2.5B need?
The smallest quant (Q4_K_M) is 25.05 GB, so you need roughly 26 GB for an 8k context including KV cache and overhead. At 256k context the f16 KV cache alone is 7.6 GB.
Can I run Flint 40B-A2.5B on a RTX 5090 32GB?
Yes: on RTX 5090 32GB it fits fully as UD-Q5_K_XL at about 391 tok/s (scaled from measured). 15 of 29 hardware configurations fit it fully at 16k context.
What is the maximum context for Flint 40B-A2.5B?
256k tokens natively. Memory, not the model, is usually the limit: see the KV cache table above.
All models: model index · by memory: 8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB · 192 GB · 256 GB · 512 GB. Numbers use the same fit math as the advisor; sources and methods.