Nemotron 3.5 Super VL 120B-A12B: VRAM requirements & speed
Nemotron 3.5 Super VL 120B-A12B (124.45B MoE, 12B active, openmdw-1.1). Native context 1M. Smallest GGUF: 70.1 GB.
Memory needed by quantization and context
GGUF file size plus f16 KV cache (and fixed overhead of ~1 GB per GPU not shown). Source: lmstudio-community/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF.
| Quant | File | +8k ctx | +32k ctx | +128k ctx | +256k ctx | +1M ctx |
|---|---|---|---|---|---|---|
| Q4_K_M | 70.1 GB | 70.3 GB | 70.5 GB | 71.3 GB | 72.4 GB | 78.5 GB |
KV cache size by context length
Computed from the model's attention layout, including a fixed 0.17 GB linear-attention state.
| Context | f16 | q8_0 | q4_0 |
|---|---|---|---|
| 8k | 0.24 GB | 0.21 GB | 0.19 GB |
| 32k | 0.44 GB | 0.32 GB | 0.25 GB |
| 128k | 1.25 GB | 0.74 GB | 0.47 GB |
| 256k | 2.32 GB | 1.31 GB | 0.78 GB |
| 1M | 8.36 GB | 4.52 GB | 2.48 GB |
Which hardware runs Nemotron 3.5 Super VL 120B-A12B? · 16k context
Discrete GPUs are modelled with 32 GB of system RAM available for offload. Does not fit even with offload: RTX 3090 24GB, RTX 4090 24GB, RTX 5090 32GB, RTX 3060 12GB (32GB RAM), 8GB VRAM class (RTX 4060 8GB, 32GB RAM), 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM), RTX 5070 12GB, RTX 4070 12GB, RTX 5070 Ti 16GB, RTX 5080 16GB, AMD RX 7900 XTX 24GB, Mac Studio (2025) M3 Ultra 96GB, Mac mini / iMac M4 16GB, Mac mini (2024) M4 Pro 64GB, MacBook Pro 14/16 M5 Pro 48GB, Mac mini M6 32GB, MacBook Air M5 32GB, RTX 5060 Ti 16GB.
MLX conversions for Apple Silicon
No MLX conversion yet: we found no verified mlx-community 4-bit or 8-bit repo for Nemotron 3.5 Super VL 120B-A12B. Use the GGUF files with llama.cpp or ollama on a Mac. How we check.
Links
Model card · GGUF files · Open in the advisor · last verified 2026-10-20
FAQ
How much VRAM does Nemotron 3.5 Super VL 120B-A12B need?
The smallest quant (Q4_K_M) is 70.1 GB, so you need roughly 71 GB for an 8k context including KV cache and overhead. At 1M context the f16 KV cache alone is 8.4 GB.
Can I run Nemotron 3.5 Super VL 120B-A12B on a RTX PRO 6000 96GB?
Yes: on RTX PRO 6000 96GB it fits fully as Q4_K_M at about 162 tok/s (scaled from measured). 9 of 28 hardware configurations fit it fully at 16k context.
What is the maximum context for Nemotron 3.5 Super VL 120B-A12B?
1M tokens natively. Memory, not the model, is usually the limit: see the KV cache table above.
All models: model index · by memory: 8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB · 192 GB · 256 GB · 512 GB. Numbers use the same fit math as the advisor; sources and methods.